Requiem for the Media: Semantic Computing in Entertainment

Transcription

1 Requiem for the Media: Semantic Computing in Entertainment Shlomo Dubnov" UCSD, Music SC Workshop, ISM 2012!

2 Semantic Computing in Entertainment Meaning of Media What people respond to? How the data can be meaningfully manipulated Group intent: Consumption patterns Collaborative authoring

3 Socio-Economical view The mystique of the socialist predestination of the media is opposite but complementary to the Orwellian myth of their terrorist manipulation by authority Everyone become a manipulator, in the sense of active operator, producer, etc., in brief, move from receiver status to that of producertransmitter - Jean Baudrillard

4 The Medium is the Message? This scientific construction is rooted in a simulation model of communication. It excludes, from its inception, the reciprocity and antagonism of interlocutors, and the ambivalence of their exchange. Jean Baudrillard s response to McLunan s maxim and Enzensberger s Constituents of a Theory of the Media (The New Media Reader)

5 Baudrilliard s Requiem Go beyond a dialectic of transmitter and receiver... there are neither transmitters, nor receivers, but only people responding to each other nor, for that matter, is there any longer any message, any corpus of information to decode univocally under the aegis of a code. The symbolic consists precisely in breaching the univocality of the message, in restoring the ambivalence of meaning and in demolishing in the same stroke the agency of the code Changing the contents of the message serves no purpose; it is necessary to modify the reading codes, to impose other interpretive codes.

6 Technology in Music Industry Production: DAW Delivery: CD and Internet Consumption: Music Recommendation Semantic Computing breaking the food chain?

7 Market Research "Seeing that music industry is more diverse and fragmented, monetising the new music experience has become increasingly complex Dominique Leguern, Director of MIDEM Almost one in five music fans 'will do anything' to meet their idols, and many are happy to view ads and even share their personal information for access to free music Synovate, Music Matters, MIDEM 2010

8 Music Consumption Patterns Cultural diversity 63% consider themselves passionate about music 'I couldn't care less' (1) to 'I'd listen every minute of the day if I could' (10). rank 6 and above: top Brazil (80%), Spain and the UK (79% each), lowest in Australia (27%) Celebrity culture 'give anything' to meet their favorite music artists US (33%), UK (32%) and Spain (30%), China, Hong Kong, and Hungary (7%) Brands sponsorship 47% of people globally don't think this is a good idea, with Hungarians the most opposed, at 63% 30% of people globally look out for competitions or promotions that feature their favourite artists and bands, topped by China at 49%

9 Changes in Music Industry 1980s, the "Big 6" EMI, CBS, BMG, PolyGram, WEA and MCA 1998, the "Big 5 - Sony bought CBS Records in 1987, PolyGram merged into Universal Music Group 2004, the Big 4 - Sony acquired BMG 2011, the Big 3 - Universal acquired EMI Sony CD 1976 MP3 1991

10 Music as a social product True fan is a person who spends $100 on his favorite artist per year 1000 True Fans Tribe / Herd

11 Integrated Music Experience New Composition Practices Multi-Layer Database Search User Modeling Interactive Sets Generative Music Group Intent and Coordination

12

13

14 Multi-Layer Representation IEEE 1599 Standard Layers Layer synchronization XML Symbols for Synchronization

15 Semantic Web Music

16 Semantic HiFi Browsing, Listening, Interacting, Performing, Sharing on Future Hifi Systems

17 New Content Models Use of Social Media during Performance Canned Contents Directed Participation NLP Exegetic Strategies Joint Authorship Commentary as Significance Giver Linking to OpenLinked Data

18 System of NicoNico NicoNicoDouga provides a unique commenting system that renders comments directly onto the video. Those comments are synchronized to a specific playback time. The commenting system contributes to create a sense of a shared watching experience among anonymous users. Main Banner Advertisemen t Social Tag Versioning System for Comments Any users can write comments over the video Affiliate for amazon.co.jp Affiliate for Google 18

19 Provocateur

20 public canvas - shows event title, chat log, promoted comments, tag cloud for a publicly viewable screen"

21 Distributed DJ

22 Music Information Dynamics X Z Source Channel. Receiver Y Z I(X,Z) = H(X) H(X Z) I(X,Y Z) = H(Y Z) H(Y X,Z) X Z Z-past X-present Y-future Analysis of signal complexity and prediction is an indicator or emotional and cognitive reactions to music Can be measured through mutual information between past, present and future

23 InfoDyn as Meta- Control Interface

24 Music Meta-Creation IRCAM / UCSD: OMax, Pyoracle Sony s Continuator [F.Pachet] Mimi [E.Chew] Band out of Box [B.Thomas] Resources:

25 Machine Improvisation Memex: Piano Violin Composer Duets:

26 Mining Screenplays Create tools to access a large database of screenplays (film, television, theatre) Allowing for new kinds of narrative and emotional structure analysis in drama and storytelling 50% of what we understand comes from watching it being said - McKee (1999) From Story Line to Box Office Eliashberg et al. (2007)

27 Casablanca scene 43

28 Narrative scatter plot The Fellowship of the Ring has a linear story structure with one main story line and a subplot. Merry and Pippin are located Very close to each other in the main plot, seemingly because of how indistinguishable they are in this movie. Aragorn, Legolas, and Gimli are all located together at the right lower end of the plot. The ring is located in the center of the plot. Lord of the Rings Fellowship of the Ring

29 Narrative scatter plot Lord of the Rings Return of the King Return of the King has more pronounced three main subplots represented in the scatter plot. The hierarchical cluster graphs suggest this too but they also break the clusters up somewhere in the middle of the movie. Looking at the characters shows Frodo, Sam and Gollum taking the ring to Mordor to be destroyed, Merry, Pippin and Gandalf dealing with the crazy king, and Aragorn, Legalos, and Gimli fighting hords of story protagonists.

30 Browsing by Narrative

31 Model of Aesthetic Experience Leder et al., British Journal of Psychology, 2004

32 O=V+E+R+HV-F V - vertical E - equilibrium R - rotation HV - network F - form Alfred Lord Tennyson ( ) O=aa+2r+2m-2ae-2ce Antonio Correggio Danae 1531 aa - alliteration/assonance r - rhyme m - musical sounds ae - alliterative excess ce - consonant sounds

33 Conclusion Understanding social trends and context awareness in multimedia consumption New content models Social interaction with canned contents Semantic interfaces and mediation technologies TBD Business models