3/2/11. CompMusic: Computational models for the discovery of the world s music. Music information modeling. Music Computing challenges

Transcription

1 CompMusic: Computational for the discovery of the world s music Xavier Serra Music Technology Group Universitat Pompeu Fabra, Barcelona (Spain) ERC mission: support investigator-driven frontier research. CompMusic is funded with an ERC Advanced Grant for a period of 5 years and with a budget of 2,5 million Euros. Music Computing challenges Music information modeling Music Computing research (IT in general) does not respond to the world's multi-cultural reality. Data, cognition, user, interaction, ontologies, are culturally biased. Cognitive Musicology Cognition User Ontologies Music information is more than CDs and metadata. Computational Musicology Music Human Computer Interaction Interaction Data Music Information Processing Sound and Music Computing Sound and Music Computing Understanding, modelling and generating sound and music through computational approaches. Processing of sound and music signals Understanding and modelling sound and music Interfaces for sound and music Assisted sound and music creation Music Information Processing Analysis and modelling of music related information. Audio content processing (Klapuri & Davy, 2006) Multi-modal processing Music context processing User/Community profiling Music representations, Ontologies (Raimond, 2008) 1

2 Computational musicology Music modelling originating from music theory. Tonal music (Lerdhal, 2001) Performance studies (Gabrielsson, 2003) User centred studies (Lesaffre, 2005) Computational tools ( Cognitive musicology Understand music perception formalizing the mental processes involved. Perceptual and Cognitive (Purwins et al., 2008) Embodied cognition (Leman, 2008) Music emotion (Juslin & Västfjäll, 2008) Music HCI Understanding and modelling music interaction. Information Foraging Theory (Pirolli, 2007) Interactive information retrieval (Cole et al. 2005) Collaborative creativity (Mamykina et al., 2002) CompMusic objectives Approach Music Computing from a multicultural point of view. Advance in the description and formalization of music making it more accessible to computational approaches. Reduce the gap between audio signal descriptions and semantically meaningful music concepts. Develop information modelling techniques for different music repertories. Develop computational to represent culture specific music contexts. Design culture-driven music discovery systems. Proposed approach Combination of academic disciplines: Computational Musicology, Cognitive Musicology, Music Information Processing, Music Interaction. Combination of methodologies: qualitative and quantitative; scientific and engineering. Combination of information sources: audio features, symbolic scores, text commentaries, user evaluations, etc Combination of music repertoires: Indian (hindustani, carnatic), Turkish-Arab (ottoman, andalusian), Chinese (han). Combination of cultural perspectives: Research teams and users immersed in the different music cultures. Why these musical repertoires? Belong to formalized classical traditions with strong influence on current society. Musicological and cultural studies available. Alive performance practice traditions. Exists within active social/cultural contexts. Possibility to challenge current western centred information paradigms. 2

3 3/2/11 Turkish-Arab music Indian music Melodic structure: Raga Melodic structure: Maqam Rhythmic structure: Tala Rhythmic structure: Wazn Texture: monophonic Texture: Monophonic Style: pre-composed and improvisatory. Style: pre-composed and improvisatory. Carnatic: Sudha Ragunathan Ottoman classical music Hindustani: Ravi Shankar Andalusian classical music CompMusic tasks Han Chinese music Melodic structure: heptatonic (not pentatonic!!) Harmony: five harmonies Rhythmic structure: duple Texture: Polyphonic Liu Ji Hong, Erhu concerto Task 1: Music repertoires Task 2: Musicological framework Gathering and organizing audio recordings, metadata, descriptions, scores, plus all the needed contextual information. Musicological studies to understand the chosen repertories within their cultural context. MTG-DB framework (MTG-UPF) Turkish art music (Signell, 2004) Arab-Andalusi music (D Erlanger, 1930) Chinese music (Shen, 1991) Open data movement: Wikipedia, Musicbrainz, Wikibooks, DBLP Bibliography, DBTune, DBpedia, Carnatic music (Sambamoorthy, 1998) Rasas in Indian art (Rangacharya, 2010) Resource Description Framework ( Grid Computing 3

4 Task 3: Music ontologies Building the ontologies needed for annotating the gathered collections. Task 4: Audio description Audio content analysis to describe the music collections chosen. The music ontology specification ( (Raimond, 2007) Community-based ontologies (Mika, 2006) Essentia & Gaia framework (MTG-UPF) Joint estimation of audio features (Papadopoulos & Peeters, 2011) Top-down and knowledge-based processing (in Klapuri & Davy, 2006) Task 5: User profiling Characterization of users and communities, modelling their musical preferences and behaviours. Task 6: Music interaction Interaction by studying user behaviour in musical tasks. Social Computing (Chai et al., 2010) Theory of music preferences (Rentfrow & Gosling, 2003) Community modelling Cultural Computing (Nakatsu et al., 2010) Table-top interfaces (Reactable) Task 7: Music discovery Conclusions Active and systems for culture-based music discovery. Collaborative creativity Online learning (Moh et al., 2008) Recommendation systems (Celma, 2009) Big and challenging project!!!! But hopefully we can contribute with our music research to develop better IT for our multicultural world. 4

5 References (1 of 4) Celma, O Music Recommendation and Discovery in the Long Tail. PhD thesis. Cole, C., et al Interactive information retrieval: Bringing the user to a selection state. in A. Spink & C. Cole (Eds.), New directions in cognitive information retrieval. Springer. Chai, S., et al. (Eds.) Advances in Social Computing. Springer. D Erlanger, R La Musique Arabe. Libraire Orientaliste Paul Geuthner (reissue 2001). 7 volumes. Gabrielsson, A Music Performance Research at the Millennium. Psychology of Music. Juslin, P. N. & D. Västfjäll Emotional Responses to Music: The Need to Consider Underlying Mechanisms. Behavioral and Brain Sciences. References (2 of 4) Klapuri, A., Davy, M. (Eds.) Signal Processing Methods for Music Transcription. Springer. Lerdahl, F Tonal Pitch Space. Oxford University Press. Lesaffre, M Music Information Retrieval: Conceptual framework, Annotation and User Behaviour. PhD Thesis. Mamykina, L. et al Collaborative creativity. Communications of the ACM. Mika, P Ontologies are us: A unified model of social networks and semantics. Web Semantics: Science, Services and Agents on the World Wide Web. Moh, Y., Orbanz, P., Buhmann, J. M "Music Preference Learning with Partial Information". ICASSP. Nakatsu, R. et al. (Eds.) Cultural Computing. Springer. References (3 of 4) Pachet, F Knowledge Management and Musical Metadata. Encyclopedia of Knowledge Management. Papadopoulos, H. and G. Peeters Joint Estimation of Chords and Downbeats from an Audio Signal. IEEE Transactions on Audio, Speech, and Language Processing. Pirolli, P Information Foraging Theory: Adaptive Interaction with Information. Oxford University Press. Purwins, H. et al Computational Models of Music Perception and Cognition I : The Perceptual and Cognitive Processing Chain. Physics of Life Reviews. Raimond, Y A Distributed Music Information System. PhD Thesis. Rangacharya, A The Natyasastra. Munshiram Manoharlal Publishers. References (4 of 4) Rentfrow, P. J., Gosling, S. D The Do Re Mi's of Everyday Life: The Structure and Personality Correlates of Music Preferences. Journal of Personality and Social Psychology. Sambamoorthy, P South Indian Music. The Indian Music Publishing House. (6 volumes) Shen, S Chinese Music and Orchestration: A Primer on Principles and Practice. Chinese Music Society of North America. Signell, K Makam: Modal Practice in Turkish Art Music. Usul editions. 5