11/1/11. CompMusic: Computational models for the discovery of the world s music. Current IT problems. Taxonomy of musical information

Transcription

1 CompMusic: Computational models for the discovery of the world s music Xavier Serra Music Technology Group Universitat Pompeu Fabra, Barcelona (Spain) ERC mission: support investigator-driven frontier research. CompMusic is funded with an ERC Advanced Grant for a period of 5 years and with a budget of 2,5 million Euros. Current IT problems Taxonomy of musical information IT research does not respond to the world's multicultural reality. Data models, cognition models, user models, interaction models, ontologies, are culturally biased. Music information is not just CDs and metadata. (Lesaffre, 2005) 1

2 Computational music modeling Cognitive Musicology Data models Cognition models Ontologies Computational Musicology Music Information Processing Human-Computer Interaction Interaction models Sound and Music Computing CompMusic objectives Promote a multicultural approach to IT research. Advance in the description and formalization of music to make it accessible to computational approaches. Reduce the gap between audio signal descriptions and semantically meaningful concepts for music. Develop data modelling techniques for different music repertories. Develop computational models to represent culture specific musical contexts. Design culture driven music discovery systems. Proposed approach Combination of academic disciplines: Computational Musicology, Cognitive Musicology, Music Information Processing, Music Interaction. Combination of methodologies: qualitative and quantitative; scientific and engineering. Combination of information sources: audio features, symbolic scores, text commentaries, user evaluations, etc Combination of music repertoires: Indian (hindustani, carnatic), Turkish-Arab (turkish, andalusian), Chinese (han). Combination of cultural perspectives: Research teams and users immersed in the different music cultures. Why these musical repertoires? Belong to formalized classical traditions with strong influence on current society. Musicological and cultural studies available. Alive performance practice traditions. Exists within active social/cultural contexts. Possibility to challenge current western centred information paradigms. 2

3 11/1/11 Turkish-Arab music Indian music Melodic structure: Raga Melodic structure: Maqam Rhythmic structure: Tala Rhythmic structure: Wazn Texture: monophonic Texture: Monophonic Style: pre-composed and improvisatory. Style: pre-composed and improvisatory. Carnatic: Sudha Ragunathan Ottoman classical music Hindustani: Ravi Shankar Andalusian classical music CompMusic tasks Han Chinese music Melodic structure: heptatonic (not pentatonic!!) Harmony: five harmonies Rhythmic structure: duple Texture: Polyphonic Liu Ji Hong, Erhu concerto 3

4 Task 1: Music repertoires Gathering and organizing audio recordings, metadata, descriptions, scores, plus all the needed contextual information. MTG-DB framework (MTG-UPF) Open data movement: Wikipedia, Musicbrainz, Wikibooks, Wordnet, DBLP Bibliography, DBTune, Geonames, Resource Description Framework ( Grid Computing Task 2: Musicological framework Musicological studies to understand the chosen repertories within their cultural context. Tonal pitch space theory (Lerdahl, 2001) Performance studies (Gabrielsson, 2003) Embodied cognition (Leman, 2008) Humdrum toolkit ( Rasas in Indian art (Rangacharya, 2010) Task 3: Music ontologies Building the ontologies needed for annotating the gathered collections. The music ontology specification ( (Raimond, 2007) Community-based ontologies (Mika, 2006) Knowledge management and metadata (Pachet, 2005) Task 4: Audio description Audio content analysis to describe the music collections chosen. Essentia & Gaia framework (MTG-UPF) Music transcription (Klapuri & Davy, 2006) Top-down and knowledge-based processing (in Klapuri & Davy, 2006) Computational auditory scene analysis (Wang & Brown, 2006) 4

5 Task 5: User profiling Characterization of users and communities, modelling their musical preferences and behaviours. Social Computing (Chai et al., 2010) Theory of music preferences (Rentfrow & Gosling, 2003) Task 6: Music interaction Interaction models by studying user behaviour in musical tasks. Cultural Computing (Nakatsu et al., 2010) Information Foraging Theory (Pirolli, 2007) Interactive Information Retrieval (Cole et al. 2005) Table-top interfaces (Reactable) Task 7: Music discovery Conclusions Active models and systems for culture-based music discovery. Collaborative creativity Online learning (Moh et al., 2008) Recommendation systems (Celma, 2009) Big and challenging!!!! But hopefully we can contribute with our music research to develop better IT for our multicultural world. 5

6 References (1 of 3) Celma, O Music Recommendation and Discovery in the Long Tail. PhD thesis. Cole, C., et al "Interactive information retrieval: Bringing the user to a selection state". in A. Spink & C. Cole (Eds.), New directions in cognitive information retrieval. Springer. Chai, S., et al. (Eds.) Advances in Social Computing. Springer. Gabrielsson, A Music Performance Research at the Millennium. Psychology of Music. Klapuri, A., Davy, M. (Eds.) Signal Processing Methods for Music Transcription. Springer. Leman, M Embodied music cognition and mediation technology. The MIT Press. Lerdahl, F Tonal Pitch Space. Oxford University References (2 of 3) Lesaffre, M Music Information Retrieval: Conceptual framework, Annotation and User Behaviour. PhD Thesis. Mika, P Ontologies are us: A unified model of social networks and semantics. Web Semantics: Science, Services and Agents on the World Wide Web 5, no. 1 (March): Moh, Y., Orbanz, P., Buhmann, J. M "Music Preference Learning with Partial Information". ICASSP Nakatsu, R. et al. (Eds.) Cultural Computing. Springer. Pachet, F Knowledge Management and Musical Metadata. Encyclopedia of Knowledge Management. Pirolli, P Information Foraging Theory: Adaptive Interaction with Information. Oxford, Oxford University Press. References (3 of 3) Raimond, Y A Distributed Music Information System. PhD Thesis. Rangacharya, A The Natyasastra. Munshiram Manoharlal Publishers. Rentfrow, P. J., Gosling, S. D The Do Re Mi's of Everyday Life: The Structure and Personality Correlates of Music Preferences. Journal of Personality and Social Psychology 84, no. 6: Wang, D. L. and Brown, G. J. (Eds.) Computational auditory scene analysis: Principles, algorithms and applications. IEEE Press/ Wiley-Interscience. 6