Music Representation and Music Information Retrieval

Size: px

Start display at page:

Download "Music Representation and Music Information Retrieval"

Juniper Shaw
5 years ago
Views:

1 Music Representation and Music Information Retrieval Nikoleta HABUDOVÁ* Slovak University of Technology Faculty of Informatics and Information Technologies Ilkovičova 3, Bratislava, Slovakia Abstract. Several computer file formats for musical notation and their storage and interchange have been developed in the world. One of them, text-based MusicXML format is robust, and user friendly, designed for interchange, analysis, and performance of musical information. It shows potential for being developed as a standard for musical notation interchange. Musical themes could be represented in formal and graph models and used in music information retrieval, which is mostly based on XML platform. 1 Introduction There have been many methods used to represent musical notes and information on computers. One method of representation to have emerged over recent years is the MusicXML file format, which is based on the conventions of the extensible Mark-up Language (XML) format. In this paper we present and describe several existing computer file formats for the representation of music. We show the possible formal and graph representation of melodic themes, which could be used in music information retrieval. 2 Music application formats The major technical problem was to desing a format that was complete enough for both commercial and academic use, while usable enough to be approachable for developers. Computers can represent music in two basic ways: * Doctoral degree study programme in field: Informatics Supervisor: Professor Pavol Návrat, Institute of Informatics and Software Engineering, Faculty of Informatics and Information Technologies STU in Bratislava IIT.SRC 2010, Bratislava, April 21, 2010, pp. 1 8.

2 2 Nikoleta Habudová An audio file represents music as sound in order to play recorded music. A symbolic file represents music in terms of musical concepts relevant to performers who read music. Converting symbolic files into audio format is important for classical composers who need to create demos in order to interest performers in their music. Symbolic files can also be converted into image/based formats like PDF and JPEG. These formats can display the printed music notation, but contain no music semantics. Converting audio files to symbolic format, on the other hand, has more in common with some of the most difficult problems in artificial intelligence. No application solve this problem satisfactorily at this time. Symbolic and audio formats are related, but they are not interchangeable. Many types of music applications use symbolic formats: Notation editors are used both to compose music and to prepare music for performance and publication. Music scanners convert printed music into symbolic format using optical music recognition (OMR), similar to converting printed text into computer text files using optical character recognition (OCR). Sequencers combine audio and symbolic formats for composers who work more independently of music notation. Sequencers focus on sound output. Digital music stands display music electronically, listen to the performance, and automatically turn pages as needed. Digital sheet music applications like are used to sell sheet music online [9]. 3 Audio and symbolic formats of music representation There are several audio and symbolic representations of music. 3.1 MIDI Music Instrument Digital Interface (MIDI) is an industry-standard protocol that enables electronic musical instruments to communicate, control, and synchronize with each other. MIDI does not transmit an audio signal or media it transmits "event messages" such as the pitch and intensity of musical notes to play. MIDI does not provide the necessary musical semantic information to be used as a comprehensive format for music representation. It is unsuitable for the precise notation of musical information [4][4]. Today, MIDI remains the only symbolic music interchange format in wide use [7]. 3.2 SMDL The creation purpose of Standard Music Description Language (SMDL) was to standardize musical information by means of the capabilities offered by SGML (Standard Generalized Markup Language) format.

3 Music Representation and Music Information Retrieval 3 SMDL is an instance of HyTime (Hypermedia Time/Based Description Language) and it inherits the functionality of this standard. It devides the aspects of music representation in four domains: Logical (cantus), Visual, Gestual and Analytical [11]. 3.3 NIFF Notation Interchange File Format (NIFF) is a music notation file format used primarily for transferring music notation between different scorewriters. The format is based upon RIFF (Resource Interchange File Format). 3.4 XML for music representation Extensible Markup Language (XML) is a subset of SGML. Its goal is to enable generic SGML to be served, received and processed on the Web in the way that is now possible with HTML. Logical structure of an XML document is a hierarchical structure with a system of hyperlinks. XML has obvious appeal as a technology to help solve the music interchange problem. It is designed to represent complex, structured data in standardized way. MusiXML is the format for music representation for exchanging musical score information. The structure is subdevided into three sections: Bibliographic data section, Logical data section and Filter data section. Authors of the paper [11] consider the structure of the MusiXML for incomplete. MusiCat is a catalogue oriented encoding format. The structure of a MusiCat-document is organised as a catalogue document and has four main sections: header, body, front and back. The body element contains the great part of the encoded information [11]. MusicXML attempts to do for online sheet music and music software what MIDI did for electronic musical instruments. By using XML, it is more Internet-friendly that proprietary binary formats. MusicXML serves as an interchange format for applications in music notation, music analysis, music information retrieval and musical performance. Therefore it augments, but does not replace, existing specialized formats for individual applications. It is designed to be adequate, not optimal, for these diverse applications [7][7]. More about MusicXML is in separate chapter. Other proposals There have been written several other musical representation proposals: Musical Notation Markup Language (MNML) Musical Markup Language (MML) Music Tagging Type Definition (MuTaTeD) MusicML

4 4 Nikoleta Habudová ChordML [11][11] Humdrum MuseData 4 MusicXML Company Recordare LLC, first started to develop MusicXML in 2000[4]. MusicXML is currently in version 2.0 as of January The main development goals were to create a file format for musical representation and interchange that would be Internetfriendly (easy to transport and use across Internet transfer mediums) and is a nonbinary format [4]. The design of MusicXML folowed two main strategies: 1. The design was based on two of the most powerful academic music formats for music notation: MuseData and Humdrum. Both formats have large music repertoires available, and have been used for diverse music applications. 2. The MusicXML definition was developed iteratively with MusicXML software [7]. Haus and Longari [11] consider that an XML DTD and an XML Schema definition of MusicXML are available. MusicXML Schema is richer than the MusicXML DTD because XML Schema language provides tools that XML DTD language does not. 4.1 Elements of MusicXML design Here is then musical equivalent of C's "Hello, world" program for MusicXML to show how MusicXML represents musical scores. The music file is the most simplest: one instrument, one measure, one note, a whole note on middle C. Figure 1. "Hello, world" in MusicXML. Here is the musical score represented in MusicXML: <?xml version="1.0" encoding="utf-8" standalone="no"?> <!DOCTYPE score-partwise PUBLIC "-//RECORDARE//DTD MusicXML 0.5 Partwise//EN" " <score-partwise> <part-list> <score-part id="p1"> <part-name>music</part-name>

5 Music Representation and Music Information Retrieval 5 </score-part> </part-list> <part id="p1"> <measure number="1"> <attributes> <divisions>1</divisions> <key> <fifths>0</fifths> </key> <time> <beats>4</beats> <beat-type>4</beat-type> </time> <clef> <sign>g</sign> <line>2</line> </clef> </attributes> <note> <pitch> <step>c</step> <octave>4</octave> </pitch> <duration>4</duration> <type>whole</type> </note> </measure> </part> </score-partwise> MusicXML can represent scores either partwise (measures within parts) or timewise (parts within measures), with XSLT stylesheets to go back and forth between the two. The attributes element contains musical attributes of score, such as the divisions, the key signature, time signature, and clef. After the attributes there is the one and only note in the score: a C in octave 4 [7][7]. 4.2 Structure The structure of the document is composed of individual score files (Score element) collected together within the structure defined in the Opus element. The Opus element allows direct and indirect arbitrary nesting of other Opus and Score elements. An XSLT program allows an automatic translation between the two orderings. The definition of DTD is split-up in several files, each one containing a modular part of the definition: Common definitions: elements and entities that are shared among multiple components (date formats, spacing units, position coordinates formats, embelischments structures and others); Score identifications: element definition of key and time signature, clefs, transpositions and other part's attribute elements;

6 6 Nikoleta Habudová Note: elements defining a consistent amount of property of the note, from the graphical representation to the logical aggregation between different parts, from articulations to performance infomation and associated lyric information; Barline: element defining the several types of barlines and their functionality like repeats, multiple ending etc... Directions and general purpose elements: contains elements describing several sign not directly related to the notes (e.g. segno, pedal, others), harmony structures (chords), printing and sound elements to specify general parameters associated to the rendering of the piece. The authors of the paper [11][11] consider the MusicXML as a sufficient interchange format for notation, analysis, retrieval and performance application. The content represented by the format is score oriented, e.g. the notes are represented as symbolic and graphical objects [11]. 4.3 Adoption MusicXML has succeeded in becoming the de facto interchange standard for symbolic music formats. MusicXML had achieved more success as a symbolic music interchange format than any prior effort besides MIDI. MusicXML in 2010 works with over 120 music applications, including all the market leaders for music notation editing and scanning [13]. 5 Formal and graph models in music information retrieval Musical themes are not pure sequences of bytes or symbols, they represent relationships among scale degrees. There are several formal and graph models of musical themes. Among formal models range functional and reductionistic approach. In functional model are musical themes treated like functions. In this context melodies are finite sequences, that is, function defined on a finite subset of N and whose values are in mono or n-dimensional spaces. Reductionistic approach exploits additional musicological information related to the piece (e.g., time signature) and assigns different levels of relevance to single notes of the melody. Graph model is typically geometric, is the study of a geometric object up to the specific transformations acting on it. Let M be a theme that we call the musical graph representing M. Musical graphs could be Eulerian, connected, oriented multigraphs. The melody is a sequence of intervals, so such a sequence represents an oriented trail in the graph which uses every edge once and once only. A musical theme M contains a series iff its representative graph G(M) is Hamiltonian. Let us consider the dodecaphonic series in Figure 2 [Schoenberg (1911)]. Figure 3 shows its representative graph, C 12, which is Hamiltonian. Cyclic graphs C n are a trivial example of series [12][12].

7 Music Representation and Music Information Retrieval 7 Figure 2. Dodecaphonic series [Schoenberg (1911)]. Figure 3. The cyclic graph C Conclusions There are many formats used for representation of music. Some represent the sound, another notation, some both of them. For the representation of musical notation seems to be the most suitable MusicXML format. XML platform offers enough possibilities for developing methods for music information retrieval. Common music information retrieval methods do not take into account the inner structure of melodic themes and the metric relationships between notes. Above mentioned formal or graph models of musical themes (building a representative graph for every theme and working with graphs instead of themes) supposed to be possible way for developing music information retrieval methods. Acknowledgement: This work was partially supported by the Scientific Grant Agency of Slovak Republic, grant No. VG1/0508/09.

8 8 Nikoleta Habudová References [1] Akerkar, R., Lingras, P.: Building an Intelligent Web, Theory and Praxis. USA: Jones & Barlett Publishers, Inc, 2008 [2] Baraté, A., Ludovico, L.A., Advanced Interfaces for Music Enjoyment. Proceedings of the working conference on Advanced visual interfaces, pp [3] Belliny, P., Della Santa, R., Nesi, P., Automatic Formatting of Music Sheets. Proceedings of the First International Conference on WEB Delivering of Music (WEDELMUSIC'01), p. 170 [4] Cunningham, S., Suitability of MusicXML as a Format for Computer Music Notation and Interchange, Proceedings of IADIS International Conference on Aplied Computing. Lisbon, Portugal, pp. III-7. [5] Cunningham, S., Gebert, N., Picking, R., Grout, V., Web-based Music Notation Editing. Proceedings of IADIS - International Conference on WWW/Internet, Murcia, Spain. [6] Extensible Markup Language 1.0. Available from: [7] Good, M., MusicXML: An Internet-Friendly Format for Sheet Music. Proceedings of XML 2001 International Conference. Orlando, USA. [8] Good, M., Actor, G., Using Music XML for File Interchange. Proceedings Third International Conference on Web delivering of Music. Leeds, UK, IEEE Press, Los Alamitos, Ca, pp [9] Good, M., Lessons from the Adoption of MusicXML as an Interchange Standard. Available from: [10] Gotoh, T., Minamikawa-Tachino R., Tamura, N., An Internet Environment for Braille Translation of Digital Music Scores. Proceedings of the 2008 Second International Symposium on Universal Communication, pp [11] Haus, G., Longari, M., Music Information Description by Mark-up Language within DB-Web Applications. First International Conference on Web Delivering of Music, Proceedings. IEEE Press, pp [12] Pinto, A., Haus, G., A Novel XML Music Information Retrieval Method Using Graph Invariants. ACM Transactions on Information Systems, Vol. 25, No. 4, Article 19. [13] Recordare MusicXML. Available from: [14] Watanabe, A., Katoh, T., Bista, B.B., Takata, T., On a Watermarking Scheme for MusicXML, Proceedings of the 20th International Conference on Advanced Information Networking and Applications (AINA 06), IEEE Press.

Representing, comparing and evaluating of music files

Representing, comparing and evaluating of music files Nikoleta Hrušková, Juraj Hvolka Abstract: Comparing strings is mostly used in text search and text retrieval. We used comparing of strings for music