ETHNOMUSE: ARCHIVING FOLK MUSIC AND DANCE CULTURE

Transcription

1 ETHNOMUSE: ARCHIVING FOLK MUSIC AND DANCE CULTURE Matija Marolt, Member IEEE, Janez Franc Vratanar, Gregor Strle Abstract: The paper presents the development of EthnoMuse: multimedia digital library of Slovenian folk music and dance culture. The main scope of the project concerns the digitization of production and post-production processes that relate to collecting, documenting and archiving of folk heritage and development of multimedia applications for various content types (folk song, music and dance) and formats (image, audio, video, notation, MIDI etc.). The main objective of this paper is to discuss the former, focusing on the conceptual design of a flexible data model the archive is based on. We also briefly describe the tools developed to support the workflow of researchers involved in collecting and archiving of folk music related contents. Index Terms: digital libraries, folk music, FRBR, CIDOC CRM. I. INTRODUCTION The Institute of Ethnomusicology is the oldest institute of the Scientific Research Centre of the Slovene Academy of Sciences and Arts; it was founded in The founding charter of the Institute includes among its basic functions the 'compiling of as complete as possible a collection of Slovenian musical folklore'. In line with basic folkloristic and ethnomusicological research, the main activities of the Institute are monitoring contemporary activities relating to folk music and dance traditions and professional intervention in those areas where folklorism draws on the heritage of folk song, music and dance. Further, vital part of research is focused on collection, preservation, research and publication of gathered Slovenian folk song, music and dance collections, and construction of the EthnoMuse [1] digital archive. The Institute s archive also contains diverse records of folk songs and music of other nations and various documentary field recordings. Since its founding, the Institute has built a large archive that includes content in various forms: folk song manuscripts with song transcripts (around items starting from 1906); audio recordings of folk songs and melodies, with the oldest on wax cylinders from 1914 and around field recordings on magnetic tape and digital media dating from 1955 onwards. The collection includes detailed metadata of recording processes, transcriptions, places of origin, themes, first verses, beginnings of the melodies, meters etc. visual documentation, including a photo archive and a collection of video documents. The goal of the EthnoMuse project is to assist with the digitization of the entire archive, provide tools for searching the archive and support the ongoing production and postproduction processes that relate to collecting, documenting and archiving Slovenian folk songs, music and dances. In order to make the EthnoMuse valid as a basis and a tool for scientific research work at the Institute, specific concepts and methods that apply to research of folk song, music and dance had to be implemented. This is due to the fact that scientists and researchers of the Institute of Ethnomusicology are active contributors (makers of the archive) and primary users of the EthnoMuse at the same time. On one hand, the nature of digitized material (folk song, music and dance) demands implementation of specific concepts and relationships into the underlying data model to enable effective description of content. On the other hand, these descriptions cannot be isolated from the production processes that this material depends on; they cannot exist outside the events of their creation (collecting, recording), documentation and archiving. They require a flexible data model and implementation of both, CIDOC Conceptual Reference Model (CIDOC CRM) [2] and Functional requirements for bibliographic records (FRBR_er) [3]; thus enabling a formal structure for description of implicit and explicit concepts and relationships between concepts and events of production processes mentioned. In the paper, we first propose a flexible data model that meets these requirements and then describe some applications built on top of this model. II. THE FLEXIBLE DATA MODEL It is in the nature of a folk song to be performed and mediated verbally. This has some major consequences: Non-performing arts products are physical objects that carry conceptual objects; such physical objects are the basic documents that are preserved in a collection and described in a catalogue; through such documents, the conceptual objects they carry are preserved and described as well; it is also possible to gather documentation about them, their creation, and the way they were perceived. Performing arts products are events that convey conceptual objects; there is therefore no basic document, but only some documentation, which, consequently, becomes primordial, as the conceptual object conveyed by a show can be somewhat preserved and described only by preserving and describing that documentation [4]. Audio recordings remain the basic scientific source for folkloristic research [5] and a core feature

2 of folk song representation. Since there is no document, the representation of a particular folk song in a recorded variant is only the representation of its physical characteristics audio documentation of the event of singing, not (of) the event/performance itself. Because of this strong interdependence, the concept of event represents a core feature of the conceptual design of the proposed flexible data model. It serves as an association between a recorded variant, a person/actor involved (performer, recording team etc.) and place and time of the recording session. Based on CIDOC CRM, the concept of event enables the representation and integration of various production (i.e. field recording session) and post-production processes through time. While the conceptual structure of CIDOC CRM proved to be efficient for modeling events, such as production and post-production processes, it doesn t support complex folkloristic descriptions of folk song and music variants. Entities and relationships that form between the performed folk song and its recorded variant (as the embodiment of the folk song s immaterial nature) in the event of recording go well along the lines of the FRBR_er model. Every new performance creates a new variant that belongs to a particular folk song variant type Work [3]. Following FRBR_er structure, this Work is represented by Expression, embodied in Manifestation (recorded variant) and carried by an Item (carrier). Such a framework represents the hierarchical structure and description of content on various levels. The proposed data model is therefore a combination of two conceptual models, with the aim to enable: 1. representation of entities and relationships describing folk song and music, based upon FRBR_er, especially 1st group entities [3]: Work, Expression, Manifestation and Item; and 2. representation of production and postproduction processes based on the CIDOC CRM concept of Event. Fig. 1 - The data model Fig. 2 - Mapping a recording session in CIDOC CRM Combining ideas of both conceptual models, FRBR_er and CIDOC CRM, Fig. 1 represents the basic framework of the proposed data model. As Fig. 1 shows, the production of an individual audio recording and description of the recorded variant are (in the data model) linked with the overall documentation (field record) of a production event. This way, not only is each individual recording session associated with other recording sessions, but FRBR_er entities describing recorded variants are associated as well. Each Manifestation is indirectly linked with the production event (individual recording session) through the Item (audio recording) - the information carrier of the recorded variant produced in a particular production event. Consequently, performers (singers, musicians etc.) belong to the Manifestation level (of recorded variant), while recording team, transcribers, collectors belong to the production event level (particular recording session). A. Mapping to FRBR_oo: Issues The existing combination of FRBR_er and CIDOC CRM represented in the flexible data model (Fig. 1) doesn t correspond to the formal definition of FRBR_er nor CIDOC CRM: the initiative of combining both models is represented in the current version of FRBR_oo [6], prepared by the International Working Group on FRBR/CIDOC CRM Harmonisation. At the time of construction of the flexible data model, FRBR_oo was still in its initial stage, and the combination of FRBR_er and CIDOC CRM seemed like a logical solution (Fig. 1). Initially, the data model was founded on CIDOC CRM, mainly because of our need to represent production and postproduction processes, as well as audio recordings thus created. We ve mapped FRBR_er onto existing

3 Fig. 3 - Flexible data model (FRBR_oo version) CIDOC CRM structure mostly because the E73 Information Object (see Fig. 2) is insufficient (and too broad) for representing complex bibliographic relationships that arise from folkloristic descriptions of folk music, its numerous variants and carriers. In the attempt to map the existing data model (Fig. 1) onto FRBR_oo, some changes had to be made, especially due to the way FRBR_oo interprets Manifestation. While in FRBR_er Manifestation can be interpreted as a representation of either material or immaterial (an Item or Information Object), in FRBR_oo Manifestation is split: into two distinct classes, corresponding to the two possible ways of interpreting the ambiguous definition provided for Manifestation in FRBR ER, namely F3 Manifestation Product Type and F4 Manifestation Singleton. Whereas F3 Manifestation Product Type is declared as a subclass of the CIDOC CRM class E55 Type, and therefore as a subclass, too, of the CIDOC CRM class E28 Conceptual Object (a merely abstract notion), F4 Manifestation Singleton is declared as a subclass of the CIDOC CRM class E24 Physical Man-Made Thing, and therefore as a subclass, too, of the CIDOC CRM class E18 Physical Thing. [6] Following above, the F3 Manifestation Product Type represents the immaterial level (subclass of E28 Conceptual Object), while F4 Manifestation Singleton represents the material level (a subclass of E18 Physical Thing). In the data model (Fig. 1) the distinction between an audio recording (Manifestation) and concrete information carrier (Item) is kept. This follows the initial FRBR_er interpretation of Manifestation and Expression as the set of concepts expressed in one particular set of signs, independently of the materialisation of that set of signs [3]. Consequently, the Expression level conveys different realisations or communicative forms of a particular folk song variant sung, instrumental or spoken, whereas their materialisation is modelled on the Manifestation level (in form of an audio recording, melody and/or text transcription etc.). Individual information carriers with their unique IDs are represented on Item level. In FRBR_oo the original audio recording is defined as a manuscript, thus F4 Manifestation Singleton overlaps with an Item. However, mapping of the flexible data model onto FRBR_oo solved some inconsistencies of existing model. First, the creation of a folk song and music variant is now modeled simultaneously with its materialization the production of the original audio recording. As already mentioned, an audio recording represents the most authentic documentation of a particular variant, while other forms of variant representations, such as text and melody transcriptions, are considered as variant s partial realizations. Further, the production of an audio recording and the production of a transcription represent two different events, the latter usually based on the analysis of the former. The new flexible data model (based on FRBR_oo) enabled these conceptual distinctions as shown in Fig. 3. Such a combination arose mainly due to the existing audio-archiving practices, where documentation and preservation of

4 Fig. 4 - Manipulating manuscripts in EthnoMuse audio recordings on various carriers, as well as techniques and procedures used in production and post-production events, is of greatest importance. III. DEVELOPING ETHNOMUSE TOOLS The data model presented in the previous section represents the core upon which a set of tools to be used by researchers at the Institute was built. The first tools developed in the course of the EthnoMuse project were tools for storage and manipulation of digitized contents and its metadata, based on the data model. We decided to use a modern commercial development platform that is well proven in the software industry, so we based the software on the Microsoft.NET framework and Microsoft s SQL Server relational database. This gave us a set of good development tools, as well as a very flexible platform for future enhancements. Support for various content types was added in stages; manuscripts were supported first, followed by audio recordings and finally images and dances. A. Manuscripts Manuscripts represent the oldest items in the archive. A manuscript is a handwritten document of a song s lyrics and musical notation, as well as some accompanying metadata on singers. The handwriting used is far too difficult for current OMR and OCR tools, so the collection was first scanned and then manually transcribed. Sibelius software was used to input music notation, because its easy to use features were well appreciated by the researchers at the Institute. Scores are stored in Sibelius native format, as well as in MusicXML [7]. MusicXML is a platform independent score representation format and is used for further parsing and score processing by query-bymelody and other applications, while the native format enables simple score visualization and playback in desktop and web applications with Sibelius Scorch plugin. Next to scores and scans of original manuscripts, a large amount of metadata is also stored in the database, ranging from musical properties such as meter, tempo and melodic structure to genre, variant and content types, geographical characteristics and performers. Fig. 4 shows a screenshot of the application for manipulation of manuscripts in the archive; the scanned manuscript, its Sibelius transcription as well as some of the metadata describing each item are shown. B. Field Recordings Field recordings represent the largest part of the archive s contents. Researchers are still recording new materials, so support had to be added for archival, as well as collection and annotation of new recorded materials. As recordings are also being published on CDs and used in various other projects, we are also adding support for postproduction processes, which will enable researchers to trace a recording from its initial capture over all postproduction steps (copying, editing, mastering...) to its final issued form. In addition to audio, field recordings are accompanied by a written report that contains recording metadata, which are also included as part of the field recording support built into the EthnoMuse. Field recordings are typically several hours long, as they include all of the audio from a recording session. Audio may be split across several carriers

5 (i.e. magnetic tapes, DATs or mp3 files). In our proposed data model. we defined the entire recording session as an recording event. The tools therefore support metadata related to such an event, including names of field researchers, technical properties of recording devices, microphone placement, etc. All of these metadata are stored in the database to enable efficient indexing and searching. The digitized audio of a recording session resides on external storage and is linked to the database record of a session by a URI. Field recordings available to researchers for listening and annotation are digitized and downsampled from their original media to 128 kbps mp3 streams. Each field recording is segmented into smaller items (e.g. speech, songs, etc.) by placing markers within the recording. We developed semiautomatic annotation and labeling tools to help researchers with this process. Individual items have properties similar to manuscripts, including Sibelius transcriptions. C. Visual Documents The application also supports visual documents. Images, including scanned photos and digital images, as well as videos of folk dances, including specific metadata and Labanotation scores are supported. D. Search and Retrieval While tools for archive maintenance are only intended to be used by researchers at the Institute, searching and browsing (parts of) the archive will be made available to a larger audience. We have therefore decided to build a web interface to the archive, which enables browsing, as well as searching through the archive s contents. Full-text search of textual metadata is supported, as well as melody-based searching, which is currently implemented with standard n-gram techniques [8]. Manuscripts have been converted to pdf format and Sibelius scores to MIDI to enable users to use standard tools while searching and browsing the archive. The web interface is built on AJAX and ASP.NET architectures and carries several resemblances to the digital archive. They both gather data from the same database, but due to the large amount of metadata residing in the EthnoMuse archive, the web application only displays the most common properties such as the contributor, genre and various classifications. At this stage, support is limited to manuscripts, but our future aim is to give support to other content types as well. IV. CONSLUSION AND FUTURE WORK The EthnoMuse digital archive represents a solid background for further folkloristic research into folk song and music. While the strength of the proposed flexible data model lies in a robust framework for representation of various content types and production and post-production processes mentioned, an added value lies in its FRBR_oo and CIDOC CRM formal conceptual structure, which makes the integration of the EthnoMuse archive with similar national and international (also European) archives more feasible. The main future emphasis will be on the development of tools for semantic search, analysis, representation and visualization of this content, especially in relation to music information retrieval (MIR) research. REFERENCES [1]. EthnoMuse: multimedia digital archive of Slovenian folk, music and folk dance culture. Slovenian Government Founded R&D Project, [2]. ICOM/CIDOC Documentation Standards Group; & CIDOC CRM Special Interest Group. Definition of the CIDOC Conceptual Reference Model: version 4.0, April [Heraklion, Greece]: [ICS-FORTH], Available online at: [3]. IFLA Study Group on the functional requirements for bibliographic records. Functional requirements for bibliographic records: final report [printed text]. Munich, Germany: K. G. Saur, Also available online: ( [4]. Patrick Le Boeuf, "... That struts and frets his hour upon the stage and then is heard no more." : the elements that should be accounted for in a conceptual model for performing arts and the information relating to their archives, Workshop at the Centre de documentation de la musique contemporaine, Paris, France, January 13th, Available as: cidoc.ics.forth.gr/docs/2006_leboeuf_eng.pdf [5]. D. Schüller, ''Methodik und Technik der phonographischen Feldforschung''. In: Deutsch, W. and Walcher, M. (Ed.), Sommerakademie Volks-kultur 1993 (p ). Wien: Volkslied, 1994 [6]. FRBR object-oriented definition (draft version 0.9). International Working Group on FRBR and CIDOC CRM Harmonisation, ( [7]. M. Good, MusicXML: An Internet-Friendly Format for Sheet Music, XML Conference Proceedings, Orlando Florida, [8]. J. Stephen Downie, Evaluating a Simple Approach to Music Information Retrieval: Conceiving Melodic N- Grams as Text, Ph.D. Thesis, University of Western Ontario, Canada, July Matija Marolt is assistant professor with the Faculty of Computer and Information Science, University of Ljubljana, Trzaska 25, 1000 Ljubljana, Slovenia; ( matija.marolt@fri.uni-lj.si). His research interests are in the field of music information retrieval. Janez Franc Vratanar is researcher with the Faculty of Computer and Information Science, University of Ljubljana, Trzaska 25, 1000 Ljubljana, Slovenia; ( janez.vratanar@fri.uni-lj.si). His research interests are in the field of music information retrieval. Gregor Strle is researcher with the Institute of Ethnomusicology of Scientific Research Centre of the Slovenian Academy of Sciences and Arts, Novi Trg 2, 1000 Ljubljana, Slovenia ( grega.strle@zrc-sazu.si). His research interests are in the field of conceptual spaces and representations.