Similarity matrix for musical themes identification considering sound s pitch and duration MICHELE DELLA VENTURA Department of Technology Music Academy Studio Musica Via Terraglio, 81 TREVISO (TV) 31100 Italy michele.dellaventura@tin.it Abstract: A large number of algorithms have been developed for the segmentation of a musical piece. The main aim is the identification of the themes (or motifs) that characterized a musical piece: motifs are never explicitly indicated by the composer of the score and this lack of indication gives to the very motif a mysterious character as if it were a secret that listening and analysis have the assignment to reveal. Even if identified by means of different research, the resulting segments typically have their composing notes disposed in a sequential manner (one after another), without any other note being interposed between them. A model of melodic analysis able to explore progressively the symbolic level of the musical text will be presented in this article, identifying, by starting from a list of previously found segments, the same motifs, which are yet concealed among the notes of melodic figurations. This approach was initiated at a melodic level, taking in consideration, together with the melody, the concept of rhythm as well. Key-Words: motif, musical surface, voice separation, similarity matrix, theme. 1 Introduction What triggers the analysis of a musical piece is essentially a pragmatic impulse: that is working something out on its own terms rather than on the terms of something else. Musical analysis focuses on a certain musical structure (that may be represented by same beats or by a phrase or by an entire composition), aiming at defining its constituents and explaining the way they operate. In a musical piece, one of the important elements is the theme which is the most identifiable and singable musical structure. The theme represents the fundamental motif, often a recurrent one, of a certain composition, especially if it is a far-reaching composition [1]: it is a melodic fragment endowed with individuality and recognizability, often to such an extent as to characterize the entire musical piece [2]. One of the main purposes of the automatic segmentation of the score is the identification of these motifs. The algorithms realized for such purpose are mainly based on two well-defined concepts: the psycho-receptive aspect and the repetitivity (recurrence). In the first case, the sonorous perception is seen as a filter of musical themes: referring therefore to the Generative Theory of Tonal Music proposed by Lerdahl and Jakendoff [3] based on the concept of functional hierarchy. The central hypothesis is that the listener, whose goal is to comprehend and memorize a tonal musical phrase, is trying to pin down the important elements of the structure by reducing what he is listening to a highly hierarchized economical scheme. Therefore the idea is that the listener is performing a mental operation of simplification that allows him not only to comprehend the complexity of the surface, but, when it is necessary, also to reconstruct such complexity starting from a simplified scheme and to produce other musical surfaces, other phrases of the same type, through a reactivation of the memorized structure. The study of these mechanisms lead to the construction of a formal grammar capable to describe the main rules observed by the human mind in order to recognize structures within a musical piece. The second aspect, an immediate consequence of the first, is the repetitivity: in order for the listener to recognize and memorize a certain sequence of notes, it is necessary to present the sequence several times [4]. The sequence must be clear, in its different manifestations, that is it must always be equally presented even if it is subject to non retrogradous translation or inversion (see figure 1) [5], insofar as retrogradous transformations make the theme hardly recognizable, most of all at the beginning, i.e. when the listener is not yet able to remember it well: if we were then to take into account the length of the theme, at times not even its continuous repetition in ISBN: 978-960-474-354-4 125
the original state, as it is the case with fugues, allows its recognition if retrograded. concealed motif. Section 4 shows some experimental tests that illustrate the effectiveness of the proposed method. Finally, conclusions are drawn in Section 5. Fig. 1: Bach's Two-Voice Invention in C Major, BWV 772. The sequence of notes (beat 1) is represented on the first staff, its transposition is represented on the second staff (beat 2) and its inversion on the third staff (beat 3). 2 The concealed motif The identification of a motif by the various algorithms, by virtue of the considerations in the preceding paragraph, occurs only if the notes making it up are sequential, that is one after the other (fig. 3). Fig. 3: Example of a melodic passage. Generally speaking, the musical theme (or motif) is a melody composed of sounds and a rhythm: a pitch (sound), in fact, never shows up by itself, but the succession of pitches is always organized in time (rhythm). The segmentation of the score is therefore dealt with by analyzing two distinct but united elements: the melody and the rhythm. Melody and rhythm are two fundamental components for the musical structuration, two almost inseparable components [6]: melody develops on the rhythm and without it, melody does not exist [7] (fig. 2). Fig. 2: Excerpt from the score of ''Bolero" by Ravel. The initial notes of the theme are represented on the first staff, without any indication of rhythm; the same notes with the rhythm assigned by the composer are represented on the second staff. Musical grammar, nevertheless, provides the composer with a series of tools allowing him to vary, within the same musical piece, an already presented melodic line, by inserting notes which are extraneous to harmony [8]. The sounds of a melodic line, in fact, may belong to the harmonic construction or may be extraneous to it. The former sounds, which fall in the chordal components, are called real, while the latter sounds, which belong to the horizontal dimension, take the name of melodic figurations (passing tones, turns or escape tones). They are complementary additional elements of the basic melodic material that lean directly or indirectly on real notes and also resolve on them. The use of melodic figurations, therefore, allows achieving greater freedom of the melody, bestowing upon it a better profile, yet at the same time making it hardly recognizable and, consequently, difficult to identify (fig. 4) [9]. Motifs are never explicitly indicated by the composer of the score and this lack of indication gives to the very motif a mysterious character as if it were a secret that listening and analysis have the assignment to reveal. On the basis of the last consideration, this article will present a model of melodic analysis able to explore the symbolic level of the musical text, identifying by starting from a list of previously found segments, the same motifs, which are yet concealed among the notes of melodic figurations. This paper is organized as follows. Section 2 describes the concealed motif. Section 3 describes the similarity matrix used to identify the + + + + Fig. 4: Comparison of two melodic passages. In the first staff, all the notes are real, while in the second staff, which is a variation of the first, the notes marked by the sign + do not belong to the harmonic structure and therefore are melodic figurations. Segmentation, as an analysis tool, does not only have to look for the single motifs, but also identify their position within the score while the non- ISBN: 978-960-474-354-4 126
indication of the concealed motifs would compromise the value of the very analysis. The model proposed in this article aims at presenting a research methodology based on the comparison of the pieces of information that every single fragment identified carries within: the pitch of every sound and its duration. 3 The similarity matrix In order to be able to search for the presence of motifs hidden inside the score, which were previously identified through segmentation, one must continue by building a similarity matrix for every single element [10]. Given a motif M with n sounds, the similarity matrix is defined as follows: A x, y where x (number of rows) equals the number of sounds of the motif M and y (the number of columns) equals 2 because the elements taken into consideration are two: the pitch of every sound and its duration. These two pieces of data, considered together, represent the coordinates of a point on the Cartesian plane, where the first sound will have the coordinates 0,0 [11]. The coordinates of the following sounds will be respectively: x the value corresponding to the number of semitones between the i-th note and the preceding one: this value will be respectively positive or negative depending on whether the note is higher or lower than the preceding note; y the values corresponding to the duration from the i-th note and the preceding one to the origin (the start). Since it is a score that must be analyzed, the duration of the sound will not be expressed in seconds but calculated (automatically by the algorithm) as a function of the musical sign (be it either a sound or a rest) having the smallest duration existing in the musical piece. The duration of every sign will therefore be a number (an integer) directly proportional to the smallest duration (fig. 5 and 6) [7]. which the value 1 is (automatically) associated: it follows that the quarter note will have the value 2. A 5,2 0 5 = 4 2 0 0 2 4 5 6 Fig. 6: Similarity matrix for the segment of figure 8. The first column displays the values corresponding to the number of semitones from the i-th note to the preceding note; the second column shows the values corresponding to the duration from the i-th note and the preceding one to the start. After having defined the similarity matrix for one single element, the algorithm continues with the exploration of the score from the beginning to the end note by note, considering every single note as the origin (coordinates 0, 0) of a new matrix B (havin the same dimensions as the matrix A) having in the second column exactly the same values of the matrix A (fig. 7). Fig. 7: Matrix B. B 5,2 0... =......... 0 2 4 5 6 The algorithm will fill in the data of the 1st column of the matrix, reading in the score the value corresponding to the number of semitones between the i- th note and the following one and after that the duration specified in the 2nd column of the i-th row. At the end of this procedure there will be a comparison between the first column of the two matrices (A and B) and if all the values are equal, an index will be drawn to indicate the point in which the segment is present within the score. Fig. 5: Melodic segment and its related graphic representation. In this example, the sign having the smallest duration is represented by the eighth note to 4 The results obtained The model of analysis set forth in this article was verified by realizing an algorithm the structure of which takes, most of all, in consideration each and every single aspect described above: the algorithm does not provide for any limitation with respect to the dimensions of the similarity matrix, but, on the ISBN: 978-960-474-354-4 127
contrary, it will be automatically dimensioned on the basis of the characteristics of every single previously identified element. Finally, one other important aspect considered in order to define the logical bases of operation of the algorithm is the nature of the data: aside from the difference of pitch among the different sounds that is defined considering the semitone as the absolute measurement unit, in the case of sound duration no predefined minimum value is provided; it will instead be calculated automatically on the basis of the smallest duration existing in every single musical piece analyzed (see the previous paragraph). Two examples of analysis are shown below (fig. 8 and 9). A) A 1 ) B) A) C) B) C) Fig. 8: A shows a melodic passage with its related graphic representation, an excerpt from J. Haydn's Trumpet Concert in E flat major, belonging to a list of segments that were previously identified by means of melodic analysis. In B there is a subsequent melodic passage in which it is possible to identify the same notes of the melodic segment A, alternating with melodic figurations. C displays the comparison between two melodic segments. Fig. 9: A shows a melodic passage out of Knecht Ruprecht belonging to the Album for the Youth by R. Schumann. A 1 shows the motif (together with its graphic representation) existing in the first staff (right hand). In B there is a preceding melodic passage in which it is possible to identify the same notes of the melodic segment A, alternating with melodic figurations. Unlike the previous example, the notes of the melodic figurations belong to the same harmonic structure of the real notes. C displays the comparison between the two melodic segments. 5 Conclusions This article has examined the notion of "motif" and the criteria for its identification within a certain score. Then, one of the potential melodic analysis problems was exposed: the recognition of a motif hidden among the notes of the musical piece. This problem was dealt with by using an important mathematical tool, namely the matrix, which allowed us to represent a motif in its two constitutive elements: melody and rhythm. The fundamental assumption, considered for this type of analysis, is the existence of a list of motifs, previously elaborated by an algorithm and for this reason, this research presents itself not as an ISBN: 978-960-474-354-4 128
autonomous program, but as an integration to the current systems of melodic segmentation of a score. Extending this methodology to the harmonic analysis of a musical piece might help us identify motifs that develop through the passage from a voice to another voice. References: [1] M. Della Ventura, Analysis of algorithms implementation for melodical operators in symbolical textual segmentation and connected evaluation of musical entropy, in In Proceedings of the International Conference on Mathematics (IAASAT 11) (pp. 66-73). Drobeta Turnu Severin, Romania. [2] M. Della Ventura, L impronta digitale del compositore, GDE, Italy, 2010. [3] F.Lerdhal, R. Jackendoff, A Generative Theory of Tonal Music, The MIT Press, 1983. [4] O. Lartillot-Nakamura, Fondements d un système d analyse musicale computationnelle suivant une modélisation cognitiviste de l écoute, Doctoral Thesis, University of Paris, 2004. [5] U. Hahn, M. Ramscar, Similarity and Categorization, Oxford University Press, Oxford (2001). [6] C. Orff, Schulwerk, elementare Musik, Hans Schneider, Tutzing, 1976. [7] M. Della Ventura, Rhythm analysis of the sonorous continuum and conjoint evaluation of the musical entropy, in In Proceedings of the International Conference on Acoustic & Music: Theory & Applications (AMTA 12) (pp. 15-21). Iasi, Romania. [8] B. Coltro, Lezioni di armonia complementare, Zanibon, 1979. [9] S. Ahlback, Melody beyond notes: A study of melodic cognition, Ph.D. thesis, Goteborgs Universitet, Sweden, 2004 [10] S.N. Nikolskij, Corso di Analisi Matematica, Edizioni MIR. [11] J. P. Hornak, The Basics of MRI, 1996. ISBN: 978-960-474-354-4 129