CONTINUOUS WAVELET-LIKE TRANSFORM BASED MUSIC SIMILARITY FEATURES FOR INTELLIGENT MUSIC NAVIGATION

Similar documents
A QUERY BY EXAMPLE MUSIC RETRIEVAL ALGORITHM

INTER GENRE SIMILARITY MODELLING FOR AUTOMATIC MUSIC GENRE CLASSIFICATION

HUMAN PERCEPTION AND COMPUTER EXTRACTION OF MUSICAL BEAT STRENGTH

Robert Alexandru Dobre, Cristian Negrescu

MUSI-6201 Computational Music Analysis

TOWARD AN INTELLIGENT EDITOR FOR JAZZ MUSIC

Automatic Rhythmic Notation from Single Voice Audio Sources

Drum Sound Identification for Polyphonic Music Using Template Adaptation and Matching Methods

Computational Models of Music Similarity. Elias Pampalk National Institute for Advanced Industrial Science and Technology (AIST)

Tempo and Beat Analysis

Instrument Recognition in Polyphonic Mixtures Using Spectral Envelopes

The song remains the same: identifying versions of the same piece using tonal descriptors

IMPROVING RHYTHMIC SIMILARITY COMPUTATION BY BEAT HISTOGRAM TRANSFORMATIONS

THE importance of music content analysis for musical

OBJECTIVE EVALUATION OF A MELODY EXTRACTOR FOR NORTH INDIAN CLASSICAL VOCAL PERFORMANCES

Outline. Why do we classify? Audio Classification

Music Radar: A Web-based Query by Humming System

Music Recommendation from Song Sets

International Journal of Advance Engineering and Research Development MUSICAL INSTRUMENT IDENTIFICATION AND STATUS FINDING WITH MFCC

POST-PROCESSING FIDDLE : A REAL-TIME MULTI-PITCH TRACKING TECHNIQUE USING HARMONIC PARTIAL SUBTRACTION FOR USE WITHIN LIVE PERFORMANCE SYSTEMS

EE391 Special Report (Spring 2005) Automatic Chord Recognition Using A Summary Autocorrelation Function

MUSICAL INSTRUMENT IDENTIFICATION BASED ON HARMONIC TEMPORAL TIMBRE FEATURES

Statistical Modeling and Retrieval of Polyphonic Music

Normalized Cumulative Spectral Distribution in Music

Automatic music transcription

A PERPLEXITY BASED COVER SONG MATCHING SYSTEM FOR SHORT LENGTH QUERIES

Automatic Laughter Detection

CSC475 Music Information Retrieval

Topic 10. Multi-pitch Analysis

Subjective Similarity of Music: Data Collection for Individuality Analysis

DAY 1. Intelligent Audio Systems: A review of the foundations and applications of semantic audio analysis and music information retrieval

APPLICATIONS OF A SEMI-AUTOMATIC MELODY EXTRACTION INTERFACE FOR INDIAN MUSIC

ON FINDING MELODIC LINES IN AUDIO RECORDINGS. Matija Marolt

Tempo and Beat Tracking

Music Source Separation

Topics in Computer Music Instrument Identification. Ioanna Karydi

Supervised Learning in Genre Classification

Music Similarity and Cover Song Identification: The Case of Jazz

2. AN INTROSPECTION OF THE MORPHING PROCESS

A prototype system for rule-based expressive modifications of audio recordings

Automatic Commercial Monitoring for TV Broadcasting Using Audio Fingerprinting

AUTOREGRESSIVE MFCC MODELS FOR GENRE CLASSIFICATION IMPROVED BY HARMONIC-PERCUSSION SEPARATION

Classification of Timbre Similarity

Music Segmentation Using Markov Chain Methods

Reducing False Positives in Video Shot Detection

Semi-supervised Musical Instrument Recognition

Music Database Retrieval Based on Spectral Similarity

Improving Frame Based Automatic Laughter Detection

Computational Modelling of Harmony

Efficient Vocal Melody Extraction from Polyphonic Music Signals

Recognising Cello Performers using Timbre Models

Transcription of the Singing Melody in Polyphonic Music

Automatic Piano Music Transcription

Chord Classification of an Audio Signal using Artificial Neural Network

2 2. Melody description The MPEG-7 standard distinguishes three types of attributes related to melody: the fundamental frequency LLD associated to a t

Automatic Music Clustering using Audio Attributes

Automatic Labelling of tabla signals

Query By Humming: Finding Songs in a Polyphonic Database

Repeating Pattern Extraction Technique(REPET);A method for music/voice separation.

GCT535- Sound Technology for Multimedia Timbre Analysis. Graduate School of Culture Technology KAIST Juhan Nam

Keywords Separation of sound, percussive instruments, non-percussive instruments, flexible audio source separation toolbox

Music Genre Classification and Variance Comparison on Number of Genres

SINGING PITCH EXTRACTION BY VOICE VIBRATO/TREMOLO ESTIMATION AND INSTRUMENT PARTIAL DELETION

Speech To Song Classification

MUSICAL INSTRUMENT RECOGNITION WITH WAVELET ENVELOPES

POLYPHONIC INSTRUMENT RECOGNITION USING SPECTRAL CLUSTERING

Speech and Speaker Recognition for the Command of an Industrial Robot

Voice & Music Pattern Extraction: A Review

Melody Extraction from Generic Audio Clips Thaminda Edirisooriya, Hansohl Kim, Connie Zeng

Data Driven Music Understanding

Hidden Markov Model based dance recognition

6.5 Percussion scalograms and musical rhythm

HUMMING METHOD FOR CONTENT-BASED MUSIC INFORMATION RETRIEVAL

Classification of Musical Instruments sounds by Using MFCC and Timbral Audio Descriptors

Onset Detection and Music Transcription for the Irish Tin Whistle

A CLASSIFICATION APPROACH TO MELODY TRANSCRIPTION

Melody Retrieval On The Web

Week 14 Music Understanding and Classification

Application Of Missing Feature Theory To The Recognition Of Musical Instruments In Polyphonic Audio

An ecological approach to multimodal subjective music similarity perception

EVALUATION OF FEATURE EXTRACTORS AND PSYCHO-ACOUSTIC TRANSFORMATIONS FOR MUSIC GENRE CLASSIFICATION

Singer Traits Identification using Deep Neural Network

Week 14 Query-by-Humming and Music Fingerprinting. Roger B. Dannenberg Professor of Computer Science, Art and Music Carnegie Mellon University

Audio-Based Video Editing with Two-Channel Microphone

AN ARTISTIC TECHNIQUE FOR AUDIO-TO-VIDEO TRANSLATION ON A MUSIC PERCEPTION STUDY

WHAT MAKES FOR A HIT POP SONG? WHAT MAKES FOR A POP SONG?

Methods for the automatic structural analysis of music. Jordan B. L. Smith CIRMMT Workshop on Structural Analysis of Music 26 March 2010

Polyphonic Audio Matching for Score Following and Intelligent Audio Editors

Piano Transcription MUMT611 Presentation III 1 March, Hankinson, 1/15

UNIVERSITY OF DUBLIN TRINITY COLLEGE

Content-based music retrieval

Content-based Music Structure Analysis with Applications to Music Semantics Understanding

Music Complexity Descriptors. Matt Stabile June 6 th, 2008

Automatic Laughter Detection

Efficient Computer-Aided Pitch Track and Note Estimation for Scientific Applications. Matthias Mauch Chris Cannam György Fazekas

FULL-AUTOMATIC DJ MIXING SYSTEM WITH OPTIMAL TEMPO ADJUSTMENT BASED ON MEASUREMENT FUNCTION OF USER DISCOMFORT

MODELING RHYTHM SIMILARITY FOR ELECTRONIC DANCE MUSIC

Effects of acoustic degradations on cover song recognition

CLASSIFICATION OF MUSICAL METRE WITH AUTOCORRELATION AND DISCRIMINANT FUNCTIONS

STRUCTURAL CHANGE ON MULTIPLE TIME SCALES AS A CORRELATE OF MUSICAL COMPLEXITY

Transcription:

CONTINUOUS WAVELET-LIKE TRANSFORM BASED MUSIC SIMILARITY FEATURES FOR INTELLIGENT MUSIC NAVIGATION Aliaksandr Paradzinets 1, Oleg Kotov 2, Hadi Harb 3, Liming Chen 4 Ecole Centrale de Lyon Departement MathInfo { 1 aliaksandr.paradzinets, 3 hadi.harb, 4 liming.chen}@ec-lyon.fr, 2 elfsoft@tut.by ABSTRACT Intelligent music navigation is one of the important tasks in today s music applications. In this context we propose several high-level musical similarity features that can be used in automatic music navigation, classification and recommendation. The features we propose use Continuous Wavelet-like Transform as a basic time-frequency analysis of a musical signal due to its flexibility in timefrequency resolutions. A novel 2D beat histogram is presented in the paper as a rhythmic similarity feature which is free from dependency on recording condition and does not require sophisticated adaptive algorithms of threshold finding in beat detection. This paper also describes a CWT based algorithm of multiple F0 estimation (note detection) and corresponding melodic similarity features). Evaluation of the both similarity measures is done in automatic genre classification context and playlist composition. 1. INTRODUCTION Similarity-based music navigation is becoming crucial for enabling easy access to the always-growing amount of digital music available to professionals and amateurs alike. A professional user, such as a radio programmer, may want to search for a different interpretation of one song to include in a radio playlist. Also, a radio programmer has the need to discover new songs and artists to help his listeners to discover new music. The music amateur on the other hand has different needs, ranging from music discovery for the geeks, to the simple seed song playlist generation of similar items. In this context we present in this paper different similarity measures capturing the ryhtmic and the melodic information. The algorithms are evaluated using two similarity experiments: genre classification and reinterpreted compositions search. For all our features we use algorithms based on Continuous Wavelet Transform (CWT). There are actual works which propose a series of various acoustic measurements to catch the different aspects of perceptive similarity of the music [1][2][3][4]. But the difficulty is always that the perceptive similarity is semantic and holds a good part of subjectivity. The rhythmic aspect of a music piece may be considered as a crucial component in a perceptive similarity measure. Considering the problem of rhythmic similarity we can refer to the following works. [5] describes a method for describing the rhythm and tempo of music as well as a rhythmic similarity measure based on the beat spectrum. Another work [6] presents the similarity of rhythmical patterns. In the current paper one of acoustic features we propose is a representation of rhythmic image. Extraction of melodic characteristics is based on automatic approximate transcription. The question of automated music transcription is the question of multiple F0 (pitch) estimation. Being a very difficult problem and being not resolved for the general case it is widely addressed in the literature. Lots of works are dedicated to the monophonic case of pitch detection in singing/speech [7][8]. The polyphonic case is usually considered with a number of limitations like the number of notes played simultaneously or an assumption of instruments involved [9][10]. The general case, for example, CD recordings [11] remains less explored. 2. ACCOUSTIC SIMILARITY FEATURES The most known acoustic characteristics generally used for audio similarity measures are Mel Frequency Cepstral Coefficients (MFCC). In a previous work [12] we have proposed the use of the statistical distribution of the audio spectrum to build feature vectors in what we call the Piecewise Gaussian Modeling (PGM) features. PGM features constitute an interesting alternative for the MFCC features. In this paper we propose several new acoustic features 2D beat histogram and note succession histogram. Unlike simple spectral features, these new

measures take into account semantic information such as rhythm, tonality etc. ( ) m a = L k e 2 k a max 1 (2) 2.1. Continuous wavelet transform vs. FFT The Fast Fourier Transform and the Short-Time Fourier Transform have been the traditional techniques in signal analysis for detecting pitches. However, the frequency and time resolution is linear and constant across the frequency scale (Figure 1) while the frequency scale of notes as well as human perception of a sound is logarithmic. Figure 1. Time-frequency resolution of the Fourier Transform The rule of calculating the frequencies of notes is wellknown. So if we consider a frequency range for different octaves, it is growing as the number of octave is higher. Thus, to cover well with frequency grid the wide range of octaves large sized windows are necessary in the case of FFT; this affects the time resolution of the analysis. On the contrary, the use of small windows makes impossible to resolve frequencies of neighboring notes in low octaves. The Continuous Wavelet Transformation (CWT) was introduced 15 years ago in order to overcome the limited time-frequency localization of the Fourier-Transform (FFT) for non-stationary signals and was found to be suitable in a lot of applications [13]. Unlike the FFT, the Continuous Wavelet Transformation has a variable timefrequency resolution grid with a high frequency resolution and a low time resolution in low-frequency area and a high temporal/low frequency resolution on the other frequency side. In that respect it is similar to the human ear which exhibits similar time-frequency resolution characteristics [14]. Also the scale of frequencies can be chosen as logarithmic which fits well for the note analysis. In that case the number of frequency bins is constant for each octave. In our works an experimental wavelet-like function with logarithmic frequency scale was used to follow the musical note system: ψ Frequency ( x, a ) =, m( a ) Time resolution H x e jw( a )x where a relative scale of wavelet, H(x,m) function of Hanning window of length m: (1) a a + 1 ( a ) = L L w (3) max min Here k 1, k 2 time resolution factors, L max and L min range of wavelet absolute scales. We have chosen this function because it has elements of windowed Fourier transform (with Hanning window) and classical wavelets. The frequency scale here is always logarithmic while the time resolution scale can be adjusted to be from liner to logarithmic. Time/frequency scale of the transform is shown on the Figure 2. Frequency Time resolution Figure 2. Time-frequency resolution of the Transform used in our work The use of CWT, however, has a negative point of costly computations. 2.2. Beat detection and rhythmic similarity features Existing approaches of beat detection can be divided into some sort of general classes, such as signal energy envelope-based, signal autocorrelation-based [15][16], based on a mathematical model of resonators set [17], non-linear resonators [18] etc. In the work [17] there is a comparison of autocorrelation and resonators set approaches in tempo and beat analysis. An approach of beat detection using image treatment techniques is presented in [19]. These methods are aimed to detect periodicities in order to determine the main tempo of the composition (in BPM). However a global view on the rhythmical image or pattern is required. One of such view is the beat histogram, proposed by [20]. Example of application of musical knowledge in the issue of beat tracking is given in [21]. Theoretical aspects of rhythmic similarity issue are mentioned in [22] The beat/onset detection algorithm being described in this paper is based on Continuous Wavelet-like Transform described in the previous chapter. An example of the wavelet representation of a musical excerpt is depicted on Figure 4.

Figure 4. Wavelet representation of musical excerpt. Since the information about beats and onsets is assumed to be concentrated in vertical constituent of wavelet spectrogram, an image treatment technique can be applied to mark out all fragments in this spectral image connected with beats and onsets. Usage of image treatment technique has been described in literature by few works. In their work [19] the authors apply edge enhancement filter on the Fast Fourier Transform (FFT) image in the preprocessing phase. In the current work, preliminary experiments with wavelet spectrum showed good results with the use of Sobel X operator. The result of an enhancement by Sobel operator is depicted on Figure 5. All beats in that musical excerpt are now clearer. Numerous beat curves may be computed separately by dividing the spectrum into bands. For the general question of beat detection the only one beat curve is used. The probable beats are situated in beat curve s peaks. However, the definition of a threshold for beat detection is problematic. Adaptive and none-adaptive algorithms for peak detection may be unstable. Many weak beats can be missed while some false beats can be detected. Later it is shown how we can overcome this difficulty in a manner which is compatible to our aim: rhythmical music similarity. We have applied the same technique of image treatment on FFT in order to compare the efficiency of CWT. The following example will show the difference in enhanced spectral image and obtained beat curve. Figure 6. Processed FFT (top) and CWT (bottom) spectral image (excerpt from Nightwish Come Cover Me ) Figure 5. Enhanced wavelet spectrogram of musical excerpt. Subsequently, the enhanced spectrogram W (t,scale) is treated by calculating a small-windowed sum to obtain one ore more beat curve(s) with time resolution of 10 ms. 3 N 1 i= 0 scale= 0 ( t + i, scale) c ( t) = W (4) Enhanced spectral images from FFT and CWT are shown on Figure 6. In the second case, implementation of Continuous Wavelet Transform is showing 100% detection of percussion instrument beats in the test excerpt. This example suggests that the CWT is better suited than the FFT for beat detection. Recall that our aim is the use of the rhythmical information for music similarity estimation. One of rhythmical information representation is the beat histogram. Beat histograms were presented in [20] and [23]. This kind of representation is quite useful for

example for genre classification (as it carries information about number of beats with different periods) or tempo determination by finding a maximum. Further evolution of beat histogram is being investigated and described in the current work. A classical one-dimensional beat histogram provides some knowledge only about the different beat periods while the distribution of beats in the meaning of their strength is not clear. At the same time beat detection algorithm and its parameters affect the form of the histogram. It is evident to be of use to bring some knowledge about the strength of beat periods into the histogram and avoid the dependency from the beat detection algorithm parameters. Thus, a 2D form histogram can be built with a beats period on the X axis and with amplitude (strength) of a beat on the Y axis (Figure 7). The information about beat strength in the proposed histogram is implicit since the histogram is computed upon the threshold used in beat detection. It is hence possible to avoid the disadvantage of recording conditions dependency (e.g. volume) and peak detection method. The range of threshold variation is taken from 1 to the found maximum-1. Thus, the beat strength is taken relatively and the volume dependency is avoided. Comparison of histograms showed only 10% of relative difference. The described rhythmical image representation foresees a resemblance measure of two musical compositions in the meaning of rhythm. As the 2D beat histogram is not affected neither by volume of music nor by conditions of recording (e.g. frequency pass band), it can be used directly in a distance measure. The measure of rhythmic distance can be defined in numerous was. In our experiments we have find out the following equation which takes into account slight variation of rhythm of musical pieces being compared. Dist 1 ( min( H1x, y H 2( x, y) + R ) + min( H1( x, y) + R H x, y ) N, M H1, H 2 = 2 R R x= 1, y= 1 2 (5) where H1, H2 beat histograms to compare N, M beat histogram size R an area of the following form (-2, -1) (2, -1) (-2, 1) (2, 1) Beat strength 100 90 80 70 60 50 40 30 20 10 0 1 11 21 31 41 51 61 71 81 91 101 111 121 131 141 Beat period Figure 7. A 2-D beat histogram. S71 S57 S43 S29 S15 S1 Such histogram can likely be a feature vector for example in genre classification or music matching. The main tempo is easily found from the histogram. In the case of Figure 5 it is equal to 34 points (means 340 ms) and in BPM it makes 176 while the strongest peak with 170ms (352 BPM) presents the most probable beat period which is out of usual tempo range. However, for beat-less musical pieces like classic or new age without clear beat or onset accents the tempo estimation is not evident. To prove the independency from recording conditions another set of experiments with beat histogram has been carried out. For this purpose a musical composition has been filtered with treble and bass cut filters. The resulting histograms of beats still had the same forms and peaks. 2.3. Melodic similarity This chapter covers aspects of melodic similarity measures. Algorithms described in the chapter are based on automated transcription (multiple F0 estimation) of polyphonic music with the use of Continuous Waveletlike Transform. Numerous algorithms for F0 estimation (pitch detection) exist in the literature. [11] describes an advanced method PreFEst. Using EM algorithm, it basically estimates the F0 of the most predominant harmonic structure in the input sound mixture; it simultaneously takes into consideration all the possibilities of F0 and considers that the input mixture contains every possible harmonic structure with different weights (amplitude). Another pitch model based system is presented in [9]. In the paper [10], authors describe a computationally inexpensive scheme of transcribing monophonic and polyphonic music produced from a single instrument. The scheme is based on two steps (track creation and grouping) and uses discrete variable window-size combfilter together with sharpening filter. We use a technique inspired by harmonic pitch models. The analysis procedure is divided into two parts (its diagram is shown on Figure 5). The first part consists of

model generation. The model is simply a fence of peaks situated at the places where F0 and its harmonics 2F0, 3F0 etc. can be found on a CWT spectrogram. Recall that our CWT spectrogram has a logarithmic frequency scale, so the distances between corresponding harmonics on such spectrogram remain constant with the change of absolute value of F0. Only the forms of each peak are changing over the frequency scale due to the change of frequency and time resolution of wavelets. To obtain these forms we are using the CWT applied on sine waveforms with appropriate frequencies. The shape of the harmonic model- fence may be used either flat where all amplitudes are similar for all the harmonics or with raised low harmonics part (ratios 3 2 1.5 1 1 etc for corresponding harmonics) which actually gives better results in general case. In particular, the shape of the harmonic model can be adjusted to the instrument assumed to be used in the play. In general case such assumption cannot be made. The second part of the analysis lies in analyzing of input wave signals for transcription. The input signal (16KHz, 16bit PCM) is processed by the CWT which has 1024 bins for frequencies lying in the range of 34-5500Hz every 25 ms. Obtained spectrum slices are analyzed in the following way. The above-mentioned harmonic structure is moved across the frequency scale of the CWT spectrogram slice and the correlation between the model and the spectrogram is computed. The place where the correlation has a maximum value on the spectrogram is assumed to be an F0 candidate. As it is found, the harmonic fence is subtracted from the currents slice of the spectrogram with the values on its peaks being taken from the actual values on the spectrogram. The procedure is repeated until no more harmonic-like structures are found in the spectrum (above the certain threshold) or the maximum number of harmonic structures to be searched defined in the algorithm is reached. We limit the maximum number of notes searched to four in our works. The described algorithm has an advantage of its rapidity and it is working well in detection of multiple pitches with non-integer rates. However, there is a disadvantage of such algorithm. Notes with F0s being in integer rates whose partials intersect are rarely can be completely detected together. At the same time, two notes with a distance of an octave hardly can be separated, because the second note does not bring any new harmonics into the spectrum, but changes the amplitude of existent harmonics of the lower note, so some knowledge of instruments involved in the play might be necessary to resolve the problem. Another possibility of the search for F0 candidates has been studied. Instead of successive subtractions of dominant harmonic structures found, one can use a local maximums search on the correlation curve. Every peak above a defined threshold is considered as an F0 candidate. Such approach can partly solve the problem of notes with overlapping partials while it generates phantom notes in one octave down to the note which actually present. With subtracting algorithm such notes never appear. Finally, all pitch candidates are filtered in time in order to remove any noise notes with duration below a defined threshold. Further, several melodic similarity measures can be proposed. The simplest way to calculate such distance is to calculate a distance between note profiles (histograms). Note profile (histogram) is computed across the whole musical title and serves for estimation of musical similarity by tonality as well as tonality (musical key) itself. Tonality in music is a definition of note set used in a piece which is characterized by tonic or key note and mode (e.g. minor, major). Each tonality has its own distribution of notes involved in a play and it can be obtained from the note histogram [24] (Figure 11). 500 450 400 350 300 250 200 150 100 50 0 350 300 250 200 150 100 50 0 Do Do# Re Mib Mi Fa Fa# Sol Sol# La Sib Si Do Do# Re Mib Mi Fa Fa# Sol Sol# La Sib Si Figure 11. Note profiles for major (C-dur, top) and minor (Cmol, bottom) tonalities (approximate). To compare two musical titles in the meaning of tonal similarity we calculate a similarity of two note profiles. These profiles must be either aligned by the detected tonality s key note (e.g. by Re for D-dur or D-mol) or a maximal similarity across all possible combinations of tonalities must be searched Another musical similarity measure studied in the current work is a similarity based on note successions histogram. Here the probability of 3-note chains is collected and its histogram is then used as a fingerprint of musical title. A musical basis of such similarity measure is that if same passages are frequent in two musical compositions, it

gives a chance that these two compositions have similarities in melody or harmony. The procedure is note successions histogram calculation is following. First, note extraction over the whole piece is carried out with a step of 320 samples (20ms). Then detected notes are grouped in local note histograms in order to find a dominant note in each grouping window. The size of the grouping window may vary from 100ms to 1 sec. Finally, all loudest notes are extracted from local histograms and their chains are collected in the note successions histogram. The resulting histogram is 3- dimensional histogram where each axe is a note of 3-note chain found in the musical piece being analyzed (Figure 12). note3 note1 Figure 12. Note successions histogram example. 3. EXPERIMENTS note2 Recall, that that our paper is dedicated to similarity estimation of musical pieces. In developing beat and note detection algorithms we do not aim to achieve 100% accuracy, however experiments on evaluation of the algorithms have been carried out. Our main experiments have in aim an estimation of musical similarity accuracy. They consist of two evaluation parts subjective and objective. 3.1. Subjective evaluation Preliminary experiments with musical similarity search were carried out. A database of 1000 musical composition of different genres, rhythms and types has been processed. Then series of playlists have been created using following two laws. - 1 st group of playlists: each next composition in playlist is the nearest in the meaning of rhythmical distance to the previous - 2 nd group of playlists: each next composition in playlist is the most distant from the previous. Comparison of two groups of playlists by a human listener showed resemblance of compositions in the first group and dissemblance in the second. In playlists of the first group classical compositions were mainly put together with other classical compositions, dance music was together with other dance music, slow melodic pieces were near other pieces of such kind and mood. 3.2. Objective evaluation In the case of rhythmic similarity an objective evaluation would be an experiment with genre classification. Notice that a music title from one genre has more probability of being rhythmically similar to another title from the same genre. We have built fro this task a reference database of 1873 musical excerpts containing musical titles of 6 genres from 822 different artists. The genres were chosen as being the six genres we generally found on several online music stores. The selected list of genres includes: Rock (Pop), Rap (HipHop, R&B), Jazz (Blues), Classic, Dance (Disco, Electro, House), Metal (Hard Rock, Heavy Metal) Each of these general genres consists of several subgenres which have more precise definition. For example, the Rap genre consist of such sub-genres as Rap, HipHop, R&B, Soul etc each sub-genre corresponds to a specificity which means that two songs of the given subgenre are closer at least from musical edition s point of view than two songs from different sub-genres. Unfortunately, detailed genre taxonomy can be defined in multiple ways [25] which is a limit for the definition of a universal musical genres taxonomy. Hence, we propose to choose from each general genre a well defined subgenre which represents the main genre. The choice of subgenres lies on the most representative sub-genre in the meaning of number of songs associated to it by a musical distributor, for instance fnacmusic. For each representative sub-genre we have selected the list of artists associated to it on the music distributor store. This list was then used to capture music from webradios [www.shoutcast.com]. The musical segments were captured as 20-seconds records starting from the 50 th second of the play and saved as PCM 8KHz 16bit Mono files. In total the reference database consists of 1873 titles from 822 artists which make 37480 seconds in total. It is crucial to note an important variability of musical titles in this reference database owing to an important number of artists. As far as we know, this is the first reference database where the attribution of genres to each title is not made in subjective manner by one person but takes into account the musical distribution attribution. Also, in comparison with other databases like magnatune, the current reference database is better balanced in the meaning of representation of classes (~1000 classic vs. ~70 for jazz in the case of magnatune). The algorithm of classification by rhythmical analysis is a basic knn classifier based on the 2D beat histogram and

the rhythmical similarity measure described in this paper. The rhythmical distances between musical files to be classified and musical files from the test set are calculated. The probability of belonging of the file in question to a class (genre) is proportional to the number of files of the same class returned in the top 15. Hence, this is a 15-NN classifier. The results of the classification using 2D beat histogram are presented in the following table. Table 1. Genre classification confusion matrix (average 52.7%) Classic Dance Jazz Metal Rap Rock Classic 89 5 20 36 8 12 Dance 0 57 2 2 5 3 Jazz 3 12 53 11 22 15 Metal 6 8 4 31 6 19 Rap 0 11 13 7 55 20 Rock 2 7 8 13 4 31 In fact, comparing these results with classification rates, obtained using acoustic features based classifier [26] we got the following confusion matrix and average rate: Table 2. Genre classification confusion matrix using PGM-MLP (average 49.3%) Classic Dance Jazz Metal Rap Rock Classic 53 5 12 2 5 10 Dance 3 40 7 7 11 8 Jazz 23 4 38 2 6 21 Metal 7 24 15 75 16 19 Rap 2 16 12 7 55 7 Rock 12 11 16 7 7 35 With the object of an objective comparison of the 2D and 1D beat histograms as well as comparison of FFT and CWT transforms for music classification, a series of experiments with genre classification have been made using the same database and classification algorithm. The use of CWT instead of FFT and proposed 2D instead of 1D beat histogram showed an increase of average classification rates by 8.4% and 3.3% respectively. Evaluation of melodic similarity measures was based on composing of similarity playlists for musical titles that have multiple reinterpretations. The database of these titles used in this work is a certain number of musical files in MP3 format. The list is as follows. 1. Ennio Morricone Chi Mai, 3 interpretations 2. Roxette Listen to Your Heart, DHT Listen to Your Heart, DHT Listen to Your Heart (dance) 3. Rednex Wish You Were Here, Blackmore s Night Wish You Were Here 4. Tatu Not Gonna Get Us (Eng), Tatu Nas Ne Dogonyat (Rus) 5. Tatu All the Things She Said (Eng), Tatu Ya Soshla s Uma (Rus), Tatu - Remix 6. Tatu 30 minutes (Eng), Tatu Pol Chasa (Rus) 7. Archie Shep, Benny Golson, Dexter Gordon, Mike Nock Trio, Ray Brown Trio Cry Me a River (ver.1 jazz instrumental) 8. Diana Krall, Tania Maria, Linda Ronstadt, Bjork, Etta James, July London Cry Me a River (ver. 2. vocal) In this experiment the different interpretations of the same title are considered as similar. In the experiment playlists with 30 similar titles corresponding to each musical title in the database were built. Appearance of a priori similar titles at the top of playlist was considered as successful similarity output. The following table shows the result of playlist composition. It gives the information about position of appearance of similar titles in the associated playlist (1 is the original music file). Table 3. Objective evaluation results of music similarity measures. Original music composition Chi Mai (1), 2, 3 Listen To Your Heart (1), 3, 12 Wish You Were Here (1), 2 Not Gonna Get Us (1), 2 All the Things She Said (1), 2, 3 30 minutes (1), 2 Positions of appearance of similar titles Cry Me a River (ver. 1) (1), 2, 3, 4, 6 Cry Me a River (ver. 2) (1), 2, 4, 7, 8, n/p Presence of similar songs in first positions of playlists signifies good performance of given melodic similarity measure. Further combinations of various similarity measures can be helpful in task of intelligent music navigation and recommendation, automatic classification and categorization. 4. CONCLUSION In this paper we described CWT-based approaches of automated music analysis. Rhythmic and melodic similarity measures have been proposed and evaluated. Brief comparison of proposed CWT-based algorithms with the same algorithms but based on FFT transform showed an important results improvement. REFERENCES [1] Harb H., Chen L. (2003). A Query by Example Music Retrieval Algorithm. 4th European Workshop on Image Analysis for Multimedia Interactive Services

(WIAMIS03), Eds. World Scientific, ISBN 981-238-355-7, April 9-11, Queen Mary, University of London, UK, pp.122-1283 [2] Berenzweig A., D.P.W. Ellis & S. Lawrence, (2003) Anchor Space for Classification and Similarity Measurement of Music. In Proceedings of the IEEE International Conference on Multimedia and Expo, ICME 03, Baltimore, July 2003 [3] Logan B., Salomon A. (2001) A music similarity function based on signal analysis. In Proceedings of IEEE International Conference on Multimedia and Expo ICME 01, August 2001 [4] Pachet, Laburthe F., Amaury, Aucouturier, Jean- Julien, (2003). The Cuidado Music Browser: an end-toend electronic music distribution system. In INRIA, editor, Proceedings of CBMI 03, IRISA Rennes, France 2003 [5] Foote, J., M. Cooper, and U. Nam. (2002). Audio retrieval by rhythmic similarity. In Proceedings of the International Conference on Music Information Retrieval [6] Paulus, J., and A. Klapuri. (2002). Measuring the similarity of rhythmic patterns. In Proceedings of the International Conference on Music Information Retrieval [7] Abe T. et al., Robust pitch estimation with harmonics enhancement in noisy environments based on instantaneous frequency, in ICSLP 96, pp. 1277 1280, 1996 [8] Hu J., Sheng Xu., Chen J. A Modified Pitch Detection Algorithm IEEE COMMUNICATIONS LETTERS, VOL. 5, NO. 2, FEBRUARY 2001 [9] Klapuri A. Pitch Estimation Using Multiple Independent Time-Frequency Windows Proc. 1999 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, New York, Oct. 17-20, 1999 [10] Lao W., Tan E.T., Kam A.H. Computationally inexpensive and effective scheme for automatic transcription of polyphonic music ICME 2004: 1775-1778 [11] Goto M. A Predominant-F0 Estimation Method for CD Recordings: MAP Estimation Using EM Algorithm for Adaptive Tone Models ICASSP 2001 Proceedings, pp. V-3365-3368, May 2001 [12] Hadi Harb, Liming Chen, Voice-Based Gender Identification in Multimedia Applications, Journal of Intelligent Information Systems JIIS, special issue on Multimedia Applications, 24:2, 179-198, 2005 [13] Kronland-Martinet R., Morlet J. and Grossman A. Analysis of sound patterns through wavelet transform, International Journal of Pattern Recognition and Artificial Intelligence,Vol. 1(2), 1987, 237-301 [14] Tzanetakis G., Essl G., Cook P. Audio Analysis using the Discrete Wavelet Transform Proc. WSES Int. Conf. Acoustics and Music: Theory 2001 and Applications (AMTA 2001) Skiathos, Greece [15] Brown, J. C. (1993). Determination of the meter of musical scores by autocorrelation. J. Acoust. Soc. Am. 94 [16] Gouyon F., Herrera P. (2003). A beat induction method for musical audio signals. Proceedings of 4th WIAMIS-Special session on Audio Segmentation and Digital Music London, UK [17] Scheirer E. (1997), Tempo and beat analysis of acoustic musical signals. Machine Listening Group, E15-401D MIT Media Laboratory, Cambridge, Massachusetts [18] Large, E. W., Kolen, J. F. (1994). Resonance and the perception of musical meter. Connection Science 6(2), 177-208 [19] Nava G.P., Tanaka H., (2004) Finding music beats and tempo by using an image processing technique, ICITA2004 [20] Tzanetakis G., Essl G., Cook P., (2001), Automatic Musical Genre Classification of Audio Signals, ISMIR2001 [21] Goto M. (2001). An Audio-based Real-time Beat Tracking System for Music With Or Without Drum sounds. Journal of New Music Research, Vol.30, No.2, pp.159-171 [22] Hofmann-Engl L. (2002). Rhythmic Similarity: A Theoretical and Empirical Approach. In: Proceedings of the 7 th International Conference on Music Perception and Cognition. [23] Tzanetakis G., Essl G., Cook P., (2002), Human Perception and Computer Extraction of Musical Beat Strength, Conference on Digital Audio Effects (DAFx- 02) [24] Ching-Hua Chuan, Elain Chew, Polyphonic Audio Key Finding Using the Spiral Array CEG Algorithm, ICME2005 [25] Pachet F., Cazaly D., A Taxonomy of Musical Genres, Proceedings of Content-Based Multimedia Information Access Conference (RIAO) Paris, France 2000 [26] Harb H., Chen L., Auloge J-Y., Mixture of experts for audio classification: an application to male female classification and musical genre recognition, In the Proceedings of the IEEE International Conference on Multimedia and Expo, ICME 2004