Audio Engineering Society Convention Paer Presented at the nd Convention 0 Aril 6 9 Budaest, Hungary This aer was eer-reviewed as a comlete manuscrit for resentation at this Convention. Additional aers may be obtained by sending request and remittance to Audio Engineering Society, 60 East nd Street, New York, New York 065-50, USA; also see www.aes.org. All rights reserved. Reroduction of this aer, or any ortion thereof, is not ermitted without direct ermission from the Journal of the Audio Engineering Society. Evaluation of cultural similarity in laylist generation Mariusz Kleć Polish-Jaanese Institute of Information Technology, Warsaw 0-008, Poland mklec@jwstk.edu.l ABSTRACT Choosing aroriate songs that satisfy one s needs is often frustrating, tiresome and ineffective due to the increasing number of music collections and their sizes. Successive songs in laylists should fit the situation, our mood or at least have common features. Consequently, there is a need to develo solutions that would enrich our exerience of listening to music. In this aer, musical similarity has been studied at the cultural level in the laylist generation rocess. Also, an author s rogram designed for testing different laylists will be described. It is used to erform an exeriment examining quality of laylists created using a cultural similarity model.. INTRODUCTION In their interactions with music eole always form ersonal oinions. Even if not musically trained, eole form views about the artists, rhythm, musical style etc very quickly []. In this aer, cultural similarities between articular artists are studied. Musical similarity has been studied so far from the score level, the audio level, and the cultural level. The outcome of this research has been resented at various conferences, articularly at annual International Conferences on Music Information Retrieval ISMIR. A number of aers exist concerning musical similarity measurement based on audio signal and music classification [5-7]. There is also a lot of research about laylist generation based on these measurements [7-9]. Similarity between certain airs of artists can be erceived as greater than others. Before considering various aroaches, we need to examine some roblems connected with artists similarity mentioned in []. There, it is ointed out that subjective judgment of the similarity between airs of artists may vary with individual s mood and life exerience. As discussed in [], subjective similarity is often asymmetric, mean that we usually comare less known artists to more known ones and not the other way round. For examle if we had to answer a question who Tony Banks is similar to, we would intuitively oint Phil Collins. But if one asked who Phil Collins was similar to, very few eole would oint out Tony Banks. Additionally, variability is another roblem. Namely, a lot of artists undergo changes during their careers and may include multile styles within a single album. Trying to define a single distance between such artists seems to be
unreliable. Desite these roblems, we believe that alying cultural similarity in laylist generation is worth consideration. The method described in this aer, adds only a small brick to this research area, but undoubtedly increases the awareness about the imortance of using cultural similarity in laylist generation. This aer is organized as follows. In section, the revious work in the field of cultural similarities and other methods of laylist generation is discussed. Next, an author s rogram referred to as the Playlist generator is described. Following is a section describing the exeriment that comares manually created laylists with those which were created using automated techniques described in this aer. Finally, conclusions and future work are resented.. RELATED WORK There is musicological and sychological evidence that cultural associations and exectations outside the content of musical signals lay an essential role in how humans interret and organize music []. In [] webbased similarity was used in facilitating to find desired music by allowing semantic queries like rock with great riffs etc. To meet this challenge, the authors assigned semantically related information to individual music ieces by finding relevant information on the Web. Extracted features were comlemented by audiobased similarity to imrove the results. In [] subjective assessments of artist similarities were investigated. This consideration had a strong influence on the research discussed in this aer. Different methods to obtain co-occurrences between terms such as artists and genres are discussed in []. In [], authors resented an aroach to accelerating automatic laylist generation by incororating musical artist similarity based on web data. The laylist can be defined as finite set of successive ieces of music. Main aim for which laylists are created is to satisfy listeners exectations of the order of songs or music itself. The quality of a laylists deends heavily on the metadata of songs in the database and the algorithm used for their selection and sorting. In aer [8] authors roosed means of organizing large music collections and way to generate laylists by drawing aths on mas reresenting similarities between songs. A similar aroach was roosed in [5]. Very interesting aroach to automatic laylist generation was resented in [9] where the user s hysiological resonse and urose in the music selection algorithms were described. We should mention jmir roject which is used in music information retrieval (MIR) research. This roject heled us to measure artist similarity to achieve intended objective. The rimary emhasis of jmir is on roviding software for general research in automatic music classification and similarity analysis [].. PLAYLIST GENERATOR The Playlist generator is the author s rogram written entirely in JAVA so that it can be used on every system where the JRE is installed. Its architecture is divided into two modules: the database manager (DB) and the laylist layer (PP). Interaction between the DB and the PP is done using the drag and dro method. We must have at least one seed song to generate the laylist. The order of songs is determined by the cultural similarity matrix, calculated beforehand with the jwebminer tool [0]. We can listen to the songs in the laylist or exort them to a file comatible with the WinAm laylist format. Figure User interface of Playlist generator JWebMiner is a tool develoed to erform cultural feature extraction from the Internet. This is a software comonent that comes with a suite called jmir, mentioned in Section []. It is an oen-source software suite imlemented in Java used in music information retrieval research (MIR). This tool was created by Cory McKay from McGill University, Montreal, Canada. jwebminer is the first out-of-thebox cultural feature extractor designed for generalurose MIR research. At its most basic level, the software oerates by using web services to extract statistics on how often articular strings co-occur on the same web ages (comared to how often they occur AES nd Convention, Budaest, Hungary, 0 Aril 6 9 Page of 7
individually). This can rovide us with insights into the relative similarity between the musicians. To build such statistics, a web service offered by Yahoo was used in our exeriment. JWebMiner allows users to choose between different scoring functions and aly different filters in web searching. Those functions and filters will be used to evaluate cultural similarity in laylist generation. Detailed descritions of these scoring functions are resented in the next section.. EXPERIMENT An exeriment was erformed to evaluate the influence of cultural features in the rocess of laylist generation. The jwebminer tool allows users to choose between two scoring systems which will be described later. The exeriment was used to evaluate which of the scoring functions better reflects artist similarity erceived by eole. In this way we can evaluate the quality of the Playlist generator because it uses one of these scoring functions to generate laylists. The aim of this rogram is to create laylists where ieces of songs are arranged in similar way to our intuitive ercetion of artist similarity. To evaluate this, it is necessary to comare the laylists generated by the Playlist generator with some reference laylists. Such reference laylists should be based on songs ordered according to artist similarity erceived intuitively by a grou of eole. Four eole were asked to create different laylists manually. Their task was to arrange 0 ieces of music by their similarity to seed music artists: Bach, Queen and Phil Collins. Hence, each erson has created the laylists. Artist Elvis Presley Diana Kral Queen Benassi Bros Tina Turner Phil Collins Bach Metallica Bon Jovi Paul Van Dyk Table Title In the Ghetto East Of The Sun The Show Must Go On Every Single Day The Best Another Day in aradise Minuet & Badinerie Nothing Else Matters Misunderstood We Are Alive List of songs for ordering by articiants in exeriment The lineu of music ieces may be set u in different ways according to the erson s subjective oinion. The articiants were given instructions saying that they should be guided by general knowledge about articular artists. If someone didn t know anything about the given artist, he was allowed to listen to the song... Algorithm This chater describes the quality measure of the laylists created by the Playlist generator. It uses the jwebminer comonent as its engine for feature extraction from the Internet. Two scoring functions for calculating artist similarity were tested: f ( a b) co( a, b) co( a, y), = ( ) + co( x, b) ( a, b) ( b) C C f ( ) = a, b C y, y a x, x b ( a) C( b) C c ( ) In the first function ( co a, b denotes the number of web ages that have an artist in their content. This number is divided by the roduct of sums of indeendent occurrences of two artists in web ages, lus one to assure that division by zero does not occur. The function ( f ) is described in more detail in []. In the second function ( f ), C ( artist) denotes the number of web ages that have the given artist in their contents. C ( a, b), likewise, is the number of ages that have both artists a and b in their contents. C is the c most oular among all of the artists who were taken into consideration in the measurement. The introduction of C is an attemt to alleviate a roblem with c extremely oular artists (like Madonna) weighing down the similarity of less-known artists []. In order to calculate the results it was also necessary to create reference laylists for each of the seed artists (Queen, Bach, Phil Collins); in other words, to calculate the average arrangement from arrangements done by eole. This averaging was achieved by using the following measures. f ), ( )... Calculating the average order of songs for Queen as seed artist Firstly the order number o was assigned arbitrarily to each of the songs in the laylist (staring from ). Then, AES nd Convention, Budaest, Hungary, 0 Aril 6 9 Page of 7
the order number was normalized to the range of <0, > using the following equation: o = R x R max( o) = ( ) assuming R >, x - normalized value of order number. We got the value of x for every song that is a art of the laylist. The calculations of x were erformed for only one, freely selected laylist from the grou of a laylists based on the Queen as a seed artist. For other laylists in a grou of the same seed artist, the order number y based on the order of x was derived (see y, y, y in Table ). The next ste was to average out the values for the individual order in each laylist in the grou: x + y + y + y = ( ) As a result, we get an order of each song being average order of all laylists in the same grou of the same seed artist. Based on this order we can say that this is a model laylist that can be used to comare it with the laylist created by Playlist generator. Below is an examle of calculating the model order based on seed artist Queen (see Table ) and the same examle but before normalization (see Table ). Queen Phil Collins 0 6 6 Bon Jovi 6 0 0 Paul Van Dyk 0 7 7 Metallica Benassi Bros 7 Diana Kral 5 5 Tina Turner 6 8 Elvis Presley 7 5 5 Bach 8 8 8 Table Arrangement of songs in laylists based on seed artist Queen created by articiants. (,,, ) denotes order numbers before normalization i o Queen x y y y Phil 0.5.75.75.06 Collins Bon Jovi..75 0 0.8 Paul Van Dyk.5 0.875.875.5 Metallica.7.75.5.75. 5 Benassi Bros 5.5.875.75.5.68 6 Diana Kral.6.65.5.5.7 7 Tina Turner.75.5.5.687 8 Elvis Presley.87.5.65.65.59 9 Bach 5.5.875 Table Calculation of the average laylist model for articiants given Queen as the seed artist. For the other grous of laylists (for other seed artists) the average order was derived as well. It was derived relatively to first laylist in every case. The final results for the average order, for each of the seed artists are as follows: Queen: {.06,.8,.5,.,.68,.7,.687,.59,.875} (see Table ) Bach: {.5,.7,.8,.8,.5,.,.8,.875, 0,75} Phil Collins: {.87,.75,.5,.87,.5,.56,.656,.78,.78}... Calculating similarity between laylists created by articiants and by Playlist generator Next, the same laylists were generated with the Playlist generator. The similarity between the two laylists was measured with Euclidean distance: S N ( A B) = ( b i a ) i = i, (5 ) Figure shows results of calculations erformed on artists shown in Table. In order to generate the automatic laylists, the following rocedure was used. Calculations were erformed searately for two functions f and f, as described in the beginning of section.. A few different settings in the jwebminer AES nd Convention, Budaest, Hungary, 0 Aril 6 9 Page of 7
tool were also tested. Figure reresents 6 similarity matrices for two scoring functions and different regional and file tye filters. Lighter colors corresond to higher similarity between artists. If the name of the artist aears more often on the same web age, the similarity will be higher. Table Similarity matrices generated from Playlist generator for 0 artists for two tyes of scoring functions (vertical division) and different filter settings (horizontal division). Each row reresents similarity values which will be later used to generate the laylist with this artist as the seed artist. Lighter colors corresond to higher similarity. The goal was to answer the question, which settings of the jwebminer software corresond best to the natural human intuition in determining the similarity between artists when building a music laylist. After comaring those settings with the human reference, we obtained the results as resented in Figure, and 5.... Results From the results of conducted exeriments, the difference between the laylists for different starting artists is clearly visible, see Figure. An excetion is the case of laylist generated using the f function and no limit as the filter setting. When we add restrictions to the regional limit or file tye, similarities for articular laylists diversify. From Figure we can conclude that function f is slightly more aroriate of the two to use it in laylist generation. The average for all the settings using this function is below the average for settings of using f everywhere in the grah for all grous of laylists. The best result was obtained using f function with no AES nd Convention, Budaest, Hungary, 0 Aril 6 9 Page 5 of 7
limits otion. It is esecially noticeable in Figure where all grous of laylists demonstrate the highest factor of similarity to the laylist created by eole. From Figure 5 we can conclude that the laylists based on Phil Collins as a seed artist indicate higher degree of similarity to reference laylists than the laylists based on other seed artist. Figure The bar chart showing distances between laylists created by Playlist generator and laylists created manually by research team. Figure 5 Linear regression, showing degree of similarity of laylists based on the individual artists. Horizontal axis reresents the following settings: f : no limits, f : regional limits, f :file tye limits, f : no limits, 5 f :regional limits, 6 f :file tye limits. 5. SUMMARY AND FUTURE WORK The main aim was to evaluate quality of laylists generated by Playlist generator with different settings of cultural similarity measurements. The settings reresent scoring functions f and f and also regional and file tyes filters alied to jwebminer during searching the Internet. This research clearly showed that the results of the automatic laylist generation through searching the network, evaluated using f, indicate higher similarity to the reference laylists generated by the articiants of the exeriments than the results obtained for f. Figure The chart showing trend to which moves similarity of laylists at various otions, limiting the search network and using different scoring functions. The vertical axis reresents distances between columns shown in Figure. Additionally, average values are also resented. The analysis, although limited in the number of exeriments, oints to the trend, to which we should move towards in future research. Particiants were given instructions to use their own knowledge in ordering the artists. They were allowed to listen to the songs if they didn t know the articular artist. However, if that haened, listening to the song was the only criterion in deciding where to ut that articular iece of music in the laylist. Particiants of the exeriment were of different ages (from 5 to 55) and different AES nd Convention, Budaest, Hungary, 0 Aril 6 9 Page 6 of 7
level of musical knowledge. To arrange 0 ieces of music is not an easy task to erform manually. Instead of focusing on their intuition, such a erson may be guided by sound or genre. It may introduce some noise to the reference laylist which ought to be created in the way that listeners erceive (intuitively) cultural similarity. The Playlist Generator doesn t take into account musical content at all only the name of the artist. Desite this fact we still could obtain clear results. We lan to extend Playlist generator with other similarity measure tools and extend abilities of this software to monitor environment features such as temerature, humidity, lights etc. to investigate their influence on eole and their choices of articular kinds of music in secific environment. 6. ACKNOWLEDGEMENTS This work was artially suorted by the Research Center of PJIIT, suorted by the Polish National Committee for Scientific Research (KBN). The author would like to thank Alicja Wieczorkowska, Danijel Korzinek and Krzysztof Marasek for their suort. 7. REFERENCES [] D.Scheirer, Eric. (000). Music-Listening Systems. Phd. Thesis. Massachusetts Institute of Technology, 007. [] P. Knees, T. Pohle, M Scheld, and G. Widmer. A Music Search Engine uon Audio-based and Webbased Similarity Measures. Austrian Research Institute for Artificial Intelligence, 007 [] P. Knees, T. Pohle, M Scheld, and G. Widmer. Combining Audio-based Similarity with Web-based Data to Accelerate Automatic Music Playlist Generation. Austrian Research Institute for Artificial Intelligence, 006 [6] B. Logan, A. Salomon. A music similarity function based on signal analysis. Cambridge Research Laboratory, USA, 00 [7] D. Schnitzer. High-erformance music similarity comutation and automatic laylist generation. Vienna University of Technology, 007 [8] R. Neumayer, T. Rauber. Content-based organization of digital audio collections. Vienna University of Technology, Austria, 005 [9] N. Olivier, L. Kreger-Stickles. PAPA: Physiology and Purose-Aware Automatic Playlist Generation. Microsoft Research, 007 [0] C. McKay, I. Fujinaga. JWebMiner: a web-based feature extractor. McGill University, Canada, 007 [] D. P.W. Ellis, B. Whitman, A. Berenzweig, S. Lawrence. The quest for ground truth in musical artist similarity. Proceedings of the International Conference on Music Information Retrieval. 70 7, 00 [] Geleijnse, G., and J. Korst. Web-based artist categorization. Proceedings of the International Conference on Music Information Retrieval. 66 7, 006 [] Schedl, M., T. Pohle, P. Knees, and G. Widmer. Assigning and visualizing music genres by webbased co-occurrence analysis. Proceedings of the International Conference on Music Information Retrieval. 60 5, 006 [] A. Tversky, Features of similarity. Psychological Review, 8():7-5 July 977. [5] S.Pauws, B.Eggen. PATS: Realization and User Evaluation of an Automatic Playlist generator. Philis Research Eindhoven, Netherlands, 00 [] Cory McKay. Automatic Music Classification with jmir. Phd. Thesis. McGill University, Montreal, 00 [5] E. Pamalk, A. Flexer. Imrovements of audiobased music similarity and genre classification. Austrian Research Institute for Artificial Intelligence, Vienna, 005 AES nd Convention, Budaest, Hungary, 0 Aril 6 9 Page 7 of 7