Scalable Music Recommendation by Search

Transcription

1 Scalable Music Recommendation by Seach Rui Cai, Chao Zhang, Lei Zhang, and Wei-Ying Ma Micosoft Reseach, Asia 49 Zhichun Road, Beijing , P.R. China {uicai, v-chaozh, leizhang, ABSTRACT The gowth of music esouces on pesonal devices and Intenet adio has inceased the need fo music ecommendations. In this pape, aiming at poviding an efficient and geneal solution, we pesent a seach-based solution fo scalable music ecommendations. In this solution a music piece is fist tansfomed to a music signatue sequence in which each signatue chaacteizes the timbe of a local music clip. Based on such signatues, a scale-sensitive method is then poposed to index the music pieces fo similaity seach, using the locality sensitive hashing (LSH). The scale-sensitive method can numeically find the appopiate paametes fo indexing vaious scales of music collections, and thus can guaantee a pope numbe of neaest neighbos ae found in seach. In the ecommendation stage, epesentative signatues fom snippets of a seed piece ae extacted as quey tems, to etieve pieces with simila melodies fo suggestions. We also design a elevance-anking function to sot the seach esults, based on the citeia that include matching atio, tempoal ode, tem weight, andmatching confidence. Finally, with the seach esults, we popose a stategy to geneate a dynamic playlist which can automatically expand with time. Evaluations of seveal music collections at vaious scales showed that ou appoach achieves encouaging esults in tems of ecommendation satisfaction and system scalability. Categoies and Subject Desciptos H.5.5 [Infomation Intefaces and Pesentation]: Sound and Music Computing methodologies and techniques, systems; H.3.3 [Infomation Stoage and Retieval]: Infomation Seach and Retieval etieval models, seach pocess; H.3.1 [Infomation Stoage and Retieval]: Content Analysis and Indexing indexing methods Geneal Tems Algoithms, Pefomance, Expeimentation Pemission to make digital o had copies of all o pat of this wok fo pesonal o classoom use is ganted without fee povided that copies ae not made o distibuted fo pofit o commecial advantage and that copies bea this notice and the full citation on the fist page. To copy othewise, to epublish, to post on seves o to edistibute to lists, equies pio specific pemission and/o a fee. MM 07, Septembe 23 28, 2007, Ausbug, Bavaia, Gemany. Copyight 2007 ACM /07/ $5.00. Keywods Scalable music ecommendation, automated playlist geneation, content-based music seach, music signatue, music snippet, locality sensitive hashing (LSH) 1. INTRODUCTION Music is moe pevasive than eve as an impotant entetainment medium. With the incease in online music stoes and sevices, and the populaization of potable music devices, we can moe conveniently access music than eve befoe. Fo example, we can puchase compact discs fom electonic maketplaces such as itunes and Amazon.com, and listen to Intenet adio sevices such as Pandoa [3] and Last.fm [1], o enjoy music anywhee with potable mp3 playes like ipod and Zune. All these channels, including potable devices, povide massive music esouces. Fo example, a typical mp3 playe with 30GB had disk can hold moe than 5, 000 songs. With such a vast scale of collections, a long tail distibution can be obseved in use listening histoy. That is, in thei collections, except fo a few pieces that ae fequently played, most ae aely visited, due to the inconvenient opeating functions of potable devices. Even on desktop computes, it is usually a tedious task to select a goup of favoite pieces fom a lage music collection. Theefoe, music ecommendations ae highly desied because uses a peceived need fo suggestions to find and oganize pieces close to thei tastes. Much eseach effot has been devoted to music ecommendations in ecent yeas [4,10,13,16,18,19,23 25,27]. Two majo ecommendation technologies have been epoted in the liteatue: collaboative filteing (CF)-based and contentbased ecommendation. CF-based methods [10] ecommend music by investigating use goup atings histoy, and ae widely adopted in online stoes and music societies [1]. To achieve easonable suggestions, CF-based methods should be based on lage-scale ating data and an adequate numbe of uses. Howeve, it is difficult to extend CF-based methods to applications like ecommendations on local pesonal music collections. Moeove, CF-based methods still suffe fom poblems like lack of data and poo vaieties of ecommendation esults [18, 27]. Content-based methods can meet the equiements of moe application scenaios, since they focus on the popeties of the music itself. Content-based methods can be futhe divided into metadata-based [4, 23 25] and acoustic-based methods [13, 16, 18, 19, 27]. Metadata, which includes popeties such as atists, gene, and tack title, ae global catalog attibutes supplied by music publishes. Based on such attibutes, some citeia o constaints can be set up to fil- 1065

2 te favoite pieces. Howeve, building optimal suggestion sequences based on multiple constaints is an NP-had poblem [4]. Although acceleation algoithms like simulated annealing [23] have been poposed, it is still difficult to extend such methods to a scale with thousands of pieces and hundeds of constaints. Othe methods based on metadata, utilize statistical leaning to constuct ecommendation models fom existing playlists [24, 25]. Due to the limitation of taining data, such leaning-based appoaches ae also difficult to scale up. Futhemoe, metadata is too coase to descibe and distinguish individual pieces chaacteistics. And in pactice, it is difficult to obtain complete and accuate metadata in most situations. In compaison, acoustic-based methods have fa few estictions and ae moe suitable fo situations in which consumes o sevice povides own the music themselves [3]. In geneal, acoustic-based methods fist extact some physical featues fom audio signals, and then constuct distance measuements o statistical models to estimate the similaity of two music objects in the acoustic space. In ecommendation, music pieces with simila acoustic chaacteistics ae gouped as suggestion candidates. Fo example, Knees et al. [16] extacted Mel fequency cepstal coefficients (MFCCs) on shot-time audio segments, and each music tack is then modeled using a Gaussian mixtue model (GMM), based on which pai-wise distances between pieces ae finally computed. In [13], music tacks ae gouped using the LBG quantization based on MPEG-7 audio featues, and the goup closest to the seed piece is etuned as suggestions. Simila to [13], Logan [19] constuct music clustes using MFCCs and K-means. And, in [8], some mid-level acoustic desciptions like tempo, mete, and hythm pattens ae computed, and a global measue of similaity is then defined by assigning specific weights to these divese featue dimensions. By investigating vaious ecommendation scenaios, we found that scales of music collection ae quite diffeent. Fo example, a music fan needs help to automatically ceate an ideal playlist fom hundeds of pieces on an ipod; while an online music adio povide should do the same job but fom seveal million pieces. Howeve, almost all the afoementioned methods on music ecommendation face the poblem of scalability, eithe when scaling down o scaling up. CF-based methods must ely on lage-scale use data, and pefomance will decease significantly when the data scale dops. Content-based appoaches mainly use linea scan to find candidates fo suggestions, and pocessing time will incease linealy with the data scale. To acceleate the pocessing time on lage-scale music collections, most content-based appoaches utilize tack-level desciptions of pieces, i.e., a whole music piece is chaacteized with one featue vecto [4, 8, 23, 24] o one model [16]. Some appoaches futhe goup music pieces into clustes, and the similaity seach is caied out on the cluste-level [13, 19]. To ou knowledge, the best pefomance epoted in one state-of-the-at wok is tenths of a second fo one match ove a million pieces [8]. Although the pocessing speed is impoved, such high-level desciptions may not be able to povide enough infomation to chaacteize and distinguish vaious pieces. On the one hand, music is a time sequence and the tempoal chaacteistics should be taken into account when estimating the content similaity. On the othe, some high-level desciptions ae too coase and ae incapable of filteing an ideal suggestion fom many simila candidates. Futhemoe, anothe disadvantage of cuent appoaches is that they ae bound to given music collections, and ae basically gounded on pe-computed pai-wise similaities. Theefoe, update costs ae consideable. While in eal situations, the membes of a music collection usually change fequently, especially in pesonal collections. A pefect solution should combine all the advantages of CF-, meta-, and acoustic-based methods. Howeve, in this pape we mainly focus on the poblem of acoustic-based music ecommendation, and leave the multi-modality ecommendation poblem as futue wok. Specifically, we ty to povide a scalable scheme to meet ecommendation equiements on vaious scales of music collections. The main idea hee is to convet the ecommendation poblem to a scalable seach poblem, o, in bief, ecommendation-byseach. Actually, web seach can be consideed a kind of ecommendation. That is, uses submit equests (queies) and the ecommende (seach engines) etuns suggestions (web pages). Analogously, a musical piece can be egaded as a web page, and can be indexed based on its local melody segments (just like a web page is indexed based on keywods) fo efficient etieval. Compaed with cuent methods, ecommendation-by-seach has the following advantages. Fist, seach technologies have been poven efficient and can be scaled fom a local desktop, intanet, to the entie Web. Second, as uses select and oganize queies (ecall the scenaio of quey-by-humming (QBH) [17], uses decide which pat of a piece they will hum as a quey), use inteaction can be seamlessly integated into seach-based ecommendation. We noticed that a simila idea has been evealed in [2]. Moeove, updating is moe convenient and cheape by means of the seach-based appoach. We only need to incementally update the index, without the need to go though the whole music collection to e-estimate pai-wise similaities. Howeve, fo the pupose of scalable music ecommendation, thee ae still seveal issues that should be addessed: Fist and the most impotant, is how to design a pope index stuctue based on data scale. Unde diffeent data scales, the citeion of similaity between music segments should be adaptively changed to guaantee a pope numbe of candidates etieved fo suggestion. Second, how to pepae seeds fo quey is anothe key poblem in ecommendation-by-seach. As mentioned befoe, it may be no a good idea to take an entie musical piece as a seed, since in most cases only pats of sections in a piece impess uses. Thid, with a list of etieval esults, a well-designed stategy is equied to ank these esults based on thei similaities to the seed, and finds the most appopiate one fo ecommendation. This is simila to the dynamic anking of esulting pages in web seach. To constuct a seach-based system fo scalable music ecommendation, we popose a solution to addess the above poblems. In the poposed solution, a musical piece is fist epesented with a music signatue sequence in which the signatue chaacteizes one local music segment. Then, the local sensitive hashing (LSH) method [11, 14] is applied to index signatues to conside thei L 2 distances. To addess the fist poblem, we popose an algoithm to adaptively estimate appopiate paametes fo LSH indexing on a given scale of the music collection. In ecommendation, we extact epesentative signatues as quey tems fom a seed piece by 1066

3 Collections Seed Piece Playlist Data Scale n tio c t a x E e t u a n ig S Scale-sensitive Paamete Estimation LSH Indexing Snippet-based Quey Selection Automated Playlist Ceation g in y e u Q s lt u s e h c a e S Index Relevance Ranking Figue 1: The flowchat of the poposed solution fo scalable music ecommendation, which consists of two pats: (I) scale-sensitive music indexing; and (II) ecommendation-by-seach. using a music snippet analysis [5,20] and design a elevanceanking function to integate citeia such as matching atio, tempoal ode, tem weight, andmatching confidence. In addition, we also popose a stategy fo dynamic playlist geneation based on the seach esults, in which the equiements of stick to the seed and dift fo supise [22] ae well balanced. Finally, we evaluate ou appoach on vaious collections fom aound 1,000 pieces to moe than 100,000 pieces, and the expeimental esults showed that ou appoach achieves pomising pefomance on both ecommendation satisfaction and system scalability, with elatively low CPU and memoy costs. The est of this pape is oganized as follows. In Section 2, an oveview of the poposed appoach is pesented. In Section 3, we intoduce ou method and implementation fo scale-sensitive music indexing. Section 4 descibes the detailed pocess of the ecommendation-by-seach, and how to automatically constuct a playlist using the poposed method. Evaluations and discussions ae given in Section 5. In the last section, we daw conclusions and point out futue eseach diections. 2. OVERVIEW OF OUR APPROACH The flowchat of ou solution is illustated in Figue 1, which mainly consists of two pats: scale-sensitive music indexing and ecommendation-by-seach. In the indexing stage, we fist extact a sequence of signatues fo each piece in the music collection, just like the pocess of tem extaction in text document indexing. Hee, a signatue is a compact epesentation of a shot-time music segment based on low-level spectum featues. With a signatue sequence, the local spectal chaacteistics and thei tempoal vaiation ove a music piece ae peseved so as to povide moe infomation than pevious tack-level desciptions. All these signatues ae then oganized by inveted indexes based on hash codes geneated by LSH. We utilize LSH because it theoetically guaantees signatues that ae close to one anothe will fall into the same hash-bucket with high pobability. A key poblem hee is how to define the citeion of closeness in LSH, which diectly affects the system pefomance. In ou solution, we automatically estimate such a closeness bounday based on the scale of a music collection, to ensue a pope numbe of esults can be etieved fo ecommendation. Intuitively, the bounday of such closeness in indexing should be somewhat elaxed fo a small music collection and tightened fo a massive collection. I II In the ecommendation stage, the seed piece is also conveted to a signatue sequence, based on which snippets of the piece ae extacted. Snippets (o thumbnails) ae those epesentative segments in a music piece, and ae usually the main chous o highlights pats. Thus, we select signatues fom the snippets instead offomthewholepiece,to constuct queies fo etieval. The etuned seach esults ae then soted though a elevance-anking function. In the anking function, besides some sophisticated citeia widely used in text seach, we also intoduce seveal new citeia to meet the specialties of music seach. Finally, we constuct a dynamic playlist using the anked seach esults. In the implementation, we build an efficient disk-based indexing stoage, and only a small cache is dynamically kept in memoy to speed up the seach pocess. In this way, ou system can opeate on most off-the-shelf PCs. 3. SCALE-SENSITIVE MUSIC INDEXING In this section, we descibe in detail the poposed method and the implementation of the scale-sensitive music indexing. This is the offline pat of the poposed scheme. 3.1 Music Signatue Geneation Some wok elated to music signatue extaction has been discussed in the liteatue on audio fingepinting [6,7]. Howeve, the fingepints defined in vaious aticles ae quite diffeent fom each othe. Fo example, some ae based on the distotion between two adjacent 10-ms audio fames and some ae based on the statistics of a whole audio steam [7]. In this wok, we adopt a method simila to the two-laye oiented pincipal component analysis (OPCA) pesented in [6], as it is based on a length suitable fo ou equiement and is obust enough to ovecome noise and distotions caused by music encoding. In the implementation, all music files ae conveted to 8 khz, 16-bit, and mono-channel fomat, and ae divided into fames of 25.6 ms with 50% ovelapping. Fo each fame, 1024 modulated complex lapped tansfom (MCLT) coefficients [21] ae fist computed and ae then tansfomed to a 64-dimensional vecto though the fist-level OPCA. Futhe, to chaacteize the tempoal vaiation, such 64- dimensional vectos fom 32 adjacent fames (aound 4.2seconds) ae concatenated and again tansfomed to a new 32- dimensional vecto though the second-level OPCA. Hee, the MCLT coefficients ae used to descibe the timbe chaacteistics on spectum fo each fame; and the time window is expeimentally selected as 4.2 seconds to chaacteize the tendoftempoalevolution.inthisway,bothspectaland tempoal infomation of the coesponding audio segment is embedded in the last 32-dimensional vecto, which is taken as a signatue in ou wok. Thus, a piece is finally conveted to a sequence of signatues by epeating the above opeation though the whole audio steam. 3.2 Music Indexing The objective of music indexing is to build an efficient data stuctue to acceleate similaity seach. It is woth noticing that the music indexing in this wok is quite diffeent to those intoduced in audio fingepinting elated woks. In fingepinting systems, the key diffeence is that only identical fingepints ae allowed to be indexed togethe, and two fingepints with only small diffeences may have quite diffeent index efeences [8, 25]. In ou appoach, we use similaity seach and ty to goup those close signatues in the 1067

4 e c n ta is D L The K-NN Distance on Datasets of Vaious Size The K th Neaest Neighbo dataset of 100,000 pieces dataset of dataset of dataset of 10,000 pieces 5,000 pieces 1,000 pieces Figue 2: The aveage K-NN L 2 distances on fou music collections with vaious scales. indexing. We also investigate how to contol the toleance of such closeness to ensue a pope numbe of signatues can be indexed togethe in the same hash bucket Locality Sensitive Hashing Locality sensitive hashing (LSH) was fist poposed in [14] and was extended in [11,15], as an efficient appoach to solve the poblem of high-dimensional neaest neighbo seach. LSH is based on a family of hash functions H = {h : S U}, which is called locality sensitive fo the distance function D(, ), if and only if fo any p, q S, itsatisfies: P H(h(p) =h(q)) = f D(D(p, q)) (1) whee f D(D(p, q)) is monotonically deceasing with D(p, q). Given a (R, λ, γ)-high dimensional neaest neighbo seach poblem 1, LSH unifomly and independently selects L K hash functions fom H, and hashes each point into L sepaate buckets. Thus two close points will have highe collision pobabilities in the L buckets. Moe details can be found in [11]. It has been theoetically poven that given acetain(r, λ, γ), the optimal L and K can be automated estimated [15]. In the neaest neighbo seach poblem, the pobabilities λ and γ can be expeientially selected, and the last poblem is how to select a pope R. The value of R diectly affects the expectation of how many neighbos can be etieved with pobability λ using LSH. Figue 2 shows such an example. In Figue 2, we andomly sampled 1000 signatues as quey tems fom fou music collections with diffeent scales 2, espectively, and then compute the aveage L 2 distance of a tem to its K th neighbo fo each collection. Fom Figue 2, it is clea that to etun a given numbe of neighbos, diffeent boundaies should be set fo diffeent data scale. Intuitively, such a bounday should be elaxed fo small set while tightened when data scale inceases, to ensue an expected numbe of neighbos can be etuned. Specifically, it a equiement of ecommendation-by-seach is to pomise a pope numbe of pieces will be etuned fo suggestion on whateve scale of music collections. 1 That is, fo any given quey point q, each point p satisfying D(p, q) R should be etieved with pobability at least λ, and each point satisfying D(p, q) >Rshould be etieved with pobability at most γ [15]. 2 The detailed infomation of the fou collections please efes to Section 5.1. Table 1: The mean μ and standad deviation σ of the pai-wise distances of signatues on vaious scales of music collections. Scale 1,000 5,000 10, ,000 μ σ y t i li b a b o P The Distibution of the Pai-wise Distance on the Dataset of 100,000 pieces V x V 0 L 2 Distance Oiginal Distance Histogam Gamma Distibution Appoximation Figue 3: The distibution of the pai-wise distance of music signatues on a collection with moe than 100, 000 pieces. It can be appoximated using a Gamma distibution Scale Sensitive Paamete Estimation In this wok, we popose a numeical method to automatically estimate the value of R fo a given scale of music collection. An assumption hee is, whateve the data scale is, the distibution of the pai-wise L 2 distances among signatues should be elatively stable. A simila conclusion has been dawn in [9]. To veify such an assumption, we checked the pai-wise distances on the fou collections in ou expeiments, and list the coesponding mean μ and standad deviation σ in Table 1. Fom Table 1, it is clea that the means and standad deviations of the pai-wise distances ae close on vaious scales of the collections. We also daw the histogam of the distance distibution on the collection that contains moe than 100,000 pieces in Figue 3. Such a distibution is simila to a Gaussian distibution. Howeve, it is asymmetic since the L 2 distance is always lage o equal to zeo, and it can be bette appoximated by a Gamma distibution, as shown in Figue 3. The pobability density function (pdf )ofa Gamma distibution is: e t/θ g(t; α, θ) =t α 1 (2) Γ(α)θ α whee the two paametes α and θ can be estimated as: α = μ 2 /σ 2 ; θ = σ 2 /μ (3) Based on the above assumption, we can conside that fo vaious music collections, the pai-wise L 2 distances of ou signatues follow a same Gamma distibution g(t; α, θ). Thus, given the data scale V 0 and the expected esult numbe V, the optimal value of R can be obtained by solving the following equation (R is eplaced by x fo claity): f(x) = x 0 g(t; α, θ) dt ρ = x t α 1 e t/θ 0 Γ(α)θ α dt ρ (4) 1068

5 and ρ = V/V 0 is the expected atio of the etuned esults. In ou expeiments, V issetto20foallthedatasets. Let s = t/θ, equation (4) is futhe tansfomed to: f(x) = 1 Γ(α) x/θ 0 s α 1 e s ds ρ = 1 Γ(α) γ(α, x θ ) ρ (5) whee γ(α, x) is a lowe incomplete Gamma function, and can be solved numeically. Thus, x can be iteatively achieved using the Newton-Raphson method with a andom initial value x 0,as: x n+1 = x n f(x n)/f (x n) (6) whee the deivative f (x) =g(t; α, θ). In this way, we can estimate a pope R and constuct a LSH-based index, accoding to the scale of a given music collection. In the seach stage, a quey signatue is hashed by the same set of LSH hash functions, and its neighbos ae independently etieved fom the coesponding L buckets. 4. RECOMMENDATION-BY-SEARCH Music in a simila style usually adopts some typical hythm pattens and instuments. Fo example,fastdumbeatpattens ae widely used in most heavy metal music. Simila instuments usually geneate simila spectal timbes, and simila hythms will lead to simila tempoal vaiation. As music signatue descibes tempoal spectal chaacteistics of a local audio clip, it is expected that music pieces of a simila style will shae some simila signatues, just like documents on simila topics usually shae simila keywods. Thus, it is possible to make music ecommendation pactical by etieving pieces with simila signatues. In othe wods, in ou system, the citeion fo ecommendation is to find out music pieces with simila time-vaying timbe chaacteistics. In this section, we descibe in detail the poposed idea of ecommendation-by-seach, and how it is implemented, based on the efficient music indexing scheme intoduced in Section 3. In the following, we fist intoduce how to select signatues as quey tems fom a seed piece, and then pesent the citeia fo designing the elevance-anking function. Finally, we pesent the poposed stategy fo automated playlist ceation. 4.1 Music Snippet-based Quey Selection How to select pope signatues as quey tems fom a piece is not a tivial poblem. Fist, not all the signatues in a piece ae epesentative to its content. Second, too many quey tems will dop the seach pefomance significantly (on aveage, a piece aound 5 minutes has moe than 2,000 signatues). Actually, people like and emembe one piece mostly because some shot but impessive melody clips ecu in the piece. Theefoe, it is easonable to select quey tems only fom such typical and epetitive segments, which have been called music snippets o thumbnails. Thee have been seveal epoted appoaches fo music snippet extaction [5, 20]. Hee, we simply utilize a evised algoithm of that poposed in [5], as thei appoach was also based on audio signatues. In the implementation, we extact thee snippets fom the font, middle, and back pats of a piece 3, and each snippet is a segment of aound Thee ae usually seveal epetitive segments fo a piece, and the snippet detection algoithm also etuns multiple candidates. To cove moe easonable snippets, we expeiseconds [5]. Howeve, we still face the long quey poblem as thee ae still about 100 signatues in such a 15-second segment. It is still a lage buden fo the seach engine. Consideing that music is a continuous steam and the two adjacent signatues have aound 4-second ovelaps, the L 2 distances between adjacent signatues ae usually small, unless some distinct changes happen in the signal. Thus, such signatues can be futhe compacted by gouping signatues close enough to each fo educing the numbe of quey tems. In ou system, a bottom-up hieachical clusteing is pefomed on signatues fom one snippet, and the clusteing is stopped when the maximum distance between clustes is lage than R/2. Fo each cluste, the signatue closest to the cente is eseved as a quey tem. Expeimentally, the quey tems can be educed to 1/10 afte the clusteing. Finally, by combining adjacent signatues in a same cluste, a music snippet is conveted to a quey, which is epesented with a sequence of (tem, duation) pais, as: Q [(q Q 1,t Q 1 ),, (q Q i,tq i ),, (qq N Q,t Q N Q )],q Q i S Q (7) whee q Q i and t Q i ae the signatue and the duation of the i th tem, S Q = {s 1,s 2,,s NUQ } is the set of all the N UQ unique tems in the quey, and N Q is the quey length. 4.2 Rank Citeia Relevance anking is a cucial component of almost all seach elated poblems. In text seach, elevance anking has been well studied such as the BM25 algoithm in [26]. The elevance anking in music seach has simila featues with that in text seach, but also has its own chaacteistics. As shown in (7), quey tems hee have duation infomation, and thei tempoal ode is also impotant. Moeove, as the seach is similaity-based but not identical matching, the confidence of such a matching should also be consideed in the anking. Fist, let us look back the seach pocess, and descibe how the seach esults ae obtained and oganized fo anking. As intoduced in Section 3.2, a quey tem (signatue) will be hashed into L buckets with LSH, and the pieces indexed in these L buckets will be meged as a esult list of this tem. Fo a hit point (also a signatue in a piece in the index), its similaity to the quey tem can be appoximated by the numbe of buckets it belongs to ove the whole L buckets (accoding to the LSH theoy, the close two signatues ae, the highe pobability they ae in a same bucket). Such a similaity can be consideed as a confidence of this matching. Afte going though all the unique tems in the quey, thei esult lists ae futhe combined to a candidate set 4 fo elevance anking. Fo each candidate piece in the set, its matching statistics can be epesented with a tiple sequence by meging adjacent hit points of a same tem into a segment, as shown in Figue 4. A tiple is in the fom of (q R,t R,c R ), whee q R is the matched tem, t R is the segment duation, and c R is the aveage matching confidence of the hit points in this segment. In anking, each candidate piece is futhe divided into fagments, as shown in Figue 4, if the time inteval Δt between two matching segments is lage than a pe-defined mentally select thee most possible candidates fom diffeent pats of a piece. 4 Hee, it is assumed the seach opeation is OR, as it cannot be expected all these tems in a quey will exist in one anothe piece. 1069

6 Quey: Q Q ( q i, t ) i Quey Tems Tem Pais Result: R R R ( q, t, c ) j j j Fagment A Δt > T min Fagment B Figue 4: An example of the music seach esult matching. The candidate piece has two fagments (A and B) that possibly match the quey. theshold T min (which is set as 15 seconds). Then, we compute the elevance scoes fo all the fagments, and the maximum is etuned as the scoe of the candidate piece. Consideing of the chaacteistics of music seach, in ou appoach, the elevance of a fagment is mainly based on the matching atio and tempoal ode, and also integates the tem weight and the matching confidence above. Fist, simila to the Robetson/Spack weight [26] in text etieval, the weight of the i th tem in S Q is defined as: V0 ni +0.5 w i =log (8) n i +0.5 whee V 0 is the total numbe of pieces in the dataset (i.e., the data scale defined in Section 3.2.2, and n i is the length of the esult list of the i th tem. The sum of all the tem weights in S Q is futhe nomalized to one. In this way, we assign lowe weights to popula tems while highe weights to special tems, just like the invese document fequency (idf ) utilized in text etieval. Then, ou anking function is defined as a linea combination of the measuements of the matching atio f atio and the tempoal ode f ode,as: f anking = f atio + f ode (9) In details: f atio is defined as: f atio = 1 N UQ min (d Q i,dr i ) N UQ max (d Q wi (10) i=1 i,dr i ) whee d Q i and d R i ae the duations of the i th tem occuing in the quey and in the fagment, espectively: d Q i = t Q k ; dr i = t R k (11) k q Q k =s i k q R k =s i In (10), the matching atio and combined with the tem weight. f ode is defined as: f ode = N Q 1 1 P occu(q Q i N Q 1,qQ i+1 ) (12) i=1 whee P occu(q Q i,qq i+1 ) is the maximum confidence of the pai (q Q i,qq i+1 ) occued in ode in the esult fagment, as: P occu(q Q i,qq i+1 )= max j q j R=qQ i &qr j+1 =qq i+1 (c R j c R j+1) (13) In (13), the tempoal ode and matching confidence ae combined togethe. In this way, fagments with lage matching atio and moe odeed tem pais ae anked with highe elevance scoes, based on which coesponding candidate pieces ae soted fo futhe ecommendation. 4.3 Automated Playlist Ceation Up till now, we have implemented a seach-based appoach to find ecommendations fo a given piece fom a music collection. In pactice, uses still need a continuous playlist, which can automatically expand with time. Hee, we also pesent one stategy fo automated playlist ceation using the ecommendation-by-seach. Vaious appoaches have been poposed to automatically geneate music playlists in pevious woks [4, 16, 23 25]. In geneal, a key idea is to povide an optimum compomise between the desie fo epetition and the desie fo supise [22]. In anothe wod, a good ecommende should suggest both popula pieces with simila attibutes ( stick to the seed ), and new pieces to povide fesh feeling ( dift fo supise ). Howeve, fo most content-based ecommendation systems, finding novel songs becomes an unavoidable poblem as thei citeion is to find out simila pieces (while fo CF-based ecommendation, this issue can be well solved using the powe of social community). Thus, ou appoach does have such a limitation in finding novel songs. To impove the divesity of ecommendation, hee, we heuistically add some dynamics when ceating playlists. In details, the playlist is geneated as following steps: 1. Use manually assigns a piece as a seed. 2. Extact thee snippets fom the seed piece, as descibed in Section 4.1, to constuct thee queies fo seach. The fist esult of each quey is added to the playlist. Thus, the suggestions ae still acoustically simila with the seed. In this step, we intend to meet the equiement of stick to the seed. 3. Randomly select a piece fom the top thee suggestions as a new seed, and go to step 2. Thus, the new seed is diffeent with the oiginal one, and will dive the playlist to a somewhat new style. In this step, we intend to meet the equiement of dift fo supise. Use inteactions can be easy integated into the above pocess. Fo example, uses can also tag any pats they ae inteested in a piece, and the playlist will be dynamically updated using queies geneated fom the tagged pats, instead of fom the snippets by default. 5. EVALUATIONS AND DISCUSSIONS In this section, we pesent esults of the poposed ecommendationby-seach, including the system efficiency, quantitative evaluations on both acoustic and gene consistencies, and sub- 1070

7 jective evaluations fom a use study. Based on these evaluations, we want to show that: 1) ou appoach is effective and efficient on vaious scales of music collections; and 2) the ecommendation quality is also pomising and is close to some state-of-the-at commecial systems. 5.1 Datasets and Expeimental Settings In expeiments, we collected 114, 239 pieces (fom 11, 716 albums) in mp3 and wma fomats. To simulate music collections with diffeent scales, we andomly sample some albums fom all the 11, 716 albums, and constuct fou collections: C 1 (1,083 pieces in 106 albums), C 2 (5,126 pieces in 521 albums), C 3 (9,931 pieces in 1007 albums), and C 4 (all the pieces). These collection scales ae selected to simulate the scenaios of ecommendation on potable devices, pesonal PCs, and online adio sevices. To evaluate the pefomances of the poposed solution on vaious scales of collections, fo each collection, we ceated 20 playlists with the seed pieces listed in Table 2. Fo compaison, we also ecoded the ecommendation lists fom a state-of-the-at online music ecommendation sevice, Pandoa [3], using the same 20 seeds. In addition, we still geneated 20 playlists in shuffle model by andomly selecting pieces fom the collections. The length of all the playlists is fixed to 10. Thus, in the expeiment, we constucted 6 playlist collections, with 20 playlists in each. As a side note, hee we want to give moe explanation to why we chose Pandoa fo compaison. Although thee ae some elated techniques in the liteatue fo automated and acoustic-based music ecommendation, it is still had to compae ous with them, since thei implementation details and paamete settings ae unavailable. Moeove, such compaisons ae also unfai because the data collections used in vaious papes ae diffeent. This is why thee ae few such coss-system compaisons in music ecommendation liteatue. In this pape, we ty to situate the ecommendation quality of ou appoach using two elatively fai efeencesandom shuffle and Pandoa. Pandoa is public fo access, and it is a top-anked commecial ecommendation sevice. It should be noted that we don t expect ou automated and only acoustic-based system can exceed the pefomance of Pandoa, as it leveages metadata and acoustic-elated infomation, as well as many expet annotations. Thus, Pandoa acts as a efeee in the following evaluations. Moe discussions please efe to Section System Efficiency In the expeiment, we employ a PC with 3.2 GHz Intel Pentium 4 CPU and 1GB memoy to evaluate the system efficiency. We fist evaluate the pefomance of the font-end, i.e., audio pocessing and music signatue extaction. We andomly select 100 pieces in eithe mp3 o WMA fomat fom the dataset. The aveage duation is 5.2 minutes. It took 3 minutes and 51 seconds fo the font-end (including the steps of mp3/wma decoding, down-sampling, MCLT, OPCA, and LSH-hashing) to pase all the 100 pieces. If the snippet extaction is also included, the total time cost is 5 minutes and 57 seconds. That is, 3.57 seconds ae equied on aveage to pocess a seed piece in ecommendation. Howeve, in most applications the seed piece is also a membe of the music collection, and the snippets and quey tems can be pe-geneated and stoed. The indexing time of the lagest collection C 4 is about 87 hous, and the detailed index size of each collection is listed in Table 3. To evaluate the online seach pefomance, fo each collection, we caied out 1, 000 queies (with aound 13.4 tems each) and the aveage pefomances ae shown in Table 3. Fom Table 3, it is fist obseved that the memoy costs of ou system on vaious collections ae elatively stable, and such memoy cost is also acceptable fo most desktop applications on PCs. Second, the aveage seach time inceases with the data scale, but is also acceptable fo most applications. The seach time hee includes etieving inveted indexes fom (#tem L) hush buckets, meging, and anking the seach esults. In C 1, as most of the index can be cached in memoy, the speed is quite fast. When index inceases with the data scale, the seach time becomes longe, as moe disk I/O ae needed fo cache exchange. Actually, when the data scale is extemely lage, the seach opeation can be easily distibuted to multiple machines to acceleate the pocess time. Anothe statistic shown in Table 3 is the aveage numbe of etuned esults. As discussed in Section 3.2, we need to assue enough esults ae etuned fo ecommendation on vaious scale of collections. Fom Table 3, the esulting numbe can be oughly kept in the ange of Moe detailed, thee ae aound 45% of pieces in C 1 ae etuned fo each quey; while with C 4 the pecentage is only aound 0.9%. Howeve, the numbe of esults is still inceased with the data scale, as the LSH is designed to bind the wost conditions, while in eal data the hitting pobability is much highe than expected. In geneal, it indicates ou method of the scale-sensitive music indexing is effective in pactice. In vaious music scales (application scenaios), ou system can guaantee a etun of a pope numbe of suggestions within an acceptable esponse time. In the following, we evaluate both the quantitative and subjective qualities of ou ecommendation esults in Section 5.3 and Section Quantitative Evaluation To the best of ou knowledge, thee is still not a sophisticated method to give a quantitative evaluation to music ecommendation. In this wok, we tied to utilize some indiect evidence fo quantitative compaisons. One is the acoustic consistency, to veify the suggestions fom the acoustic-level. The othe is the gene consistency, to veify the suggestions fom the metadata-level Acoustic Consistency The acoustic consistency is to veify how close those suggested pieces ae in the low-level acoustic space, and also has been utilized in some pevious woks fo music ecommendation [16, 19]. Hee, we adopted a GMM-based appoach [12, 16] to measue the distance between two pieces. In implementation, each piece in a playlist is modeled with agmminthed = 64 dimensional MCLT spectum space (the same one we adopted in Section 3.1 fo music signatue extaction), as: f(x) = k α in (x; μ i, Σ i)= i=1 k α if i(x) (14) whee μ i,σ i,andα i ae the mean, covaiance, and weight of the i th Gaussian component f i(x), espectively; and k is the numbe of mixtues (which is set as 10 expeimentally). The distance between two GMMs f(x) and g(x) is then defined: i=1 d(f,g) = 1 2 ( d(f, g)+ d(g,f)) (15) 1071

8 Table 2: Infomation of the seed pieces in expeiments. No. Tack Atist Gene 1 Lemon Tee Fool s Gaden Pop 2 My Heat Will Go On Celine Dion Pop 3 Candle in the Wind Elton John Pop 4 Soledad Westlife Pop 5 Say You, Say Me Lionel Richie Pop 6 Eveytime Bitney Speas Pop 7 As Long As You Love Me Backsteet Boys Pop 8 Right Hee Waiting Richad Max Rock 9 Yesteday Once Moe Capentes Rock 10 It s My Life Bon Jovi Rock 11 Teas in Heaven Eic Clapton Rock 12 Take Me to You Heat Michael Leans to Rock Rock 13 What d I Say Ray Chales R&B 14 Beat It Michael Jackson R&B 15 Fight Fo You Right Beastie Boys Rap 16 Does Fot Woth Eve Coss you Mind Geoge Stait County 17 Coss Road Blues Robet Johnson Blues 18 Bon Slippy Undewold Electonic 19 Scaboough Fai Saah Bightman Classical 20 So What Miles Davis Jazz Table 3: The usages of disk, memoy, and CPU on C 1 C 4. C 1 C 2 C 3 C 4 Index on Disk 70M 414M 787M 9.16G Runtime Memoy in Seach 42.5M 43.3M 43.5M 47.1M Aveage Seach Time 0.27s 1.41s 1.72s 2.53s Aveage Result Numbe whee d(f,g) is the diect distance fom f to g: d(f,g) = k α i min j,1 j k i=1 KL(fi gi) (16) Hee, the Kullback-Leible (KL) divegence between two Gaussian components is defined as: KL(N (x; μ 1, Σ 1) N (x; μ 2, Σ 2)) = 1 Σ2 [log 2 Σ + 1 t(σ 1 2 Σ 1)+(μ 1 μ 2) T Σ 1 2 (μ 1 μ 2) d] (17) In this way, fo each playlist we compute all the pai-wise distances between pieces. Afte going though all the 20 playlists in a collection, the distibution of such GMM-based distances on the collection is obtained and can be appoximated by a Gamma distibution, simila to that intoduced in Section Figue 5 illustates the appoximate distance distibutions on all the six playlist collections in ou expeiments. Fom Figue 5, it is found that the aveage pai-wise distance in shuffle is the lagest, while C 4 is the smallest. This indicates that pieces suggested by ou seach-based appoach still have simila acoustic chaacteistics in the tack-level, although only signatues in snippet pats ae used fo seach. This indicates ou ecommendation-by-seach can satisfy the assumption of acoustic-based music ecommendation. With the decease of the data scale, fom C 4 C 1, the aveage distance becomes lage, as well as the deviation of the distibution. In Figue 5, the distibution of Pandoa is in the middle of the shuffled appoach and those geneated using ou appoach. This indicates acoustic featues may also Pabability C1 C2 C3 C4 Pandoa Shuffle GMM-based Distance Figue 5: The distibutions of the GMM-based paiwise distance of pieces in a playlist, fo the six playlist collections, espectively. be consideed in Pandoa, but thei ecommendations ae not only based on the acoustic attibutes. This obsevation is consistent with the online intoduction of Pandoa [3], that is, it also leveages expet annotations such as cultue and emotion to geneate thei playlists. Thus, in Pandoa, pieces with simila annotations ae also possibly selected fo ecommendation, although thei low-level acoustic featues may be quite diffeent Gene Consistency A music gene is a categoy of pieces of music that shae a cetain style, and is one of the basic tags in music industy. Although the gene classifications ae sometimes abitay and contovesial, it is still possible to note similaities be- 1072

9 Table 4: Entopy of the gene distibution on the six playlist collections. Pandoa Shuffle C 1 C 2 C 3 C 4 Mean Std Table 5: Statistics of the subjective atings fo the six playlist collections. Pandoa Shuffle C 1 C 2 C 3 C 4 Mean Std tween musical pieces, and thus is widely used in metadatabased music ecommendation [4, 23 25]. To guaantee the genes used in the expeiment ae as accuate as possible, we utilized AllMusic ( the most authoitative commecial music diectoy to manually veify the gene of each piece. In total, nine basic gene categoies: Pop,Rock,R&B,Rap,County,Blues,Electonic,Classical, and Jazz, ae adopted fo classification. The evaluation of gene consistency hee is simila to that pesented in [16]. That is, the Shannon entopy is utilized to measue the gene distibution of pieces in a playlist. The Shannon entopy is defined as: H(x) = p(x)log 10 p(x) (18) x whee p(x) is the pecentage of a given gene in a playlist. Hee, consideing the length of a playlist is 10, we adopt log 10 ( ) in (18) thus the entopy of the wost case (the 10 pieces in a playlist ae fom 10 diffeent genes) is 1. And fo the ideal case (all the 10 pieces ae fom a same gene), the entopy is 0. The statistics of the entopies on the 6 collections ae listed in Table 4. Thee is not an authoitative citeion to descibe what the gene distibution should be like fo an ideal playlist [16]. Hee, by compaing the aveage entopies of playlists fom Pandoa and in shuffle model, we just assume the lowe the entopy, the bette the playlist quality. In Table 4, the entopy of playlists in shuffle is the highest and with small deviation, and it indeed should be close to the gene distibution of the whole music collection. The gene entopies of the playlists fom C 1 C 4 ae aound , and ae between Pandoa and the shuffle one. As gene is actually one of the citeia utilized fo ecommendation in Pandoa [3], the distibution on Pandoa is the most concentated. Though the compaison, it indicates that ou appoach can still keep the gene consistency, to a cetain extent. 5.4 Subjective Evaluation To evaluate the pefomance in pactice, we also conducted a small use study by inviting 10 college students as testes. Consideing the wok load, we andomly selected 5 playlists fom each collection fo each teste. Thus, each teste evaluated 30 playlists though listening to them one by one, and the collection infomation was blind to the testes. The testes wee asked to assign a ating anging scoefom1to5toeachplaylist.theatingciteiaae: 1 ( totally unacceptable ), 2 ( maginally acceptable, but still inconsistent ), 3 ( acceptable, and basically consistent ), 4 ( acceptable, with some good suggestions ), and 5 ( almost all good suggestions ). Hee, acceptable is defined as it is OK to finish the playlist without inteuption. To emove the individual bias, atings fom each teste ae fist e-nomalized befoe analysis. Then, the nomalized atings fom vaious testes ae aveaged on each playlist collection, and the coesponding mean and standad deviation ae kept fo compaison, as shown in Table 5. Fom Table 5, it is clea that the highest subjective ating is achieved on Pandoa, with an aveage ating close to 4.3. The atings fom C 1 C 4 ae aound 3.85, which indicates that with ou appoach, the suggestion qualities ae still acceptable and suffe little fom the data scales, especially when the scales ae lage enough (such as C 3 and C 4). The pefomance of the playlists in shuffle is the wost, thei aveage anking is lowe than 2. Howeve, an inteesting phenomenon is that the standad deviation on the shuffle collection is the smallest, which suggests subjective judgments ae moe consistent using it. Similaly, the subjects also show consistent satisfaction fo Pandoa. While in compaison, such deviations of C 1 C 4 ae notably highe. It indicates that the suggestion qualities of ou appoach ae not as stable as that of Pandoa, and need to be impoved in the futue wok. 5.5 Discussion The above evaluations have shown ou solution can achieve encouaging and stable pefomance on vaious scales of music collections, and is efficient in pactice. The geneal pefomance is much bette than that in shuffle, and is close to Pandoa. Howeve, it still needs impovement in compaison with Pandoa. Pandoa was ceated by the Music Genome poject [3], which is tying to ceate the most compehensive analysis of music eve. In Music Genome, a goup of musicians and music-loving technologists wee invited to caefully listen to pieces and label eveything fom melody, hamony and hythm, to instumentation, ochestation, aangement, lyics, and of couse the ich wold of singing and vocal hamony. Thus, the ecommendation of Pandoa has integated both meta- and acoustic- infomation, as well as human knowledge fom music expets. That is why it achieved the best subjective satisfaction in the expeiments. Howeve, Pandoa equies a geat amount of manual/expet labeling woks, which is expensive and is not available without geat difficulty in many applications, such as music ecommendations on pesonal PCs o potable devices. In compaison, ou solution can be conveniently deployed to both desktop and web sevices. Especially fo desktopbased applications, ou appoach can be natually integated into desktop seach, to facilitate uses seach, bowsing, and discovey local pesonal music esouce. Futhemoe, if metadata and use listening pefeences ae available, ou solution can be futhe impoved by combining local acousticbased seach esults with CF-based and meta-based infomation etieved fom the Web. This is one of ou diections in the futue. 6. CONCLUSIONS In this pape we have pesented a seach-based solution fo scalable music ecommendation. In this solution, though acoustic analysis, music pieces ae fist tansfomed to sequences of music signatues. Based on this, an LSH-based scale-sensitive method is pesented to index the music pieces 1073

10 fo effective similaity seach. Accoding to a given data scale, this method can numeically estimate the appopiate paametes to index vaious scales of music collections, and thus guaantees that an optimum numbe of neaest neighbos can be etuned in seach. In the ecommendation stage, epesentative signatues fom the snippets of a seed piece ae fist selected as quey tems to etieve pieces with simila melodies fom the indexed dataset. Then, a elevance function is designed to sot the seach esults by consideing citeia like matching atio, tempoal ode, tem weight, and matching confidence. In addition, we also popose a stategy to geneate dynamic playlists using the seach esults. Expeimental evaluations have shown that the poposedappoachachievedpomisingpefomanceonsystem efficiency, content consistency, and subjective satisfaction fo vaious music collections fom aound 1,000 music pieces to moe than 100,000 pieces. Although the poposed solution has been veified to be feasible fo music ecommendations, thee is still consideable oom fo futhe investigation and impovement. Fo example, besides the elevance (dynamic) anking, static anks such as sound quality and music populaity can also be integated to find bette suggestions. Moeove, moe sophisticated acoustic featues should be evaluated to discove those that ae moe suitable fo music ecommendation. Lasting pesonalized music ecommendation, use pefeences should be modeled by tacking opeational behavio and listening histoies. These ae diections of ou futue wok. 7. REFERENCES [1] Last.fm The Social Music Revolution. [2] Owl multimedia use YOUR music to find New music! [3] Pandoa Intenet Radio and Music Genome Poject. [4] J.-J. Aucoutuie and F. Pachet. Scaling up music playlist geneation. In Poc. IEEE ICME 02, volume 1, pages , Lausanne, Aug [5] C.J.C.Buges,D.Plastina,J.C.Platt,E.Renshaw, and H. S. Malva. Using audio fingepinting fo duplicate and thumbnail geneation. In Poc. IEEE ICASSP 05, volume 3, pages 9 12, Philadelphia, PA, USA, Ma [6] C. J. C. Buges, J. C. Platt, and S. Jana. Distotion disciminant analysis fo audio fingepinting. IEEE Tans. Speech and Audio Pocessing, 11(3): , May [7] P. Cano, E. Batlle, E. Gómez, L. Gomes, and M. Bonnet. Audio fingepinting: concepts and applications, chapte 17, pages Computational Intelligence fo Modelling and Pediction. Spinge-Velag, [8] P. Cano, M. Makus, and N. Wack. An industial-stength content-based music ecommendation system. In Poc. SIGIR 05, pages , Salvado, Bazil, Aug [9] P. Ciaccia, M. Patella, and P. Zezula. A cost model fo similaity queies in metic spaces. In Poc. PODS 98, pages 59 68, Seattle, USA, Jun [10] W. Cohena and W. Fana. Web-collaboative filteing: ecommending music by cawling the Web. Compute Netwoks, 33(1-6): , Jun [11] M. Data, N. Immolica, P. Indyk, and V. S. Miokni. Locality-sensitive hashing scheme based on p-stable distibutions. In Poc. SCG 04, pages , Booklyn, NY, USA, Jun [12] J. Goldbege and S. Roweis. Hieachical clusteing of a mixtue model. In Poc. NIPS 04, pages , Vancouve, Canada, Dec [13] Y.-C. Huang and S.-K. Jeng. An audio ecommendation system based on audio signatue desciption scheme in MPEG-7 audio. In Poc. IEEE ICME 04, pages , Taipei, Taiwan, Aug [14] P. Indyk and R. Motwani. Appoximate neaest neighbos: towads emoving the cuse of dimensionality. In Poc. SOTC 98, pages , Dallas, Texas, USA, Jun [15] D. Jiang and G. Xu. Tunable locality sensitive hashing: a unified appoach to nea duplicate detection. Submit to ICDE 08, Cancún, México, Ap [16] P.Knees,T.Pohle,M.Schedl,andG.Widme. Combining audio-based similaity with Web-based data to acceleate automatic music playlist geneation. In Poc. ACM MIR 06, pages , Santa Babaa, CA, USA, Oct [17] N. Kosugi, Y. Nishihaa, T. Sakata, M. Yamamuo, and K. Kushima. A pactical quey-by-humming system fo a lage music database. In Poc. ACM Multimedia 00, pages , Los Angeles, Oct [18] Q.Li,B.M.Kim,D.H.Guan,andD.Oh.Amusic ecommende based on audio featues. In Poc. SIGIR 04, pages , Sheffield, Jul [19] B. Logan. Music ecommendation fom song sets. In Poc. ISMIR 04, pages , Bacelona, Oct [20] L. Lu and H.-J. Zhang. Automated extaction of music snippet. In Poc. ACM Multimedia 03, pages , Bekeley, USA, Oct [21] H. S. Malva. Fast algoithm fo the modulated complex lapped tansfom. IEEE Signal Pocessing Lettes, 10(1):8 10, Jan [22] E. Pampalk, T. Pohle, and G. Widme. Dynamic playlist geneation based on skipping behavio. In Poc. ISMIR 05, pages , London, Sept [23] S. Pauws, W. Vehaegh, and M. Vossen. Fast geneation of optimal music playlist using local seach. In Poc. ISMIR 06, Victoia, Canada, Oct [24] J. C. Platt, C. J. C. Buges, S. Swenson, C. Weae, and A. Zheng. Leaning a Gaussian pocess pio fo automatically geneating music playlists. In Poc. NIPS 01, pages , Vancouve, Canada, Dec [25] R. Ragno, C. J. C. Buges, and C. Heley. Infeing similaity between music objects with application to playlist geneation. In Poc. ACM MIR 05, pages 73 80, Singapoe, Nov [26] S. Robetson, H. Zaagoza, and M. Taylo. Simple BM25 extension to multiple weighted fields. In Poc. CIKM 04, pages 42 49, Washington, D.C., USA, Nov [27] K. Yoshii, M. Goto, K. Komatani, T. Ogata, and H. Okuno. Hybid collaboative and content-based music ecommendation using pobabilistic model with latent use pefeence. In Poc. ISMIR 06, Victoia, Canada, Oct