Searching for Similar Phrases in Music udio an Ellis Laboratory for Recognition and Organization of Speech and udio ept. Electrical Engineering, olumbia University, NY US http://labrosa.ee.columbia.edu/ 1. Motivation: Similar Phrases. Phrase Matching System 3. Experiments. onclusions & uture Similar Phrases in Music - Ellis 7-1-1 p. 1 /1
1. Motivation: Similar Phrases Idea: Music is a sequence of reused pieces e.g. melodic runs, chord sequences,... an we identify them in large music databases?... which we have i.e. machine learning pplications classification and matching of pieces compressed representation data-driven musicology Similar Phrases in Music - Ellis 7-1-1 p. /1
ommon Phrase iscovery Beat tracking Music audio hroma features Key normalization Landmark identification Locality Sensitive Hash Table hop up music into short descriptions of musical content -beat beat-chroma matrices? hoose a few that appear to be starts Put into LSH table (similar items fall in same bin) ind the bins with most entries Similar Phrases in Music - Ellis 7-1-1 p. 3 /1
. Phrase Matching: Beat Tracking oal: One feature vector per beat (tatum) for tempo normalization, efficiency Onset Strength Envelope sumf(max(, difft(log X(t, f) ))) freq / mel 3 Ellis, 7 5 15 time / sec utocorr. + window global tempo estimate 1.5 BPM 3 5 7 9 lag / ms samples Similar Phrases in Music - Ellis 7-1-1 p. /1
Piano scale hroma eatures hroma features convert spectral energy into musical weights in a canonical octave freq / khz 3 1 i.e. 1 semitone bins time / sec 3 5 7 time / frames an resynthesize as Shepard Tones level / db Piano chromatic scale all octaves at once 1 Shepard tone spectra - - -3 - -5-5 15 5 freq / Hz chroma freq / khz 3 1 I chroma Shepard tone resynth time / sec Similar Phrases in Music - Ellis 7-1-1 p. 5 /1
Key Estimation ovariance of chroma reflects key Normalize by transposing for best fit single aussian model of one piece find ML rotation of other pieces model all transposed pieces iterate until convergence aligned chroma Taxman Eleanor Rigby I'm Only Sleeping Love You To ligned lobal model She Said She Said ood ay Sunshine nd Your Bird an Sing Similar Phrases in Music - Ellis 7-1-1 p. /1 Ellis ISSP 7 Yellow Submarine aligned chroma
Landmark Location Looking for beginnings of phrases e.g. abrupt change in harmony, instruments, etc. use likelihood ratio test: weighted windows either side of boundary vs. all hoose top freq / khz ome Together - Spectrogram, Beat-sync chromogram, and top segment points 3 locally-normalized peaks 1 1.. to control data size 5 Similar Phrases in Music - Ellis 5 7-1-1 15 p. 7 /1 time / sec 5
Locality Sensitive Hashes oal: Quantize high-dimensional data so similar items fall into same bin.. for fast and scalable nearest-neighbor search Idea: Multiple random scalar projections each one will tend to keep neighbors nearby items close together in all projections are probably neighbors from Slaney & asey Similar Phrases in Music - Ellis 7-1-1 p. /1
3. Experiments ata artist - artist x albums = 113 tracks (up to) landmarks/track = 1,7 patches each patch = 1 s x beats ( dims) # neighbors within. - 7 a patches Performance 9 feature calculation: ~ min 7 LSH 1k NNs: 5 ~ 3 sec 51 patches have 3 > NNs within r =. count 5 15 5 3 35 # near neighbors Similar Phrases in Music - Ellis 7-1-1 p. 9 /1
Results - artist radiohead 1-You 19.7-19.5s green day -Blood Sex nd Booze 1.3-19.s 1 1 radiohead 7-Ripcord 177.-1.5s radiohead 11-enchildren hidden 3.9-35.s 1 1 5 15 beat mainly sustained notes 5 15 beat Similar Phrases in Music - Ellis 7-1-1 p. /1
Results - Beatles Only the Beatles tracks ll beat offsets = 1,75 patches LSH takes 3 sec - approx NlogN in patches? High-pass along time to avoid sustained notes Song filter remove hits in same track 1 1 -I Should Have Known Better 9.-97.7s 9-Martha My ear 9.9-9.s 5 15 beat Similar Phrases in Music - Ellis 7-1-1 p. 11/1 1 1 5-Here There nd Everywhere 1.1-.5s 1-Piggies.-9.s 5 15 beat
Summary / onclusions Beat tracking Music audio hroma features Key normalization Landmark identification Locality Sensitive Hash Table Lots of data find motifs by counting near neighbors ommon patterns e.g. melodic/harmonic-beat sequences uture different features and/or pre-emphasis better landmark points complete dictionary Similar Phrases in Music - Ellis 7-1-1 p. 1/1