dvanced ourse omputer Science Music Processing Summer Term 2 Meinard Müller, Verena Konz Saarland University and MPI Informatik meinard@mpi-inf.mpg.de hord Recognition spects of Music Melody Piece of music Rhythm Harmony Harmony: The asis of Music Musical hords Pachelbel s anon ombination of three or more tones which sound simultaneously hord classes Triads including, minor, diminished, augmented chords many other more complex chords such as seventh chords Here: focus on and minor triads oversong ie ine (ie irma) Musical hords The chord Musical hords The minor chord erived from the scale erived from the minor scale ---- the root ---- the () third ---- the fifth ---- the root b ---- the (minor) third ---- the fifth
Structure of the 24 Major/Minor hords hord Recognition evelopment of automatic methods for the harmonic analysis of audio data minor 2 3 4 5 6 7 8 9 # # # # # pplications in the field of music information retrieval: music segmentation cover song identification audio matching music structure analysis hord Recognition iven: udio file Output: Segmentation and chord labeling :min :min :min :min :min :min hord Templates # # minor # minor # # # # # aseline Method for hord Recognition aseline Method for hord Recognition (2, 2 minor) (2, 2 minor) hroma feature extraction (framewise)
aseline Method for hord Recognition aseline Method for hord Recognition (2, 2 minor) hroma feature extraction (framewise) (2, 2 minor) hroma feature extraction (framewise) ompute for each frame the distance of the feature vector to the 24 templates ompute for each frame the distance of the feature vector to the 24 templates Selected chord according to template with minimal distance to respective feature vector Problems in hord Recognition xample: hopin Mazurka Op. 68 No.3 Problems in hord Recognition xample: xcerpt of Wagner s Meistersinger hromagram # # # #.5.2 # 2 3 4 5 6. Problems in hord Recognition xample: eethoven s ppasionata (mm.6-9, f minor) Problems in hord Recognition Problem: rame-wise chord analysis may not be meaningful hromagram 6-9:f xample: ach: Prelude, WV 846 # # #.5 Problem: roken chords # #.2. Measure-wise chord analysis necessary 5 52 53 54 55 56 57 58 59
Problems in hord Recognition Problem: mbiguity of chords Problems in hord Recognition xample: The eatles ``Let it be minor minor Problems in hord Recognition Problem: Reduction to the 24 /minor chords makes the recognition of more complex chords difficult/impossible! xample: xcerpt of Wagner s Meistersinger xample: Prelude, WV 846, mm.9-25 hromagram (from MII) # # # #.5.2 #. 2 3 4 5 6 xample: xcerpt of Wagner s Meistersinger xample: xcerpt of Wagner s Meistersinger hromagram (from audio) hromagram (from audio) # # Problem: udio is tuned more than half a semi-tone upwards # # Problem: udio is tuned more than half a semi-tone upwards # # # 2 3 4 5 6.5.2. # # # 2 3 4 5 6.5.2. Solution: djust frequency-chroma binning (e.g., by shifting filter bank)
ataset: 8 eatles songs (manually annotated) Usage of 6 differently shifted pitch filter banks (fractional semitone shifts,.25, 3,.5, 7, 5) in combination with the 2 cyclic chroma shifts 72 chroma feature versions omputation of chord labels for all 72 chroma versions using template-based chord recognizer xample: The eatles ``Lovely Rita Without tuning With tuning omputation of -measures for all 72 chroma versions onsider chroma version resulting in maximal -measure Key Relations: ircle of ifths Song Title Year Tuning -measure (original) -measure (tuning) Lovely Rita 967 7.3 42 Strawberry ields orever 967.25.59.547 Wild Honey Pie 968.5.78 66 Ticket To Ride 965.25 6 4 nother irl 965 7.8 46 oys 963 3 4 34 You ve ot To Hide Your Love way 965 3.577 4 o You Want To Know Secret 963.5 46.53 verage.53 verage for all 8 songs.526.559 rom http://en.wikipedia.org/wiki/ircle_of_fifths Key Relations: ircle of ifths Hidden Markov Models Observation: or tonality reasons, some chord progressions are more likely than others. X X2 X3 rom http://en.wikipedia.org/wiki/ircle_of_fifths Idea: Usage of Hidden Markov Models (HMMs) to model chord dependencies y Model Parameters: X: States y: Possible observations a: State transition probabilities b: Output probabilities y2 y3 y4
Hidden Markov Models xample: Weather States Probability P(Rainy) = P(Sunny) = Transition Probability Rainy Sunny Rainy Sunny Observation Probability Walk Shop lean Rainy..5 Sunny.5. Hidden Markov Models xample: hords States Probabilities P() = P(:min) = Transition Probabilities :min :min Observation Probabilities b :min.2 start :min.2,,,, b, Hidden Markov Models Two computational problems. Training: set model parameters (orward-ackward lgorithm) 2. valuation: find optimal state sequence (Viterbi lgorithm) Multi-Perspective valuation or automatically evaluating chord recognizers one needs chord-labeled ground truth data Training ata (Wav/ Midi + labels) Music Knowledge MIR researcher often need ground truth labels for audio data time-consuming task HMM hord Models Training Test ata (Unseen Wav/ Midi) Musicians usually annotate chords on the basis of a musical score (not audio data) valuation Recognized hord Multi-Perspective valuation iven: hord labels for a musical score Uninterpreted MII file representing the score omputed chord-labels for an audio recording Multi-Perspective valuation Score-based groundtruth chord-labels hopin Mazurka Op. 68 No. 3 (mm.-4) Overlayed score and audio chord labels udio chord-labels Strategy Synchronize the MII file with the audio recording Transfer the computed audio chord-labels to the MII time-axis (measure-axis) by using the synchronization result Warped audio chord-labels Multi-perspective overlay of score and audio chord labels
Multi-Perspective valuation xample: eethoven s ifth (mm.47-475) Joint chord recognition result (37 different performances) Multi-Perspective valuation xample: eethoven s ifth (mm.484-49) Joint chord recognition result (37 different performances) Multi-Perspective valuation Summary ridges the gap between musicians and MIR researchers llows for an in-depth error analysis on a musically meaningful time-axis (given in measures or beats) llows for comparing chord labeling procedures across different domains and different performances