Rhythm related MIR tasks

Transcription

1 Rhythm related MIR tasks Ajay Srinivasamurthy 1, André Holzapfel 1 1 MTG, Universitat Pompeu Fabra, Barcelona, Spain 10 July, 2012 Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

2 1 Rhythm 2 Onset detection 3 Tempo Estimation and Beat Tracking 4 Meter and Time Signature Recognition 5 Rhythmic Similarity/Classification 6 Structural Analysis Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

3 Outline 1 Rhythm 2 Onset detection 3 Tempo Estimation and Beat Tracking 4 Meter and Time Signature Recognition 5 Rhythmic Similarity/Classification 6 Structural Analysis Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

4 Rhythm Clayton (1996) Cooper and Meyer (1960) Rhythm: the way one or more unaccented events are grouped in relation to an accented one. London (2001) Rhythm: pattern of durations that is phenomenally present in the music. There is rhythm without meter, periodicity, or even without pulse! Examples: Turkish taksim, Beijing opera, Indian alap MIR research concentrated on rhythm in music with meter (mainly Western music, typically 4/4) It is not only us : Clayton (1996) reports the same tendency for musicology. Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

5 Outline 1 Rhythm 2 Onset detection 3 Tempo Estimation and Beat Tracking 4 Meter and Time Signature Recognition 5 Rhythmic Similarity/Classification 6 Structural Analysis Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

6 Onset detection Bello (2005) The figure depicts the flowchart of a standard onset detection algorithm. Evaluation happens using either manually annotated onsets, or onsets derived from instruments with MIDI outputs. 4.1 freq/khz (a) Cello Spectr. Magn time/s (b) Guitar Spectr. Magn. Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

7 Onset Detection Pre-processing Multiband processing Separation of percussive/harmonic content Reduction Using temporal, spectral magnitude, phase, or F0 information. Feature fusion was proposed to improve performance. Probabilistic models and neural networks were also applied for reduction. Rhythmic structure was used to improve onset detection recently. Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

8 Outline 1 Rhythm 2 Onset detection 3 Tempo Estimation and Beat Tracking 4 Meter and Time Signature Recognition 5 Rhythmic Similarity/Classification 6 Structural Analysis Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

9 What is Beat Tracking? Tapping one s foot in time to music" [Davies-07] Extract a sequence of beat instants and the corresponding inter-beat intervals given an audio file Perceptually accurate beat times Locally constant inter-beat intervals (IBI) Causal v/s non-causal beat tracking Beat Tracking on Symbolic v/s audio data Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

10 What is Beat Tracking? (contd...) Challenges Tempo variation On-beat and off-beat Genre variation Different time signatures Before we begin - Fundamental Questions Is the problem well defined? - Metrical levels Is it an intuitive and an easy task for humans? Ground truth for evaluation? All (most) beats occur at onsets, but not all onsets are beats" Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

11 Components of a Beat Tracking System Rhythmic Feature Extraction Extract relevant rhythm features Onset detection Onset Salience estimation Tempo Induction Determine the basic tempo/tempo hypotheses Tempo definition for Indian music? Beat Tracking Estimate Beat times Beat phase? Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

12 Approaches to Beat Tracking Tempo Induction Pulse selection methods e.g BeatRoot [Dixon-06] Periodicity functions e.g. [Klapuri-06, Davies-07, Ellis-07] Beat Tracking Multiple agent architecture e.g BeatRoot Statistical Model [Klapuri-06] Dynamic Programming [Ellis-07] Context Dependent Tracking [Davies-07] Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

13 Beat Tracking Systems Davies (2007) Dixon (2006) - BeatRoot Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

14 Examples of Beat Tracking Money (Pink Floyd) Clip Beats Charleston Dance piece Clip Beats Mahaganapatim Naata raga, Chaturashra Eka taala Clip Beats Light Indian classical Shivaranjani raga, Jhap like taal (pentuple meter) Clip Beats Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

15 Outline 1 Rhythm 2 Onset detection 3 Tempo Estimation and Beat Tracking 4 Meter and Time Signature Recognition 5 Rhythmic Similarity/Classification 6 Structural Analysis Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

16 Meter and Time Signature Recognition Long-term, high level rhythm description Beat similarity based approach Onset detection, Tempo estimation and Beat Tracking tracking necessary Beat Similarity matrix based approach: [Gainza-09] Recent work in Indian Music: [Gulati-11], [Miron-11] Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

17 Outline 1 Rhythm 2 Onset detection 3 Tempo Estimation and Beat Tracking 4 Meter and Time Signature Recognition 5 Rhythmic Similarity/Classification 6 Structural Analysis Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

18 Rhythm Similarity Figure: Example for periodicity descriptors (J.H.Jensen) Periodicity descriptors Can be considered state-of-the art in MIR. No beat estimation necessary can be applied to all types of signals Tempo robustness can be achieved in various degrees However, phase information is lost. Usually evaluated on dance music, or in music similarity tasks. Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

19 Rhythm Similarity: Sequential Description General Properties No information loss because not based on transform magnitudes. Disadvantage: Music must have a pulse which can be estimated. Paulus and Klapuri (2002) Using spectral centroid and loudness to derive pattern description. Dynamic Time Warping is used to compare patterns. Whiteley et al.(2007) Probabilistic framework is proposed to infer tempo and rhythmic patterns Only applied to MIDI signals in this paper, rhythmic patterns are given. Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

20 Outline 1 Rhythm 2 Onset detection 3 Tempo Estimation and Beat Tracking 4 Meter and Time Signature Recognition 5 Rhythmic Similarity/Classification 6 Structural Analysis Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

21 Structural Analysis Chorus detection Main goal: Detect repetitions of parts of a song Most common: Self-similarity matrix analysis Cover song detection Identify if two songs are different interpretations of the same composition Features: e.g. chroma features, chord estimations. Similarity measures e.g. dynamic programming, string matching. Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

22 Rhythm Structural segmentation Main goal: Obtain a musical meaningful segmentation of a song into large time-scale structures. Again, self similarities play a big role. Also, HMM were applied for labeling states at beat level, and then find similarities in the state distributions to obtain segments (Levy and Sandler (2008)). Figure: Compare structure of a query in form of a score to the structure of audio (Martin et al. 2009) Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23

23 References yon-05 Gouyon, F. A Computational Approach to Rhythm Description. PhD Thesis, Pompeu Fabra University, Barcelona, 2005 ixon-06 Dixon, S. Evaluation of The Audio Beat Tracking System Beatroot. Journal of New Music Research 36 (1), pp , 2006 vies-07 Davies, M. and Plumbley, M. Context-Dependent Beat Tracking of Musical Audio. IEEE Transactions on Audio, Speech, and Language Processing 15 (3), pp , Ellis-07 Ellis, D. P. W. Beat Tracking by Dynamic Programming. Journal of New Music Research, 36(1), pp , inza-09 Gainza M., Automatic musical meter detection, in Proc. ICASSP 2009, pp , Taipei, Taiwan EX-06 hle-03 Christian Uhle, Juergen Herre, Estimation of Tempo, Micro Time and Time Signature from Percussive Music, in Proc. of 6th Int. Conference on Digital Audio Effects (DAFX-03), London, UK, September 8-11, 2003 ulati-11 Sankalp Gulati, Vishweshwara Rao and Preeti Rao, Meter Detection from Audio for Indian Music, CMMR/FRSM 2011, Bhubhaneswar, March 2011 iron-11 M. Miron, Automatic Detection of Hindustani Talas. Master s thesis, Universitat Pompeu Fabra, Barcelona, Spain, Srinivasamurthy et al. (UPF) MIR tasks 10 July, / 23