Computational analysis of rhythmic aspects in Makam music of Turkey

Computational analysis of rhythmic aspects in Makam music of Turkey André Holzapfel MTG, Universitat Pompeu Fabra, Spain hannover@csd.uoc.gr 10 July, 2012 Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 1 / 29

1 Rhythm Workshop 2 Fundamental Approaches 3 Proposed Tasks Segmentation Usul recognition Fundamental pulse recognition Onset Detection Usul recognition Usul tracking/annotation 4 Conclusions Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 2 / 29

Outline 1 Rhythm Workshop 2 Fundamental Approaches 3 Proposed Tasks Segmentation Usul recognition Fundamental pulse recognition Onset Detection Usul recognition Usul tracking/annotation 4 Conclusions Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 3 / 29

Some examples for meter in Turkish makam music 1 Şarkı: Usul Düyek, Makam Kürdilihicazkar 2 Saz semaisi: Usul Aksaksemai, Makam Uşşak 3 Peşrev: Usul Devrikebir, Makam Isfahan Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 4 / 29

Fundamental Approaches LAYER 3 LAYER 2 LAYER 1 Figure: Concept of layers that build up the temporal structure in music 1.: Top-Down(-Top) Sentence Structure (L4) Usul Layer (L3) Beat Layer (L2) Onset Layer (L1) L2... Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 6 / 29

Fundamental Approaches 2.: Dual 1 MIDI-data Currently 1700 compositions available. Dataset can be extended according to our needs. 2 Parallel audio data Recordings of compositions available as MIDI. Currently 452 pieces with related MIDI. Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 7 / 29

Fundamental Approaches: Duality Some statistics Figure: Distribution in parallel data according to the form sirto yuruksemai turku sazsemaisi pesrev sarki 0 50 100 150 200 250 Number of pieces Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 8 / 29

Fundamental Approaches: Duality Some statistics Figure: Distribution in parallel data according to the makam suzinak evic acemkurdi acemasiran suzidil isfahan sehnaz neva sedaraban muhayyerkurdi ferahfeza saba mahur hicazkar muhayyer nihavent ussak kurdilihicazkar huzzam segah rast hicaz huseyni 0 5 10 15 20 25 30 35 Number of pieces Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 9 / 29

Fundamental Approaches: Duality Some statistics Figure: Distribution in parallel data according to the usul fahte semai musemmen agirduyek hafif turkaksagi senginsemai sofyan nimsofyan muhammes agiraksak yuruksemai devrikebir curcuna duyek aksaksemai aksak 0 10 20 30 40 50 60 70 80 Number of pieces Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 10 / 29

Fundamental Approaches 1.: Top-Down(-Top) Sentence Structure (L4) Usul Layer (L3) Beat Layer (L2) Onset Layer (L1) L2... 2.: Dual Parallel MIDI-Audio corpus 3.: Comparative What are the differences to styles in other music? Regular discussions between those people in Compmusic working on rhythm must happen. Proposed tools should be evaluated not only on data of the specific culture. A first step should be: How good can current approaches handle that problem? Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 11 / 29

The Research Menu Tasks in proposed chronological order 1 Segmentation (L4) 2 Usul recognition (L3) 3 Fundamental pulse recognition (L2) 4 Detection of Ornamentations (L1) 5 Usul tracking/annotation (L2) 6 Usul recognition revisited (L3) LAYER 3 LAYER 2 LAYER 1 Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 12 / 29

Segmentation Preparation of aligned data Current state: 82 Peşrev, 72 Saz semaisi, 217 Şarkı Manual annotation of sections necessary Systematic evaluation and improvement of current segmentation approach. How does the approach compare to methods tailored for Western music? Can our approach be of advantage for Western music? Examine signal characteristic at phrase/segment boundaries. Extension: Examine self-similarity on audio. Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 14 / 29

Usul recognition Usul The meter of a composition in makam music of Turkey is defined by a verbal sequence of certain length that defines a series of weak and strong intonations in time. Example for an usul: Aksaksemai Figure: Velveleli Figure: Simple form Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 15 / 29

Usul recognition Preliminary study on symbolic data In what aspects differ compositions when they follow different usul? 3 Possible aspects Onset locations Note durations Inter-onset-interval histograms Metrical contradiction, Example 1, Example 2 Note intervals? Mean Duration (1/16) Normalized Count Weight 2.5 2 1.5 1 0.5 0 4 3 2 1 0 3 2 1 0 2 4 6 8 10 12 14 16 18 Location (1/16) (a) Onset location 2 4 6 8 10 12 14 16 18 Location (1/16) (b) Duration 2 4 6 8 10 12 14 16 18 Weight location (1/16) (c) Theory Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 16 / 29

Fundamental pulse Do Turks beat it? This task is similar to beat tracking in Western music. But... Can we speak of a beat in Turkish music? Would Turkish musicians/listerners tap to some regular pulse? Do they agree in that? Approach Expert interviews with Turkish musicians and listeners. Tapping experiment: Turkish and non-turkish listeners will be asked to tap a pulse to music. Can we obtain a consistent answer from interviews and computational analysis of taps? Do Turkish and non-turkish tap sequences differ? Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 17 / 29

Fundamental pulse Do our algorithms beat it? How well do existing state-of-the-art approaches in finding annotated pulses? How can their performance be improved? Approach Pulse annotate a set of pieces that we have the taps for. How well do algorithms and tappers agree with that pulse? Can we improve algorithms by providing knowledge about onset distributions or tempo priors? How is algorithmic performance in comparison with Western music? Can a committee of beat trackers help that use various signal characteristics as input? How does performance on real audio compare with performance on synthesized audio? Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 18 / 29

Ornamentation detection Is there anything specific about Turkish onsets? Transient characteristics can be considered widely the same as for Western music (exception: Ney). Playing style differs widely. Figure: A short oud sample Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 19 / 29

Ornamentation detection Approach Derive an alignment between onsets existent in a score and those detected in audio. Combine signal aspects for onset detection, concentrating on F0 characteristics. Detect areas where number of onsets in audio deviates from that in the score. Figure: Onset Example Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 20 / 29

Ornamentation detection Questions 1 Which of those overdetections are related to ornamentations? 2 What type of ornamentations do exist? 3 Do ornamentations appear in specific parts of the meter? 4 How strong do they depend on the type of instrument? 5 Can we automatically detect them? 6 Can we remove them?? (a risky task...) Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 21 / 29

Usul recognition (audio) Figure: Example for periodicity descriptors (J.H.Jensen) Periodicity descriptors Periodicity based models describe the strength of pulses at different tempi. As they basically describe a spectral magnitude, they loose information contained in phase. This implies that they cannot describe the sequential order of events that are related to the magnitudes. Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 22 / 29

Usul recognition (audio) Sequential descriptors Example: Onset frequency count histograms, note length histograms, or combinations thereof. In a first step, compare those descriptors with those derived from symbolic data! Use them to derive a classification or similarity measurement. How do these descriptors perform compared with periodicity descriptors? What are the additional difficulties/benefits we have on audio? Observe dependence on good fundamental pulse recognition. Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 23 / 29

Usul tracking/annotation Tracking vs. annotation Usul annotation: Assign the syllables of the usul to time instances in the piece off-line. Usul tracking: Causal, real time process, useful for automatic accompaniment of performance. It should be discussed if tracking would be of practical value. Did Turks beat it? Nowadays, we almost always have percussion in Şarkı, sometimes even drums. In old recordings, percussion are less common. This presents us with a wide variety of signal characteristics for tracking! Surely, an annotation of the usul-syllables would be more meaningful for musicians than generating a pulse. Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 24 / 29

Usul tracking/annotation Case 1: percussive accompaniment Example Try to separate percussive and harmonic elements (e.g. Fitzgerald). Do an usul annotation using the percussive onsets, possibly supported by harmonic part characteristics. Cluster percussive sounds in strong and weak beats. Feasible even without knowing the usul, might even be used for classification. Case 1: only melodic instruments Is it easy to find fundamental pulse? 1 Yes: align usul pattern with pulse synchronous features. 2 No: What we are going to do has nothing to do with beat tracking... Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 25 / 29

Usul tracking/annotation Music with meter, but without obvious pulse This is apparently a contradiction, as music having a meter is a subgroup of music having a pulse. Parallels to Western music: often encountered in solo performances of e.g. Romantic music. However, the way musicians perceive the underlying meter is likely to be different in Turkish music. Some thoughts: As pulse cannot be detected, use information from other layers (onsets, phrase boundaries) Try to find periodic regions. Task is dramatically simplified when having symbolic data as well The task bears high resemblance to seyir analysis (melodic development in improvisation). Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 27 / 29

Conclusions Presenting the work In forms of Journal and conference contributions Demo platform for musicians: 1 An automatic alignment between score and audio enables for flexible browsing 2 Ornamentation detection enables to propose specific phrases for focussed study 3 Platform: RepoVizz? 4 Also makam analysis results should be included into the platform. Personal goal Do usul fit into our common understanding of meter in music? Does it make sense to impose a hierarchical structure to meter in makam music? Holzapfel et al. (MTG/UPF) Rhythm research in Turkish music 10 July, 2012 29 / 29