Musical Data Bases Semantic-oriented Comparison of Symbolic Music Documents

Transcription

1 Semantic-oriented Comparison of Symbolic Music Documents ISST Chemnitz University of Technology Information Systems & Software Engineering Informatiktag 2006

2 Content Project Approaches in Music Information Retrieval (MIR) Our approach: The Lead Sheet Model Results Further Development

3 Project Chair Data Management Dr. Frank Seifert Chair Information Systems & Software Engineering Michael Rentzsch Algorithms and Methods: Analysing music data Comparing music documents Indexing in music data bases

4 Project Chair Data Management Dr. Frank Seifert Chair Information Systems & Software Engineering Michael Rentzsch Algorithms and Methods: Analysing music data Comparing music documents Indexing in music data bases

5 Existing Approaches Symbolic representation Contour of Melody Edit distance Musical Edit-Distanz (Melody/Rhythm/Harmonisation) Geometric Modelling Piano Sub-symbolic representation Audio Fingerprinting Tone and harmony recognition Methods from symbolic representation

6 Existing Approaches Symbolic representation Contour of Melody Edit distance Musical Edit-Distanz (Melody/Rhythm/Harmonisation) Geometric Modelling Piano Sub-symbolic representation Audio Fingerprinting Tone and harmony recognition Methods from symbolic representation

7 Existing Approaches Symbolic representation Contour of Melody Edit distance Musical Edit-Distanz (Melody/Rhythm/Harmonisation) Geometric Modelling Piano Sub-symbolic representation Audio Fingerprinting Tone and harmony recognition Methods from symbolic representation

8 Critical Evaluation Fingerprinting: Original pieces (recordings) only Simple Edit distance: No musical background based on statistics No degree of equality (motifs/patterns) No musical context No comparison of entire pieces of music Alternatives are required Knowledge about musical context Equality of music documents

9 Critical Evaluation Fingerprinting: Original pieces (recordings) only Simple Edit distance: No musical background based on statistics No degree of equality (motifs/patterns) No musical context No comparison of entire pieces of music Alternatives are required Knowledge about musical context Equality of music documents

10 Lead Sheet Model (1/2) PhD-Thesis Musikalische Datenbanken Grundlagen semantischer Indexierung von Tondokumenten Basic idea: characteristic motif (CM) Musical pattern Melody Rhythm Tempo Harmonisation 4 = 90 F C7

11 Lead Sheet Model (2/2) Term lead sheet from jazz and pop music Adds structural and sematic (relation) aspect to model Modelling as a graph

12 Lead Sheet Templates 1 Starting point: Original piece (e. g. traditional) 2 Finding charakteristic motifs {CM i } 3 Analysing semantic context template

13 Lead Sheet Templates 1 Starting point: Original piece (e. g. traditional) 2 Finding charakteristic motifs {CM i } 3 Analysing semantic context template CM 1 CM 2 CM 2 CM 3 CM 4

14 Lead Sheet Templates 1 Starting point: Original piece (e. g. traditional) 2 Finding charakteristic motifs {CM i } 3 Analysing semantic context template

15 Equality of Music Documents Central Idea Every piece of music can be represented as a template using the identified motifs and the information in the lead sheet graph. Thus, comparing two pieces of music can be reduced to comparing these templates. Requires Methods to identify motifs (paying tribute to small variations) Distance metric for lead sheet templates

16 Equality of Music Documents Central Idea Every piece of music can be represented as a template using the identified motifs and the information in the lead sheet graph. Thus, comparing two pieces of music can be reduced to comparing these templates. Requires Methods to identify motifs (paying tribute to small variations) Distance metric for lead sheet templates

17 Equality function for motifs (1/2) 4? 4 Comparing 4 parameters of music: Melody Rhythm Tempo Harmonisation Voice 5 degrees of equality: E M, E R, E T, E H, E V [0...1] Weighting of parameters W p

18 Equality function for motifs (1/2) 4? 4 Comparing 4 parameters of music: Melody Rhythm Tempo Harmonisation Voice 5 degrees of equality: E M, E R, E T, E H, E V [0...1] Weighting of parameters W p

19 Equality function for motifs (1/2) 4? 4 Comparing 4 parameters of music: Melody Rhythm Tempo Harmonisation Voice 5 degrees of equality: E M, E R, E T, E H, E V [0...1] Weighting of parameters W p

20 Equality function for motifs (1/2) 4? 4 Comparing 4 parameters of music: Melody Rhythm Tempo Harmonisation Voice 5 degrees of equality: E M, E R, E T, E H, E V [0...1] Weighting of parameters W p

21 Equality function for motifs (1/2) 4? 4 Comparing 4 parameters of music: Melody Rhythm Tempo Harmonisation Voice 5 degrees of equality: E M, E R, E T, E H, E V [0...1] Weighting of parameters W p

22 Equality function for motifs (1/2) 4? 4 Comparing 4 parameters of music: Melody Rhythm Tempo Harmonisation Voice 5 degrees of equality: E M, E R, E T, E H, E V [0...1] Weighting of parameters W p

23 Equality function for motifs (1/2) 4? 4 Comparing 4 parameters of music: Melody Rhythm Tempo Harmonisation Voice 5 degrees of equality: E M, E R, E T, E H, E V [0...1] Weighting of parameters W p Melody 55% Rhythm 25% Tempo 10% Harmonisation 10%

24 Equality function for motifs (2/2) 4? 4 Determine E P Calculate overall degree of equality E = E V W p E p p p {M,R,T,H} Empirically determined: E 0, 75 similar

25 Comparing entire pieces of music For each piece Analyse document List of identified motifs and degree of equality D = [(m 1,e 1 ),(m 2,e 2 ),...,(m n,e n )] + Information from lead sheet graph Abstraction to template Comparing pieces Comparing templates Subgraph isomorphism Graph edit distance Self-defined distance metric

26 Comparing entire pieces of music For each piece Analyse document List of identified motifs and degree of equality D = [(m 1,e 1 ),(m 2,e 2 ),...,(m n,e n )] + Information from lead sheet graph Abstraction to template Comparing pieces Comparing templates Subgraph isomorphism Graph edit distance Self-defined distance metric

27 Comparing entire pieces of music For each piece Analyse document List of identified motifs and degree of equality D = [(m 1,e 1 ),(m 2,e 2 ),...,(m n,e n )] + Information from lead sheet graph Abstraction to template Comparing pieces Comparing templates Subgraph isomorphism Graph edit distance Self-defined distance metric

28 Comparing entire pieces of music For each piece Analyse document List of identified motifs and degree of equality D = [(m 1,e 1 ),(m 2,e 2 ),...,(m n,e n )] + Information from lead sheet graph Abstraction to template Comparing pieces Comparing templates Subgraph isomorphism Graph edit distance Self-defined distance metric

29 Self-defined Distance Metric Starting point: Two templates T 1, T 2 First step: Determine common sub-template (CST) Analyse CST on 4 levels: 1 Time Level 2 Structural Level 3 Semantic Level 4 Motif Level 4 distance values D E, Overall distance value: D(T 1,T 2 ) = (D Time, D Struct, D Semant, D Motif )

30 Time Level Evaluates length of CST in proportion to length of document (piece) t i Length (CST) / Length (piece) D Time = 1 2 (t 1 + t 2 ) States: The bigger D Time, the more similar T 1 and T 2

31 Structural Level Abstraction to lead sheets despite small modifications These modifications are rated (Type Relation) d i Modifications in sub-template i: D Struct = d i d States: The smaller D Struct, the more similar T 1 and T 2

32 Semantic Level What has been found in T 1 and T 2? Different values average States: The bigger D Semant, the more similar T 1 and T 2

33 Motif Level Average degree of equality of all identified motifs e i equality values D Motif = e i e States: The bigger D Motif, the more similar T 1 and T 2

34 Results (Example) Example from [1] Test set: 4 different, monophonic pieces of music Auld lang syne original Auld lang syne motif variations Medley: Auld lang syne and Oh, when the saints Oh, when the saints Compared to Auld lang syne Similarity calculated using prototype application Results illustrated as a 3d chart: without structural Level [1] Rentzsch, M., Seifert, F.: Semantic-based Similarity of Music. Pattern Recognition in Information Systems, Paphos (Cyprus), May 2006.

35 Similarities

36 Similarities

37 Similarities

38 Similarities

39 Similarities

40 Futher Development Build up test repository MELDEX library Improve equality function for motifs (musical aspects ) Use deduction when searching for motifs Apply methods to (symbolic) audio data Goal: Indexing documents in a music data base according to musical aspects

41 Thank you for your attention! Further information about our project can be found at