Automated Transcription of a Lyric s Melody. David Branner. Hacker School, New York

Size: px

Start display at page:

Download "Automated Transcription of a Lyric s Melody. David Branner. Hacker School, New York"

Gabriel Potter
5 years ago
Views:

1 Automated Transcription of a Lyric s Melody David Branner Hacker School, New York

2 Automated* Transcription of a Lyric s Melody David Branner Hacker School, New York

3 Automated* Transcription of a Lyric s Melody (*if you ve done a lot of preliminary manual work) David Branner Hacker School, New York

4 In September I reported on an experiment to transcribe the solo singing voice with automated tools. Branner, Automated* Transcription of a Lyric s Melody p. 4

5 In September I reported on an experiment to transcribe the solo singing voice with automated tools. I have recordings that I want to study specifically, I want to study how the melody reflects the organization of the words that the music is set to. Branner, Automated* Transcription of a Lyric s Melody p. 5

6 In September I reported on an experiment to transcribe the solo singing voice with automated tools. I have recordings that I want to study specifically, I want to study how the melody reflects the organization of the words that the music is set to. I want the text, annotated with its melody. A normal musical transcription supplies the opposite of that: a melody annotated with words. Branner, Automated* Transcription of a Lyric s Melody p. 6

7 My earlier transcription (audio => MIDI => Python editing) went very badly because of tremolo in the singer s voice and reverberation in the recording. Branner, Automated* Transcription of a Lyric s Melody p. 7

8 My earlier transcription (audio => MIDI => Python editing) went very badly because of tremolo in the singer s voice and reverberation in the recording. In the end I transcribed the piece by hand, using an open-source program called MuseScore. Branner, Automated* Transcription of a Lyric s Melody p. 8

9 My earlier transcription (audio => MIDI => Python editing) went very badly because of tremolo in the singer s voice and reverberation in the recording. In the end I transcribed the piece by hand, using an open-source program called MuseScore. Example: Branner, Automated* Transcription of a Lyric s Melody p. 9

10 Initially I was pleased because MuseScore can play back the melody I transcribe, speeding the work. Branner, Automated* Transcription of a Lyric s Melody p. 10

11 Initially I was pleased because MuseScore can play back the melody I transcribe, speeding the work. But I was in luck because it can also export the score to an open-source format called MusicXML. Branner, Automated* Transcription of a Lyric s Melody p. 11

12 Initially I was pleased because MuseScore can play back the melody I transcribe, speeding the work. But I was in luck because it can also export the score to an open-source format called MusicXML. This turned out to allow me to do exactly what I had originally wanted to study the melody of each syllable, quantitatively. Branner, Automated* Transcription of a Lyric s Melody p. 12

13 Here is a single note of the 1100-odd note performance I transcribed (the text is Taiwanese in Church romaji): Branner, Automated* Transcription of a Lyric s Melody p. 13

14 Here is a single note of the 1100-odd note performance I transcribed (the text is Taiwanese in Church romaji): Branner, Automated* Transcription of a Lyric s Melody p. 14

15 Here is a single note of the 1100-odd note performance I transcribed (the text is Taiwanese in Church romaji): and here is the XML for that note: Branner, Automated* Transcription of a Lyric s Melody p. 15

16 Here is a single note of the 1100-odd note performance I transcribed (the text is Taiwanese in Church romaji): and here is the XML for that note: <measure number="17"> <attributes> <time> <beats>2</beats> <beat-type>4</beat-type> </time> </attributes> <note> <pitch> <step>d</step> <octave>4</octave> </pitch> <duration>12</duration> <voice>1</voice> <type>quarter</type> <dot/> <stem>down</stem> <lyric number="1"> <syllabic>single</syllabic> <text font-family="times New Roman">chuì</text> </lyric> <lyric number="2"> <syllabic>single</syllabic> <text font-family="times New Roman"> </text> </lyric> </note> Branner, Automated* Transcription of a Lyric s Melody p. 16

17 Here is a single note of the 1100-odd note performance I transcribed (the text is Taiwanese in Church romaji): <measure number="17"> <attributes> <time> <beats>2</beats> <beat-type>4</beat-type> </time> </attributes> <note> <pitch> <step>d</step> <octave>4</octave> </pitch> <duration>12</duration> <voice>1</voice> and here is the XML for that note: <type>quarter</type> <dot/> <stem>down</stem> <lyric number="1"> <syllabic>single</syllabic> <text font-family="times New Roman">chuì</text> </lyric> <lyric number="2"> <syllabic>single</syllabic> <text font-family="times New Roman"> </text> </lyric> </note> Branner, Automated* Transcription of a Lyric s Melody p. 17

18 That s exactly what I need to turn Branner, Automated* Transcription of a Lyric s Melody p. 18

19 That s exactly what I need to turn words set to music (traditional score-notation) Branner, Automated* Transcription of a Lyric s Melody p. 19

20 That s exactly what I need to turn words set to music (traditional score-notation) into Branner, Automated* Transcription of a Lyric s Melody p. 20

21 That s exactly what I need to turn words set to music (traditional score-notation) into melody on each syllable. Branner, Automated* Transcription of a Lyric s Melody p. 21

22 That s exactly what I need to turn words set to music (traditional score-notation) into melody on each syllable. The point is to have a representation of the melody organized by words, rather than the other way around. Branner, Automated* Transcription of a Lyric s Melody p. 22

23 Another example: Branner, Automated* Transcription of a Lyric s Melody p. 23

24 Another example: Branner, Automated* Transcription of a Lyric s Melody p. 24

25 Another example: <note> <pitch> <step>e</step> <octave>4</octave> </pitch> <duration>6</duration> <voice>1</voice> <type>eighth</type> <dot/> <stem>down</stem> <beam number="1">begin</beam> <notations> <slur type="start" number="1"/> </notations> <lyric number="1"> <syllabic>end</syllabic> <text font-family="times New Roman">bông</text> </lyric> <lyric number="2"> <syllabic>single</syllabic> <text> </text> </lyric> </note> <note> <pitch> <step>g</step> <octave>4</octave> </pitch> <duration>2</duration> <voice>1</voice> <type>16th</type> <stem>down</stem> <beam number="1">continue</ beam> <beam number="2">forward hook</beam> </note> <note> <pitch> <step>e</step> <octave>4</octave> </pitch> <duration>4</duration> <voice>1</voice> <type>eighth</type> <stem>down</stem> <beam number="1">continue</ beam> </note> <note> <pitch> <step>g</step> <octave>4</octave> </pitch> <duration>4</duration> <voice>1</voice> <type>eighth</type> <stem>down</stem> <beam number="1">end</beam> <notations> <slur type="stop" number="1"/> </notations> </note> Branner, Automated* Transcription of a Lyric s Melody p. 25

26 Another example: <note> <pitch> <step>e</step> <octave>4</octave> </pitch> <duration>6</duration> <voice>1</voice> <type>eighth</type> <dot/> <stem>down</stem> <beam number="1">begin</beam> <notations> <slur type="start" number="1"/> </notations> <lyric number="1"> <syllabic>end</syllabic> <text font-family="times New Roman">bông</text> </lyric> <lyric number="2"> <syllabic>single</syllabic> <text> </text> </lyric> </note> <note> <pitch> <step>g</step> <octave>4</octave> </pitch> <duration>2</duration> <voice>1</voice> <type>16th</type> <stem>down</stem> <beam number="1">continue</ beam> <beam number="2">forward hook</beam> </note> <note> <pitch> <step>e</step> <octave>4</octave> </pitch> <duration>4</duration> <voice>1</voice> <type>eighth</type> <stem>down</stem> <beam number="1">continue</ beam> </note> <note> <pitch> <step>g</step> <octave>4</octave> </pitch> <duration>4</duration> <voice>1</voice> <type>eighth</type> <stem>down</stem> <beam number="1">end</beam> <notations> <slur type="stop" number="1"/> </notations> </note> Branner, Automated* Transcription of a Lyric s Melody p. 26

27 It is easy to parse the XML using the lxml library; lxml s root.xpath method allows me request all the note elements as a generator, and I can step through the generator s output, identifying syllables and then retrieving the melody to assigned to those syllables. Branner, Automated* Transcription of a Lyric s Melody p. 27

28 Important edge cases: Branner, Automated* Transcription of a Lyric s Melody p. 28

29 Important edge cases: 1. rests they have no pitch content; also, since they don t correspond to syllables, consecutive rests can be collapsed into one; Branner, Automated* Transcription of a Lyric s Melody p. 29

30 Important edge cases: 1. rests they have no pitch content; also, since they don t correspond to syllables, consecutive rests can be collapsed into one; 2. notes tied (but not slurred) together since they correspond to one or part of one syllable and are all the same pitch, they can be collapsed into one; Branner, Automated* Transcription of a Lyric s Melody p. 30

31 Important edge cases: 1. rests they have no pitch content; also, since they don t correspond to syllables, consecutive rests can be collapsed into one; 2. notes tied (but not slurred) together since they correspond to one or part of one syllable and are all the same pitch, they can be collapsed into one; 3. melisma more than one note per vowel or syllable represented as a series of notes on one syllable. Branner, Automated* Transcription of a Lyric s Melody p. 31

32 Output of the first note above: Branner, Automated* Transcription of a Lyric s Melody p. 32

33 Output of the first note above: Branner, Automated* Transcription of a Lyric s Melody p. 33

34 Output of the first note above:... ('chuì', [{'pitch_data': {'octave': '4', 'step': 'D'}, 'lyric_2': {'syllabic': 'single', 'text': ' '}, 'duration': 12}]),... Branner, Automated* Transcription of a Lyric s Melody p. 34

35 Output of the more melismatic second example above: Branner, Automated* Transcription of a Lyric s Melody p. 35

36 Output of the more melismatic second example above: Branner, Automated* Transcription of a Lyric s Melody p. 36

37 Output of the more melismatic second example above:... ('bông', [{'pitch_data': {'octave': '4', 'step': 'E'}, 'lyric_2': {'syllabic': 'single', 'text': ' '}, 'duration': 6}, {'pitch_data': {'octave': '4', 'step': 'G'}, 'duration': 2}, {'pitch_data': {'octave': '4', 'step': 'E'}, 'duration': 4}, {'pitch_data': {'octave': '4', 'step': 'G'}, 'duration': 4}]),... Branner, Automated* Transcription of a Lyric s Melody p. 37

38 Output of the more melismatic second example above:... ('bông', [{'pitch_data': {'octave': '4', 'step': 'E'}, 'lyric_2': {'syllabic': 'single', 'text': ' '}, 'duration': 6}, {'pitch_data': {'octave': '4', 'step': 'G'}, 'duration': 2}, {'pitch_data': {'octave': '4', 'step': 'E'}, 'duration': 4}, {'pitch_data': {'octave': '4', 'step': 'G'}, 'duration': 4}]),... Branner, Automated* Transcription of a Lyric s Melody p. 38

39 Final output is list of tuples: [(syllable, [note-dicts]), ]. Branner, Automated* Transcription of a Lyric s Melody p. 39

40 Final output is list of tuples: [(syllable, [note-dicts]), ]. Next steps: Branner, Automated* Transcription of a Lyric s Melody p. 40

41 Final output is list of tuples: [(syllable, [note-dicts]), ]. Next steps: 1. Conversion of 'octave' and 'step' attributes of 'pitch_data' to MIDI pitch numbers. Branner, Automated* Transcription of a Lyric s Melody p. 41

42 Final output is list of tuples: [(syllable, [note-dicts]), ]. Next steps: 1. Conversion of 'octave' and 'step' attributes of 'pitch_data' to MIDI pitch numbers. 2. Analysis of the melodic values of syllables and language phrases. Branner, Automated* Transcription of a Lyric s Melody p. 42

43 End

Assessment Schedule 2017 Music: Demonstrate knowledge of conventions in a range of music scores (91276)

NCEA Level 2 Music (91276) 2017 page 1 of 8 Assessment Schedule 2017 Music: Demonstrate knowledge of conventions in a range of music scores (91276) Assessment Criteria Demonstrating knowledge of conventions