Functional Generation of Harmony and Melody

Size: px

Start display at page:

Download "Functional Generation of Harmony and Melody"

Gwenda Willis
6 years ago
Views:

1 Functional Generation of Harmony and Melody José Pedro Magalhães Hendrik Vincent Koops Functional Art, Music, Modeling and Design September 2014 José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

2 An extremely useful model of musical harmony A couple of years ago I worked on a modelling musical harmony as a GADT in Haskell (together with W. Bas de Haas). From this model came: José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

3 An extremely useful model of musical harmony A couple of years ago I worked on a modelling musical harmony as a GADT in Haskell (together with W. Bas de Haas). From this model came: A pretty-printer that shows the tree structure of a sequence of chords (ICFP 2011); José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

4 An extremely useful model of musical harmony A couple of years ago I worked on a modelling musical harmony as a GADT in Haskell (together with W. Bas de Haas). From this model came: A pretty-printer that shows the tree structure of a sequence of chords (ICFP 2011); A system for finding cover songs based on harmonic similarity (ISMIR 2011); José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

5 An extremely useful model of musical harmony A couple of years ago I worked on a modelling musical harmony as a GADT in Haskell (together with W. Bas de Haas). From this model came: A pretty-printer that shows the tree structure of a sequence of chords (ICFP 2011); A system for finding cover songs based on harmonic similarity (ISMIR 2011); An improved chord recogniser from audio that is aware of the rules of musical harmony (ISMIR 2012); José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

6 An extremely useful model of musical harmony A couple of years ago I worked on a modelling musical harmony as a GADT in Haskell (together with W. Bas de Haas). From this model came: A pretty-printer that shows the tree structure of a sequence of chords (ICFP 2011); A system for finding cover songs based on harmonic similarity (ISMIR 2011); An improved chord recogniser from audio that is aware of the rules of musical harmony (ISMIR 2012); A company ( José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

7 An extremely useful model of musical harmony A couple of years ago I worked on a modelling musical harmony as a GADT in Haskell (together with W. Bas de Haas). From this model came: A pretty-printer that shows the tree structure of a sequence of chords (ICFP 2011); A system for finding cover songs based on harmonic similarity (ISMIR 2011); An improved chord recogniser from audio that is aware of the rules of musical harmony (ISMIR 2012); A company ( A system for automatic melody harmonisation (FARM 2013); José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

8 An extremely useful model of musical harmony A couple of years ago I worked on a modelling musical harmony as a GADT in Haskell (together with W. Bas de Haas). From this model came: A pretty-printer that shows the tree structure of a sequence of chords (ICFP 2011); A system for finding cover songs based on harmonic similarity (ISMIR 2011); An improved chord recogniser from audio that is aware of the rules of musical harmony (ISMIR 2012); A company ( A system for automatic melody harmonisation (FARM 2013); FComp: a system for automatic generation of harmony and accompanying melody (this talk). José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

9 An example: visualising harmonic structure Piece Phrase Ton Dom Ton I : Maj Sub Dom I : Maj C: Maj III : Min IV : Maj II : Dom 7 V : Dom 7 C: Maj E: Min F : Maj D: Dom 7 G: Dom 7 You can see this tree as having been produced by taking the chords in green as input... José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

10 An example: generating harmonic structure Piece Phrase Ton Dom Ton I : Maj Sub Dom I : Maj C: Maj III : Min IV : Maj II : Dom 7 V : Dom 7 C: Maj E: Min F : Maj D: Dom 7 G: Dom 7 You can see this tree as having been produced by taking the chords in green as input... or the chords might have been dictated by the structure! José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

11 System structure Define harmony model Take all possible melody notes per chord Set rule weight Pick one focal melody note per chord Filter melody notes Embellish melody José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

12 A functional model of harmony Piece M [Phrase M ] (M {Maj, Min}) José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

13 A functional model of harmony Piece M [Phrase M ] (M {Maj, Min}) Phrase M Ton M Dom M Ton M Dom M Ton M José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

14 A functional model of harmony Piece M [Phrase M ] (M {Maj, Min}) Phrase M Ton M Dom M Ton M Dom M Ton M Ton Maj I Maj Ton Min I m Min José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

15 A functional model of harmony Piece M [Phrase M ] (M {Maj, Min}) Phrase M Ton M Dom M Ton M Dom M Ton M Ton Maj I Maj Ton Min I m Min Dom M V 7 M V M VII 0 M Sub M Dom M II 7 M V 7 M José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

16 A functional model of harmony Piece M [Phrase M ] (M {Maj, Min}) Phrase M Ton M Dom M Ton M Dom M Ton M Ton Maj I Maj Ton Min I m Min Sub Maj II m Maj IV Maj III m Maj IV Maj Sub Min IV m Min Dom M V 7 M V M VII 0 M Sub M Dom M II 7 M V 7 M José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

17 A functional model of harmony Piece M [Phrase M ] (M {Maj, Min}) Phrase M Ton M Dom M Ton M Dom M Ton M Ton Maj I Maj Ton Min I m Min Dom M V 7 M V M VII 0 M Sub M Dom M II 7 M V 7 M Sub Maj II m Maj IV Maj III m Maj IV Maj Sub Min IV m Min I Maj IMin m VM 7 C: Maj C: Min G: Dom7 VII 0 M B: Dim José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

18 A functional model of harmony Piece M [Phrase M ] (M {Maj, Min}) Phrase M Ton M Dom M Ton M Dom M Ton M Ton Maj I Maj Ton Min I m Min Dom M V 7 M V M VII 0 M Sub M Dom M II 7 M V 7 M Sub Maj II m Maj IV Maj III m Maj IV Maj Sub Min IV m Min I Maj IMin m VM 7 C: Maj C: Min G: Dom7 VII 0 M B: Dim Simple, but enough for now, and easy to extend. José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

19 Now in Haskell I A GADT encoding musical harmony: data Mode = Maj Mode Min Mode data Piece = µ :: Mode.Piece [ Phrase µ ] José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

20 Now in Haskell I A GADT encoding musical harmony: data Mode = Maj Mode Min Mode data Piece = µ :: Mode.Piece [ Phrase µ ] data Phrase (µ :: Mode) where Phrase IVI :: Ton µ Dom µ Ton µ Phrase µ Phrase VI :: Dom µ Ton µ Phrase µ José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

21 Now in Haskell I A GADT encoding musical harmony: data Mode = Maj Mode Min Mode data Piece = µ :: Mode.Piece [ Phrase µ ] data Phrase (µ :: Mode) where Phrase IVI :: Ton µ Dom µ Ton µ Phrase µ Phrase VI :: Dom µ Ton µ Phrase µ data Ton (µ :: Mode) where Ton Maj :: SD I Maj Ton Maj Mode Ton Min :: SD I Min Ton Min Mode José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

22 Now in Haskell I A GADT encoding musical harmony: data Mode = Maj Mode Min Mode data Piece = µ :: Mode.Piece [ Phrase µ ] data Phrase (µ :: Mode) where Phrase IVI :: Ton µ Dom µ Ton µ Phrase µ Phrase VI :: Dom µ Ton µ Phrase µ data Ton (µ :: Mode) where Ton Maj :: SD I Maj Ton Maj Mode Ton Min :: SD I Min Ton Min Mode data Dom (µ :: Mode) where Dom 1 :: SD V Dom 7 Dom µ Dom 2 :: SD V Maj Dom µ Dom 3 :: SD VII Dim Dom µ Dom 4 :: SDom µ Dom µ Dom µ Dom 5 :: SD II Dom 7 SD V Dom 7 Dom µ José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

23 Now in Haskell II Scale degrees are the leaves of our hierarchical structure: data DiatonicDegree = I II III IV V VI VII data Quality = Maj Min Dom 7 Dim data SD (δ :: DiatonicDegree) (γ :: Quality) where SurfaceChord :: ChordDegree SD δ γ José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

24 Generating harmony Now that we have a datatype representing harmony sequences, how do we generate a sequence of chords? José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

25 Generating harmony Now that we have a datatype representing harmony sequences, how do we generate a sequence of chords? QuickCheck! We give Arbitrary instances for each of the datatypes in our model. José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

26 Generating harmony Now that we have a datatype representing harmony sequences, how do we generate a sequence of chords? QuickCheck! We give Arbitrary instances for each of the datatypes in our model.... but we don t want to do this by hand, for every datatype, and to have to adapt the instances every time we change the model... so we use generic programming: José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

27 Generating harmony Now that we have a datatype representing harmony sequences, how do we generate a sequence of chords? QuickCheck! We give Arbitrary instances for each of the datatypes in our model.... but we don t want to do this by hand, for every datatype, and to have to adapt the instances every time we change the model... so we use generic programming: gen :: (Representable α, Generate (Rep α)) Gen α José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

28 Generating harmony Now that we have a datatype representing harmony sequences, how do we generate a sequence of chords? QuickCheck! We give Arbitrary instances for each of the datatypes in our model.... but we don t want to do this by hand, for every datatype, and to have to adapt the instances every time we change the model... so we use generic programming: gen :: (Representable α, Generate (Rep α)) [ (String,Int) ] Gen α José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

29 Examples of harmony generation I testgen :: Gen (Phrase Maj Mode ) testgen = gen [("Dom4", 3), ("Dom5", 4)] example :: IO () example = let k = Key (Note C) Maj Mode in sample testgen >= mapm (printonkey k) printonkey :: Key Phrase Maj Mode IO String José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

30 Examples of harmony generation I testgen :: Gen (Phrase Maj Mode ) testgen = gen [("Dom4", 3), ("Dom5", 4)] example :: IO () example = let k = Key (Note C) Maj Mode in sample testgen >= mapm (printonkey k) printonkey :: Key Phrase Maj Mode IO String > example [C: Maj, D: Dom 7, G: Dom 7, C: Maj ] [C: Maj, G: Dom 7, C: Maj ] [C: Maj, E: Min, F : Maj, G: Maj, C: Maj ] [C: Maj, E: Min, F : Maj, D: Dom 7, G: Dom 7, C: Maj ] [C: Maj, D: Min, E: Min, F : Maj, D: Dom 7, G: Dom 7, C: Maj ] José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

31 Examples of harmony generation II????? # n # n n # n José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

32 Generating a melody for a given harmony We then generate a melody in 4 steps: José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

33 Generating a melody for a given harmony We then generate a melody in 4 steps: 1. Generate a list of candidate melody notes per chord; José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

34 Generating a melody for a given harmony We then generate a melody in 4 steps: 1. Generate a list of candidate melody notes per chord; 2. Refine the candidates by filtering out obviously bad candidates; José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

35 Generating a melody for a given harmony We then generate a melody in 4 steps: 1. Generate a list of candidate melody notes per chord; 2. Refine the candidates by filtering out obviously bad candidates; 3. Pick one focal candidate melody note per chord; José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

36 Generating a melody for a given harmony We then generate a melody in 4 steps: 1. Generate a list of candidate melody notes per chord; 2. Refine the candidates by filtering out obviously bad candidates; 3. Pick one focal candidate melody note per chord; 4. Embellish the candidate notes to produce a final melody. José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

37 Generating a melody for a given harmony We then generate a melody in 4 steps: 1. Generate a list of candidate melody notes per chord; 2. Refine the candidates by filtering out obviously bad candidates; 3. Pick one focal candidate melody note per chord; 4. Embellish the candidate notes to produce a final melody. These four steps combine naturally using plain monadic bind: melody :: Key State MyState Song melody k = gencandidates >= refine >= pickone >= embellish >= return Song k More details in the paper! José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

38 Example I &? n # Ton Phrase Dom Ton I : Maj Sub Dom I : Maj C: Maj III : Min IV : Maj II : Dom 7 V : Dom 7 C: Maj E: Min F : Maj D: Dom 7 G: Dom 7 José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

39 Example II & #?# Phrase Ton Dom Ton I : Maj Sub Dom I : Maj G: Maj III : Min IV : Maj Sub Dom G: Maj B: Min C: Maj III : Min IV : Maj V : Maj B: Min C: Maj D: Dom 7 José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

40 Example III & #?# ## # n Phrase Ton Dom Ton I : Min Sub Dom I : Min E: Min IV : Min Sub Dom E: Min A: Min IV : Min II : Dom 7 V : Dom 7 A: Min F : Dom 7 B: Dom 7 José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

41 Conclusion FComp: a simple and easy to understand and improve functional system for automatic generation of harmony and accompanying melody. José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

42 Conclusion FComp: a simple and easy to understand and improve functional system for automatic generation of harmony and accompanying melody. Lots of room for improvement: Voice leading and counterpoint Handle repetition Improve embellishment Rhythm, form, instrumentation, dynamics José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

43 Conclusion FComp: a simple and easy to understand and improve functional system for automatic generation of harmony and accompanying melody. Lots of room for improvement: Voice leading and counterpoint Handle repetition Improve embellishment Rhythm, form, instrumentation, dynamics Thank you for your attention! José Pedro Magalhães Functional Generation of Harmony and Melody, FARM / 15

WPA REGIONAL CONGRESS OSAKA Japan 2015

!!!!!!!!! -1- WPA REGIONAL CONGRESS OSAKA Japan 2015 "! " -2- !!!! -3- " " "!! " " -4- !!!!!!!!!!!!!! -5- !!!!!!!!!!!!!!! -6- WPA REGIONAL CONGRESS OSAKA Japan 2015! -7- ! "!! -8- -9- WPA REGIONAL CONGRESS