MorpheuS: constraining structure in automatic music generation

MorpheuS: constraining structure in automatic music generation Dorien Herremans & Elaine Chew Center for Digital Music (C4DM) Queen Mary University, London Dagstuhl Seminar, Stimulus talk, 29 February 4 March 2016 D. Herremans (C4DM, QMUL, London) Structure & generation 1 / 24

Automatically generating music? Why don t we listen to automatically generated music? D. Herremans (C4DM, QMUL, London) Structure & generation 2 / 24

Automatically generating music? Why don t we listen to automatically generated music? Long-term structure D. Herremans (C4DM, QMUL, London) Structure & generation 3 / 24

Automatically generating music? Composing music = combinatorial optimization problem Decide on which notes Objective: fit a style/structure Solved by metaheuristic such as variable neighbourhood search D. Herremans (C4DM, QMUL, London) Structure & generation 4 / 24

How do we evaluate music? Human evaluation Music theory Machine learning objective function D. Herremans (C4DM, QMUL, London) Structure & generation 5 / 24

Outline 1 Structural constraints Global structure Tension profile Pattern detection D. Herremans (C4DM, QMUL, London) Structure & generation 6 / 24

Outline 1 Structural constraints Global structure Tension profile Pattern detection D. Herremans (C4DM, QMUL, London) Structure & generation 7 / 24

Global structure Bagana: Ethiopian Lyre Match an expectancy profile (Markov model) Fixed structure (A 1 A 2 A 1 A 2 A 3 A 1 A 3 A 1 ) Hard constraint during optimization 8 A1 4 3 2 2 2 4 A2 5 4 2 2 2 2 A3 4 2 3 3 1 1 5 1 1 3 2 2 4 A2 5 4 2 2 2 2 D. Herremans (C4DM, QMUL, London) Structure & generation 8 / 24

Outline 1 Structural constraints Global structure Tension profile Pattern detection D. Herremans (C4DM, QMUL, London) Structure & generation 9 / 24

Tension profile Complex composite concept Tonal, melodic, harmonic, rhythmic, expressive,... Useful for complete compositions & video/game music T R Tonal tension model based on the spiral array (Chew, 2000) D. Herremans (C4DM, QMUL, London) Structure & generation 10 / 24

Spiral array, 3D mathematical model for tonality D. Herremans (C4DM, QMUL, London) Structure & generation 11 / 24

3 aspects of tonal tension Cloud diameter Cloud momentum D. Herremans (C4DM, QMUL, London) Structure & generation 12 / 24

3 aspects of tonal tension Tensile strain (distance to key) D. Herremans (C4DM, QMUL, London) Structure & generation 13 / 24

Tonal tension tristan chord Wagner s opera Tristan und Isolde Bass note and augmented 4th, 6th and 9th D. Herremans (C4DM, QMUL, London) Structure & generation 14 / 24

Tonal tension tristan chord D. Herremans (C4DM, QMUL, London) Structure & generation 15 / 24

Tension profiles soft constraints D. Herremans (C4DM, QMUL, London) Structure & generation 16 / 24

Outline 1 Structural constraints Global structure Tension profile Pattern detection D. Herremans (C4DM, QMUL, London) Structure & generation 17 / 24

Pattern detection Compression algorithm: COSIATEC (Meredith, 2013) Point set representation of a piece Computes a compressed encoding of the piece maximal translatable patterns D. Herremans (C4DM, QMUL, London) Structure & generation 18 / 24

Polyphonic example Bach prelude 20 (book 2) D. Herremans (C4DM, QMUL, London) Structure & generation 19 / 24

Putting it all together: MorpheuS Problem: find pitches Objective: match tension profile to template Hard constraint: detected patterns Test: Bach 1st Prelude: D. Herremans (C4DM, QMUL, London) Structure & generation 20 / 24

Preliminary results: pattern detection D. Herremans (C4DM, QMUL, London) Structure & generation 21 / 24

Preliminary results: random starting piece D. Herremans (C4DM, QMUL, London) Structure & generation 22 / 24

Preliminary results: fit in tension profile & patterns D. Herremans (C4DM, QMUL, London) Structure & generation 23 / 24