University of Dublin TRINITY COLLEGE. Algorithmic Music Composition Using Genetic Algorithms and Machine Learning

Transcription

1 ! University of Dublin TRINITY COLLEGE Algorithmic Music Composition Using Genetic Algorithms and Machine Learning Eoghan Hynes B.A.(Mod.) Computer Science Final Year Project April 2015 Supervisor: Dr. Mike Brady School of Computer Science and Statistics O Reilly Institute, Trinity College, Dublin 2, Ireland

2 Declaration I hereby declare that this thesis is entirely my own work and that it has not been submitted as an exercise for a degree at any other university. Eoghan Hynes April 2015

3 Acknowledgements I would like to thank my supervisor Dr. Brady for his enthusiasm and wisdom throughout the course of this project. My parents for their support and patience even when I might be difficult to get a hold of. My siblings for all their advice through my four years of college and beyond. To Theo for encouraging my love of music throughout my life. And finally my classmates for always striving to be their best and motivating me to do the same.

4 Abstract Algorithmic Composition is a concept wherein computers are the main composers of music, as opposed to humans. In this report we explore the applicability of genetic algorithms and other machine learning techniques in the generation of style-imitative musical compositions, based on a corpus of pre-existing scores.

5 Table of contents List of figures vi 1 Introduction 1 2 Background Related Works Early Research Pre-Computer Early Computational Compositions Methods of Algorithmic Composition Grammars Knowledge-Based Systems Markov Chains Genetic Algorithms Related Work EMI ACSSM [1] ACSSM2 [2] Design Implementation System Overview MusicXML Clarification Score Analyser Score Deconstructor Recombination/ Composition Genetic Algorithm Design Fitness Function Crossover Operator Mutation Operator

6 Table of contents v Considerations Conclusions Results Markov-Based Recombination Genetic Algorithm-Based Recombination Limitations Music Theory MusicXML Concluding Remarks Future Work References 21 Appendix A Explanation of Terms 24 A.1 Genetic Algorithms Overview A.2 Glossary Appendix B Composition Examples 27 B.1 Corpus used B.2 Markov-Based Recombination B.2.1 Sample B.2.2 Sample B.2.3 Sample B.3 Genetic Algorithm-Based Recombination B.3.1 Sample B.3.2 Sample B.3.3 Sample

7 List of figures 2.1 Block diagram of EMI Block diagram of ACSSM Block diagram of ACSSM Block diagram of the system

8 Chapter 1 Introduction Computational creativity is a field of research which is the cross-section of art and science. The difficulty surrounding this area is that individuals of an artistic disposition rarely find themselves in the realm of scientific research, and vice versa. In fact researchers who do attempt to create systems which mimic human creativity often discover themselves at the centre of controversy over what many consider to be a uniquely human characteristic. Researchers have created systems which attempt to mimic almost every form of human creativity, from poetry and literature, to visual and musical works and although each are interesting in their own right, this report focuses solely on musical compositions. Different forms of computational music creativity exist with unique goals and varying levels of complexity and human interaction. Three such areas include Computer-Aided Algorithmic Composition (CAAC), Generative Music and Algorithmic Composition. CAAC is a very broad and vague term used to describe tools which are generally included in music composition software for composers to aid in their compositions or to provide inspiration to them. Their main aim is to assist human composers rather than compose music themselves, and they can range in sophistication from a simple arpeggiator to more complicated systems using grammars and Petri Nets. [3] Generative Music is a style of music made popular by Brian Eno and is similar to algorithmic composition, though rather than trying to create a self-contained musical piece it instead composes a non-deterministic, ever-changing piece of work. Generally it works by defining the initial parameters and then allowing the system to improvise as it pleases from then on, the idea being that no two performances would be exactly the same. [4] Algorithmic composition is a subsection of computational music creativity which concerns itself with creating musical works with minimum or no human interaction, either through building on a pre-existing musical corpus or by generating entirely new works independently. Algorithmic composition of this sort is one of the most active research areas

9 2 in computational music creativity and the main focus of this report and specifically style sensitive algorithmic composition, which will be explained later. In this report we will describe a system which can read in a corpus of MusicXML sheet music in a specific style of music, and produce new music in a similar style. In 2.1 we go through a brief history of algorithmic composition, 2.2 will describe some of the techniques which have been used by others, 2.3 goes through some of the previous research that has been done which is closely related to this project. Chapter 3 will go through an overview of the design of my project, with 3.2 delving deeper into the Genetic Algorithm used. Chapter 4 will give an overall evaluation of this project as well as a description of the further work which could be done. Appendix A also includes a glossary of terms, as well as a quick tutorial on Genetic Algorithms, and finally Appendix B has examples of the sheet music produced by our system.

10 Chapter 2 Background Related Works 2.1 Early Research Algorithmic composition has a history which intertwines with the long history of music as well as the beginnings of the field of Artificial Intelligence itself Pre-Computer Non-creative musical composition (that is, music composed through pre-defined structured systems or rules rather than purely human creativity), and "algorithms" for music composition, have been around since the 18th century. One of the first examples of this is Musikalisches Würfelspiel or "Musical Dice Game", a popular game throughout Western Europe at the time which is often accredited to Mozart. In Musikalisches Würfelspiel the player is given a sheet which includes a table of numbered measures of music. The player then rolls a dice and compiles the measures based on the numbers which are thrown, thus creating an entirely new composition using the pre-defined measure table.[5] This method, and variations of it, are the basis for many algorithmic composition techniques, including my own. Fast-forwarding to the 20th Century, Arnold Schoenberg developed a compositional system called the twelve-tone technique whose aim was to utilise all twelve tones of the chromatic scale throughout the piece, and to ensure that each note was sounded equally so that no dominant key could be distinguished in the piece.[6] John Cage, a student of Schoenberg, then went on to compose one of the first indeterminate pieces "Music of Changes" using the classic Chinese text "I Ching" as his source of indeterminacy. Cage would "ask" the book different questions and would add to his composition based on the answers provided. [7]

11 2.2 Methods of Algorithmic Composition Early Computational Compositions Around the time that Cage was experimenting with indeterminism through literary means, two chemists Lejaren Hiller and Leonard Issacson were experimenting with indeterminism through mathematical and computational means. Hiller and Issacson are credited with one of the first musical pieces to be composed by a computer. Composed in 1957, "Illiac Suite" (so called as it was an ILLIAC I computer which was used in the composition), a four-part piece for string quartet, uses the Monte Carlo algorithm for data generation and Information Theory for selection and runs in three stages: Initialisation, in which music theory rules and the personal rules of the human "composers" are defined; Generation, in which the computer generates data, at a rate of a thousand per second; Verification, in which the generated data is compared to the parameters defined in the Initialisation stage, and if not suitable, the Generation stage is repeated. Finally the completed work is decoded from its initial alphanumeric representation to traditional sheet music score. [8] An article written by Hiller for Scientific American on Illiac Suite was picked up by press and caused a huge deal of controversy, at a time when AI and future speculation was much discussed. [9] Around that same time Burrough s, an early computer company, had developed a system for composing "Tin Pan Alley" style compositions on their DATATRON computer, creating a melody called "Push Button Bertha". This system was one of the first style-imitative to be developed though it required the manual analysis of previous compositions by a human operator, rather than automatic analysis by the system as in the systems described later. The system worked by "inspiring" the DATATRON computer by inputting a 10-digit random number which it then used to generate 1000 single digits representing notes, sharps and flats. The system then picked notes at random, testing them against the rules defined by the previous musical analysis for their suitability. [10][11] Another example of style-imitative composition was done by a 16 year-old Raymond Kurzweil who appeared on the 1965 TV show "I ve Got A Secret" demonstrating a piano composition which his system generated.[12] Unfortunately due to his young age, details of the system aren t available, though sound snippets from his appearance on "I ve Got A Secret" available online gives positive results. [13] 2.2 Methods of Algorithmic Composition Many methods exist for algorithmic composition each with their own specific goals and implementations. Methods exist for generating everything from real-time jazz improvisations to creating beats based off binary subdivision.[14] The system described in this report

12 2.2 Methods of Algorithmic Composition 5 attempts to create western piano music and so we will only discuss the prevalent techniques which are used to create music in this style Grammars Formal Grammars were originally a linguistic concept, developed by linguist Noam Chomsky, but they were found to be extremely useful by computer scientists for their recursive properties, and particularly useful in the designing of programming languages. Broadly speaking, formal grammars describe a set of symbols and acceptable transitions from those symbols to other symbols, which can be traversed to create potentially infinitely large structures (depending on the grammar). One of the difficulties surrounding grammars in music is that, while Chomsky had developed a grammar for natural languages, no such grammar had been developed to represent musical structure.[15] Early researchers attempted to develop grammars by hand using known music theory principles but one of the first major attempts at musical grammar generation, inspired by lectures given by Leonard Bernstein, was done by music theorist Fred Lerdahl and linguist Ray Jackendoff in "A Generative Theory of Tonal Music" (GTTM). In GTTM Lerdahl and Jackendoff generated a hierarchical grammar for Western music which attempt to mimic what Chomsky had developed for natural language, with four structures describing different characteristics of music.[16] Although their work was not concerned with algorithmic computation directly, it was very influential in grammar-based algorithmic composition from then on, inspiring works such as Pope (1991)[17], Leach and Fitch (1995)[18], Hamanaka et al (2008)[19], and Chan (2004) which we discuss in In 1998 Cruz-Alcázar and Vidal-Ruiz implemented a system which used several methods of grammatical induction (namely k-tsi, ALERGIA, and ECGI algorithms) on small fragments of pre-existing corpus of musical works. Experimenting with Gregorian, J.S. Bach and Scott Joplin works they used the grammars generated by these algorithms to generate short style-imitative melodies to the composer from which the grammar was generated.[20] Other interesting experiments with grammars include the use of graphical L-Systems, converting the images generated by the L-system into music. One example of this is by Prusinkiewicz (1986) who used L-systems to algorithmically render a "turtle graphic" (a type of computer-generated vector graphic) which he then interpreted into a musical score using horizontal line segments as notes, the y-coordinate of the segment as its pitch, and note duration as its length.[21]

13 2.2 Methods of Algorithmic Composition Knowledge-Based Systems Knowledge-based systems are probably the most intuitive way of designing algorithmic composition systems. The term broadly describes any system which uses pre-defined known music theory rules to generate its music. One such example is the system used to compose the Illiac suite described in 2.1.2, but it is very common for algorithmic composition systems to include composition rules at some point. Many systems have been built to implement pre-existing music theory rules such as Schoenberg s twelve-tone technique (Gill 1963)[22], improvisation of eighteenth century unfigured bass (Rothgeb 1968)[23], and composition of early-classical piano minuets (Löthe 1999)[24] Markov Chains Markov chains are a relatively simple idea which can be extremely useful in machine learning. Their basic operation is a of a memoryless, stochastic, transition network which has a finite set of possible states and whose next state depends only on the current state. Many uses of Markov chains can be generalised as an automatic Musikalisches Würfelspiel where the states are represented as pre-defined excerpts of music, manually or computationally defined and transitions between these states are governed by acceptable musical transitions defined by the programmer, such as in Jones (1981)[25] or Langston (1989)[14]. Verbeurgt et al. (2004) used Markov chains by first doing pattern matching across a piece, using the patterns identified as states in the Markov chain, and refining it through the use of an artificial neural network.[26] Wooler and Brown (2005) also used "Markov Morphing" to generate transitions between two distinct musical excerpts by using markov chains learned through the analyses of the provided excerpts and interpolating between them. [27] Genetic Algorithms Genetic algorithms (GA) are a subset of AI that attempt to model the biological evolutionary process and are generally used in optimisation problems. They represent a unspecific framework for various problems through which specific functions which simulate reproduction, mutation, and selection in a population of data must be designed on a problem-by-problem basis (A more in-depth explanation of GA is given in Appendix A for those unfamiliar with the concept). Genetic algorithms have been used in algorithmic composition since the early 90s and most of the more interesting parts of these systems lie in the different methods used to

14 2.3 Related Work 7 represent music within the GA framework and also the different fitness functions used. Many systems use a weighted some of certain feature vectors (such as those described by Towsey et al. in 2001) as their fitness function. In Johnson et. al (2004) individuals in the population are initialised as randomly generated melodies, and a fitness score is assigned to it based on a note in the melody s relationship with its following note, using a pre-determined weighted criteria.[28] Gartland-Jones (2002) used an alternative approach, where population is created from a 2 bar phrase provided to the system, and a fitness score is given based on its similarity to a target 2 bar phrase (which is also provided), the aim being is to create a new phrase which is a hybrid of both phrases.[29] One of the most successful approached to GA-based composition comes from the Iamus computer cluster which uses many different biology-inspired techniques to compose contemporary classical pieces. Iamus released its first album "Hello World" in 2011, and now sells compositions created by the cluster, as well as the full rights to the piece to buyers.[30] 2.3 Related Work EMI Experiments in Musical Intelligence is a software program which has been developed by former University of California professor of music David Cope since 1981, affectionally referred to as "Emmy".[31] EMI was developed by Cope as a result of a composers block he was experiencing at the time. My initial idea involved creating a computer program which would have a sense of my overall musical style and the ability to track the ideas of a current work such that at any given point I could request a next note, next measure, next ten measures, and so on. My hope was that this new music would not just be interesting but relevant to my style and to my current work. [32] EMI is a multi-stage system which generates entirely new compositions in the style of a corpus of music which is fed to it. Cope s design is inspired by the Musikalisches Würfelspiel and, like some of the other examples described in 2.2.3, could be described as a sophisticated automatic implementation of it. The analysis stage of EMI goes through each piece in the corpus of music (represented in the MIDI format in his case) and analyses the harmonic characteristics of the music by using SPEAC identifiers.

15 2.3 Related Work 8 SPEAC Signature dictionary Lexicon Corpus Analysis Pattern Matching Deconstruction Reconstruction Output Fig. 2.1 Block diagram of EMI SPEAC identifiers are a system developed by cope to give more context to harmony than traditional chord structures could provide. SPEAC stands for statement,preparation, extension, antecedent, and consequent. A statement can occur anywhere in a piece, a preparation is followed by a statement, an extension extend other identifiers, and antecedent is followed by a consequent, which itself is similar to a statement but always follows an antecedent that it resolves.[33][34][1] SPEAC ID s are used by both my system and those used by ACSSM described in If you were to imagine a sample of sheet music in front of you SPEAC ID s describe the vertical elements of the music (the chords or harmony) where as the pattern matching attempts to describe the horizontal elements (melody). The pattern matching stage of EMI attempts to analyse music for recurring melodies or "signatures" of a composer, adding to a separate signature database to preserve the structure for the recombination stage. The deconstruction stage breaks the original music into small segments and is stored as a lexicon, which is then used to reconstruct a new piece using augmented transition networks, along with the signatures identified by the pattern matching stage. EMI has led to some very positive results and was used by Cope right up to 2004, retaining 11,000 of its composed pieces. He used EMI as the basis for his next project Emily Howell (Emmy s "daughter") which he is currently still working on. Unfortunately from a research standpoint, Emily Howell is far more interactive than EMI and cannot independently compose without regular input from a user. [31] ACSSM [1] "Automated Composer of Style Sensitive Music" is a system built by Michael Chan, a post-graduate student in the University of New South Wales and is based on EMI. ACSSM deviates from EMI in that it removes the pattern matching stage, due to what it cites to be a lack of suitable scores for this stage to be effective, uses MusicXML rather than MIDI, and adds extra steps in the Analysis stage. ACSSM uses GTTM (described in 2.2.1) in tandem

16 2.3 Related Work 9 SPEAC Lexicon Harmonic Analyzer Corpus MusicXML Parser Deconstructor Reconstructor Output GTTM Engine Fig. 2.2 Block diagram of ACSSM Segments Corpus MusicXML Parser CBMS generator Deconstructor Features Generator Reconstructor Output Fig. 2.3 Block diagram of ACSSM2 with SPEAC ID s to glean more information from the MusicXML corpus and aid in more intelligent deconstruction by the system. ACSSM was successful in creating quite pleasant, style-imitative music despite it s small corpus, but it acknowledges that it is still quite a primitive system and that a lot of further work could be done to improve it ACSSM2 [2] Chan developed a second version of ACSSM in conjunction with John Potter and Emery Schubert for the 9th International Conference of Music Perception and Cognition. ACSSM2 added many interesting improvements on the original system including using The Cognition of Basic Musical Structures (CBMS) over GTTM, the use of feature vectors, and the use of GA. CBMS is a book by David Temperley and is based on GTTM but focuses more on the aural perception of music rather than the analysis of music structure.[35] The feature vectors were based on previous work by Chan and Potter in Recognition of Musically Similar Polyphonic Music[36] which in turn was based on 160 features identified

17 2.3 Related Work 10 by Cory McKay in his thesis [37]. In addition to a small selection of pitch and rhythm features used from McKay s work Chan and Potter added eight of their own rhythm features. Chan et al. used GA in an interesting way. They identified that recombination based on feature vectors was analogous with path searching in a map, with the musically similar segments identified representing cities, and the transition probability corresponding to the distance between each city. They then used a GA for finding the shortest path between the cities based on the work done by Ben Mowery in 2001[38], with the musical recombination being based on these paths. According to their evaluations, their modifications based on the original ACSSM resulted in an average score 1.68 higher, with ACSSM scoring 5.66 and ACSSM2 scoring 7.34 in overall musicality (in the range 0 to 10).

18 Chapter 3 Design Implementation A high level description of the flow of my system: 1. Reads in a corpus of sheet music (a directory of MusicXML files). 2. Clarifies these scores, ensuring that they are in a standard format. 3. Analyses these scores for SPEAC and traditional chord identifiers. 4. Deconstructs the scores based on the previous analyses. 5. Composes an entirely new piece by reconstructing the piece. 3.1 System Overview MusicXML Clarification Music comes in a wide range of key and time signatures, but before we can use a piece of music for composition we must ensure that all pieces are all standardised. To do this each SPEAC Traditional Chord No. Beats Markov-Based Reconstruction Output Corpus MusicXML Clarifier Analysis Deconstruction Genetic Algorithm Reconstruction Output Fig. 3.1 Block diagram of the system

19 3.1 System Overview 12 piece must be transposed to the correct key, in our case the key of C major(cmj) or its relative minor A (Am). No attempt was made to transpose minor keys to major keys or vice versa as it can be sometimes musically desirable for pieces to transition between their relative major and minor keys. "Notations" tags, which are used to notate dynamics, articulations, slurs etc. were also removed from the MusicXML scores as they were found to cause issues with the rendering of the MusicXML in the recombination stage. A high level description of the operation of the clarification program: 1. Program iterates through piece measure by measure 2. Checks the current key of the measure, and if already in Cmj/Am continues until a key change is found, finishing if a key change is not found (i.e. the piece is already in the key of Cmj/Am). 3.. Iterates through each note in the measure. 4. For every note it transposes the note up or down depending on its distance from Cmj/Am using the circle of fifths. [39] 5. Once every note is transposed the program then removes any "notations" if they exist in the measure. 6. Repeats until each measure has been processed. 7. Stores the clarified score in $PROJECTROOT/data/clarified-scores. An attempt was made to test the correct transposition of this program by making use of the Krumhansl-Schmuckler key-finding algorithm, but the implementation was found to be generally unreliable.[40] Score Analyser The score analyser part of this system is responsible for iterating through each of the clarified scores processed by the clarification stage before it, and assigning meaning to the music within. The score analyser assigns both SPEAC identifiers and traditional chord numbers to each beat in the piece, and modifies the "divisions" and "duration" 1 of each note to ensure each note in every piece is using the same divisor for its note duration, to aid in recombination. A high level description of the operation of the analyses program: 1 refer to MusicXML DTD for further details

20 3.1 System Overview Program checks to ensure that the piece has two staves (non-keyboard music is not supported) exiting if not. 2. Program then iterates through piece measure by measure. 3. For each measure, it does further splitting by beat. 4. For each beat it analyses the notes in the beat, adds an appropriate SPEAC ID element, traditional chord ID element as well a "beatnumber" attribute to aid in deconstruction. 5. Once each measure has been analysed the program then does a second sweep of the measures to ensure that each note duration is using our standard divisor, 48. The analyses stage is one of the most important parts of this method of algorithmic composition and can have a huge influence over the musicality of the final composition. A huge amount of music theory can be added to this stage (such as the use of GTTM and CBMS in ACSSM and ACSSM2 for instance) which can ease with a more graceful deconstruction in the next stage, but these more advanced music theory techniques are outside the scope of this project Score Deconstructor The deconstruction stage of this system is quite straightforward, it takes the analyses done in the previous stage and uses it to categorise the beats according to their corresponding SPEAC IDs. A more in-depth description of the program is: 1. Program goes through piece measure by measure 2. For every measure it groups the beats in the measure by the "beatnumber" attribute added in the previous stage. 3. For each beat it gets the SPEAC ID element described in the previous step and adds the beat to a dictionary with the corresponding SPEAC ID as the key. 4. The program then writes each value in the SPEAC ID dictionary to file grouping them by the SPEAC ID key Recombination/ Composition Two different approaches were attempted in recombining the beats into a new composition, a markov-chain based approach, and a genetic algorithm approach. Two approaches were

21 3.1 System Overview 14 implemented for comparison and to allow for consideration as to which approach, if any, generates the most musical compositions. (a) Markov Chain Method Our Markov-chain based approach works by going through the corpus of composition, analysing the SPEAC and traditional chord structure of each piece and learning a Markovchain from these structures. The system then generates a new SPEAC/chord structure using the learnt Markov-chain, and uses this new structure to slot in corresponding SPEAC beats, and generate a new composition. A more in-depth description of the program is: 1. Program goes through every piece in the clarified score directory and learns the SPEAC and chord structure of each in a list of tuples of the form (SPEAC_ID, Chord_ID). 2. It then goes through each of the structures and generates triples in the form of (IDs_1,IDs_2):[List of possible transitions], (IDs_2,IDs_3):[List of possible transitions] and so on. (For example ((P1,I), (S2,V)):[(E2,VII), (S3,VI), (A1,I), (A2,V)]). 3. System then uses this dictionary to create an entirely new SPEAC structure of a certain length (in our case 72 beats). 4. Once the specified length has been reached it then attempts to find a final cadence for the piece, giving up after a certain amount of attempts if there are no suitable cadences which can be chosen. 5. Program then uses the generated structure to choose beats from the SPEAC categories and slots these beats into each of their beat descriptors in the structure (like a jigsaw of sorts). (b) Genetic Algorithm Method The GA approach to the reconstruction and composition of new pieces was used to select beats for recombination based on their similarity to previous beats. At its worst the GA simply searches for beats of an acceptable similarity to the "fitness-beat", at its best the GA modifies the beats, creating entirely new variations of the original beats. This GA-based program is one of the more complicated sections of the system so for the sake of clarity, details of the GA are explained in 3.1. The basic flow of the program is as follows:

22 3.2 Genetic Algorithm Design Program selects a seed beat randomly from the SPEAC-categorised beats. 2. It then feeds this beat into the GA (described below). 3. A new beat is received from the genetic algorithm. 4. This new beat is added to the piece, and fed back into the GA, repeating from 2. until the required number of beats are generated, writing the results to file as a new composition. 3.2 Genetic Algorithm Design 2 Similar to actual musical composition, the design of GA requires a certain amount of creativity and experimentation. Many different approaches have been attempted for using GA in composition as described in Chapter 2, but we attempted to create our own unique design to aid in the "creativity" of the system. The population in our GA is each beat which was deconstructed in Fitness Function For each beat in the population a 3-dimensional feature vector is calculated from its contents. These features are inspired by those described by Towsey et al. in their 2001 paper on designing GA for music composition,[41] and are as follows: 1. Pitch Median. Median pitch is used over average pitch to avoid skewing by outlying notes, and it is usual for the median note to be close to the main melody of the piece. 2. Average Note Duration. This is to ensure that beats with longer notes are matched with the same. 3. Number of notes. Again this is to aid in flow so that beats with may notes (i.e. faster beats) flow into other beats with a similar amount of notes. This can sometimes lead to a gradual slowing down or speeding up which can make the composition more interesting. Although Towsey et al. described 21 features, many of these were unsuitable for our particular implementation and it was found that the three features described above were adequate for flow between each of the population beats. 2 A general, high-level explanation of the operation of Genetic Algorithms is given in Appendix A

23 3.2 Genetic Algorithm Design 16 The feature vector is calculated for the seed beat and the fitness function uses the Euclidean distance between the seed beat and each of the population beats as its fitness score Crossover Operator Our GA uses quite a simple crossover operator. Selection for crossover is Fitness proportionate [42] and crossover occurs by randomly swapping between treble clefs of both parents and bass clefs. The amount swapped is randomly chosen between an index 0 and minlen(parent1), len(parent2)} swapping at that index for treble notes, and repeating again for bass notes Mutation Operator The mutation operator is also quite simple, and operates by randomly choosing a note in the beat and increasing or decreasing by 2 notes (ignoring sharps and flats) Considerations Due its design the GA had a tendency to repeatedly choose the same small selection of beats in the population. To alleviate this a first-in-first-out fixed-size queue was implemented which beats were added to when they were chosen by the GA. When the GA considered an appropriate beat it would first check to make sure that that beat was not in the queue before choosing it for the composition. An appropriately large queue proved to be quite successful for ensuring musical-diversity in the final composition with minimal musical looping. The population is also reset to its Generation 1 properties each time a new beat is to be produced to avoid the population from becoming corrupted in cases where a lot of generations are traversed over the course of its operation.

24 Chapter 4 Conclusions 4.1 Results The final system was successful in creating style-imitative music with both the Markov-based approach and the GA-based approach with varying levels of success. A MusicXML corpus of Bach s keyboard Inventions were used for recombination. Both methods of recombination had both merits and faults which I will discuss individually. Examples of the produced sheet music are available in Appendix B Markov-Based Recombination Successful recombination using Markov Chains relies heavily on the analysis and deconstruction stage to categorise the music adequately. Unfortunately in our case deconstructing a piece beat-by-beat may not have been the best the optimal method for recombination using Markov chains. It is very rare for composers to have adequate time to develop an idea within the timeframe allowed by one beat of music, especially in the counterpointal music of our Bach corpus, and as a result we ended up with very "unfocused" music which would regularly jump between different unrelated sections of music. Tests were put in place with the recombination of the music which attempted to ensure that they chordal transitions between the states of the Markov chains were acceptable but, although the chords may have made sense musically, any melodies that resulted from this were quite incoherent.

25 4.2 Limitations Genetic Algorithm-Based Recombination The GA algorithm in general produced more regular and less-"jumpy" music but suffered from almost the opposite problem of the Markov method, tending to be quite monotonous musically, and rarely changing as the piece progressed. Depending on the seed beat which was chosen, it was possible for the GA program to create and recombine a new piece using beats present in the first generation alone. In cases like this the GA was effectively operating simply as a search algorithm rather than utilising the "creative" potential of the GA when it iterates through different generations. Cases where the genetic algorithm iterated through generations regularly produced sheet music which was more interesting but tended to break some musical rules in terms of number of beats per bar, and as a result tended to misalign the sheet music in certain sections when it was put into software which could render the MusicXML generated. As mentioned in one limitation of the design of the genetic algorithm was its tendency to converge on a small subsection of beats, looping between them until the desired composition length was reached. There is a certain amount of tradeoff between looping and diversity of the music. In music it is very common for sections of music to loop for a certain amount of bars, and can be very musical. Unfortunately allowing the GA to loop between beats causes it to do so without change so it was necessary to enforce some limitations on the algorithm to avoid this looping (i.e. the queue mentioned in 3.2.4). As for the operation and effectiveness of the GA itself, an appropriate beat was usually found within 3 generations. If the program ran beyond this the population tended to diverge from an optimal fitness score rather than converge as it should (generation limit was capped at 10 generations for this reason). Usually this would pose a problem for a GA but in our case this is not entirely "incorrect", so to speak. Diverging from the seed beat provided has the potential to produce more interesting music as beats following this divergent beat will follow the precedent set by it, and may result in a tempo change or some other musically-desirable consequence. Of course this is not always the case, but this is considered a "quirk" of the system rather than a bug per se. 4.2 Limitations Throughout the project certain bumps in the road were discovered which had to be overcome or bypassed depending on their severity. In general the problems posed could have been overcome with adequate time but unfortunately a work-around had to be adopted in most cases to avoid delaying the project as a whole.

26 4.3 Concluding Remarks Future Work Music Theory This project exists within the cross-section of music theory and computer science, and the background of many researchers of algorithmic composition either varies between the two fields or includes a member of both. Unfortunately in my case, although I did have a certain level of music theory knowledge from my own interests, a large amount of initial music theory reasearch was required to begin the project at all. Basing most of the music theory on what was deemed adequate for both EMI and ACSSM (discussed in 2.3) allowed me to spend more time on the more computer science related sections of the project, but unfortunately it did take up a lot of projecs time nonetheless. An entire final year project (or even a masters) could be spent on researching appropriate music theory techniques for algorithmic composition, but this was outside the scope of this project MusicXML The MusicXML format was chosen as the main music representation as the main aim of this project was to create sheet music for a human musician to then perform. Most musicians are not developers and aren t concerned with what format their digital sheet music is in (the most common being pdf), and so it was extremely difficult to find an adequate corpus of sheet music in MusicXML. For the music produced to be style-imitative it was necessary for the corpus to be by a single composer (or single genre at the very least). Thankfully I was able to come across a resource online which had an adequate amount of Bach sheet music in the MusicXML format, but because of this Bach is the only composer which the system was tested on. Without an adequate amount of different scores in one style the music produced by the system would recognisable as being taken from one score or another. An attempt was made to convert MIDI files to MusicXML but the resulting sheet music was of poor quality and would have required too much manual editing for it to be acceptable for the system. 4.3 Concluding Remarks Future Work Evaluation of a system such as the one described in this book is quite subjective, but as a whole the original aim of the project was achieved: to create music in a similar style to the corpus of music being analysed. The project was extremely ambitious and as a result suffered from many scope problems. To put this in perspective, 11 years passed between David Cope beginning work on EMI in 1981 and publishing details of the system in 1992, where as the time allocated for this project was just over 4 months.

27 4.3 Concluding Remarks Future Work 20 To aid in the musicality of this project a lot of further work and experimentation could be done in the analyses and deconstruction stages of my system. These sections are the parts which require the largest amount of music theory, but additions such as ACSSM s and ACSSM2 s use of GTTM and CBMS respectively, or even an EMI-esque pattern matching stage, could improve the final results greatly. A modification which could greatly improve the versatility of the project was to allow the use of MIDI input as well as (or instead of) MusicXML. Although MusicXML does allow for nice sheet music rendering, it isn t as ubiquitous as MIDI. The addition of MIDI could also aid in attempts at composition of contemporary non-classical pieces, where the use of MIDI is far more common. With the GA a simple modification (which unfortunately there wasn t enough time to implement) to aid in the diversity of the music would be, in the deconstruction stage, to store both the current beat and also its destination beat (i.e. the beat immediately after it). The GA would then use this destination beat as the comparison beat for its fitness score rather than the current beat. Along with this, further experimentation with the feature vectors used for the fitness score (and even simply different mutation and crossover rates) could aid with its musicality. The further work discussed here is by no means exhaustive and a project such as this could be experimented and worked on almost indefinitely, but what has been achieved in this project could serve as useful as a foundation for any other algorithmic composition researchers to bootstrap them straight into the more experimental phases of research such as this. Although as a whole the system may not create exactly performance-ready music, it achieves a similar goal as David Cope s original vision for EMI, to create style-imitative music, which could be used by a composer to inspire and get them through composition-blocks.

28 References [1] Michael Chan. Automated composer of style sensitive music. In Proceedings of the Second Australian Undergraduate Students Computing Conference, page 40. Citeseer, [2] Michael Chan, John Potter, and Emery Schubert. Improving algorithmic music composition with machine learning. [3] Stephen Travis Pope. Fifteen years of computer-assisted compositions. Computer Music Journal, [4] Brian Eno. Generative music. June [5] Zsofia Ruttkay. Composing mozart variations with dice. Teaching Statistics, 19(1):18 19, [6] Richard S. Hill. Schoenberg s tone-rows and the tonal system of the future. The Musical Quarterly, 22(1):14 17, January [7] John Cage. Silence, Lectures and Writings. Wesleyan University Press, [8] Lejaren Hiller and Leonard Isaacson. Experimental music; composition with an electronic computer. McGraw-Hill Book Company Inc., New York, [9] Roger T. Dean, editor. The Oxford Handbook of Computer Music. Oxford University Press, [10] Charles Ames. Automated composition in retrospect: Leonardo, pages , [11] Alex Di Nunzio. Push button bertha. push-button-bertha.php, January [12] John Rennie. Ray kurzweil s slippery futurism. IEEE Spectrum, 47(12):24 28, [13] I ve got a secret, TV game show, produced for CBS television. [14] Peter Langston. Six techniques for algorithmic music composition. In 15th International Computer Music Conference. Citeseer, [15] Noam Chomsky. Three models for the description of language. Information Theory, IRE Transactions on, 2(3): , 1956.

29 References 22 [16] Ray Lerdahl Fred Jackendoff. A generative theory of tonal music. Cambridge (MA), MIT Press, [17] Stephen Travis Pope. A tool for manipulating expressive and structural hierarchies in music (or, tr-trees in the mode: A tree editor based loosely on fred s theory ). In Proceedings of the International Computer Music Conference, pages , [18] Jeremy Leach and John Fitch. Nature, music, and algorithmic composition. Computer Music Journal, pages 23 33, [19] Masatoshi Hamanaka, Keiji Hirata, and Satoshi Tojo. Melody morphing method based on gttm. In Proc. of ICMC, pages , [20] Pedro P Cruz-Alcázar and Enrique Vidal-Ruiz. Learning regular grammars to model musical style: Comparing different coding schemes. In Grammatical Inference, pages Springer, [21] Przemyslaw Prusinkiewicz. Score generation with L-systems. Ann Arbor, MI: MPublishing, University of Michigan Library, [22] Stanley Gill. A technique for the composition of music in a computer. The Computer Journal, 6(2): , [23] John Rothgeb. Harmonizing the unfigured bass: A computational study. PhD thesis, [24] Mathis Löthe. Knowledge based automatic composition and variation of melodies for minuets in early classical style. In KI-99: Advances in Artificial Intelligence, pages Springer, [25] Kevin Jones. Compositional applications of stochastic processes. Computer Music Journal, pages 45 61, [26] Karsten Verbeurgt, Mikhail Fayer, and Michael Dinolfo. A hybrid neural-markov approach for learning to compose music by example. In Advances in Artificial Intelligence, pages Springer, [27] RW Wooller and Andrew R Brown. Investigating morphing algorithms for generative music. In Third Iteration: Third International Conference on Generative Systems in the Electronic Arts, Melbourne, Australia. Citeseer, [28] MATTD Johnson, Daniel R Tauritz, and R Wilkerson. Evolutionary computation applied to melody generation. In Proc. of the ANNIE, [29] Andrew Gartland-Jones. Can a genetic algorithm think like a composer In Generative Art, [30] = Melomics official site [Accessed 20th April, 2015]. [31] Tim Adams. David cope: you pushed the button and out came hundreds and thousands of sonatas. david-cope-computer-composer, July 2010.

30 References 23 [32] David Cope. David Cope s personal website [Accessed 13 April 2015]. [33] David Cope. Experiments in musical intelligence, volume 12. AR editions Madison, WI, [34] DR Hofstadter. Staring emmy straight in the eye and doing my best not to flinch. Creativity, Cognition and Knowledge, pages , [35] David Temperley. The cognition of basic musical structures. MIT press, [36] Michael Chan and John Potter. Recognition of musically similar polyphonic music. In Pattern Recognition, ICPR th International Conference on, volume 4, pages IEEE, [37] Cory McKay. Automatic genre classification of MIDI recordings. PhD thesis, McGill University, [38] Ben Mowery. Solving the generalized graph search problem with genetic algorithms. [39] Claudia R Jensen. A theoretical work of late seventeenth-century muscovy: Nikolai diletskii s" grammatika" and the earliest circle of fifths. Journal of the American Musicological Society, pages , [40] David Temperley. What s key for key the krumhansl-schmuckler key-finding algorithm reconsidered. Music Perception, pages , [41] Michael W Towsey, Andrew R Brown, Susan K Wright, and Joachim Diederich. Towards melodic extension using genetic algorithms. Educational Technology Society, 4(2):54 65, [42] Adam Lipowski and Dorota Lipowska. Roulette-wheel selection via stochastic acceptance. Physica A: Statistical Mechanics and its Applications, 391(6): , [43] Melanie Mitchell. An introduction to genetic algorithms. MIT press, [44] James E Baker. Reducing bias and inefficiency in the selection algorithm. In Proceedings of the second international conference on genetic algorithms, pages 14 21, 1987.

31 Appendix A Explanation of Terms A.1 Genetic Algorithms Overview General operation of a genetic algorithm: 1. For every member in the population a fitness score is assigned to it. 2. If an adequate solution isn t found at this stage individuals in the population are selected for the next generation based on some criteria (usually relative to their fitness score). 3. Crossover or "reproduction" occurs between two "parents" in the selected population based on a predefined crossover rate. 4. Mutation occurs on individuals based on a predefined mutation rate (usually very low) 5. Process is repeated with this new population until a certain amount of generations has been reached, or until a suitable solution has been found to the problem. [43] Crossover and mutation rates are usually defined before GA operation and are often experimented with to see which rates produce the best solutions. An explanation of the terms mentioned above: Population Chromosomes The population in a GA is a dataset of candidate solutions known as chromosomes (or sometimes genomes). Traditionally the chromosomes are a binary string of 0s and 1s, analogous with a biological chromosome, but can be represented in any way deemed appropriate. Population can be of any size but is usually in the hundreds or thousands.

32 A.1 Genetic Algorithms Overview 25 Fitness Score For every chromosome in the population a fitness score is assigned to it. The fitness criteria is problem-specific but usually is the chromosome s similarity or closeness to some known value. For example if the GA were attempting to converge on a binary representation of some ASCII character then the fitness function would be how many 1 s or 0 s the chromosome has in common with that ASCII character, high similarity receiving a high score, low similarity receiving a low. Selection The selection function defines the criteria which chromosomes will be considered for the next generation. Chromosomes with high fitness score have a higher chance of being chosen for the next population, so that the desirable "traits" of that chromosome continue to the next generation. There are many methods for selection, the most common being Fitness proportionate selection (also known as Roulette Wheel Selection). This works by mapping chromosomes across a line (metaphorically), each given space proportionate to its fitness, producing a random number, and selecting the chromosome which is at that number. Chromosomes of higher fitness have a much better chance of being picked but it s not guaranteed. Other methods include stochastic universal sampling (which works similarly to roulette wheel selection though rather than one point spun multiple times, it is multiple points spun once), tournament selection and many others. An entirely new selection method can also be used in specific solutions if it is deemed more appropriate to the problem. [44] Crossover Operator Crossover operator is analogous with biological reproduction and works by choosing two (or more) parent chromosomes and mixing the contents of both to produce children chromosomes. Different methods are used for this but most follow the basic formula of choosing a random point or points in the chromosomes and swapping between parents at that point. Mutation Operator The mutation operator is used to maintain diversity in the GA population and avoid local minima by preventing chromosomes from becoming too similar. A classic mutation operator for a string of bits would be to choose a random bit in the string and invert it, though there are many varieties of mutation operator about depending on the problem at hand.