University of Manchester

Transcription

1 University of Manchester Author: Sarah King Supervisor: Dr. Andrea Schalk A third year project report submitted for the degree of BSc.(Hons) Computer Science and Mathematics in the School of Computer Science. April 24, 2015

2 This report considers the link between Computer Science, Mathematics, and classical music. It looks at programming a computer to algorithmically generate music, and attempts to determine whether music produced by a computer can mimic a piece composed by a human. The aim of this project is to research good practice from previous attempts of algorithmic composition, such as the Illiac Suite by Hiller and Isaacson, and Cope s Experiments in Musical Intelligence (as discussed in Chapter 2). The project then shifts focus onto building on these practices and developing algorithms to generate compositions based on a particular composer s style. These algorithms take various aspects of musical theory into consideration and use tactics from probability theory in order model a string of music. The compositions generated by these algorithms are compared against human compositions. There is evidence to suggest that the algorithms used in this project create the illusion of a human generated piece of music, although respondents were not completely fooled. An attempt is made to suggest retrospective improvements to the project, along with ideas for future developers to consider.

3 Firstly, I would like to thank my supervisor Dr. Andrea Schalk for being a constant support throughout this project and my time at university. Secondly, to Ben Allott for putting up with my midnight ramblings and always being there to pick up the pieces when things went wrong. Thirdly, to Rebecca Doran for sticking with me through thick and thin. Your friendship is worth the world to me, and I hope to never lose that. Fourthly, to all my friends and family who have acted as coding ducks during this past year: I could not have got to this stage without you all. Thank you for keeping me sane. Finally, to Richard Hartnell and Merle Calderbank thank you for introducing me to the wonderful worlds of mathematics and music. You are both a great inspiration and without your enthusiasm, the idea for this project could never have been born.

4 Abstract 1 Acknowledgements 2 1 Introduction Motivation Aims and Objectives Structure of the Report Background Research Previous Attempts Canonical Composition (c. 16 th Century) Dice Music (c. 1780) Twelve tone Music (c. 1910) The Illiac Suite (c. 1955) Musicomp (c. 1960) Formalised Music (c. 1960) Experiments In Music Intelligence (c. 1980) Genetic Programming (c. 1995) Analysis and Lessons Learned Design Musical Theory Note Selection Note Duration Cadences Markov Modelling Overview The Mathematics Behind The Models Representing Music Selecting a Note Critic Function Note Repetition Cadences

5 4 Implementation Software Design Handling Music The JFugue API JFugue MusicString Improving Algorithm Output Random Chance (A1) Basic Markov Modelling (A2) Markov & Critic Function (A3) Markov, Critic & Variable Note Lengths (A4) Summary Parser Testing & Statistical Analysis Random Chance Basic Markov Modelling Markov & Critic Function All Improvements Study of Musical Ability Furthering The Project Retrospective Look at Development Features To Include Suggestions for Future Projects Self Reflection Conclusion Bibliography 36 A Music Terminology 40 B JFugue Details 42 C Raw Data for Chapter 5 43 C.1 Identifying Compositions C.2 Plaintext Responses C.2.1 Random Chance C.2.2 Basic Markov Modelling C.2.3 Markov & Critic Function C.2.4 Markov, Critic & Variable Note Lengths C.3 Musicality of Respondents C.4 Responses by Musical Ability

6 The dictionary definition of music is a pattern of sounds made by musical instruments, voices, or computers, or a combination of these, intended to give pleasure to people listening to it [Dic15]. As musical tastes have developed over time, so have the combinations of instruments and voices that are considered good music. Human composers regularly push the boundaries of music, so it stands to reason that a computer could do the same thing. By teaching a computer the rules of musical theory which are essentially mathematics music can be generated. The interesting question is if the computer has created music that is passable as a human composition. This report shows how a computer can be programmed to generate music, and eventually fool people into believing that a computer generated composition was indeed composed by a human. Throughout this report, a number of musical terms are used. Please refer to Appendix A for a glossary of all of these terms. Mathematics is used in almost every academic discipline, so it follows that mathematics is heavily involved in the creation of music. The Ancient Greek mathematician Pythagoras is credited with the creation of the musical scale, noticing that strings split in the ratio of 3 : 2 produce notes that are a perfect fifth apart. Different ratios then produce different note intervals [Fra01]. Obviously, mathematics and mathematical theory are also heavily involved within computer science. Without mathematics giving us the ability to express ideas to a computer, new technologies and the field of computer science itself would not be as widespread. There are plenty of examples from history of people composing music using a computer. The earliest example of this is the ILIAC computer at The University of Illinois [Edw11]. The Iliac Suite for String Quartet was completed in 1956 and makes use of Markov chains to generate random-

7 walk pitch generation algorithms [Edw11]. These are considered further in Chapter 2. From these previous attempts, it is clear that music can be composed algorithmically. Knowing this, if the computer can trick a human with a composition. This forms the second part of the report where attempts to improve a composition previously generated by the computer are discussed, with focus on if these improvements aid the illusion of a human composition. The aims of the project evolved from research into the history of algorithmic composition (discussed further in Chapter 2), and specifically focus on the development and improvements of stochastic algorithms. The aims are: Produce a stochastic algorithm (an algorithm with random chance) that is an approximation of human-created music. Produce a function to alter a previous composition based on musical theory (discussed further in Chapter 3). Improve the random probabilities used in this algorithm in real time. Input pieces from human composers and have the ability to load the random probabilities with probabilities relating to a certain composer. Allow the user to change tempo, instruments and style of music created. Chapter 2 looks at the previous attempts to create algorithmically generated music, and reflects on lessons learned from these attempts. Chapter 3 considers what makes good music and discusses some of the musical theory choices were made throughout the project. Chapter 4 delves into the minutiae of the development, and looks at the most influential parts in creating a human-sounding composition. In Chapter 5, the statistics gathered from asking various people to listen to attempt to distinguish between human composed and computer generated music are discussed. Chapter 6 looks at how the project could be developed with more time, and takes a retrospective look at how time was spent throughout the development phase of this project.

8 Algorithmic composition is the use of a well defined algorithm when composing music [Jac96]. There is a long history of composing algorithmically in both the pre and post digital computer age [Edw11], and although algorithmic composition became more popular with the rise of the computer, algorithmic thinking is certainly a lot older [Ess07]. There have been numerous attempts at algorithmically generating music either by computer, or by hand. The main attempts have been listed and analysed here. From the 16 th century, the word canon was used to describe music that was generated by any kind of imitative musical counterpoint [Bri93]. Canons are generated from one melodic line, and follow a generative rule that controls the number of voices, various entry points and the tempo of successive voices (amongst other aspects) [Bri93]. The rule used to generate the other melodic lines was usually given verbally, and in later compositions, was denoted by markings in the score. A modern day canon can be seen in the nursery rhyme Three Blind Mice. This can be sung in a round, where the same melody is sung by a number of voices, with each voice starting at a different point. Formally, this is called a perpetual canon as the voices can begin again when they reach the end of their melodic phrase. Mozart s Musikalisches Würfelspiel was designed to compose, without the least knowledge of music, so many waltzes or ländler as one pleases, by

9 throwing a certain number with two dice [Moz87]. Musikalisches Würfelspiel consists of numerous two bar fragments of music, and contains instructions for creating music using these fragments [Nie09]. Other composers of the time were also experimenting with this concept, such as Kirnberger with The Ever Ready Minuet and Polonaise Composer [Nie09]. The Austrian born composer Arnold Schoenberg is considered the inventor of twelve tone music. This method of composing music was thought to provide a basis for the structure of a piece, and uses all 12 tones of the scale (as shown in Figure 2.1) equally. A different ordering of these 12 tones was used for each composition, and this ordering became the idea that can be found throughout the whole composition [EB14]. There are almost 500,000,000 unique permutations of a 12 tone basis that can occur, giving the composer a large scope for creating the structure of a piece of music. Figure 2.1: The Twelve Notes Used in a Scale. The Illiac Suite composed by the ILLIAC computer was the first musical composition for traditional instruments created through computer assisted composition [Nun11]. The suite was an experiment to test various composition algorithms, with the four movements of the suite being the results of these experiments. Hiller & Isaacson experimented with generation of music, canonical music, music with dynamics, and generative grammars [HI59]. The generative algorithms used in these experiments were the first reported examples of using computer algorithms for generating music. They laid the foundation of all algorithmically generated music in the future, but showed no real break from the algorithms used before [Bag98]. In this way, the Illiac Suite is sometimes considered a continuation of tradition. Musicomp was a piece of software developed by Leonard Hiller as a way of furthering his work on the Illiac Suite [Nun13]. The first musical work composed with Musicomp was entitled Computer Cantata, and was an example

10 of the various compositional procedures that could be used. Musicomp was written as a library of subroutines, giving composers the flexibility to inject some of their own ideas into the piece, rather than relying solely on the computer (as in the Illiac experiments) [Mau99]. These subroutines include a Musikalisches Würfelspiel inspired selection procedure, amongst others [Ari05]. Formalised music refers to software created by Iannis Xenakis to produce data for stochastic algorithms. Using the computer s ability to calculate at high speeds, Xenakis focussed on various probability theories to improve compositions [Xen63]. The program would work out a score from a list of notes and probabilistic weights, and then make a decision based on a random number generator [Alp95]. This method of composition combines stochastic algorithms and the rule based systems similar to those used in Illiac. However, in the compositions produced using formalised music, the computer has not actually produced the resultant sound; it has only aided the composer by virtue of its high-speed computations [Cop84]. David Cope s Experiments in Music Intelligence (EMI) software mimics composers and creates works that are similar sounding [Coc01]. It was initially developed by Cope as a way to overcoming his composer s block, and would be used to track the current mood of a piece, whilst being able to generate the next note/measure/phrase [Cop81]. The EMI software contains a large database of music, descriptions and styles. Using this database, the EMI software deconstructs and recombines musical phrases, giving the illusion of mimicking a particular composer. The deconstructions and recombinations requires human intervention, however, in order to ensure that the recombinations are not utter gibberish [Cop81]. Genetic algorithms evolve in ways that resemble natural selection [Hol15]. Initially, the population of music is filled with a randomly selected (or human composed) piece. Then, iterations of the algorithm are performed over this data in order to produce something that sounds more musical. This process is shown in Figure 2.2. Genetics algorithms are fairly new, and as such have only been used to create harmonisations and accompaniments to existing pieces of music [Dos03]. Genetic algorithms are able to generate their own musical materials as well as form their own grammars. Composers must program their own critic

11 function which listens to the music generated by the algorithm and decides if it is a suitable representation of music [Mau99]. Figure 2.2: A simplified version of a genetic program run. Canonical Composition (2.1.1) is a perfect example of the use of a deterministic algorithm when algorithmically generating music, i.e. using an algorithm with no random choice. If one considers singing in the round, it is known beforehand how the music will sound. Thus, this music can be considered very prescriptive, which is the reason that deterministic algorithms were discounted as a viable way of generating good music. With the development of stochastic algorithms came the creation of Dice Music (2.1.2) and Twelve tone Music (2.1.3). The element of random choice here would enable an algorithm to force choices that a human may (or may not) make, and would therefore create more interesting pieces of music than a deterministic algorithm. However, creating a piece of music completely randomly does not necessarily mean it is a good piece of music. This would be more suited to building up a piece of music using small phrases of good music that have been developed using another algorithm. Moving onto more modern attempts, the Illiac Suite and Musicomp both show great advances in the algorithms used for creating music. The use of Markov Models shown in Musicomp became prevalent and became the basis for preparing an algorithm for generating music. However, both Illiac and Musicomp were considered by their developers as excused from aesthetic scrutiny as the studies were designed to test the efficiency and ease of use of the algorithm [Ari05]. Hence, it is unclear how The Illiac Suite and

12 Computer Cantata were received by audiences at the time. Finally, Formalised Music (2.1.6), Experiments in Musical Intelligence (2.1.7) and Genetic Algorithms (2.1.8) all have similar positives and negatives. They all show a break from the traditional algorithms used by Illiac and Musicomp. However, these solutions require greater computational time and power than I had at my disposal, so I have not been able to fully pursue these solutions. But, the Critic function used in Genetic Programming models seemed feasible to complete in the given time, so this idea will be taken forward into the development process. In summary, the elements from this research that are implemented in the project are: Markov models for note and duration selection. Critic function to analyse a composition created by the computer. This judges the compositions based on various aspects of musical theory as discussed in Chapter 3. A database of previous note combinations used to provide increasingly accurate data for the Markov model solutions. This makes these solutions more accurate, as the probabilities from each piece can be stored and added to existing probabilities. The ability to parse a piece of music in real time to improve the Markov probabilities.

13 There has been a lot of debate about what makes good music good. This chapter looks at some of the bigger concepts in this debate, and then discuss how these concepts were combined into a single Critic Function used to judge a piece of music generated by a computer. There are various different elements that make up a piece of music, from the basic notes that are used in the piece to the more complex chord progressions used to create a texture (or timbre). A lot of the information here was found in [BBC14]. A note used in a piece of music is selected from the key signature. The key signature ensures that the notes selected are all from the same scale, and hence sound nice in comparison with each other. In a piece of music, there is normally one main key signature that is used, with occasional modulations into another key. These modulations can create tension or change the mood of the piece as required. It is also important to select notes that are commonly used after each other, as evidence (in the form of other pieces of music) suggests that these note combinations work well together. A note s duration is the length of time that a note is played for. There are 5 main durations that are used in music semibreves, minims, crotchets, quavers and semi quavers [The15]. A semibreve is a note with the longest duration typically 4 beats long. A minim is a note with half the duration of a semibreve. Next, there is the crotchet, which has half the duration of

14 a minim. Quavers and semi-quavers are the quickest moving notes, with a quaver having half the duration of a crotchet and a semi quaver having half the duration of a quaver. This hierarchy is shown in Figure 3.1. Figure 3.1: The Hierarchy of Notes. [The15] A cadence is a sequence of notes or chords that generally signifies the end of a musical piece or phrase. There are 4 types of cadence that are commonly used in music. Chord progressions are generally written as Roman numerals, where major chords are upper case numerals, and minor chords are lower case numerals. Finally, diminished chords have a small circle to signify that they are diminished.the application of Roman numerals to the C major scale is depicted in Figure 3.2. Figure 3.2: Roman Numerals Applied to a C Major Scale [The15]

15 A perfect cadence is a chord progression from V to I. This creates the feeling that the music has come to a definitive end, and as such are usually used at the end of a piece of music. A plagal cadence is a chord progression from IV to I. This also creates the feeling that the music has come to a definitive end, and can also be found at the end of a piece of music. The plagal cadence was traditionally used in plainchant songs that emerged around 100 A.D. [Est15] as it is commonly sung at the end of hymns to the A men. An imperfect cadence is a chord progression from I to V. Unlike the perfect or plagal cadences, an imperfect cadence does not sound finished. They are used at the end of movements (as the music is carrying on into another movement) or in the middle of a piece at the end of a particular section. Imperfect cadences sound as though they want to carry on to complete the music properly. An interrupted cadence is a chord progression from vi to vii. An interrupted cadence does not provide a satisfactory end to a piece of music, and is used in the same way as an imperfect cadence. The algorithm used to create music relies heavily on the use of Markov models. Markov chains were used in previous attempts such as Musicomp and Formalised Music outlined in Chapter 2 as they provide an effective mechanism for creating and using stochastic matrices in musically satisfying ways [SB15]. The information that follows was primarily taken from from Victor Powell s excellent interactive tutorial [Pow14]. Markov chains, named for Andrey Markov, are mathematical systems that move between states which represent a situation or some values. Alongside state names, there is also a set of probabilities that represent the chance of moving from one state to the next. Markov models take into consideration the events that occurred immediately before (and the probabilities of these events happening), implying that the outcome could be changed dramatically depending upon the events that precede a particular event. In a two-state system, there are 4 possible transitions that the model must take into consideration: A A, A B, B A, and B B (as states can always transition to themselves). In this simple system, depicted

16 Figure 3.3: A Simple Two State System A B in Figure 3.3, the probability of transitioning from one state to any other is 0.5, as at each state there are two places it can transition to (with even weighting). Expanding this model, if a state has N links, there is a 1 / N chance of transitioning to another state. Of course, it can be that a certain path is more favourable than another, and weight the transition probabilities accordingly. The skewing of transition probabilities helps to model real life situations accurately. The theory of Markov models can now be generalised into mathematics, using probability theory to model the effect created by a Markov chain diagram. First, consider the chain rule of conditional probability. Suppose there are indexed sets A 1,..., A n. Using the definition of joint probability [Tri12], the value of a joint distribution of these indexed sets is: P (A n,..., A 1 ) = P (A n A n 1,..., A 1 ) (3.1) Iterating this process with the final terms gives: ( n ) n k 1 P A k = P A k k=1 k=1 j=1 To explain this fully, suppose there are three variables: A j (3.2) P (A 3, A 2, A 1 ) = P (A 3 A 2, A 1 ) P (A 2 A 1 ) P (A 1 ) (3.3) where P (A B) is the probability of event A happening given that event B has already happened [Wik15]. As an example, suppose there are two buckets. Bucket 1 has 4 white balls and 5 black balls, and bucket 2 has 1 white ball and 7 black balls. Let A be the event that the first bucket is selected: P (A) = 0.5. Let B be the event that a black ball is picked randomly. The chance of picking a black ball, given that the first bucket was selected is: P (B A) = 5 9

17 So, P (A, B) = P (B A) P (A) = = 5 18 However, Markov models differ in the sense that they only consider the event immediately prior in the calculation, and then sum all of these probabilities. Applying this to Equation 3.2 gives (adapted from [Lee10]): ( n ) n P A k = P (A k A k 1 ) (3.4) k=1 k=1 and applying this to Equation 3.3 gives: P (A 3, A 2, A 1 ) = P (A 3 A 2 ) P (A 2 A 1 ) P (A 1 ) (3.5) Music can be separated, very simplistically, into notes and duration, making the application of Markov models to a piece of music very easy. For the remainder of this section, the traditional nursery rhyme Twinkle Twinkle Little Star will be used; the musical notes for which are displayed in Figure 3.4. The piece of music can be represented in matrix form. Equation 3.6 is the result of parsing this piece of music into a matrix A, and incrementing a position A ij if the note represented by row i was followed by the note represented by column j. C D E F G A B C D E F G A B (3.6) To calculate probabilities of one note following another, the matrix is made row stochastic (each row sums to 1). The stochastic matrix is shown in Equation 3.7.

18 C D E F G A B C D E F G A B (3.7) Figure 3.4: Twinkle Twinkle Little Star In order to choose the correct, or the most probable, note in a sequence, a random number is generated in the range of 0 1. Then, considering the row of the stochastic matrix that corresponds to the current note, the algorithm steps along the row and sums the probabilities of notes it could move to as it reaches them. If the sum of these probabilities becomes greater than the random probability, the algorithm moves to the state which had the probability that tipped the sum over. Initially, a random starting position is selected for the beginning of the piece and then Markov modelling is applied on every note after that. The full pseudocode can be seen in Algorithm 1. Note that allnotes described in the algorithm is the stochastic matrix of probabilities. The Critic function judges a composition that is generated by the computer and changes various aspects to suit the musical theory rules outlined in Section 3.1. This function focusses on minimising the number of repeated notes and applying a cadence at the end of the piece, although there is scope to increase the number of features the Critic function checks for.

19 Algorithm 1 Pseudocode for Markov Models [SB15] Require: number of notes to produce called sizeofpiece Require: double array of all notes called allnotes seednote random note (initially) outputnote for int i 0 to i sizeofpiece do targetprob random number between 0 1 currentsum 0.0 for outputnote 0 to outputnote allnotes.length do currentsum += allnotes[seednote][outputnote] if targetprob currentsum then break end if end for seednote = outputnote end for To achieve minimal repetition of notes, the Critic function analyses the notes used in the composition and then decides if a note has been repeated too many times, in particular checking if a note is repeated more than three times consecutively. The pseudocode for this can be found in Algorithm 2. This function should not change for the majority of compositions that are created. As new pieces are created, and existing pieces are analysed, the probabilistic weights assigned to the notes should become more realistic. Thus, this would reduce the number of notes that are repeated numerous times, as this is a feature not normally found in a traditional piece of music. Algorithm 2 Pseudocode for Finding Repeated Notes Require: non empty byte array of notes, called notes for int i 1 to i notes.length do currentnote notes[i] previousnote notes[i - 1] nextnote notes[i + 1] if currentnote = previousnote and currentnote = nextnote then Return the position i. else There are no repeats in these three notes. end if end for

20 The application of a cadence is something that requires a little more thought. As is discussed in subsection 3.1.3, there are a number of cadences that can be applied at the end of a composition generated by a computer. The cadence to be selected will be chosen using a random number generator, with the emphasis on selecting a finished cadence. If an unfinished cadence is selected, the Critic function will add on more notes (using Markov modelling) and apply a new cadence at the end of this extended piece. This continues until a finished cadence is added to the end of a piece. This process guarantees that the composition will always end of a satisfying note. Another thing to consider is the chord progression that is used to when applying a cadence. Initially, the composition is in a fixed key, allowing us to select the first, fourth, fifth, sixth, and seventh of a scale as required. In order to select the correct octave to apply to the cadence, the octave of the last note used in the composition is calculated. Finding the octave can be achieved simply, by finding the floor value of the midi value of the note divided by 12 (the number of octaves achievable by midi values). Once the octave is calculated, the cadence is created by simply working with the midi values and adding (or subtracting) intervals as required. The pseudocode for this algorithm can be found in Algorithm 3. Algorithm 3 Pseudocode for Applying a Cadence Require: non empty byte array of notes, called notes rand random number in range 0 1 lastmidi notes[notes.length - 3] octave octave of lastmidi if rand < 0.4 then Apply perfect cadence: notes[length - 2] = fifth and notes[length - 1] = root else if rand >= 0.4 and rand < 0.8 then Apply plagal cadence: notes[length - 2] = fourth and notes[length - 1] = root else if rand >= 0.8 and rand < 0.9 then Apply imperfect cadence: notes[length - 2] = first and notes[length - 1] = fifth else Apply imperfect cadence: notes[length - 2] = sixth and notes[length - 1] = seventh end if

21 This chapter is concerned with the interesting or problematic aspects of implementation of the Critic, Parser and Markov functions, and also touches briefly on the software design principles used throughout development. The software design principles discussed in this chapter were mainly taken from John Sargeant s third year course entitled Software Design using Patterns [Sar15]. The project was split into a number of iterations which lasted 3 4 weeks each in keeping with the Agile method of software development. In each iteration, the algorithms that generated music were developed, improved, and even completely thrown away in some cases. This resulted in more aspects of musical theory being implemented into the algorithms. The hardest iteration was the third, as this saw the implementation of the Critic function, which is discussed further in subsection The Critic function was difficult to implement because it required a complete rethink of how the music was being stored, as well as changing large parts of the logic in the program such as how notes were being played by the JFugue player (discussed in section 4.2). The third iteration forced all of the code to be refactored into the classes that are present within the code now. Using the Agile principles of low coupling and high cohesion, classes were divided into smaller, specific entities, some of which acted as helpers to the other more important classes. Finally, I used the Agile principle of Information Expert to encapsulate all code for a particular function in one class, making the program more cohesive. This principle could also be used in reverse to decide where some functionality should go.

22 The JFugue API is an open source library that enables the composition of music in the Java programming language without having to worry about the MIDI conversions etc [Wik14]. As the use of JFugue makes dealing with the compositions a lot easier, the development of the algorithms became the focus of the project. A number of features from the JFugue API that certainly eased the development process are discussed below. The following discussion is mainly taken from The Complete Guide to JFugue [Koe08]. The JFugue MusicString is a specially formatted String object that consists of music instructions. The MusicString can consist of notes, durations, and can also control the tempo and instrumentation of a piece. In order to specify a note, it is enough to specify the note name: C, D, E, F, G, A, B, or R (to specify silence). After this specification, it is simple to sharpen (by appending a # ) or flatten (by appending a b ) a note. Appending a number in the range of 0 10 after the complete note name selects the octave that the note will sound from. The notes available in JFugue are shown in Appendix B, Figure B.1. A duration or length of a note can be appended to the note in the MusicString after the octave marking. There are 8 different lengths of notes that can be applied, which are shown in Appendix B, Figure B.2. As more markings are added to the MusicString, it becomes less readable by humans. But, the format is very easy to build up using String objects in Java. The strict pattern that is followed in order to create a detailed music string is easy to implement, allowing complicated strings to be created easily Music played by JFugue makes use of MIDI in order to render the Music- String into a playable form. There are 128 different instruments that are standard across MIDI devices, although the sound quality varies between the instruments. To select an instrument, an optional argument is placed at the beginning of the MusicString Ix, where x is an integer between 0 and 127. The tempo of a piece of music can drastically change the way a piece of music sounds. Generally, a faster piece of music is more stimulating and

23 creates a heightened physiological response than a slower piece even if the same notes are played [vwv11]. To add a tempo, an optional argument is prepended to the MusicString, of the form Ty, where y is an integer. There are, however, limitations created by using JFugue. Additional markings that a musician would typically expect in a piece of music, such as markings showing accents placed on a note, are not yet supported by JFugue. This lowers the realism of a composition that can be produced by the algorithms. In order to investigate how a good piece of music can be created, a number of different algorithms are used. This section briefly discusses each improvement of the complete algorithm. Each composition is judged on the following qualities, with justification for each found in the specified sections. Tonality is the piece constrained to one key? (section 3.1.1) Variety in pitch but within a playable bound (section 3.1.1) Variety in duration of notes (section 3.1.2) Ending of piece feels final and conclusive (section 3.1.3) Notes are not repeated multiple times (section 3.3.1) This algorithm takes a very simplistic view on how notes are selected. A random integer between is generated, converted into a MIDI note and played. Figure 4.1 shows a typical output from this algorithm 1. The output of the random algorithm is compared against the five qualities of good music (as shown in Table 4.1), and does not achieve any of these qualities due to the completely random nature of the algorithm. The Markov algorithm (as explained in Section 3.2) chooses a random starting point for the composition, and then uses each previous note as a seed to 1 The two lines of music shown are played concurrently, and are shown on two staves as this is how notes in these octaves would normally be displayed.

24 Figure 4.1: Typical Output of Random Algorithm generate the next most likely note. Figure 4.2 shows a typical output from this algorithm 2. Some of the five qualities for a good piece of music have been achieved. The music now has some degree of tonality as all the probabilities in the Markov matrices are realistic, and become increasingly realistic as more pieces analysed. Secondly, the Markov models enable the algorithm to select notes that commonly follow a particular note, achieving the second aim. Figure 4.2: Typical Output of Markov Algorithm Applying the Critic function, which minimises the number of repeated notes in a piece and enforces a cadence ending, drastically improved the quality of the piece of music. Whilst the difference in pieces cannot truly be seen by looking at the sheet music (in Figure 4.3), the difference can be heard when the two pieces are played consecutively. Now, considering the five qualities that should be aimed for in a composition, this algorithm has now ensured there is always a cadence ending, and that notes are not repeated more than three times consecutively. As a final improvement to the algorithm, Markov models were applied to the note lengths. This is done by maintaining a separate matrix for the 2 This output is shown on one stave, as there is no need to go onto the lower bass stave.

25 Figure 4.3: Typical Output of Markov Algorithm & Critic Function probabilities of particular note lengths following another. This models the effect of having running quavers or semiquavers. This is a common compositional feature, as it creates fast moving sections of music that help the piece of music flow. The Critic function was adapted to ensure that the application of a cadence to the end of a piece, also tied in with the application of a crotchet followed by a minim (one beat note followed by a two beat note). This further adds to the conclusive feel of the end of a composition. A typical output of the final algorithm is shown in Figure 4.4. This algorithm achieves all the aims of good music outlined at the beginning of section 4.3. Figure 4.4: Markov Applied to Notes & Lengths, & Critic Function This table summarises how each algorithm met the aims of good music. Table 4.1: Algorithms Compared Against the Aims of Good Music Aim A1 A2 A3 A4 Tonality Pitch Note Lengths Cadence Ending Note Repetition An important aspect of the project is the Parser, as it allows the Markov models to be updated in real time, and adds extra functionality to the overall

26 piece of software. When the program starts running, the user is presented with a choice of generating a piece of music or adding information to the database. The user inputs a piece of music in the format of a MusicString, and in order to support the addition of Markov modelled note lengths, the notes and durations are separated. Temporary Markov matrix are created, and then combined with the existing matrices. As an example, consider the matrices used in eqs. (3.6) and (3.7), and then combine these existing matrices with the temporary matrices created when parsing Frère Jacques, the notes for which are: {G4 A4 B4 G4 G4 A4 B4 G4 B4 C5 D5 B4 C5 D5 D5 E5 D5 C5 B4 G4 D5 E5 D5 C5 B4 G4 G4 D4 G4 G4 D4 G4} Formulating a matrix based off the newly parsed piece gives: C 4 D 4 E 4 F 4 G 4 A 4 B 4 C 5 D 5 E 5 C D E F G A B C D E (4.1) The Parser function performs matrix addition on the new and existing matrices of integers (eqs. (3.6) and (4.1)) and then recalculates the new probabilities by summing the rows of this combined matrix and dividing each element by the sum. This yields: C 4 D 4 E 4 F 4 G 4 A 4 B 4 C 5 D 5 E 5 C D E F G A B C D E (4.2) Thus, the probabilities become more accurate for the notes A 4... D 5 as these are the most common notes in the parsed pieces. The new notes are

27 included in the combined matrix, but the probabilities cannot be expected to be as accurate, but they are still considered in the calculation. The same process is used to improve the probabilities of the note lengths.

28 This chapter focuses on testing the output of the algorithm, rather than detailing software tests. An online survey was created, which linked to a number of computer and human generated pieces in an open Dropbox folder. The instructions in the survey told the respondent which pieces to play for each question, and invited them to select which they thought was computer generated (with a option of Can t Decide for the indecisive). Each question had a human and computer generated piece, so there were no trick questions. In total, there were 54 respondents of various musical ability. Raw data can be found in Appendix C. The random output from the program was compared to Steve Reich s Phrygian Gates. This helped to level the playing field slightly as both pieces were atonal and were unlikely to end on a conclusive note. The table of results for this question can be found in Table C.1, and the plaintext comments can be found in full in Appendix C.2.1. Figure 5.1 shows the distribution of responses. Piece 1 was highlighted as computer generated by 62.96% of the respondents, which was accurate. Looking at the plaintext responses, this seems to be because Piece 1 was deemed too random and lacking a humanly perceived notion of harmony. The intervals appeared more haphazard in Piece 1. However, some respondents said Piece 1 was much more pleasing to listen to, but the general consensus showed that Piece 1 was obviously a computer generated piece of music. There were a few comments pertaining to the respondents not being entirely sure which of the pieces was computer generated, as both seemed haphazard.

29 Figure 5.1: Distribution of Responses (Piece 1 & 2) The basic Markov output was compared to a more structured minimalist piece; a stripped down version of New York Counterpoint by Steve Reich. The table of results for this question can be found in Table C.2, and the plaintext comments can be found in full in Appendix C.2.2. Figure 5.2 shows the distribution of responses. Piece 3 was highlighted as computer generated by 48.15% of the respondents, narrowly beating Piece 4 (with 44.44% of the vote). This was the incorrect choice as Piece 4 was computer generated, but as the margin was so narrow; there is no significant statistical evidence suggesting the algorithm truly confused respondents. Respondents thought that Piece 3 feels more fluid than Piece 4, more so that a human would have composed it and that they were sure that Piece 3 is generated by a computer because there are sequences of notes which sound unnatural. Piece 4 was considered more complex and that it was used as inspiration for creating Piece 3. However, there were comments suggesting that Piece 4 seemed too chaotic, and that it didn t have much of a flow. This suggests that whilst the respondents were (marginally) tricked by the pieces in this question, it may have been down to sheer luck in the choice of the human generated piece. Based on the comments, the need for the Critic function as a means of improving the piece becomes apparent.

30 Figure 5.2: Distribution of Responses (Piece 3 & 4) The output obtained from applying the Markov algorithm and the Critic function was compared to an excerpt from Vaughan William s English Folk Song Suite. The output from the combined algorithm is a lot more sophisticated, and hence it should stand up against a truly classical piece of music. The table of results for this question can be found in Table C.3, and the plaintext comments can be found in full in Appendix C.2.3. Figure 5.3 shows the distribution of responses. This was the most surprising response. Piece 6 was highlighted by the 37.04% of the respondents, with one third of all respondents being unable to decide between the two pieces. Both were pleasant to listen to, and both sounded human to the respondents. Piece 6, however, was the Vaughan William s excerpt. One respondent stated that Piece 5 finished nicely, and had a nice continual tempo. It seemed to fit a lot of music I had heard in the past, making it more believable. The two pieces were not expected to cause as much consternation as they did. The fact that one third of the respondents were confused between the two pieces, and more than half of the remaining respondents highlighted the incorrect piece as computer generated is surely a victory for this algorithm. It was said by a few respondents that Piece 6 sounded a bit synthetic and more robotic, but the general consensus of the comments was that there was little or no difference between these pieces.

31 Figure 5.3: Distribution of Responses (Piece 5 & 6) Prior to the creation of the final piece, thirty bars of fifteen distinct Mozart pieces were parsed in an attempt to get a good idea of his style. The output from this algorithm was compared to Mozart s Divertimento in Bb major, no part of which was included in the Markov matrices. The table of results for this question can be found in Table C.4, and the plaintext comments can be found in full in Appendix C.2.4. Figure 5.4 shows the distribution of responses. As this was theoretically the best combination of algorithms, this should have been the piece to confuse the respondents. But, 62.96% of the respondents correctly identified Piece 8 as the computer generated piece of music, saying that Piece 8 did not seem to have the melody that Piece 7 had. The addition of note lengths seems to have been the downfall here. Considering some of the plaintext comments, it was observed that The awkward timing of Piece 8 and the fluid-ness of Piece 7 made me sure Piece 8 was generated by a computer. Note lengths applied using Markov modelling seems to have been a slightly primitive solution. As is discussed in Section 6.1.1, there are more accurate ways that note lengths could have been applied.

32 Figure 5.4: Distribution of Responses (Piece 7 & 8) There was a good mix of musical ability in those who responded to the survey. Only 27.78% of respondents had never played/sung before, with all other respondents having some degree of musical background. There were, however, no professional musicians which responded to the survey even though some were specifically targeted. Figure 5.5 shows the full distribution of musical ability amongst the respondents. Before creating the survey, a positive correlation between musical ability and ability to identify a computer generated piece of music was expected. However, it seems to be in reverse. The greater the musical exposure, the worse respondents seemed to do. Respondents who used to play/sing had the greatest ability to correctly identify a piece of computer generated music. There were 21 respondents, and on average they were able to correctly identify 2.52 pieces correctly, with a standard deviation The small standard error of the mean (0.18), suggests there is reasonable confidence that the mean generated reflects the true sample. Five of these respondents were able to identify all four computer generated pieces. Depending upon the level of ability they reached, the respondents in this category could have had a large exposure to music before they stopped playing. Hence they would have an idea of what makes a piece of music good. Coming next in ability to identify computer generated music were the

33 respondents who had never played/sung before. There were a total of 14 respondents in this subgroup. An average of 2.43 pieces were identified correctly, with a standard deviation of However, the standard deviation and standard error (0.31) were the largest of all the subgroups, suggesting that there was more variance within their responses. There were two respondents who could correctly identify the four computer generated pieces. The variance in these responses may be influenced by external factors, such as music preference. This was not considered by the survey, but for the respondents who have a preference for electronic music, they may have been more able to identify the computer generated music. Bringing up the rear are the respondents who currently play/sing. There were 17 respondents in this category. On average, they were able to identify 2.18 pieces correctly, with a standard deviation of This was the smallest standard deviation, and also yielded the smallest standard error suggesting that the representation of the mean of the sample is accurate. There were no respondents in this category that could correctly identify all four computer generated pieces. The poor accuracy of the most musical respondents could be explained by over exposure. As musicians, they have to play a wide variety of pieces from obscure to well known. Thus, their exposure to unconventional pieces of music would be greater, and could therefore be skewing their results. Figure 5.5: Distribution of Musical Ability

34 This chapter retrospectively considers the development process and comments on the features of the project that could have been included/improved. There are a number of features that would further improve the quality of the music that the computer can produce. These weren t included as either JFugue does not yet support the particular feature, or there was no time to add the feature. A piece of music is naturally split into bars. These are divisions of the music that contain a number of notes, with durations that total the number of beats in a bar (as specified by the time signature). A phrase is a group of bars that contain a particular motif (or theme) that is carried through the piece of music. Phrases are generally repeated throughout a piece, although they may be modulated by a certain interval, or played by a different instrument. The addition of bars and phrases would enable the piece of music have some repetition a point that was highlighted by a number of respondents to the survey. This feature was researched after the development of the generation algorithms towards the end of the project and so there was not enough time to fully implement this feature. With hindsight, the algorithms should have developed with this in mind from the beginning.