Computing, Artificial Intelligence, and Music A History and Exploration of Current Research Josh Everist CS 427 5/12/05
Introduction. As an art, music is older than mathematics. Humans learned to manipulate primitive instruments long before writing or mathematics existed. Indeed, one of the first uses of mathematics was to define music in terms of pitch and rhythm. In ancient Greece, the study of music was as important as arithmetic, astronomy, and geometry. The concept of the music of the spheres was a symbolic effort by the Greeks to tie together the seven known planets, the seven colors of the rainbow, and the seven notes of the diatonic scale. The origin of this concept is attributed to Pythagoras (c. 582-497 BC). One of its most important proponents was Plato, whose Timeaus contains enigmatic references to the Pythagorean ideas. In particular, Pythagoras, along with discovering the famous Pythagorean theorem, with which all budding mathematicians and computer scientists are familiar, also discovered the harmonic properties of the octave, the perfect fourth and the perfect fifth properties that sadly many mathematicians and computer scientists are not familiar with. The octave is the basis of all modern music, or at least music that has been composed in Western civilization since the time of the Greeks. The octave is simply a doubling of musical pitch. A tone played at 440 Hz(cycles per second). is the same note as a tone played at 880 Hz., only an octave down. Fig 1. An A note, played at 440 Hz. (Sound sample:http://www.cs.unm.edu/~jeverist/music/a440.wav)
Fig 2. An A note, played at 880 Hz. (Sound sample: http://www.cs.unm.edu/~jeverist/music/a880.wav) The diatonic scale is a division of seven notes over the octave. The perfect fourth and perfect fifth are the fourth and fifth notes of the scale. Fig 3. A diatonic scale(c major) (Sound sample: http://www.cs.unm.edu/~jeverist/music/cmajor.wav) Here, we come to an extremely interesting feature of music the ability of music to convey emotion. While the major scale is described as conveying happy, joyful sounds, the minor scale is described as conveying sad and reflective sounds. It is something worthwhile for mathematicians and computer scientists, especially those involved in artificial intelligence, to consider: one can express emotion with purely mathematical language!
Fig. 4. The A minor scale. This scale uses the same notes as the C major scale, but the octave is played from A to A, rather than from C to C. (Sound sample: http://www.cs.unm.edu/~jeverist/music/aminor.wav) Beyond being able to describe pitch, which is the basis of melody and harmony, mathematics can describe virtually every other aspect of music. Rhythm, for instance, is the division of notes into figures like eighth notes(eight notes per bar), quarter notes(four notes per bar), and triplets(a division of quarter notes into three notes per quarter note). The timbre of music meaning the tonal difference between instruments like the violin and the trumpet can also be precisely defined in mathematical terms. The simplest mathematical term describing timbre is the sine wave. Along the audio frequencies, the sine wave produces a distinct sound. Most electronic synthesizers consist of waveform generators that produce only three different waveforms the sine wave, the square wave, and the sawtooth wave. Add the ability to adjust attack, decay, sustain, and release to these basic waveforms, along with a few basic electronic filters, and virtually any sound can be duplicated. These days, sounds can be replicated with startling accuracy, good enough to fool most listeners and even many experts. The ability to make a sound wave sound like a trumpet or a cello or a grand piano is yet another thing that can be expressed in purely mathematical terms.
Computers, Artificial Intelligence, and Music Pitch, rhythm, and timbre are all that is required to describe a musical composition. A musical score will contain only combinations of these elements. Since the fundamentals of music can be precisely defined in mathematical terms, musicians have used computers to help them write and compose music ever since it became practical to do so. But just like symbols and numbers are meaningless unless they are arranged into written works or computer programs, the basic elements of music too are meaningless until they are skillfully arranged into a musical composition. The central question for researchers investigating artificial intelligence as it applies to music is: can computers be programmed to compose music? Can they be trained to compose music that would pass the Turing test meaning a listener would not be able to tell the difference between a human-composed piece of music and one that was generated by a computer program? The answer to this is yes, to a certain extent. As computers become more powerful and artificial intelligence techniques grow more sophisticated, the ability of computers to compose original music will grow. Already though, there are programs exist that can effectively mimic certain styles of music, and even pass the Turing test in many cases. One reason computers are able to produce music in certain styles is as follows: Just as many of the fundamental algorithms of computer science were invented long before the advent of electronic computers, algorithms were also invented to aid in the composition of classical music. The eighteenth century Musikalisches Würfelspiel, or musical dice game, was one of the first examples.
The Musikalisches Würfelspiel is a form of music composed with a previously composed work, a matrix, and two 6-sided dye. The numbers down the left side of the matrix represent the eleven possible results from rolling two dice. Each number in the matrix links to a previously composed measure of music. To generate new measures of music, one rolls the dice, and then takes a corresponding measure of music from the previously written score. A typical Würfelspiel is 16 measures: the following matrix could therefore generate 11^16 different new pieces. Fig. 5. A matrix for a first phrase from a Musikalisches Würfelspiel attributed to Franz Josef Haydn. A B C D E F G H 2 96 22 141 41 105 122 11 30 3 32 6 128 63 146 46 134 81 4 69 95 158 12 153 66 110 24 5 40 17 113 85 161 2 159 100 6 148 74 163 45 80 97 36 107 7 152 60 171 53 154 68 118 911 8 152 60 171 53 99 133 21 127 9 119 84 114 50 140 86 169 94 10 98 142 42 156 75 129 62 123 11 3 87 165 61 135 47 147 33 12 54 130 10 103 28 37 106 5 Besides Haydn, CPE Bach and Mozart were other notable composers of the Musikalisches Würfelspiel.
In the past fifty years or so, composers wishing to use mathematical algorithms and artificial intelligence techniques have done so with computers. Some of the pioneering researchers and composers in artificial intelligence and music were the following: Iannis Senakis is particularly remembered for his pioneering electronic and computer music, and for the use of stochastic mathematical techniques in his compositions, including probability (Pithoprakta), aleatory distribution of points on a plane in (Diamorphoses), minimal constraints(achorripsis), Gaussian distribution(st/10, Atrèes), Markovian chains(analogiques), game theory (Duel and Stratégie), group theory (Nomos Alpha), and Boolean algebra (Herma, Eonta). Kemal Ebcioglu, primarily a compiler researcher at IBM, used predicate calculus to develop more than 350 rules of voice leading for creating chorales in the style of J.S. Bach. His program effectively portrays the techniques of four-part writing. William Schottstaedt, a Stanford university researcher, created a program called Counterpoint Solver, which produces counterpoint in the Baroque style of the early 18 th century. He is also the inventor of Common Lisp Music, a sound synthesis language that is fast, efficient, has real-time capabilities and runs on many different types of computers. Charles Ames, a researcher for the Kurzweil Group, created a program called the Cybernetic Composer that composes in four different popular styles: "standard" jazz, Latin jazz, ragtime, and rock. The Cybernetic Composer program works in four layers-- solo, background chords, bass line, and drums. Each layer works (almost) independently
of the remaining lines. Pitch, rhythm, thematic structure, and other compositional choices are calculated by a series of rules. Christopher Yavelow is the creator of an interesting children s program called Push Button Bach(PushBtnBach). Primarily for children, PushBtnBach will keep generating new minuets in the style of Bach simply pressing the Compose button in the program. Composed music can be played directly on a MIDI synthesizer if one is connected to the computer, or it can just play through a computers soundcard. PushBtnBach was one of the only programs I have mentioned so far that is still available for download it can be obtained at http://www.kidsdomain.com. In recent years, Dominik Hornel and Wolfram Menzel have made use of neural networks to create music with stylistic similarities to composers of the Renaissance and Baroque periods. While their focus has been primarily on harmonization and melodic variation, their work departs from other approaches based on programmed rules. Rather, their program is fed one or more examples of music and their neural network then learns through the process of backpropagation. Eduardo Miranda, a research scientist now with Sony, began researching artificial intelligence and music at the University of Edinburgh. His research primarily involves the use of cellular automata to compose music. One of the most interesting facets of Miranda s research is that he is not only applying artificial intelligence techniques to the creation of musical structures, but to the actual synthesis of the sounds used in the compositions as well.
The last researcher I will mention is David Cope, a professor at USC. He has written a program called Experiments in Musical Intelligence, affectionately known as Emmy. Although Emmy uses more traditional techniques of artificial intelligence than some of the ones I have mentioned above, such as combinatorics, pattern matching, and probability, his program has succeeded in passing the Turing test, and has been praised by many other researchers in the field. To demonstrate how well his program works, I have made available two sets of files that can be downloaded from http://www.cs.unm.edu/~jeverist/music. The first set is two pieces of music by J.S. Bach, and two pieces that were composed by the Emmy program. The second set is two pieces by Frederic Chopin and two pieces composed by Emmy. The answers as to which is which can be found on the site, however, I would encourage the reader to try to guess which is which before looking. Conclusion. Music is a language that can be expressed precisely through mathematics. Because of that, the use of computers in music came about as a natural evolution. As the field of artificial intelligence grew, so did its applications toward the composition of music. It is unfortunate however, that many mathematicians, computer scientists, and artificial intelligence researchers completely ignore music in their careers. I believe research in music and artificial intelligence might help those involved in computational linguistics better approach the problem of grounding because as I demonstrated above, music can be a precise way to mathematically model emotions. It would be interesting to see what
kind of interdisciplinary research could come about from people working in music and other areas of artificial intelligence because of this. Bibliography Virtual Music Computer Synthesis of Musical Style, David Cope, MIT Press 2001 Godel, Escher, Bach, Douglas Hofstader, Basic Books 1999 Wikipedia, http://www.wikipedia.org