Final Project: Musical Memory

Transcription

1 Final Projet: Musial Memory Jeff Kaufman May 12, 2008 Astrat This paper presents a mahine learning system for notes, apale of learning some aspets of tunes. Input is in the form of notes played on a penny whistle, output is a predition of the next note. In testing the system performed far aove hane, ut there are some important forms of generalization that it annot perform. 1 Introdution In this paper we present a learning system for notes, along with software and hardware for using it on live musi. The overall projet goal, of whih this is only a portion, is to reate a root that an learn to play penny whistle. 1 This root would, starting with ompletely unanalyzed sensors and effetors in the manner of SODA [2], learn in an unsupervised manner to play oth along with other people and alone. An overview of the system as implemented so far is shown in Figure 1. Waveform Note Preditions Mi Sequene Learner Note Categorizations Fourier Transform Power Spetrum Growing Neural Gas (UDP) Computer 2 Computer 1 Figure 1: System Overview 1 A penny whistle is a six-hole fipple flute, apale of playing a D-major sale (with the addition of C natural) over two otaves. We hose it for this projet eause it produes lear notes that respond well to omputer analysis and eause it has simple fingering. 1

2 For the lass midterm projet we implemented the first two stages of this proess. The first stage is a low level stage with no learning omponent. First a omputer samples the analog output of a mirophone at Hz using 16 it samples. Then, every 1102 samples or 40 times a seond, it omputes a power spetrum with a Fourier transform on the sampled waveform. The spetrum onsists of 501 frequeny ukets, eah representing the strength of a 40 Hz wide slie of the sound spetrum. The seond stage is an implementation of a Growing Neural Gas (GNG) learning system as in [1] that reates ategories orresponding to notes. It reeives a power spetrum over UDP 40 times a seond, updating and adding internal prototype nodes. Over time it learns to distinguish ommon frequeny ominations (notes), whih an then e given as input to the next stage. While a iologially realisti system would likely have the GNG ontinuing to form and update ategories, this is experimentally inonvenient, as it an make later experiments less onsistent etween runs. Instead, we trained the GNG on twenty minutes of human penny whistle playing and then froze it. After initial training it funtioned only as a ategorization system, produing symols 40 times a seond indiating how it was ategorizing the urrent sound. The third stage and fous of this paper is fundamentally a predition system. The goal is to predit what note will e deteted next given the sequenes of notes that have een heard in the past and the reent history. A iologial approah suggests short-term and long-term memory omponents. Short term memory is implemented as a simple list of the previous notes heard, going ak fifty notes. Long term memory is implemented as a kind of trie. An example predition trie is shown in Figure 2. The trie egins with only a root node, indiating an empty memory. Initial preditions are null and so always inorret. Upon experiening a sequene, if the final note (t 1 or p) is predited orretly y the trie given its history (t 0 through t n ) 2 The final note of a sequene is then the trie is left unhanged. Otherwise, a new node is added at t (n+1) as a hild of the t n node the searh ended at, prediting p. 2 Note that here we are indexing in the opposite manner ustomary. If the sequene d a is deteted, for example, then t 1/p is, t 0 is and so on. 2

3 t 0 t -1 t -2 p a a Given this d d a a Predit this Figure 2: An example predition trie 2 Experiment In order to evaluate how well the system an learn sequenes, we tested it on six tunes. Eah tune was played six to ten times on a penny whistle. After hearing every transition etween notes, the learning agent predited what the next transition would e. We an evaluate auray y looking at the fration of transitions predited orretly. The six tunes do not give the appearane of uniform omplexity. Tale 1 shows the tunes ordered y the average note duration. Tune Name Tune Length Numer of Notes Average Note Length Row Row Row Your Boat (Em) 54s s Row Row Row Your Boat (Em) (omplex) 44s s Jamie Allen 105s s Canadian Barn Dane 103s s Evit Gariel (A part) 32s s Swallowtail Jig 67s s Tale 1: Complexity of Tunes 3 Results The learning system was ale to learn all of the tunes to some extent. Tale 2 shows how the system did on eah of the tunes overall starting from a lank slate, while Figure 3 shows a graph 3

4 of auray over time for one tune. Tune Name Perent Inorret Row Row Row Your Boat (Em) 44.1 Row Row Row Your Boat (Em) (omplex) 58.3 Jamie Allen 42.3 Canadian Barn Dane 57.4 Evit Gariel (A part) 61.3 Swallowtail Jig 57.7 Tale 2: Perent of Note Transitions Predited Inorretly Figure 3: Perent of transitions in Row row row your oat (Em) inorretly predited, 30 transition moving average In looking at Tale 2 it is important to note that on the first few transitions error is going to e 100% eause the system starts with a lank slate. A perent inorret sore of zero then would not make sense, as it would indiate a psyhi learning mehanism. There is ertainly room for improvement in this system, ut these rates are not as ad as they might appear. One urious feature of Figure 3, and in fat of the predition patterns for all the tunes, is that while predition error goes down heavily at first it often reverses its path later. We are not ompletely sure why this happens, ut suspet it is due to our system not taking into aount errors in the input stream. A person playing a penny whistle may oasionally play a wrong note, and the predition trie is not very roust in the fae of error. Lookups in the trie do use preferentially more reent samples and so errors a few notes ak may not ause too many errors. The transition 4

5 immediately after an inorret note, however, will nearly always e mispredited. Use of a different learning mehanism that opes with a noisy input stream ould help with this and would e an interesting diretion for future work. Other diretions of future work inlude making the learning system aware of timing and duration, modifying the low level spetrum produing algorithm to use wavelets for etter auray, finishing onstrution of the hardware for rooti penny whistle playing, and implementation of an IAC-style learning mehanism for supervision and ontrol of the hardware. 4 Conlusion With this projet we have shown that one an make a system apale of learning to predit note ategories starting with ompletely unknown sensors. This system would e an essential omponent of a full rooti penny whistle learning system. Referenes [1] Bernd Fritzke. A growing neural gas network learns topologies. In G. Tesauro, D. S. Touretzky, and T. K. Leen, editors, Advanes in Neural Information Proessing Systems 7, pages MIT Press, Camridge MA, [2] Jefferson Provost, Benjamin J. Kuipers, and Risto Miikkulainen. Developing navigation ehavior through self-organizing distintive state astration. Connetion Siene, 18(2),