PROCEEDINGS OF THE INTERNATIONAL COMPUTER MUSIC CONFERENCE 2012 IRZU _INSTITUTE FOR SONIC ARTS RESEARCH

Transcription

1 ROCEEDINGS OF THE INTERNATIONAL COMUTER MUSIC CONFERENCE 2012 IRZU _INSTITUTE FOR SONIC ARTS RESEARCH

2 THE XYOLIN, A 10-OCTAVE CONTINUOUS-ITCH XYLOHONE, AND OTHER EXISTEMOLOGICAL INSTRUMENTS Steve Mann an Ryan Janzen University of Toronto, Faculties of Engineering, Arts&Sci., an Forestry ABSTRACT A class of truly acoustic, yet computational musical instruments is presente. The instruments are base on physiphones (instruments where the initial soun-prouction is physical rather than virtual), which have been outfitte with computation an tactuation, such that the final soun elivery is also physical. In one example, a single plank of woo is turne into a continuous-pitch xylophone in which the initial soun prouction originates xylophonically (i.e. as vibrations in woo), as input to a computational user-interface. But rather than using a louspeaker to reprouce the computerprocesse soun, the final soun elivery is also xylophonic (i.e. the same woo itself is set into mechanical vibration, riven by the computer output). This xylophone, which we call the Xyolin, prouces continuously variable pitch like a violin. It also covers more than 10 octaves, an inclues the entire range of human hearing, over its 122 centimeter length, logarithmically (1 semitone per centimeter). Other examples inclue pagophones in which initial soun generation occurs in ice, an final soun output also occurs in the ice. More generally, we propose an existemological (existential epistemology, i.e. learn-by-being ) framework where any foun material or object can be turne into a highly expressive musical instrument in which soun both originates an is output iiophonically in the same material or object, which may inclue some or all of the player s own boy as part of the instrument. 1. NON-COCHLEAR SOUND The theme of this year s ICMC conference is Non-Cochlear soun. The notion of non-cochlear soun is suggestive of two things: 1. soun that is perceive by other than the cochlea, e.g. tactile soun (soun that can be felt through the whole boy rather than only hear); an 2. a metaphor likene to Marcel Duchamp s non-retinal visual art, broaening our perception of what is meant by art, through Reaymaes (orinary foun objects as art, for example). Likewise Non-Cochlear Sonic Art can be thought of as broaening our unerstaning of sonic art in the Seth Kim-Cohen sense of Non-Cochlear [Kim-Cohen, 2009]. This paper presents a methoology an philosophy of instrument-builing that embraces non-cochlear in both these senses, i.e. instruments that are tactile (an can thus, for example, be playe an enjoye without the ear they can even be enjoye by the eaf), an instruments that are Reaymaes in the Duchamp/Seth Kim-Cohen sense (with the existential self-eterimination of the DIY maker culture). 2. BACKGROUND AND RIOR WORK The work presente in this paper can be thought of as an extension of the concept of physiphones [Mann, 2007] (using the natural acoustic soun prouction in physical material an objects for computer input evices), which itself may be regare as an extension of hyperinstruments [Machover, 1991] Computer music an user-interfaces Traitional computer-music is generate by using various kins of Human-Computer Interfaces (input evices), connecte to a computer system, which synthesizes the soun we hear through a louspeaker system. See Fig. 1, in relation to Fig , to be escribe in what follows. Some of the input evices use for computer music are very creative. For example, Hiroshi Ishii of the MIT (Massachusetts Institute of Technology) Meia Lab has worke extensively to evelop TUIs (Tangible User Interfaces) [Ishii an Ullmer, 1997]. TUIs have been extensively use as user-interfaces [Vertegaal an Ungvary, 2001] [Alonso an Keyson, 2005]. Many of these user-interfaces are extensive an creative, an use real-worl objects as input evices. For example, Luc Geurts an Vero Vanen Abeele have use a bowl of water with electrical contacts in the water as a computer input evice so that splashing the water triggers the playback of a pre-recore soun sample [Geurts an Abeele, 2012]. Others have create systems that allow anyone to easily turn any objects such as fruit, plants, human skin, water, paintbrushes, or other objects into musical instruments [Silver et al., 2012]. Thus the piano keyboar symbol of Fig. 1 is meant to stan for any of the wie variety of Human-Computer input evice in common usage, which can inclue real worl physical objects, such as a bowl of water, as input evices Machover s Hyperinstruments In 1986, To Machover, from the MIT Meia Lab evelope the concept of hyperinstruments, in which real physical objects such as a violin, cello, or piano, are fitte with sensors as input evices to a computer which _451

3 USER INUT DEVICE COMUTER SEAKER Figure 1: A common computer music methoology: A user interacts with a userinterface that is connecte to a computer, which generates soun through amplification an a speaker system. USER OUTUT (ACOUSTIC) OUTUT (SUNTHETIC) USER MIC. OR ICKU OTIONAL ADDITIONAL SENSORS COMUTER SEAKER Figure 3: Mann s physiphones (hyperacoustic instruments): A user interacts with a real physical object such as block of woo, ice, earth, water fountain, or the like [Mann, 2007][Mann et al., 2007]. The sensor is a microphone or other listening evice (soun pickup). Rather than synthesize soun, the computer moifies the souns actually generate by the real physical object, such as by pitch-correction (pitch transposing) to notes on a musical scale. The natural physiphonically generate souns [Mann, 2007] are hear by an amplifier an speaker system, after being moifie by the computer. COMUTER INUT DEVICE Figure 2: Machover s hyperinstrument: A user interacts with a real physical object such as a violin or cello. The real physical object inclues various sensors that also function as input evices to a computer, which synthesizes computer music that is reprouce by a speaker system, an can be hear simultaneously with (i.e. in aition to) the real physical object s own acoustic soun prouction. synthesizes soun to accompany the real physical instrument [Machover, 1991]. One example is the hyperpiano, in which MIDI ata generate by performer on a Yamaha Disklavier is manipulate by various Max/MS processes as accompaniment an augmentation of keyboar performance ( Machover) [Machover, 1991] Mann s Hyperacoustic insruments Throughout the 1980s an 1990s Steve Mann create a variety of input evices that use the real worl itself as the user-interface, for which he coine the terms Reality User Interface an Natural User Interface (NUI) [Mann, 2001] (before Microsoft Corporation began using this term in a narrower sense to enote tabletop interfaces.). In this paper, we use the term Natural User Interface in its original sense to enote interfaces that both (1) use natural human capabilities (i.e. capabilities which come naturally to us), an (2) use nature itself as a user-interface (i.e. real-worl physical objects, an natural philosophy, i.e. physics). Some emboiments of these interfaces use the acoustic isturbances in real physical objects as computer input [Mann, 2001] [Mann, 2007]. See Fig 3. Some of these Natural User Interfaces inclue turning public fountains such as Dunas Square (Canaa s cultural an civic centre, akin to Times Square in the Unite States), various municipal ice rinks, an Lake Simcoe (Ontario, Canaa) itself, into giant musical instruments. These were not merely input evices to control soun synthsizers, but, rather, instruments in which mechanical vibrations in the water, ice, earth, concrete, or the like, were capture with listening evices, an moifie by computer in such a way as to make an expressive musical instrument. Such instruments are calle hyperacoustic instruments [Mann et al., 2007]. It is easy to make an instrument that makes new an unfamiliar souns. But just as a painter like icasso ha to first prove himself with realism, before creating something new, the hyperacoustic instruments were first use MIC. OR ICKU COMUTER USER OTIONAL ADDITIONAL SENSORS Figure 4: ropose system: acoustic physiphones. A hyperacoustic system is both the original source of the computer-moifie soun, as well as the elivery mechanism of that moifie soun. There is no speaker. Instea the physical object itself vibrates, both to generate the original soun, as well as to eliver the processe soun to the auience an player(s). to play various classical music an jazz stanars, in orer to prove to the worl that they were real instruments, an then were subsequenty use to play new music compose for them. One example performance was Mann s Aaggio for Fingernails an Chalkboar (performe 2010 May 8th) in which actual acoustic soun, capture by contact microphones on each of his fingernails was pitch-transpose an pitch-correcte to musical notes, first to play some familiar classical an jazz repertoire, an then to play the new Aaggio. Another example of a hyperacoustic instrument is the use of one or more wooen blocks or any other foun scraps of woo as a xylophone in which the natural soun of the woo is pitch-transpose to a musical pitch. In this way, any foun object can be turne into a non-electrophic musical instrument, i.e. an iiophone, in which the soun is generate acoustically, then moifie by computer. It shoul be note that such instruments are not merely input evices to computerize soun generators, as shown in Fig 1, but, rather, use the original soun itself, an are thus much more expressive an natural. For example, an orinary esk can be turne into a xylophone in which tapping on the esk can make souns like a bell, whereas rubbing on it can make more sustaine notes like that of a violin or cello. This sonic expressivity is ue to the fact that the original soun, not a synthsize soun, is use. Various foun objects, such as a bath tub that were foun in a umpster, were turne into expressive musical instruments that coul play any classical or jazz repertoire, intricate Bach fugues, etc., as well as being able to play newly compose music written specifically for the new instruments. A hyperacoustic instrument built into a SpaBerry hot tub was use as the main instrument for the main act in North America s largest winter festival, to perform for Canaa s rime Minister an Governor General, in front _452

4 of an auience of more than 10,000 people. The resulting instrument was a variation of the hyraulophone known as the balnaphone. Aitionally, hyperacoustic instruments facilitate truly natural user interfaces such as, for example, turning a living tree into a xylophone in which the soun originates xylophincally. layers strike the tree branches with mallets, an the actual souns from the tree are picke up by listening evices attache to the tree. The natural souns prouce by tapping, scratching, or rubbing the tree are pitchtranspose to musical notes. The target pitch of the pitch transposition is epenent on where the tree is struck. This is etermine by using an array of listening evices with soun localization (time-of-flight), an/or a vision system (camera(s) an computer input frame grabber) that also watches to see where the tree is struck. With regars to Fig. 3, the camera(s), if present, is/are the optional aitional sensor(s). Iniviual parts of the tree can then be labele with chalk, e.g. A, B-flat, B, C., C-sharp, etc Orchestrions, player-pianos, an other actuate instruments Our work iffers from computer-controlle musical instruments like player-pianos, solenoi-activate xylophones, an other computer actuate musical instruments [Overholt et al., 2011] in the sense that we are not trying to get the computer to play the instrument. In fact, quite the opposite: we re trying to get the computer to help us get closer to nature! 3. ROOSED INSTRUMENTS In this paper, we propose acoustic physiphones which are natural user-interfaces in which: the initial soun prouction (soun generation) is natural, i.e. acoustic, as with physiphones; the final soun elivery (soun reprouction) is by way of the natural material. Thus if the soun originate xylophonically (from vibrations in woo), the processe soun is also reprouce xylophonically (i.e. by way of vibrations in woo). referably the same woo that is use to generate the original soun is use to eliver the processe (e.g. pitchtranspose) soun. See Fig 4. The software use for the work one in this paper was written in the C programming language, on specialize embee computers that we esigne an built to be completely waterproof an environmentally seale, so as to operate in a natural environment. We use GNU Linux an wrote our own evice rivers to exten the operating system to aapt to the new harware we built The Xyolin We now present an example of an acoustic physiphone, which we call the Xyolin, name an invente by author S. Mann. It is a xylophone, but it has infinitely con- MALLET WITH TRANSDUCER INSIDE IT XYLOHONE LANK, WHICH IS ALSO ITS OWN SOUNDBOARD COMUTER TRANSMIT/RECV DULEXER HIGH VOLTAGE AMLIFIER Figure 5: System architecture of the Xyolin (single-plank xylophone). Four small transucers, one in each corner of the plank, capture acoustic vibrations in the plank an convey these to the computer. A high voltage amplifier was aapte from an ol vacuum tube amplifier foun in a umpster. The computer thus rives one large transucer locate in the mile of the plank. All of the transucers are capable of being transmitters or recievers, but the soun hear by the auience is primarily ue to vibrations inuce in the boar by the large central transucer. tinous pitch like a violin. It can be playe by striking, or by rubbing or bowing (thus giving it the capability to be playe either percussively or with infinite sustain for notes of whatever uration are esire). Various single-plank xylophones were built from high quality Sitka Spruce sounboars. But one of these instruments was mae from a piece of rough plywoo foun in a garbage umpster. It was fitte with four transucers, one in each corner, which coul sense an effect vibrations in the woo. Originally these were use as both listening evices an excitatory evices, but later a much larger transucer was put in the center of the boar. See Fig 5. In aition to position tracking by listening (time-of-arrival ifferences in the various receive transucers, etc.), various other position sensing technologies were use in this work. These inclue a GHz home-mae raar set aapte for close range, an ultrasonic range sensor, an an overhea camera to improve the position-sensing (especially while rubbing, where the onset of soun was less iscernible), an to recognize various mallets, sticks, gestures, etc.. Aitionally, fine granules of brightly colore san were often place on the boar, so as to form cymatics, visible to the overhea camera. In this way the camera can see the noal patterns in the vibrating woo, an this information can be use as part of the feeback loop in riving the transmit transucer(s) to affect the vibrations in the woo. Other variations use ripple tanks as, or on, the vibrating meium of the instrument. The boar becomes both the input evice as well as the sounboar for the instrument, elivering a variety of public performances without the nee to use a A (public aress) system. See Fig 6. When hitting the boar with one or more mallets or sticks, the surface texture ha little effect on the soun prouction or soun elivery. But when rubbing the surface with a mallet or stick, the surface texture of the boar was foun to be very important. It was foun that rough plywoo, covere in violin rosin, worke best for generating long sustaine violinlike notes, through rubbing with a stick also coate in violin rosin. _453

5 g T g R Threshol 1:1 Output Level (B) 2:1 4:1 (a) (b) Input Level (B) Figure 7: (a) Uncontrolle feeback through an acoustic physical material (having transfer function ), using amplifie transmit an receive transucers with gains g T an g R, respectively. (b) Input/output relationship of a simple ynamic range compressor, with various compresion ratios. [secon image in the public omain, via Wikimeia Commons] :1 Figure 6: Xyolin uring an evening performance. A single wooen plank is fitte with position sensors that sense the position of one or more mallets or sticks. The result is a simple uncluttere artistic performance instrument. The plank an some of the mallets or sticks are fitte with listening evices that capture the actual soun of the woo being struck or rubbe with the mallets or sticks. The acoustic soun from hitting or rubbing the woo is passe through one or more position-epenent banpass filters, implemente on a computer system. The final output from the computer is amplifie an fe back to the very plank that first generate the soun. The xylophone picture in Fig 6 covers just over 10 octaves, with a resolution of exactly one centimeter per semitone (i.e. 12 centimeters per octave). The centimeters are marke with lines, as is every octave (in boler lines) but the user can hit the plank between markings to get quarter tones or any other microtonal intervals. The frequency range of the instrument is from E-flat 0 (19.45 Hz) to E10 (21, Hz). Thus it spans the entire range of human hearing from less than 20Hz to greater than 20kHz, over its 122 cm (122 semitone) length. osition is etermine by an array of listening evices on the unersie of the plank (using initial time-of-flight estimation in the woo, correcte for the ifferences in the spee of soun going along the grain versus going crossgrain, etc.). Aitionally, a sie-looking K-ban complex (in-phase an quarature) raar set an an overhea camera run a machine vision algorithm with backgroun subtraction [Yao an Oobez, 2007]. This provies improve tracking accuracy an istinguishes between various mallets an sticks which each have a uniquely colore ban attache near the tip, or a Luneberg raar lens (or both). The stick in the player s right han (the stick picture to the auience s left) in Fig 6 is equippe with its own pickup. This pickup fees back at a high enough gain to provie infinite sustain if it is kept touching the woo. In this way it will cause the woo to vibrate at any frequency from 20 Hz to 20kHz epening on its position. The other stick (the one without the pickup) simply excites the pickups in the wooen plank Acoustic feeback, with ynamic range compression, an position-epenent banpass filter A ynamic range compressor is a evice that makes quiet souns louer an lou souns quieter, thereby compressing an auio signal s ynamic range. Compressors are often use, for example, to process the output of vocal microphones to reuce the ynamic range of a human voice. Orinarily in auio applications, acoustic feeback is highly unesirable, an ynamic range compression can be precarious in a live theatre because it can lea to feeback. However, eliberate use of feeback is often use (e.g. when a guitarist stans next to a speaker to get long sustaine violinesque tones). INITIATING VIBRATIONS x TRANSMIT TRANS DUCER g T D TIME DELAY HYSICAL ACOUSTIC MATERIAL R REFLECTIONS, RESONANCE g GAIN OSITION DEENDENT FILTER g R RECEIVE TRANS DUCER F C ENV LF COMRESSOR C * GAIN Figure 8: Acoustic feeback assiste by ynamic range compression. We also use a position epenent banpass filter F to tune the resonance accoring to the position of the player s han or mallet, as etecte by raar set an computer vision. A simple feeback system is shown in Fig. 7(a), with g T representing an amplifie transmit transucer (turns an electrical signal into acoustic vibrations), representing the physical material through which the soun is fe back, an g R representing a receive transucer with amplifier (turns acoustic vibrations into an electrical signal). In control theory is often use to represent a plant (e.g. a joint in a robot), an here literally is a plant when we are using a tree branch. The system in Fig 7(a) is typically unstable an ifficult to operate. That is, if we turn up the gains g T an g R high enough such that a vibration occurs, the vibration can suenly grow out of control, in the positive feeback loop, an the transucers have to be lifte off the acoustic material before amage occurs! Compressors typically act on a signal in the manner shown in Fig 7(b), acting on the amplitue of a signal (etermine over several perios of the waveform) rather than being applie at each point in time through the waveform (which woul a harmonics ue to a nonlinear effect on the shape of the waveform itself). Therefore the natural soun of the acoustic process is preserve, an feeback is controlle an maintaine. In this paper we present controlle feeback in iiophonic meia, using aaptive computational processing (e.g. compression, filtering, etc.). to control an sustain feeback with a pitch, timbre, an amplitue that can be accurately an reliably controlle by the player. See Fig 8. Even though the compressor is nonlinear, we can take a small segment of time over which the compressor s gain C is static, to first orer (it graually varies over the course of many waveform cycles), thus creating a linear feeback system. Over the course of a single waveform, then, the input-output transfer function simply becomes: g T g R FC +1. This mathematically escribes the acoustic response to a mallet strike or any vibration create by the player, represente by input x in Fig 8. The compressor ajusts C to ensure the feeback is sustaine. _454

6 Figure 9: Derivation of a new shape for the 10-octave Xyolin Reshaping the Xyolin The emboiment of the one-plank xylophone picture in Fig 6 works quite well, but we wishe to improve both its soun, an its aesthetic form. There is something nice about the aesthetics of a stanar xylophone, as the higher notes have shorter bars. We wish to mimick this exponential shape, both for appearance an for improve soun. Conceptually, imagine we make a xylophone that has 12 wooen bars per octave. A two-octave xylophone will have 25 bars (12*2 + 1 to complete the octave), as shown in Fig 9(leftmost). Notice that the rightmost bar is half the length of the leftmost bar, since the funamental frequency of vibration varies inversely with the square of the length, i.e. half the length results in four times the frequency [Lapp, 2010]. Thus length f (length is inversely proportional to the square root of the frequency). No suppose we make a microtonal xylophone, with quartertones, thus having 51 bars for the same two octaves (the rightmost bar still being half the length of the leftmost bar. In the limit, as the pitch increment approaches zero, an the number of bars approaches infinity, we obtain the arrangement shown in Fig 9(center). In Fig 9(center) we have just one piece of soli woo. The rightmost sie is half the height of the leftmost sie. Now if we actually ha a xylophone that ran 10 octaves from 20Hz to 20480Hz ( Hz), the lowest (longest) bar woul be 32 times longer than the shortest bar. This frequency range is really amazing when we think about it, an it is ue to the fact that length f, i.e. the ratio of longest to shortest bar is much smaller than the ratio of highest to lowest frequency. Therefore, we generate a continuous exponential shape that runs over the entire 10 octave range, as shown in Fig 9(rightmost). The left sie of this shape is 32 times taller than the right sie A single-plank exponentially shape xylophone Cutting out the plank in this shape, gives our instrument a nice new shape, although the number of receive transucers was reuce from 4 own to 3 (an the transmit transucer was move to a new location closer to the fatter en of the plank). The new artistic aesthetic serves a practical purpose. For example, it is now obvious which en is the en for low notes an which en is the en for high notes. The extreme ifferences between the two ens also helps to make apparent the extreme range of pitches that the instrument is capable of proucing. See Fig. 10. However, the shape goes beyon mere aesthetics. Now the lower moes of vibration in the woo ten to occur Figure 10: The Xyolin, a single plank xylophone with an exponential taper. This shape has both aesthetic value (e.g. it is obvious which en of the plank is for low notes, which en is for high notes, an the extremes in size clearly inicate its broa compass), as well as functional value. Rightmost: we see the view from the overhea camera use for computer-vision (tracking positions of the mallets an sticks, etc.). Figure 11: Acoustic physiphone mae from a fallen tree branch foun in a forest. The transmit transucer is shown towar the left, hanging ownwars. The receive transucers were acoustically couple to various smaller branches with pipe clamps. more strongly at the larger en, an the higher moes of vibration ten to occur more strongly at the smaller en. Thus we hear low notes emanate mainly from the large en, high notes mainly from the small en, while miones emanate mainly from the mile of the plank. Moreover, when using a stick or mallet with a pickup in it, the infinite sustain actually works better with this new tapere shape. For example, the very narrow en can vibrate easily at very high frequencies, up to an beyon the range of human hearing. The large en works better at low frequencies, especially as it can move more of the surrouning air in the room, in orer to better reprouce low pitches. We also preferre the timbral changes to the soun arising from the tapere shape, especially the improve clarity of long sustaine high notes Natural User Interfaces A walk in the forest with a rubber mallet will often reveal fallen tree branches that are very sonorous. Accoringly, a fallen branch of Sitka Spruce was foun, which soune quite nicely on its own. This piece of fallen tree was mae into an acoustic physiphone, by fitting it with a transmit transucer an a number of receive transucers. See Fig 11. The result is a highly expressive an sonorous instrument that can be use to play highly intricate recognizable songs an classical or jazz reperetoire (incluing intricate Bach fugues, etc.) as well as new experimental music, owing to the microtonal character an high egree of timbral variability. _455

7 Figure 12: An ensemble of xylophones was constructe from real living trees in a forest. Here we see a transmit transucer hanging from a branch at the left, a receive transucer acoustically couple to the branch near the right, an an overhea camera assisting with the ientification an position tracking of a variety of sticks an mallets. Aitionally a ata projector is incorporate into the camera for use in late-night concerts, as well as turning the branch into an interactive touch screen of sorts. Figure 14: ublic performance with Reaymae instruments: acoustic physiphones mae from items supplie by auience members. A transmit transucer excites the object to regenerate its own acoustic vibrations as picke up by a receive transucer. An overhea camera tracks an active or passive stick or mallet. In this figure, the stick is a magic wan containing an active illumination source tracke by the camera, as well as an auio pickup to sense vibrations in the supplie objects. An overhea camera an projection system mounte to a microphone boom can be place over any supplie objects to turn them into interactive touch surfaces that augment these acoustic physiphones. Leftmost: a rubber boot; Rightmost: a smartphone (we also wrote a smartphone app that turns anything into a musical instrument). Figure 13: ublic pagophone performance mae using ice that was mae more sonorous by computer processing. Transucers embee in the ice cause it to vibrate at musical pitches. The two large slabs of ice each prouce 12 perfectly tune musical notes that remain in perfect tune even as the ice melts. The smaller slabs each prouce a single note. Figure 15: Reaymae bath instrument showing innars. Finally, a forest concert was prepare, in which numerous trees were turne into xylophonic ensemble of musical instruments. Special mounting brackets were evelope to attach transmit an recieve transucers to tree branches, to softly grasp the gree branches without amaging them. See Fig OTHER ACOUSTIC HYSIHONES The same principles that apply to our Xyolin, in all its Reaymae emboiments, from office esks, to wooen planks, to branches, to forests, etc., can also be applie to other materials. This work was the opening keynote for ACM (Association of Computing Machinery) TEI conference, by way of a performance using ice as an interactive musical meium. In this performance, we use four transmit transucers, an 12 receive transucers, arrange on an in blocks of ice. Some of the transucers were frozen right into the ice blocks, an others were couple acoustically to the ice. See Fig. 13. We also invite auience members to bring forwar any object that they wishe to turn into a musical instrument. We took requests, e.g. Can you play achelbel s Canon on this rubber boot? or Can you play Gershwin s Summertime on this soft-cover book?, which we i. We then performe some original music on the ice, an on the objects selecte or supplie by the auience members. See Fig READYMADE FOUNTAINS The propose metho of creating acoustic physiphones from nearly any foun objects is not limite to iiophonic soun creation. As an example of another form of soun creation, a musical instrument was mae from a bath tub foun in a umpster. After cleaning out the tub it was fitte with various hyrophones (12 receive hyrophones an two transmit hyrophones), an some waterproof computer equipment. Four wheels were installe, uner the tub, one in each corner, to create a kin of bathmobile. A propane heater was fitte to the tub, so that it coul be rolle aroun while being playe. A circulatory system was create from electric pumps running from a car battery an power inverter installe in the unersie of the tub, together with the various computational an sensory equipment. The resulting reaymae bathmobile is an instrument in which soun: originates as vibrations in water, by playing any of the 12 water jets installe on the tub; is elivere to the auience by vibrations in the same water. See Figs. 15 an 16. Soun prouction an soun elivery are thus hyraulophonic, with computational capabilities an a wie range of acoustic timbres an capabilities. Moreover, the soun is truly tactile, in the sense that participants can feel the soun in their fingertips, an also _456

8 Figure 16: Reaymae bath instrument uring a rolling street performance. Figure 17: The Reaymae bath instrument is inherently tactile an visual. As well as hearing, we can also feel an see the vibrations in the water which prouce the soun. As a result, hearing impaire musicians can also enjoy the instrument. see the soun vibrations in the water. See Fig. 17. As a result, hearing impaire musicians can also enjoy the instrument. For example, hearing impaire percussionist Evelyn Glennie playe on the instrument, an was able to play an feel meloies an harmonies on it. Thus, like the iiophones presente in this paper, the bath instrument is non-cochlear in both senses of the wor: it can be experience without the cochlea, an it also truly references the work of Marcel Duchamp, in many ways! 6. SCIENCE OUTREACH STEM is an acronym for Science, Technology, Engineering, an Mathematics, an an agena of public eucation is integrating these isciplines. Other interisciplinary efforts like MIT s Meia Laboratory focus on Art + Science + Technology. Design is also an important iscipline, so we might consier DAST = Design + Art + Science + Technology. DAST coul put a heart an soul into STEM, e.g. going beyon multiisciplinary to something we call multipassionary or interpassionary or transpassionary, i.e. passion is a better master than iscipline (Albert Einstein sai that love is a better master than uty ). Consier, for example, DASTEM = Design + Art + Science + Technology + Engineering + Mathematics ( astemology ), or perhaps DASI = Design + Art + Science + In(ter)vention or Innovation. erhaps what we want to nurture is the inventopher (inventor+philosopher), through existemology (existential epistemology), i.e. learn-by-being. This goes beyon the learn by oing (the constructionist eucation of Minsky an appert at MIT). A simple example of putting existemology into practice is when we teach our chilren how to measure something, using anthropomorphic units (measurements base on the human boy) (wikipeia.org/wiki/anthropic units) like inches (wih of the thumb) or feet. The human boy itself becomes the ruler. We learn about rulers an measurement by becoming the measurement instrument. Consier a four-year-ol learning about water pressure: Day: This gauge is in kilopascals. Christina (age 4): Why kill a pascal? Day: Kilo means 1000, so its 1000 pascals. Christina: What s pascal? Day: A French physicist, also one newtwon per square meter. Christina: What s newton? Day: Another physicist... The same chil ha no problem unerstaning water pressure in pouns per square inch or Christinas (her own boy weight) per square Stephanie (her sister s area). The very inaccuracy of anthropomorphic units, especially when use across various age groups, is why the concept is so powerful as a teaching tool: it is OK to make mistakes, to take guesses, an to get a rough imprecise unerstaning of the worl aroun us. Another example of existemology is wearable computing: we learn about computers by becoming the technology in the cyborg sense, Learning by Being: Thirty Years of Cyborg Existemology, INTERNATIONAL HAND- BOOK OF VIRTUAL LEARNING ENVIRONMENTS, 2006, art IV, Much like the Suzuki metho for teaching music, the Mann metho (author S. Mann) of teaching is base on existemology. The human boy itself becomes a musical instrument that teaches physics, states-of-matter, mathematics, an the like. An example aroun this iea is ipe Dreams, a series of performances an emonstrations in 2011, in which author S. Mann playe instruments while sleeping. A skull cap with 64 brainwave electroes was connecte to a computer that playe four instruments, one in each state-ofmatter: chimes mae of pipes (soli matter); a hyraulophone (liqui matter); a pipe organ (gaseous matter); an a plasmaphone (soun from the fourth state-of-matter). When the soli, liqui, an gas pipes are arraye together aroun the sleeping subject, they form an interesting sculptural form as well. The tubular glockenspiel has pipes that vary in length inversely as the square root of the frequency, whereas the pipe organ has pipes that vary inversely with linear frequency, an the hyraulophone pipes vary inversely with the square of the frequency: Xylophone or glockenspiel ipe organ Hyraulophone length f length f length f 2 Moreover, the chimes (glockenspiel) are velocity-sensing, whereas the pipe organ is isplacement sensing, an the hyraulophone is absement sensing. Absement is the timeintegral of isplacement. More generally, hyraulophones give rise to a new kinematics ( scitamenik ) that inclues negative erivates-of-isplacement, in the sequence: {..., absounce, abserk, abseleration, absity, absement, isplacement, velocity, acceleration, jerk, jounce,...}. See Fig 18. These simple an funamental aspects like state-of-matter an kinematics allow us to see the worl in new ways, beyon music. For example, others have recognize the iactic value of this new kinematics philosophy: _457

9 Kinematics an Musical Instruments Two-stage Hyraulophone... Abseleration Absity Absement Displacement (Distance) Velocity (Spee) Acceleration... Hyraulophone is Absement-sensitive a f(x) Integration + y x b Organ is Displacement-sensitive iano is Velocity-sensitive Differentiation Figure 18: Hyraulophones reveal an exhibit a completely new way of unerstaning an thinking about kinematics: negative erivatives of isplacement! Although time-integrate charge is a somewhat unusual quantity in circuit theory, it may be consiere as the electrical analogue of a mechanical quantity calle absement. Base on this analogy, simple mechanical evices are presente that can serve as iactic examples to explain memristive, meminuctive, an memcapacitive behavior.[jeltsema, 2012] 6.1. Water, Forestry, an First Nations instruments The water instruments allow a natural element - water - to itself become a musical instrument. We are working to combine water an forestry in a series of musical performances in various forests. One such performance contextualizes the forest canopy as a catheral of sorts, where native flutes are playe high in the forest canopy, along a canopy walkway. Aitionally, various water instruments are playe on an in natural boies of water in the forest. In one of the compositions there are three elements: Earth: Native Drums, forest, an tree instruments, incluing the Xyolin. These instruments are playe on the groun; Water: Hyraulophones, which are playe on an in natural boies of water in the forest. Some of these instruments are actually playe unerwater; Air: Native Flutes playe high in a forest canopy walkway. Thus we have Earth on the groun, Water on an in the water, an Air up in the air. The use of the five Elements (Earth, Water, Air, Fire, Iea) is part of our work at the nexus of art, science, technology (engineering), an esign to support DAST (Design, Art, Science, an Technology) outreach. Lateral thinking within this new states-of-matter musical instrument ontology (physical organology) can lea to the invention an rapi prototyping of many new musical instruments in a DIY reaymae context well-suite to existemological outreach. 7. CONCLUSION We have create several instances of a new kin of computerbase musical instrument in which the soun (a) originates acoustically, an (b) is conveye to the auience acoustically, i.e. by acoustic vibrations in the physical boy of the instrument. Examples inclue the Xyolin, a xylophone that has infinitely many notes an covers the entire auio range of human hearing, where soun originates as vibrations in woo, an is conveye to the auience by vibrations in woo, as well as the pagophone, in which soun originates in vibrations in ice, an is conveye to the auience by way of vibrations in ice. The instruments can play any jazz or classical repertoire, intricate Bach fugues, etc., but they can also play a wie range of original works not possible on any other instrument. Moreover, these new instruments give rise to a new way of thinking about an learning about science, such as states-of-matter, an a new perspective on kinematics that inclues negative erivatives of isplacement. 8. ACKNOWLEDGEMENTS The authors wish to thank Anrew Kmiecik, Jason Huang, Valmiki Rampersa, Raymon Lo, Queen s University, NSERC, an AMD. References [Alonso an Keyson, 2005] Alonso, M. B. an Keyson, D. V. (2005). MusicCube: making igital music tangible. ACM CHI. [Geurts an Abeele, 2012] Geurts, L. an Abeele, V. V. (2012). Splash controllers: Game controllers involving the uncareful manipulation of water. In roceeings of the ACM Tangible Embee an Emboie Interaction, pages , Kingston, Ontario, Canaa. [Ishii an Ullmer, 1997] Ishii, H. an Ullmer, B. (1997). Tangible bits: Towars seamless interfaces between people, bits an atoms. roceeings of the ACM CHI 97 Human Factors in Computing Systems Conference, pages March 22 27, 1997, Atlanta, Georgia, pp [Jeltsema, 2012] Jeltsema, D. (February 15-17, 2012). Memory elements: A paraigm shift in lagrangian moeling of electrical circuits. Vienna, Austria. In proc. 7th Vienna Conference on Mathematical Moelling (MathMo), Nr. 448,. [Kim-Cohen, 2009] Kim-Cohen, S. (2009). In the Blink of an Ear: Towar a Non-Cochlear Sonic Art. Continuum. [Lapp, 2010] Lapp, D. R. (2010). In THE HYSICS OF MUSIC AND MU- SICAL INSTRUMENTS, pages , WRIGHT CENTER FOR INNOVATIVE SCIENCE EDUCATION TUFTS UNIVERSITY MEDFORD, MASSACHUSETTS. [Machover, 1991] Machover, T. (1991). Hyperinstruments: A composer s approach to the evolution of intelligent musical instruments. In Freeman, W., eitor, Cyberarts. Spartan Books, San Francisco. [Mann, 2001] Mann, S. (2001). Intelligent Image rocessing. John Wiley an Sons. ISBN: [Mann, 2007] Mann, S. (2007). hysiphones... In roc. New Interfaces for Musical Expression. [Mann et al., 2007] Mann, S., Janzen, R., an Meier, J. (2007). The electric hyraulophone: A hyperacoustic instrument with acoustic feeback. In roc. International Computer Music Conference, ICMC 07, August 27-31, Copenhagen, Denmark, volume 2, pages [Overholt et al., 2011] Overholt, D., Berahl, E., an Hamilton, R. (2011). Avancements in actuate musical instruments. Organize Soun, 16(2): [Silver et al., 2012] Silver, J., Rosenbaum, E., an Shaw, D. (2012). Makey makey: Improvising tangible an nature-base user interfaces. In roceeings of the ACM Tangible Embee an Emboie Interaction, pages , Kingston, Ontario, Canaa. [Vertegaal an Ungvary, 2001] Vertegaal, R. an Ungvary, T. (2001). Tangible bits an malleable atoms in the esign of a computer music instrument. In CHI 01: CHI 01 extene abstracts on Human factors in computing systems, pages , New York, NY, USA. ACM ress. [Yao an Oobez, 2007] Yao, J. an Oobez, J.-M. (2007). Multi-layer backgroun subtraction base on color an texture. CVR, pages 1 8. _458