UC San Diego UC San Diego Electronic Theses and Dissertations

Transcription

1 UC San Diego UC San Diego Electronic Theses and Dissertations Title To un-button : strategies in computer music performance to incorporate the body as remediator of electronic sound Permalink Author Oliver La Rosa, Jaime Eduardo Publication Date Peer reviewed Thesis/dissertation escholarship.org Powered by the California Digital Library University of California

2 UNIVERSITY OF CALIFORNIA, SAN DIEGO To Un-button: Strategies in Computer Music Performance to Incorporate the Body as Re-Mediator of Electronic Sound A thesis submitted in partial satisfaction of the requirements for the degree Master of Arts in Music by Jaime Eduardo Oliver La Rosa Committee in charge: Miller Puckette, Chair Philippe Manoury Shlomo Dubnov 2008

3 Copyright Jaime Eduardo Oliver La Rosa, 2008 All rights reserved.

4 The thesis of Jaime Eduardo Oliver La Rosa is approved and is acceptable in quality and form for publication on microfilm: Chair University of California, San Diego 2008 iii

5 DEDICATION to my wife... iv

6 EPIGRAPH It is through my body that I understand other people; just as it is through my body that I perceive things. The meaning of a gesture thus understood is not behind it, it is intermingled with the structure of the world, outlined by gesture, and which I take up on my own account. It is arrayed all over the gesture itself (Merleau-Ponty 1962:186) v

7 TABLE OF CONTENTS Signature Page Dedication iii iv Epigraph v Table of Contents List of Figures Acknowledgements Abstract vi vii viii ix Chapter 1. Introduction Chapter 2. Instrument - Gesture - Sound Chapter 3. Gesture, the Studio and the Computer Chapter 4. Live Performance, the Body and Concert in Computer Music Chapter 5. Conclusions References vi

8 LIST OF FIGURES Figure 2.1. Model of a Digital Musical Instrument Figure 4.1. Acousmonium Figure 4.2. Circularity in Performance vii

9 ACKNOWLEDGEMENTS I gratefully acknowledge all faculty and students in UCSD for their contribution to my knowledge. viii

10 ABSTRACT OF THE THESIS To Un-button: Strategies in Computer Music Performance to Incorporate the Body as Re-Mediator of Electronic Sound by Jaime Eduardo Oliver La Rosa Master of Arts in Music University of California, San Diego, 2008 Miller Puckette, Chair This thesis reviews the current literature for instrumental gesture and digital musical instruments. It then elaborates on the gestures of electronic and computer music in the studio and as a result of algorithmic procedures as well as briefly review the composer-performer-listener model of music production. This is followed by a brief history of performance and the concept of live electronic music, from which a theory is derived whereby computer music is incorporating the body as its primary means of expression, elaborating on its perceptual impacts. ix

11 Chapter 1 Introduction Before recording and sound synthesis devices, music performance consisted almost exclusively of instruments performed by people. With the advent of music technology, the practice of public audition of music started undergoing a set of radical changes, from having no people in stage to having no concert hall at all. Computers, machines that have been capable of real-time production of high quality sound for a few decades, have introduced radical changes in the performance of computer/electronic music that we are still exploring and coming to grasp. The sound generating possibilities that music technology opened up in the middle of the 20th century changed the way we understand, produce and consume music and sound. Computer or electronic textures and timbres have had a tremendous impact on the way we understand and process gesture, gradually distancing from the culturally embedded to the unknown. Tape music, either as a product of manipulation of sound recordings or through the synthesis of sounds through the electronic medium, opened up a world of sounds impossible to imagine in the strictly acoustic world. With the use of computers, the ability to represent gestures as functions of time or computer automations as well as the development of other sound generating techniques native to the new medium, these new gestures and organizational possibilities increased. This new sound world developed 1

12 2 its own gestures, shaped at first by the human gestures that produced the sounds in the studio, although usually unrecognizable due to the complex layering process and tape construction procedures, thus creating remote and surrogate gestures. However, the fast emergence of the fields of controller design and HCI, act both as a need to feed computers with live human gestures that control sounds, but also to provide the computer with elements of decision to generate interaction. The concept of live performance of technologically created or mediated material has become an issue of interest not only in the music discipline, but also in other performative arts. New ways of presenting new media are continually emerging and evolving. These new practices, namely, performance with gestural controllers and interactive computer systems will be analyzed as attempts by composers and performers to re-incorporate the body as principal medium of expression and as mediator between the known acoustic instrumental world and the unknown electronic sound. There is a substantial body of theoretical reflection outside the computer music discipline, which can help understand this and which will be reviewed. This thesis is an exploration of the issues surrounding the performance of live computer music as it is evolving, evaluating the impact in the traditional composerperformer-listener model, from the point of view of humans who practice and listen to electronic and computer music. The second chapter reviews some of the literature on the conceptualization of acoustic instruments, instrumental gesture and digital musical instruments. The third chapter explores the new gestures that the electronic music studio and the computer created as a result of the new musical practices that emerged from that technology mainly in acousmatic music. The fourth chapter, explores briefly the history of live electronics, exploring the concepts of interactionism in cognitive science, our understanding of the word live and the impact of the body in the perception and performance of music. This thesis, does not pretend to solve the problems of electronic music performance, but to explore them. As Heidegger once said, we can not know the forest, we can only walk through its paths. And it is in this spirit that certain theorization needs to

13 be achieved to understand why we are doing what we are doing. 3

14 Chapter 2 Instrument - Gesture - Sound Acoustic Instruments Musical instrument is a self-explanatory term for an observer in his own society (Kartomi, 2008) We understand acoustic musical instruments as organized physical objects that are played to produce musical sounds. From this definition, we can extract the idea that a gesture is performed on an object to obtain sound. We cannot imagine an acoustical musical instrument without a human performing it, that is, without human actions that bring out the potential sound that the instrument affords. We can think of an instrument in terms of its material qualities and the sound it makes. When seeing a clarinet we imagine its sound, and when we hear a clarinet we imagine it being played. This sound has a quality that varies with playing technique and material characteristics, but exists within a well-defined set of boundaries that makes it impossible, for example, for a clarinet to make the sound of a guitar and for us to confuse it with one. We usually refer to the sound produced by the clarinet with common playing technique as the timbre of the clarinet and to alternate ways of playing it as extended techniques. These gestures are a set of movements that include those of the respiratory system, mouth, arms and fingers. Specific configurations of these body parts produce a 4

15 5 particular sound. The instrument is then defined by its material limits as well as by the human limits of the performer. The material limits are somehow clear, acoustic instruments are acoustic because the sounds they produce are a physical acoustic phenomenon. These organized physical objects have the function of: (1) transducing a gesture into vibrations; like a plucking a string, blowing into a reed or hitting a membrane; vibrations that become sound waves. (2) Other objects like keys and frets change the size of these waves for pitch control. Finally, some sort of (3) resonating body amplifies and models the sound. In the words of Choi (2000) Musical Instruments provide physical action-spaces with auditory affordances. Instrumental performance is then a negotiation between what the instrument affords in terms of acoustic phenomena and the extent to which a performer controls/explores these affordances. In performing the instrument, the performer needs to exert physical energy upon it, resulting in various types of proprioceptive movement. We call this movement instrumental gesture. Most occidental musical instruments have reached a level of standardization that allows performers to perform a wide repertoire of music that can be composed with certain knowledge of its performing boundaries. This standardization consists of a specific organization of its material elements (resulting for example in specific tunings, range, etc.); all of these have a reflection on technique. The consequences of standardization have shifted our cultural understanding of instruments in several ways. Culturally, we understand instruments through their repertoire and it is this repertoire that seems to determine what is and what is not an instrument and how a particular instrument should be played. Contemporary music practice has significantly extended what we understand as music, and therefore, the practices that define what an instrument sounds like. Standardization has also brought a view of instruments that are perfectly deterministic, ones that are defined by the high predictability of its output relative to a performer s controls. (Chadabe, 2002)

16 6 Finally, the small variation from instrument to instrument has led us to define instruments as singular physical objects in Schick s words. Percussion practice extends and challenges some of our ideas of what instruments are, especially those ideas that stem out of standardization. There is not a single instrument that defines percussion playing in the same way that the piano, for example the singular physical object of the piano, ubiquitous and universal defines piano playing (Schick, 2006) In Schick (2006), the author states the impossibility of defining percussion through a single physical object or through the reduced contemporary solo percussion repertoire and proposes the following: The most succinct definition of percussion comes from the German, Schlagzeug; Schlag means hit and Zeug means stuff This definition is particularly interesting for it has the duality we stated to begin with. An instrument is an object-gesture compound. It is interesting how sound is implicit when defining this sound producing activity. What we usually call musical instruments are object-gesture compounds that have been standardized. Marimbas and vibraphones, sets of bars of similar material and arranged in ascending size, are clear examples of standardized percussion instruments. This brings us to the concept of percussion setup. This concept is very common amongst percussionists. It consists of arranging percussion instruments within the reach of the percussionist; a sort of modular construction of a supra-instrument; an instrument made of instruments. Composers have gladly adopted this practice. When Stockhausen and Boulez talked of composers not only organizing sounds, but also making these sounds themselves, a new perspective on music making was being forged. The construction of the percussion setup seems to be in the same spirit, the construction of the percussion metainstrument allows constructing an instrument for a specific piece; choosing the sounds an instrument will have is in a similar spirit as making the sounds. This concept is however not new, the orchestra, string quartet, wind quintet are clear examples of standardized meta-instruments. These supra-instruments could how-

17 7 ever be considered simply an instrument; a violin is an element of the string quartet and a string an element of that violin. It is still an object-gesture compound and the art of writing string quartets, from Haydn to Lachennmann, provide us with the palette of gestures that are possible. In any event, the practice of choosing instruments for a piece grew very quickly in the 20th century in comparison to previous ones, where standardized formations were the rule. Gesture In our definition of acoustic musical instruments as object-gesture compounds we made a brief definition of instrumental gesture as proprioceptive movement of a performer that brings out the sound potential in an object. In this way, gestures are dependent on the instrument that is being played and the context of a score or plan. In this section I will attempt to review some ideas and conceptualizations about gesture, emphasizing certain facts that pertain to percussion playing and computer tracking strategies. The study of instrumental gesture in music brings out a consciousness of the body that isn t present in the traditional analysis of music. Gestures seem to be studied and classified in terms of bodily affordances and types of movement or in terms of the function they perform. Cadoz and Wanderley (2000) reviewed the current literature on gesture, serving as an excellent guide for this section on gesture. Bonnet et al. (1994) recognize two kinds of motor units and corresponding movements. Slow motor units and movements are related to posture and fast motor units and movements, like fingers or hands, related to gestures such as strikes. Goldstein proposes a similar idea with the terms Current control and Ballistic control : The root of all gesture is muscular action, and neurophysiologists have recognized two types of muscle control [3]. Current control movements are sustained and can be changed while they are being performed. Ballistic control movements are short and energetic. They send a limb on a trajectory while the ballistic movement itself ends before the limb has completed its action. (Goldstein, 1998)

18 8 This brings us to the question of gesture and posture: The coordination between posture and movement condition the efficiency of the gesture. In fact, postural muscular activities, static or dynamical, anticipate, accompany and follow the execution of the movement in an automatic manner. (Bonnet et al., 1994) The question of posture and gesture, specifically in percussion performance is of importance in that it helps us determine what to track. There are many approaches to tracking the movement of a performer that include the tracking of several limbs. However, from this idea we can derive that the movement of a mallet head, hand or finger contains somehow all the other movements including posture; as if it was a derivation or final expression of the general movement of the body towards the gesture that actually produces the sound. Viviani (1994) states a very interesting fact: the bigger the curvature of a movement, the slower its speed. This fact, especially in percussion will give us two basic types of gesture as extremes in the speed continuum, where slow movements produced curved gestures, fast ones linear ones and combinations of these in between. Choi (2000) proposes the idea of gestural primitives as fundamental human movements that relate the human subject to dynamic responses in an environment, presented by the author as an attempt to bring a formalization of human motion in terms of performance gesture into the computable domain. Choi recognizes three types of gestural primitives: (1) Trajectory-based primitives, (2) Force-based primitives and (3) Pattern-based primitives. These are related to rotation or changes of orientation, gradient or linear changes and period, respectively. Choi understands these primitives as a set of resources that can be used across instruments and as substructures of individual gestures or gesture sequences. He exemplifies it in the following way... a trill may be performed as a gradient event by its crescendo/decrescendo property, a force-based primitive. However, a trill could also be performed emphasizing the rate of repetition of individual notes, a patternbased primitive.

19 9 In a more functional approach, Delalande (1988) classifies gestures in three levels from purely functional to purely symbolic : Effective Gesture - necessary to mechanically produce the sound bow, blow, press a key, etc. Accompanist gesture - body movements associated to effective gestures - chest, elbow movements, mimics, breathing for a piano player, etc. Figurative Gesture - perceived by the audience, but without a clear correspondence to a physical movement - a melodic balance, etc. Cadoz considers instrumental gesture as effective gesture, however, it must be quite clear that this separation is somehow artificial, which is of great use for academic analysis, but not directly applicable in describing real gestures; in a performance situation, these are inseparable. For example in percussion, only the down-stroke produces the actual sound, but it needs a previous lift and a movement that positions the mallet and performer in the space of the object that will be hit. Cadoz proposes as well an Instrumental Gesture Typology based on its function: 1. Excitation Gesture: Instantaneous - Sound starts when gesture finishes Continous - gesture and sound coexist. 2. Modification Gesture: Parametric - continuous variation of a parameter. (e.g. vibrato) Structural - when modification is related to categorical differences (e.g. inserting a mute in a trumpet) 3. Selection Gesture - Choice among similar elements in an instrument Perhaps Cadoz s most interesting contribution, besides his comprehensive view of the research literature, is the concept of the gestural channel. The gestural channel is unique if compared to other human communication channels (Visual, auditory, and Vocal) in that it is both a means of action on the physical world and a means of communication of information. In this second role, the gestural channel has a double direction emmision and reception of information. It is therefore impossible to dissociate action from perception. (Cadoz and Wanderley, 2000)

20 10 He recognizes three different functions in the gestural channel: Ergotic function - material action, modification and transformation of the environment. Epistemic function - perception of the environment. Semiotic function - communication of information towards the environment. (Cadoz and Wanderley, 2000) These ideas bring us to the concept of feedback. Having defined an instrument as a gesture-object compound that results in musical sounds, gesture as an action performed on an object implies the response of that object in the form of physical feedback as well as sonic feedback. That impossibility of dissociating action from perception is key in our understanding of gesture. Delalande brings forth two important facts. The first is that a performer perceives his performance with the whole body. The second is that this perception constantly shapes the gesture; that is, in the process of shaping a gesture in rehearsal or during the execution of a performance, his assimilation of perceptual information or feedback helps him accommodate his gesture. Or in Piagets words The hand adapts to the shape of the object. The semiotic function presented by Cadoz refers to the fact that gestures address an audience and is similar to Delalande s Figurative Gesture, which is described as purely symbolic. There is a general conception that gesture is emotional and communicative. From a linguistic point of view, we use gestures to complement speech as non-verbal communication, which gives us the ability to change or alter the meaning of the verbal. It is clear that an excess of gestures is read as exaggeration or dramaticness and a lack of gestures as inexpressiveness or mechanical, but there doesn t seem to be a convincing way of addressing expressivity and emotion in gesture. For Choi, the traditional relationship of gesture and expression is that expressions are said to be the product of gestures. He however proposes the following:

21 11 With respect to Gestural Primitives we propose the inverse of this tradition: an expression is not the child of a gesture, rather a gesture is the child of an expression, where the rehearsal and planning to perform an expression is defined by the performer s orientation to a gestural primitive. Gestural Primitives provide a movement substrate that defines expression resources. (Choi, 2000) Finally, it is interesting to read Delalande s appreciation of Glenn Gould s performance. From this description it is clear that the measure of expressivity relies also in gestures that do not produce any sound, but that are read by the audience as expressive: Pianist Glenn Gould progressively reduced the range of gestures he used to a certain number of types corresponding to what musical terminology would call expressive traits. What is meant by an expressive trait? Merely a type of productive gesture that has become generalized. When Glenn Gould struck the keyboard either vigorously or lightly, the sound produced was not the only indicator - in addition, body position, facial expression -including movements of the eyebrows - expressed vigor or lightness. In a vigorous body position, his shoulders would contract and his head pull in. A contraction of the whole upper body could be observed. On the other hand, to play light, successive notes, Gould s head would no longer be drawn back into his shoulders, and he adopted another typical posture, leaning over the keyboard, and wrinkling his forehead. Thus his whole body was involved. (Delalande, 2003) Digital Instruments As opposed to acoustic musical instruments, electric and digital musical instruments are those in which the gestural input and the sound production are independent linked together by mappings. Sound production independent from acoustical means, that is recording, synthesis and transformation, has basically been practiced for artistic creation since the 1950 s. We ll come back to this in the next chapter. A digital musical instrument implies that the gestural input and sound production units involve digital technology and therefore electricity. Although historical instruments like the theremin didn t use a computer (for obvious reasons), computers have become the most flexible way of recording and producing sound. For that reason I ll refer indistinctly with the term digital musical instrument to both electric and digital, even

22 12 Figure 2.1: General Model for a Digital Musical Instrument though the analog sound world produces sounds without one. A common representation of a musical instrument is the one seen in Figure 2.1. The gestural controller is the unit in charge of transducing the continuous physical gesture of the real world into digital data through some sort of sensor technology. This process is commonly referred to as gesture acquisition, also called by Choi as Gesture extensive research. According to Miranda and Wanderley (2006), gesture acquisition can be of three types: direct, indirect and physiological. In direct acquisition sensors are used to monitor the actions of the performer. A comprehensive description of available sensors for direct acquisition is found in chapter three of the same book. In indirect acquisition, gestures are monitored through the sounds they produce (eg. through a microphone). Physiological acquisition is concerned with the capture of bio-signals and not of physical gestures and is done with specialized equipment like ECGs, etc. To this classification I would distinguish between sensors that are obtrusive and non-obtrusive. This extra category is important when designing controllers that dont interfere with the performers gesture. In this sense microphones and video cameras aren t obtrusive. Another important factor in the design of digital and electric musical instruments is that of feedback. Digital Musical instruments provide at least auditory, in most cases

23 13 visual and in some haptic feedback. This feedback allows the performer to evaluate his actions. Auditory feedback is enabled by an auditory display mechanism, which offers a fine degree of resolution for the data field, and sensitive responsiveness to the observer s performance with low lag time. Only then do we have an environment where the observer is able to construct auditory percepts and relate them to her own performance. (Choi, 2000) Miranda and Wanderley (2006) adopt a classification of new gestural controllers or digital musical instruments based on the degree of similarity to existing acoustic instruments in four categories: 1. Augmented musical instruments - acoustic instruments augmented by the use of various sensors 2. Instrument-like gestural controllers - gestural controllers that are modeled after the control surfaces of acoustic instruments, with the goal of completely reproducing their initial features. 3. Instrument-inspired gestural controllers - gestural controllers inspired by existing instruments or that intend to overcome some intrinsic limitation of the original models, but that do not attempt to reproduce them exactly 4. Alternate gestural controllers - that do not bear strong resemblance to existing instruments. Tanaka (2000) proposes a classification into physical and non-physical, based on the mode of interaction with the controller and independently, mechanical or nonmechanical. In his examples, a potentiometer is a physical mechanical controller, while gestures that modulate light captured by a photocell is, from the point of view of the sensor, neither physical nor mechanical. Choi proposes as well a gesture intensive research concerned with the application to sound production of movement data retrieved from a measurement and storage system. This is also referred to as mapping.

24 14 Iazzeta defines mapping in the following way: While in traditional acoustic instruments the effects of the performer s physical activity on an instrument are already established by the physical properties of the instrument, in electronic instruments this relation must be previously designed. (Iazzetta, 2000) While in acoustical instruments sound production is a direct result of the physicalmechanical application of energy to an object that creates sounds by acoustical means, in a digital instrument gestures are transformed directly into data-streams. The job of mapping is to assign these data streams or gesture parameters to parameters of sound production. Most authors on mapping issues claim that this is a critical feature of electronic instruments. Rovan et al. (1997) classify mapping strategies into three categories: (1) One-to- One (2) Divergent, and (3) Convergent. In One-to-One mappings each independent gestural output is assigned to one musical parameter. It is regarded as the least expressive. In divergent mapping, also known as One-to-Many, one gestural output is used to control more than one simultaneous musical parameter. Although it may initially provide a macro-level expressivity control, this approach nevertheless may prove limited when applied alone, as it does not allow access to internal (micro) features of the sound object. In convergent mapping, or Many-to-One, many gestures are coupled to produce one musical parameter. This scheme requires previous experience with the system in order to achieve effective control. Although harder to master, it proves far more expressive than the simpler unity mapping. Hunt et al. (2000) explored mapping models for expert interaction, concluding that while complex mappings are harder to learn they allow control of complex parameter spaces. Mappings can be implemented in two ways. Using generative mechanisms, such as neural networks or high dimensional interpolators or to define the mappings explicitly. (Hunt and Wanderley, 2003). In most musical situations controllers produce a different number of controls (usually less) than the number of parameters that need to

25 15 be used to control a sound, simply because a human being can only pay attention to a limited number of things at a time. Both gesture control and sound parameters can be represented as their derivative or their integral, increasing the ways of looking at these parameters, but also increasing the number of parameters to be mapped. The question is then how to match the number of controls to the number of mappings using complex mappings? Goudeseune (2003) proposes a high dimensional interpolator that allows the user to specify pairs of groups of values; that is to specify what values we want in the sound variables when the control variables have another set of values, or, as Goudessene puts it: when the controls have these values, make this sound. This process continues by adding other points in the map and then interpolating through them with a high dimensional simplicial interpolator. With this method, the number of control parameters a performer needs to pay attention to is reduced, increasing his intuitive understanding of the instrument. In a similar spirit, Lee et al. (1991) proposed mappings with multi-layer neural networks trained by back propagation to control sound synthesis. Besides the strategy of mapping that is used, there is the question of the kind of sound we are trying to control. Goldstein (1998) proposes observing the gestural affordance of real instruments to design our controllers or observing the sound s structural qualities to choose or design the controller. This is done as an attempt to obtain what he calls gestural coherence, avoiding for example having violin sounds being controlled by keyboards. Levitin et al. (2003) define the role of mapping as a means to exploit some intrinsic property of the musicians cognitive map so that a gesture or movement in the physical domain is tightly coupled in a non-arbitrary way with the intention of the musician. For him, our brains have evolved by incorporating certain specific physical principles of the world and developing cognitive maps that condition our perception. These cognitive maps include notions such as harder means louder (for breathing or striking), gestural wiggling yields pitch or timbre wiggling (such as in creating vibrato on a stringed instrument), and tighter means higher in pitch (such as when stretching

26 16 a membrane on a drum, or tightening the embouchure on a wind instrument). The problem with both Goldstein and Levitin s approach is that by focusing on the way we control the physical sounds of acoustic instruments we leave certain kinds of sounds aside. For example, audio feedback is a phenomenon exclusive of the electronic medium, impossible to model in terms of an acoustic instrument and its gestures. The kinds of gestures used to control feedback are well exemplified by Ostertag in his description of Jimi Hendrix s playing style: Hendrix s crucial innovation was playing at high volume and standing close to the speaker to obtain feedback that he could control in an extremely nuanced way with the position and angle of the guitar, the weight and position of his fingers on the strings, even the exact position of his entire body. (Ostertag, 2002) Evaluating a mapping strategy is an issue that has to be addressed in a case-bycase basis based on the sounds to which we are trying to map. The scope of this thesis can not include a review od all sound production techniques. However, we can still abstract ways of approaching sound production that mapping does not fully address. Hunt et al. (2002) raise the issue of mapping as a specific feature of a composition or as an integral part of the instrument. They consider the second point of view as the best, giving the instrument consistency through time and offering the performer the possibility of learning these mappings at an expert level. It is clear that this issue raises all sorts of questions surrounding the concepts of instrument, which will be discussed later in more detail, but some questions can be posited now. Computers are machines capable of performing many tasks, many of which include to a certain extent, making decisions. It is clear that in certain mappings-for example in controlling a granular texture-we are bound to map our gestures onto higher level parameters like density, envelope shape, delay times, random generators, etc. This is simply because if we were to map open air hand gestures to each of the grains, either the sounds would not be as interesting-or at least would not sound granular-or we would be forced to move hysterically and fail in the attempt. So for this matter, we entrust the computer with the task of generating the texture from our general indications.

27 17 Chadabe recognizes two types of instruments, deterministic and indeterministic ones. As mentioned previously, certain determinism characterizes standardized acoustic instruments in the hands of trained performers. However in digital musical instruments, the sounds generated by the gestures are not always entirely determined. For Chadabe an indeterministic instrument outputs a substantial amount of unpredictable information relative to a performer s controls. In working with such an instrument, a performer shares control of the music with algorithms as virtual co-performers... (Chadabe, 2002) In mapping strategies like Goudessene s interpolation or Lee and Wessel s cellular automata, the designer doesn t fully describe the outcome desired, but only certain reference points. The computer then complements the final behavior of the controller and the performer is left to discover them in a learning process. But the overall behavior of the instrument is the same unless the reference points change. But Chadabe goes further in assessing the limitations of mapping in determining the way an instrument works: To the extent that an automatic mechanism generates information, even while remaining obedient to a performer s commands, it becomes more difficult to conceptualize a performer s control gestures as mapped onto an output the computer s ability to expand simple but powerful instructions into coordinated controls for multitudes of variables, to redefine controls in different contexts, and to maintain goal-orientation while introducing enough unpredictability to keep the instrument interesting. (Chadabe, 2002) The question then starts to reveal itself. Can a button be considered an instrument? Can pressing a button be considered a performance?

28 Chapter 3 Gesture, the Studio and the Computer The model Composer-Performer-Listener is the product of a long standing tradition in the history of music and indeed, when no means of generating sound independently from instruments existed, a concert/performance was the only way to deliver sounds to an audience. Although composer/performers existed, mainly due to the fact that traditional music education proposed the knowledge of the piano as an indispensable pre-requisite, the model persevered as a kind specialization system, where one was trained in composition and the other in performance. This also led to the concert ritual, the space and moment in which the performer delivered the music from the composer to the audience. It is necessary to state however that this model is to say the least simplistic, but it still reflects somehow the ideas of occidental culture towards music. In his Introduction to the Sociology of Music, Adorno (1976) states that works are objectively structured things and meaningful in themselves, things that invite analysis and can be perceived and experienced with different degrees of accuracy. This point of view subscribes to the idea that the work is an abstract idea, a product of the mind, perfect in itself and independent from its sound realization. This is supported by a tradition of sound analysis that focuses exclusively on scores and particularly on a form that is determined by the organization of pitch as the main material. This leaves the performer as a body in charge of realizing this idea into sound, which can then be received 18

29 19 by an audience, the minds that can, at least in Adorno s view, perceive and experience it with certain degrees of accuracy. The 20th century also saw an increased desire from composers to obtain new sounds. This was reflected in the increased use of percussion instruments, extended techniques, prepared pianos, timbral use of the orchestra, etc. There are many accounts of composers complaining about the inability of the instrumental medium to realize their ideas. In many ways the search for new ways of producing sound was a result of technology as much as music s use of technology was a search for new sounds, specially given that these technologies existed for several years and, in the case of recording, decades before. In the 1950s two major research centers concentrated the research on electronic sounds. On the one hand the Radio Television Franaise (RTF) in Paris, directed by Pierre Schaeffer, associated with Musique Concrète, which focused in the manipulation of recorded sounds as the main means of sound production. On the other Nordwestdeutscher Rundfunk in Cologne, directed by Herbert Eimert, associated with Electronische Musik, which focused on the use of sound generators and modulators to generate sounds. Other studios and research centers gradually evolved and the electronic music studio gradually formed itself into a sort of standard with some form of synthesizer and tape recording device as central units. The unifying element in both cases was the fact that the music produced, regardless of the sound sources, had an almost strict dependence on the use of magnetic tape as a medium to fix the sounds not only as a final product, but also throughout the composition process. Although we could argue that the recording process changes the sound, by fixing a sound in a digital or analog format, the concrete character of sound is preserved (as opposed to abstracted and notated) and acquires a material existence. This materiality not only allowed the possibility of sound reproduction but also of transformation and organization. In this sense, composition shifted to work directly with sounds and not with abstract representations or ideas of sounds, as in the case of instrumental musical scores.

30 20 In one of Schaeffers first definitions of Musique Concrète published in the magazine Polyphonie in December, 1948 We apply (...) the term abstract to conventional music, because it is first conceived by the spirit, then theoretically notated, finally realized in an instrumental performance. We have named our music concrete, in opposition, because it is constructed from pre-existing elements, regardless of what sounding material caused it, be it noise or conventional music, then composed experimentally by direct construction. (quoted in Chion 1991) When Schaeffer refers to a direct construction he not only refers to the physical cutting and pasting of the tape, but to the fact that musical notation and instrumental performance were not part of the composition process, the composer would work directly with the sound material that would constitute the piece; with the concrete material fixed in a recording format. Musique Concrète and music fixed in a recording format was later coined as acousmatic by composer Franois Bayle. This originally greek term refers to the phenomenon of hearing a sound which we can not see the source or cause. Various composers then adopted the term to describe music that comes out of loudspeakers. So at the outset of acousmatic music the music produced by these composers challenged the composer-performer-audience model in many ways, but principally in the fact that there was no visible performer, and at a first glance, the performer was removed completely: the music that was produced in the studio was in practice the same to be heard by the audience, it would not be performed for them. Several issues arise from these facts. Music, an art form characterized by the performance of music in a concert hall was not the same anymore. Furthermore, this performance practice, which traditionally featured a human being playing a musical instrument through musical gestures, didn t seem to have a place anymore. But what if we try to picture the electronic music studio as an instrument. Luigi Nono considered the studio as new instruments which need to be studied over time to learn and study again and again, to explore other possibilities, different from the ones usually chosen and given, other musical thoughts, other infinite spaces...

31 21 (Nono, 1984) In our initial definition of musical instrument, we talked about object-gesture compounds that produce sound. In our distinction of electric and digital musical instruments, we talked about the independence of gestural input and sound production units. Electronic music studios then, fit perfectly well in these definitions. Knobs, faders and switches constituted the gestural input unit. The mapping layer was the patching bay and the cables that connected the unit generators, modulators to speakers or to the magnetic tape. Magnetic tape manipulation on the other hand presented similar features with knobs for playback speed, direction and volume, providing even the possibility of pressing the tape on the reader with hands and fingers. Most studios quickly incorporated keyboards and Pierre Schaeffer even developed his own tape-controlling keyboard, the phonogene. The studio provided as well many limitations. The first limitation was the size of these studios and the specificity of the equipment, factors that reduced the possibilities of moving it anywhere (and therefore onto a stage). Although the set of possible sounds that could be produced with this instrument increased dramatically, it was impossible to jump from one sound to the other, for it implied a change in the patching bay. On the other hand, the amount of sounds that could be produced at the same time was limited to the number of unit generators, modulators and tape decks available. For this reason the tape became the medium to store sounds temporarily, giving the composer time to produce another sound that could be mixed with this one, to change the mappings for the next sound, to change the tapes that were being manipulated, to store all the sounds that would then be assembled into the final piece and to fix that final result on tape. In a way, the composer performed the sounds into the tape, which recorded his original gestures. All of these facts resulted in a kind of music that had a temporal and spatial dislocation, between the place and moments in time when the sounds were recorded to the place and moment in time in which the audience heard the sounds. While in the traditional composer-performer-audience model, the composer worked on a composition or a

32 22 score in a separate, previous time, the audience perceived the sounds in the performance immediately after they were produced by the performer in the concert. Now, the sound perceived by the audience in a concert was the result of sound produced in other spaces and times. So if we were to sketch the model again we would have to create a hybrid composer/performer, if our model reflected the time process, we would have to include performance-composition as much as composition-performance, or even more to establish a more dialectic way: composition-performance where each one renders the other in a slow process. This is precisely the process from which the audience is removed, left to hear the result through speakers. Although human gestures are employed in the construction of electronic sounds in the studio, there are several points to consider. The first is that the manipulation of electronic sounds with machines often involves gestures that are different from our known instrumental gestures, first because the interface is different and second because the sound material is different. Even when using recorded physical sounds (and therefore physical gestures), the results can become so distant from the source sound that the original gesture is lost. The process of construction is usually so layered that the resulting sound has probably no direct relation to what we have learned to recognize as musical gestures in acoustic instruments. Composer Denis Smalley describes this process in the following way: the working gestures of the acousmatic compositional process do not carry perceptual information equivalent to an intuitive knowledge of the physical gestures of traditional sound-making. Therefore, while in traditional music, sound-making and the perception of sound are interwoven, in electroacoustic music they are often not connected. (Smalley, 2001) Smalley describes the final sounds produced by acousmatic music as spectromorphologies referring to the interaction between sound spectra (spectro-) and the ways they change and are shaped through time (-morphology). The spectro- cannot exist without the -morphology and vice versa: something has to be shaped, and a shape must have sonic content.

33 23 Parallel with the growth of electronic music studios, several research centers in the USA were researching the production of sound with computers. Amongst those studios were Bell Telephone Laboratories in New Jersey. The first computers were as big as several rooms and rendering one sound could take hours or days. As early as 1963 Max Mathews proposed the idea of using the computer as a musical instrument (Mathews, 1963) and in 1970, along with F. Richard Moore published their GROOVE system: A Program to Compose, Store, and Edit Functions of Time (Mathews and Moore, 1970). This system had the virtue of being able to produce sounds by capturing gesture input by a computer that controlled analog devices for sound production. Again, the size of the machinery made it impossible to stage in a concert. The first line in Mathews and Moore (1970) states: Many tasks now done by people are best described simply by one or more functions of time. This concept of functions of time is key to computer technology and especially to computer music. Although in their program, gestures inputted to the system were represented as functions of time to be used immediately, it also allowed to store, edit and compose them. That is, gestures could be manipulated and composed as much as sounds as functions of time and therefore the control of analog sound devices did not necessarily depend on humans at knobs, but could now be automated. Due to the rapid improvement in computer technology, the electronic music studio has gradually become just a computer. The practice of acousmatic or tape music is now usually realized directly on the computer. (The value judgments of analog vs. digital technology are not part of this thesis.) This increase in computer speed also allows for freer and less mechanic ways of composing acousmatic/tape music, with no need of using magnetic tape for storage or assemblage. This also means that gestures and sounds are now commonly automated not only in the sense that control functions are not performed by human performers, but also through the use of algorithms. In this way computers became the primary tool for music produced in the studio and then played back in a diffusion system or concert hall, but also a tool to produce sequences of instructions that could assist a composer in the

34 24 production of a composition, either as a score intended for instrumental performance or directly modeling the sounds. The gestures performed by the computer or even the gestures we create as a result of the direct construction mentioned by Schaeffer are not the gestures we have trained ourselves to recognize as coming from humans playing instruments. So what are these gestures and how do we make sense out of them? Dennis Smalley proposes a classification of gesture in terms of the perceptual distance it presents from the gestures we know from experience. For him listening is a sense making activity whereby we do not only think of the gesture process as causesource-spectromorphology, but also as a reverse process spectromorphology-sourcecause. We listen to sounds in a referral process by which we hear a spectromorphology, recognize the source and detect human activity behind it, building the gesture that caused it. An example of this is when we hear to recorded instrumental music. The listeners experience of listening to instruments is a cultural conditioning process based on years of (unconscious) audiovisual training. A knowledge of sounding gesture is therefore culturally very strongly embedded. (Smalley, 2001) But when we hear acousmatic sounds, the causes can become remote or detached from known. Smalley calls this increasing remoteness continuum gestural surrogacy. Gestural surrogacy then has the following categories: First-order Surrogacy: The original, primal gesture, on which sounding gesture is based, occurs outside music in all proprioceptive perception and its allied psychology. Traditionally, this first level does not become music; it develops into second-order surrogacy. To consider them first-order we need to recognize source and gestural cause. Second-order surrogacy: Traditional instrumental gesture, a stage removed from the first order, where recognizable performance skill has been used to develop an extensive registral articulatory play. Much music which uses the simulation of instrumental sound can also be regarded as second-order since, although the

35 25 instrument may not be real, it is perceived as the equivalent of the real. Third-order surrogacy: where a gesture is inferred or imagined in the music. The nature of the spectromorphology makes us unsure about the reality of either the source or the cause, or both. In his examples we can infer a cause for an unknown source or we can only partially its behavior. Remote surrogacy: concerned with gestural vestiges. Source and cause become unknown and unknowable as any human action behind the sound disappears. The listener may instead be concerned with non-sounding extrinsic links, always, of course, based on perceived spectromorphological attributes. But some vestiges of gesture might still remain. To find them we must refer to tensile, proprioceptive properties, to those characteristics of effort and resistance perceived in the trajectory of gesture. Thus, remote surrogacy, while distanced from the basic, musical first order, can yet remain linked to the psychology of primal gesture. (Smalley, 2001) While most of Smalley s ideas are based on experience and could be debatable in many ways, his intuition and ability to identify, categorize and name phenomena is one of the few that exist. Before leaving his ideas two more points need to be made. In this sense-making activity and in relation to the extrinsic links he mentions in remote surrogacy, Smalley created the term source bonding to refer to connections between intrinsic qualities of the sound and extrinsic qualities of the world. In this way, the connections a listener makes between a granular texture and rocks falling are a form of source bonding. The second point is that of texture. For him, if textures are weak, if they become too stretched out in time, or if they become too slowly evolving, we lose the human physicality. We seem to cross a blurred border between events in the human scale and events on a more worldly, environmental scale Smalley (2001) To this I would add that the conglomeration of small gestures in a small or repetitive way are also textures, although

36 26 not necessarily detached from the human scale. In this sense, long orchestral notes or even repetitive figures like the alberti bass act as a texture. However, we can deduce the many possibilities in which textures can be created in the acousmatic medium, either as granular processes created by a computer or through cutting and pasting magnetic tape like Xenakis Concret PH, Cage s Radiomusic or Tenney s Blue Suede Shoes. As stated previously, computers are machines able to receive and produce sequences of instructions that could assist a composer in the production of a composition, either as a score intended for instrumental performance or directly modeling the sounds. The use of algorithms in composing is not entirely new in the sense that there have been innumerable attempts at formalizing compositional processes from D Arezzo and Serialism to Xenakis as the most cited examples. Computers are machines perfectly fit for this task and therefore, since the advent of computers in music, some of the first tasks they were to perform were those of algorithmic organization of sounds. The specific applications of algorithmic composition have included the generation of melodic, rhythmic and harmonic material created by algorithms that are then analyzed and modeled by the composer. Other approaches have attempted more radical positions, that is, where the composer doesn t modify the computer result. The question of to what extent instrumental composition is determined by the physiscal gestures of humans playing acoustic instruments is one that needs to be assessed. The traditional disciplines of music education provide us with hints on this issue. Besides music language courses, the basic courses in traditional conservatory curriculums for music composition are music theory, harmony, counterpoint, analysis, instrumentation and orchestration. Instrumentation and orchestration are concerned with issues of performability in acoustic instruments, and furthermore, the issues of comfortable ranges, and dynamic and articulatory possibilities in those ranges. Gestures like trills, glisses, tremolos, chords are instrument dependent and have intrinsic limitations within each instrument, limitations that are determined both by the instrument and the corporeal limits of the player. Iazzeta points out these issues in the following way:

37 27 Concerning music, one can say that physical gesture is directly related to music interpretation while composition is much closer to mental gesture: As Bernadete Zagonel stresses, if the composer goes from gesture to the composition, the performer goes the opposite way, that means, he goes from the score to the gesture. To this statement we can add that the listener completes this chain by mentally recreating the performer s physical gestures while listening to music. (Iazzetta, 2000) This mental gesture produced by the composer before writing in the score is based on how a particular musical idea would sound on an instrument, but is also based on his cultural training on the way instruments work and sound, on the analysis, perceptual or formal of the way in which instruments work and the limits they present. To write a crescendo going from ppp in the highest register of the flute to fff in the lowest is not adequate and won t work. These kinds of considerations form what has been called idiomatic composition, which we could define in very similar ways to instrumentation, but adding the consideration of what kinds of phrases and passages are most appropriate for the instrument. Tanaka (2000) This approach not only defines the instrument through composition, but also composition through the instrument. In other words, mental gestures are those instrumental gestures we have processed and retained through past experience. In this way, could the embedding of instrumental gesture in our cognitive system be determining the gestures we make in electronic music? Could electronic music be hiding a body beneath it? So what kinds of mental gestures can a computer have? The computer s unawareness of instrumentation rules (even of its own performative capabilities) renders it incapable of good acoustic instrumentation unless coded specifically for the task as a constraint to the composition algorithm. Such a task is better performed by the composer, who as a human, shares the body with the performer and can asses, by imagination, previous experience, education, score analysis and case-specific testing, the particularities and affordances each instrument presents. So the composer can act on the computer in the following ways: either specifying parameters for the creation of material, encoding the boundaries and general concept of

38 28 his mental gesture or encoding a general idea that is then mediated by the composer to make it adequate for the instrument. But, can we have non-instrumental mental gestures? The computer can easily perform certain things that are impossible for acoustic instruments and humans. Can we talk about gestures of the instrument-computer or even about a computer-composer? I will leave these questions partially unanswered. For Smalley, acousmatic composition is a sort of negotiation between levels of surrogacy: Acousmatic music, therefore, can stay close to traditional, gestural cause-source relations, but at its most adventurous extends into thirdorder ambiguity and beyond to a music which, although remote from traditional sound-making activity, can nevertheless maintain a humanity. I venture to suggest that an electroacoustic music which is confined to the second order does not really explore the potential of the medium, while a music which does not take some account of the cultural embedding of gesture will appear to most listeners a very cold, difficult, even sterile music. (Smalley, 2001)

39 Chapter 4 Live Performance, the Body and Concert in Computer Music History: Electronics and Performance The advances introduced by the computer and technology in general, namely the radical expansion in sound material and new spectromorphologies with the ability to create surrogate gestures independent from instrumental ones and their organization through algorithmic methods presented new problems: how to perform or present them to the public, the audience. Due to convention, music produced in studios was presented in the traditional space for presenting music: the concert hall. The first concerts introduced the concept of a loudspeaker orchestra as opposed to traditional performance where a human was in charge of bringing the sounds out of the score. The concept of a loudspeaker orchestra had inevitable direct connotations with the idea of going to a regular orchestral performance. As we can see in Figure 4.1., the loudspeakers where placed in the space of the orchestra and the audience remained seated. This format of presentation, the concert, with an audience facing a stage in a darkened hall, and in the stage a dissimilar array of speakers, seemed to contrast with instead of assimilate the performing tradition. Audiences were already familiar with radio and record playback. Since the ad- 29

40 30 Figure 4.1: François Bayle in the Acousmonium in 1974 vent of recording technology, music had acquired a dual nature, recorded and live. Where recorded music was the recording of a live performance and live music was the actual performance. Now music presented new sounds, produced through a process unknown to the audience and whose sonic results were usually unknown as well. Moreover, the resulting pieces were fixed in a format associated with recorded music, they were actually records, the same the audience had at home, in their living room. So why go to a concert which is a sort of expanded living room? There seemed to be an association whereby the concert hall belonged to the live sphere and the living room, where the radio and the record player lived, to the recorded one. Sound spatialization, the distribution of sound in a space, became key in the diffusion of acousmatic music and recorded sound in general. Initially, the works of composers like Schaeffer and Henry, who came from the radiophonic tradition were presented from the front, monophonic or at best multi-monophonic Bayle (2007). This situation set the sounds to come from one place, each one from one loudspeaker. The acousmonium, created in the 1970 s by François Bayle, was based on the concept of the