19 th INTERNATIONAL CONGRESS ON ACOUSTICS MADRID, 2-7 SEPTEMBER 2007 EXPERIMENTS ON THE UTILIZATION OF SPACE AS A MUSICAL LANGUAGE RESOURCE

Transcription

1 19 th INTERNATIONAL CONGRESS ON ACOUSTICS MADRID, 2-7 SEPTEMBER 2007 EXPERIMENTS ON THE UTILIZATION OF SPACE AS A MUSICAL LANGUAGE RESOURCE PACS: Cd Miralles Bono, José Luis 1 ; Payri, Blas 2 ; Redondo, Javier 3 ; Picó Rubén 4 1 Universidad Politécnica de Valencia; Cami de Vera s/n, Valencia, 46022, Spain; josmibo@posgrado.upv.es 234 Universidad Politécnica de Valencia, Campus de Gandía, Grao de Gandía, Spain; 2 bpayri@har.upv.es; 3 fredondo@fis.upv.es; 4 rpico@fis.upv.es ABSTRACT This paper explores the possibilities of spatialization and intrinsic space perception as a musical form-bearing dimension. Grounding the experiments on the conditions defined by McAdams on the psychological constraints on form-bearing dimensions in music, we define a set of conditions on our sound material. After analyzing the usage of space in actual musical works and the techniques of spatialization in electroacoustic and instrumental music, we define a set of experiments using the general frame of Auditory Scene Analysis. First we explore the continuous evolution of spatial perception as opposed to discrete jumps in the spatial position. Second, we define the material to understand the utilization of space as a form-bearing dimension per se, as opposed to the static spatial position that is a resource to improve the discrimination of musical events within the acoustic surface. Listeners must use the Acousmographe tool to segment the acoustic surface into significant units and group elements together. We define different space combinations to understand whether spatialization breaks the continuity of a sound stream or whether a set of sounds with different space positions may be grouped together and recognized as a pattern. INTRODUCTION "I love the sound of the flute: it is beautiful when one hears it gradually approaching from the distance, and also when it is played near by and then moves far away until it becomes very faint." The Pillow Book, Sei Shônagon (Japan, XIth century) It is a classical assumption that music is a time-art (Zeitkunst), and painting, sculpture, architecture, etc are space-arts (Raumkunst). If this opposition is quite valid historically, music composers have now a large array of possibilities to use spatial features. Space has been thoroughly studied in psychoacoustics [1], and widely used in contemporary and electroacoustic music. But its possibilities to be used as a form-bearing dimension have not been yet seriously studied. As composer François Donato [2] points out talking about the projection of electroacoustic works during concerts The essential parameter that we use in our concerts, namely space, remains rather mysterious. In almost each case, we proceed in an empirical, completely intuitive way, by using our basic tool - hearing - to control the effect of our actions at the projection console table. It is already rich, carrying an already interesting potential of musical quality, but it is not sufficient. We miss information on the real effects of the perception of space, on the possible archetypes of hearing, whereas for centuries, we piled up a wide knowledge on the perception of the usual parameters, pitch, intensity, durations and more recently (in Europe), on the timbre. We are at the moment at a rather naive stage in the use of space for the enhancement of music. We want to place in this paper the basis to test if the space can be considered or not a form-bearing dimension., as Pope points out it is important to remember that space, and spatialization as a sound parameter, can and should be used compositionally in computer music. In scientific words; in this paper we want to test the usability of space as a form-bearing element, and in musician words; we want to know if the space can be used in composition or interpretation as the other compositional parameters: pitch, rhythm, dynamics, timbre, etc... And

2 if yes; with which (in scientific words) psychological constraints or (in music compositors words) how could be used. Form-bearing elements Stephen McAdams defines, in his paper [3] about what should be a form-bearing dimension, the following constraints: - A form-bearing dimension should be closely correlated with the sensory dimensions that effect perceptual grouping in other words to organize the acoustic surface into musical events, connecting the events into musical streams or chunking the event streams into musical units (simultaneous, sequential and segmentational grouping). - A form-bearing dimension should be susceptible to being organized into perceptual categories and relations among these categories should be easily encoded. - certain recurrent sequential patterns of values should be easily learned as a kind of lexicon of forms. As McAdams says on the factors of utility of the dimension as a form-bearer, we find that if the dimension affords a greater number of perceivable, and discernible, configurations it will be more valuable to a composer than a dimension which only has a small number of perceivable configurations. And must present the capacity to encode patterns along a given dimension in the presence of changes along other dimensions. USES OF THE SPACE IN MUSIC Some historical uses The conscious use of space in music is not an actual discover, it is certain that the new technological innovations made possible an easily use, but we can find examples of the conscious use of space in the history of western music, all most related to an use of a specific place. The most famous case in the traditional studies in music history are the italian polichoral techniques used in in St Mark's Basilica in Venice. The First composer who makes famous the compositional possibilities of St Mark's Basilica was his Chapel Master Adrian Willaert ( ) with different chorus disposed in strategic places in the Basilica dialoguing with the organ. In the Monteverdi's Visperas is used too this little game with chorus who dialogues. Berlioz's Grande Messe des morts disposes in the first part of the work 200 musicians fanfares in the four transept s vertex down the dome of the Église Saint-Louis Des Invalides. The musicians must wear the victim's mortal remains of The 3 Glorious (before the French revolution). The second part must be interpreted in the square during the burial ceremony. In Verdi's Requiem, at the Tuba Mirum, the italian composer uses trumpets surrounding the stage to produce the call to Final Judgment. As we can see, the principal function of these uses of space are theatrical. Some 20 th & 21 st Century instrumental and electroacoustic uses At the moment, there are different aesthetic streams closely related to the use of space [8]. First we can have a normal work, without any compositional considerations of the use of space, but with specific loudspeaker diffusion, as Die Soldaten of B. A. Zimmermann; or with a specific instrumental distribution, like Gruppen of K. Stockhausen. And also the use of a specified place, with some characteristical properties, to play the work. With instrumental uses we can consider works with an specified movement of the musicians; an early example of this can be the 4 th Symphony of C. Ives, in which there are two orchestras, one with constant pulse at the scene; and the other with not equal pulsation and walking down the scene. Finally with instrumental works we can afford the combination with electroacoustics; like some interpretations of E. Varèse s Deserts, on which the electroacoustic sounds surrounds the public. On the other side, we have the works only with electroacoustics, with a complete fixation of the sounds in multitrack (A. Vande Gorne s Vox Alia) [7], ambisonics (cinema s THX), etc Space simulation or spatial synthesis with software (with Spat s IRCAM for example) [5] or even the interpretation of an electroacoustic work in a specific place, like L. Nono s Fabrica Illuminata, on which concrete sounds were recorded in the same place the work was played the first time. An in depth study of space uses in contemporary music can be found at [14]. 2

3 TOWARDS AN EXPERIMENT ON MUSICAL USES OF SPACE Notions of space Before presenting the experiments, it is necessary to define the concepts we use. Duchêne [5] distinguishes between intrinsic and extrinsic space: the former is the space as recorded in a work (here we are talking about electroacoustic music), the latter refers to the spatial combinations that are due to the actual space where the work is played: loudspeakers, distances, room acoustics. We will only test the intrinsic properties of space, and will try to eliminate any extrinsic influence by having the subjects listening with closed headphones. Also, Duchêne defines the notion of spatial mass, that is, the perceptual size of the space that a given sound seems to occupy, having on one side sounds that seem to come from everywhere and on the other side sounds that are very localized. For the time being, we shall skip the parameter of spatial mass, although if the experimental results bear fruit, it will be a very interesting parameter to study as a musical resource. We cannot forget one of the most important notions of space: the interaction between the surroundings acoustical properties and a specific sound source. Namely, reverberation and other filters that influence the feeling of space, have a wide utilization in electroacoustic music and in other fields of sound design, like movie sound design or soundscapes. This kind of use of space that includes the room response is maybe musically more important in its actual use than other ways of creating a space, and should also receive attention in a later stage as a possible musical resource. Finally, a very important factor in spatial perception comes from the recording conditions of well known sound sources. For instance, the distance microphone-speaker can be perceived by the listener due to its familiarity with the acoustic characteristics of voice, and the balance of energy, the presence of mouth and other articulator noises and other Sound Material When choosing the sound material, we must be aware of the influence in the perception of space due to the harmonic, frequency related and temporal characteristics of the sounds. Bregman [10] discusses amply the conditions of sound separation in the auditory scene analysis. Blauert [1] focuses more specifically in sound localizability. For instance, pitch has an influence in space perception and low pitch sounds are localized poorly because head size limits azimuth resolution and because the loose correlation of acoustic degradation with distance limits accurate estimation of auditory distance. [13] Also, as Cheng and Wakefield [15] point out, many limitations of moving sound source synthesis are closely related to some well-known limitations of stationary sound source synthesis, in which the synthesized sounds do not move through space: listeners often report that there is a lack of presence in spatially synthesized sounds. Also, more complex phenomena are described as the fact that signals processed to sound as though they originate from in front of or above a listener actually sound like they originate from in back of or below the listener (the so-called front-back and up-down confusions) [15] On the election of the material, we take sounds that have obvious sound sources: the reason is that this would help stream segregation as listeners will readily recognize the sound source regardless of its spatial position. Also, we choose pitches that are in the central part of the audible range which are the easiest to localize. And we avoid completely stationary sounds even when we perform synthesis or sound transformation to get our sound material. We propose five different kind of material to use: voice (sustained voiced sounds, either singing a sustained vowel or sound transformation) speech (different medium pitch as the sustained voice) whispered sustained voice pitched sound - abstract synthesized or from a given instrument pitchless sound - abstract synthesized or from a given instrument 3

4 Conditions to be tested: sequences versus isolated sounds Defining the conditions we want to test is the heart of this paper. First, we must point out that spatial hearing has been very well documented and studied in psychoacoustics [1]. Our experimental design is not aiming at solving classical psychoacoustical issues on space perception but rather define the musical use of space. We could hypothesize that spatial parameters fulfil the conditions of a form bearing element as defined above using the conditions of McAdams [3], as indeed we can have vectors of space, define relative values, and transpose spatial positions. But as the experiments with timbre show that they timbre has a the possibility of being a form bearing element, the actual experiments use isolated sounds starting by Grey [16] and even McAdams [17], and do not explore the actual composition of timbre melodies in a musical sequence, as should be the case when exploring the Klangfarbenmelodie. In our experiment we propose to use sequences and not isolated sounds so that we can infer the possible uses of space as a musical language element: the goal is to recognize space sequences that would be the equivalent of a theme, a tune that can be recognized and can be distorted with variations to some extent. Our listening conditions will be contextual, as we believe that context is very important in musical listening as discussed in [11] and [12]: experiments can show that a specific feature is recognized with isolated items, but then the actual use of the same sounds in a musical sequence may produce a different perception, and the features that are important with isolated items become inaudible or irrelevant compared to other elements. The first condition is on the space changes that we want to test. As mentioned before, we will not use reverberation and other filters, and the basis of our spatial movement will be the leftright opposition, that can be achieved with any stereo sound editor. That forces our sounds to be mono recordings. Left-right motion is perhaps still the best known, most straightforward, and most convincing of spatial trajectories to synthesize. As shown in [15] spatial locations which are directly opposite the left and right ears (azimuth 90 and azimuth 90, respectively) both spatialize and externalize extremely well. To study further if a space sequence can be recognized with more complex movements, we may add to the left-right other spatial trajectories like front-back, and updown which Cheng and Wakefield claim they can express certain spatially based musical ideas. [15] There are several well-known techniques that can be used to spatialize moving sound sources. Simple panning methods are still very effective for many types of left-right motions, and dynamic reverberation manipulation produces effective illusions of varying depth. More complex, pitch-based processing, such as Doppler shifting, can also produce compelling examples of moving sounds. Other methods for producing moving sound sources, such as ambisonics and interaural cross-talk cancellation rely on systems designed for loudspeaker reproduction of spatial sound. An excellent production-quality software package that incorporates these and other spatialization techniques is IRCAM s Spatialisateur ( Spat ) program. [15] The second condition refers to the space sequences themselves, and we want to test 3 space configuration possibilities. Continuous position change Discrete position change No position change The sounds appear with different space patterns, that vary along the two following elements: 1) variation type: continuous and discrete variations of position change. We use 7 position changes. Discrete, triangular continuous, sinusoidal (Figure 1), and 4 interpolations of discrete with the triangular (2 interpolations) and the sinusoidal (2 interpolations more) Figure 1.-Discrete variation (left), triangular continuous variation (center) and sinusoidal variation (right) types of spatial figures 2) temporal patterns: we define a set of temporal patterns with left-right different position in time; accelerated and irregular (Figure 2), and regular (Figure1). These patterns are applied with each variation type. 4

5 Figure 2.-Discrete variation type with accelerated temporal pattern (left) and with irrgular temporal pattern (right) It would be interesting to apply different speed transformations (figure 3) of the spatial patterns, in order to test the recognition of the patterns with those transformations. Also, ASA predicts different results depending on the speed of the iteration in the segregation of streams. But to avoid a surge in the combinations to be tested, we skip these transformations for a later study, and only consider 3 speed factors. Figure 3.-Sinusoidal variation type with slow speed factor (left) and with fast speed factor (right) Out of context listening So with these: 5 sources (S), 7 types (T), 3 patterns (P), 3 speed factors (Sf); we create 18 sounds blocking (x) each time 3 of all the 4 variables. These sounds are presented by pairs (with one repeated control sound pair); and listeners must mark the similarity with a yes-no test, and rate the security of the answer in a 1-6 degree scale. With the results we will be able to know the influence of velocity, if the pattern can be abstracted from the source or velocity; and the differences between the discrete and continuous types, and their interpolations. Contextual listening This second phase of the experiment tests the actual musical perception of space. For that matter, we build a musical sequence where the sounds that were build previously appear in temporal order, with some overlappings to test if we can process counterpoint uses of space. The goal of contextual listening is to understand if the space patterns that were recognized when presented with pairs of sounds, are still relevant when the subject has a complex sequence and has to decide what is meaningful. Indeed, we may prove that subjects can recognize spatial patterns in an abstract experiment, but this may prove to be irrelevant when listening to a musical sequence. Also we must be aware of the increased complexity of contextual listening with complex sequences of space. Denis Smalley [18] and Stevenson [19] point out the problems of spatial superimposition due to the complexities of room acoustics which is somehow reduced when using headphones, although superimposition of sounds will always create difficult conditions for spatial sequences recognition: this problem must be tackled and not avoided. It is essential that all the difficulties of hearing spatial motives in a complex sequence be studied, as this is essential in music composition: space will not be a real element of musical language (with the conditions that we have set) if the perception fades in favour of more salient features as pitch melodies and timbre sequences. CONCLUSIONS Our paper presents an experimental design aimed at delimiting the usability of space as a musical language parameter. We restrict the conditions to a set of space parameters that are a subset of all the possible parameters in order to avoid an explosion of combinations. Using the words of Cheng and Wakefield [15], it is sometimes easy for the authors, as scientists trained in inquiry for inquiry s sake, to forget about music for music s sake. Science and technology may be art forms in themselves, but we must not forget that in the ideal case, electroacoustic music should not have to exist simply to express the intricate technology behind it. In our case, although we follow most of the experimental design common to psychoacoustic experiments, the most salient feature of our design is to use spatial sequences instead of isolated sounds, and going further, contextual listening of those spatial sequences in a complex musical 5

6 combination. We claim that this contextual listening is paramount to understand the actual musical possibilities of space, as many features that are discernible out of context may loose their salience when combined with other features that are known to be more salient and robust: pitch and rhythm. Our experiments will tell whether spatialization of sound is a musical resource, and to what extent it is irrelevant in complex musical constructs, which could explain the lack of musical works that do use extensively and systematically space features, as maybe as Donato [2] claimed there is a large gap in understanding spatial hearing in the context of music language, and this lack of understanding might explain why composers cannot think the space and thus compose the space. References: [1] J. Blauert, Spatial Hearing. The Psychophysics of Human Sound Localization. MIT Press, Massachusetts (1997) [2] F. Donato et al., L interprétation des oeuvres acousmatiques. In Table ronde organisée par Thélème Contemporain. Futura (1996) [3] S. McAdams, Psychological constraints on form-bearing dimensions in music. Contemporary Music Review, (1989) [4] B. Merlier, Réflexions à propos de la mise en espace de la musique électroacoustique dans les logiciels audionumériques. JIM (2005) [5] JM. Dûchenne, Des outils pour composer l espace. JIM (2005) [6] L. Austin, Sound Diffusion in Composition and Perfomance Practice II: An Interview with Ambrose Field. Computer Music Journal, 25:4 (2001) [7] A. Vande Gorne, L interprétation spatiale. Essai de formalisation méthodologique. Revue DEMéter. (December 2002) [8] B. Merlier, A la conquête de l espace; (2004) [9] J. Arnau, Espacios para la música. Nausícaä. Spain (2005) [10] A.S. Bregman, Auditory Scene Analysis: The perceptual organisation of sound. MIT press, 2nd ed [11] B. Payri, Auditory Scene Analysis and Sound Source Coherence as a frame for the perceptual study of electroacoustic music language. EMS07 Electroacoustic Music Studies 4th International Conference Series (2007) [12] B. Payri, JL. Miralles Bono, R. Picó, J. Redondo. Modifications in timbre perception depending on listening context ISMA07. International Symposium on Musical Acoustics (2007) [13] B.S. Nelson, P.K. Stoddard. Accuracy of auditory distance and azimuth perception by a passerine bird in natural habitat. Animal Behaviour 56, (1998) [14] M. Trochimczyk. From Circles to Nets: On the Signification of Spatial Sound Imagery in New Music. Computer Music Journal, 25, 4 (2001), [15] C.I. Cheng, G.H. Wakefield. Moving Sound Source Synthesis for Binaural Electroacoustic Music using Interpolated Head-Realted Transfer Functions (HRTFs) Computer Music Journal, 25, 4 (2001), [16] J. M. Grey, J. W. Gordon: Perceptual effects of spectral modifications on musical timbres. Journal of the Acoustical Society of America 63 (1978) [17] S. McAdams, S. Winsberg, S. Donnadieu, G. De Soete, J. Krimphoff. Perceptual scaling of synthesized musical timbres: common dimensions, specificities, and latent subject classes. Psychological Research, 58, (1995) [18] D. Smalley, The Listening Imagination: Listening in the Electroacoustic Era. Contemporary Music Review 13, 2 (1997) [19] I. Stevenson. A dialectic of audible space. Headwize (1997) [20] S. T. Pope,. About This Issue. Computer Music Journal 19, 4 (1995) 1. 6