International Symposium on Performance Science ISBN 978-2-9601378-0-4 The Author 2013, Published by the AEC All rights reserved Piano touch, timbre, ecological psychology, and cross-modal interference Richard Parncutt Centre for Systematic Musicology, University of Graz, Austria The piano has a wide timbral range, and performance quality is often judged in timbral terms. Yet, despite decades of research, there are still fundamental disagreements about the nature and origin of piano touch. Scientists (acousticians) maintain that the timbre of a single tone cannot be varied independently of its loudness. Performers, humanities scholars, and concert audiences take the opposite for granted: timbre and loudness can be independently varied by gestural means. Both sides are right, but their implicit definitions of timbre differ, and both fail to clearly distinguish between physical measures and descriptions of subjective experience. Scientists assume that timbre depends only on physical sound parameters, but experiential parameters generally depend on concurrent input from other senses, the listener s relevant knowledge and expectations, and immediately preceding and following events. The paradox of timbre disappears if we accept, based on empirical evidence, that timbre generally depends on input from more than one sensory modality (weak synesthesia). Embodied corporality and conceptual metaphors are the norm, not the exception. Gestural and ecological approaches to timbre perception pose existential challenges to disembodied cognitive orientations. Keywords: timbre; piano; touch; ecology; synesthesia The piano has a remarkable timbral range. Piano timbre depends strongly on pitch (higher is brighter) and loudness (louder is brighter) in ways that are unique to the instrument. Piano timbre is also affected by complex physical interactions (among strings, soundboard, internal resonances) and perceptual interactions (among sensations and emotions), which in turn depend on timing, dynamics and pedaling.
764 WWW.PERFORMANCESCIENCE.ORG The way a pianist strikes a key ( touch ) seems to influence timbre, even when loudness (key velocity) is constant. In fact, the hammer hits the string in free flight, so any such effect must be tiny; empirical studies suggest that if such an effect exists, it is inaudible. We can sometimes hear fingertips hitting keys ( touch precursor or early noise in staccato touch; Goebl et al. 2004) but that is a small and probably negligible aspect of touch. Weak synesthesia MAIN CONTRIBUTION To account for the richness of piano timbre and demystify piano touch, we need an ecological, multimodal concept that acknowledges the role of vision, proprioception, the somatic sense, and gesture. Weak synesthesia, or crossmodal interference, occurs when perceptual input in one modality (seeing, hearing, tasting, and so on) influences perceptual judgments in another. Weak synesthesia can occur in all perceptual modalities (Martino and Marks 2001) and is probably innate (Walker et al. 2010). Although we physically pick up information via different sensory modalities, and within each modality there are separable sensations (e.g. pitch, timbre), we cannot completely separate modalities or sensations. The reason is that ecological and evolutionary: sounds are only interesting (and consciously perceived) if they carry information about environmental interaction that could affect survival or reproduction. We tend to perceive environmental objects holistically, focusing on their affordances what we can do with them (Gibson 1979). Both sport and music performance can benefit when attention is directed to the effects of movements (external focus of attention: distal stimulus) rather than the movements themselves (internal focus of attention: proximal stimulus; Wulf and Prinz 2001). Golf players make faster progress when their attention is directed to the ball and its goal rather than body movements. Pianists can be more successful if they concentrate on sound rather than technique. The sophisticated motor control mechanisms that regulate our movements are largely unconscious, which allows us to focus on external goals. But the perception of a motoric goal cannot be separated from proprioception (kinesthesia) perception of the relative position of body parts and corresponding muscular effort. Similarly, pianists perception of timbre cannot be separated from their perception of the gestures used to achieve it. Research on audiovisual mirror neurons (Kohler et al. 2002) and auditorymotor interactions (Zatorre et al. 2007) further implies that listeners at a piano recital share the performer s proprioception.
INTERNATIONAL SYMPOSIUM ON PERFORMANCE SCIENCE 765 Cognition is embodied when it is deeply dependent upon features of the physical body of an agent (Wilson and Foglia 2011) a central issue in music psychology (Leman 2008). Understanding sound via the body is an example of conceptual metaphor: ideas in one domain are understood in terms of ideas in another (Lakoff and Johnson 1980). Weak synesthesia in perception and performance The feel of the piano keys under the fingers of a concert pianist can change with the hall acoustics, even if the piano is identical (Brendel 1976). Pianists are highly sensitive to the touch-sound relationship; that is a major aspect of their art. For a pianist, the sense of touch the sense of the keys under the fingers is generally inseparable from the produced sound. There are many examples of weak synesthesia in music. For example, musical pitch is understood in spatial terms: it rises and falls. The timbre of a jazz voice is compared with familiar environmental objects (e.g. round ) or the body or the singer (e.g. relaxed; Prem and Parncutt 2008). Unlike pianists, wind and string players have a high degree of independent control over the exact pitch and timbre of individual tones. But even the best performers do not clearly separate intonation from timbre (Ely 1992, Platt and Racine 1985); musicians who play with good timbre are judged by experts and amateurs to have good intonation and vice versa. Another example: in the best performances of Renaissance choral music, intonation is close to 12-tone equal temperament (Devaney et al 2011), perhaps because 12ET offers an optimal compromise between the clarity of Pythagorean tuning (in which scale steps are clear and stable) and just tuning (in which roughness and beats are minimized). The special feel of just intonation as idealized by Renaissance music aficionados (Duffin 2007) may be a timbral illusion another example of synesthesia. Ecological psychology is also relevant for musicology and aesthetics. An example: acousmatic music is abstract, electronically synthesized sound heard from loudspeakers. Listeners constantly guess and imagine sources or causes of musical sounds just as we do in everyday life when we hear a sound that could be important. Electronically generated sounds sound less strange and more musical when we notice their similarity to familiar sounds and imagine their sources. [T]he acousmatic curtain does not merely serve to obscure the sources of sounds. Indeed, it can be seen to intensify our search for intelligible sources, for likely causal events (Windsor 2000, p. 31). Modern approaches to music theory and the psychology of musical structure have been disembodied by cognitive epistemologies, ignoring envi-
766 WWW.PERFORMANCESCIENCE.ORG ronmental interaction. An ecological approach might start instead with an empirical study of the relationship between physics and experience, as we perceive complex tones in real speech and music (Terhardt 1984). Redefining timbre These diverse examples suggest that weak synesthesia is the rule rather than the exception. Anything that we experience in any sensory modality can be influenced by any other modality. If that is true, we need a new, explicitly ecological definition of timbre. Timbre depends generally on input from other senses (weak synesthesia) not to mention the listener s relevant knowledge and expectations, as well as immediately preceding and following events. These dependencies are not errors they are intrinsic to timbre. Discussion of musical timbre often begins by apologizing for current definitions. A more appropriate definition might include the following: (1) Like pitch, loudness, and (in vision) color, timbre is purely experiential. It has no physical existence, but corresponds to physical states and events. (2) Timbre is a holistic property of a sound source or auditory image that can depend on concurrent input from all relevant senses: hearing, vision, touch, gesture perception. Our ability to consciously separate sensory inputs is limited. Timbre often depends on feelings in the body while performing, or an audience s projections of those feelings. Timbre can also be affected by acoustic or other aspects of a listening space, emotional reactions, and associations with other music or events. (3) A complete description of timbre includes quantitative and qualitative elements. Both are indispensable, and both are intrinsically vague and intangible. From a quantitative viewpoint, timbre is multidimensional; the axis labels are part of timbre s qualitative description. More generally, timbre descriptions refer to the physical environment and the human body, including speech (Traube 2004). (4) Like loudness, timbre is a mixture of sound quality (proximal perception) and sound source quality (distal perception). Psychoacousticians traditionally study proximal loudness and timbre in experiments with artificial sounds heard on headphones, and then consider neural foundations. But in everyday life and music, loudness and timbre usually refer to sound sources not sound as sensation. Timbre generally depends on imagined visual and tactile properties of sound sources, and the listener s past experience of those sources. An example: the temporal and spectral characteristics of the clarinet sound vary enormously from one register to an-
INTERNATIONAL SYMPOSIUM ON PERFORMANCE SCIENCE 767 other, but we still recognize the sound as belonging to one category called clarinet. That in turn suggests that our experience of timbre is also generally influenced by spontaneous, learned categorizations. IMPLICATIONS The long-standing failure of scientists and musicians to agree about touch in piano music may be part of a broader failure to come up with a realistic operational definition of timbre. This failure is inhibiting interdisciplinary interaction. One solution might be to inform both sides about basics of experiential psychophysics and ecological psychology. Another might be to agree on a new definition of timbre. Scientists may be the most resistant to change, given the current dominance of the philosophical worldview known as materialism, according to which the only things that exist are matter and energy as defined by physicists. But with that worldview it is impossible to study artistic experience. You cannot study something that does not exist. The impression that piano timbre depends on gesture, arm weight, and touch is valid if we accept that experiences exist in their own right, and are generally multimodal. But we must also agree that in the physical world, the spectral and temporal envelopes of an isolated piano tone cannot be changed independently of physical intensity. A rational discussion of the relationship between physics and experience will become possible when both sides agree that the previous sentences are complementary and not contradictory. A new interdisciplinary platform will enable more effective and realistic investigations of musical interpretation in practice, on the basis of subjective interactions among performers proximal sensations (tactile, auditory, visual, proprioceptive) and distal perception and cognition (performance space, communication with the audience, cultural context). Address for correspondence Richard Parncutt, Centre for Systematic Musicology, University of Graz, Merangasse 70, Graz 8010, Austria; Email: parncutt@uni-graz.at References Brendel A. (1976). Musical Thoughts & Afterthoughts. London: Robson. Devaney J., Mandel M. I., Ellis D.P.W. et al. (2011). Automatically extracting performance data from recordings of trained singers. Psychomusicology, 21, pp. 108 136. Duffin R. W. (2007). How Equal Temperament Ruined Harmony. New York: Norton.
768 WWW.PERFORMANCESCIENCE.ORG Ely M. C. (1992). Effects of timbre on college woodwind players intonation performance and perception. Journal of Research in Music Education, 40, pp. 158-167. Gibson J. J. (1979). The Ecological Approach to Visual Perception. Boston: Houghton Mifflin. Goebl W., Bresin R. and Galembo A. (2004). Once again: The perception of piano touch and tone. Proceedings of the International Symposium on Musical Acoustics. Nara, Japan. Kohler E., Keysers C., Umiltà M. A. et al. (2002). Hearing sounds, understanding actions: Action representation in mirror neurons. Science, 297, pp. 846-848. Lakoff G. and Johnson M. (1980). Metaphors We Live By. Chicago: University of Chicago Press. Leman M. (2008). Embodied Music Cognition and Mediation Technology. Cambridge, Massachusetts, USA: MIT Press. Martino G. and Marks L. E. (2001). Synesthesia: Strong and weak. Current Directions in Pyschological Science, 10, pp. 61-65. Platt J. R. and Racine R. J. (1985). Effect of frequency, timbre, experience, and feedback on musical tuning skills. Perception & Psychophysics, 38, pp. 543-553. Prem D. and Parncutt R. (2008). Corporality in the timbre vocabulary of professional female jazz vocalists. In M. M. Marin, M. Knoche, and R. Parncutt (eds.), Proceedings of the First International Conference of Students of Systematic Musicology. Graz, Austria. Terhardt E. (1984). The concept of musical consonance: A link between music and psychoacoustics. Music Perception, 1, pp. 276-295. Traube C. (2004). An Interdisciplinary Study of the Timbre of the Classical Guitar. Unpublished doctoral thesis, McGill University, Montreal. Walker P., Bremmer J. G., Mason U. et al. (2010). Preverbal infants sensitivity to synaesthetic cross-modality correspondences. Psychological Science, 21, pp. 21-25. Wilson R. A. and Foglia L. (2011). Embodied cognition. In E. N. Zalta (ed.), Stanford Encyclopedia of Philosophy, accessed at http://plato.stanford.edu/archives/ fall2011/entries/embodied-cognition/. Windsor L. (2000). Beyond the acousmatic: The interpretation of electronic sounds. In S. Emmerson (ed.), Music, Electronic Media and Culture (pp. 7-35). Aldershot, UK: Ashgate. Wulf G., and Prinz W. (2001). Directing attention to movement enhances learning: A review. Psychonomic Bulletin and Review, 8, pp. 648-660. Zatorre R. J., Chen J. L., and Penhune V. B. (2007). When the brain plays music: Auditory motor interactions in music perception and production. Nature Reviews Neuroscience, 8, pp. 547-558.