The Neural Code of Pitch and Harmony

Transcription

1 The Neural Code of Pitch and Harmony Harmony is an integral part of our auditory environment. Resonances characterized by harmonic frequency relationships are found throughout the natural world, and harmonic sounds are essential elements of speech, communication and, of course, music. Providing neurophysiological data and theories that are suitable to explain the neural code of pitch and harmony, the author demonstrates that musical pitch is a temporal phenomenon and that musical harmony is a mathematical necessity based on neuronal mechanisms. Moreover, he offers new evidence for the role of an auditory time constant for speech and music perception as well as for similar neuronal processing mechanisms of auditory and brain waves. Successfully relating current neurophysiological results to the ancient ideas of Pythagoras, this unique title will appeal to specialists in the fields of neurophysiology, neuroacoustics, linguistics, behavioural biology and musicology, as well as to a broader audience interested in the neural basis of music perception. Gerald Langner received a diploma in physics from the Technical University of Munich in He then worked at the Max-Plank Institute in Göttingen and at the TU Darmstadt, where he studied hearing in birds and electroreception in fish. In 1985, during a research stay in Canberra, Australia, he discovered together with Henning Scheich the electric sense in platypus. From 1988 to 2008 he was Professor of Neurobiology in Darmstadt, with his research focusing on spatial and temporal aspects of processing in the auditory system.

2

3 The Neural Code of Pitch and Harmony GERALD LANGNER Technische Universität, Darmstadt, Germany EDITED BY CHRISTINA BENSON

4 University Printing House, Cambridge CB2 8BS, United Kingdom Cambridge University Press is part of the University of Cambridge. It furthers the University s mission by disseminating knowledge in the pursuit of education, learning and research at the highest international levels of excellence. Information on this title: / Gerald Langner 2015 This publication is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First published 2015 Printed in the United Kingdom by TJ International Ltd. Padstow Cornwall A catalogue record for this publication is available from the British Library Library of Congress Cataloguing in Publication data Langner, Gerald, 1943 The neural code of pitch and harmony / Gerald Langner, Technische Universität, Darmstadt, Germany ; edited by Christina Benson. pages cm Includes bibliographical references and subject index. ISBN Bioacoustics. 2. Neurobiology. 3. Sound Physiological effect. 4. Sound Psychological aspects. 5. Harmonic analysis (Music) 6. Contentment. I. Benson, Christina, editor. II. Title. QH510.5.L dc ISBN Hardback ISBN Paperback Cambridge University Press has no responsibility for the persistence or accuracy of URLs for external or third-party internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate.

5 Contents Preface Foreword page xi xiii 1 Historical aspects of harmony The origin of music The power of music and harmony Music as a universal language Musical harmony and whole numbers Universal harmony Harmony of the Spheres Harmony in modern astrophysics 8 2 Sound and periodicity Sound is movement The periodicity of sound Nature of periodic sounds Perception of periodic sounds Fourier analysis Sounds of speech Production of speech Perception of speech 22 3 The discovery of the missing fundamental The sound of sirens The pitch quarrel Hermann von Helmholtz A mechanical basis of pitch? The cochlear piano Place and resonance Combination tones and the missing fundamental A mechanical basis of harmony? Helmholtz s influence on music 33

6 vi Contents 4 The pitch puzzle The telephone theory The Residue Revisited The dominance region The pitch shift Spectral coding Temporal coding 43 5 The auditory time constant A quantum effect of pitch shift Pulling effect and absolute pitch The auditory time constant in vowel formants The auditory time constant in Chinese tone language The mystery of flute tuning The auditory time constant in bird calls 57 6 Pathways of hearing From the cochlea to the cortex The ear The receiving system The cochlea The travelling wave The organ of Corti The cochlear amplifier The auditory nerve Spectral coding A limited dynamic range Temporal coding Cochlear nucleus Functional organization The ventral part The dorsal part Olivary nuclei Lateral lemniscus Inferior colliculus Functional organization Tonotopy in the midbrain The fine structure of tonotopy Tonotopic fine structure and critical bands Thalamus, the gateway to the cortex The cortex Tonotopy in the cortex Wernicke s area 84

7 Contents vii Broca s area What- and where-streams A music centre? 86 7 Periodicity coding in the brainstem Periodicity coding in the auditory nerve Temporal coding Intensity effects Population coding and lateral suppression Temporal coding of vowels Periodicity coding in the cochlear nucleus Faithful synchronization Diversity of periodicity coding Bushy cells Octopus cells Stellate cells Dorsal cochlear nucleus Periodicity coding in the midbrain Coding of complex sounds Processing of species-specific vocalizations Neuronal mechanisms of feature detection Synchronization and rate Stimulus parameters and response features Periodicity coding Temporal response patterns Coincidence effect Intrinsic oscillations Best modulation period, intrinsic oscillation and onset latency Theories of periodicity coding Synchronization and harmony The Licklider model The model of Hewitt and Meddis The periodicity model The functional principle The trigger The oscillator The reducer The coincidence neuron Simulations of the periodicity model Simulation of the components Simulation of the synchronization effect Simulation of the BMF shift 138

8 viii Contents 9.6 Pitch effects explained by the periodicity model The missing fundamental The dominance region Pitch-shift effects Absolute and relative pitch Periodotopy Spatial representation of pitch Mapping from time to place Orthogonality of pitch and timbre Mapping the inferior colliculus Electrophysiology Metabolic labelling Pitch neurons Periodotopy and tonotopy: a model Cortical maps Periodotopy in the mynah bird Periodotopy in the gerbil Periodotopy in the cat Periodotopy in the human cortex Above the auditory cortex The neural code of harmony The pitch helix Comb filters in the midbrain Synchronized inhibition The periodicity model, including inhibition The auditory double helix The neural pitch helix Consonance Harmony The oscillating brain Grandmother cell and cocktail-party problem Binding and oscillations A historical note Binding and neuronal correlation analysis Helical brain structures The blue helix The helix for eye movement The helix of emotion The helical web of memory The black helix 193

9 Contents ix 12.4 The mind is the music of the brain Periodicity analysis beyond the auditory system Delay mechanisms and coincidence neurons The top-down control of oscillation frequency A map for periodicity in the striatum? Neuronal space and harmony Is grandmother a frequency composition? 199 References 202 Index 222

10

11 Preface Sound is a vital tool for humans and animals. We communicate with each other through speech, we convey emotion by laughing or crying, but we also purposefully create sounds using our voices or musical instruments just because we perceive them to be appealing or beautiful. The pitch, rhythm and melody of speech and music can communicate emotions like fear, pleasure and anger quite quickly and efficiently. Moreover, as humans we seem to have a powerful urge to fill the world with sounds of our own creation, with the result that these days music surrounds us virtually everywhere. The need to make, listen and dance to music stretches back to the very beginnings of our history: for many thousands of years music has played an essential role in our social interactions, rituals and ceremonies. The sixth-century Roman philosopher and great musical theorist Boethius stated quite simply: it appears beyond doubt that music is so naturally united with us that we cannot be free from it even if desired. We all know that some combinations of musical tones sound particularly good when played together or subsequently; we call these consonant or harmonious, while others sound harsh or dissonant. If asked what combinations of sounds they find pleasant, or at least interesting, people from different cultural backgrounds may not completely agree. Different forms of music prevail in different regions of the world, and musical instruments and composition have become progressively more sophisticated as civilization advances. Nevertheless, there are certain combinations of tones that seem to have universal appeal. They are preferred everywhere and form the basis of musical systems throughout the world. Clearly, there must be some universal rules that are crucial to our perception of musical harmony. The question of what these rules are and what might be the role of whole numbers dates back to the time of the ancient Greeks. They believed that the mathematical rules of musical harmony are the very same that govern the entire universe. Besides neurophysiologic data and theories that are suitable to explain auditory processing of pitch and harmony, this book provides new evidence for this ancient philosophical concept. The conclusion is that our sense for musical harmony is an unavoidable consequence of mathematical rules underlying temporal processing in our hearing

12 xii Preface system. As we progress through this book, theories and models of pitch perception and harmonic perception, both historical and current, will be presented and explained. Finally, in the last chapter I will suggest that neuronal dynamic processes similar to those in the hearing system are involved in other crucial brain functions: motor control, emotion and memory processing. The book is intended not only for neuroscientists and musicologists, but also for a broader audience interested in the perceptual basis of music. Therefore boxes in various chapters contain additional information that may be helpful, although perhaps unnecessary for the specialist. Moreover, in the times of internet it should be quite easy to obtain additional information for those who want to go into details.

13 Foreword Human sensing abilities have been shaped and refined over long evolutionary periods. Hearing has adapted to serve us well in many different tasks and situations, helping us to orient ourselves and to survive in the world. The general properties of peripheral and central mechanisms of hearing are highly conserved across vertebrates due to very similar environmental conditions. Species-specific variations do exist, such as the use of ultrasound for orientation in bats and cetaceans, but they are usually founded on quantitative and not qualitative differences to generally applicable principles of hearing and brain mechanisms. Basic hearing tasks for survival include detecting, localizing and identifying sound sources in cluttered or noisy environments. Another critical role of vertebrate hearing is the control and analysis of communication sounds which, in humans, lead to the highly developed ability of speech production and perception. Speech, like many other sounds involving resonance phenomena, contains harmonic elements, i.e. frequency components that are integer multiples of a common fundamental frequency. These sounds can evoke a perceptual phenomenon, periodicity or virtual pitch that is distinct from other perceptual dimensions of sounds. A most human endeavour, the production and enjoyment of music, is fundamentally based on this perceptual phenomenon. Studies of the brain mechanisms that lead to this perception, its psychophysical manifestation and, eventually, cognitive and emotional benefits have progressed for more than a century, as is outlined in this volume, but still many aspects remain unresolved. A helpful aspect in resolving this matter may be found in the fact that humans have surrounded themselves with an environment of their own creation. Based on our ability to use tools we have created artificial soundscapes that serve, entertain and move us. Unsurprisingly, many of those sound aspects have been, often inadvertently, chosen to match or most effectively engage our biological sound analysis system. Examples include the choice for frequency transitions in ambulance sirens to catch our attention, or the relationship of voices in polyphonic music. Both of these examples can be traced to specific psychophysically verified and physiologically implemented principles of sound processing. Furthermore, instrumental music is a solely human development that emerged early in our evolution to become human, as indicated by the recovery of Palaeolithic flutes created more than years ago. The sound effects emanating from these old and current, electronic artefacts of musical sound generation also must reflect and potentially reveal basic properties of our auditory system.

14 xiv Foreword The author of this book has been fascinated by these aspects for a long time and has tried to create a unifying perspective. In the early 1980s, I joined the Coleman Memorial Laboratory at the University of California in San Francisco, which is dedicated to the study of the physiological basis of hearing and deafness. Shortly thereafter, Dr Gerald Langner arrived for his first of many extended visits to explore sound processing in the central auditory system, especially in the auditory midbrain, an obligatory processing station between the inner ear and the auditory cortex. Over the years we embarked on several studies, especially with regard to the processing of amplitude-modulated sounds, a simplified exemplar of a harmonic sound. As a trained physicist, Gerald was keen to approach biological phenomena from a theory-driven perspective. A theory of pitch processing, understood as a construct of hypotheses based on physical, psychophysical and physiological aspects, should be able to provide verifiable predictions of the processing and role of harmonic coding in animals and humans. I recall many discussions of new data points, derived over long days and nights in the laboratory, in which he invoked his credo: Never trust data, unless you have a theory. In this book the author outlines his conclusions from this lifelong pursuit of potential links between aspects of our neural machinery of pitch processing and their reflection in our self-created sound environment. Drawing on theoretical, computational, physiological, psychophysical and music-historical evidence, he has created a compelling scenario of the properties of some brain mechanisms and their expression in our percepts as well as their reflection in the cultural world we have created around us. He provides a fascinating journey into the history and future of pitch and brain studies and suggests intriguing interactions of fine-scale neural processes in our brain with our cultural history of sound creation. Christoph E. Schreiner San Francisco November 2014