CTP431- Music and Audio Computing Musical Acoustics Graduate School of Culture Technology KAIST Juhan Nam 1
Outlines What is sound? Physical view Psychoacoustic view Sound generation Wave equation Wave properties Sound perception Ears and auditory system Properties of sounds Amplitude and loudness Frequency and pitch Waveform and timbre 2
What Is Sound? Vibration of air that you can hear Change (or vibration) of air pressure that we can hear with ears Generation Propagation Perception Vibration on materials (e.g. string, pipe, membrane) Traveling via the air Sensation of the air vibration through ears Physical Psychological https://www.youtube.com/watch?v=yydxaurxyaa http://www.kidsent.com/pediatricent/ear_infections/index.html 3
Sound Generation Governed by Newton s law of motion and Wave properties of sound Sound generation on musical instruments (or vibrating objects) 1. Drive force on a sound object 2. Vibration by restoration force 3. Propagation 4. Reflection 5. Superposition 6. Standing Wave (modes): generate a tone 7. Radiation from the object 4
Sound Generation One-dimensional ideal vibrating string String instrument: piano, guitar and harp Wave Equation c = K ε (string tension) (linear mass density) K 2 y x = ε 2 y 2 t 2 Boundary Conditions Fixed or open ends y 0,0 = 0 y L, 0 = 0 Initial Conditions Plucking, Striking, Solution y x, t is the resulting vibration 5
Sound Generation https://www.youtube.com/watch?v=_x72on6csl0 6
Wave Propagation Explained by wave equation on the vibrating string K 2 y x = ε 2 y 2 t 2 General Solution: y(x,t) = y r (t x / c)+ y l (t + x / c) The general solution means that any left-traveling wave, any right-traveling wave and the sum of the two satisfy the wave equation. 7
Wave Propagation Why they are left-traveling and right-traveling waves? Right-traveling case y(x,t) = y r (t x / c) y(x,t + Δt) = y r (t + Δt x / c) = y r (t (x cδt) / c) = y r (t (x Δx) / c) = y r (x Δx,t) Δx = cδt Wave propagation 8
Wave Reflection Explained by the boundary conditions Displacement = 0 at the boundary y(0,t) = y r (t)+ y l (t) = 0 y r (t) = y l (t) f (0,t) = f r (t)+ f l (t)= Rv r (t) Rv l (t)= 0 y r (t) = y l (t) Force = 0 at the boundary Animation: http://www.acs.psu.edu/drussell/demos/reflect/reflect.html 9
Wave Superposition Two traveling waves can cancel or reinforce each other. Depends on their relative phases and frequencies Constructive interference Destructive interference Constructive interference with different shapes Animation: http://www.acs.psu.edu/drussell/demos/superposition/superposition.html 10
Standing Wave The sum of two waves travelling in opposite directions with the same frequency creates a stationary oscillation The standing wave forms a mode Node Anti-node Animation: http://www.acs.psu.edu/drussell/demos/swr/swr.html 11
Complex Harmonic Oscillation Combination of modes are determined by the initial conditions The wavelengths of standing waves are determined by the boundary conditions λ = 2L, L, 2L 3, L 2,... f = c 2L, c L, 3c 2L, 2c L,... Plucked String (Initial Conditions) Modes 12
Oscillation in Pipe Analogous to ideal 1-D string Woodwind or brass instrument: flute, clarinet, trumpet Blowing: continuous excitation Longitudinal pressure wave to travel in air column Modes Open-pipe: e.g. flute f = c 2L, 2c 2L, 3c 2L, 4c 2L,... Semi-open pipe: e.g. clarinet f = c 4L, 3c 4L, 5c 4L, 7c 4L,... 13
Oscillation in Membrane 2-D wave equation: y x, y, t Drum, percussion Boundary condition: by the shape of membrane Circular harmonic oscillation (generate inharmonic tone) 14
Sound Visualization Demos high-speed camera https://www.youtube.com/watch?v=qxjdgbzqvlc Artifact by camera rolling shutter https://www.youtube.com/watch?v=dk6o5raiaj4 Chladni Plate https://www.youtube.com/watch?v=cgiislmffli https://www.youtube.com/watch?v=wvjagrubf4w Visualization by fire https://www.youtube.com/watch?v=2awbkq2dlre Awesome sound visualization performance http://www.nigelstanford.com/cymatics/ 15
Sound Perception Governed by ears (physiological sense) and brain (cognitive sense) human auditory system Ears A series of highly sensitive transducers Three parts Outer, middle and inner ears Transform sound into subband signals Brain Segregate and organize the auditory stimulus Recognize loudness, pitch and timbre Air Mechanical Fluid Electric (Cook, 1999) 16
Outer Ear Pinnae Collect sounds http://www.douglas-self.com/museum/comms/ear/ear.htm Related to recognize the direction of sound c.f. Head-related transfer function (HRTF) Auditory canal Protect ear drums Quarter-wave resonance: boost the vibration around 3kHz by 15-20 db Ear drum Membrane that transduces air vibration to mechanical vibration Malleus (hammer) is attached to it 17
Middle Ear Ossicles malleus (hammer), incus (anvil) and stapes(stirrup) The smallest bones in human body Impedance matching: between air pressure (outer) and fluid (inner) Without ossicles, only about 1/30 of the sound energy would have been transferred to inner ears Amplification Work as a lever: membrane size changes from the large (ear drum) to the small (oval windows) Muscles Reduce the sound transmission in response to loud sounds 18
Inner ears Cochlea: transduces fluid vibration to nerve firing Basilar membrane Fluctuate at different positions selectively according to the frequency of incoming vibration Similar to a bank of band-pass filters http://acousticslab.org/psychoacoustics/pmfiles/module03a.htm Frequency resolution becomes worse as frequency increases Organ of Corti One row of inner hair-cell: fire neural spikes Three rows of outer hair-cell: gain control Oval window Round window 19
Auditory Transduction Video Auditory Transduction http://www.youtube.com/watch?v=petrigtenoc 20
Sound Properties Amplitude Loudness Frequency Pitch Waveform Timbre Physical Psychological 21
Amplitude and Loudness Sound Pressure Level (SPL) Objective measure of sound amplitude Velocity Amplitude of musical sounds in MIDI 0-127 (128 steps) SPL meter Dynamics Amplitude of musical sounds in music score 8 degrees (ppp, pp, p, mp, mf, f, ff, fff) Source: http://www.audioholics.com/home-theater-connection/basic-home-theater-setup-guide/splmeter500x332.jpg/image_view_fullscreen Source: https://en.wikipedia.org/wiki/dynamics_(music) 22
Amplitude and Loudness Loudness is perceptual correlate of sound intensity Log-scale is natural to human SPL has decibel unit 20 log 10 (P / P 0 ) P 0 = 20µPa : threshold of human hearing Loudness is proportional to SPL but not exactly Equal-Loudness Curve Most sensitive to 2-5KHz tones Threshold of hearing Equal-Loudness Curve (also called Fetcher-Munson Curve) 23
Frequency and Pitch Pitch Defined as auditory attribute of sound according to which sounds can be ordered on a scale from low and high (ANSI, 1994) One way of measuring pitch is finding the frequency of a sine wave that is matched to the target sound in a psychophysical experiment thus, subject to individual persons: e.g. tone-deaf Fundamental Frequency Physical attribute of sounds measured from periodicity Often called F0 Pitch should be discriminated from F0: However, in practice, they are exchangeably used. 24
Frequency and Pitch Pitch Scale Human ears are sensitive to frequency changes in a log scale Ex) Piano note scale frequency Hz 4000 3500 3000 2500 2000 1500 1000 500 Frequency Range of Hearing 20 to 20kHz 0 120 10 20 30 40 50 time [second] Chromatic Scale of Piano notes (Linear Frequency) 100 MIDI note number 80 60 40 20 10 20 30 40 50 time [second] Chromatic Scale of Piano notes (Log Frequency) 25
Waveform and Timbre Definition of Timbre Attribute of sensation in terms of which a listener can judge that two sounds having the same loudness and pitch are dissimilar (ANSI) Tone color or quality that defines a particular sound Associated with classifying or identifying sound sources Class: piano, guitar, singing voice, engine sound Identity: Steinway Model D, Fender Stratocaster, Michael Jackson, Harley Davisson 26
Waveform and Timbre Determined by multiple physical attributes Time envelope (ADSR) Spectral envelope (or formant) Changes of spectral envelope and fundamental frequency Harmonicity: ratio between tonal and noise-like characteristics The onset of a sound differing notably from the sustained vibration Inharmonicity ADSR Inharmonicity (Vibraphone) Changes of spectral envelope 27
Semantic Description of Timbre Verbally describe different characteristics of timbre using words Dull Brilliant Cold Warm Pure Rich (Pratt and Doak, 1976) Dull Sharp Compact Scattered Full Empty Colorful Colorless (von Bismark, 1974) (T. Rossing s music150 slides) 28
Timbre Space Perceptual multi-dimensional attributes based on measuring similarity Ask human to listen a pair of sounds and judge the degree of similarity as a score The similarity matrix is processed using multidimensional scaling (MDS), a dimensionality reduction algorithm which determines the timbre space Acoustic correlation with the three (reduced) dimensions Spectral energy distribution Attack and decay time Amount of inharmonic sound in the attack (Grey, 1977) 29
Waveform and Timbre Determined by a number of parameters Perspective of sound synthesis Source: http://www.matrixsynth.com/2011/05/kid-with-buchla.html 30
References Andy Farnell, Designing sound Tom Rossing, The science of sound John R. Pierce, The science of musical sound Julius O. Smith, Physical audio signal processing Perry R. Cook, Real Sound Synthesis for Interactive Applications 31