EVTA SESSION HELSINKI JUNE 06 10, 2012 Reading Spectrograms FINATS Department of Communication and Arts University of Aveiro Campus Universitário de Santiago 3810-193 Aveiro Portugal ipa Lã (PhD) Department of Communication and Arts University of Aveiro, INET-MD - Portugal filipa.la@ua.pt
What information does a spectrum display reveals? 2/ 60
UNDERLYING INFORMATION Any musical sound is characterised by four interrelated properties Duration Frequency (objective measure of pitch) Amplitude (objective e measure e of loudness) Spectral envelope (objective measure of timbre) 3/ 60
SOUND WHAT IS IT? Duration: the periodicity of a musical sound needs adequate duration to be perceived by the brain as pitch Brain areas invlolced in music perception and memorization [adapted from http://images.google.pt/imgres?imgurl, in 02/05/2010] 4/ 60
UNDERLYING INFORMATION Frequency: resulting from periodicity; distinguishes noise from musical sounds Period (time) 5/ 60
UNDERLYING INFORMATION Amplitude: magnitude of the compressions within a sound wave Amplitude (magnitude of compression) Frequency (measuring time) 6/ 60
SOUND WHAT IS IT? Spectral envelope: assessing the timbre (i.e. tone color) of a given sound Pure tone (adapted from McCoy, 2004) 7/ 60
UNDERLYING INFORMATION Musical sounds: result from many simultaneously-occurring, interrelated frequencies at different amplitudes Am plitude [db] Harmonics = additional vibratory frequencies that are whole-number multiples of F0 Frequency [Hz] Spectral envelop of a cello (power- spectrum) (adapted from McCoy, 2004) 8/ 60
UNDERLYING INFORMATION The human voice: source/ filter theory The glottal sound, before being filtered by the vocal tract resonances, have harmonics which diminish in amplitude as they increase in frequency Harmonics do not diminish at a constant rate as they increase in frequency: some are strongly amplified whereas others are dampened SOURCE FILTER THEORY: Vocal tract subareas (A); Primary sound spectrum (B) modified by the resonances of the vocal tract (C). The result is a radiated sound spectrum that has certain partials enhanced and other dampened (adapted from Urrutia & González, 1996: 80) 9/ 60
UNDERLYING INFORMATION The human voice: primary sound VOICE SOURCE SPECTRUM Sound level ([10 db/ /division n] Fundamental Octave 1 Fifth Octave 2 Major Third Fifth Septime Octave 3 Frequency [partial number] 10/ 60
UNDERLYING INFORMATION The human voice: modified sound Velar narrowi ng Pharynx Epilarynx Glottis Gotts Oral cavity Subareas of the vocal tract A formant is a resonance of the vocal tract Lips F1 F2 F3 F4 Vocal tract A resonator with its own resonance frequencies (formants) F1,2,3,4, 5 = (V/L) x ¼ (V/L) x ¾ (V/L) x 5/4 (V/L) x 7/4 (V/L) V = 350 ms (sound velocity) L = tube length 11/ 60
UNDERLYING INFORMATION Formants of the human voice: equivalent to the amplifier and tone controls in a stereo Frequency Measured at the centre, top of the peak Provide vowel accuracy y( (F1 and F2) Provide individual timbre and may assist in vocal projection (F3, F4 and F5) Formant s frequency peak (adapted from McCoy, 2004: 41) Bandwith Measures the with of the peak at a specific distance from the top (eg. 10dB) An harmonic near the vicinity of a formant will be amplified 12/ 60
UNDERLYING INFORMATION The human voice: unique musical instrumental in which articulation also affects resonance Formant frequencies change when altering the shape of the vocal tract through articulation i A constriction in the front of the vocal tract (e.g. palatal moving of the tongue) lowers F1 and raises F2 A constriction in the back of the vocal tract t (e.g. pharyngeal movement of the tongue) raises F1 and lowers F2 Tongue position captured by magnetic resonance for different vowels showing the effects on vocal tract constrictions (adapted from Lindblom & Sundberg, 2007: 683) 13/ 60
UNDERLYING INFORMATION All formant frequencies lower uniformly when the vocal tract is lengthened (e.g. lowering the larynx and protruding the lips) All formant frequencies rise uniformly when the vocal tract is shortened (e.g. raising the larynx and spreading the lips) Vowel /a/ spoken and sung Am mplitude [db] Frequency [KHz] 14/ 60
UNDERLYING INFORMATION Larynx height increases with increasing pitch for a soprano t [mm] Laryn nx heigh Resting position Fundamental frequency [Hz] Change in larynx position with raising F0 (adapted from Sundberg, 1987) 15/ 60
UNDERLYING INFORMATION Lowering the jaw (opening the mouth) increases F1 This is needed to avoid F0 > F1 (which causes instabilities in the vibration i of the vocal folds) Vowel [i] Vowel [u] Visualisation of jaw opening for vowels /i/ and /u/ with raising F0 (adapted from Sundberg, 1987) 16/ 60
UNDERLYING INFORMATION Jaw op pening [m mm] Pitc ch Jaw opening during two octave scale Time [s] Jaw opening during a two octave scale singing (Courtesy from Sundberg, 2010: Summer course) 17/ 60
UNDERLYING INFORMATION High voices need to open their jaw as F0 increase Ja aw open ning [mm m] Spok en S u n g Fundamental frequency [Hz] Jaw opening with increasing F0 (Courtesy from Sundberg, 2010: Summer course) Opening of the jaw constricts the pharynx and thus raises F1 to track F0 (adapted from Thurman & Welch, 2000: 481) 18/ 60
UNDERLYING INFORMATION Tongue shape also varied Spoken Lower jaw contour Sung Fundamental frequency [Hz] 230 465 940 Jaw opening with increasing F0 (Courtesy from Sundberg, 2010: Summer course) 19/ 60
UNDERLYING INFORMATION The consequence is vowel modification! Vowel accuracy is defined by F1 and F2 Vowel F1, F2 and F3 according with APEX model (adapted from Lindblom & Sundberg, 2007: 686) Vowel chart and corresponding vocal ranges of different singers (adapted from Lindblom & Sundberg, 2007: 686) 20/ 60
UNDERLYING INFORMATION Female and male vocal tract and vocal folds length Different vocal tract lenghts (left) and vocal folds length (right) (adapted from Roers, Mürbe & Sundberg, 2008) VoceVista 3.lnk 21/ 60
Spectrum displays 22/ 60
UNDERLYING INFORMATION Fast Fourier Transform mathematical formula which converts sound into its component parts it is used to perform: power spectrum spectrogram data collected by means of a microphone is: converted from an electric sound to a digital signal (computer sound card or external device) the signal is processed applying FFT analysis results are plotted in a graph displayed d on the computer screen 23/ 60
UNDERLYING INFORMATION Effects of microphone placement placing the microphone near to the mouth increases energy of all partials the effect we see is not related with the student s changing behaviours best to measure distance to the mouth with the student and use the same distance always with that student best to use omnidirectional microphones with a flat response over a wide range of frequency band (e.g. Audio-Technica ATR-3350 Lavalier Omnidirectional Condenser Microphone) 24/ 60
UNDERLYING INFORMATION Power Spectrum Amplitude of the F0 and partials on the vertical axis Frequency along vertical axis It shows sound components during a single moment in time (some ms) Leve l [db] Frequency [Hz] 25/ 60
REAL-TIME FEEDBACK APPLICATIONS IN SINGING LESSONS useful to display formant tuning strategies, although it cannot show location of formants Spectrogram of a baritone voice singing a scale up to G4, vowel [a], with power spectrum at G4 showing different harmonic partials (using Don Miller s VoceVista program, version 3.3) 3) 26/ 60
UNDERLYING INFORMATION Long-Term Average Spectrum (LTAS) Amplitude of the F0 and partials on the vertical axis Frequency along vertical axis It shows sound components during several cycles of vibration; it is therefore an average acoustic output t Am mplitude [db] Vowel /ae/ spoken and sung Frequency [KHz] 27/ 60
UNDERLYING INFORMATION Spectrogram Time along horisontal axis Frequency along vertical axis Dark for formant frequencies of different vowels /a/ ------- ------/i/ -------------- -/u/ ---------- ncy [Hz] Frequen Time [s] 28/60
APPLICABILITY IN TEACHING Different types of spectrograms Frequ uency [H Hz] Wide band Time [s] Narrow band Wide (left) and arrow (right) band spectrograms of a vowel /a/ sung in chest register D4 Wide band: used to display formant frequencies with clarity (because it divides the frequency spectrum into broad swaths) Narrow band: clear visualization of harmonics and vocal elements (because it divides the frequency spectrum into narrow segments) 29/ 60
Applicability of spectrum displays in teaching 30/ 60
APPLICABILITY IN TEACHING Important considerations Spectral displays are easier to understand for male than for female voices (frequency range that allows the display of many harmonics) Spectral displays are quite useful for working with the student in learning to control: Phonation types Vocal timbre (singer s spectrum peak; chiaroscuro tone) Vibrato Legato Staccato (voice onset and offset) Vowel accuracy Synchronicity with accompaniment Acoustical efficiency 31/ 60
APPLICABILITY IN TEACHING Free Paid wavesurfer.exe Spead3.exe VoceVista 3.lnk 32/60
APPLICABILITY IN TEACHING Let us try Wavesurfer! wavesurfer.exe http://www.speech.kth.se/wavesurfer/ 33/ 60
APPLICABILITY IN TEACHING Procedures Open the program Create a pane Enlarge the window size Define type of spectrogram Wide band: FFT = 1024 points Window = 1024 points Cut spectrum at 5000Hz 34/ 60
APPLICABILITY IN TEACHING Define type of spectrogram Narrow band: FFT = 512 points Window = 64 points Analysis window type: rectangle Cut spectrum at 5000Hz 35/ 60
APPLICABILITY IN TEACHING Applicability of wide band spectrograms 36/ 60
APPLICABILITY IN TEACHING 1. Phonation types: determine voice timbre and correspond to different levels of vocal effort can be demonstrated using a synthesizer software (Madde, by Svante Granqvist) Weak Adduction Courtesy of Professor Johan Sundberg Type of phonation Breathy Flow Neutral Strong F0 Firm Pressed Weak F0 37/ 60
APPLICABILITY IN TEACHING Neutral Good morning ladies and gentlemen, we are going to speak about voice Pressed Good morning ladies and gentlemen, we are going to speak about voice Flow Good morning ladies and gentlemen, we are going to speak about voice 38/60
APPLICABILITY IN TEACHING 2. Singer s spectrum peak: the fine art of clustering resonances by male classically trained singers can be displayed in wide band spectrograms In ntendity [db] 10 0-10 -20-30 -40-50 -60 Clustering of F3, F4 & F5-70 0 1000 2000 3000 4000 5000 Frequency [Hz] (Courtesy of Professor Sundberg, Distinguished Lecturer, CIRMMT, 2009) 39/ 60
FEEDBACK EM TEMPO REAL SISTEMA RESSOADOR Singer s spectrum peak: one strategy that male singers use to be heard over a loud accompaniment ntensity [db] Mean i 0-10 -20-30 LTAS of singer and orchestra Orchestra Singer + orchestra 1º - Sound corresponding to the level of an orchestra 2º - Singing i without and with singer s spectrum peak -40 100 1000 10000 Frequency [khz] 3º - First and second examples together (Cortesy from Professor Sundberg, Distinguished Lecturer, CIRMMT, 2009) 40/ 60
APPLICABILITY IN TEACHING Physiologically: lowering of the larynx and thus widening of the pharynx widening of laryngeal ventricle and the sinus piriformes (i.e. Bottom part of the vocal tract surrounding the larynx tube) Comparison between speaking and singing of a male x-ray frontal tracings (adapted from Sundberg 1987: 121) 41/ 60
APPLICABILITY IN TEACHING /u/ vowel sung with singer s spectrum peak /u/ vowel sung without singer s s spectrum peak 42/60
APPLICABILITY IN TEACHING Aspects of language accuracy Intelligible pronunciation of lyrics Correct pronunciation of different languages Accuracy of vowel production in each language 43/ 60
APPLICABILITY IN TEACHING Aspects of language accuracy Intelligible pronunciation of lyrics Correct pronunciation of different languages Accuracy of vowel production in each language [o] [i] [o] [a] [ae] [e] [oe] 44/60
APPLICABILITY IN TEACHING Formant frequencies of vowels Narrow band spectrogram iii aaa ooo uuu 45/ 60
APPLICABILITY IN TEACHING Applicability of narrow band spectrograms 46/ 60
APPLICABILITY IN TEACHING Vibrato: regular variation of F0 the perceived frequency corresponds to the mean F0 variation the mean must fall into a tolerated rate (usually between 5 and 6 Hz) F0 [H Hz] Time [secs] Representation of vibrato (adapted from Lã & Sundberg, 2010: Perceptual evaluations of Summer course Function of the Singing Voice, 2010) 47/ 60
APPLICABILITY IN TEACHING Vibrato: regular variation of F0 Activation of the cricothyroid muscle F0 Frequency [Hz] 48/ 60
APPLICABILITY IN TEACHING Vibrato: regular variation of F0 49/60
APPLICABILITY IN TEACHING Legato: visualised in the spectrogram through the continuity of the signal wavesurfer.exe 50/ 60
APPLICABILITY IN TEACHING Voice onset and offset Hard attack Staccato Breathy attack Adduction Subglottal pressure Adduction first Adduction and subglottal pressure simultaneously l Subglottal pressure first Vocal fold vibration (Adapted from Lã & Sundberg, 2010: Perceptual evaluations of voices- Summer course Function of the Singing Voice, 2010) 51/ 60
APPLICABILITY IN TEACHING Different vocal onsets Aspirated Staccato Hard 52/ 60
APPLICABILITY IN TEACHING Synchrony with accompaniment Realising intended timing of tones Requires perfect control of respiratory apparatus, laryngeal function and vocal tract articulation 53/ 60
APPLICABILITY IN TEACHING Almost perfect synchrony between vowel onset and the piano by Fischer-Dieskau s recording of Dichterliebe by Schumann (adapted from Lindblom & Sundberg, 2007: 694) 54/ 60
APPLICABILITY IN TEACHING READING SPECTRO Fi Applicability of power spectrum 55/ 60
APPLICABILITY IN TEACHING Acoustical efficiency Singers use register equalization to avoid discontinuities in voice quality between registers (e.g. Schutte & Miller, 1990; Titze, 1988;Schutte et al., 2005; Neuman et al., 2005) Previous studies suggest different formant tuning strategies around the male s passaggio for open vowels: Falling of F1 below second partial (H2) (Miller & Schutte, 1994; Hertegård, Gaufin & Sundberg 1990; Neuman et al., 2005) F2 tracking H4 below the passaggio ( chest ) and F2 tracking H3 at the level of the passaggio ( head ) (Neuman et al., 2005) Vocal fry has been used to measure the frequencies of the two lowest formants in relation to the frequencies of the spectrum partials (Miller, 2000) 56/ 60
APPLICABILITY IN TEACHING H1 H2 H3 (Sundberg, Lã & Gill, 2001) Level [db B] Frequency [Hz] el [db] Leve H1 Using a correct tuning strategy for the passaggio note in a baritone (G4 392 Hz) H2 H3 Using an incorrect tuning strategy for the passaggio note (G4 392 Hz) Frequency [Hz] 57/ 60
(Sundberg, Lã & Gill, 2001) Where are the formants? 58/60
APPLICABILITY IN TEACHING Inverse filtering by Decap Flow glottogram Derivative of EGG (deeg) (Sundberg, Lã & Gill, 2001) Criteria for setting the filters: Unfiltered spectrum Time [10 ms] tered spectrum Ripple free closed phase /division] ndwidth Lev vel [10 db Invers se filter ba F 1 F 2 F 3 F 4 F 5 Continuously falling voice source spectrum envelop Synchrony between deeg peak and the flow discontinuity at glottal closure Frequency [Hz] 59/60
APPLICABILITY IN TEACHING VOWEL /a/ 10 Hz] F1 & F2 [se emitones re 1 nes re 110 Hz z] & F2 [semito 42 36 30 24 42 36 30 H7 Singer 1 H6 H5 MV rating 84.6%, SD 7.5% MV rating 12.4%, SD 10.8% H4 H3 6 12 18 24 H7 Singer 2 H6 H5 MV rating 83.0%, SD 7.9% MV rating 25.0%, SD 22.4% H4 H2 H3 H2 (Sundberg, Lã & Gill, 2001) F1 and F2 are typically lower in the Classical than in the Non-classical tuning, especially in the passaggio Formants do not change systematically between scale tones For the top pitches, F1 falls below H2 in the Classical tuning For the top pitches, F2 is just above, just below or right on H3 in Classical tuning 42 H7 H6 H5 H4 36 MV rating 83.1%, SD 8.6% 30 Singer 4 MV rating 32.0%, SD 17.8% H3 H2 F1 24 6 12 18 24 F0 [semitones re 110 Hz] 24 6 12 18 24 F0 [semitones re 110 Hz]