Music, Hearing Loss, and Cochlear Implants The Next Frontier Charles J. Limb, M.D. Francis A. Sooy Professor Chief of Otology, Neurotology and Skull Base Surgery Director, Douglas Grant Cochlear Implant Center Department of Otolaryngology-Head and Neck Surgery and Neurological Surgery University of California San Francisco San Francisco, CA Disclosures Advanced Bionics Corporation Consultant (2006-present) Research support Med-El Corporation Research grants (2012-present) Research support (2016) Oticon Medical Conference chair/consultant, cochlear implants and music (2016) Frequency Therapeutics, Inc. Consultant (2016) 1
2
Treatment for hearing loss has evolved 1800s 2000s Methods Secondary Reconstruction 3
Why music? 4
First musical instrument? Bird bone flute Recovered from southern Germany (Giessenklösterle) ~35,000 years old Snowball gets his groove on Patel, et al., 2009 Current Snowball gets his groove on Patel, et al., 2009 Current 5
Despite success in language perception, most cochlear implant users cannot hear music well. Percentage Correct Doesn t Work For Music! 500-1500 Hz BPF unfiltered stimuli 6
7
What does music sound like for a CI user? In normal hearing, place and rate pitch mechanisms are seamlessly integrated and support one another. In electric hearing, place and rate pitch mechanisms are both disrupted. Pitch perception is the single biggest obstacle for CI-mediated perception of music Relative pitch > absolute pitch Difference between middle C and one semitone above or below ~15 Hz 8
PURE TONES REAL TIME SPECTROGRAM SUPERSTITION - REAL TIME SPECTROGRAM 9
10
Typical CI frequency map covers range of 60+ semitones Rachmaninoff Prelude Op. 3, No. 2 in C# minor: original Rachmaninoff Prelude Op. 3, No. 2 in C# minor: original 11
Rachmaninoff Prelude Op. 3, No. 2 in C# minor: +/- 1 s.t. Rachmaninoff Prelude Op. 3, No. 2 in C# minor: +/- 1 s.t. Caldwell M. et al, 2015. 12
CI Users Utilize Tempo Rather Than Mode to Interpret Musical Emotion Caldwell M. et al, 2015. 13
Caldwell MT et al. 2016 14
CI Users Demonstrate Fusion of Polyphonic Pitch CI Confusion Matrix Presented Identified Cnt Cor Single Interval Class. % Chord 199 63 26 Single 69.1% 21.9% 9.0% 136 84 67 Interval 47.2% 29.2% 23.3% 112 86 89 Chord 38.9% 29.9% 30.9% NH Confusion Matrix Presented Identified Cnt Cor Single Class. % Interval Chord Single 260 24 2 90.3% 8.3% 0.7% Interval 63 174 53 21.9% 60.4% 18.4% Chord 23 122 143 8.0% 42.4% 49.7% Donnelly et al., 2009 15
How do we measure musical sound quality in CI users? Musical Sound Quality is Poor in CI Users CI users subjectively report poorer musical sound quality following implantation (Gfeller et al 2002; Lassaletta et al 2008) Sound quality traditionally assessed via questionnaires or rating scales (Lassaletta et al 2008; Gfeller et al 2008; Looi et al 2008, 2011) Assessment of sound quality can be used as a tool, not an indicator of preference or enjoyment 16
17
Cochlear Implant-MUltiple Stimulus with Hidden Reference and Anchor (CI-MUSHRA) 25 full quality musical stimuli are increasingly degraded: Example: Removal of bass frequencies Hidden reference (best sound quality) No Alterations 200 Hz HPF 400 Hz HPF 600 Hz HPF 800 Hz HPF 1000 Hz HPF Anchor (1000-1200Hz band-pass filter) Highly Altered 18
MUSHRA: ITU-R Recommendation BS.1534 For evaluation of lossy audio compression algorithms CI-MUSHRA 25 excerpts, from 5 genres (pop/rock, country, jazz, classical, hip hop) Roy et al., 2012 Roy et al., 2012 19
How do we improve music for CI users? Change the CI Change the brain Change the music Can we tune cochlear implants? 20
FPCT distinguishes all 216 individual electrode contacts Jiam NT et al. Otol Neurotol 2016 (In press) Frequency-place mismatch increases in apical and basal electrodes Jiam NT et al. Otol Neurotol 2016 (In press) 21
Kretzmer, E. A. et al. Arch Otolaryngol Head Neck Surg 2004;130:499-508. A radiograph of a cat with a cochlear implant shows the 6 electrode contacts (electrode 1 at white arrowhead) within the cochlea Kretzmer, E. A. et al. Arch Otolaryngol Head Neck Surg 2004;130:499-508. Courtesy of the Ryugo laboratory, Center for Hearing Sciences 22
PET is quiet, non-magnetic Parsons LM, 2001 Positron Emission Tomography H2-[15]O molecules Positron-emitting [15]O atom Positron Emission Tomography 23
Is CI listening similar to normal listening? Normal listeners CI listeners 10 postlingual CI users / 10 controls Limb et al., 2010 24
Roy et al., 2014 25
Conclusions Music is the pinnacle of hearing Critical impairments in pitch, timbre and sound quality are observed in CI users These impairments reveal limitations of CI processing that speech testing does not Cochlear implants are severely out of tune We must work to improve CI designs, processing strategy, musical training, and even the music itself Thank You Johns Hopkins University School of Medicine The Dana Foundation NIDCD/NIH The Brain Science Institute National Endowment for the Arts Peabody Conservatory of Music University of California San Francisco Baltimore Symphony Orchestra San Francisco Jazz Center San Francisco Conservatory of Music Advanced Bionics Corporation Cochlear Corporation Med El Corporation Oticon 26
Thank You Judy Doong Alice He Nicole Jiam Tina Munjal Meredith Caldwell Patpong Jiradejvong Joseph Heng Mickael Deroche Gabe Donnay Stephen Dunlap Diane Hwang Irene Kim Matthew Sachs Karen Barrett Chan Nicholas Ryugo Gabriela Cantarero Monica Lopez-Gonzalez Lindsay Scattergood Jonathan Zwi Juan Huang Michael Williams Patrick Donnelly Summer Rankin Malinda McPherson Andrew Landau Fred Barrett Alexis Roy Melanie Gilbert Music, Hearing Loss, and Cochlear Implants The Next Frontier Charles J. Limb, M.D. Francis A. Sooy Professor Chief of Otology, Neurotology and Skull Base Surgery Director, Douglas Grant Cochlear Implant Center Department of Otolaryngology-Head and Neck Surgery and Neurological Surgery University of California San Francisco San Francisco, CA 27