Music for Cochlear Implant Recipients: C I Can! Valerie Looi British Academy of Audiology National Conference. Bournemouth, UK. 19-20 Nov 2014
Let s Put It In Context
Outcomes Speech perception in quiet is excellent for CI recipients both adults & children However, performance is poorer for more complex stimuli and listening environments such as: Pitch perception (Gfeller et al., 2000; Kong et al., 2004; Leal et al., 2003; Looi et al, 2008am 2012; McDermott, 2004; McDermott & McKay, 1997; Shannon et al., 2004; Smith et al., 2002) Music perception (Fujita & Ito, 1999; Gfeller et al., 1998, 2000, 2002; Looi et al., 2012; McDermott, 2004)
Music vs. Speech signals Both speech & music have spectral and temporal envelopes that vary in time. Speech & music share similarities, but also differ: F0 range and loudness for music > speech. Accurate F0 perception is not imperative to speech recognition for non-tonal languages. For accurate melody recognition, information related to pitch, F0, duration, and timing of the pitches are critical. (Chasin & Russo, 2004; Wolfe, 2002)
Perceptual Consequences Of A S/NHL A significant S/N HL assoc. with: pitch discrim. & perceptual anomalies (e.g. non-monotonic pitch-scaling). Degree is unpredictable & not strongly correlated with hearing thresholds (Moore & Carlyon, 2005; Moore, 1995) freq. selectivity due to auditory filter bandwidths (Moore, 1995, 1996). Thresholds ~40-50 dbhl asymmetrically shaped filters & bandwidths ~2x wider than NH. Broader bandwidths the resolvability of low-order harmonics, affecting F0 perception (Moore, 1995). Affects BOTH pitch AND timbre perception - spectral shape perceived would be altered (Moore, 1995).
Technical limitations of CIs CI - sound percepts different from acoustic hearing. Existing research: Temporal resolution skills = to NH. But CI poorer on pitch and spectral-based tasks. (McDermott, 2004; Looi et al., 2012; Limb & Rubinstein, 2014) Impacts on music - western music requires discriminating frequency modulations of 6% (1 semitone). Pitch perception - extract F0 info. via 2 different mechanisms: 1. Resolve individual frequency components (place pitch). 2. Extract temporal pitch information from the signal. For CI recipient, both mechanisms affected by numerous factors.
What do we know?
General Music Listening (Adults) Time spent listening post-ci is significantly < pre-ci (Gfeller et al., 2000; Looi & She, 2010). Many CI recipients report music to sound strange, noisy, tinny, mechanical, unnatural. Some deliberately avoid it. (Gfeller et al., 2000; Leal et al., 2003; Looi & She, 2010; Looi et al., 2007, 2012).
Music Overall No indication that one type of CI and/or manufacturer is better or worse (Gfeller et al., 2000, 2008, 2010). No strategy is any better or worse (Gfeller et al., 2008. 2010; Lassaletta et al., 2008; Looi et al., 2008). No indication that electrode insertion depth of standard arrays, including deep insertion have any benefit for music (Gfeller et al., 2008, 2010; Lassaletta et al., 2008).
Rhythm & Pitch Adults with CIs similar to NH or HA for rhythm or temporal discrimination (Gfeller & Lansing, 1991, 1992; Leal et al., 2003; Looi et al., 2008a). However CI users score significantly < NH on pitch-based tasks. HOWEVER are NH results a fair comparison or realistic expectation as CI recipients have a significant S/NHL?
Pitch Perception Research Looi et al. (2008a,b; 2012) CI users significantly poorer than HA users in pitch-ranking 1, ½ & ¼ oct. intervals. CI: chance performance for ¼ oct. interval (51.75%). HA: performance for all 3 intervals was significantly above chance. NB* - HA pitch results in all of these studies were NOT as good as NH listeners.
Timbre Multi-dimensional; related to differences in sound spectra. Assessed using music instrument tests. Sound Quality is related to timbre & contributes more to satisfaction & appreciation than identification skills. Music Appreciation & Identification are DIFFERENT!
Timbre Research Gfeller et al. (2002) compared CI & NH - recognition & appraisal of 8 instruments: NH: 91%; CI: 47% & lower appraisal scores. No correlation to speech perception. Weak correlation between recognition and music experience or length of HL. No significant correlation accuracy & appraisal scores.
Timbre Research Looi et al. (2008a) - CI vs. HA - identification of 12 instruments & 12 ensembles. CI: 61% (inst); 43% (ensembles). HA: 69% (inst); 47% (ensembles). No significant difference CI & HA. Looi et al. (2008b): no significant difference pre post CI surgery in identification scores. Looi et al. (2007): compared CI & HA quality ratings: For newly implanted, ratings significantly higher post- than pre-implant (with HAs) (p = 0.026). For experienced CI and HA groups, CI group higher ratings than HA group (but not statistically significant).
Recent CI vs. HA results Test CI (n=17) HA (n=11) Instrument Identification 76% 83% Ensemble Identification 54% 72% Style Identification 55% 72% Pitch ½ Oct. 71% 91% Pitch ¼ Oct. 54%* 83% Instrument Rating 6.7/10 # 7.4/10 # Ensemble Rating 5.9/10 # 7.4/10 # Looi, King et al. (2012) Things to note: *CI ¼ oct. pitch ranking at chance level. CI ensemble & style identification & rating scores lower than single inst. Scores not much better than results 5-10 years ago. HA, although better than CI, are not close to perfect AND HA users had much better hearing levels (mild mod-severe losses) # Ratings of CI & HA not close to 10.
Melody Recognition Research Gfeller et al. (2002): closed-set recognition 12 melodies: CI: 19% correct. 66% of correct items rhythmic. Looi et al. (2008a): compared CI vs. HA - closed set recognition 10 melodies: CI (M=52%) significantly poorer than HA (M=91%) (p<0.001).
Correlations The only consistent correlations between music perception and various patient characteristics: Age (Gfeller et al., 1997, 2002, 2005; Looi et al., 2008a). Post-implant (or current ) music listening habits (Gfeller et al., 1998, 2000, 2005; Looi & She, 2010; Looi et al., 2007). Use of a hearing aid (Gfeller et al., 2008).
Example Simulation Normal Hearing CI-only Radford - http://thelisteningtree.wordpress.com/
Summary CI & HA users = NH rhythm; poorer pitch, timbre & melody. Reflected in appraisal ratings - music reported to sound empty, rough, tinny, unpleasant, & disappointing. BUT large variability; no single variable or explanation. Only consistent correlations: age, listening post-ci, & simultaneous HA. Time with CI doesn t improve music perception incidental exposure is NOT enough.
So What Can We Do About it?
Music Training MUSIC TRAINING, and focused listening practise helps both identification & appreciation. (Gfeller et al., 2000; Galvin & Fu, 2007, 2008; Looi et al., 2014). Some recipients love music, and are successful with it. Children can learn to perceive music accurately, & enjoy music. In lieu of a formal music training program, recipients can still self train or work at their music listening skills.
Music Training Although sound processors may limit pitch and timbre perception, studies have shown that music listening can be improved with training. (Gfeller et al., 2000; Galvin & Fu, 2007, 2008; Looi et al., 2012). Maximise potential from current device, with current technology.
Example of Music Training Research Looi et al. (2012) developed a music appreciation training program. Aim: to improve enjoyment (rather than recognition) of instruments, ensembles and styles. After training (10wks, 4x wk, 30mins), in training group s enjoyment ratings significantly > control group s (p = 0.043). Suggests music enjoyment can be improved with training.
Reintroducing Music to Adults
Tips for reintroducing music to adults Tips for new recipients or reintroducing music Start simple i.e. 1 instrument/singer, or small # inst Start with music you know or remember Start with music with words Try to have visual cues available (e.g. Score, song lyrics, DVD or video) Start with music with a strong rhythm/beat Use good quality equipment if possible Optimise the listening environment Don t be afraid to experiment DON T GIVE UP!!!
Considerations for adults for Self training Frequency: spaced sessions better than massed practise (Moore & Amitay, 2007). Aim for 30min sessions, 2-3 times a wk (Looi & She, 2010): Duration: Longer time frame better (Moore & Amitay, 2007). Difficulty: Too easy does NOT enable optimal learning; difficult tasks result in more robust learning (Moore & Amitay, 2007).
Do-at-home tasks to try Listen to: The same piece with different listening modes. E.g. stereo, computer, MP3, DAI. Which prefer? Why? What s the difference? 2 CDs of styles have never listened to, or wouldn t normally listen to. Compare. What s similar? What s different? Which sounds better? Why? What elements of the new styles do you like/dislike? 2 pieces from a different culture e.g. Chinese, African. What features are distinctive? What sounds different? How does it compare to the music you normally listen to? 2 radio music stations you've never listened to for 30 mins each. Describe the music played on each station. 2 contrasting CDs in your preferred style (e.g. different singers, groups, or types of music). e.g. Classical: orchestra vs. string quartet. Pop/Rock: heavy metal vs. soft rock.
Do-at-home tasks for recipients to try cont. Compare your everyday listening program to the music program whilst listening to music. Differences? Similarities? Experiment on a keyboard or piano. Start with the lowest note and go up 1-by-1. Is there an in pitch? Or do a series of notes sound the same? Or do some notes sound 'out' or wrong? Try to find the same song recorded with different instrumentations e.g. i) solo instrument (melody, no lyrics); ii) singer (lyrics) + simple accompaniment; iii) a karaoke version with subtitled lyrics; iv) a larger group version (e.g. band) with lyrics; and v) instrumental-only larger group version (no lyrics).
Introducing music to children
Pediatric Results Direct comparisons of adults vs. children are not valid: Congenital or pre-lingual HL. No internal representation of NH sound; learnt to hear with CI. Concepts such as pitch & timbre? Brain plasticity & sensitive period for learning.? cues to recognise music may not be same cues we use. If they don t like music or low music test scores, may not only be due to limitations of CI: Little exposure to music? No understanding of musical concepts? Maturation of auditory system and/or musical skills?
Music Training Research NH adults: music training primes brain for music sounds. Training functional & structural changes in auditory system. E.g: neural activity in fmri studies Music training benefits non-music domains (e.g. speech, language, emotion & auditory processing) (Kraus & Chandrasekaran, 2010). Hyde et al. (2009) NH children after 15mths music training: Structural changes in their auditory & motor cortex areas, relative to no training group. Music training promotes neuroplasticity. (Chermak, 2010; Galvin et al., 2007)
Music Training for children with CI Based on NH findings, professionals recommend musicbased training for children with CIs. (Limb & Roy, 2014; Looi et al., 2012) Children with CIs benefit from being involved in music; evidence of skill transference to non-musical tasks. o Consider age-appropriate music development, speech & language abilities, and limitations of CI. o Make it a positive, fun experience. o Focus on what they CAN do, rather than what they can t do, and build on this.
Ideas for children First emphasise exploration & enjoyment, not accuracy. Gradually introduce auditory concepts e.g. pitch, treble/ bass, low/high etc. Introduce simple auditory segregation tasks e.g. separating melody/words from accompaniment. Start with single instrument accompaniment (e.g. guitar, piano). Harmony: 2-3 simple chords only. Gradually complexity e.g. # instruments, harmonic complexity, speed of melody. Also consider the contrast between timbral qualities (e.g. same vs. different inst. families), or singers (e.g. gender).. (Rocca, 2012)
Ideas for children Singing in tune is complex requires BOTH pitch perception and control of vocal expression. NH child later developed skill. Singing/vocal exercises that can help: Vocal and pitch exploration, pitch glissandos, melodic contour matching, imitation, singing simple, known melodies. Playing instruments can help CI children develop listening skills, from initial sound awareness to discriminating between instruments, and then identifying and naming instruments. (Rocca, 2012)
In summary Adults with CI can improve their music listening skills. Children with CIs DO enjoy music. They CAN learn music, musical instruments & singing. Music Training and Focused Listening Practise DOES help. Consider limitations of CI, but don t let this stop participation. A CI is NOT a contraindication to music involvement.
References Chasin M, Russo FA. Hearing aids and music. Trends Amplif. 2004;8(2): 35-47. Chermak G. Music and auditory training. Hear J 2010;63:57-58 Galvin JJ, Fu QJ, Nogaki G. Melodic contour identification by cochlear implant listeners. Ear Hear. 2007;28(3):302-19. Galvin JJ, Fu QJ, Oba S. Effect of instrument timbre on melodic contour identification by cochlear implant users. J. Acoust. Soc. Am. 2008;124(EL):189-95 Gfeller K, Christ A, Knutson JF, Witt S, Murray KT, Tyler RS. Musical backgrounds, listening habits, and aesthetic enjoyment of adult cochlear implant recipients. J. Am. Acad. Audiol. 2000;11(7):390-406. Gfeller K, Christ A, Knutson J, Witt S, Mehr M. The effects of familiarity and complexity on appraisal of complex songs by cochlear implant recipients and normal hearing adults. J. Music Ther. 2003;40:78-112. Gfeller K, Oleson J, Knutson JF, Breheny P, Driscoll V, Olszewski C. Multivariate predictors of music perception and appraisal by adult cochlear implant users. J. Am. Acad. Audiol. 2008;19(2):120-34. Gfeller K, Olszewski C, Rychener M, Sena K, Knutson JF, Witt S, et al. Recognition of "real-world" musical excerpts by cochlear implant recipients and normal-hearing adults. Ear Hear. 2005;26(3):237-50. Gfeller KE, Olszewski C, Turner C, Gantz B, Oleson J. Music perception with cochlear implants and residual hearing. Audiol. Neurootol. 2006;11 Suppl 1:12-5. Gfeller K, Turner C, Oleson J, Zhang X, Gantz B, Froman R, et al. Accuracy of cochlear implant recipients on pitch perception, melody recognition, and speech reception in noise. Ear Hear. 2007;28(3):412-23. Gfeller K, Witt S, Stordahl J, Mehr M, Woodworth G. The effects of training on melody recognition and appraisal by adult cochlear implant recipients. J. Acad. Rehabil. Audiol. 2000b;33:115-138. Gfeller K, Witt S, Woodworth G, Mehr MA, Knutson J. Effects of frequency, instrumental family, and cochlear implant type on timbre recognition and appraisal. Ann. Otol. Rhinol. Laryngol. 2002;111(4):349-56. Kraus N, Chandrasekaran B. Music training for the development of auditory skills. Nat Rev:Neuroscience. 2010;11(8):599 605.
References Lassaletta L, Castro A, Bastarrica M, Perez-Mora R, Herran B, Sanz L et al. Otolaryngol Head Neck Surg. 2008;183:363-7. Leal MC, Shin YJ, Laborde ML, Calmels MN, Verges S, Lugardon S, et al. Music perception in adult cochlear implant recipients. Acta Otolaryngol. (Stockh). 2003;123:826-835. Limb CJ, Rubinstein JT. Current research on music perception in cochlear implant users. Otolaryngol Clin North Am 2012;45:129-140 Limb CJ, Roy AT. Technological, biological, and acoustical constraints to music perception in cochlear implant users. Hear Res. 2014;308:13-26. Looi V. The effect of cochlear implantation of music: A review. Otorinolaringologia. 2008;58(4):169-190. Looi V, Gfeller K, Driscoll G. Music Appreciation And Training For Cochlear Implant Recipients: A Review. Semin Hear. 2012; 33(4):307-334. Looi V, King J, Kelly-Campbell RA. Music Appreciation Training Program Developed For Clinical Application With Cochlear Implant Recipients And Hearing Aid Users. Semin Hear. 2012; 33(4);361-380. Looi V, McDermott H, McKay C, Hickson L. Music perception of cochlear implant users compared with that of hearing aid users. Ear Hear. 2008;29(3):421-34. Looi V, McDermott H, McKay C, Hickson L. The effect of cochlear implantation on music perception by adults with usable pre-operative acoustic hearing. Int J Audiol. 2008b;47(5):257-268. Looi V, McDermott H, McKay C, Hickson L. Comparisons of quality ratings for music by cochlear implant and hearing aid users. Ear Hear. 2007;28(2 Supplement):59S-61S. Looi, V. & She, J. Perceptions of Music for Adult Cochlear Implant Users: A Questionnaire, and its implications for a music training program. Int J Audiol. 2010;49(2):116-128 McDermott H. Music perception with cochlear implants: A review. Trends Amplif. 2004;8(2):49-82 Mirza S, Douglas SA, Lindsey P, Hildreth T, Hawthorne M. Appreciation of music in adult patients with cochlear implants: A patient questionnaire. Cochlear Implants Int. 2003;4(2):85-95.
References Moore BCJ. Perceptual Consequences of Cochlear Damage. Oxford: Oxford University Press; 1995. Moore BCJ, Carlyon RP. Perception of pitch by people with cochlear hearing loss and by cochlear implant users. In: Plack CJ, Oxenham AJ, Fay RR, Popper AN, editors. Pitch: Neural Coding and Perception. New York: Springer-Verlag; 2005. p. 234-277. Moore DR, Amitay S. Auditory training: Rules and applications. Semin Hear 2007;28:99 Rocca CA. Different Musical Perspective: Improving Outcomes in Music through Habilitation, Education and Training for Children with CIs. Semin Hear. 2012;33(4), Wolfe J. Speech and music, acoustics and coding, and what music might be 'for'. Paper presented at The 7th International Conference on Music Perception and Cognition, Sydney, Australia. 2002.
THANK YOU! Questions???
NH Developmental Milestones for Music Age Milestone 9-12 m Spontaneous singing 12-18 m Attends to lyrics, sings extracts of songs, rough pitch matching 18-24 m Coarsely imitate songs lyrics more accurate than pitch Infant Detect change of direction in melody contour, but difficulty higher / lower concept 2-3 yr Some instrument discrimination 3-4 yr Pitch match consistently & reproduce intervals 4-5 yr Start to recognise familiar melodies without lyrics 5-6 yr Better pitch accuracy in singing; can remember songs in head 6-7 yr Develop tonal centre (can sing a song in 1 key) Singing range focuses around 5-6 notes 7-9 yr Vocal range expands. Pitch ranking ability matures. ~9yrs develop preference for a music style. 11 yr Consistent singing in tune (Andrews & Madeira, 1997; Barton, 2010; Dowling, 1999; Rocca, in press; Stalinski et al., 2008)