Language and Music: Differential Hemispheric Dominance in Detecting Unexpected Errors in the Lyrics and Melody of Memorized Songs

Transcription

1 Human Bain Mapping 30: (2009) Language and Music: Diffeential Hemispheic Dominance in Detecting Unexpected Eos in the Lyics and Melody of Memoized Songs Takuya Yasui, 1,2,3 Kimitaka Kaga, 2,4 and Kuniyoshi L. Sakai 1,3 * 1 Depatment of Basic Science, Gaduate School of Ats and Sciences, The Univesity of Tokyo, Komaba, Komaba, Meguo-ku, Tokyo , Japan 2 Depatment of Otolayngology, Gaduate School of Medicine, The Univesity of Tokyo, Hongo, Bunkyo-ku, Tokyo , Japan 3 CREST, Japan Science and Technology Copoation, Kawaguchi-shi , Japan 4 National Institute of Sensoy Ogans, Higashigaoka, Meguo-ku, Tokyo , Japan Abstact: Using magnetoencephalogaphy (MEG), we epot hee the hemispheic dominance of the auditoy cotex that is selectively modulated by unexpected eos in the lyics and melody of songs (lyics and melody deviants), theeby elucidating unde which conditions the latealization of auditoy pocessing changes. In expeiment 1 using familia songs, we found that the dipole stength of esponses to the lyics deviants was left-dominant at 140 ms (M140), wheeas that of esponses to the melody deviants was ightdominant at 130 ms (M130). In expeiment 2 using familia songs with a constant syllable o pitch, the dipole stength of fequency mismatch negativity elicited by oddballs was left-dominant. Thee wee significant main effects of expeiment (1 and 2) fo the peak latencies and fo the coodinates of the dipoles, indicating that the M140 and M130 wee not the fequency mismatch negativity. In expeiment 3 using newly memoized songs, the ight-dominant M130 was obseved only when the pesented note was unexpected one, independent of peceiving unnatual pitch tansitions (i.e., peceptual saliency) and of selective attention to the melody of songs. The consistent ight-dominance of the M130 between expeiments 1 and 3 suggests that the M130 in expeiment 1 is due to unexpected notes deviating fom well-memoized songs. On the othe hand, the left-dominant M140 was elicited by lyics deviants, suggesting the influence of top-down linguistic infomation and the memoy of the familia songs. We thus conclude that the leftlatealized M140 and ight-latealized M130 eflect the expectation based on top-down infomation of language and music, espectively. Hum Bain Mapp 30: , VC 2008 Wiley-Liss, Inc. Key wods: hemispheic dominance; auditoy pocessing; tempoal cotex; magnetoencephalogaphy; mismatch negativity Contact gant sponso: Ministy of Education, Cultue, Spots, Science, and Technology of Japan; Contact gant numbes: ; Contact gant sponso: Japan Science and Technology Agency (JST). *Coespondence to: Kuniyoshi L. Sakai, Depatment of Basic Science, Gaduate School of Ats and Sciences, The Univesity of Tokyo, Komaba, Komaba, Meguo-ku, Tokyo , Japan. sakai@mind.c.u-tokyo.ac.jp Received fo publication 17 July 2007; Revised 14 Novembe 2007; Accepted 15 Novembe 2007 DOI: /hbm Published online 2 Januay 2008 in Wiley InteScience (www. intescience.wiley.com). INTRODUCTION The highe bain functions elated to language and music ae thought to involve uniquely human abilities [Ledahl and Jackendoff, 1983; Patel, 2003], and they ae known to have a stong tendency fo hemispheic dominance in the bain. Syntax, semantics, and phonology ae geneally latealized with left hemispheic dominance [Gazzaniga, 2000; Geschwind, 1979; Pice, 2000; Sakai, 2005], wheeas the ight hemispheic dominance in pocessing posodic infomation has been epoted in fontal VC 2008 Wiley-Liss, Inc.

2 Latealization in Monitoing Lyics and Melody and tempoal egions [Hesling et al., 2005; Nicholson et al., 2003; Plante et al., 2002; Ross, 1981]. On the othe hand, ight tempoal lesions ae epoted to cause amusia, o deficits in the discimination of melodies [Ayotte et al., 2000; Dennis and Hopyan, 2001; Giffiths et al., 1997; Liégeois-Chauvel et al., 1998; Muayama et al., 2004; Nicholson et al., 2003]. A positon emission tomogaphy study on healthy paticipants has also epoted that activation of the ight supeio tempoal cotex is enhanced duing passive listening to melodies [Zatoe et al., 1994]. Active listening to instumental music activates both hemisphees, with ight-hemispheic weighting [Koelsch et al., 2002; Kaeme et al., 2005], and pevious lesion studies have indicated that long-tem memoy of musical infomation, in contast to vebal infomation, is not clealy latealized in one hemisphee [Samson, 1999; Stewat et al., 2006]. To establish the ight hemispheic dominance of auditoy pocessing moe conclusively, cotical activation should be futhe examined with sufficient tempoal esolution. Recent magnetoencephalogaphy (MEG) studies with speech sounds have epoted that phoneme o voice oddballs induce the left-dominant mismatch negativity (MMNm) [Knösche et al., 2002; Näätänen et al., 1997]. This type of fequency MMNm is geneated by an automatic detection of fequency eos (deviants) occuing in a seies of standad stimuli, and the fequency MMNm can be elicited even by auditoy oddballs eliciting no N1 esponse [Näätänen et al., 2005]. An MEG study has epoted that the fequency MMNm fo chod oddballs was lage than that fo phoneme oddballs in the ight hemisphee, but the ight dominance of the MMNm fo chod oddballs was not significant [Tevaniemi et al., 1999]. In these pevious studies, speech sounds wee used fo language stimuli, while the instumental sounds o chods wee used fo music stimuli. Thus the bottom-up pocesses fo the acoustic featues of stimuli can explain the contasting hemispheic dominance between language and music pocessing. Speech sounds ae highly dependent on apidly changing sounds wheeas tonal music pattens tend to be slowe; thus, hemispheic dominance of the supeio tempoal cotex may depend on the tempoal and spectal popeties of the acoustic stimuli and associated bottomup pocesses [Boemio et al., 2005; Zatoe et al., 2002]. Howeve, it emains unclea how the hemispheic dominance is influenced by top-down pocesses including attention and memoy. A pevious event-elated bain potential (ERP) study examined the effect of familia melodies with a egula but unexpected ending, o with an iegula tone, and epoted that a late positive component with the peak latency of 300 ms was elicited by these wong notes [Besson and Faïta, 1995]. Anothe ERP study epoted that the N400 component was elicited by memoy violations, wheeas an ealy ight anteio negativity (ERAN) with the peak latency of 200 ms was elicited by out-of-key violations [Mianda and Ullman, 2007]. The ERAN was also associated with expectancy based on musical egulaity [Koelsch et al., 2000; Maess et al., 2001]. It emains to be elucidated how the expectancy of melody itself based on stoed memoy affects ealy components. If the exact timing of the fist component eflecting the top-down pocesses based on stoed memoy could be claified, this would futhe elucidate basic auditoy pocessing. To examine such top-down pocesses, expeiments in which paticipants listen actively to song stimuli with both popeties of lyics and melody would be ideal, because the tempoal and spectal popeties of the song stimuli can be equated as much as possible. Figue 1A shows one example of song stimuli used in the pesent study, which was made with a speech synthesis pogam that can be configued to poduce songs by assigning musical notes and lyics. Using this pogam, the duation, pitch, and powe of each note can be held constant. We used a foced-choice eo-detection paadigm, which has been established in pevious studies [Embick et al., 2000]; in each tial, thee wee always one o two unexpected eos in the lyics and melody of songs (lyics and melody deviants). In expeiment 1 using familia songs, we fist compaed the hemispheic dominance between language and music tasks, whee identical song stimuli wee used without any instumental accompaniment o chods. We detected event-elated MEG esponses to eithe the lyics o melody deviants in two tasks pefomed sepaately: a lyics and a melody task (Fig. 1B). Hee the stoed memoy of songs was equied, since thee was no cue in the acoustic featues pe se to disciminate deviants fom the nomal efeence stimuli. In expeiment 2 using familia songs with a constant syllable o pitch, which did not equie stoed memoy of songs, we examined the effect of oddballs in two tasks pefomed sepaately: a syllable and a pitch task (Fig. 1C). We tied to claify whethe the deviant-induced fields obseved in expeiment 1 wee diffeent fom the fequency MMNm. In expeiment 3 using unfamilia and newly memoized songs, we futhe tied to sepaate the top-down expectation pocesses fom the bottom-up peception of unnatual pitch tansitions in the melody task (Fig. 1D), theeby confiming the consistency of the deviant-induced fields between expeiments 1 and 3. MATERIALS AND METHODS Paticipants Sixteen native Japanese speakes paticipated in the pesent study (aged yeas, mean 27 yeas; eight females). Fouteen paticipants paticipated in expeiment 1, nine of whom also paticipated in expeiment 2 along with an additional paticipant (a total of 10 paticipants). Eleven paticipants paticipated in expeiment 3, eight of whom also paticipated in both expeiments 1 and 2 but one of them was discaded due to low pefomance at the taining stage (a total of 10 paticipants), and anothe two of whom had paticipated in eithe expeiment 1 o 2. All paticipants showed ight-handedness by the Edinbugh inventoy 589

3 Yasui et al. The expeimental paadigm with songs to examine the diffeences between lyics and melody pocessing. A: An example of 12 oiginal songs used in expeiments 1 and 2 (e.g., a Fench folk song Twinkle, twinkle, little sta with typical Japanese lyics). Each song has seven notes, and is sung without any instumental accompaniment and chods. The time line of one tial is shown below the musical scoe. In expeiment 3, a diffeent set of songs was used with the same time sequence. B: Expeiment 1, employing lyics and melody tasks on familia songs. Thee wee one o two deviants in lyics (a blue cicle) o melody (a ed cicle) among the thid to the seventh notes. The nomal note Figue 1. peceding the fist appeaing deviant was used as the efeence fo deviant-induced fields (gay ectangles). C: Expeiment 2, employing syllable and pitch tasks on familia songs with a constant syllable o pitch. In the syllable task, lyics wee eplaced with the constant syllable of na. A blue cicle denotes a syllable oddball. In the pitch task, the melody was eplaced with the constant note of A4 (440 Hz). A ed cicle denotes a pitch oddball. The note peceding the fist appeaing oddball was used as the efeence (gay ectangles). D: Expeiment 3, employing the melody task on newly memoized songs. See Mateials and Methods fo full explanation. [Oldfield, 1971], and had nomal heaing abilities without pofessional singing o instument taining besides school education. The study was appoved by the institutional eview boad of The Univesity of Tokyo, Komaba. Stimuli and Tasks The song stimuli without accompaniment wee poduced with SMARTTALK softwae (Oki Electic Industy, Tokyo, Japan). Fo each tial in all expeiments, a song stimulus with seven notes (600 ms each) was pesented at a sound pessue level of about 85 db (Fig. 1A). Thee wee always one o two deviants among the thid to the seventh notes, and the second deviant appeaed two o thee sounds apat fom the fist one, but in a pseudoandomized position, so that the melody afte the fist deviant clealy etuned to the oiginal one. Afte the song stimulus, a pue tone of 1,000 Hz was pesented fo 50 ms to initiate the button pess within the following 2 s. The paticipants wee infomed that thee would always be one o two deviants in each tial, and they indicated the numbe of deviants by pushing one of two buttons. In expeiments 1 and 2, MEG esponses to the fist appeaing deviant/oddball and those to a efeence note just befoe it wee used fo the analysis; the esponses to the second deviant/oddball wee not used because they might eflect piming o habituation effects. In expeiment 3, MEG esponses to the thid note and those to the second efeence 590

4 Latealization in Monitoing Lyics and Melody note wee used fo the analysis. The tial inteval was andomized within the ange of 62% at 6.8 s to educe any peiodical noises. The stimuli wee deliveed binaually to paticipants via plastic tubes (length, 6.5 m), which wee connected to inset eaphones (Etymotic ER-30; Etymotic Reseach, Elk Gove Village, IL). Stimulus pesentation and behavioal data collection wee contolled using the LabVIEW softwae and inteface (National Instuments, Austin, TX). Expeiment 1 We pepaed fo a sequence of 12 familia songs (oiginal songs), each consisting of seven quate notes. The ode of these 12 song stimuli was always fixed, so that the paticipants could expect lyics and melody even fom the beginning of each song. Fo the lyics task, lyics deviants wee made by changing both consonants and vowels in two ways fo each syllable (e.g., wa and bu fom ki ), while etaining the oiginal melody (Fig. 1B). The songs with lyics deviants wee phonotactically legal, but they wee syntactically and semantically anomalous. On the basis of such top-down linguistic infomation and the memoy of familia songs, the paticipants esponded to one o two lyics deviants. Fo the melody task, melody deviants wee made by aising o loweing the oiginal note by five semitones, while etaining the oiginal lyics (Fig. 1B). On the basis of the memoy of familia songs, the paticipants esponded to one o two melody deviants. Fo each task, 228 tials wee conducted fo about 25 min; the last thee tials wee excluded fom the analysis because of technical poblems with the pogam. Behavioal data wee not successfully collected fom thee paticipants due to technical poblems with the buttons. The ode of the two tasks was countebalanced among the paticipants. Expeiment 2 In the syllable task, all lyics wee eplaced with the constant syllable of na while keeping the oiginal melody used in expeiment 1. The paticipants esponded to one o two oddballs consisting of the syllables ki o so in each song stimulus (Fig. 1C). In the pitch task, all melodies wee eplaced with the constant note of A4 (440 Hz) while keeping the oiginal lyics used in expeiment 1. The paticipants esponded to one o two oddballs, which wee made by aising o loweing the constant note by five semitones. The location and numbe of oddballs in this expeiment wee the same as those of the deviants in expeiment 1. Fo each task, 228 tials wee conducted fo about 25 min; the last thee tials wee excluded fom the analysis due to technical poblems with the pogam. The ode of the two tasks was countebalanced among the paticipants. Expeiment 3 As shown in Figue 1D (oange column), the paticipants memoized a sequence of the eight unfamilia songs (a, b, a,...) just befoe the MEG ecodings. The ode of these eight songs was always fixed, so that the paticipants could expect lyics and melody even fom the beginning of each song. On the basis of the memoized sequence, the paticipants esponded to one o two melody deviants. Hee, we define the expected note condition as pesenting a note fom the memoized sequence (the thid note in the oange column), and the unexpected note condition as pesenting a melody deviant (the thid note in the puple column). If the expected note condition consisted of the oiginally composed notes alone wheeas the unexpected note condition included the expeimentally modified notes, it was not possible to sepaate the top-down expectation pocesses fom the bottom-up peception of unnatual pitch tansitions. To equate the melody stimuli between these conditions, each condition included both oiginally composed notes (geen cicles) and expeimentally modified notes (blue cicles), wheeas the second notes wee used as the efeence (gay ectangles). Fom an oiginal song (a, b, c, and d), each modified song (a, b, c, and d ) was made by aising o loweing the thid note alone by five semitones. This pocedue was identical with that fo making melody deviants, but the pesence of modified songs did not hampe memoization, since the oiginal songs wee totally unfamilia to the paticipants befoe the expeiment. Moeove, a melody deviant at the thid note of a modified song (e.g., a ) was made always identical with the thid note of the coesponding oiginal song (a fo a ), theeby stictly equating the stimulus popeties. Melody deviants wee also pesent in the fouth to the seventh notes as in expeiment 1, but they wee not used fo the analysis. Afte an initial self-paced memoization block fo 10 min, a confimation test consisting of eight tials of the melody task was pefomed twice. If paticipants failed to scoe seven out of eight, anothe memoization block was administeed. All but one paticipant successfully scoed seven out of eight in two consecutive tests, within two o thee memoization blocks (20 30 min in total). The MEG ecodings wee then stated fo the successfully passed paticipants. Duing the MEG ecodings, only the melody task was pefomed, using the same design as in expeiment 1. In each of thee sepaate blocks, 128 tials wee pefomed fo about 15 min each. MEG Recodings and Analyses The MEG data wee acquied with a 160-channel wholehead ecoding system (MEGvision; Yokogawa Electic Copoation, Kanazawa-city, Japan). Signals wee digitized on-line with a bandwidth of 0.3 1,000 Hz at a sampling ate of 2,000 Hz. Fom the MEG data, only atifact-fee tials (peak-to-peak amplitude <3,000 ft and without eye movement) wee selected and aveaged sepaately fo the diffeent types of deviant/oddball and efeence stimuli. In expeiments 1 and 2, both coect and incoect tials wee analyzed, wheeas in expeiment 3, only coect tials 591

5 Yasui et al. wee used fo analyses because the accuacies wee lowe. The aveaged data wee futhe filteed with an off-line band-pass filte of 1 20 Hz [Sinkkonen and Tevaniemi, 2000]. The MEG esponses fom 2100 ms to 600 ms (0 ms at the onset of each note) wee analyzed with a time window of 0.5 ms, and those fom 2100 ms to 0 ms wee egaded as a baseline. Fo each MEG component (e.g., P1m), a tempoal peak latency and a spatial peak channel (the posteio peak of a souce/sink pai) wee simultaneously seached and detemined in each hemisphee of a paticipant. The selected peak channels did not necessaily coincide among the paticipants. With a softwae package (MEG Laboatoy; Yokogawa Electic Copoation, Kanazawa-city, Japan) based on peviously established pocedues [Savas, 1987], an equivalent cuent dipole (ECD) at the latency of each component was calculated fo atifactfee channels coesponding to a souce/sink pai of MEG esponses in each hemisphee, and a spheical model was used to detemine ECDs without the use of seed-points o othe constaints. In expeiments 1 and 2, the citeion fo an acceptable dipole solution was a goodness-of-fit of at least 90% fo each paticipant, wheeas in expeiment 3, a goodness-of-fit of at least 85% was equied. Each ECD location was plotted as a point on individual MR images using MRIco softwae ( Individual MR images wee then nomalized with SPM2 softwae (Welcome Depatment of Imaging Neuoscience, London, UK), and the ECD points wee tansfomed with the same nomalization paametes using the MRIco suboutine (lesionmask.m). In the pesent study, we used the MNI (Monteal Neuological Institute) coodinate system. The MNI coodinates of each ECD wee plotted onto the standad bain, and thei positions elative to Heschl s sulcus wee detemined on the standad bain. Fo the statistical analysis within each expeiment, epeated measues analysis of vaiance (ANOVA) was pefomed. If an inteaction among seveal factos was significant, futhe analyses with one-way ANOVAs wee pefomed. Regading the ECD location in each hemisphee, the unsigned x coodinates (moe medial o lateal within the hemisphee), as well as the signed y and z coodinates, wee used fo compaisons. RESULTS Expeiment 1 The accuacies of the lyics and melody tasks fo the 11 paticipants wee ( )% (mean 6 SEM) and ( )%, espectively, and these two values wee not significantly diffeent (F(1, 10) , P 5 0.8). Figue 2A shows event-elated MEG esponses to the efeence stimuli, lyics deviants, and melody deviants; these esponses wee taken fom the same posteio peak channel in each hemisphee of one epesentative paticipant. The fist detectable peak, a P1m, was obseved with the latency of 50 ms fo all types of stimuli, wheeas the second peak, an N1m, was obseved with the latency of 100 ms fo efeence stimuli (Fig. 2A, black lines). Fom all 28 hemisphees, we obtained the peak latencies of the P1m induced by efeence stimuli ( ms), as well as those induced by both types of deviants (lyics deviants, ms; melody deviants, ms). Accoding to a two-way epeated measues ANOVA, neithe the main effects of task and hemisphee no thei inteaction wee significant fo the peak latencies (Table I). The gand-aveage of the ECD locations fo the P1m is shown in Figue 2B. The ECDs in the lyics task [MNI coodinates, left: (x, y, z) 5 ( , , ); ight: ( , , )] and those in the melody task [left: ( , , ); ight: ( , , )] wee located in the posteio egion of Heschl s gyus. The significant main effects of task fo the ECD locations wee obseved, such that the ECDs of the P1m in the lyics task wee significantly supeio to those in the melody task (Table I). Neithe the main effects of task and hemisphee no thei inteaction wee significant fo the ECD stength (Fig. 2C, Table I). The N1m fo efeence stimuli (peak latency, ms) was not always induced (nine hemisphees), pesumably due to the apid succession of notes [Hai et al., 1982]. The P1m and N1m ae also known as P50m and N100m [McEvoy et al., 1994]. In contast, pominent deviant-induced fields with the latency of ms wee elicited by both the lyics and melody deviants (Fig. 2A, blue and ed lines). The distibution of these magnetic fields with an invese polaity in each hemisphee suggests the existence of symmetical dipoles that point to the infeo-posteio diection (Fig. 2A, uppe panel). To extact pue deviant-induced fields, the event-elated esponses to efeence stimuli wee subtacted fom the esponses to deviants (Fig. 2D, blue and ed lines), which esulted in a simila distibution patten of magnetic fields fo this epesentative paticipant (Fig. 2D, uppe panel fo melody deviants). Because the efeence notes include both featues of syllable and pitch used in the deviants, the subtacted MEG esponses ae selective to the tempoal popeties of eithe lyics o melody, which ae independent fom paticula languages, songs, and constant acoustic featues. In all 14 paticipants tested, these deviant-induced fields wee consistently obseved. In the lyics task, thei tempoal peak latencies in the left and ight hemisphees, simultaneously detemined with spatial peak channels fom the subtacted data, wee ms and ms, espectively. In the melody task, the peak latencies in the left and ight hemisphees wee ms and ms, espectively. The significant main effects of task fo the peak latencies wee obseved, such that the peak latency in the lyics task was significantly longe than that in the melody task (Table I). We thus named the deviantsinduced fields in the lyics and melody tasks the M140 and M130, espectively. To evaluate the hemispheic dominance of deviantinduced fields in the lyics and melody tasks, we calculated 592

6 Latealization in Monitoing Lyics and Melody ECDs of the M140 and M130 fom the subtacted data. The ECDs of the M140 [left: ( , , ); ight: ( , , )] and those of the M130 [left: ( , , ); ight: ( , , )] wee located anteio to Heschl s gyus (Fig. 2E). Accoding to a two-way epeated measues ANOVA, the significant main effects of task fo the z coodinate wee significant, such that the ECDs of the M140 wee significantly supeio to those of the M130 (Table I). On the othe hand, the main effects of task and an inteaction between task and hemisphee wee maginally significant fo the x coodinate. The ECDs of the M140 wee significantly moe lateal to those of the M130 in the ight hemisphee (F(1, 13) 5 16, P ), while thee was no such diffeence in the left hemisphee (F(1, 13) , P 5 0.9). Next, we compaed the ECD locations between the P1m and M140/M130. Accoding to a thee-way epeated measues ANOVA fo the y coodinate (Table I), the significant main effects of both component and hemisphee wee obseved. The ECDs of the M140/M130 wee significantly moe anteio to those of the P1m, while those of the M140/M130 and P1m wee significantly moe anteio in the ight hemisphee to those in the left hemisphee. A significant inteaction between component and task fo the x and z coodinates, as well as that between task and hemisphee fo the x coodinate, was also obseved. The ECDs of the M140 wee significantly moe dosal to those of the Figue 2. The esults of expeiment 1, showing the left-dominant M140 and ight-dominant M130. A: MEG esponses to the lyics deviants (blue lines), melody deviants (ed lines), and efeence stimuli (black lines), taken fom one epesentative paticipant. The uppe panel shows the magnetic field distibution at 130 ms fom the onset of the fist melody deviants (souce: ed lines; sink: geen lines). The lowe left and lowe ight panels show the esponses at the same posteio channels in the left and ight hemisphees, espectively. B: The gand-aveaged ECD locations of the P1m in the lyics (blue cicles) and melody tasks (ed cicles). The ECDs wee supeimposed on a stuctual image of the standad bain (Monteal Neuological Institute) showing a hoizontal slice at z 5 9. C: The ECD stength (mean 6 SE) of the P1m in the lyics and melody tasks. D: Time couses of the M140 (blue lines) and M130 (ed lines), taken fom one epesentative paticipant. The data wee calculated by subtacting event-elated esponses to efeence stimuli fom those to the fist appeaing deviants. The uppe panel is the magnetic field distibution of the M130 (afte subtaction) at 130 ms fom the onset of the fist melody deviants. E: The gandaveaged ECD locations of the M140 (blue cicles) and M130 (ed cicles). F: The ECD stength of the M140 and M130. The M140 was significantly lage in the left than in the ight hemisphee (*P < 0.05). In contast, the M130 was significantly lage in the ight than in the left hemisphee. 593

7 Yasui et al. TABLE I. Repeated measues ANOVAs fo the P1m and M140/M130 in expeiment 1 x y z df F P F P F P P1m Task (lyics, melody) 1, * Hemisphee (L, R) 1, Task 3 Hemisphee 1, Latency ECD stength Task (lyics, melody) 1, Hemisphee (L, R) 1, Task 3 Hemisphee 1, M140/M130 Task (lyics, melody) 1, * Hemisphee (L, R) 1, Task 3 Hemisphee 1, Latency ECD stength Task (lyics, melody) 1, ** Hemisphee (L, R) 1, Task 3 Hemisphee 1, ** Component (P1m, M140/M130) 1, ** Task (lyics, melody) 1, Hemisphee (L, R) 1, * Component 3 Task 1, * ** Component 3 Hemisphee 1, Task 3 Hemisphee 1, * Hemisphee 3 Task 3 Component 1, df, degee of feedom; L, left; R, ight; *, P < 0.05; **, P < P1m in the ight hemisphee (F(1, 13) 5 5.6, P ). The ECDs of the M130 wee maginally moe vental to those of the P1m in the left hemisphee (F(1, 13) 5 4.3, P ), and those of the M130 wee significantly medial in the ight hemisphee (F(1, 13) 5 28, P ), while thee was no significant diffeence fo the othe cases (P > 0.1). Theefoe, the ECD locations of the M140 and M130 wee significantly diffeent fom those of the P1m. To evaluate the hemispheic dominance of M140 and M130, we compaed the ECD stength of these two components between hemisphees. We found a significant inteaction between task (i.e., component) and hemisphee fo the ECD stength of the M140/M130 (Fig. 2F, Table I). In the lyics task, the ECD stength of the M140 was significantly lage in the left than in the ight hemisphee (F(1, 13) 5 6.4, P ), wheeas that of the M130 was significantly lage in the ight than in the left hemisphee in the melody task (F(1, 13) 5 8.8, P ). These esults demonstate the evidence of the left-dominant M140 in the lyics task and the ight-dominant M130 in the melody task. Expeiment 2 The latencies of M140 and M130 wee within the ange of those of the peviously epoted fequency MMNm ( ms), and the deviants might be egaded as oddballs fom the natual flow of songs. Moeove, it has been epoted that the fequency MMNm was elicited even by oddballs at the second position (i.e., diffeent fom the fist stimulus) [Jääskeläinen et al., 2004]. It is thus necessay to examine whethe o not the M140 and M130 wee identical with the fequency MMNm. Fo this pupose, we tested syllable and pitch tasks with an oddball paadigm, which did not equie judgment based on the stoed memoy of songs. In the syllable task, all lyics wee eplaced with a constant syllable, wheeas in the pitch task, all notes wee eplaced with a constant pitch (Fig. 1C). Paticipants detected the oddballs by monitoing the constant syllable o pitch, whee othe featues of the songs wee identical with those tested in expeiment 1. If the ight dominance obseved in the melody task in expeiment 1 wee due to the chaacteistics of the fequency MMNm, then the ight dominance would be obseved in the pitch task in expeiment 2 as well, because the fequency MMNm is moe likely to be poduced by the pitch oddballs. The accuacies in the syllable and pitch tasks fo the 10 paticipants wee ( )% and ( )%, espectively, and these two values wee significantly diffeent (F(1, 9) 5 5.1, P ). Figue 3A shows event-elated MEG esponses to pitch oddballs and efeence stimuli in 594

8 Latealization in Monitoing Lyics and Melody the pitch task (Fig. 3A, ed and black lines, espectively); these esponses wee taken fom the same posteio channel in each hemisphee of one epesentative paticipant. At 120 ms, the oddballs at the positions of the thid/fouth and fifth-seventh notes elicited lage esponses than the efeence stimuli. The distibution of these magnetic fields with an invese polaity in each hemisphee suggests the existence of symmetical dipoles that point to the infeoposteio diection (Fig. 3A, uppe panel). To extact the MMNm, event-elated esponses to the efeence stimuli, which wee the notes just befoe the fist appeaing oddballs at all stimulus positions, wee subtacted fom the event-elated esponses to the oddballs (Fig. 3B, blue and ed lines), which esulted in a simila distibution patten of magnetic fields fo this epesentative paticipant (Fig. 3B, uppe panel fo pitch oddballs). In all 10 paticipants, both syllable and pitch oddballs elicited lage magnetic esponses than the efeence stimuli in the left and ight tempoal cotices. In the syllable task, the tempoal peak latencies of the MMNm in the left and ight hemisphees, simultaneously detemined with spatial peak channels fom the subtacted data, wee ms and ms, espectively. In the pitch task, the peak latencies in the left and ight hemisphees wee ms and ms, espectively. Regading the peak latencies of the MMNm, neithe the main effects of the task and hemisphee no thei inteaction wee significant (Table II). With these individually detemined tempoal peak latencies and spatial peak channels, the wave amplitudes of MEG esponses wee calculated fo oddball and efeence stimuli, as well as fo stimulus positions (fist and second halves of the five notes used fo analyses), by using the data befoe the subtaction. In both tasks and hemisphees, the significant main effects of stimulus type fo the wave amplitudes wee obseved, such that the wave amplitudes fo oddballs wee lage than those fo efeence stimuli. On the othe hand, the main effects of stimulus position wee not significant (Table II). An inteaction between stimulus type and stimulus position Figue 3. The esults of expeiment 2, showing the MMNm in the syllable and pitch tasks. A: MEG esponses to pitch oddballs at the thid and fouth notes (solid ed lines), pitch oddballs fom the fifth to seventh notes (dashed ed lines), efeence stimuli at the second and thid notes (solid black lines), and efeence stimuli fom the fouth to sixth notes (dashed black lines), all taken fom one epesentative paticipant in the pitch task. The uppe panel shows the magnetic field distibution at 130 ms fom the onset of the pitch oddballs at the thid and fouth notes. The lowe left and lowe ight panels show the esponses at the same posteio channels in the left and ight hemisphees, espectively. B: Time couses of the MMNm in the syllable (blue lines) and pitch tasks (ed lines), taken fom one epesentative paticipant. The data wee calculated by subtacting eventelated esponses to efeence stimuli fom those to pitch oddballs. The uppe panel shows the magnetic field distibution of the MMNm (afte subtaction) at 130 ms fom the onset of the fist pitch oddballs. C: The gand-aveaged ECD locations of the MMNm in the syllable (blue cicles) and pitch tasks (ed cicles). The ECDs wee supeimposed on a stuctual image showing a hoizontal slice at z 5 9. D: The ECD stength of the MMNm in the syllable (blue bas) and pitch tasks (ed bas). 595

9 Yasui et al. TABLE II. Repeated-measues ANOVAs fo the MMNm in expeiment 2 Left Right df F P F P F P Syllable task Stimulus type (oddball, efeence) 1, ** ** Stimulus position (fist, second half) 1, Stimulus type 3 Stimulus position 1, Pitch task Stimulus type (oddball, efeence) 1, ** 51 <0.0001** Stimulus position (fist, second half) 1, Stimulus type 3 Stimulus position 1, x y z Task (syllable, pitch) 1, Hemisphee (L, R) 1, Task 3 Hemisphee 1, Latency ECD stength Task (syllable, pitch) 1, Hemisphee (L, R) 1, * Task 3 Hemisphee 1, Regading the stimulus position, oddballs at the thid and fouth notes coespond to the fist half of the five notes used fo analyses (thee wee no oddballs at the fist and two notes), and those fom the fifth to the seventh notes coespond to the second half. Refeence stimuli at the second and thid notes coespond to the fist half, and those fom the fouth to the sixth notes coespond to the second half (efeence notes just befoe the fist appeaing oddballs wee used fo analyses). *, P < 0.05; **, P < was maginally significant in the ight hemisphee fo the pitch task; the wave amplitudes fo the oddballs on the second half wee significantly smalle than that on the fist half (F(1, 9) 5 11, P ), wheeas no significant effect of stimulus position was obseved fo the efeence stimuli (F(1, 9) , P 5 0.7). Theefoe, the effect of stimulus position cannot account fo the lage wave amplitudes of the oddballs that appeaed late than the efeence stimuli; moeove, this esult was obseved in the two tasks and both hemisphees (Table II). These esults confim that the obseved MMNm was indeed due to the pesence of oddballs. Next, we calculated ECDs of the MMNm fom the subtacted data, and the gand-aveage of the ECD locations fo the MMNm is shown in Figue 3C. The ECDs of the MMNm in the syllable task [left: ( , , ); ight: ( , , )] and those in the pitch task [left: ( , , ), ight: ( , , )] wee located in the posteio egion of Heschl s gyus. Neithe the main effects of task and hemisphee no thei inteaction wee significant fo all coodinates (Table II). Regading the ECD stength of the MMNm, the significant main effects of hemisphee wee obseved, showing that the ECD stength was significantly lage in the left hemisphee than in the ight hemisphee (Fig. 3D, Table II). To compae the esults of expeiments 1 and 2, theeway ANOVAs of expeiment (1 and 2) 3 task 3 hemisphee fo the peak latency, dipole coodinates, and ECD stength wee pefomed (Table III). By compaing the M140/M130 and MMNm, the significant main effects of expeiment wee obseved, such that the M140/M130 showed significantly longe peak latencies, and the ECD locations of the M140/M130 wee significantly moe medial, anteio, and maginally dosal to those of the MMNm. An inteaction between task and expeiment, as well as that between task and hemisphee, wee significant fo the ECD stength. The ECD stength of the MMNm in the pitch task was significantly smalle than that of the M130 in the ight hemisphee (F(1, 9) 5 5.7, P ), while thee was no significant diffeence between expeiments 1 and 2 fo the othe cases (P > 0.2). These esults stongly suggest that the M140 and M130 wee unique and independent of the popety of the fequency MMNm. Expeiment 3 To examine which factos modulated the M130 obseved in expeiment 1, we intoduced newly memoized songs in the melody task (Fig. 1D). It is possible that the M130 was elicited by unexpected notes deviating fom well-memoized songs. An altenative possibility is that the M130 was induced by peceiving unnatual pitch tansitions, independent fom the stoed memoy of songs. In expeiment 3, the paticipants wee tained with a fixed sequence of oiginal and modified songs befoe the MEG ecodings, and pefomed the melody task based on these newly memoized songs. If the M130 was elicited by unexpected notes deviating fom well-memoized songs, a ight-dominant M130 would be obseved unde the unex- 596

10 Latealization in Monitoing Lyics and Melody TABLE III. ANOVAs fo the compaison between the M140/M130 in expeiment 1 and the MMNm in expeiment 2 x y z df F P F P F P Expeiment (1, 2) 1, ** * Task (lyics/syllable, melody/pitch) 1, * Hemisphee (L, R) 1, Expeiment 3 Task 1, 88 < Expeiment 3 Hemisphee 1, Task 3 Hemisphee 1, Expeiment 3 Task 3 Hemisphee 1, Latency ECD stength Expeiment (1, 2) 1, * Task (lyics/syllable, melody/pitch) 1, ** Hemisphee (L, R) 1, Expeiment 3 Task 1, ** Expeiment 3 Hemisphee 1, Task 3 Hemisphee 1, ** Expeiment 3 Task 3 Hemisphee 1, *, P < 0.05; **, P < pected note condition (the puple column in Fig. 1D), but not unde the expected note condition (the oange column). On the othe hand, if the M130 was induced by peceiving unnatual pitch tansitions, independent fom the stoed memoy of songs, a ight-dominant M130 would be obseved fo the modified songs (blue cicles), but not fo the oiginal songs (geen cicles). The accuacies unde the expected note and unexpected note conditions fo the 10 paticipants wee ( )% and ( )%, espectively, and these two values wee not significantly diffeent (F(1, 9) 5 2.3, P 5 0.2). To extact pue deviant-induced fields, the event-elated esponses to the second notes wee subtacted fom the esponses to the thid notes. Regading the peak latencies of the M130, neithe the main effects of the expectation and hemisphee no thei inteaction wee significant (Table IV). Next, the ECDs wee calculated fom the subtacted data. The gand-aveage of the ECD locations fo the M130 is shown in Figue 4A. The ECDs of the M130 unde the unexpected note condition [left: ( , , ); ight: ( , , )] wee located anteio to Heschl s gyus. The ECDs of the M130 unde the expected note condition [left: ( , , ); ight: ( , , )] wee located anteio to Heschl s gyus in the left hemisphee, and located in the posteio egion of Heschl s TABLE IV. Repeated measues ANOVAs fo the effect of expectation o melody on the M130 in expeiment 3 x y z df F P F P F P Effect of expectation Expectation (expected note, unexpected note) 1, Hemisphee (L, R) 1, Expectation 3 Hemisphee 1, Latency ECD stength Expectation (expected note, unexpected note) 1, * Hemisphee (L, R) 1, Expectation 3 Hemisphee 1, Effect of melody Melody (oiginal, modified) 1, Hemisphee (L, R) 1, Melody 3 Hemisphee 1, < Latency ECD stength Melody (oiginal, modified) 1, Hemisphee (L, R) 1, Melody 3 Hemisphee 1, *, P < 0.05; **, P <

11 Yasui et al. Figue 4. The esults of expeiment 3, showing the ight-dominant M130 unde the unexpected note condition alone. A: The gand-aveaged ECD locations of the M130 unde the expected note (oange cicles) and unexpected note conditions (puple cicles). The ECDs wee supeimposed on a stuctual image showing a hoizontal slice at z 5 9. B: The ECD stength of the M130 unde the expected note (oange bas) and unexpected note conditions (puple bas). Unde the unexpected note condition, the M130 was significantly lage in the ight than in the left hemisphee. C: The gand-aveaged ECD locations of the M130 fo the oiginal (geen cicles) and modified songs (blue cicles). D: The ECD stength of the M130 fo the oiginal (geen bas) and modified songs (blue bas). gyus in the ight hemisphee. Neithe the main effects of expectation and hemisphee no thei inteaction wee significant fo any of the coodinates (Table IV). Regading the ECD stength, the main effects of expectation (expected note and unexpected note) wee significant, while an inteaction between expectation and hemisphee was maginally significant (Fig. 4B, Table IV). Unde the unexpected note condition, the ECD stength of the M130 was significantly lage in the ight than in the left hemisphee (F(1, 9) 5 5.8, P ), wheeas thee was no hemispheic dominance unde the expected note condition (F(1, 9) , P 5 0.8). In the ight hemisphee, the ECD stength was significantly lage unde the unexpected note than unde the expected note condition (F(1, 9) 5 9.1, P ), wheeas thee was no such diffeence in the left hemisphee (F(1, 9) , P 5 0.4). Next, we examined whethe o not the ight-dominant M130 was elicited automatically by the modified songs. The accuacies fo the oiginal and modified songs fo the 10 paticipants wee ( )% and ( )%, espectively, and these two values wee not significantly diffeent (F(1, 9) , P5 0.4). Neithe the main effects of melody and hemisphee no thei inteaction wee significant fo the peak latency, any of the ECD coodinates, o the ECD stength (Fig. 4C,D, Table IV). To claify whethe the M130 obseved unde the unexpected note condition was consistent with the M130 in expeiment 1, in which the same melody task was employed, we pefomed two-way ANOVAs of expeiment (1 and 3) 3 hemisphee fo the peak latency, dipole coodinates, and ECD stength (Table V). We confimed that the peak latency of the M130 wee consistent between expeiments 1 and 3. Regading the ECD locations, an inteaction between expeiment and hemisphee was maginally significant fo the z coodinate alone. In the left hemisphee, the ECDs in expeiment 3 wee significantly moe supeio to those in expeiment 1 (F(1, 22) 5 5.3, P ); no significant diffeence was obseved fo the z coodinate in the ight hemisphee (F(1, 22) , P 5 0.6). We also examined whethe the ECD stength of the M130 was consistent between expeiments 1 and 3. Regading the main effects of task (P 5 0.1), the diffeence between expeiments 1 and 3 fo the ECD stength in the ight hemisphee was nam coveing zeo (with the confidence level of 95%). The main effects of hemisphee wee significant fo the ECD stength (P ), as expected fom the ight-dominant M130. The esults of the ight-dominant M130, as well as its matched peak latency, ECD locations, and ECD stength in the ight hemisphee, suggest that the M130 in expeiment 1 is consistent with that in expeiment 3. Since the ight-dominant M130 was clealy obseved only unde the unexpected note condi- TABLE V. ANOVAs fo the M130 unde the unexpected note condition in expeiment 3 and the M130 in the melody task in expeiment 1 x y z df F P F P F P Expeiment (1, 3) 1, Hemisphee (L, R) 1, Expeiment 3 Hemisphee 1, Latency ECD stength Expeiment (1, 3) 1, Hemisphee (L, R) 1, * Expeiment 3 Hemisphee 1, *, P < 0.05; **, P <

12 Latealization in Monitoing Lyics and Melody tion in expeiment 3, we conclude that the M130 in expeiment 1 was indeed modulated by memoy-elated factos. DISCUSSION In the pesent MEG study, we demonstated the hemispheic dominance of the auditoy cotex that is selectively modulated by deviant types of songs consisting of both lyics and melody, theeby elucidating unde which conditions the latealization of auditoy pocessing changes. In expeiment 1 using familia songs, we found that the ECD stength of the M140 to the lyics deviants was left-dominant, wheeas that of the M130 to the melody deviants was ight-dominant (see Fig. 2). Moeove, the ECDs of the M130 wee located significantly anteio to the P1m foci. In expeiment 2 using familia songs with a constant syllable o pitch, the ECD stength of the fequency MMNm elicited by oddballs was left-dominant (see Fig. 3). Thee wee significant main effects of expeiment (1 and 2) fo the peak latencies and fo the ECD coodinates, indicating that the M140 and M130 wee not the fequency MMNm. In expeiment 3 using newly memoized songs, the ightdominant M130 was obseved only unde the unexpected note condition (see Fig. 4), independent of peceiving unnatual pitch tansitions (i.e., peceptual saliency). Moeove, the effect of selective attention to the melody of songs can also be excluded, as discussed in detail below. The consistent ight-dominance of the M130, as well as its matched peak latency, ECD stength, and ECD coodinates between expeiments 1 and 3, suggests that the M130 obseved in expeiment 1 is due to unexpected notes deviating fom well-memoized songs, but not due to unnatual pitch tansitions. On the othe hand, the left-dominant M140 was elicited by lyics deviants, suggesting the influence of top-down linguistic infomation and the memoy of the familia songs. We thus conclude that the left-latealized M140 and ight-latealized M130 eflect the expectation based on top-down infomation of language and music, espectively. The effect of selective attention has been examined by pevious neuoimaging studies, and it is known that selective attention to auditoy stimuli enhances neual esponses in the auditoy cotex [Alho et al., 1999; Hashimoto et al., 2000; Johnson and Zatoe, 2005; Neelon et al., 2006]. The diffeential modulation of hemispheic dominance in detecting the lyics and melody deviants evealed in expeiment 1 might also be due to selective attention to paticula components of songs, although thee have been no pevious epot compaing language and music components of the same stimuli. Howeve, the fequency MMNm obseved in the pitch task was compaable between the hemisphees (Fig. 3D), even when the selective attention to the music component was equied. Moeove, the selective attention to the melody was equied unde all of the conditions in expeiment 3, but nevetheless the ight-dominant M130 was obseved only unde the unexpected note condition (Fig. 4B, D). Theefoe, ou esults clealy established that memoy-elated components, independent of selective attention, specifically modulate the ight-dominant M130. A ecent MEG study has epoted an activity in the bilateal supeio tempoal aea within ms of auditoy stimulus onset, which was obseved only when paticipants could pedict an incoming sound and the pesented sound was an unexpected one [Aoyama et al., 2006]. The authos in this pevious study suggested that this component was simila to the MMNm, wheeas the ECD stength was not significantly diffeent between hemisphees. Fequency MMNm esponses ae known to oiginate in the tempoal cotex, and they ae automatically elicited by oddballs even when paticipants ae asked to ignoe the stimuli [Alho et al., 1998; Imaizumi et al., 1998; Knösche et al., 2002; Näätänen et al., 2005]. Howeve, the unexpected notes in the pesent expeiments 1 and 3 wee not fequency deviants, and the peak latency of the M140/M130 in expeiment 1 wee significantly longe than that of the MMNm in expeiment 2. This diffeence might be due to the fact that the deviants in expeiment 1 wee hade to detect than the oddballs in expeiment 2, but the ight dominance of the M130 confimed by expeiment 3 suggests the expectation based on top-down infomation athe than bottom-up featues like detectability of stimuli. Indeed, thee was no cue in the acoustic featues pe se to disciminate deviants fom the efeence stimuli in the melody task of expeiments 1 and 3. The melody deviants could be disciminated by using expectation based on stoed memoy of the melody sequence, and this pocess would pobably elicit the M130. These esults indicate the pesence of memoy-elated souces in the supeio tempoal cotex, which ae functionally distinct fom the fequency MMNm. The ECDs of the M140/M130 wee located significantly anteio to that of the P1m located in the posteio egion of Heschl s gyus (Fig. 2B, E). One putative auditoy egion located anteio to the human pimay auditoy aea (A1) has been poposed as the medial aea (MA), based on the patten of cytochome oxidase and acetylcholinestease activity [Rivie and Clake, 1997]. This pevious anatomical study egaded the MA as an upsteam association aea, defined as a unimodal association cotex eceiving diect input fom A1, and as an intemediate level between A1 and the supeio tempoal aea. One might speculate that the ECDs of the M140/M130 ae located in the MA, which subseves intemediate auditoy pocessing and combines both bottom-up infomation fom A1 and top-down infomation fom the highe auditoy aeas. Futhemoe, the M140/M130 eflecting top-down pocesses occus at the latency of 140/130 ms, which is clealy late than the P1m. These esults suggest that top-down infomation stats to influence auditoy pocessing as ealy as 130 ms, which may be useful in a fast identification and detection of stimulus changes. The ight hemispheic dominance of fontal and tempoal egions in pocessing posodic infomation has been 599