Modulation of the N400 potential during auditory phonological/ semantic interaction

Transcription

1 Cognitive Brain Research 17 (2003) locate/ cogbrainres Research report Modulation of the N400 potential during auditory phonological/ semantic interaction Fabien Perrin a,b, *, Luis Garcıa-Larrea a EA-1880, University Claude Bernard, Lyon, France b Sleep Disorders Unit, Neurological Hospital, 59bd. Pinel, Lyon, France c INSERM (EMI), Human Neurophysiology Laboratory at the CERMEP, 59bd Pinel, Lyon, France Accepted 15 January 2003 a,c Abstract The processing of phonological and semantic word attributes has been commonly explored with electrophysiological methods using simple contexts where competition between features is eliminated. Conversely, the interaction between phonological and semantic attributes has not been systematically examined. We therefore recorded an event-related electrophysiological marker of word discordance, the N400, in response to sequences of auditory word pairs containing semantic incongruences, phonological discordances, or a mixture of the two. N400 enhancement to semantically unrelated words was systematically observed, whether the subjects heard the sequences passively (no instruction) or actively (semantic judgement task), and even in contexts where the task did not concern semantic attributes. In contrast, the N400 effect to phonologically unrelated (non-rhyming) words was exclusively obtained in the active situation (phonological judgment), while it disappeared in passive conditions and during semantic/ phonological interference. This suggests that the detection of semantic incongruences is a more robust and automatized mechanism than that of phonological ones, and tends to occlude this latter when both features are in competition. Our data also provide new elements supporting the persistence of the semantic N400 during shallow word processing tasks, i.e. tasks that discourage analysis of semantic aspects of the words Elsevier Science B.V. All rights reserved. Theme: Neural basis of behavior Topic: Cognition Keywords: Event-related potentials (ERPs); Auditory processing; Phonological priming/ detection; Semantic priming/ detection 1. Introduction a particular sentence [39], Kutas and Hillyard [18] noted that the more a word was unexpected, the more the N400 Words that are semantically anomalous relative to a amplitude increased. This led the authors to consider the given context enhance a negative wave in event-related N400 amplitude as inversely related to the subject s brain potentials (ERPs) peaking at about 400 ms post- semantic expectancy. stimulus, and labeled accordingly N400 [16]. The am- Negative brain potentials of similar characteristics have plitude of the N400 has been shown to be directly been described in response to items (words or pseudocorrelated to the word s semantic incongruence, either words) that either end [35,36] or start [9,32] by a when the target word arrives at the end of a sentence phonological deviance relative to the preceding word. [16,21] or as the second word in a paired-words paradigm These phonological N400 (sometimes labeled N450 for [1,2]. Using the cloze probability, which corresponds to the rhyme and PMN for the alliteration) appear to be the proportion of subjects using a given word to complete dependent to some extent on the sensory modality of the stimulus. Thus, in rhyme/ non-rhyme paradigms the N400 effect to visually presented words was observed exclusive- *Corresponding author. Present address: Cyclotron Research Centre/ ˆ ly when the subjects actively detected the presence or Liege University/ Batiment B30, Sart Tilman/Allee du 6 aout, 8, B-4000 Liege, Belgium. Tel.: ; fax: absence of a rhyme [35 38], while in the auditory address: fabien.perrin@univ-lyon1.fr (F. Perrin). modality this effect appeared not only during a rhyme /03/$ see front matter 2003 Elsevier Science B.V. All rights reserved. doi: / S (03)

2 F. Perrin, L. Garcıa-Larrea / Cognitive Brain Research 17 (2003) judgement task but also during a lexical detection task cal and semantic aspects interfere mutually. The relations [31,13]. Since the non-rhyming N400 effect was obtained between different mismatch markers when several types also when using non-words, Rugg [36] hypothesized that of language-related discordances are simultaneously pres- in the rhyme matching task, subjects generate or attend to ent (i.e. in more ecological contexts), has not been a set of candidate words likely to match with the first systematically studied so far. This is an important issue word and develop an enhanced negativity when the word since placing different stimulus features in competition is a they hear/ see does not belong to this set. In this line, straightforward method of assessing whether one is more several authors have demonstrated an orthographic N400 robust (i.e. occludes) the other. In an elegant study, in response to the second word of a pair when it does not Connolly and Philips [9] recorded ERPs while subjects look alike (i.e. is orthographically unrelated) to the first listened to sentences terminal words which were, or not, one [14,38]. phonologically identical to the highest semantically-ex- As a result of this previous literature, an unified concept pected sentence completion over the first several hundred of discordance negativities has emerged [29,15] which milliseconds, and which were semantically congruous or considers the N400 potential as a member of the stimulus incongruous to the sentence context. Although they demmismatch family of responses, first described by Naatanen onstrated an enhanced early negativity (the phonological et al. [24] for acoustic mismatch. However, if all these mismatch negativity ) to words that started with noncomponents share the sensitivity to environmental mis- concordant phonemes, no real competition between matches, each of them seems to have specific properties, phonological and semantic features could be tested since notably regarding their respective behavior in situations alliteration is detected at the initial phoneme, and thus where the mismatch is not consciously detected. The earlier than sense, which becomes available only about the mismatch negativity (MMN) to acoustic deviance is end of the word. This difficulty was partially overcome in evoked even in the absence of attention directed to the a recent study by Radeau et al. [34] who compared ERPs stimulus, as when the subject reads a book (for a review in conditions of simultaneous phonological and semantic see Ref. [26]). In contrast, N400 amplitude differences discordances with those obtained to phonological or between semantically concordant and discordant words are semantic discordances alone. The discordance negativity greatly attenuated or absent when subjects attention is was maximal in case of double incongruity, and minimal directed toward other stimuli, as in dichotic listening or to phonological mismatch alone; however the absence of visual selective attention paradigms [3,22,28]. However, control (congruent) and no-task conditions precluded to there is also converging evidence that the N400 to compare these N400 against a baseline, and to evaluate semantic discordances may appear, both for visual and the automaticity of these processes. auditory stimulus presentation, even in the absence of In the present study, we investigate the influence of active attention to the words meaning, such as in tasks auditory semantic and phonological features, separately that discourage analysis of semantic aspects of words and in combination, on both behavioral responses and [4,7,8,20], or even when subjects are sleeping [5,30] (but cortical ERPs, in particular the discordance negativities see also Ref. [6]). Conversely, no such concordant evi- of the N400 type. Notably, the modulation of the N400 dence exists regarding the phonological N400, whose potential was explored during three different phonological/ behavior in non-attentive conditions seems to be modality- semantic conditions, by using sequences of word-pairs dependent. While in the visual modality the phonological consisting of (a) words phonologically related or not (by N400 to rhyme has been reported only during active rhyme); (b) words semantically related or not (by meanphonological discrimination tasks [15,35], Praamstra and ing), and (c) a mixture of phonological and semantic colleagues [31,32] described a phonological N400 for concordances/ discordances. Our main purpose was therenon-rhyming words presented acoustically even when fore to study the interactions between semantic and attention was diverted from phonological aspects by a phonological features of auditory stimuli and their relative lexical detection task. capability to influence behavioral parameters and N400. One could suggest that the existence of semantic and phonological N400s in the absence of active attention is the sign that subjects can automatically detect both seman- 2. Methods tic and phonological discordances. While this automatic mechanism would be very useful to improve auditory 2.1. Subjects semantic detection in everyday life, it is more difficult to explain the usefulness of a similar mechanism for Ten volunteers (five women and five men, phonological detection. It may be alternatively hypoth- years), without audiological or neurological disease, all but esized that each of the two systems may be triggered one right-handed, and all native French speakers particiautomatically in simple contexts (when phonological and pated to the experiments. All gave informed consent to the semantic concordances/ discordances are not in competi- study, which was conducted in agreement with the guidetion) but not in more complex situations where phonologi- lines of the Declaration of Helsinki.

3 38 F. Perrin, L. Garcıa-Larrea / Cognitive Brain Research 17 (2003) Stimulation procedure (10 pairs of each combination) were constructed for this sequence type. Stimuli consisted of 200 word pairs, formed by one In Sequences of type 2 all pairs were phonologically prime and one target. Four different types of prime- unrelated, but could be or not semantically related target association were constructed. Thus, each prime (S1Ph2 and S2Ph2). Five blocks of 20 pairs each word could be associated with either: (10 pairs of each combination) were constructed for this sequence type. Sequences of type 3 were formed by a mixture of the (a) one semantically (meaning) and phonologically (rhym- four possible combinations (S1Ph1, S2Ph1, S1 ing) related target word (S1Ph1) [ex: animal-cheval Ph2 and S2Ph2). Five blocks of 40 pairs each (10 (animal-horse)]; pairs of each possible combination) were constructed (b) one semantically unrelated but phonologically related for this sequence type. target word (S-Ph1) [ex: animal-fiscal (animal-fiscal)]; Thus, Sequence 1 explored the phonological axis only, (c) one semantically related but phonologically unrelated and Sequence 2 the semantic axis only, while both target word (S1Ph-) [ex: animal-brebis (animal-ewe)]; semantic and phonological match/ mismatch were included and in Sequence 3. This last sequence was therefore used to (d) one semantically and phonologically unrelated target test semantic/ phonological interaction. word (S-Ph-) [ex: animal-juge (animal-judge)] Recording Previous to the study, the degree of semantic and Electroencephalographic (EEG) signals from 31 tin phonological association between each pair of words was electrodes (International system) referenced to the tested on 10 subjects different from those participating to nose, and electrooculogram (EOG) from two electrodes, the ERP experiment. In this pre-test, subjects had to make diagonally above and below the right eye, were amplified a judgment on the strength of semantic or phonological ( ) and sampled at 500 Hz using an analogue association on a 5-point scale (from 05no association to bandpass of Hz. A ground electrode was placed 55very strong association). Word pairs scoring higher than between Fz and Fpz and impedance at all electrodes was 3.9 for semantic and 4.0 for phonology were retained as kept below 5 kv. the related pairs, and those scoring lower than 1.1 for semantic and 0.8 for phonology were retained as the 2.5. Experimental procedure unrelated pairs. The overall degree of semantic and After installation of the head cap and of two miniphonological concordance/ discordance across the test was earphones inserted into the external acoustic canals, subequated by matching each semantic word pair with one jects were comfortably seated on a chair. They were phonological one of similar score (within 0.2 points). instructed to fix their gaze to a point in front of them. Two Words were selected from a table of frequency for French attentional conditions were used. In the passive condition words [10] and it was checked that the mean occurrence of no particular task was given to the subjects, who were use in French language was in the same frequency range simply asked to listen attentively to the 15 blocks of for the four types of target words (20 to 30 per million). stimuli. The order of presentation of blocks in passive All words were recorded by the same male voice and were situation proceeded always from sequence 1 to sequence 3 digitized and replayed binaurally at 70 db SPL maximal (to minimize the influence of semantic relations upon intensity. All target words were disyllabics, had a mean rhyming relations). In the active condition, participants length of 452 ms, whatever the type of target word (S1 were asked to press, with their dominant hand, a right or Ph1, S1Ph2, S2Ph1, S2Ph2), and appeared 900 ms left button according to whether the pairs rhymed or not after the beginning of the prime, which had a mean length (blocks of sequence 1 ), or were semantically related or not of 515 ms. Pairs were separated by a silence of 1900 ms, (blocks of sequence 2 ). For blocks of sequence 3, which this implying that the interval between the onset of two contained both phonological and semantic mismatches, successive primes was 3300 ms. subjects were instructed first to respond on semantic features (two blocks), and then on phonological features 2.3. Stimulus sequences (two other blocks). The order of presentation of blocks was Three types of sequences were elaborated with these counterbalanced across subjects, but passive conditions word pairs: always preceded active conditions. This experimental design allowed to study the auditory N400-type responses to: In Sequences of type 1 all pairs were semantically related, but could be or not phonologically related (S1Ph1 and S1Ph2). Five blocks of 20 pairs each (a) phonological discordances alone,

4 F. Perrin, L. Garcıa-Larrea / Cognitive Brain Research 17 (2003) (b) semantic incongruences alone, Since previous work has suggested a lateralized am- (c) phonological discordances in the presence of semantic plitude predominance for the phonological N400 [36] and congruences/ incongruences, for the semantic N400 [17,19], separate ANOVAs were (d) semantic incongruences in the presence of phonologi- performed to test the existence of laterality effects on the cal concordances/ discordances, N400 under our experimental conditions. For the three types of sequences the mean areas under the curve within and this in both passive and active conditions. the three consecutive N400 windows (N400a, N400b and N400c) were submitted to similar repeated measures 2.6. Data analysis ANOVA than described above, but using lateral temporal Mean reaction times (RTs), 6standard deviation, were electrodes (T3 and T4) as levels for the electrode site obtained for the different types of target words in all factor. For all analyses, no significant effect nor interacconditions. For sequences 1 and 2, t-tests were calculated tions were noted for the electrode side factor, and therefore between RTs to related targets and RTs to unrelated ones. the core analysis was restricted to midline electrodes in For sequence 3, a two-way, repeated measures ANOVA order to maximise the N400 effect, which is known to using the presence or absence of semantic link (related vs. predominate over the midline. unrelated words) and the presence or absence of Additionally, we calculated the difference of N400 areas phonological link (rhyming vs. non-rhyming words) as between related and unrelated target words (at Pz site), and within-subject factors were performed on RTs, and post- submitted values to a 3-way ANOVA with repeated hoc t-tests were applied only when main effects reached measures on the type of condition (passive vs. active) the significance. presence or absence of relatedness/ unrelatedness Auditory evoked potentials to the second (target) word (phonological vs. semantic) and the number of changing of each pair were analysed over a 1700 ms window (which features in the sequence (one only phonological or only included a prestimulus baseline of 200 ms), after being semantic vs. two semantic and phonological simultandigitally filtered between 0.8 and 30 Hz (23 db, 24 eously ). db/ octave). Only results reaching significance at P,0.05, after a Four ERP components were assessed for each sequence Geisser Greenhouse correction, are presented in the Reand condition: N1, N2, N400 and the late positive com- sults section; post-hoc t-tests were applied only when main ponent (LPC) (see for example Fig. 2, upper right). The effects reached significance or two factors interacted magnitude of these components was evaluated by measur- significantly. ing the mean areas under the curve within six consecutive latency windows. The area of N1 was calculated between 50 and 200 ms; and that of N2 between 200 and 300 ms. 3. Results For N400, three consecutive areas were calculated, corresponding to three consecutive 100 ms windows: the N400a 3.1. Behavioral data ( ms), N400b ( ms) and N400c ( ms). Finally, the LPC area was calculated between 600 and Mean reaction times (RTs) were analyzed for correct 800 ms. These intervals were chosen on the basis of responses exclusively, since incorrect responses repregrand-averaged traces across sequences and conditions. sented only 2% of the data. In sequences of type 1 (when Analyses of variance (ANOVAs) on area values were the source of match/ mismatch was phonological) subjects performed separately for each sequence. For sequence 1 responded significantly faster to rhyming (i.e. phonologi- ( phonological), values within each window were submitted cally related) than non-rhyming words (t ; P,0.02; to a three-way, repeated measures ANOVA using the type Fig. 1, upper left panel). Similarly, in sequences of type 2 of condition (passive vs. active), the presence or absence (semantic context) RTs were faster to semantically congruof phonological link (rhyming vs. non-rhyming words) and ous words than to incongruous words (t ; P, the electrode position (Fz vs. Cz vs. Pz vs. Oz) as within ; Fig. 1, lower left panel). subject factors. For sequence 2(semantic), values in each During the mixed context sequences (sequences of type window were submitted to a three-way, repeated measures 3, where phonological and semantic mismatch coexisted), ANOVA using the type of condition (passive vs. active), the subjects were asked to respond to phonological or to the presence or absence of semantic link (related vs. semantic features of the word pairs (see Methods). When unrelated words) and the electrode position (Fz vs. Cz vs. they had to respond on phonological features, subjects Pz vs. Oz) as within-subject factors. For sequence 3(mixed were faster to detect rhyming than non-rhyming words phonological/semantic), a 4-way ANOVA was used, with (F1, ; P,0.004; Fig. 1, upper right panel) but type of condition (passive vs. phonological active vs. only when words were also congruous (t ; P,0.008; semantic active), presence or absence of semantic link n.s. for incongruous words). Moreover, RTs were faster to (related vs. unrelated words), presence or absence of semantically congruous words than to incongruous words phonological link (rhyming vs. non-rhyming words), and (F1, ; P,0.02) only when they were phonologicalelectrode position as within-subject factors. ly related (t ; P,0.006). In the same mixed con-

5 40 F. Perrin, L. Garcıa-Larrea / Cognitive Brain Research 17 (2003) Fig. 1. Mean reaction times (6standard deviation) obtained for the different types of target words (S1Ph1, S2Ph1, S1Ph2, S2Ph2), during sequence 1 (phonological context), sequence 2 (semantic context) and sequence 3 (mixed contexts both in response to phonological and semantic features). Significant differences (P,0.05) are represented by asterisks. texts, when subjects had to respond on semantic features, dow, but only during its central segment, N400b (P5 RTs were faster to semantically related than unrelated 0.008). This reflected a higher N400 amplitude in response words (F1, ; P ; Fig. 1, lower right panel), to phonologically unrelated word pairs than to related when they were also phonologically related (t ; P, ones. There was also a significant interaction between 0.005) but not when they were phonologically unrelated. phonological link and condition, since the enhanced N400b RTs were never faster to rhyming than non-rhyming to non-rhyming words was only significant during the words. active listening paradigm (post-hoc tests: t ; P, in active condition; n.s. in passive condition). A 3.2. Event related potentials significant effect of the phonological link (rhyming vs. non-rhyming words) was observed for the LPC (P50.01), Globally, a significant effect of the type of condition its mean amplitude being greater after non-rhyming words. (active vs. passive listening) was observed on all com- Again, this was only the case in the active condition as ponents (N1, N2, N400a, N400b, N400c and LPC), mean shown by the interaction between the phonological link amplitudes being greater in active than passive conditions and the condition and by post-hoc tests (t ; P,0.005 (with the exception of the ms window values in active condition; n.s. in passive condition) (Table 1, Fig. which were close to zero in both cases). The electrode 2). position had also a significant effect on all components (save for N1), reflecting their restricted scalp distribution Semantic context: sequence Phonological context: sequence 1 A significant effect of the semantic link (related vs. A significant effect of the phonological link (rhyming unrelated words) was observed all along the N400 window, vs. non-rhyming words) was observed in the N400 win- N400a, N400b and N400c (P , P and P5

6 F. Perrin, L. Garcıa-Larrea / Cognitive Brain Research 17 (2003) Table 1 F and P values (after G G correction) of the ANOVAs performed on N1, N2, N400 (each 100 ms) and LPC mean areas obtained during sequence 1 Main effects Interactions [1] [2] [3] [1]3[2] [1]3[3] [2]3[3] Active Phon. related Electrode vs. passive vs. unrelated position N1 [ ms] F(1,9) P N2 [ ms] F(1,9) F(3,27) F(3,27) P P P N400 N4a [ ms] F(1,9) F(3,27)54.81 P P N4b [ ms] F(1,9) F(1,9) F(3,27) F(1,9)59.07 P P P P N4c [ ms] F(1,9) F(3,27) P P LPC [ ms] F(1,9) F(3,27) F(1,9) F(3,27) P P P P Only significant effects of the three factors [type of condition (passive vs. active), presence or absence of phonological link (rhyming vs. rhyming words) and electrode position (Fz vs. Cz vs. Pz vs. Oz)], and their interactions are presented. Fig. 2. Grand-averages of AEPs to phonologically related (S1Ph1)/ unrelated (S1Ph2) words during passive and active conditions of Sequence 1, and to semantically related (S1Ph2)/ unrelated (S2Ph2) words during passive and active conditions of Sequence 2. Significant differences (P,0.05) obtained between related and unrelated words were hatched and represented with scalp topography.

7 42 F. Perrin, L. Garcıa-Larrea / Cognitive Brain Research 17 (2003) , respectively), reflecting higher N400 amplitudes in passive vs. phonological active : F1, ; P,0.03). response to semantically unrelated words than to related This suggests that the late N400 (N400c) to phonologiones. This was the case only in active condition for N400a, cally unrelated words was significantly more negative than as shown by the interaction between semantic link and to phonologically related words only during the phonologicondition and post-hoc tests (t ; P,0.02 in active cal active condition (post-hoc tests: t952.9; P,0.02 in condition; n.s. in passive condition); and both during phonological active condition; n.s. in passive and semantic passive and active conditions for N400b and N400c (no active conditions). A significant effect of semantic link interaction between semantic link and condition). Con- (related vs. unrelated words) was observed for the LPC cerning the LPC, a significant effect of the semantic link (P,0.02), this reflecting higher amplitude after semanti- (related vs. unrelated words) was observed (P50.03), its cally unrelated words. However, there was also a signifimean amplitude being greater after semantically unrelated cant interaction between the semantic link and the conthan related words. However, this was the case only in dition, indicating that enhanced LPC to semantically active condition as shown by the interaction between the unrelated words was only significant during both passive semantic link and the condition and post-hoc tests (t 952 and semantic active condition, but not in the phonological ; P,0.02 in active condition; n.s. in passive con- ly active situation (post-hoc tests: t ; P,0.02 in dition) (Table 2, Fig. 2) Mixed ( phonological and semantic) context: 3, Fig. 3). sequence 3 passive condition; t ; P,0.007 in semantic ac- 9 tive condition; n.s. in phonological active condition) (Table All relatedness/ unrelatedness effects are summarized in Fig. 4. A significant effect of the semantic link (related vs. unrelated words) was noted in the N400 window, during 3.5. Comparison between semantic and phonological the two first segments, N400a and N400b (P50.01 and N400 P , respectively), the mean amplitude of the wave being enhanced in response to semantically unrelated The N400 effect was isolated in difference waves words. This effect was observed during both passive and obtained by subtraction of ERPs to related words from active conditions in the N400a window, but only in the those to unrelated words. Comparison of such difference semantic active condition in the N400b window as shown waves between pooled phonological and semantic contexts by the interaction between presence or absence of semantic is shown in Table 4. A significant effect of the condition link and type of condition and post-hoc tests (t ; (active vs. passive) was observed in the central segment of P in semantic active condition; n.s. in passive and N400 (N400b, ms) reflecting a greater effect in phonological active conditions). Also, a near significant active than in passive conditions. A near-significant differinteraction was noted between presence or absence of ence in N400b was also noted between phonological and phonological link and type of condition in the N400c semantic contexts, the N400b effect being greater in the window (this effect becoming significant with another latter (F1, , P50.06). Finally, a significant inter- ANOVA where the type of condition was restricted to action between the number of features in the sequence (one Table 2 F and P values (after G G correction) of the ANOVAs performed on N1, N2, N400 (each 100 ms) and LPC mean areas obtained during sequence 2 Main effects [1] [2] [3] Interactions Active Sem. related Electrode [1]3[2] [1]3[3] [2]3[3] vs. passive vs. unrelated position N1 [ ms] F(1,9) P N2 [ ms] F(1,9) F(3,27) P P N400 N4a [ ms] F(1,9) F(3,27) F(1,9) P P P N4b [ ms] F (1,9) F(1,9) F(3,27) F(3,27) P P P P N4c [ ms] F(1,9) F (3,27) F(3,27) P P P LPC [ ms] F (1,9) F(1,9) F(3,27) F(1,9) F(3,27) P P P P P Only significant effects of the three factors [type of condition (passive vs. active), presence or absence of semantic link (related vs. unrelated words) and electrode position (Fz vs. Cz vs. Pz vs. Oz)], and their interactions are presented.

8 Table 3 F and P values (after G G correction) of the ANOVAs performed on N1, N2, N400 (each 100 ms) and LPC mean areas obtained during sequence 3 main effects [1] [2] [3] [4] interactions pass vs. phon phon. related sem. related electrode [1]3[2] [1]3[3] [1]3[4] [2]3[3] [2]3[4] [3]3[4] act vs. sem act vs. unrelated vs. unrelated position N1 [ ms] F(2,18) P F(6,54) P N2 [ ms] F(2,18) P F(3,27) F(6,54) P P N400 N4a [ ms] F(2,18) F(1,9) F(3,27) F(2,18) F(6,54) F(1,9) P P P P P P N4b [ ms] F(1,9) F(3,27) F(2,18) F (3,27) F(3,27) P P P P P N4c [ ms] F (2,18) F(3,27) F(2,18) F(6,54) P P P P LPC F (2,18) F(1,9) F(3,27) F(2,18)55.7 F(3,27) P P P P P Only significant effects of the four factors [type of condition (passive vs. phonological active vs. semantic active), presence or absence of semantic link (related vs. unrelated words), presence or absence of phonological link (rhyming vs. rhyming words) and electrode position (Fz vs. Cz vs. Pz vs. Oz)], and their interactions are presented. F. Perrin, L. Garcıa-Larrea / Cognitive Brain Research 17 (2003)

9 44 F. Perrin, L. Garcıa-Larrea / Cognitive Brain Research 17 (2003) Fig. 3. Grand-averages of AEPs to phonologically related (average of S1Ph1 and S2Ph1)/ unrelated (average of S1Ph2 and S2Ph2) words, and to semantically related (average of S1Ph1 and S1Ph2)/ unrelated (average of S2Ph1 and S2Ph2) words during passive and the two active conditions of Sequence 3. Significant differences (P,0.05) obtained between related and unrelated words were hatched and represented with scalp topography. vs. two) and the presence or absence of relatedness/ 4. Discussion unrelatedness (phonological vs. semantic) was noted in the last segment of N400, with a greater N400c semantic effect 4.1. Behavioral responses in simple than in mixed contexts combining semantic and phonological features (post-hoc tests: t ; P, In simple contexts i.e. when phonological or semantic for semantic relatedness/ unrelatedness; n.s. for matches/ mismatches were presented alone both phonological relatedness/ unrelatedness) (Table 4). phonologically related (rhyming) words and semantically Fig. 4. Qualitative representation of significant differences (P,0.01 and P,0.05) of all components (N1, N2, N400a, N400b, N400c and LPC) between related and unrelated words (Ph1/Ph or S1/S2), relative to each condition (passive and active conditions of sequences 1, 2 and 3).

10 F. Perrin, L. Garcıa-Larrea / Cognitive Brain Research 17 (2003) Table 4 F and P values (after G G correction) of the ANOVAs performed on the difference waves obtained by subtraction of N400 (each 100 ms) to related words from those to unrelated ones [1] [2] [3] Interactions one vs. two active phonological changing features vs. passive vs. semantic [1]3[3] N4a diff. [ ms] N4b diff. [ ms] F(1,9) P F(1,9) P N4c diff. [ ms] F(1,9) P Only significant effects of the in N400 amplitude area (each 100 ms), for the three factors [number of changing features in the sequence (one vs. two), on the type of condition (passive vs. active) and on the presence or absence of relatedness/ unrelatedness (phonological vs. semantic)] and their interactions are presented. related (congruous) words were detected faster than unre- response to semantic incongruence than to phonological lated words. These results are in accordance with the mismatch. In other words, our data suggest that semantic general finding that, in a number of contexts, the detection processing is a default mode, and that rhyme processing is of identity requires less time than the detection of differ- optionally engaged. This conclusion is supported by three ences [23,27,33]. sets of results that are discussed in what follows. Firstly, When both features (phonological and semantic) were in only the semantic N400 appeared in both passive and competition within the same sequence (sequence 3 ), active conditions. Conversely, the phonological N400 was congruous words were detected faster than incongruous obtained only in the active listening paradigm (i.e. when words (when semantics was the task), and the detection of subjects had to pay actively attention to phonological rhyming words required less time than that of non-rhyming features) but disappeared in the passive condition (Fig. 2). words (when phonology was the task). Furthermore, these Secondly, in active conditions where both semantic and effects were observed only when the other feature was also phonological N400 were obtained, the semantic N400 concordant but not when it was discordant, suggesting that effect was significantly greater and lasted longer than its when one of the feature is discordant the facilitation effect phonological counterpart (Fig. 2, Table 4). Finally, in dissipates. Finally, it is interesting to note that, in mixed mixed contexts (when phonological and semantic discordcontexts, RTs were faster for semantically congruous than ances were intermingled), a semantic N400 was always for incongruous words even when subjects paid attention elicited in response to semantically incongruous words, selectively to phonological concordant features. This latter even in passive conditions or when the subjects task was effect suggests that semantic priming [15] is operant when phonological. Conversely, a very small and hardly signifithe subjects attention was focused on the phonological cant phonological N400 appeared in mixed contexts, and aspects of the stimuli, i.e. that a semantic field of this only in the phonological active condition, suggesting congruous words was more automatically opened when- that when the semantic and phonological axes of processever a word was heard. ing were in competition the semantic N400 tended to occlude the phonological N400 (Fig. 3). Our data provide some new elements to the long-lasting 4.2. Event related potentials debate on the possible existence of a semantic N400 during shallow word processing tasks, i.e. with tasks which No amplitude differences between related and unrelated discourage analysis of semantic aspects of the words [11]. stimuli were observed in N1 and N2 components whatever While some studies noted the occurrence of semantic N400 the context (semantic, phonological, or mixed) or condition with tasks that would not require a semantic analysis (active and passive). This suggests that these early ERP [4,7,8,20], other authors did not observe any N400 under components, which are known to be sensitive to the similar conditions [6,12]. In our study, paying attention physical characteristics of auditory stimuli [25], are on the specifically to the phonological aspects of words (i.e. contrary unaffected by the phonological or semantic discouraging their active semantic analysis) decreased the analysis proper (Fig. 4). Thus only N400 (and in some amplitude of the semantic N400, but did not prevent its 1 contexts the LPC ) appeared sensitive to phonological and occurrence, which remained statistically significant. This semantic matches/ mismatches. The main result of the present study was that the N400 was more robust in 1 The differences observed in the LPC area were probably related to those obtained to the N400 component. This has been explained in previous literature by the fact that the end of the N400 potential is partially overlapped by the subsequent LPC (see for example Ref. [1]). result strongly suggests that the mechanism of mismatch detection underlying the semantic N400 is a very powerful one, which is set up automatically even in the absence of any explicit instruction, and even when the subject is explicitly oriented toward a different task. The auditory modality of stimulus presentation has been shown to be more robust than the visual modality to

11 46 F. Perrin, L. Garcıa-Larrea / Cognitive Brain Research 17 (2003) produce phonological N400 effects [15,31,35], and there- As in most previous work in this field, the experiments fore the use of auditory stimuli in this study should have reported herein were conducted in highly trained and maximized the chances to enhance this component. In spite educated western individuals. This clearly limits the doof this, the phonological auditory N400 proved to be a main where our results may be considered valid, and much less robust phenomenon than its semantic counter- prompts the necessity of further studies with similar design part, since it was never observed in passive conditions but applied to other population subclasses, such as children (Figs. 2 and 3, left columns), and, during the active or illiterate persons. Also, it should be stressed that most situations appeared exclusively when the subjects atten- experimental and natural language contexts are not tion was specifically focused on the phonological aspects phonologically-oriented, as rhyming words do not comof the stimuli. This suggests that the brain mechanisms monly occur together in natural sentences, and therefore underlying the detection of phonological discordance are responses to phonological discordances may be smaller not automatically set up in every context, but come into because phonological expectancies are lower. This situaview when the phonological attributes become task rel- tion may be different within rhyming contexts, such as evant. Previous literature on this issue is scanty and when listening to an Elizabethan or Romantic theatre play, controversial. Thus, in the study by Kramer and Donchin and such situations would of course warrant specifically [15], subjects had to decide if two visually presented words designed studies. either rhymed or looked alike. Similarly to the present results, a phonological N400 was observed only when phonology was the task (i.e. when subjects had to decide on the basis of rhyme) and not when the task was Acknowledgements orthographic (decide whether the words were visually Fabien Perrin was supported by the Dotation Française similar). Conversely, Praamstra and colleagues [31,32] de Recherche Veille et Sommeil Sanofi-Synthelabo 1999 observed an auditory phonological N400 for non-rhyming Part of this work was supported by a grant on words even during a lexical detection task (where subjects «Apprentissage des langues: dysfonctionnements et had to decide if the second stimulus of a pair was a word remediations» (programme cognitique: ecole et sciences or a non-word). This difference with our data may be cognitives; project n8 LA47) to Luis Garcia-Larrea, from explained by the extreme simplicity of the lexical task in the French Ministry of Research and Technology. We Praamstra et al. s studies (decide whether the stimulus was thank one of the anonymous reviewers for his precious or not a word) as compared to the more elaborated comments. semantic discrimination required by our procedure (decide whether a word belongs to the same semantic class as a previous one). Taken together, these and our present results support the notion that the phonological N400 tends to be References occluded in case of concomitant and significant semantic [1] S. Bentin, G. McCarthy, C.C. Wood, Event-related potentials, lexical processing of the words, although it might be present in decision and semantic priming, Electroenceph. Clin. Neurophysiol. case of concomitant very simple lexical tasks. 60 (1985) The lack of phonological N400 effect in purely passive [2] S. Bentin, M. Kutas, S.A. Hillyard, Electrophysiological evidence for task effects on semantic priming in auditory word processing, (no task) situations might have been favored by a relatively Psychophysiology 30 (1993) long ISI, since in previous literature this effect has been [3] S. Bentin, M. Kutas, S.A. Hillyard, Semantic processing and obtained only for short ISIs, in lexical or shadowing tasks memory for attended and unattended words in dichotic listening: [13]. Although ultra-short ISIs may increase the robustness behavioral and electrophysiological evidence, J. Exp. Psychol. Hum. of phonological priming, they are however non ecologi- Percept. Perform. 21 (1995) cal, and probably irrelevant for the understanding of [4] M. Besson, I. Fischler, T. Boaz, G. Raney, Effects of automatic associative activation on explicit and implicit memory tests, J. Exp. mechanisms sub serving human language processing in Psychol. Learn. Mem. Cogn. 18 (1992) everyday life. An ISI of 900 ms between prime and target [5] J. Brualla, M.F. Romero, M. Serrano, J.R. Valdizan, Auditory eventwords was therefore chosen in this study, as it was related potentials to semantic priming during sleep, Electroenceph. 2 compatible with common spoken language conditions. Clin. Neurophysiol. 108 (1998) Under such constraint, our results indicate a lack of [6] D.J. Chwilla, C.M. Brown, P. Hagoort, The N400 as a function of the level of processing, Psychophysiology 32 (1995) automaticity of phonological comparison processes in [7] J.F. Connolly, S.H. Stewart, N.A. Phillips, The effects of processing passive conditions, which contrats with the spontaneous requirements on neurophysiological responses to spoken sentences, tendency to process the semantic features of the stimuli, to Brain Lang. 39 (1990) the detriment of their phonological attributes. [8] J.F. Connolly, N.A. Phillips, S.H. Stewart, W.G. Brake, Event- related potential sensitivity to acoustic and semantic properties of terminal words in sentences, Brain Lang. 43 (1992) It is noteworthy that the temporal decay of automatic electrophysiologi- [9] J.F. Connolly, N.A. Phillips, Event-related potential components cal detectors of auditory mismatch, such as the MMN, is of about 10 s reflect phonological and semantic processing of the terminal word of [26], i.e. much longer than the ISI used in this study. spoken sentences, J. Cogn. Neurosci. 6 (1994)

12 F. Perrin, L. Garcıa-Larrea / Cognitive Brain Research 17 (2003) [10] A. Content, P. Mousty, M. Radeau, BRULEX: Une base de donnees [25] R. Naatanen, T. Picton, The N1 wave of the human electric and lexicales informatisee pour le Français ecrit et parle, L annee magnetic response to sound: a review and an analysis of the Psychologique 90 (1990) component structure, Psychophysiology 24 (1987) [11] F.I.M. Craik, R.S. Lockhart, Levels of processing: A framework for [26] R. Naatanen, Mismatch negativity (MMN): perspectives for applicamemory research, J. Verbal Learn. Verbal Behav. 11 (1972) 671 tion, Int. J. Psychophysiol. 37 (2000) [27] R.S. Nickerson, On the time it takes to tell things apart, in: J. Requin [12] D. Deacon, F. Breton, W. Ritter, H.G. Vaughan, The relationship (Ed.), Attention and Performance, Vol. VII, Erlbaum, Hillsdale, NJ, between N2 and N400: Scalp distribution, stimulus probability, and 1978, pp task relevance, Psychophysiology 28 (1991) [28] T. Okita, T. Jibu, Selective attention and N400 attenuation with [13] N. Dumay, A. Benraiss, B. Barriol, C. Colin, M. Radeau, M. spoken word repetition, Psychophysiology 35 (1998) Besson, Behavioral and electrophysiological study of phonological [29] M.C. Perez-Abalo, R. Rodriguez, M.A. Bobes, J. Gutierrez, M. priming between bisyllabic spoken words, J. Cogn. Neurosci. 13 Valdes-Sosa, Brain potentials and the availability of semantic and (2001) phonological codes over time, NeuroReport 5 (1994) [14] K.D. Emmorey, Auditory morphological priming in the lexicon, [30] F. Perrin, L. Garcıa-Larrea, H. Bastuji, Detection of verbal discord- Lang. Cogn. Proc. 4 (1989) ances during sleep, NeuroReport 13 (2002) [15] A.F. Kramer, E. Donchin, Brain potentials as indices of orthographic [31] P. Praamstra, D.F. Stegeman, Phonological effects on the auditory and phonological interaction during word matching, J. Exp. Psychol. N400 event-related brain potential, Cogn. Brain Res. 1 (1993) Learn. Mem. Cogn. 13 (1987) [16] M. Kutas, S.A. Hillyard, Reading senseless sentences: brain po- [32] P. Praamstra, A.S. Meyer, W.J.M. Levelt, Neurophysiological manitentials reflect semantic incongruity, Science 207 (1980) festations of phonological processing: latency variation of a negative [17] M. Kutas, S.A. Hillyard, The lateral distribution of event-related ERP component timelocked to phonological mismatch, J. Cogn. potentials during sentence processing, Neuropsychologia 20 (1982) Neurosci. 6 (1994) [33] R.W. Proctor, A unified theory for matching task phenomena, [18] M. Kutas, S.A. Hillyard, Brain potentials during reading reflect word Psychol. Rev. 88 (1981) expectancy and semantic association, Nature 307 (1984) [34] M. Radeau, M. Besson, E. Fonteneau, S.L. Castro, Semantic, [19] M. Kutas, C. Van Petten, M. Besson, Event-related potential repetition and rime priming between spoken words: behavioral and asymetries during the reading of sentences, Electroenceph. Clin. electrophysiological evidence, Biol. Psychol. 48 (1998) Neurophysiol. 69 (1988) [35] M.D. Rugg, Event-related potentials in phonological matching tasks, [20] M. Kutas, S.A. Hillyard, An electrophysiological probe of incidental Brain Lang. 23 (1984) semantic association, J. Cogn. Neurosci. 1 (1989) [36] M.D. Rugg, Event-related potentials and the phonological process- [21] W.C. McCallum, S.F. Farmer, P.V. Pocock, The effects of physical ing of words and non-words, Neuropsychologia 22 (1984) and semantic incongruities on auditory event-related potentials, [37] M.D. Rugg, S.E. Barrett, Event-related potentials and the interaction Electroenceph. Clin. Neurophysiol. 59 (1984) between orthographic and phonological information in a rhyme- [22] G. McCarthy, A.C. Nobre, Modulation of semantic processing by judgment task, Brain Lang. 32 (1987) spatial selective attention, Electroenceph. Clin. Neurophysiol. 88 [38] T.F. Sanquist, J.W. Rohrbaugh, K. Syndulko, D.S. Lindsley, Electro- (1993) cortical signs of level processing: Perceptual analysis and recogni- [23] D.E. Meyer, R.W. Schvaneveldt, Facilitation in recognizing pairs of tion memory, Psychophysiology 17 (1980) words: Evidence of a dependance between retrieval operations, J. [39] W.L. Taylor, Cloze procedure: a new tool for measuring Exp. Psychol. 90 (1971) readabibility, Journalism Q. 30 (1953) [24] R. Naatanen, M. Simpson, N.E. Loveless, Stimulus deviance and evoked potentials, Biol. Psychol. 14 (1982)