Facing the Facts: Assessing the role of appearance-behavior incongruities through the modulation of the N400. Senior Thesis.

Transcription

1 Facing the Facts: Assessing the role of appearance-behavior incongruities through the modulation of the N400 Senior Thesis Presented to The Faculty of the School of Arts and Sciences Brandeis University Undergraduate Program in Neuroscience Dr. Angela Gutchess, Advisor In partial fulfillment of the requirements for the degree of Bachelor of Science by Jennifer Crawford May 2015 Copyright by Jennifer Crawford

2 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 2 Abstract A growing body of literature has begun to study the neural dynamics of person perception when people are asked to integrate social information from several domains simultaneously as opposed to studying how person perception is affected by appearance, behavioral, or non-verbal cues separately. Previous fmri work has revealed the role of the dorsal-medial prefrontal cortex (dmpfc) in updating social information, showing greater dmpfc activation for incongruent compared to congruent social and emotional information (Cassidy & Gutchess, in press). However, the neural mechanism involved in the updating of face- and behavior-based impressions has not been elucidated in a way that can capture the temporal dynamics of impression formation processes. We used event-related potentials (ERPs) to explore how incongruent versus congruent face-behavior information is processed as a function of the amplitude of the N400, an ERP component traditionally involved in semantic processing. Participants viewed faces (varying on degree of trustworthiness) paired with behavioral sentences (reflecting positive or negative character), which allowed us to identify the neural activity underlying the processing of both congruent and incongruent face-behavior relationships involved in social cognition by time-locking the presentation of the congruent and incongruent stimuli. ERP analyses indicate that when presented with incongruent face-behavior information, the amplitude of the N400 increases anteriorly compared to congruent face-behavior information. This suggests that when confronted with conflicting information about a person s facial characteristics and behavior, frontal networks may be recruited to resolve this incongruency.

3 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 3 Facing the Facts: Assessing the role of face-behavior incongruities through the modulation of the N400 Impressions shape social interactions in a myriad of ways. Much of the work on impression formation underlies how individuals form appearance-based (e.g. why I construe certain faces to convey untrustworthy qualities) and behavior-based (e.g. why I think a man is kind when he helps an elderly woman cross the street) impressions. One relies upon facial appearances when forming impressions of others and many of these appearance-based impressions are formed immediately, leading to instantaneous judgments of the person viewed (Willis & Todorov, 2006). Given that appearances influence how people value behavioral information, the implications are meaningful, whether it be the pursuit of a personal relationship, a job interview, a political candidate running for office, or the outcome of a criminal trial (Brownlow, 1992; Olivola & Todorov, 2010; Nisbett & Wilson, 1977; McCulloch et al., 2008; Ballew & Todorov, 2007; Downs & Lyons, 1991; Zebrowitz & McDonald, 1991; Blair et al., 2004). As with impressions formed from faces, behaviors also add important information that can inform inferences about character. People are more likely to use the behaviors of actors to inform traits about them and predict future behaviors using the traits inferred from the observed behavior even without conscious recollection of the behavior in question (Todorov & Uleman, 2002; McCarthy & Skowronski, 2011). While these studies provide evidence for appearanceand behavior-based cues and their relation to impression formation and provide valuable information about each of these processes by themselves, very little research has explored how people integrate appearance- and behavior-based cues together when forming impressions. Naturalistically, forming initial impressions of others involves integrating pertinent social information across many different domains (e.g., appearances, behaviors, non-verbal cues)

4 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 4 simultaneously. It is therefore important to understand the ways in which observers integrate cues from a variety of domains to learn about and form opinions of others in a meaningful social context. Recent work has begun to investigate how both facial features and behaviors affect character judgments, showing that people make more extreme judgments about others when their face and behavior are congruent (e.g., both suggest trustworthy or positive traits, rather than the face looking untrustworthy and the behavior suggesting positive character), and are more likely to remember impressions of others and approach people with appearance-behavior congruency (Cassidy & Gutchess, in press; Cassidy, Zebrowitz, & Gutchess, 2012). This line of research has been further extended using fmri to show that dorsal medial prefrontal cortex (dmpfc) activation is influenced by appearance and behavioral congruity (Baron, Gobbini, Engell, & Todorov, 2010; Vandekerckhove, Van Hoeck, & Van Overwalle, 2012; Cassidy & Gutchess, in press). Given the dynamic nature of person perception, research has begun to address the neural mechanisms involved in updating behavior-based impressions. When incoming behavioral information contradicts previously learned behaviors, there is an increase in activity in dmpfc and a broad impression formation network (rostrolateral PFC, superior temporal sulcus, and posterior cingulate cortex) compared to when incoming behavioral information is congruent with one s expectations about a person (Ma et al., 2012; Mende-Siedlecki, Cai, & Todorov, 2012). While this body of work largely seeks to study the neural dynamics involved in the congruity of behavior-based impression formation, it doesn t take the social information contained within the facial characteristics of the person viewed into account in a way that can accurately assess the rapid neural processing and context updating mechanisms taking place during impression formation. The present study aims to elucidate the neural mechanism involved in person

5 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 5 perception when facial appearances (varied on levels of trustworthiness) and behavioral information contradict each other through the modulation of event-related potentials (ERPs). ERPs offer an element of temporal specificity that other behavioral and neural measurements cannot provide in humans. Thus, ERPs are an ideal measurement tool to study the different components of information processing involved in person perception because they provide a way to identify the different, often fast-paced, cognitive processing mechanisms that mediate the links between the social context (appearance- and behavioral-based cues) and the resulting impression that is formed (Rugg & Coles, 1995; Ito & Cacioppo, in press). ERPs extend beyond the information contained within spatially-specific neural activations that other neuroimaging methods, like fmri, may provide in elucidating the mechanism underlying the resolution of conflicting appearance-behavior information. Using ERPs to study person perception processes helps to illuminate how incongruent versus congruent social information is resolved on a precise temporal scale. While the synchronous network-level activity in the brain is poorly characterized by region-specific analyses, like, fmri, ERPs are able to characterize these multimodal and complex neural processes and can begin to tell a story of the multifaceted, network-driven processes involved in the mechanism of updating person perception. In this regard, ERPs provide an advantage over more traditional measures of cognitive processing, like response latencies, because they are independent of behavioral responses. While behavioral responses have provided an integral portion of the surrounding work done within the field of person perception, ERPs present brain activity that is generally less controllable than selfreport measures and are not confounded by response-related motor responses and task-specific requirements of other behavioral measures (McCarthy & Donchin, 1975; McCarthy & Donchin, 1981; Ito & Cacioppo, 2000). Previous studies have shown that while congruency affects ratings

6 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 6 for approachability, behavioral measures, such as memorability aren t as sensitive as one might hope in characterizing person perception processes (Cassidy & Gutchess, in press). By providing a direct measure of cortical activity, ERPs bypass many of the confounds present for self-report measures and begin to more accurately characterize the phenomena the behavioral measures aim to assess. The N400 is an ERP component ranging from approximately ms that has been reliably shown to reflect the processing of semantic associations (Kutas & Federmeier, 2011; Kutas & Hillyard, 1980; Holcomb & Neville, 1991). The amplitude of the response is most susceptible to manipulation and is likely to vary with many of the same measures that influence reaction time in behavioral studies, such as expectancy, semantic priming, and attention (Kutas & Federmeier, 2011). While much of the N400 literature pertains to the amplitude changes related to semantic context violation (Kutas & Federmeier, 2011), N400 effects have been found to generalize across many different types of stimuli, including faces, and domains of research, such as stereotyping and trait inferences (van den Brink et al., 2010; Goto, Yee, Lowenberg, & Lewis, 2013; White, Crites, Taylor, & Corral, 2009; Wieser, Gerdes, Büngel, Schwarz, Mühlberger, & Pauli, 2014). The present study continues this trend of extending the N400 into social contexts by studying the incongruity of face- and behavior-based cues in impression formation and the underlying neural mechanism used to resolve this conflict. Unlike previous N400 studies, the present study further extends the canonical, unimodal stimulus, paradigm used to study the effects of congruity by incorporating two different types of stimuli that together establish congruence: behavioral sentences and faces. This allows for the social information contained within faces to be more accurately assessed during impression formation, reflecting a

7 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 7 more naturalistic impression updating mechanism as indexed by the modulation of the N400 in response to appearance-behavior incongruity. It is important to note that in many of the studies that used the N400 as a neural marker for the processing of valenced stimuli, differences in electrode topography across the scalp gave evidence to a fn400 as opposed to what is generally seen as the N400 (Kutas & Federmeier, 2011). The fn400 is characterized by greater frontal electrode activation as opposed to traditional centro-parietal activation of electrodes as evidenced in language studies (Kutas & Federmeier, 2011). This helps to shape our ideations of where we believe the strongest network activations needed to resolve appearance-behavior incongruency reside and why examining the N400 from frontal midline sites (Fz) in addition to the more traditional N400 sites (Cz, Pz) will be most advantageous in characterizing the neural mechanisms involved in the resolution of appearance-behavior incongruity. While N400 is has typically been regarded as an excellent electrophysiological marker of processing in a distributed semantic memory system, its extension to the literature on social and cognitive processing has been underdeveloped. This creates an exciting opportunity to build upon the conceptual ideations of the N400 when examining how online neural processing differs when face and behavioral information are incongruent versus congruent. In the present study, we predict a larger amplitude of the N400 in response to incongruent face- and behavior-based pairs (e.g. untrustworthy face and positive behavior) in comparison to congruent face and behavioral pairs (e.g. untrustworthy face and negative behavior). The stronger response to the incongruent relationship likely results from the more effortful processing and, potentially, additional network activation needed to resolve the disparity between the conflicting social information of the faces and corresponding behaviors viewed.

8 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 8 It is possible, however, that the N400 may not reflect the incongruity of face and behavioral cues, but rather the appearance-behavior incongruity could give rise to the late positive potential (LPP). The LPP is subcomponent of the P300, which is an ERP component associated with brain s response to novelty, commonly described as an indicator of context updating in working memory (Luck, 2005). This subcomponent has been given special characterization for the increased positive amplitude ranging from ~ ms and is modulated by the emotional intensity of a stimulus (Gray et al., 2004). Further research has also established that scenes including people elicit larger LPPs than those scenes without people (Gray et al., 2004). The working hypothesis for LPP is that it reflects a temporary increase in attention, which serves to facilitate the processing of the affective stimulus that elicited the LPP [enhanced perception hypothesis]. A recent study by Delaney-Busch & Kuperberg (2013) hypothesizes that the late positivity merely reflects greater processing of emotional salience (e.g., emotionally charged words are arousing or motivating enough to trigger a greater allocation of attentional resources required for further analysis) that cannot be processed in the narrow time constraints of the N400 window. The prioritization of emotional salience may reduce the semantic analysis of the N400 in some contexts; prioritizing access to emotional information can lead to relatively superficial semantic analysis, but can quickly trigger an affective response. Conversely, Van Overwalle et al. (2011) argue that while previous studies on trait inferences have not reported the modulation of the N400, it is likely because it was overshadowed by the larger and broader LPP component. The effortful and widespread neural processing needed to update one s impressions of people is likely spread across many different neural systems and processes and is thus distributed throughout many different brain regions (and ERP components) after encountering incongruent face and behavioral information. However, the N400 still provides valuable

9 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 9 information by providing an operationalized construct used to evaluate the initial stages of effortful neural processing related to person perception that can show very clear differences in the amount, and timing, of semantic processing when faces and behaviors are congruent versus incongruent. Method Participants Twenty-six younger adults recruited from Brandeis University participated and provided informed consent. Of the twenty-six original participants, six were excluded due to excessive artifacts (>30% of trials), leaving twenty participants whose data were included in the final analyses (18-32 years old (M=20.17); 14 females). Participants were right-handed, native English speakers (or had learned before the age of five), and had no history of neurological, affective, or psychiatric problems. All participants had normal or corrected-to-normal vision. The Brandeis University Institutional Review Board approved this study. Stimuli Faces. Forty-eight trustworthy, forty-eight neutral, and forty-eight untrustworthy faces were chosen from three differing sets of faces that were generated using the software FaceGen version 3.1 ( and acquired from a publically available lab database of generated faces ( All faces were male, Caucasian, and had a unique identity. All 144 faces used in this experiment had been previously rated for trustworthiness and emotionality (please refer to Cassidy & Gutchess, in press, for more detailed analyses). Behaviors. Forty-eight positive, forty-eight neutral, and forty-eight negative behaviors were chosen from a database of behavioral sentences (Somerville, Wig, Whalen, & Kelley,

10 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES ). The behavioral sentences were selected using the arousal and valence ratings of 185 total behaviors by 23 younger adults (please refer to Somerville et al., 2006 for detailed analyses). Face-behavior pairs. The forty-eight positive and forty-eight negative behaviors were assigned evenly among the trustworthy and untrustworthy faces. The forty-eight neutral faces were paired only with the forty-eight neutral behaviors. Trustworthy faces paired with positive sentences (24 trials) and untrustworthy faces paired with negative sentences (24 trials) were classified as congruent. Trustworthy faces paired with negative sentences (24 trials) and untrustworthy faces paired with positive sentences (24 trials) were classified as incongruent. Neutral faces paired with neutral behaviors were classified as neutral. There were two task versions, each counterbalanced on the assignment of faces and behaviors for congruent and incongruent trials for a total of 48 congruent, 48 incongruent and 48 neutral face-behavior pairs. Procedure Participants were seated in a dark, quiet room 0.7m in front of a computer monitor. All stimuli were presented using E-Prime 2.0 (Psychology Software Tools, Pittsburgh, PA). Participants viewed twenty face-behavior pairs as a part of a practice task before viewing the congruent, incongruent, and neutral pairs described above. No face-behavior pair was viewed more than once by each participant. Participants were instructed to read and comprehend each scenario presented to them; no other demands were imposed. The 144 trials were broken up into three blocks, each approximately six minutes in duration. Each behavioral sentence was shown for 1500ms. The sentence was followed by a fixation cross that was jittered ( ms); the corresponding face was then shown for 1000ms (see Figure 1). There was a 1500ms pause in

11 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 11 between each trial. Participants were advised to avoid eye movements, shifting, and other larger movements while words and/or pictures were on the screen. ERP Recording & Data Analysis EEG signals were recorded using Ag/AgCl electrodes from thirty-two channels according to the International system (see Figure 2). Two supplementary electrodes were used as supports in signal processing: one below the left eye and one at the outer right canthus. Electrode impedances were below 5kΩ. Signals were recorded using a BioSemi ActiveTwo amplifier (Cortech Solutions, Wilmington, NC) using a 512-Hz sampling rate. After off-line re-referencing of the EEG signals to the left and right mastoids, the EEG signals were filtered using a 1-Hz high-pass filter and ICA was run to correct for ocular artifacts using the runica algorithm in EEGLAB b (Delorme & Makeig, 2004), a freely available open source toolbox ( running under Matlab (MathWorks, Natick, MA) The following analyses were all performed in ERPLAB , an open-source toolbox for the analysis of event-related potentials (Lopez-Calderon & Luck, 2014). Epochs were created ranging from 200ms before the onset of the face to 800ms after. Each epoch was baseline corrected by subtracting the mean amplitude from -200ms to 0ms pre-stimulus interval and semi-automatically inspected for eye movements, muscle artifacts, shifting, or other anomalies. Segments containing such artifacts (±100µV) were rejected (8.2%). The remaining EEG signals were averaged per participant and condition, generating averaged ERPs. The remaining ERP signals were filtered with a 30Hz low-pass filter. The mean amplitude of the N400 ( ms) was analyzed from three channels for each participant for incongruent and congruent face-behavior pairs: Fz, Cz, and Pz. This allowed for the direct comparison of online neural processing when participants viewed congruent versus incongruent face-behavior pairs.

12 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 12 Results Behavioral Data After ERPs were recorded, participants were asked to rate how well they thought each face-behavior pair went together using a nine-point scale (e.g. 1=face-behavior pair does not go together at all, 9= face-behavior pair goes together very well). Congruent face-behavior pairs (M=6.56, SD=0.90) were rated higher than incongruent face-behavior pairs (M=3.72, SD=0.88), t(19)= 8.018, p< However, there was no significant difference between the reaction time when rating congruent face-behavior pairs (M=2.57s, SD=0.27s) and incongruent face-behavior pairs (M=2.56s, SD=0.23s), p=0.76. These findings indicate that participants are able to distinguish between congruent and incongruent face-behavior pairs and reliably rate them as being such. This provides a manipulation check verifying that participants viewed the congruent and incongruent stimuli as we had intended. ERP Data The grand-average ERP waveforms of the three selected midline channels (Fz, Cz, Pz) for the congruent and incongruent conditions in addition to each respective difference wave (incongruent-congruent) are shown in Fig. 3, with twenty participants and 48 trials per condition before artifact rejection. 1 A paired samples t-test revealed that the average amplitude during the a priori defined N400 window ( ms) was significantly more negative following incongruent appearance-behavior information (M=-1.31, SD=4.69) compared to congruent appearancebehavior information (M=0.53, SD=2.23) at Fz, t(19) = 2.173, p=0.044, reflecting an increased N400 effect for incongruent pairs. While the same pattern of greater negativity for incongruent as compared to congruent appearance-behavior pairs is observed at Cz, the effect is not 1 After artifact rejection, the number of trials per condition were as follows: M congruent =44.01, M incongruent =44.21, M neutral =44.37.

13 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 13 significant (p>0.2); no such effect is observed at Pz (p>.71). Taken together, these patterns of results show an anterior N400 effect reflected through enhanced frontal negativity for incongruent as compared to congruent face-behavior pairs at Fz (see Figure 4). 2 Exploratory Analyses Research has extensively shown that increased levels of empathy are related to greater levels of prosocial behaviors and attitudes towards strangers (Eisenberg & Miller, 1987); however, the ways in which empathy may affect how easily one s expectancies about a stranger are violated when faced with dissonant information about the stranger has yet to be fully explored. Participants were asked to complete the Interpersonal Reactivity Index, IRI, (Davis, 1980) to compare how one s levels of empathy correlate with how easily his or her expectancy is violated. Using the previously defined N400 window, the mean amplitude of the difference wave at Fz was correlated with each of the four sub-scales of empathy (perspective taking, fantasy, empathetic concern, personal distress) as defined by the Interpersonal Reactivity Index. We predicted that empathetic concern would have the strongest relationship to the N400 response because the empathetic concern subscale aims to assess other-oriented feelings and concern, which is likely subject to violations when there is conflicting face-behavior information. However the prediction was not supported (p=.196), nor were there significant relationships with any other subscales (ps>.538), (see Table 1). 2 Additional analyses were performed to characterize the effects of gender and the affective properties of the stimuli (positive/negative sentences, trustworthy/untrustworthy faces) on the amplitude of the ERP waveforms collected from ms. No significant difference was found for the N400 difference wave amplitude between female and male participants at Fz, Cz, or Pz, ps>.283. The contribution of positive and negative behaviors and trustworthy and untrustworthy faces were compared within congruency (congruent, incongruent) at Fz, Cz, and Pz; no significant effects were found, ps>.243.

14 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 14 Discussion This study assessed whether appearance-behavioral incongruities generate increased N400 amplitudes as compared to congruent appearance-behavioral information. This hypothesis was supported at the anterior, midline channel, Fz, showing increased N400 amplitude during the defined window ( ms) for incongruent relative to congruent trials. Scalp activations provide further support in establishing this pattern of frontal negativity during the N400 window for incongruent versus congruent face-behavior pairs. In line with the more traditional view of the N400, these results are consistent with the idea that the increased amplitude of the N400 following appearance-behavioral incongruity likely reflects the additional processing needed to integrate the competing stimuli from preexisting schemas into a working conceptual framework (Federmeier & Laszlo, 2009) in order to resolve conflicting social cues. The results of this study indicate that during impression formation, it is more difficult to take competing facial and behavioral cues and form a working person perception concept. Prior behavioral work supports this claim in that during impression formation, observers anticipate the traits and behaviors of the observed to be consistent and that the processing of incongruent appearance-based information requires additional effort (Hamilton & Sherman, 1996; Macrae, Bodenhausen, & Milne, 1994; Macrae et al., 1999). Recent ERP work on stereotypes (Van Berkum, van den Brink, Tesink, Kos & Hagoort, 2008; White et al., 2009) and spontaneous trait interferences (Baetens, Van der Cruyssen, Achtziger, Vandekerckhove, & Van Overwalle, 2011) also find similar N400 effects that are attributed to this additional network activity needed to resolve competing sources of social information. While much of the work using the N400 has studied the effect in lexico-semantic domains, this experiment contributes to a growing body of literature that extends these N400 effects to social

15 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 15 domains and opens up new ways to conceptualize and study the modulation of the N400 in social contexts. It is important to note that the distribution of the scalp activity within the N400 time window was more anterior than the usual centro-parietal regions observed in other N400 studies (Kutas & Federmeier, 2011). However, many of these studies are comprised of stimuli lacking positive or negative affect and utilize very simple sentence presentations of stimuli (Federmeier & Laszlo, 2009). Many researchers have argued that valenced stimuli elicit more frontal scalp activations than those conveying no affect (Delaney-Busch & Kuperberg, 2013; Kanske & Kotz, 2007; Pratt & Kelly, 2008). It is possible that when confronted with affective stimuli, additional resources are needed to process this information. The affective stimuli used in this study may account for why an anterior shift is observed in scalp localization. There have, however, been N400 studies using affective stimuli that have elicited the more traditional parietal activation (Holt, Lynn, & Kuperberg, 2009). This leads us to question whether it is the affective properties of the stimuli that modulate this anterior N400 effect or if the greater functional significance of the social context studied in these experiments can be used to explain this anterior shift as a result of frontally-recruited networks used to resolve conflicting sources of social information. Rather than attributing all of the anterior distribution of the N400 to the properties of the affective stimuli used in this study, it is possible that there may be more frontally based activity needed to resolve competing face-behavioral information or similar processes that can explain this effect. Our work provides an alternative account for the anterior shift in the scalp localization of the N400 by attributing this effect to the differential neural networks that mediate cortical activity used for conflict-updating processes involved in person perception, extending the anterior shift hypothesis beyond just the presence of affective stimuli alone. The data show

16 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 16 greater anterior negativity when people are confronted with incongruent relative to congruent face-behavior information. This complements a recent finding by Cassidy & Gutchess (in press) that found increased medial prefrontal cortex (mpfc) activity when presented with incongruent appearance-behavior information as compared to congruent appearance-behavioral information. While ERP scalp localization should not be used as an approximation of the activity of the neural structures immediately surrounding the regions analyzed, the correspondence of increased mpfc activation to the significant N400 effect observed anteriorly may inform a potential mechanism used to resolve appearance-behavior incongruity during person perception and explain the anterior shift in the N400. Previous work has shown that mpfc activity can be extended to reflect mentalizing. When considering the findings of Cassidy & Gutchess (in press), a study that used the same stimuli employed in this study, that found increased mpfc activation for incongruent (versus congruent) face-behavior information, it is possible that increased amplitude of the N400 for incongruent relative to incongruent pairs could contribute to the working hypothesis that more mentalizing is needed to integrate social cues coming from different (e.g. appearance, behavioral) domains. Our work provides important information about the network-based activity used in the processing of incongruent social cues that, when applied to prior neuroimaging work, provides a strong link to frontal-mediated mentalizing process when resolving incongruent facebehavioral information. While our exploratory goal to relate individual differences in empathy to the amplitude of the N400 response was not supported by the data, it is still worth considering how empathy may be related to the congruency effects in person perception and through what methodologies this relation might be best measured. While there was a correlation (r = -.320) between empathetic

17 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 17 concern and the N400 amplitude, this value did not approach significant. Given the relatively small number of questions (7) used in the IRI to address empathetic concern and the relatively small sample size tested (N=20), it s possible that this measure fails to illuminate a more nuanced relationship of one s empathy levels relative to his or her levels of expectancy violations indexed by the amplitude of the N400 when using such a small sample size to disentangle moderate effects. Future experiments may be able to utilize the IRI to mediate an understanding of the relationship of an individual s empathy to the amplitude of the N400 with greater specificity by incorporating a larger participant pool to study this effect. Much of the prior work using the N400 has utilized very simple experimental constructs to study lexico-semantic processing. While the extension of the N400 into social domains remains an exciting endeavor, there s a persistent gap in our knowledge of what exactly underlies the activity modulated by the N400 in social contexts. Related to this underlying cortical activity, it still remains unclear why the N400 is anterior in some contexts and not others; whether this anterior shift is related directly to the affective content of the stimuli or rather the larger network activations needed to process other types of social or affective information is not fully understood. However, this study is able to provide added support for frontal network activity in mediating the resolution of incongruent relative to congruent facebehavioral information, further reinforcing the hypothesis that additional mentalizing is needed to integrate conflicting cues across different modalities in person perception. It s very possible that the blunt, synchronous neural activity conveyed by ERPs through the mean amplitude of the waveform may not be enough to accurately convey the types networks activated during person perception. Further research may be able to utilize the general time course of activity described

18 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 18 in this study to elucidate the spatial nuances and connectivity patterns during person perception to get a better quantifiable measure of congruency effects in person perception.

19 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 19 References Baetens, K., Van der Cruyssen, L., Achtziger, A., Vandekerckhove, M., & Van Overwalle, F. (2011). N400 and LPP in spontaneous trait interferences. Brain Research, 1418, Ballew, C., & Todorov, A. (2007). Predicting political elections from rapid and unreflective face judgments. Proceedings of the National Academy of Sciences, 104(46), Baron, S., Gobbini, M., Engell, A., & Todorov, A. (2010). Amygdala and dorsomedial prefrontal cortex responses to appearance-based and behavior-based person impressions. Social, Cognitive, & Affective Neuroscience, 6, Blair, I. V., Judd, C. M., & Chapleau, K. M. (2004). The influence of afrocentric facial features in criminal sentencing. Psychological Science, 15, Brownlow, S. (1992). Seeing is believing: facial appearance, credibility, and attitudechange. Journal of Nonverbal Behavior, 16, Cassidy, B.S., & Gutchess, A.H. (in press). Influences of appearance-behaviour congruity on memory and social judgments. Memory. Cassidy, B.S., & Gutchess, A.H. (in press). Neural responses to appearance-behavior congruity. Social Cognition. Davis, M. H. (1980). A multidimensional approach to individual differences in empathy. JSAS Catalog of Selected Documents in Psychology, 10, 85. Delaney-Busch, N. & Kuperberg, G. (2013). Friendly drug-dealers and terrifying puppies: Affective primacy can attenuate the N400 effect in emotional discourse contexts. Cognitive, Affective, and Behavioral Neuroscience, 13, 3,

20 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 20 Delorme, A., & Makeig, S. (2004). EEGLAB: an open source toolbox for analysis of single trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134, Downs, C., & Lyons, P. (1991). Natural observations of the links between attractiveness and initial legal judgments. Personality and Social Psychology Bulletin, 17, 5, Eisenberg, N., & Miller, P. A. (1987). The relation of empathy to prosocial and related behaviors. Psychological Bulletin, 101(1), 91. Federmeier, K.D., Laszlo, S. (2009). Time for meaning: electrophysiology provides insights into the dynamics of representation and processing in semantic memory. Psychology of Learning and Motivation, 51, Finholt, T. A., & Olson, G. M. (1997). From laboratories to collaboratories: A new organizational form for scientific collaboration. Psychological Science, 8, 1, Goto, S.G., Yee, A., Lowenberg, K., & Lewis, R.S. (2013). Cultural differences in sensitivity to social context: Detecting affective congruity using the N400. Social Neuroscience, 8, 1, Gray, H.M., Ambady, N., Lowenthal, W.T., & Deldin, P. (2004). P300 as an index of attention to self-relevant stimuli. Journal of Experimental Social Psychology 40, 2, Hamilton, D., & Sherman, S. (1996). Perceiving persons and groups. Psychological Review, 103, 2, Holcomb, P.J., & Neville, H.J. (1991). The electrophysiology of spoken sentence processing. Psychobiology, 19,

21 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 21 Holt, D.J., Lynn, S.K., Kuperberg, G.R. (2009). Neurophysiological correlates of comprehending emotional meaning in context. Journal of Cognitive Neuroscience, 21,11, Ito, T.A., & Cacioppo, J.T. (2000). Electrophysiological evidence of implicit and explicit categorization processes. Journal of Experimental Social Psychology, 36, 6, Ito, T. A., & Cacioppo, J. T. (in press). Physiological measures of implicit attitudes. In B. Wittenbrink & N. Schwartz (Eds.), Implicit measures of attitudes. New York: Guilford Press. Kanske, P., Kotz, S.A. (2007). Concreteness in emotional words: ERP evidence from a hemifield study. Brain Research, 1148, Kutas, M., and Federmeier, K. D. (2011). Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). Annu. Rev. Psychol. 62, Kutas, M. & Hillyard, S.A. (1980). Event-related brain potentials to semantically inappropriate and surprising words. Biological Psychology, 11, Luck, S. J. (2005). An introduction to the event-related potential technique. Cambridge, MA: MIT Press. Ma, N., Vandekerckhove, M., Baetens, K., Van Overwalle, F., Seurinck, R., & Fias, W. (2012). Inconsistencies in spontaneous and intentional trait inferences. Social Cognitive and Affective Neuroscience, 7, 8,

22 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 22 Ma, N., Vandekerckhove, M., VanOverwalle, F., Seurinck, R., & Fias, W. (2011). Spontaneous and intentional trait inferences recruit a common mentalizing network to a different degree: Spontaneous inferences activate only its core areas. Social Neuroscience, 6(2), Macrae, C. N., Bodenhausen, G., & Milne, A. (1994). Stereotypes as energy-saving devices: A peek inside the cognitive toolbox. Journal of Personality and Social Psychology, 66, Macrae, C. N., Bodenhausen, G., Schloerscheidt, A., & Milne, A. (1999). Tales of the unexpected: executive function and person perception. Journal of Personality and Social Psychology, 76(2), McCarthy, G. & Donchin, E. (1975). Differences between sinistrals' and dextrals' ability to infer a whole from its parts: A failure to replicate. Neuropsychologia, 13, McCarthy, G. & Donchin, E. (1981). A metric for thought: A comparison of P300 latency and reaction time. Science, 211, McCullough, M. E., Kimeldorf, M. B., & Cohen, A. D. (2008). An adaptation for altruism? The social causes, social effects, and social evolution of gratitude. Current Directions in Psychological Science, 17, Mende-Siedlecki, P., Cai, Y., & Todorov, A. (2012). The neural dynamics of updating person impressions. Social Cognitive and Affective Neuroscience. Lopez-Calderon, J., & Luck, S. J. (2014). ERPLAB: An open-source toolbox for the analysis of event-related potentials. Frontiers in Human Neuroscience, 8, 213. Nisbett, R. E., and Wilson, T. D. (1977). Telling more than we can know: verbal reports on mental processes. Psychological Review, 84,

23 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 23 Olivola, C., & Todorov, A. (2010). Fooled by first impressions? Reexamining the diagnostic value of appearance-based inferences. Journal of Experimental Social Psychology, 46, Pratt, N.L., Kelly, S.D. (2008). Emotional states influence the neural processing of affective language. Social Neuroscience, 3(3 4), Rugg, M. D., & Coles, M. G. H. (Eds.) (1995). Electrophysiology of mind: Event-related brain potentials and cognition. New York: Oxford University Press. Somerville, L., Wig, G., Whalen, P., & Kelley, W. M. (2006). Dissociable Medial Temporal Lobe Contributions to Social Memory. Journal of Cognitive Neuroscience, 18(8), Todorov, A., & Uleman, J. (2002). Spontaneous trait inferences are bound to actors' faces: Evidence from a false recognition paradigm. Journal of Personality and Social Psychology, 83(5), Van Berkum, J.J.A, van den Brink, D., Tesink, C.M.J., Kos, M. & Hagoort, P. (2008). The neural integration of speaker and message. Journal of Cognitive Neuroscience, 20, van den Brink, D., Van Berkum, J.A., Bastiaansen, M.C., Tesink, C.M., Kos, M., Buitelaar, J.K., & Hagoort, P. (2012). Empathy matters: ERP evidence for inter-individual differences in social language processing. SCAN, 7, Vandekerckhove, M., Kestemont, J., Weiss, R., Schotte, C., Exadaktylos, V., Haex, B., Verbraecken, J., & Gross, J.J. (2012). Experiential versus analytical emotion regulation and sleep: Breaking the link between negative events and sleep disturbance. Emotion, Vol 12, 6, 1263.

24 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 24 Van Overwalle, F., Van Duynslaeger, M., Coomans, D., & Timmermans, B. (2011). Spontaneous goal inferences are often inferred faster than spontaneous trait inferences. Journal of Experimental Social Psychology. White, K.R., Crites, S.L., Taylor, J.H., & Corral, G. (2009). Wait, what? Assessing stereotype incongruities using the N400 component. SCAN. Wieser, M.J., Gerdes, A.B., Büngel, I., Schwarz, K.A., Mühlberger, A., & Pauli, P. (2014). Not so harmless anymore: How context impacts the perception and electrocortical processing of neutral faces. NeuroImage, 92, Willis, J., & Todorov, A. (2006). First Impressions: Making Up Your Mind After a 100-Ms Exposure to a Face. Psychological Science, 17, 7, Zebrowitz, L., & McDonald, S. (1991). The impact of litigants' baby-facedness and attractiveness on adjudications in small claims courts. Law and Human Behavior, 15(6),

25 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 25 Table 1. Empathy subscores correlated to individual mean amplitude of difference wave during N400 window ( ms). Perspective Taking Fantasy Empathetic Concern Personal Distress Correlation: Correlation: Correlation: Correlation: p=.985 p=.937 p=.196 p=.538

26 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 26 Figure 1. Presentation of the stimuli beginning with the presentation of the behavioral sentence (1500ms), followed by fixation cross (jittered ms), and completed with presentation of the face, the time-locked stimulus (1000ms).

27 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 27 Figure 2. Electrode montage with regions used for analysis shaded.

28 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 28 Figure 3. Effects of congruence;n400 effect elicited at Fz. Grand averages per condition and difference waves (incongruent-congruent) for three midline sites. Grand average waveforms for congruent (black) and incongruent (red) [left plots] and grand average difference waveforms (black) [right plots] for three midline sites; positive plotted upwards.

29 N400 MODULATION OF APPEARANCE-BEHAVIOR INCONGRUITIES 29 Figure 4. Effects of congruence; greater anterior negativity for incongruent relative to congruent face-behavior pairs. Scalp activations shown for the mean amplitude of the congruent and incongruent grand average waveforms during the N400 window ( ms).