Sentences and prediction Jonathan R. Brennan Introduction to Neurolinguistics, LSA2017 1
Grant et al. 2004 2
3
! Agenda»! Incremental prediction in sentence comprehension and the N400» What information guides predictions?» Clues towards how predictions are (and aren't) made» Prediction affects multiple stages of processing» Building sentence structure Introduction to Neurolinguistics, LSA2017 4
Kutas & Hillyard 1980 Science 5
Kutas & Federmeier 2000 Trends Cog Sci 6
"The day was breezy so the boy went outside to fly..." Expected article noun Unexpected article noun a kite an airplane "Cloze" 86% 89% <50% <50% DeLong et al. 2005 Nat Neurosci; see also Wicha et al. 2004 Neurosci Lett, but cf. Ito et al. 2015 Lang Cog Neurosci 7
8
Take-away Higher N400 for pre-noun article indicates prediction of subsequent noun based on linguistic context DeLong et al. 2005 Nat Neurosci 9
Other ERP evidence for prediction: If the N400 is reduced when words are predicted then misspellings will also lead to reduced N400s even though they are semantically content-less. Expected Completion Unexpected Neighbour Pseudoword Neighbour Non-word Neighbour Before lunch he has to deposit his paycheck at the... Every morning he gets up at six and goes for a ten mile... It was a beautiful summer day without not a cloud in the... bank bark pank bxnk run rug ron rcn sky spy smy sko Lazlo & Federmeier 2009 J Mem Lang 10
Lazlo & Federmeier 2009 J Mem Lang 11
The N400 ERP component is larger when lexical access is more difficult. This can be for lexical factors (like uncommon vs. common words) but also may reflect contextual prediction: predicted words are pre-activated and thus easier to access when encountered. Next: The N400 provides clues as to what kind of information guides prediction, inluding:» Linguistic context» Social factors» Abstract linguistic structure, not surface properties Introduction to Neurolinguistics, LSA2017 12
! Agenda» Incremental prediction in sentence comprehension and the N400»! What information guides predictions?» Clues towards how predictions are (and aren't) made» Prediction affects multiple stages of processing» Building sentence structure Introduction to Neurolinguistics, LSA2017 13
Target words equally matched against immediate sentence context but differ in terms of compatability with whole storycontext: "As agreed upon, Jane was to wake her sister and her brother at five o clock in the morning. But the sister had already washed herself, and the brother had even got dressed. Jane told the brother that he was exceptionally quick / slow." Van Berkum et al. 2003 Cog Brain Res 14
Next: test whether information about who the speaker is affects the predictions that are indexed by the N400 Strategy: Use sentences with topics that are more or less appropriate for different speakers voice male/female upper-/lower-class child/adult sentence "If only I looked like Brittany Spears in her latest video." "I have a large tatoo on my back." "Every evening I drink some wine before I go to sleep." Van Berkum et al. 2008 J Cog Neurosci 15
Van Berkum et al. 2008 J Cog Neurosci 16
Take-away speaker identify information affects the N400 component in the same time-window/location (but not same amplitude) as linguistic information. Van Berkum et al. 2008 J Cog Neurosci 17
What kind of linguistic information guides these predictions? Brennan & Hale In prep. 18
Brennan & Hale In prep. 19
Brennan & Hale In prep. 20
Brennan & Hale In prep. 21
So, the N400 ERP componet is affected by word frequency, semantic context, syntactic context, story-context, who the speaker is, etc. Predictive coding explains why all of these different types of information lead to the same brain response Next Clues from the N400 about how predictions are made Introduction to Neurolinguistics, LSA2017 22
! Agenda» Incremental prediction in sentence comprehension and the N400» What information guides predictions?»! Clues towards how predictions are (and aren't) made» Prediction affects multiple stages of processing» Building sentence structure Introduction to Neurolinguistics, LSA2017 23
Evidence for how long it takes to form lexical predictions takes advantage of several properties of Japanese» Optional argument drop» Case marking to indicate subject/object» Flexible word order Typical Argument Order Reverse Argument Order...and vary whether Noun-Verb presented 800 ms or 1200 ms apart Chow et al. 2016 Lang Cog Neurosci; Momma 2017 Unpublished Dissertation 24
Evidence that predictions constrain content but not (necessarily) timing of some linguistic input:» Words are primed or not primed and» There is a constant or variable interval between prime and target Prime Target 150 ms variable intervals constant intervals 800-1400 ms grass 50 250 green nurse 450 250 doctor hammer 250 250 nail zebra 650 250 stripes (Note: MEG, not EEG) Lau & Nguyen 2015 Cortex 25
26
Summary so far The N400 varies depending on whether lexical access for a word is easier/harder. This often (but not always) reflects prediction. Next Lexical access isn't the only process affected by prediction! What else? Introduction to Neurolinguistics, LSA2017 27
! Agenda» Incremental prediction in sentence comprehension and the N400» What information guides predictions?» Clues towards how predictions are (and aren't) made»! Prediction affects multiple stages of processing» Building sentence structure Introduction to Neurolinguistics, LSA2017 28
Prediction propogates through multiple levels of linguistic representation [Dikker & Pylkkanen 2012 Brain Lang 29
Evidence for stages of prediction from semantic priming. Studie varies whether:» A prime picture highly constrains or moderately constrains prediction and whether» the target word matches or mismatches prime Dikker & Pylkkanen 2012 Brain Lang 30
Dikker & Pylkkanen 2012 Brain Lang 31
Evidence that syntactic predictions can also propogate down to lower levels. For example, don't expect a noun after an adverb like "beautifully": The beautiful princess was painted (Ctrl) The beautifully princess was painted (Viol) Vary whether noun has» noun morphology ("princ-ess", farm-er")» typical noun-y form properties ("movie" "soda")» neutral noun-or-verb form properties Dikker et al., 2010 Psych Sci 32
We also make predictions about upcoming sentence structure Introduction to Neurolinguistics, LSA2017 33
We also make predictions about upcoming sentence structure The P600 ERP component peaks 600-800 ms after stimulus onset for words that are syntactically unexpected Standard interpretation: P600 reflects syntactic reanalysis Neville et al. 1991 J Cog Neurosci 34
Perhaps surprisingly, a P600 is also found for words that appear to be semantically unexpected See Kuperberg 2007 Brain Res for a summary 35
See Kuperberg 2007 Brain Res for a summary 36
These "Semantic" P600 effects follow from predictive coding: "Every morning at breakfast the eggs would eat" See Chow et al. 2013 Brain Res 37
Taking stock of where we are We predict upcoming words and when we activate unexpected lexical semantics there is a larger N400 ERP component that is linked with lexical access Different types of information can guide predictions: immediate sentence context including structure, broader linguistic context, who is speaking, etc. Predictions might be made at various levels of representations, including word semantics, syntactic structures, or form features, and research is narrowing down the time it takes for predictions to propogate "from the top-down" Unexpected syntactic structures are associated with a P600 component that is linked with syntactic reanalysis Introduction to Neurolinguistics, LSA2017 38
! Agenda» Incremental prediction in sentence comprehension and the N400» What information guides predictions?» Clues towards how predictions are (and aren't) made» Prediction affects multiple stages of processing»! Building sentence structure Introduction to Neurolinguistics, LSA2017 39
So far, we have looked at ERP components that reflect violated responses when expectations are not met. These processes are a consequence of synatic analsysis. So what sort of work goes into building these sentence representations in the first place? Introduction to Neurolinguistics, LSA2017 40
Bemis & Pylkkanen 2011 J Neurosci 41
Bemis & Pylkkanen 2011 J Neurosci 42
Bemis & Pylkkanen 2011 J Neurosci 43
Summary Basic composition leads to increased anterior temporal processing at around 200-300 ms Bemis & Pylkkanen 2011 J Neurosci 44
Brennan 2016 Brain Lang 45
Brennan et al. 2012 Brain Lang; Brennan et al. 2016 Brain Lang; Brennan 2016 Lang Ling Compass 46
Brennan et al. 2016 Brain Lang 47
Adapted from Brennan et al. 2016 Brain Lang; 48
Nelson et al. 2017 Proc Nat Acad Sci 49
Nelson et al. 2017 Proc Nat Acad Sci 50
Nelson et al. 2017 Proc Nat Acad Sci 51
Nelson et al. 2017 Proc Nat Acad Sci 52
Summary Prediction peremates all aspects of sentence processing, and can be probed systematically when expectations are violated leading to N400 (lexical semantic, P600 (syntactic reanalysis) and other ERP components Ongoing work probes how linguistic representations are built in real-time by pairing computational models of sentence parsing with sentence comprehension brain data Introduction to Neurolinguistics, LSA2017 53
Last topics?» More on modeling sentence parsing» Teasing apart syntactic and semantic composition» Processing long-distance dependencies: evidence from imaging and from deficits Introduction to Neurolinguistics, LSA2017 54