Mixed Linear Models. Case studies on speech rate modulations in spontaneous speech. LSA Summer Institute 2009, UC Berkeley

Transcription

1 Mixed Linear Models Case studies on speech rate modulations in spontaneous speech LSA Summer Institute 2009, UC Berkeley Florian Jaeger University of Rochester

2 Managing speech rate How do speakers determine how fast to talk at a given moment? Beyond speech rate difference between speakers, speech rate could be used strategically to slow down when planning/retrieving difficult upcoming material in order to avoid disfluency to slow down if the current word is unexpected to provide more signal to the interlocutors Speech rate may also be affected by segmental or supra segmental interference. Mixed Linear Models An example (T. Florian Jaeger) [2]

3 Corpus & Data Switchboard corpus 357 speakers 650 dialogues 800k words 100k utterances Automatically time aligned transcription (40k words hand corrected) Today: High frequency function word: the, a, they, it, etc. Mixed Linear Models An example (T. Florian Jaeger) [3]

4 Step size = 0.01 seconds = 10 msecs Mixed Linear Models An example (T. Florian Jaeger) [4]

5 Mixed Linear Models An example (T. Florian Jaeger) [5]

6 Mixed Linear Models An example (T. Florian Jaeger) [6]

7 Speakers vary Mixed Linear Models An example (T. Florian Jaeger) [7]

8 Instances within speakers vary Mixed Linear Models An example (T. Florian Jaeger) [8]

9 Preparing the data Mixed Linear Models An example (T. Florian Jaeger) [9]

10 Subset ing (1): Missing information Exclude cases with missing variable information: d <- subset(d, SpeechRate > 0 &!is.na(id_duration) & ID_duration > 0 & WORDpreceding!= "" & WORDfollowing!= "" ) Mixed Linear Models An example (T. Florian Jaeger) [10]

11 Subset ing (2): Stratification Only words in the center of prosodic phrases of sufficiently long clauses: d <- subset(d, TOPlength > 4 & ID_spWindowSyllables > 7 & ID_spWindowSyllables < 40 & ID_spWindowSyllablePosition > 3 & ID_spWindowSyllables - ID_spWindowSyllablePosition > 3 ) Exclude disfluent words: d <- subset(d, Dform!= 1 ) Mixed Linear Models An example (T. Florian Jaeger) [11]

12 Subset ing (3): Exclude outliers based on distributional information: d<- subset(d, abs(scale(lspeechrate)) < 2.5 & abs(scale(id_duration)) < 2.5 ) Mixed Linear Models An example (T. Florian Jaeger) [12]

13 Data 9,460 7,685 5, 876 5,443 3,605 2,290 1,930 1,730 the a I that (determiner) it they for we Syntactic annotation available Mixed Linear Models An example (T. Florian Jaeger) [13]

14 A simple model > lmer(log(id_duration) ~ lspeechrate + (1 Speaker_ID), the) Linear mixed model fit by REML Formula: log(id_duration) ~ lspeechrate + (1 Speaker_ID) Data: the AIC BIC loglik deviance REMLdev Random effects: Groups Name Variance Std.Dev. Speaker_ID (Intercept) Residual Number of obs: 9460, groups: Speaker_ID, 357 Interpretation? Fixed effects: Estimate Std. Error t value (Intercept) lspeechrate Mixed Linear Models An example (T. Florian Jaeger) [14]

15 Interpretation of random effects Mixed Linear Models An example (T. Florian Jaeger) [15]

16 MCMC sampling $fixed Estimate MCMCmean HPD95lower HPD95upper pmcmc Pr(> t ) (Intercept) lspeechrate $random Groups Name Std.Dev. MCMCmedian MCMCmean HPD95lower HPD95uppe 1 Speaker_ID (Intercept) Residual Mixed Linear Models An example (T. Florian Jaeger) [16]

17 Preparing the data Mixed Linear Models An example (T. Florian Jaeger) [17]

18 Was log transform of speech rate justified? Linear mixed model fit by REML Formula: log(id_duration) ~ SpeechRate + (1 Speaker_ID) Data: the AIC BIC loglik deviance REMLdev Random effects: Groups Name Variance Std.Dev. Speaker_ID (Intercept) Residual Number of obs: 9460, groups: Speaker_ID, 357 Fixed effects: Estimate Std. Error t value (Intercept) SpeechRate cf for log transformed speech rate Mixed Linear Models An example (T. Florian Jaeger) [18]

19 Other ways of testing the log log linearity assumption l.rcs <- lmer(log(id_duration) ~ rcs(speechrate, 4) + (1 Speaker_ID), the) plotlmer.fnc(l.rcs) Non linearity goes away for log transformed speech rate Mixed Linear Models An example (T. Florian Jaeger) [19]

20 Let s s add some more controls Formula: log(id_duration) ~ lspeechrate + Dpreceding + Dfollowing + (1 Speaker_ID) Data: the AIC BIC loglik deviance REMLdev Random effects: Groups Name Variance Std.Dev. Speaker_ID (Intercept) Residual Number of obs: 9460, groups: Speaker_ID, 357 Fixed effects: Estimate Std. Error t value (Intercept) lspeechrate Dpreceding Dfollowing cf for speech rate only model Pretty much unchanged cf for speech rate only model Mixed Linear Models An example (T. Florian Jaeger) [20]

21 Preparing the data Mixed Linear Models An example (T. Florian Jaeger) [21]

22 Collinearity? Linear mixed model fit by REML Formula: log(id_duration) ~ lspeechrate + Dpreceding + Dfollowing + (1 Speaker_ID) Correlation of Fixed Effects: (Intr) lspchr Dprcdn lspeechrate Dpreceding Dfollowing Mixed Linear Models An example (T. Florian Jaeger) [22]

23 MCMC $fixed Estimate MCMCmean HPD95lower HPD95upper pmcmc Pr(> t ) (Intercept) lspeechrate Dpreceding Dfollowing $random Groups Name Std.Dev. MCMCmedian MCMCmean HPD95lower HPD95upper 1 Speaker_ID (Intercept) Residual Mixed Linear Models An example (T. Florian Jaeger) [23]

24 And some social variables Formula: log(id_duration) ~ lspeechrate + Dpreceding + Dfollowing + SpeakerMale * lspeakerage + (1 Speaker_ID) Fixed effects: Estimate Std. Error t value (Intercept) lspeechrate Dpreceding Dfollowing SpeakerMale lspeakerage SpeakerMale:lSpeakerAge Mixed Linear Models An example (T. Florian Jaeger) [24]

25 Collinearity! Effects: (Intr) lspchr Dprcdn Dfllwn SpkrMl lspkra lspeechrate Dpreceding Dfollowing SpeakerMale lspeakerage SpkrMl:lSpA Mixed Linear Models An example (T. Florian Jaeger) [25]

26 Mixed Linear Models An example (T. Florian Jaeger) [26]

27 Mixed Linear Models An example (T. Florian Jaeger) [27]

28 Mixed Linear Models An example (T. Florian Jaeger) [28]

29 Mixed Linear Models An example (T. Florian Jaeger) [29]

30 Mixed Linear Models An example (T. Florian Jaeger) [30]

31 Mixed Linear Models An example (T. Florian Jaeger) [31]

32 Collinearity is gone (nice) Correlation of Fixed Effects: (Intr) lspchr Dprcdn Dfllwn cspkrm clspka lspeechrate Dpreceding Dfollowing cspeakermal clspeakerag cspkrml:csa Mixed Linear Models An example (T. Florian Jaeger) [32]

33 After centering tion) ~ lspeechrate + Dpreceding + Dfollowing + cspeakermale * clspeakerage + (1 Speaker_ID) Fixed effects: Estimate Std. Error t value (Intercept) lspeechrate Dpreceding Dfollowing cspeakermale clspeakerage cspeakermale:clspeakerage Here: no change in significance (social effects still insignificant) but now we can trust the results Mixed Linear Models An example (T. Florian Jaeger) [33]

34 Driven by the phonological complexity of surrounding coda/onsets? Addition of phonological complexity: χ 2 (2)=577.5, p< Removal of OCP effects: χ 2 (3)=117.1, p< Partial shadowed effect or collinearity? Fixed effects: Estimate Std. Error t value (Intercept) clspeechrate Dpreceding Dfollowing consetprecedingcodaocp consetprecedingonsetocp consetfollowingonsetocp ccodaclusterpreceding consetclusterfollowing cspeakermale clspeakerage cspeakermale:clspeakerage Mixed Linear Models An example (T. Florian Jaeger) [34]

35 Mild collinearity Correlation of Fixed Effects: (Intr) clspcr Dprcdn Dfllwn copcoc copooc cofooc ccdclp clspeechrat Dpreceding Dfollowing constprcocp constproocp constfloocp ccdclstrprc constclstrf Mixed Linear Models An example (T. Florian Jaeger) [35]

36 What to do if centering is not going to help? Mixed Linear Models An example (T. Florian Jaeger) [36]

37 the$ronsetfollowingonsetocp <- residuals(lm(consetfollowingonsetocp ~ consetclusterfollowing, the)) Correlation of Fixed Effects: (Intr) clspcr Dprcdn Dfllwn copcoc copooc rofooc ccdclp clspeechrat Dpreceding Dfollowing constprcocp constproocp ronstfloocp ccdclstrprc constclstrf Mixed Linear Models An example (T. Florian Jaeger) [37]

38 Does availability affect pronunciation? Two measures of availability: Frequency of next work (trigram) predictability of next work the$rlcndp_1forward <- residuals(lm(clcndp_1forward ~ clfqfollowing, the)) l.avail.r <- lmer(log(id_duration) ~ clspeechrate + Dpreceding + Dfollowing + consetprecedingcodaocp + consetprecedingonsetocp + consetfollowingonsetocp + consetprecedingcodaident + consetprecedingonsetident + consetfollowingonsetident + ccodaclusterpreceding + consetclusterfollowing + clfqfollowing + rlcndp_1forward + cspeakermale * clspeakerage + (1 Speaker_ID) + (1 WORDpreceding) + (1 WORDfollowing), the) Mixed Linear Models An example (T. Florian Jaeger) [38]

39 Addition of availability: χ 2 (2)=32.3, p< Estimate Std. Error t value (Intercept) clspeechrate Dpreceding Dfollowing consetprecedingcodaocp consetprecedingonsetocp consetfollowingonsetocp consetprecedingcodaident consetprecedingonsetident consetfollowingonsetident ccodaclusterpreceding consetclusterfollowing clfqfollowing rlcndp_1forward cspeakermale clspeakerage cspeakermale:clspeakerage Mixed Linear Models An example (T. Florian Jaeger) [39]

40 Does redundancy affect pronunciation? Mixed Linear Models An example (T. Florian Jaeger) [40]