Music and Language Perception: Expectations, Structural Integration, and Cognitive Sequencing

Transcription

1 Topics in Cognitive Science 4 (2012) Copyright Ó 2012 Cognitive Science Society, Inc. All rights reserved. ISSN: print / online DOI: /j x Music and Language Perception: Expectations, Structural Integration, and Cognitive Sequencing Barbara Tillmann Lyon Neuroscience Research Center CRNL, CNRS UMR5292, INSERM U1028, Université Lyon 1 Received 6 August 2010; received in revised form 27 September 2011; accepted 7 January 2012 Abstract Music can be described as sequences of events that are structured in pitch and time. Studying music processing provides insight into how complex event sequences are learned, perceived, and represented by the brain. Given the temporal nature of sound, expectations, structural integration, and cognitive sequencing are central in music perception (i.e., which sounds are most likely to come next and at what moment should they occur?). This paper focuses on similarities in music and language cognition research, showing that music cognition research provides insight into the understanding of not only music processing but also language processing and the processing of other structured stimuli. The hypothesis of shared resources between music and language processing and of domain-general dynamic attention has motivated the development of research to test music as a means to stimulate sensory, cognitive, and motor processes. Keywords: Music and language processing; Cognitive expectations; Structural integration; Temporal processing; Shared neural correlates; Priming; Implicit processing; Expertise 1. Music cognition Language cognition Music cognition research provides insight into cognitive processes and neural correlates of learning and perceiving complex acoustic, nonverbal structures. Music cognition is asking questions similar to those addressed in language cognition: How do listeners acquire knowledge via simple exposure and how does this implicitly acquired knowledge allow listeners to process structures, create mental representations, and develop expectations? This article presents some parallels between structures of musical and linguistic systems and Correspondence should be sent to Barbara Tillmann, Lyon Neuroscience Research Center, Team Auditory Cognition and Psychoacoustics, 50 Av. Tony Garnier, F Lyon Cedex 07, France. btillmann@ olfac.univ-lyon1.fr

2 B. Tillmann Topics in Cognitive Science 4 (2012) 569 perceivers acquired knowledge thereof. In this line of research, the definition of syntax has been adapted and defined as a set of principles governing the combination of discrete structural elements (such as words or musical tones) into sequences (Patel, 2003, p. 674). To process syntactic structures, perceivers need to be acculturated to their culture s musical linguistic systems and use their knowledge about these systems to construct structured representations (see also Patel, 2008). For both domains, expectations and structural processing have been studied by manipulating structural contexts. The findings from these studies led to the hypothesis of shared syntactic integration resources (Patel, 2003), which motivated new research on music and language processing. Considering additional data sets leads to the hypothesis of more general shared structural integration resources that extend from syntax to semantics and other structured materials. The focus on structures highlights that music and language processing require cognitive sequencing: Perceivers have to process individual events and to store them in short-term memory; they are influenced by context effects, develop perceptual expectations, perform structural integration, update the currently developed mental model, and use their knowledge about the relevant (e.g., musical, linguistic) system. The final section extends the consideration of tonal processing to temporal processing (both require cognitive sequencing) and opens to new research avenues testing music as a means to improve sensory, cognitive, or motor processing, thus benefitting other domains. 2. Perceivers structural knowledge of musical and linguistic systems In comparison to descriptions of musicologists and linguists, the cognitive psychology approach to musical and linguistic structures is necessarily simplified. The goal of this approach is to highlight statistical regularities (e.g., frequencies of co-occurrence, frequencies of occurrence) and structural organization (e.g., hierarchical structures, syntactic structures) and to focus on how the brain processes these musical and linguistic structures. The cognitive capacity of implicit learning enables perceivers to learn regularities in their environment through mere exposure to materials that obey the rules of a given system whether language (e.g., Gomez & Gerken, 2000; Pacton, Perruchet, Fayol, & Cleeremans, 2001) or music (i.e., tonal acculturation, Francès, 1958; Krumhansl, 1990; Tillmann, Bharucha, & Bigand, 2000). Perceivers knowledge about the regularities and structures of their culture s linguistic and musical systems allows the development of expectations, which facilitate the processing of expected events over unexpected events (faster and more accurate processing, which requires fewer neural resources). In language, statistical regularities between words can be related to semantic concepts, differences in associative strengths (frequencies of co-occurrence) between words, and differences in frequencies of occurrence (e.g., Thompson-Schill, Kurtz, & Gabrieli, 1998). Additionally, words have syntactic functions (e.g., verbs), and sentences contain syntactic structures, which can be described using structural trees or phrase structures (e.g., Chomsky, 1965; Gibson & Pearlmutter, 1998). Syntactic rules create constraints and regularities that perceivers can use to develop expectations for upcoming words. Behavioral and neural

3 570 B. Tillmann Topics in Cognitive Science 4 (2012) studies have shown that perceivers (listeners and readers) process structures related to semantic, associative, and syntactic regularities, and they develop expectations, which facilitate processing of expected events (e.g., Kaan & Swaab, 2002; McNamara, 2005). While it is debatable to search directly for semantics in a musical system, syntactic-like structures and functions (e.g., Lerdahl, 2001; Patel, 2008; Winograd, 1968) as well as differences in frequencies of occurrence and frequencies of co-occurrence (between tones, chords, and keys) can be described (e.g., Budge, 1943; Krumhansl, 1990; Rohrmeier & Cross, 2008; Tillmann et al., 2000). Pitch is the most obvious form- and structure-bearing dimension of Western tonal music, beyond the temporal dimension (structures based on duration, rhythm, and meter; McAdams, 1989). The pitch dimension contains the tonal hierarchies and provides the basis to define typical, grammatical progressions as well as event hierarchies (Bharucha, 1984; Lerdahl & Jackendoff, 1983; Rohrmeier, 2011). The pitch-structures of the Western tonal system are based on 12 tones, used according to the rules of music theory, thus making structures and statistical regularities emerge: Tonalities (keys) are defined as subsets of 7 tones that, as a consequence, frequently occur together, and chords are defined as subsets of at least 3 tones that frequently occur together. For tones and chords, music theory describes a hierarchy of tonal function: Tones and chords with important tonal functions (e.g., the tonic) are at the top of the hierarchy and are referred to as stable events, which induce a sense of finality and resolution, while events lower in the hierarchy are unstable events, which induce a sense of tension and a need for resolution (e.g., Bharucha, 1984; Piston, 1978; Schenker, 1935). Tonal hierarchies correlate, at least partially, with frequencies of occurrence (e.g., Krumhansl, 1990). Tones and chords with important tonal functions (particularly, tonic or dominant) occur more frequently in a given context than others (Budge, 1943; Krumhansl, 1990; Rohrmeier & Cross, 2008; see also Huron, 2006). Via mere exposure, listeners acquire implicit knowledge of the tonal hierarchies that are common to musical pieces of their culture. Once a tonal hierarchy is activated in a given context, events that are more stable are more strongly expected to occur (e.g., Bharucha, 1984, 1987). The tonic functions as a cognitive reference point to which other events are perceived in relation (Krumhansl, 1990; Tillmann, Janata, Birk, & Bharucha, 2008; see also Lerdahl & Jackendoff, 1983, for a formal account). The structures and regularities go beyond single tonal events and extend to specific progressions and sequences in which they occur and co-occur (Bharucha & Todd, 1989). They have been estimated, for example, as transition probabilities between chord functions (as used in Western tonal music; Piston, 1978). They have been further described as more complex hierarchical organizations spanning longer sequences (e.g., in terms of a generative grammar; Rohrmeier, 2011; Steedman, 1984) and that can also be included in event hierarchies. Event hierarchies are defined on the basis of tonal hierarchies (including typical progressions of tonal events) together with temporal structures. They are structural organizational trees specific to a given musical piece and in which each musical event (tone, chord) has its position (Bharucha, 1984; Lerdahl & Jackendoff, 1983; Schenker, 1935): Unstable events are subordinated to stable events, which are assigned more prominent positions. These structural trees can be compared with structural trees proposed in linguistics (see Patel, 2003).

4 B. Tillmann Topics in Cognitive Science 4 (2012) 571 Investigating listeners musical structure processing can thus distinguish two aspects of musical syntax, which are respectively related to tonal and event hierarchies: (a) listeners understanding of the syntactic function of a given event (tone, chord) in the currently instilled tonality, such as the tone C acting as the tonic in the tonality of C Major; and (b) the integration of each tone or chord in the event hierarchy that considers the event s tonal function (e.g., the tonic), its temporal position, and the resulting progression with surrounding events. To date, studies investigating musical structure processing and tonal expectancy formation have mostly performed manipulations that can be described in terms of tonal hierarchy: notably by comparing events that are higher or lower in the tonal hierarchy (e.g., the tonic versus the dominant or an out-of-key chord). Some studies have investigated the processing of event hierarchies (see Bigand, 1993) but without studying processing benefits (e.g., faster response times) for events expected on the basis of their position in the structural tree. One approach scrambled the sequential chord order, thereby modifying the harmonic progression and the event hierarchy, but not the tonal function of the final chord (i.e., its position in the tonal hierarchy, that is, tonic or subdominant). However, this manipulation did not affect the processing speed of the final chords, suggesting the relevance of tonal hierarchy for the observed effects (Tillmann & Bigand, 2001). 3. Contextual expectations leading to processing benefits for music and language Music cognition research has adopted a methodology of psycholinguistics to investigate listeners understanding of musical structures: a to-be-processed event is presented in different contexts, in which it is either related (and supposed to be expected) or unrelated (and supposed to be unexpected). 1 The rationale is that the context activates listeners knowledge about a system s structures and functions, and this activation allows expectancy formation for future events, which then influence event processing. For language, it has been shown that word processing is more efficient when the word is expected in a given context than when it is syntactically (e.g., West & Stanovich, 1982) or semantically unexpected (McNamara, 2005 for a review). For music, the same tone or chord (the same acoustic information) is presented in contexts instilling different tonalities, which thus change its tonal function. The priming paradigm is an implicit investigation method to study the effects of context and perceivers expectations on the efficiency of perception (i.e., accuracy and processing speed). This paradigm, extensively used in psycholinguistics (see Neely, 1991), was introduced in music perception research by Bharucha and Stoeckig (1986) and has since been further developed (e.g., Bigand & Pineau, 1997; Marmel, Tillmann, & Dowling, 2008). It allows studying nonmusicians implicit musical knowledge, which may be more sophisticated than explicit judgments suggest. Use of this method has provided evidence for implicit musical knowledge in children younger than shown when using explicit methods (Schellenberg, Bigand, Poulin, Garnier, & Stevens, 2005) and in amusic individuals who are impaired in explicit musical tasks (Tillmann, Peretz, Bigand, & Gosselin, 2007).

5 572 B. Tillmann Topics in Cognitive Science 4 (2012) The basic design consists of a prime context (i.e., a word, a sentence, a chord, or a chord sequence) and a target event (i.e., a word or chord). The relations between prime and target are systematically manipulated: for language, this manipulation concerns syntactic structure, semantic relatedness, or strength of associations; for music, it concerns tonal relatedness or tonal functions as defined by music theory. Participants are not required to make direct judgments on the relation between the prime context and the target but rather focus solely on the target. Psycholinguistic studies frequently use lexical-decision tasks, in which half of the targets are words and Half are nonwords and participants make speeded decisions regarding whether the target is a word from their native language or a nonword (see Neely, 1991; for a review). In music studies, participants make speeded judgments on a perceptual feature of the target chord, such as sensory consonance dissonance (e.g., Bharucha & Stoeckig, 1986; Bigand, & Pineau, 1997): Half of the targets are consonant (i.e., well-tuned, correctly constructed chords), and half are rendered acoustically dissonant (by mistuning or adding outof-key tones). Alternative priming tasks have required, for example, judgments of temporal asynchrony (Tillmann & Bharucha, 2002) or timbre-discrimination (Tillmann, Bigand, Escoffier, & Lalitte, 2006). Musical priming research has shown that nonmusicians musical expectations are not limited to expected, in-key events when contrasted with unexpected, out-of-key events, but even nonmusicians develop differentiated in-key expectations: The processing of a target (i.e., the last tone or chord of a musical sequence) is facilitated when it is related to the context and functions at the top level of the tonal hierarchy (i.e., the tonic) than when it is less related and has a subordinate position in the tonal hierarchy (i.e., the subdominant) (e.g., Bigand, Madurell, Tillmann, & Pineau, 1999; Marmel, Tillmann, & Delbé, 2010). Adapting a similar rationale proposed in psycholinguistic research (e.g., Jonides & Mack, 1984), comparisons with neutral baseline contexts (i.e., atonal sequences) have enabled investigations of costs and benefits associated with musical priming in tonal contexts (Tillmann et al., 2003b, 2008). In a sensory priming approach, processing is facilitated for targets having occurred in the context (repetition priming) or sharing perceptual features with the context. In a cognitive priming approach, expectations are based on listeners knowledge about possible relations between events independent of repetition. Cognitive and sensory components of expectations are not mutually exclusive, as is reflected by the physical relation between semantically related words (e.g., nurse and nursery) or by the correlation between musical relatedness (as defined by music theory) and psychoacoustic relatedness (even if it is possible to experimentally manipulate them separately). In a series of priming experiments, highly controlled experimental stimuli have allowed reducing (Marmel et al., 2010), keeping constant (e.g., Bigand, Tillmann, Poulin, D Adamo, & Madurell, 2001; Marmel & Tillmann, 2009), or favoring sensory influences (e.g., Bigand, Poulin, Tillmann, & D Adamo, 2003; Tekman & Bharucha, 1998). Findings support the hypothesis that musical priming and expectations are driven by listeners knowledge about the musical system, and not by sensory information stored in a sensory memory buffer. Indeed, repetition priming, the strongest form of sensory priming, is overruled by musical structures, thus by cognitive priming (Bigand, Tillmann, Manderlier, & Poulin, 2005).

6 B. Tillmann Topics in Cognitive Science 4 (2012) 573 This observation contrasts the dominant effects of repetition in language or visual processing (e.g., Bruce & Valentine, 1985; Dannenbring & Briand, 1982). 4. Shared neural correlates for musical and linguistic structure processing Neural correlates of musical and linguistic structure processing have been measured for a target (e.g., word or chord) presented in a context with maintained or violated expectations. Electrophysiological and brain imaging data suggest similar neural resources for structure processing in music and language. Event-related potential (ERP) studies have shown that the late positivity P600, evoked by an unexpected event (e.g., requiring enhanced syntactic integration), can be elicited by linguistic- and music-syntactic violations (e.g., Patel, Gibson, Ratner, Besson, & Holcomb, 1998). In addition, an early anterior negativity has been reported for syntactic violations of both material types (bilaterally, but differing in hemispheric weightening): the left-lateralized (early) anterior negativity (i.e., (E)LAN) elicited by linguistic-syntactic violations (e.g., Friederici, Pfeifer, & Hahne, 1993) and the right-lateralized early anterior negativity (ERAN) elicited by music-syntactic violations (e.g., Koelsch et al., 2001). These components are thought to arise in the inferior frontal regions around Broca s area and its right-hemisphere homolog (Friederici, 1995; Friederici, Meyer, & von Cramon, 2000; Maess, Koelsch, Gunter, & Friederici, 2001). Functional magnetic resonance imaging (fmri) data further confirmed the involvement of inferior frontal regions in linguistic and music processing. The inferior frontal gyrus (frontal operculum) was activated more strongly for semantically unrelated words than related ones in word pairs (Kotz, Cappa, von Cramon, & Friederici, 2002) and for words that created a syntactic violation than for words in syntactically correct sentences (e.g., Friederici, Rueschemeyer, Hahne, & Fiebach, 2003). While this activation was mostly bilateral, but stronger in the left-hemisphere, the reverse was observed for musical violations: In comparison to an expected event, a musically unexpected event resulted in increased activation of the inferior frontal cortex (frontal operculum, anterior insula), bilaterally, but with an asymmetry to the right hemisphere (Koelsch, Gunter, Wittfoth, & Sammler, 2005; Koelsch et al., 2002; Tillmann, Janata, & Bharucha, 2003a; Tillmann et al., 2006a). In contrast to language violations, musical structure violations can create acoustic violations (i.e., introduction of new tones). While the necessity to control for sensory influences had been highlighted in behavioral studies (see above), most neuroscience research investigating the neural correlates of musical structure processing has used experimental material confounded by sensory influences, thus leading to alternative interpretations in terms of sensory deviance detection instead of musical syntax processing (see Bigand et al., 2006). Only some recent ERP and fmri studies, which used controlled musical stimuli, have allowed rejecting this alternative sensory hypothesis and confirmed the implication of the inferior frontal regions in musical syntax processing (Koelsch, Jentschke, Sammler, & Mietchen, 2007; Tillmann et al., 2006a). 2 Based on the observed similar neural correlates in linguistic and musical structure processing, Patel (2003) proposed the Shared Syntactic Integration Resource Hypothesis

7 574 B. Tillmann Topics in Cognitive Science 4 (2012) (SSIRH): Music and language share neural resources for processes linked to the syntactic, structural integration of events (i.e., processing of structural relations between events). The music- and linguistic-syntactic representations would be stored in distinct neural networks and can be selectively damaged, thereby accounting for the double dissociations observed in patients (e.g., Peretz, Belleville, & Fontaine, 1997). Two new hypotheses have been developed on the basis of the SSIRH: (1) If musical and linguistic syntax processing do share neural resources, patients with linguistic syntax processing deficits should also have musical syntax processing impairments. In accordance with this hypothesis, Jentschke, Koelsch, Sallat, and Friederici (2008) reported that children with specific language impairment also show impaired music-syntactic processing (missing ERAN for musically unexpected events). Similarly, Patel et al. (2008) reported a lack of musical priming in patients with Broca s aphasia who have difficulties in linguistic syntax processing. (2) If musical and linguistic syntax processing share neural resources (and considering the hypothesis of limited resources), the simultaneous processing of music-syntactic and linguistic-syntactic structures should interfere with each other. ERP studies, in which visually presented sentences were synchronized with auditory chord sequences, provide support for this hypothesis by demonstrating interactions between simultaneous processing of music-syntactic and linguistic-syntactic structures (Koelsch et al., 2005; Steinbeis & Koelsch, 2008). Beyond the ERPs observed for the processing of music and language violations (i.e., LAN and ERAN, respectively), the simultaneous presentation of a music-syntactically unexpected chord reduced the amplitude of the LAN, and the simultaneous presentation of a linguistic-syntactically unexpected word reduced the amplitude of the ERAN. Recent cross-modal behavioral studies confirmed interactive relations between linguistic-syntactic and music-syntactic processing (Hoch, Poulin-Charronnat, & Tillmann, 2011; Slevc, Rosenberg, & Patel, 2009). The nature of the interactive patterns seems to be influenced by the type of linguistic-syntactic manipulation (garden-path sentences or syntactic violations) and music-syntactic manipulation (in-key or out-of-key). 5. An extended hypothesis of shared resources for structural integration A key component of syntactic processing for language and music is structural integration (i.e., connecting an incoming event, X, to one or more events, Y, Z; Patel, 2003). While the SSIRH focuses on syntactic processing and predicts interference for simultaneous musical and linguistic syntax processing, Slevc et al. (2009) considered the alternative hypothesis that language and music share resources for a more general type of processing (e.g., for a process of integrating new information into any type of evolving representation) (p. 375), which should then lead to interactions of musical syntax with linguistic syntax and with semantics. While some results support syntactic specificity of shared resources, particularly

8 B. Tillmann Topics in Cognitive Science 4 (2012) 575 by showing interactive influences between simultaneous processing of music-syntactic and linguistic-syntactic structures, but not between simultaneous processing of music-syntactic and linguistic-semantic processing (Koelsch et al., 2005; Slevc et al., 2009), other results have shown interactive influences also between music-syntactic and linguistic-semantic processing (Poulin-Charronnat, Bigand, Madurell, & Peereman, 2005; Steinbeis & Koelsch, 2008). 3 Thus, the results on semantics contrast with the consistent observation of interactive influences between the simultaneous processing of music and language syntax. Taken together, these findings suggest that shared resources might be dedicated to more general, structural and temporal integration, for music as well as syntax and semantics in language. For music and language, each event must be integrated online into an updated mental representation of the context (Friederici, 2002; Hagoort, 2005; Patel, 2003, 2008; Tillmann, 2005). For language, both syntactic and semantic processing require structural integration of information over time (e.g., Friederici, 2002; Gibson, 1998; Hagoort, 2005; Jackendoff, 2002), and readers listeners integrate incoming information to create coherent situational models (e.g., van Dijk & Kintsch, 1983; Kintsch, 1988). This integration is more difficult and demanding when an incoming event is unexpected than when it is expected. Structural and temporal integration is also required for the processing of other materials, such as arithmetic, movies, dance or action sequences, as well as newly acquired artificial structures (see Jackendoff, 2009; Fazio et al., 2009). 4 If resources are shared for structural, temporal integrative processes, interactive patterns are predicted for the simultaneous processing of structured materials beyond music and language. This more general hypothesis can be further supported by imaging data showing inferior frontal cortex activation during sequential manipulation and structuring of notes, syllables, visuo-spatial stimuli, or action sequences (e.g., Gelfand & Bookheimer, 2003; Tettamanti & Weniger, 2006); syntactic violations in artificial sequence structures (Petersson, Forkstam, & Ingvar, 2004; see also Christiansen, Kelly, Shillcock, & Greenfield, 2010); and in temporal sequence perception and production (Coull, 2004; Fuster, 2001; Janata & Grafton, 2003; Schubotz, Friederici, & von Cramon, 2000). Furthermore, late positivities (P600) have been reported not only for musical and linguistic violations (Patel et al., 1998) but also for violations in numerical sequences (Núñez-Peña & Honrubia-Serrano, 2004), abstract non-linguistic material (Lelekov, Dominey, & Garcia-Larrea, 2000; Lelekov-Boissard & Dominey, 2002; see also Besson & Faïta, 1995), and visual sequential patterns (Christiansen, Conway, & Onnis, 2007). The increased positivity for the unexpected event (based on the contextual structure) has been interpreted as a difficulty to integrate the event in the previous structure. Thus, these findings suggest an overlap in resources for the processing of music, language, and other complex sequential regularities. Accordingly, recent studies have investigated language and action (Fazio et al., 2009), music and action (Sammler, Harding, D Ausilio, Fadiga, & Koelsch, 2010) and music and arithmetic (Hoch & Tillmann, 2012). For example, Fazio et al. (2009) reported that aphasic patients with deficits in linguistic syntax processing also have deficits in action syntax processing. Another study adapted the cross-modal paradigm (as used for music and language, see above) to music and arithmetic: Number series were visually presented synchronously with musical sequences, and numerical processing time revealed interactive influences between expectancy violations in musical sequences

9 576 B. Tillmann Topics in Cognitive Science 4 (2012) (ending on an expected or unexpected chord) and expectancy violations in arithmetic sequences (Hoch & Tillmann, 2012). 6. Cognitive sequencing: Sound as a scaffolding framework and music as a favorite candidate Similar to speech, music perception requires auditory sequencing: The incoming stream must be segmented (chunked) into events and phrases, the timing and ordering of events must be processed and memorized, relations and structures need to be processed, and each incoming event must be integrated (also using knowledge about the relevant system) into the structure of the context. Understanding sequencing behavior in music benefits from and contributes to research in psychology and neuroscience investigating timing, attention, and sequence learning. In addition, the strong coupling between perception and action further suggests music as a model for sensorimotor coupling (Janata & Grafton, 2003; Zatorre, Chen, & Penhune, 2007). Conway, Pisoni, and Kronenberger (2009) recently highlighted the importance of sound for cognitive sequencing abilities by proposing that sound acts as a type of cognitive scaffolding to support learning how to process and interpret sequential and temporal information in the environment. Most important, this supporting framework is not restricted to auditory processing but more generally influences learning and manipulation of serial-order information in other modalities. The capacity of sound and speech processing thus has an additional unspecified influence on the development of general cognitive sequencing abilities. Their hypothesis is supported by the finding that auditory deprivation in deafness results in deficits of auditory perception and spoken language abilities and in disturbances of domain-general sequencing functions: Children with cochlear implants displayed impaired implicit learning capacity for visual, non-linguistic regularities and motor sequences. 5 Their hypothesis receives additional support from research investigating the effect of musical expertise. Musical training, which involves training in sound analyses and enhanced sound exposure, is linked to cortical changes (e.g., Hyde et al., 2009; Schlaug, 2001; Sluming et al., 2002) and improved processing in cognitive non-musical tasks, such as verbal memory (e.g., Chan, Ho, & Cheung, 1998), prosody perception (Magne, Schön, & Besson, 2006; Marques, Moreno, Castro, & Besson, 2007), scores on mathematical tests (Cheek & Smith, 1999) and linguistic syntax perception (Jentschke & Koelsch, 2009). Though numerous studies have only compared musicians and nonmusicians, other studies have studied the same individuals before and after musical training and provided converging evidence (e.g., Moreno et al., 2009; Schellenberg, 2004; Schlaug, Norton, Overy, & Winner, 2005). Benefits of auditory, music-like training also contribute to a decrease in deficits in cognitive pathologies. For example, timing and pitch discrimination deficits in dyslexic children were reduced after musical training (Overy, 2003; Santos, Joly-Pottuz, Moreno, Habib, & Besson, 2007), and active music listening was beneficial for the recovery of cognitive functions and mood after stroke (Särkämö, Tervaniemi, Laitinen, et al., 2008). Both lines

10 B. Tillmann Topics in Cognitive Science 4 (2012) 577 of research, demonstrating costs or benefits of reduced or enhanced sound processing, thus suggest the role of sound in cognitive sequencing abilities. Jones (1976; Jones & Boltz, 1989; Large & Jones, 1999) proposed a theoretical framework of auditory attention and, more generally, temporal attention (i.e., dynamic attending). It suggests that auditory attention is not evenly distributed over time but develops cyclically. Structures of sensory input (such as tonal or temporal accents in music) induce attentional cycles, which are conceptualized as internal oscillators. These oscillators synchronize to the regularities of external stimuli and help direct attention over time. They facilitate expectations and support segmentation as well as structural and temporal integration. While the research presented above focused on tonal structures and expectations, other studies investigated expectations for temporal structures. For example, processing a musical event is faster and more accurate for events occurring in a temporally structured, regular sequence (vs. irregular sequence) and when occurring on time rather than too early (e.g., Schmuckler & Boltz, 1994; Tillmann & Lebrun-Guillaud, 2006). Even pitch discrimination can be facilitated when the temporal occurrence of the tones is consistent with the temporal regularity of the context (Jones, Moynihan, MacKenzie, & Puente, 2002). Jones theory of dynamic attending has been described for music and, via temporal structures, also applied to speech and movement (e.g., Jones & Boltz, 1989; Port, 2003; Quené & Port, 2005). For language, it has been suggested that the timing of metric stress supports segmentation (Cutler, 1994), and regular predictable presentation influences syntax processing (Schmidt-Kassow & Kotz, 2008, 2009). The use of strongly metrical stimuli (e.g., a marching rhythm), which provided sensory predictable cues and were presented as musical primes in the experimental session, can help to compensate for deficits in speech analysis (syntax processing) in patients with basal ganglia lesions or developmental disorders (Kotz, von Cramon, & Friederici, 2005; Przybylski et al., unpublished data) and to facilitate gait patterns in patients with Parkinson s disease (McIntosh, Brown, Rice, & Thaut, 1997; Thaut, 2003) and movement in apraxic patients (Bernardi et al., 2009). Advances in the cognitive psychology and neuroscience of music have increasingly motivated the investigation of beneficial effects of music (training or listening) for cognition, and sensory, cognitive, and motor rehabilitation (e.g., symposium at the Neurosciences and Music conference [Schlaug, 2009], special issue of Music Perception [Schlaug, Altenmueller, & Thaut, 2010]). Beyond exploring the benefits of music-related activities, this research investigates potential explanatory frameworks, such as those linked to arousal and motivation (e.g., Thompson, Schellenberg, & Husain, 2001), overlapping resources between music and other mental activities, such as language (e.g., Patel, 2008), and or shared dynamic attention, which influences cognitive sequencing (e.g., Jones, 1976). Notes 1. The focus here is on processing benefits and costs that are created by listeners contextual expectations. Musical expectations have been further studied using production tasks, singing or playing (on a piano) the most expected continuation (Carlsen, 1981;

11 578 B. Tillmann Topics in Cognitive Science 4 (2012) Schmuckler, 1990), and using perception tasks requiring explicit subjective judgments (Schellenberg, Adachi, Purdy, & McKinnon, 2002; Schmuckler & Boltz, 1994; see also Huron, 2006; for a review). Furthermore, it is important to note that musical expectations have been also attributed a role for musical expressivity (see Meyer, 1956). 2. Beyond the inferior frontal regions, imaging data have further suggested that the processing of musical structures requires a larger neural network, including bilateral temporal regions (anterior and posterior superior temporal gyrus sulcus, middle temporal gyrus) and right parietal regions (supramarginal gyrus; Koelsch et al., 2002, 2005; Tillmann et al., 2003a, 2006a). This is comparable to neural activation patterns observed during language processing (with frontal, temporal, and parietal regions, see Friederici, 2002). In particular, posterior temporal areas have been linked to processes of on-line integration (e.g., Constable et al., 2004; Friederici et al., 2003; Wise et al., 2001). This overlap in neural correlates further suggests that music and language processing share resources necessary for structural integration, especially over time. 3. Variability in data patterns for semantics might be linked to differences in types of stimuli violations or attentional demands (i.e., dual vs. single task). It is important to note that the same differences led to consistent observations of interactive influences for linguistic syntax. 4. This can also be linked to the proposed relation between natural language and planned action (see, e.g., Roy & Arbib, 2005; Steedman, 2002). 5. Performance was not impaired in nonsequencing tasks, such as visual-spatial memory. Acknowledgments I would like to thank my collaborators of the presented research, and in particular Lisianne Hoch, Bénédicte Poulin-Charronnat, Emmanuel Bigand, Daniele Schön, Sonja Kotz, and Ani Patel for our discussions on the ideas developed here. Partly funded from the European Community s Seventh Framework Programme under the EBRAMUS project-grant agreement References Bernardi, N. F., Aggujaro, S., Caimmi, M., Molteni, F., Maravita, A., & Luzzatti, C. (2009). A new approach to rhythm cueing of cognitive functions: The case of ideomotor apraxia. The Neurosciences and Music III: Disorders and Plasticity: Annals of the NewYork Academic Science, 1169, Besson, M., & Faïta, F. (1995). An event-related potential (ERP) study of musical expectancy: Comparison of musicians with nonmusicians. Journal of Experimental Psychology: Human Perception and Performance, 21, Bharucha, J. J. (1984). Event hierarchies, tonal hierarchies, and assimilation: A reply to Deutsch and Dowling. Journal of Experimental Psychology: General, 113,

12 B. Tillmann Topics in Cognitive Science 4 (2012) 579 Bharucha, J. J. (1987). Music cognition and perceptual facilitation: A connectionist framework. Music Perception, 5 (1), Bharucha, J. J., & Stoeckig, K. (1986). Reaction time and musical expectancy: Priming of chords. Journal of Experimental Psychology: Human Perception and Performance, 12, Bharucha, J. J., & Todd, P. (1989). Modeling the perception of tonal structures with neural nets. Computer Music Journal, 13, Bigand, E. (1993). Contributions of music to research on human auditory cognition. In S. McAdams & E. Bigand (Eds.), Thinking in sound: The cognitive psychology of human audition. Oxford, England: Claredon Press. Bigand, E., Madurell, F., Tillmann, B., & Pineau, M. (1999). Effect of global structure and temporal organization on chord processing. Journal of Experimental Psychology: Human Perception and Performance, 25, Bigand, E., & Pineau, M. (1997). Global context effects on musical expectancy. Perception & Psychophysics, 59, Bigand, E., Poulin, B., Tillmann, B., & D Adamo, D. (2003). Cognitive versus sensory components in harmonic priming effects. Journal of Experimental Psychology: Human Perception and Performance, 29, Bigand, E., Tillmann, B., Manderlier, D., & Poulin, B. (2005). Repetition priming: Is music special? Quarterly Journal of Experimental Psychology, 58, Bigand, E., Tillmann, B., Poulin, B., D Adamo, D. A., & Madurell, F. (2001). The effect of harmonic context on phoneme monitoring in vocal music. Cognition, 81, B11 B20. Bigand, E., Tillmann, B., & Poulin-Charronnat, B. (2006). A module for syntactic processing in music? Trends in Cognitive Science, 10, Bruce, V., & Valentine, T. (1985). Identity priming in face recognition. British Journal of Psychology, 76, Budge, H. (1943). A study of chord frequencies (Contributions to Education No. 882). New York: Columbia University, Teacher College. Carlsen, C. (1981). Some factors which influence melodic expectancy. Psychomusicology, 1, Chan, A. S., Ho, Y. C., & Cheung, M. C. (1998). Music training improves verbal memory. Nature, 396, 128. Cheek, J. M., & Smith, L. R. (1999). Music training and mathematics achievement. Adolescence, 34, Chomsky, N. (1965). Aspects of the theory of syntax. Cambridge, MA: MIT Press. Christiansen, M. H., Conway, C. M., & Onnis, L. (2007). Overlapping neural responses to structural incongruencies in language and statistical learning point to similar underlying mechanisms. In D. S. McNamara & J. G. Trafton (Eds.), Proceedings of the 29th annual cognitive science society conference (pp ). Austin, TX: Cognitive Science Society. Christiansen, M. H., Kelly, M. L., Shillcock, R. C., & Greenfield, K. (2010). Impaired artificial grammar learning in agrammatism. Cognition, 116, Constable, R. T., Pugh, K. R., Berroya, E., Mencl, W. E., Westerveld, M., Ni, W., & Shankweiler, D. (2004). Sentence complexity and input modality effects in sentence comprehension: An fmri study. NeuroImage, 22, Conway, C. M., Pisoni, D. B., & Kronenberger, W. G. (2009). The importance of sound for auditory sequencing abilities. Current Directions in Psychological Science, 18, Coull, J. T. (2004). fmri studies of temporal attention: allocating attention within, or towards, time. Cognitive Brain Research, 21, Cutler, A. (1994). Segmentation problems; rhythmic solutions. Lingua, 92, Dannenbring, G. L., & Briand, K. (1982). Semantic priming and the word repetition effect in a lexical decision task. Canadian Journal of Psychology, 36, van Dijk, T. A., & Kintsch, W. (1983). Strategies of discourse comprehension. New York: Academic Press. Fazio, P., Cantagallo, A., Craighero, L., D Ausilio, A., Roy, A. C., Pozzo, T., Calzolari, F., Granieri, E., & Fadiga, L. (2009). Encoding of human action in Broca s area. Brain, 132, Francès, R. (1958). La perception de la musique (2nd ed.). Paris: Vrin.

13 580 B. Tillmann Topics in Cognitive Science 4 (2012) Friederici, A. D. (1995). The time course of syntactic activation during language processing: A model based on neuropsychological and neurophysiological data. Brain and Language, 50, Friederici, A. D. (2002). Towards a neural basis of auditory sentences processing. Trends in Cognitive Science, 6, Friederici, A. D., Meyer, M., & von Cramon, D. Y. (2000). Auditory language comprehension: An event-related fmri study on the processing of syntactic and lexical information. Brain and Language, 74, Friederici, A. D., Pfeifer, E., & Hahne, A. (1993). Event-related brain potentials during natural speech processing: effects of semantic, morphological and syntactic violations. Cognitive Brain Research, 3, Friederici, A. D., Rueschemeyer, S.-A., Hahne, A., & Fiebach, C. J. (2003). The role of left inferior frontal and superior temporal cortex in sentence comprehension: Localizing syntactic and semantic processes. Cerebral Cortex, 13, Fuster, J. M. (2001). The prefrontal cortex An update: Time is of the essence. Neuron, 30, Gelfand, J. R., & Bookheimer, S. Y. (2003). Dissociating neural mechanisms of temporal sequencing and processing phonemes. Neuron, 38, Gibson, E. (1998). Linguistic complexity: Locality of syntactic dependencies. Cognition, 76, Gibson, E., & Pearlmutter, N. J. (1998). Constraints on sentence comprehension. Trends in Cognitive Sciences, 2, Gomez, R. L., & Gerken, L. A. (2000). Infant artificial language learning and language acquisition. Trends in Cognitive Science, 4, Hagoort, P. (2005). On Broca, brain, and binding: A new framework. Trends in Cognitive Sciences, 9, Hoch, L., Poulin-Charronnat, B., & Tillmann, B. (2011). The tonal function of a task-irrelevant music influences language processing: syntactic versus semantic structures. Frontiers Psychology - Auditory Neuroscience, 2, 112. Hoch, L., & Tillmann, B. (2012). Shared structural and temporal integration resources for music and arithmetic processing. Acta Psychologica, 140, Huron, D. (2006). Sweet anticipation: Music and the psychology of expectation. Cambridge, MA: MIT Press. Hyde, K. L., Lerch, J., Norton, A., Forgeard, M.,. Winner, E., Evans, A. C., & Schlaug, G. (2009). Musical training shapes structural brain development. Journal of Neuroscience, 29, Jackendoff, R. (2002). Foundations of language: Brain, meaning, grammar, evolution. New York: Oxford University Press. Jackendoff, R. (2009). Parallels and nonparallels between language and music. Music Perception, 26, Janata, P., & Grafton, S. T. (2003). Swinging in the brain: Shared neural substrates for behaviors related to sequencing and music. Nature Neuroscience, 6, Jentschke, S., & Koelsch, S. (2009). Musical training modulates the development of syntax processing in children. Neuroimage, 15, Jentschke, S., Koelsch, S., Sallat, S., & Friederici, A. D. (2008). Children with specific language impairment also show impairment of music-syntax processing. Journal of Cognitive Neuroscience, 20, Jones, M. R. (1976). Time our lost dimension. Psychological Review, 83, Jones, M. R., & Boltz, M. (1989). Dynamic attending and responses to time. Psychological Review, 96, Jones, M. R., Moynihan, H., MacKenzie, N., & Puente, J. (2002). Temporal aspects of stimulus-driven attending in dynamic arrays. Psychological Science, 13, Jonides, J., & Mack, R. (1984). On the cost and benefit of cost and benefit. Psychological Bulletin, 96, Kaan, E., & Swaab, T. Y. (2002). The brain circuitry of syntactic compréhension. Trends in Cognitive Science, 6, Kintsch, W. (1988). The role of knowledge in discourse comprehension: A construction-integration model. Psychological Review, 95, Koelsch, S., Gunter, T. C., Schröger, E., Tervaniemi, M., Sammler, D., & Friederici, A. D. (2001). Differentiating ERAN and MMN: An ERP study. NeuroReport, 12,

14 B. Tillmann Topics in Cognitive Science 4 (2012) 581 Koelsch, S., Gunter, T. C., von Cramon, D. Y., Zysset, S., Lohmann, G., & Friederici, A. D. (2002). Bach speaks: A cortical language-network serves the processing of music. NeuroImage, 17, Koelsch, S., Gunter, T. C., Wittfoth, M., & Sammler, D. (2005). Interaction between syntax processing in language and in music: An ERP study. Journal of Cognitive Neuroscience, 17, Koelsch, S., Jentschke, S., Sammler, D., & Mietchen, D. (2007). Untangling syntactic and sensory processing: An ERP study of music perception. Psychophysiology, 44, Kotz, S. A., Cappa, S. F., von Cramon, D. Y., & Friederici, A. D. (2002). Modulation of the Lexical Semantic Network by auditory semantic priming: An event-related functional MRI study. NeuroImage, 17, Kotz, S., von Cramon, D., & Friederici, A. (2005). On the role of phonological short-term memory in sentence processing: ERP single case evidence on modality-specific effects. Cognitive Neuropsychology, 22, Krumhansl, C. L. (1990). Cognitive foundations of musical pitch. New York: Oxford University Press. Large, E. W., & Jones, M. R. (1999). The dynamics of attending: How people track time-varying events. Psychological Review, 106, Lelekov, T., Dominey, P. F., & Garcia-Larrea, L. (2000). Dissociable ERP profiles for processing rules vs. instances in a cognitive sequencing task. NeuroReport, 11, Lelekov-Boissard, T., & Dominey, P. F. (2002). Human brain potentials reveal similar processing of non-linguistic abstract structure and linguistic syntactic structure. Neurophysiological Clinic, 32, Lerdahl, F. (2001). Tonal pitch space. New York: Oxford University Press. Lerdahl, F., & Jackendoff, R. (1983). A generative theory of tonal music. Cambridge, MA: The MIT Press. Maess, B., Koelsch, S., Gunter, T. C., & Friederici, A. D. (2001). Musical syntax is processed in Broca s area: An MEG study. Nature Neuroscience, 4, Magne, C., Schön, D., & Besson, M. (2006). Musician children detect pitch violations in both music and language better than non-musician children: Behavioral and electrophysiological approaches. Journal of Cognitive Neuroscience, 18, Marmel, F., & Tillmann, B. (2009). Tonal priming beyond tonics. Music Perception, 26, Marmel, F., Tillmann, B., & Delbé, C. (2010). Priming in melody perception: Tracking down the strength of cognitive expectations. Journal of Experimental Psychology: Human Perception & Performance, 36, Marmel, F., Tillmann, B., & Dowling, W. J. (2008). Tonal expectations influence pitch perception. Perception & Psychophysics, 70, Marques, C., Moreno, S., Castro, S. L., & Besson, M. (2007). Musicians detect pitch violation in a foreign language better than nonmusicians: Behavioral and electrophysiological evidence. Journal of Cognitive Neuroscience, 19, McAdams, S. (1989). Contraintes psychologiques sur les dimensions porteuses de formes en musique. In S. McAdams & I. Deliège (Eds.), La musique et les sciences cognitives (pp ). Bruxelles: Mardaga. McIntosh, G. C., Brown, S. H., Rice, R. R., & Thaut, M. H. (1997). Rhythmic auditory-motor facilitation of gait patterns in patients with Parkinson s disease. Journal of Neurology Neurosurgery & Psychiatry, 62, McNamara, T. P. (2005). Semantic priming: Perspectives from memory and word recognition. Sussex, England: Psychology Press. Meyer, L. B. (1956). Emotion and meaning in music. Chicago: University of Chicago Press. Moreno, S., Marques, C., Santos, A., Santos, M., Castro, S. L., & Besson, M. (2009). Musical training influences linguistic abilities in 8-year-old children: More evidence for brain plasticity. Cerebral Cortex, 19, Neely, J. H. (1991). Semantic priming effects in visual word recognition: A selective review of current findings and theories. In D. Besner & G. W. Humphreys (Eds.), Basic processes in reading: Visual word recognition (pp ). Hillsdale, NJ: Lawrence Erlbaum. Núñez-Peña, M. I., & Honrubia-Serrano, M. L. (2004). P600 related to rule violation in an arithmetic task. Cognitive Brain Research, 18, Overy, K. (2003). Dyslexia and music. From timing deficits to musical intervention. Annals of the New York Academy of Sciences, 999,

15 582 B. Tillmann Topics in Cognitive Science 4 (2012) Pacton, S., Perruchet, P., Fayol, M., & Cleeremans, A. (2001). Implicit learning in real world context: The case of orthographic regularities. Journal of Experimental Psychology: General, 130, Patel, A. D. (2003). Language, music, syntax and the brain. Nature Neuroscience, 6, Patel, A. D. (2008). Music, language, and the brain. Oxford, England: Oxford University Press. Patel, A. D., Gibson, E., Ratner, J., Besson, M., & Holcomb, P. J. (1998). Processing syntactic relations in language and music: An event-related potential study. Journal of Cognitive Neuroscience, 10, Patel, A. D., Iversen, J. R., Wassenaar, M., & Hagoort, P. (2008). Musical syntactic processing in agrammatic Broca s aphasia. Aphasiology, 22, Peretz, I., Belleville, S., & Fontaine, S. (1997). Dissociation entre musique et langage après atteinte cérébrale: un nouveau cas d amusie sans aphasie. Revue canadienne de psychologie expérimentale, 51, Petersson, K. M., Forkstam, C., & Ingvar, M. (2004). Artificial syntactic violations activate Broca s region. Cognitive Science, 28, Piston, W. (1978). Harmony (4th ed.). New York: Norton. Port, R. F. (2003). Meter and speech. Journal of Phonetics, 31, Poulin-Charronnat, B., Bigand, E., Madurell, F., & Peereman, R. (2005). Musical structure modulates semantic priming in vocal music. Cognition, 94, B67 B78. Quené, H., & Port, R. F. (2005). Effects of timing regularity and metrical expectancy on spoken-word perception. Phonetica, 62, Rohrmeier, M. (2011). Towards a generative syntax of tonal harmony. Journal of Mathematics and Music, 5, Rohrmeier, M., & Cross, I. (2008). Statistical properties of harmony in Bach s chorales. In Ken ichi Miyazaki, Yuzuru Hiraga, Mayumi Adachi, Yoshitaka Nakajima, & Minoru Tsuzaki (Eds.), Proceedings of the 10th international conference on music perception and cognition (pp ). Roy, A. C., & Arbib, M. A. (2005). The syntactic motor system. Gesture, 5(1 2), Sammler, D., Harding, E. E., D Ausilio, A., Fadiga, L., & Koelsch, S. (2010). Music and action: Do they share neural resources? International Conference of Music Perception and Cognition 11, Abstract booklet edited by S. M. Demorest, S. J. Morrison & P. S. Campbell, p. 87. Santos, A., Joly-Pottuz, B., Moreno, S., Habib, M., & Besson, M. (2007). Behavioural and event-related potential evidence for pitch discrimination deficit in dyslexic children: Improvement after intensive phonic intervention. Neuropsychologia, 45, Särkämö, T., Tervaniemi, M., Laitinen, S., Forsblom, A., Soinila, S., Mikkonen, M., Autti, T., Silvennoinen, H. M., Erkkilä, J., Laine, M., Peretz, I., & Hietanen, M. (2008). Music listening enhances cognitive recovery and mood after middle cerebral artery stroke. Brain, 131, Schellenberg, E. G. (2004). Music lessons enhance IQ. Psychological Science, 15, Schellenberg, E. G., Adachi, M., Purdy, K. T., & McKinnon, M. C. (2002). Expectancy in melody: Tests of children and adults. Journal of Experimental Psychology: General, 131, Schellenberg, E. G., Bigand, E., Poulin, B., Garnier, C., & Stevens, C. (2005). Children s implicit knowledge of harmony in Western music. Developmental Science, 8, Schenker, H. (1935). Der Freie Satz. Neue musikalische Theorien und Phantasien. (N. Meeùs, Trans.). Liège: Margada. Schlaug, G. (2001). The brain of musicians. A model for functional and structural adaptation. Annals of the New York Academy of Science, 930, Schlaug, G. (2009). Part VI. Listening to and making music facilitates brain recovery processes. In S. Dalla Bellaet al. (Eds.), The neurosciences and music III disorders and plasticity (pp ). Annals of the New York Academy of Sciences, Volume Schlaug, G., Altenmueller, E., & Thaut, M. (2010). Special issue: Music listening and music making in the treatment of neurological disorders and impairement. Music Perception, 27, Schlaug, G., Norton, A., Overy, K., & Winner, E. (2005). Effects of music training on brain and cognitive development. Annals of the New York Academy of Science, 1060,