Music Improvisation: Spatiotemporal Patterns of Coordination. A thesis submitted to the. Division of Graduate Education and Research

Transcription

1

2 Music Improvisation: Spatiotemporal Patterns of Coordination A thesis submitted to the Division of Graduate Education and Research of the University of Cincinnati in partial fulfillment of the requirements for the degree of MASTER OF ARTS in the Department of Psychology of the McMicken College of Arts and Sciences by Ashley Walton B.S., B.I.S., University of Cincinnati, 2011 December, 2015 Committee Chair: Michael J. Richardson, Ph.D. Committee: Anthony Chemero, Ph.D., Heidi Kloos, Ph.D., Peter Langland-Hassan, Ph.D. Readers: Elaine Hollensbe, Ph.D., Richard C. Schmidt, Ph.D.,

3 Abstract Interpersonal coordination plays a key role in the dynamics and effective outcome of musical performance. Such coordination is not only characteristic of musicians performing highly practiced and structured musical scores, but it is also a dominant feature of improvised musical performance. The current study was designed to investigate the multi-scaled dynamics of the movement coordination that supports collaborative musical improvisation and performance. The spatiotemporal dynamics of movement coordination that occur between improvising musicians was captured by recording the playing behavior and body movements of pairs of improvising pianists. The structure and complexity of the performance context was manipulated (i.e., musical key, chord progression, and rhythm), as well as visual information about a musician s co-performer. The patterning and dynamic stabilities of the inter-musician coordination that emerged from the different improvisation contexts was then examined across multiple spatial and temporal scales using a range of process-oriented nonlinear time-series techniques. It was hypothesized that the dynamic structure of a musical performance, including the emergence and stability of novel musical expressions, would not only be constrained by the structure of the musical context (i.e., backing-track, visual information), but also by the dynamic structure of inter-musician movement coordination and synergistic responsiveness (i.e., reciprocal adaptation to unexpected behavioral fluctuations). In general, the results revealed that the structure of the musical context determined the playing dynamics and movement coordination that emerged between musicians, such that when there was a rhythmic and harmonic foundation musicians engaged in more variable playing behavior. In contrast, the musicians playing behavior was less variable and more tightly coordinated when they had to co-construct this foundation on their own (drone backing track). 2

4 The results also highlight how the performative and function role of each limb and body movement differed across fast and slow timescales, synergistically and complementarily defined to enhance the structural integrity or creativity of the musical performance. Finally, the current findings demonstrate how the contemporary, nonlinear time- and event-series methods employed and developed in this thesis provide a new and exciting way to explore the dynamics of the complex coordinative processes that occur during multi-agent musical performance and how these dynamics shape and constrain novel musical expression and improvisation. 3

5

6 Acknowledgements This research would not have been possible without the continued commitment and guidance of Michael J. Richardson, as well as my committee members. I would also like to thank Auriel Washburn, who was integral in data collection and analysis. This work was supported by the National Institutes of Health (R01MH094659). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. 4

7 Table of Contents Abstract... 2 Acknowledgements... 4 Table of Contents... Error! Bookmark not defined. List of Figures List of Tables CHAPTER Introduction Understanding Improvisation Movement coordination in music performance Social Motor Coordination Measuring musical spontaneity Current study CHAPTER METHOD AND ANALYSES Analyses CHAPTER MIDI Results & Discussion Recurrence Results Cross Recurrence Results MIDI Analyses Section Summary CHAPTER Principal Component Analysis Results & Discussion Principal Component Analysis Section Summary CHAPTER Cross Wavelet Results & Discussion Head Movement Left Arm Movement Right Arm Movement Full Body Movement CHAPTER General Discussion Coordination in playing behavior Movement coordination and dimensional compression Movement coordination at target frequencies

8 Qualitative Analysis of post-session interviews Conclusion References APPENDIX A APPENDIX B

9 Figure 1. Cross wavelet plot. (p. 20) List of Figures Figure 2. Illustration of circular variance. (p. 22) Figure 3. Cross wavelet plot with bands. (p. 24) Figure 4. Real and virtual pairs. (p. 26) Figure 5. Auto recurrence plot. (p. 29) Figure 6. RQA results for keys/combinations of keys pressed. (p. 31) Figure 7. RQA results for average key press velocity. (p. 34) Figure 8. RQA results for key press timing. (p. 36) Figure 9. Cross recurrence plot. (p. 38) Figure 10. CRQA %REC for keys/combinations of keys pressed. (p. 39) Figure 11. CRQA MaxLine for keys/combinations of keys pressed. (p. 40) Figure 12. CRQA %REC for average key press velocity. (p. 42) Figure 13. CRQA MaxLine for average key press velocity. (p. 43) Figure 14. CRQA %REC for key press timing. (p. 45) Figure 15. CRQA MaxLine for key press timing. (p. 46) Figure 16. PCA illustration. (p. 49) Figure 17. PCA results for individual movements. (p. 52) Figure 18. PCA results for individual movements by condition. (p. 52) Figure 19. PCA results for pairs, number of components that accounted for 80%. (p. 52) Figure 20. PCA results for pairs, variance accounted for by first component (p. 56) Figure 21. PCA results for pairs, variance accounted for by second component (p. 58) Figure 22. PCA results for pairs, variance accounted for by each component (p. 59) Figure 23. PCA results intra- versus inter-personal dimensional compression (p. 61) Figure 24. Inv. circular variance, head movements,.5 to 1.5 sec. freq. (p. 66) Figure 25. Inv. circular variance, head movements, 3.5 to 4.5 sec. freq. (p. 68) 7

10 Figure 26. Inv. circular variance, head movements, 7.5 to 8.5 sec. freq. (p. 69) Figure 27. Inv. circular variance, head movements, 15.5 to 16.5 sec. freq. (p. 70) Figure 28. Inv. circular variance, left arm movements,.5 to 1.5 sec. freq. (p. 72) Figure 29. Inv. circular variance, left arm movements, 3.5 to 4.5 sec. freq. (p. 73) Figure 30. Inv. circular variance, left arm movements, 7.5 to 8.5 sec. freq. (p. 74) Figure 31. Inv. circular variance, left arm movements, 15.5 to 16.5 sec. freq. (p. 75) Figure 32. Inv. circular variance, right arm movements,.5 to 1.5 sec. freq. (p. 75) Figure 33. Inv. circular variance, right arm movements, 3.5 to 4.5 sec. freq. (p. 78) Figure 34. Inv. circular variance, right arm movements, 7.5 to 8.5 sec. freq. (p. 79) Figure 35. Inv. circular variance, right arm movements, 15.5 to 16.5 sec. freq. (p. 81) Figure 36. Inv. circular variance, full body 1 st component,.5 to 1.5 sec. freq. (p. 85) Figure 37. Inv. circular variance, full body 1 st component, 3.5 to 4.5 sec. freq. (p. 86) Figure 38. Inv. circular variance, full body 1 st component, 7.5 to 8.5 sec. freq. (p. 87) Figure 39. Inv. circular variance, full body 1 st component, 15.5 to 16.5 sec. freq. (p. 88) Figure 40. Inv. circular variance, full body 2 nd component,.5 to 1.5 sec. freq. (p. 91) Figure 41. Inv. circular variance, full body 2 nd component, 3.5 to 4.5 sec. freq. (p. 93) Figure 42. Inv. circular variance, full body 2 nd component, 7.5 to 8.5 sec. freq. (p. 94) Figure 43. Inv. circular variance, full body 2 nd component, 15.5 to 16.5 sec. freq. (p. 95) 8

11 List of Tables Table 1. Summary of RQA results. (p. 47) Table 2. Summary of CRQA results. (p. 48) Table 3. Summary of intra-personal PCA results. (p. 62) Table 4. Summary of inter-personal PCA results. (p. 63) Table 5. Summary of cross wavelet results for head movements. (p. 71) Table 6. Summary of cross wavelet results for left arm movements. (p. 76) Table 7. Summary of cross wavelet results for right arm movements. (p. 83) Table 8. Summary of cross wavelet results for head and right arm movements. (p. 84) Table 9. Summary of cross wavelet results for 1 st component of full body. (p. 90) Table 10. Summary of cross wavelet results for 1 st component of full body. (p. 95) 9

12 CHAPTER 1 Introduction Interpersonal coordination plays a key role in the dynamics and effective outcome of musical performance. This coordination requires that musicians demonstrate a kind of precise flexibility with respect to both auditory structure and the patterning of their body and limb movements. In other words, musical competence demands the collective synchronization of both the auditory and kinesthetic dimensions, whereby the music-making body and the sonic traces it leaves behind are pivotal to this co-articulation (Iyer, 2004). Such coordination is not only characteristic of musicians performing highly practiced and structured musical scores, but it is also a dominant feature of improvised musical performance, despite the spontaneous, unplanned melodic and temporal exploration that characterizes an improvised exchange. For example, when a jazz trio performs an improvisational piece the musicians behaviors are not fully prescribed in advance. Nonetheless their actions become so tightly coordinated and their decisions so seamlessly intertwined that the trio behaves as a single synergistic unit rather than a collection of individuals. This kind of cohesive, yet highly flexible coordination also occurs between band members jamming in a garage or between pairs of friends improvising a sequence on drums, piano, trumpet or violin. Understanding Improvisation Even everyday, planned tasks require continuous moment-to-moment improvisation. You can never navigate your car through the exact same highway traffic, or cook your favorite meal the exact same way life is a continual improvisation (Agre & Chapman, 1987, p. 268). Understanding the skills behind musical improvisation is vastly more complex. Pressing (1998, 10

13 2000) proposed a theory for how musicians can engage in such spontaneous musical action, describing skills and tools that musicians use to overcome the limitations of their informationprocessing capacities. Berkowitz (2010) has also proposed that musicians have a knowledge base which optimizes speed and efficiency by organizing musical materials into higher-level categories according to their function (Berkowitz, 2010, p. 54). He claims that this knowledge base is refined through practicing variants of different musical expressions, as well as how these units can be combined. The ability to combine these units is developed through statistical learning, where transitional probabilities that describe the likelihood of one musical event following another form patterns, which then constitute the hierarchical structure of the musician s knowledge base. Berkowitz s explanation also addresses how at the peak of performance musicians are often unaware of their actions and the details of how this skill is being executed. Berkowitz (2010, p.125) calls this the creator- witness phenomenon ; Jeff Pressing also describes how often in improvised performance... the hands appear to have a life of their own, (Pressing, 2000, p. 139). Berkowitz claims that when musicians let go the automated components and processes that make up a musician s expert knowledge base are what drive their musical action. However he doesn t provide a clear account for how this knowledge base is at play during these creatorwitness experiences, saying only that musical flow magically [emphasis added] manifests, without a need to know or remember where one has been or where one is going (Berkowitz, 2010, p. 130). This explanation of how a stable knowledge base supports improvised musical expression forgets the lives of the hands (Sudnow, 1978), the choices of the ears (Berkowitz, 2010) the body s own actions (Berliner, 1994). This study will attempt to explore how the attribution of agency to the body and its limbs goes beyond simple metaphor to provide new 11

14 possibilities for discovering stable patterns that support the spontaneous and creative production of music improvisation. Movement coordination in music performance. The importance of body movement in understanding musical performance has become well accepted, Vijay Iyer provides detailed account of how musical bodies tell stories (Iyer, 2002, 2004), and the dynamics of movement and force in musical performance have been widely examined experimentally (see Keller, 2012; Palmer, 2013 for review). Kinesthetic patterns have been found to be the primary determinant of everything from musical genres, to structures of instruments, as well as musician s personal identities (Baily, 1985; Dalla Bella and Palmer, 2011). Additionally, these patterns are essential to understanding not only the production but also the perception of music, in that listeners directly experience the articulators of performers the changes in rate and force of the bodily movements that produce musical sounds (Shove & Repp, 1995). For instance, Thompson, Graham and Russo (2005) have demonstrated how visual information about a performer s movement influences a listeners perception of the music performed by examining the bodily-mediation of facial expressions and limb gestures used by musicians to highlight and articulate phrases within the performance. These kinds of affective displays, such as when musician BB King opens his mouth and shakes his head to match the vibrato of a note, serve to constrain the perceptual experience of listeners. Furthermore, these musical movements are not considered peripheral, but rather provide a listener with direct access to the perceptions, moods and feelings of the musician (Maes et al., 2014). These coordinative patterns are not only important with respect to musicians performing highly practiced and structured musical scores (Keller & Appel, 2010; Loehr & Palmer, 2011; Ragert et al., 2013; Palmer & Loehr, 2013), but also with regard to how improvising musicians spontaneously produce cohesive musical expressions while simultaneously interpreting and 12

15 coordinating with those produced by other performers. Previous experimental investigations have focused on individual improvisers (e.g. Norgaard, 2011; 2014; Keller et al., 2011). Yet the paradigmatic example of improvisation is a duet or jazz trio, where multiple musical bodies must spontaneously coordinate while simultaneously engaging in both musical perception and action. In such situations, musicians are engaged in a continuous and complex time-evolving negotiation anticipating and coordinating their playing behavior without the guide of musical notation. In other words, the improvised musical performance emerges within a context of social collaboration, where the ongoing inter-musician interactions operate to construct and constrain the flow of the performance from moment-to-moment (Sawyer, 2003). Social Motor Coordination. Many of our daily activities are performed in a social setting and require that individuals coordinate their movements and actions with those of other people. Such social or interpersonal behavioral coordination is a natural and inherent part of navigating a crowded sidewalk, clearing a table with friends and family, or dancing together with a romantic partner, and is essential for everyday social functioning. Indeed, social motor coordination seems to be intrinsically important to how we share time with others (e.g., Marsh, et al., 2009; Semin, 2007), with research from a broad range of disciplines demonstrating how social motor coordination is related to feelings of interpersonal rapport (e.g., Bernieri & Rosenthal, 1991; Hove & Risen, 2009; Miles et al., 2011) and cooperation (Miles et al., 2009; Wiltermuth & Heath, 2009) and also to the breakdown of social cognitive functioning in autism (Isenhower et al., 2009; Fitzpatrick, et al., 2013; Trevarthen & Daniel, 2005) and schizophrenia (Condon & Ogston, 1966; Varlet et al, 2012). Moreover, cognitive psychologists studying conversational interactions have found that social movement coordination appears to play a fundamental role in verbal 13

16 comprehension, social understanding and perception, as well as the cognitive performance of interacting individuals (Duran, et al., 2014; Fowler, et al., 2008; Shockley, D.C. Richardson & Dale, 2009). Over the last two decades, attempts to understand the mechanisms that support social motor coordination have largely focused on identifying the neural and representational mechanisms that support social action observation and the development of shared intentional states (e.g., Graf, et al., 2009; Newman-Norlund, et al., 2007; Rizzolatti & Craighero, 2004; Sebanz & Knoblich, 2009; Knoblich, et al., 2011). However, studies of intra as well as interpersonal coordination of body movement began early as the 1960s (see Condon & Ogston 1966; 1967 ;1971). Schmidt & Fitzpatrick (in press) describe how interpersonal coordination research in the 1980s and 1990s began to focus on entrainment. Studies of the dynamical processes of inter-limb rhythmic coordination (see Schmidt, Carello & Turvey, 1990; Schmidt & Turvey, 1994; Schmidt & O Brian, 1997) made way for exploring the role of perceptual coupling in the formation of synergies where human agents engaged in a coordinated social activity behave as a single, unified behavioral system (Riley, et al., 2011; Schmidt & Richardson, 2008). The latter research by Schmidt and colleagues as well as more contemporary research has also placed a strong emphasis on how the behavioral order of coordinated social activity is often selforganized naturally emerging from the physical interactions, mutual relations and informational couplings that exists between co-actors (see e.g.; Schmidt, Carello & Turvey, 1990; Chemero, 2009; Dale, et al., 2013; Richardson, et al., 2010; Riley, et al., 2011). The significance of this research is that it highlights how investigating the processes that support effective social coordination and communication requires that researchers develop a clear understanding of the time-evolving dynamics of such behavior and the degree to which these 14

17 dynamics structure the behavioral possibilities of effective social interaction (Anderson, et al., 2012; Dale, et al., 2013; Richardson, et al., 2014; Richardson & Chemero, 2014). Improvising musicians are human agents engaged in social movement coordination. Thus, investigating the nature of how they are informationally coupled and the time-evolving dynamics of their behavioral coordination can provide insight into how they effectively collaborate to produce innovative and cohesive musical expressions from the overwhelming number of combinatorial possibilities. This self-organized, interpersonal movement coordination can also provide a way of understanding exactly what is happening when musicians let go and submit to musical spontaneity. Measuring musical spontaneity. In his book The Ways of the Hand, David Sudnow documents his experience learning to improvise jazz on the piano. He describes observing performances by the New York jazz piano player Jimmy Rowles: I watched him night after night, watched him move from chord to chord with a broadly swaying participation of his shoulders and entire torso, watched him delineate waves of movement, some broadly encircling, others subdividing the broadly undulating strokes with finer rotational movements... As his foot tapped up and down his head went through a similar rotational course, and the strict up-and-down tapping of the foot was incorporated in a cyclical manner of accenting his bodily movements. In an anchored heel you could see only the up-and-down movements of the foot, but in the accompanying head rotation and shoulder swaying you could see a circularly undulating flow of motion... Sudnow (1978, p. 82) Sudnow s description includes the full spectrum of Jimmy Rowles s bodily articulations, observing the rotational movements of not just the hands, but also the head, shoulders and feet. 15

18 Head-bobbing or foot-tapping may usually be considered more gestural or peripheral to musical production, but examining how the dynamics of a performer s, as well as their co-performer s, musical movements may provide insight into how this interpersonal coordination constrains the range of expressions within musical performance. As Sudnow describes the subdivisions in the rotations of Jimmy Rowle s different body movements, there are multiple time scales at which this interpersonal coordination constrains musical performance. The dynamics of this coordination is not easily isolated into components, nor can it be strictly defined by content or by a particular frame of time. Musician s movements may at times involve explicit communicative signals such as a touch to the head that signals back to the top, or eye contact and nodding of the head before or after solos. But these are just a small part of a continuous flow of information about a co-performer that supports adaptive coordination and communication across the multiple time scales of an improvised musical performance. Given the complexity of these spatiotemporal patterns of movement coordination that often involve non-stationary and aperiodic behavioral sequences, very little research has attempted to examine the interpersonal coupling that emerges between improvising musicians (Shockley & Riley, 2015; Walton et al., 2015). Process-oriented nonlinear time-series methods provide tools to evaluate the dynamics of these spatiotemporal processes, and have been previously applied in understanding individual and rehearsed musical performance (Demos et al., 2011; 2014; Beauvois, 2007; Hennig, 2014; Rankin et al., 2009; Ruiz et al., 2014; Serrà, et al., 2009; 2011; Glowinski et al., 2013; Keller, et al., 2011), as well as in the case of solo improvisation (Borgo, 2005; Pressing, 1999, 2000). However, non-linear time series methods also make possible detailed examinations of how the dynamics of movement coordination between improvising musicians evolve across the time span of musical performance. 16

19 Uncovering how the dynamics of these spontaneous coordinative behaviors unfold provides a way of better understanding the exchanges between order and violations of order that potentiate the novelty that characterizes improvisatory expression. Without the guide of notation, improvising musicians must be engaged in a continuous negotiation, anticipating and coordinating with changes in different aspects of each other s musical expression. This anticipatory coordination can result in dramatic transitions towards unexpected trajectories when musicians act upon information about their co-performer, as well as adapt their playing in order to re-contextualize and even take advantage of musical errors or noise. Movement coordination is an important part of the information that initiates these transitions to novel modes of expression: saxophonist Even Parker claims sometimes the body leads the imagination (Borgo, 2005). Quantifying these spatiotemporal patterns can provide an understanding of what kinds of dynamics make possible this spontaneous emergence of previously unimagined forms of order. Current study. The goal of the current study was to identify the multi-scaled dynamics of the movement coordination that support collaborative musical improvisation and performance. The spatiotemporal dynamics of movement coordination that occur between improvising musicians was captured by recording the playing behavior and body movements of pairs of improvising pianists. The structure and complexity of the performance context was manipulated (i.e., musical key, chord progression, and rhythm), as well as visual information about co-performer. The patterning and dynamic stabilities of the inter-musician coordination that emerged from the different improvisation contexts was then examined across multiple spatial and temporal scales using a range of process-oriented nonlinear time-series techniques (e.g., cross-recurrence 17

20 analysis, principal component analysis, and cross-wavelet analysis,). It was hypothesized that is the dynamic structure of a musical performance, including the emergence and stability of novel musical expressions, will be not only be determined and constrained by the structure of the musical context (i.e., backing-track, visual information), but also by the dynamic structure of the movement coordination and mutual responsiveness (i.e., reciprocal adaptation to unexpected behavioral fluctuations) that occurs between the musicians. It is important to note that this is the first research attempting to examine body movement in improvised music performance, thus it can be considered exploratory as it necessitated the testing and development of new applications of non-linear methods that could meaningfully capture inter-musician coordination. The results presented here represent a small portion of the potential analyses, and an initial attempt for which the methods can continue to be further developed. 18

21 CHAPTER 2 METHOD AND ANALYSES Participants. 7 pairs of musicians were recruited from the local music community as well as the University of Cincinnati s College-Conservatory of Music (CCM). One pair was eliminated because one of the players did not meet the minimum years of experience improvising (at least one year). The 6 pairs that were analyzed had 8 to 46 years of training in piano performance (M = 19.7, SD = 12.5) and 4 to 46 years of experience with improvisation (M = 14.9, SD = 13.5) and ranged in age from 18 to 59 years (M = 30.4, SD = 14.3). Procedure and Design. Participants played standing with an Alesis Q88, 88-key semiweighted USB/MIDI keyboard controller, directly facing one another while their movements were recorded using a Latus Polhemus, wireless motion tracking system (at 96 samples per second). Participants were equipped with motion sensors attached to their forehead, and both their left and right forearms (positioned directly below the point where their wrist bends). Ableton Live was used to record all of the MIDI key press commands and the resulting audio signal during the musical improvisation. Pairs were instructed to develop 2-minute improvised duets under vision and no-vision conditions, over different backing tracks. The vision and no-vision manipulation simply involved placing a curtain between musicians for half of the performances. Musicians were required to improvise along to three different backing tracks: an ostinato, a swing and a drone backing track. The ostinato backing track was a short melodic phrase consisting of the four ascending chords (Cm11; BbM7/D, EbM7#11, Fadd4) that was looped every four seconds, in 7/8 time, as opposed to the more common 4/4 time signature. The swing backing track was the bass line of a chord progression from the jazz standard used by Keller, 19

22 Weber and Engel (2011), titled: There s No Greater Love. This track has a key and time signature (4/4), with the standard s chord changes played on loop in a lower octave. Finally, the drone backing track was a pair of pitches, D and A, that were played for the entire duration of the two minutes. At the beginning of the experiment, the musicians first performed three individual warmup trials, where they improvised over each backing track while the other sat outside the performance room. Then together the pairs performed two blocks of six two-minute improvised duets, for a total 12 duets (2 conditions x 3 backing tracks x 2 blocks). The six condition (vision, no-vision) by backing track (swing, ostinato, drone) trials within each block were pseudorandomized, such that there are no instances where identical backing tracks were performed sequentially. In between each trial musicians completed a brief survey that asked them to rate how coordinated they felt with their co-performer, how smooth the improvisation felt, and how difficult it felt to improvise with their co-performer, all on a scale of 1-9. There was also space for them to write any comments they wanted to share about their experience improvising for that trial. This survey is included in the Appendix X, however, none of the results are presented as part of this thesis. Following the completion of all the experimental trials, both musicians filled out a survey asking them questions regarding their feelings of affiliation towards their coperformer (adapted from Aron, Aron & Smallon, 1992), their experience improvising to each backing track, and their perception of the difficulties and/or success in coordinating with each other (see Appendix X). These survey results are also not presented in this document. After the musicians completed all of the improvisational trials they each were interviewed separately. During these interviews the video and audio from the last block of trials was played back using a laptop computer, so they were interviewed once for each experimental 20

23 condition. The interview method used was an adaptation of that used by Norgaard (2011). The following was be read by the experimenter before the interview: As you are watching and listening to your performance, try to narrate your conscious thinking, considering questions like, Where did that come from? We are looking for a narration similar to a director s commentary on a DVD. We are particularly interested in how you are able to play with the backing track as well as with your co-performer. We are also interested in your creative process. How are you making decisions about what to play and when in your improvisation? As the musicians watched the videos of their performances, the experimenters asked follow-up questions to explore and clarify points, as well as probed on interesting themes that emerged in their descriptions (Charmaz, 2006). Analyses The coordination that emerged between the improvising musicians with respect to their body movements and their musical production was analyzed using categorical recurrence quantification analysis (RQA), categorical cross recurrence quantification analyses (CRQA), principal component analyses (PCA) and cross wavelet spectral analysis. A brief description each of these analyses and how their measures were employed to capture the coordination dynamics that emerged between improvising musicians are included below. Additional details of each analysis method are also provided in each of the related results chapters (i.e., chapters 3 to 5). Categorical Recurrence and Cross Recurrence Analysis (Chapter 3). The MIDI output was used to create times series for each musician that captured the keys and combinations of keys pressed, when the keys were pressed, and the velocity of the musicians key presses. 21

24 These times series were then used to create both auto recurrence plots and cross recurrence plots that display when these various key press events recur over time, or visit the same states or sequence of states. Auto recurrence plots data events onto themselves, to see how an individual system repeats or revisits its own states across time. Cross recurrence takes two data event series and plots them against each other, to see when two systems repeat and revisit each other s states across time. Information about the dynamic structure of system behavior is provided through visual inspection of the patterns of these recurrence plots (Eckmann et al., 1987). More importantly, these dynamic patterns can be mathematically quantified using recurrence quantification analysis (RQA). RQA, or CRQA in the case of cross recurrence quantification, results in numerous statistics that each quantifies a different aspect of the dynamics entailed in the time-series data. The statistics percent recurrence (%REC): the ratio between the number of recurrent points found and the total number of recurrent points possible), and MaxLine: the length of the longest sequence of recurrent points, were employed here to quantify the recurrent dynamics observed. Principal Component Analysis (Chapter 4). The Polhemus motion tracking sensors were used to capture the movements of each musician s head, right forearm and left forearm. These sensors provided the x, y and z position for each body part throughout the course of the improvisations. These times series were then used to perform a principal component analysis (PCA). PCA is a widely used statistical technique to identify co-variation within high dimensional datasets and to remap the data into a space whose axes (principal components) represent the dataset s primary dimensions of variation (Daffertshofer et al., 2004). If the original variables are correlated, PCA yields a dimension reduction fewer principal components are required to account for most of the variance in the dataset than the number of original variables. For coupled perceptual-motor systems (musicians in this case), this reduction corresponds to 22

25 dimensional compression. How many principal components were required to capture 80% of the variance in the musician movements, the amount of variance account for by the first principal component, as well as the amount of variance accounted for by the second principal component were examined to evaluate differences in dimensional compression across experimental conditions. Additionally, the dimensional compression observed for the musician s intrapersonal movement coordination versus inter-personal movement coordination between the musicians were compared to evaluate whether the coordination that occurred was the result of intra-personal or interpersonal synergistic processes (or both). Cross Wavelet Spectral Analysis (Chapter 5). Cross-wavelet spectral analysis is a nonlinear time series initially developed within the fields of geological sciences and physiology. It assesses coordination between two time series through spectral decomposition, and subsequent examination of the strength (coherence, circular variance) and patterning (relative phase) of the coordination that occurs between participants across multiple time scales (see Grinsted et al., 2004; Issartel et al., 2014, for a more detailed introduction). More recently it has been employed as a tool for understanding the movement coordination that occurs between co-actors during joke-telling and dancing (Schmidt et al., 2014; Washburn et al., 2014). Schmidt et al. (2014) demonstrated the ability of cross wavelet analysis to reveal common periodicities in behavioral coordination at nested time scales, detecting local micro-scale structures within global macroscale patterns. It does so without assuming the time series is stationary, meaning it is able to capture the time-evolving behavior in time series that are noisy, contain a drift or sudden change in the mean. These qualities are highly characteristic of the complex patterns of coordination that emerge during spontaneous musical performance. Of particular importance for the current study, is that the method allows one to assess the magnitude of inter-musician coordination at different component time scales for the individual 23

26 movements of the head, left arm, and right arm, as well as fully body movement coordination (Walton et al., 2015). For example, in Figure 1 the level of coherence between the movements of the two musician s right arms over time is denoted by color (red for high coherence, dark blue for low to no coherence) and is displayed as a function of period (in units of seconds) on the y- axis. The arrows correspond to the relative phase of the coordination. Right arrows equal inphase coordination (the two systems are visiting the same states in perfect synchrony) and left arrows equal anti-phase coordination (the phases at which the two system are visiting the same states are in perfect opposition). Period (sec) Time (secs) Figure 1. Cross wavelet plots of the lateral movements of the musicians right forearms, displaying the strength of coherence at each period (red for high coherence = 1, dark blue for low to nocoherence = 0), as well as relative phase angle (right arrows equal in-phase coordination, left arrows equal anti-phase coordination). The cross wavelet measure used to evaluate coordination in the musician s movements was the inverse of circular variance, which represents the proportion of relative phase relationships visited between two time series. It is thus a measure of the variability of relative phase between two time series: more variability in relative phase relationships can be understood as less coordination between movements, and less variability in relative phase relationships can 24

27 be understood as more coordination between movement time series. Note that the inverse of circular variance was chosen over the coherence measure because it provides a better summary variable of spatiotemporal coordination stability (coherence provides only a temporal estimate of coordination stability). However, cross-spectral coherence and measures of relative phase variability and stability, such as inverse of circular variance, are known to be highly correlated (an r =.9 to.99; see e.g., Schmidt & O Brien, 1997; Schmidt et al., 2007; Richardson et al., 2005; Richardson et al., 2007). How to interpret values of inverse of circular variance is illustrated in Figure 2. An inverse of circular variance of zero means that two time series never visited the same relative phase relationship more than once. Higher values of the inverse of circular variance indicate that two time series visited a smaller set of relative phase relationships; a value of one would mean that two time series maintained the same relative phase relationship for the entire duration of a trial. It is important to note that there is not one phase relationship for which the time series are considered more or less coordinated, so for example in-phase (zero degrees phase relationship) or anti-phase (180 degrees relative phase relationship) are not of special significance. The circular variance only indicates the overall variability in relative phase relationships. 25

28 Figure 2. Illustration of the variability of relative phase represented by the inverse of circular variance. If two time series never revisit any relative phase relationship more than once, the value of the inverse of circular variance is zero. As two time series visit a smaller set of relative phase relationships the value of the inverse of circular variance approaches 1. As noted above, what is especially advantageous about the use cross wavelet analysis is that it allows one to assess the variability in relative phase relationships of the coordination that occurs between participants across multiple time scales. For example the variability in relative phase between the musician s movements every 1-second, every 2-seconds, every 4-seconds, etc., can be evaluated separately. This is made possible by only assessing the relative phase relationships within a specific frequency band on the cross wavelet plot. Accordingly, one can determine how movement coordination relates to the shorter- and longer-term temporal structure and phrasing of the musical performance. Hypotheses with respect to the relevant intervals at which you might see more coordination and thus less variability in relative phase can be developed based upon the timing and structure of the backing tracks provided for improvisation. The ostinato backing track is a melody repeated every four seconds, so one could hypothesize relevant musical intervals to include multiples of four (4-seconds, 8-seconds, 16-seconds), as well as 7-seconds given the 7/8 time signature. For the swing backing track one measure of the chord progression lasted 1.8 seconds, so two measures equals 3.6 seconds, four measures 7.2 seconds, 8 measures 14.4 seconds all of which could be considered intervals at which there might be more stability in movement coordination. For the drone backing track there is no rhythmic structure, but one would expect the musicians might create this rhythmical structure themselves through their playing, and that patterning may correspond to certain time intervals standard to a familiar genre of performance, like 4/4 time in jazz. While each of the backing tracks can be analyzed individually according to time intervals specific to their rhythmic structure, intervals were chosen such that movement coordination could be compared across backing tracks. Figure 3 displays the same cross wavelet plot from 26

29 Figure 1 but highlights in pink the specific frequency bands for which the variability in relative phase was evaluated for each movement time series. The targeted frequency bands were.5 to 1.5 seconds, 3.5 to 4.5 seconds, 7.5 to 8.5 seconds, and 15.5 to 16.5 seconds. Thus the variability in relative phase between the improvising musician s heads, left arms, right arms, and full body movements was evaluated for these four frequency bands. This allowed for a better understanding of the relevant component frequencies with respect to coordination of musicians different body parts, as well as how this coordination develops across trials, and how it is affected by the backing track and the availability of visual information about a co-performer. Figure 3. Cross wavelet plots of the lateral movements of the musicians right forearms, with the targeted frequency bands highlighted in magenta. The values of the inverse of circular variance are evaluated for only the region on the plot designated by the frequencies bands of interest (i.e..5 seconds to 1.5 seconds). A common issue in the analysis of movement coordination data is which movement dimension to employ, i.e., the x, y or z dimension or some combination of the three movement dimensions. Given that movement in the x, y and z dimensions often co-vary, one solution is to employ PCA analysis in order to extract a principle component time-series for limb or body 27

30 movement or sensor. This first principal component time series captures the change over time along the dimension of primary variation and thus provided the best collective measure of a limb or body movements over time (Riley et al., 2011). Accordingly, prior to conducting the crosswavelet analysis of the movement time-series, a PCA analysis was conducted on the z-score normalized x, y, and z dimensions for each sensor (i.e., head, right arm, left arms sensor) in order to extract a principal component time series for each sensor. Cross wavelet spectral analysis was used to quantify the coordination between these principal competent time series. Virtual Pairs Analysis. When considering inter-musician coordination, it is important to also measure the degree to which the coordination observed is related to musical structure of backing track itself, and in turn, the degree to which the coordination observed is a result of being coupled to and improvising to the same backing track. Note that such coordination does not correspond to chance coordination per say, but rather reflects the degree to which the musical coordination observed between improvising musicians is defined and constrained by a particular backing track. In order to understand how much coordination between players could be understood as driven by the musical context of the backing track, a virtual pairs analysis was conducted for each measures of interpersonal coordination (CRQA, PCA, and Cross Wavelet) in addition to the real pair CRQA analysis whereby the data for player 1 (always on the left side of the room) from each pair was analyzed with respect to player 2 (always on the right side of the room) data from every other pair. Conversely, the data for player 2 from each pair was analyzed with respect to player 1 data from every other pair. Figure 5 demonstrates how the virtual pairs were generated from the real pairs of piano players. In this case, for real pair #1 the coordination analysis was run between their time series first the real-pair analysis. Then their time series were matched up with the time series of the players from other pairs that were playing in the same block, condition, and with the same backing track the virtual-pair 28

31 analysis. The same coordination analysis was then run with each of these virtual pairs, and the resulting measures were averaged. This average represented the amount of coordination to be expected between two musicians playing behavior just by virtue of doing the same task, in the same block, under the same vision conditions, and playing with the same backing track. Comparing this to the coordination observed between musicians that were actually coupled in performance helped to capture how much of that coordination can be attributed to musicians improvising with each other. It is important to keep in mind that there isn t a directional hypothesis for the virtual pairs analyses for any of these measures it isn t clear how more successful musical production relates to more or less coordination in playing behavior, and most likely will vary depending on how the playing behavior is being measured. Figure 4. Illustration of the generation of real pair (thick arrow) and virtual pairs (thin arrows). Statistical Analysis. To assess whether the results from the virtual pairs analysis were significantly different from the real pairs analysis for the CRQA, interpersonal PCA and crosswavelet DVs, a 2 x 2 x 2 x 3 repeated measures ANOVA was conducted for each dependent measure of interpersonal coordination, with the factors of pair type (real vs. virtual), block, condition, and backing track. For shorthand this analysis will be referred to in the following chapters as the Real/Virtual Omnibus. Of particular significance in this analysis was whether 29

32 there was a main effect of pair type and any interaction effects with pair type. If not main or interaction effects of pair type were found, no other effects or subsequent analysis was performed with regards to this Real/Virtual Omnibus analysis fro each DV. That is, the Real/Virtual Omnibus was only employed to evaluate instances for which two perceptual motor systems actually being coupled made a difference in the interpersonal coordination. For the RQA and intrapersonal PCA DVs, and following the Real/Virtual Omnibus analysis for the CRQA, interpersonal PCA and cross-wavelet DVs, planned 2 (block) x 2 (vision condition) x 3 (backing track) repeated measures ANOVA on the real pairs data was always conducted, irrespective of the result found for the Real/Virtual Omnibus analysis. Due to the lack of power attributed to only having 6 pairs, results were considered significant if the p value was anything below.100, accompanied by a moderate to large effect size (.300 < η 2 p ). Subsequent simple effects analysis and LSD post-hoc tests were employed where necessary. 30

33 CHAPTER 3 MIDI Results & Discussion The results of RQA and CRQA are presented below in separate sections. Auto recurrence or RQA was used to capture how individual players repeated themselves in the structure of their musical behavior, and cross recurrence analysis or CRQA was used to show how paired musicians repeated the same or similar musical behaviors as each other. Specifically, the recurrent or co-recurrence of what keys/combination of keys they pressed, the velocity of their key presses, and the timing of their key presses. The resulting values of the RQA statistics %REC and MaxLine for the keys pressed, key press velocity, and key press timing were analyzed and are reported here. Recurrence Results Recurrence Analysis on Notes. The MIDI output provided the number of each key (from 1 to 88) as well as the on and off time for each key pressed. This was used to create a time series for each player that captured what keys and/or combinations of keys they pressed across the time span of the performance. Then the total number of unique combinations of keys pressed by both players was determined and each was assigned a code number. This code number was then substituted for the key or key combination it represented in order to generate a time series of code numbers for each player (see Figure 4). The MIDI key numbers were the only information used to define the musical states. Notes or chords formed by the keys pressed were not identified. Timing of the playing behavior was preserved by using random numbers to represent points in the performance when no keys were being pressed. Because RQA is used to examine patterns 31

34 that recur within a time series, random numbers instead of zeros results in non-playing moments as not being quantified as recurrent states. Figure 4 illustrates how the time series containing the notes or groups of notes played by each musician at each time point in the improvisation are mapped in an auto recurrence plot. Figure 5. The MIDI output was used to create a time series where the keys and combinations of keys pressed (B) were used to create a code number for each unique combination of notes played (C) and then a time series for each player was created using these code numbers as well random numbers when no notes were being played (D). This allowed for the quantification of when the musicians were repeating themselves throughout the performance. The results of the recurrence analysis on the notes the musicians played (keys pressed) are displayed in Figure 5. For %REC there were no main effects of block and condition (both F<2.17, p >.201, η 2 p <.302), but a significant two-way interaction between condition and backing track F(2,10) = 5.35, p =.026, η 2 p =.517, and a significant three-way interaction between block, backing track and condition F(2,10) = 5.16, p =.029, η 2 p =.508. The analysis was then split into two separate 2 (condition) x 3 (backing track) repeated measures ANOVA for each block. In the first block of trials there was a significant main effect of backing track F(2,10) = 4.40, p =.043, η 2 p =.468, with musicians repeating their note sequences significantly less often 32

35 when improvising to the swing track compared to the ostinato (p =.018) or drone track (p =.044). There was no main effect of condition, nor an interaction effect (both F<1.17, p >.328, η 2 p <.190). For the second block there was a significant interaction between backing track and condition, F(2,10) = 6.43, p =.016, η 2 p =.563. Accordingly, three separate simple effects ANOVAs were conducted for each backing track. This analysis revealed a significant main effect of condition for the ostinato backing track, F(1,5) = 25.31, p =.004, η 2 p =.835, a nonsignificant result but a moderate effect size for the drone backing track, F(1,5) = 3.40, p =.124, η 2 p =.405, and no significant effect of condition for the swing backing track F(1,5) =.918, p =.382, η p2 =.155. Thus, in the second block of trials for the ostinato track, musicians repeated their note structures more when they couldn t see each other, but for the drone track they repeated themselves more when they could see each other. For MaxLine there was a significant main effect of backing track F(2,10) = 8.82, p =.006, η 2 p =.638, with MaxLine for the swing backing track being significantly lower than for the ostinato (p=.016) and drone (p=.002) backing tracks. Thus, for the swing backing track musicians repeating themselves in shorter sequences than for the other two backing tracks. There were no other significant main effects or interactions (all F<2.66, p >.118, η p2 <.348). 33

36 Figure 6. Results of auto recurrence analysis on keys/combinations of keys pressed: %REC of the keys and combinations of keys pressed (top row) and MaxLine (bottom row). In summary, when improvising with the swing backing track musicians repeated their own note structures less, as well as produced shorter sequences of repeated note structures. This appears to be due to the fact that the swing backing track established a more consistent and familiar rhythmic foundation than the ostinato and drone backing tracks, which provided an opportunity for the musicians to more freely explore a wider variety of musical expressions. Difference in the breadth of musical (note) exploration also appears to underlie the effects of condition (vision vs. non vision) when improvising with the ostinato backing track in the second block of trials. Because the ostinato is harmonically and rhythmically dominant, when musicians have visual information about their co-performer they seemed to vary their note structures more to both compliment the track and each other. This effect of visual information may only arise in the second block of trials because musicians after having become practiced with coordinating 34

37 with the harmonic and rhythmic structure of the backing track in the first trials, are then capable of exploiting new information and exploring more variation in the later trials. Recurrence Analysis on Average Velocity. The MIDI data provides a velocity value for each key press, ranging from 1 to 127. A time series was created for each musician with the average velocity of the keys pressed at each time point. In order for a point to be considered recurrent the average velocity of the key press had to be the exact same value (the radius for the recurrence analysis was zero). The results of the recurrence analysis on the average velocities of the musicians key presses are displayed in Figure 6. For %REC there was a significant main effect of backing track F(2,10) = 9.68, p =.005, η p2 =.659, and a significant three-way interaction between block, backing track and condition F(2,10) = 5.45, p =.025, η 2 p =.521, with no other significant main effects nor interactions (all F<2.05, p >.212, η 2 p <.291). To unpack the three-way interaction, two separate 2 (condition) x 3 (backing track) ANOVAS were conducted for each block. In the first block there was a significant main effect of backing track F(2,10) = 3.31, p =.079, η 2 p =.398, and a significant interaction between condition and backing track F(2,10) = 6.47, p =.016, η p2 =.564. A simple effects analysis of backing track for each condition revealed that for the no vision condition there was a main effect of backing track F(2,10) = 9.73, p =.005, η 2 p =.661, where recurrence for the drone track was significant higher than the ostinato (p =.018) and swing backing track (p =.013). For the vision condition there was also a main effect of backing track F(2,10) = 5.02, p =.031, η 2 p =.501, where recurrence for the drone backing track was also significant higher than the swing track (p =.025). These %REC results for the first block indicated that when improvising along to the drone backing track musicians key press velocities were more similar overall across the vision and no vision conditions. 35

38 For the second block of trials there was a main effect of backing track, F(2,10) = 3.79, p =.060, η 2 p =.431; where %REC for the drone was higher than for the swing backing track (p =.094). There was no main effect of condition, nor an interaction effect (both F<1.28, p <.342, η 2 p <.193). Thus, consistent with the findings for the first block of trails, musicians key press velocities in the second block of trials were more similar when improvising along to the drone than the swing backing track, regardless of availability of visual information about their coperformer. For MaxLine there was a significant interaction between block and backing track F(2,10) = 3.64, p =.065, η p2 =.421, but no significant effect of condition, nor any other interaction effects (F<1.59, p <.263, η 2 p <.242). Although there was not a significant three-way interaction, the planned 2 (condition) by 3 (backing track) ANOVAS were still conducted for each block. For the first block of trials, there was significant main effect of backing track F(2,10) = 7.95, p =.009, η 2 p =.614, significantly lower MaxLine values observed for the swing backing track, compared to the ostinato (p =.017) and drone backing tracks (p =.0113). Thus, in the first block of trials the musicians repeated their own key press velocities in shorter sequences for the swing track compared to when playing with the other tracks. For the second block there were no significant main effects, nor interactions (all F<1.79, p <.217, η 2 p <.264). 36

39 Figure 7. Results of auto recurrence analysis on average key press velocities: %REC of the average velocity of the musician s key presses (top row) and MaxLine (bottom row). With regards to the recurrence analysis of key-press velocities, overall the results indicated that musicians repeated their own key press velocities more often when playing with the drone backing track. With respect to the swing track, the results were also consistent with those observed in the above analysis of notes played, with musicians repeating their own key press velocities significantly less when improvising to the swing backing track compared to the ostinato and drone backing tracks. Recurrence Analysis of Key Press Timing (On/Off). In order to capture a more general measure of coordination in the musician s playing behavior, a time series was created for each player that represented the timing of their key presses. If a key was pressed at a given time point, that time point was given the value of 1. If a key wasn t being played it was assigned a random number so that it would not be quantified as a recurrent point. Therefore recurrence in this case demonstrates more broadly when the musicians were pressing keys in the same pattern, regardless of which key or the key press velocity. This analysis also did not take into consideration how many keys were being pressed, only the combined timing. The results of the recurrence analysis on the key press behavior are displayed in Figure 7. For %REC there was a significant main effect of backing track F(2,10) = 6.49, p =.016, η 2 p =.565, a significant two-way interaction between block and condition F(1,5) = 5.56, p =.065, η 2 p = 37

40 .526, as well as a three-way interaction between block, backing track and condition F(2,10) = 7.15, p =.012, η 2 p =.588. There were no other significant main effects, nor interactions (both F<1.60, p >.262, η 2 p <.242). Separate 2 x 3 ANOVAS where then conducted to unpack this three way interaction. For the first block there was not a main effect of condition, but a significant main effect of backing track F(2,10) = 3.31, p =.079, η 2 p =.398, and a significant interaction between condition and back track F(2,10) = 6.08, p =.019, η 2 p =.549. Follow-up analyses revealed that for the drone track there was a significant effect of condition F(1,5) = 6.05, p =.057, η p2 =.547, but no significant effect of condition for the swing or ostinato backing tracks (both F<.714, p >.437, η 2 p <.125). A simple effects analysis of backing track for the no-vision condition was also significant F(2,10) = 5.11, p =.030, η 2 p =.505, with the magnitude of recurrence for the swing backing track being significant lower than that observed for the drone (p =.016) or ostinato backing track (p =.094). That is, in the first block of trials musicians repeated the patterns of their key presses more less often for the swing backing track when they couldn t see each other. There was no effect of backing track for the vision condition (F=2.37, p =.143, η 2 p =.322). For the second block of trials there was a significant main effect of backing track, F(2,10) = 4.56, p =.039, η 2 p =.477, where for the swing track there was a significantly lower recurrence than for the ostinato (p =.085) or drone track (p =.044). That is, musicians were repeated the patterns of their key presses less often for the swing track in the second block of trials. There was no main effect of condition or interaction effects (all F<.510, p <.640, η 2 p <.047). The analysis of MaxLine resulted in a significant main effect of backing track, F(2,10) = 5.42, p =.025, η p2 =.520, with MaxLine for the swing track being significantly lower than for the ostinato (p =.057) or drone backing tracks (p =.005). In short, musicians were repeating patterns 38

41 of their key presses in shorter sequences when playing with the swing track across both blocks of trials.there was no main effect of block, condition or interaction effects (all F<1.31, p <.303, η p 2 <.208). Figure 8. Results of auto recurrence analysis of key press timing: %REC of the patterns of musician s key presses, (top row) and MaxLine (bottom row). Recall that RQA provides a way of identifying the way in which musicians create intrapersonal dynamic stabilities during improvised musical play (i.e., the degree to which there are repeating patterns in their own playing behavior). It therefore captures the degree to which the joint performance shapes an individual musician s own playing dynamics. In summary, the results of the various RQA analysis reveals that the musicians playing behavior when improvising with the swing track not only differs in the keys pressed, and the key press velocity, but overall timing compared to playing behavior observed for the drone and ostinato backing tracks. This implies that the musical context of the swing-backing track therefore affords very different playing behavior compared to the ostinato and drone backing tracks. One might 39

42 intuitively hypothesize that the drone backing track would have allowed for more variability in playing because there is no rhythmic structure and very little harmonic information. However, the current data clearly suggest otherwise, with the drone backing track necessitating that the musicians repeated themselves more to create and maintain a musical structure with which to coordinate. With regards to the ostinato backing track the results were similar to those observed for the drone track. Thus, although the ostinato backing tracking provided the musicians with a harmonic and rhythmic structure to improvise along to (i.,e., the musicians didn t have to create or maintain the harmonic or rhythmic structure), this harmonic and rhythmic structure didn t afford the same opportunities as the swing track for exploring different or complementary modes of playing behavior. Cross Recurrence Results While RQA helps identify the way musicians create dynamic stabilities by repeating patterns in their own playing behavior, cross recurrence analysis helps capture how they repeat aspects of each other s playing behavior. In other words, CRQA captures the degree to which the playing behavior of the two musicians was similar and/or synchronized over time. Accordingly, the keys pressed, key press velocity, and key press timing data event-series were all submitted to CRQA in order to determine that presence and magnitude of such inter-musician coordination. Cross Recurrence Analysis on Keys/Combination of Keys Pressed. As stated above, the MIDI output was used to create a time series for each player that captured what keys and/or combinations of keys they pressed across the time span of the performance. The key difference in the CRQA as opposed to the RQA is that both players MIDI output was used to determine the set of unique keys and combination of keys pressed. Then each of these unique combinations of keys pressed by both players was assigned to a code number. Figure 8 illustrates how the time 40

43 series containing the code numbers for unique keys and combination of keys pressed by each musician at each time point in the improvisation are mapped onto one another using a cross recurrence plot. Figure 9. The MIDI output was used to create a time series where the keys and combinations of keys pressed (B) were used to create a code number for each unique combination of notes played (C) and then a time series for each player was created using these code numbers as well random numbers when no notes were being played (D and E). This allowed for the quantification of when the musicians were playing the same thing at the same time (i.e. code number 718), as well as how they repeat each other at different delays (i.e. code number 607) through a cross recurrence plot (right). The results of the cross recurrence analysis for %REC of the keys pressed are displayed in Figure 9. For Real/Virtual Omnibus that includes pair type as a factor, there was a significant main effect of pair type F(1,5) = 27.47, p =.003, η 2 p =.846, such that keys and combinations of keys pressed were repeated more among the real pairs than in the virtual pairs. There were no significant interaction effects with pair type (all F<3.09, p>.139, η 2 p <.382). For the real pairs analysis there were no significant differences for %REC (all F<1.64, p >.256, η 2 p <.247). 41

44 Figure 10. Results of cross recurrence analysis on keys/combinations of keys pressed: %REC of the keys and combinations of keys pressed for the real pairs (top row) and virtual pairs (bottom row). The results for MaxLine are displayed in Figure 10. For the virtual pair analysis there was a significant main effect of pair type F(1,5) = 27.47, p =.003, η 2 p =.846, such that longest sequence of keys and combinations of keys pressed was significant longer for the real pairs than for the virtual pairs. There were no significant interaction effects with pair type (all F<2.13, p>.205, η 2 p <.298). For the real pairs analysis there were no significant differences for MaxLine (all F<1.43, p >.285, η p2 <.222). 42

45 Figure 11. Results of cross recurrence analysis on keys/combinations of keys pressed: MaxLine of the keys and combinations of keys pressed for the real pairs (top row) and virtual pairs (bottom row). In summary, the results of the virtual pairs analysis revealed that the coordination in playing behavior that emerges between two interacting musicians, is above and beyond coordinating musical structures with the backing track and is unaffected by block or vision condition. Cross Recurrence Analysis on Average Velocity. Similar to the RQA analysis, a time series was created for each musician with the average velocity of the keys pressed at each time point (see Figure 7). Given the scale of 1 to 127, it needed to be determined what level of precision is necessary to capture the dynamics of the key press velocity. For example, should a recurrent point only represent when the average velocity of key presses between the two players is exactly the same value, or if it is within a certain range. To determine the appropriate level of precision CRQA was run separately with the radius values 1-10, and the changes in significance 43

46 values for the main effect of backing track for %REC and MaxLine were observed. The p values for the main effect of backing track for %REC were stable across radius values, for MaxLine the values fluctuated randomly as the radius was increased. A radius of ten was chosen, such that if the average key press velocities of each player were within ten values of each other. The results of the cross recurrence analysis for %REC of the key press velocities are displayed in Figure 11. For the Real/Virtual Omnibus there was no main effect of pair type, no interaction effects with pair type (all F<2.43, p>.180, η p2 <.327). As can be seen from inspection of Figure 11, the magnitude of %REC for the virtual pairs mirrored the %REC observed for the real pairs, suggesting that the coordination key press velocities are driven by the players coordination with the backing track. For the real pairs analysis their was a significant main effect of backing track F(2,10) = 5.82, p =.021, η 2 p =.538, such that musicians were pressing keys with the same velocity significantly less when playing with the swing backing track than the ostinato (p =.045) or drone backing track (p =.040). There was also a main effect of block F(1,5) = 4.09, p =.099, η 2 p =.450, where the musician s key press velocities were more similar in the second block than in the first block of trials. There was no main effect of condition or interaction effects (all F<2.46, p <.135, η 2 p <.330). Thus the musicians repeated each other s key press velocities significantly less for the swing track, which is consistent with the auto recurrence results, and they repeated each other s key press velocities more in the second block of trials. This could be due to the fact that they had better developed the ability to coordinate with the backing track and each other by the later trials. 44

47 Figure 12. Results of cross recurrence analysis on average key press velocities: %REC of the average velocity of the musician s key presses for the real pairs (top row) and virtual pairs (bottom row). The results of the cross recurrence analysis for MaxLine of the key press velocities are displayed in Figure 12. For the Real/Virtual Omnibus ANOVA there was a significant interaction between pair type and condition F(1,5) = 5.58, p =.065, η 2 p =.527, with no main effect of pair type or other significant interactions with pair type (all F<2.71, p>.115, η p2 <.352). When split by condition there was a significant simple effect of pair type for the no vision condition F(1,5) = 4.94, p =.077, η 2 p =.497, such that the real pairs repeated longer sequences of key press velocities than the virtual pairs (all interactions with pair type F<2.44, p>.137, η 2 p <.328). For the vision condition there were no main effects or interactions with pair type (all F<.371, p>.569, η 2 p <.069). For the planned real pairs analysis there was a significant main effect of condition F(1,5) = 5.14, p =.073, η p 2 =.471, such that when the musicians could see each other the patterns of key 45

48 press velocities were less similar than when they couldn t see each other. There was also a significant main effect of backing track F(2,10) = 5.71, p =.022, η 2 p =.533, such that the patterns of the musicians keys press velocities were less similar when playing with the swing track than the ostinato (p =.062) or drone track (p =.002). The effect of vision indicates that when musicians have visual information about their co-performer they do not repeat each other s key press velocities as much because they are playing things that are more complimentary. Whereas in the no vision condition they may play more similar things because they don t have as much information to anticipate what their co-performer is doing, and playing something different. Figure 13. Results of cross recurrence analysis on average key press velocities: MaxLine of the average velocity of the musician s key presses for the real pairs (top row) and virtual pairs (bottom row). In summary the cross recurrence analysis on the average key press velocities reveal similar patterns as the auto recurrence results in that there is significantly less repeating of playing behaviors when playing with the swing backing track. When comparing cross recurrence 46

49 on the key press velocities to the cross recurrence on the keys pressed, it is interesting to note how for the keys pressed there were significant differences between the real and virtual pairs, but in most conditions for the key press velocity, there was not. This means that the interaction that occurs between real pairs may be better captured by analyzing the harmonic aspects of their playing (what keys, notes, chords), as opposed to something like key press velocity which is driven mostly by the backing track. Cross Recurrence Analysis of Key Press Timing (On/Off). As with the RQA, a time series was created for each player such that if a key was pressed at a given time point, that time point was given the value of 1, so recurrence captures when the musicians were pressing keys in the same pattern, regardless of note or key press velocity. The results of the cross recurrence analysis for %REC of the key press timing are displayed in Figure 13. For the Real/Virtual Omnibus there was no significant main effect of pair type or interaction effects with pair type (all F<.943, p>.376, η p2 <.159). For the planned analysis of real pairs there was a three way interaction between block, condition and backing track F(2,10) = 6.32, p =.017, η p2 =.558. The analysis was then split into two 2 (condition) x 3 (backing track) ANOVAs for each separate block. In the first block of trials there was a significant interaction between condition and backing track, F(2,10) = 5.97, p =.020, η 2 p =.544. A simple effects analysis revealed that there was a main effect of backing track for the vision condition (F=.145, p =.719, η 2 p =.028), where %REC was significantly higher for the drone backing track than the ostinato (p=.029) and the swing backing track (p=.008). Thus the patterns of the musicians key presses in the first block were more similar when playing with the drone backing track, when they couldn t see each other. There was no effect of backing track for the no vision condition, F(2,10) = 10.03, p =.004, η 2 p =.667. For the second block of trials there was no 47

50 interaction effect, only a main effect of backing track F(2,10) =.007, p =.021, η 2 p =.538, with %REC significantly lower for the swing track than for the ostinato (p=.090) or drone backing track (p=.024). Thus for the second block of trials the musicians patterns of key presses were less similar when playing with the swing track, regardless of whether they could see each other. Figure 14. Results of cross recurrence analysis of on/off key press timing: %REC of the patterns of musician s key presses for the real pairs (top row) and virtual pairs (bottom row). The results of the cross recurrence analysis for MaxLine of the key press timing are displayed in Figure 14. For the Real/Virtual Omnibus there was no main effect of pair type, but a significant interaction between pair type, block and backing track F(2,10) = 3.12, p =.088, η 2 p =.384. Separate, 2 x 3 repeated measures ANOVAs for each block, revealed no significant main effects or interactions with pair type in the first block (all F<3.27, p <.130, η 2 p <.396), but a significant interaction between pair type and backing track for the second block of trials, F(2,10) = 3.09, p =.090, η 2 p =.382. However, follow-up analysis revealed that the only backing track for which there was a significant main effect of pair type was the ostinato backing track F(1,5) = 48

51 12.47, p =.017, η p 2 =.714, such that MaxLine was significantly lower for the real pairs as compared to the virtual pairs. For the planned real pairs analysis there was a significant main effect of backing track F(2,10) = 9.16, p =.005, η 2 p =.647, with the longest sequences of recurrent key press patterns being significant lower for the swing backing track compared to the ostinato (p =.003) and drone backing tracks (p =.001). There was no main effect of block or condition, nor an interaction effect (all F<2.69, p <.162, η 2 p <.350). In summary, like the differences in key press velocities, the differences in key press timing may also be attributed to the constraints of the backing track, given the similarities between the real and virtual pairs analyses. The results are also consistent with respect to the swing backing track, where musicians are repeating themselves less, and each other less, with respect to key press velocity and key press timing. Figure 15. Results of cross recurrence analysis of on/off key press timing: MaxLine of the patterns of musician s key presses for the real pairs (top row) and virtual pairs (bottom row). 49

52 MIDI Analyses Section Summary The results of the categorical recurrence quantification analysis of how often musicians repeated their own note structures, key press velocities, and patterns of key pressing timing are summarized in Table 1. It can be observed that overall both the harmonic and rhythmic qualities of their playing behavior were less variable when improvising with the drone backing track, and more variable when improvising with the swing backing track. As mentioned above, the familiar rhythmic foundation of the swing backing provided an opportunity for the musicians to more freely explore a wider variety of playing behavior with respect to what keys they pressed as well as how they pressed them. Where as the lacking of structure for the drone may have demanded that they repeated certain rhythmic aspects of their playing behavior in order to provide the grounding for improvisation play. Table 1. Results of recurrence quantification of musician s MIDI data with respect to what keys they pressed, key press velocity, and key press timing. The results of the cross recurrence quantification analysis of how often musicians repeated their own note structures, key press velocities, and patterns of key pressing timing are summarized in Table 2. It can be observed how for key press velocities and key press timing 50

53 there are similar patterns as the RQA results with respect to the swing and drone backing tracks. In particular musicians repeated. Finally, while the results for the rhythmic qualities of their playing behavior are similar between CRQA and RQA, the dynamics of how they repeat their own note structures as compared to each other s note structures is different. That is, for CRQA while there were no significant differences in how the real pairs repeated each other s note structures across backing track, block, or vision condition, the real pairs did repeat each other s note structures significantly more than the virtual pairs for all the experimental conditions. Table 2. Results of cross recurrence quantification of musician s MIDI data with respect to what keys they pressed, key press velocity, and key press timing. Also included are the results for the virtual pairs analysis. 51

54 CHAPTER 4 Principal Component Analysis Results & Discussion PCA was used to determine how many principal components were required to capture 80% of the variance in the original dataset, as well as how much variance is accounted for by the first and second principal components. Changes in these measures were used to indicate changes in the mutual co-adaptation and synergistic coupling across the different task conditions (backing, condition, and block). PCA was also used to quantify the synergistic coupling among the movements of the musicians, where changes in dimensional compression as indicated by the number of principal components needed to account for the 80% variability, was evaluated for both the musicians individually, and as a pair, as illustrated in Figure 15. These various PCA analyses are presented below in separate sections. 52

55 Figure 16. For the Individual PCA (left) the analysis is run for each musician separately, and the number of principal components and percent variance accounted for by each component are averaged across the musicians in each pair for each experimental condition. For the Pairs PCA (right) the movements from the musicians in each pair are analyzed together for each experimental condition. PCA on Individual Players (Intrapersonal). The x, y and z position recorded by the motion sensors attached to the musicians right forearm, left forearm, and forehead were used evaluate how many principal components were needed to account for 80% of the variance in the musicians movements individually. Two separate PCAs were run for each musician, which consisted of 9 degrees of freedom for each individual player: 3 time series (the x, y and z position) for the 3 body parts (head, and each of the arms). The number of principal components that accounted for 80% of the total variance, percentage of variance accounted for by the first principal component, and the percentage of variance accounted for by the second principal component were averaged across the two musicians within each pair to obtain one set of intrapersonal PCA measurements for each pair. The results of the PCA on the musician s movements are displayed in Figure 16. For the number of principal components, there was a main effect of backing track F(2,10) = 4.65, p =.037, η 2 p =.482, whereby the swing backing track needed significantly more principal components to account for 80% of the variance in the data compared to the ostinato (p=.064) or drone backing track (p=.013). Thus, there was significantly less dimensional compression in the musician s intrapersonal movements when they were improvising over the swing as compared to the other two backing tracks. There was no main effect of condition or interaction effects (all F<2.55, p <.128, η 2 p <.388). 53

56 For percent variance accounted for by the first principal component there were no main effects, but a significant interaction between block and condition F(1,5) = 12.30, p =.017, η 2 p =.711, as well as between block and backing track F(2,10) = 3.00, p =.095, η 2 p =.375. When split into two separate 2 (block) x 3 (backing track) ANOVAs for each condition, there was a significant main effect of block for the no vision condition F(1,5) = 6.03, p =.058, η 2 p =.547, such that when musicians couldn t see each other significantly more variance was accounted for by the first principal component in the second block of trials. For the vision condition there was a significant interaction between block and backing track F(2,10) = 6.36, p =.016, η 2 p =.560. Simple effects analyses revealed that when the musicians could see each other for the ostinato backing track significantly more variance was accounted for by the first principal component in the first block F(1,5) = 7.34, p =.042, η p2 =.595. No effects were observed for swing backing track F(1,5) =.022, p =.888, η p2 =.004, but for the drone less variance was accounted by the first principal component in the first block, F(1,5) = 7.88, p =.038, η 2 p =.612. Figure 17 displays the results for percentage of variance accounted for by the first principal component with separate graphs for the vision and no vision condition, so these interaction effects are more easily observed. Of particular interest, is that the variability in the musician s movements that is captured by the first principal component does not change as a function of the backing track when the musicians can t see each other, but increases as trials progress (a higher percentage of variance is accounted for in the second block). In contrast, the way the first principal component captures the variability when the musicians can see each other is affected by which backing track they are playing along too, with the first principal component capturing more variability in the first block for the ostinato, but less for the drone backing track. For percent variance accounted for by the second principal component there were no significant main effects or interactions (all F<2.32, p <.188, η p2 <.318). 54

57 Figure 17. Results of PCA analysis on individual musicians movements: Number of principle components (top row), percentage of variance accounted for by the first principal component (middle row), percentage of variance accounted for by the second principal component (bottom row). 55

58 Figure 18. Results of PCA analysis on individual musicians movements: specifically the percentage of variance accounted for by the first principal component in the no vision condition (left) and vision condition (right). PCA on Players together (Interpersonal). The x, y and z position recorded by the motion sensors attached to the musicians right forearm, left forearm, and forehead were used evaluate how many principal components were needed to account for 80% of the variance in the musicians movements. PCA was run on the musicians as a pair, which consisted of 18 degrees of freedom: 3 time series (the x, y and z position) for the three body parts (head, and each of the arms) of both musicians. The same virtual pairs analysis employed for the MIDI and cross wavelet analyses present in the previous chapter was used for this analysis, where x, y and z position of the head, left arm and right arm of a player was matched up with the x, y and z positions of players from other pairs and submitted to the same PCA analysis. This was meant to capture the covariance in movement you would see as a function of musicians playing with the same backing track, in the same experimental condition. The results of the PCA for the number of components needed to account for 80% of the variance of the musicians head, left arm, and right arm movements are displayed in Figure 18. For the Real/Virtual Omnibus there was a significant main effect of pair type F(1,5) = 26.55, p =.004, η 2 p =.842, but also a significant three-way interaction between pair type, block and backing track F(2,10) = 5.09, p =.030, η 2 p =.505. The analysis was then split into two separate 2 x 3 ANOVAs for each block, where there were no significant interactions with pair type, but a main effect of pair type for both blocks (both F>18.33, p <.008, η 2 p >.786), such that the real pairs of musicians needed significantly more components to capture 80% of the variance than for the virtual pairs. 56

59 For the real pairs analysis, there was a significant interaction between block and backing track F(2,10) = 4.46, p =.041, η 2 p =.471. Follow-up analysis revealed that for the first block there was no effect of vision, nor an interaction between vision and backing track (both F<.217, p <.661, η 2 p <.042). However, there was a significant main effect of backing track F(2,10) = 6.03, p =.019, η 2 p =.548, with more principal components needed to account for eighty percent of the variance for the swing compared to the ostinato (p =.015) and drone backing track (p =.057). Thus, in the first block there was less dimensional compression at the pair level when the musicians were improvising with the swing compared to the other two backing tracks. In the second block there were no significant main effects, nor interactions (all F<.607, p <.564, η 2 p <.108). Figure 19. Number of principal components needed to account for 80% of variance in the players head, left arm, and right arm movements. The top row displays the number of principal components needed for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the number of principal components needed for the virtual pairs. 57

60 In summary, the results for the number of principal components needed to account for 80% of the variance in the pairs of musicians movements are consistent with the analysis of their individual movements, in that there was significantly less dimensional compression when playing with the swing track. However, the virtual pairs analysis also indicates how the dimensional compression exhibited between the real pairs, may largely reflective the constraints of the backing track alone, rather than due to the playing with a co-musician. The results of the PCA for the amount of the variance account for by the first principal component are displayed in Figure 19. For the Real/Virtual Omnibus there was a significant main effect of pair type F(1,5) = 4.46, p =.015, η p2 =.727, but also a three-way interaction between pair type, condition and backing track F(2,10) = 5.49, p =.025, η 2 p =.523, so the analysis was broken up into two separate 2 (block) x 3 (backing track) ANOVAs for each condition. For the vision condition there was a significant main effect of pair type F(1,5) = 5.20, p =.072, η 2 p =.510, with the first principal component accounting for more variance for the virtual pairs as compared to the real pairs. There were no other significant interactions with pair type (all F<1.14, p >.186, η 2 p <.310). For the no vision condition there was also a main effect of pair type F(1,5) = 50.57, p =.001, η p2 =.910, as well as an interaction between pair type and block F(2,10) = 3.39, p =.075, η 2 p =.404. A subsequent set of 2 (pair type) x 2 (block) ANOVAs for each backing track was then conducted. For the ostinato backing track there was only a significant main effect of pair type F(1,5) = , p =.000, η 2 p =.965, with the first principal component accounting for more of the variance for the virtual pairs compared to the real pairs. The same was true for the swing backing track, where there was a main effect of pair type F(1,5) = 4.59, p =.085, η 2 p =.479 and no interaction effects. For the drone backing track, however, there were no significant main effects, and no interaction effect (all F<3.02, p >.134, η p2 <.390). Thus, for all playing conditions except for the when playing with the drone in the no vision condition, 58

61 the first principal component captured significantly more variance in the movements of the virtual pairs compared to the movements of the real pairs. Finally, the planned real pairs analysis of the percent variance accounted for by the first principal component revealed no main effects, nor any interactions (all F<2.25, p >.157, η p 2 <.310). Collectively then, the latter results reveal that while there were no significant differences among the real pairs for the percentage of variance accounted for by the first component, the percentage of variance accounted for in the musician s movements was significantly higher for the virtual pairs. This is consistent with the results for the number of components needed to accounted for 80% of the variance, in that for the virtual pairs their movements exhibit higher co-variation attributable to the structure of the backing track. Figure 20. The amount of variance in the players head, left arm, and right arm movements accounted for by the first principal component. The top row displays the amount of variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the amount of variance for the virtual pairs. 59

62 The results of the PCA for the amount of the variance account for by the second principal component are displayed in Figure 20. For the Real/Virtual Omnibus there was a significant main effect of pair type F(1,5) = 18.56, p =.008, η 2 p =.788, but also a significant interaction between pair type and backing track F(2,10) = 7.93, p =.009, η 2 p =.613. Follow-up analyses for each backing track were then conducted. For the ostinato backing track there was no significant main effect of pair type (all F<.934, p <.344, η 2 p <.039). For the swing backing track there was a significant main effect of pair type F(1,23) = 43.94, p =.000, η 2 p =.656, such that for the virtual pairs the second principal component accounted for significantly more variance than for the real pairs. The second principal component also accounted for more variance in the virtual pairs for the drone backing track, where there was also a significant main effect of pair type F(1,23) = 6.06, p =.022, η p2 =.208. Thus, for the swing and drone backing tracks the second principal component accounted for significantly more variance in the virtual pairs movements than for the real pairs. With regards to the planned real pairs analysis of the percent variance accounted for by the second principal component there was only a main effect of backing track F(2,10) = 23.80, p =.000, η p2 =.826 (all other main and interaction effects F<1.83, p <.234, η p2 <.268). Post hoc analysis revealed that for swing backing track there was significantly less variance accounted for by the second component compared to the ostinato (p=.001) and the drone backing tracks (p=.010). There was also significantly more variance accounted for by the second principal component for the ostinato compared to the drone backing track (p=.046). 60

63 Figure 21. The amount of variance in the players head, left arm, and right arm movements accounted for by the second principal component. The top row displays the amount of variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the amount of variance for the virtual pairs. Together, the results of the 2 nd principal component further explicate the differences in the number of principal components needed to account for 80% of the variance when the musicians play with the swing backing track. When playing with the swing backing track more components were needed to account for the variability in the musician s movements, and while there were no significant differences in the variation captured by the first component, less variation is captured by the second principal component. Thus for the swing backing track the variability in the musicians movements must be more distributed across the remaining principal components (i.e. third, fourth fifth) than for the trials when the musicians are playing with the other backing tracks (see Figure 21). As mentioned above, however, when considering the results of the virtual pairs analysis this seems to be attributable to the co-variance between movements that would occur by virtue of 61

64 playing with the same backing track. The movements of the virtual pairs demonstrate higher covariation than the real pairs given that they needed less principal components to account for eighty percent of the variance, and often less variance was accounted for by the first and second principal component. This might be because the real pairs are engaging in playing behavior complimentary to their co-performer and as a result introduce more variability into their movements above what is dictated by playing with the same backing track. Figure 22. Results of PCA analysis on musicians movements as pairs: Number of principle components (top row), percentage of variance accounted for by the first component (middle row), percentage of variance accounted for by the second component (bottom row). Dimensional Compression. Principal component analysis of the musician s movements as pairs provides an (indirect) index of how coupled the musicians movements are during the performances the greater the dimensional compression, the more synergistically coupled. Similarly, at the intrapersonal level the greater the dimensional compression, the more synergistically coupled each musicians intra-limb and intra-body movements were. Thus, comparing the magnitude of intra-personal and inter-personal dimensional compression provides a measure of whether the musical outcome was more or less dominated by intra- vs. interpersonal synergistic processes. In other words, a comparison of the dimensional compression observed at the intra- and interpersonal levels enables ones to determine the degree to which the 62

65 players formed an interpersonal synergy, rather than simple two coordinated intra-personal synergies (Romero et al., 2015; Ramenzoni et al.,2011; Riley, et al., 2011). The magnitude of dimensional compression for the pair (interpersonal) and individual (intrapersonal) conditions was evaluated by taking the number of degrees of freedom (9 for the individual, 18 for the pair) and dividing it by the number of components needed to account for 80% of the variance for each experimental condition. This provided a magnitude ratio of dimensional compression such that if there were more principal components needed to account for the variability in the musician s movements, the resulting ratio would be smaller. Conversely, if there were fewer principal components needed the resulting ratio of dimensional compression would be larger. For example if four principal components were needed to account for 80 percent of the variance in a pair system with 18 degrees of freedom, this would be equal to the magnitude of dimensional compression in a intra-personal system with 9 degrees of freedom for which two principal components are needed to account for 80% of the variance (18/4 = 9/2). Thus this ratio allows for the magnitude of dimensional compression to be directly compared across the intrapersonal (individual) and interpersonal (pair) PCA results. The dimensional compression ratios are shown in Figure 22 and were submitted to a three-way mixed measures ANOVA with block, condition and backing track as within-subjects factors and coupling type (intra vs. interpersonal) as a between-subjects factor. This analysis revealed a significant effect of coupling type F(1,10) = 5.57, p =.04, η 2 p =.358, with significantly more dimensional compression at the interpersonal level compared to the intrapersonal level. However, there was a significant four-way interaction between block, condition, backing track and coupling type F(2,20) = 2.86, p =.081, η 2 p =.223, so the analysis was split into two separate 2 x 3 ANOVAs for each block. For the first block there was still a 63

66 significant effect of coupling type F(1,10) = 4.16, p =.069, η p 2 =.294, as well as for the second block F(1,10) = 4.44, p =.061, η p 2 =.308, where in both cases the dimensionality of the interpersonal coupling was significantly greater than the intra-personal coupling. A 2 x 2 x 3 repeated measures ANOVA was then used to evaluate how the interpersonal coupling (18df) changed as a function of the experimental conditions. There were no significant main effects, nor interaction effects (all F<2.57, p <.126, η p 2 <.339). The dimensional compression that results when performing principal component analysis on the musician s movements as pairs (18 df), instead of individuals (9 df), seems to support the hypothesis that when two musicians improvise music their movement dynamics form an interpersonal synergy. Figure 23. Results of a three-way mixed measures ANOVA with block, condition and backing track as within subject factors, and type of coupling (intra-personal vs. inter-personal) as a between subjects factor. Shown in blue is the magnitude of dimensional compression from the PCA analysis of the pairs, or inter-personal condition. The dimensional compression is greater than for the intrapersonal PCA of the musicians individually, shown in purple. Principal Component Analysis Section Summary The results of the principal component analysis of musician s individual movements (9 df) are summarized in Table 3. Of particular importance was finding that significantly more 64

67 components were needed to account for 80% of the variance of the musician s movements when improvising with the swing backing track compared to the ostinato and drone backing tracks. This findings was consistent with the RQA and CRQA results for the key press playing detailed in the previous chapter and seems to individuate the musicians overall behavioral movements was more variable when improvising along to the swing backing track. Table 3. Results of principal component analysis of musician s individual movements (9 df). The results of the principal component analysis of musician s individual movements (18 df) are summarized in Table 4. Again, when improvising with the swing back track there seems to be less dimensional compression given that more principal components are needed to account for the variance in their movements. Moreover, for the swing track there was significantly less variance accounted for by the second principal component accounts for the real pairs compared to the virtual pairs, which further suggests that the pairs explored a greater range of musical improvisation when playing along to the swing backing track. Finally, the fact that number of principal components needed to capture 80% of the variance was significantly greater for real pairs compared to the virtual pairs in general (particular for the swing and drone tracks) seems to 65

68 indicate the PCA analysis maybe tapping into the magnitude of complementary coordination exhibited by musicians during real time improvisation. That is, the current results may indicate that there is an inverse relationship between dimensional compression and complementary coordination (and possibly reciprocal compensation; see general discussion section) such that less dimensional compression is related to increased exploratory musical play and musically related movement variability. Table 4. Results of principal component analysis of musicians movements as pairs (18 df). 66

69 CHAPTER 5 Cross Wavelet Results & Discussion Cross wavelet spectral analysis was employed to assess coordination between the first principal component times series for each musician s individual limbs. The variability in relative phase as captured by the inverse of circular variance was measured for the musician s heads, left arms and right arms every one second, four seconds, eight seconds, and sixteen seconds. The coordination at each of these intervals between the full body movements of the musicians was also explored by calculated the inverse of circular variance for both the first and second component time series when all the limbs for each player are submitted together to a principal competent analysis (9 degrees of freedom). Head Movement Circular Variance of Head Movement every one second. The results of the cross wavelet analysis for both the Real/Virtual Omnibus and the planned real pairs analysis of the musicians head movements at the.5 to 1.5 second frequency band are shown in Figure 23. There was a significant main effect of pair type F(1,5) = 17.61, p =.009, η 2 p =.779, with significantly less variable relative phase relationships observed between the real pairs than virtual pairs. There were also three-way interactions between pair type, condition, and backing track F(2,10) = 3.19, p =.085, η 2 p =.390, and pair type, block, and condition F(1,5) = 24.47, p =.004, η 2 p =.830, as well as a two-way interaction between pair type and backing track F(2,10) = 5.12, p =.024, η 2 p =.524. To further unpack these results, separate 2 (pair type) x 2 (condition) x 3 (backing track) ANOVAs for each block. 67

70 For the first block there was a significant interaction between pair type and condition F(1,5) = 4.40, p=.090, η 2 p =.468, and no main effect or other significant interactions with pair type (all F<2.05, p>.180, η 2 p <.290). Follow-up analyses revealed that for the no vision condition there was no main effect nor interaction with pair type (both F<.422, p>.667, η 2 p <.078), but for the vision condition there was both a main effect of pair type F(1,5) = 7.21, p=.044, η 2 p =.423, and an interaction with pair type and backing track F(2,10) = 7.71, p=.009, η 2 p =.607. When split into three separate one-way ANOVAs for each backing track in the vision condition, there was no significant differences between the real and virtual pairs for the ostinato and drone (both F<1.15, p>.332, η p2 <.187). There was a significant main effect of pair type for the swing backing track F(1,5) = 11.69, p=.019, η 2 p =.700, however, such that for the real pairs demonstrated less variable relative phase relationships in their head movements than the virtual pairs in the first block.. For the second block there was a main effect of pair type F(1,5) = 5.61, p=.064, η 2 p =.529, a significant interaction between pair type and condition F(1,5) = 21.04, p=.006, η 2 p =.808, and a significant interaction between pair type and backing track F(2,10) = 3.31, p=.079, η 2 p =.399. When split into two separate ANOVAs for each condition, for the no vision condition there was not an interaction with backing track but a significant main effect of pair type F(1,5) = 14.65, p=.012, η 2 p =.746, such that head movements of the real pairs demonstrated les variable relative phase relationships than for the virtual pairs. For the vision condition there was no main effect nor significant interaction effect with pair type (both F<2.57, p>.126, η 2 p <.339). For the real pairs analysis there was a significant main effect of backing track F(2,10) = 7.20, p =.012, η p 2 =.590; the musicians head movements were more coordinated when playing with swing track than the ostinato (p =.051) or drone track (p =.013). There was also a 68

71 significant interaction between block and condition F(2,10) = 24.48, p =.004, η 2 p =.830. Simple effects analyses revealed a main effect of condition in the first block F(1,5) = 6.64, p =.05, η 2 p =.571, where the musicians head movements were significantly more coordinated when they could see each other. In the second block there was also a main effect of condition F(1,5) = 8.29, p =.035, η 2 p =.624, but in the second block their head movements were more coordinated when they could not see each other. Figure 24. Inverse circular variance between the musicians head movements for the.5 to 1.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. In summary, head movement coordination seems to play a role at the faster time scale (1- second frequency), and was exploited more when playing with the swing backing track, possibly because the rhythmic quality or groove or this swing style of music. The virtual pairs analysis revealed that this coordination in the swing backing track is beyond how players might move their heads just by virtue of playing with the same backing track alone or with a different player. This is consistent with the observed importance of head movement in jazz performance 69

72 (Sudnow, 1970) and may be related to the musical groove (Janata et al., 2012). The musicians could be exploiting visual information about their partners head movements in the first block of trials to establish coordination. Once this is established visual information is not exploited in the later trials, thus their head movements are then equally coordinated in both the vision and no vision condition in the second block. Circular Variance of Head Movement every 4 seconds. The results for the 3.5 to 4.5 second frequency band are shown in Figure 24. For the Real/Virtual Omnibus there was a threeway interaction between pair type, condition and backing track, F(2,10) = 7.49, p =.010, η p2 =.600; with no significant main effect of pair type or other interactions with pair type (all F<2.72, p>.160, η 2 p <.352). When broken up into two separate 2 (pair type) x 2 (block) x 3 (backing track) ANOVAs for each condition, the analysis revealed that for the no vision condition there were no main effects of pair type, nor any interaction effects (all F<2.62, p>.122, η 2 p <.344). For the vision condition there was also not a significant main effect of pair type (F=1.21, p=.322, η 2 p =.195), but there was a significant interaction between pair type and block F(1,5) = 4.76, p =.081, η 2 p =.488, and pair type and backing track F(2,10) = 4.65, p =.037, η 2 p =.482. The analysis was then split into two separate ANOVAs for each block. For the first block there was no significant main effect of pair type but a significant interaction between pair type and backing track F(2,10) = 3.11, p =.089, η 2 p =.384. Simple effects analyses revealed no significant effects of pair type for any of the backing tracks. For the second block there was a significant main effect of pair type F(1,5) = 13.64, p =.014, η 2 p =.732, such that there was significantly less variable relative phase relationships between the head movements of the virtual pairs than the real pairs in the second block of trials. 70

73 For the planned real pairs analysis there were no significant main effects, but a significant interaction between condition and backing track F(2,10) = 4.23, p =.047, η 2 p =.458. Simple effects analysis revealed that for the no vision condition there were no significant effects or interactions (all F<2.62, p>.121, η 2 p <.344). For the vision condition there was a significant main effect of block F(1,5) = 5.95, p =.059, η 2 p =.543, such that the musicians head movements were more coordinated when they could see each other, in the first block. Figure 25. Inverse circular variance between the musicians head movements for the 3.5 to 4.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. In summary the musicians head movements for the 4-second frequency in the first block were significantly more coordinated when they could see each other, mirroring the results for the 1-second frequency. Yet in this case the variability in the phase relationships between the real pairs was not significantly different than for the virtual pairs, so this head movement 71

74 coordination may be attributable to the constraints introduced by just performing with the same backing track. Circular Variance of Head Movement every 8 seconds. The results for the 7.5 to 8.5 second frequency band are shown in Figure 25. For the Real/Virtual Omnibus there was no main effect of pair type or interaction effects with pair type (all F<2.90, p>.102, η 2 p <.367), thus only the planned real pairs analysis is reported. For the real pairs analysis there were no significant main effects, but a significant interaction between block, condition and backing track F(2,10) = 3.81, p =.059, η p2 =.432. When broken up into separate 2 x 3 ANOVAs for each block, for the first block there were no main effects or interaction effects (all F<1.35, p>.304, η p2 <.212). In the second block there was a significant interaction between backing track and condition F(2,10) = 3.86, p =.057, η 2 p =.436, with simple effects analyses revealing a significant main effect of backing track for the vision condition F(2,10) = 4.01, p =.053, η 2 p =.445, but not for the novision condition, F(2,10) =.405, p =..667, η 2 p =.075. For the vision condition, post hoc analysis revealed that when playing with the ostinato backing track, musicians head movements were significantly less coordinated than when playing with the swing (p=.032) and drone backing track (p=.075). 72

75 Figure 26. Inverse circular variance between the musicians head movements for the 7.5 to 8.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. Circular Variance of Head Movement every 16 seconds. The results for the 15.5 to 16.5 second frequency band are shown in Figure 26. For the Real/Virtual Omnibus there was no main effect of pair type or interaction effects with pair type (all F<1.50, p>.280, η p 2 <.227). For the real pairs analysis there were no significant main effects or interaction effects (all F<.087, p>.473, η p 2 <.139). 73

76 Figure 27. Inverse circular variance between the musicians head movements for the 15.5 to 16.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. The results of the cross wavelet analysis on the musicians head movements every 1 second, 4 seconds, 8 seconds, and 16 seconds are summarized in Table 5. Given the majority of the significant differences are for the 1 second and 4 second intervals, it seems head movement coordination plays an important role at the faster time scales rather than the slower time scales (i.e. 16 second interval), and especially for the swing backing track. As previously mentioned this may have to do with particular rhythmic qualities or groove of the track. But this head movement is not only the result of coordinating with the musical structure of the backing track the relative phase relationships between the musicians head movements in the first block, when they had visual information about their co-performer, were significantly less variable for the real pairs when compared to the virtual pairs. This provides the strongest evidence for how musicians exploit visual information about each other s body movements when coordinating together in musical performance. 74

77 Table 5. Results of cross wavelet spectral analysis on the first principal component time series of the musicians head movements for the.5 to 1.5, 3.5 to 4.5, 7.5 to 8.5 and 15.5 to 16.5 second frequency bands. Left Arm Movement Circular Variance of Left Arm Movement every one-second. The results of the cross wavelet analysis on the musician s left arm movements for both the real and virtual pairs at the.5 to 1.5 second frequency band are displayed in Figure 7. For the Real/Virtual Omnibus there was no main effect of pair type, nor any interaction effects with pair type (all F<1.50, p>.275, η 2 p <.231). For the planned real pairs analysis, there were no significant main effects or interactions (all F<2.15, p>.167, η p2 <.301). 75

78 Figure 28. Inverse circular variance between the musicians left arm movements for the.5 to 1.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. Circular Variance of Left Arm Movement every 4 seconds. The results of the cross wavelet analysis on the musician s left arm movements for the real and virtual pairs at the 3.5 to 4.5 second frequency band are displayed in Figure 8. For the Real/Virtual Omnibus there was no main effect of pair type, nor any interaction effects with pair type (all F<2.87, p>.103, η p2 <.365). For the planned real pairs analysis, there were no significant main effects but there was a significant interaction between block and backing track F(2,10) = 3.81, p =.059, η p 2 =.433. When split into two separate 2 (block) x 3 (backing track) ANOVAs for each block, no main effects or interactions were found for the first block (all F<.201, p>.365, η p 2 <.182). For the second block, however, there was a significant main effect of backing track F(2,10) = 4.43, p =.042, η p 2 =.470, such that the musicians left arms were more coordinated when playing over the drone track than with the swing (p =.019) or ostinato track (p =.082). 76

79 Figure 29. Inverse circular variance between the musicians left arm movements for the 3.5 to 4.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. In summary the musicians left arm movements for the 4-second frequency were more coordinated when playing with the drone track in the second block, which may result from the need to establish a rhythmic foundation when playing with only a drone, the role of accompaniment normally designated to the left hand. Though the virtual pair analyses revealed that this coordination is not beyond the movement coordination you would expect when any two players are playing with the drone backing track in the same condition, the sparse musical context does necessitate the production of some kind of rhythmic or harmonic foundation that in the case of the swing and ostinato backing track is already provided. Circular Variance of Left Arm Movement every 8 seconds. The results of the cross wavelet analysis on the musician s left arm movements for the real and virtual pairs at the 7.5 to 8.5 second frequency band are displayed in Figure 29. For the Real/Virtual Omnibus there was 77

80 no main effect of pair type, nor any interaction effects with pair type (all F<1.45, p>.282, η p 2 <.225). For the planned real pairs analysis there were no significant main effects or interactions (all F<1.20, p>.322, η p 2 <.194). Figure 30. Inverse circular variance between the musicians left arm movements for the 7.5 to 8.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. Circular Variance of Left Arm Movement every 16 seconds. The results of the cross wavelet analysis on the musician s left arm movements for the real and virtual pairs at the 15.5 to 16.5 second frequency band are displayed in Figure 30. For the Real/Virtual Omnibus there was no main effect of pair type, nor any interaction effects with pair type (all F<1.99, p>.187, η p 2 <.285). For the planned real pairs analysis there were no significant main effects or interactions for the 15.5 to 16.5 second frequency band (all F<1.69, p>.234, η p 2 <.252). 78

81 Figure 31. Inverse circular variance between the musicians left arm movements for the 15.5 to 16.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. The results of the cross wavelet analysis on the musicians left arm movements every 1 second, 4 seconds, 8 seconds, and 16 seconds are summarized in Table 6. The only significant effects were for the 4-second interval, which makes sense given the role of the left hand in piano performance; if the left hand is engaged with accompanying and/or playing chord changes, it would be expected for this coordination to occur consistently at 4 second intervals, intervals that could be treated as a measure for both the swing and ostinato track. Therefore the significantly higher coordination between the left arms every 4 seconds for the drone backing track can be attributed to the fact that the drone doesn t provide chord changes (as opposed to the ostinato or swing) so the musicians have to create this progression themselves, demonstrating how the structure of the musical context constrains the movement coordination that emerges in musical performance. 79

82 Table 6. Results of cross wavelet spectral analysis on the first principal component time series of the musicians left arm movements for the.5 to 1.5, 3.5 to 4.5, 7.5 to 8.5 and 15.5 to 16.5 second frequency bands. Right Arm Movement Circular Variance of Right Arm Movement every one second. The results of the cross wavelet analysis on the musician s right arm movements for the real and virtual pairs at the.5 to 1.5 second frequency band are displayed in Figure 31. For the Real/Virtual Omnibus there was no main effect of pair type, nor any interaction effects with pair type (all F<1.92, p>.188, η 2 p <.284). For the planned real pairs analysis there were no significant main effects or interaction effects (all F<2.37, p>.143, η 2 p <.322). 80

83 Figure 32. Inverse circular variance between the musicians right arm movements for the.5 to 1.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. Circular Variance of Right Arm Movement every 4 seconds. The results of the cross wavelet analysis on the musician s right arm movements for the real and virtual pairs at the 3.5 to 4.5 second frequency band are displayed in Figure 32. For the Real/Virtual Omnibus there were no significant main effects of pair type, nor any interaction effects (all F<2.26, p>.154, η p 2 <.312). For the planned real pairs there was a significant interaction between block and backing track F(2,10) = 3.67, p =.064, η 2 p =.424, with no significant main effects or other interaction effects (all F<2.00, p>.186, η 2 p <.286). When split into two separate 2 x 3 ANOVAs for each block, for the second block of trials there were no main effects or interactions (all F<2.26, p>.155, η 2 p <.311), but for the first block of trials there was a significant main effect of backing track F(2,10) = 3.06, p =.092, η 2 p =.380, such that there was significantly more coordination 81

84 between the musician s right forearms when playing with the swing compared to the drone backing track (p =.083). Figure 33. Inverse circular variance between the musicians right arm movements for the 3.5 to 4.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. Circular Variance of Right Arm Movement every 8 seconds. The results of the cross wavelet analysis on the musician s right arm movements for the real and virtual pairs at the 7.5 to 8.5 second frequency band are displayed in Figure 33. For the Real/Virtual Omnibus there were not a significant main effect of pair type, nor any interaction effects (all F<2.80, p>.155, η p 2 <.359). For the real pairs analysis there were no significant main effects, nor interaction effects (all F<1.49, p>.271, η p 2 <.230). 82

85 Figure 34. Inverse circular variance between the musicians right arm movements for the 7.5 to 8.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. Circular Variance of Right Arm Movement every 16 seconds. The results of the cross wavelet analysis on the musician s right arm movements for the real and virtual pairs analysis at the 15.5 to 16.5 second frequency band are displayed in Figure 34. For the Real/Virtual Omnibus there was not a significant main effect of pair type, but there was a significant interaction between pair type and block F(1,5) = 31.66, p =.002, η p 2 =.864. When split into separate 2 (pair type) x 2 (condition) x 3 (backing track) ANOVAs for each block, for the first block of trials there was significant main effect of pair type F(1,5) = 9.63, p =.027, η p 2 =.658, such that the virtual pairs demonstrated less variable relative phase relationships than the real pairs in the first block of trials. While there was no main effect of pair type for the second block of trials, there was a significant interaction between pair type, condition and backing track F(2,10) = 4.38, p =.043, η p2 =.467. When split into separate 2 x 3 ANOVAs for each condition, for the vision condition there was a significant main effect of pair type F(1,5) = 5.40, p =.068, η p2 =.519, such 83

86 that there was significantly less variability in the relative phase relationships between the right arms of the real pairs than between the virtual pairs. For the no vision condition there was no main effect of pair type, but a significant interaction between pair type and backing track F(1,5) = 3.45, p =.072, η 2 p =.408. Simple effects analyses revealed no effect of pair type for the ostinato or swing track (both F<.993, p>.365, η 2 p <.166), but a significant effect of pair type for when the musicians were improvising with the drone track F(1,5) = 6.79, p =.048, η 2 p =.579, such that virtual pairs exhibited less variable phase relationships than the real pairs. In summary the real pairs right arm movements demonstrated less variable phase relationships than the virtual pairs for all conditions in the first block as well as for the vision condition in the second block. However, the relative phase relationships between the right arm movements of the real pairs was more variable than the virtual pairs when playing with the drone track, in the second block, in the no vision condition. For the real pairs analysis there were no interactions effects, but a significant main effect of block F(1,5) = 6.77, p =.048, η p 2 =.575; the coordination between the musician s right forearms was significantly more coordinated in the second block than in the first block of trials. 84

87 Figure 35. Inverse circular variance between the musicians right arm movements for the 15.5 to 16.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. The results of the cross wavelet analysis on the musicians right arm movements every 1 second, 4 seconds, 8 seconds, and 16 seconds are summarized in Table 7. Given that the significant differences are for 4- and 16- second frequencies, right arm movement coordination appears to play an important role at the slower time scales intervals that the musicians may treat as musical measures (4 sec) or sections (16 sec). The movement coordination for the 3.5 to 4.5 frequency band when playing with the swing track can be understood as coordination of right arm movements at the interval of two measures (which occurs every 3.6 seconds as described previously). There was not a significant difference between the real and virtual pairs, thus this decreased variability in the relative phase relationships between the right arms can be attributed to the structure of the backing track. Indeed, it is easy to imagine how a musician could coordinate with the measures of the swing track without reference to their co-performer. 85

88 As opposed to musical measures, the 16-second interval may emerge as an interval representing the transition between musical sections where the musicians make more pronounced movements with their right arms to play something in a new region of the keyboard, or switch between roles of leading versus accompanying. This coordination developed across trials, as evidenced by the increased coordination in the second block. The fact that in the second block the right arms of the real pairs demonstrated less variable phase relationships every 16 seconds than the virtual pairs, but only when they could see each other, further contributes to understanding under what conditions visual information about co-performers is exploited during musical performance. The coordination of the right arms at the 4-second interval doesn t necessitate visual information about co-performer s movements, but for an interval like 16- seconds where musician s right arms are engaged in switching and/or coordinating who is playing the lead melody, it is here that musicians exploit visual information to achieve this coordination that gets better with time. 86

89 Table 7. Results of cross wavelet spectral analysis on the first principal component time series of the musicians right arm movements for the.5 to 1.5, 3.5 to 4.5, 7.5 to 8.5 and 15.5 to 16.5 second frequency bands. To conclude the results of the cross wavelet analysis, Table 8 summarizes the results for specifically the musicians head movements as well as right arm movements. A first glance demonstrates how the oscillatory movements of the different effectors of musicians play different roles depending on the time scale of coordination within musical performance. Head movement is important for the faster time scales, while right arm coordination is crucial at slower time scales as evidenced by the list of significant effects for these regions in the table. The head movement at the one-second interval and the right arm movement at the 16 second interval demonstrated significantly less variable relative phase relationships for the real pairs than the virtual pairs, and this coordination was also often significantly greater when the 87

90 musicians could see each other. This co-occurrence of significant differences between real and virtual pairs and significant effects of visual information confirms the role of the virtual pairs analysis as a way of capturing when movement coordination is beyond what is attributable to playing with the same track. Thus when the real pairs are more coordinated there is often a significant effect of condition because it is only the real pairs that have visual information available to guide/constrain their movement coordination. Table 8. Results of cross wavelet spectral analysis on the musicians head movements, and right arm movements for the.5 to 1.5, 3.5 to 4.5, 7.5 to 8.5 and 15.5 to 16.5 second frequency bands. Full Body Movement In order to capture at a more global level how the musicians coordinated their movements during improvised performance, principal component analysis was performed on each individual 88

91 player s head, left arm, and right arm movements, in each x, y and z dimension (9 degrees of freedom). The principal component time series for each individual player for each possible combination of block, condition and backing track were then submitted to cross wavelet spectral analysis as with the individual limbs, in order to examine how the inverse circular variance (variability in relative phase relationships) related to the block, condition and backing track for the different frequency bands. The results for the cross wavelet analysis of both the first principal component time series as well as the second principal component time series are presented below. First principal component time series of Full Body Movement. The results of the cross wavelet analysis on the first principal component time series of the musician s full body movements for the real and virtual pairs analysis at the.5 to 1.5 second frequency band are displayed in Figure 35. For the Real/Virtual Omnibus there was not a significant main effect, nor any interaction effects with pair type (all F<2.87, p>.151, η 2 p <.365). For the planned real pairs analysis there were no significant main effects, nor interactions (all F<2.57, p>.170, η 2 p <.339). 89

92 Figure 36. Inverse circular variance between the first principal component time series of the musicians full body movements for the.5 to 1.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. The results of the cross wavelet analysis of the 3.5 to 4.5 second frequency band are displayed in Figure 36. For the Real/Virtual Omnibus there was not a significant main effect, nor any interaction effects with pair type (all F<1.38, p>.294, η p 2 <.216). For the planned real pairs analysis there were no significant main effects or interaction effects for the 3.5 to 4.5 second frequency band (all F<1.41, p>.290, η p 2 <.220). Figure 37. Inverse circular variance between the first principal component time series of the musicians full body movements for the 3.5 to 4.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. The results of the cross wavelet analysis of the 7.5 to 8.5 second frequency band are displayed in Figure 37. For the Real/Virtual Omnibus there was no significant main effect, nor any interaction effects with pair type (all F<2.37, p>.135, η p 2 <.283). For the real pairs analysis 90

93 their was a significant main effect of block F(1,5) = 25.87, p =.004, η p 2 =.838, such that the musicians full body movements were more coordinated in the first block than in the second block of trials. Figure 38. Inverse circular variance between the first principal component time series of the musicians full body movements for the 7.5 to 8.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. The results of the cross wavelet analysis of the 15.5 to 16.5 second frequency band are displayed in Figure 38. For the Real/Virtual Omnibus there was no significant main effect, nor any interaction effects with pair type (all F<3.87, p>.106, η p 2 <.436). For the real pairs analysis there were no significant main effects, but a significant interaction between block and backing track F(2,10) = 10.35, p =.004, η p 2 =.674, as well as a significant three-way interaction between block, condition and backing track F(2,10) = 3.16, p =.087, η p 2 =.384. When split into separate 2 (condition) x 3 (backing track) ANOVAs for each block, for the second block of trials there were no main effects or interactions (all F<2.89, p>.102, η p 2 <.336), but in the first block of trials there 91

94 was a significant main effect of backing track F(2,10) = 6.30, p =.017, η 2 p =.558, as well as a significant interaction between condition and backing track F(2,10) = 4.70, p =.036, η 2 p =.484. Simple effects analyses revealed that for the no vision condition there was no effect of backing track. But for the vision condition there was a main effect of backing track, such that in the first block of trials when the musicians could see each other, the musician s body movements were significantly less coordinated when playing with the swing than the ostinato (p=.034) and drone backing track (p=.004). Additionally, the musicians body movements when playing with the ostinato were less coordinated compared to when they were playing with the drone backing track (p=.086). Figure 39. Inverse circular variance between the first principal component time series of the musicians full body movements for the 15.5 to 16.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. To summarize these results it is useful to compare them to the coordination observed between the each player s individual limbs, as opposed to the full body movement. For the 7.5 to 92

95 8.5 second frequency, there were no significant main effects of block for the individual limbs, but their overall their body movements were more coordinated in the first block of trials. Because there were no significant differences between the real and virtual pairs for that frequency, it may be that musicians are engaging in full body movements to coordinate with the structure of the backing track, and consequently move similarly. For the 15.5 to 16.5 second frequency, there was no main effect of backing track for the first block for any of the individual limbs, and so instead the full body coordination every 16 seconds may be an important part of improvising over the drone track such that it creates and constrains their playing structure at slower time scales when the track itself provides very little structure. Comparing these results to those of the individual limbs also provides an idea of whether full body movement coordination is a straightforward addition of the coordination among individual limbs, or a higher-order form of coordination that is not easily understood through evaluation of the contribution of its components. This question can be further explored by examining the percentage accounted for by each time series (head x, head y, head z, left arm x, ect ), but the results presented here seem to indicate the existence of a higher-order informational variable constituted by the players full body movements. It is important to note that for none of these frequencies were the full body movements of the real pairs significantly different from the virtual pairs, thus the coordination captured by the first principal component could be largely attributed to how the variability in relative phase is constrained by the structure of backing track. 93

96 Table 9. Results of cross wavelet spectral analysis on the first principal component time series of the musicians full body movements for the.5 to 1.5, 3.5 to 4.5, 7.5 to 8.5 and 15.5 to 16.5 second frequency bands. Second principal component time series of Full Body Movement. The results of the cross wavelet analysis on the second principal component time series of the musician s full body movements for the real and virtual pairs analysis at the.5 to 1.5 second frequency band are displayed in Figure 39. For the Real/Virtual Omnibus there was no significant main effect, nor any interaction effects with pair type (all F<1.91, p>.225, η p 2 <.277). For the planned real pairs analysis there were no significant main effects or interactions (all F<1.28, p>.321, η p 2 <.203). 94

97 Figure 40. Inverse circular variance between the second principal component time series of the musicians full body movements for the.5 to 1.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. The results of the cross wavelet of the 3.5 to 4.5 second frequency band are displayed in Figure 40. For the Real/Virtual Omnibus there was not a significant main effect of pair type, however, there were significant interactions between pair type and block F(1,5) =4.27, p =.094, η p 2 =.461, and pair type and condition F(1,5) =16.41, p =.010, η p 2 =.763. Accordingly, two separate 2 (pair type) x 2 (condition) x 3 (backing track) ANOVAs were run for each block. For the first block there was a significant main effect of pair type F(1,5) =4.93, p =.077, η p 2 =.496, such that the full body movements of the real pairs demonstrated less variable relative phase relationships than the virtual pairs. For the second block of trials there was not a main effect of pair type, but a significant interaction between pair type and condition F(1,5) =17.20, p =.009, η p 2 =.775, as well as an interaction between pair type and backing track F(2,10) =3.56, p =.068, η p2 =.416. Two separate 2 (pair type) x 3 (backing track) ANOVAs for each condition revealed that for the no vision condition there was no main effect of pair type, nor any significant 95

98 interactions with pair type (all F<2.83, p>.154, η 2 p <.361).There was also no main effect of pair type for the vision condition. However, there was a significant interaction between pair type and backing track F(2,10) =8.47, p =.007, η 2 p =.629, for the vision condition. A simple effects analysis (for each backing track, in the vision condition, in the second block of trials) found no effect of pair type for the ostinato or drone backing track (both F<.428, p>.542, η 2 p <.079), but a significant effect of pair type for the swing backing track F(1,5) =6.95, p =.046, η 2 p =.582. Thus, the full body movements of the real pairs as captured by the second principal component time series for the 3.5 to 4.5 frequency band demonstrated less variable relative phase relationships than the virtual pairs in the first block, and for the vision condition when performing with the swing backing track in the second block. For the planned real pairs analysis there was a significant main effect of condition F(1,5) =11.12, p =.021, η 2 p =.690, as well as a three-way interaction between block, condition and backing track F(2,10) = 6.00, p =.019, η 2 p =.545. The analysis was split into two separate 2 (condition) x 3 (backing track) ANOVAs for each block. For the second block there were no significant main effects or interaction effects (all F<3.87, p>.106, η 2 p <.436). For the first block there were no main effects, but a significant interaction effect between condition and backing track F(2,10) = 3.88, p =.056, η 2 p =.437. Simple effects analyses revealed no effects of condition for the ostinato or drone backing track (both F<2.25, p>.194, η 2 p <.311), but a significant effect of condition for the swing track F(1,5) = 7.30, p =.043, η 2 p =.593, with the musicians body movements being more coordinated when they could see each other. 96

99 Figure 41. Inverse circular variance between the second principal component time series of the musicians full body movements for the 3.5 to 4.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. The results of the cross wavelet of the 7.5 to 8.5 second frequency band are displayed in Figure 41. For the Real/Virtual Omnibus there were no main effects of pair type but a four-way interaction between pair type, block, condition, and backing track F(2,12) = 4.45, p =.036, η p 2 =.426. When split into two separate 2 (pair type) x 2 (condition) x 3 (backing track) ANOVAs for each block, in the first block there was not a significant main effect of pair type, nor any interactions with pair type (all F<2.49, p>.124, η p 2 <.294). For the second block of trials there was no a main effect of pair type, nor any significant interaction with pair type (all F<2.23, p>.186, η p 2 <.271). For the real pairs analysis there were no significant main effects or interactions (all F<2.45, p>.136, η p 2 <.329). 97

100 Figure 42. Inverse circular variance between the second principal component time series of the musicians full body movements for the 7.5 to 8.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. The results of the cross wavelet analysis on the second principal component time of the 15.5 to 16.5 second frequency band are displayed in Figure 42. For the Real/Virtual Omnibus there was not a main effect of pair type but a significant interaction between pair type and block F(1,5) = 6.29, p =.054, η p 2 =.557. When split into two separate 2 (pair type) x 2 (condition) x 3 (backing track) ANOVAs for each block, there were no significant main effects or interaction effects with pair type in the second block (all F<3.83, p>.108, η p 2 <.434), but a main effect of pair type in the first block of trials F(1,5) = 10.15, p =.024, η p 2 =.670, where there the movements of the virtual pairs demonstrated less variable relative phase relationships than the real pairs. For the real pairs analysis there were no significant main effects or interactions (all F<2.02, p>.241, η p2 <.288). 98

101 Figure 43. Inverse circular variance between the second principal component time series of the musicians full body movements for the 15.5 to 16.5 second frequency band. The top row displays the inverse circular variance for the no vision (purple) and vision (green) conditions, for each block with the standard error bars. The bottom row shows the inverse circular variance between the virtual pairs. 99