When the Leading Tone Doesn t Lead: Musical Qualia in Context

Transcription

1 When the Leading Tone Doesn t Lead: Musical Qualia in Context Dissertation Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University By Claire Arthur, B.Mus., M.A. Graduate Program in Music The Ohio State University 2016 Dissertation Committee: David Huron, Advisor David Clampitt Anna Gawboy

2 c Copyright by Claire Arthur 2016

3 Abstract An empirical investigation is made of musical qualia in context. Specifically, scale-degree qualia are evaluated in relation to a local harmonic context, and rhythm qualia are evaluated in relation to a metrical context. After reviewing some of the philosophical background on qualia, and briefly reviewing some theories of musical qualia, three studies are presented. The first builds on Huron s (2006) theory of statistical or implicit learning and melodic probability as significant contributors to musical qualia. Prior statistical models of melodic expectation have focused on the distribution of pitches in melodies, or on their first-order likelihoods as predictors of melodic continuation. Since most Western music is non-monophonic, this first study investigates whether melodic probabilities are altered when the underlying harmonic accompaniment is taken into consideration. This project was carried out by building and analyzing a corpus of classical music containing harmonic analyses. Analysis of the data found that harmony was a significant predictor of scale-degree continuation. In addition, two experiments were carried out to test the perceptual effects of context on musical qualia. In the first experiment participants rated the perceived qualia of individual scale-degrees following various common four-chord progressions that each ended with a different harmony. While scale-degrees were still shown to elicit relatively stable qualia, there was a significant effect for the role of the local chord context. Importantly, this experiment was carried out using participants both with ii

4 and without music-theoretic training, supporting the notion that the identification of scale-degrees was not responsible for the evoked qualia. This experiment also partially replicated a component of Krumhansl & Kessler s (1982) study examining the goodness of fit of scale-degrees within a key. However, the authors claim that scale-degrees 1, 3, and 5 were best fitting due to the tonal stability of the tonic triad could not be fully supported here. In fact, the results from the present study found that the goodness of fit effect could perhaps be better explained by other factors. In the second experiment participants rated the perceived qualia of either composed inter-onset patterns or recorded song clips presented in different metrical contexts. Both inter-onset interval pattern and meter were shown to be significant influences on judgments of qualia. In addition, syncopation was found to be a strong predictor for certain components of qualia. The overall results from these studies show that musical context is an important contributor to musical qualia, and therefore, while isolated musical events may still be capable of creating relatively stable qualia, in real musical contexts these may change dramatically. iii

5 Acknowledgments I would first and foremost like to thank my advisor, David Huron, for his continuous guidance and support, and for his infectious curiosity and excitement for all things musical. I would also like to thank my fellow CSML colleagues, both past and present, for their feedback, advice, and camaraderie. In addition, many thanks are due to the other members of my committee, Anna Gawboy and David Clampitt, for their insights and encouragement throughout my time at Ohio State. I would also like to acknowledge the support of the Social Sciences Humanities Research Council of Canada, whose funding made possible my final year of study. Finally, I especially need to thank my better half, Nat Condit-Schultz, for all of his help with statistics and programming, for his unfailing optimism, and above all, for inspiring me to become a better scholar and musician. iv

6 Vita Graduate Research Assistant, Ohio State University School of Music Graduate Teaching Associate, Ohio State University School of Music Private Piano and Music Theory Instructor M.A., Music Theory, University of British Columbia B.Mus., Music Theory and History, University of Toronto Publications Research Publications Arthur, C., & Huron, D. (2016). The direct octaves rule: Testing a scene analysis interpretation. Musicae Scientiae. Advance online publication. doi: / Devaney, J., Arthur, C., Condit-Schultz, N., & Nisula, K. (2015). Theme and Variations Encodings with Roman Numerals (TAVERN): A new data set for symbolic music analysis. In M. Muller & F. Wiering (Eds.), Proceedings of the International Society of Music Information Retrieval (ISMIR) Conference. Malaga, Spain: Arthur, C. (2014). Does harmony affect scale-degree qualia?: A corpus study investigating the relation of scale-degree and harmonic support. In M.K. Song (Ed.), Proceedings of the 13th International Conference for Music Perception and Cognition. Seoul, Korea: Yonsei University, v

7 Fields of Study Major Field: Music Area of Specialization: Music Theory vi

8 Table of Contents Page Abstract Acknowledgments Vita List of Tables ii iv v x List of Figures xi 1. Introduction On the Philosophy and Science of Musical Qualia What Are Qualia? The Problem Music Raises for the Study of Qualia (and Vice-Versa) Conceptual Knowledge and Qualia Qualia as Synthesis of Sensory and Cognitive Processing Introspection, Observation, and Converging Evidence Why Study Musical Qualia? Theories of Musical Qualia Scale-degree Qualia and Implicit Learning Rhythm Qualia Chapter Summary A Corpus Study The Corpus Analysis Overview Sampling vii

9 3.1.3 Methodology Evaluating the Models Global Hypothesis Test Descriptive Statistics Mode Classification Zeroth-order Probabilities First-order Probabilities Change in Melodic Probabilities when Harmony is Considered Conclusions Discussion A Perceptual Study of Scale-degree in Context Introduction Method Participants Stimuli Procedure Results Discussion Rhythm Qualia Introduction Background A Perceptual Study: Rhythm in Context Introduction Method Results and Discussion Post-hoc Exploration of Rhythm Qualia Chapter Summary General Summary Recapitulation Discussion Implications for Music Pedagogy Areas for Future Research on Musical Qualia Works Cited viii

10 Appendices 150 A. The Corpus Data ix

11 List of Tables Table Page 3.1 Illustration of Encoding Key Changes Melody and Harmony Encodings at Key Changes Result Statistics by Dependent Variable Tally of Opt-outs by Dependent Variable Correlations with the K&K Profile Similarity Measures Between Dependent Variables Syncopation Scores for Rhythm Stimuli x

12 List of Figures Figure Page 2.1 Example of Identical Acoustic Information Generating Unique Qualia Flow Chart of Diatonic Scale-degree Probabilities from Huron (2006) Zeroth-order Distribution of Scale-degrees in the Corpus Zeroth-order Distribution of Harmonies in the Corpus First-order Probabilities for Scale-degrees Predicted and Observed Probabilities for ˆ1 in Tonic Context Effect of Harmony on Scale-degree Probability (continued on next page) Image of Digital Interface Used in Experiment Scale-degree Qualia Ratings (continued on next page) Consistency Between and Across Participants Intra-Subject Correlations Key Profiles from Krumhansl & Kessler (1982) Present Data Compared to Krumhansl & Kessler s (1982) Key Profiles Three Identical Onset Patterns in Different Metrical Contexts Rhythm Stimuli (Composed) xi

13 5.3 Rhythm Stimuli (Borrowed) List of Rhythmic Descriptor Terms Rhythm Qualia Ratings - Composed Rhythms (continued on next page) Rhythm Qualia Ratings - Song Clips Amount of Syncopation and Grooviness Additional Examples of Syncopation Predicting Qualia xii

14 Chapter 1: Introduction This dissertation examines whether certain fundamental components of musical structure, such as scale-degree and rhythm, can generate unique, qualitative musical experiences, or qualia, and in particular whether the musical context in this case harmony and meter, respectively can affect those experiences. Chapter 2 introduces the topic of qualia, briefly mentions its philosophical origins, and discusses its role in music. In particular, the concepts of scale-degree and rhythm pose a challenge to many traditional accounts of what constitutes qualia on account of their being referential. In addition, many philosophers who claim qualia to be ineffable pose a challenge to empirical researchers wishing to study musical qualia. This chapter also reviews the literature on music and qualia, and summarizes theories about the generation of musical qualia, giving special attention to the role of implicit learning. It then proposes an operational definition of qualia, similar to those used by Zentner (2012) and Dowling (2010), that supports a perspective conducive to the evaluation of qualia as an object for scientific study. Chapters 3 and 4 are devoted to the topic of scale-degree and harmony interaction, in particular questioning how a harmonic context might influence a scale-degree s qualia. Building on Huron s (2006) theory of the role of implicit learning in the generation of musical qualia, Chapter 3 begins by examining how melody and harmony 1

15 interact from a statistical standpoint. Specifically, prior research has examined pitch distributions and/or first-order probabilities for scale-degrees (or pitch) in various corpora (e.g. Krumhansl, 1990; Aarden, 2003; Pearce, 2005; Huron, 2006; Temperley, 2007; Albrecht & Huron, 2014), with some of these researchers finding that a listener s expectations can be largely predicted by these pitch distributions. However, it is unclear for music that typically is set with a harmonic accompaniment whether the underlying harmonic context may influence the melodic behavior (and therefore, by extention, the melodic expectations), and if so, to what extent? Therefore, a corpus of classical music with both melodic and harmonic information was created in order to analyze the relative contribution of harmony in influencing the statistical probabilities of melodic continuations. Chapter 4 reports a perceptual experiment that examines the qualia of scaledegrees set in various harmonic contexts. One question that arises from both philosophical and cognitive literature is whether conceptual knowledge can influence qualia. In the case of scale-degree in particular, trained musicians can not only identify scaledegrees, but have conceptual knowledge about them (e.g., functional properties), and have already been largely exposed to vocabulary commonly applied to them within the field of music theory. Therefore, by using participants with and without musical training, this chapter also investigates the question of how music-theoretic training might influence descriptions of musical qualia, and in fact, questions whether non-musicians are capable of reporting scale-degree qualia at all. In addition, this experiment partially replicates a component of Krumhansl and Kessler s (1982) probe-tone study. As such, the chapter ends with a comparison of the results of their study with the results of the present experiment. 2

16 Chapter 5 discusses the related referential problem of musical qualia as it applies to rhythm in a metrical context. It then details an experiment on the perception of rhythm qualia in various metrical contexts, which makes use of both experimentercontrolled (i.e., composed) and ecologically valid (i.e., preexisting) stimuli. In the work as a whole, I attempt to go beyond simply looking for statistically significant results, which merely suggest the existence of some effect, by investigating and analyzing the stimuli themselves via theoretical analyses and post-hoc exploratory comparisons to look for features that might be causing or contributing to the effect. Finally, Chapter 6 summarizes the research on musical qualia. Given that real music consists of a multitude of contexts (meter, duration, melody, rhythm, key, harmony, etc.) happening simultaneously, it is proposed that while certain components in music such as scale-degree or rhythm may elicit relatively stable qualia in isolation, these may change dramatically when encountered in more complete musical contexts. Nevertheless, it is suggested that in order to build a general picture of how musical qualia are evoked, it is appropriate to study the qualitative effects of musical rudiments independently. The chapter ends with a discussion of the implications for the findings, and suggestions for future areas of study related to musical qualia. 3

17 Chapter 2: On the Philosophy and Science of Musical Qualia Abstract In this chapter, the topic of musical qualia is reviewed, taking into consideration philosophical, theoretical, and cognitive standpoints. Theories about the origins of musical qualia are discussed, and relevant literature is reviewed. Concerns are raised regarding various stipulations on the traditional definition of qualia, and the role of conceptual knowledge. A working definition of qualia is proposed that would make qualia amenable to scientific study. 2.1 What Are Qualia? Qualia is a term, borrowed from philosophy, that generally refers to what it is like to experience something. The term is often used synonymously with phenomenal character. In philosophy of mind qualia is a hotly debated concept, as it features prominently in issues related to consciousness and the mind-body problem. The most common points of debate are which mental states have qualia, whether qualia are intrinsic qualities of their bearers, and how qualia relate to the physical world both inside and outside the head (Tye, 2015). While the full debate of what are or aren t qualia will not be expounded on here, some frequently-made claims about qualia will be mentioned since they necessarily bear on the topic of musical qualia. 4

18 Before engaging in further discussion, some definitions of terminology with specialized meanings in philosophy will be useful. In philosophy, intrinsic (or non-relational) is a property that an object or thing has of itself, independently of other things, including its context. An extrinsic (or relational) property, on the other hand, is a property that does depend on the things around it (i.e., context). Ineffable (opposite: effable) refers to a common problem in aesthetics and philosophy of mind where it is argued that certain aspects of experience are incommunicable. The notion of representation in philosophy is a complex one. It comes from the theory of representationalism, which holds that we only come to understand the physical world through our minds and our ideas. (Note that this is not the same as Platonic idealism, where it is held that there is no physical realm but only the realm of ideas.) A simplistic explanation of the term representation is to say that objects (both physical and non-physical) have isms, likenesses or features that we come to internalize. These isms often can be described using propositional language. That is, we can say I feel x about y or I believe x has/does y. Some philosophers, however, believe that qualia are made up of more than these representations (or that they exist as something completely independent of them), and this view is described as non-representational (Hansberry, in press). Theories of non-representational accounts of qualia therefore typically hold that qualia cannot be described in propositional language. While there are many claims made about qualia within the field of philosophy, it is commonly argued that qualia are intrinsic, non-relational properties, and many believe that they are ineffable and non-representational (Tye, 2015), and it is these claims that prove particularly problematic when describing musical qualia. 5

19 2.2 The Problem Music Raises for the Study of Qualia (and Vice-Versa) What qualia are elicited by music? There are many instances where a listener might recognize a distinctive feeling, character, or quality associated with some musical moment. For example, in the striving or effortful intensity of a sung high pitch, or in the paradoxical feel of both repose and impetus evoked by a deceptive cadence. In these and many other musical situations, listeners can experience seemingly ineffable yet characteristic subjective states that that would be appropriate to call qualia. Yet, considering some of these experiences as qualia may raise concern for some philosophers who hold a strict viewpoint on what constitutes qualia, as described in the section above. There is a problem with the notion of musical qualia as being intrinsic, specifically, that an object s quale cannot exist in reference to anything else. The claim of qualia as intrinsic is perhaps the single most difficult problem for musical qualia, in particular for the two types of qualia I investigate in this dissertation: scale-degree qualia and rhythm qualia. Our experience of scale-degree and rhythm are not like that of color, or even of timbre, in that they are not absolute but in fact only exist in relation to something else. In the case of scale-degree, it exists only in relation to a tonic, key, or other scale-step; in the case of rhythm, it typically relies on its relation to a meter (see Figure 5.1 for an example and Section 5.1 for discussion). Of course, that these aspects of music are capable of eliciting qualia does not appear to be under debate. The problem of ineffability, if true, would certainly pose a problem for anyone wishing to study musical qualia empirically, since experiments typically rely on language as expressed by participants (e.g., how does this sound? or how does this 6

20 feel? ). Even when examinations of qualia attempt to avoid procedures that collect language as a dependent variable (such as descriptive language offered by participants or words rated for applicability), language is nevertheless used in instructions for experiments that indirectly measure qualia (e.g., adjust the brightness until the object on the left is twice as bright ). It seems, however, that the (typical) philosopher s objection to the notion of effability is that it is considered impossible to describe a phenomenal experience in words to another individual who has never had that experience. While I agree that it is likely impossible to communicate the qualia of, for example, scale-degree 7 (or anything for that matter) to someone who has never experienced it, I will argue that those of us who have experienced hearing scale-degree 7 are able to communicate with each other some aspect of that shared experience. The idea of musical qualia (and perhaps all qualia) as being non-representational as defined above, is challenged if one asks what remains after the representational content is removed. Hansberry (in press), for instance, states that qualia ought not to extend to propositional content, conceptual content, emotions, associated memories, affordances, or any other part of experience. Tye (2015), on the other hand, makes an opposing argument (specifically in reference to the qualia of linguistic understanding): The phenomenal aspects of understanding derive largely from linguistic (or verbal) images, which have the phonological and syntactic structure of items in the subject s native language. These images frequently even come complete with details of stress and intonation. As we read, it is sometimes phenomenally as if we are speaking to ourselves. (Likewise when we consciously think about something without reading). We often hear an inner voice. Depending upon the content of the passage, we may also undergo a variety of emotions and feelings. We may feel tense, bored, excited, uneasy, angry. Once all these reactions are removed, together with the images of an inner voice and the visual sensations produced 7

21 by reading, some would say (myself included) that no phenomenology remains. Music, like language, is able to evoke emotions and feelings, though arguably, in a more abstract way, since in the case of music without lyrics there lacks semantic meaning 1. Music also has its own syntax and prosody (Thaut, 2008), and is capable of inducing visual imagery (Gabrielsson, 2011; Osborne, 1980; Quittner & Glueckauf, 1983). Thus, although Tye s argument refers to language, it might apply equally well to music. These aspects of music, combined with others that I describe below, intuitively seem to make up a rather large contribution to their qualia, and after removing all propositional content, conceptual content, emotions, memory, affordances, etc., it is difficult to comprehend what would remain. Discussions and debates on qualia in philosophy frequently arise due to unresolved issues in perception and consciousness, and a particular problem arises over the explanation of phenomena such as illusion, memory, and hallucination. While these phenomena bear a relationship to objects of reality, they technically only exist in one s mind. Common points of argument typically center on whether the qualitative experience of these phenomena are distinct from their original (or physical) counterparts, and/or whether these phenomena have representational content in the same way that their originals do. Scale-degrees and rhythms pose a similar problem for qualia perception, since, in a way, they too only exist in the mind. That is, they by definition rely on interpretation. However, they are unlike memory and hallucinations in that they are typically evoked in direct response to real environmental stimuli, and 1 Michael Thaut, in his book Rhythm, Music, and the Brain asserts that most likely the most important difference between speech and music lies in the lack of explicit semantic or referential meaning in music. (p.2) However, he does argue that music fits the definition of communication. 8

22 they are unlike illusions since those are usually thought of as perceptual errors, whereas scale-degrees and rhythms are typically not. Indeed, a trained musician can hear a single complex tone (e.g., a single key struck on the piano) and impose upon it the phenomenal experience of any scale-degree entirely through mental gymnastics. Importantly, this is not simply an artificial exercise, but occurs during real listening at moments of transposition, presumably not only for trained musicians, but for all listeners enculturated into the Western musical tradition. The same phenomenon of mental manipulation occurs with metric reinterpretations of rhythms, or fakeouts, as will be discussed in Chapter 5. This problem, where seemingly the same physical stimulus can represent two things simultaneously poses a serious challenge for philosophical theories that hold qualia to be intrinsic. 2.3 Conceptual Knowledge and Qualia A famous thought experiment in philosophical literature is that of Mary the color scientist who lives in a room without color, knows everything there is to know about color, and then one day she leaves the room and finally experiences color. The argument goes that the actual experience of seeing color must extend beyond her knowledge of the physical properties of color. This argument intuitively seems to have merit. However, it does not disprove the role of conceptual information in shaping our qualitative experiences. That is, what if the argument is flipped around? For someone who knows nothing about wavelengths and frequencies as properties of light and color, but can experience color, how is their experience of seeing color different from Mary s (if at all)? Surprisingly, there is not much discussion on the role of conceptual knowledge as contributing to qualia. Many philosophers contend that 9

23 the content of each experience and its phenomenal properties are thus two different features that may be present independently from each other (Schiavio & van der Schyff, 2016), and perhaps this is why there is a reluctance to discuss the role of conceptual knowledge. However, simply because two things might exist independently does not mean that they cannot bear influence on each other. So does conceptual information affect or influence qualia? Of course, there is a distinction between knowing facts and having beliefs about things, and the two may have very different effects (or lack of) on phenomenal experience. Since I do not wish to digress further into philosophical debates, I will assume that conceptual knowledge can imply both factual (including semantic) knowledge and held beliefs, that it relies on declarative memory, and typically can be expressed in propositional language. I argue that while conceptual knowledge may not always directly influence qualia, even casual observation suggests knowledge can, at least, indirectly influence it. Take, for example, what might seem to be a completely random, nonsensical sound. While one person might be able to describe their experience based on the feel of the timbre, loudness, and so on, the qualia will almost certainly be different for a second listener for whom the sound is not random, but is an expletive in his/her native language. Take this personal anecdote as another example: While a child around the time of Halloween, there was an exhibit that consisted of boxes with holes cut out that allowed one to stick their hand inside. Each box had a label of some human body part, insinuating its contents. However, the objects were simple everyday objects: a human eye was a skinned grape; human intestines were a ball of slick noodles; and so on. The point is that thinking that I was touching these objects 10

24 was a horrific and disgusting experience in a way that simply touching those common objects normally would not be. A more simple, everyday example of the same scenario unfolds when someone makes a rather poorly timed suggestion or comment in relation to some food you might be eating. How does the experience of seeing or tasting that food change after this concept has been brought to mind? Based on these examples, I argue that knowing something (or thinking/imagining) about an object, has the potential to influence its qualia. Further support for conceptual information influencing phenomenal experience comes from recent literature of synaesthesia, a rare phenomenon where certain individuals have sensory experiences that are interlaced, either in the same modality or across different ones, such as seeing colors associated with certain letters or numbers, or associating certain tastes with sounds. Research has pointed to regions of the cortex associated with conceptual knowledge as playing an important role in synaesthesia, suggesting that associations form (at least in part) due to conceptual associations, which would somewhat explain the wide differences and subjectivity in reports of the condition (Chiou & Rich, 2014). Of course, there is a lot of variation in the type of experiences we have, and in the way that one individual s experience might differ from another s. However, if conceptual knowledge has the potential to influence musical qualia, then do musicians with a deep theoretical knowledge of music experience some musical passage in a phenomenally different way compared to a group with no theoretical understanding of music? This question is addressed in this dissertation, and a discussion of the issue is carried out in Chapter 6. 11

25 2.4 Qualia as Synthesis of Sensory and Cognitive Processing The concerns raised in this chapter are not unique. Despite differing points of view, Goguen (2004), Zentner (2012), Raffman (1993), and Dowling (2010), all discuss the problem with traditional accounts of qualia for empirical musicologists. Goguen, for instance, holds that qualitative experience is situation dependent, and believes that emotion... is the essence of qualia. Zentner argues that contrary to the claim that musical experiences are ineffable...musical qualia may be amenable to linguistic description and objectification. Raffman believes that empiricism has much to contribute to aesthetics, and in particular criticizes the view of qualia as non-representational, saying some sensory states have... legitimate representational contents (although she also believes these are consciously accessible but not reportable ). Dowling, in fact, is quite audacious in dismissing the stipulations of ineffability ( I tend to think something truly ineffable would resist being manipulated ), determinacy ( there aren t any incorrigible facts ), and intrinsic properties ( there is almost no area of human perception which is not context dependent ). In order to study musical qualia, it seems that one must either take a position on what qualia is or is not, or propose to use the word out of tradition, but define it to mean something other than what it usually means. While I have opposed various stipulations on the traditional definition of qualia and in so doing imply a definition other than what it usually means I propose that it remains appropriate to use the term qualia, since, as Dowling (2010) states: This use of the term clearly is not consonant with some of the ways it has been used in the history of philosophy, but nevertheless it is difficult to see what term could better be used to point to the functions described. 12

26 In terms of an operational definition of qualia, mine can be taken as largely synonymous with Dowling s, who proposes qualia as so-called intervening variables, or inferred processes in the causal chain leading from stimulus to response. Specifically, I propose that while qualia are the resulting subjective experience of something, there are a multitude of factors that can contribute to that phenomenal aspect of musical experience, including sensory information (bottom-up information); conscious and unconscious (implicit) knowledge, memory, and awareness (top-down information); and the resulting inference or interpretation (including possible accompanying bodily changes, such as heartrate) that are a result of the synthesis of this sensory and cognitive processing. Furthermore, I informally define qualia to be aspects of conscious experience, available via introspection, fleeting or temporary, extrinsic (relational), at least partially communicable, and as having the potential to be mediated both by context and conceptual knowledge (explicit or implicit). In addition, while I regard qualia as necessarily subjective, the fact that persons can have similar experiences in response to the same stimulus suggests there may be (theoretically) measurable features of the stimulus that are seemingly absorbed by our senses and that may lead to these common, or overlapping, aspects of perception. 2.5 Introspection, Observation, and Converging Evidence What it is like to undergo some phenomenal experience is available to us by introspection (Tye 2015), and I argue that we can study qualia by examining common language and observing common reactions to stimuli from multiple persons experiences and introspections. Zentner (2012) argues that although the inner experience of an emotion is a private and subjective one, its expressions are amenable to scientific 13

27 description, quantification, and analysis. Chalmers (1996), on the other hand, claims that models of the mind as posed by scientific research are incapable of explaining the human experience of consciousness in any satisfactory way (Montague, 2011). I tend to agree, however, with Goguen (2004), who says Chalmers discusses the hard problem which is to explain qualia in the language of the hard sciences... [however] we argue that cognitive and qualitative aspects of experience are inseparable, even though first and third person approaches artificially separate them. In particular, as already mentioned, the converging evidence provided by similar descriptions independently offered by multiple individuals reaction to the same stimulus surely points to some common component of an experience. This point of view is similarly shared by Zentner, who says: Although we cannot know what people s inner experience... might feel like, the assumption is that the similarity in emotion expression is subtended by an interpersonally similar inner experience of the emotion. In other words, the similarity and dissimilarity of inner subjective experiences across individuals, while not directly accessible, can nonetheless be inferred. In this way, I proceed by assuming that some aspects of musical qualia can be obtained using this converging evidence methodology, even if the resulting descriptions might be crude and/or incomplete, and that musical qualia are amenable to scientific analysis. 2.6 Why Study Musical Qualia? I believe that one goal of music cognition is to attempt to bridge the explanatory gap between musical structure and musical perception. For composers who wish to compose in a way that generates a specific effect in the listener, a better understanding 14

28 of the relationship between the music s structure and the qualia they elicit can only be beneficial. Of course, music theory and musical composition have a complementary relationship. Montague (2011), discusses the language used in formalist interpretations of musical events, complaining that frequently the language used does not offer much connection to the experience of attending to this music, and suggests that a connection of the totality of musical experience with analytical explanations would make analyses themselves more meaningful. While this connection to musical experience is not always possible in analysis (nor, in my opinion, always relevant), a comprehensive, phenomenally-informed analysis of music remains a formidable goal not only for music theorists but for systematic musicologists as well. Of course, this type of approach to music analysis is certainly not novel, having largely been brought to the foreground of music theory by Meyer (1956), who was perhaps one of the first music theorists to examine music from a cognitive perspective, attempting to interpret the progression from structure to anticipation to emotion. 2 In yet other applications of qualia research, if practitioners in the field of music therapy can better understand the link from musical stimulus to physiological reaction and phenomenal experience, it could lead to innovations and/or potential increases in the reliability of music therapies. Theories of musical expectation have been used to attempt to explain the existence of some musical qualia (Huron, 2006), however, musical qualia can also be used to generate theories. That is, if musical qualia can generate reliably elicited effects from multiple individuals responses to the same stimuli, it may be useful in helping shape theories of music cognition. In sum, if perceptual reactions 2 Note that outside of music theory, Berlyne s Aesthetics and Psychobiology (1971) takes on a similar lofty goal to Meyer in his investigation of the biological flow from perceptual variables in artworks to physiological variables and their associated behavioral states to affective-aesthetic responses. 15

29 can be reliably tied to musical features, then this information could be used to inform musical composition, music therapy, theories of music cognition and musical expectation, as well as musical analysis. Of course, one s understanding of music often begins in a classroom, and therefore all of the above implications are also relevant for music pedagogy, in particular for music theory and composition pedagogy. In addition, there may be certain aspects of musical qualia that are particularly relevant for aural skills training, however these will become clearer after a comprehensive investigation of the topic of qualia in context, and so this discussion is reserved for Chapter 6. Our reactions and experiences to music are almost as varied as music itself. While certainly not a simple undertaking, I believe that understanding the range of musical experiences and the factors that might contribute to them to be a valuable endeavor not only for musicians and music scholars, but also for philosophers and psychologists. 2.7 Theories of Musical Qualia Where do musical qualia come from? What gives each note or passage a unique characteristic, or quality? For instance, what makes the scale-degrees in each measure of Figure 2.1 feel different? 16

30 Figure 2.1: Example of identical acoustic informtaion generating unique qualia. Three intervals are presented that are identical in pitch, but composed of different scale-degree pairs. Despite having the same acoustic information, each generates a distinct qualia. Of course, scale-degrees and intervals can be defined in a mathematical sense as a point on a scale, or a unit of measure, respectively. However, this is not typically how one perceives them. Indeed, in music theory pedagogy, teachers often focus on the overall qualities of the sounds, or what they sound like. In the above examples, the acoustic information in each measure is identical. Only the musical context in this case, key has changed. In order to perceive these three pairs of scaledegrees as having different qualia, then, one must be able to imagine them within their appropriate positions within the given key (or scale). But what would cause one note within a scale to sound different from any other in the first place? Scale-degree Qualia and Implicit Learning Huron (2006) argues that scale-degree qualia arise at least in part from statistical learning. As applied to melody, the theory of implicit learning assumes that, through exposure, we come to internalize the statistical probabilities for where a given scale-degree will go next. In tonal music, of course, the progression of scale-degrees is 17

31 Figure 2.2: Flow chart of diatonic scale-degree probabilities from Huron (2006). Diatonic scale-degree flow chart according to first-order probabilities. The strength of a scale-degree s tendency is marked by the arrow s width. non-random, with scale-degrees exhibiting a range of probable (or improbable) behaviors, or tendencies to proceed in a predictable way. For instance, ˆ7 tends to move to ˆ1, and ˆ4 tends to move to ˆ3. Huron argues that because the brain ceaselessly attempts to predict what will happen next, our sense of anticipation and the unconscious confirmation or denial of the arrival of some scale-degree are, in part, what lead to scale-degree qualia. In other words, because of statistical regularities, the qualia of an individual scale-degree become closely tied to feelings of tension or resolution that have become associated with it. Huron, of course, was not the first person to suggest the role of anticipation in leading to phenomenal experience. Dennett (1991) has famously said that brains are, in essence, anticipation machines, and Meyer (1956) has proposed that emotion in music is evoked when... an expectation is not met. 18

32 Of course, the entire concept of scale-degree requires a connection not only to the other scale-degrees and where they might proceed, but to a particular position within the scale. Browne (1981) theorized about what has come to be known as the rare interval hypothesis. Browne proposed that it is the unique intervallic properties of the diatonic collection that allow for position finding, or a sense of maintaining one s bearings with regard to a particular position within the scale or reference to a tonic. Specifically, the subset of all possible intervallic (dyadic) possibilities within the diatonic collection can be tabled into a set of interval-classes, where each interval class appears a unique number of times. Thus, the rare intervals the tritone and semitones function as the position finding elements. This theory was tested empirically by Butler and Brown (1981) who found that listeners were best able to correctly infer the tonic from three note subsets when they included the rare interval of the tritone. Shepard (2009) argued that we build an internalized mental representation of the pattern of the scale, which is what leads us not only to understand the music we hear, but also to the generation of qualia. Note that Shepard s position also implies that simple exposure (implicit learning) is at work in the generation of these internalized representations. While Browne doesn t explicitly mention an internalized representation of the scale, he does imply that the structural properties of the scale are what generate their qualia: One might look at a [tonal] usage, even one which is merely a feeling long noted, and attempt to provide the structural differentiation which might account for that usage in terms of stateable facts. It seems clear that an event when perceived or imagined in context, is somehow enriched by its context. Here Browne provides a footnote in which a colleague comments on the relation between Browne s theory and her own observations in the classroom: 19

33 But of course! Students have always insisted that the I chord doesn t sound like the IV chord or the V chord even though they are all obviously major triads. Thus Browne and his colleague are suggesting that this internalized information about a note s (or chord s) position in the scale (here referred to as context ), contribute to the generation of unique qualia. Raffman (1993) argues there is a structural musical ineffability which arises, for example, when an untrained musician is trying to describe some aspect of music but lacks the proper understanding and/or terminology; or when a performer feels compelled to perform a passage in a particular way without knowing why. What Raffman seems to be describing is musical intuition. Since musical understanding and cultural norms are not hard-wired at birth, presumably musical intuition arises from experience. I therefore argue that musical intuition is simply a form of unconscious knowledge, or, in other words, a form of implicit learning. While Raffman takes these scenarios as examples of music s ineffability, it does not follow that all qualia (or all aspects of some quale) are ineffable, as instances of music s effability have already been discussed. (Note that musical intuition might best be explained via Daniel Dennett s line of thinking which claims that all knowledge could be capable of expression in verbal form if we only possessed the necessary vocabulary and paid close attention to the details of our experience since many musicians can express the nuances of those qualia with proper training.) Since the notion of implicit or statistical learning features prominently in theories of musical expectation, and expectation has been implicated in the generation of scale-degree qualia, Chapter 3 will continue this inquiry by investigating the statistical probabilities of scale-degrees in a harmonic context. In particular, the notion 20

34 of musical context itself poses an interesting question for theories of qualia: If musical objects such as scale-degrees evoke seemingly unique qualia that are largely brought about through implicit learning, do their qualia remain stable across different contexts, or would the statistical frequency of the context itself bear on the qualia of a scale-degree? For instance, the quintessential scale-degree qualia are perhaps those evoked by scale-degree 7, which typically are characterized with terms such as leading, leaning, pulling, and restless. However, scale-degree 7 is unique in that, within Western classical music, it is the only scale-degree that is so strongly associated with chords of dominant function, which inherently carry strong tendency to resolve to tonic. Interestingly, however, if the so-called leading tone (i.e., ˆ7) is placed in a mediant context, or, more rarely, in a tonic context, it appears to lose those original leading-type qualia, and in fact in those contexts (iii and I7) scale-degree 7 tends to resolve downwards. Thus, one question which will be addressed in this dissertation is how the role of context might shape certain aspects of musical qualia Rhythm Qualia While most theories of musical qualia have focused on scale-degree, aspects of rhythm, meter, and timing also have much to contribute to the dialogue of musical qualia, although, their effects are rarely referred to as qualia, but rather, are more likely to be described using terms such as groove, nuance, and feel. Roholt (2014), for example, devotes an entire book to the phenomenology of rhythmic nuance, and while he avoids the term qualia, he nevertheless grapples with the problem of ineffability. Roholt, like Raffman, believes that musical nuance is ineffable, and argues that the feel of a rhythm is something that arises only through our 21

35 embodied engagement with the music, or, more specifically: the feel of a groove is the affective dimension of the relevant motor-intentional movements (p.105). London (2016), however, criticizes Roholt s logic, arguing that ineffability is not necessary to his claim of embodied cognition: If I am aware of the extent to which a groove is pushing or pulling (it can be pushing a little or a lot, violently and jerkily, or steadily and so forth), then my sense of that groove/nuance can be fairly determinate and hence effable. Many scholars have written extensively about the nature of embodied cognition and its role in music, and especially with regards to our phenomenological experience of rhythm and meter (e.g. Abraham, 1995; Iyer, 2002; Thaut, 2008; Grant, 2010; Janata et al., 2012; Schiavio & van der Schyff, 2016). In addition, many works writing about rhythmic nuance or groove, typically focus on the role of microtiming, meaning tiny deviations from a metronomic, square pulse (e.g. Iyer, 2002; Fruhauf et al., 2013; Roholt, 2014). (Roholt, for example, argues that it is precisely these deviations in microtiming that create groove. ) However, while bodily motion and expressive timing likely both play an important role in our experience of rhythm, neither of these factors are investigated in this dissertation. As such, further review of this literature here would only dilute the aims of this chapter. Of course, microtiming is only one contributing factor to our phenomenal experience of rhythm. And knowing that the notion of groove relates to a desire for physical movement (Janata et al., 2012), does not elucidate what it is about the music that provokes movement. Indeed, London (2016) points out that both Roholt (2014) and Janata et al. (2012) remain agnostic as to what groove actually is; they do not, for example, analyze the most and least groovy songs in their survey to determine the 22

36 structural requirements for grooviness. In addition, while the study of microtiming and nuance can give us insight into the differences between multiple performances of the same music (with the same rhythms), presumably two pieces composed with completely different rhythms will have a much larger effect, or perceived change, compared with changes in microtiming. That is, the study of microtiming and nuance is necessarily a study of musical minutiae, and, while a fascinating topic in and of itself, it is curious that more attention has not been given to the study of the perceived differences between more obvious changes in rhythm; namely, the perceived effects of different rhythms and meters. Aristides Quintilianus, writing as far back as the third century, attempted to describe the feel and affective influence of various rhythms, even prescribing (or warning against) certain rhythms for ethical purposes: a quieting of the heart...useful in war dances...sacred...the most healthful...bring the heart into not a little disorder...pull against the soul...fearful and deadly...orderly and manly...indulgent...lowly and ignoble...stimulating to actions...supine and flabby... (Matheisen, 1983). It would seem, then, that indeed different rhythms (and meters) give rise to different qualitative experiences. Thus, this dissertation will, in part, investigate the variety of qualia (if any) evoked by rhythm and meter. The topic of rhythm and meter will be given a more complete introduction in Section Chapter Summary This chapter has provided a brief explication of qualia as a term with origins in philosophy. The topic of qualia features prominently in current discussions in the philosophy of mind, as it is commonly used as a central argument in claims about the mind-body problem and the nature of consciousness. While those debates are beyond 23

37 the scope of this dissertation, there are commonly-made claims about qualia that prove problematic when applied to certain aspects of music such as scale-degree and rhythm. In this chapter, those claims were pointed out, and shown to be incompatible with known properties of certain musical qualia (especially scale-degree qualia), which poses a problem for musical qualia as an object of empirical study, as well as posing (yet another) problem for philosophical arguments about qualia, which frequently define qualia as ineffable, non-representational, and intrinsic. An operational definition of qualia is proposed, following a similar stance taken by both Dowling (2010) and Zentner (2012). In particular, a crucial distinction between my stance on understanding qualia in comparison with traditional philosophical approaches, is that I do not propose to uncover or capture the essence of what constitutes the purely phenomenological component of some experience, but rather, I propose that various top-down and bottom-up factors, including conscious and unconscious memory (implicit knowledge), conceptual knowledge, emotional and physiological reactions, can all contribute to the resulting perceived experience. In addition, I propose, like Zentner, that language can be useful as a means of communicating common reactions to the same stimulus, and that this can provide converging evidence in support of some component that might either contribute to the qualia, or help describe the overall qualitative experience. In this dissertation I aim to uncover some common ground (or lack thereof) for the perception of both scale-degree qualia and rhythm qualia. 24

38 Finally, this chapter has provided a brief background on the work of others who have proposed some theoretical explanation for the origin of musical qualia, in particular, the role of statistical learning (or implicit knowledge) as an important component in the shaping of our musical experiences. In the following chapter, I begin by first investigating the functional and resolutional properties of scale-degrees from a statistical perspective. 25

39 Chapter 3: A Corpus Study Abstract Probabilistic models have proved remarkably successful in modeling melodic organization (e.g. Pearce, 2005; Huron, 2006; Temperley, 2008). However, the majority of these models rely on pitch information taken from melody alone. Given the prevalence of homophonic music in Western culture, however, little attention has been directed at exploring the predictive power of harmonic accompaniment in models of melodic organization. The research presented here uses a combination of three main approaches to empirical musicology exploratory analysis, modeling, and hypothesis testing to investigate the influence of the harmonic accompaniment on melodic behavior. In this study a comparison is made between models that use only melodic information and models that consider the melodic information along with the underlying harmonic accompaniment to predict melodic continuations. A test of overall performance shows a significant improvement using a melodic-harmonic model. When individual scale-degrees are examined, the major diatonic scale-degrees are shown to have unique probability distributions for each of their most common harmonic settings. That is, the results suggest a robust effect of harmony on scale-degree tendency. If scale-degree tendencies originate in part from their statistical probabilities, then the finding that these probabilities are mediated by the harmonic context suggests that the qualia they elicit will differ depending on the supporting harmony. Research has suggested that statistical learning plays a substantial role in forming musical expectations (e.g., Krumhansl, 1990; Huron, 2006; Temperley, 2007). Given that implicit knowledge can arise from probabilistic exposure to sequences and patterns (Saffran et al., 1999; Romberg & Saffran, 2010), it is appropriate to begin 26

40 an investigation of musical expectations by examining the statistical properties of melody. (For a review of implicit and probabilistic learning see Reber, 1993). The majority of these models focus on the evaluation of information taken from the melody alone (e.g., pitch height, interval size, interval direction, etc.). However, research has also established the importance of rhythm and phrasing in contributing to melodic expectation. For example, rhythmic information can help predict the location of phrase endings (Palmer & Krumhansl, 1987; Krumhansl & Jusczyk, 1990; Jusczyk & Krumhansl, 1993; Krumhansl, 2000), and pitches located near phrase endings will have an increased probability to move towards their note of resolution (Aarden, 2003; Pearce, 2005). Given the prevalence of homophonic music (i.e., melody with accompaniment) in Western musical cultures, an obvious avenue for further exploration in models of melodic organization would be that of harmonic context. In other words, in modeling melodic expectancy, it may prove beneficial to examine melody not just as isolated lines, but as lines embedded in a harmonic context. Accordingly, this chapter investigates the role of the supporting harmonic accompaniment in shaping melodic organization. Taking a probabilistic approach, in this study a model is constructed that combines harmonic information from the accompaniment with first-order melodic information from a digital corpus of encoded musical scores from the common practice period. This model is compared to a model that uses solely first-order melodic information in order to investigate the effect of harmony on the predictability of melodic continuations. To anticipate the results, it appears that melodic prediction is significantly improved when harmonic information is taken into account. Following the overall model comparison, an exploratory analysis considers unique interactions 27

41 of harmony and scale-degree, in order to examine the specific effect of the former on the latter. 3 There are many ways to quantify and dissect melodic information. In cognitive models of musical expectation in particular those based on Narmour s Implication- Realization theory (1990; 1992) pitch information is tallied, but also the specific interval and direction from pitch to pitch. The current approach, however, is based on ideas of statistical learning proposed by Huron (2006) where only first-order scaledegree information is considered. Using scale-degree, like pitch-class, assumes octave equivalence and removes information about the size and direction of an interval. This means that the model does not distinguish between, for instance, a major second and a minor seventh. However, one should bear in mind that the possible interval and direction between two scale-degrees has only two options (ignoring compound intervals, which are extremely rare in melodies). Furthermore, it has been shown that small melodic intervals are much more common than large ones (Ortmann, 1926; Merriam et al., 1956; Dowling, 1967; Huron, 2001; Temperley, 2008). Thus, most scale-degree successions will represent the smaller of the two intervallic possibilities. As will be discussed in further detail in the following sections, harmonic information in this case refers to a reduction of the accompanimental texture to a Roman numeral. This approach has several limitations which should be acknowledged; such as the reliance on a single (subjective) analysis, the marginalization of voice-leading practices, and the insertion of a presentist bias which discards relevant stylistic 3 Notice that the causal influence between melody and harmony can go in both directions: That is, melody might be expected to affect harmony as well as harmony affecting melody. For the purposes of this chapter, however, only the probabilities relationship of harmony on melody will be investigated, leaving aside the reverse analysis for another occasion. 28

42 information (Gjerdingen, 2014). In this case, however, using Roman numerals affords a simple method that significantly reduces the number of necessary comparisons. It should be emphasized that the ensuing model of melodic probability is not being presented as an optimal model of melodic prediction. Any realistic model will certainly consider more than first-order scale-degree successions and a crude harmonic analysis. Rather, the goal is to learn whether the harmonic information in the accompaniment can improve the predictive power of a model of melodic continuation. The motivation for this research is to better understand melodic tendency as it functions in a (relatively) ecologically valid context. If scale-degree tendencies originate in part from their statistical probabilities, and the probability of any given melodic event turns out to be largely dependent on the underlying harmonic context, then their qualia would be expected to differ depending on that context. The hypothesis tested here is that harmonic context plays a sizable role in shaping melodic behavior. This hypothesis is tested using a statistical (i.e., probabilistic) approach. 3.1 The Corpus Analysis Overview The ultimate goal of the project is to better understand the relationship between harmony and melody; more specifically, to identify possible influences of the supporting harmonic context on melodic organization. The approach taken here is a strictly probabilistic one. That is, the probabilities of melodic continuation are evaluated in two conditions: the first examines melody in isolation, the second examines melody along with the underlying harmony in the accompaniment. In this way, if 29

43 the probabilities of melodic continuation are significantly altered in the latter condition (where underlying harmony is taken into account) then one can infer that the melodic behavior is influenced by the harmonic context. To this end, a musical corpus was assembled containing melodies set within unambiguous harmonic contexts. To test the main hypothesis, two models were needed: one with first-order melodic succession probabilities calculated using only melodic information, and another which combined first-order melodic information with the harmonic information taken from the accompaniment of the antecedent scale-degree. In order to have some measure of improvement, a zeroth-order melodic model was also included. Since it is well established that first-order models perform better than zeroth-order models, the zerothorder model provides a baseline against which to calculate each successive model s improvement. Thus, probabilities were calculated for melodic antecedent-consequent pairs of scale-degrees, first in isolation, and then with the harmony supporting the antecedent scale-degree of the consequent tone, in order to investigate whether the harmonic support of the antecedent note can help predict the melodic behavior of the consequent. In brief, the results of the study will show that harmonic context appears to have a strong influence on melodic behavior Sampling In light of the research hypothesis, a corpus of music featuring melodies with harmonic accompaniment was needed. While there are many Western forms of music that have rich harmonic traditions, such as rock, pop, and jazz, classical music makes an ideal jumping-off point for this research, as it forms a large body of notated music that is easily accessible. In addition, the use of classical scores provide a 30

44 convenience sample, since the pitch information for many works is already encoded in digital format. Therefore, this study specifically investigates the statistical properties of classical melodies from the common practice period. The corpus thus excluded material typically considered outside of the bounds of the common practice era, such as 20th century and Renaissance music, as well as solo and two-part works (e.g., Bach inventions) where harmonies are incomplete or implied. Although it can be argued that the common practice period represents a wide range of harmonic practices (for instance, some composers included in the corpus were not yet thinking in terms of triads with roots), it is nevertheless commonly thought of and taught as one coherent tonal system. Given the aims of the study, there were several features that were desirable in the corpus: 1) Voice-leading and use of harmony should be representative of the common practice period; 2) The melody could be easily determined and, as much as possible, clearly distinct from the harmonic accompaniment; 3) Several composers should be represented whose works span the time frame of the common practice period; 4) Both vocal and instrumental styles should be included in roughly equal proportions; and 5) Homorhythmic and non-homorhythmic styles should be included in roughly equal proportions. Note that proportion is in relation to the total number of melody notes, not the number of pieces, since melody tones provide the unit of measure in the study. In assembling the desired sample described above, for practical purposes it was preferred to use any existing digitally encoded scores. A subset of Bach chorales were chosen as a convenience sample, as they had harmonic analyses which were already encoded by other scholars. This meant that the works only had to be checked 31

45 for errors, rather than encoded entirely from scratch. Similarly, the Schubert lieder had melodic information already encoded, and only the harmonic analyses had to be added. The remainder of the corpus was supplemented by consulting Burkhart s Anthology for Musical Analysis (1994), specifically searching for pieces composed between nominally Baroque and Romantic eras (roughly ), preferring those which had clear separation of melody and harmony. In this process, an effort was made to gather works which represented a variety of composers, styles, and instrumentation. The Anthology was an appropriate source since it contains many works featuring harmonic accompaniment by many composers in different styles and instrumentation dating from the common practice period. The corpus used in this study yielded close to 10,000 melody notes and was comprised of 68 pieces, including: 50 Bach Chorales, 1 Haydn string quartet, 2 Mozart sonata movements, 3 Beethoven sonata movements, 6 Schubert Lieder, 3 Schumann piano miniatures, 1 Clementi Sonatina, and 2 Mendelssohn Songs without words. (See Appendix A for the complete list of works included in the corpus). All musical information in the corpus was encoded in Humdrum format (Huron, 1995). All pieces (aside from the Bach chorales) were harmonically analyzed and encoded by the author. Details of the analytic procedure are outlined in the following section. It should be mentioned that while the corpus may appear to be somewhat biased due to the overrepresentation of Bach chorales, the chorales are much shorter in length than the remaining pieces, meaning that, in fact, the chorales do not constitute the majority of notes or measures in the corpus (e.g., the chorales constitute a total of 726 measures; the remaining pieces make up 1306 measures.) This means that, bar-for-bar (and note-for-note), there may in fact be an overrepresentation of non-homorhythmic 32

46 music, where melodies commonly continue over an unchanging harmony. However, given that the majority of music in the classical genre is not homorhythmic, this slight imbalance may prove more representative of melodic expectations in general Methodology In assembling the corpus, the goal was to be able to compare adjacent slices of the musical texture. Since melodic tendency was the primary area of interest, each slice would be one melodic tone (or attack), regardless of duration. Two models were created which both consider a prior musical state to predict the subsequent melodic tone. The first model looks only to the prior melodic note to predict its successor; the second model (referred to as the harmonic model) looks to the prior melodic tone and that tone s harmonic accompaniment to predict the successor. Of course, most melodic tones in the corpus do not coincide with an onset in the harmonic accompaniment. In order to best represent the harmonic information present in the accompaniment at any given point in a melody, every melodic slice is assigned a Roman numeral based on the most recent sounding harmony present in the accompaniment. As will be explained in the paragraphs below, determining scale-degrees and Roman numerals relies on the determination of a key, which was resolved via human analysis (as opposed to automatic methods such as feature extraction.) Furthermore, apart from the chorales which have clear harmonic onsets (i.e., homorhythmic vocal entrances), the remainder of works in the corpus typically have arpeggiated (or non-homorhythmic) harmonic textures. In these cases harmonic labels were assigned based on the harmonic rhythm of the accompaniment. Currently, the most accurate method for extracting information about harmonic relationships from homophonic 33

47 musical textures (in classical music) is via manual analysis. While such methods are necessarily subjective, the level of difficulty posed by the analysis of these works was trivial, suggesting that error rates would be minimal. In addition, although works were not analyzed in duplicate, a recent collaborative project by the author involving duplicate Roman numeral analyses on a similar data set suggests that the level of discrepancy between subsequent analyses would have been minor (Devaney et al., 2015). 4 Since the primary purpose of this study is to investigate the influence of harmony on scale-degree behavior, it is imperative to limit the number of variables in order to have enough data to be able to make generalizable conclusions. As such, certain decisions were made to simplify the harmonic context. For instance, there may be a difference in melodic behavior for scale-degree depending on whether the supporting harmony is in root position or not. However, classifying all harmonies separately based on both Roman numeral and inversion would divide the data too much, reducing statistical power and making it more difficult to discover patterns. Thus, in the analysis all inversion information was collapsed to root position. Although it is possible that the bass tone may be a more accurate predictor than the generic triad, as a preliminary corpus study of homophonic music, a simpler model using fewer parameters was preferred over a complex one to start with. In the same vein, rhythmic information, such as the duration of melodic tones or chords, was removed. Other simplifications included collapsing seventh chords together with triads. For example, instances of V7 and V were all labeled as V. It should be noted, however, that since 4 In fact, the texture in the TAVERN data set was more complex than the textures in the present corpus, and discrepancies between analyses were still minor. The most common discrepancies involved questions of inversion (typically during complex left hand passages) and tonicization versus true modulation. 34

48 the scale-degrees were tallied and paired with their associated harmonies, it becomes clear when a given scale-degree is functioning as a harmonic seventh. Like any empirical experiment, corpus studies must also operationalize concepts in order to render the hypothesis testable. However, operationalizing the various parameters raises a number of complicated issues, which are described in the following paragraphs. The Problem of Multiple Contexts In this study, scale-degrees and harmonies are examined in their musical contexts. This means both in a vertical context (i.e., which scale-degree sits above a given harmony?), and a horizontal context (i.e., which melodic event precedes another?). This simple scenario is complicated by the presence of rests, since it would not be beneficial to count every instance where a scale-degree proceeds to a rest (i.e., there would be too many). If rests were to be removed, however, care must be taken to preserve the synchronization of the melodic and harmonic information. In order to preserve the original melodic-harmonic alignment, other musical information would need to either be deleted, or added, or both. Thus, rests were treated in the following way: Rests were removed whenever they were present simultaneously in both melody and accompaniment. For rests comprising less than one measure, the melody note prior to the rest was copied, as if it were present for the remainder of the measure. This procedure of extending harmonic or melodic material was preferred over deleting material, since it was decided that the latter would create discontinuities in the analysis of sequential progressions. Rests of less than one measure make up the majority of rests in the corpus. Thus, two notes with a rest in between are tallied such that the first note is understood to proceed to the second note. Rests (in either the melody or accompaniment) greater than one measure in duration were dealt with 35

49 on an individual basis. For instance, the majority of longer rest periods were due to breaks in the melodic lines of Lieder, such as during introductions, codas, or pauses between phrases. If this type of break consisted of a complete solo phrase in the accompaniment, and did not involve any transition or change of key, then the entire accompanimental phrase was deleted. Often a break comprised only part of a phrase, where the melodic line was passed to the accompaniment, in which case the material was not deleted and the melody could be traced. Using this procedure, harmonic and melodic alignment was preserved as best as possible. Another challenge was how to handle repeating sections of music. For every section of music that repeats, should it be encoded (and thus tallied) once or twice? The most common scenario was that of repeat signs written into the score. However, there were some pieces with repeats that were written-out. The procedure for handling repeats was as follows: ignore repeat signs but include repeated musical segments that are written-out, unless the written-out portion comprises more than eight measures. Given that most phrases are four or eight measures, it seemed that eight measures would be sufficiently long to avoid discarding most written-out repeated segments. The rationale for ignoring repeat signs was that long segments of music that are typically repeated (such as the entire A and B sections of many binary forms, or complete sonata expositions) would occupy a disproportionate volume of the corpus, thereby reducing data independence and potentially biasing the data. The Problem of Modulation and Tonicization In the musical analyses for this study, melodies are represented as scale-degrees. Since scale-degrees rely on a reference to a tonic, it is important to reflect, as accurately as possible, the implied key at each moment. Of course, in real musical 36

50 composition, key determination can be rather complicated. Music theorists typically distinguish three categories of key migration: modulation, tonicization, and temporary application of a chord (or chords) from outside the primary key area (Laitz, 2008). With modulation, there may or may not be an explicitly notated change of key in the score. Furthermore, what sets modulation apart from tonicization is not always clear. Although not all theorists agree on the criteria for classifying key changes under one label or another, for the purposes of this analysis it was decided that in order for the scale-degrees to accurately reflect their most appropriate harmonic context, and to maximize consistency, there should be a systematic procedure in place. Thus, the working definition of key change for the purposes of this analysis was as follows: for any segment with four or more harmonies in a row (regardless of harmonic rhythm) that applied to a secondary key area, a key change was deemed to occur. For example, Table 3.1 shows the following arrangement of harmonies: The section begins in the original key of D minor, but a C7 harmony appears as a repeated applied dominant to F (III) after which it appears to go back to D minor (at A7-Dm), but then continues in F. According to the four-harmonies-in-a-row protocol, the resulting Roman numeral analysis is given in the right-most column of Table 3.1 below. Note this means that, in the example below, the first appearance of A7 Dm are given different Roman numeral encodings from when they return towards the end of the passage, where the same chords are coded as applied chords in F. One other problem arises in the process of changing keys; wherever a key change occurs, it will affect the flow of both Roman numerals and scale-degrees. Thus, when pitch classes are represented as scale-degrees, and compared in their horizontal 37

51 (antecedent-consequent) context, a problem arises at key-change locations. Table 3.1: Illustration of Encoding Key Changes. An illustration of how key changes were operationalized for the purposes of encoding the harmonic progressions of complete pieces that included applied chords, temporary tonicizations, and modulations to other keys. When four or more harmonies in a row belonged to a secondary key area, they were labeled as local chords in the secondary key. If there were less than four chords belonging to a secondary key, they were labeled as applied chords. Chord R.N. in key of Dm R.N. used in analysis Dm i i Eø7 iiø7 iiø7 A7 V7 V7 Dm i i Eø7 iiø7 iiø7 (viiø7) *key of F C7 V7/III V7 F III I C7 V7/III V7 F III I A7 V7 V7/vi Dm i vi BbM7 IV7/III IV7 F III I C7 V7/III V7 F III I Consider a scenario represented in the two illustrations below, with each column representing a measure of music, where the third measure (C7) initiates a key change from D minor to F major. D Minor: F Major: Melody: A, Bb, A, F, A G, A, G, E, D C, Bb, A, G A Chords: Dm Eø7 C7 F 38

52 D Minor: F Major: Scale-degrees: 5, b6, 5, b3, 5 4, 5, 4, 2, 1* *5, 4, 3, 2 3 Roman Numerals: i iiø7* *V7 I As indicated by the asterisks (*), (and despite that, by chance, the resulting Roman numeral progression is a logical one), this analysis does not accurately represent the melodic nor the harmonic progression, since the Roman numerals and scaledegrees asterisked do not belong to the same key. To remedy this, at the point where the music changes key, the harmony (and scale-degree) immediately prior to the key change is notated as a pivot chord regardless of whether it would be interpreted as such by a music theorist and then given a boundary separation, followed by a second representation (as a distinct Roman numeral) in the second key area. In this way, false progressions can be ignored (i.e., skipped over) in the process of tallying the first-order melodic and harmonic contexts. Take again the following progression from Table 3.1, where the half-diminished ii chord pivots into a vii chord: D minor: i iiø7 *F major: viiø7 V7 I A side-by-side sliding window of antecedent-consequent pairs of harmonies and scale-degrees can thus be represented as shown in Table 3.2. Rows showing the chords or scale-degrees to be ignored in the analysis are shown with asterisks. By preceding key changes with a notated pivot chord, and ignoring antecedent-consequent pairs which cross a key boundary, the correct harmonic and melodic contexts are preserved, and spurious progressions are avoided. Although it is recognized that this solution causes the pivot chord to be counted twice once 39

53 Table 3.2: Melody and Harmony Encodings at Key Changes. Illustration of how antecedent-consequent pairs of Roman numerals or scale-degrees were encoded at key changes. Sliding windows show each consequent as it becomes the antecedent. At a key change, a boundary separation is indicated by a dash, and the pivot chord (or scale-degree) gets reinterpreted in the second key. Rows with dashes are skipped in the tallying process so that illogical progressions (chords or scale-degrees from different keys) are never counted. Harmony Antecedent Consequent i iiø7 *iiø7 viiø7 V7 V7 I Scale-degree Antecedent Consequent 5 b6 b6 5 5 b3 b * with each label the ratio of pivot to non-pivot chords in the corpus is sufficiently small that the chord duplication should not raise cause for concern. Furthermore, this method appears to be the best way of accurately capturing the antecedentconsequent harmonic and melodic relationships as they relate to the implied key at any given moment without unnecessarily discarding information. Thus, preservation of immediate key structures was given priority over preservation of higher-order key interpretations. 40

54 3.2 Evaluating the Models Global Hypothesis Test The primary research question asks: is melodic tendency dependent on harmonic context? In order to test this hypothesis overall, two models predicting melodic continuations were created: The first uses first-order melodic information, and the second uses first-order melodic information plus the harmonic information from the accompaniment of the antecedent melody note. In order to have some measure of the degree of improvement of the harmonic model, a third model was included which uses only zeroth-order melodic information, since it is well established that firstorder models tend to perform better than zeroth-order models (e.g. Pearce, 2005; Huron, 2006; Temperley, 2007). In the model tested, all scale-degrees and harmonies were included from the complete corpus. Each model assigns a probability to each melodic note, and the overall strength of the model can be determined by evaluating the product of all the probabilities. Since the resulting products are infinitesimally small, the standard procedure is to instead take the log of each probability, and then sum them together to generate a log-likelihood. The log-likelihood values are then divided by the length of the model s data set and converted to a positive number. The resulting values are labeled as cross-entropy scores to be consistent with similar techniques in recent literature (e.g. Temperley, 2007). 5 Lower cross-entropy values indicate a better model fit. Using this methodology the following cross-entropy values were obtained: zeroth-order model = 2.22, first-order model = 1.87, harmonic model 5 This use of the term cross entropy may not fit the traditional definition (see Rubenstein & Kroese (2004), p. 29), however, it is consistent with Temperley s usage in Music & Probability (2007). 41

55 = The lower cross-entropy score comparing the first-order model to the zerothorder model demonstrates that the former is more accurate at predicting the melodic continuation. What is relevant to the primary hypothesis is that the harmonic model generates a substantially lower cross-entropy value compared to the first-order model. Notice that the difference in moving from the first-order model to the harmonic model is similar to the difference between the zeroth-order model and the first-order model. Of course, a model with more parameters will always fit equivalent to, or better than, a model with fewer parameters. Thus, there is some possibility that the improved model fit might be due to chance. As such, it is appropriate to perform a statistical test. A log-likelihood ratio test was conducted to compare the first-order model to the harmonic model. This test compares a simpler model with a more complex model by evaluating the difference of log-likelihoods between models. This difference is multiplied by -2 to produce a value known as the deviance, which is known to be χ 2 distributed, with degrees of freedom equal to the difference in the number of parameters between the two models. This ratio test produced a deviance of Unfortunately, determining the degrees of freedom for this test is problematic. One would expect the degrees of freedom to represent the difference in the number of parameters in the two models, which in this case is However, many of these parameters are highly correlated with each other, and furthermore, many of the parameter estimates are zero (i.e., not every scale-degree actually appears with every possible chord type). Consequently, this number exaggerates the degrees of 6 The first order model has parameters: 16 antecedent scale-degrees (e.g., ˆ1 and ˆ2 are considered separately) and 16 potential consequent scale-degrees. The harmonic model has parameters (the same scale-degrees from the first-order model, but with 30 possible chord types). 42

56 freedom. Since a theoretical χ 2 distribution could not be calculated, an empirical distribution was calculated through Monte Carlo computer simulation. Five thousand log-likelihood tests were conducted using the first-order model and a scrambled version of the harmonic model where the harmony paired with the antecedent scaledegree was randomized, producing 5000 null χ 2 values. The distribution of these 5000 χ 2 values were centered around 2350 with the top five percent of values falling within the range of The χ 2 value of 4840 observed using the actual data is far above this range, allowing us to conclude that the improvement of the model is significantly better than chance, even without the precise calculation of a p value. This procedure follows the traditional approach of statistical testing, where a critical test applied to the complete data set determines the likelihood of a given outcome appearing by chance. Modeling techniques in computer science, however, typically use a form of evaluation that reserves a portion of the data set to test the efficacy of the model using the probabilities calculated from the remainder of the data set. Thus, this complementary method was performed as well. The complete data set was divided into five portions, where each fifth was rotated through as a reserve set, and tested against the remainder which acted as the training set. If a parameter (i.e., scale-degree + harmony) in the test set was not encountered in the training set, the probability for that parameter was pulled from the higher-order (i.e., first-order) probability for the scale-degree alone. For each test set this was only necessary for approximately.05% of the data (or 10 in 1813) on average. Cross-entropy values were calculated for each model on each of the five test sets, and an overall value for each model was obtained by taking the average of all five. The cross-entropy values found using this method were: Zeroth-order model, 2.22; First-order model, 1.88; Harmonic 43

57 model, As can be seen by comparing with the earlier test, the cross-entropy values found (and the differences between them) using both testing methodologies are very similar. These global tests show that evaluating a melodic tone along with its underlying harmony allows one to make more accurate predictions about an ensuing melodic tone than simply considering the first-order distributions of melody alone. This is consistent with the main hypothesis. Of course, this overall test does not tell us anything of interest about the specific interactions of harmony and scale-degree, and in particular, how the former might affect the latter. In order to investigate this, the subsequent paragraphs make an exploratory examination of the scale-degree distributions in each model. 3.3 Descriptive Statistics Mode Classification The complete corpus consists of pieces both in major and minor keys. In order to evaluate the summary statistics, a question arises as to whether to keep the corpus as a whole, or divide it into two parts, with one part comprising the pieces written in the major mode, and the other part comprising pieces in minor mode. Of course, major key works can include brief transitions into minor keys and vice-versa, so classifying entire works based on modality may appear to be an arbitrary decision. However, it is possible that the distribution of harmonies and/or scale-degrees might differ depending on the primary modality. Furthermore, if the corpus remains undivided, the tallying of all harmonies will produce an enormous table which may prove difficult to interpret. For instance, there would be three versions of the submediant chord: vi, 44

58 VI, and bvi. This would make it difficult to distinguish, for example, the proportion of harmonies originating in minor key contexts from those that are a result of modal mixture. Accordingly, it was decided to split the corpus into two, based on the primary modality of the piece. Finally, there remained one additional problem with mode classification, even after the division of the corpus into two. A given piece that modulates to its relative key would contain both major and minor tonic harmonies which would be tallied separately (i.e., I in a major key; i in a minor key). However, dominant function chords, and a few other harmonies such as applied chords or Italian 6th chords, share the same symbols regardless of the mode they are employed in. The result of tallying all instances of such chords, is that while the tonic harmonies would be subdivided and grouped by modality, the dominant harmonies would all be pooled together, possibly distorting the true ratio of dominant to tonic function harmonies. As such, harmonies that commonly appear in either mode were specially labeled in the analysis according to the subsequent harmony in the context of the piece. For example, if V is followed by i then that V chord is marked with an additional symbol to indicate that it preceded a minor chord. These V chords would be tallied separately from V chords that were followed by a major chord. In this way, common harmonies with labels that could apply to either the major or the minor mode were distinguished so as to most accurately represent the proportion of harmonies in the corpus. In the end, despite dividing the corpus in two, the total number of harmonies found in either part of the corpus remained unwieldy. In order to simplify the results, harmonies which represented less than one percent of their respective major or minor corpus were omitted from the graphs and tables. Overall, these rare harmonies tended 45

59 to be comprised of: modal mixture chords, Neapolitan chords, uncommon applied chords, and harmonies resulting from modulations to opposite-mode key areas. Lastly, in an attempt to retain as much musical information as possible, applied chords in the form of leading-tone chords were merged with dominant applied chords (e.g., counts of V/vi and viio/vi were pooled together as dominant function applied chords) Zeroth-order Probabilities The zeroth order probabilities (i.e., the overall distribution) of both scale-degrees and harmonies (independent of each other) were extracted from the corpus in order to determine their relative proportions. These are shown in Figure 3.1 and Figure 3.2. Recall that the corpus was divided into two sub-corpora representing the major mode pieces and the minor mode pieces, respectively. Thus, the percentages and counts shown in the figures below are labeled with regards to their respective corpora totals. The x-axis lists all possible scale-degrees (Figure 3.1) or harmonies (Figure 3.2) found in the corpus, and the y-axis represents the proportion of the corpus that is made up by each of those scale-degrees or harmonies. As mentioned above, harmonies making up less than one percent of the corpus have been omitted from the figures. Not surprisingly, neither scale-degree nor harmony follow a simple uniform distribution. Rather, some scale-degrees and harmonies occur more often than others. Specifically, scale-degrees from the diatonic collection appear more often than chromatic scale-degrees. Even within the diatonic collection, the first five scale-degrees (1 to 5) are more common than ˆ6 and ˆ7. This is finding is consistent with pitch distributions reported by other scholars (e.g. Huron, 2006; Temperley, 2007; Albrecht & Huron, 2014). 46

60 Figure 3.1: Zeroth order distribution of scale-degrees in the corpus. Bar graphs show the overall proportions of the scale-degrees as a percentage of the whole that each scale-degree makes up in the major or minor corpus, respectively. The numeric labels above each scale-degree tally the actual number of instances found for that given scale-degree. These tallies demonstrate the under-representation of minor key works in the corpus overall. 47

61 Figure 3.2: Zeroth order distribution of harmonies in the corpus. These charts show the overall proportions of the harmonies as a percentage of the whole that each harmony makes up in the major or minor corpus, respectively. The numeric labels above each harmony give a count of the actual number of instances found for that given harmony. These counts demonstrate the under-representation of minor key works in the corpus overall. 48

62 Similarly, the use of non-diatonic harmonies is mostly outweighed by diatonic harmonies. Interestingly, the use of dominant and tonic chords grossly outweigh all other harmonies combined. This finding is relatively consistent with the work of Budge (1943). 7 Since the tonic chord supports ˆ1, ˆ3 and ˆ5, and the dominant chord supports ˆ5, ˆ7, ˆ2 (and sometimes ˆ4), one might expect, given this abundance of tonic and dominant chords, that ˆ6 ought to be the least common scale-degree. This does appear to be the case in the minor mode corpus, where ˆ6 and ˆ6 are used less frequently than nearly all other diatonic scale-degrees. (The least common diatonic scale-degree is ˆ7, which is not surprising given the common practice of raising ˆ7 to ˆ7 in the minor mode.) However, this is not the case in the major mode corpus, where ˆ7 is the least used, despite the abundance of dominant chords in the corpus. This suggests that composers may be avoiding scale-degree 7 in the melodic line. However, what appears as avoidance may simply be the result of voice-leading preferences; a possibility that cannot be ruled out with the information at hand. In any case, this finding is worthy of further study First-order Probabilities The conditional first order probability for each scale-degree was independently tallied. That is, given some scale-degree x, what scale-degree is likely to follow? For the sake of clarity and brevity, and to ensure sufficient statistical power, the remaining analyses only consider the data from the major-mode portion of the corpus. Figure 3.3 illustrates the percentage of the time that a given scale-degree proceeds to another. The x-axis shows the antecedent scale-degree (i.e., the note of 7 Budge found that all inversions of I and V combined made up roughly 45% of her corpus. 49

63 Figure 3.3: First-order probabilities for scale-degrees. Visual representation of the likelihood of a consequent scale-degree (y-axis), given the antecedent (x-axis). The size of the circle is roughly proportional to its probability (expressed in percent), as shown in the legend. For example, the probability of ˆ2 moving to ˆ3 is 100%, whereas the probability of ˆ1 moving to ˆ2 is 19%. Clustering of large circles around a line with a slope of 1 arises from high probabilities for note repetitions and step-wise motion, consistent with previous literature. 50

64 origin) and the y-axis represents the consequent note. Circle sizes represent the likelihood (expressed in percent) for a given antecedent-consequent pair, with an empty space representing 0% and the largest circle representing 100% likelihood. Note that Figure 3.3 reveals certain musical features that may not be evident from examining purely vocal corpora. For example, the tendency for note repetition in these melodies is quite high, and to a lesser extent, the tendency for arpeggiation (moving between scale-degrees that are a third apart). A line with a slope of 1 represents note repetitions, with clustering around that line by 1 representing motion by step. Note also that the scale-degrees with the most predictable melodic continuations are those that music theorists would classify as tendency tones (e.g., scale-degrees 2, 4, and 5 all have 70% or higher probabilities of moving upwards by one semitone). This is in part because there are relatively few instances of these scale-degrees in the corpus overall, nevertheless, they do have highly predictable behavior. This high predictability can be seen by looking at the vertical spread of each scale-degree. The diatonic scaledegrees show more spread (i.e., more possibilities for melodic continuation) compared with the non-diatonic tones which show less spread. Scale-degree 2, for instance, shows all 19 occurrences move upwards to scale-degree 3. (Refer to Figure 3.1 for numeric counts of scale-degrees and their distributions). Finally, this analysis is consistent with existing literature that shows step-motion and repetition to be more common than motion by leap (Ortmann, 1926; Merriam et al., 1956; Dowling, 1967; Huron, 2001; Temperley, 2008). The trend for step-wise melodic motion can be seen in Figure 3.3 by the clustering of larger circles around the diagonal. Note also that, 51

65 aside from the tendency tones which all move upward, there is slightly more clustering below the diagonal than above it, implying that downward step motion is slightly more common than upwards step motion overall. Notice that the zeroth- and first-order distributions of scale-degrees provide useful null distributions against which harmonically informed melodic practice can be contrasted. By comparing the distributions of purely melodic scale-degree continuations with those of scale-degrees set in a harmonic context, we can see what effect the harmonic accompaniment has on the likelihood of the ensuing melodic tone. In other words, the zeroth and first order probabilities of melody alone provide a sample of predicted melodic behavior which is then compared, using the melodic-harmonic data, with the observed melodic continuation for each scale-degree in their most common harmonic contexts Change in Melodic Probabilities when Harmony is Considered In order to facilitate interpreting the results, Figure 3.4 shows an enlarged portion of Figure 3.5. Here, the predicted melodic continuations for scale-degree 1 are compared with the observed continuations for all scale-degree 1s when supported by tonic harmony (I). The y-axis shows the probability of occurrence that the given scaledegree (in this case, ˆ1) will proceed to any of the diatonic scale-degrees, listed along the x-axis. The shaded bars represent the predicted, or expected, melodic behavior (again, from considering only the first-order melodic information), whereas the white bars represent the observed melodic behavior when the given chord (in this case, I) is supporting the antecedent scale-degree. 52

66 Figure 3.4: Predicted and observed probabilities for ˆ1 in tonic context. Probabilities for both the predicted and observed melodic continuations for scaledegree 1. The predicted probabilities, shown with shaded bars, are taken from the first-order distribution of melody alone. The observed probabilities, shown in white bars, are calculated from the first-order conditional probabilities of scale-degree 1 when it is embedded in a tonic (I) harmonic context. Reported sample size represents the total number of instances for the observed condition. The p value indicates the results of a χ 2 test (see footnote 8) comparing the predicted and observed distributions (df = 6). 53

67 Figure 3.4 shows that there is a statistically significant difference between melodic probability distributions for scale-degree 1 when embedded in a tonic harmony context compared with the corresponding melody-only distribution. Specifically, the melodyonly distribution predicts that ˆ1 is most often followed by a repeat of ˆ1, with ˆ7 and ˆ2 being the second- and third-most likely consequent scale-degrees, respectively. When scale-degree 1 is embedded in a tonic harmony context, however, the probability for a repeat of ˆ1 increases, and there is a decrease in probability for continuing to either ˆ7 or ˆ2. In fact, the observed distribution shows a roughly equal probability of moving to scale-degrees 7, 5, 3, or 2. Figure 3.5 shows the predicted and observed continuations for all diatonic scaledegrees in the major mode. On the far left column, Arabic numbers represent the antecedent scale-degree. Each row of graphs examines the behavior of that single antecedent scale-degree in three different harmonic contexts. For instance, the first row of graphs shows the probabilities of the different melodic trajectories for scaledegree 1 when either I, vi, or IV is the supporting harmony. Using this graph, one can see that ˆ1 is more likely to ascend to ˆ2 when it is supported by IV or vi than when it is supported by a tonic harmony (I). A more dramatic example can be seen in the graphs for scale-degree 6, where it appears far less likely to move down to ˆ5 when supported by submediant harmony (vi). Some unexpected findings come from examining these graphs. For instance, in examining the probable continuations for scale-degree 2, we can see that ˆ2 is most likely to repeat when supported by ii. (This is not surprising given the propensity for ii to move to V, and that ˆ2 is a common tone to both harmonies.) However, we find this same likelihood for ˆ2 to remain stationary when supported by vii. In 54

68 Figure 3.5: Effect of Harmony on Scale-degree Probability (continued on next page) 55

69 Figure 3.5: Effect of harmony on scale-degree probability. Each graph shows the observed and predicted distributions of melodic continuations for a given antecedent scale-degree in three different harmonic contexts: as the root, 3rd, or 5th of a given harmony. (An exception is scale-degree 4, where V replaces vii.) The y-axes show the probability of occurrence that the given scale-degree will proceed to any of the diatonic scale-degrees, listed along the x-axes. Shaded bars represent the predicted melodic probabilities (using melodic-only data), and white bars represent the observed (from the melodic-harmonic data). A missing graph indicates that there were fewer than 50 observations in the given context, and so a test was not performed. The counts represent the total number of instances for the observed conditions. p values indicate the results of χ 2 tests (see footnote 8) comparing the predicted and observed distributions (df = 6). Non-significant values are shown in parentheses. 56

70 comparison, ˆ2 is most likely to descend to scale-degree ˆ1 when paired with dominant harmonic support (V). Given that vii and V have the same harmonic function (i.e., dominant function), one might expect the distributions for scale-degree 2 supported by vii or V to look similar, yet they do not. Likewise, the consequent behavior of scaledegree 7 supported by either vii or V again show differing distributions despite the fact that these chords share the same harmonic function and tendency for resolution. However, the observed distributions for ˆ2 and ˆ7 in these dominant function contexts are dramatically different from each other. In this latter example, ˆ7 over vii has more than a 60% chance of moving to ˆ1, the highest probability found in the results, whereas ˆ2 over vii has only about an 8% chance of proceeding to ˆ1. Recall that all inversion information was discarded from the model, and that seventh chords were collapsed into triads. Although it is possible that this is a result of unequal sample sizes, it suggests that for the melodic succession ˆ7 ˆ1, scale-degree 7 may be more likely to be supported by vii (or some inversion of vii or viio7) while for the melodic succession ˆ2 ˆ1, scale-degree 2 may be more likely to be supported with V (or some inversion of V or V7). A series of χ 2 tests were performed comparing the first-order melodic distribution of each diatonic scale degree, with the first-order conditional distribution of that scale-degree in three different harmonic contexts. 8 The p values are reported on each graph, with non-significant values shown in parentheses. If there were less than 50 8 The counts for the melody-only condition far outweighed the melody+harmony condition. In order to compute a χ 2 test, the total counts from the expected and observed distributions must match. The solution was to apply the overall proportions from the melody-only (predicted) distribution to the total count from the melody+harmony distribution thereby reducing the counts for the melody-only condition while retaining its original proportions. In this way, the expected and observed distribution totals were matched. For example, if the melody-harmony count was 100, and the melody-only condition was 1000, a hypothetical distribution of 100, 200, 300, 400 would be reduced to 10, 20, 30 and 40, respectively. 57

71 observations for a scale-degree in a given harmonic context, the graph was omitted and no tests were conducted. As evident in Figure 3.5, when the supporting harmony is taken into account, the observed melodic behavior differs significantly from what was predicted in 15 out of 19 cases. Of course, the tests themselves are not particularly important given the global test already conducted. Rather, the graphs were meant to provide an illustration of how the underlying harmony can impact the movement of specific scale-degrees. Since these tests follow the main hypothesis test and are exploratory in nature, the p values reported in Figure 3.5 have not been corrected for multiple tests. 3.4 Conclusions This chapter presented a study in which melodic trajectories were examined empirically through use of a classical music corpus. Compared with other corpus-based approaches to the study of melodic probability, this study looked beyond the surface features of the melody itself and considered the impact of the harmonic accompaniment on the likely trajectory of a melodic tone. It was found that observing a melodic tone within its harmonic context conveys significantly more information about its continuation than looking at the melody in isolation. This is consistent with musical intuition. In order to evaluate the main hypothesis, three models were created to test whether harmonic information contributed significantly to the successful prediction of melodic continuations. The first model used only zeroth-order information to predict melodic continuations, the second model used first-order information from melody alone, and the third model combined the first-order melodic information along with 58

72 information about the harmonic support of the antecedent scale-degree. In each model the cross-entropy (Temperley, 2007) was calculated. The cross-entropy values decreased significantly with each subsequent model, with the harmonic model showing a substantial decrease in cross-entropy. The improvement in moving from the first-order (melodic) model to the harmonic model is roughly comparable to the improvement between the zeroth-order model and the first-order model, suggesting that harmony is playing a sizable role in predicting melodic continuations. In addition to this global test of the overall effect of harmony on melodic behavior, further analyses were carried out in order to investigate the effects for each diatonic scale-degree. Specifically, a distribution of predicted consequent melodic behavior was calculated for each diatonic scale-degree based on the purely melodic information from the major-only portion of the corpus. Each of the predicted distributions was then compared against an observed consequent distribution for each diatonic scale-degree in three different harmonic contexts, such that the given scale-degree was either the root, third, or fifth of the supporting harmony. These graphs indicate the specific changes in probability for each diatonic scale-degree under the different harmonic accompaniments. Many of the observed contexts had a small number of occurrences in the corpus, yet, when tested, many of those contexts produced very small p values, suggesting that the effect size may be quite robust for certain scale-degrees in certain harmonic contexts. Once again, the findings from this study are consistent with the notion that melodic organization is not independent of harmonic support. Several caveats should be reiterated regarding the methodology for analyzing the corpus. As mentioned in Section 3.1.3, several decisions had to be made about how to best encode the musical analysis, given that these are complex works that contain 59

73 tonicizations, temporary applied chords, and modulations to secondary keys. The works also contain a fair amount of repetition, which might be expected to reduce the data independence. The methods for analyzing and encoding necessarily involved making decisions such as including or discarding sections that were repeated, or how to encode and represent harmonies that bordered two different key areas. Of course, these decisions are subjective and therefore open to question. However, although some decisions were made in order to minimize the segregation of data and maximize power such as pooling Roman-numerals regardless of inversion the majority of decisions were made with the intention of how to best represent the music as one might hear it in real time. It is worth noting that in any large-scale analysis of complex corpora, this decision-making process is inevitable, and some interpretive decisions must be made in order to reach the final stage of testing the hypothesis. Although music theorists may find the results of this chapter unsurprising, there is a wealth of literature in which conjectures are made about melodic expectations, statistical learning processes, etc., where typically the only musical parameter examined is melody in isolation. Furthermore, some might imagine that all the harmonic tendency information is contained in the melody alone, and therefore that considering the harmonic accompaniment does not add much information. However, the results of this study suggest that is not the case, as measured by the relative changes in log-likelihood from the zeroth, to first, to harmonic model. It is acknowledged that, in terms of modeling, using only one musical parameter (i.e. melody) makes the task at hand substantially easier. Furthermore, there are now several accessible corpora which are monophonic (e.g., Essen Folksong Collection), and therefore offer researchers a convenient sample from which to test a theory or build a model about 60

74 melody. This chapter hopefully will not only contribute to the existing body of literature on melodic expectation, but also support existing research (e.g. Aarden, 2003; Pearce, 2005; Albrecht & Huron, 2014) promoting the point of view that melodic distributions do not come in one size fits all, and that factors such as rhythmic duration, metric position, phrase position, and even historical period, all contribute information to melodic expectancy. Although often challenging as demonstrated by my own collapsing of information in the corpus the more parameters that one can include, the more accurate those distributions will be. 3.5 Discussion Given these findings of the effect of harmonic context on the organization of melody, 9 a logical next step would be to consider the relative weightings of factors such as bassline (or chordal inversion) and voice-leading. Certainly voice-leading and chord doubling play an important role in governing the behavior of melodic tone succession, and it would be useful to know how important they are in terms of predicting melodic successions. For instance, perhaps the finding that scale-degree 7 appears to be avoided in the melodic line partly arises from the voice-leading principle which warns against the doubling of tendency tones. Or, take as another example the finding that scale-degree 6 is far less likely to move to scale-degree 5 when supported by submediant harmony: If submediant harmony frequently moves to dominant harmony, the avoidance of ˆ5 as a consequent tone in this context may arise from an 9 Note that this was a correlational study, and as such it might be thought that the melody could be equally influencing the harmony. However, recall that antecedent information was used to predict consequent information. As such, while an influence of the consequent on the antecedent is still possible, it is less plausible, since we typically assume that composers don t write melodies in reverse. 61

75 avoidance of parallel fifths. Unfortunately, in this study key pieces of information such as that of chordal inversion were discarded in the process of simplifying the musical texture in order to examine the main hypothesis. As such, questions pertaining to the importance of basslines and voice-leading cannot be tested here. Nevertheless, the role of voice-leading implied by the findings mentioned above warrant future study. Having found these statistical effects, it would be appropriate to test the perceptual effects of harmony on melodic continuations. As mentioned, the differences found in the exploratory analysis suggest that there may be moderate effect sizes for some scale-degrees in certain harmonic contexts. If this is true, and the sample used in the corpus is sufficiently representative of classical music, then perceptual effects would likely be quantifiable. Given the evidence in support of statistical learning, these findings suggest that listeners may be sensitive to the influence of harmony on melodic continuations. That is, if a given scale degree regularly and consistently tends to move (in the context of real music) in a particular way when framed with harmonic support x, but not y, then a classical listeners expectations could be tested, for example, with the use of reaction-time studies. More importantly for the present study, if a particular scale-degree appears to carry different probabilities for resolution depending on the harmonic framework, then a related question arises, which is: can a melodic tone elicit a different feel or qualia in a listener when it is framed in different harmonic contexts? We turn next to this question in Chapter 4 62

76 Chapter 4: A Perceptual Study of Scale-degree in Context Abstract A perceptual study investigated the ability of scale-degrees to evoke qualia, and the impact of harmonic context in shaping a scale-degree s qualia. In addition, the following questions were addressed: What role does musical training have in shaping qualia? Are listeners consistent in their descriptions? Are experiences similar across participants, or are they individual and subjective? Listeners with or without music-theoretic training were asked to rate the qualia of scale-degrees following various chord progressions, each ending with a different final harmony. Scale-degrees were found to exhibit relatively consistent musical qualia; however, the local chord context was found to significantly influence qualia ratings. In general, both groups of listeners were found to be fairly consistent in their ratings of scale-degree qualia; however, musician listeners were more consistent than non-musician listeners. Finally, a subset of the musical qualia ratings were compared against Krumhansl and Kessler s (1982) scale-degree profiles. While profiles created from the present data, overall, were correlated with the K&K profiles, their claim that tonal stability accounts for the high ratings ascribed to tonic triad members was found to be better explained by the effect of the local chord context. 4.1 Introduction Qualia are, by definition, subjective. Yet, it seems feasible that a majority of individuals might describe the qualia of a sunset, or of eating a pear, in similar ways. Perhaps the qualia of scale-degrees, then, might similarly be described using common language across individuals? 63

77 Huron (2006) describes an informal study in which he asked ten experienced musicians (all professors and graduate students in a music department) to imagine each scale-degree, and to free associate words or phrases that they felt described that particular scale-degree. He then analyzed the responses by grouping words and phrases that were alike in meaning, and found that there was a clustering of similar responses according to scale-degree. This suggests that experienced musicians not only have the ability to hear qualia, but also that their personal experiences of the qualia of scale-degree appear to be similar. When teaching scale-degree (or interval) identification in music, it seems like what we attempt to do as teachers is to bring about an awareness in our students of these shared attributes of experience. Identifying scale-degrees, however, comes more easily for some than for others. Perhaps students that have difficulties with scale-degree identification tend to confuse the qualia of scale-degrees? Or perhaps scale-degrees with similar functions (e.g., ˆ4 and ˆ7) tend to elicit similar qualia? However, there is a possibility that these labels that we attach to certain scale-degrees do not arise organically from experience, but rather perhaps from generations of teachers passing down learned vocabulary scale-degrees become imbued with the representations we attach to them. Perhaps the qualia are not useful for identification, and instead we come to identify scale-degrees by other means and then, once identified, gain access to all the associated features we have learned. In this chapter, I aim to investigate, firstly, whether everyone is capable of experiencing scale-degree qualia. It may be that certain listeners are incapable of hearing in this way. For those that can distinguish the qualia of scale-degrees, do those listeners do so in consistently similar ways? And what about the role of learning and 64

78 experience? Are the participants in Huron s study merely responding according to learned associations? Can individuals without any music-theoretic training distinguish the qualia of scale-degrees in consistent ways? And if so, are their responses similar to those who do have musical training? Finally, do scale-degree qualia remain stable within real musical contexts, where typically at least in Western music a melody is most commonly embedded in a harmonic context? In addition to investigating the exploratory questions listed above, this chapter will also evaluate a formal hypothesis: that changes to the immediate harmonic context will modify the perceived qualia of scale-degrees. In a series of well-known experiments, Carol Krumhansl, along with Edward Kessler and Roger Shepard, tested participants responses to perceived goodness of fit of a scale-degree after a given harmonic progression in an attempt to investigate the properties of scale-degrees as they relate to the overall key. (Krumhansl & Shepard, 1979; Krumhansl & Kessler, 1982; Krumhansl, 1990) (This work directly led to the use of the famous Krumhansl and Kessler key profiles which are still widely in use today.) Krumhansl and Kessler claim that their goodness of fit ratings confirm a tonal hierarchy theory, in which they propose that what listeners are really responding to are the varying levels of tonal stability of the scale-degrees within a given key, which fit into three categories: tonic chord, remaining diatonic tones, and remaining chromatic tones. However, the various harmonic contexts in the experiments that led to the key-profiles were all variations of a predominant-dominant-tonic progression, and therefore all ended with the tonic chord. As discussed in Chapter 3, a listener s expectations for a scale-degree s resolution (or progression), may be tied to the harmonic context. Therefore, in order to ensure that their key profiles truly 65

79 represent effects of key, it would be prudent to examine the goodness of fit using a greater variety of chord progressions, and specifically, using chord progressions that end on a chord other than tonic. Accordingly, this chapter will also examine ratings related to goodness of fit in various harmonic contexts, and compare them to Krumhansl and Kessler s findings. In sum, this chapter ultimately has three goals. The first is to re-investigate Huron s (2006) findings that musicians generally agree on qualitative terms for the various scale-degrees, and then to explore whether listeners without musical training describe the qualia of scale-degrees using similar terms, or whether they can do the task at all, given that they cannot identify scale-degrees. If descriptions are relatively consistent across all levels of experience, it would suggest that scale-degree qualia are not dependent on training. The second goal is to test the role of harmony in the perception of scale-degree qualia. If scale-degree qualia are found to be altered by the harmonic context, then this finding would carry implications for models of melodic expectation. In addition, it might suggest a revision to current models of aural skills pedagogy. Lastly, goodness of fit judgments for scale-degrees in different harmonic contexts will be compared with the findings from Krumhansl and Kessler to re-examine the relation of scale-degree to chord and chord to key. 4.2 Method Prior to the main experiment, an informal pilot study was conducted with 10 participants of mixed musical backgrounds. Some were undergraduate students pursuing a music degree, and others were persons with little-to-no musical experience and no training in music theory or aural skills. The primary purpose of the pilot 66

80 study was to figure out the descriptive terms that should be used in the main experiment, and to evaluate whether individuals without musical experience would be able to perform the task. Therefore, an approach similar to that of Huron (2006) was taken, where participants were asked to free-associate words with various scaledegrees. In contrast to Huron s approach, however, the scale-degrees were actually heard (as opposed to imagined) with key contexts first established by performing simple scales or short chord progressions at the keyboard. After setting up a key context, a single scale-degree was played, after which the participant would respond with their free-associated terms which were then written down by the experimenter. Given the goals of the present research, and that perceived scale-degree qualia may be a highly individual experience, an appropriate design for the main experiment might take the large list of adjectives given by these participants, and ask listeners in the main experiment to check all that apply. However, it would be useful to establish, at first, whether scale-degree qualia are experienced in any kind of similar ways. Furthermore, since the task is rather abstract (according to feedback from the participants in the pilot study), responses are likely to contain a large amount of variation, and the more terms given, the more variation one is likely to find. Although there may be subtle differences between words like jarring and harsh, or gloomy and sad, a primary goal of this research is to establish whether there are general similarities in descriptions across participants, especially given differences in musical training. Therefore, the participants responses from the pilot study were subjected to content analysis, in an attempt to come up with a representative set of descriptive words that would be able to be rated by participants in the main experiment. 67

81 Thus, the complete set of vocabulary obtained in the pilot study was grouped by similarity into larger categories, and then the categories were given names. For example, the following words were grouped into a category designated as relating to the concepts of strength and/or stability : confident, centered, weak, heavy, strong, stable, light, unsure, unsteady, airy, tipsy, fragile, unstable. Remarkably, despite a few unusual and unexpected responses from the non-musician group, the results of the pilot study suggest that even when participants have no musical training, listeners came up with similar descriptive terms that could be put into similar categories as found in Huron (2006). The content analysis initially resulted in five categories: movement, strength/stability, emotional valence (primarily happy and sad), lightness/darkness, and tense/relaxed. After comparing these results with Huron s categories (which were: certainty, tendency, completion, mobility, stability, power, and emotional valence), it was acknowledged that the movement category held the largest number of terms, and therefore could easily be divided into the subcategories: tendency and completion. Thus, since the majority of descriptions given by participants could fit within these categories, it was decided that these seven category names themselves would be the terms used as dependent variables in the main experiment. Notice that many of the terms given by the participants in the pilot study, as well as the resulting vocabulary that make up the dependent variables, can be considered metaphorical. Some scholars (especially those who believe in the strict definitions of qualia discussed in Chapter 2) might consider the reliance on metaphor in describing aspects of music as providing a supporting argument for the claim that musical qualia are ineffable. However, I have already noted that the language gathered and provided in these experiments is 68

82 in no way meant to directly represent their qualia. Rather the vocabulary used (and collected) are thought of as aspects contributing to the overall qualia, and are simply used as a way of measuring converging evidence. In addition, metaphors of motion, energy, brightness or darkness, tension or relaxation have permeated pedagogical language for centuries (as evidenced by historical pedagogical treatises,) and therefore appear perfectly appropriate to use as descriptive elements of music in these studies Participants Sixty-five participants were recruited for this study. Of the total participants, 43 were second-year undergraduate music students, and 22 were graduate and undergraduate students from various other disciplines. All participants were given the Ollen Musical Sophistication Index (OMSI), primarily to distinguish those who had received formal music-theoretic training from those who had not. For the purposes of this study, only two questions from the OMSI questionnaire were considered: one which asks them to self-identify as either a non-musician, or a musician (of varying degrees of competency); and another which inquires about how many years (if any) of college-level coursework they have completed. The main factor for including persons in the non-musician group was that they had not been exposed to the language and terminology commonly used in music theory pedagogy, and that they would not have developed the skill of identifying scale-degrees by name. Thus, in addition, participants were informally asked during pre- and post-experiment interviews whether they had ever received any music theory or aural skills training of any kind, and whether they believed they possessed absolute pitch. Based on their responses to the above questions, participants were then divided into two groups: musicians and 69

83 non-musicians. Note that a few individuals in the non-musician group did claim to have vocal or instrumental training, but did not consider themselves musicians, nor had they had any experience with music theory or aural training. Of the total participants, five identified themselves as having perfect pitch. These participants data were first examined in comparison with the musician group, to see if, as a group, their responses were markedly different from the rest of the musician group. A priori, it was determined that if the absolute-pitch group responded in a significantly different way, they would be considered separately from the other two groups. If not, their responses would be included in the musician group. In the end, two participants were excluded from the experiment due to technical malfunctions. Of the remaining participants, 41 music students fell in the musician group and the other 22 students fell in the category of the non-musician group Stimuli The study was broken into two blocks of 35 trials each. The stimuli followed a context+probe design. In each block, participants were presented with a keydefining progression in a major key. All progressions were four chords in length, began with the tonic triad, implied the same key, but each ended with a different harmony. The possible chord progressions (contexts) consisted of: I IV V I, I V I IV, I IV V vi, I IV I V, or I V I ii. Thus, each progression contained the tonic and dominant chord, but did not necessarily end on the tonic harmony. Each trial consisted of one such key defining progression, immediately followed by a single probe tone, which could be any one of the seven diatonic (major) scale-degrees, or one of three chromatic scale-degrees: ˆ1, ˆ4, or ˆ5. Note that since these chromatic notes 70

84 are being presented aurally, they may equally be interpreted as their enharmonic equivalents. However, for the sake of clarity these chromatic scale-degrees will only be given a single representation (as sharp scale-degrees) in all figures and text. It was decided that the sounds should be as realistic as possible. As such, all stimuli were recorded using a Yamaha P-90 keyboard, Sonar sound editing software, and VST instruments by Roland s Sonic Cell (VST used was the ultimate grand 003). The stimuli were not quantized, however they were recorded with the use of a metronome. Although this meant that the inter-onset intervals and coordination of voice onsets for chords would be inexact, it was decided that any effects resulting from these liberties would likely be negligible for this particular experiment; and given the already abstract nature of the task, would in fact be preferred over more highly constrained and less realistic-sounding stimuli. However, the velocity of the stimuli were equalized such that all chord progressions and scale-degrees would have equal velocity (and so roughly equivalent apparent loudness). Chords were voiced in a traditional keyboard spacing, with the left hand playing a single bass note and the right hand playing a close-position triad approximately one octave above the left. In order to prevent qualia responses from being biased towards describing pitch height, the original stimuli which were all recorded in the key of C were randomly transposed to a new key after every 10 trials. This was preferred over changing key after every trial, since it was informally noted in the pilot study that participants (especially those in the non-musician group) found the task more difficult immediately following a key change. Thus, this design introduced a randomization of keys while still allowing participants enough time in one key to familiarize themselves with it before moving on to a new one. Key presentation order, and the order of stimuli 71

85 within each key group, were randomly assigned for each participant. The scaledegrees were all transposed along with the chord progression in order to maintain equivalent spacing from the chord progressions across transposition levels. Scaledegrees (probes) were always at least a perfect fourth higher than the upper-most tone of the chord progression in order to minimize response biases due to pitch proximity (see Krumhansl, 1982). Participants heard each scale-degree (total 10) in each key context (total 5) once each for a total of 50 trials, plus an additional 20 trials which were repeated and arranged randomly throughout the two blocks in order to gather a measure of within-subject variability Procedure As explained in Section 4.2, it was decided that the best method of approach was to use a condensed version of the most commonly used adjectives to describe the qualia of scale-degrees, and have participants rate the appropriateness of the terms. Limiting the variables also makes the somewhat daunting task a bit easier for those with no musical training. For statistical purposes, a rating task simplifies the analysis by having numerical values on a continuous scale, rather than counts of categorical variables. In addition, using a rating scale rather than yes/no categories allows possible subtle differences in scale-degrees to come to the fore that might otherwise be described with similar terms. That is, using a checkbox approach, one might find that ˆ1 and ˆ3 both tend to be described as stable, relaxed, solid, etc. However, by using a rating scale it becomes possible to discern whether, for example, ˆ1 might be more stable, relaxed, solid, etc., than ˆ3. 72

86 Figure 4.1: Image of digital interface used in experiment. Rating terms are listed in opposite-facing pairs with a slider in-between. Participants were instructed to move the sliders to indicate the degree to which they believed a particular term described the qualia of the probe tone. Participants were asked to rate the qualia of each scale-degree using a digital interface with sliders that could be dragged to one side or the other indicating that a given scale-degree was heard as either more x or more y, where x and y represent opposite qualia terms (e.g., happy or sad). As can be seen in Figure 4.1, the interface contained 14 terms, arranged in opposite-facing pairs, and participants were instructed to move sliders to indicate the degree to which they believed each particular term described the qualia of the probe tone. Participants were also given the option to opt-out of using any particular rating scale on any given trial by checking does not apply / I don t know if they felt that it was not a useful or applicable descriptive term. 73