Hearing and visual complementation: a discussion of accent in Chinese opera by XUEFENG ZHOU Citation Zhou, X. 'Hearing and visual complementation: a discussion of accent in Chinese opera'. In: R. Timmers, N. Dibben, Z. Eitan, R. Granot, T. Metcalfe, A. Schiavio, & V. Williamson (Eds.). Proceedings of ICMEM 2015. International Conference on the Multimodal Experience of Music. Sheffield: The Digital Humanities Institute, 2015. ISBN 978-0-9571022-4-8. Available online at: <https://www.dhi.ac.uk/openbook/chapter/icmem2015-zhou> Abstract The research literature concerning accent increasingly reports that structural, joint and particular accent contain regularly salient onsets. This paper aims to explore how salient onsets work together in the creation of accent structure in a Chinese musical context. For this purpose, observational methods and Sonic Visualiser were employed. Using a classical song of kunqu opera, an intangible cultural heritage with six hundred years history in China, two famous singers recordings were compared and analyzed. The results revealed that: (1) ban - an idiophone percussion instrument made of woodblock - is a marker of salient onsets, (2) serial ban onsets can be classified as isometric and symmetric types with adjacent or nonadjacent features, and (3) such types generate rhythmic accent and extended rhythmic accent. The findings suggest the existence of an accent structure used to shape music over time, which can be theorised in relation to looseness theory in phonetics and accent theory in musicology. 1
Hearing and visual complementation: a discussion of accent in Chinese opera by XUEFENG ZHOU Background Accent is an important perceptual and performing feature in music. The taxonomies of the categories of accent have been much-discussed, but the creation of accent in a cross-cultural context has been relatively neglected. Cooper and Meyer have defined the accent as 'a marking for consciousness of a particular point in the music, achieved through a change of state in any parameter' (Cook, 2013, p. 167). Such a change of state in any parameter can be described as auditory sequences over time along a general dimension (e.g., pitch, loudness, duration). Taking this approach further, each general dimension can express particular accents 'to mean a family of salient local changes'; for example, 'a salient time change might occur if three successive short IOIs were followed by a long IOI' (Ellis & Jones, 2009, p. 265). That is to say, a relatively salient dimension, e.g. inter-onset interval, creates a change in auditory sequences over time, which creates a particular accent. In the domain of musical performance, specific accent types are studied, e.g. creating accentuation, joint accent structures, and other accent structures. Here, accent is not limited to a narrow definition; rather, an onset is a stress which equals an accent. 'In mazurka performance this essentially translates to creating accentuation through dynamic emphasis (playing louder), agogic emphasis (prolonging a note or beat), and articulation (emphasising a note by clipping it)' (Cook, 2013, p. 167). Drake and Palmer observed similar systematic performance variations - of intensity, interonset timing, and articulation - in relation to three accent structures: rhythmic grouping, melodic, and metric accent structures: 'Variations corresponding to rhythmic grouping accents were most consistent across musical contexts and dominated when the accent structures conflicted' (Drake & Palmer, 2003, p. 143). Joint accent structures provide a structural basis for anticipatory attending and the generation of expectancies. 'Because joint accents recur with temporal regularity, a listener is able to anticipate not only the "what" of upcoming accents but also "when" they should occur' (Jones, 1976; Jones & Boltz, 1989, cited in Boltz, 1992, p. 91). This suggests that combined serial onsets can be perceived as structure through temporal regularity. 2
The term 'accent' has a different meaning in a Chinese context than it does in a Western musical context. Deriving from an ancient Chinese aesthetic category, the term 'accent' was intended as a blanket term for phrase/sectional structural meaning, similar to notions of Schenkerian prolongation.this is supported by recent evidence that performers choose individual accents according to structural location (Zhou, 2013a). In China, accent was also valued in phonetics. Wang (2004) has argued that 'Chinese belongs to the looseness type, whereas English belongs to the stress type' (p. 21). This is because Chinese words are combined with two or more characters, and each character might have a salient phoneme of length, pitch and so on. In summary, the aforementioned literature concerning accent increasingly reports that structural, joint and particular accents contain regularly salient onsets. Aims This paper aims to explore how salient onsets work together in the creation of accent structure in a Chinese musical context. To achieve this aim, an interest in understanding how salient onsets are produced and perceived through observation has led to two questions: 1. What is the best method to capture onsets from sound? 2. Does regularity exist in serial onsets? If so, how does it relate to accent structure? Method Observation and audio analysis software - Sonic Visualiser, which provides onset times and global dynamic values (Cook & Leech-Wilkinson, 2009) - were the basic methods employed, together with manual statistical analysis. Materials Using a classical song of a kunqu opera, an intangible cultural heritage with six hundred years' history in China, two famous singers recordings - Mei Lanfang (1960) and Shen Fengying (2004) - were compared and analysed. This study focused on two recordings (film and video) of the song Zaoluopao 3
of Peony Pavilion. Two recordings were taken from footage of film and video. The singer Mei was one of the greatest vocal artists in modern China; in the performance of Zaoluopao, he is accompanied by the Ensemble of Shanghai traditional opera school. The singer Shen is younger but famous and has successfully performed Peony Pavilion since 2004. In the performance of Zaoluopao, she is accompanied by the Suzhou Kunqu Opera Troupe. It is worth mentioning that both recordings used the musical score of Nashu Studio Theatrical Music (Yu, 2008); this letter notation did not record rhythm. In other words, the rhythm heard and seen through sonic analysis has been created by the performers. This reflects pedagogical practice in which performers learned by oral transmission and memorization prior to the rise of state-run opera schools, which adapted aspects of Western musical training. Prior to such opera schools, entrants to the kunqu profession were trained as apprentices in private schools attached to opera troupes (see Mackerras, 1972; Mackerras, 1975). Procedure This study consisted of three stages: (1) on the basis of close listening, a visually macroscopic analysis of ban (a percussion idiophone made of woodblock) was performed; (2) data regarding ban onsets were translated into perceived types; and (3) these types were used to create an analysis representing the temporal structure of the music. Stage 1 Since much sonic information relating to an ensemble is mixed, it was difficult to capture individual instrument parameters by listening or using an auto onset tracker separately. Therefore, these methods were used in combination, alongside careful visual inspection. The prominent timbre of ban, and its definition of strong beat, e.g. one ban (strong beat) three yan (weak beats), recalled the memory of my fieldwork: (1) in kunqu opera, the ban performer acts as a conductor; and (2) this case, Zaoluopao is mild and exquisite, and mainly used ban as opposed to other forms of percussion. Thus, ban, together with a few other percussion instruments, conveyed important information in a manner similar to that of a conductor. Next, onset times of each ban were detected using Sonic Visualiser's 'Add New Time Instants Layer' and 'Audio Onset Detector' features (Cook & Leech-Wilkinson, 2009). I observed two narrowing analysis graphics - recordings of Mei and Shen - in one screen; counted data originating from the Time Instants Layer and compared pitch structure and librettos. 4
Stage 2 To create a taxonomy of ban onsets, it was necessary to identify the values of temporal intervals. This is the ban interval between the current onset (t C ) and the previous onset (t P ): interonset interval (IOI) B = t C - t P. With this, I obtained all values of temporal intervals of ban onsets - IOI B - in turn. Clusters featuring regularity were arranged into separate categories (Table 1). Clusters were collected by a round criterion of equivalence: differences that surpassed 10% of the smaller value were excluded. Table 1. Clusters of serial ban onsets In the Table, all sequence numbers correspond to the marks of delineation in Figures 4 and 5. The numbers on both sides of '+' are temporal values of adjacent ban onset intervals. Black numbers denote data as exported by Sonic Visualiser. Grey numbers denote sums which were added by several numbers under and on the right side of '='. Therefore, sequences with grey 5
numbers belong to the nonadjacent type. Sequences 1-2 in the left column and 1-7 in the right column belong to a type of adjacent isometric interval because values on both sides of every '+' are approximately equivalent. Sequences 3-4 in the left column and 8 in the right column belong to a type of adjacent symmetric interval because two approximately equivalent lateral numbers stand on both sides of one different number and two '+'. Sequence 6 in the left column and 9-10 in the right column belong to a type of nonadjacent isometric interval because they contain grey numbers. Lastly, Sequence 5 in the left column belongs to a type of nonadjacent symmetric interval because it contains grey numbers. Stage 3 In stage three, visual observation in macroscopic and synchronic context came first. In this context, values of temporal intervals of ban onsets were delineated and marked (as shown in Figures 4 and 5), and the observed structures were analysed and discussed. Results Hearing and visual complementation Based on the manual determination of ban onset times and Audio Onset Detector, two recordings were analysed and exported by Sonic Visualiser. These contours show: (1) the relationship between automatically-detected onsets and ban onsets in Shen s recording; (2) the complementarity between automatically-detected onsets and ban onsets in Mei s recording, which looks like a sketchy harmonic structural graph. The relationship between ban and onset, and between the two recordings are revealed by the following data: Mei: 164 seconds /25 ban per ban = 6.56 seconds; Shen: 144 seconds /36 ban per ban = 4 seconds Mei: 164 seconds /432 onset per onset = 0.48 seconds; Shen: 144 seconds /369 onset per onset = 0.39 seconds Ban: Mei/Shen = 1.64 seconds Onset: Mei/Shen = 1.23 seconds 6
I therefore suggest that ban in Mei s recording is more structured than that of Shen - something that I explore in the rest of this paper. Figure 1. Automatically-detected onsets and ban onsets. The former are denoted by the blue wave and the latter by the black wave. Lines closer to vertical denote increasing density of serial ban onsets. In Figure 2, the blue vertical lines show boundaries of sections between the introductory sentence, first segment and second segment. Tonality and librettos are indicated at the end of segments and sentences. Structure indicated by the serial ban onsets in both Mei and Shen s recordings related to both the harmonic and semantic structure. Figure 2. Musical form and onsets of ban. The most obvious difference is a short section of overlapping music where a characters voice overlaps Shen s voice, at the end of Shen's recording. This feature will be explained in further detail later. In short, ban provides a 7
method by which to capture onsets from ensemble music, via careful listening and visual analysis. Serial ban onset types Four kinds of cluster of serial ban onsets are shown in Table 1, which are split into two essential types: isometric and symmetric, with adjacent or nonadjacent features. Isometric intervals are similar to short beats, which are easy to understand. Symmetric intervals need further explanation (see Figure 3). There are three spectrogram images, and each image is marked with four red lines (ban onsets) with the libretto shown in blue along with the vocal contours. Seeing the left image, together with listening to the ending of the introductory (joint) sentence of Mei s recording, I found that the function of the serial ban onsets is to articulate both the preceding section and the following section. Mei sung the [si] (beginning of the first segment) later than the four bans. The latter two bans fill in the lower duration of the ending, which causes an inconspicuous boundary between the joint sentence and the first segment. In the middle image, though the adjacent symmetric intervals connect two phrases, they function as an 'and'. In Shen's recording the adjacent symmetric intervals are boundaries of one phrase and yet also fulfil the function of 'and'. These observations support a viewpoint that, unlike Pogorelich, who chose to articulate the preceding section when playing Brahms Op.117 No.1 bars 1 and 9 (Zhou, 2013b), Chinese rhythmic accents connect/articulate both preceding and following sections. Are the functions of three adjacent symmetric intervals to 'connect/articulate the preceding or the following section' (Zhou, 2013b)? The answer is 'no'. If performers choose 'or', there must have been obvious contrast to create separation. On the contrary, 'and' is selected, which means a weak salient and unclear boundary. Choosing 'and' creates a looseness of accent - its salient onset comes from length, timbre and other salient phonemes in Chinese phonetics. In this case, I take adjacent symmetric intervals as rhythmic accents together with adjacent isometric intervals. 8
Figure 3. Adjacent symmetric intervals. In addition to isometric and symmetric types with adjacent features as above, there are also isometric and symmetric types with nonadjacent features, which are shown in Table 2. These values are calculated as percentages of duration, as follows: sum of IOI B of the type total length of the piece 100. Table 2. Types of serial ban onsets The adjacent type belongs to rhythmic accent, and the nonadjacent type belongs to extended rhythmic accent. The proportion of each performance displaying the adjacent type is: Mei: 5.3 %+13.2% = 18.5% Shen: 25.6%+5.9%= 31.5% The proportion of each performance displaying the nonadjacent type is: Mei: 36.1%+49.7% = 85.8% Shen: 49.7%+0 = 49.7% These data show that, in the category of regularity, the extended rhythmic accent accounts for half or more of the accents across the two recordings, whilst the rhythmic accent accounts for a minority (especially in Mei s 9
recording). But how are these serial ban onsets arranged in whole works? I argue that the organisational regularity conveys structural meaning. Accent structure of ban In this study, 'serial onsets' refers to onsets that form part of a series. Different patterns of relatively equal ban onsets establish different categories of event (adjacent symmetric, nonadjacent isometric, and so on) that are used to articulate and link sections. According to phonetics, the accent structure of ban can be described as looseness short and long durations alternate as relatively equal onsets. Firstly, looseness is a Chinese term regarding accent corresponding to the stress accent in English or pitch accent in Japanese. Secondly, the melodic forms of kunqu are related to the forms of librettos which contain free-flowing rhythm and many changes of pace. In other word, looseness has more freedom of duration than that of rubato in western classical music. Thirdly, the previous findings of clusters of serial ban onsets (Table 1) show the features of looseness. But how does accent structure of ban of looseness appear? Figures 4 and 5 illustrate these features: based on the data in Table 1, all regular serial ban onsets described are marked. To observe their organisation, I itemized ban onsets by red or blue vertical lines, and itemized clusters of serial ban onsets according to Table 1 by black delineations. That is to say, clusters in Table 1 are mapped onto Figure 4 and 5 by the same ordinal number. The horizontal plane denotes time. Four kinds of black delineations (short line, ellipse, triangle and trapezoid) each refer to adjacent isometric, adjacent symmetric, nonadjacent isometric and nonadjacent symmetric intervals. As shown in Figure 4, symmetric features of Mei s recording were shown not only in adjacent symmetric intervals but also in nonadjacent symmetric intervals. The first segment, symmetric trapezoid, manifests fold-unfold-fold feature. A similar feature, but to a lesser degree, is shown in triangle 6. On the other hand, there are two overlaps. One is located between ellipse 3 and trapezoid 5, and another between trapezoid 5 and triangle 6. These two overlaps function similarly to connect the preceding and following sections. With more than 85.8% extended rhythmic accent, it is concluded that organizational regularity exists in Mei s recording. 10
Figure 4. Looseness accent structure of Mei s recording. Figure 5 illustrates looseness of adjacent isometric intervals; short line 1-7, holds 25.6% of time. Nonadjacent isometric intervals, triangle 9 and 10, hold 50.1% of time. That is to say, more than half the time was spent in extended rhythmic accent and 25.6% in rhythmic accent. These short lines appear as continuously punctiform features. On the other hand, there is no overlap within nonadjacent intervals, which implies clear structural boundaries. As can be heard, the first segment starts on the ban, but an overlap part is located at the end of the works, as previously mentioned and shown in Figure 2. This is more evidence to support the idea of looseness and structural meaning of continuity, which has punctiform features. In brief, these serial ban onsets are associated with unfolding and punctiform features by organisational regularity of symmetry and continuity which are rooted in rhythmic accent. Figure 5. Looseness accent structure of Shen s recording. 11
Conclusions In China, people generally think of rhythm in Western countries as involving equal temporal distribution, whereas Chinese traditional music is associated with unequal temporal distribution. This study attempted to uncover more regularity in traditional Chinese music, and contradicted this viewpoint. The results revealed that: (1) ban is a cue to capture salient onsets, (2) serial ban onsets have isometric and symmetric types with adjacent or nonadjacent features, and (3) such types generate rhythmic accent and extended rhythmic accent which demonstrates looseness accent of structural meaning with unfolding and punctiform features. Owing to traditional notation in China omitting rhythm, the regularity of rhythmic accent and extended rhythmic accent are created by performers. Thus, the regularity reflects how performers of ban shape the music over time. In terms of regularity, ban in Mei s recording is more structured than that of Shen. However, why large differences exist between the two recordings and to what degree the types of temporal intervals are perceived by current audiences, are questions that require further exploration. Notes Address for correspondence: Xuefeng Zhou, Faculty of Music of Southwest University, No.2 Tiansheng Road, Beibei, Chongqing 400715, P.R. China. Email: xuefeng_zhou@outlook.com. References Boltz, M. (1992). Temporal accent structure and the remembering of filmed narratives. Journal of Experimental Psychology: Human Perception and Performance, 18(1), 90-105. Cook, N. (2013). Beyond the Score: Music as Performance. New York: Oxford University Press. Cook, N., & Leech-Wilkinson, D. (2009). A musicologist's guide to Sonic Visualiser. Retrieved February 2, 2014, 12
from http://www.charm.kcl.ac.uk/analysing/p9_3.html Drake, C. & Palmer, C. (1993). Accent structures in music performance. Music Perception, 10(3), 344-378. Ellis, R. & Jones, M. (2009). The role of accent salience and joint accent structure in meter perception. Journal of Experimental Psychology: Human Perception and Performance, 35(1), 264-280. Huron, D. & Royal, M. (1996). What is melodic accent? Converging evidence from musical practice. Music Perception: An Interdisciplinary Journal, 13(4), 489-516. Mackerras, C. (1972). The rise of the Peking Opera: social aspects of the theatre in Manchu China. Oxford: Clarendon Press. Mackerras, C. (1975). The Chinese theatre in modern times: from 1840 to the present day. London: Thames and Hudson. Mei, L. (1960). The Peony Pavilion. Beijing: Beijing Movie Studio Audio & Video Co., Ltd. Shen, F. (2004). The Peony Pavilion. Hangzhou: Zhejiang Audiovisual Publishing House. Wang, H. (2004). On the metrical type of Modern Standard Chinese: a type based on looseness. Linguistic Science, 10(3), 21-28. Yu, J. (2008). A review to the Kunqu Opera The Peony Pavilion & Dream Garden. MA thesis of Chinese National Academy of Arts. Zhou, X. (2013a). Observation on musical performance: a case study. In Marinho, H., & Domenici, L. (Ed.), Congresso Internacional Performa (CD). Porto Alegre: Evangraf. Zhou, X. (2013b). A collection of performing knowledge for application: a further observational case study. London: Online PitS Conference. 13