Experiment on adjustment of piano performance to room acoustics: Analysis of performance coded into MIDI data.

Toronto, Canada International Symposium on Room Acoustics 203 June 9- ISRA 203 Experiment on adjustment of piano performance to room acoustics: Analysis of performance coded into MIDI data. Keiji Kawai (kkawai@kumamoto-u.ac.jp) Kazunori Harada (28d898@st.kumamoto-u.ac.jp) Kumamoto University 2-39- Kurokami, Kumamoto 860-8555 Kosuke Kato (kato@uic.osaka-u.ac.jp) Osaka University 2-8 Yamadaoka Suita, Osaka 565-087 Kanako Ueno (uenok@isc.meiji.ac.jp) Meiji University -- Higashimita Kawasaki, Kanagawa 24-857 Tetsuya Sakuma (sakuma@k.u-tokyo.ac.jp) The University of Tokyo 5--5 Kashiwanoha Kashiwa, Chiba 277-8563 ABSTRACT An experiment was carried out in which 2 professional piano performers played test phrases in five types of real and vurtual acoustic conditions, and the movements of the keys and pedals were recorded as MIDI signals simultaneously. The test phrases were the extracts from three classical music pieces. For each of the five acoustic conditions, reverberation time (T30) and stage parameters (STEarly and STLate) were measured. Four performance factors derived from MIDI signals, namely, performance duration, note-on velocity and standard deviation, and full pedal time ratio, were analysed in relation with these acoustic factors. While the variations in different acoustic conditions between the performers were observed, significant correlations between note-on velocity and STEarly and between full pedal time ratio and T30 were observed as a common trend over the performers. A further analysis was performed to find particular patterns of individual differences pertaining to the correlation of performance factors with the performed sound fields. In the result, some group trends, such as a diverse correlation with STEarly between groups, were observed. INTRODUCTION While most studies on concert hall acoustics assume that the music performances are constant regardless of the hall s acoustics, it seems common among professional performers that they adjust their performance to the suit room acoustics. From this point of view, our previous study investigated the relationship between room acoustics and performer's adjustment of performance through an experiment in which five professional performers of four types of instruments played in virtually simulated concert hall sound fields and the result indicated their conscious or unconscious adjustment of their performance. In this study, we focused on the piano performance with a reason that the physical action of piano play can be recorded digitally

by use of MIDI coding system. This feature is significant among other instruments as the piano performance can be analysed numerically on the computer basis. Bolzinger 2,3 utilized this system and found relationships between the performance and the acoustic factors such as reverberation time and direct/reflection sound energy ratio. In this study, for investigating more detailed relationship between various musical expression and acoustic conditions, a performance experiment was carried out in which professional piano performers played test phrases in real and virtual sound fields. 2 EXPERIMENT The experiment was carried out in June and October 200 with an identical condition setting and, in total, 2 professional piano performers participated as the subjects. All the subjects were selected among piano teachers who received exclusive musical education. Acoustic conditions: Five types of acoustic conditions were presented, which are: ) a highly absorbed piano practice room using polyester-fiber sound absorbing boards and wedges (S0), 2) the same room as S0 but real-time reverberation sound was provided from four surrounding loudspeakers simulating a small hall (S), 3) the same as S but simulating a large hall with a longer reverberation (), 4) a real small recital hall (RH) and 5) a real small room(rr). RR is a piano storage room of RH and the piano placed closely to the concrete walls that provides instant reflections. For each of the conditions, reverberation time (T30), and stage parameters (STEarly and STLate) were measured and the arithmetic of those in 500 Hz and khz octave band were selected as the representative values (Table ). Figure shows the relation between the acoustic factors in each of the conditions. It is seen that the STEarly of S0, S and are similar probably because the early reflection was determined not by the additional reverberation from the loudspeakers but by the room reflection. RR has the greatest STEarly and STLate among all, and T30 of RH and S are similar but ST of RH is 2-3 db greater than S. Procedure: In the experiment, a grand piano, with which a YAMAHA Disklavier MIDI system was installed, was located at one of the three rooms of S0-, RH or RR. The piano was tuned every time after moved to another room. Before playing the test phrases in a condition, each subject took a few minutes for practicing test pieces and examining the characteristics of the acoustics. Then they played three kinds of test phrases three times in a condition and the performance action was recoded as MIDI signals simultaneously. Following the performance in each of the conditions and after -25 0.0 0.5.0.5 2.0 2.5 3.0 Figure : Profiles of five acoustic conditions Table : Acoustic conditions in the experiment Field Volume T30 STEarly STLate Features S0 4 0.29-8.0-24. Highly absorbed piano practice room S 4.06-6.7-2.2 Simulated small hall in S0 room 4 2.52-7. -4.2 Simulated large hall in S0 room RH 572 0.95-4.4-8.9 Real small recital hall RR 22 0.52 3.6-6.0 Real small room (a piano storage of RH) 2 ST(500-K) (db) 5 0-5 -0-5 -20 S0 RR RH S STEarly STLate

finishing all the conditions, the subjects were interviewed on the adjustment that they made to their performance in response to the acoustic condition. Test phrases: The test phrases are extracts from three classical piano music pieces as follows. - Ba: Bach, The Well-Tempered Clavier, Book - Prelude No. in C major (BWV 846) (beginning 8 bars) - Be: Beethoven, Piano Sonata No.8 in C minor, Op.3 the first movement (beginning 4 bars) - Ch: Chopin, Waltz No.9 in A-flat major, Op.69- (beginning 6 bars) A characteristic of Ba is arpeggio with regular tempo. Be has wide dynamic range and a variety of note length. Ch is a piece with agogic expression. 3 ANALYSIS Four indices, presumably related to musical expressions, were calculated from the recorded MIDI signal data as follows. - Performance duration (PD): Performance duration corresponds to overall playing tempo and, here, this was simply defined as the time from the onset of the first played note to the onset of final note using the MIDI time code. - Note-on velocity and standard deviation (NOV and NOVsd): Note-on velocity is a MIDI code with the digital value ranged from 0 to 255. This corresponds to the strength of a key touch and indicates how loudly the note was played. In the analysis, the and standard deviation of NOV for all the notes played in each performance was calculated. The value corresponds to the overall sound volume and the s.d. is assumed to reflect the dynamics in a performance. - Full pedal time ratio (FPR): The depth of the sustain pedal pressed was encoded into MIDI signal with the value from 0 to 255. The threshold value at which the pedal fully works was 70, which was examined for the piano after the performance experiment, and consequently the full pedal time ratio was defined as the time ratio of the period when the value was 70 or more to the entire performance duration of a performance. 4 RESULTS AND DISCUSSION 4. Effect of acoustic factors A series of analysis of variance (ANOVA) were conducted with the subjects and five sound fields as explanatory variables and each of the four performance factors as response variables by each of the test phrases. When the interaction between the subjects and the sound field was put in the ANOVA, the effects of all the explanatory variables including the interaction were significant at less than % level for all the test phrases. This result was rather trivial so then only the main effects of the explanatory variables were put into ANOVA with the interaction was pooled in the error. Also the correlation coefficients between the performance factors averaged for each of the acoustic conditions and the acoustic factors were calculated. Table 2 summarized these results. Furthermore, to visualize the relation between the performance factors and the sound fields, the values of performance factors were standardized by dividing them by the overall for each of the test phrases, and then the values were averaged over the subjects and plotted with the T30 as the horizontal axis (Figure 2). 3

Result of ANOVA shows significant difference in most of the performance factors by the main effect of sound field. The effect was rather small on the performance duration. FPR had a strong correlation with the T30 while other performance factors had more correlations with STEarly rather than T30. In Figure 2, both of the standardized NOV and NOVsd were the lowest in RR, which indicates that the subjects played softly and with less dynamics in RR. This implies that small rooms like RR might not be suitable for piano practice because they couldn t perform in the same way in RH or in the other simulated acoustic conditions. As the correlation coefficient indicates, it is also seen in Figure 2 that the performance duration didn t change with the T30 Standardized Note-on velocity Standardized Performance duration RH S Ba 47.6 Be 59. Ch 47.0 0.0 0.5.0.5 2.0 2.5 3.0 RH S Ba 26.0(s) Be 33.4 Ch 27. 0.0 0.5.0.5 2.0 2.5 3.0 Standardized Note-on velocity s.d. Figure 2: Standardized performance factors averaged over performers 4 RH S Ba 8.3 Be 7.5 Ch 4.8 0.0 0.5.0.5 2.0 2.5 3.0 () Note on velocity (2) Note on velocity s.d. Standardized Full pedal time ratio +8% RH S Ba 0.8 Be 0.64 Ch 0.74-8% 0.0 0.5.0.5 2.0 2.5 3.0 (3) Performance duration (4) Full pedal time ratio Table 2: Effect of acoustic field on performance factors. ANOVA Correlation coefficient with acoustic factors Performance factor **:p<0.0, *:p<0.05 T30 STEarly STLate Ba Be Ch Ba Be Ch Ba Be Ch Ba Be Ch Note-on velocity ** ** ** 0.02 0.33 0.9-0.85-0.97-0.96-0.66-0.72-0.76 Note-on velocity s.d. ** ** 0.22 0.4 0.34-0.82-0.77-0.92-0.36-0.27-0.5 Performance duration * * -0.07 0.69 0.67 0.94-0.55-0.75 0.79 0.06-0.55 Full pedal time ratio ** ** ** -0.97 -.00-0.94 0.23 0.28 0.09-0.27-0.8-0.35

and the FPR was negatively proportional to the T30. 4.2 Individual variations The results above are the general trend regardless of the individual variation with the acoustic conditions so here we discuss the individual variation and similarity by examining the similarity between the subjects with respect to the correlation between the performance factors and the acoustic conditions. For this analysis, the performance duration of Ch, of which the variance of the interaction of the subjects and the acoustic conditions was the largest among the performance factors in the ANOVA, and FPR of a part of Be, from the beginning of the 3rd bar to the 3rd beat of the 4th bar where the sustain pedal action seemed to be most varied, were subjected to this analysis. In the analysis, the values of the three trials in each of the acoustic conditions were averaged and thus each of the subjects was associated with five values of the performance factors for each of the sound fields. The similarity between individuals was defined here as the correlation Standardized Performance duration Table 3: Subject groups based on cluster analysis Factor Cluster ID for the subjects 2 3 4 5 6 7 8 9 0 2 PD(Ch) A A B C - - - B C - A B FPR(Be) D D - E D F D E F F D E A F are the cluster ID and the subjects are in the same group when the ID is the same. - indicates the subjects showed little correlation with all the others and didn t join any group. 0% +5% -5% -0% RH S Cluster A Cluster B Cluster C Overall (:27.) -5% 0.0 0.5.0.5 2.0 2.5 3.0 Table 4: Correlation between acoustic factors and averaged performance factors of each clusters of each cluster () Performance duration of Ch Cluster T30 STEarly STLate A 0.28-0.78-0.83 B -0.44 0.89 0.4 C 0.40-0.93-0.55 () Performance duration of Ch (2) Full pedal time ratio in part of Be Figure 3: Performance factors averaged within the clusters 5 Standardized Full pedal time ratio +5% +0% +5% -5% -0% RH S Cluster D Cluster E Cluster F Overall (:0.66) -5% 0.0 0.5.0.5 2.0 2.5 3.0 (2) Full pedal time ratio in part of Be Cluster T30 STEarly STLate D -0.9-0.05-0.40 E -0.75 0.7 0.37 F -0.9 0.24-0.3

coefficient of each of the performance factors. A cluster analysis and principal component analysis was performed to determine the groups. Table 3 shows the result and the correlation coefficients of the member in the same cluster were from 0.64 to 0.98. Finally the performance factors were averaged within each of the groups and the correlation coefficients between each of the clusters and the acoustic factors were calculated. The result is shown in Figure 3 and Table 4. In the result of the performance duration of Ch, a general trend was playing faster (thus the shorter performance duration) in RR and RH than the simulated sound fields, which corresponded to the negative correlation of PD with STEarly. This trend was the most obvious for cluster A, and on the other hand, only cluster B indicated the opposite trend with a high and positive correlation with STEarly. As for the FPR of Be, a general trend was the less sustaining notes in the longer T30, which corresponded to the negative high correlation with T30. However there is a group (cluster E) whose FPR was high in RR and RH with a rather high positive correlation (r=0.7) with STEarly as well as T30 and cluster D indicated a weak opposite trend. Thus, for both of the performance factors, some different group trends as described above were observed in the variation of performance by the sound field. 5 CONCLUSIONS In this study, the MIDI signals, obtained in a performance experiment in the real and simulated sound field, were analysed in relation with the acoustic factors of the sound fields. With respect to the adjustment of the performance to the acoustic conditions, general trends were found such as a positive high correlation of note on velocity with STEarly and a negative high correlation of full pedal time ratio and T30. As well as such common trends among the subjects, group trends were also observed such as the full pedal time ratio of a group changed in the correlation with both T30 and STEarly and the performance duration correlated either positively or negatively with STEarly depending on the groups. Further study will address performers intention of the adjustment and analysis of the sound heard at the performers position as well as the MIDI signal. ACKNOWLEDGMENTS The authors thank all the piano performers who participated in this study for their willing cooperation and also N.Shime, T.Hirose and M.Yasutomo of Yamaha Music Foundation for their great support for this study. REFERENCES K.Ueno, K.Kato and K.Kawai, Effect of room acoustics on musicians performance. Part : Experimental investigation with a conceptual model. Acta Acustica united with Acustica, Vol. 96, 505-55(200). 2 S. Bolzinger and J.C.Risset, A preliminary study on the influence of room acoustics on piano performance, Journal de Physique IV Colloque C, supplement au Journal de Physique III, Vol.2 C-93-96 (992). 3 S. Bolzinger, O.Warusfel and E.Kahle, A study on the influence of room acoustics on piano performance, Journal de Physique IV Colloque C5, supplement au Journal de Physique III, Vol.4 C5-67-620 (994). 6