Quantitative Evaluation of Violin Solo Performance Yiju Lin, Wei-Chen Chang and Alvin WY Su SCREAM Lab, Deartment of Comuter Science and Engineering, ational Cheng-Kung University, Tainan, Taiwan, ROC E-mail: alvinsu@mailnckuedutw Tel: +886-6-757575-6537 Abstract Evaluation of erformances of musical instruments is usually subjective It may be easier for keyboard instruments For bowed-string instruments such as a violin, delicate articulations are required in its erformance such that there exhibit much comlexities in its sound, making the evaluation more difficult In this aer, a note searation algorithm based on sectral domain factorization is used to extract the notes from recordings of violin solo erformances Each note can then be quantitatively evaluated based on a set of metrics that is designed to rovide various asects of violin erformances including itch accuracy, bowing steadiness, vibrato deth/rate, bowing intensity, temo, and timbre characteristics and so on The tools should be useful in musical instrument erformance education I ITRODUCTIO Music erformance review and criticism have been a difficult and subjective task []-[4] It may be arguable that objective evaluation of truly artistic erformances is aroriate However, it is definitely useful that certain hysical asects of sounds of music erformances can be accurately delineated if a layer wants to know exactly how he/she lays the instruments Furthermore, it is ossible to disclose the differences among violinists in order to rovide more objective descrition of styles In the research areas of music information retrieval, welldeveloed technologies such as onset detection [5], itch estimation [6], score alignment [7], beat tracking [8] and so on can be alied to extract hysical features related to musical sounds Quantitative indices can be derived based on the features resented in [9], too In this aer, violin solo erformances are studied and some useful quantitative indices will be created to describe both the sound and the erformance Because of the sustaining driven characteristics of violins, roerties such as itch, timbre, and volume kee changing during the lay [] Therefore, it is also desired that the indices should be time varying, too A major difficulty of analysis of violin solo erformances is to searate the violin art from the accomaniment art of acoustic recordings Then, it is also necessary to divide the violin solo art into isolated notes Audio source searation has been a very oular toic in the last few years []-[4] In [5], a MF (onnegative Matrix factorization) based method is roosed for the analysis of bowed-string instruments In [6], a time deendent recursive regularization (TD-RR) aroach for musical note searation is roosed In the above methods, the magnitude sectrum of the signal is divided into two nonnegative matrices which reresent the sectral temlate information and the associate intensity information, resectively We will use the temlate matrix and the intensity matrix obtained by using TD-RR method to generate the quantitative indices of violin erformances The rest of the aer is organized as follows Section reviews the note isolation algorithm emloyed in this aer Section 3 resents the quantitative indices used to describe violin solo erformances In Section 4, exeriments using different commercial recordings of Bach s violin solo work are resented Conclusion and future works are in section 5 II OTE SEPARATIO OF MUSICAL RECORDIGS In [6], for the magnitude sectrogram of a mixture signal M V R > and the number of tone models R ℵ, factorization of V generates two nonnegative matrices, the R sectral temlate matrix W and the intensity matrix M R R > R > H, as shown in () M is the number of frequency bins and is the number of audio frames involved in the analysis V V ~ = H W The cost function in () contains the distance between V and H W in the mean square sense and two additional regularization terms W + H C H D = V H W + λ W C γ With the above cost function minimized, one can obtain W and H by W = H T T T ( H H + λ I ) ( H V + λ C ) = T T T ( W W + γ I ) ( W V + γ C ) In (), the two regularization terms are used to constraint the form of the target matrices Unlike conventional MF methods, equation (3) and (4) may not always roduce nonnegative results By setting C W and C H as nonnegative matrices, the results tend to be nonnegative too For temoral smoothness, it is desired that both temlate matrix and W H () () (3) (4)
intensity matrix don t vary too much in a short eriod of time Therefore, one can set C W and C H to be the analysis results of the revious time instant Moreover, if the timbre of the target musical instrument is known, one could constraint the temlate matrix by using the timbre information as follows Let the reference temlate matrix be ur C W [ u u u u ] T = r R, where is used to reresent the constraint temlate column vector for the r-th tone model In [4] and [6], u r can be obtained as the weighted sum of bell-shae functions, u r = g G ( f, σ ) r, r In (6), G is Gaussian function, f r is the fundamental frequency of the r-th tone model, is the artial index, and σ is used to control the width of Gaussian function g r, is the arameter to control the intensity of the -th artial of the r-th tone model If the timbre of the tone model is known in advance, g r, can be a fixed arameter Otherwise, one has to estimate these arameters during the analysis rocess using (5) (6) the method roosed in [7] Hence, one could combine several constraints as u ( t) = α W ( t ) + r r ( α) ( f ( t ), ), gr, ( t ) G r σ (7) where W r ( t ) is the temlate column vector of the r-th tone model obtained in the revious time instant In (7), we emloy W r ( t ), the estimated fundamental frequency and the estimated timbre obtained at time t- to have the constraint temlate column vector in order to calculate W r (t) in (3) Fig shows the intensity matrix derived from the analysis result of a short cli of Bach Sonata o3 in C major, BWV 5, Adagio, the 4-th measure, layed by Hillary Hahn There are 7 notes in the cli Each note is searated to give a clear view of how the note was layed The temlate matrix is obtained without the knowledge of timbre information It shows how the timbre changes in the eriod of a note One can also observe how vibrato was executed The intensity matrix shows the onset time, the offset time and how loud the note was layed We will describe how the information can be converted into the quantitative indices in the later section ote - D5 3 3 4 5 6 ote - E5b 3 3 4 5 6 3 ote - C5 3 3 4 5 6 4 ote - D4 3 3 4 5 6 5 ote - B4b 3 3 4 5 6 6- ote - A4 (Trill) 3 3 4 5 6 6- ote - B4b (Trill) 3 3 4 5 6 6 5 Hilary Hahn - Pitch Contour 7 ote - G4 3 3 4 5 6 4 3 3 4 5 6 Fig Searated results from a cli of Bach Sonata o3 in C major, BWV 5, Adagio, the 4-th measure, layed by Hillary Hahn There are seven notes and their note names are shown corresondingly The first 3 sub-figures show the intensity matrix The fourth sub-figure shows the derived itch contour
III OBJECTIVE IDICES OF VIOLI PERFORMACE Many erformances can be characterized by itch and intensity evolution of their sounding notes In this aer, we roose a new quantitative descrition, called the steadiness index, which indicates how stable the sounding note is over time when the erformer is laying the instrument It is also designed to rovide an overall rank in the eriod of a note To be secific, the itch contour P () and the intensity contour I () can be fundamental start oints for our quantitative descrition of a violin note Since a note is said sounded or voiced, both contours are exected to be smooth and continuous curves over a limited time eriod T Both contours behave secific characteristics in different sounding modes For examle, a violin note can be either a vibrato note or a non-vibrato note, so we need to evaluate a note in two different ways In Figure, the converted itch contour which is derived from the intensity matrix and temlate matrix is shown One is a vibrato note and the other one isn t In this aer, steadiness measure is used as the erformance index For a non-vibrato segment of a note whose duration is T P n and variance T s =, we estimate the itch mean ( ) σ ( n) as well as the intensity mean I ( n) and variance ( n) σ The steadiness measure of a non-vibrato segment is defined as s nv σ σ = I n= P % 48 ( n) I ( n) If the itch variation is smaller than a quarter of semitone Sigiswald Kuijken-st note D4(799) 3 3 4 Hilary Hahn-st note D4(799) 3 3 4 Arthur Grumiaux-st note D4(34) 3 3 4 I (8) Fig Bach Sonata o3 in C major, BWV 5, Adagio, the 4- th measure and the intensity variation is smaller than ercentage of the intensity average, the steadiness value should be smaller than one, indicating that the note is steady enough Similarly, for a vibrato segment of a note, the vibrato rate D are first estimated Then, the V () and the vibrato deth () vibrato rate mean V ( n) and variance ( n) σ are consequently calculated Similarly, we also have the vibrato deth mean D ( n) and variance σ D ( n) The steadiness measure of a vibrato segment is calculated as s v σ σ = V D n= V ( n) % D( n) % If variations of the vibrato deth and the vibrato rate are smaller than ercentages of their averages, the steadiness value should be smaller than one, indicating that the vibrato note is steady enough Sigiswald Kuijken-nd note B4b(444) 3 5 5 Hilary Hahn-nd note B4b(4575) 3 5 5 Arthur Grumiaux-nd note B4b(4683) 3 5 5 V (9) Sigiswald Kuijken-3rd note A4(498) 3 5 5 5 3 35 Hilary Hahn-3rd note A4(444) 3 5 5 5 3 35 Arthur Grumiaux-3rd note A4(444) 3 5 5 5 3 35 Sigiswald Kuijken-4th note B4b(444) 3 4 6 8 Hilary Hahn-4th note B4b(4683) 3 4 6 8 Arthur Grumiaux-4th note B4b() 3 4 6 8 Fig 3 Intensity matrices of 4 different notes layed by three violinists
IV AALYSIS RESULTS OF PERFORMACES OF BACH S VIOLI SOLO WORK In the exeriments of this aer, we use commercial recordings of Bach Sonata o3 in C major, BWV 5, Adagio, the 4-th measure shown in Fig, recorded by different famous violinists, Sigiswald Kuijkan [8], Hillary Hahn [9] and Arthur Grumiaux [] Sigiswald Kuijken - D4 Hilary Hahn - D4 Arthur Grumiaux - D4 Sigiswald Kuijken - B4b Hilary Hahn - B4b Arthur Grumiaux - B4b 9 9 8 8 7 7 6 5 4 6 5 4 3 3 3 3 3 5 5 5 5 5 5 Sigiswald Kuijken - A4(Trill) Hilary Hahn - A4(Trill) Arthur Grumiaux - A4 Sigiswald Kuijken - B4b(Trill) Hilary Hahn - B4b(Trill) Arthur Grumiaux - o Trill 9 9 8 8 7 7 6 5 4 6 5 4 3 3 5 5 5 5 5 5 4 6 8 4 6 8 4 6 8 Fig 4 Temlate matrices of 4 different notes layed by three violinists Sigiswald Kuijken - D4 8 75 7 5 5 5 3 35 Hilary Hahn - D4 3 95 9 5 5 5 3 35 34 3 96 Arthur Grumiaux - D4 5 5 5 3 35 Sigiswald Kuijken - B4b 445 44 435 5 5 47 465 46 47 Hilary Hahn - B4b 5 5 Arthur Grumiaux - B4b 46 5 5 Sigiswald Kuijken - A4(Trill) 4 45 5 5 445 44 Sigiswald Kuijken - B4b(Trill) 435 4 6 8 44 438 Hilary Hahn - A4(Trill) 5 5 475 465 455 Hilary Hahn - B4b(Trill) 4 6 8 446 Arthur Grumiaux - A4(o Trill) 44 5 5 5 Arthur Grumiaux - o Trill 4 6 8 Fig 5 Pitch contours of 4 different notes layed by three violinists
Sigiswald Kuijken - Snv 5 5 5 3 35 Hilary Hahn - Snv 5 5 5 3 35 Arthur Grumiaux - Snv 5 5 5 3 35 Sigiswald Kuijken - Sv 5 5 5 5 Hilary Hahn - Sv Arthur Grumiaux - Sv 5 5 Fig 6 indices of the non-vibrato(d4) and vibrato(b4b) cases layed by three violinists The resective temlate matrices and intensity matrices after the note searation rocessing can be obtained We show only four notes, D4, B4b, A4 and another B4b The first two notes are layed at the same time though D4 note was layed with an oen string and started a little bit earlier The last notes are layed at the same time using a technique called trill, as indicated in Fig 3 One can comare the differences of their erformances of the same iece of music Fig 3 shows the intensity matrices, Fig 4 shows the temlate matrices and Fig 5 shows the itch contours, for all three layers By observing the above figures, we found lots of differences For D4 note, Kuijkan did not use vibrato, but the other two layers used vibrato For B4b note, all three layers used vibrato, but Kuijkan s vibrato started at 5 second after B4b note was layed For A4 and B4b notes, because trill was used and the trill seed was high, it is difficult to use steadiness measure However, we can see that Grumiaux didn t erform the trill and the B4b note was not layed Finally, Kuijkan layed one semitone lower than the other two layers Fig 6 shows the steadiness indices of 5-second segments of two samle notes There is a 5% overla between two adjacent segments Table shows the average steadiness values of non-vibrato and vibrato notes layed by three violinists The clis are available at our web site so that one can listen and comare their erformances V COCLUSIO AD FUTURE WORKS In this aer, a note searation algorithm is resented and the quantitative evaluation aroach of violin solo erformances is also roosed Commercial recordings of Bach s violin solo works layed by various violinists are used to demonstrate our work Though one may be able to tell the differences of the erformances by observing the temlate matrix and the intensity matrix obtained in the note searation rocess, quantitative indices are still necessary for those who are not familiar with signal rocessing technologies In this aer, steadiness indices are roosed to delineate the characteristics of violin solo erformances in the more conclusive way such that it is easy for most eole to understand and objectively describe erformances It is noted TABLE I AVERAGE STEADIESS VALUES OF THE O-VIBRATO(D4) AD VIBRATO(B4B) CASES PLAYED BY THREE VIOLIISTS Violinist ame Sigiswald Kuigken Hilary Hahn Arthur Grumiaux s nv (D4) 3885 55376 5896 s v (B4b) 597 59483 655 that the roosed work is not intended to touch the artistic art of erformances which should still remain in the conventional domain of music erformance assessment In the future, we wish to imrove our note searation algorithm and rovide more tyes of quantitative indices We also want to consult violin teachers to validate the roosed quantitative indices Finally, the techniques used in this aer can be extended to the evaluation of erformances of other musical instruments REFERECES [] B H Re, Diversity and commonality in music erformance: An analysis of timing microstructure in Schumann s Träumerei, Journal of the Acoustical Society of America, 9, 546-568, 99 [] B H Re, Patterns of note onset asynchronies in exressive iano erformance, Journal of the Acoustic Society of America, oo, 397-393, 996 [3] C S Sa, Comarative analysis of multile musical erformances, in Proc of the 8th International Conference on Music Information Retrieval, Vienna, Austria, 497 5, 7 [4] C S Sa, Hybrid numeric/rank similarity metrics for musical erformance analysis in Proc of 9 th International Conference on Music Information Retrieval (ISMIR 8), Vienna, Austria, 5 56, 8 [5] J Bello, L Daudet, S Abdallah, C Duxbury, M Davies, and M Sandler, A tutorial on onset detection in musical signals, IEEE Trans Seech and Audio Processing, vol 3, no 5, 35 47, 5
[6] A de Cheveigne and H Kawahara, YI, a fundamental frequency estimator for seech and music, Journal of the Acoustical Society of America, vol, 97 93, [7] Orio, S Lemouton, and D Schwarz, Score Following: State of the Art and ew Develoments, in Proc of the International Conference on ew Interfaces for Musical Exression (IME), Montreal, May 3 [8] M E P Davies and M D Plumbley, Context-Deendent Beat Tracking of Musical Audio, IEEE Transactions on Audio, Seech and Language Processing, vol 5, no 3, 9-, 7 [9] G Peeters, A large set of audio features for sound descrition (similarity and classification) in the CUIDADO roject, Cuidado roject reort, Institut de Recherche et de Coordination Acoustique Musique (IRCAM), 4 [] Y-S Siao, W-C Chang, and Alvin W Y Su, Pitch Detection/Tracking Strategy for Musical Recordings of Solo Bowed-String and Wind Instruments, Journal of Information Science and Engineering 5, 39-53, 9 [] D D Lee and H S Seung, Learning the arts of objects by nonnegative matrix factorization, ature, vol 4, 788-79, 999 [] B Wang and M D Plumbley, Musical audio stream searation by non-negative matrix factorization, in Proc DMR Summer Conference, Glasgow, 5 [3] T Virtanen, Monaural sound source searation by nonnegative matrix factorization with temoral continuity, IEEE Transactions on Audio, Seech and Language Processing, vol 5, no 3, 66 74, March 7 [4] R Hennequin, R Badeau, and B David, Time-deendent arametric and harmonic temlates in non-negative matrix factorization, in Proc of the 3th Int Conference on Digital Audio Effects, Graz, Austria, [5] Y-L Chen, T-M Wang, W-H Liao, and Alvin WY Su, Analysis and Trans-synthesis of Acoustic Bowed-String Instrument Recordings A Case Study Using Bach Cello Suites, in Proc of the 4th International Conference on Digital Audio Effects (DAFx-), Paris, France, Se 9-3, [6] T-M Wang, T-C Chen, Y-L Chen, and Alvin WY Su, Time-Deendent Recursive Regularization for Sound Source Searation, in Proc of the 3rd International Conference on Audio, Language and Image Processing (ICALIP), Shanghai, China, Jul 6-8, [7] Yiju Lin, W-C Chang, T-M Wang, Alvin W Y Su and W-H Liao, Timbre-Constrained Recursive Time-Varying Analysis for Musical ote Searation, in Proc of the 6th International Conference on Digital Audio Effects (DAFx-3), Maynooth, Ireland, Se -6, 3 [8] Sigiswald Kuijkan, "Bach: Sonatas & Partitas, BWV - 6," Deutsche Harmonia Mundi Records, CD, 99 Barcode: 3567743 [9] Hillary Hahn, "Hilary Hahn Plays Bach," Sony Bmg Music Entertainment, 5 Barcode: 746467938 [] Arthur Grumiaux, "Bach: Sonatas & Partitas for solo violin," Decca Music Grou Limited, CD, 6 Barcode: 894757558