Drum Transcription in the presence of pitched instruments using Prior Subspace Analysis

Transcription

1 ISSC 2003, Limerick. Juy -2 Drum Transcription in the presence of pitched instruments using Prior Subspace Anaysis Derry FitzGerad φ, Bob Lawor*, and Eugene Coye φ φ Music Technoogy Centre, Dubin Institute of Technoogy, Rathmines Road, Dubin, IRELAND E-mai: φ * Department of Eectronic Engineering, Nationa University of Ireand, Maynooth, IRELAND E-mai: Abstract -- This paper demonstrates the use of Prior Subspace Anaysis (PSA) as a method for transcribing drums in the presence of pitched instruments. PSA uses prior subspaces that represent the sources to be transcribed to overcome some of the probems associated with other subspace methods such as Independent Subspace Anaysis (ISA) or sub-band ISA. The use of prior knowedge resuts in improved robustness for transcription purposes and enabes the method to work more readiy in the presence of pitched instruments than other subspace methods. The system presented in this paper attempts to extend the use of PSA to transcribe drum sounds in the presence of interfering pitched instruments. I INTRODUCTION In the past few years a number of subspace methods such as Independent Subspace Anaysis (ISA) and Prior Subspace Anaysis (PSA) have been proposed for sound source separation in singe channe mixtures [,2]. The underying assumptions of these methods make them particuary suited to attempting the task of transcribing drums from singe channe audio mixtures, as has been shown in [2,3]. However these methods have deat soey with the case where drums ony are present. This system presented in this paper attempts to extend the PSA method to transcribe drums robusty in the presence of pitched instruments. II SUBSPACE METHODS FOR SOUND SOURCE SEPARATION The subspace methods described in [,2,3] attempt to represent sound sources as ow dimensiona independent subspaces in the time-frequency pane. These methods make a number of assumptions about the signa. An input signa containing a number of sound sources is transformed to a timefrequency representation such as a magnitude spectrogram. It is assumed that the mixture signa spectrogram Y can be decomposed into statisticay independent spectrograms Y. These spectrograms are assumed to be represented by the outer product of an invariant frequency basis function f, and a corresponding invariant ampitude basis function t which describes the variations in ampitude of the frequency basis function over time. This yieds: = Y = Y = f t () = T These independent basis functions represent features of the individua sources. Each source is made up of a number of these basis functions which form a ow dimensiona subspace that represents the sound source. Where the subspace methods differ is in how decomposition of the origina mixture spectrogram Y into outer product basis functions is achieved. ISA achieves the decomposition by performing Principa Component Anaysis on the mixture spectrogram. Components of ow variance are then discarded to achieve ow dimensionaity. Independent Component Anaysis (ICA) [4] is then performed on the remaining components to obtain independent subspaces. The above decomposition is performed in a totay bind manner and makes no use of information about sources known to be present in the

2 mixture. A detaied description of the above decomposition can be found in []. Though an effective means of separating sound mixtures there are significant imitations to the ISA method. Firsty the assumption that the basis functions are invariant means no pitch changes are aowed in the overa spectrogram. However this is not a probem when deaing with sources that can be considered stationary in pitch such as drum sounds, making ISA suited to deaing with drum sounds. Secondy estimating the number of components to retain from PCA remains a probem. The number of components required for separation varies with the frequency and ampitude characteristics of the source sounds, and the threshod method proposed in [] cannot adequatey predict the required number of components. This resuts in the necessity of an observer to decide the number of components to retain. An attempt to overcome this probem by means of sub-band preprocessing is described in [3]. Despite these imitations ISA provides a method of overcoming the probem of identifying mixtures of drums encountered by Sianpää et a when trying to identify and transcribe mixtures of drums [5]. On the other hand PSA assumes that there exists known prior frequency basis functions f p that are good initia approximations to the actua basis functions of the sources of interest. Substituting these f p for the f in equation yieds: T Y f pt (2) = Mutipying the overa spectrogram Y by the pseudoinverse of the prior frequency subspaces yieds estimates of the ampitude basis functions, tˆ : tˆ = fppy (3) where f pp is the pseudoinverse of f p. However the ampitude basis functions returned are not independent and so ICA is carried out on tˆ to give t = Wtˆ (4) where W is the unmixing matrix obtained using ICA. Improved estimates of the frequency basis functions can then be obtained from f = ( Yt ) T p (5) PSA uses prior knowedge to obtain the most important information specificay on the sources of interest and so overcomes the probem of estimating the amount of information needed for separation that is associated with ISA. PSA aso reaxes the assumption that no pitch changes are aowed in the overa spectrogram. Instead it assumes that ony the sources of interest are stationary in pitch. This makes PSA suitabe for attempting to transcribe drums in the presence of pitched instruments. PSA is demonstrated in Figure, which shows the ampitude enveopes obtained from anaysing a drum oop using PSA. Figure. Drum oop separation using PSA The prior subspaces used in this exampe were created by anaysing arge numbers of each type of drum. An ISA-type anaysis such as described in [6] was carried out on each exampe. As mentioned previousy this amounts to carrying out PCA foowed by ICA on the spectrogram of the exampe. The first three principa components retained from the PCA step were passed to the ICA agorithm and the resuting independent frequency subspace with the argest proected variance was taken to represent the exampe. K-means custering was then carried out on the frequency subspaces for a given drum type to yied a singe subspace that best characterised a given drum type. PSA was initiay tested on 5 drum oops containing snares, kick drums and hi-hats. It achieved an overa success rate of 92.5% in successfuy identifying the drums present. This represents an improvement over the 89.5% success rate achieved using sub-band ISA on the same signas [2]. PSA was found to be better than subband ISA in correcty identifying hi-hats and was aso significanty faster than ISA or sub-band ISA due to the fact that PSA does not require the use of PCA. In tests on the same signas PSA was found to be approximatey ten times faster than sub-band ISA and five times faster than ISA. III PSA IN THE PRESENCE OF PITCHED INSTRUMENTS It was previousy noted that as the basis functions obtained by ISA are invariant no pitch changes are aowed within the sources present. It was aso noted that PSA provides a reaxation of this assumption in that this restriction now ony appies to the sources being searched for. As aready noted drum sounds meet this criterion, making PSA a vauabe too for drum transcription. As it is no onger required that a the sources present be stationary in pitch, ony the sources being searched for, it is possibe to extend PSA to work in the presence of pitched instruments.

3 However a number of issues must be addressed before PSA can be used to transcribe drums in the presence of pitched instruments. The first of these is to note that the presence of a arge number of pitched instruments wi cause a partia match with the prior subspace used to identify a given drum. This causes interference in the recovered ampitude enveope, which can in turn make detection of the drums more difficut. However it shoud be noted that pitched instruments have harmonic spectra with resuting regions of ow intensity between partias. Furthermore due to the rues of harmony used in popuar music many of the pitches payed simutaneousy wi be in harmonic reation to each other and so wi have many overapping partias. As a resut every time pitched instruments occur there wi be regions in the frequency spectrum where itte or no energy is present due to pitched instruments. It can therefore be seen that using a higher frequency resoution reduces the interference due to the pitched instruments, and as a resut improves the ikeihood of recognition of the drums. This is demonstrated in Figure 2, which shows the snare ampitude enveopes obtained from spectrograms of an excerpt from a pop song. The spectrograms had FFT sizes of 52 and 4096 respectivey. The interference due to other instruments can be seen to be greaty reduced at the higher frequency resoution, and as a resut the snare drum is more easiy identified at the higher frequency resoution. However the use of higher frequency resoution comes at the price of a reduction in the time resoution, which eads to inaccuracies in the detected onset times of the drum events. out over a greater range of the spectrum than the snare and kick drum, making it more sensitive to the presence of pitched instruments. However by noting that most of the energy of pop songs is contained in the ower region of the spectrum, it is possibe to overcome this probem. The power spectra density (PSD) of a signa gives an estimate of the average power at each point in the spectrum [6]. Dividing a spectrogram by the PSD wi emphasise those regions of the spectrum where there is ess power, in this case the upper regions of the spectrum. This resuts in improved recognisabiity of the hi-hats. This is demonstrated in Figure 3 which shows the hi-hat ampitude enveopes obtained from an excerpt from a pop song both with and without PSD normaisation. The PSD was obtained using an eigenvector method using a sma number of eigenvectors to capture ony the broad regions where most of the energy occurs. During testing of the modified PSA agorithm it was discovered that whie successfu in many cases, in some cases the agorithm did not perform correcty. Further anaysis reveaed that this was as a resut of the sensitivity of the ICA agorithm to the interference or noise due to the presence of pitched instruments remaining in the snare and kick drum ampitude enveopes. To overcome this probem a vaues in the ampitude enveope beow a set threshod are set to zero. A normaised ampitude of 0.4 was found to be a suitabe threshod for both the snare and kick drum. This operation is not carried out on the hi-hats as the interference was found to have been sufficienty eiminated by the PSD normaisation step. Figure 2. Snare enveopes at different frequency resoutions Despite the use of high frequency resoution the interference present in the hi-hat subspace was in some cases found to be consideraby greater than that in the bass drum or snare subspaces. This caused probems in trying to identify hi-hat events. The extra interference appears to be as a resut of the fact that the hi-hat prior subspace has its energy spread Figure 3. Hi-hat ampitude enveopes with/without PSD step However the threshoding operation was found to have another consequence. The resuting snare and kick drum enveopes contained arge areas of no activity, with sudden and sharp peaks occurring when a snare or kick occurred. This contrasts with the more natura peaks and decays occurring in the hihat enveope. When these very different ampitude enveopes were input to an ICA agorithm the

4 resuting independent signas contained unusua artifacts such as numerous sudden arge ampitude moduations which were detected as events where none was present. To eiminate this probem it was necessary to carry out ICA on ony the snare and bass drum ampitude enveopes, as they are comparabe in that they both contain sharp peaks and arge areas of no activity. This resuted in the correct separation of bass drums and snare drums in most cases. The hi-hat enveope is passed directy to the onset detection agorithm. Whie this gives good resuts in genera it can resut in extra errors in detection of hi-hats. As the hi-hat ampitude enveope no onger undergoes ICA the agorithm oses the abiity to distinguish between a snare occurring on its own and a snare and hi-hat occurring simutaneousy. However in many cases a hi-hat does occur simutaneousy with the snare, so this ony resuts in a sma reduction in the efficiency of the transcription agorithm. IV DRUM TRANSCRIPTION IN THE PRESENCE OF PITCHED INSTRUMENTS To test the abiity of PSA to transcribe drums in the presence of pitched instruments a drum transcription system was impemented in Matab. The system impemented deas ony with snares, bass drums and hi-hats. Due to the source signa ordering probem in the ICA step it is assumed that the bass drum has a ower spectra centroid than the snare. The system was tested on 20 excerpts taken at random from pop songs from as wide a range of styes as possibe ranging from pop to disco and rock. The drum patterns from these excerpts were transcribed by an expert istener. Because of the imperfect separation of the ICA step the ampitude enveopes were normaised and onsets over a given threshod were taken to be a drum onset. The same threshod was used for both snare and kick drums whie a ower threshod was used for the hi-hats. This refects the fact that the ampitude of the hi-hats in rea word exampes can vary widey depending on the stye of drumming. The resuts obtained are outined in Tabe. Though the resuts demonstrate the effectiveness of PSA as a method for transcribing drums in the presence of pitched instruments a greater number of errors occur than for PSA with drums ony. Possibe reasons for this are discussed beow. Type Tota Missing Incorrect % Snare Kick Hi-hats Overa Tabe. Drum Transcription Resuts In the case of the bass drums, six snare events were incorrecty identified as bass drums. These errors occurred in excerpts where a disco stye of drumming was empoyed. In these excerpts the snare drum is typicay ess bright than in the other genres of music, and so a greater chance of incorrect identification is the resut. Ony one of the incorrect bass drum detections was as a resut of a bass guitar note being identified as a bass drum. The missing four undetected bass drum events were visibe on the ampitude enveope of the excerpts in question, but were beow the threshod for detection. The bass drums at these points were audiby ower than the other bass drum events in the excerpts. In the case of the snare drum, five of the incorrect snares were as a resut of the combination of a bass drum and a hi-hat occurring simutaneousy being mistaken for snares. This happened in two excerpts. The remaining errors occurred as a resut of noise due to pitched instruments. With regards to the hi-hats the maority of incorrect identifications were as a resut of interference that had not been eiminated in the PSD normaisation step. In two cases an event with the characteristics of a hi-hat was ceary visibe in both the spectrogram and the recovered ampitude enveope, but no event of this type was audibe to the istener. These events may be genuine hi-hat events that have been masked by other audio events, but as there is no way of determining this for excerpts from commercia recordings, these onsets have been cassed as incorrect detections. In the case of the undetected hi-hats the maority of the hats were ceary visibe in the ampitude enveopes, but beow the threshod required for identification. Further improvements may be possibe by adusting the threshods for detection but there is a trade-off between reducing the number of incorrect identifications and increasing the number of missed events. Due to the imitations in the time resoution of the STFT, the detection of onset times had an average error of 0ms. It shoud be noted that this error tended to be consistent across a the drums in a given oop, so that inter-onset intervas remained consistent within a given oop. However it is sti desirabe to improve the accuracy of onset detection in PSA. It shoud be noted that these resuts were obtained without the use of any form of rhythmic modeing to predict when a given drum was most ikey to occur. V CONCLUSIONS & FUTURE WORK Prior Subspace Anaysis has been shown to be a viabe approach for the transcription of drums in the presence of pitched instruments, overcoming some of the probems associated with Independent Subspace Anaysis. Further work needs to be done to improve the correct identification of the drums and to increase the accuracy of the onset times. It is aso proposed to

5 generaise the method to dea with an increased number of drum types. REFERENCES [] Casey, M. & Westner, A., Separation of Mixed Audio Sources By Independent Subspace Anaysis Proceedings Of ICMC 2000, pp. 54-6, Berin, Germany, [2] FitzGerad, D., Lawor, B., Coye, E., Prior Subspace Anaysis for Drum Transcription, 4th AES Conference Amsterdam March 22 nd 25 th 2003 [3] FitzGerad, D., Coye E, Lawor B., Sub-band Independent Subspace Anaysis for Drum Transcription, Proceedings of the. Digita Audio Effects Conference (DAFX02), Hamburg, pp , [4] Hyvärinen A. & Oa E., Independent Component Anaysis: Agorithms and Appications. Neura Networks, 3(4-5): pp , [5] Sianpää J., Kapuri A., Seppänen J., Virtanen T., Recognition of acoustic noise mixtures by combining bottom-up and top-down processing. In proc. European Signa Processing Conference, EUSIPCO [6] Casey, M., Auditory Group Theory: with Appications to Statistica Basis Methods for Structured Audio, Ph.D. Thesis, MIT Media Lab, February 998. [7] Vaseghi, Saeed V., Advanced Digita Signa Processing and Noise Reduction, 2 nd ed. John Wiey & Sons Ltd. pp