A STUDY OF TRUMPET ENVELOPES Roger B. Dannenberg, Hank Pellern, and Istvan Dereny School of Computer Scence, Carnege Mellon Unversty Pttsburgh, PA 15213 USA rbd@cs.cmu.edu, hank.pellern@andrew.cmu.edu, dereny@cs.cmu.edu Abstract: Most synthess technques provde for some amount of parametrc control. Generatng sutable controls s a dffcult problem, especally for nstruments that admt contnuous control by the performer. The tradtonal approach to control generaton n computer musc has been note-based, but note-by-note synthess tends to overlook the nteracton between notes n a phrase. Ths study consders factors, ncludng melodc contour, artculaton, and dynamcs, that affect the shape of ampltude envelopes n trumpet performance. After showng statstcally sgnfcant varaton due to these factors, a model for trumpet envelopes s descrbed. Ths model s used wth Spectral Interpolaton Synthess to synthesze realstc trumpet performances. 1. Introducton In our efforts to synthesze the classcal trumpet, we have conducted a study of ampltude and frequency envelopes. Prevous efforts, orgnatng wth Rsset (1985), have analyzed ndvdual tones, and resynthess of ndvdual tones can sound very realstc. When solated notes are smply joned together, however, the results are nether realstc nor muscal. It s mportant to study and explan the observed varaton n envelopes so that muscal phrases can be syntheszed n a muscal manner. Chafe performed smlar studes on strngs (Chafe 1989), and Clynes ntroduced the dea that envelopes change based on local ptch contours and metrcal poston (Clynes 1987). Sundberg and colleagues created the dea of performance rules (Sundberg, Askenfelt, and Fryden 1983), whch can be appled to envelopes, and numerous studes have examned expressve performance. Researchers have also used analyss/synthess technques on extended sounds, ncludng muscal phrases, and there has been recent nterest n relatng the analyss to muscal structure (Arcos, Mantaras and Serra 1997) and performer s ntenton (Canazza, De Pol, and Vdoln 1997). Our approach uses sgnal analyss and statstcs n controlled studes of envelopes. Ths study s part of a larger project to explore the use of spectral nterpolaton n the synthess of acoustc nstrument performances. In ths approach, we model an acoustc nstrument as a mappng from two or more control parameters (e.g. ptch and ampltude) to a correspondng nstantaneous spectrum. (Serra, Rubne, and Dannenberg 1990 and Dereny and Dannenberg 1998) Realstc, tme-varyng spectra are produced smply by varyng the control parameters. An mportant challenge n ths approach s to produce approprate control parameters. A smple approach s to capture parameters from acoustc performances, but ths s lmted to the reproducton of exstng sounds. In contrast, the goal of synthess s to produce new sounds from hgh-level specfcatons. In ths work, we want to produce sounds from the nformaton typcally found n a muscal score: notes, slurs, dynamcs, and tempo. Therefore, we chose to study envelopes, how they vary, and how ther varaton mght relate to the underlyng muscal score Ths paper presents some of our fndngs. The next secton descrbes some analytcal work n whch we appled statstcal measures to extracted envelopes. Secton 3 dscusses the synthess of envelopes from symbolc score nformaton. Fnally, we descrbe future drectons for research, a summary and conclusons. 2. Analyss of Ampltude Envelopes Our frst study was nspred by Clynes (1985) envelope model n whch the overall weght of the envelope s shfted later when the note occurs n a rsng ptch contour, and earler n a fallng ptch contour. Ths makes ntutve sense: The player wll ncrease breath pressure to prepare to move upward, makng the latter part of the note louder. The opposte happens wth a fallng contour. We wanted to study ths phenomenon n a controlled experment and look for other factors that affect envelope shape. In the experment, the analyzed note s always the second note n a 3-note contour, and s always an AI 4 quarter note. The ptches of the frst and thrd notes determne the melodc contour. The frst note was ether a half step above or below or 5 to 6 half steps above or below the mddle note. Smlarly, the thrd note was ether a half step Dannenberg, R. B., H. Pellern, I. Dereny. 1998. A Study of Trumpet Envelopes. In Proceedngs of the Internatonal Computer Musc Conference. San Francsco: Internatonal Computer Musc Assocaton.
Dannenberg, Pellern, and Dereny A Study of Trumpet Envelopes 2 above or below, or 5 to 6 half steps above or below the mddle note. In addton to varyng the contour, we vared the dynamc levels and artculaton n hopes of observng the effect of these varables on envelope shape. 2.1. Data Collecton and Analyss A set of contours were performed, recorded to DAT, and transferred to sound fles for analyss. The AI 4 tones were manually extracted and analyzed. Snce we were already performng spectral analyses wth the phase vocoder n the SNDAN program (Beauchamp 1993), we used the RMS ampltude output of SNDAN, normalzed to a maxmum ampltude of 1, for our envelope data. Each envelope was automatcally trmmed, startng when the AI 4 ntally crossed a threshold of 0.1 and endng when the subsequent note crossed a threshold of 0.1. The envelopes were then normalzed to a duraton of 1.0. A total of 125 contours were recorded, varyng n nterval sze, drecton, and artculaton. 67 of these were performed mezzo forte wth normal artculaton. The center of mass or frst moment of each envelope was calculated accordng to: a t ) t a( t ) (. Statstcal tests were performed on the centers of mass to see f there was a sgnfcant dfference between the centers of mass n one category and the centers of mass n another category. The purpose of these tests was to determne whether melodc contour had a sgnfcant effect on the shape of the envelopes. Table 1 shows mean center of mass grouped usng Tukey s HSD means test. Dfferent letters ndcate that means probably come from sgnfcantly dfferent populatons (usng a 1% rsk of error). These tables show that small nterval phrases had hgher or later mean centers of mass than dd large nterval phrases, and that, for each sze, the up-up contours had sgnfcantly hgher centers of mass than the other contours. Category Mean Center of Mass Sgnfcant Groupngs Small Interval 0.443 A Large Interval 0.409 B Up-Up 0.448 A Down-Down 0.425 B Down-Up 0.417 B Up-Down 0.415 B Table 1. Separaton of Means for nterval sze and contour. All contours played mf wth normal artculaton. Table 2 shows a smlar analyss of the effect of dynamcs, and Table 3 shows an analyss of artculaton. Note that forte and pano were grouped together wth low means and mezzo s a separate group wth a hgh mean center of mass. In Table 3, artculaton clearly accounts for large varatons n the center of mass. Category Mean Center of Mass Sgnfcant Groupngs mezzo 0.452 A forte 0.420 B pano 0.415 B Table 2. Separaton of Means for dynamc level. Category Mean Center of Mass Sgnfcant Groupngs legato 0.520 A normal 0.452 B staccato 0.313 C Table 3. Separaton of Means for artculaton style. 2.2. Dscusson of Statstcal Results Overall, these results clearly show that dynamcs, artculaton, and melodc contour have sgnfcant effects upon the shape of the ampltude envelope. Notce that snce the envelopes were all normalzed n maxmum ampltude and duraton that t s n fact changes n shape that we are observng. It should be no surprse that artculaton has a major mpact upon the observed center of mass (compare Fgures 1 and 2). Recall that the note duraton s taken to be the nter-onset nterval, so the slence (f any) followng a staccato note s taken to be part of the envelope. Staccato notes are short, so the center of mass s low (early), and legato notes are sustaned, so ther center of mass s relatvely hgh.
Dannenberg, Pellern, and Dereny A Study of Trumpet Envelopes 3 More surprsng s the nteracton between dynamcs and center of mass. Both loud and soft notes show lower centers of mass. We suspect there are two factors at work here. In the case of loud notes, fast attacks are relatvely easy to perform usng the tongue as a valve to release ar suddenly, but sudden stops n the arflow are to be avoded. Perhaps n loud notes, the performer needs more tme to reduce the ar pressure n preparaton for the next note. Ths would place more of the hgh ampltude early n the note, thus lowerng the center of mass. Soft notes present the player wth another problem: attacks are smpler and cleaner wth ncreased ar pressure, but ths makes the note louder. A soluton s to use a relatvely loud attack but quckly dmnsh the sound level. Ths would also shft the center of mass toward the begnnng of the note. It s curous that, after normalzaton, loud and soft envelopes have smlar frst moments; more study s called for. The prmary motvaton for ths study was to look at contour. Whle the Up-Up contour dd n fact have a hgher center of mass as expected, Down-Down dd not have the lowest, and nterval sze had a large effect ndependent of drecton. If we assume large ntervals are preceded by a slght pause (Sundberg, Askenfelt, and Fryden 1983), then ths would lead to a lower center of mass as observed. It s nterestng that nterval sze and melodc drecton can act n opposton, an observaton whch mght be helpful n future studes. 3. Syntheszng Envelopes As descrbed earler, our goal s to model trumpet (and other) envelopes n order to drve Spectral Interpolaton Synthess. The precedng study shows that there are many parameters affectng the envelope. Many of these parameters can be obtaned drectly and easly from a symbolc muscal score. The hope s that ths nformaton s suffcent to synthesze realstc nstrumental performances. By plottng multple envelopes normalzed to unt ampltude and duraton, we get a sense for features of envelopes and the nature of varaton among envelopes. Overall, we fnd a very consstent smooth arch shape (see Fgure 1). Supermposed upon ths curve s a rapd attack, explaned by the release of stopped ar by the tongue. In consecutve notes, we also observe a short rse followed by a rapd drop at the end of the envelope, all supermposed onto the smooth arch shape. Slurs have a smlar dp and rse, but the ampltude does not drop to zero, and the dp s shorter wth slurs than n the tongued case (see Fgure 2). 3.1. The Breath and Tongue Model Fgure 1 shows a typcal trumpet envelope taken from a sequence of tongued quarter notes. Ths shape can be nterpreted based upon some smple elements of trumpet performance. Frst, the overall arch shape can be attrbuted to ar pressure n the lungs. Snce the regulaton of ths ar pressure nvolves most of the torso, t should not be surprsng to fnd a smooth overall shape as opposed to a rapdly modulated one. The rapd rse s due to the tongue, whch acts as a valve n releasng ar. As expected, there s much less of ths seen n slurred note transtons. We were surprsed to see a relatvely fast release, almost lke that caused by a damper on a pano strng, at the end of tongued notes. Ths can be explaned as the effect of stoppng ar wth the tongue n preparaton for the next note. In fact, ths dampng feature s not found on notes that are not mmedately followed by a tongued note, and t does not appear n classcal analyses of the trumpet such as the one by Moorer, Grey, and Strawn (1978). Normalzed RMS Ampltude Normalzed RMS Ampltude Normalzed Tme Normalzed Tme Fgure 1. A trumpet envelope for a mezzo forte AI 4 taken from a sequence of ascendng tongued notes. Fgure 2. A trumpet envelope for a mezzo forte C 4 taken from a sequence of ascendng slurred notes. Puttng these observatons together leads to a smple but effectve model for trumpet envelopes. A low-rate-ofchange curve orgnatng from the chest and daphragm s combned wth hgh-rate-of-change curves due to the
Dannenberg, Pellern, and Dereny A Study of Trumpet Envelopes 4 acton of the tongue. Overall ampltude s nomnally based on ptch, wth addtonal modfcatons based on dynamc markngs. Fgure 3 shows an example synthetc envelope llustratng these deas. 3.2 Fne Tunng Most of our studes looked at normalzed envelopes, so we also looked at varatons n absolute ampltude as a functon of ptch. Of course, the player has a great deal of control over dynamcs, so we analyzed a scale played at a comfortable level. We found an almost lnear relatonshp between RMS ampltude and fundamental frequency, so ths forms the bass for overall ampltude. To ths model, we must add some more subtle parameters. We found some varaton n shape wth duraton. Bascally, longer notes have a more rounded envelope wth slowly ncreasng and decreasng ampltude, ndcatng the player has more tme to shape the note wth the breath. Shorter notes have a more constant ampltude except where the tongue provdes artculaton. The general trend of the envelope (ncreasng or decreasng) s affected by the neghborng ptch contour as descrbed n the prevous secton. Breath Ampltude Attack Decay Ampltude Tme Fgure 3. A synthetc envelope showng the breath (ncludes dotted lnes) modulated by an attack and decay envelopes. The sold lne shows the result. Tme Fgure 4. The breath envelope s constructed by extractng a segment, ndcated by arrows, of a longer envelope and stretchng t to the desred duraton. Based on observatons of actual data, we developed a 10-parameter envelope model that ncorporates the noton of a breath envelope and tongue envelope. To model the breath, we smply take an actual trumpet envelope (Fgure 4), but we extract only a porton (shown by arrows) and stretch that porton to the desred duraton. If a small regon s taken from the center, the resultng envelope s farly flat. If a regon s taken from earler or later portons, the resultng envelopes wll be generally rsng (later center of mass) or fallng (earler center of mass), respectvely. Thus, a varety of shapes are avalable. The resultng breath envelope has an nstantaneous attack and decay, but these are modfed by the tongue envelope as descrbed n the next paragraph. For the tongue envelope, we used analytc functons composed from snes and exponentals, but the functons are ft (by hand) to actual data n order to derve sets of typcal parameters. Some of these parameters depend upon context. For example, notes at the end of a phrase have a short release (n ths case the breath envelope wll be nearly zero), whereas notes precedng a tongued note wll have a longer release tme that s proportonal to the note s duraton. The tmngs of these features generally vary by only tens of mllseconds, but ths small varaton has mportant audble effects, especally at note transtons. 4. Future Work At the begnnng of ths study, we had no dea how to thnk about envelopes n a systematc way, but we began work belevng that envelopes are the key to mproved musc synthess. It was thus very exctng to dscover some smple prncples that seem to be effectve n envelope synthess, most mportantly the breath and tongue model. Ths conceptual framework allows us to construct an envelope model wth relatvely few parameters. Ths, n turn, should make t possble to use machne learnng to construct functons from score parameters to envelope parameters. Another mportant drecton for future work s to apply these technques to other nstruments. It seems lkely that the envelopes of other wnd nstruments wll share many features wth those of the trumpet. If so, then t should be possble to make an effectve collecton of Spectral Interpolaton Synthess wnd nstruments. Ths technque may also apply to strngs, although there are obvous dfferences that must be accommodated.
Dannenberg, Pellern, and Dereny A Study of Trumpet Envelopes 5 Ptch varaton has not receved much attenton n our work, but vbrato s an especally mportant area for future study. 5. Summary and Conclusons We have studed trumpet envelopes as part of a larger effort to create a hgh-qualty syntheszer for the trumpet and other nstruments. Statstcal analyss of the trumpet shows that the shape of the envelope (not just the ampltude and duraton) changes systematcally accordng to melodc context, dynamc level, and artculaton. Informed by ths study, we created a computer model of trumpet envelopes n whch propertes of a symbolc score determne a number of envelope parameters. The resultng envelopes are used to drve a Spectral Interpolaton Synthess model, resultng n realstc trumpet phrases. These nclude approprate dynamcs, spectral varaton, tongued attacks, and slurred note transtons. Examples can be found at http://www.cs.cmu.edu/~rbd/musc. An mportant contrbuton of ths work s an approach n whch synthess s based on phrases rather than ndvdual notes. By lookng to melodc contour and other smple features that are readly apparent n the score, we can account for a great deal of varaton n envelope shape. Ths s an mportant step toward the synthess of muscal phrases. Acknowledgments Elzabeth Bunn, at the South Carolna Governor s School for Scence and Mathematcs, provded assstance wth the statstcal analyss of trumpet envelopes and served as academc advsor to the second author. Jm Beauchamp provded us wth the SNDAN sgnal analyss tools. References Arcos, J. L., R. L. de Mantaras, and X. Serra. 1997. SaxEx: a case-based reasonng system for generatng expressve muscal performances, n Proceedngs Internatonal Computer Musc Conference 1997. San Francsco: Internatonal Computer Musc Assocaton, pp. 329 336. Beauchamp, J. 1993. Unx Workstaton Software for Analyss, Graphcs, Modfcaton, and Synthess of Muscal Sounds. Audo Engneerng Socety Preprnt, No. 3479 (Berln Conventon, March). Canazza, S., G. De Pol, A. Roda, and A. Vdoln. 1997. Analyss and synthess of expressve ntentons n muscal performance. In Proceedngs of the Internatonal Computer Musc Conference 1997. San Francsco: Internatonal Computer Musc Assocaton, pp. 113 120. Chafe, C. 1989. Smulatng Performance on a Bowed Instrument. In M. Mathews and J. Perce (Eds.): Current Drectons n Computer Musc Research. Cambrdge MA: M.I.T. Press, pp. 185 198. Clynes, M. 1985. Secrets of Lfe n Musc: Muscalty Realsed by Computer. In Proceedngs of the 1984 Internatonal Computer Musc Conference, Computer Musc Assocaton, pp 225 232. (Note: 1984 proceedngs were publshed n 1985.) Clynes, M. 1987. What Can a Muscan Learn About Musc Performance From Newly Dscovered Mcrostructure Prncples (PM and PAS)? n A. Gabrelsson (Ed.): Acton and Percepton n Rhythm and Musc, pp. 201 233. Publcatons ssued by the Royal Swedsh Academy of Musc No. 55. Dereny, I. and R. B. Dannenberg. 1998. Syntheszng Trumpet Performances. In Proceedngs of the Internatonal Computer Musc Conference. San Francsco: Internatonal Computer Musc Assocaton. Moorer, J. A., J. Grey, and J. Strawn. 1978. Lexcon of Analyzed Tones (Part III: The Trumpet). Computer Musc Journal 2(2), pp. 23 31. Rsset, J.-C. 1985. Computer Musc Experments 1964. Computer Musc Journal 9(1) (Sprng), pp. 11 18. Serra, M.-H., D. Rubne, and R. B. Dannenberg. 1990. Analyss and Synthess of Tones by Spectral Interpolaton,. Journal of the Audo Engneerng Socety, 38(3) (March), pp. 111 128. Sundberg, J., A. Askenfelt, and L. Fryden. 1983. Muscal Performance: A Synthess-by-Rule Approach. Computer Musc Journal 7(1), pp. 37 43.