Melody Recognition: Effects of Articulation Format

Transcription

1 Meody Recognition: Eect o Articuation Format Stephen Wee Hun im (gmwh@nu.edu.g) Winton D. Goh (pygohw@nu.edu.g) Department o Pychoogy, Nationa Univerity o Singapore Bock AS4, 9 Art ink, Singapore interpretation o that rendition. Whie Raman (1993) ha uggeted that ony the atract tructura inormation i encoded into ong-term memory (TM), other have reported that urace eature, uch a timre (e.g., Peretz, Gaudreau, & Bonne, 199) and tempo (e.g., Hapern and Müenieen, 200), are ao encoded into TM during a meody recognition tak. In muic, the way a meody i articuated hape it urace appearance. In the extant iterature that examined the eect o urace characteritic on meody recognition perormance, it i urpriing that no tudy ha expored the eect o articuation ormat, even though it i a eature that i commony manipuated y oth compoer and perormer. Trained muician commony deine articuation a whether the muic (e.g., meody) i payed in a egato (i.e., continuou) or taccato (i.e., detached) ormat. Becaue no one ha tudied the inuence o articuation on meody recognition, our initia motivation wa to add to that iterature. Thu ar, memory repreentation that uerve expicit recognition o meodie appear to e ormed y a highy peciaized aociation that ind together characteritic uch a timre and tempo with meody identity. It i thu attractive to ak whether the articuation eature i tied to a meody identity and computed during the perceptua anayi o the meodic input. By addreing thi quetion, we hope to expicate more uy the centra idea that variaiity in urace eature, aong with the ideaized canonica tructure o muic, i important in muic perception and proceing. To examine the eect o articuation ormat on meody recognition, we deigned the meody to occur either uy in egato orm, uy in taccato orm, or in mixed articuation ormat (i.e., a comination o egato and taccato component). When the meody wa payed in taccato orm, the duration o each note in the meody wa manipuated to at 10% o the u duration when the note wa payed in egato orm. The chematic o the eight dierent articuation ormat i hown in Figure 1. Thee ormat are coded a,, a,, c, d, e, and : The egato and taccato ormat are areviated a ormat and, repectivey, whie the ix mixed-articuation ormat oow an aphaetica ytem o coding or eae o reerence. Each et o our oxe repreent equentiay the our ar o the meody repectivey. Taking ormat or intance, the meody open in taccato orm (i.e., the note o the meody are articuated y the intrument in a dijointed ahion) or the irt ar, witche to egato orm (i.e., the note are now articuated moothy Atract Variou urace eature timre, tempo, and pitch inuence meody recognition memory, ut articuation ormat eect, i any, remain unknown. For the irt time, thee eect were examined. In Experiment 1, meodie that remained in the ame, or appeared in a dierent ut imiar, articuation ormat rom tudy to tet were recognized etter than were meodie that were preented in a ditinct ormat at tet. A imiar articuation ormat adequatey induced matching. Experiment 2 reveaed that initia perceptua (di)imiarity a a unction o the ocation o articuation (mi)match etween two intance o the meody did not accuratey determine dicrimination perormance. An important oundary condition o the matching proce wa deined: Whether matching occur depend on the phyica quantity, rather than ocation, o it etween the memory trace and the recognition proe, uggeting a goa matching advantage eect. Keyword: Meody recognition memory; articuation ormat eect; goa matching advantage Introduction When we hear a piece o muic, we detect and occaionay rememer phrae, moti, theme, yncopation, upenion, tonic chord, cadence, and o on. We recognize the intrument paying the meody, or even identiy with the emotion o the peciic muician perorming the work. To thi end, what exacty i the nature o menta repreentation that underie the muic experience? To addre thi quetion, it i ueu to irt recognize that there are two kind o inormation in muic, namey atract tructure and urace characteritic (Trainor, Wu, & Tang, 2004). The atract tructure entai the reative pitche and ratio o the duration etween adjacent muica note, regarde o the individua note aoute pitch eve or ength per e. Surace characteritic, in contrat, contain the non-tructura apect o the muic, uch a aoute pitch, tempo, and timre. Both the atract tructure and urace characteritic contriute toward muica interpretation. Repreenting the atract tructure enae recognition o a meody acro dierent perormance, and muica variation o a moti within a muica compoition (arge, Pamer, & Poack, 1995). For exampe, Happy Birthday retain it identity and i readiy recognized even when it i payed or ung in variou key and tempo, or y dierent voice or intrument. Yet, thee very urace characteritic ead u to identiy the peciic muician and unique perormance o the work, deining the emotiona 423

2 in a continuou manner) y the econd ar, return to taccato mode in the third ar, and inay coe with a ong-ounding note in the ina ar. compared to when the meodie appeared in a ditinct articuation ormat. Method a c d e egato taccato Participant Forty-even introductory pychoogy tudent participated or coure credit. Materia The timuu et contained 4 nove monophonic meodie (ee Figure 2 or ampe). An equa numer o our-ar meodie were compoed in the tonaity (key) o C major, C minor, G major, or G minor. The meodie tarted either on the tonic, mediant, or dominant, ut away ended with a inge ong note on the tonic o their home key. Each meody wa written in impe tripe or impe quadrupe time, ating approximatey ix econd or 7.2 econd repectivey. The meodie were contructed uing the Finae 2009 otware, and aved a.wav ound ie. inge ong note Figure 1: Schematic o the eight dierent articuation ormat manipuation. Experiment 1 In Experiment 1, we aked two quetion: (1) I articuation eature inormation retained in TM, and (2) what i the roe o eature imiarity in meody recognition memory? Our irt goa wa to invetigate the eect o manipuating articuation context on meody recognition. The hypothei wa that to the extent that articuation ormat inormation i not eraed rom, ut i in act preerved in, TM, dicrimination perormance ought to improve when od meodie are repeated in the ame articuation ormat, a compared to when the meodie appeared in a ditinct articuation ormat during the recognition tage. In addition, we recognized that extant tudie that examined urace eature eect have ued tet timui that were denoted a either o the ame or dierent ormat, negecting eect that coud arie rom varying magnitude o intermediate perceptua dierence. For intance, Peretz et a. (199) preented meodie in timre at tet that were either the ame a, or ditinct rom, thoe ued at tudy; Hapern and Müenieen (200) made the tempo change in atered tune arge enough to e perceptie (p. 137). Eect o ine-grained perceptua detai o urace eature, uch a tempo or timre, have een omewhat overooked, o it i uncear whether thee detai actuay contriuted to the diparate urace eature eect oerved in the iterature. A uch, a econd goa wa to ae the contriution o ine perceptua detai in meody recognition memory, y incuding a imiar-articuation-ormat condition. We pecuated that to the extent that articuation imiarity contitute an integrated part o the matching and retrieva procee invoved in meody recognition, perormance ought to improve even when od meodie are teted with a dierent ut imiar articuation ormat, a Figure 2: Sampe o the 4 meodie ued. Prior to conducting Experiment 1, we irt derived a mutidimeniona articuation map uing MDS technique (Kruka & Wih, 197) that how the imiarity reation etween the individua articuation ormat that wi e ued a the timuu materia. Thi procedure wa neceary to enure that the eection o peciic articuation ormat or 2 a Dimenion d c -1 e Dimenion 1 Figure 3: Two-dimeniona MDS oution or eight articuation ormat. 424

3 the vioin. The eion conited o two part the memorization phae and the recognition phae. The orthcoming recognition tet wa made known to participant eore the memorization phae tarted. Participant were tod to ienty memorize each meody that wa payed through the headphone. At the tart o each tria, a ready prompt wa dipayed on the monitor or one econd, ater which it wa deeted. One econd ater, a meody wa payed over the headphone; the meody wa repeated 00 m oowing it irt preentation. Participant then preed the pace key to proceed to iten to the next meody. Thi equence perited unti a 24 meodie had een preented. The meody preentation equence wa randomized acro participant. Foowing the memorization phae, participant were irt preented with verion o two we-known meodie Mary had a itte am and ondon ridge i aing down that varied in their articuation ormat to cariy the deinition o orm. Ater which, the recognition tet egan. On each tria, the ready prompt appeared or one econd and diappeared. 00 m ater, the quetion Did you hear thi meody in Part 1? wa dipayed, and a inge meody wa payed through the headphone. Participant were tod to pre the Ye utton on the Seria Repone Box i they thought they had heard the meody earier, regarde o the origina orm (i.e., articuation ormat) that the meody wa preented in. Otherwie, they were tod to pre the No utton. Participant were tod to repond a accuratey a poie. No eedack wa provided on any o the tria. A new tria wa tarted ater a utton repone. ue in the uequent main experiment can e aed on ojective meaure o the degree o perceived imiarity among dierent articuation ormat. Sixteen tudent rom the ame popuation ampe ut who did not participate in the main experiment rated the pairwie imiarity o the eight articuation ormat acro our dierent meodie uing a 7-point ikert cae. The two-dimeniona MDS oution (Kruka tre =.15, R2 =.5) or the eight articuation ormat appear in Figure 3. The interpretation i that the urther away two articuation ormat are poitioned rom each other in pace, the more perceptuay ditinct they are. Two dierent comination o articuation ormat were eected or meody preentation. For each comination, the articuation ormat are ited in the order that contitute the ame-, imiar-, and ditinct-articuation context condition, repectivey: (1),, and (2),,. Thee et were created or counteraancing purpoe decried in the procedure. Apparatu Computer equipped with 16-it ound card were ued or the experiment. Participant received the igna through a pair o Beyerdynamic DT150 headphone at approximatey 70 db SP. The timui were preented uing E-prime 1.2, and data were coected uing the PST Seria Repone Box (Schneider, Echman, & Zuccootto, 2002), with the et- and right-mot utton o the uttonox aeed No and Ye repectivey. Deign The 4 meodie were divided equay into two it. One it wa deignated to conit o od meodie whie the other to conit o new meodie. At tudy, a the 24 od meodie were preented uing a inge articuation ormat. In the tet phae, the 24 new meodie were divided among three articuation ormat, where eight meodie were aigned to e preented in the ame ormat, eight in a imiar ormat, and the remaining eight in a ditinct ormat. For the 24 od meodie, ikewie, eight were aigned to the ame-articuation context condition, eight to the imiararticuation context condition, and the remaining eight to the ditinct-articuation context condition (ee Tae 1). Reut and Dicuion Tae 2 preent the pattern o reut or d' perormance acro the three articuation-context condition. There wa a reiae main eect o articuation context, F(2, 90) = 3.94, MSe = 0.36, p <.05. Pairwie comparion reveaed that participant were igniicanty etter at dicriminating meodie preented with the ame articuation ormat than they were at dicriminating meodie preented with a ditinct articuation ormat, t(46) = 2.42, p <.05; participant ao perormed etter when meodie appeared in a imiar articuation ormat than they did when meodie appeared in a ditinct ormat, t(46) = 2.03, p <.05. Procedure Ha o the participant were randomy aigned to iten to meodie payed y the carinet, whie the other ha were randomy aocated to iten to meodie payed y Tae 1: Summary o Experiment 1 Deign Memorization Study meodie Recognition Tet meodie (Od) Tet meodie (New) Articuation ormat context Simiar Ditinct Simiar Ditinct Set comination 1 articuation ormat Set comination 2 articuation ormat 425

4 Dicriminaiity did not dier etween the ame- and imiar-articuation context condition, t < Thi pattern o reut indicate that dicriminaiity increaed igniicanty o ong a meodie were teted in at eat a imiar articuation ormat. Experiment 2 A irt examination o the articuation imiarity caing oution hown in Figure 3 revea that the greater the amount o phyica articuation match etween two intance o a meody, the more imiar they were perceived to e. For intance, ormat d and, each containing two ar o taccato component, were perceived a imiar to each other. But a coer ook at the caing oution revea that ony when the articuation ormat o two intance o the meody matched at the meody onet woud the two intance o the meody e perceived a imiar to each other. Thi interpretation can expain why ormat e wa perceived a rather dierent rom ormat d and even though each o thee ormat contained two ar o taccato component. Thi oervation i intriguing ecaue two articuation ormat, given the ame quantitative amount o articuation match, coud in act e perceived a dierent rom each other due to the act that the match did not occur at the meody onet. We thereore purued a third quetion here: Woud thi perceptua diimiarity etween two intance o the meody (e.g., in ormat d and e) due to the ocation o the (mi)match hamper dicrimination perormance during the tet tage, even when oth intance contain the exact ame quantity o articuation match (e.g., two ar o taccato component)? The goa wa to iuminate the underying nature o the matching proce in meody recognition memory, and we hypotheized that to the extent that perceptua diimiarity, a a unction o the ocation o (mi)match in ormat, aect matching etween tudy and tet, dicrimination perormance ought to e hampered when od meodie that were originay payed in, ay, ormat are repeated in ormat e (i.e., perceptuay diimiar ormat) at tet, a compared to when the meodie are repeated in ormat d or (i.e., perceptuay imiar ormat) at tet, athough ormat d, e, and each contain the exact ame quantity (i.e., two ar) o taccato component. Tae 2: Dicrimination Perormance (d') Acro Articuation-Context Condition in Experiment 1. M SD Articuation context Simiar Ditinct The preent data reveaed an advantage in meody recognition or ame-articuation repetition over ditinctarticuation preentation. There wa ao an advantage in meody recognition or imiar-articuation preentation over ditinct-articuation preentation. An interpretation aed on the the now-caic encoding peciicity ramework (Tuving & Thompon, 1973) i apt. Under thi ramework, the eectivene o a retrieva cue depend on it degree o reatedne to the encoding o an item at irt. Our view i that urace (articuation) and tructura attriute o a meody are tored together in the TM trace. Meody recognition i reiae when a peciic match etween the epiodic memory trace and the proe occur, ut i hampered when there i a mimatch. The comparion o hared propertie etween the memory trace and the proe impie that item imiarity per e contitute an integra part o the retrieva proce. In act, the degree o imiarity among the eature o the exempar trace in memory and the target proe orm a centra apect in exempar mode o memory and categorization (Giund & Shirin, 194; Hintzman, 19). Memory theorit have aumed that memory or a timuu i reay memory or eature contained in that timuu. The goa matching approach (ee Cark & Gronund, 1996) ugget that thee eature in a tet item, when matched with the eature that have earier een tored in memory, evoke a amiiarity igna. Speciicay, the greater the degree o match i, the tronger the igna wi e. In our cae, when a meody wa re-payed in the ame or in a imiar articuation ormat at tet, there are many overapping eature etween the articuation ormat o the two meody intance rom tudy to tet. Thee overap preumay contriute to a trong ene o amiiarity igna evoked y reemance to the tudied meody (ee Ceary, 2004). In contrat, when the meody appeared at tet in a ditinct ormat, there are ew overapping eature with the meody origina ormat. A uch, the amiiarity igna i preumay weaker, which hinder meody dicrimination. The preent experiment ugget that when matching occur, meody recognition perormance i reiae at tet. Experiment 2 wa deigned to etaih an important oundary condition which determine whether thi matching proce woud prevai (or ai). Method Participant Sixty-our pychoogy undergraduate participated. None had participated in Experiment 1. Materia, Apparatu, Deign, and Procedure The materia and procedure were eentiay the ame a thoe o Experiment 1, with a ight modiication in materia. Baed on Figure 3, our dierent comination o articuation ormat were eected or meody preentation. For each comination, the articuation ormat are ited in the order that contitute the ame-, imiar-, and ditinctarticuation context condition repectivey: (1), d, e, (2),, e, (3),, a, and (4), c, a. Set comination wa counteraanced acro participant. Reut and Dicuion Tae 3 preent the pattern o reut or d' perormance acro the three articuation-context condition. There wa no reiae main eect o articuation context, F <

5 Dicriminaiity etween the ame-, imiar-, and ditinctarticuation context condition did not dier reiay. Articuation ormat did not inuence perormance. condition o matching oerved in meody recognition under which matching woud (or woud not) e ucceu. Tae 3: Dicrimination Perormance (d') Acro Articuation-Context Condition in Experiment 2. Severa tudie have demontrated that the ateration o the initia part o a ound can aect the recognition o muica intrument (e.g., Berger, 1964; Grey & Moorer, 1977). Thee inding ugget that tempora eature are important in timre perception and muic proceing at arge. Yet, Experiment 2 ugget that atering the initia part o the articuation ormat (i.e., at the onet o a meody) did not inuence dicrimination perormance. In expaining thee data, we oer a goa matching advantage interpretation which ind it root in Getat pychoogy. A aic poition o the Getat view i that a whoe i quaitativey dierent rom the compex that one might predict y conidering ony it part. Under thi view, whoe are organized prior to perceptua anayi o their propertie and component in perceptua organization. Navon (1977) propoed that perceptua proceing tart with goa tructuring and ater move toward more ine-grained anayi. Thi propoa wa termed a the goa precedence hypothei. Thi hypothei ha een teted y tudying the perception o hierarchica pattern in which arger igure are contructed y uitae arrangement o maer igure. An exampe i a et o arge etter contructed rom the ame et o maer etter having either the ame identity a the arger etter or a dierent identity (ee Figure 4). The arger etter i conidered a higher-eve unit reative to the maer etter, which are, in turn, ower-eve unit. Propertie o the higher-eve unit are conidered more Genera Dicuion M SD Articuation context Simiar Ditinct Experiment 1 uggeted that articuation propertie are ound with the meody tructura identity. Surace eature inormation o the meody i irt encoded and tored in the memory trace, and ater ued to retrieve the meody. Becaue a ame- or imiar-eature repetition contitute an exact, or at eat a coe, match with the memory trace or the od meody, the trace ecome more aient than the other competing trace, enhancing dicrimination perormance. On the other hand, a ditinct-eature preentation woud not match with the trace or the od meody, thu perormance i hampered. The interpretation i that given a retrieva cue that coincide with the initia encoding o the meody in term o it urace propertie, the cue woud hep the meody to e recovered at tet. But Experiment 2 reveaed that initia perceptua (di)imiarity, a a unction o the ocation o eature (mi)match etween two intance o the meody, did not accuratey determine dicrimination perormance. When two intance o the meody are perceived a dierent rom each other rom tudy to tet, matching preumay woud not occur. Yet, ome orm o matching mut have occurred depite the perceptua mimatch ecaue the overa dicrimination perormance (in the ditinct articuation condition) wa good, average d' = Vaue o d' etween 1 and 2 uuay repreent good yeno recognition perormance (Neath & Surprenant, 2003, p. 202). To urther jutiy that thi wa good perormance, we conducted three panned comparion on the d' data. The irt and econd comparion etaihed that the data et etween Experiment 1 and 2 were comparae: Perormance in the ame-articuation condition, a we a perormance in the imiar-articuation condition, acro oth experiment did not dier, t < 1.2, p >.21. The third comparion ued perormance in Experiment 1 ditinct articuation condition a aeine, and reveaed that perormance in Experiment 2 ditinct-articuation condition reiay exceeded perormance in thi aeine condition, t(109) = 3.44, p <.01, impicating good dicrimination perormance in thi cae. Thu, the ogica inerence i that whether matching woud occur i ikey to e contingent on the aoute phyica quantity o match etween the memory trace and the recognition proe per e, rather than the perception o diimiarity due to the ocation o (mi)match in the eature attriute. Thee data deined an important oundary Figure 4: An exampe o Navon (1977) type hierarchica timui. arge E and H are compoed uing ma E and H. 427

6 goa than propertie o the ower-eve unit y virtue o their poition in the hierarchica tructure. In a typica experiment, oerver are preented with uch timui and are required to identiy the arger (i.e., goa) or the maer (i.e., oca) etter in dierent tria. Goa advantage i oerved, where the goa etter i identiied ater than the oca etter. Our view i that an anaogou goa advantage mechanim operate in the matching proce ound in meody recognition. The genera articuation ormat o the meody (i.e., whether the meody i overa preented in a taccato or egato ormat) i conidered a higher-eve unit reative to the peciic ormat o individua ar, which are, in turn, ower-eve unit, and propertie o the higher-eve unit are conidered more goa than propertie o the owereve (oca) unit aed on their poition in the hierarchica tructure. In order or matching to occur, that there i a goa match aed on the aoute quantity o match etween the memory trace and the recognition proe per e i more critica, a compared with whether there i a oca match etween the articuation ormat at the onet o the tet meody and the ormat at the onet o the tudy meody. Once goa matching attain, meody dicrimination perormance i enhanced. The preent goa matching advantage hypothei can e veriied in a uture tudy that manipuate the overa (goa) and oca matche in, ay, timre etween two intance o a meody, y peciicay atering the timre at variou tempora point (e.g., the onet) o the meody. Other coud ae the eect o urace eature that have yet to receive attention, uch a the ue o ornament or phrae oundarie. More roady, uture invetigation can extend to the domain o peech perception. There had een coniderae work which argued or a commonaity etween muic and peech proceing (ee Pate, 2003), and comparing thee two procee can ead to an undertanding o wider (and potentiay hared) principe o perceptua categorization and tempora organization acro rain area (McMuen & Saran, 2004; Pate, 2003). Thu, it i o interet whether the preent eect woud emerge in peech. There i a arge ody o data uggeting that taker voice, a urace eature o poken anguage, i encoded into TM. Speciicay, od word were recognized etter when they were teted in a voice that matched with the origina voice that originay poke the word at tudy, than when the voice did not match (ee Goh, 2005 or a review). Yet, the oundarie that permit (or prevent) thi match in a peech context are not we deined. It i worthwhie to expore the extent to which peech recognition perormance i driven y the aoute match in the phyica propertie o voice etween two intance o peech and/or the ocation o match per e (e.g., in a entence context). Cark, S. E., & Gronund, S. D. (1996). Goa matching mode o recognition memory: How the mode match the data. Pychonomic Buetin & Review, 3, Ceary, A. M. (2004). Orthography, phonoogy, and meaning: Word eature that give rie to eeing o amiiarity in recognition. Pychonomic Buetin & Review, 11, Giund, G., & Shirin, R. M. (194). A retrieva mode or oth recognition and reca. Pychoogica Review, 91, Goh, W. D. (2005). Taker variaiity and recognition memory: Intance-peciic and voice-peciic eect. Journa o Experimenta Pychoogy: earning, Memory, and Cognition, 31, Grey, J. M., & Moorer, J. A. (1977). Perceptua evauation o yntheized muica intrument tone. Journa o the Acoutica Society o America, 62, Hapern, A. R., & Müenieen, D. (200). Eect o timre and tempo change on memory or muic. The Quartery Journa o Experimenta Pychoogy, 61, Hintzman, D.. (19). Judgment o requency and recognition memory in a mutipe trace memory mode. Pychoogica Review, 95, Kruka, J. B., & Wih, M. (197). Mutidimeniona caing. Newury Park, CA: Sage. arge, E. W., Pamer, C., & Poack, J. B. (1995). Reduced memory repreentation or muic. Cognitive Science, 19, McMuen, E., & Saran, J. R. (2004). Muic and anguage: A deveopmenta comparion. Muic Perception, 21, Navon, D. (1977). Foret eore tree: The precedence o goa eature in viua perception. Cognitive Pychoogy, 9, Neath, I., & Surprenant, A. M. (2003). Human memory: An introduction to reearch, data, and theory. Toronto: Wadworth. Pate, A. D. (2003). anguage, muic, yntax and the rain. Nature Neurocience, 6, Peretz, I., Gaudreau, D., & Bonne, A. (199). Expoure eect on muic preerence and recognition. Memory & Cognition, 26, Raman, D. (1993). anguage, muic, and mind. Camridge, MA: MIT Pre. Schneider, W., Echman, A., & Zuccoott, A. (2002). EPrime Uer Guide. Pitturg: Pychoogy Sotware Too Inc. Trainor,. J., Wu,. & Tang, C. D. (2004). ong-term memory or muic: Inant rememer tempo and timre. Deveopmenta Science, 7, Tuving, E., & Thompon, D. M. (1973). Encoding peciicity and retrieva procee in epiodic memory. Pychoogica Review, 0, Reerence Berger, K. W. (1964). Some actor in the recognition o timre. Journa o the Acoutica Society o America, 36,