25 Chapter Two: Long-Term Memory for Timbre Task In a test of long-term memory, listeners are asked to label timbres and indicate whether or not each timbre was heard in a previous phase of the experiment (i.e., whether the timbre is a target or a lure). Hypotheses Target timbres that are consistently labeled will receive higher recognition scores than will target timbres that are inconsistently labeled or not labeled at all. Lure timbres that are labeled will be correctly rejected with greater frequency than lure timbres that are not labeled. Synthesized timbres that are explicitly named during training will receive higher recognition scores than will synthesized timbres that are not explicitly named during training. 1 Participants All participants were volunteers enrolled in the Musicianship and Theory program at the University of Iowa School of Music. 2 The 58 participants (29 male, 29 female) ranged in age from 18-42 years (average age 20.3 years). All but four participants were music majors. 1 No training effects are hypothesized for acoustic instruments because both their timbres and names are assumed to be familiar prior to this experiment. 2 Specifically, Musicianship and Theory I (regular first-semester theory) and Musicianship and Theory III (third-semester theory), which are required courses for undergraduate music majors.
Chapter Two 26 Stimuli Twenty-three acoustic instruments (played by faculty and students at the University of Iowa) were recorded performing the same ten-note diatonic pattern in C major. 3 Thirteen synthesized timbres were chosen from Studio Vision s CyberSynth patch library and programmed to play the ten-note diatonic pattern in C major. 4 The volume of each note was randomized within a narrow range in order to imitate some of the irregularities of acoustic instruments. (A complete list of all sound sources used in this dissertation, including brief descriptions of synthesized timbres, is included in appendix A.) Each ten-note pattern from each sound source (synthesized and acoustic) was normalized in volume and edited to a length of 4.00 seconds. Each stimulus for all phases of the experiment consisted of one of these four-second recordings of the same ten-note melody. Equipment Acoustic instruments and speech were recorded monophonically using a high-quality microphone. Although the microphone itself was analog, sounds were recorded in a digital format using Sound Designer II on a Macintosh computer. All synthesized sounds were produced on a Macintosh 3 More correctly, twenty-three acoustic timbres were recorded. Certain instruments were recorded twice, with and without mutes. A single ten-note melody was used throughout this experiment: Performers selected the octave that was most comfortable for each instrument. Thus, each instrument was played in its most characteristic range. 4 Instruments that produced parallel fifths were transposed, if necessary, so that the most prominent portion of the sound was in C major.
Chapter Two 27 computer using CyberSynth patches from Studio Vision. All editing was done digitally on a Macintosh computer using Sound Designer II and/or Sound Edit 16. The final CD was burned using a Yamaha CD-RW drive and was played for participants over a high-quality stereo system in a quiet classroom. Design and procedure Figure 2.1 outlines the design of this experiment. During an initial training period, listeners heard the same ten-note melody in 24 different timbres and learned to identify twelve of these timbres by name. (Names were presented both verbally and in writing.) Twelve of the 24 timbres were explicitly named, and the remaining twelve timbres were heard but not explicitly named. Sixteen of the 24 timbres heard during training were played by acoustic instruments, and the remaining eight timbres were produced electronically. Half of the acoustic timbres and half of the electronic timbres were explicitly named during training, and the other half were heard but not explicitly named. Each named timbre was first presented individually. Participants then heard this target timbre juxtaposed with another timbre and indicated whether the first or second timbre of this pair was the target timbre. The same target timbre was then juxtaposed with a different timbre and participants again indicated whether the first or second timbre was the target. The same training procedure was repeated for three additional target timbres. (This component of the training procedure follows Logan, Lively, and Pisoni [1991].) Once four named timbres had been presented using this method, listeners heard a series of sixteen different timbres the four timbres just learned plus twelve additional timbres. Whenever they heard one of the four
Chapter Two 28 target timbres, participants named it; for all other timbres, participants responded, None of the above. This pattern of presenting four timbres individually, followed by two paired comparisons, followed by a sixteenelement recognition series was repeated until twelve timbres were learned. At the end of the training period, listeners heard all twelve of the named timbres in a random order. Immediately after hearing each timbre, listeners responded with the timbre s name. By the end of the training period, participants had heard 24 timbres exactly five times each. Twelve of the timbres were explicitly named (as described above), while the other twelve were heard in the paired comparisons or the sixteen-element recognition series and were never named. The primary purpose of the none of the above response was to prevent students from explicitly naming the additional timbres. 5 Twelve additional timbres used in later phases of the experiment were neither heard nor mentioned during training. Immediately after the training period, listeners heard Series #1, an assortment of 24 timbres playing the same ten-note melody. Timbres fell into one of three categories: eight had been explicitly named during training, eight had been heard but not explicitly named, and the remaining eight were not heard at all during training. Each stimulus was preceded by a spoken question number and followed by 15 seconds of silence during which listeners were asked to write the name of the timbre. If they could not identify any given timbre, listeners were specifically instructed to invent a name even if it would make sense only to them. When this phase of the experiment was 5 Of course, listeners were free to identify these timbres silently.
Chapter Two 29 completed, listeners were told to study their written lists for one minute before handing them in. In Series #2, the final portion of the experiment, listeners heard another assortment of 24 timbres playing the same ten-note melody. As before, timbres fell into one of three categories: eight had been explicitly named during training, eight had been heard but not explicitly named, and eight were not heard at all. Stimuli were again preceded by a spoken question number. This time after each timbre presentation, listeners had 15 seconds of silence during which to write the name of the timbre (again inventing a name if necessary) and indicate whether they had heard this timbre as part of Series #1 by circling the appropriate number on their printed response sheets: 1 = I m sure I heard this timbre on the first series 2 = I think I heard this timbre on the first series 3 = I m guessing I heard this timbre on the first series 4 = I m guessing I didn t hear this timbre on the first series 5 = I think I didn t hear this timbre on the first series 6 = I m sure I didn t hear this timbre on the first series A script for the spoken directions is included in appendix B. Twelve of the 24 timbres were new and the other twelve had been heard during Series #1. Four of the new timbres had been explicitly named during training, four had been heard but not named, and four had not been heard; this ratio was exactly the same for the recurring timbres. (Please refer to figure 2.1.) Every effort was made to keep the three different training categories (explicitly named, heard but not named, and not heard during training) as equivalent as possible with regard to timbre familiarity. All three training categories contained the same proportion of commonly heard timbres (e.g., trumpet, flute, violin), less commonly heard timbres (e.g., English horn, bass clarinet, alto flute), and completely unfamiliar timbres (i.e., electronic
Chapter Two 30 instruments not reminiscent of any known Western acoustic instrument). The same care was taken to make Series #1 and Series #2 as equivalent as possible with regard to timbre familiarity. Scoring For clarity, circled responses about whether each timbre had been heard on Series #1 were converted to scaled scores that reflect whether, and with how much confidence, participants judged the stimulus correctly. A score of 2.5 represents a correct response (i.e., either a hit or a correct rejection) at the highest level of confidence, -2.5 represents an incorrect response (i.e., either a miss or a false alarm ) at the highest level of confidence, and all other responses fall at one-point intervals in between. 6 For example, if a new timbre were presented and a participant circled 5 I think I didn t hear this timbre on the first series this would be scored as 1.5. A response of 3 for the same stimulus would be scored as -.5. No individual response is recorded as a zero, but it is possible for multiple responses to average at zero, indicating chance performance. Written responses for the Series #2 were classified in several ways: whether the label provided was correct, whether it was applied consistently (if applicable), and whether it appeared on the response sheet for Series #1. Labels were assigned to one of four categories according to their accuracy: correct, slightly incorrect, incorrect, and no label. These categories are probably best explained with an example. One of the timbres heard was a 6 This change in scoring does not affect the experimental results, because in either case responses are expressed on an equally-spaced six-point scale. The primary advantage of this system is that there is no need to remember whether 1 or 6 was the most correct response for any given stimulus.
Chapter Two 31 trumpet with a Harmon mute. Labels of muted trumpet were classified as correct, trumpet was classified as slightly incorrect, and horn was classified as incorrect. 7 If a participant didn t attempt to identify the sound or wrote don t know, this was classified as no label. In general, labels were either quite clearly correct or incorrect and required little interpretation. All acoustic instruments and those electronic sounds that had been explicitly named during training were included in this analysis; electronic sounds that were not explicitly named during training were not included since correct and incorrect labels could not reasonably be distinguished in these cases. It was possible, however, to assess whether listeners labeled unfamiliar electronic sounds consistently that is, whether listeners applied the same label when they heard a timbre during different phases of the experiment. As alluded above, participants were specifically instructed before both Series #1 and Series #2, If you don t know the correct name for an instrument, feel free to invent something even it if will only make sense to you. Most listeners supplied a name for every timbre, regardless of whether it had been heard during training, and regardless of whether it had been produced by an acoustic or an electronic instrument. It was therefore possible to compare labels for recurring timbres and classify them as consistent, slightly inconsistent, or inconsistent. In most cases, it was possible to apply the same standards described earlier (for instance, trumpet and muted trumpet would be classified as slightly inconsistent). However, invented labels for electronic sounds were sometimes harder to classify. When labels used essentially the same words, 7 To ensure consistency and reduce bias, an independent scorer was consulted when a label was not clearly correct or clearly incorrect.
Chapter Two 32 they were considered consistent (for example, the labels dark and fuzzy and fuzzy and dark would be scored as consistent, as would the labels bell with long echo and bell with long reverb ). A label that included a substantial subset of the original description would be considered slightly inconsistent (for example, wavy sound with a pop and a buzz vs. wavy sound with a buzz ). Labels that were at least half different were considered inconsistent. Although readers might disagree with a few assessments, the majority of labels were either clearly consistent or clearly inconsistent. 8 Finally, labels were categorized according to whether or not they appeared in association with a timbre on Series #1 (regardless of whether the label was applied consistently or inconsistently). If most of a label from Series #2 also appeared somewhere on Series #1, the label was considered re-used; if not, it was considered new. Once again, most labels were either clearly re-used or clearly new. Results and discussion Participants in this experiment were compelled to associate verbal labels with sounds. As described previously, the stimuli used were designed to present a variety of labeling and familiarity conditions. For instance, some timbres were produced by European acoustic instruments that presumably could be easily named by trained musicians, while others were produced by synthesizer patches with no familiar associated labels. Some timbres were heard and identified during training, some were heard but not explicitly identified, and some were neither heard nor mentioned during training. inconsistent. 8 Again, an independent scorer was consulted when a label was not clearly consistent or clearly
Chapter Two 33 Type of training condition produced a significant main effect. As depicted in figure 2.2, timbres that had been heard during training but were not explicitly named received significantly lower scores on Series #2 than did timbres that had been explicitly named and timbres that had not been heard at all during training. Although it is counterintuitive that timbres not heard during training would score significantly higher that timbres that were heard but not explicitly named during training, this result seem consistent with Melcher & Schooler (1996): vaguely familiar timbres were apparently more vulnerable to verbal overshadowing than were unfamiliar timbres. 9 Figure 2.3 provides another perspective on these results. The three white bars show the scores associated with targets (those timbres that were heard in both Series #1 and Series #2). Scores for target timbres are very close for all three conditions (named, heard, and not heard during training). The three shaded bars show scores associated with correct rejection of lures (those timbres in Series #2 that were not heard in Series #1). These scores are significantly lower, particularly for the lures that had been heard but not explicitly named during training a combination that resulted in only chance performance. These data suggest that verbal encoding is particularly helpful in making correct rejections that is, remembering what was not heard. Label accuracy also produced a significant main effect: when participants correctly identified a timbre by name after hearing it in Series #2, they were much more likely to remember whether or not it had been included in Series #1. The striking differences are shown in figure 2.4. This association between accurate labels and high scores applied to each of the three training 9 See chapter 1 for a summary of Melcher & Schooler (1996). Not surprisingly, timbres that had not been heard during training were particularly easy to reject as lures (see next paragraph).
Chapter Two 34 conditions. Inaccurate labels, on the other hand, were associated with performance not significantly different from chance. An incorrect label is apparently quite detrimental to auditory memory worse, in fact, than no label at all. It was originally hypothesized that lures receiving labels would receive higher scores than would lures that were left unlabeled, but figure 2.5 shows that this was not the case. Labeled targets were recognized whether the labels were correct or incorrect, but lures were less likely to be correctly rejected when they were labeled incorrectly. Correct labels appear to be helpful, but under certain conditions incorrect labels are evidently worse than no labels. Once again, these data suggest that accurate verbal labels are particularly helpful in making correct rejections. This finding may explain the unanticipated result depicted in figure 2.6: overall, electronic timbres received slightly higher scores than did acoustic timbres. Some participants were apparently more inclined to leave electronic sounds unlabeled rather than guess a label that would misidentify them. Although accurate labels resulted in higher scores regardless of sound source, unlabeled and mislabeled electronic timbres scored higher than did unlabeled and mislabeled acoustic timbres, as shown in figure 2.7. 10 Electronic timbres that were explicitly named during training received particularly high scores, as shown in figure 2.8. However, only one third of the timbres used in this experiment were electronically produced, and only four of these were explicitly named during training; the results might be quite different if listeners were exposed to a greater number of unfamiliar timbres and required to memorize a greater number of unfamiliar labels. Notice that among timbres performance. 10 The apparently missing shaded bar for slightly incorrect is set at zero, indicating chance
Chapter Two 35 that were not heard during training, electronic sounds scored lower than acoustic sounds. It is possible that training was particularly helpful for these unfamiliar timbres. Training was less helpful for familiar acoustic timbres that most listeners could already easily name. I suspect that many participants implicitly labeled the acoustic timbres heard but not explicitly identified during training, and that this implicit labeling may be largely responsible for the interaction between timbre source and training condition shown on figure 2.8. As discussed above, only electronic sounds that had been explicitly named during training and acoustic instruments could reasonably be addressed in terms of labeling accuracy. It was possible, however, to assess whether any recurring sound was labeled consistently. Figure 2.9 depicts the scores associated with consistently and inconsistently labeled target timbres. As hypothesized, labeling consistency produced significant main effects: consistently labeled target timbres received very high recognition scores (recall that 2.5 is the maximum possible score) substantially higher than other target timbres. The differences between target timbres that were labeled slightly inconsistently, very inconsistently, or weren t labeled at all were not significant. 11 While figure 2.9 focuses on whether labels were re-used consistently, figure 2.10 instead focuses on whether labels were re-used at all (either 11 Vivid idiosyncratic labels (such as ghost ) were more likely to be applied consistently, especially when they were relatively short. Objective descriptions of sound quality (such as muffled electronic sound ) were much less likely to be applied consistently, particularly when the description was fairly long. These results were not sufficiently quantifiable to serve as data, but they suggest that labels may be easier to remember (and consequently more helpful) when they are evocative.
Chapter Two 36 consistently or inconsistently). 12 Overall, there was no statistical difference in scores between timbres that were identified with a new label and timbres that were identified with a re-used label. However, there was significant interaction between the re-use of labels and the recurrence of timbres (that is, targets vs. lures). When a timbre that had been heard on Series #1 was identified by a label that had been produced on Series #1, listeners were very likely to recognize the sound with a high degree of confidence. Target timbres with re-used labels received the highest scores, while target timbres that were identified with a new label or no label received significantly lower scores. When a timbre that had not been heard on Series #1 was given a new label, listeners were very likely to identify the sound as new with a relatively high degree of confidence. Lure timbres with new labels received the second highest scores, while lures that were assigned labels that had been used during Series #1 were likely to be incorrectly recognized with a fairly high degree of confidence. The combination of lures associated with re-used labels produced the lowest scores by far. This false alarm rate strongly suggests that listeners reacted to their memory of the verbal label rather than their memory of the sound itself. No gender differences were hypothesized, and indeed men and women scored nearly identically overall. However, there were some interaction effects involving gender. Figure 2.11 shows that both men and women identified target timbres best when they applied consistent labels, but men s performance was substantially worse (not significantly different from chance) 12 For purposes of this discussion, a re-used label is one that a participant wrote on both Series #1 and Series #2; a new label is one that a participant wrote on Series #2 but not on Series #1.
Chapter Two 37 when they did not label target timbres. On the other hand, figure 2.12 shows that men did not generally perform poorly when they did not supply a label, but instead performed poorly when they supplied an incorrect label. Women also performed poorly when they supplied incorrect labels, but the difference was not as great. Figure 2.13 provides another perspective on the data depicted in figure 2.12, separating the results by gender and dividing label accuracy by target/lure condition. Among female participants, recognition rates for targets were similar regardless of label accuracy. Lures, however, were only correctly rejected by female participants when they were labeled accurately; lures that received no label or slightly incorrect labels were rejected at near-chance levels, while incorrectly labeled lures were rejected at below-chance levels. Results for male participants are not easily summarized. Among men, correct labels were associated with high recognition rates for targets and moderately high correct rejection rates for lures, while slightly incorrect labels were associated with below-chance performance for targets and performance not significantly different from chance for lures. Unlabeled lures were correctly rejected at a high rate, but unlabeled targets were recognized at levels not significantly different from chance. This relationship was reversed for incorrect labels: men recognized incorrectly labeled targets at a high rate, but rejected incorrectly labeled lures at a below-chance level. The relative consistency of results among female participants and the relative inconsistency of results among male participants suggests that women may be more likely to employ verbal encoding strategies. Further experimentation is indicated to clarify these results.
Chapter Two 38 Summary Timbres that were heard but not explicitly named during training received significantly lower scores; in particular, these timbres were less likely to be correctly rejected when presented as lures. Correctly labeled timbres received significantly higher scores overall, while incorrectly labeled timbres were associated with near-chance performance. Consistently labeled timbres received significantly higher scores. Target timbres that were identified with a re-used label were particularly likely to be recognized, while lure timbres that were identified with a new label were very likely to be correctly rejected. However, lure timbres that were identified with a re-used label were mistakenly recognized (i.e., a false alarm response). Surprisingly, electronic timbres received higher scores than did acoustic timbres. As hypothesized, there were no main effects for gender, but some interaction was observed involving gender and labeling accuracy.