Rubato: Towards the Gamification of Music Pedagogy for Learning Outside of the Classroom Peter Washington Rice University Houston, TX 77005, USA peterwashington@alumni.rice.edu Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. Copyright is held by the owner/author(s). Each submission will be assigned a unique DOI string to be included here. Abstract Musical concepts are often difficult to teach, as they usually require frequent feedback between the instructor and student that is not possible in the traditional weekly lesson setup. Often, students will give up on the musical learning process due to a lack of significant and timely progress. In order to facilitate musical reinforcement outside of lessons, I have built a simple gamified system, Rubato, which requires users to identify an incorrect note played in a melody written out as sheet music displayed on the screen. Rubato uses adaptive learning to adjust the difficulty of the game based on the user s performance, much as a teacher would. To evaluate Rubato, I recruited several participants to take a music exam before and after playing the game. Through observing the scores of users on these exams as well as recording game performance statistics, I have found that the gamification of music theory education using adaptive learning with a system such as Rubato can be a fast, effective, and engaging way to reinforce musical concepts outside of the classroom. Author Keywords Education Systems; Gamification; Music Theory; Music Education; Adaptive Learning
Figure 1: A green box surrounds the current note played to the user. A red box surrounds incorrect guesses. ACM Classification Keywords H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous Introduction Learning to read music and understanding the fundamentals of music theory takes a long time. A great deal of dedication is required from the student as well as the instructor. Often, students will give up on the musical learning process due to a lack of significant and timely progress. Younger students in elementary school and middle school may lack the motivation and discipline to practice music outside of the context of infrequent (weekly) lessons. If the parents of the student do not have substantive musical knowledge, then oftentimes the student will forget the majority of the material learned during the lessons due to a lack of reinforcement of the material in between the lessons. This results in a continuous lack of motivation for the student due to the absence of weekly progress, even if the student genuinely has a desire to learn how to play an instrument and read music. In order to facilitate musical reinforcement outside of lessons, I have built a simple musical game called Rubato. The goal of Rubato is to solve musical puzzles by identifying the incorrect note played in a melody that is written out on a staff displayed on the screen and played back to the user. The Rubato game utilizes adaptive learning by giving the user more difficult musical puzzles if they are performing well and easier musical puzzles if they are performing poorly. While the task of identifying the incorrect note may seem tedious in a traditional educational setting, the inclusion of game-like elements such as a score, a timed setting, a finite number of guesses, adaptive levels, and the option to share the player s score on a social networking website can make this process fun, and provide a means for the player to learn in an engaging manner. Related Work Researchers have been exploring ways in which computers can be used to aid in teaching methodologies since the 1960 s [7]. Recently, the idea of gamification has been gaining significant traction. Gamification can be formally defined as the use of design elements characteristic for games in non-game contexts [3]. Gamification has been shown to be an effective educational tool in numerous studies [4, 6, 8]. Gamification can be preferred to traditional learning methods for a variety of reasons, including the creation of an addictive and engaging learning process and the minimization of the negative emotions that a learner may feel in a classroom setting or in front of an instructor [5]. Adaptive learning can be used to augment gamified education technology [1]. Adaptive learning is the process of using a computer to adjust to the learner s educational needs. Adaptive learning has been used in many previous educational contexts, such as web-based adaptive textbooks [2] and in integration with ubiquitous computing paradigms [6]. Rubato The goal of the Rubato game is to identify the incorrect note in a particular melody. There are a series of musical puzzles in the game consisting of these short, 16-note melodies. The game player has the ability to listen to the melody as many times as they wish. There is a single note in the melody that does not match the
Figure 2: The Rubato game. sheet music displayed on the screen. It is the job of the player to identify this note as quickly as possible and in as few guesses as possible. Rubato has the goal of teaching the user two important skills in basic music theory: the ability to read notes and the ability to hear note intervals (relative pitch). As the game becomes more difficult, the goal shifts from learning to read notes and focuses more on the ability to distinguish note intervals, a more advanced musical skill. As can be seen in Figure 2, the user interface is quite simple, with only a few buttons for the user to click. Although instructions are provided, the interface is simple enough for the user to figure out without too many explicit instructions. The user starts the game by pressing the Play button. This will start the stopwatch timer, and the melody displayed on the screen will begin to play. As each note is played, a green box surrounds it. For every level, there will be one incorrect note played that is, a note will be played which does not match the note written on the sheet music. The challenge for the game player is to identify the incorrect note as quickly and in as few guesses as possible. The user has only 3 guesses per level, in order to prevent random guesses and to encourage the player to really pay attention to the game. The game starts out relatively easy, playing familiar melodies (Bach s Minuet in G Major, Frere Jacques, and other popular melodies). This allows the user to
The 11 multiple choice questions are split up into the following categories: Three questions covering the ability to read basic notes. One question involving written intervals. One question involving identifying the quality of a written chord (major, minor, augmented, diminished, etc.). Three questions testing the ability to hear intervals. This is a precursor to relative pitch. Two questions testing relative pitch. One question testing perfect pitch. By splitting up the questions into different categories, we were also able to keep track of the specific types of questions that users correctly answered. become accustomed to the game s interface. As the game progresses, the melodies become more difficult. The game difficulty levels can be thought of in the following stages: Stage Zero The melodies are recognizable to the user. The correct note can be determined without any ability to correctly read notes. This stage is primarily used to get the user accustomed to the Rubato game and start the process of associating notes on a staff to their corresponding note name. Stage One The melodies are no longer recognizable, but are still relatively simple. No difficult musical modes are incorporated, and there are no interval leaps besides thirds, fifths, and octaves. Stage Two The melodies lose their simplicity but retain a certain structure. For example, the melody may be in a different musical mode (minor, Mixolydian, etc.) and contain more complicated melodic intervals (fourths, sixths, sevenths, ninths, and elevenths). However, the melody will most likely begin and end on the root tonic, contain short phrases, and otherwise have the characteristics of a coherent melody. Stage Three The melodies no longer have any structure or form. The notes played are almost completely random. This will truly require the listener to have a good ear and pay attention to each note as it relates to the previous note. The rate at which the musical puzzles become more difficult is determined by a simple adaptive learning algorithm which takes into consideration the time in which the user spends on a particular level, the number of guesses at a particular difficulty level, and whether the player s score is increasing or decreasing. At the end of the experience, the user has the ability to share their results on Facebook, as approved by the IRB. If the user chooses to do this, none of their Facebook data is recorded. Sharing the score provides a two-fold benefit. Firstly, it allows the participants of the study to advertise Rubato, which brings in more users to study. More importantly, it creates a collaborative learning experience among students who wish to have this experience. This friendly competition among students who wish to have it provides an incentive for a student to perform at their best ability when playing the game. Experiment Design The IRB-approved study was published online for a period of two weeks, during which 74 users participated in some way. There was no special requirement regarding the musical ability of the participants that played Rubato. Instead, there was a diagnostic music theory examination given prior to the experiments that gauged the initial musical knowledge of each participant. The data of any participant who made a perfect or near-perfect score on the diagnostic exam was thrown out, as there would be no room for that user to learn more about what we were testing for. An evaluation exam was also given after the experiments. The difference in the scores between the diagnostic exam and the evaluation exam was recorded in our database. The amount of participation in the experiments was entirely up to the individual participant. When the user felt that they were done playing the game, they could As long as the user
Time (secs) Score Increase 64 +1 9 56 +2 9 65 +2 9 58 +3 5 44 +3 7 71 +4 7 Initial Score Figure 5: The initial score of the users who had a relatively high performance increase despite spending a relatively short amount of time playing Rubato. completed the short exit quiz, the participation counted as valid. Multiple data points were collected throughout each use of Rubato. The following data was collected from the user every time they played Rubato: The score on both of the quizzes. The specific types of questions that the user The amount of time it took to complete each puzzle and the difficulty level of the corresponding puzzle. The speed at which the user advanced through the difficulty levels. Results There were a total of 74 participants who decided to play Rubato during the two-week trial period. Out of these 74 participants, we had 32 valid trials. The data from a trial was thrown out (considered invalid) if the user made a perfect score on the initial examination (as this would not allow for any room for improvement). Frequency 12 10 8 6 4 2 0 Figure 3: Net change in game performance before and after playing Rubato. Improvements in Quiz Performance -9% 0% 9% 18% 27% 36% Percent Improvements in Quiz Performance Time Spent on Rubato vs. Performance Increase Figure 3 below shows the performance gains from these 32 users. It can be seen that the majority of valid users in the initial study performed better on the quiz after playing Rubato rather than before. 20 of the 32 users (62.5%) made some sort of performance gain on the quiz. Only 3 of the 32 users (9.375%) decreased their quiz performance, while the remaining 9 users (28.125%) did not make any performance gain whatsoever. The 62.5% of users who displayed a net improvement indicates that Rubato has some potential to be used as a musical educational tool. Performance Increase 40% 30% 20% 10% 0% -10% -20% 0 100 200 300 400 Time Spent (seconds) Figure 4: Time spent on Rubato vs. net quiz performance increase playing Rubato.
The relatively large percentage of users who did not show a net improvement (37.5%) could initially be viewed as somewhat troublesome. However, Figures 4 and 5, which factor in the time that the user spent playing the game, provide an explanation for the lack of improvement. Figure 4 shows that there is a relatively strong positive correlation between the time spent playing Rubato and the quiz performance increase. Despite this correlation, there are still six data points that show a large performance increase while only spending about a minute playing the game. Looking at the initial scores of these six users provides some insight into the causes of this phenomenon, as detailed in Figure 5. All of the users who made a significant increase in quiz performance while only spending a small amount of time playing the game had a fairly high initial score. Furthermore, all of the question types that were initially missed for all of these users were the more difficult question types involving intervals, relative pitch, and perfect pitch. This data indicates that these users had prior education in music theory before playing the game. Playing Rubato for a short amount of time could have refreshed their memory of the material. Conclusions The results indicate that educational games augmented with adaptive learning can be used as an effective pedagogical tool for teaching music theory. This has the potential to greatly enhance the music learning process by providing a means for students at home to practice and reinforce musical concepts taught during lessons. Future Work There are many future directions for expanding Rubato. Currently, the game is only meant to teach very basic musical concepts such as note recognition, interval identification, and relative pitch. However, the game could be expanded to include chords accompanying the melody in an effort to teach basic harmonization principles. The machine learning used in the game could also be expanded. For example, the player could provide the computer feedback regarding which note and chord combinations sound pleasing and which do not. The act of having a student teach a computer which notes sound correct given a chord progression can give the student significant insights about the theoretical reasons as to why certain note sequences sound pleasant given a set of chords. Acknowledgements I would like to acknowledge the Rice Undergraduate Scholars Program (RUSP) at Rice University for providing the funding to build the system and conduct the experiments as well was Professor Lin Zhong for discussing the project with me. References 1. Berlanga, Adriana, and Francisco J. Garcia. "Learning technology specifications: semantic objects for adaptive learning environments." International Journal of Learning Technology 1.4 (2005): 458-472. 2. Brusilovsky, Peter, John Eklund, and Elmar Schwarz. "Web-based education for all: a tool for development adaptive courseware." Computer Networks and ISDN Systems 30.1 (1998): 291-300. 3. Deterding, Sebastian, et al. "From game design elements to gamefulness: defining gamification."
Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments. ACM, 2011. 4. Denis, Guillaume, and Pierre Jouvelot. "Motivationdriven educational game design: applying best practices to music education." Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology. ACM, 2005. 5. Huang, Wendy Hsin-Yuan, and Dilip Soman. "Gamification Of Education." (2013). 6. Jones, Vicki, and Jun H. Jo. "Ubiquitous learning environment: An adaptive teaching system using ubiquitous technology." Beyond the comfort zone: Proceedings of the 21st ASCILITE Conference. No. s 468. 2004. 7. Kuhn, Wolfgang E., and Raynold L. Allvin. "Computer-assisted teaching: A new approach to research in music." Journal of Research in Music Education (1967): 305-315. 8. Landers, Richard N., and Rachel C. Callan. "Casual social games as serious games: The psychology of gamification in undergraduate education and employee training." Serious games and edutainment applications. Springer London, 2011. 399-423.