700 Years Later: Using Multimedia to Bring the Stories of Medieval Asian Scrolls to Today s Students

Transcription

1

2 This paper is a preliminary version of the following publication. Please use the official version for citing. Travers, K. & Finneran, C.M. (2006). 700 years later: Using multimedia to bring the stories of medieval scrolls to today s students. In Technologies for E-Learning and Digital Entertainment: Proceedings of the First International Conference, Edutainment 2006, April 16-19, 2006, (pp ). Hangzhou, China. Springer. 700 Years Later: Using Multimedia to Bring the Stories of Medieval Asian Scrolls to Today s Students Kevin Travers and Christina M. Finneran Educational Research & Development Information Technology Bowdoin College Brunswick ME USA {ktravers, cfinnera}@bowdoin.edu Introduction This paper first discusses storytelling, its pedagogical benefits, and how these benefits can be enhanced with multimedia technology. Then we detail our development of a multimedia scroll viewer that enables students to interact with medieval Asian scrolls. Two courses that use the multimedia scroll viewer are described, demonstrating the pedagogical benefits of multimedia narrative tools. Finally, we close with our plans for future work on the multimedia scroll viewer. In talking about the film industry and preservation, Ben Davis (2000) articulates that, storytelling is the mechanism for preserving our collective memory as well as the communication tool of the imagination. The word storytelling focuses on the imagination that occurs during the telling of a story. However, the implicit side is the creation of the story, which requires as much, if not more, imagination. The distinction between creating and telling stories created by another party is a matter of degree; one can adapt a story or create one from whole cloth. The act of creating a story versus experiencing a story has different pedagogical implications for students, which will be demonstrated in this paper. Davis also talks about the value of stories for our collective memory. The collective memory that narratives provide illustrates the group aspect of stories. Stories are usually shared among cultures and across generations to bring groups of people together physically and mentally, if only for a transient period of time. On the creation side, stories are not necessarily created by a single individual. As technologies enable simultaneous constructions of documents (e.g., wikis) and gaming interactions (e.g., massively multi-player online games) and social software continues to emerge, the creation of narratives is positioned to become more of a collaborative, or joint storytelling, effort. Finally, the nature of stories, and their frequent telling, enables them to evolve to meet contemporary audiences. Thus, we can think of stories as dynamic entities, narratives that are intended for a particular time and place, potentially altered depending on the audience. Multimedia

3 technologies not only provide an additional means for telling or evolving stories, but they also have the potential to make the dynamic nature of stories explicit to students. Pedagogical Benefits of Narrative Heckman, Finneran, and Marshall (2003) identify five characteristics of narratives that make them pedagogically beneficial. Below we explain each of the characteristics and demonstrate how multimedia technologies can enhance them. Narratives facilitate comprehension and memory David Snowden, who has promoted storytelling for documenting and sharing knowledge within organizations, states that stories are a patterning device for human brains (Snowden, 2000). Stories facilitate comprehension and memory through their plot sequence and their concreteness (Sadoski, Goetz, and Rodriguez, 2000). When we add multimedia to stories or texts, what was textual can now have colorful images and audio. These additional features increase the amount of sensory input and thus, can further facilitate memory and concreteness. Additionally, animation has the potential to facilitate comprehension. For example, portraying a long journey with an animated map can bring more meaning to a trek over mountains and through a snowstorm than a static illustrated map. When students create their own narrative as opposed to being given a story, the process will require more cognitive effort and imagination, and thus, will be ingrained more in their memories. Narratives engage the affect Learning is not merely a cognitive activity, but involves the affect, or emotional aspect (Goleman, 1994). Students learn better when motivated and engaged (decharms, 1976; Thomas, 1980). Stephen Denning used short, vivid stories at the World Bank to facilitate change; he explained, It is through empathy for the protagonist in difficulty that the audience s emotions become engaged, and so get a deeper level of meaning (Denning, 2001, pg. 125). Multimedia can increase the impact of the affect by relying on more human senses. As with films, music and sound effects can be added to engage the affect of audiences. Multimedia imagery can bring the narrative more to life, furthering the emotional aspects of the story. Narratives are experiential Bandura s (1989) Social Learning Theory states that much of learning occurs through observation and modeling others. By being a part of stories, students are learning vicariously through others actions and consequences. With multimedia stories, students can come closer to experiencing the event than with solely oral or written stories, especially when the multimedia narratives are interactive so each student can experience the narrative in a particularly unique way. These interactive stories make each encounter with the narrative a unique one, allowing students to have multiple experiences with the same story. Stories that are created collaboratively bring this experiential aspect to a new level. The narrative is not only experienced as part of a group but the story unfolds, in perhaps unexpected ways, during the story creation process. Narratives embody the tacit Tacit knowledge (Polyani, 1958) is what one knows but cannot easily articulate in words. Because of the difficulty in conveying such knowledge, it is less easily taught in formal learning environments like schools. Narratives enable tacit knowledge to be shared in the classroom because stories embody the tacit within them. Multimedia has the potential to convey more of the tacit 2

4 because textual words are not required. As in films, a multimedia narrative can depict mood or sentiment through artistic methods of color, texture, and sound. When students create narratives, they can express themselves with more than words, thus giving students who are less apt with verbal expression a chance to shine. Narratives demand active construction Because narrative focuses on interpretations, it is constructivist in nature. As Bruner (2002, p. 20) suggests, narrative is an invitation to problem finding, not a lesson in problem solving. When interpreting narratives, students are identifying questions and not necessarily seeking the correct answer. They quickly realize that each student must construct her own meaning from the story. Further, the fact that narratives evolve over time, as they are used in different times and places, demonstrates the importance of constructivist thinking. With multimedia websites, multiple interpretations of a story can be displayed nearly simultaneously so that a student can easily identify disparities. Multimedia technologies also enable students to relatively easily create basic narratives, particularly if they are given a template to use. With a template students can create a narrative focusing on the story itself rather than the creation tools (e.g., paintbrush, PhotoShop). Medieval Asian Scrolls We briefly discuss the role of scrolls for storytelling in medieval Asia and our development of a multimedia scroll viewer to bring these story telling devices to the Web. Scrolls as Story Telling Devices During the 12th and 13th century Asian scrolls were used to depict important events and to convey these narrative accounts to future generations. Many of the scrolls have not survived in original form and those that have are not available to undergraduate students in the United States or Europe. In this century, most Western students studying 12th and 13th century Asia must rely on monographs, which display the scroll as a multitude of disparate pages. The students are rarely given the opportunity to experience the narrative in its original form of a scroll. Design of the Multimedia Scrolls With current multimedia and compression technologies, we developed a scroll viewing mechanism that allows users to view scrolls via the Web. Because the scrolls are displayed in the traditional continuous linear format and read right to left, the viewing mechanism depicts the narrative as it was originally intended to be read, and without the disruption of flipping pages. Utilizing Macromedia Flash ( and image compression technology from Zoomify ( users can interactively view the scrolls and zoom in for more detail. The monographs containing the scroll content are scanned and brought into an interface that is designed to provide interactive functionality but gives precedence to the content and narrative of the scrolls. After scanning the prints, the high resolution images are tiled into hundreds of images at several different sizes. These images are fed to the browser on demand as the user interacts with the scroll. Given the size of the images and the potential need for a detailed view of any particular part of the scroll, the technology is designed to decrease download time and increase the speed of viewing the scrolled images. Because of the tiling, the user only needs to download the section currently being viewed. Also by storing the tiled images at different resolutions, only the level of 3

5 detail which is currently required is downloaded. The scrolls can be viewed in extremely close detail or zoomed way out, enabling new perspectives from which to view the scroll. Applications of Multimedia Scroll Narratives We will discuss two classroom applications of the multimedia scroll viewer that use narrative, and the multimedia scroll viewer, in different ways. The first, Scrolls of the Mongol Invasions of Japan, provides students with a historical narrative to experience and interpret. The second, Spring Festival on the River, offers students the opportunity to contribute their own personal accounts to an medieval artist s scroll painting. Scrolls of the Mongol Invasion of Japan The Scrolls of the Mongol Invasions of Japan interactive web site ( is an ongoing project that brings 13th century Japanese scrolls to today s undergraduate students. These scrolls were originally commissioned by Takezaki Suenaga to tell the warrior s account of the Mongol invasions of Japan in 1274 and 1281 and his victory over the Mongols. Because the fragments of these particular scrolls were lost over the centuries, their true order has long been disputed. This website allows students to compare the different versions of scroll fragments that have been reordered, and even altered, into different versions over time while maintaining the linear narrative format of the scroll. Relevance of Narrative The multiple versions of the scrolls depict the same historical account from different centuries. The stories conveyed are different, because of the changing self-identity of the Japanese (Conlan, 2001). Conlan (2001) interprets the 19th century version of the narrative to indicate that the Japanese were qualified to win the fight against the Mongols and did not need divine intervention, as previous interpretations argued. The analysis of the various changes to the story that were made over time make it clear that changes in the narrative had the power to alter the perception of Japanese history. This shows the important role that storytelling has played throughout history. While visual depiction of events in history through a scroll is more compelling than just written words, the narrative perspective of this story lead to an even higher level of engagement. The story is told from the viewpoint of a great warrior in exciting battles. The importance of those events was diluted by poor presentation when it was in a book format. The student becomes engaged in the experience by presenting the scrolls in the format in which they were originally intended. When the student is able to see the story flowing in its continuous format with translations immediately available, it brings to life a time and place that is culturally and temporally distant from college students of today. As a part of his class, The Origins of Japanese Culture and Civilization, Professor Tom Conlan introduces the scrolls to his students and leads discussion on specific disparities of the scroll versions. The students, who view the scrolls outside of class, can make use of an interactive glossary and translations that provide insight into the culture and warfare of medieval Japan. Design Decisions A limitation with the book-based Mongol Scrolls has been the inability to view them in an order different from whichever century they were created in. Because there are at least three variations of the Mongol Scrolls created in different centuries, with the addition and/or removal of page, the 4

6 narrative has changed considerably. The scroll viewer application allows for careful observation of these changes in a comparative format making the underlying changes much clearer to the student. The experience is no longer constrained to one variation of the narrative; therefore, technology is impacting the narrative experience and making the dynamic nature of stories explicit to the students. Figure 1. Comparison View within the Scrolls of the Mongol Invasions of Japan One of the initial reasons for designing the multimedia scroll viewer was to facilitate the comparison of the various versions of the Scrolls (see Figure 1) and the different orders in which they were assembled. Such functionality to compare the different narrative versions would show how even small changes to the story could mean significant changes in the perception of history. Conlan states, "The thing that I found interesting was that even when people view this [in book form], they didn't always know what the proper order was. It's only through the Internet that instead of having one definitive version, you can compare a number of versions so that indeterminacy helps you get a better understanding of the scrolls to begin with. Secondly, the multimedia scroll viewer enables a student to view the scroll in its original intended format. The student can view the scroll as a continuous document without the interruption of page turning that would break up some of the action of the narrative. The right to left direction of Asian scrolls is also maintained, giving the scroll more authenticity. Having the translation of Japanese text on the Mongol Scrolls instantly available is paramount to enhancing the narrative experience. In previously published reproductions, translations had to be referred to in an index following numbers. With the scroll viewer key translations can be accessed instantly just by clicking on the text. The words and images are juxtaposed, enabling their meaning to be easier to comprehend. Immediate identification and translation of important objects is available through the Glossary feature. The student is not taken out of the narrative experience. The information is available right next to the object on screen. Specific objects, which have been identified as important to the story or historically, are included in the Glossary. 5

7 Even in the original scroll form, it would be extremely difficult to see large portions of the scroll all at once. The scroll would have to be rolled out, and viewed from a very high perspective and even then, it would be difficult to see. With the multimedia scroll viewer, students can get a holistic view and also zoom into incredible detail. The multimedia scroll allows not only a new viewpoint on the narrative of the story in the scroll, but a new way to analyze the varying versions that were assembled and changed over the centuries. Spring Festival on the River Scroll The Chinese Scrolls website ( showcases the scroll titled, Spring Festival on the River. This 12th century Beijing handscroll is believed to have been created during the Northern Song dynasty before the dynasty was overthrown. Due to the paucity of records on the scroll, much debate has occurred over the creator, the date and place of origin, and the meaning of the work. The technology used for viewing this scroll is similar to the Scrolls of the Mongol Invasions of Japan, yet this project also enables students to contribute their own work and append it to the scroll. Relevance of Narrative In Bowdoin College s Stories and Scrolls introductory seminar to Art History and Asian Studies, Professor De-nin Lee has her students create colophons (textual responses that were appended to handscrolls during that time period) for the Spring Festival on the River, depicting their personal interpretations of or reactions to the scene. These colophons, assessed on their content, imagination, and beauty, are appended to the scroll. During each semester that the course is taught, more colophons are appended. As the scroll builds, it becomes not only a way to display a scroll from the past, but also a way to build onto the joint story as more people contribute to it. Part of the specific use of narrative can be seen in the Chinese Scroll, where students are encouraged to assume the role of a writer/artist of that time period, going through similar experiences to relay their personal reaction through the use of words and images. The narrative medium itself is given to them as a powerful means for expressing ideas and emotions. As Professor Lee s students continue to add to the colophons of the scroll each semester, it becomes a document that can be used to observe the trends of the authors and their contributions. It becomes a map of the reactions to the Chinese Scroll from a specific age group over a long period of time. The students are contributing to a dynamic story constructed of their own personalized narratives. Design Decisions The design of the Spring Festival on the River multimedia scroll was based on the same model for the Mongol Scrolls. Where the Mongol Scroll's focus is on observation and analysis of specific details of the narrative, the Spring Festival on the River multimedia scroll focuses on adding to a dynamically building story. Because of the multimedia scroll viewer s ability to display the entire Spring Festival on the River scroll in its originally intended format, the viewer gets a better sense of the artist s true intention. The experience would not be as powerful if the scroll remained sliced into separate pages in a book. When a student sees the progressing story in scrolling form, it comes to life and better presents itself as an engaging narrative medium. When the students get a visual sense of how the original story will flow into their added page, it will encourage them to take part in the experience. 6

8 Figure 2. Colophons added to the Spring Festival on the River scroll Instead of just being given the task of writing a response to the scroll for a text based colophon addition, the students were given a template created in Adobe Photoshop that includes a paper background, sample text and images laid out in a simple format. All of the elements are organized into layers. With a brief introduction to the software the students are encouraged to experiment, add new elements, color, or poetry. Hand drawn or written material is also encouraged and would be scanned into their colophons separately. The multimedia scroll viewer was built with Macromedia Flash, which is an animation program at heart. Because of the support of a wide range of file formats, the application could easily accommodate the addition of multimedia within the colophons created by the students, taking the narrative to the next level. Responses to the experience of the scrolls could be recorded immediately with audio and video. A simple drawing tool built within the application could be used to do traditional calligraphy directly on the screen, combining modern and traditional tools. From a visual perspective, the scroll viewer allows for a unique approach to arranging text and graphic elements on the colophons. Because of the viewer's ability to zoom very close to small details and very far out to view the scroll and colophons from a distance, new opportunities for creativity are available. What could be hidden in the finest details of an image? What does the colophon look like when zoomed far away? Just as visual tools can be used in film production to set emotional tensions and ambient moods, the narrative tools in a multimedia application such as the scroll viewer allows for new ways to contribute to the scroll. Future Work Our future work in enhancing the multimedia scrolls, and these particular applications of the scrolls, includes adding features to facilitate memory, enhance the narrative experience, and increase the student s degree of active construction. Given the detail and length of each of the Mongol Scrolls, we will add bookmarking and annotation devices for the students to use. These bookmarking devices, or memory dots, are graphical and enable the student to return to particular parts of the scroll that she may find particularly important or challenging. The annotation device can be used for the student to record notes that are connected to a particular point in the scroll. The bookmarking and annotation features will help students navigate the scroll and remember notable events in the story. Finally, to aid the 7

9 students comprehension, we would like to add an animated map giving the geographic context of the Mongol invasion. To enhance the student s interaction with the narrative, we will also add audio to supplement the Scrolls of the Mongol Invasions of Japan project. Ambient sound along with voice narration will deliver material through auditory channels and enable students to be told the story, if they prefer. Further, audio translation will enable students who are fluent in Japanese to hear the story in its original language and to learn the pronunciation of particularly important elements to the story (e.g., medieval Japanese weapons). Lastly, we will continue to work on the features that enable students to create their own stories - - additions or interpretations of the narratives -- as well as create collaborative narratives. A feature that we are considering is the ability of a user (instructor or student) to rearrange the order of the Mongol Scroll fragments. Given the longstanding debate about the order of these fragments, having the fragments available for students to rearrange would demonstrate the constructive aspects of narratives while offering a real academic exercise that historians must face with primary source documents. The annotations mentioned above will allow students to note their interpretations of the events unfolded in the narratives. By availing the personal annotations to groups of students, the annotation feature can be used to develop joint interpretations of the narratives. Summary Multimedia technologies can be used to bring 12th and 13th century Asian stories told through traditional scrolls to life for undergraduates whose access to the original scrolls is quite limited. The multimedia scroll viewer enables students to experience the scroll closer to its original linear format, interact with the narrative accounts, and to create their own narratives. Pedagogical benefits will occur because these multimedia applications of narrative facilitate comprehension and memory, engage the affect, are experiential, embody the tacit, and demand active construction. References Bandura, A. (1989). Human agency in social cognitive theory. American Psychologist, 44(9), Bruner, J. (2002). Making Stories: Law, Literature, Life. New York: Farrar, Straus and Giroux. Conlan, T.D. (2001). In Little Need of Divine Intervention: Scrolls of the Mongol Invasions of Japan (Translation with Interpretive Essay). Ithaca, New York: Cornell University East Asia Program. decharms, R. (1976). Enhancing motivation: Change in the classroom. New York: Irvington. Davis, B. (2000). Digital Storytelling, Razorfish Report (024). Retrieved January 11, 2006, from Denning, S. (2001). The Springboard: How storytelling ignites action in knowledge-era organizations. Boston: Butterworth Heinemann. Eislele, T. (1990). Wittgenstein's Instructive Narratives: Leaving the Lessons Latent. Journal of Legal Education, 40(1-2), Goleman, D. (1994). Emotional Intelligence. New York: Bantam. Heckman, R., Finneran, C.M., and Marshall, T. (2003 November). Narrative in online education. Paper presented at the 9th Annual Sloan-C International Conference on Asynchronous Learning Networks, Orlando, FL. Polanyi, M. (1958). Personal knowledge: Towards a post-critical philosophy. Chicago: University of Chicago Press 8

10 Sadoski, M., Goetz, E. T., & Rodriguez, M. S. (2000). Engaging texts: effects of concreteness on comprehensibility, interest, and recall in four text types. Journal of Educational Psychology, 92(1), Snowden, D. (2002). Plenary Presentation. Philadelphia, PA: American Society for Information Science & Technology 2002 Annual Meeting: Informations, Connections, and Community. Thomas, J.W. (1980). Agency and achievement: Self-management and self-regard. Review of Educational Research, 50,