Creatures of the Abyss Evaluation Report Executive Summary Amy Henson Katrina Pisani Chantal Barriault Holly Baker Julie Fisowich Science North March 2010
A. INTRODUCTION Creatures of the Abyss is a 600 square meter (6,000 square feet) travelling exhibit that takes visitors on a journey through the depths of our planet s ocean. Exhibits and experiences include full-scale models of sea creatures and their environments, preserved specimens, mechanical interactives, multimedia experiences, large full-image graphic panels and maps. The exhibit takes visitors on a journey to discover and learn about both the physical features of the deep ocean and the fascinating creatures that inhabit these special places. Creatures of the Abyss takes visitors down into the deep, across the vast sea floor, up submarine mountains, and into canyons and hydrothermal vents. It is a mysterious, immersive and awe-inspiring glimpse into another world. This evaluation report is based on video data collected at Science North from October to December 2009. The visitor survey data was collected during the holiday break of 2009-2010. It is important to note that not all exhibits were filmed during this evaluation. Due to unforeseen circumstances, the number of subjects in both video recording and from visitor surveys is low at less than 50 visitors. All results presented in this report are based on this and must be interpreted with this in mind. B. GOALS FOR THE VISITOR EXPERIENCE The exhibit development team set out the following visitor experience goals: The Creatures of the Abyss exhibit should: 1. Introduce visitors to the amazing creatures of the deep sea. 2. Involve visitors with adaptation and behaviour. 3. Increase visitorsʼ understanding of the oceans and the importance of these ecosystems. 4. Involve visitors with how scientists explore the deep ocean. C. METHOD - DATA COLLECTION In order to assess the impact of the Creatures of the Abyss exhibition on the visitor learning experience, and to enable us to get a more complete picture of the visitor experience, data was collected in two different ways. Visitors surveys were conducted through a computer-based feedback kiosk and observations of visitor behaviour were collected through the use of video recording of visitors interacting with exhibits. Visitor behaviours were then coded for learning behaviours. Creatures of the Abyss Evaluation Report 2
1. Visitor Feedback Kiosk A computer comment kiosk was placed in the Science North special exhibits hall asking visitors to comment on their visit to the Creatures of the Abyss exhibition. The survey asked visitors knowledge questions based on information and concepts presented in the exhibition. In addition, visitors were asked to self-evaluate their own learning and understanding of information in the exhibit based on certain key messages. This method of self-reporting gives us a glimpse into how the visitors feel the exhibit has impacted their understanding of the concepts in the exhibition. Survey Section A - Self-reports of understanding Based on the goals of the exhibit, visitors were asked to rate their knowledge and understanding of the diversity of life in, threats to and science research in the deep ocean before visiting the Creatures of the Abyss exhibition. Visitors were asked to rate their response on a 5-point Likert scale based on their evaluation of their understanding. The scale ranged from very good to very poor. Using the same scale, visitors were asked to rate their knowledge and understanding of the diversity of life in, threats to and science research in the deep ocean after having experiences the Creatures of the Abyss exhibition. Survey Section B - Concepts retained This section of the survey tested visitors understanding of key concepts in the exhibition. Visitors were asked to rate their agreement / disagreement to the statements: 1. In the deep ocean, creatures commonly produce bioluminescence, which is creature created light. and 2. Humans have explored most of what exists in the deep ocean. A multiple choice question asked visitors to recall what species is the primary predator of the colossal squid: angler fish, sperm whale, vampire squid, or giant larvacean. A multiple answers question asked visitors to recall concepts they had encountered in the Cephalopod exhibit. Visitors were given eight different statements (Table 1) corresponding to the eight different stations at the exhibit. Creatures of the Abyss Evaluation Report 3
Table 1. Statements regarding cephalopods visitors were asked to recall. Cephalopods have special adaptations for life in the ocean. Which of the following do you remember? Octopuses usually have 8 arms and no tentacles. Many cephalopods can change the colour and texture of their skin. The only hard body part in cephalopods is the beak. All cephalopods have hard, sharp beaks for feeding. Squid arms and tentacles can have both suckers and hooks. All cephalopods can use jet propulsion to move. Cephalopods have similar eyes to humans. Researchers still don t know why deep-sea squids use ink. Survey Section C The final section of the survey consisted of two open-ended questions. The first question asked visitors what they found most interesting today during their visit to the Creatures of the Abyss exhibition. The second question asked visitors to comment on how Science North could improve the exhibition. Both were openended questions where visitors could type as much or as little as they wanted. 2. Observations of Visitor Behaviour A video camera was set up in the exhibit hall at various times throughout the duration of the Creatures of the Abyss exhibition at Science North. Video footage of visitors interacting with exhibits in 3 locations was collected in November and December of 2009. Ethical protocols were adhered during video recording. Creatures of the Abyss Evaluation Report 4
D. DATA ANALYSIS AND RESULTS 1. Multiple Choice Questions - Data Analysis The total number of completed surveys received was 44. Each completed survey contained either completed multiple choice questions or completed essay questions. All responses were used, whether the entire survey was completed or not. All comments that were 5 not completed or contained non-relevant information 4 37.8% were deleted and not included in the comment analysis. 3 2 1 1 13.5% 13.5% 16.2% 18.9% very poor poor average good very good Chart 1. Rating of previous knowledge and understanding of the diversity of life in the deep ocean. 5 4 3 2 1 1 5.9% 26.5% 32.4% 35.3% very poor poor average good very good Chart 2. Rating of knowledge and understanding of the diversity of life in the deep ocean after visiting the exhibition. Knowledge and Understanding of the Diversity of Life in the Deep Ocean Visitors were asked to rate their knowledge and understanding of the diversity of life in the deep ocean before their visit to the Creatures of the Abyss exhibition and after their visit. Chart 1 shows the percentage of visitors who rated their previous knowledge and understanding. Just over 51% of visitors rated their knowledge and understanding as poor or very poor ; 13.5% of visitors rated themselves average while 35% of visitors rated their knowledge as good or very good. When visitors were asked to rate their knowledge and understanding of the diversity of life in the deep ocean following their visit to Creatures of Creatures of the Abyss Evaluation Report 5
the Abyss, substantially fewer visitors rated their knowledge as poor to very poor (from 51% to 6%), while many more visitors rated themselves as having good and very good knowledge and understanding (from 35% to 67.7%) of the diversity of life in the deep ocean (Chart 2). Knowledge and Understanding of Threats to the Ocean Visitors were asked to rate their knowledge and understanding of threats to the ocean before and after their visit to the Creatures of the Abyss exhibition. Chart 3 shows the percentage of visitors who rated their previous knowledge and understanding from very poor to very good. 5 4 3 2 1 1 8.3% 22.2% 27.8% 19.4% 22.2% very poor poor average good very good Chart 3. Rating of previous knowledge and understanding of threats to the ocean. 5 4 38.2% 35.3% The results show that more visitors reported a good or very good knowledge and understanding (41.6%), while 30.5% of visitors reported having a poor or very poor knowledge and understanding. When asked to rate their knowledge and understanding of threats to the ocean after visiting Creatures of the Abyss, most visitors (73.5%) rated themselves as very good or good (Chart 4). The percentage of those that rated themselves as poor or very poor dropped dramatically to 5.9%. 3 2 1 1 5.9% 20.6% very poor poor average good very good Chart 4. Rating of knowledge and understanding of threats to the ocean after visiting the exhibition. Knowledge and Understanding of Research Conducted in the Ocean Visitors were asked to rate their knowledge and understanding of research conducted in the ocean before and after their visit to the Creatures of the Abyss exhibition. Chart 5 shows the Creatures of the Abyss Evaluation Report 6
numbers and percentage of visitors who rated their previous knowledge and understanding from very poor to very good. 5 4 3 2 1 1 5 4 3 2 16.7% 21.2% 13.9% 42.4% Nearly 42% percent of visitors rated their knowledge as very poor or poor ; 33.3% of visitors rated their knowledge as good or very good ; 25% of visitors rated their knowledge as average. When visitors rated their knowledge of research being conducted in the ocean after visiting the Creatures of the Abyss exhibition, the responses changed dramatically with 72.7% of visitors rating their knowledge and understanding as good or very good (Chart 6). The numbers of visitors rating their knowledge as very poor or poor dropped to 6%. Knowledge Questions 1 1 3.0% 3.0% Visitors were asked three specific questions based on information presented in the exhibit. These questions very poor poor average good very good were designed to gain an understanding of specific Chart 6. Rating of knowledge and understanding of ocean research after visiting the exhibition. knowledge gained. The first question asked visitors the degree to which they agreed with the statement: In the deep ocean, creatures commonly bioluminescence, 19.4% very poor poor average good very good Chart 5. Rating of previous knowledge and understanding of research being conducted in the ocean. 30.3% Creatures of the Abyss Evaluation Report 7
5 4 3 2 1 1 6.1% 6.1% 15.2% 39.4% 36.4% totally disagree disagree not sure agree totally agree Chart 7. Responses to the statement that bioluminescence is creature-created light. 5 4 3 2 1 1 21.2% 45.5% 9.1% 6.1% 15.2% totally disagree disagree not sure agree totally agree Chart 8. Responses to the statement that humans have explored and understand most of what exists in the deep ocean. which is creature-created light. Most recognized that bioluminescence is creature created light ; 75.8% of visitors agreed or totally agreed with this statement (Chart 7) while 15.2% of visitors were not sure and 12.2% of visitors disagreed or totally disagreed with the statement. The next statement also asked visitors the degree to which they agree or disagree with the statement that humans have explored and understand most of what exists in the deep ocean. In this instance, the statement is false and therefore the desired answers should be disagree or totally disagree. More visitors disagreed or totally disagreed to this statement (66.7%). In addition, 9.1% of visitors were unsure of the statement, while 21.3% of visitors agreed or totally agreed to this statement. Creatures of the Abyss Evaluation Report 8
&#'$% ()#*$%!"#!$% +,-./0%123% 24/05%63+./% 7+5480/%29:8;% -8+,<%.+07+=/+,% The third question visitors were asked was a standard multiple choice question asking visitors to name the primary predator of the colossal squid. In this case, there is only one correct answer: the sperm whale. Chart 9 shows that most visitors chose the correct answer (82%). Chart 9. Answers chosen by visitors in response to the question of what is the primary predator of the colossal squid. The final question asked visitors what points of information they recall from visiting the Creatures of the Abyss exhibition. Table 2 shows the percentage of visitors recalling each statement, which is fairly evenly distributed without any statement being recalled significantly more than another. Table 2. Percentage of visitors who recalled cephalopod statements. Cephalopods have special adaptations for life in the ocean. Which of the following do you remember? Percentage of visitors who recalled the information Octopuses usually have 8 arms and no tentacles. 38.6% Many cephalopods can change the colour and texture of their skin. 34.1% The only hard body part in cephalopods is the beak. 38.6% All cephalopods have hard, sharp beaks for feeding. 36.4% Squid arms and tentacles can have both suckers and hooks. 29.5% All cephalopods can use jet propulsion to move. 38.6% Cephalopods have similar eyes to humans. Researchers still don t know why deep-sea squids use ink. 36.4% Creatures of the Abyss Evaluation Report 9
2. Visitor Observations Visitor Engagement Profiles (Barriault, 1998, Barriault & Pearson, 2010) were created using the data obtained from the videos taken of visitors using three exhibits in Creatures of the Abyss exhibit. Dive into the Abyss Exhibit!"#$"%&'(")*+),-.-&*#.) '!!" &!" %!" $!" #!"!",-.-&*#)4%('("5"%&)!#*67") 8-2")9*)91"):;<..) ()*+,+-)""./,)0*+-)" 1/2,345/-675" /"'#%-%()0"1'2-*3#.) Chart 10. Visitor Engagement Profile for the Dive into the Abyss exhibit. Fifteen visitors were filmed at the ʻDive into the Abyssʼ exhibit. The Visitor Engagement Profile for the ʻDive into the Abyssʼ exhibit shows that a high number of visitors, 73%, engaged in Breakthrough learning behaviours (Chart 10). Visitors were often seen deeply engaged with the exhibit. When children approached the exhibit, they often thought it was a game and parents would often sit down with their children and go through the exhibit, exploring each of the animals highlighted. They would often ask each other questions about the animals, engage in dialogue and then continue to explore. Eighty percent of visitors reached transition behaviour and 73 % of visitors reached breakthrough behaviour. Creatures of the Abyss Evaluation Report 10
Ocean Issues Globe Exhibit!"#$"%&'(")*+),-.-&*#.) '!!" &!" %!" $!" #!"!" There were 43 visitors filmed interacting with the ʻOcean Issues Globeʼ. Nearly 50% of visitors showed Transition Behaviours and 39.5% of visitors engaged in Breakthrough Behaviours (Chart 11). This percentage of Breakthrough Behaviours shows that this exhibit is quite successful at attracting and retaining visitors in a learning experience. Most visitors who showed Breakthrough Behaviours were often described as ʻseeking and sharing informationʼ with other visitors. The visitors would often show others what was on the globe and became engaged by exploring the different facets of the experience. Cephalopods Exhibits,-.-&*#)4%('("5"%&)!#*67") 8$"'%)9..3.".)!#*:"$;*%)<7*=") ()*+,+-)"./,)0*+-)" 1/2,345/-675" /"'#%-%()0"1'2-*3#.) Chart 11. Visitor Engagement Profile for the Ocean Issues Globe exhibit.!"#$"%&'(")*+),-.-&*#.) '!!" &!" %!" $!" #!"!",-.-&*#)4%('("5"%&)!#*67") 8"91'7*9*:.;)<=3-:>.)4?"),-"@A)8"91'7*9*:)B%CA)8"91'7*9*:)0"'C.) ()*+,+-)"./,)0*+-)" 1/2,345/-675" /"'#%-%()0"1'2-*3#.) Chart 12. Visitor Engagement Profile for the Cephalopods exhibit. Of the eight exhibits that focused on cephalopods, three of the exhibits were filmed. The exhibit filmed were the ʻSquidʼs Eye Viewʼ, ʻCephalopod Inkʼ, and ʻCephalopod Beaksʼ. These three exhibits were grouped together for analysis as part of the larger ʻCephalopodʼ exhibit. Twenty seven visitors were filmed interacting with these exhibits. Thirty seven percent of visitors showed transition behaviours through repeating the activity and showing positive emotional responses. A moderate number of visitors engaged in Breakthrough Behaviours, 30% at this exhibit. It was observed that older people were more likely to stop to watch the videos as well as look at the objects, while children looked more at the objects than the video. Creatures of the Abyss Evaluation Report 11
E. DISCUSSION Visitors self reported that they did gain knowledge and understanding in the Creatures of the Abyss exhibition. While some visitors reported higher knowledge on different topics before visiting Creatures of the Abyss, visitors still reported an increase in their knowledge and understanding after visiting the exhibition. Visitors were also able to answer knowledge-based questions regarding the exhibition. Due to the nature of the evaluation methods, it is not possible to say with confidence that this knowledge was gained in the exhibition however the findings are encouraging considering the goals and objectives of the exhibition seem to resonate as take away messages for our visitors. The ʻDive into the Abyssʼ exhibit had the greatest number of visitors showing Breakthrough Behaviours at 73%. This is an extremely high percentage and shows the learning successes of the exhibit. However, it was very interesting to notice how children approached the exhibit. They often thought the exhibit was a game, where they were expected to shoot and kill the different animals. This may have been due to the controls of the exhibit that consisted of a joystick and a bulls-eye graphic on the exhibit screen. After they were shown by an adult or a staff member the actual purpose of the exhibit, they began to get really involved and engaged in learning from the exhibit. The ʻOcean Issues Globeʼ exhibit had a good number of visitors reaching Breakthrough Behaviours while the ʻCephalopodsʼ exhibits had an adequate but lower number of visitors reaching breakthrough behaviours. Creatures of the Abyss Evaluation Report 12