Why not Conduct a Survey?

Size: px

Start display at page:

Download "Why not Conduct a Survey?"

Joanna Henry
5 years ago
Views:

1 Introduction Over the past decade, electronic books (e-books) have become increasingly popular in the academic community. In response to this demand, Columbia University Libraries/Information Services (CUL/IS) provides access to over two million e-books that support research, teaching, and learning activities across campus and within the wider scholarly community. Data collected through COUNTER usage statistics and the LibQUAL+ service quality survey tells us that faculty, graduate students, and undergraduates value access to the growing e-book collection at CUL/IS. While the aggregate results indicate that e-book use continues to increase, usage rates are not uniform across disciplines. Anecdotal evidence suggests that while e-book use has grown in the sciences and social sciences, scholars in the arts and humanities rely heavily on print books. Given the highly diverse research needs of the university community, we want to understand scholarly e- book usage in different disciplines.

2 Why not Conduct a Survey? Our initial thought was to develop a campus-wide survey. Three key factors influenced our assessment strategy and motivated us to tap into existing data sources instead. 1. Due to our interest in continuously monitoring our user base in an ever-changing e-book landscape, reliance on readily available, continuous, and accurate data was an important factor in creating a sustainable assessment plan. 2. Survey research has experienced significant challenges that impact its use in library assessment plans. The quality of the data begins to deteriorate when potential respondents do not make the effort to submit a completed survey. 3. Surveys are of little, or no use, if the response rate is low or the data is inaccurate. Based on the low response rates from a recent survey, and in an attempt to avoid survey fatigue, we investigated alternative approaches of data collection.

3 Research Design The study utilized text data from two sources: users e-book search queries entered into CLIO and e-book title words provided by the COUNTER Book Report 2 (R4) usage reports. Data collection was limited to six major platforms: Cambridge Books Online, Ebrary, EBSCOhost, ScienceDirect, Oxford Scholarship Online, and SpringerLink. We began analysis by identifying and quantifying words from users search queries with the purpose of exploring usage, and completed our analysis by examining the contexts within which these words were used. Using the mixed method analysis software tool, NVivo, we ran two word frequency queries using two different text match options; exact-match and stemmed-word match (grouping the words with the same stem), to generate lists of the most frequently occurring search terms and e-book title words. Then, we examine frequently occurring search terms to understand the meaning, purpose, and significance of these words, and the implications for e-book search, discovery, and delivery process.

4 Study Findings 1. Search Query Length Operational Definition: The number of words in a search query, where a word is defined as a string of characters delimited either by a space or by the end of the query. We discovered that the users searching behavior when using a library catalog is different than users searching behavior on the Web. The average search query length when searching CLIO (m=3.62 terms) compared to the average search engine query length (m=2.4 terms) may indicate that library users are more sophisticated in how they structure their queries when they are looking for very specific items or specific answers. Query length in words Number of occurrences % of queries 1 7, % 2 15, % 3 10, % 4 6, % 5 4, % 6 2, % 7 1, % 8 1, % % % % % % % % % % % % % 20 or more words % Total 53, %

5 Study Findings Rank Exact-match word Count Stemmed-word Count Similar Words 2. The Nature of E-Book Use We ran two word frequency queries using both exact-match and stemmed-word match options to generate a list of most frequently occurring e-book search terms. The prominence of topical words such as history, social, and politics in the list was an interesting reflection on the kinds of works users were looking for, as were the terms handbook, guide, and manual. The high frequency of these words lead us to believe that users were searching for broad topics, reference works, or other collections of instructions, all of which are intended to provide ready reference. 1 history 1,062 history 1,096 histories, history 2 theory 787 theory analysis 766 analysis 766 analysis 4 new 681 new 681 new 5 introduction 669 introductions social 638 statistics health 566 socialization handbook 556 politics american 539 americans 570 theorie, theories, theory introduction, introductions statistic, statistical, statistics 10 research 497 health 566 health 11 management 450 handbooks 561 social, socialism, socialization, socializing polite, politeness, political, politics american, americanizing, americans handbook, handbooks 12 statistics 419 managment 529 managed, management, manager, managers, managing, managment

Study Findings 2. The Nature of E-Book Use, Cont d Using NVivo, we created word clouds to graphically display what types of e-books users were searching for (e.g. broad topic, level of academic use, and genre).

sciences, and sciences, were searching for e-books.

6 Study Findings 2. The Nature of E-Book Use, Cont d Using NVivo, we created word clouds to graphically display what types of e-books users were searching for (e.g. broad topic, level of academic use, and genre). A word cloud generated by using exact match option for the top one thousand most frequently occurring search terms implied that users across from all major disciplines, namely humanities, social sciences, and sciences, were searching for e-books. To determine how well our search term findings correlate with what actually being used, we conducted a similar analysis using COUNTER Book Report 2 data. The prominence of topical words such as history in both lists was an interesting reflection on the kinds of works being used, as were the terms handbook, guide, and manual.

7 Study Findings 3. Contextualizing Title Words We performed a visual scan of e-book titles with the most heavily requested book chapters. An evaluation of highest-ranking e-book titles revealed that the titles were reference materials (e.g. Oxford English Dictionary) and undergraduate texts (e.g. Real Analysis and Applications). Title Publisher Platform YTD Total Oxford English Dictionary Oxford U Press OED 57,138 Epidemiology Matters: A New Introduction to Methodological Foundations Oxford U Press EBRARY 26,298 Encyclopedia of New York City (2nd Edition) Yale U Press EBRARY 17,619 Real Analysis and Applications Springer SpringerLink 15,136 Introductory Statistics for the Behavioral Sciences (7th Edition) John Wiley & Sons EBRARY 11,705 Advanced Calculus Springer SpringerLink 11,176 Social Work Values and Ethics Columbia U Press EBRARY 10,263 Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty PublicAffairs EBRARY 9,745 Wiley Series in Probability and Statistics: Analysis of Financial Time Series (3rd Edition) John Wiley & Sons EBRARY 9,236 Greenwood Publishing Group EBRARY 8,621 Managing Organizational Behavior Spatially Integrated Social Science Oxford UPress EBRARY 8,528 Principles of Turbomachinery John Wiley & Sons EBRARY 8,486 Japan to 1600: A Social and Economic History U of Hawaii Press EBRARY 8,454 Craft of Research (3rd Edition) U of Chicago Press EBRARY 7,853 PCR Cloning Protocols Springer SpringerLink 7,394 Wealth and Welfare States: Is America a Laggard or Leader? Oxford U Press EBRARY 7,292 Information Security and Privacy Springer SpringerLink 7,065 Effective Public Manager: Achieving Success in Government Organizations (5th Edition) John Wiley & Sons EBRARY 6,582 Business and Environmental Policy: Corporate Interests in the American Political System MIT Press EBRARY 6,340 Elusive Quest for Growth: Economists' Adventures and Misadventures in the Tropics MIT Press EBRARY 6,269 Prenatal and Postnatal Care John Wiley & Sons EBRARY 6,264

8 Conclusion The strength of the study is that it involves real users, using real queries, with actual information needs, and actual usage. The ability to analyze e-book search queries allowed us to discover many search patterns that we wouldn t otherwise observe. For instance, we discovered that the users searching behavior when using a library catalog is different than users searching behavior on the Web. Text analysis of search terms and requested title words provided insight into the nature of e-book use across disciplines, including broad topic (e.g. history), academic level of use (e.g. introductory), and genre/type (e.g. reference). It is challenging to deduce reader intent from word frequencies, as text data remain widely open for interpretation. However, responses to open-ended questions from the most recent LibQUAL+ survey are consistent with our findings that e-book collections are widely used across all major disciplines to support instruction and learning.

9 Future Studies Search phrases provided a good deal of information about what types of e-books users search for, but much less information about why searches are conducted or how satisfied users are with the discovery process as a whole. This knowledge gap must be taken into account in analyses and complemented by other techniques to provide a more complete understanding of search behaviors. Despite these limitations, the methodology is extremely effective at capturing actual user behavior, not recalled behaviors or subjective impressions of interactions.

10 Acknowledgements We would like to thank Bob Scott, Digital Humanities Librarian, for sharing his NVivo expertise and generating many of the word clouds used in the study. Image source:

Nisa Bakkalbasi, Assessment Coordinator Melissa Goertzen, E-Book Program Development Librarian. *Photo credit: M. Goertzen

Nisa Bakkalbasi, Assessment Coordinator Melissa Goertzen, E-Book Program Development Librarian *Photo credit: M. Goertzen Since 2010, there has been marked growth in Columbia University Libraries (CUL)