AC : ANALYSIS OF ASEE-ELD CONFERENCE PROCEEDINGS:

AC 2010-1047: ANALYSIS OF ASEE-ELD CONFERENCE PROCEEDINGS: 2000-2009 David Hubbard, Texas A&M University David E. Hubbard is an Assistant Professor and Science & Engineering Librarian at the Sterling C. Evans Library, Texas A&M University, College Station, TX. He received his B.A. in chemistry from the University of Missouri-St. Louis in 1988 and M.A in library science from the University of Missouri-Columbia in 2003. American Society for Engineering Education, 2010 Page 15.177.1

Analysis of ASEE-ELD Conference Proceedings: 2000-2009 Abstract This study examines the papers and posters from the annual American Society for Engineering Education Engineering Libraries Division (ASEE-ELD) programs over the last ten years. The bibliometric analysis provides an overview of authorship and content, as well as insights into the Division and what it values. It also gives context that is useful to both newer and longtime ASEE-ELD members. Unlike many other studies of single publications, contributions to annual ASEE-ELD programs were not systematically indexed and tangible artifacts do not exist for all contributions. Primary sources were consulted to identify the contributors and their contributions; however, this was often limited to just bibliographic information. The advent of publish-to-present for all papers and posters in 2009 will provide systematic archiving in the future, but is of limited use for the period studied. Considering the move to publish-to-present and a decade that brought significant change to the profession, it seems appropriate to reflect upon the past decade through such an analysis. The contributors were summarized both qualitatively and quantitatively in terms of authorship, co-authorship, and institutional/organizations affiliations. Since full-length papers do not exist for all contributions prior to 2009, content analysis is based on titles of the papers and posters. The titles were analyzed using both a standard classification scheme and textual analysis software to identify topics and keywords/phrases, respectively. These topics and keywords/phrases were further analyzed for patterns and trends. The analysis not only presents a snapshot of where our profession and Division has been, but potentially identifies future directions. Introduction Bibliographic analysis of a single publication can provide valuable insights into a publication, as well as the interests and values of an organization if associated with a particular professional society. Single publication studies are performed for a variety of reasons, though most often to study the characteristics, trends, impact, and authorship within a specific publication or discipline. The objective of this study is to examine authorship and content of the American Society for Engineering Education Engineering Libraries Division (ASEE-ELD) papers and posters from 2000 to 2009 in order to obtain a snapshot of the state of the publication and identify trends. The idea of a single bibliometric publication study is not new. To date, there have been over 180 single journal bibliometric studies and the field has matured to the point that there are at least two review articles summarizing these studies. Tiew 1 reviewed the literature from 1969 to 1997 and identified 102 single publication studies covering library and information science, medicine, science and technology, and the arts, social sciences, and humanities. He categorized the 102 studies into four general categories: bibliometric studies on single journals (40 studies), citation analysis of single journals (45 studies), content analysis of single journals (11 studies), and other aspects of bibliometric study on single journals (6 studies). These studies were reviewed and discussed in terms of the four categories and the aforementioned disciplines. Anyi, Zainab, and Anuar 2 examined the literature from 1998 to 2008, essentially continuing the work of Tiew. Page 15.177.2

They identified 82 single publication studies covering: arts, humanities, and social sciences (12 studies), medical and health sciences (16 studies), science and technology (25 studies), library and information science (21 studies). It should be noted that the 182 studies do not correspond to 182 unique titles, since some publications were revisited several times over the years. Based on the analysis of the 82 studies, Tiew 1 identified seven general bibliometric measures for single publication studies: article productivity, author characteristics, author s productivity, coauthorship pattern, content analysis, citation analysis, and characteristics of the editorial board. Within each of the major categories there were one or more specific measures totaling over 40 individual measures (or methodologies). Appendix A lists all the measures identified, though it is important to note that not all of these measures are explored in every study and some studies only focused on a specific aspect (or used a specific methodology). ASEE-ELD was organized in 1967 though there was no requirement for authors to submit fulllength papers for ASEE-ELD sessions prior to 2009; however, some authors voluntarily submitted full-length papers to the annual proceedings. Since 2009, all authors are required to submit full-length, peer reviewed papers for all ASEE paper and poster presentations. Preconference, panel discussion, and distinguished speaker sessions are exempted from this publication requirement. So unlike many other studies of single publications, contributions to annual ASEE-ELD programs were not systematically indexed and no tangible artifacts (e.g., papers or abstracts) exist for all contributions. In the case of ASEE-ELD papers and posters, the tangle artifact is often limited to just bibliographic information (title, author, and affiliation). The lack of papers and indexing limits the types of bibliometric analyses that can be performed. Despite the apparent limitations, various aspects of at least five measures outlined in Appendix A were examined in this study. The five measures examined in this paper focus on article productivity, author characteristics, author s productivity, co-authorship pattern, and content analysis. Only one of the measures, citation analysis, was not possible due to the lack of citation indexing. Another measure, characteristics of the editorial board, was not explored since the publication does not have an editorial board per se. The descriptions of these measures are provided in Appendix A and will not be reproduced in this section; however, content analysis requires further comment and is discussed below. The other measures employed will be discussed in more detail within the context of the methodology. There are numerous definitions for content analysis, but one of the classic definitions is a research technique for the objective, systematic, and quantitative description of the manifest content of communication. 3 Allen and Reser 4 found that the meaning of content analysis varies considerably within library and information science (LIS) literature as does its application. Approaches to content analysis in the LIS literature are almost as numerous as the individuals that use the methodology. According to Allen and Reser, there are two basic types of content analysis: classification analysis and elemental analysis. 4 The former uses methods to assign documents to various subject groups based on an exhaustive and mutually exclusive subject or classification scheme, whereas the latter identifies words and word frequencies of documents. Anyi, Zainab, and Anuar 2 identified 19 different measures or methodologies used for content analysis among the 82 bibliometric studies examined (See Appendix A). A number of the Page 15.177.3

measures cannot be used in this study for the aforementioned reasons (i.e., lack of a full-length papers and indexing). Since the only information available for each paper and poster is bibliographic, one cannot determine such measures as types of models, theories and framework used or the number of pages per article. Some of the methods that can be used on the ASEE- ELD papers and posters are subject area of articles and article title analysis based on paper and poster titles. Several classification schemes have been developed and used for content analysis within the LIS literature (e.g., Atkins 5 ; Feehan, Gragg, Havener, and Kester 6 ; Nour 7 ; and Pertiz 8 ), however, the scheme developed by Jarvelin and Vakkari 9 has gained widespread acceptance for its systematic approach to subject analysis compared to the more theoretical focus of other studies. The classification scheme of Jarvelin and Vakkari 9 is outlined in Table 5 and used in this study. Textual analysis is a form of content analysis that examines a text and its words for meaning. Identification of the words and their frequency is the most basic form of textual analysis it is what Allen and Reser 4 would refer to as elemental analysis. There is a wide array of software available for performing textual analysis, both free and commercial. Several websites list commercial and freely available software for textual analysis (e.g., www.textanalysis.info, www.kdnuggets.com, and http://www.content-analysis.de/). In addition to standard textual analysis software, Wordles 10 are increasing being used for textual analysis to better visualize data as word clouds and to obtain word frequencies. For example, Harris, Lecroq, Kucherov and Lonardi 11 used Wordle to analyze the titles of conference proceedings for the Symposium of Combinatorial Pattern Matching. This study will also employ Wordle to perform article title analysis, as well as use freely available textual analysis software called Textalyser. 12 Using bibliometric measures and content analysis methodologies outlined above, this paper will provide a snapshot of the ASEE-ELD Annual Conference proceedings and identify trends. Methodology The analysis was limited to papers and posters presented at ASEE-ELD annual conferences from 2000 to 2009. The data were obtained from the annual conference program guides, except for the 2001 poster session that was omitted from the original program guide and obtained from the ASEE-ELD archives. Only papers and posters that included authors in the index of conference program guide were included, so almost all pre-conference programming, panel discussions, and forums were omitted from this study. These were omitted for several reasons, but mainly because they often lacked specific authors and individual titles beyond the session title. Due to the similarity of paper and poster presentations at ASEE-ELD sessions and to simplify the language throughout this paper, papers and posters will hereafter be referred to as papers unless specified otherwise. The authors, paper titles, affiliations, year, and other bibliographic information were entered into an Excel spreadsheet based on the annual ASEE conference program guides. The data were then sorted and analyzed in several ways to explore authorship and content. Page 15.177.4

The authorship was first examined quantitatively by determining the overall number of papers and posters, as well as the average number of authors per paper and poster for 2000 to 2009. Authorship was also explored on an individual basis by determining the number of times each author presented a paper. A similar approach was used to examine affiliation and the number of times each institution/organization was represented. While authorship was credited each time an author s name appeared as an author or co-author, affiliation was handled a little differently. For affiliation, the institution was only credited once for each paper; otherwise a paper written by five authors from the same institution would be counted five times. On the other hand, each of the five authors would each be counted once for their contribution. Co-authorship was examined in terms of both the number of co-authors and the types of co-authorship (or collaboration). The various types of co-authorships were plotted and linear regression used to identify any significant trends over time. The papers were categorized based on a standard classification scheme developed by Jarvelin and Vakkari 9, which has been widely used, adopted, and cited for this purpose. The 30 classes (or topics) are listed in Table 5. One major limitation with this particular analysis is that the content analysis is based solely on the titles of the papers since full-length papers generally do not exist. Textual analysis of the paper titles was performed using Textalyser for single and two-word phrase frequencies. This was accomplished by cutting and pasting all the titles of the ASEE-ELD papers from the spreadsheet into a Word document and then ultimately the Textalyser software. The most frequently used words and phrases were then identified. In a similar manner, Wordle was used to provide a better visualization of the word frequencies. Results and Discussion There were 258 papers and posters presented at the ASEE-ELD sessions during ASEE Annual Conferences from 2000-2009. Of the 258, there were 170 (66%) papers and 88 (34%) posters. Figure 1 shows the annual number of papers and posters presented at each annual conference from 2000-2009. The authorship was examined in a number of ways. Figure 2 presents the number of authors per publication. The majority of the publications (72.5%) were single authored. The number of authors per paper averaged 1.50 and ranged from 1 to 9 authors. Figure 3 and Figure 4 show plots of the average number of authors per paper and poster over the 10-year period, respectively. The average number of authors was found to increase from 1 to 2 authors per paper and poster over the last 10 years and the trend is statistically significant. There were 258 individual authors or co-authors responsible for the 258 papers from 2000-2009. Considering that there are over 600 colleges and universities with ABET-accredited engineering programs in the United States 13, each with at least one engineering librarian, the number of ASEE-ELD papers contributed compared to the total number of engineering librarians is relatively small or from a select group. Furthermore, some of the authors and co-authors of the Page 15.177.5

258 papers are not engineering librarians, but instruction librarians, engineering faculty, vendors, etc. Figure 1. Number of Papers and Posters, 2000-2009 Number of Paper/Poster r 35 30 Paper Poster 25 20 15 10 5 0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 Year Figure 2. Number of Authors per Publication, 2000-2009 Number of Papers sss 200 150 100 50 0 1 2 3 4 5+ Number of Authors Page 15.177.6

Figure 3. Average Number of Authors per Paper, 2000-2009 Number of Authors s 3.0 2.5 r = 0.8767, p = 0.0009 2.0 1.5 1.0 0.5 0.0 2000 2002 2004 2006 2008 2010 Year Figure 4. Average Number of Authors per Poster, 2000-2009 Number of Authors s 3.0 2.5 r = 0.8217, p = 0.0035 2.0 1.5 1.0 0.5 0.0 2000 2002 2004 2006 2008 2010 Year Table 1 summarizes the number of times an individual authored or co-authored a paper for an ASEE-ELD session. The productivity of individual authors loosely follows Lotka s Law, which predicts that the number of authors contributing a particular number of papers is inversely proportional to the number of papers contributed. For comparison, percentages of authors contributing a certain number of papers as predicted by Lotka s Law are also presented in Table 1. Page 15.177.7

Table 1. Number of Individuals Authoring or Co-Authoring a Specified Number of Papers, 2000-2009 Number of Papers Number of Individuals Lotka s Prediction (%) 1 184 (73%) 60 2 42 (15%) 15 3 15 (5%) 7 4 8 (3%) 4 5 8 (3%) 2.4 6 0 (0%) 1.7 7 1 (0.4%) 1.2 There were 127 institutions/organizations associated with the 258 papers. Table 2 presents the number of individual institutions/organizations associated with each of the 258 papers and the institutions/organizations associated with 5 or more papers are presented in Table 3. Table 2. Number of Institutions/Organizations Associated with a Specified Number of Papers, 2000-2009 Number of Papers Number of Institutions/Organizations 13 1 12 2 11 0 10 0 9 1 8 0 7 5 6 3 5 4 4 5 3 16 2 23 1 67 Table 4 describes the relationships and summarizes the frequency of the six different types of coauthorships identified in the ASEE-ELD papers. Of the 258 papers, 71 papers were authored by 2 or more authors. The nature of these collaborations was explored based on the relationship of the co-authors. Page 15.177.8

Table 3. Institutions/Organizations Associated with 5 or More Papers, 2000-2009 Rank Institution/Organization Number of Papers 1 Purdue University 13 2 Massachusetts Institute of Technology 12 North Carolina State University 12 3 Pennsylvania State University 9 4 Drexel University 7 University of Arizona 7 University of California - Berkeley 7 University of Illinois at Urbana-Champaign 7 5 University of Minnesota - Twin Cities 7 Colorado School of Mines 6 Cornell University 6 University of Wisconsin - Madison 6 6 Bucknell University 5 Northern Illinois University 5 University of Washington 5 Virginia Polytechnic Institute and State University 5 Table 4. Type and Frequencies of Co-Authorship Type of Co-authorship Librarian + Faculty Librarian + Librarian (Same Institution) Librarian + Librarian (Different Institution) Librarian + Vendor Librarian + Other Other Description Co-authorship involving at least one librarian and faculty member outside of the library. Co-authorship involving two or more librarians at the same institution. Co-authorship involving two or more librarians at different institutions. Co-authorship involving at least one librarian and one vendor. Co-authorship involving at least one librarian and one other individual (e.g., students and/or staff outside the library). Co-authorship that does meet any of the other five criteria (e.g., two engineering faculty). Number of Papers 21 32 10 5 2 1 Page 15.177.9

All six types of co-authorship were plotted for the 10-year period and linear regression was used to explore trends; however, only one proved to be statistically significant. The one type of coauthorship that that showed a statistically significant increase over time is the number of Library + Faculty co-authorships (Figure 5), which represented 8.1% of all 258 papers and 30% of the 71 co-authored papers. Figure 5. Librarian + Faculty Co-Authorships, 2000-2009 Number of Papers sss 6 5 4 3 2 1 r = 0.8657, p = 0.0012 0 2000 2002 2004 2006 2008 2010 Year Table 5 summarizes the classification of the 258 papers based on the Jarvelin & Vakkari 9 classification scheme. The four largest topics were user education, information/ reference services, other types of databases (i.e., non-bibliographic), and collections. These four categories represent over 50% of the topics covered within in the 258 papers. All but 4 of the 30 topics from the Jarvelin and Vakkari 9 classification scheme were represented. The four topics that were not represented were: methodology (i.e., study of research methods), analysis of LIS (i.e., empirical and theoretical methods used), automation study, and cataloging. Considering that the Jarvelin and Vakkari 9 study was conducted almost 20 years ago and used 37 core journals covering a range of LIS topics, one must be cautious about comparing results and drawing inferences between the two studies. If one does compare what Jarvelin and Vakkari 9 referred to as Professional LIS articles to the ASEE-ELD papers in this study, many more ASEE-ELD papers were found under the major categories of Library and Information Service Activities (59.3% versus 35.2%) and Information Seeking (10.9% versus 2.3%). The majority of the former was primarily due to a large number of papers focusing on user education and information/reference services. Information Seeking was buoyed by a number of user studies. One area that was noticeably lower was Information Storage and Retrieval despite a number of papers on various databases and search interfaces. It appears that the absence of cataloging and classification might have led to a lower percentage in this category compared to LIS literature in general. The underrepresented topics may represent areas for future papers, though there may be an inherent preference for the areas currently represented by the ASEE-ELD membership. Page 15.177.10

Table 5. Distribution of Topics in ASEE - ELD Papers and Professional LIS Articles Topics ASEE - ELD Papers Jarvelin and Vakkari n % n % The profession 15 5.8 16 4.2 Library history 2 0.8 5 1.3 Publishing 4 1.6 23 6.0 Education in LIS 2 0.8 10 2.6 Methodology 0 0.0 0.0 Analysis of LIS 0 0.0 2 0.5 Library and information service activities 153 59.3 135 35.2 Circulation or interlibrary loans 3 6 Collections 21 30 Information/reference services 26 9 User education 63 6 Library buildings and facilities 9 1 Administration or planning 2 31 Automation study 0 14 Other L&I service activities 11 1 Several interconnected activities 18 37 Information storage and retrieval 33 12.8 96 25.0 Cataloguing 0 17 Classification and indexing 1 21 Information retrieval 4 19 Bibliographic databases/bibliographies 6 24 Other types of databases 22 15 Information seeking 28 10.9 9 2.3 Dissemination of information 2 3 Use/users of channels/sources of info 16 Use of L&I services 1 Information seeking behavior 5 Use of information 3 Information management, IRM 1 6 Scientific & professional communication 13 5.0 6 1.6 Scientific/professional publishing 9 4 Citation pattern & structures 1 Other aspects 3 2 Other aspects of LIS 8 3.1 82 21.4 Totals 258 100.0 384 100.0 Page 15.177.11

The results of the textual analysis on the titles are presented in Table 6 and Table 7. Textalyser found 831 unique words among the titles of the ASEE-ELD papers, but the results presented in Table 6 and their frequencies are limited to the twelve most frequent words. This was done to create a manageable list with a logical break in the data. The Textalyser analysis was hampered in a number of ways, including the inability to omit stop words from 2-word phrase frequencies. For example, of engineering and in the were the second and third most common 2-word phrases, respectively. To make more sense of the results, these types of 2-word phrases were omitted from list of the five highest ranked two-word phrases list in Table 7. Table 6. Words and Frequencies Appearing in Titles of ASEE-ELD Papers Rank Word Frequency Percent of Total Words 1 engineering 111 5.9 2 information 59 3.2 3 library 58 3.1 4 literacy 26 1.4 5 students 25 1.3 6 digital 19 1.0 7 new 17 0.9 collection 17 0.9 8 faculty 16 0.9 9 science 15 0.8 10 using 14 0.7 university 14 0.7 Table 7. Two-Word Phrases and Frequencies Appearing in Titles of ASEE-ELD Papers Rank Two-Word Phrases Frequency 1 information literacy 26 2 engineering students 15 3 engineering library 7 4 digital library 6 library instruction 6 collection development 6 5 first year 5 engineering design 5 case study 5 More sophisticated textual analysis software might be able to eliminate these stopwords from phrase lists, as well as create categorical hierarchies of subjects. However, this was beyond the functionality of Textalyser. The results of this textual analysis point to an emphasis on information literacy/instruction/education, engineering students, faculty, collection development, Page 15.177.12

reference, information, libraries, and various digital aspects. It might be just as interesting to consider what words or phrases (i.e., topics) are absent from the lists. The results of the Wordle analysis are presented in Figure 6. To ensure the figure was legible, the analysis was limited to the 50 most frequent words from the ASEE-ELD paper titles. The results provide greater visual impact compared to the list presented in Table 6 but is essentially same information using different software. While neither of the textual analyses is particularly insightful, both support many of the results found during the classification analysis. Figure 6. Wordle of the Fifty Most Frequent Words Appearing in Titles of ASEE-ELD Papers Conclusion This study provided a snapshot of the ASEE-ELD conference papers and posters from 2000 to 2009. The analysis of these conference proceedings posed several unique challenges compared to other single publication bibliometric studies. This was primarily due to the lack full-length papers and indexing. This limited the types of bibliometric and content analyses that could be performed. Despite these limitations, several methodologies were successfully applied and significant trends identified. One of the most significant trends is the increase in co-authorship. Co-authorship doubled over the 10-year period. More interesting was the increase in co-authorship with faculty outside the library, which also showed a statistically significant increase. While not surprising, authorship did loosely follow Lotka s Law in that the proportion of individual authors making a contribution decreased according to an inverse power law. The classification scheme for the content analysis was subjective, but did point to the emphasis on information literacy, information/reference services, collections and novel databases. While textual analysis was limited in terms of results, it did provide further support for the classification analysis based on word and phrase frequencies of the paper titles. A future study might apply a more rigorous textual analysis methodology (and software) to obtain topic hierarchies and to perform more sophisticated analyses. Further work could compare these results to other LIS publications in the field of science and engineering librarianship. Page 15.177.13

References 1. Tiew, S. 1997. Single journal bibliometric studies: A review. Malaysian Journal of Library & Information Science 2 (2): 93-114. 2. Anyi, K., A. Zainab, and N. Anuar. 2009. Bibliometric studies on single journals: A review. Malaysian Journal of Library & Information Science 14 (1): 17-55. 3. Berelson, B. 1971. Content Analysis in Communication Research. New York: Hafner, 18. 4. Allen, B., and D. Reser. 1990. Content analysis in library and information science research. Library & Information Science Research 12 (3): 251 262. 5. Atkins, S. 1988. Subject trends in library and information science research, 1975-1984. Library Trends 36 (4): 633-658. 6. Feehan, P., W Gragg, W. Havener, and D. Kester. 1987. Library and information science research: An analysis of the 1984 journal literature. Library & Information Science Research 9 (3): 173-185. 7. Nour, M. 1985. A quantitative analysis of the research articles published in core library journals of 1980. Library & Information Science Research 7 (3): 261-273. 8. Peritz, B. 1980. The methods of library science research: Some results from a bibliometric survey. Library Research 2 (1): 251-264. 9. Jarvelin, K., P. Vakkari. 1990. Content analysis of research articles in library and information science. Library & Information Science Research 12 (4): 395-421. 10. Feinberg, J. 2009. Wordle. http://www.wordle.net/ [accessed October 1, 2009]. 11. Harris, E., T. Lecroq, G. Kucherov, and S. Lonardi. 2009. CPM s 20th anniversary: A statistical retrospective. In 20th Annual Symposium on Combinatorial Pattern Matching, edited by G. Kucherov and E. Ukkonen, 1-11. Berlin: Springer. 12. Textalyser.net. 2004. Textalyser. http://textalyser.net/ [accessed October 1, 2009]. 13. ABET. 2010. ABET. http://www.abet.org/ [accessed March 2, 2010]. Page 15.177.14

Appendix A. Bibliometric Measures Identified by Tiew 1 1. Article productivity Number of articles published by issues, volumes and years 2. Author characteristics Authors gender, profession, rank, academic title Authors geographical affiliations by institutional names and types of institutions (academic, professionals) Authors location by region, or country 3. Author s productivity Rank list of core and active authors Authorship productivity pattern may be tested with Lotka s Law of authorship distribution 4. Co-authorship pattern Types of co-authored works Degree of collaboration Local and foreign collaboration activities among authors by country and institution Internationalization status of the journal 5. Content analysis Subject areas of articles, keyword analysis, keyword co-occurrence network Article title analysis, number of words, punctuation usage, word frequency and preposition usage Number of pages per article Journal circulation Journal frequency Types of research methodology used Types of models, theories and framework used Analysis of acknowledgement Analysis of funding received Analysis of article appendices Analysis of article abstracts Acceptance rate Analysis of indexation and abstraction information Language of publication 6. Citation analysis Number and distribution of citations per article, volumes and years Authorship pattern of citations Author co-citation analysis network Most cited author Types of literature cited Age of cited literature Cited literature s half-life Rank list of core journals using Bradford s Law Extent and growth of web citations Journal citation identity, analysis of references in articles from the journal Journal citation image, analysis of citations to the journal Journal influence and diffusion in other subject areas Geographical location and language distribution of cited literature Journal self-citation Author self-citations Journal performance, quality and prestige as measured by journal impact factor, prestige index, trajectory index, immediacy index, journal attraction power, journal consumption power and discipline contribution score 7. Characteristics of the editorial board List and geographical distribution of editorial board members List and geographical distribution of reviewers Editorials and reviewer s gender, profession, qualification, academic rank, publication productivity prior and post appointment Editorial policy Page 15.177.15