Introduction to Citation Metrics Library Tutorial for PC5198 Geok Kee slbtgk@nus.edu.sg 6 March 2014 1
Outline Searching in databases Introduction to citation metrics Journal metrics Author impact metrics Managing your publication history 2
Research topic Searching in Databases 1 List search 2 Form search terms statement 3 Choose a database Keywords, synonyms, alternative words 3
Operators AND OR Truncation *,?, $ Phrase searching Proximity Operators NEAR, SAME Parenthesis, ( ) Function Results must contain all search terms (To narrow searches) waves and interference Result contains either of the search terms (To broaden searches) handphone or mobile phone enzym* enzyme, enzymes, enzymatic, enzymology (zero to many characters) synthesi?e synthesise, synthesize (one character) colo$r color, colour (zero or one character) Search for exact phrase energy conservation Used to specify search terms appearing in the same sentence. fiber NEAR composites Use for grouping of words/phrases & to specify order of search 4
Topic Study of electronic structure of graphene using spectroscopic ellipsometry and ultrafast pumpprobe techniques 5
Search statements electronic structure graphene electronic structure AND graphene ( ) graphene AND ellipsometry OR pump probe 6
Web of Science Core Collection One of the largest citation index 52 million papers, 800 million citations Multidisciplinary Organised into 250 categories covering Science, Social Sciences, Arts & Humanities Our subscription Science Citation Index Expanded (1900-present) Social Sciences Citation Index (1900-present) Arts & Humanities Citation Index (1975-present) Conference Proceedings Citation Index- Science (1990-present) Conference Proceedings Citation Index- Social Science & Humanities (1990-present) Updated weekly (Thomson Reuters, 2013) 7
Search history 8
Which journal articles to read? Editors Edited Subject experts Peer Reviewed Review Process Author Self- Published 9
Definition of peer review What is peer review? Peer review is the critical assessment of manuscripts submitted to journals by experts who are not part of the editorial staff. International Committee of Medical Journal Editors (ICMJE). Uniform Requirements for Manuscripts Submitted to Biomedical Journals Source: Hames, Irene (2007). Peer Review and Manuscript Management in Scientific Journals: Guidelines for Good Practice. Blackwell Publishing. 10
Nature Physics 11
12
13
Which journal articles to read? Read peer reviewed articles Read review articles for historical development of a subject Primary vs secondary sources Retrieve relevant papers in your subject area Identify high impact papers Trace research progress using citations 14
Citation network in Web of Science Paper A 2008 2010 2004 Cited References 2007 Times Cited 2013 Citing articles 2006 Related Records 2004 2008 1996 2011 1998 15
Related Records 16
17
Introduction to Citation Metrics Cited Reference Searching Guide: http://libguides.nus.edu.sg/citedref 18
Journal metrics Journal Citation Reports (Thomson Reuters) Impact Factor, 5-Year Impact Factor, Immediacy Index Eigenfactor Score http://www.eigenfactor.org/ Article Influence Score SCImago Journal Rank (SJR) & Source Normalised Impact per Paper (SNIP) Can be accessed via Scopus http://www.scimagojr.com/ 19
Journal Citation Reports Quantitative tool or resource for ranking, evaluating, categorising and comparing journals Published annually, compiled using citation data drawn from over 11,000 scholarly and technical journals from more than 3,300 publishers in over 80 countries 20
Journal Impact Factor Average number of times articles from the journal published in the past two years have been cited in the JCR year Impact Factor = (2012) Cites in 2012 to 2010 & 2011 articles No. of articles published in 2010 & 2011 Used in evaluating or comparing a journal's relative importance to others in the same field/subject category Underlying assumption No. of citations is an indication of the importance & influence of a journal 21
Reviews of Modern Physics 22
Journals in Physics ranked by impact factor 23
Journal impact factor: Trend graph 24
Exercise: Journal rankings in a subject category Launch Journal Citation Reports Select View a group of journals by: Subject category Physics, Multidisciplinary View Journal Data by impact factor Repeat the steps for Mathematics Computer Science Chemistry Engineering Physiology, What do you observe? 25
Exercise Find the impact factor of the journal, Acta Crystallographica Section A Look for the trend graph What can you deduce? 26
Example of a journal where its impact factor is inflated Year Total Citations Impact Factor 2007 260 2.385 2008 201 2.051 2009 6,091 49.926 2010 6,520 54.333 2011 245 2.076 URL: http://libguides.nus.edu.sg/citedref 27
5 Year Journal Impact Factor Based on cites in the current year received by articles published in the prior 5 years For journals Where body of citations is not large enough to make reasonable comparisons With consistently late publication schedules Of subject fields which require longer citation period, e.g. Mathematics, Geology or Management or Sociology, etc 28
Journal immediacy index Average number of times an article is cited in the year it is published Indicates how quickly articles in a journal are cited Immediacy Index (in current year) = no. of citations (in current year) no. of articles (in current year) 29
Issues to note when using impact factor Different disciplines have different publication cultures & citation practices Cannot compare journals across disciplines Impact factor varies considerably among disciplines Short citation window favours rapidly moving fields In some disciplines, many citations occur outside of the 2-year window Not all journals are indexed, analyses are limited to the journals listed in JCR, few publications from languages other than English are included 30
Issues to note when using impact factor May be subject to manipulation Eg Self citations; publish a larger percentage of review articles which receive more citations Review journals tend to have higher impact factors than those publishing original research in the same field Citation profile of journals often skewed Distribution of citation counts for individual papers in a journal is highly skewed 31
Some misconceptions Articles published in a high impact journal will receive more citations No correlation between publishing in a high impact journal and the citations an article receives Individual articles published in a high impact journal must also be of excellent quality It is inappropriate to use impact factors to assess quality of individual articles or evaluate/compare researchers 32
Other Journal Metrics Eigenfactor Score Rates journals according to the number of incoming citations, similar to page-rank algorithm used by Google & citations from highly-ranked journals are weighted higher Article Influence Score SCImago Journal Rank (SJR) Source Normalised Impact per Paper (SNIP) Measures the average influence, per article, of the papers in a journal Also uses algorithms similar to page-rank in Google. Considers the prestige of the journal based on the ranking of the sources of the incoming citations Accounts for subject differences by weighting citations based on the total number of citations in a subject field (Citations in subject field less likely to receive citations are given more weight) URL: http://libguides.nus.edu.sg/citedref 33
Researcher impact metrics Citation based metrics Number of publications Citation Count h-index Other measures: Peer review (Gold standard) - Assessment by fellow researchers Underlying assumption No. of times a publication is cited by other researchers is an indication of the influence or impact the research has within the field Measures of esteem such as invitations, membership on editorial boards, funding grants & awards 34
Cited Reference Search 35
Professor Wee Thye Shen, Andrew Professor Andrew Wee T S 36
Cited Reference Search 37
Issues & limitations of citation metrics Citations take time to accrue Citation comparisons are only meaningful if comparing like with like, i.e. researchers in the same field of research and at similar career stages Raw count of citations & analyses depend on coverage of the data sources/citation databases 38
Citation Data Sources/Databases Web of Science Scopus Google Scholar > 12,000 Journals > 148,000 Proceedings ~ 19,500 Journals ~ 250 Proceedings From 1900 - From 1996 - Not stated Worldwide N. American bias English Language bias (Chinese Science Citation DB) Worldwide 50% from Europe, Latin America and Asia-Pacific English Language bias No. not stated - Journals, Papers, Theses, Acad. Books, Conf. Proceedings, Abstracts & Tech. Reports Not stated Multiple Languages Multidisciplinary Multidisciplinary Multidisciplinary Cited Reference Searching Guide: http://libguides.nus.edu.sg/citedref Publish or Perish 39
Issues & limitations of citation metrics Different disciplines have different citation behaviours Science: Dominated by international, peer reviewed journals published in English; sometimes involve multiple co-authors Social Sciences & humanities: Publish more in books & conference papers which are less likely to have citation counts; usually single author No. of references/paper published varies with field e.g. Mathematics low, Biomedical - high 40
A scientist has index h if h of [his/her] Np papers have at least h citations each, and the other (Np - h) papers have at most h citations each h-index Hirsch, J.E. An index to quantify an individual's scientific research output, PNAS, vol. 102 no. 46 (2005) 41
Case 1: Consider this Researcher A publishes exactly 10 papers of 10 citations each. Researcher B publishes 10 papers with 10 citations and an additional 90 papers with 9 citations each. Case 2: Researcher C publishes exactly 10 papers of 10 citations each. Researcher B publishes exactly 10 papers of 100 citations each. Adler, R., Ewing, J. & Taylor, P. (2008). Citation Statistics: A report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS). 42
From Hirsch himself. Obviously a single number can never give more than a rough approximation to an individual s multifaceted profile, and many other factors should be considered in combination in evaluating an individual. This and the fact that there can always be exceptions to rules should be kept in mind especially in life-changing decision such as the granting or denying of tenure. Jorge Hirsch (Creator of the h-index) Hirsch, J.E. An index to quantify an individual's scientific research output, PNAS, vol. 102 no. 46 (2005) 43
Putting Citation Counts in Perspective http://www.jbc.org/content/280/28/e25.full as at 06 Mar 2014 44
Putting Citation Counts in Perspective it is flattering to be the most cited author but I am afraid it does not signify great scientific accomplishment Although method development is usually a pretty pedestrian affair, others doing more creative work have to use methods and feel constrained to give credit for the same Oliver Lowry (Devised the Lowry Protein Assay) Lowry OH, Rosebrough NJ, Farr AL & Randall RJ. Protein measurement with the Folin phenol reagent. J. Biol. Chem. 193: 265-75, 1951.- Classic Paper. Citation Classic Commentary http://garfield.library.upenn.edu/classics1977/a1977dm02300001.pdf 45
Google 46
Citation based metrics Impact factor Times cited h-index Journal Paper Author/ Institution 47
Manage your publication history Be consistent in the use of your name syntax in all your publications, e.g. James Tiberius Kirk Kirk, James T. Kirk, JT Kirk, J Distinguish yourself from other authors by choosing a name syntax that is complex enough List your institutional affiliations over the course of your career correctly, including 48
Manage your publication history Name of Institution National University of Singapore Faculty Faculty of Science instead of Science Faculty, Sci Fac, etc Department Department of Physics instead of Phy Dept For Research Group Include the Faculty (if applicable) and Department 49
Manage your publication history Check that your publications are correctly listed in databases such as Web of Science & Scopus Promote your publications by participating in researcher identification registries ORCID (Open Researcher and Contributor ID), http://orcid.org/ ResearcherID, http://www.researcherid.com/ Google Scholar researcher profile 50
ORCID (Open Researcher and Contributor ID), http://orcid.org/ 51
ResearcherID 52
Google Scholar researcher profile 53
Thank you Let us have your feedback at http://tinyurl.com/l6yjehe 54