Running a Journal... the right one
Overview Peer Review History What is Peer Review Peer Review Study What are your experiences New peer review models 2
What is the history of peer review and what role does it serve? Cornerstone of the whole scholarly publication system Maintains integrity in the advancement of science Well-established process over 300 years old 3
Authors Peer Review has two key functions: 1,000 new editors per year 15-20 new journals per year What is Peer Review? Acts as a filter by ensuring only good research is published. Helps to determine validity, significance and originality 10 million articles available 10 million researchers 4,500+ institutions 180+ countries 386+ million downloads per year 2.5 million print Improves the quality of the research submitted for publication by giving reviewers the opportunity to suggest improvements pages 4 per year >700,000+ article submissions per year Pre-Submission 1 million Peer Review reviewer Editor Publication Reviewers 500,000 reviewers reports per year 40%-90% of articles rejected 7,000 editors 70,000 editorial board members 6.5 million author/publisher communications per Ultimately, however, the satisfaction of each of us as authors depends year upon our collective behavior as referees. So we say it again: Thank you, thank you, thank you, dear referees. Charles B. Duke, Editor Surface Science, January 26, 1996 290,000 new articles produced per year 180 years of back issues scanned, processed and data-tagged
Different Types of Peer Review 1. Single blind peer review 2. Double blind peer review 3. Open peer review Experimental 1. Post-publication peer review POST-PUBLICATION Comments: 2. Dynamic peer review 1. And the reviewer 5 star rating is 5 PRE-PRINT 2. 3.5 star rating Etc. www.naboj.com
Peer Review study The Peer Review Survey was conducted by Elsevier in partnership with Sense About Science in August 2009. A global study with 4,037 responses. (Margin of error ± 1.5% at 95% confidence levels Too few and overworked reviewers Climategate General erosion of peer review Overall satisfaction with Peer Review. holding back innovative research.there is is bias against developing countries 8% 1% 8%.research articles are not improved not good at stopping plagiarism or fraud 22% 69% satisfied (64% in 2007) Exploits of Jan Hendrik Schön The overly ambitious Woo Suk Hwang 61% 6 Very Satisfied Neither satisfied nor dissatisfied Very Dissatisfied Satisfied Dissatisfied
Journal Metrics and Ranking What Impact? Which Ranking?
Metrics in our discussion today How is it calculated? What is it influenced by? How is it used (and abused)? How are they calculated? What do they add? 11
Number of Active, Peer-Reviewed Journals 25000 20000 Growth of scholarly journals ~3% per annum 15000 10000 5000 0 Source: <1900 1900s 1910s 1920s 1930s 1940s 1950s 1960s 1970s 1980s 1990s >2000 Decade This is truly the decade of the journal and one should seek to limit their number rather than to increase them, since there can be too many periodicals. 1789 12
Use of the Impact Factor 1) For journal editors and publishing houses the impact factors give market research information. Editors are able to approximate their journals standing in comparison to other thematically closely related periodicals. 2) For libraries impact factors are useful for collection development, especially in combination with further indicators, primarily journal prices. Combined indicators, for example, Euros per unit of impact factor or a combination of this indicator with cost-per-use, may be effective for serial selection.[15] 3) Authors get hints on journals in which they can publish their research results (in case there are any authors in academia who do not know their appropriate periodicals). In no case is it possible to use a journal impact factor on the article level to evaluate the influence of an article, an author or an institution. 13
The Impact Factor calculation & anomaly A ratio between citations and recent citable items published in a journal; the average number of citations received per published article. IF 200X = citations in 200X to all items published in 200X-1+200X-2 citable items published in 200X-1+200X-2 Citations to non-source items (editorials, letters, news items, book reviews, abstracts, etc) may inflate the IF 14
Citations Influences on the IF: Article Types Impact Factor Reviews Notes Articles 0 2 4 6 8 10 12 14 16 18 20 Years after publication
Influences on the IF: Subject Category Aggregate 2009 IF 16
Impact Factor doubts BMJ 1997;314(7079):498 (15 February) Cumulative contribution of articles with different citation rates to total journal impact. 17
Impact Factor doubts October 14, 2005 June 5, 2006 21 August 2009 JASIST 61 (2), pp. 424-427 http://arxiv.org/abs/0908.3177 Journal of Documentation Year: 2008 Volume: 64 Issue: 2 18
New metrics : Prestige Eigenfactor Free at eigenfactor.org; also on JCR 5 years Citations weighting Self-citations excluded SCImago Journal Rank Free (scimagojr.com or info.scopus.com/journalmetrics) 3 years Citations weighting 19
New metrics : Popularity ource ormalized mpact per aper Free (info.scopus.com/journalmetrics) Contextual to: field citation potential field immediacy database coverage Field defined by citation network Peer-reviewed papers only 20
h-index Fig. 1. Schematic curve of number of citations versus paper number, with papers numbered in order of decreasing citations. The intersection of the 45 line with the curve gives h. The total number of citations is the area under the curve. Copyright 2005 by the National Academy of Sciences 21
Usage Factor? Nascent metrics calculation not yet fixed May be similar to Impact Factor 22
Why do we have editorial boards? Just a set of distinguished names? Nice but how relevant for an established journal? Handling of peer review and decision making in many cases. Active, senior peer reviewers. Resolvers of disputes. Advocates for the journal.
Publishing ethics & plagiarism Publishing ethics policies somewhat ad-hoc before 2000 Recognizing need to raise bar, we introduced new common approaches Conflicts of interest (2005) Ethical Guidelines (2006) Codes included in EES (2006) Helpdesk experiment 2007-2008 Launched PERK site 2009 Full membership COPE 2009
Publishing ethics & plagiarism 2 Cross Check initiative (2009) Huge database: 26.6 million articles from 49,000 journals from 124 publishers Ithenticate software shows similarities between the article and previously published articles 400 Editors piloted in 2009, now widely available
Publishing ethics & plagiarism 3 On the front lines: Complaints may come in through variety of methods Editors will need to assess Publishers (& lawyers) will provide support & tools (PERK/ COPE/ Cross Check) But judgment/context is indispensible
Publishing ethics & plagiarism 4 PERK guidelines process: Gather relevant information Consider due process for authors Involve other bodies or agents, if necessary Consider appropriate remedies & sanctions
Publishing ethics & plagiarism 5 Table shows statistics for midyear 2010 from our S&T journals (145 cases ) Formal retractions/removals (S&T & HS): 63/2 Article withdrawals
Alternative calculation of the IF 40