Running a Journal.... the right one

Transcription

1 Running a Journal... the right one

2 Overview Peer Review History What is Peer Review Peer Review Study What are your experiences New peer review models 2

3 What is the history of peer review and what role does it serve? Cornerstone of the whole scholarly publication system Maintains integrity in the advancement of science Well-established process over 300 years old 3

4 Authors Peer Review has two key functions: 1,000 new editors per year new journals per year What is Peer Review? Acts as a filter by ensuring only good research is published. Helps to determine validity, significance and originality 10 million articles available 10 million researchers 4,500+ institutions 180+ countries 386+ million downloads per year 2.5 million print Improves the quality of the research submitted for publication by giving reviewers the opportunity to suggest improvements pages 4 per year >700,000+ article submissions per year Pre-Submission 1 million Peer Review reviewer Editor Publication Reviewers 500,000 reviewers reports per year 40%-90% of articles rejected 7,000 editors 70,000 editorial board members 6.5 million author/publisher communications per Ultimately, however, the satisfaction of each of us as authors depends year upon our collective behavior as referees. So we say it again: Thank you, thank you, thank you, dear referees. Charles B. Duke, Editor Surface Science, January 26, ,000 new articles produced per year 180 years of back issues scanned, processed and data-tagged

5 Different Types of Peer Review 1. Single blind peer review 2. Double blind peer review 3. Open peer review Experimental 1. Post-publication peer review POST-PUBLICATION Comments: 2. Dynamic peer review 1. And the reviewer 5 star rating is 5 PRE-PRINT star rating Etc.

6 Peer Review study The Peer Review Survey was conducted by Elsevier in partnership with Sense About Science in August A global study with 4,037 responses. (Margin of error ± 1.5% at 95% confidence levels Too few and overworked reviewers Climategate General erosion of peer review Overall satisfaction with Peer Review. holding back innovative research.there is is bias against developing countries 8% 1% 8%.research articles are not improved not good at stopping plagiarism or fraud 22% 69% satisfied (64% in 2007) Exploits of Jan Hendrik Schön The overly ambitious Woo Suk Hwang 61% 6 Very Satisfied Neither satisfied nor dissatisfied Very Dissatisfied Satisfied Dissatisfied

7 Journal Metrics and Ranking What Impact? Which Ranking?

8 Metrics in our discussion today How is it calculated? What is it influenced by? How is it used (and abused)? How are they calculated? What do they add? 11

9 Number of Active, Peer-Reviewed Journals Growth of scholarly journals ~3% per annum Source: < s 1910s 1920s 1930s 1940s 1950s 1960s 1970s 1980s 1990s >2000 Decade This is truly the decade of the journal and one should seek to limit their number rather than to increase them, since there can be too many periodicals

10 Use of the Impact Factor 1) For journal editors and publishing houses the impact factors give market research information. Editors are able to approximate their journals standing in comparison to other thematically closely related periodicals. 2) For libraries impact factors are useful for collection development, especially in combination with further indicators, primarily journal prices. Combined indicators, for example, Euros per unit of impact factor or a combination of this indicator with cost-per-use, may be effective for serial selection.[15] 3) Authors get hints on journals in which they can publish their research results (in case there are any authors in academia who do not know their appropriate periodicals). In no case is it possible to use a journal impact factor on the article level to evaluate the influence of an article, an author or an institution. 13

11 The Impact Factor calculation & anomaly A ratio between citations and recent citable items published in a journal; the average number of citations received per published article. IF 200X = citations in 200X to all items published in 200X-1+200X-2 citable items published in 200X-1+200X-2 Citations to non-source items (editorials, letters, news items, book reviews, abstracts, etc) may inflate the IF 14

12 Citations Influences on the IF: Article Types Impact Factor Reviews Notes Articles Years after publication

13 Influences on the IF: Subject Category Aggregate 2009 IF 16

14 Impact Factor doubts BMJ 1997;314(7079):498 (15 February) Cumulative contribution of articles with different citation rates to total journal impact. 17

15 Impact Factor doubts October 14, 2005 June 5, August 2009 JASIST 61 (2), pp Journal of Documentation Year: 2008 Volume: 64 Issue: 2 18

16 New metrics : Prestige Eigenfactor Free at eigenfactor.org; also on JCR 5 years Citations weighting Self-citations excluded SCImago Journal Rank Free (scimagojr.com or info.scopus.com/journalmetrics) 3 years Citations weighting 19

17 New metrics : Popularity ource ormalized mpact per aper Free (info.scopus.com/journalmetrics) Contextual to: field citation potential field immediacy database coverage Field defined by citation network Peer-reviewed papers only 20

18 h-index Fig. 1. Schematic curve of number of citations versus paper number, with papers numbered in order of decreasing citations. The intersection of the 45 line with the curve gives h. The total number of citations is the area under the curve. Copyright 2005 by the National Academy of Sciences 21

19 Usage Factor? Nascent metrics calculation not yet fixed May be similar to Impact Factor 22

20 Why do we have editorial boards? Just a set of distinguished names? Nice but how relevant for an established journal? Handling of peer review and decision making in many cases. Active, senior peer reviewers. Resolvers of disputes. Advocates for the journal.

21 Publishing ethics & plagiarism Publishing ethics policies somewhat ad-hoc before 2000 Recognizing need to raise bar, we introduced new common approaches Conflicts of interest (2005) Ethical Guidelines (2006) Codes included in EES (2006) Helpdesk experiment Launched PERK site 2009 Full membership COPE 2009

22 Publishing ethics & plagiarism 2 Cross Check initiative (2009) Huge database: 26.6 million articles from 49,000 journals from 124 publishers Ithenticate software shows similarities between the article and previously published articles 400 Editors piloted in 2009, now widely available

23 Publishing ethics & plagiarism 3 On the front lines: Complaints may come in through variety of methods Editors will need to assess Publishers (& lawyers) will provide support & tools (PERK/ COPE/ Cross Check) But judgment/context is indispensible

24 Publishing ethics & plagiarism 4 PERK guidelines process: Gather relevant information Consider due process for authors Involve other bodies or agents, if necessary Consider appropriate remedies & sanctions

25 Publishing ethics & plagiarism 5 Table shows statistics for midyear 2010 from our S&T journals (145 cases ) Formal retractions/removals (S&T & HS): 63/2 Article withdrawals

26 Alternative calculation of the IF 40