Impact Factors: Scientific Assessment by Numbers

Transcription

1 Impact Factors: Scientific Assessment by Numbers Nico Bruining, Erasmus MC,

2 Impact Factors: Scientific Assessment by Numbers I have no disclosures

3 Scientific Evaluation Parameters Since a couple of years it is has become more-and-more important that the work by scientists is evaluated against that of their peers, of which the results are being used by policy makers to distribute funds and to select those who excels in their field to give them an opportunity to pursue a scientific career

4 Application of The Scientific Evaluation Parameters by The Policy Makers Distribution of research funds according to the scientific achievements of the sub-specialities

5 Scientific Evaluation Parameters The currently two most used parameters are: 1. To evaluate Journals: The Impact Factor (IF) Often used by scientists to decide to which journal to submit their work To evaluate Scientists (even groups): The h-factor Used by research institutes for decision making if a researcher is eligible to make the next step in a scientific career

6 The Impact Factor of Journals

7 Impact Factor Thomson Reuters Web-Of-Science (WOS)

8 Impact Factor The IF of a journal is the average number of citations received per paper published in that journal during the two preceding years. The formula is the following: 2011 impact factor = A/B A = the number of times articles published in 2009 and 2010 were cited by indexed journals during 2011 B = the total of citable items published by that journal in 2009 and Citable items are usually original articles, expert reviews, proceedings or notes, sometimes editorials and/or letters-to-the-editor are noncitable items.

9 Impact Factor in Cardiology Alphabetical Order (July 2012) Thomson Reuters Web-Of-Science (WOS)

10 Impact Factor in Cardiology Ranking on Impact Factor (July 2012) Thomson Reuters Web-Of-Science (WOS)

11 Impact Factor Trends Thomson Reuters Web-Of-Science (WOS)

12 Impact Factor The top 3 journals in general medicine are (numbers of 2012): The New England Journal of Medicine: 53.3 Lancet: 38.8 JAMA: 30 In Cardiology the top 3 ranking is: Circulation: 14.7 JACC: 14.2 European Heart Journal: 10.5 Of note is that half of the indexed journals into the Web-of-Science database have an impact factor equal or lower than 1.

13 Distribution of Research Funds At the Erasmus MC you receive Scientific credits if you have publications in journals which are in the first quartile (e.g. the top 25%) of impact factors within your sub-speciality. For cardiology that is currently an IF > 3.7 (2012)

14 The h-factor of Scientists

15 The H(irsch)-Factor Is an scientific evaluation parameter used to evaluate individual scientists, a group of scientists, a department or even the complete university (proposed by Hirsch in 2005). It measures both the productivity and the impact of the published work of a scientist or scholar. The index is based on the set of the scientists most cited papers and the number of citations that they have received in other peoples publications. Hirsch, J.E. PNAS, 2005, vol. 102; no. 46:

16 The H(irsch)-Factor Application The h-factor for professors in clinical cardiology is 20 or higher EHJ; cardiopulse section

17 The H(irsch)-Factor Calculation To calculate the H-factor Hirsch writes: A scientist has index h if h of (his/her) N p papers have at least h citations each, and the other (N p h) have at most h citations each No need to calculate this yourself, you can look it up in: Web of Sciences Scopus or Google Scholar Hirsch, J.E. PNAS, 2005, vol. 102; no. 46:

18 The H(irsch)-Factor Calculation Example I Prof. Erbel Prof. van der Wall Thomson Reuters Web-Of-Science (WOS)

19 The H(irsch)-Factor Example N. Bruining Where are the results for Prof. Lüscher?? Thomson Reuters Web-Of-Science (WOS)

20 The H(irsch)-Factor Example Thomson Reuters Web-Of-Science (WOS)

21 The H(irsch)-Factor Example I have not yet figured out the correct syntax for this query, for which I am very sorry about that and my apologies Thomson Reuters Web-Of-Science (WOS)

22 How to Increase your H-Factor? The proper way Get your paper published in a journal with a high impact factor (IF), as that indicates that the chances that your work is cited are relatively higher than when you publish your work in lower IF journals.

23 How to Increase your h-factor? Anecdotal By example: Do not write books, they do not count in for the h-factor But we need books for teaching, so keep on writing! Work in a popular and wide area, breakthroughs in niche areas will not be as many times cited as those in popular areas But we desperately also need experts in lesser attractive sub-specialities Use Google Scholar Citations to calculate your own h-factor instead the Web of Sciences, because it includes more papers and uses a better search engine, resulting in more citations hits and thus a higher h-factor

24 The H(irsch)-Factor Example Web-of-Sciences Scopus Google Scholar

25 The H(irsch)-Factor Application It has been suggested that in order to deal with this variation in h for a single academic, that one should assume false negatives in the databases are more problematic than false positives and take the maximum h measured for this academic. What would be the h-factor needed within Cardiology? (to make career.) A personal professorship in clinical cardiology needs a h-factor 20* Most department chiefs do have a h-factor around 40* The top has a h-factor between (* of note that these are numbers based on the Dutch situation) Hirsch, J.E. PNAS, 2005, vol. 102; no. 46:

26 The H(irsch)-Factor Application Mentioned Problems It is obvious that the h-factor will be higher according to the length of the scientific career. This makes it difficult to compare scientists with a different work history. To correct for this Hirsch proposed the m-factor: Divide h by scientific age Scientific age is the number of years since the scientists appeared for the first time onto an author list. M can be thought of as the speed with which a researcher's h index increases. Hirsch, J.E. PNAS, vol. 102, no. 46,

27 Are these two scientific evaluation parameters perfect? Or do we also need to take into account other possible parameters or perhaps even develop new metrics to evaluate our scientific performances? As a consequence, the citation index has established itself as the most appropriate measure to assess the importance of publications. Lüscher, T.F. et al., EHJ, 2012, vol. 33:

28 Other, currently not taken into account Parameters

29 EuroIntervention Citations are telling us how scientifically important or hot a paper is. But how about appreciation of papers published in less citable areas such as by example: How-To-Treat? Bruining, N. et al., Eurointervention, 2011, vol. 7, issue 1: 143-7

30 EuroIntervention Top Cited Papers 140 Top 30 Cited papers Period Bruining, N. et al., Eurointervention, 2011, vol. 7, issue 1: 143-7

31 EuroIntervention Top Downloaded Papers Downloaded Papers From Eurointervention Website Since 2008 All Bruining, N. et al., Eurointervention, 2011, vol. 7, issue 1: 143-7

32 EuroIntervention Top Downloaded vs. Top Cited Papers Cited Downloaded Period Bruining, N. et al., Eurointervention, 2011, vol. 7, issue 1: 143-7

33 EuroIntervention Top Downloaded Papers Are downloaded papers related to original new scientific research submissions only? Sub-category Number Clinical Research 8 Focus Article 3 Expert review 3 WG 10 Column 3 Editorial 3 Experimental Research 2 How and Why 2 Supplement 2 Special Report 1 Technical Report 1 No, as we can see also practical medicine is highly appreciated as well, which is not taken into account into a citation index Bruining, N. et al., Eurointervention, 2011, vol. 7, issue 1: 143-7

34 Summary I Scientific evaluation parameters are becoming more and more important, whether we like it or not. However, they should be interpreted with caution as medicine is not only new original academic research only, but there is also a need to translate the new findings to daily clinical application. Unfortunately, this is not expressed into any parameter yet (other than what we showed here, e.g. the number of downloads, indicating the high interest into such translational /teaching paper).

35 Summary II What does Hirsch say himself about his h-factor: One single number can never be more than a rough estimate of the profile of a single individual and to judge someone correctly other parameters have to be taken into account. This together with the fact that there are always exceptions onto every rule must also be taken into mind when taking decisions to approve or disapprove a persons career in science or to evaluate a group of scientists or a complete department.