Horizon 2020 Policy Support Facility Bibliometrics in PRFS Topics in the Challenge Paper Mutual Learning Exercise on Performance Based Funding Systems Third Meeting in Rome 13 March 2017 Gunnar Sivertsen
Agenda for this session 1. Bibliometric indicators in PRFS 2. Bibliometric data sources 3. Models for the use of bibliometrics 4. The importance of field normalisations 5. Discussion Page 2
Agenda for this session 1. Bibliometric indicators in PRFS 2. Bibliometric data sources 3. Models for the use of bibliometrics 4. The importance of field normalisations 5. Discussion Page 3
Bibliometrics a field of research n International journals n International conferences Page 4
Bibliometrics a commercial field Page 5
Most used bibliometric indicators 1. Impact indicators a. Total number of citations b. Number of citations compared to average in the field c. Proportion of publications among the most cited in the world Page 6
Page 7
Most used bibliometric indicators 1. Impact indicators a. Total number of citations b. Number of citations compared to average in the field c. Proportion of publications among the most cited in the world n n n n Always relate to: The year of the publication (time to get cited) The type of publication (e.g. review articles are more cited than original research articles) The field of research (field normalization to be discussed later on) Page 8
Most used bibliometric indicators 1. Impact indicators a. Total number of citations b. Number of citations compared to average in the field c. Proportion of publications among the most cited in the world 2. Productivity indicators 1. Total number of publications 2. Total number of publications compared to input variables (resources for research) Page 11
Page 12
Other often used bibliometric indicators 1. Research profile: Number of publications per field compared to the general distribution in the database or in other units of assessment 2. Share of publications with co-authors in certain relations 3. Frequencies of co-authored publications in certain relations Page 13
Page 14
Most used bibliometric indicators 1. Impact indicators a. Total number of citations b. Number of citations compared to average in the field c. Proportion of publications among the most cited in the world 2. Productivity indicators 1. Total number of publications 2. Total number of publications comparted to input variables (resources for research) n n n n Always relate to: Total number of authors/institutions per publication Share of authors from evaluated unit The field of research (field normalization to be discussed later on) Page 15
Often used, but not recommended 1. Journal Impact Factor, when used to evaluate individual researchers and their output 2. H-index, when used for the same purpose Page 16
Distribution of 4,845 citations to 405 articles in Science, Technology and Human Values 300 250 200 150 100 40 articles (10 per cent) have received half of the citations. The JIF measures the average for all articles. 50 0 Page 17
Page 18
Distribution of 8,215 citations to 162 publications by one author (Per O. Seglen) 500 450 400 350 300 250 H-index = 49 200 150 100 50 0 Page 19
Agenda for this session 1. Bibliometric indicators in PRFS 2. Bibliometric data sources 3. Models for the use of bibliometrics 4. The importance of field normalisations 5. Discussion Page 20
Page 21
Page 22
Google Scholar is of great help to the scientist as reader but not to the bibliometrician: No control over data and methods Page 23
Harzing s Publish or Perish Page 24
Page 25
Local or national Current Research Information Systems (CRIS) may give Page 26
Page 27
Page 28
Page 29
Page 30
Page 31
Page 32
Agenda for this session 1. Bibliometric indicators in PRFS 2. Bibliometric data sources 3. Models for the use of bibliometrics 4. The importance of field normalisations 5. Discussion Page 33
Models for the use of bibliometrics in PRFS A. Used only to inform peer review by expert panels B. Used directly in the funding formula C. A+B (mixed) D. Not categorized: Austria, Cyprus, Moldova, Spain A B A+B (mixed) Scopus/WoS Portugal, Turkey Sweden Estonia CRIS (+Scopus/Wos) Croatia, Czech Republic, Norway Armenia, Italy Slovenia Page 34
Page 35
Page 36
Additional metrics often used as information for panels or as input to the formula in PRFS n n n n n External competitive funding External revenues Prizes Doctoral degrees Educational statistics Page 37
Agenda for this session 1. Bibliometric indicators in PRFS 2. Bibliometric data sources 3. Models for the use of bibliometrics 4. The importance of field normalisations 5. Discussion Page 38
Page 39
Universities have different research profiles Page 40
Large variations in average number of citations per field CELL BIOLOGY NUCLEAR PHYSICS IMMUNOLOGY BEHAVIORAL SCIENCES GEOLOGY PSYCHOLOGY ECONOMICS POLITICAL SCIENCE MATHEMATICS LINGUISTICS PHILOSOPHY HISTORY 0 10 20 30 Page 41
Field differences in the use of publication types Page 42
Variations in the average number of authors per publication Page 43
Variations in the average number of authors per publication Page 44
Fractional counting Page 45
Fractional counting with balance Page 46
Agenda for this session 1. Bibliometric indicators in PRFS 2. Bibliometric data sources 3. Models for the use of bibliometrics 4. The importance of field normalisations 5. Discussion Page 47
For Breakout 1: Most used bibliometric indicators 1. Impact indicators a. Total number of citations b. Number of citations compared to average in the field c. Proportion of publications among the most cited in the world 2. Productivity indicators 1. Total number of publications 2. Total number of publications compared to input variables (resources for research) Page 48
For breakout 2: Additional metrics often used as information for panels or as input to the formula in PRFS n n n n n External competitive funding External revenues Prizes Doctoral degrees Educational statistics Page 49
Page 50