Horizon 2020 Policy Support Facility

Transcription

1 Horizon 2020 Policy Support Facility Bibliometrics in PRFS Topics in the Challenge Paper Mutual Learning Exercise on Performance Based Funding Systems Third Meeting in Rome 13 March 2017 Gunnar Sivertsen

2 Agenda for this session 1. Bibliometric indicators in PRFS 2. Bibliometric data sources 3. Models for the use of bibliometrics 4. The importance of field normalisations 5. Discussion Page 2

3 Agenda for this session 1. Bibliometric indicators in PRFS 2. Bibliometric data sources 3. Models for the use of bibliometrics 4. The importance of field normalisations 5. Discussion Page 3

4 Bibliometrics a field of research n International journals n International conferences Page 4

5 Bibliometrics a commercial field Page 5

6 Most used bibliometric indicators 1. Impact indicators a. Total number of citations b. Number of citations compared to average in the field c. Proportion of publications among the most cited in the world Page 6

7 Page 7

8 Most used bibliometric indicators 1. Impact indicators a. Total number of citations b. Number of citations compared to average in the field c. Proportion of publications among the most cited in the world n n n n Always relate to: The year of the publication (time to get cited) The type of publication (e.g. review articles are more cited than original research articles) The field of research (field normalization to be discussed later on) Page 8

9

10

11 Most used bibliometric indicators 1. Impact indicators a. Total number of citations b. Number of citations compared to average in the field c. Proportion of publications among the most cited in the world 2. Productivity indicators 1. Total number of publications 2. Total number of publications compared to input variables (resources for research) Page 11

12 Page 12

13 Other often used bibliometric indicators 1. Research profile: Number of publications per field compared to the general distribution in the database or in other units of assessment 2. Share of publications with co-authors in certain relations 3. Frequencies of co-authored publications in certain relations Page 13

14 Page 14

15 Most used bibliometric indicators 1. Impact indicators a. Total number of citations b. Number of citations compared to average in the field c. Proportion of publications among the most cited in the world 2. Productivity indicators 1. Total number of publications 2. Total number of publications comparted to input variables (resources for research) n n n n Always relate to: Total number of authors/institutions per publication Share of authors from evaluated unit The field of research (field normalization to be discussed later on) Page 15

16 Often used, but not recommended 1. Journal Impact Factor, when used to evaluate individual researchers and their output 2. H-index, when used for the same purpose Page 16

17 Distribution of 4,845 citations to 405 articles in Science, Technology and Human Values articles (10 per cent) have received half of the citations. The JIF measures the average for all articles Page 17

18 Page 18

19 Distribution of 8,215 citations to 162 publications by one author (Per O. Seglen) H-index = Page 19

20 Agenda for this session 1. Bibliometric indicators in PRFS 2. Bibliometric data sources 3. Models for the use of bibliometrics 4. The importance of field normalisations 5. Discussion Page 20

21 Page 21

22 Page 22

23 Google Scholar is of great help to the scientist as reader but not to the bibliometrician: No control over data and methods Page 23

24 Harzing s Publish or Perish Page 24

25 Page 25

26 Local or national Current Research Information Systems (CRIS) may give Page 26

27 Page 27

28 Page 28

29 Page 29

30 Page 30

31 Page 31

32 Page 32

33 Agenda for this session 1. Bibliometric indicators in PRFS 2. Bibliometric data sources 3. Models for the use of bibliometrics 4. The importance of field normalisations 5. Discussion Page 33

34 Models for the use of bibliometrics in PRFS A. Used only to inform peer review by expert panels B. Used directly in the funding formula C. A+B (mixed) D. Not categorized: Austria, Cyprus, Moldova, Spain A B A+B (mixed) Scopus/WoS Portugal, Turkey Sweden Estonia CRIS (+Scopus/Wos) Croatia, Czech Republic, Norway Armenia, Italy Slovenia Page 34

35 Page 35

36 Page 36

37 Additional metrics often used as information for panels or as input to the formula in PRFS n n n n n External competitive funding External revenues Prizes Doctoral degrees Educational statistics Page 37

38 Agenda for this session 1. Bibliometric indicators in PRFS 2. Bibliometric data sources 3. Models for the use of bibliometrics 4. The importance of field normalisations 5. Discussion Page 38

39 Page 39

40 Universities have different research profiles Page 40

41 Large variations in average number of citations per field CELL BIOLOGY NUCLEAR PHYSICS IMMUNOLOGY BEHAVIORAL SCIENCES GEOLOGY PSYCHOLOGY ECONOMICS POLITICAL SCIENCE MATHEMATICS LINGUISTICS PHILOSOPHY HISTORY Page 41

42 Field differences in the use of publication types Page 42

43 Variations in the average number of authors per publication Page 43

44 Variations in the average number of authors per publication Page 44

45 Fractional counting Page 45

46 Fractional counting with balance Page 46

47 Agenda for this session 1. Bibliometric indicators in PRFS 2. Bibliometric data sources 3. Models for the use of bibliometrics 4. The importance of field normalisations 5. Discussion Page 47

48 For Breakout 1: Most used bibliometric indicators 1. Impact indicators a. Total number of citations b. Number of citations compared to average in the field c. Proportion of publications among the most cited in the world 2. Productivity indicators 1. Total number of publications 2. Total number of publications compared to input variables (resources for research) Page 48

49 For breakout 2: Additional metrics often used as information for panels or as input to the formula in PRFS n n n n n External competitive funding External revenues Prizes Doctoral degrees Educational statistics Page 49

50 Page 50