esss european summer school for scientometrics 2013 Prof. Dr. Hans-Dieter Daniel

Size: px

Start display at page:

Download "esss european summer school for scientometrics 2013 Prof. Dr. Hans-Dieter Daniel"

Hugo Lyons
5 years ago
Views:

1 Research Evaluation at the University of Zurich esss european summer school for scientometrics 2013 Prof. Dr. Hans-Dieter Daniel

2 Higher Education in Switzerland

3 University of Zurich Key Figures 2012 Teaching # Students (excl. MAS) New enrollments Degrees awarded Funding Mio. CHF Total funds External funds 257 (20% of total funds) Staff FTEs Professors 527 (whereof 55% international) Non-professorial academic staff Administrative and technical staff 2 051

League of European Research Universities «Membership in the League of European Research Universities (LERU), which is by invitation, is periodically evaluated against a broad set of quantitative and

4 League of European Research Universities «Membership in the League of European Research Universities (LERU), which is by invitation, is periodically evaluated against a broad set of quantitative and qualitative criteria, such as research volume, impact and funding, strengths in PhD training, size and disciplinary breadth, and peer-recognised academic excellence.» University of Amsterdam Universitat de Barcelona University of Cambridge University of Edinburgh University of Freiburg Université de Genève Universität Heidelberg University of Helsinki Universiteit Leiden KU Leuven Imperial College London University College London Lund University University of Milan Ludwig-Maximilians-Universität München University of Oxford Pierre & Marie Curie University Université Paris-Sud University of Strasbourg Utrecht University University of Zurich

5 LERU membership requirements «The first hurdle is that a LERU member university must have an excellent record in terms of bibliometric i data the amount of leading research that a university s staff publish in internationally regarded publications.» Prof. Jan Carlstedt-Duke, Chair of the LERU Membership Committee, Joining a League of Excellence, LERU Annual Report 2006, p. 6

6 Citation analysis of major European universities & Science and Medicine All Disciplines University CPP/FCSm Rank CPP/FCSm Rank Cambridge Oxford Oxford Cambridge Geneva Edinburgh Strasbourg TU München Edinburgh Geneva Heidelberg Zürich TU München Helsinki Leiden Heidelberg Zürich Karolinska Institutet Helsinki Leiden Karolinska Institutet Strasbourg n/a n/a CPP/FCSm: Mean citation impact of the university compared to the world citation average in the subfields in which the university is active. Source: Centre for Science and Technology Studies (CWTS), Universiteit Leiden

7 Citation analysis of major European universities & Definitions iti CPP: average number of citations per publication FCS: mean citation rate of all papers published in all journals of the field(s) in which the institute is active CPP/FCSm: Mean citation ti impact of the university it compared to the world citation average in the subfields in which the university is active («crown indicator»). Source: Center for Science and Technology Studies (CWTS), Universiteit Leiden Seite 7

8 Send all bibliometrics to the darkest black hole... the personal wish of the author remains to send all bibliometrics and its diligent servants to the darkest omnivoric black hole that is known in the entire universe, in order to liberate academia forever from this pestilence. Richard R. Ernst Ernst, Richard R. (2010). The follies of citation indices and academic ranking lists a brief, ( ) g commentary to Bibliometrics as Weapons of Mass Citation. CHIMIA,64(1/2), p. 90.

Reviewers' Report Follow-up Follow-up Agreement Surveys Students t Alumni Academic Staff Personnel Customers Comprehensive Evaluation Report

9 Evaluation Process at UZH green blue brown = Activity of the unit under evaluation = Organized by the Evaluation Office = Organized by the University Executive Board Self-evaluation Documentation Statements Statements Board, Executive Board, Dean Reviewers (Peers) Site Visit Reviewers' Report Follow-up Follow-up Agreement Surveys Students t Alumni Academic Staff Personnel Customers Comprehensive Evaluation Report Publication Activity Bibliometry Course Evaluation Regular Courses Continuing Education (after two years) Monitoring i Monitoring Report Semester 1 Semester 2 Semester 3

10 Self-Evaluation Documentation at UZH Retrospective and profile of strengths and weaknesses (last five years) Retrospective and future perspective Structure and resources Research Teaching and study programs Management and administration Academic career development Services Brief portraits of the chairs Research o Research focus and projects o Expenditure of third-party funds for research o Research co-operations o List of the 10 most important publications of the professorship o Self-assessment of research performance in national and international comparison Brief academic CVs of senior academic staff

11 Analysis of the Publication Activity at UZH Data List of publications as provided by the unit (7 years): «Current potential analysis» Publication window (5 years for citation analyses, starting 7 years back) Citation window (citations from date of publication; i.e., between 2 and 7 years) Citation window Citations -7 years -2 years Date of analysis Publication window for citation analysis List of publications

12 Analysis of the Publication Activity at UZH Indicators Number of Citations ti (CIT) Journal-based Reference Citation Value (J-AVG) Average Journal Impact Factor (AVG-IF) Weighted Average Journal Impact Factor for all Subject Categories (IF- CATS) Journal-based Relative Citation Eminence (JRCE) Publication Strategy Index (PSI) Subject-category-based Relative Citation Eminence (SRCE) Statistical Outliers Highly Cited Papers and Highly Cited Scientists

13 ctional Data Fic Publications by Document Type and Year

14 Data Fictional D Citation ti Rates and Bibliometric i Indicators

15 Data Fictional D Citation ti Rates and Bibliometric i Indicators

16 INSPIRE The Case of High Energy Physics

17 INSPIRE applied / 1 The Case of High Energy Physics: Comparison of Professorships at UZH Data Fictional D

18 Data Fictional D INSPIRE applied / 2 The Case of High Energy Physics: Comparison of Professorships at UZH

19 Lifetime publication activity MathSciNet and WoS: The Case of Mathematics: Comparison of Professorships Data Fictional D

20 Data Fictional D MathSciNet and WoS Indicators The Case of Mathematics Journal Metrics

21 Data Fictional D Journal Analysis with ERIH: The Case of Theology

22 Data Fictional D Citation Analysis with Google Scholar (by Year) The Case of Computer Science

23 Data Fictional D Citation Analysis with Google Scholar (by Document Type) The Case of Computer Science

24 Reference standards based on the sections of Chemical Abstracts The Case of Biochemistry Seite 24

25 Correlation between number of publications and the average citation The Case of Biochemistry For the 20 biochemistry sections in Chemical Abstracts r =.57, p < Seite 25

26 Comparative citation analysis of three institutes for chemistry at UZH Note. Field-normalized citation counts, applying reference standards based on the subject categories of the Science Citation Index ( ) and the sections of Chemical Abstracts ( ). The points represent the mean estimated by negative binomial regression, and the vertical lines show the corresponding 95% confidence interval. The horizontal lines show the ranges of values for an impact about ( ), above ( ), and far above (> 1.5) the international average of the fields. A B Institute C Seite 26 26

27 Leiden Ranking «A short history of SHELX» / 1

28 Leiden Ranking: «A short history of SHELX» / 2

29 Leiden Ranking «A short history of SHELX» / 3

30 Leiden Ranking «A short history of SHELX» / 4

31 Citation Analysis Examples of Outliers at UZH

32 Citation Analysis Indicators With Outliers at UZH

33 Citation Analysis Indicators Without Outliers at UZH

34 When comparing research groups / institutions: Always check the distribution of citation counts for individual papers! Seite 34

Using Bibliometric Analyses for Evaluating Leading Journals and Top Researchers in SoTL

Georgia Southern University Digital Commons@Georgia Southern SoTL Commons Conference SoTL Commons Conference Mar 26th, 2:00 PM - 2:45 PM Using Bibliometric Analyses for Evaluating Leading Journals and