Research Evaluation at the University of Zurich esss european summer school for scientometrics 2013 Prof. Dr. Hans-Dieter Daniel
Higher Education in Switzerland
University of Zurich Key Figures 2012 Teaching # Students (excl. MAS) 25 634 New enrollments 4 433 Degrees awarded 5 213 Funding Mio. CHF Total funds 1 278 External funds 257 (20% of total funds) Staff FTEs Professors 527 (whereof 55% international) Non-professorial academic staff 3 175 Administrative and technical staff 2 051
League of European Research Universities «Membership in the League of European Research Universities (LERU), which is by invitation, is periodically evaluated against a broad set of quantitative and qualitative criteria, such as research volume, impact and funding, strengths in PhD training, size and disciplinary breadth, and peer-recognised academic excellence.» University of Amsterdam Universitat de Barcelona University of Cambridge University of Edinburgh University of Freiburg Université de Genève Universität Heidelberg University of Helsinki Universiteit Leiden KU Leuven Imperial College London University College London Lund University University of Milan Ludwig-Maximilians-Universität München University of Oxford Pierre & Marie Curie University Université Paris-Sud University of Strasbourg Utrecht University University of Zurich www.leru.org
LERU membership requirements «The first hurdle is that a LERU member university must have an excellent record in terms of bibliometric i data the amount of leading research that a university s staff publish in internationally regarded publications.» Prof. Jan Carlstedt-Duke, Chair of the LERU Membership Committee, Joining a League of Excellence, LERU Annual Report 2006, p. 6
Citation analysis of major European universities 1995-1999 1999 & 2000-2004 2004 Science and Medicine 1995-19991999 All Disciplines 2000-20042004 University CPP/FCSm Rank CPP/FCSm Rank Cambridge 160 1.60 1 Oxford 167 1.67 1 Oxford 1.57 2 Cambridge 1.64 2 Geneva 1.55 3 Edinburgh 1.44 3 Strasbourg 1.41 4 TU München 1.42 4 Edinburgh 1.38 5 Geneva 1.42 4 Heidelberg 1.36 6 Zürich 1.40 6 TU München 1.36 6 Helsinki 1.39 7 Leiden 1.28 8 Heidelberg 1.31 8 Zürich 1.26 9 Karolinska Institutet 1.24 9 Helsinki 1.25 10 Leiden 1.24 9 Karolinska Institutet 1.16 11 Strasbourg n/a n/a...... CPP/FCSm: Mean citation impact of the university compared to the world citation average in the subfields in which the university is active. Source: Centre for Science and Technology Studies (CWTS), Universiteit Leiden
Citation analysis of major European universities 1995-1999 & 2000-2004 Definitions iti CPP: average number of citations per publication FCS: mean citation rate of all papers published in all journals of the field(s) in which the institute is active CPP/FCSm: Mean citation ti impact of the university it compared to the world citation average in the subfields in which the university is active («crown indicator»). Source: Center for Science and Technology Studies (CWTS), Universiteit Leiden. 10.09.2013 Seite 7
Send all bibliometrics to the darkest black hole... the personal wish of the author remains to send all bibliometrics and its diligent servants to the darkest omnivoric black hole that is known in the entire universe, in order to liberate academia forever from this pestilence. Richard R. Ernst Ernst, Richard R. (2010). The follies of citation indices and academic ranking lists a brief, ( ) g commentary to Bibliometrics as Weapons of Mass Citation. CHIMIA,64(1/2), p. 90.
Evaluation Process at UZH green blue brown = Activity of the unit under evaluation = Organized by the Evaluation Office = Organized by the University Executive Board Self-evaluation Documentation Statements Statements Board, Executive Board, Dean Reviewers (Peers) Site Visit Reviewers' Report Follow-up Follow-up Agreement Surveys Students t Alumni Academic Staff Personnel Customers Comprehensive Evaluation Report Publication Activity Bibliometry Course Evaluation Regular Courses Continuing Education (after two years) Monitoring i Monitoring Report Semester 1 Semester 2 Semester 3
Self-Evaluation Documentation at UZH Retrospective and profile of strengths and weaknesses (last five years) Retrospective and future perspective Structure and resources Research Teaching and study programs Management and administration Academic career development Services Brief portraits of the chairs Research o Research focus and projects o Expenditure of third-party funds for research o Research co-operations o List of the 10 most important publications of the professorship o Self-assessment of research performance in national and international comparison Brief academic CVs of senior academic staff
Analysis of the Publication Activity at UZH Data List of publications as provided by the unit (7 years): «Current potential analysis» Publication window (5 years for citation analyses, starting 7 years back) Citation window (citations from date of publication; i.e., between 2 and 7 years) Citation window Citations -7 years -2 years Date of analysis Publication window for citation analysis List of publications
Analysis of the Publication Activity at UZH Indicators Number of Citations ti (CIT) Journal-based Reference Citation Value (J-AVG) Average Journal Impact Factor (AVG-IF) Weighted Average Journal Impact Factor for all Subject Categories (IF- CATS) Journal-based Relative Citation Eminence (JRCE) Publication Strategy Index (PSI) Subject-category-based Relative Citation Eminence (SRCE) Statistical Outliers Highly Cited Papers and Highly Cited Scientists
ctional Data Fic Publications by Document Type and Year
Data Fictional D Citation ti Rates and Bibliometric i Indicators
Data Fictional D Citation ti Rates and Bibliometric i Indicators
INSPIRE The Case of High Energy Physics
INSPIRE applied / 1 The Case of High Energy Physics: Comparison of Professorships at UZH Data Fictional D
Data Fictional D INSPIRE applied / 2 The Case of High Energy Physics: Comparison of Professorships at UZH
Lifetime publication activity MathSciNet and WoS: The Case of Mathematics: Comparison of Professorships Data Fictional D
Data Fictional D MathSciNet and WoS Indicators The Case of Mathematics Journal Metrics
Data Fictional D Journal Analysis with ERIH: The Case of Theology
Data Fictional D Citation Analysis with Google Scholar (by Year) The Case of Computer Science
Data Fictional D Citation Analysis with Google Scholar (by Document Type) The Case of Computer Science
Reference standards based on the sections of Chemical Abstracts The Case of Biochemistry 10.09.2013 Seite 24
Correlation between number of publications and the average citation The Case of Biochemistry For the 20 biochemistry sections in Chemical Abstracts r =.57, p <.001 10.09.2013 Seite 25
Comparative citation analysis of three institutes for chemistry at UZH Note. Field-normalized citation counts, applying reference standards based on the subject categories of the Science Citation Index ( ) and the sections of Chemical Abstracts ( ). The points represent the mean estimated by negative binomial regression, and the vertical lines show the corresponding 95% confidence interval. The horizontal lines show the ranges of values for an impact about (0.8-1.2), above (1.2-1.5), and far above (> 1.5) the international average of the fields. A B Institute C 10.09.2013 Seite 26 26
Leiden Ranking «A short history of SHELX» / 1
Leiden Ranking: «A short history of SHELX» / 2
Leiden Ranking «A short history of SHELX» / 3
Leiden Ranking «A short history of SHELX» / 4
Citation Analysis Examples of Outliers at UZH
Citation Analysis Indicators With Outliers at UZH
Citation Analysis Indicators Without Outliers at UZH
When comparing research groups / institutions: Always check the distribution of citation counts for individual papers! 10.09.2013 Seite 34