Google Scholar and ISI WoS Author metrics within Earth Sciences subjects Susanne Mikki Bergen University Library
My first steps within bibliometry Research question How well is Google Scholar performing compared to Thomson Reuters ISI WoS? Limitation Limit to author searches within Earth Sciences subjects
Research objective Analyse search results of the two services by overlap coverage ranking Similarities in ranking are carried out in terms of Spearman s footrule Mean citation count h-index
Data material 29 (26) authors, avoiding common names and names containing special characters Most of the authors related to UoB Mainly working with climate and petrology issues In Google Scholar Author search, which includes citation searching Result list includes items marked [BOOK], [CITATION] and [PDF] (representing about 55% of all results) Citation count obtained by Cited by In ISI WoS Author search, not Cited Reference Search Citation count obtained by Times cited (not including conference proceedings)
Data treatment Export titles from ISI WoS and GS (cut and paste) Matlab for further data treatment and analysis Filter data Remove duplicates (7,7% in GS) For identification of identical records - Compare titles Remove all punctuations Compare max first 50 characters
Unique items and overlap of search results
Comparing search results - Methods and indicators Overlap O = P( ISI) P( ISI) P( GS) P( GS) Coverage - defined as the recall of ISI records in Google Scholar Similarity of rank defined by the Normalized Spearmans s footrule F = 1 P( ISI) P( GS) C = P( ISI) Z i = 1 σ GS i F max σ ISI i h-index A scientist has index h if h of his or her papers have at least h citations each. The h-index is introduced as a measure which ignores the impact of inflated citation counts of single documents, and a long tail of rarely cited documents h N C ( h) while N ( h + 1) < h + 1 C
Citation count versus rank of publication Skewed distribution or Mathew effect in science Number of publications 52 (ISI), 117(GS) Number of citations 775(ISI), 835 (GS) Mean citation count ca 15 (ISI), 7(GS) h-index 16(ISI), 13 (GS)
Citation count for overlapping items Higher citation counts in ISI WoS
ISI WoS and Google Scholar compared autho r ISI sum publ N P(ISI ) GS sum publ N P(GS ) ISI citation count N C(ISI ) GS citation count N C(GS ) ISI h- index h ISI GS h- index h GS relative h-index h h ISI GS ISI mean citation count CC ISI GS mean citation count CC GS relative citation count N N C( ISI ) C( GS ) footrule coverage py>2000 cover age F C py> 2000 C (Σ 29 ) (1573) (5048) (43028) (40908) (16.7) (16.0) (1.04) (20.3) (6.5) (0.93) (0.80) (0.75) (0.84) Σ 26 1264 4196 27189 29053 15.3 15.0 1.03 18.3 5.9 0.90 0.81 0.77 0.86 1 9 20 43 107 4 3 1.33 4.8 5.4 0.40 0.63 0.50 0.89 2 67 209 350 668 12 13 0.92 5.2 3.2 0.52 0.74 0.94 0.88 3 61 154 1479 1204 23 22 1.05 24.2 7.8 1.23 0.84 0.80 0.86 4 65 265 541 1136 13 14 0.93 8.3 4.3 0.48 0.80 0.73 0.87 5 52 117 775 835 16 13 1.23 14.9 7.1 0.93 0.81 0.75 0.85 For all authors 6 20 35 421 292 9 8 1.13 21.1 8.3 1.44 0.75 0.40 0.40 7 99 196 1125 639 16 10 1.60 11.4 3.3 1.76 0.78 0.77 0.49 Coverage 0,86 8 20 79 161 134 6 6 1.00 8.1 1.7 1.20 0.86 0.70 0.88 9 26 112 142 296 7 6 1.17 5.5 2.6 0.48 0.67 1.00 1.00 Footrule 0,81 10 55 127 941 918 17 17 1.00 17.1 7.2 1.03 0.85 0.63 0.83 h-indeks 15,3(ISI); 15,0(GS) 11 42 184 445 884 13 15 0.87 10.6 4.8 0.50 0.79 0.87 0.93 12 60 173 1971 1277 25 20 1.25 32.9 7.4 1.54 0.87 0.86 0.93
Summery The amount of earth science content is comprehensive in Google Scholar. It covers about 85% of content indexed by ISI WoS. For impact studies the h-index has proofed to be a robust measure leading to similar values for the two sources. The ranking of the two services similar. However, for overlapping items, ISI WoS accumulates significantly more citing articles, by which it confirms its position as the leading citation index. The number of search results and their citations is otherwise higher in Google Scholar. The service returns highly cited sources not indexed by ISI WoS, but also a long tail of minor relevant items, barely matching the search expression. Even if the citation counts in this study are comparable, the citing documents in Google Scholar and ISI WoS will differ. Citing records were not verified and examined in this study.
Future work Rerun the calculations Automate Google Scholar searches Combine GS citation counts with the institutional data Improve the duplicate control
Thank you for your attention susanne.mikki@ub.uib.no Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences, 102(46), 16569-16572. Mikki, S. (2009). Google Scholar compared to Web of Science. A literature review. Journal, 1(1), 41-51. Retrieved from https://noril.uib.no/index.php/noril/article/view/10/6 Mikki, S. (2010). Comparing Google Scholar and ISI WoS for Earth Sciences. Scientometrics, 82, 321-331, doi: 10.1007/s11192-009-0038-6.