The Eect on Citation Inequality of Dierences in Citation Practices across Scientic Fields

Size: px

Start display at page:

Download "The Eect on Citation Inequality of Dierences in Citation Practices across Scientic Fields"

Julius Cannon
5 years ago
Views:

1 The Eect on Citation Inequality of Dierences in Citation Practices across Scientic Fields Juan A. Crespo 1, Yunrong Li 2, Javier Ruiz-Castillo 2 2 Universidad Carlos III de Madrid, Spain October 10, 2013 Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

2 Outline 1 Motivation 2 Model 3 Empirical with Raw Data 4 Normalization of Raw Data Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

3 Motivation How to assess the scientic inuence of a research paper? Citation impact: the number of citations received by the paper within a certain period of time after its publication. A Scientic Field: a collection of papers published in a set of closely related professional journals. Empirical Regularity: highly skewed citation distributions (Albarrán,2011), citation inequality is very large within a eld as well as in all-elds case. Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

4 Skewness of Citation Distribution Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

5 Motivation The large citation inequality may be due to dierent papers have dierent scientic inuence, or papers belong to dierent elds. The eld dependence of citation impacts: Size of the eld: average number of papers per author in a given period of time. Average number of references per paper. The speed at which the citation process evolves. To introduce a simple model to see how important is the eld dependence of citation impacts on citation inequality. Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

6 Motivation The large citation inequality may be due to dierent papers have dierent scientic inuence, or papers belong to dierent elds. The eld dependence of citation impacts: Size of the eld: average number of papers per author in a given period of time. Average number of references per paper. The speed at which the citation process evolves. To introduce a simple model to see how important is the eld dependence of citation impacts on citation inequality. Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

7 Motivation The large citation inequality may be due to dierent papers have dierent scientic inuence, or papers belong to dierent elds. The eld dependence of citation impacts: Size of the eld: average number of papers per author in a given period of time. Average number of references per paper. The speed at which the citation process evolves. To introduce a simple model to see how important is the eld dependence of citation impacts on citation inequality. Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

8 Statistics of Field Citation Distribution Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

9 Roemer s Income Inequality Model Roemer s (1998) model: Income of individuals depends on eorts and circumstances (e.g. parents wealth, education. ), partition the population by type, Income ti = (type t, effort ti ). Eort distribution within a type is a characteristic of the type. A1: Within a type, individuals at the same quantile of eort distribution implement the same degree of eort. A2: Within a type, income is monotonic in eort. Quantiles of eort distribution correspond to quantiles of income distribution. Holding constant the degree of eort/income, income inequality across types is due to circumstances, inequality of opportunity. Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

10 Analogy of Our Model with Roemer s Individuals= Articles Income= Citation impact A income distribution= A citation distribution Circumstances/Types= Fields Eort= Scientic inuence Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

11 Assumptions Citation fi = (field f, Scientific Influence fi ), f = 1,...,F ; i = 1,...,N. A1: Within a eld, articles at the same quantile of the scientic inuence distribution reect the same degree of scientic inuence. A2: Citation fi is monotonic in scientific influence fi. Quantiles of scientic inuence distribution correspond the quantiles of citation distribution. Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

12 Assumptions Citation fi = (field f, Scientific Influence fi ), f = 1,...,F ; i = 1,...,N. A1: Within a eld, articles at the same quantile of the scientic inuence distribution reect the same degree of scientic inuence. A2: Citation fi is monotonic in scientific influence fi. Quantiles of scientic inuence distribution correspond the quantiles of citation distribution. uan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

13 Assumptions Citation fi = (field f, Scientific Influence fi ), f = 1,...,F ; i = 1,...,N. A1: Within a eld, articles at the same quantile of the scientic inuence distribution reect the same degree of scientic inuence. A2: Citation fi is monotonic in scientific influence fi. Quantiles of scientic inuence distribution correspond the quantiles of citation distribution. uan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

14 Double Partition Partition the all-elds citation distribution by elds f and quantiles π. Field 1 1st 2nd quantiles. Πth. Field F 1st 2nd quantiles. Πth Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

15 Double Partition sort, citations are in asending order within each eld. _pctile, create 1000 quantiles within each eld. merge to merge all elds together. All-elds citation distribution is a matrix of cells, (f, π). Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

16 Decomposable Inequality Index Generalized Entropy (GE) family of inequality indices, the Theil index. I 1 (C) = N 1 l c l µ log c l µ, citation inequality of all-elds case, µ is the mean citation of all articles. For articles with 0 ciation, we follow the convention 0 log0 = 0. I 1 (C) is decomposable, I 1 (C) = W + S + IDCP. Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

17 Decomposable Inequality Index Generalized Entropy (GE) family of inequality indices, the Theil index. I 1 (C) = N 1 l c l µ log c l µ, citation inequality of all-elds case, µ is the mean citation of all articles. For articles with 0 ciation, we follow the convention 0 log0 = 0. I 1 (C) is decomposable, I 1 (C) = W + S + IDCP. uan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

18 Decomposable Inequality Index Generalized Entropy (GE) family of inequality indices, the Theil index. I 1 (C) = N 1 l c l µ log c l µ, citation inequality of all-elds case, µ is the mean citation of all articles. For articles with 0 ciation, we follow the convention 0 log0 = 0. I 1 (C) is decomposable, I 1 (C) = W + S + IDCP. uan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

19 Decomposable Inequality Index W = π f ν π,f I 1 (cf π ), Within-Group term. Weight:ν π,f is share of total citations received by articles in cell (f, π). For large Π, W is small. S = I 1 (µ 1,...µ π,...µ Π ), Between-Group term. Each paper is given the mean citation of articles in its own quantile. Citation inequality is due to articles belonging to dierent quantiles, i.e. skewness of the all-elds citation distribution. IDCP = π ν π I 1 (µ π 1,...µπ f,...µπ F ). Each paper is given the mean citation of articles in its own cell. Within any quantile π, citation inequality is due to the dierences in citation practices across elds. Weight: ν π is the share of total citations received by articles in quantile π. Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

20 Decomposable Inequality Index W = π f ν π,f I 1 (cf π ), Within-Group term. Weight:ν π,f is share of total citations received by articles in cell (f, π). For large Π, W is small. S = I 1 (µ 1,...µ π,...µ Π ), Between-Group term. Each paper is given the mean citation of articles in its own quantile. Citation inequality is due to articles belonging to dierent quantiles, i.e. skewness of the all-elds citation distribution. IDCP = π ν π I 1 (µ π 1,...µπ f,...µπ F ). Each paper is given the mean citation of articles in its own cell. Within any quantile π, citation inequality is due to the dierences in citation practices across elds. Weight: ν π is the share of total citations received by articles in quantile π. Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

21 Decomposable Inequality Index W = π f ν π,f I 1 (cf π ), Within-Group term. Weight:ν π,f is share of total citations received by articles in cell (f, π). For large Π, W is small. S = I 1 (µ 1,...µ π,...µ Π ), Between-Group term. Each paper is given the mean citation of articles in its own quantile. Citation inequality is due to articles belonging to dierent quantiles, i.e. skewness of the all-elds citation distribution. IDCP = π ν π I 1 (µ π 1,...µπ f,...µπ F ). Each paper is given the mean citation of articles in its own cell. Within any quantile π, citation inequality is due to the dierences in citation practices across elds. Weight: ν π is the share of total citations received by articles in quantile π. Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

22 Data Only research papers. One paper is assigned into only one eld. About 4.4 million articles pubilshed in A common ve-year citation window for every year, about 35 million citations. 22 broad elds: 20 for natural sciences and 2 for social sciences, distinguished by Thomson Scientic. Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

23 Results with Raw Data Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

24 Exchange Rates Dene exchange rate, e f (π), for articles in cell (f, π): e f (π) = µπ f µ π, how many citations for an article at quantile π of eld f are equivalent on average to one citation in all-elds case (the reference situation). If e f (π) varies dramatically across π within a eld, no common factor of all quantiles can be estimated. Empirically, e f (π) remain suciently constant over quantiles [706th, 998th] % of citations in each eld. Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

25 Exchange Rates Dene exchange rate, e f (π), for articles in cell (f, π): e f (π) = µπ f µ π, how many citations for an article at quantile π of eld f are equivalent on average to one citation in all-elds case (the reference situation). If e f (π) varies dramatically across π within a eld, no common factor of all quantiles can be estimated. Empirically, e f (π) remain suciently constant over quantiles [706th, 998th] % of citations in each eld. uan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

26 Exchange Rates Dene exchange rate, e f (π), for articles in cell (f, π): e f (π) = µπ f µ π, how many citations for an article at quantile π of eld f are equivalent on average to one citation in all-elds case (the reference situation). If e f (π) varies dramatically across π within a eld, no common factor of all quantiles can be estimated. Empirically, e f (π) remain suciently constant over quantiles [706th, 998th] % of citations in each eld. uan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

27 Normalization Factor Dene an average-based exchange rate (ER) over [706th, 998th]: e f = 1 π M π m π e f (π) Normalize raw citations: c fi = c fi e f Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

28 Results after Normalization Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

29 Thank you! Juan A. Crespo, Yunrong Li, Javier Ruiz-Castillo Dierences (UC3M) in Citation Impacts October 10, / 19

A systematic empirical comparison of different approaches for normalizing citation impact indicators

A systematic empirical comparison of different approaches for normalizing citation impact indicators Ludo Waltman and Nees Jan van Eck Paper number CWTS Working Paper Series CWTS-WP-2013-001 Publication