Contribution of Chinese publications in computer science: A case study on LNCS

Transcription

1 Jointly published by Akadémiai Kiadó, Budapest Scientometrics, Vol. 75, No. 3 (2008) and Springer, Dordrecht DOI: /s Contribution of Chinese publications in computer science: A case study on LNCS YING HE, a JIANCHENG GUAN b a School of Management, Beijing University of Aeronautics and Astronautics, Beijing (P. R. China) b School of Management, Fudan University, Shanghai (P. R. China) Conference proceedings are one of the key communication channels in computer science. This paper aims to analyze the Chinese outputs in the context of conference papers in computer science through an exploration of the conference proceedings series book Lecture Notes in Computer Science (LNCS) in the period of Results indicate that: 1. The number of Chinese papers in LNCS keeps growing in the studied period; the share of Chinese papers in LNCS in recent years is much higher than that of Chinese SCI papers in the world; In sharp contrast with remarkable growth of the share of Chinese papers in LNCS, the share of SCI articles in top journals of computer science published by the scientists of mainland China is negligible during the same period. 2. Chinese researchers are more likely to collaborate with domestic fellows; 3. In spite of the increasing amounts of Chinese papers in LNCS, they receive only a few citations; 4. The articles are strikingly more cited by authors themselves and international authors citations are more than Chinese authors non-self-citations in the first three years after publication; 5. Based on the new indicator Impact Index (II) the authors proposed, the relative impact of Chinese articles in LNCS is increasing although the average impact of Chinese papers in LNCS is obviously less than that of the publications in LNCS in each year during the studied period. Introduction By all accounts, computer science and technology sector is a rapidly developing area. It has been considered as a source of economic growth and a strategic tool to enhance the quality of life of the population [GU, 2002; ROJO & GÓMEZ, 2006]. In the Received July 30, 2007 Address for correspondence: JIANCHENG GUAN School of Management, Fudan University, 670 Guoshun Road, Shanghai , P. R. China guanjianch@buaa.edu.cn; guanjianch@fudan.edu.cn; guanjianch@sina.com /US $ Copyright 2008 Akadémiai Kiadó, Budapest All rights reserved

2 context of late industrializing countries, such as China, the well acknowledged strategy technology catching-up succeeds in the areas of mechanical engineering and electronics [CHOUNG & AL., 2003]. In China, computer science and technology is playing an increasingly important role in the science and technology development as well as in economic growth (ANNUAL REPORT OF SCIENCE AND TECHNOLOGY DEVELOPMENT OF CHINA, 2004). The evidences show that the most notable successes have occurred in upgrading of China s industrial competitiveness in the information industry, particularly in computer industry [LU & LAZONICK, 2001]. Along with technology development, scientific activities have also risen significantly. In the study of AMSDEN & MOURSHEN [1997], the strength of the East Asia countries including China resides in chemistry, physics, engineering and computer science. Another research conducted by GUAN & MA [2004] leads to the conclusion that China has shown the character of low level of beginning and high speed of developing in computer science. Computer science remains an area of highest priority in China for several decades, as identified by Natural Science Foundation of China and Ministry of Science and Technology of China [GUAN & WANG, 2004]. The development of computer science is also listed in Chinese top research programs, such as Chinese National Key Basic Research Program (973 program) and Chinese High Technology Development Program (863 program) [SONG, 1997]. The basic research is generally published in journal articles while applied research preferentially in conference papers. According to GOODRUM & AL. [2001], some conference proceedings are considered as more timely, more cutting-edge, and even more strictly refereed than some journals. They provide a main instrument for scholarly communication. As DROTT [1995] has pointed out, in the areas of information science and computer science where much work is done in private firms, researchers are preferred to present their work at conferences because of the intellectual proprietary of the ideas. ROJO & GÓMEZ [2006] have also found that in the domains of ICT (Information and Communication Technology), EU-15 countries produced more conference papers than journal papers during Since proceedings of conference is an important communication channel in computer science, this study will concentrate on the analysis of the conference papers produced by Chinese authors in a famous scholarly publication of conference proceedings Lecture Notes in Computer Science (LNCS). Citation analysis, as a scientometric technique, plays an important role in the assessment of the performance of scientific research [VAN RAAN, 1990]. According to RUSSELL & ROUSSEAU [2002], the number of received citations is related to the visibility as well as impact. Citation analysis is originated from Garfield s seminal work for creating a citation index [GARFIELD, 1955, 1970; GARFIELD & SHER, 1963] and extended by a number of practices [SCHUBERT & AL., 1986; SCHUBERT & BRAUN, 1993; MOED & VAN RAAN, 1986; BRAAM & AL., 1988, 1991; VAN RAAN, 1991, 1996; 520 Scientometrics 75 (2008)

3 KOSTOFF, 2002]. Several studies have dealt with citation analysis to Chinese research. ZHANG & ZHANG [1997] measured Chinese research performance in terms of counts and cited times of SCI-covered papers between 1987 and The results showed that the mean citation rate of was only 0.45 and the number of citations varied in the different disciplines. HE [2003] analyzed the citation rate of articles published in the journal of Chinese Science Bulletin during 1995 and 1999 in SCIE database. The author found that there were relatively high self-citation rates of the papers and most citations were from Chinese journals. In this contribution, we carry out a further analysis on the Chinese publications in LNCS by using some scientometric indicators and methods. Research objectives The objectives of this study are four folded: 1. to explore the trend of paper counts produced by Chinese author(s) in LNCS over the studied period; 2. to ascertain the authorship patterns of these papers, including the number of author(s), the nationality of the author(s), as well as the institutions of the author(s) working in; 3. to identify the citation behavior of the papers, such as the proportion of self-citation vs. non-self-citation, and the contribution of citations from international authors in the non-self-citation; 4. to examine the relative influence of the papers published by Chinese, comparing with the average impact of the total papers in LNCS. Materials and methods In this study we will analyze the conference papers published in Lecture Notes in Computer Science (LNCS) with Chinese author(s) by using several scientometric indicators and methods. LNCS is a book series where many important workshop series and conference series in computer science and information science are published and it is issued by German famous publication company Springer Verlag. It deals with the latest results of original research reported in proceedings ( LNCS commenced publication in 1973 and was indexed by SCI from Hence, the information about papers in LNCS from 1997 can be derived from the on-line edition of Science Citation Index Expanded (SCIE) database published by Institute of Science Information (ISI) ( Scientometrics 75 (2008) 521

4 By restricting the source title, address of authors, and publication year to Lecture Notes in Computer Science, PR China and each year between 1997 and 2005, respectively, the records of the Chinese articles published in LNCS have been downloaded for each year in the period of A total of 5,916 articles with Chinese author(s) are contributed in the present study. In He s analysis on articles published in Chinese Science Bulletin [HE, 2003], several notes were proposed as rules to specify the authors and citations for the papers. Following He s suggestions, some remarks are also developed for this study. Chinese authors article is an article published in LNCS which includes at least one Chinese author from mainland or Hong Kong Special Administration District or Macao Special Administration District. Articles with author(s) from Taiwan are not a part of the Chinese articles. During calculating the citation rates of the articles, a citation is counted as a self-citation if and only if at least one of the authors in the citing article matches an author in the cited article in terms of both name and address. Regarding non-self-citation, an international citation is that the author(s) of the citing article are all international author(s), other wise it is a domestic citation. As mentioned above, the authors from Hong Kong and Macao are Chinese authors, while Taiwanese are international authors. Research findings General trend of Chinese publications in LNCS Table 1 shows the chronological distribution of publications in LNCS and the articles with Chinese author(s) published in it during the period of Paper counts in LNCS Paper counts with Chinese author(s) Chinese articles /All articles (%) Source: Table 1. Chinese publication counts in LNCS Total Totally, 65,975 papers are published in LNCS over the considered period in which 5,916 papers are with Chinese author(s), contributing a share of 8.5 per cent. An upward trend is observed both in the number of total publications in LNCS and in the paper amounts with Chinese author(s) in LNCS. According to each annual contribution of Chinese publications in LNCS, it gains a remarkable improvement after Between 522 Scientometrics 75 (2008)

5 1997 and 2003, the share of Chinese papers in LNCS is less than 5 per cent in each year, while this proportion jumps up to 12 per cent in 2004 and then gains a remarkable share of 14.8 per cent in According to the data from ISI s Essential Science Indicators (ESI) (July 2006), China ranked 6 th in computer science in terms of the number of published papers over the period January 1996 to April Table 2 compares the total number of Chinese papers in computer science in ESI (denoted as N x ) with the number of Chinese papers published in LNCS (denoted as N y ) in every five-year-interval. A clear increasing trend appears in the share of Chinese papers in LNCS relative to the total Chinese papers in computer science (N y /N x ). In the period of , the sum of Chinese papers in LNCS in the five years contributed 18.57% to the total Chinese papers in computer science, while during , the share of Chinese papers in LNCS to the total number of Chinese papers in computer science reached 60.24%. Therefore, in the recent years, more than a half of Chinese SCI papers in computer science were published in LNCS. Furthermore, we investigate the Chinese publications in top journals in Computer Science. From Table 1 we can see that Chinese publications in LNCS gained remarkable increasing in 2004 and We can retrieve the Chinese publications in the top 5 journals in the category of Computer Science, Theory and Methods in which LNCS is located during 2004 and By searching in JCR-2004 Edition and JCR-2005 Edition, the top 5 journals in the selected category have a slight variation in the two years. The number of the articles with at least one Chinese author in the top 5 journals is presented in Table 3 in the two years. Chinese authors published no articles in the top two journals in 2004 and in the top 3 journals in Totally, within the top 5 journals, there are 123 articles and 218 articles published in 2004 and 2005, respectively. Only 5 articles with Chinese author(s) are published in the top 5 journals in each year, with a share of 4.07% in 2004 and 2.29% in In each year, 3 out of 5 Chinese articles (60%) are published by Hong Kong authors. Table 2. Chinese publication counts in Computer Science Number of Chinese papers in computer science (N x) Number of Chinese papers in LNCS (N y) N y/n x (%) Citations number of Chinese papers in computer science (C x) Citations number of Chinese papers in LNCS (C y) C y/c x (%) Source: Scientometrics 75 (2008) 523

6 Table 3. Chinese publications in top 5 journals in Computer Science (theory and methods) in 2004 and 2005 Rank 2004 Total articleschinese articles b/a Hong Kong Coauthors (a) (b) (%) articles articles 1 ACM Computing Surveys Human-Computer Interaction IEEE Transactions on Evolutionary Computation Evolutionary Computation Quantum Information & Computation Total Rank 2005 Total articleschinese articles b/a Hong Kong Coauthors (a) (b) (%) articles articles 1 ACM Computing Surveys Human-Computer Interaction Quantum Information & Computation IEEE Transactions on Evolutionary Computation IEEE Transactions on Networking Total Source: Among these Chinese publications in the top journals, 1 article is coauthored with foreign researchers in 2004 and 2 articles are coauthored in In sharp contrast with remarkable growth of the share of Chinese papers in LNCS, the share of SCI articles in top journals of Computer Science published totally by the scientists of mainland China is negligible during the same period. We found that although the Chinese researchers published more than 12% of the articles in LNCS, they contributed a rather low proportion in the top journals in Computer Science. Therefore, the Chinese SCI publications in Computer Science are concentrated in conference proceedings and low quality journals. Authorship patterns of Chinese publications in LNCS As BEAVER & ROSEN [1979] has pointed out, scientific collaboration plays a key role in the advancement of knowledge in any field. In modern society, the increasing cooperation in science has been a well-known phenomenon. It is necessary to study the authorship patterns of the publications. In the study of GUAN & MA [2004], the scientific papers have been divided into four categories according to their number of authors: single authored papers; two authored papers; multi-authored papers with 3 or 4 authors; and mega-authored papers including more than 4 authors. Along the same line, Figure 1 presents the breakdown by number of authors of the 5,916 Chinese papers published in LNCS. More than a half (54%) of the papers include 3 or 4 authors, and 21% of papers have even more authors. 524 Scientometrics 75 (2008)

7 Figure 1. Chinese articles in LNCS by the number of authors The papers with two authors contribute the share of 22%, while only 3% of papers are single-authored papers. This result supports the conclusion from Guan & Ma (2004) that in the field of computer science, Chinese scientists seldom worked alone and they preferred to collaborate with 2~3 collaborators. Since Chinese researchers prefer to work in groups, in the following context, we analyze the composition of the authors in the collaboration. Figure 2 illustrates the international collaboration pattern for the Chinese articles in LNCS during Except 188 single-authored papers, we find that 4,814 papers in our study are all Chinese authored, accounting for 84%, while 914 papers have at least one author with foreign address, taking a share of 16%. That is to say, most of the Chinese scientists are more likely to work with domestic fellows rather than with foreigners. Table 4 gives the distribution of international collaborators in this study. There are 1,137 foreign authors took part in Chinese joint publications in LNCS in The researchers Chinese scientists collaborated with most frequently in LNCS are from USA, accounting for nearly 30%. Singapore and United Kingdom follows, either of them takes a share of 11%. Figure 2. Chinese articles in LNCS by collaboration patterns Scientometrics 75 (2008) 525

8 Table 4. Distribution of international collaborators Country Counts Country Counts Country Counts United States 330 France 38 Taiwan 8 Singapore 130 South Korea 23 Greece 7 United Kingdom 130 Netherlands 16 New Zealand 7 Canada 105 India 12 Sweden 7 Japan 103 Finland 11 Others 37 Australia 83 Italy 10 Germany 72 Belgium 8 Total 1137 Source: Figure 3 displays the breakdown by institution of authors of the 5,916 Chinese papers published in LNCS. Most of the authors come from universities, taking a proportion of 87%. Authors from public research institutions and companies account for 9% and 4%, respectively. Therefore, with regard to the conference papers in computer science, researchers from universities are the main producers, and researchers from public research institutions and companies also play a role in the scientific community. Figure 3. Chinese articles in LNCS by institution of authors Citation analysis of Chinese publications in LNCS The research results linked each other by mutual citations in their references. One of the reasons for the occurrence of the citations is to find some evidence from the cited articles to prove or support the conclusions in the citing article. Hence, citations, to a certain extent, demonstrate the scientific value and quality of the research results. The analysis of scientific citations can be used to assess the impact of the research results. Table 2 also gives the total citation times of Chinese papers received in computer science in ESI database (denoted as C x ) and the citation times received by Chinese papers in LNCS (denoted as C y ) in every five-year-interval. Although the citation 526 Scientometrics 75 (2008)

9 times of Chinese papers in LNCS increased significantly from 200 in to 1153 in , the share of the citation times gained by Chinese papers in LNCS to those gained by all the Chinese papers in computer science remained within the range of [16%, 20%]. In Figure 4, we compare the share of Chinese papers published in LNCS relative to the total Chinese papers in computer science (N y /N x ) with the share of citation times Chinese articles in LNCS received relative to the total cited times Chinese articles received in computer science (C y /C x ). In the first interval , the two ratios have the similar values (N y /N x = 18.57%, C y /C x = 19.19%). Then N y /N x increases while C y /C x remains stable relatively. In the last interval , there exists a great distance between the two indicators (N y /N x = 60.24%, C y /C x = 19.09%). This implies in the recent years, more and more Chinese researchers preferred to publish in LNCS, but only a few Chinese articles in LNCS were cited. Figure 4. Shares of Chinese papers and citations in LNCS Table 5. Summary of the citation of the Chinese papers in LNCS Year Total A B C B/A (%) C/A (%) Source: A: Total number of Chinese papers in LNCS B: Total number of citations within the first three years C: Number of non-self-citations within the first three years Table 5 reports the total number of Chinese articles, citation times of the articles and number of non-self-citations of the articles during According to HE [2003], Scientometrics 75 (2008) 527

10 the citation window was defined as 3 years. In the present analysis, 91% of the total citations occurred within the first three years after the articles published. 5,916 Chinese articles in this study make an average of 0.2 citation times within the first three years, only 741 papers (as a percentage of 12.5%) owning citations (including self-citations as well as non-self-citations). For each year of publication, the citation rates (including self-citation) over the total of articles within the first three years cited are 10.0%, 33.3%, 26.0%, 21.3%, 33.1%, 43.1%, 34.5, 19.3% and 15.3%, respectively. Among them, non-self-citation rates are 10.0%, 22.2%, 14.0%, 10.2%, 18.5%, 21.5%, 18.7%, 7.6%, and 7.0%, respectively, indicating that the impact of the Chinese articles published in LNCS is low. However, it is optimistic to find that the tendency for both citations and non-self-citations within the first three years is increasing. Figure 5 highlights the ratio of the non-self-citations relative to the total cited times of the Chinese articles in LNCS within the first three years (C/B). A declined trend is found in Figure 5, revealing that the number of self-citations grows more rapidly than non-selfcitations. Figure 5. Rates of the non-self-citations relative to the total citations Among all citations within the first three years, 54% is self-citation, 20% is domestic non-self-citation, and 26% is international non-self-citations (see Figure 6). This illustrates that the number of self-citations is even more than non-self-citations for the Chinese articles published in LNCS within the first three years. Furthermore, the international non-self-citations are more than domestic ones, which means Chinese authors are not likely to cite the domestic articles. 528 Scientometrics 75 (2008)

11 Figure 6. Citations distribution for the Chinese papers in LNCS Figure 7 describes the trends of domestic and international non-self-citations. Both of them show growing trends during the studied period. In the initial years ( ), both the domestic and international non-self-citations showed the low levels. After 2001, international non-self-citations exceed Chinese non-self-citations and further increase. Apparently, few international papers cited Chinese articles in LNCS in 1990s, and more and more citations from abroad in the most recent years. Figure 7. Chinese and foreign non-self-citations for the Chinese papers in LNCS Scientometrics 75 (2008) 529

12 Table 6. Summary of the countries involved in the non-self-citations for the Chinese papers in LNCS Countries Citations Countries Citations Countries Citations U.S. 43 France 15 Austria 8 U.K. 24 Italy 13 Taiwan 8 South Korea 23 Canada 10 Belgium 6 Germany 20 Switzerland 10 Brazil 6 Spain 18 Greece 10 Denmark 5 Japan 17 Netherlands 10 Finland 4 Australia 16 Singapore 9 Others 25 Total 300 Source: Table 6 exhibits the international authors citations for the Chinese publications in LNCS within the first three years. 43 citations are from the U.S., followed by U.K., contributing 24 citations, and South Korea with 23 citations. From Table 6, we find that, the majority of the countries, citing Chinese articles in LNCS, are developed countries from Europe and North America. Thus, Chinese research in computer science is attracting more attentions from the international scholars of the mainstream science. The relative impact of Chinese publications in LNCS Journal impact factor (IF) is a well known indicator which measures the mean citations of the articles in the journal in a given period. In the ISI, this given period is two years. As a citation based indicator, IF can to some extent reflect the quality and international visibility of a journal. Using the ISI s calculation method, we define the Impact Index (II) of the Chinese articles published in LNCS to indicate the relative impact of the Chinese articles in LNCS. II can be described as follows: II China = Citatins in year Y to Chinese articles published in LNCS in year (Y 1) and (Y 2) Total Chinese articles published in LNCS in year (Y 1) and (Y 2) IF measures the mean frequency of the articles in a journal being cited. II China >IF LNCS indicates that the impact of Chinese articles is higher than the average impact of the articles published in LNCS, and vice versa. Table 7 presents the general tendency of Impact Factor of LNCS and Impact Index of Chinese articles in LNCS. Table 7. IF of LNCS and II of the Chinese articles in LNCS for each year Impact Factor * Impact Index II/IF Source: and authors calculation Remarks: * indicates the impact factor in 2003 is calculated by authors 530 Scientometrics 75 (2008)

13 Impact factors of LNCS remain relative steady during 7 years and the values fall in the range of (0.39, 0.52) for 5 years in the studied period. An upward trend is observed in terms of the II, indicating the increasing impact of Chinese articles in LNCS. Figure 8 compares the IF of LNCS and II of the Chinese articles published in it. We find that although the II value is obviously less than IF value in each year during the studied period, the gap between the two parameters is diminishing. The ratio between II and IF for each year is also presented in Figure 8. An obviously increasing trend can be seen in terms of II/IF. Thus, the relative impact of Chinese articles published in LNCS is increasing compared with the average impact of the publications in LNCS. Figure 8. IF of LNCS and the impact of Chinese papers in LNCS Conclusions This paper analyzes the Chinese outputs in the context of conference papers in computer science through an exploration of one of the journals in this field Lecture Notes in Computer Science in the past 9 years ( ). Based on some scientometric methods, some conclusions can be made as follows: 1. During the studied period, the number of Chinese papers in LNCS kept growing and the share of it gained a remarkable improvement since In the recent years, the Chinese SCI publications in Computer Science were concentrated in conference proceedings and low quality journals. 2. According to the analysis on authorship patterns, Chinese researchers seldom work alone, and they are more likely to collaborate with domestic fellows than foreign scientists. Furthermore, the researchers from university are the major producer. Scientometrics 75 (2008) 531

14 3. Comparing the Chinese papers in LNCS with the total Chinese papers in the whole computer science in ISI, we find that in spite of the increasing ratio between paper amounts in LNCS and in the whole computer science, only a few citation times received by the Chinese papers in LNCS relative to the papers in the whole computer science. 4. Among all citations, 54% is self-citation, 20% is Chinese authors non-selfcitation, and 26% is international authors non-self-citations. Moreover, the number of self-citations grows more rapidly than non-self-citations. In terms of non-self-citations, more and more citations are from abroad in the recent years. 5. In order to examine the relative impact of the Chinese papers in LNCS, a new indicator Impact Index (II) is defined. Comparing the II of the Chinese papers in LNCS with the Impact Factor of the LNCS, we find that the relative impact of Chinese articles is increasing although the average impact of Chinese papers in LNCS is obviously less than that of the publications in LNCS in each year during the studied period. The booming of the Chinese scientific research during recent years has drawn worldwide attention. This paper also gives us some implications on further research, not only in China, but beyond. First, it seems that Chinese citation rates are not compatible with its number of publications in LNCS. The average impact of the Chines articles in LNCS should be qualified as very low, paricularly in terms of non-self-citations. It is thus suggested that China should pay more attention to increasing the international visibility of the research and to improving the quality of research, in addition to continued attention to enlarging the number of SCI papers. Second, it seems that the share of Chinese papers in LNCS is far from being compatible with the share of SCI articles in top journals of Computer Science published by the scientists of mainland China. Publishing papers in international top journals is another way to increase the international visibility of Chinese papers. Previous evidences showed that international collaboration does contribute a lot to the improvement of the mainstream connectivity and international visibility for China s research [MA & GUAN, 2005]. However, the evidences in the study also present that Chinese researchers are more likely to collaborate with domestic fellows than foreign scientists. Therefore, the communication and collaboration among Chinese research community and international scholars in computer science should be enhanced. According to GOODRUM & AL. [2001] and GLÄNZEL & AL. [2006], proceedings papers became more and more substitutes for journals articles in computer science. The current work supports their results in the context of China. Since scientific conferences are important channels for communicating research results, the future study on the applied and engineering sciences, such as computer science, should pay more attention on the proceedings papers. Moreover, the analysis on the proceedings papers is mostly 532 Scientometrics 75 (2008)

15 based on ISI Proceedings database or SCI database, but papers published in non-serials are not yet available [GLÄNZEL & AL., 2006] and the citation statistics are also limited. Therefore, more comprehensive and systematic citation research should be conducted to extend the citation analysis to the papers of conference proceedings. * This research is substantially supported by National Natural Science Foundation of China (Project No ). Authors are grateful for the valuable comments and suggestions of both the reviewer and the editors. References AMSDEN,A., MOURSHED,M. (1997), Scientific publication, patents, and technological capabilities in lateindustrializing countries, Technology Analysis & Strategic Management, 9 : BEAVER, D. DE B., ROSEN, R. (1979), Studies in scientific collaboration. Part III. Professionalization and natural history of modern scientific co-authorship, Scientometrics, 1 : BRAAM, R. R., MOED H. F., VAN RAAN A. F. J. (1988), Mapping of science by combined co-citation and word analysis. 1. Structural aspects, Science Technology & Human Values, 13 (1 2) : BRAAM, R. R., MOED, H. F., VAN RAAN A. F. J. (1991), Mapping of science by combined cocitation and word analysis. 1. Structural aspects, Journal of the American Society for Information Science, 42 (4) : CHOUNG, J., MIN, H., PARK, M. (2003), Patterns of knowledge production: the case of information and telecommunication sector in Korea, Scientometrics, 58 (1) : DROTT, M. C. (1995), Reexamination of the role of conference papers in scholarly communication, Journal of the American Society for Information Science, 46 (4) : GARFIELD, E. (1955), Citation indexes for science new dimension in documentation through association of ideas, Science, 122 (3159) : GARFIELD, E. (1970), Citation indexing for studying science, Nature, 227 (5259) : GARFIELD, E., SHER, I. H. (1963), New factors in evaluation of scientific literature through citation indexing, American Documentation, 14 (3) : GLÄNZEL, W., SCHLEMMER, B., SCHUBERT, A., THIJS, B. (2006), Proceedings literature as additional data source for bibliometric analysis, Scientometrics, 68 (3) : GOODRUM, A. A., MCCAIN, K. W., LAWRENCE, S., GILES, C. L. (2001), Scholarly publishing in the Internet age: a citation analysis of computer science literature, Information Processing & Management, 37 (5) : GU, Y. (2002), An exploratory study of Malaysian publication productivity in computer science and information technology, Journal of the American Society for Information Science and Technology, 53 (12) : GUAN, J. C., MA, N. (2004), A comparative study of research performance in Computer science, Scientometrics, 61 (3) : GUAN, J. C., WANG, J. X. (2004), Evaluation and interpretation of knowledge production efficiency, Scientometrics, 59 (1) : HE, T. (2003), Difficulties and challenges of Chinese scientific journals: statistical analysis of Chinese literatures using Chinese Science Bulletin as example, Scientometrics, 57 (1) : KOSTOFF, R. N. (2002), Citation analysis of research performer quality, Scientometrics, 53 (1) : LU, Q., LAZONICK, W. (2001), The organisation of innovation in a transitional economy: business and government in Chinese electronic publishing, Research Policy, 30 : Scientometrics 75 (2008) 533

16 MA, N., GUAN, J. C. (2005), An exploratory study on collaboration profiles of Chinese publications in Molecular Biology, Scientometrics, 65 (3) : MOED, H. F., VAN RAAN, A. F. J. (1986), Observations and hypotheses on the phenomenon of multiple citation to a research groups oeuvre, Scientometrics, 10 (1 2) : ROJO, R., GÓMEZ, I. (2006), Analysis of the Spanish scientific and technological output in the ICT sector, Scientometrics, 66 (1) : RUSSELL, J. M., ROUSSEAU, R. (2002). Bibliometrics and institutional evaluation. In : Encyclopedia of Life Support Systems (EOLSS). Part 19.3 Science and Technology Policy, edited by Rigas Arvantis. Developed under the auspices of the UNESCO, Eolss Publishers, Oxford, UK. SCHUBERT, A., BRAUN, T. (1993), Reference-standards for citation based assessments, Scientometrics, 26 (1) : SCHUBERT, A., GLÄNZEL, W., BRAUN, T. (1986), Relative indicators of publication output and citation impact of European physics research: , Czechoslovak Journal of Physics, 36 (1) : SONG, J. (1997), Science and technology in China: the engine of rapid economic development, Technology in Society, 19 (3/4) : VAN RAAN, A. F. J., (1990), Bibliometric indicators as research performance evaluation tools, In: Proceedings of the European University Institute Conference on Research Management in Europe Today, Florence, VAN RAAN, A. F. J. (1991), Fractal geometry of information space as represented by co-citation-clustering, Scientometrics, 20 (3) : VAN RAAN, A. F. J. (1996), Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises, Scientometrics, 36 (3) : ZHANG, H. Q., ZHANG, Y. H. (1997), Scientometric study on research performance in China, Information Processing and Management, 33 (1) : Scientometrics 75 (2008)