Best of both worlds? Domestic relevance vs. international visibility of local science journals in developing countries

Transcription

1 Best of both worlds? Domestic relevance vs. international visibility of local science journals in developing countries Robert J.W. Tijssen Centre for Science and Technology Studies (CWTS), Leiden University, PO Box 9555, 2300 RB Leiden, The Netherlands Centre for Research on Science and Technology (CREST), Stellenbosch University, Private Bag X1, Matieland 7602, South Africa Abstract Scientists and scholars produce research publications to disseminate new knowledge and communicate with their peers, increasingly through research articles in peer-reviewed international scientific and scholarly journals. The vast majority of these journals is Englishlanguage, and tends to focus on mainstream science relevant to the advanced countries. They do little justice to domestic and regional science of the lesser developed countries, the kind of science often published in local journals. Are these journals, even the English language journals, therefore doomed to play a peripheral role in global science? And are research papers published in those journals therefore insignificant within international scholarly communication? How can one assess local journals in terms of their global significance? These key questions drive the empirical study presented in this chapter which introduces measurement model to directly compare the international visibility of local journals and international journals. The case studies deal with African science and African journals, with a special focus on South Africa. The recent on-going trends in publication output indicate that South African researchers now often prefer international journals over their local journals. Results of citation impact analyses reveal that some of South Africa s local journals enjoy a surprisingly large degree of visibility within the international scholarly literature. Keywords Scholarly communication, developing countries, citation analyses, journal impact factor, research assessment * This paper draws on two of the author s recent publications: R.J.W. Tijssen, Africa s contribution to the worldwide research literature: new analytical perspectives, trends, and performance indicators, Scientometrics, 71, 2007), and R.J.W. Tijssen et al., How relevant are local scholarly journals in global science? A case study of South Africa, Research Evaluation, 15, , Version 3/26/2007 page 1

2 1 Introduction Many of the less developed nations of the world face enormous difficulties in gaining access to international research networks and benefiting from new scientific knowledge - either for teaching and training, building research capabilities, or for doing domestic scientific research (let alone for improving medical and health systems, combating poverty, or generating sustainable research-based economic development). Scientific research in these countries is often seen as science on the periphery; not so much because of its lesser societal relevance or insufficient scientific quality, but mostly because of its lack of visibility and lack of standing within the global research community, especially amongst those working in the advanced countries at the cutting edge of international research frontiers. One of the reasons for this dismal state of affairs arises from the fact that a very significant share of their publications, often an overwhelming majority, are disseminated through local or domestic science journals, or other publication outlets with a geographically or linguistically restricted range of dissemination (e.g. Arunachalam, 2004). Africa s peripheral position in the world of science is of course a striking case in point. African researchers have every reason to be concerned about their poor visibility in global science; not only do they lack comprehensive local and regional bibliographic databases, indexing services and ICT infrastructures that facilitate the processing and dissemination of domestic scientific literature (e.g. Nwagwu, 2005), their research activities and outputs are also not adequately covered in the large international databases which are produced in the advance industrialized nations of the Northern hemisphere (e.g. Arvanitis et al., 2000; ASSAf, 2006). Most of these information systems and indexing services comprise of research articles in peerreviewed international journals, which usually focus on mainstream Englishlanguage science. These research publications meet the scope and quality standards of the Northern databases, and are not necessarily a good reflection of all internationallevel publications produced by developing countries, let alone of all their research outputs that may be of some interest to the global research community. Consequently, the highly developed science systems of the advanced countries of the North, which have many of their local research journals indexed by such quality-controlled databases, are more visible than countries in the South whose journals are few. 1 The vast majority of their domestic or regional journals are either not included, or are only very selectively covered by disciplinary international bibliographical databases such as CAB abstracts and PubMed, or their multidisciplinary counterparts such as the Science Citation Index or PASCAL. Nowadays, scientific performance is increasingly assessed and benchmarked on the basis of publication output indexed by the Northern databases (e.g. UIS 2005, Gaillart et al., 2006). Most analysts recognize that the international databases do little justice to science in the South, especially African science. Unfortunately, there is no easy fix to this problem because assessments based exclusively on national or regional databases will also be misleading because they too suffer from insufficient coverage of all science within the country. For practical reasons, much of the cross-country comparative statistical data on African science is therefore still derived from analyses of international bibliographic databases and indexing services. This situation has a compounded negative effect on African science; on top of the lack of sufficient 1 The industrialised high-income countries account for about 85% of global R&D expenditure and contribute 90% of all research articles in international science journals. Version 3/26/2007 page 2

3 investment in research capacity, the results of research efforts are often invisible to the wider (international) community, both of which give a lower return on investment and less incentive for funding agencies and donors to invest in future research capacities. Obviously, there is a great need of supplementary systematic and empirical evidence from the primary outputs of African researchers with regards to the quantity and quality of their outputs. The challenge is therefore to identify meaningful information sources and to develop indicators, qualitative or quantitative, to generate more comprehensive coverage of scientific activities of Africa while taking into account the local realities of science and quality criteria by which R&D capabilities and outputs of developing countries can be assessed in a fair and transparent manner (cf. Tijssen and Hollanders, 2006). This paper introduces an analytical framework and measurement methodology to help fill this North-South assessment gap in science by focusing on the divide between international and local scholarly communication. Following an introduction of the methodology in section 2, quantitative empirical data on the African contribution to international science are presented in section 3 which are linked to measurements on the relevance of local research journals within South Africa. Section 4 concludes with remarks about the major pros and cons of employing research journals as an information source for indicator-based assessments and comparisons of science systems within the less developed countries. 2 Analytical framework and information sources 2.1 Research literatures Modern scholarly communication technologies disseminate huge volumes of written information on the findings of original scientific and scholarly research (many of which are now often available both in hard-copy and electronically). When crudely classified in terms of their coverage by domestic or international bibliographic databases, four broad classes of research literatures can be defined: Type 1 - international research journals and conference proceedings, usually written in English and mostly peer-reviewed by independent international experts; Type 2 - domestic or regional peer-reviewed journals and conference proceedings, often in a local language and peer reviewed by national experts; Type 3 - monographs and (edited) books issued by international publishing houses and commercial publishers, usually written in English and mostly peerreviewed by international experts; Type 4 other periodicals, books, reports, and grey literature documents usually in a local language and non peer-reviewed. Each class represents a valuable source of information for domestic and institutional assessments of scientific capacities and achievements, although these classes differ significantly as to their international accessibility and within-class comparability. Research publications disseminated through Types 1 and 2 serial literatures are usually submitted to quality control systems and are typically subjected to impartial assessments by editors, editorial boards and external peers. Type 1 literature is often the only source of reasonably reliable cross-country comparative statistics on the international visibility and scholarly impact of science in developing countries. However, most local journals are classified as Type 2. Types 3 and 4 literatures lack sufficient coverage of the most relevant research publications and are therefore Version 3/26/2007 page 3

4 unsuited for large scale systematic assessments of science with a domestic or regional focus, neither at the mesolevel of universities, research institutes nor the macrolevel entire countries or regions. What then is the role of local Type 2 African journals as dissemination vehicles of information and knowledge within the international scientific community? Should these periodicals simply be dismissed as being of minor importance, irrelevant maybe, or do they serve a purpose in bridging the divide between local science and the international research frontiers? The domestic journals and other local publication outlets are often the main vehicles for disseminating relevant results of indigenous research activities dealing with issues or problems of predominantly or exclusively local relevance. The results of a survey by Alemna et al. (1999) revealed that African university researchers highly valued publications in key international journals for their own research, but there was also strong support for local publications to ensure that their discoveries were disseminated and sufficiently embedded within the local environment. Many of the local journals represent a distinct niche in local science, covering one or more topical areas within research fields, often representing domestic or regional research communities, professional societies or specific institutes (e.g. an Academy of science). In many cases, these indigenous journals are focal points of small communities, acting as recognized flag ship outlets of their research outputs. These journals are often issued by professional publishers, and comply with international quality standards, including the adoption of peer-reviewed based selection procedures. 2 It stands to reason that researchers may prefer to publish in their local journals for many good reasons. The findings of a survey by Pouris and Richter (2000) provide the following list of motives in the case of South African scholars and scientists: paper was not good enough for (or refused by) an international peer-reviewed journal; assessment of the journal as the best outlet for the paper; special issue of the journal; papers on a topic or domestic or regional significance; targeted towards a regional or domestic audience; rapid publication time; wish to contribute to support a local journal and/or encourage local scholarship; accredited journals, i.e. those in which publication output is eligible for state subsidy. Some local journals may also be institute-specific. For example, recent work by Mouton et al. (2006) revealed that several of South Africa s largest journals including single-institute shares up to 40%. The many cases the journal is published by the same institution/unit that produces the majority of articles, suggesting protectionist publishing where the institution acts as an intentional gatekeeper to its own external articles. Alternatively, such over-representations may also simply arise from an insufficient external supply for publications. The majority of these in-house journals were found in the fields of the arts and humanities. Clearly, the large variety of local journals will also cover a wide spectrum in terms of scientific impact, ranging from journals that have succeeded in reaching 2 The article focuses on printed versions of scholarly and professional journals, thereby discarding e- journals, open access journals, or other online versions of journals, and associated electronic archiving systems. Version 3/26/2007 page 4

5 international standards to those that would seem to cater for a small, parochial audience. Table 1 summarizes the degree of significance of these four literatures as an information source for researchers, practitioners and other stakeholders. The findings indicate significant differences in relevance for the varied audiences. Overall, it would seem that there is little common ground for comprehensive descriptions and sufficiently in-depth assessments of science in developing countries, unless of course key information on individual journals belonging to Types 1 and 2 can be connected and (partially) integrated into a single analytical framework. Citation impact analysis of Type 1 and 2 journals provides a diagnostic tool to help accomplish this objective as will become apparent in the next subsection. Table 1. General typology of research literatures by their relevance as information sources within developing countries For international For local R&D R&D communities communities (researchers and (researchers and practitioners) practitioners) Type 1 Peer-reviewed international journals and conference proceedings Type 2 Peer-reviewed domestic or regional journals and For external users and stakeholders (analysts, evaluators, administrators, policy makers, funding agencies) very high (very) high very high moderate to high (very) high very high conference proceedings Type 3 Books and book chapters low to moderate moderate to high (very) high Type 4 Reports and other research documents low low to moderate low to high 2.2 Assessments of African science with Citation Indexes As indicated above, the peer-reviewed international science and engineering journals are one of the very few available sources of internationally comparative information on African scientific outputs. Only a small minority are Type 2 journals. In fact, out of a total of some 8,000 journals that are listed in the Science Citation edition and/or Social Sciences edition of Thomson Scientific s Journal Citation Reports 2004, a mere 0.3% are African journals (23 journals). Nonetheless, Thomson Scientific s Citation Index (CI) databases offer a unique and useful window on African science. The quantitative-statistical ( bibliometric ) analyses of CI-listed African research papers were conducted by querying CWTS in-house edition of Thomson Scientific s Citation Index database (CWTS/CI-database) which covers the publication years from 1980 onwards. African research papers are defined as those papers that list at least one author affiliate address in Africa. This international bibliographical database consists of some 9,000 peer-reviewed journals, including conference proceedings published in journals. This is the set of source journals. These sources are either fully indexed ( cover-to-cover ) or partially indexed by Thomson Scientific for inclusion in the CI, or the web-based editions of these databases (Web of Science, and Version 3/26/2007 page 5

6 Web of Knowledge). As of January 1 st 2006, this CDROM-based database was replaced by the Web of Science upgrade. The bibliographic information in the CI-records not only includes all author names but all affiliate addresses as well. Virtually all research articles in CI-listed journals also include footnotes, endnotes, or reference lists that acknowledge the information sources the authors considered to be of some relevance for the research reported in the paper. The quantities of citations these publications have received from other CIcovered articles, during a specific time-interval, is often used as a quantitative indicator of international scientific impact, which reflects, to a certain degree, the scientific relevance and impact of the research, the associated researchers, and of course also of the journal as a whole (Box 1 provides more technical details about citation analysis).. Box 1 Citation impact analysis with the CWTS Citation Index database The large majority of the citations refer to other Type 1 CI-indexed source items, i.e. cited publications in journals that were CI-indexed during a specified time interval. Most of the remainder consists of non-ci citations issued by CI-listed publications, either to non source items such as books, reports, or to publications in non source journals that were not CI-indexed (Types 2, 3 and 4). 3 These two categories are not mutually exclusive; there are also CI/non-CI citations to source items in journals that were only indexed for a limited number of years.4 Many of these partially indexed journals are of Type 2.Citation analysis enables us to establish how often research publications in Type 1 and Type 2 journals, either CI-listed, non-indexed, or partially indexed journals - are cited in the international CI-listed scientific literature. The citation analysis presented is this paper deals only with the aggregate level of journals as a whole (rather than at the level of individual publications in those journals). In most cases, each non-ci cited journal that was cited in research publications in CI-listed source journals are represented by a large quantity of name variants, truncations and abbreviations (the names of CI-listed journal are standardized by Thomson Scientific). Hence, the list of cited sources in the CIdatabase needs extensive cleaning and standardization to produce reliable citation impact data. The work was done at CWTS and relates only the titles of the journals. Both the outputs of CI-listed research publications, and citations between Type 1 and Type 2 literatures, may help gauge the international scientific visibility of African research activities. This information is applied to develop a set of quantitative measures which allow us to characterize and classify journals, and to compare them according to their international scientific impact. Most importantly, these citation links between Type 1 and 2 journals enable a direct comparison of the scientific visibility and impact of CI-listed Type 1 journals and local Type 2 journals within a common analytical framework, in terms of each journal s citation impact on the 3 Recent empirical studies indicate that a significant fraction of non-journal document types (especially conference proceedings, monographs and book chapters) are also likely to be cited by the research articles that are published in local and international journals (cf. Butler and Visser, 2006). 4 Some journals are listed for only one or two years, or intermittently, depending on the selection criteria and policies of Thomson Scientific. Hence, in some cases, you may find journals with both source and non source citations. Furthermore, sources citations are not identified as such by Thomson Scientific data entry procedures due to typographical or bibliographical errors or ambiguities within the original references added by the authors, as a result of which even the fully covered source journals tend to have a small fraction of non source citations. Version 3/26/2007 page 6

7 international scientific literature indexed by Thomson Scientific s Citation Indexes (Tijssen et al., 2006). 3 African research publications in science journals 3.1 General trends in publication output and citation impact Africa s contribution to global knowledge production measured in terms of CIlisted publications is still at an extremely low level compared to the other continents. The long-term trends in African publication output, shown in Figure 2, indicates that the fraction of Sub-Saharan Africa has dropped quite dramatically - from 1% in 1987 to 0.7% in 1996 with little sign of recovery. 5 Africa has lost some 10% of its share in global science since its peak in 1987; Sub-Saharan science has lost almost a third of its share. 6 Figure 2. Trends in African research article output in the international journal literature ( , % of worldwide publication output) 1,4% 1,3% 1,2% 1,1% 1,0% 0,9% 0,8% 0,7% 0,6% 0,5% 0,4% 0,3% 0,2% All African countries Sub-Saharan Africa Northern Africa Source: CWTS/Thomson Science Citation Index databases (excluding the Arts and Humanities Citation Index). A range of interrelated socio-economic factors has been suggested over the years to measure and to explain this demise (e.g. Gailliard et al., 1996). First, many researchers in African countries suffer from poor working environments, low pay, tight budgets, a lack of equipment and career prospects, which are damaging morale and are encouraging talented people to migrate to industrialized nations. Several countries simply lack the resources or willingness to invest in infrastructure and equipment to retain workers at universities, research laboratories and health institutions. Another contributing factor to the significant decline of Sub-Saharan publication output is the removal of African journals from the Citation Indexes; 5 Tijssen (2007) provides a more extensive bibliometric overview of Africa s current scientific performance. 6 Arvanitis et al. (2000) observed similar rates of decline within the PASCAL database, in large part due to the cancellation of several African journals in PASCAL, notably those from Egypt and South Africa. Version 3/26/2007 page 7

8 notably, the number of South African journals dropped from 35 to 19 during the years So, the significant downward trend is probably a too gloomy picture of reality. Moreover, because these diminishing shares of African science do not necessarily reflect decreases in any absolute sense, but rather reflect increases in publication output albeit at a (considerably) slower pace than worldwide growth rates. On the whole, these developments in African science are probably best described as steady state or leveling off. Naturally, these aggregate-level trends obscure significant changes at the level of individual countries, or scientific disciplines within those countries. Some countries and fields will fare better than others. Several African countries exhibited significant increases during recent years in real terms. The publication output trends in CI-listed journals (Figure 3) indicate that research output in South Africa, Algeria and Tunisia has grown during the years , although South Africa s gradual increase seems to have come to a halt in 2002 (Figure 4 in the next section displays the breakdown of SA s trend by outputs in local and international journals). Figure 3. Recent publication output trends of major African countries South Africa Egypt Morocco 2000 Tunisia Nigeria Algeria Kenya Source: CWTS/Thomson Science Citation Index database (excluding the Arts and Humanities Citation Index). Given the slow but steady upward trend in the output of research articles in international journals in several African countries, it appears that African science is, ever so gradually, becoming more internationalized. How do these general trends fit it with domestic developments of countries and, particularly, how do they impact on the (perceived) significance of local journals? The next section provides an in-depth statistical analysis for the case of South Africa. 3.2 Trends in South African science South Africa is Africa s leading scientific nation, with a fairly advanced Northern-type science system and home of some 250 scholarly and scientific journals that are recognized by the Government of South Africa s Department of Version 3/26/2007 page 8

9 Education (DoE) as meeting the minimum requirements for state subsidy under the policy of rewarding academics who publish in these journals. Only 20 of those DoE accredited journals were CI-listed in ; the CI-coverage of an additional 15 journals was (temporarily) discontinued by Thomson Scientific during the period Figure 4 shows that about 60% of South African research articles in journals that were published between 1990 and 2002 appeared in CI-listed journals, almost a quarter of which are of South African origin (for more details, see Mouton et al., 2006). The share of South African authored articles that were published in CIlisted foreign journals has increased significantly, from 37% in 1990 to 47% in A reverse trend occurred in share of the publication output disseminated through CIlisted SA journals. The share of articles in local South African journals dropped from 17% to 13%, at a steady state level of about 1000 articles per year. Hence, a significant share of South African research continues to be published in the domestically-oriented Type 2 literature. Figure 4. Trends in the South African output of journal articles by journal type CI-listed SA journals CI-listed Non-SA journals Non CI-listed SA journals Source: CREST, Univ. Stellenbosch These two interdependent trends are the outcome of several conflicting determinants impacting on South Africa s Type 2 scholarly literature: (a) the constraints of limited international scope and visibility of journals, (b) associated peer review-based) quality assurance systems, and (c) monetary incentives offered by DoE s accreditation system. Neither the peer-review system nor the DoE system are based entirely on considerations of quality: the choice of journals by Thomson Scientific is partially determined by commercial and market interests. Although the numbers of citation received from other CI-listed journals is the main criterion for inclusion in the Citations Indexes, it is common knowledge that journals with large 7 An additional three South African journals that are indexed by Thomson Scientific in 2004 belong to research fields in the arts and humanities: South African Archaelogical Bulletin, South African Journal of Philosophy, and South African Historical Journal. Since the arts and humanities are not an integral part of this study, these three journals are excluded from further analyses. Version 3/26/2007 page 9

10 subscription bases and published by big commercial publishers are more likely to be included. As far as the South African/DoE accreditation system goes, there is also sufficient evidence to suggest that a rigorous concern for quality has not always been applied in the selection of eligible journals (Mouton et al., 2006). The fact that substantial proportions of articles still appear in local journals in many fields of science, which is undoubtedly due to the state subsidies that reward publication output these journals. The overall outcome of the mix of incentive systems provides and interesting model for other developing countries and an example of what may occur should they decide to implement such a system: an increasing preference among researchers towards scholarly communication within the peer-refereed CIlisted foreign journals, while maintaining a presence within the local niche journals. These preferences and trends are field-specific: not surprisingly, more than two thirds of research articles in the natural, engineering and health sciences are published in CIlisted journals, whereas the high proportions of articles in the social science and humanities appearing in local non-ci listed journals, which is not surprising either since many of these fields that are locally embedded (e.g. indigenous languages, theology, law and performing arts). 3.3 How (in)visible is local science at the global level? Measuring the international citation impact of African local journals Although South Africa may represent an atypical case for the entire African continent, the increasing preference of South African scientists to publish in Type 1 journals, to the detriment of their local Type 2 journals, certainly warrants a closer look at the relevance of Africa s local journals as a whole. More specifically, which Type 2 journals (if any) seem to be bridging the communication divide between local science and global science? One way of answering this question is to examine the geographical impact of local journals and their contribution to information sources that are used by research worldwide. There are at least three information sources available for such empirical analyses. First, the coverage of African science in international bibliographical databases, especially in disciplinary databases such as CAB Abstracts (Agriculture) and PubMed (medicine and health), either in terms of individual research articles or the cover-to-cover indexing of journals. Secondly, by looking at the geographical breakdown of downloads or document deliveries of research papers that are stored in repositories of African science and technology journals. Box 2 African Journals OnLine African Journals OnLine began in May 1998 as a pilot project managed by INASP (International Network for the Availability of Scientific Publications). It aimed to promote the awareness and use of African-published journals in the sciences by providing access to tables of contents on the Internet, backed by a document delivery service. AJOL-indexed journals cover the agricultural sciences and resource management, arts, culture, language and literature, health science and technology, and social sciences. The National Inquiry Services Centre (NISC) of South Africa has taken over management of AJOL in AJOL records contain a range of information on each journal, including its sponsor, publisher, editor or co-editors, subject field, language(s) of publication, country and city of focus (see for more information). Version 3/26/2007 page 10

11 One the largest of those repositories is the African Journals OnLine database (see Box 2), a compilation of 228 scholarly journals (viewed on November 24 th 2005) that are edited and/or published in Africa, or with content dealing with Africa (Smart, 2005). The third option, the one applied in this study, is to investigate the citation impact of African research articles published in non CI-listed journals on the international research literature as represented in the CI-listed journals. This is where Type 1 and Type 2 journals directly connect. Although the vast majority of AJOL s journals are of Type 2, many of research articles published in those journals are cited regularly by articles in the Type 1 serial literature. The findings show that 111 AJOL journals which were not CI-listed (in 2004) produced research articles that were cited at least once in the international research literature during the time-interval ; 31 of those cited journals are South African, the 80 others are mostly from other Sub-Saharan countries. Nonetheless, the majority of the journals receive very few citations during those 5 years. In fact, almost 100 of them receive less than 50 citations. Only six highly cited journals received more than 150 citations within a 5-year time-interval. Table 5 displays the top 20 most highly cited journals according to their citation frequencies. Table 5. Citation-based rankings of African science journals: Top 20 AJOL journals cited in the international scholarly literature ( )*,** Journal title Country of publisher Citation frequency** Philosophical Papers South Africa 415 South African Forestry Journal South Africa 402 West African Journal of Medicine Nigeria 389 Quaestiones Mathematicae South Africa 271 African Crop Science Journal Uganda 264 African Journal of Reproductive Health Nigeria 171 East African Medical Journal Kenya 150 Ethiopian Journal of Health Development Ethiopia 115 Zimbabwe Veterinary Journal Zimbabwe 115 SINET: Ethiopian Journal of Science Ethiopia 104 Discovery and Innovation Kenya 87 Africa Insight South Africa 86 East African Agricultural and Forestry Journal Kenya 81 African Journal of Health Sciences Kenya 73 Africa Development Senegal 61 Malawi Medical Journal Malawi 49 African Journal of Library, Archives and Information Science Botswana 43 Tropical Veterinarian Nigeria 39 South African Journal of Clinical Nutrition South Africa 36 African Journal of Range and Forage Science South Africa * Citations received in Thomson Scientific Citation Indices (CI), author self-citations included. AJOL-indexed non-ci listed journals only. ** Citing years: ; cited years Although the frequencies are fairly low in many cases, these first findings make it abundantly clear that local African journals publish research articles with a noticeable impact the CI-listed international research literature. For a fuller appreciation of these citation impact linkages, should take note of the fact that these citation frequencies may critically depend on several determinants, not least the volume of papers published in a journal, the citing and cited fields of science (picking up on interdisciplinary citations), and self-citations by authors referring to their previous Version 3/26/2007 page 11

12 articles in international journals 8. The next section addresses these issues by analyzing South Africa s CI-listed international journals and non-ci local journals included in AJOL Citation-based rankings of South African journals The findings of this citation analysis, comprising 166 SA journals, show that the non-ci journals account for 48% of all references found within the worldwide CIlisted journal literature that have cited SA authored research articles during the years This share compares quite favorably to the 14-17% contribution of non- CI journals to the South African publication output (see Figure 4). Hence, SA s local journals account for more than a fair share of SA s citation impact at the global level, a surprising outcome that requires some explanation. First, the distribution of these citation frequencies across cited non-ci journals is highly skewed; a few journals receive relatively large quantities, the bulk of these journals are hardly ever cited or never at all. Secondly, this outcome is affected by the cited journal s field of science or, more specifically, the international orientation of those fields. The medical, life, and natural sciences are characterized by worldwide research communities that operate and communicate on the basis of common scientific paradigms (i.e. theories and hypotheses, research themes, methods and investigative approaches) as a result of which topical high-quality research articles tend to attract heavier citation traffic. Conversely, the research activities within the other fields of science are usually of a more applied nature and/or are focused on local or regional problems or themes, more often than not characterized by fragmentation into small and distinct schools of thought each with own local paradigms. Hence, one would expect journals in, say, social sciences to accumulate significantly lower numbers of citations compared to journals of similar size and background operating in, say, the medical sciences. Table 6 displays the summary results of the citation analyses for a subset of 107 cited journals, 88 non-ci journals and the 19 CI-listed ones, classified by broad field of science (the numbers of CI-listed journals are indicated between parentheses). 10 Each of these selected journals received at least one citation in the CI-literature during the interval The majority of all journals are cited infrequently. Although the 37 journals attributed to the natural sciences show a wide range in citation frequencies, most of these journals are CI-listed none of which collected fewer than 25 citations. It is also interesting to note that the majority of SA s medical journals are among the lesser cited journals, with the exception of the CI-listed journals Although these observed differences in citation frequency reflect field-dependent phenomenon in scholarly communication, one cannot simply compare citation frequency data of journals within fields of science without taking into account the publication output magnitude of the cited journals. Naturally, citation frequency data require sizecorrected normalizations to arrive at fair and robust comparisons of journal impacts. 8 An unknown fraction of the citations originating from CI-listed Type 1 journals are in fact author selfcitations i.e. where African researchers are citing previous work published in Type 2 journals. 9 The citation data refer to research publications that were published in the period and cited in the years The cited articles include international co-publications that list authors with South African affiliations (for more details, see also Tijssen et al., 2006). 10 The annual publication output of each journal was collated by staff at CREST (Univ. Stellenbosch, South Africa) from several sources including CREST s South African Knowledgebase, Index to South African Periodicals and from SA Studies. Version 3/26/2007 page 12

13 Table 6. Citation impact of South African journals: summary statistics ( )* Medical and Life Sciences Natural Sciences Engineering Sciences Social Sciences Share Citation frequency* High >250 1 (1) 7 (7) 8% 100 <Moderate< (1) 6 (5) 2 (1) 8% 25<Few<100 2 (1) 11 (1) 2 (1) 7 (1) 21% 1<Very few< % 0 citations % Journal Impact Factor** CE-JIF>0.5 5 (5) 7% 0.25<CE-JIF< (2) 8 (5) % 0.1<CE-JIF< (1) 7 (1) 2 (1) 1 (1) 19% 0<CE-JIF< (1) 2 21(1) 56% Share of SA citations*** >50% % 20%-50% 1 (1) 13 (10) 1 (1) 3 (2) 58% <20% 2 (1) 5 (4) % * Citing years: 2002 and 2003; cited years: ** Concerns 107 cited journals, i.e. the 88 non-ci journals and the 19 CI-listed ones that were attributed to broad fields. The number of CI-listed SA journals in 2004 indicated between brackets. *** Citations originating from research articles published in CI-listed with one or more South African author addresses. Concerns the 31 most highly cited journals, i.e. receiving at least 25 citations during the years to articles published in The number of CI-listed SA journals in 2004 is indicated between parentheses. Source: CWTS/Thomson Science Citation Index databases (excluding the Arts and Humanities Citation Index). Citation frequencies need to be corrected for the corresponding quantities of citable publication output. There are many methods to do so but one of those metrics, the Impact Factor (IF), is particularly noteworthy and useful. This index is probably the prime example of a CI-dependent and size-corrected measure of citation impact (see Box 3). The IF s two-year citation window was originally designed to measure the short-term impact of the prestigious mainstream journals, more specifically those originating from the advanced countries, especially the USA, and gathering significant quantities of citations in such a short period of time. In contrast, most journals from SA (and other less developed countries) play a modest role in worldwide science and therefore fail to meet these citation impact criteria; publications in these journals usually take considerably more time to attract sufficient numbers of citations (if any) from the international scientific literature to produce a statistically robust metrics. In dealing these technical drawbacks, the IF s two-year citation window was expanded to eight years, while two consecutive years cited years were merged to deal with possible large annual fluctuations within citation frequencies. 11 This Composite Extended Journal Impact Factor (CE-JIF) is more suited to produce statistically robust measures of the international citation impact generated by Type 2 journals. 11 The third modification relates to the journal self-citations, i.e. the citations from one article to another published within the same journal. Since these journal self-citations are not registered within Citation Indexes in the case of the non-ci listed journals, these self-citations are also not included in the citation counts of the CI-listed journals in order to arrive at a fair comparison of CI-listed and non-ci listed journals. Version 3/26/2007 page 13

14 Box 3 Impact Factor The IF was developed in the 1960s by ISI (Garfield and Sher, 1963), and is applied extensively for evaluating and comparing journal impact performances across the globe. The IF is computationally defined as follows: the ratio of all CI-indexed citations received in year t referring to CI-indexed publications from the years t-1 and t-2 and the number of research articles published in those two years. Despite its overwhelming success, the IF suffers from the fact that no single measure can describe all relevant aspects of scientific visibility of a journal, let alone its scientific quality. Applications of the IF as measure of journal quality have therefore been subject of fierce debate and controversy in the last two decades both within the academic world and amongst journal publishers (e.g. Moed, 2005). CE-JIF presents a tailored yardstick to compare and rank Type 1 and Type 2 journals according to their international scientific impact and visibility. We can now, for instance, check whether some of SA s non-ci journals are among the nation s most highly cited. As it turns out, only six journals, each of them dealing with the natural sciences, manage to attract an average of 0.5 citation per article (CE-JIF score 0.50), each which are (partially) CI-listed. Another six CI-listed SA journals have CE-JIF values less than Remarkably, two non CI-listed journals: South African Journal of Enology and Viticulture and Concrete attract an average of 0.47 citations per research article. South Africa s highly cited AJOL-indexed non-ci SA journals have considerably lower values: South African Forestry Journal (0.13); Philosophical Papers (0.12). 12 The origin of the citing literatures also provides clues as to the geographical and institutional distribution of SA s scientific impact. 13 Citations issued by articles published in CI-listed journals may refer to local scholarly communication and knowledge dissemination, rather than international level knowledge flows. It stands to reason that a non-negligible share of those citations is likely to be of SA origin where SA researchers cite their own work, previous articles of their colleagues and coworkers, or that of other members within the SA research community. 14 Clearly, these self-citations do not constitute international scientific impact in the narrow sense, but since they are included in reference lists of papers published in CI-listed journals, they may indirectly help raise the international visibility of the cited SA science and associated researchers. By calculating the shares of citations (author self-citations included) originating from SA CI-listed research articles to local SA journals, a crude measure of a journal s geographical impact can be developed to help ascertain the degree to which the citation impact actually constitutes a local impact. The findings are summarized at the bottom of Table 6, indicating that the majority of the citations were of SA origin in only 13% of the journals; SA citations accounted for 20-50% of all citations in 58% of the journals. SA authors represent the majority of the citers in only five cases: South African Archaeological Bulletin, Koedoe, Southern African Journal of Aquatic 12 The CE-JIF value of Quaestiones Mathematicae could not be computed due to missing data on the annual publication production. 13 SA citations in this case also include author self-citations, i.e. citations from later publications by the same author(s) published in CI-listed journals. These self-citations could not be filtered out due to lack of accurate information on the institutional affiliation of the authors listed on the cited (non-ci) publications. 14 A fair share of these author self-citations will relate to references in domestic or international coauthored publications in which the citing SA author participated. Such citations should be seen as partial self-citations given the limited contribution of a single author in multi-author publications. Version 3/26/2007 page 14

15 Sciences, Southern African Journal of Epidemiology and Infection, and South African Journal of Higher Education. Interestingly, four South African journals also seem to exert a significant impact on African science at the regional level, attracting more than 10% of their citations from research articles produced by authors based in other African countries: African Plant Protection, Water SA, Durban Museum Novitates, and African Sociological Review. In 29% of the cases do we find the impact to be predominantly international; South African citations represent less than 20% of all citations in nine journals, four of which are not CI-listed, indicating that in exceptional cases local journals may indeed also enjoy a significant degree of international citation impact. This outcome clearly merits a geographical breakdown of citing courses when assessing the true nature of international scientific impact of journals in less developed countries. As for the previously highlighted AJOL-indexed SA journals, Quaestiones Mathematicae received 14% of its citations from SA authors, 35% of the citations to South African Forestry Journal are of South African origin, while these self-citations account for a mere 2% in the case of Philosophical Papers which is remarkably high for an arts and humanities journal. Table 7. Top 20 South African journals cited in the international scholarly literature, ranked by CE-JIF score* Citation frequencies Source journal Thomson Scientific CE-JIF** Source Non source Citation Indices Impact Factor*** Journal title (abbreviation) ( ) ( ) ( ) (2004) SA J Marine Science partially 0.64 SA J Geology no 1.11 African Zoology fully 0.39 Onderstepoort J Vet Research fully 0.49 Water SA fully 0.46 SA Archaeological Bulletin partially N/A SA J Science fully 0.55 SA J Enology and Viticulture no N/A Concrete no N/A SA Med J fully 1.11 SA J Wildlife Research fully 0.24 SA J Chemistry fully 0.37 SA J Psychology fully N/A J SA Veterinary Assoc partially 0.32 African Entomology partially 0.41 Ostrich partially 0.62 SA J Botany partially 0.46 Social Dynamics partially 0.45 Afr J Range and Forage Sci no N/A SA J Economics fully * CE-JIF scores of each journal are based on the total of citations received from source publications indexed for in Thomson Scientific Citation Indices in addition to non-source citations from non-indexed publications (author self-citations included) normalized for the publication output in the corresponding cited years. ** Citing years: 2002 (cited years ) and 2003 (cited years ), excludes journal self-citations. *** Citing year: 2004; cited years , includes journal self-citations. Table 7 compares various citation statistics of South Africa s leading journals. The top 20 journals according to their CE-JIF ranking include three non CI-listed Version 3/26/2007 page 15

16 journals: South African Journal of Enology and Viticulture, Concrete, and African Journal of Range and Forage Science. The first two are even among the top 10. The statistics also illustrate that citation-based journal rankings based on Thomson Scientific s Impact Factor are often not as straightforward and applicable as they might seem at first sight. Owing to partial indexing and annual changes in Thomson Scientific s coverage of South African journals, we find many cases where CI-listed source journals are not fully indexed. Consequently, the CE-JIF scores often comprise of a mixed bag of source citations (from one CI-listed journal to the other) and non source citations (from CI-listed to non CI-listed journals) Conclusions and lessons 4.1 Case study on South African journals Several salient observations and conclusions emerge from this particular case study on South Africa: Measuring a journal s citation impact on global science helps to determine the relative standing of domestic and regional research journals within a unified internationally comparative framework. A journal performance indicator that is specifically designed to deal with the citation impact specifics of local journals in peripheral countries (like the CE-JIF index), introduces an external metric for objective assessments (and rankings!) of journals in terms of their global relevance which may also help to evaluate the academic performance of local scientists and scholars who publish regularly in these journals; Thomson Scientific does not always select the most heavily cited local journals for processing in its Citation Index databases. Overall, we find no evidence of a clear cut distinction between a small elite collection of CI-listed journals that are relatively highly cited in the international literature and the large majority of poorly cited non-ci journals. Locally oriented journals can be competitive with the internationally-oriented CI-listed journals in terms of citation impact, especially when normalized for the journal s annual publication output; Journal classification and ranking systems should incorporate as many qualityrelated criteria as possible (international visibility, spread of local and overseas authors, frequency of issues, rejection rates of submitted articles, composition of editorial boards, and so on) covering both their international standing and local significance; Implementing such an information system will require dedicated and comprehensive databases, infrastructures and investments that will not be readily available in most developing countries; ideally, an integrated system (an African Citation Index!?) that combines the best of both worlds : local information sources, like AJOL, and international sources such as Thomson Scientific s Citation Indexes. In conclusion, the metrics and indicators introduced in this paper offer a comprehensive analytical framework to categorize and rank local journals according to their international scientific impact. The results also tell an important cautionary tale; rather than treating local journals as equivalent entities, with little or no 15 The occurrence of small numbers of non source citations to fully indexed journals (e.g. African Zoology and Water SA) is caused by references that were not identified at first by Thomson Scientific but subsequently identified by the CWTS data cleaning routines. Version 3/26/2007 page 16