The impact factor of rheumatology journals: an analysis of 2008 and the recent 10 years

Abstract

Despite various weaknesses, the impact factor (IF) is still used as an important indictor for scientific quality in specific subject categories. In the current study, the IFs of rheumatology journals over the past 10 years were serially analyzed and compared with that from other fields. For the past 10 years (1999–2008), the IFs published by the Institute for Scientific Information in the Science Citation Index—Journal Citation Report were analyzed. For the majority of rheumatology journals, the IF shows a gradually increasing trend. The mean and median level of increase of IF from 1999 to 2008 is 233.9 and 66.5%, respectively. The increase in IF from 1999 or the first year with IF documentation to that in 2008 was higher for European journals than for the USA journals. The aggregate IF and the median IF of rheumatology journals remained within the top 30% and top 15% in clinical medical and all the scientific categories, respectively. Over the past 10 years, rheumatology journals showed a general increase in IF and rheumatology remained a leading discipline. For journals in the English language, those from Europe had an even higher increase than those from USA.

Introduction

The impact factor (IF), an attempt to quantify and rank journal quality, has been published since 1961 by the Institute for Scientific Information (ISI). It is a measure of the frequency with which an average article in a journal has been cited in a particular year. The IF of a journal, computed annually by the Science Citation Index (SCI), is the average number of times that articles published in that journal in two consecutive years are cited during the following year. It has been widely used as an index for evaluating the quality of publications in scientific journals [1, 2]. Although it has been criticized for various weaknesses [3–7], the IF does provide an objective measure of the citation rate of the average published article in a specific journal, and it has not been replaced by any other index of rating the quality of journals.

To give a further basis for the discussion on the significance of the IF, the IF of rheumatology journals of 2008 as well as those of the past 10 years was serially analyzed and compared with that from other fields given the relative paucity of information in this area.

Methods

For the past 10 years (1999–2008), the IFs published by the ISI in the SCI—Journal Citation Report (JCR) [8] were descriptively analyzed. All journals listed in the 2008 SCI under the subheading “Rheumatology” were included. For each journal, the IF for the years 1999–2008 (if available), language (i.e., English, multiple languages (including English), or non-English) and country of origin (according to the item of “Journal Country/Territory” of each journal in the Journal Citation Report) were determined. The aggregate IF and median IF of each category were also extracted from the JCR each year (only available since 2003).

Additionally, all categories of scientific journals in the “JCR Science Edition” and categories related to clinical medicine (so-called clinical medicine journals listed under the following subheadings: Allergy; Andrology; Anesthesiology; Cardiac & cardiovascular system; Clinical neurology; Critical care medicine; Dentistry, oral surgery & medicine; Dermatology; Emergency medicine; Endocrinology & metabolism; Gastroenterology & hepatology; Geriatrics & gerontology; Hematology; Infectious diseases; Medicine, general and internal; Obstetrics & gynecology; Oncology; Ophthalmology; Orthopedics; Otorhinolaryngology; Pediatrics; Peripheral vascular disease; Psychiatry; Psychology; Radiology, nuclear medicine & medical imaging; Rehabilitation; Respiratory system; Rheumatology; Surgery; Transplantation; Tropical medicine; Urology & nephrology) were also analyzed with regard to changes in the ranking of the top 50% rheumatology journals (according to the IF in 2008), as well as to the ranking of the aggregate and median IF of rheumatology journals over the past 10 years.

Results

General data

Twenty-two journals in rheumatology were analyzed according to the 2008 JCR Science Edition (Table 1). Among the 22 journals, 20 were in English; one was in German, and the other one was in “multiple languages”. Eight journals originated from USA, 7 from UK, and the other 7 originated from 5 different countries.

Table 1 Change in IF of rheumatology journals from 1999 to 2008

Among the 22 journals in the field of rheumatology, the only non-English one was entitled “Aktuelle Rhematologie”, whose IF remains at the bottom throughout the 10 years.

“Arthritis and Rheumatism”, the official journal of the American College of Rheumatology, always remained as the top one with respect to IF among rheumatology journals until 2008, when it was replaced by “Annals of the Rheumatic Diseases”, the official journal of the European League Against Rheumatism.

Changes in IFs of rheumatology journals

Compared with 1999 (or the first year of IF documentation), the IFs in 2008 of 17/22 journals increased, with a median level of +84.7% (range +14.0 to +2457.2%). Only five journals had their IFs decreased, with a median level of −14.1% (ranging −3.8 to −64.3%). Among all the 22 journals, the mean and median level of increase in IF from 1999 (or the first year of IF documentation) to 2008 were 233.9% and 66.5%, respectively.

Journals in English language from USA and Europe

For the journals in the English language, the mean IF of those originating from the USA was higher than that from Europe before 2002. Since 2003, however, the mean IF of European journals has begun to catch up with that of USA journals. Most of the 12 non-USA journals in English originated from European countries, except one from Canada (“Journal of Rheumatology”). If we include this journal and compare the IF of English journals from the USA and that from non-USA countries, the results are similar (Fig. 1).

Fig. 1

Comparisons of the IFs of rheumatology journals from 1999 to 2008 in the English language. a Between USA and European countries; b between USA and non-USA countries

The mean and median increase of IF in 2008 from 1999 or the first year with IF documentation were higher in non-USA journals than in USA journals [1.426 ± 1.471 vs. 1.185 ± 1.896, 1.096 (range −0.551 to +5.220) vs. 0.988 (range −0.267 to +5.455)]. The median percentage of increase of IF in 2008 from 1999 or the first year with IF documentation was higher in non-USA journals than in USA journals [80.0% (range −10.9 to +1079.2%) vs. 50.1% (range −36.5 to +2457.2%)].

Effect of review articles

Among the 20 journals in the English language, 4 journals publish almost exclusively reviews rather than original papers (so-called review-journals). These are “Best Practice & Research in Clinical Rheumatology”, “Current Opinion in Rheumatology”, “Rheumatic Disease Clinics of North America”, and “Nature Clinical Practice Rheumatology”. Compared with the other 16 journals that primarily publish original papers, the “review-journals” showed more increase in their IFs. The mean and median increase of IF in 2008 from the lowest level was 2.492 vs. 1.296 and 2.256 vs. 0.944, respectively. The mean and median percentage of increase of IF in 2008 from the lowest level was 898.4 vs. 129.4% and 568.2 vs. 90.2%.

Ranking among clinical medicine journals and all scientific journals

Since 2003, the “category data” of journals of each discipline has been available, including total citation, aggregate IF, and median IF. There were 170 categories in 2003 rising to 173 categories in 2008. The aggregate IF and the median IF of rheumatology journals ranked 12–19 and 17–31 among these categories through these years, respectively. Among the 32 categories of journals related to clinical medicine, the aggregate IF and the median IF of rheumatology journals ranked 5–9 and 5–12 through the period of 2003–2008, respectively (Fig. 2).

Fig. 2

The percentile rankings of the aggregate IF and median IF of rheumatology journals from 2003 to 2008. a among clinical medicine journals; b among all scientific journals

From 1999 to 2008, 1,291–1,553 journals were registered as “clinical medicine journals” and 5,550–6,598 journals as scientific journals in the “JCR Science Edition”. The ranking of the top 50% rheumatology journals (according to the IF in 2008) in clinical medicine journals and all scientific journals in the “JCR Science Edition” is listed in Table 2.

Table 2 Absolute rankings of the IF of the top 50% rheumatology journals among all the scientific journals and clinical medicine journals

Since 2005, “Arthritis and Rheumatism” and “Annals of the Rheumatic Diseases” hold the top 2 IF among the rheumatology journals. The ranking of these two journals among clinical medicine journals and among all the scientific journals remained roughly stable or kept consistently rising within the last 10 years.

Discussion

The reliability of this IF is questionable [3–7]. For example, a journal’s IF, derived from citations to all articles in a journal, cannot tell the quality of any specific research article, nor of the quality of the work of author; review articles are heavily cited and increase the IF of journals. Moreover, there are numerous ways by which the IF can be manipulated. However, since so far there is no alternative, IFs are nowadays used as an important indicator for scientific quality of journals in specific subject categories [9].

The current study analyzed the change in IFs of rheumatology journals in the last 10 years. For the majority of these journals, the IF shows a gradual increase. This is consistent with trends in other fields [10]. Possible reasons for the increase in the IF of scientific journals include the increasing number of references per published article nowadays, modifications in the relative frequency of publication of review articles vs. original research articles, and the availability of full text of articles on the Internet [11]. The IF of the only non-English journal in rheumatology, “Aktuelle Rhematologie”, remains at the bottom throughout this 10-year period. This is in part due to bias against non-English journals noted by ISI [12, 13].

Since 2003 when the “category data” in journals of each discipline became available, the aggregated IF of rheumatology journals has stably remained within the top 30% and top 15% of categories related to clinical medicine and all scientific journals, respectively. This indicates that rheumatology is one of the active fields in research [12].

The most important finding in the current study is that, among the rheumatology journals in the English language, the IF of journals originating from European countries has gradually caught up with that originating from the USA. This phenomenon is different from what occurs in many other fields of medicine, e.g., nephrology and cardiology, and it also contrasts with some previous reports stating that journals from the USA generally show higher IF levels than journals from Europe [14]. This may be partially attributed to the citing behavior which favors journals from the USA in some fields, i.e., studies originating in the USA rarely cite research from other nations, while European researchers tend to cite more frequently paper from the other side of the Atlantic [14, 15]. Despite such bias, endeavor by the editorial boards of many European rheumatology journals is crucial for the increase in journals’ IF.

The current study also observed that “review-journals” have more rapid increase in IFs than those publishing original papers. This is consistent with findings in other fields since reviews tend to be cited more frequently [9, 12].

In conclusion, over the past 10 years, rheumatology journals showed a general increase in IF and rheumatology remains a leading discipline in clinical medicine and all scientific categories. For journals in the English language, those from Europe had an even higher increase in IF than those from the USA.