Abstract
Despite various weaknesses, the impact factor (IF) is still used as an important indictor for scientific quality in specific subject categories. In the current study, the IFs of rheumatology journals over the past 10 years were serially analyzed and compared with that from other fields. For the past 10 years (1999–2008), the IFs published by the Institute for Scientific Information in the Science Citation Index—Journal Citation Report were analyzed. For the majority of rheumatology journals, the IF shows a gradually increasing trend. The mean and median level of increase of IF from 1999 to 2008 is 233.9 and 66.5%, respectively. The increase in IF from 1999 or the first year with IF documentation to that in 2008 was higher for European journals than for the USA journals. The aggregate IF and the median IF of rheumatology journals remained within the top 30% and top 15% in clinical medical and all the scientific categories, respectively. Over the past 10 years, rheumatology journals showed a general increase in IF and rheumatology remained a leading discipline. For journals in the English language, those from Europe had an even higher increase than those from USA.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
The impact factor (IF), an attempt to quantify and rank journal quality, has been published since 1961 by the Institute for Scientific Information (ISI). It is a measure of the frequency with which an average article in a journal has been cited in a particular year. The IF of a journal, computed annually by the Science Citation Index (SCI), is the average number of times that articles published in that journal in two consecutive years are cited during the following year. It has been widely used as an index for evaluating the quality of publications in scientific journals [1, 2]. Although it has been criticized for various weaknesses [3–7], the IF does provide an objective measure of the citation rate of the average published article in a specific journal, and it has not been replaced by any other index of rating the quality of journals.
To give a further basis for the discussion on the significance of the IF, the IF of rheumatology journals of 2008 as well as those of the past 10 years was serially analyzed and compared with that from other fields given the relative paucity of information in this area.
Methods
For the past 10 years (1999–2008), the IFs published by the ISI in the SCI—Journal Citation Report (JCR) [8] were descriptively analyzed. All journals listed in the 2008 SCI under the subheading “Rheumatology” were included. For each journal, the IF for the years 1999–2008 (if available), language (i.e., English, multiple languages (including English), or non-English) and country of origin (according to the item of “Journal Country/Territory” of each journal in the Journal Citation Report) were determined. The aggregate IF and median IF of each category were also extracted from the JCR each year (only available since 2003).
Additionally, all categories of scientific journals in the “JCR Science Edition” and categories related to clinical medicine (so-called clinical medicine journals listed under the following subheadings: Allergy; Andrology; Anesthesiology; Cardiac & cardiovascular system; Clinical neurology; Critical care medicine; Dentistry, oral surgery & medicine; Dermatology; Emergency medicine; Endocrinology & metabolism; Gastroenterology & hepatology; Geriatrics & gerontology; Hematology; Infectious diseases; Medicine, general and internal; Obstetrics & gynecology; Oncology; Ophthalmology; Orthopedics; Otorhinolaryngology; Pediatrics; Peripheral vascular disease; Psychiatry; Psychology; Radiology, nuclear medicine & medical imaging; Rehabilitation; Respiratory system; Rheumatology; Surgery; Transplantation; Tropical medicine; Urology & nephrology) were also analyzed with regard to changes in the ranking of the top 50% rheumatology journals (according to the IF in 2008), as well as to the ranking of the aggregate and median IF of rheumatology journals over the past 10 years.
Results
General data
Twenty-two journals in rheumatology were analyzed according to the 2008 JCR Science Edition (Table 1). Among the 22 journals, 20 were in English; one was in German, and the other one was in “multiple languages”. Eight journals originated from USA, 7 from UK, and the other 7 originated from 5 different countries.
Among the 22 journals in the field of rheumatology, the only non-English one was entitled “Aktuelle Rhematologie”, whose IF remains at the bottom throughout the 10 years.
“Arthritis and Rheumatism”, the official journal of the American College of Rheumatology, always remained as the top one with respect to IF among rheumatology journals until 2008, when it was replaced by “Annals of the Rheumatic Diseases”, the official journal of the European League Against Rheumatism.
Changes in IFs of rheumatology journals
Compared with 1999 (or the first year of IF documentation), the IFs in 2008 of 17/22 journals increased, with a median level of +84.7% (range +14.0 to +2457.2%). Only five journals had their IFs decreased, with a median level of −14.1% (ranging −3.8 to −64.3%). Among all the 22 journals, the mean and median level of increase in IF from 1999 (or the first year of IF documentation) to 2008 were 233.9% and 66.5%, respectively.
Journals in English language from USA and Europe
For the journals in the English language, the mean IF of those originating from the USA was higher than that from Europe before 2002. Since 2003, however, the mean IF of European journals has begun to catch up with that of USA journals. Most of the 12 non-USA journals in English originated from European countries, except one from Canada (“Journal of Rheumatology”). If we include this journal and compare the IF of English journals from the USA and that from non-USA countries, the results are similar (Fig. 1).
The mean and median increase of IF in 2008 from 1999 or the first year with IF documentation were higher in non-USA journals than in USA journals [1.426 ± 1.471 vs. 1.185 ± 1.896, 1.096 (range −0.551 to +5.220) vs. 0.988 (range −0.267 to +5.455)]. The median percentage of increase of IF in 2008 from 1999 or the first year with IF documentation was higher in non-USA journals than in USA journals [80.0% (range −10.9 to +1079.2%) vs. 50.1% (range −36.5 to +2457.2%)].
Effect of review articles
Among the 20 journals in the English language, 4 journals publish almost exclusively reviews rather than original papers (so-called review-journals). These are “Best Practice & Research in Clinical Rheumatology”, “Current Opinion in Rheumatology”, “Rheumatic Disease Clinics of North America”, and “Nature Clinical Practice Rheumatology”. Compared with the other 16 journals that primarily publish original papers, the “review-journals” showed more increase in their IFs. The mean and median increase of IF in 2008 from the lowest level was 2.492 vs. 1.296 and 2.256 vs. 0.944, respectively. The mean and median percentage of increase of IF in 2008 from the lowest level was 898.4 vs. 129.4% and 568.2 vs. 90.2%.
Ranking among clinical medicine journals and all scientific journals
Since 2003, the “category data” of journals of each discipline has been available, including total citation, aggregate IF, and median IF. There were 170 categories in 2003 rising to 173 categories in 2008. The aggregate IF and the median IF of rheumatology journals ranked 12–19 and 17–31 among these categories through these years, respectively. Among the 32 categories of journals related to clinical medicine, the aggregate IF and the median IF of rheumatology journals ranked 5–9 and 5–12 through the period of 2003–2008, respectively (Fig. 2).
From 1999 to 2008, 1,291–1,553 journals were registered as “clinical medicine journals” and 5,550–6,598 journals as scientific journals in the “JCR Science Edition”. The ranking of the top 50% rheumatology journals (according to the IF in 2008) in clinical medicine journals and all scientific journals in the “JCR Science Edition” is listed in Table 2.
Since 2005, “Arthritis and Rheumatism” and “Annals of the Rheumatic Diseases” hold the top 2 IF among the rheumatology journals. The ranking of these two journals among clinical medicine journals and among all the scientific journals remained roughly stable or kept consistently rising within the last 10 years.
Discussion
The reliability of this IF is questionable [3–7]. For example, a journal’s IF, derived from citations to all articles in a journal, cannot tell the quality of any specific research article, nor of the quality of the work of author; review articles are heavily cited and increase the IF of journals. Moreover, there are numerous ways by which the IF can be manipulated. However, since so far there is no alternative, IFs are nowadays used as an important indicator for scientific quality of journals in specific subject categories [9].
The current study analyzed the change in IFs of rheumatology journals in the last 10 years. For the majority of these journals, the IF shows a gradual increase. This is consistent with trends in other fields [10]. Possible reasons for the increase in the IF of scientific journals include the increasing number of references per published article nowadays, modifications in the relative frequency of publication of review articles vs. original research articles, and the availability of full text of articles on the Internet [11]. The IF of the only non-English journal in rheumatology, “Aktuelle Rhematologie”, remains at the bottom throughout this 10-year period. This is in part due to bias against non-English journals noted by ISI [12, 13].
Since 2003 when the “category data” in journals of each discipline became available, the aggregated IF of rheumatology journals has stably remained within the top 30% and top 15% of categories related to clinical medicine and all scientific journals, respectively. This indicates that rheumatology is one of the active fields in research [12].
The most important finding in the current study is that, among the rheumatology journals in the English language, the IF of journals originating from European countries has gradually caught up with that originating from the USA. This phenomenon is different from what occurs in many other fields of medicine, e.g., nephrology and cardiology, and it also contrasts with some previous reports stating that journals from the USA generally show higher IF levels than journals from Europe [14]. This may be partially attributed to the citing behavior which favors journals from the USA in some fields, i.e., studies originating in the USA rarely cite research from other nations, while European researchers tend to cite more frequently paper from the other side of the Atlantic [14, 15]. Despite such bias, endeavor by the editorial boards of many European rheumatology journals is crucial for the increase in journals’ IF.
The current study also observed that “review-journals” have more rapid increase in IFs than those publishing original papers. This is consistent with findings in other fields since reviews tend to be cited more frequently [9, 12].
In conclusion, over the past 10 years, rheumatology journals showed a general increase in IF and rheumatology remains a leading discipline in clinical medicine and all scientific categories. For journals in the English language, those from Europe had an even higher increase in IF than those from the USA.
References
Garfield E (1955) Citation indexes for science: a new dimension in documentation through association of ideas. Science 122:108–111
Luukkonen T (1990) Bibliometrics and evaluation of research performance. Ann Med 22:145–150
Al-Awqati Q (2007) Impact factors and prestige. Kidney Int 71:183–185
Kamath PS, Bologna G (2009) Impact factor: misused and overhyped? Hepatology 49:1787–1789
[No authors listed] (2006) The impact factor game. It is time to find a better way to assess the scientific literature. PLoS Med 3: e291
Seglen PO (1997) Why the impact factor of journals should not be used for evaluating research. BMJ 314:498–502
Dong P, Loh M, Mondry A (2005) The “impact factor” revisited? Biomed Digit Libr 2:7
Science Citation Index (SCI®) (2009) Journal Citation Reports. A bibliometric analysis of science journals in the ISI® data base. ISI—Institute for Scientific Information, Philadelphia
Andersen J, Belmont J, Cho CT (2006) Journal impact factor in the era of expanding literature. J Microbiol Immunol Infect 39:436–443
Falagas ME, Zouglakis GM, Papastamataki PA (2006) Trends in the impact factor of scientific journals. Mayo Clin Proc 81:1401–1402
Mueller PS, Murali NS, Cha SS, Erwin PJ, Ghosh AK (2006) The effect of online status on the impact factors of general internal medicine journals. Neth J Med 64:39–44
Sims JL, McGhee CN (2003) Citation analysis and journal impact factors in ophthalmology and vision science journals. Clin Experiment Ophthalmol 31:14–22
Garfield E (1986) Which medical journals have the greatest impact? Ann Intern Med 105:313–320
Boldt J, Haisch G, Maleck WH (2000) Changes in the impact factor of anesthesia/critical care journals within the past 10 years. Acta Anaesthesiol Scand 44:842–849
de Jong JW, Schaper W (1996) The international rank order of clinical cardiology. Eur Heart J 17:35–42
Acknowledgments
This study is supported by a grant of the National Natural Science Fund (No. 30972733).
Conflict of interest statement
None.
Open Access
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Chen, M., Zhao, MH. & Kallenberg, C.G.M. The impact factor of rheumatology journals: an analysis of 2008 and the recent 10 years. Rheumatol Int 31, 1611–1615 (2011). https://doi.org/10.1007/s00296-010-1541-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00296-010-1541-z