Rheumatology International

, Volume 32, Issue 7, pp 1861–1867

Diversity, value and limitations of the journal impact factor and alternative metrics

  • Lutz Bornmann
  • Werner Marx
  • Armen Yuri Gasparyan
  • George D. Kitas
Review Article

Abstract

The highly popular journal impact factor (JIF) is an average measure of citations within 1 year after the publication of a journal as a whole within the two preceding years. It is widely used as a proxy of a journal’s quality and scientific prestige. This article discusses misuses of JIF to assess impact of separate journal articles and the effect of several manuscript versions on JIF. It also presents some newer alternative journal metrics such as SCImago Journal Rank and the h-index and analyses examples of their application in several subject categories.

Keywords

Bibliometrics Periodicals as topic Impact factor h-Index Biomedical journals Rheumatology 

References

  1. 1.
    Archambault É, Larivière V (2009) History of the journal impact factor: contingencies and consequences. Scientometrics 79:635–649. doi:10.1007/s11192-007-2036-x CrossRefGoogle Scholar
  2. 2.
    Gasparyan AY, Banach M (2009) A medium of science communication in our times. Arch Med Sci 5:1–2Google Scholar
  3. 3.
    Habibzadeh F (2008) Journal impact factor: uses and misuses. Arch Iran Med 11:453–454PubMedGoogle Scholar
  4. 4.
    Gasparyan AY (2010) Thoughts on impact factors and editing of medical journals. Inflamm Allergy Drug Targets 9:2–5. doi:10.2174/187152810791292908 PubMedGoogle Scholar
  5. 5.
    Bornmann L, Leydesdorff L, Marx W (2007) Citation environment of angewandte chemie. CHIMIA 61:104–109. doi:10.2533/chimia.2007.104 CrossRefGoogle Scholar
  6. 6.
    Retzer V, Jurasinski G (2009) Towards objectivity in research evaluation using bibliometric indicators: a protocol for incorporating complexity. Basic Appl Ecol 10:393–400. doi:10.1016/j.baae.2008.09.001 CrossRefGoogle Scholar
  7. 7.
    (2007) EASE statement on inappropriate use of impact factors. Eur Sci Ed 33:99–100Google Scholar
  8. 8.
    Hoeffel C (1998) Journal impact factors. Allergy 53:1225PubMedCrossRefGoogle Scholar
  9. 9.
    Garfield E (2006) The history and meaning of the journal impact factor. JAMA 295:90–93. doi:10.1001/jama.295.1.90 PubMedCrossRefGoogle Scholar
  10. 10.
    Seglen PO (1997) Why the impact factor of journals should not be used for evaluating research. BMJ 314:498–502PubMedCrossRefGoogle Scholar
  11. 11.
    Oswald AJ (2007) An examination of the reliability of prestigious scholarly journals: evidence and implications for decision-makers. Economica 74:21–31. doi:10.1111/j.1468-0335.2006.00575.x CrossRefGoogle Scholar
  12. 12.
    Metze K (2010) Bureaucrats, researchers, editors, and the impact factor: a vicious circle that is detrimental to science. Clin (Sao Paulo) 65:937–940. doi:10.1590/S1807-59322010001000002 CrossRefGoogle Scholar
  13. 13.
    Krell FT (2010) Should editors influence journal impact factors? Learn Publ 23:59–62. doi:10.1087/20100110 CrossRefGoogle Scholar
  14. 14.
    Dimitrov JD, Kaveri SV, Bayry J (2010) Metrics: journal’s impact factor skewed by a single paper. Nature 466:179. doi:10.1038/466179b PubMedCrossRefGoogle Scholar
  15. 15.
    Rossner M, Van Epps H, Hill E (2008) Irreproducible results: a response to Thomson Scientific. J Exp Med 205:260–261. doi:10.1084/jem.20080053 PubMedCrossRefGoogle Scholar
  16. 16.
    Rossner M, Van Epps H, Hill E (2007) Show me the data. J Cell Biol 179:1091–1092. doi:10.1083/jcb.200711140 PubMedCrossRefGoogle Scholar
  17. 17.
    Golubic R, Rudes M, Kovacic N, Marusic M, Marusic A (2008) Calculating impact factor: how bibliographical classification of journal items affects the impact factor of large and small journals. Sci Eng Ethics 14:41–49. doi:10.1007/s11948-007-9044-3 PubMedCrossRefGoogle Scholar
  18. 18.
    González L, Campanario JM (2007) Structure of the impact factor of journals included in the social sciences citation index: citations from documents labeled “editorial material”. J Am Soc Inf Sci Technol 58:252–262. doi:10.1002/asi.v58:2 CrossRefGoogle Scholar
  19. 19.
    Togia A, Tsigilis N (2006) Impact factor and education journals: a critical examination and analysis. Int J Educ Res 45:362–379. doi:10.1016/j.ijer.2007.02.001 CrossRefGoogle Scholar
  20. 20.
    Caramelli B, Silva MRE (2010) Brazilian medical journals are now facing a big challenge. Eur Sci Ed 36:38–39Google Scholar
  21. 21.
    Satyanarayana K (2010) Impact factor and other indices to assess science, scientists and scientific journals. Indian J Physiol Pharmacol 54:197–212PubMedGoogle Scholar
  22. 22.
    Bornmann L, Daniel H-D (2008) What do citation counts measure? A review of studies on citing behavior. J Documentation 64:45–80. doi:10.1108/00220410810844150 CrossRefGoogle Scholar
  23. 23.
    Tomlinson S (2000) The research assessment exercise and medical research. BMJ 320:636–639PubMedCrossRefGoogle Scholar
  24. 24.
    Brown H (2007) How impact factors changed medical publishing—and science. BMJ 334:561–564. doi:10.1136/bmj.39142.454086.AD PubMedCrossRefGoogle Scholar
  25. 25.
    Adler R, Ewing J, Taylor P (2009) Citation statistics. Stat Sci 24:1–14. doi:10.1214/09-STS285 CrossRefGoogle Scholar
  26. 26.
    Habibzadeh F, Yadollahie M (2011) Evidence-based journalism. Croat Med J 52:212–213. doi:10.3325/cmj.2011.52.212 PubMedCrossRefGoogle Scholar
  27. 27.
    Frey BS, Rost K (2010) Do rankings reflect research quality? J Appl Econ 13:1–38. doi:10.1016/S1514-0326(10)60002-5 CrossRefGoogle Scholar
  28. 28.
    Moed HF, Van Leeuwen TN, Reedijk J (1996) A critical analysis of the journal impact factors of angewandte chemie and the journal of the American chemical society: inaccuracies in published impact factors based on overall citations only. Scientometrics 37:105–116. doi:10.1007/BF02093487 CrossRefGoogle Scholar
  29. 29.
    Marx W (2001) Angewandte chemie in light of the science citation index. Angew Chem Int Ed 40:139–141. doi:10.1002/1521-3773(20010105)40:1<139:AID-ANIE139>3.0.CO;2-4 CrossRefGoogle Scholar
  30. 30.
  31. 31.
    Bornmann L, Neuhaus C, Daniel H-D (2011) The effect of a two-stage publication process on the journal impact factor: a case study on the interactive open access journal atmospheric chemistry and physics. Scientometrics 86:93–97. doi:10.1007/s11192-010-0250-4 CrossRefGoogle Scholar
  32. 32.
    Pöschl U (2004) Interactive journal concept for improved scientific publishing and quality assurance. Learn Publ 17:105–113. doi:10.1087/095315104322958481 CrossRefGoogle Scholar
  33. 33.
    Bergstrom CT, West JD, Wiseman MA (2008) The eigenfactor metrics. J Neurosci 28:11433–11434. doi:10.1523/JNEUROSCI.0003-08.2008 PubMedCrossRefGoogle Scholar
  34. 34.
    Rizkallah J, Sin DD (2010) Integrative approach to quality assessment of medical journals using impact factor, eigenfactor, and article influence scores. PLoS One 5:e10204. doi:10.1371/journal.pone.0010204 PubMedCrossRefGoogle Scholar
  35. 35.
    Fersht A (2009) The most influential journals: impact factor and eigenfactor. Proc Natl Acad Sci USA 106:6883–6884. doi:10.1073/pnas.0903307106 PubMedCrossRefGoogle Scholar
  36. 36.
    Mathur VP, Sharma A (2009) Impact factor and other standardized measures of journal citation: a perspective. Indian J Dent Res 20:81–85PubMedCrossRefGoogle Scholar
  37. 37.
    Licinio J (2006) Molecular psychiatry: the highest immediacy index in the field of psychiatry. Mol Psychiatr 11:324. doi:10.1038/sj.mp.4001820 CrossRefGoogle Scholar
  38. 38.
    Gasparyan AY, Ayvazyan L, Kitas GD (2011) Biomedical journal editing: elements of success. Croat Med J 52:423–428. doi:10.3325/cmj.2011.52.423 PubMedCrossRefGoogle Scholar
  39. 39.
    The SJR indicator: a new indicator of journals’ scientific prestige. Available at http://arxiv.org/ftp/arxiv/papers/0912/0912.4141.pdf. Accessed 15 Sept 2011
  40. 40.
    Gonzalez-Pereira B, Guerrero-Bote VP, Moya-Anegon F (2010) A new approach to the metric of journals’ scientific prestige: the SJR indicator. J Inf 4:379–391. doi:10.1016/j.joi.2010.03.002 Google Scholar
  41. 41.
    Gasparyan AY (2011) Get indexed and cited, or perish. Eur Sci Ed 37:66Google Scholar
  42. 42.
  43. 43.
    Falagas ME, Kouranos VD, Arencibia-Jorge R, Karageorgopoulos DE (2008) Comparison of SCImago journal rank indicator with journal impact factor. FASEB J 22:2623–2628. doi:10.1096/fj.08-107938 PubMedCrossRefGoogle Scholar
  44. 44.
    Bornmann L, Daniel H-D (2009) The state of h index research. Is the h index the ideal way to measure research performance. EMBO Rep 10:2–6. doi:10.1038/embor.2008.233 PubMedCrossRefGoogle Scholar
  45. 45.
    Hirsch JE (2005) An index to quantify an individual’s scientific research output. Proc Natl Acad Sci USA 102:16569–16572. doi:10.1073/pnas.0507655102 PubMedCrossRefGoogle Scholar
  46. 46.
    Braun T, Glänzel W, Schubert A (2006) A Hirsch-type index for journals. Scientometrics 69:169–173. doi:10.1007/s11192-006-0147-4 CrossRefGoogle Scholar
  47. 47.
    Kanter SL (2009) Understanding the journal’s impact. Acad Med 84:1169–1170. doi:10.1097/ACM.0b013e3181b7ffa6 PubMedCrossRefGoogle Scholar
  48. 48.
    http://www.bioinformation.net/. Accessed 15 Sept 2011
  49. 49.
    Chen M, Zhao MH, Kallenberg CG (2011) The impact factor of rheumatology journals: an analysis of 2008 and the recent 10 years. Rheumatol Int 31:1611–1615. doi:10.1007/s00296-010-1541-z Google Scholar
  50. 50.
    Bornmann L, Marx W (2011) The h index as a research performance indicator. Eur Sci Ed 37:77–80Google Scholar
  51. 51.
    Bornmann L, Marx W, Schier H (2009) Hirsch-type index values for organic chemistry journals: a comparison of new metrics with the Journal Impact Factor. Eur J Org Chem 2009:1471–1476. doi:10.1002/ejoc.200801243 CrossRefGoogle Scholar
  52. 52.
    Rieder S, Bruse CS, Michalski CW, Kleeff J, Friess H (2010) The impact factor ranking—a challenge for scientists and publishers. Langenbecks Arch Surg 395(Suppl 1):69–73. doi:10.1007/s00423-010-0623-4 PubMedCrossRefGoogle Scholar
  53. 53.
    Kovacić N, Huić M, Ivanis A (2008) Citation analysis of the croatian medical journal: the first 15 years. Croat Med J 49:12–17. doi:10.3325/cmj.2008.1.12 PubMedCrossRefGoogle Scholar
  54. 54.

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Lutz Bornmann
    • 1
  • Werner Marx
    • 2
  • Armen Yuri Gasparyan
    • 3
  • George D. Kitas
    • 3
    • 4
  1. 1.Max Planck Society, Administrative HeadquartersMunichGermany
  2. 2.Max Planck Institute for Solid State ResearchStuttgartGermany
  3. 3.Clinical Research Unit, Dudley Group NHS Foundation Trust (A Teaching Trust of University of Birmingham), Department of RheumatologyRussell’s Hall HospitalDudley, West MidlandsUK
  4. 4.Arthritis Research UK Epidemiology UnitUniversity of ManchesterManchesterUK

Personalised recommendations