Rank-normalized journal impact factor as a predictive tool

Varia - Scientometrics

Abstract

Citation data accumulated on articles from the top and bottom 25 of impact factor (IF)-ranked international journals are compared using 59 international geoscience journals from 1998 and 378 Polish geological papers from 1989-994. There is a minor risk of being uncited when results are published in high-IF periodicals as the average non-citation rate is 0.88 over a 10-year period in this not very rapidly developing scientific discipline. Similarly, the established error levels in the prognosis of expected citation success versus failure based on the extreme IF quartiles as an evaluation tool is low (at most 12.5). Thus the application of the rank-normalized journal IF as a proxy of real citation frequency and, accordingly, as a predictive tool in the a priori qualification of recently published publications is a rational time- and cost-saving alternative (or at least a significant supplement) to traditional informed peer review. Blanket criticism of using IF for decisions in research funding is therefore at least partly exaggerated.

Keywords

impact factor citation scientific quality research funding 

Abbreviations

ISI

Institute for Scientific Information in Philadelphia

IF

journal impact factor

r

correlation coefficient

n

number of articles

m

median citation value.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adam D (2002) The counting house. Nature 415: 726-29PubMedGoogle Scholar
  2. Cole S, Cole JR, Simon GA (1981) Chance and consensus in peer review. Science 214: 881-86CrossRefPubMedGoogle Scholar
  3. Fairchild J (2000) Celebration and consideration of citations. J Geol Soc 157: 1089-091CrossRefGoogle Scholar
  4. Falagas ME, Alexiou VG (2008) The top-ten in journal impact factor manipulation. Arch Immunol Ther Exp 56: 223-26CrossRefGoogle Scholar
  5. Fava GA, Ottolini F (2000) Impact factors versus actual citations. Psychother Psychosom 69: 285-86CrossRefPubMedGoogle Scholar
  6. Filion KB, Pless IB (2008) Factors related to the frequency of citation of epidemiologic publications. Epidemiol Perspect Innov Doi:10.1186/1742-5573-5-3 (available via http://www.epi-perspectives.com/content/5/1/3)
  7. Franck G (1999) Scientific communication — a vanity fair?. Science 286:53, 55Google Scholar
  8. Garfield E (1983) How to use citation analysis for faculty evaluations, and when is relevant? Parts 1 and 2. Essays Inf Scientist 6:354-72 (available via http://www.garfield.library.upenn.edu/essays/v6p354y1983.pdf; http://www.garfield.library.upenn.edu/essays/v6p363y1983.pdf)
  9. Garfield E (1991) To be an uncited scientist is no cause for shame. Essays Inf Scientist 14:390-91 (available via http://www.garfield.library.upenn.edu/essays/v14p390y1991.pdf)
  10. Garfield E (1996) How can impact factors be improved?. BMJ 313: 411-13PubMedGoogle Scholar
  11. Garfield E (2000) Use of Journal Citation Reports and Journal Performance Indicators in measuring short and long term journal impact. Croatian Med J 41:368-74 (available via http://www.garfield.library.upenn.edu/papers/croatianmedj41(4)p368-374y2000.pdf)Google Scholar
  12. Garfield E (2003) The meaning of the Impact Factor. Rev Int Psicol Clinica Salud 3:363-69 (available via http://garfield.library.upenn.edu/papers/meaningofif2003.pdf)Google Scholar
  13. Garfield E (2006) The history and meaning of the Journal Impact Factor. JAMA 293: 90-3CrossRefGoogle Scholar
  14. Hoeffel C (1998) Journal impact factors. Allergy 53: 1225CrossRefPubMedGoogle Scholar
  15. Holden G, Rosenberg G, Barker K et al (2006) An assessment of the predictive validity of Impact Factor scores: Implications for academic employment decisions in social work. Res Social Work Pract 16: 613-24CrossRefGoogle Scholar
  16. Kabala ZJ (1998) Know thy journals (available via http://www.agu.org/eos_elec/98073e.html)
  17. Leimu R, Koricheva J (2005) What determines the citation frequency of ecological papers?. Trends Ecol Evol 20: 28-2CrossRefPubMedGoogle Scholar
  18. Łomnicki A (2003) Impact factors reward and promote excellence — the system is unkind but effective. Others would do less good for developing countries. Nature 424: 487Google Scholar
  19. Pudovkin AI, Garfield E (2004) Rank-normalized Impact Factor: A way to compare journal performance across subject categories. Proceedings of the 67th Annual Meeting of the American Society for Information Science and Technology 041:507-15Google Scholar
  20. Racki G (2002) Parametryczny system oceny jednostek naukowych przez KBN: prognozy i postulaty. Zagadnienia Naukoznawstwa 38: 51-8Google Scholar
  21. Rothwell PM, Martyn CN (2000) Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by chance alone? Brain 123: 1964-969Google Scholar
  22. Saha S, Saint S, Christakis D (2003) Impact Factor: a valid measure of journal quality. J Med Libr Assoc 91: 42-6PubMedGoogle Scholar
  23. Seglen PO (1994) Causal relationship between article citedness and journal impact. J Am Soc Inf Sci 45: 1-1CrossRefGoogle Scholar
  24. Seglen PO (1997) Why the Impact Factor of journals should not be used for evaluating research?. BMJ 124: 498-0225Google Scholar
  25. Swart P, Carling P (2008) Citations and other musings. Sedimentology 55: 1115-116CrossRefGoogle Scholar
  26. Wróblewski AK (2008) A commentary on misuses of the impact factor. Arch Immunol Ther Exp 56: 355-56CrossRefGoogle Scholar
  27. Żlicz M (1999) Biologia molekularna w Polsce. Sprawy Nauki 2: 15-8Google Scholar

Copyright information

© L. Hirszfeld Institute of Immunology and Experimental Therapy, Wroclaw, Poland 2009

Authors and Affiliations

  1. 1.Institute of PaleobiologyPolish Academy of SciencesWarszawaPoland

Personalised recommendations