Impact factor: outdated artefact or stepping-stone to journal certification?

Abstract

A review of Garfield’s journal impact factor and its specific implementation as the Thomson Reuters impact factor reveals several weaknesses in this commonly-used indicator of journal standing. Key limitations include the mismatch between citing and cited documents, the deceptive display of three decimals that belies the real precision, and the absence of confidence intervals. These are minor issues that are easily amended and should be corrected, but more substantive improvements are needed. There are indications that the scientific community seeks and needs better certification of journal procedures to improve the quality of published science. Comprehensive certification of editorial and review procedures could help ensure adequate procedures to detect duplicate and fraudulent submissions.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

References

  1. Abraham, P. (2000). Duplicate and salami publications. Journal of Postgraduate Medicine, 46, 67–69.

    Google Scholar 

  2. Abramo, G., D’Angelo, C. A., & Di Costa, F. (2010). Citations versus journal impact factor as proxy of quality: Could the latter ever be preferable? Scientometrics, 84, 821–833.

    Google Scholar 

  3. Adler, R., Ewing, J., & Taylor, P. (2008). Joint committee on quantitative assessment of research: Citation statistics. The Australian Mathematical Society Gazette, 35(3), 166–188.

    MATH  Google Scholar 

  4. Althouse, B. M., West, J. D., Bergstrom, C. T., & Bergstrom, T. (2009). Differences in impact factor across fields and over time. Journal of the American Society for Information Science and Technology, 60, 27–34.

    Google Scholar 

  5. Altmann, K. G., & Gorman, G. E. (1998). The usefulness of impact factors in serial selection: A rank and mean analysis using ecology journals. Library Acquisitions: Practice and Theory, 22(2), 147–159.

    Google Scholar 

  6. Andrade, A., González-Jonte, R., & Campanario, J. M. (2009). Journals that increase their impact factor at least fourfold in a few years: The role of journal self-citations. Scientometrics, 80, 515–528.

    Google Scholar 

  7. Anonymous. (2004). Complacency about misconduct. Nature, 427, 1.

    Google Scholar 

  8. Anonymous (2011). The Thomson Reuters journal selection process. Thomson Reuters, http://thomsonreuters.com/products_services/science/free/essays/journal_selection_process/. Accessed 2011 September 5.

  9. ARC (2010). The excellence in research for Australia (ERA) Initiative. Australian Researh Council, Canberra. http://www.arc.gov.au/era/default.htm. Accessed 2011 August 5.

  10. Archambault, E., & Lariviere, V. (2009). History of the journal impact factor: Contingencies and consequences. Scientometrics, 79, 635–649.

    Google Scholar 

  11. Awrey, J., Inaba, K., Barmparas, G., Recinos, G., Teixeira, P. G. R., Chan, L. S., et al. (2011). Reference accuracy in the general surgery literature. World Journal of Surgery, 35, 475–479.

    Google Scholar 

  12. Bain, C. R., & Myles, P. S. (2005). Relationship between journal impact factor and levels of evidence in anaesthesia. Anaesthesia and Intensive Care, 33(5), 567–570.

    Google Scholar 

  13. Barbour, V. (2010). How ghost-writing threatens the credibility of medical knowledge and medical journals. Haematologica, 95, 1–2.

    Google Scholar 

  14. Bath, F. J., Owen, V. E., & Bath, P. M. W. (1998). Quality of full and final publications reporting acute stroke trials: A systematic review. Stroke, 29(10), 2203–2210.

    Google Scholar 

  15. Beal, J. (2011). Wiley-Blackwell announces continued growth in impact factor journals. Press release 11 July 2011, http://www.eurekalert.org/pub_releases/2011-07/w-wac071111.php. Accessed 2011 August 25.

  16. Bensman, S. J. (2007). Garfield and the impact factor. Annual Review of Information Science and Technology, 41, 93–155.

    Google Scholar 

  17. Bensman, S. J. (2008). Distributional differences of the impact factor in the sciences versus the social sciences: An analysis of the probabilistic structure of the 2005 journal citation reports. Journal of the American Society for Information Science and Technology, 59, 1366–1382.

    Google Scholar 

  18. Bensman, S. J., Smolinsky, L. J., & Pudovkin, A. I. (2010). Mean citation rate per article in mathematics journals: Differences from the scientific model. Journal of the American Society for Information Science and Technology, 61, 1440–1463.

    Google Scholar 

  19. Bernstam, E. V., Herskovic, J. R., Aphinyanaphongs, Y., Aliferis, C. F., Sriram, M. G., & Hersh, W. R. (2006). Using citation data to improve retrieval from MEDLINE. Journal of the American Medical Informatics Association, 13, 96–105.

    Google Scholar 

  20. Berquist, T. H. (2008). Duplicate publishing or journal publication ethics 101. American Journal of Roentgenology, 191, 311–312.

    Google Scholar 

  21. Bollen, J., Van de Sompel, H., Hagberg, A., & Chute, R. (2009). A principal component analysis of 39 scientific impact measures. PLoS ONE, 4(6), e6022.

    Google Scholar 

  22. Bonillo Perales, A. (2002). Spanish pediatric publications in pubmed between 1996 and 2001 [Publicaciones pediátricas españolas en PubMed en los años 1996 y 2001]. Anales Espanoles de Pediatria, 57(2), 152–156.

    Google Scholar 

  23. Bornmann, L., Nast, I., & Daniel, H.-D. (2008). Do editors and referees look for signs of scientific misconduct when reviewing manuscripts? A quantitative content analysis of studies that examined review criteria and reasons for accepting and rejecting manuscripts for publication. Scientometrics, 77, 415–432.

    Google Scholar 

  24. Braun, T. (Ed.) (2007). The Impact Factor of scientific and scholarly journals: Its use and misuse in research evaluation. Scientometrics Guidebooks Series (vol. 2, pp. 686). Budapest: Akademiai Kiado.

  25. Braun, T., & Glanzel, W. (1995). On a source of error in computing journal impact factors. Chemical Intelligencer, 1, 31–32.

    Google Scholar 

  26. Brown, H. (2007). How impact factors changed medical publishing: and science. BMJ, 334, 561–564.

    Google Scholar 

  27. Brumback, R. A. (2009a). Impact factor wars: Episode V: The empire strikes back. Journal of Child Neurology, 24, 260–262.

    Google Scholar 

  28. Brumback, R. A. (2009b). Impact factor: Let’s be unreasonable! Epidemiology, 20, 932–933.

    Google Scholar 

  29. Butakov, S., & Scherbinin, V. (2009). The toolbox for local and global plagiarism detection. Computers & Education, 52, 781–788.

    Google Scholar 

  30. Callaham, M., & McCulloch, C. (2011). Longitudinal trends in the performance of scientific peer reviewers. Annals of Emergency Medicine, 57, 141–148.

    Google Scholar 

  31. Callaham, M., Wears, R. L., & Weber, E. (2002). Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA, 287(21), 2847–2850.

    Google Scholar 

  32. Calver, M. C., & Bradley, J. S. (2010). Patterns of citations of open access and non-open access conservation biology journal papers and book chapters. Conservation Biology, 24(3), 872–880.

    Google Scholar 

  33. Cameron, B. D. (2005). Trends in the usage of ISI bibliometric data: Uses, abuses, and implications. Portal, 5, 105–125.

    Google Scholar 

  34. Campanario, J. M. (2011a). Empirical study of journal impact factors obtained using the classical two-year citation window versus a 5-year citation window. Scientometrics, 87, 189–204.

    Google Scholar 

  35. Campanario, J. M. (2011b). Large increases and decreases in journal impact factors in only 1 year: The effect of journal self-citations. Journal of the American Society for Information Science and Technology, 62, 230–235.

    Google Scholar 

  36. Campanario, J. M., Carretero, J., Marangon, V., Molina, A., & Ros, G. (2011). Effect on the journal impact factor of the number and document type of citing records: A wide-scale study. Scientometrics, 87, 75–84.

    Google Scholar 

  37. Campbell, P. (2008). Escape from the impact factor. Ethics in Science and Environmental Politics, 8, 5–7.

    Google Scholar 

  38. Cartwright, V. A., & McGhee, C. N. J. (2005). Ophthalmology and vision science research. Part 1: Understanding and using journal impact factors and citation indices. Journal of Cataract and Refractive Surgery, 31, 1999–2007.

    Google Scholar 

  39. Cashore, B., van Kooten, C. G., Vertinsky, I., Auld, G., & Affolderbach, J. (2005). Private or self-regulation? A comparative study of forest certification choices in Canada, the United States and Germany. Forest Policy and Economics, 7, 53–69.

    Google Scholar 

  40. Chalmers, I. (2006). Role of systematic reviews in detecting plagiarism: Case of Asim Kurjak. BMJ, 333, 594.

    Google Scholar 

  41. Chapman, S., Ragg, M., & McGeechan, K. (2009). Citation bias in reported smoking prevalence in people with schizophrenia. Australian and New Zealand Journal of Psychiatry, 43(3), 277–282.

    Google Scholar 

  42. Cheek, J., Garnham, B., & Quan, J. (2006). What’s in a number? Issues in providing evidence of impact and quality of research(ers). Qualitative Health Research, 16, 423–435.

    Google Scholar 

  43. Chen, C. (2006). CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature. Journal of the American Society for Information Science and Technology, 57(3), 359–377.

    Google Scholar 

  44. Chew, M., Villanueva, E. V., & Van Der Weyden, M. B. (2007). Life and times of the impact factor: Retrospective analysis of trends for seven medical journals (1994–2005) and their Editors’ views. Journal of the Royal Society of Medicine, 100, 142–150.

    Google Scholar 

  45. Cobo, M. J., Lopez-Herrera, A. G., Herrera-Viedma, E., & Herrera, F. (2011). Science mapping software tools: Review, analysis, and cooperative study among tools. JASIST, 62(7), 1382–1402.

    Google Scholar 

  46. Cohen, J. B. (1938). The misuse of statistics. Journal of the American Statistical Association, 33(204), 657–674.

    Google Scholar 

  47. Coleman, R. (1999). Impact factors: Use and abuse in biomedical research. Anatomical Record, 257, 54–57.

    Google Scholar 

  48. Coleman, A. (2007). Assessing the value of a journal beyond the impact factor. Journal of the American Society for Information Science and Technology, 58(8), 1148–1161.

    Google Scholar 

  49. Davies, J. (1955). Phrenology: Fad and science: A 19th-century American crusade (p. 203). New Haven: Yale University Press.

    Google Scholar 

  50. Dempsey, J. A. (2009). Impact factor and its role in academic promotion: a statement adopted by the international respiratory journal editors roundtable. Journal of Applied Physiology, 107, 1005.

    Google Scholar 

  51. Dong, P., Loh, M., & Mondry, A. (2005). The “impact factor” revisited. Biomedical Digital Libraries, 2, 7.

    Google Scholar 

  52. Dost, F. N. (2008). Peer review at a crossroads: A case study. Environmental Science and Pollution Research, 15, 443–447.

    Google Scholar 

  53. Drew, D. E., & Karpf, R. (1981). Ranking academic departments: Empirical findings and a theoretical perspective. Research in Higher Education, 14(4), 305–320.

    Google Scholar 

  54. Eaton, L. (2005). Medical editors issue guidance on ghost writing. BMJ, 330(7498), 988.

    Google Scholar 

  55. Elsevier (2011). Article posting policies. Elsevier BV. http://www.elsevier.com/wps/find/authorsview.authors/postingpolicy. Accessed 2011 September 10.

  56. Epstein, R. J. (2004). Journal impact factors do not equitably reflect academic staff performance in different medical subspecialties. Journal of Investigative Medicine, 52, 531–536.

    Google Scholar 

  57. Errami, M., & Garner, H. (2008). A tale of two citations. Nature, 451, 397–399.

    Google Scholar 

  58. Errami, M., Hicks, J. M., Fisher, W., Trusty, D., Wren, J. D., Long, T. C., et al. (2008). Deja vu: A study of duplicate citations in medline. Bioinformatics, 24, 243–249.

    Google Scholar 

  59. Falagas, M. E., & Alexiou, V. G. (2008). The top-ten in journal impact factor manipulation. Archivum Immunologiae et therapiae Experimentalis, 56(4), 223–226.

    Google Scholar 

  60. Fassoulaki, A., Paraskeva, A., Papilas, A., & Karabinis, G. (2000). Self-citations in six anaesthesia journals and their significance in determining the impact factor. British Journal of Anaesthesia, 84(2), 266–269.

    Google Scholar 

  61. Favaloro, E. J. (2009). The journal impact factor: Don’t expect its demise any time soon. Clinical Chemistry and Laboratory Medicine, 47, 1319–1324.

    Google Scholar 

  62. Foo, J. Y. A. (2009). The retrospective analysis of bibliographical trends for nine biomedical engineering journals from 1999 to 2007. Annals of Biomedical Engineering, 37, 1474–1481.

    Google Scholar 

  63. Foo J. Y. A. (2010). A retrospective analysis of the trend of retracted publications in the field of biomedical and life sciences. Science and Engineering Ethics. doi:10.1007/s11948-010-9212-8.

  64. Foo, J. Y. A. (2011). Impact of excessive journal self-citations: A case study on the folia phoniatrica et logopaedica journal. Science and Engineering Ethics, 17, 65–73.

    Google Scholar 

  65. Fox, M. F. (1994). Scientific misconduct and editorial and peer review processes. J Higher Educ, 65, 298–309.

    Google Scholar 

  66. Frank, M. (2003). Impact factors: arbiter of excellence? J Med Libr Assoc, 91, 4–6.

    Google Scholar 

  67. Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178, 471–479.

    Google Scholar 

  68. Garfield, E. (1994). The Thomson Reuters impact factor. http://thomsonreuters.com/products_services/science/free/essays/impact_factor/.

  69. Garfield, E. (1996). How can impact factors be improved? British Medical Journal, 313, 411–413.

    Google Scholar 

  70. Garfield, E. (1999). Journal impact factor: A brief review. Canadian Medical Association Journal, 161, 979–980.

    Google Scholar 

  71. Garfield, E. (2006). The history and meaning of the journal impact factor. JAMA, 295, 90–93.

    Google Scholar 

  72. Glänzel, W. (2009). The multi-dimensionality of journal impact. Scientometrics, 78, 355–374.

    Google Scholar 

  73. Glänzel, W., & Moed, H. F. (2002). Journal impact measures in bibliometric research. Scientometrics, 53, 171–193.

    Google Scholar 

  74. Gluud, L. L., Sorensen, T. I. A., Gotzsche, P. C., & Gluud, C. (2005). The journal impact factor as a predictor of trial quality and outcomes: Cohort study of hepatobiliary randomized clinical trials. American Journal of Gastroenterology, 100, 2431–2435.

    Google Scholar 

  75. Glynn, R. W., Chin, J. Z., Kerin, M. J., & Sweeney, K. J. (2010). Representation of cancer in the medical literature: A bibliometric analysis. PLoS ONE, 5(11), e13902.

    Google Scholar 

  76. Godlee, F. (2004). Dealing with editorial misconduct. BMJ, 329, 1301.

    Google Scholar 

  77. Goldstein, H., & Maier, G. (2010). The use and valuation of journals in planning scholarship: Peer assessment versus impact factors. Journal of Planning Education and Research, 30, 66–75.

    Google Scholar 

  78. Gollogly, L., & Momen, H. (2006). Ethical dilemmas in scientific publication: pitfalls and solutions for editors. Rev Saúde Pública, 40, 24–29.

    Google Scholar 

  79. Gøtzsche, P. C., Kassirer, J. P., Woolley, K. L., Wager, E., Jacobs, A., et al. (2009). What should be done to tackle ghost writing in the medical literature? PLoS Med, 6(2), e1000023. doi:10.1371/journal.pmed.1000023.

    Google Scholar 

  80. Greenwood, D. C. (2007). Reliability of journal impact factor rankings. BMC Medical Research Methodology, 7, 48.

    Google Scholar 

  81. Gwilym, S. E., Swan, M. C., & Giele, H. (2004). One in 13 ‘original’ articles in the journal of bone and joint surgery are duplicate or fragmented publications. The Journal of Bone and Joint Surgery, 86-B(5), 743–745.

    Google Scholar 

  82. Ha, T. C., Tan, S. B., & Soo, K. C. (2006). The journal impact factor: Too much of an impact? Annals of the Academy of Medicine, Singapore, 35, 911–916.

    Google Scholar 

  83. Habibzadeh, F., & Winker, M. A. (2009). Duplicate publication and plagiarism: causes and cures. Notfall Rettungsmed, 12, 415–418.

    Google Scholar 

  84. Hames, I. (2007). Peer review and manuscript management in scientific journals: guidelines for good practice. Oxford: Wiley. doi:10.1002/9780470750803.

    Google Scholar 

  85. Hansen, H. B., & Henriksen, J. H. (1997). How well does journal ‘impact’ work in the assessment of papers on clinical physiology and nuclear medicine? Clinical Physiology, 17(4), 409–418.

    Google Scholar 

  86. Haslam, N., & Koval, P. (2010). Predicting long-term citation impact of articles in social and personality psychology. Psychological Reports, 106(3), 891–900.

    Google Scholar 

  87. Hernan, M. A. (2009). Impact factor: A call to reason. Epidemiology, 20, 317–318.

    Google Scholar 

  88. Ho, D. (2011). Thomson Reuters: Cause for celebration! Journal of Laboratory Automation, 16(3), A7–A8.

    Google Scholar 

  89. Holsapple, C. W. (2009). Journal self-citation II: The quest for high impact:Truth and consequences? Communications of the Association for Information Systems, 25, 11–20.

    Google Scholar 

  90. Horner, J., & Minifie, F. D. (2011). Research ethics III: Publication practices and authorship, conflicts of interest, and research misconduct. Journal of Speech, Language, and Hearing S346 Research, 54, S346–S362.

    Google Scholar 

  91. Hunt, G. E., Cleary, M., & Walter, G. (2010). Psychiatry and the Hirsch h-index: The relationship between journal impact factors and accrued citations. Harvard Review of Psychiatry, 18(4), 207–219.

    Google Scholar 

  92. Jacobs, G., & Ip, B. (2005). Ring fenced research: The case of computer-assisted learning in health sciences. British Journal of Educational Technology, 36, 361–377.

    Google Scholar 

  93. Jacso, P. (2009). Five-year impact factor data in the journal citation reports. Online Information Review, 33, 603–614.

    Google Scholar 

  94. Jahangiriana, M., Eldabi, T., Garg, L., Jun, G. T., Naseer, A., Patel, B., et al. (2011). A rapid review method for extremely large corpora of literature: Applications to the domains of modelling, simulation, and management. International Journal of Information Management, 31, 234–243.

    Google Scholar 

  95. Jeacle, I., & Carter, C. (2011). In TripAdvisor we trust: Rankings, calculative regimes and abstract systems. Accounting, Organizations and Society. doi:10.1016/j.aos.2011.04.002.

  96. Johnson, C. (2006). Repetitive, duplicate, and redundant publications: a review for authors and readers. Journal of Manipulative and Physiological Therapeutics, 29, 505–509.

    Google Scholar 

  97. Johnstone, M.-J. (2007). Journal impact factors: Implications for the nursing profession. International Nursing Review, 54, 35–40.

    Google Scholar 

  98. Jones, A. W. (2002). JAT’s impact factor: Room for improvement? Journal of Analytical Toxicology, 26, 2–5.

    Google Scholar 

  99. Jones, A. W. (2003). Impact factors of forensic science and toxicology journals: What do the numbers really mean? Forensic Science International, 133, 1–8.

    Google Scholar 

  100. Jörgensen, P. (2005). Incorporating context in text analysis by interactive activation with competition artificial neural networks. Information Processing & Management, 41(5), 1081–1099.

    Google Scholar 

  101. Kapeller, J. (2010). Citation metrics: Serious drawbacks, perverse incentives, and strategic options for heterodox economics. American Journal of Economics and Sociology, 69, 1376–1408.

    Google Scholar 

  102. Ketcham, C. M. (2008). The proper use of citation data in journal management. Archivum Immunologiae et therapiae Experimentalis, 56, 357–362.

    Google Scholar 

  103. Knothe, G. (2006). Comparative citation analysis of duplicate or highly related publications. Journal of the American Society for Information Science and Technology, 57(13), 1830–1839.

    Google Scholar 

  104. Kotiaho, J. S., Tomkins, J. L., & Simmons, L. W. (1999). Unfamiliar citations breed mistakes. Nature, 400, 307.

    Google Scholar 

  105. Krell, F.-T. (2010). Should editors influence journal impact factors? Learned Publishing, 23, 59–62.

    Google Scholar 

  106. Kumar, V., Upadhyay, S., & Medhi, B. (2009). Impact of the impact factor in biomedical research: Its use and misuse. Singapore Medical Journal, 50, 752–755.

    Google Scholar 

  107. Kurmis, A. P. (2003). Understanding the limitations of the journal impact factor. Journal of Bone and Joint Surgery: Series A, 85, 2449–2454.

    Google Scholar 

  108. Kurmis, A. P., & Kurmis, T. P. (2006). Exploring the relationship between impact factor and manuscript rejection rates in radiologic journals. Academic Radiology, 13, 77–83.

    Google Scholar 

  109. Kurmis, T. P., & Kurmis, A. P. (2010). Self-citation rates among medical imaging journals and a possible association with impact factor. Radiography, 16, 21–25.

    Google Scholar 

  110. Laband, D. N. (1990). Is there value-added from the review process in economics?: Preliminary evidence from authors. The Quarterly Journal of Economics, 105, 341–352.

    Google Scholar 

  111. Lange, L. L. (2002). The impact factor as a phantom: Is there a self-fulfilling prophecy effect of impact? Journal of Documentation, 58, 175–184.

    Google Scholar 

  112. Larivière, V., & Gingras, Y. (2010). The impact factor’s Matthew effect: A natural experiment in bibliometrics. Journal of the American Society for Information Science and Technology, 61, 424–427.

    Google Scholar 

  113. Lau, S. L., & Samman, N. (2007). Levels of evidence and journal impact factor in oral and maxillofacial surgery. International Journal of Oral and Maxillofacial Surgery, 36, 1–5.

    Google Scholar 

  114. Lawrence, P. A. (2007). The mismeasurement of science. Current Biology, 17, R583–R585.

    Google Scholar 

  115. Lehmkuhl, G., Petermann, F., & Warnke, A. (2009). Research in child and adolescent psychiatry, promotion of young academics and publication practice [Kinder- und jugendpsychiatrische Forschung, Förderung des wissenschaftlichen Nachwuchses und Veröffentlichungspraxis]. Zeitschrift fur Kinder- und Jugendpsychiatrie und Psychotherapie, 37, 93–96.

    Google Scholar 

  116. Leutner, D., & Wirth, J. (2007). As mirrored by the journal: Themes and trends of educational psychology in the years 2005 to 2007 [Im Spiegel der Zeitschrift: Themen und Trends der Pädagogischen Psychologie in den Jahren 2005 bis 2007]. Zeitschrift fur Padagogische Psychologie, 21, 195–202.

    Google Scholar 

  117. Leydesdorff, L., & Opthof, T. (2010). Scopus’s source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations. Journal of the American Society for Information Science and Technology, 61, 2365–2369.

    Google Scholar 

  118. Lock, S. (1995). Lessons from the Pearce affair: Handling scientific fraud. BMJ, 310, 1547–1548.

    Google Scholar 

  119. Lomnicki, A. (2003). Impact factors reward and promote excellence. Nature, 424, 487.

    Google Scholar 

  120. Maier, G. (2006). Impact factors and peer judgment: The case of regional science journals. Scientometrics, 69(3), 651–667.

    Google Scholar 

  121. McGarty, C. (2000). The citation impact factor in social psychology: a bad statistic that encourages bad science. Current Research in Social Psychology, 5, 1–16.

    Google Scholar 

  122. McKeever, L. (2006). Online plagiarism detection services: Saviour or scourge? Assessment & Evaluation in Higher Education, 31, 155–165.

    Google Scholar 

  123. McVeigh, M. E. (2004). Journal self-citation in the journal citation reports. Thomson Reuters, http://thomsonreuters.com/products_services/science/free/essays/journal_self_citation_jcr. Accessed 2011 September 7.

  124. McVeigh, M. E., & Mann, S. J. (2009). The journal impact factor denominator: Defining citable (counted) items. JAMA, 302, 1107–1109.

    Google Scholar 

  125. Medarević, M., Banković, S., Pantić, D. & Obradović, S. (2010) Effects of the control method (goč variety) in selection forest management in western Serbia. Archives of Biological Science, 62(2), 407–418.

    Google Scholar 

  126. Mehrad, J., & Goltaji, M. (2010). Correlation between journal self citation with impact factor for the scientific publications in humanities published between 2001 and 2007 based on Persian journal citation report generated by Islamic science citation database. Information Sciences and Technology, 25, 189–206.

    Google Scholar 

  127. Mehrad, J., & Goltaji, M. (2011). Correlation between journal self-citation and impact factor in ISC’s PJCR agriculture and veterinary science journals during 2001–2007. International Journal of Information Science and Management, 9, 75–87.

    Google Scholar 

  128. Meneghini, R., Packer, A. L., & Nassi-Calo, L. (2008). Articles by Latin American authors in prestigious journals have fewer citations. PLoS ONE, 3(11), e3804.

    Google Scholar 

  129. Metze, K. (2010). Bureaucrats, researchers, editors and the impact factor: A vicious circle that is detrimental to science. Clinics, 65(10), 937–940.

    Google Scholar 

  130. Meyer zu Eissen, S., & Stein, B. (2006). Intrinsic plagiarism detection. Lecture Notes in Computer Science, 3936, 565–569.

    Google Scholar 

  131. Minelli, C., Thompson, J. R., Abrams, K. R., Thakkinstian, A., & Attia, J. (2009). The quality of meta-analyses of genetic association studies: A review with recommendations. American Journal of Epidemiology, 170, 1333–1343.

    Google Scholar 

  132. Moed, H., & Plume, A. (2011). Research assessment: The multi-dimensional research assessment matrix. Research Trends, 23, 5–7.

    Google Scholar 

  133. Moed, H. F., Van Leeuwen, Th. N., & Reedijk, J. (1996). A critical analysis of the journal impact factors of Angewandte chemie and the Journal of the American Chemical Society: Inaccuracies in published impact factors based on overall citations only. Scientometrics, 37, 105–116.

    Google Scholar 

  134. Moller, J., Retelsdorf, J., & Sudkamp, A. (2010). As mirrored by the journal: Themes and trends of educational psychology in the years 2008 to 2010 [Im Spiegel der Zeitschrift: Themen und Trends der Pädagogischen Psychologie in den Jahren 2008 bis 2010]. Zeitschrift fur Padagogische Psychologie, 24, 163–169.

    Google Scholar 

  135. Monastersky, R. (2005). The number that’s devouring science. The chronicle of higher education 52(8):A12 http://chronicle.com/free/v52/i08/08a01201.htm.

  136. Mullen, L. B. (2008). Increasing Impact of Scholarly Journal Articles: Practical Strategies Librarians Can Share. E-JASL: The Electronic Journal of Academic and Special Librarianship 9, 1, http://southernlibrarianship.icaap.org/content/v09n01/mullen_l01.html. Accessed 2011 September 1.

  137. Neuhaus, C., Marx, W., & Daniel, H.-D. (2009). The publication and citation impact profiles of Angewandte Chemie and the journal of the American Chemical Society based on the sections of chemical abstracts: A case study on the limitations of the journal impact factor. Journal of the American Society for Information Science and Technology, 60, 176–183.

    Google Scholar 

  138. Obremskey, W. T., Pappas, N., Attallah-Wasif, E., Tornetta, P., I. I. I., & Bhandari, M. (2005). Level of evidence in orthopaedic journals. Journal of Bone and Joint Surgery: Series A, 87(12 I), 2632–2638.

    Google Scholar 

  139. Ogden, T. L., & Bartley, D. L. (2008). The ups and downs of journal impact factors. Annals of Occupational Hygiene, 52(2), 73–82.

    Google Scholar 

  140. Owlia, P., Vasei, M., Goliaei, B., & Nassiri, I. (2011). Normalized impact factor (NIF): An adjusted method for calculating the citation rate of biomedical journals. Journal of Biomedical Informatics, 44, 216–220.

    Google Scholar 

  141. Patterson, M. (2009). Is the end in cite? EMBO reports, 10(11), 1186.

    MathSciNet  Google Scholar 

  142. Perneger, T. V. (2010). Citation analysis of identical consensus statements revealed journal-related bias. Journal of Clinical Epidemiology, 63(6), 660–664.

    Google Scholar 

  143. PLoS Medicine Editors. (2006). The impact factor game. PLoS Med, 3(6), e291. doi:10.1371/journal.pmed.0030291.

    Google Scholar 

  144. Poomkottayil, D., Bornstein, M. M., & Sendi, P. (2011). Lost in translation: The impact of publication language on citation frequency in the scientific dental literature. Swiss Medical Weekly, 141, w13148.

    Google Scholar 

  145. Porter, M., & McIntyre, S. (1984). What is, must be best: A research note on conservative or deferential responses to antenatal care provision. Social Science and Medicine, 19, 1197–1200.

    Google Scholar 

  146. Postma, E. (2007). Inflated impact factors? The true impact of evolutionary papers in non-evolutionary journals. PLoS ONE, 2(10), e999.

    Google Scholar 

  147. Preti, F., Dani, A., & Laio, F. (2010). Root profile assessment by means of hydrological, pedological and above-ground vegetation information for bio-engineering purposes. Ecological Engineering, 36, 305–316.

    Google Scholar 

  148. Pudovkin, A. I., & Garfield, E. (2004). Rank-normalized impact factor: A way to compare journal performance across subject categories. Proceedings of the ASIST Annual Meeting, 41, 507–515.

    Google Scholar 

  149. Racki, G. (2009). Rank-normalized journal impact factor as a predictive tool. Archivum Immunologiae et therapiae Experimentalis, 57, 39–43.

    Google Scholar 

  150. Ramsden, J. J. (2009). Impact factors: A critique. Journal of Biological Physics and Chemistry, 9, 139–140.

    Google Scholar 

  151. Reedijk, J., & Moed, H. F. (2008). Is the impact of journal impact factors decreasing? Journal of Documentation, 64, 183–192.

    Google Scholar 

  152. Reller, T. (2011). Elsevier announces 2010 journal impact factor highlights. Press release, 14 July 2011. http://www.marketwatch.com/story/elsevier-announces-2010-journal-impact-factor-highlights-2011-07-14. Accessed 2011 August 25.

  153. Resnik, D. B., Peddada, S., & Brunson, W., Jr. (2009). Research misconduct policies of scientific journals. Accountability in Research, 16, 254–267.

    Google Scholar 

  154. Resnik, D. B., Patrone, D., & Peddada, S. (2010). Research misconduct policies of social science journals and impact factor. Accountability in Research, 17, 79–84.

    Google Scholar 

  155. Rezaei-Ghaleh, N., & Azizi, F. (2007). The impact factor-based quality assessment of biomedical research institutes in Iran: Effect of impact factor normalization by subject. Archives of Iranian Medicine, 10, 182–189.

    Google Scholar 

  156. Rieseberg, L., & Smith, H. (2008). Editorial and Retrospective. Molecular Ecology, 17, 501–513.

    Google Scholar 

  157. Rieseberg, L., Vines, T., & Kane, N. (2011). 20 years of molecular ecology. Molecular Ecology, 20, 1–21.

    Google Scholar 

  158. Robert, C. P. (2010). On the relevance of the Bayesian approach to statistics. Review of Economic Analysis, 2, 139–152.

    Google Scholar 

  159. Roberts, J. (2009). An author’s guide to publication ethics: A review of emerging standards in biomedical journals. Headache, 49, 578–589.

    Google Scholar 

  160. Roosendaal, H. E., Geurts, P. A. T. M. (1997). Forces and functions in scientific communication: An analysis of their interplay. Cooperative Research Information Systems in Physics, August 31–September 4 1997, Oldenburg. http://www.physik.unioldenburg.de/conferences/crisp97/roosendaal.html.

  161. Rossner, M. (2006). How to guard against image fraud. The Scientist, 20(3), 24–25.

    Google Scholar 

  162. Rossner, M., Van Epps, H., & Hill, E. (2007). Show me the data. Journal of Cell Biology, 179, 1091–1092.

    Google Scholar 

  163. Rousseau, R. (2002). Journal evaluation: Technical and practical issues. Library Trends, 50, 418.

    Google Scholar 

  164. Rousseau, R., & Van Hooydonk, G. (1996). Journal production and journal impact factors. Journal of the American Society for Information Science, 47, 775–780.

    Google Scholar 

  165. Ruiz, M. A., Greco, O. T., & Braile, D. M. (2009). Journal impact factor: This editorial, academic and scientific influence [Fator de impacto: Importancia e influencia no meio editorial, academico e cientifico]. Brazilian Journal of Cardiovascular Surgery, 24, 273–278.

    Google Scholar 

  166. Saha, S., Saint, S., & Christakis, D. A. (2003). Impact factor: A valid measure of journal quality? Journal of the Medical Library Association, 91, 42–46.

    Google Scholar 

  167. Sample, I. (2002). People weigh less on a hard surface. New Scientist http://www.newscientist.com/article/dn2462-people-weigh-less-on-a-hard-surface.html.

  168. Saunders, R., & Savulescu, J. (2008). Research ethics and lessons from Hwanggate: what can we learn from the Korean cloning fraud? Journal of Medical Ethics, 34, 214–221.

    Google Scholar 

  169. Schoonbaert, D., & Roelants, G. (1996). Citation analysis for measuring the value of scientific publications: Quality assessment tool or comedy of errors? Tropical Medicine and International Health, 1, 739–752.

    Google Scholar 

  170. Schopfel, J., & Prost, H. (2009). Comparison of SCImago journal rank indicator (SJR) with JCR journal impact factor (IF) for French journals [Le JCR facteur d’impact (IF) et le SCImago journal rank indicator (SJR) des revues françaises: une étude comparative]. Psychologie Francaise, 54, 287–305.

    Google Scholar 

  171. Schubert, A., & Glanzel, W. (1983). Statistical reliability of comparisons based on the citation impact of scientific publications. Scientometrics, 5, 59–73.

    Google Scholar 

  172. Schumm, W. R. (2010). A comparison of citations across multidisciplinary psychology journals: A case study of two independent journals. Psychological Reports, 106, 314–322.

    Google Scholar 

  173. Scully, C., & Lodge, H. (2005). Impact factors and their significance; overrated or misused? British Dental Journal, 198, 391–393.

    Google Scholar 

  174. Seglen, P. O. (1989). Use of citation analysis and other bibliometric methods in evaluation of the quality of research [Bruk av siteringsanalyse og andre bibliometriske metoder i evaluering av forskningskvalitet]. Tidsskrift for den Norske Laegeforening, 109(31), 3224–3229.

    Google Scholar 

  175. Seglen, P. O. (1992). Journal impact: How representative is the journal impact factor? Research Evaluation, 2, 143–149.

    Google Scholar 

  176. Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. BMJ, 314, 497–502.

    Google Scholar 

  177. Simkin, M. V., & Roychowdhury, V. P. (2003). Read before you cite! Complex Syst, 14, 269–274.

    Google Scholar 

  178. Simons, K. (2008). The misused impact factor. Science, 322(5899), 165.

    Google Scholar 

  179. Skovsgaard, J. P., & Vanclay, J. K. (2008). Forest site productivity: A review of the evolution of dendrometric concepts for even-aged stands. Forestry, 81, 13–31.

    Google Scholar 

  180. Smith, D. R. (2007). Historical development of the journal impact factor and its relevance for occupational health. Industrial Health, 45, 730–742.

    Google Scholar 

  181. Sombatsompop, N., & Markpin, T. (2005). Making an equality of ISI impact factors for different subject fields. Journal of the American Society for Information Science and Technology, 56, 676–683.

    Google Scholar 

  182. Sombatsompop, N., Markpin, T., & Premkamolnetr, N. (2004). A modified method for calculating the impact factors of journals in ISI journal citation reports: Polymer science category in 1997–2001. Scientometrics, 60, 217–235.

    Google Scholar 

  183. Sonderstrup-Andersen, E. M., & Sonderstrup-Andersen, H. H. K. (2008). An investigation into diabetes researcher’s perceptions of the journal impact factor: Reconsidering evaluating research. Scientometrics, 76, 391–406.

    Google Scholar 

  184. Soreide, K., & Winter, D. C. (2010). Global survey of factors influencing choice of surgical journal for manuscript submission. Surgery, 147, 475–480.

    Google Scholar 

  185. Starbuck, W. H. (2005). How much better are the most-prestigious journals? The statistics of academic publication. Organization Science, 16, 180–200.

    Google Scholar 

  186. Statzner, B., & Resh, V. H. (2010). Negative changes in the scientific publication process in ecology: Potential causes and consequences. Freshwater Biology, 55, 2639–2653.

    Google Scholar 

  187. Stein, R. J., Haddock, C. K., Poston, W. S. C., Catanese, D., & Spertus, J. A. (2005). Precision in weighing: A comparison of scales found in physician offices, fitness centers, and weight loss centers. Public Health Reports, 120, 266–270.

    Google Scholar 

  188. Stiftel, B., & Mukhopadhyay, C. (2007). Thoughts on Anglo-American hegemony in planning scholarship: Do we read each other’s work? Town Planning Review, 78, 545–572.

    Google Scholar 

  189. Straub, D. W., & Anderson, C. (2009). Journal self-citation VI: Forced journal self-citation: Common, appropriate, ethical? Communications of the Association for Information Systems, 25, 57–66.

    Google Scholar 

  190. Tal, A., & Gordon, J. (2010). Carbon cautious: Israel’s afforestation experience and approach to sequestration. Small-Scale Forestry, 9, 409–428.

    Google Scholar 

  191. Taylor, M., Perakakis, P., & Trachana, V. (2008). The siege of science. Ethics in Science and Environmental Politics, 8, 17–40.

    Google Scholar 

  192. Thomson Reuters (2011). Journal citation reports. Thomson reuters products and services. http://thomsonreuters.com/products_services/science/science_products/a-z/journal_citation_reports/. Accessed 2011 August 16.

  193. Timuralp, B. (2010). Our first impact factor. Anadolu Kardiyoloji Dergisi, 10(4), 297.

    Google Scholar 

  194. Tobin, M. J. (2002). AJRCCM’s policy on duplicate publication. American Journal of Respiratory and Critical Care Medicine, 166, 433–437.

    Google Scholar 

  195. Todd, P. A., & Ladle, R. J. (2008). Hidden dangers of a ‘citation culture’. Ethics in Science and Environmental Politics, 8, 13–16.

    Google Scholar 

  196. Todd, P. A., Guest, J. R., Lu, J., & Chou, L. M. (2010). One in four citations in marine biology papers is inappropriate. Marine Ecology Progress Series, 408, 299–303.

    Google Scholar 

  197. Ugolini, D., Bogliolo, A., Parodi, S., Casilli, C., & Santi, L. (1997). Assessing research productivity in an oncology research institute: The role of the documentation center. Bulletin of the Medical Library Association, 85, 33–38.

    Google Scholar 

  198. Van de Sompel, H., Payette, S., Erickson, J., Lagoze, C., & Warner, S. (2004). Rethinking scholarly communication: Building the system that scholars deserve. D-Lib Magazine, 10, 9.

    Google Scholar 

  199. Van Driel, M., De Maeseneer, J., De Sutter, A., De Bacquer, D., De Backer, G., & Christiaens, T. (2008). How scientific is the assessment of the quality of scientific output using the journal impact factor? [Hoe wetenschappelijk is het beoordelen van wetenschappelijk werk aan de hand van impactfactoren van tijdschriften?]. Tijdschrift voor Geneeskunde, 64, 471–476.

    Google Scholar 

  200. Van Gyseghem, E., Baert, L., Van Remoortere, P., van ‘t Klooster, G., Rouan, M.-C., Voorspoels, J., de Kock, H., Schueller, L., Rosier, J., Grooten, L., & Van den Mooter, G. (2010) Co-administration of darunavir and a new pharmacokinetic booster: Formulation strategies and evaluation in dogs. European Journal of Pharmaceutical Sciences, 41, 193–200.

    Google Scholar 

  201. Van Leeuwen, T. N., Moed, H. F., & Reedijk, J. (1999). Critical comments on institute for scientific information impact factors: a sample of inorganic molecular chemistry journals. Journal of Information Science, 25(6), 189–198.

    Google Scholar 

  202. Vanclay, J. K. (2009). Bias in the journal impact factor. Scientometrics, 78, 3–12.

    Google Scholar 

  203. Vlosky, R. P., & Ozanne, L. K. (1997). Forest products certification: The business customer perspective. Wood and Fiber Science, 29, 195–208.

    Google Scholar 

  204. Wager, E., Fiack, S., Graf, C., Robinson, A., & Rowlands, I. (2009). Science journal editors’ views on publication ethics: Results of an international survey. Journal of Medical Ethics, 35, 348–353.

    Google Scholar 

  205. Wagner, A. B. (2009). Percentile-based journal impact factors: A neglected collection development metric. Issues in Science and Technology Librarianship 57.

  206. Walter, G., Bloch, S., Hunt, G., & Fisher, K. (2003). Counting on citations: A flawed way to measure quality. Medical Journal of Australia, 178(6), 280–281.

    Google Scholar 

  207. Wang, S., Wang, H., & Weldon, P. R. (2010). Effect of cooperation between Chinese scientific journals and international publishers on journals’ impact factor. Journal of Informetrics, 4, 233–238.

    Google Scholar 

  208. Weale, A. R., Bailey, M., & Lear, P. A. (2004). The level of non-citation of articles within a journal as a measure of quality: A comparison to the impact factor. BMC Medical Research Methodology, 4, 14.

    Google Scholar 

  209. Weller, A. C. (2001). Editorial peer review: Its strengths and weaknesses. American society for information science and technology monograph series, Information Today, Inc (p. 342). ISBN 9781573871006.

  210. Williams, G. (2007). Should we ditch impact factors? BMJ, 334, 568.

    Google Scholar 

  211. Winkmann, G., & Schweim, H. G. (2000). Biomedical databases and the journal impact factor [Medizinisch-biowissenschaftliche datenbanken und der impact-faktor]. Deutsche Medizinische Wochenschrift, 125, 1133–1141.

    Google Scholar 

  212. Winkmann, G., Schlutius, S., & Schweim, H. G. (2002). Publication languages of Impact factor journals and of medical bibliographic databanks [Publikationssprachen der Impact Faktor-zeitschriften und medizinischer literaturdatenbanken]. Deutsche Medizinische Wochenschrift, 127, 131–137.

    Google Scholar 

  213. Woelfel, J. (1993). Artificial neural networks in policy research: A current assessment. Journal of Communication, 43, 63–80.

    Google Scholar 

  214. Woelfel, J., Woelfel, J. (1997). ThoughtView version 2.0, Galileo Corporation.

  215. Woolgar, S. (1991). Beyond the citation debate: towards a sociology of measurement technologies and their use in science policy. Science and Public Policy, 18, 319–326.

    Google Scholar 

  216. Wulff, J. L., & Nixon, N. D. (2004). Quality markers and use of electronic journals in an academic health sciences library. Journal of the Medical Library Association, 92(3), 315–322.

    Google Scholar 

  217. Xiao, H., Yuan, F., & Wu, J.-G. (2009). Factors affecting citations: A comparison between Chinese and English journals in ecology. Chinese Journal of Applied Ecology, 20, 1253–1262.

    Google Scholar 

  218. Yu, G., & Wang, L. (2007). The self-cited rate of scientific journals and the manipulation of their impact factors. Scientometrics, 73, 321–330.

    Google Scholar 

  219. Yu, G., Yang, D.-H., & He, H.-X. (2011). An automatic recognition method of journal impact factor manipulation. Journal of Information Science, 37, 235–245.

    Google Scholar 

  220. Yue, W., Wilson, C. S., & Boller, F. (2007). Peer assessment of journal quality in clinical neurology. Journal of the Medical Library Association, 95, 70–76.

    Google Scholar 

  221. Zetterstrom, R. (1999). Impact factor and the future of Acta Paediatrica and other European medical journals. Acta Paediatrica, International Journal of Paediatrics, 88, 793–796.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jerome K. Vanclay.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Vanclay, J.K. Impact factor: outdated artefact or stepping-stone to journal certification?. Scientometrics 92, 211–238 (2012). https://doi.org/10.1007/s11192-011-0561-0

Download citation

Keywords

  • Thomson Reuters
  • ISI
  • JCR
  • Journal impact factor
  • Certification
  • Quality control

Mathematics Subject Classification (2000)

  • 94A17

JEL Classification

  • L86