Scientometrics

, Volume 53, Issue 2, pp 195–206 | Cite as

Advantages and limitations in the use of impact factor measures for the assessment of research performance

  • Maria Bordons
  • M. T. Fernández
  • Isabel Gómez

Abstract

Impact factor is a quasi-qualitative indicator, which provides a measurement of the prestige and international visibility of journals. Although the use of impact factor-based indicators for science policy purposes has increased over the last two decades, several limitations have been pointed out and should be borne in mind. The use of impact factor should be treated carefully when applied to the analysis of peripheral countries, whose national journals are hardly covered by ISI databases. Our experience in the use of impact factor based indicators for the analysis of the Spanish scientific production is shown. The usefulness of the impact factor measures in macro, meso and micro analyses is displayed. In addition, the main advantages, such as the great accessibility of impact factor and its ready-to-use nature are pointed out. Several limitations such as the need to avoid inter-field comparisons or the convenience of using a fixed journal set for international comparisons are also stressed. It is worth noting that the use of impact factor in the research evaluation process has influenced strongly the publication strategy of scientists.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Amin, M., Mabe, M. (2000), Impact factors: use and abuse, Perspectives in Publishing, 1: 1-6.MATHGoogle Scholar
  2. Arunachalam, K., Manorama, K. (1988), Are citation-based quantitative techniques adequate for measuring science on the periphery? Scientometrics, 15(5–6): 393-408.Google Scholar
  3. Bordons, M., BarrigÓn, S. (1992), Bibliometric analysis of publications of Spanish pharmacologist in the SCI (1984–89). Part II. Contribution to subfields other than “Pharmacology and Pharmacy”, Scientometrics, 25 (3): 425-446.CrossRefGoogle Scholar
  4. Bordons, M., Zulueta, M. A. (1997), Comparison of research team activity in two biomedical fields, Scientometrics, 40 (3): 423-436.CrossRefGoogle Scholar
  5. CamÍ, J., FernÁndez, M. T., GÓmez, I. (1993), Producción científica española en Biomedicina y Salud. Un estudio a través del Science Citation Index (1986–1989), Medicina Clínica (Barc) 101 (19): 721-731.Google Scholar
  6. CamÍ, J., Zulueta, M. A., FernÁndez, M. T., Bordons, M., GÓmez, I. (1997), Producción científica española en biomedicina y ciencias de la salud durante el período 1990–1993 (Science Citation Index y Social Science Citation Index) y comparación con el período 1986–1989, Medicina Clínica (Barc) 109: 481-496.Google Scholar
  7. Garfield, E. (1998), Long-term vs. short-term journal impact: Does it matter? The Scientist, 12 (3): 11-12.Google Scholar
  8. GÓmez, I. Bordons, M. (1996), Limitaciones en el uso de los indicadores bibliométricos para la evaluación científica, Política Científica, 46: 21-26.Google Scholar
  9. GÓmez, I., FernÁndez, M. T., Bordons, M., Cabrero, A., Morillo, F., Rojo, R. (2001), La actividad científica del CSIC a través del Science Citation Index, Social Science Citation Index y Arts & Humanities Citation Index. Estudio bibliométrico del período 1997–99. Report. CINDOC, Madrid. Institute for Scientific Information (2001) http://www.isinet.com/isi/hot/research/200012/3310412/index.hml Google Scholar
  10. LÓpez piÑero, J. M., Terrada, M. L. (1994), El consumo de información científica nacional y extranjera en las revistas médicas españolas: un nuevo repertorio destinado a su estudio, Medicina Clínica (Barc) 102: 104-112.Google Scholar
  11. Moed, H. F., Van Leeuwen, Th. N., Reedijk, J. (1998), A new classification system to describe the ageing of scientific journals and their impact factors, Journal of Documentation, 54 (4): 387-419.CrossRefGoogle Scholar
  12. Moed, H. F., Van Leeuwen, Th. N. (1995), Improving the accuracy of Institute for Scientific Information's journal impact factors, JASIS, 46 (6): 461-467.CrossRefGoogle Scholar
  13. Noma, E. (1986), Subject classification and influence weights for 3,000 journals. Report. Computer Horizons, Inc./CHI Research.Google Scholar
  14. Osareh, F., Wilson, C. S. (1995), Scientific productivity and impact of the third world countries: a citation study, In: Koenig, M. D. E., Bookstein, A. (Eds), Proceedings of the Fifth Biennial Conference of the International Society for Scientometrics and Informetrics. Learned Information, Medford.Google Scholar
  15. Seglen, P. O. (1992), How representative is the journal impact factor? Research Evaluation, 2 (3): 143-149.Google Scholar
  16. Sen, B. K., Karanjai, A., Munshi, U. M. (1989), A method for determining the impact factor of a non-SCI journal, Journal of Documentation, 45 (2): 139-141.Google Scholar
  17. Stegmann, J. (1999), Building a list of journals with constructed impact factors, Journal of Documentation, 55 (3): 310-324.CrossRefGoogle Scholar
  18. Solari, A., Magri, M. H. (2000), A new approach to the SCI Journal Citation Reports, a system for evaluating scientific journals, Scientometrics, 47 (3): 605-625.CrossRefGoogle Scholar
  19. Van Leeuwen, Th. N., Moed, H. F., Tijssen, R. J. W., Visser, M. S., van Raan, A. F. J. (2001), Language biases in the coverage of the Science Citation Index and its consequences for interntaional comparison of national research performance, Scientometrics, 51: 335-346.CrossRefGoogle Scholar
  20. Van Leeuwen, Th. N., Moed, H. F., Reedijk, J. (1999), Critical comments on Institute for Scientific Information impact factors: a sample of inorganic molecular chemistry journals, Journal of Information Science, 25 (6): 489-498.CrossRefGoogle Scholar
  21. Velho, L. (1986), The meaning of citation in the context of a scientifically peripheral country, Scientometrics, 9 (1–2): 71-89.CrossRefGoogle Scholar
  22. Zitt, M., Perrot, F., BarrÉ, R. (1998), The transition from “national” to “transnational” model and related measures of countries performances, JASIS, 49 (1): 30-42.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Maria Bordons
    • 1
  • M. T. Fernández
    • 2
  • Isabel Gómez
    • 1
  1. 1.Centro de Información y Documentación Científica (CINDOC-CSIC)Madrid
  2. 2.Centro de Información y Documentación Científica (CINDOC-CSIC)Spain

Personalised recommendations