Scientometrics

, Volume 82, Issue 3, pp 613–626 | Cite as

Journal Impact Factors for evaluating scientific performance: use of h-like indicators

Article

Abstract

This article introduces the Impact Factor squared or IF2-index, an h-like indicator of research performance. This indicator reflects the degree to which large entities such as countries and/or their states participate in top-level research in a field or subfield. The IF2-index uses the Journal Impact Factor (JIF) of research publications instead of the number of citations. This concept is applied to other h-type indexes and their results compared to the IF2-index. These JIF-based indexes are then used to assess the overall performance of cancer research in Australia and its states over 8 years from 1999 to 2006. The IF2-index has three advantages when evaluating larger research units: firstly, it provides a stable value that does not change over time, reflecting the degree to which a research unit participated in top-level research in a given year; secondly, it can be calculated closely approximating the publication date of yearly datasets; and finally, it provides an additional dimension when a full article-based citation analysis is not feasible. As the index reflects the degree of participation in top-level research it may favor larger units when units of different sizes are compared.

Keywords

Journal Impact Factor IF2-index Research evaluation Australia Cancer research h-Index h-Type indexes h-Like indexes Scientometrics 

References

  1. Arencibia-Jorge, R., Barrios-Almaguer, I., Fernández-Hernández, S., & Carvajal-Espino, R. (2008). Applying successive H indices in the institutional evaluation: A case study. Journal of the American Society for Information Science and Technology, 59(1), 155–157.CrossRefGoogle Scholar
  2. Bensman, J. S. (2007). Garfield and the impact factor. Annual Review of Information Science and Technology, 41, 93–155.CrossRefGoogle Scholar
  3. Bornmann, L., & Daniel, H.-D. (2007). What do we know about the h index? Journal of the American Society for Information Science and Technology, 58(9), 1381–1385.CrossRefGoogle Scholar
  4. Bornmann, L., Mutz, R., & Daniel, H.-D. (2008a). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830–837.CrossRefGoogle Scholar
  5. Bornmann, L., Wallon, G., & Ledin, A. (2008b). Is the h index related to (standard) bibliometric measures and to the assessments by peers? An investigation of the h index by using molecular life sciences data. Research Evaluation, 17(2), 149–156.CrossRefGoogle Scholar
  6. Braun, T., Glänzel, W., & Schubert, A. (2005). A Hirsch-type index for journals. The Scientist, 19(22), 8.Google Scholar
  7. Braun, T., Glänzel, W., & Schubert, A. (2006). A Hirsch-type index for journals. Scientometrics, 69(1), 167–173.CrossRefGoogle Scholar
  8. Burrell, Q. L. (2007a). Should the h-index be discounted? ISSI Newsletter, 3(S), 65–68.Google Scholar
  9. Burrell, Q. L. (2007b). On the h-index, the size of the Hirsch core and Jin’s A-index. Journal of Informetrics, 1(2), 170–177.CrossRefMathSciNetGoogle Scholar
  10. Burrell, Q. L. (2009). On Hirsch’s h, Egghe’s g and Kosmulski’s h(2). Scientometrics. doi: 10.1007/s11192-009-0405-3 [Online First].
  11. Costas, R., & Bordons, M. (2007). The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro-level. Journal of Informetrics, 1(3), 193–203.CrossRefGoogle Scholar
  12. Costas, R., & Bordons, M. (2008). Is g-index better than h-index? An exploratory study at the individual level. Scientometrics, 77(2), 267–288.CrossRefGoogle Scholar
  13. Cronin, B., & Meho, L. (2006). Using the h-index to rank influential information scientists. Journal of the American Society for Information Science and Technology, 57(9), 1275–1278.CrossRefGoogle Scholar
  14. Davies, H., Bignell, G. R., Cox, C., Stephens, P., Edkins, S., Clegg, S., et al. (2002). Mutations of the BRAF gene in human cancer. Nature, 417(6892), 949–954.CrossRefGoogle Scholar
  15. Egghe, L. (2006). Properties of the n-overlap vector and n-overlap similarity theory. Journal of the American Society for Information Science and Technology, 57(9), 1165–1177.CrossRefGoogle Scholar
  16. Egghe, L. (2007). Dynamic h-index: The Hirsch index in function of time. Journal of the American Society for Information Science and Technology, 58(3), 452–454.CrossRefGoogle Scholar
  17. Egghe, L. (2008). The influence of merging on h-type indices. Journal of Informetrics, 2(3), 252–262.CrossRefGoogle Scholar
  18. Egghe, L. (2010). The Hirsch index and related impact measures. Annual Review of Information Science and Technology, 44, 65–114.Google Scholar
  19. Geyer, C. E., Forster, J., Lindquist, D., Chan, S., Romieu, C. G., Pienkowski, T., et al. (2006). Lapatinib plus capecitabine for HER2-positive advanced breast cancer. The New England Journal of Medicine, 355(26), 2733–2743.CrossRefGoogle Scholar
  20. Glänzel, W. (2008). On some new bibliometric applications of statistics related to the h-index. Scientometrics, 77(1), 187–196.CrossRefGoogle Scholar
  21. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.CrossRefGoogle Scholar
  22. Jin, B. (2006). h-index: An evaluation indicator proposed by scientist. Science Focus, 1(1), 8–9.Google Scholar
  23. Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R- and ARindices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855–863.CrossRefGoogle Scholar
  24. Kaltenborn, K.-F., & Kuhn, K. (2004). The journal impact factor as a parameter for the evaluation of researchers and research. Revista Española de Enfermedades Digestivas, 96(7), 460–476. http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1130-01082004000700004&lng=en&nrm=iso [Online].
  25. Ketcham, C. M., & Crawford, J. M. (2008). Can impact factor data be trusted? Laboratory Investigation, 88, 340–341.CrossRefGoogle Scholar
  26. Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4–6.Google Scholar
  27. Lee, K. P., Schotland, M., Bacchetti, P., & Bero, L. A. (2002). Association of journal quality indicators with methodological quality of clinical research articles. The Journal of the American Medical Association, 287(21), 2805–2808.CrossRefGoogle Scholar
  28. Lehrl, S. (1999). Der Impact-Faktor als Bewertungskriterium wissenschaftlicher Leistungen- das Recht auf Chancengleichheit. Strahlentherapie und Onkologie, 175(4), 141–153.CrossRefGoogle Scholar
  29. Molinari, J.-F., & Molinari, A. (2008). A new methodology for ranking scientific institutions. Scientometrics, 75(1), 163–174.CrossRefGoogle Scholar
  30. Oppenheim, C. (2007). Using the h-index to rank influential British researchers in information science and librarianship. Journal of the American Society for Information Science and Technology, 58(2), 297–301.CrossRefGoogle Scholar
  31. Opthof, T. (1997). Sense and nonsense about the impact factor. Cardiovascular Research, 33, 1–7.CrossRefGoogle Scholar
  32. Pires Da Luz, M., Marques-Portella, C., Mendlowicz, M., Gleiser, S., Silva Freire Coutinho, E., & Figueira, I. (2008). Institutional h-index: The performance of a new metric in the evaluation of Brazilian psychiatric post-graduation programs. Scientometrics, 77(2), 361–368.CrossRefGoogle Scholar
  33. Pringle, J. (2008). Trends in the use of ISI citation databases for evaluation. Learned Publishing, 21(2), 85–91.CrossRefGoogle Scholar
  34. Rossner, M., Van Epps, H., & Hill, E. (2007). Show me the data. Journal of Cell Biology, 179(6), 1091–1092.CrossRefGoogle Scholar
  35. Rossner, M., Van Epps, H., & Hill, E. (2008). Irreproducible results: A response to Thomson Scientific. Journal of Cell Biology, 180(2), 254–255.CrossRefGoogle Scholar
  36. Rousseau, R. (2006). New developments related to the Hirsch index. Accessed October 14, 2008, from http://eprints.rclis.org/archive/00006376/01/Hirsch_new_developments.pdf [Online].
  37. Rousseau, R. (2008). Reflections on recent developments of the h-index and h-type indices. In H. Kretschmer & F. Havemann (Eds.), Proceedings of WIS 2008. Fourth International Conference on Webometrics, Informetrics and Scientometrics & Ninth COLLNET Meeting. Berlin: HumboldtUniversität zu Berlin, Institute for Library and Information Science. http://www.collnet.de/Berlin-2008/RousseauWIS2008rrd.pdf [Online].
  38. Saha, S., Saint, S., & Christakis, D. A. (2003). Impact factor: A valid measure of journal quality? Journal of the Medical Library Association, 91(1), 42–46.Google Scholar
  39. Schreiber, M. (2008). An empirical investigation of the g-index for 26 physicists in comparison with the h-Index, the A-index, and the R-index. Journal of the American Society for Information Science and Technology, 59(9), 1513–1522.CrossRefGoogle Scholar
  40. Schubert, A. (2009). Using the h-index for assessing single publications. Scientometrics, 78(3), 559–565.CrossRefGoogle Scholar
  41. Schubert, A., & Glänzel, W. (2007). A systematic analysis of Hirsch-type indices for journals. Journal of Informetrics, 1(3), 179–184.CrossRefGoogle Scholar
  42. Schubert, A., Korn, A., & Telcs, A. (2009). Hirsch-type indices for characterizing network. Scientometrics, 78(2), 375–382.CrossRefGoogle Scholar
  43. Seglen, P. O. (1994). Causal relationship between article citedness and journal impact. Journal of the American Society for Information Science, 45(1), 1–11.CrossRefGoogle Scholar
  44. Van Raan, A. F. J. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67(3), 491–502.Google Scholar
  45. Vanclay, J. K. (2007). On the robustness of the h-index. Journal of the American Society for Information Science and Technology, 58(10), 1547–1550.CrossRefGoogle Scholar
  46. Vanclay, J. K. (2008). Ranking forestry journals using the h-index. Journal of Informetrics, 2(4), 326–334.CrossRefGoogle Scholar
  47. Welberry, H., Edwards, C., Weston, A., Harvey, C., Wilson, C. S., Boell, S. K., Lo, M., & Bishop, J. K. (2008). Cancer research in New South Wales 2001–2006. Sydney: Cancer Institute NSW. http://www.cancerinstitute.org.au/cancer_inst/publications/pdfs/rm-2008-1_cancer-research-in-nsw-2001-2006.pdf [Online].
  48. Wilson, C. S. (2005). General analyses of cancer research publications in Australian states using the science and social science citations indexes. In P. Ingwersen & B. Larsen (Eds.), Proceedings of ISSI 2005 (pp. 168–176). Stockholm: Karolinska University Press.Google Scholar
  49. Wilson, C. S., Boell, S. K., & Lo, M. (2007). Systematic review of the quantity and quality of cancer research publications in New South Wales: 1999 to 2006. Sydney: University of New South Wales, School of Information Systems, Technology and Management.Google Scholar
  50. Wilson, C. S., & Pittman, S. (2000). Assessments of outputs: Quantity and quality of cancer research publications in New South Wales from 1994 to 1998. Sydney: The University of New South Wales, School of Information Systems, Technology and Management.Google Scholar
  51. Yue, W., Wilson, C. S., & Boller, F. (2007). Peer assessment of journal quality in clinical neurology. Journal of the Medical Library Association, 95(1), 70–76.Google Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2010

Authors and Affiliations

  1. 1.School of Information Systems, Technology and Management (SISTM)University of New South WalesSydneyAustralia

Personalised recommendations