Skip to main content
SpringerLink
Log in

Journal Impact Factors for evaluating scientific performance: use of h-like indicators

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

This article introduces the Impact Factor squared or IF2-index, an h-like indicator of research performance. This indicator reflects the degree to which large entities such as countries and/or their states participate in top-level research in a field or subfield. The IF2-index uses the Journal Impact Factor (JIF) of research publications instead of the number of citations. This concept is applied to other h-type indexes and their results compared to the IF2-index. These JIF-based indexes are then used to assess the overall performance of cancer research in Australia and its states over 8 years from 1999 to 2006. The IF2-index has three advantages when evaluating larger research units: firstly, it provides a stable value that does not change over time, reflecting the degree to which a research unit participated in top-level research in a given year; secondly, it can be calculated closely approximating the publication date of yearly datasets; and finally, it provides an additional dimension when a full article-based citation analysis is not feasible. As the index reflects the degree of participation in top-level research it may favor larger units when units of different sizes are compared.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. For a detailed description of the methodology, see Wilson et al. (2007), Welberry et al. (2008) or contact the authors directly for additional information.

  2. For our purpose we decided to allow real numbers for the h-like index and the h(2)-like index as they reflect the fact that IFs can be real numbers. Arguably fractions of publications do not exist and therefore the conservative criterion for both indexes had to be used to determine the number of publications. This is always the natural number before the decimal point; e.g., for the h-like index, a value of 17.808 represents 17 publications as the 18th publication does not fulfill the criterion of having an IF of 18 or more.

References

  • Arencibia-Jorge, R., Barrios-Almaguer, I., Fernández-Hernández, S., & Carvajal-Espino, R. (2008). Applying successive H indices in the institutional evaluation: A case study. Journal of the American Society for Information Science and Technology, 59(1), 155–157.

    Article  Google Scholar 

  • Bensman, J. S. (2007). Garfield and the impact factor. Annual Review of Information Science and Technology, 41, 93–155.

    Article  Google Scholar 

  • Bornmann, L., & Daniel, H.-D. (2007). What do we know about the h index? Journal of the American Society for Information Science and Technology, 58(9), 1381–1385.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., & Daniel, H.-D. (2008a). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830–837.

    Article  Google Scholar 

  • Bornmann, L., Wallon, G., & Ledin, A. (2008b). Is the h index related to (standard) bibliometric measures and to the assessments by peers? An investigation of the h index by using molecular life sciences data. Research Evaluation, 17(2), 149–156.

    Article  Google Scholar 

  • Braun, T., Glänzel, W., & Schubert, A. (2005). A Hirsch-type index for journals. The Scientist, 19(22), 8.

    Google Scholar 

  • Braun, T., Glänzel, W., & Schubert, A. (2006). A Hirsch-type index for journals. Scientometrics, 69(1), 167–173.

    Article  Google Scholar 

  • Burrell, Q. L. (2007a). Should the h-index be discounted? ISSI Newsletter, 3(S), 65–68.

    Google Scholar 

  • Burrell, Q. L. (2007b). On the h-index, the size of the Hirsch core and Jin’s A-index. Journal of Informetrics, 1(2), 170–177.

    Article  MathSciNet  Google Scholar 

  • Burrell, Q. L. (2009). On Hirsch’s h, Egghe’s g and Kosmulski’s h(2). Scientometrics. doi: 10.1007/s11192-009-0405-3 [Online First].

  • Costas, R., & Bordons, M. (2007). The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro-level. Journal of Informetrics, 1(3), 193–203.

    Article  Google Scholar 

  • Costas, R., & Bordons, M. (2008). Is g-index better than h-index? An exploratory study at the individual level. Scientometrics, 77(2), 267–288.

    Article  Google Scholar 

  • Cronin, B., & Meho, L. (2006). Using the h-index to rank influential information scientists. Journal of the American Society for Information Science and Technology, 57(9), 1275–1278.

    Article  Google Scholar 

  • Davies, H., Bignell, G. R., Cox, C., Stephens, P., Edkins, S., Clegg, S., et al. (2002). Mutations of the BRAF gene in human cancer. Nature, 417(6892), 949–954.

    Article  Google Scholar 

  • Egghe, L. (2006). Properties of the n-overlap vector and n-overlap similarity theory. Journal of the American Society for Information Science and Technology, 57(9), 1165–1177.

    Article  Google Scholar 

  • Egghe, L. (2007). Dynamic h-index: The Hirsch index in function of time. Journal of the American Society for Information Science and Technology, 58(3), 452–454.

    Article  Google Scholar 

  • Egghe, L. (2008). The influence of merging on h-type indices. Journal of Informetrics, 2(3), 252–262.

    Article  Google Scholar 

  • Egghe, L. (2010). The Hirsch index and related impact measures. Annual Review of Information Science and Technology, 44, 65–114.

    Google Scholar 

  • Geyer, C. E., Forster, J., Lindquist, D., Chan, S., Romieu, C. G., Pienkowski, T., et al. (2006). Lapatinib plus capecitabine for HER2-positive advanced breast cancer. The New England Journal of Medicine, 355(26), 2733–2743.

    Article  Google Scholar 

  • Glänzel, W. (2008). On some new bibliometric applications of statistics related to the h-index. Scientometrics, 77(1), 187–196.

    Article  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.

    Article  Google Scholar 

  • Jin, B. (2006). h-index: An evaluation indicator proposed by scientist. Science Focus, 1(1), 8–9.

    Google Scholar 

  • Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R- and ARindices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855–863.

    Article  Google Scholar 

  • Kaltenborn, K.-F., & Kuhn, K. (2004). The journal impact factor as a parameter for the evaluation of researchers and research. Revista Española de Enfermedades Digestivas, 96(7), 460–476. http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1130-01082004000700004&lng=en&nrm=iso [Online].

  • Ketcham, C. M., & Crawford, J. M. (2008). Can impact factor data be trusted? Laboratory Investigation, 88, 340–341.

    Article  Google Scholar 

  • Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4–6.

    Google Scholar 

  • Lee, K. P., Schotland, M., Bacchetti, P., & Bero, L. A. (2002). Association of journal quality indicators with methodological quality of clinical research articles. The Journal of the American Medical Association, 287(21), 2805–2808.

    Article  Google Scholar 

  • Lehrl, S. (1999). Der Impact-Faktor als Bewertungskriterium wissenschaftlicher Leistungen- das Recht auf Chancengleichheit. Strahlentherapie und Onkologie, 175(4), 141–153.

    Article  Google Scholar 

  • Molinari, J.-F., & Molinari, A. (2008). A new methodology for ranking scientific institutions. Scientometrics, 75(1), 163–174.

    Article  Google Scholar 

  • Oppenheim, C. (2007). Using the h-index to rank influential British researchers in information science and librarianship. Journal of the American Society for Information Science and Technology, 58(2), 297–301.

    Article  Google Scholar 

  • Opthof, T. (1997). Sense and nonsense about the impact factor. Cardiovascular Research, 33, 1–7.

    Article  Google Scholar 

  • Pires Da Luz, M., Marques-Portella, C., Mendlowicz, M., Gleiser, S., Silva Freire Coutinho, E., & Figueira, I. (2008). Institutional h-index: The performance of a new metric in the evaluation of Brazilian psychiatric post-graduation programs. Scientometrics, 77(2), 361–368.

    Article  Google Scholar 

  • Pringle, J. (2008). Trends in the use of ISI citation databases for evaluation. Learned Publishing, 21(2), 85–91.

    Article  Google Scholar 

  • Rossner, M., Van Epps, H., & Hill, E. (2007). Show me the data. Journal of Cell Biology, 179(6), 1091–1092.

    Article  Google Scholar 

  • Rossner, M., Van Epps, H., & Hill, E. (2008). Irreproducible results: A response to Thomson Scientific. Journal of Cell Biology, 180(2), 254–255.

    Article  Google Scholar 

  • Rousseau, R. (2006). New developments related to the Hirsch index. Accessed October 14, 2008, from http://eprints.rclis.org/archive/00006376/01/Hirsch_new_developments.pdf [Online].

  • Rousseau, R. (2008). Reflections on recent developments of the h-index and h-type indices. In H. Kretschmer & F. Havemann (Eds.), Proceedings of WIS 2008. Fourth International Conference on Webometrics, Informetrics and Scientometrics & Ninth COLLNET Meeting. Berlin: HumboldtUniversität zu Berlin, Institute for Library and Information Science. http://www.collnet.de/Berlin-2008/RousseauWIS2008rrd.pdf [Online].

  • Saha, S., Saint, S., & Christakis, D. A. (2003). Impact factor: A valid measure of journal quality? Journal of the Medical Library Association, 91(1), 42–46.

    Google Scholar 

  • Schreiber, M. (2008). An empirical investigation of the g-index for 26 physicists in comparison with the h-Index, the A-index, and the R-index. Journal of the American Society for Information Science and Technology, 59(9), 1513–1522.

    Article  Google Scholar 

  • Schubert, A. (2009). Using the h-index for assessing single publications. Scientometrics, 78(3), 559–565.

    Article  Google Scholar 

  • Schubert, A., & Glänzel, W. (2007). A systematic analysis of Hirsch-type indices for journals. Journal of Informetrics, 1(3), 179–184.

    Article  Google Scholar 

  • Schubert, A., Korn, A., & Telcs, A. (2009). Hirsch-type indices for characterizing network. Scientometrics, 78(2), 375–382.

    Article  Google Scholar 

  • Seglen, P. O. (1994). Causal relationship between article citedness and journal impact. Journal of the American Society for Information Science, 45(1), 1–11.

    Article  Google Scholar 

  • Van Raan, A. F. J. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67(3), 491–502.

    Google Scholar 

  • Vanclay, J. K. (2007). On the robustness of the h-index. Journal of the American Society for Information Science and Technology, 58(10), 1547–1550.

    Article  Google Scholar 

  • Vanclay, J. K. (2008). Ranking forestry journals using the h-index. Journal of Informetrics, 2(4), 326–334.

    Article  Google Scholar 

  • Welberry, H., Edwards, C., Weston, A., Harvey, C., Wilson, C. S., Boell, S. K., Lo, M., & Bishop, J. K. (2008). Cancer research in New South Wales 2001–2006. Sydney: Cancer Institute NSW. http://www.cancerinstitute.org.au/cancer_inst/publications/pdfs/rm-2008-1_cancer-research-in-nsw-2001-2006.pdf [Online].

  • Wilson, C. S. (2005). General analyses of cancer research publications in Australian states using the science and social science citations indexes. In P. Ingwersen & B. Larsen (Eds.), Proceedings of ISSI 2005 (pp. 168–176). Stockholm: Karolinska University Press.

    Google Scholar 

  • Wilson, C. S., Boell, S. K., & Lo, M. (2007). Systematic review of the quantity and quality of cancer research publications in New South Wales: 1999 to 2006. Sydney: University of New South Wales, School of Information Systems, Technology and Management.

  • Wilson, C. S., & Pittman, S. (2000). Assessments of outputs: Quantity and quality of cancer research publications in New South Wales from 1994 to 1998. Sydney: The University of New South Wales, School of Information Systems, Technology and Management.

    Google Scholar 

  • Yue, W., Wilson, C. S., & Boller, F. (2007). Peer assessment of journal quality in clinical neurology. Journal of the Medical Library Association, 95(1), 70–76.

    Google Scholar 

Download references

Acknowledgements

The authors acknowledge with thanks the support of the John Metcalfe Memorial Fund and helpful comments from the anonymous reviewers.

Author information

Authors and Affiliations

  1. School of Information Systems, Technology and Management (SISTM), University of New South Wales, Sydney, NSW, 2052, Australia

    Sebastian K. Boell & Concepción S. Wilson

Authors
  1. Sebastian K. Boell
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Concepción S. Wilson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sebastian K. Boell.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Boell, S.K., Wilson, C.S. Journal Impact Factors for evaluating scientific performance: use of h-like indicators. Scientometrics 82, 613–626 (2010). https://doi.org/10.1007/s11192-010-0175-y

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-010-0175-y

Keywords

Search

Navigation