Scientometrics

, Volume 109, Issue 3, pp 1815–1833 | Cite as

Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators

Article

Abstract

The h-index, introduced by Hirsch in 2005, was used by Schubert in 2009 to assess single publications. In 2011, Bornmann, Schier, Marx, and Daniel confirmed that the h-index is effective when assessing papers in chemistry. Quite a few Hirsch-type indices originate from the h-index. Can these Hirsch-type indices also be effectively used for assessing single publications? Will they behave the same or differently? In this study, the research objects were 26 kinds of Hirsch-type indices (including the original h-index) and three traditional methods, a total of 29 indicators. Based on the original definitions of these indicators and our new explanations of generations (i.e. mixed, pure, and non-pure generations of citations), we defined/redefined 29 paper-level metrics, calculated their values to assess publications, considered the correlations between those indices and the h-index or Wu’s w-index, and did factor analysis to contrast effectiveness. It was found that a few Hirsch-type indices (i.e. the f-index, rational h-index, real h-index, j-index, hg-index, Woeginger’s w-index, and tapered h-index) are highly correlated with the h-index but not close to Wu’s w-index, while some other indices (i.e. the a-index, h(5,2)-index, q2-index, r-index, maxprod, e-index, p-index, and weighted h-index) have relatively low correlations with the h-index but are close to Wu’s w-index. The normalized h-index and ph-ratio are obviously different from the other indices, and in most cases, their correlation coefficients with the h-index or Wu’s w-index are statistically non-significant (p > .05) or negative significant (p < .01). We argue that indices which are neither too near to nor too far from the h-index could be much more promising than others.

Keywords

Hirsch-type indices Single publication H-index Paper-level metrics Reference networks Citation analysis 

Notes

Acknowledgments

The authors would like to acknowledge the helpful comments of the anonymous reviewer. This research was supported in part by the National Natural Science Foundation of China (Grants 71273250 and 70973117).

References

  1. Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2010). hg-index: A new index to characterize the scientific output of researchers based on the h- and g-indices. Scientometrics, 82(2), 391–400.CrossRefGoogle Scholar
  2. Anderson, T., Hankin, R., & Killworth, P. (2008). Beyond the Durfee square: Enhancing the h-index to score total publication output. Scientometrics, 76(3), 577–588.CrossRefGoogle Scholar
  3. Banks, M. G. (2006). An extension of the Hirsch index: Indexing scientific topics and compounds. Scientometrics, 69(1), 161–168.CrossRefGoogle Scholar
  4. Bornmann, L., Mutz, R., & Daniel, H. D. (2008). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830–837.CrossRefGoogle Scholar
  5. Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H. D. (2011a). A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. Journal of Informetrics, 5(3), 346–359.CrossRefGoogle Scholar
  6. Bornmann, L., Schier, H., Marx, W., & Daniel, H. D. (2011b). Does the h index for assessing single publications really work? A case study on papers published in chemistry. Scientometrics, 89(3), 835–843.CrossRefGoogle Scholar
  7. Braun, T., Glänzel, W., & Schubert, A. (2005). A Hirsch-type index for journals. The Scientist, 19(22), 8–10.Google Scholar
  8. Cabrerizo, F. J., Alonso, S., Herrera-Viedma, E., & Herrera, F. (2010). q2-Index: Quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core. Journal of Informetrics, 4(1), 23–28.CrossRefGoogle Scholar
  9. Chan, H. F., Frey, B. S., Gallus, J., Schaffner, M., Torgler, B., & Whyte, S. (2016). External influence as an indicator of scholarly importance. CESifo Economic Studies, 62(1), 170–195.CrossRefGoogle Scholar
  10. Egghe, L. (2006a). An improvement of the h-index: The g-index. ISSI Newsletter, 2(1), 8–9.MathSciNetGoogle Scholar
  11. Egghe, L. (2006b). Theory and practise of the g-index. Scientometrics, 69(1), 131–152.MathSciNetCrossRefGoogle Scholar
  12. Egghe, L. (2010). On the relation between Schubert’s h-index of a single paper and its total number of received citations. Scientometrics, 84(1), 115–117.CrossRefGoogle Scholar
  13. Egghe, L., & Rousseau, R. (2008). An h-index weighted by citation impact. Information Processing and Management, 44(2), 770–780.CrossRefGoogle Scholar
  14. Ellison, G. (2010). How does the market use citation data? The Hirsch index in economics. National Bureau of Economic Research. http://www.nber.org/papers/w16419. Accessed October 8, 2010.
  15. Fragkiadaki, E., & Evangelidis, G. (2016). Three novel indirect indicators for the assessment of papers and authors based on generations of citations. Scientometrics, 106(2), 657–694.CrossRefGoogle Scholar
  16. Glänzel, W., & Schubert, A. (2010). Hirsch-type characteristics of the tail of distributions. The generalised h-index. Journal of Informetrics, 4(1), 118–123.CrossRefGoogle Scholar
  17. Guns, R., & Rousseau, R. (2009). Real and rational variants of the h-index and the g-index. Journal of Informetrics, 3(1), 64–71.CrossRefGoogle Scholar
  18. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.CrossRefMATHGoogle Scholar
  19. Hu, X., Rousseau, R., & Chen, J. (2011). On the definition of forward and backward citation generations. Journal of Informetrics, 5(1), 27–36.CrossRefGoogle Scholar
  20. Jacsó, P. (2009). The h-index for countries in Web of Science and Scopus. Online Information Review, 33(4), 831–837.CrossRefGoogle Scholar
  21. Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R- and AR-indices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855–863.CrossRefGoogle Scholar
  22. Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4–6.Google Scholar
  23. Kosmulski, M. (2007). MAXPROD-A new index for assessment of the scientific output of an individual, and a comparison. Cybermetrics , 11(1), 1–5.Google Scholar
  24. Kosmulski, M. (2013). Family-tree of bibliometric indices. Journal of Informetrics, 7(2), 313–317.CrossRefGoogle Scholar
  25. Lacasse, J. R., Hodge, D. R., & Bean, K. F. (2011). Evaluating the productivity of social work scholars using the h-Index. Research on Social Work Practice, 21(5), 599–607.CrossRefGoogle Scholar
  26. Mingers, J. (2009). Measuring the research contribution of management academics using the Hirsch-index. Journal of the Operational Research Society, 60(9), 1143–1153.CrossRefGoogle Scholar
  27. Molinari, A., & Molinari, J. F. (2008). Mathematical aspects of a new criterion for ranking scientific institutions based on the h-index. Scientometrics, 75(2), 339–356.CrossRefGoogle Scholar
  28. Mugnaini, R., Packer, A. L., & Meneghini, R. (2008). Comparison of scientists of the Brazilian Academy of Sciences and of the National Academy of Sciences of the USA on the basis of the h-index. Brazilian Journal of Medical and Biological Research, 41(4), 258–262.CrossRefGoogle Scholar
  29. Navon, D. (2009). The h-i index: A proposed new metric of individual scientific output. Cybermetrics, 13(1), paper 3.Google Scholar
  30. Olden, J. D. (2007). How do ecological journals stack-up? Ranking of scientific quality according to the h index. Ecoscience, 14(3), 370–376.CrossRefGoogle Scholar
  31. Panaretos, J., & Malesios, C. (2009). Assessing scientific research performance and impact with single indices. Scientometrics, 81(3), 635–670.CrossRefGoogle Scholar
  32. Prathap, G. (2010a). The 100 most prolific economists using the p-index. Scientometrics, 84(1), 167–172.CrossRefGoogle Scholar
  33. Prathap, G. (2010b). Is there a place for a mock h-index? Scientometrics, 84(1), 153–165.CrossRefGoogle Scholar
  34. Rosenberg, M. S. (2014). A biologist’s guide to impact factors. PeerJ PrePrints 2: e477v1. doi: 10.7287/peerj.preprints.477v1.
  35. Rousseau, R. (1987). The Gozinto theorem: Using citations to determine influences on a scientific publication. Scientometrics, 11(3–4), 217–229.CrossRefGoogle Scholar
  36. Rousseau, R. (2006). New developments related to the Hirsch index. Science Focus, 1(4), 23–25. (in Chinese).Google Scholar
  37. Ruane, F., & Tol, R. S. J. (2008). Rational (successive) h-indices: An application to economics in the Republic of Ireland. Scientometrics, 75(2), 395–405.CrossRefGoogle Scholar
  38. Schreiber, M. (2010). Twenty Hirsch index variants and other indicators giving more or less preference to highly cited papers. Annalen der Physik, 522(8), 536–554.CrossRefGoogle Scholar
  39. Schubert, A. (2007). Successive h-indices. Scientometrics, 70(1), 201–205.CrossRefGoogle Scholar
  40. Schubert, A. (2009). Using the h-index for assessing single publications. Scientometrics, 78(3), 559–565.CrossRefGoogle Scholar
  41. Sidiropoulos, A., Katsaros, D., & Manolopoulos, Y. (2007). Generalized Hirsch h-index for disclosing latent facts in citation networks. Scientometrics, 72(2), 253–280.CrossRefGoogle Scholar
  42. Thor, A., & Bornmann, L. (2011). The calculation of the single publicationhindex and related performance measures. Online Information Review, 35(2), 291–300.CrossRefGoogle Scholar
  43. Todeschini, R. (2011). The j-index: A new bibliometric index and multivariate comparisons between other common indices. Scientometrics, 87(3), 621–639.CrossRefGoogle Scholar
  44. Tol, R. S. J. (2009). The h-index and its alternatives: An application to the 100 most prolific economists. Scientometrics, 80(2), 317–324.CrossRefGoogle Scholar
  45. Turaga, K. K., & Gamblin, T. C. (2012). Measuring the surgical academic output of an institution: The “institutional” h-Index. Journal of Surgical Education, 69(4), 499–503.CrossRefGoogle Scholar
  46. Vinkler, P. (2009). The π-index: a new indicator for assessing scientific impact. Journal of Information Science, 35(5), 602–612.CrossRefGoogle Scholar
  47. Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365–391.MathSciNetCrossRefGoogle Scholar
  48. Watkins, M. W., & Chan-Park, C. Y. (2015). The research impact of school psychology faculty. Journal of School Psychology, 53(3), 231–241.CrossRefGoogle Scholar
  49. Wildgaard, L., Schneider, J. W., & Larsen, B. (2014). A review of the characteristics of 108 author-level bibliometric indicators. Scientometrics, 101(1), 125–158.CrossRefGoogle Scholar
  50. Woeginger, G. J. (2008). An axiomatic characterization of the Hirsch-index. Mathematical Social Sciences, 56(2), 224–232.MathSciNetCrossRefMATHGoogle Scholar
  51. Wu, Q. (2010). The w-index: A measure to assess scientific impact by focusing on widely cited papers. Journal of the American Society for Information Science and Technology, 61(3), 609–614.Google Scholar
  52. Zhang, C.-T. (2009). The e-index, complementing the h-index for excess citations. PLoS One, 4(5), e5429.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2016

Authors and Affiliations

  1. 1.School of ManagementUniversity of Science and Technology of ChinaHefeiChina

Personalised recommendations