, Volume 102, Issue 1, pp 1019–1035 | Cite as

Comparative analysis of some individual bibliometric indices when applied to groups of researchers

  • Ana Paula dos Santos Rubem
  • Ariane Lima de Moura
  • João Carlos Correia Baptista Soares de Mello


The h-index is a widely used bibliometric indicator for assessing individual scientists or other units of analysis. When evaluating aggregated authors, the h-index may produce rankings that are not consistent with the individual ones. The problem is claimed to affect all h-type indices; while the highly cited publications indicator, which comes from a different class, represents an alternative that is immune to such issue. The main objective of this work is to perform a comparative analysis of some bibliometric indicators originally designed to measure the overall impact of individual scientific production, when applied to the evaluation of groups, to investigate the consistency between the rankings at different levels of aggregation. For that, we use part of a previously reported citation database. The results indicate that, although the consistency at distinct aggregative levels is not formally complied by the h-index and all its variants, it is met with reasonable frequency.


h-Index h-Type indices Highly cited publications indicator Scientific performance 


  1. Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2010). hg-Index: A new index to characterize the scientific output of researchers based on the h- and g-indices. Scientometrics, 82(2), 391–400.CrossRefGoogle Scholar
  2. Anderson, T., Hankin, R., & Killworth, P. (2008). Beyond the Durfee square: Enhancing the h-index to score total publication output. Scientometrics, 76(3), 577–588.CrossRefGoogle Scholar
  3. Arencibia-Jorge, R., & Rousseau, R. (2009). Influence of individual researchers’ visibility on institutional impact: An example of Prathap’s approach to successive h-indices. Scientometrics, 79(3), 507–516.CrossRefGoogle Scholar
  4. Batista, P. D., Campiteli, M. G., & Kinouchi, O. (2006). Is it possible to compare researchers with different scientific interests? Scientometrics, 68(1), 179–189.CrossRefGoogle Scholar
  5. Bornmann, L., & Daniel, H.-D. (2007). What do we know about the h index? Journal of the American Society for Information Science and Technology, 58(9), 1381–1385.CrossRefGoogle Scholar
  6. Bornmann, L., & Daniel, H.-D. (2009). The state of h index research. Is the h index the ideal way to measure research performance? EMBO Reports, 10(1), 2–6.CrossRefGoogle Scholar
  7. Bornmann, L., Mutz, R., & Daniel, H.-D. (2008). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830–837.CrossRefGoogle Scholar
  8. Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H.-D. (2011). A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. Journal of Informetrics, 5(3), 346–359.CrossRefGoogle Scholar
  9. Bouyssou, D., & Marchant, T. (2014). An axiomatic approach to bibliometric rankings and indices. Journal of Informetrics, 8(3), 449–477.CrossRefGoogle Scholar
  10. Braun, T., Glänzel, W., & Schubert, A. (2006). A Hirsch-type index for journals. Scientometrics, 69(1), 169–173.Google Scholar
  11. Cabrerizo, F. J., Alonso, S., Herrera-Viedma, E., & Herrera, F. (2010). q2-Index: Quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core. Journal of Informetrics, 4(1), 23–28.CrossRefGoogle Scholar
  12. Csajbók, E., Berhidi, A., Vasas, L., & Schubert, A. (2007). Hirsch-index for countries based on Essential Science Indicators data. Scientometrics, 73(1), 91–117.CrossRefGoogle Scholar
  13. Deineko, V. G., & Woeginger, G. J. (2009). A new family of scientific impact measures: The generalized Kosmulski-indices. Scientometrics, 80(3), 819–828.CrossRefGoogle Scholar
  14. Egghe, L. (2006a). An improvement of the h-index: The g-index. ISSI Newsletter, 2(1), 8–9.MathSciNetGoogle Scholar
  15. Egghe, L. (2006b). Theory and practise of the g-index. Scientometrics, 69(1), 131–152.CrossRefMathSciNetGoogle Scholar
  16. Egghe, L. (2010). The Hirsch index and related impact measures. Annual Review of Information Science and Technology, 44, 65–114.CrossRefGoogle Scholar
  17. Egghe, L., & Rousseau, R. (2008). An h-index weighted by citation impact. Information Processing and Management, 44(2), 770–780.CrossRefGoogle Scholar
  18. Franceschini, F., & Maisano, D. A. (2010). Analysis of the Hirsch index’s operational properties. European Journal of Operational Research, 203(2), 494–504.CrossRefzbMATHGoogle Scholar
  19. Greco, S., Matarazzo, B., & Slowinski, R. (2001). Rough sets theory for multicriteria decision analysis. European Journal of Operational Research, 129(1), 1–47.CrossRefMathSciNetzbMATHGoogle Scholar
  20. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, 102(46), 16569–16572.CrossRefGoogle Scholar
  21. Hwang, Y.-A. (2013). An axiomatization of the Hirsch-index without adopting monotonicity. Applied Mathematics and Information Sciences. An International Journal, 7(4), 1317–1322.CrossRefGoogle Scholar
  22. Jin, B. (2006). H-index: An evaluation indicator proposed by scientist. Science Focus, 1(1), 8–9.Google Scholar
  23. Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R- and AR-indices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855–863.CrossRefGoogle Scholar
  24. Kinney, A. L. (2007). National scientific facilities and their science impact on non-biomedical research. Proceedings of the National Academy of Sciencies, 104(46), 17943–17947.CrossRefGoogle Scholar
  25. Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4–6.Google Scholar
  26. Kosmulski, M. (2007). MAXPROD—A new index for assessment of the scientific output of an individual, and a comparison. Cybermetrics, 11(1), 1–5.Google Scholar
  27. Marchant, T. (2009a). An axiomatic characterization of the ranking based on the h-index and some other bibliometric rankings of authors. Scientometrics, 80(2), 327–344.CrossRefGoogle Scholar
  28. Marchant, T. (2009b). Score-based bibliometric rankings of authors. Journal of the American Society for Information Science and Technology, 60(6), 1132–1137.CrossRefGoogle Scholar
  29. Miroiu, A. (2013). Axiomatizing the Hirsch index: Quantity and quality disjoined. Journal of Informetrics, 7(1), 10–15.CrossRefGoogle Scholar
  30. Norris, M., & Oppenheim, C. (2010). The h-index: A broad review of a new bibliometric indicator. Journal of Documentation, 66(5), 681–705.CrossRefGoogle Scholar
  31. Panaretos, J., & Malesios, C. (2009). Assessing scientific research performance and impact with single indices. Scientometrics, 81(3), 635–670.CrossRefGoogle Scholar
  32. Pomerol, J. C., & Barba-Romero, S. (2000). Multicriterion decision in management: Principles and practice (Vol. 25). New York: Springer.Google Scholar
  33. Prathap, G. (2006). Hirsch-type indices for ranking institutions’ scientific research output. Current Science, 91(11), 1439.Google Scholar
  34. Quesada, A. (2009). Monotonicity and the Hirsch index. Journal of Informetrics, 3(2), 158–160.CrossRefGoogle Scholar
  35. Quesada, A. (2010). More axiomatics for the Hirsch index. Scientometrics, 82(2), 413–418.CrossRefMathSciNetGoogle Scholar
  36. Quesada, A. (2011a). Axiomatics for the Hirsch index and the Egghe index. Journal of Informetrics, 5(3), 476–480.CrossRefGoogle Scholar
  37. Quesada, A. (2011b). Further characterizations of the Hirsch index. Scientometrics, 87(1), 107–114.CrossRefGoogle Scholar
  38. Roy, B., & Bouyssou, D. (1993). Aide Multicritère à la Décision: Méthodes et Cas. Paris: Economica.zbMATHGoogle Scholar
  39. Rubem, A. P. S., Moura, A. L. & Soares de Mello, J. C. C. B. (2013). Numerical analysis of some individual bibliometric indexes when applied to groups of researchers. In: XLV Simpósio Brasileiro de Pesquisa Operacional, SBPO 2013, Natal. Anais do XLV SBPO.Google Scholar
  40. Schreiber, M. (2008). A modification of the h-index: The hm-index accounts for multiauthored manuscripts. Journal of Informetrics, 2(3), 211–216.CrossRefMathSciNetGoogle Scholar
  41. Schreiber, M. (2010a). Revisiting the g-index: The average number of citations in the g-core. Journal of American Society for Information Science and Technology, 61(1), 169–174.CrossRefGoogle Scholar
  42. Schreiber, M. (2010b). Twenty Hirsch index variants and other indicators giving more or less preference to highly cited papers. Annalen der Physik, 522(8), 536–554.CrossRefMathSciNetGoogle Scholar
  43. Schubert, A. (2007). Successive h-indices. Scientometrics, 70(1), 201–205.CrossRefGoogle Scholar
  44. Thompson, D. F., Callen, E. C., & Nahata, M. C. (2009). New indices in scholarship assessment. American Journal of Pharmaceutical Education, 73(6), 111.CrossRefGoogle Scholar
  45. Tol, R. S. (2009). The h-index and its alternatives: An application to the 100 most prolific economists. Scientometrics, 80(2), 317–324.Google Scholar
  46. Van Eck, N. J., & Waltman, L. (2008). Generalizing the h- and g-indices. Journal of Informetrics, 2(4), 263–271.CrossRefGoogle Scholar
  47. Van Raan, A. F. J. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67(3), 491–502.CrossRefGoogle Scholar
  48. Vinkler, P. (2009). The π-index: A new indicator for assessing scientific impact. Journal of Information Science, 35(5), 602–612.Google Scholar
  49. Waltman, L., & Van Eck, N. J. (2012). The inconsistency of the h-index. Journal of the American Society for Information Science and Technology, 63(2), 406–415.CrossRefGoogle Scholar
  50. Woeginger, G. J. (2008a). An axiomatic characterization of the Hirsch-index. Mathematical Social Sciences, 56(2), 224–232.CrossRefMathSciNetzbMATHGoogle Scholar
  51. Woeginger, G. J. (2008b). A symmetry axiom for scientific impact indices. Journal of Informetrics, 2(4), 298–303.CrossRefMathSciNetGoogle Scholar
  52. Woeginger, G. J. (2008c). An axiomatic analysis of Egghe’s g-index. Journal of Informetrics, 2(4), 364–368.CrossRefMathSciNetGoogle Scholar
  53. Woeginger, G. J. (2009). Generalizations of Egghe’s g-index. Journal of the American Society for Information Science and Technology, 60(6), 1267–1273.CrossRefGoogle Scholar
  54. Wu, Q. (2010). The w-index: A measure to assess scientific impact by focusing on widely cited papers. Journal of the American Society for Information Science and Technology, 61(3), 609–614.Google Scholar
  55. Zhang, C.-T. (2009). The e-index, complementing the h-index for excess citations. PLoS One, 4(5), e5429. doi: 10.1371/journal.pone.0005429.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2014

Authors and Affiliations

  • Ana Paula dos Santos Rubem
    • 1
  • Ariane Lima de Moura
    • 2
  • João Carlos Correia Baptista Soares de Mello
    • 3
  1. 1.Centro de Análise de Sistemas Navais, Praça Barão de LadárioRio de JaneiroBrazil
  2. 2.Mestrado em Engenharia de ProduçãoUniversidade Federal FluminenseNiteróiBrazil
  3. 3.Departamento de Engenharia de ProduçãoUniversidade Federal FluminenseNiteróiBrazil

Personalised recommendations