Skip to main content
SpringerLink
Log in

Comparative analysis of some individual bibliometric indices when applied to groups of researchers

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

The h-index is a widely used bibliometric indicator for assessing individual scientists or other units of analysis. When evaluating aggregated authors, the h-index may produce rankings that are not consistent with the individual ones. The problem is claimed to affect all h-type indices; while the highly cited publications indicator, which comes from a different class, represents an alternative that is immune to such issue. The main objective of this work is to perform a comparative analysis of some bibliometric indicators originally designed to measure the overall impact of individual scientific production, when applied to the evaluation of groups, to investigate the consistency between the rankings at different levels of aggregation. For that, we use part of a previously reported citation database. The results indicate that, although the consistency at distinct aggregative levels is not formally complied by the h-index and all its variants, it is met with reasonable frequency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2010). hg-Index: A new index to characterize the scientific output of researchers based on the h- and g-indices. Scientometrics, 82(2), 391–400.

    Article  Google Scholar 

  • Anderson, T., Hankin, R., & Killworth, P. (2008). Beyond the Durfee square: Enhancing the h-index to score total publication output. Scientometrics, 76(3), 577–588.

    Article  Google Scholar 

  • Arencibia-Jorge, R., & Rousseau, R. (2009). Influence of individual researchers’ visibility on institutional impact: An example of Prathap’s approach to successive h-indices. Scientometrics, 79(3), 507–516.

    Article  Google Scholar 

  • Batista, P. D., Campiteli, M. G., & Kinouchi, O. (2006). Is it possible to compare researchers with different scientific interests? Scientometrics, 68(1), 179–189.

    Article  Google Scholar 

  • Bornmann, L., & Daniel, H.-D. (2007). What do we know about the h index? Journal of the American Society for Information Science and Technology, 58(9), 1381–1385.

    Article  Google Scholar 

  • Bornmann, L., & Daniel, H.-D. (2009). The state of h index research. Is the h index the ideal way to measure research performance? EMBO Reports, 10(1), 2–6.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., & Daniel, H.-D. (2008). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830–837.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H.-D. (2011). A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. Journal of Informetrics, 5(3), 346–359.

    Article  Google Scholar 

  • Bouyssou, D., & Marchant, T. (2014). An axiomatic approach to bibliometric rankings and indices. Journal of Informetrics, 8(3), 449–477.

    Article  Google Scholar 

  • Braun, T., Glänzel, W., & Schubert, A. (2006). A Hirsch-type index for journals. Scientometrics, 69(1), 169–173.

    Google Scholar 

  • Cabrerizo, F. J., Alonso, S., Herrera-Viedma, E., & Herrera, F. (2010). q2-Index: Quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core. Journal of Informetrics, 4(1), 23–28.

    Article  Google Scholar 

  • Csajbók, E., Berhidi, A., Vasas, L., & Schubert, A. (2007). Hirsch-index for countries based on Essential Science Indicators data. Scientometrics, 73(1), 91–117.

    Article  Google Scholar 

  • Deineko, V. G., & Woeginger, G. J. (2009). A new family of scientific impact measures: The generalized Kosmulski-indices. Scientometrics, 80(3), 819–828.

    Article  Google Scholar 

  • Egghe, L. (2006a). An improvement of the h-index: The g-index. ISSI Newsletter, 2(1), 8–9.

    MathSciNet  Google Scholar 

  • Egghe, L. (2006b). Theory and practise of the g-index. Scientometrics, 69(1), 131–152.

    Article  MathSciNet  Google Scholar 

  • Egghe, L. (2010). The Hirsch index and related impact measures. Annual Review of Information Science and Technology, 44, 65–114.

    Article  Google Scholar 

  • Egghe, L., & Rousseau, R. (2008). An h-index weighted by citation impact. Information Processing and Management, 44(2), 770–780.

    Article  Google Scholar 

  • Franceschini, F., & Maisano, D. A. (2010). Analysis of the Hirsch index’s operational properties. European Journal of Operational Research, 203(2), 494–504.

    Article  MATH  Google Scholar 

  • Greco, S., Matarazzo, B., & Slowinski, R. (2001). Rough sets theory for multicriteria decision analysis. European Journal of Operational Research, 129(1), 1–47.

    Article  MathSciNet  MATH  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, 102(46), 16569–16572.

    Article  Google Scholar 

  • Hwang, Y.-A. (2013). An axiomatization of the Hirsch-index without adopting monotonicity. Applied Mathematics and Information Sciences. An International Journal, 7(4), 1317–1322.

    Article  Google Scholar 

  • Jin, B. (2006). H-index: An evaluation indicator proposed by scientist. Science Focus, 1(1), 8–9.

    Google Scholar 

  • Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R- and AR-indices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855–863.

    Article  Google Scholar 

  • Kinney, A. L. (2007). National scientific facilities and their science impact on non-biomedical research. Proceedings of the National Academy of Sciencies, 104(46), 17943–17947.

    Article  Google Scholar 

  • Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4–6.

    Google Scholar 

  • Kosmulski, M. (2007). MAXPROD—A new index for assessment of the scientific output of an individual, and a comparison. Cybermetrics, 11(1), 1–5.

    Google Scholar 

  • Marchant, T. (2009a). An axiomatic characterization of the ranking based on the h-index and some other bibliometric rankings of authors. Scientometrics, 80(2), 327–344.

    Article  Google Scholar 

  • Marchant, T. (2009b). Score-based bibliometric rankings of authors. Journal of the American Society for Information Science and Technology, 60(6), 1132–1137.

    Article  Google Scholar 

  • Miroiu, A. (2013). Axiomatizing the Hirsch index: Quantity and quality disjoined. Journal of Informetrics, 7(1), 10–15.

    Article  Google Scholar 

  • Norris, M., & Oppenheim, C. (2010). The h-index: A broad review of a new bibliometric indicator. Journal of Documentation, 66(5), 681–705.

    Article  Google Scholar 

  • Panaretos, J., & Malesios, C. (2009). Assessing scientific research performance and impact with single indices. Scientometrics, 81(3), 635–670.

    Article  Google Scholar 

  • Pomerol, J. C., & Barba-Romero, S. (2000). Multicriterion decision in management: Principles and practice (Vol. 25). New York: Springer.

    Google Scholar 

  • Prathap, G. (2006). Hirsch-type indices for ranking institutions’ scientific research output. Current Science, 91(11), 1439.

    Google Scholar 

  • Quesada, A. (2009). Monotonicity and the Hirsch index. Journal of Informetrics, 3(2), 158–160.

    Article  Google Scholar 

  • Quesada, A. (2010). More axiomatics for the Hirsch index. Scientometrics, 82(2), 413–418.

    Article  MathSciNet  Google Scholar 

  • Quesada, A. (2011a). Axiomatics for the Hirsch index and the Egghe index. Journal of Informetrics, 5(3), 476–480.

    Article  Google Scholar 

  • Quesada, A. (2011b). Further characterizations of the Hirsch index. Scientometrics, 87(1), 107–114.

    Article  Google Scholar 

  • Roy, B., & Bouyssou, D. (1993). Aide Multicritère à la Décision: Méthodes et Cas. Paris: Economica.

    MATH  Google Scholar 

  • Rubem, A. P. S., Moura, A. L. & Soares de Mello, J. C. C. B. (2013). Numerical analysis of some individual bibliometric indexes when applied to groups of researchers. In: XLV Simpósio Brasileiro de Pesquisa Operacional, SBPO 2013, Natal. Anais do XLV SBPO.

  • Schreiber, M. (2008). A modification of the h-index: The hm-index accounts for multiauthored manuscripts. Journal of Informetrics, 2(3), 211–216.

    Article  MathSciNet  Google Scholar 

  • Schreiber, M. (2010a). Revisiting the g-index: The average number of citations in the g-core. Journal of American Society for Information Science and Technology, 61(1), 169–174.

    Article  Google Scholar 

  • Schreiber, M. (2010b). Twenty Hirsch index variants and other indicators giving more or less preference to highly cited papers. Annalen der Physik, 522(8), 536–554.

    Article  MathSciNet  Google Scholar 

  • Schubert, A. (2007). Successive h-indices. Scientometrics, 70(1), 201–205.

    Article  Google Scholar 

  • Thompson, D. F., Callen, E. C., & Nahata, M. C. (2009). New indices in scholarship assessment. American Journal of Pharmaceutical Education, 73(6), 111.

    Article  Google Scholar 

  • Tol, R. S. (2009). The h-index and its alternatives: An application to the 100 most prolific economists. Scientometrics, 80(2), 317–324.

  • Van Eck, N. J., & Waltman, L. (2008). Generalizing the h- and g-indices. Journal of Informetrics, 2(4), 263–271.

    Article  Google Scholar 

  • Van Raan, A. F. J. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67(3), 491–502.

    Article  Google Scholar 

  • Vinkler, P. (2009). The π-index: A new indicator for assessing scientific impact. Journal of Information Science, 35(5), 602–612.

  • Waltman, L., & Van Eck, N. J. (2012). The inconsistency of the h-index. Journal of the American Society for Information Science and Technology, 63(2), 406–415.

    Article  Google Scholar 

  • Woeginger, G. J. (2008a). An axiomatic characterization of the Hirsch-index. Mathematical Social Sciences, 56(2), 224–232.

    Article  MathSciNet  MATH  Google Scholar 

  • Woeginger, G. J. (2008b). A symmetry axiom for scientific impact indices. Journal of Informetrics, 2(4), 298–303.

    Article  MathSciNet  Google Scholar 

  • Woeginger, G. J. (2008c). An axiomatic analysis of Egghe’s g-index. Journal of Informetrics, 2(4), 364–368.

    Article  MathSciNet  Google Scholar 

  • Woeginger, G. J. (2009). Generalizations of Egghe’s g-index. Journal of the American Society for Information Science and Technology, 60(6), 1267–1273.

    Article  Google Scholar 

  • Wu, Q. (2010). The w-index: A measure to assess scientific impact by focusing on widely cited papers. Journal of the American Society for Information Science and Technology, 61(3), 609–614.

    Google Scholar 

  • Zhang, C.-T. (2009). The e-index, complementing the h-index for excess citations. PLoS One, 4(5), e5429. doi:10.1371/journal.pone.0005429.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centro de Análise de Sistemas Navais, Praça Barão de Ladário, Ilha das Cobras, Centro, Rio de Janeiro, RJ, 20091-000, Brazil

    Ana Paula dos Santos Rubem

  2. Mestrado em Engenharia de Produção, Universidade Federal Fluminense, Rua Passo da Pátria, 156, São Domingos, Niterói, RJ, 24210-240, Brazil

    Ariane Lima de Moura

  3. Departamento de Engenharia de Produção, Universidade Federal Fluminense, Rua Passo da Pátria, 156, São Domingos, Niterói, RJ, 24210-240, Brazil

    João Carlos Correia Baptista Soares de Mello

Authors
  1. Ana Paula dos Santos Rubem
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ariane Lima de Moura
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. João Carlos Correia Baptista Soares de Mello
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ariane Lima de Moura.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rubem, A.P.S., de Moura, A.L. & Soares de Mello, J.C.C.B. Comparative analysis of some individual bibliometric indices when applied to groups of researchers. Scientometrics 102, 1019–1035 (2015). https://doi.org/10.1007/s11192-014-1428-y

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-014-1428-y

Keywords

Search

Navigation