# Evaluation of *h*-index, its variants and extensions based on publication age & citation intensity in civil engineering

- 621 Downloads

## Abstract

The scientific community has proposed diversified set of parameters to rank researchers, including publications, citations, *h*-index, different variants and extensions of *h*-index. However, there is a debate in the scientific Community which index ranks authors in a better way. Current state-of-the-art depicts that these indices are evaluated on imaginary case scenarios and small datasets. Furthermore, these indices are evaluated on different datasets, making it difficult to grasp the contribution and importance of each index over the others. To analyze the individual behavior of each index, these indices should comprehensively be evaluated on some extensive data set. This study emphasizes on the scrutiny of *h*-index, some of its variants and extensions to rank authors. These indices are evaluated using a comprehensive data set of Civil Engineering field. For the evaluation of results obtained from these indices, first correlation was calculated among indices. There exists weak correlation between various indices, which demonstrates that the author’s rankings acquired from these indices are not identical. Secondly, occurrences of awardees are checked in all ranked lists. The prestigious award winners of four Civil Engineering societies are considered as a benchmark. In top 10% of ranked list, maximum 47% of the awardees were brought by Wu-index. Overall, none of the index succeeded in bringing 100% awardees to the top rankings. Highest number of awardees on top of all ranked lists are found to be from ACI (American Concrete Institute), which shows ACI might be dependent on these indices for its criterion to honor awards.

### Keywords

ASCE CEDB*h*-Index Variants of

*h*-index Civil engineering field Author ranking

### References

- Afzal, M. T., & Maurer, H. A. (2011). Expertise recommender system for scientific community.
*Journal of Universal Computer Science,**17*(11), 1529–1549.Google Scholar - Aguillo, I. F. (2011). Is Google Scholar useful for bibliometrics? A webometric analysis.
*Scientometrics,**91*(2), 343–351.CrossRefGoogle Scholar - Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2009). h-Index: A review focused in its variants, computation and standardization for different scientific fields.
*Journal of Informetrics*,*3*(4), 273–289.Google Scholar - Anderson, T., Hankin, R., & Killworth, P. (2008). Beyond the Durfee square: Enhancing the h-index to score total publication output.
*Scientometrics,**76*(3), 577–588.CrossRefGoogle Scholar - Ayaz, S., & Afzal, M. T. (2016). Identification of conversion factor for completing-h index for the field of mathematics.
*Scientometrics,**109*(3), 1511–1524.CrossRefGoogle Scholar - Balog, K., Azzopardi, L., & De Rijke, M. (2006). Formal models for expert finding in enterprise corpora. In
*Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval*(pp. 43–50). ACM.Google Scholar - Beel, J., Gipp, B., & Wilde, E. (2009). Academic search engine optimization (ASEO): Optimizing scholarly literature for Google Scholar & Co.
*Journal of Scholarly Publishing,**41*(2), 176–190.Google Scholar - Belew, R. K. (2005). Scientific impact quantity and quality: Analysis of two sources of bibliographic data. arXiv preprint cs/0504036.Google Scholar
- Bogers, T., & Van den Bosch, A. (2008). Recommending scientific articles using citeulike. In
*Proceedings of the 2008 ACM conference on Recommender systems*(pp. 287–290). ACM.Google Scholar - Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H. D. (2011). A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants.
*Journal of Informetrics,**5*(3), 346–359.CrossRefGoogle Scholar - Burrell, Q. (2007). Hirsch index or Hirsch rate? Some thoughts arising from Liang’s data.
*Scientometrics,**73*(1), 19–28.CrossRefGoogle Scholar - Cabrerizo, F. J., Alonso, S., Herrera-Viedma, E., & Herrera, F. (2010). q
^{2}-Index: Quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core.*Journal of Informetrics,**4*(1), 23–28.CrossRefGoogle Scholar - Cameron, D. H. L., Aleman-Meza, B., Decker, S., & Arpinar, I. B. (2007). SEMEF: A taxonomy-based discovery of experts, expertise and collaboration networks (Doctoral dissertation, University of Georgia).Google Scholar
- Corder, G. W., & Foreman, D. I. (2009).
*Comparing variables of ordinal or dichotomous scales: Spearman rank-order, Point-biserial, and biserial correlations*(pp. 122–154). Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach.Google Scholar - Czarnecki, L., Kaźmierkowski, M. P., & Rogalski, A. (2013). Doing Hirsch proud; shaping H-index in engineering sciences.
*Bulletin of the Polish Academy of Sciences: Technical Sciences,**61*(1), 5–21.Google Scholar - De Winter, J. C., Zadpoor, A. A., & Dodou, D. (2014). The expansion of Google Scholar versus web of science: A longitudinal study.
*Scientometrics,**98*(2), 1547–1565.CrossRefGoogle Scholar - Deng, H., Han, J., Lyu, M. R., & King, I. (2012). Modeling and exploiting heterogeneous bibliographic networks for expertise ranking. In
*Proceedings of the 12th ACM/IEEE-CS Joint Conference on Digital Libraries*(pp. 71–80). ACM.Google Scholar - Dienes, K. R. (2015). Completing h.
*Journal of Informetrics,**9*(2), 385–397.MathSciNetCrossRefGoogle Scholar - Egghe, L. (2006). An improvement of the h-index: The g-index.
*ISSI Newsletter,**2*(1), 8–9.MathSciNetGoogle Scholar - Egghe, L. (2011). Characterizations of the generalized Wu- and Kosmulski-indices in Lotkaian systems.
*Journal of Informetrics,**5*(3), 439–445.CrossRefGoogle Scholar - Ellison, G. (2013). How does the market use citation data? The Hirsch index in economics.
*American Economic Journal: Applied Economics,**5*(3), 63–90.Google Scholar - Evans, J. D. (1996).
*Straightforward statistics for the behavioral sciences*. Pacific Grove: Brooks/Cole publishing.Google Scholar - Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., & Pappas, G. (2008). Comparison of PubMed, Scopus, web of science, and Google Scholar: strengths and weaknesses.
*The FASEB Journal,**22*(2), 338–342.CrossRefGoogle Scholar - Fang, H., & Zhai, C. (2007). Probabilistic models for expert finding. In
*European Conference on Information Retrieval*(pp. 418–430). Berlin: Springer.Google Scholar - Harzing, A. W. (2014). A longitudinal study of Google Scholar coverage between 2012 and 2013.
*Scientometrics,**98*(1), 565–575.CrossRefGoogle Scholar - Henderson, J. (2005). Google Scholar: A source for clinicians?
*Canadian Medical Association Journal,**172*(12), 1549–1550.CrossRefGoogle Scholar - Hirsch, Jorge E. (2005). An index to quantify an individual’s scientific research output.
*Proceedings of the National Academy of Sciences of the United States of America,**102*(46), 16569–16572.CrossRefMATHGoogle Scholar - Jacsó, P. (2008). The pros and cons of computing the h-index using Google Scholar.
*Online Information Review*,*32*(3), 437–452.Google Scholar - Jin, B. (2006). H-index: an evaluation indicator proposed by scientist.
*Science Focus,**1*(1), 8–9.**(in Chinese)**.MathSciNetGoogle Scholar - Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R- and AR-indices: Complementing the h-index.
*Chinese Science Bulletin,**52*(6), 855–863.CrossRefGoogle Scholar - Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index.
*ISSI Newsletter,**2*(3), 4–6.Google Scholar - Kosmulski, M. (2007). MAXPROD—A new index for assessment of the scientific output of an individual, and a comparison.
*Cybermetrics,**11*(1), 1–5.Google Scholar - Kosmulski, M. (2013). Family-tree of bibliometric indices.
*Journal of Informetrics,**7*(2), 313–317.CrossRefGoogle Scholar - Liu, Y. & Rousseau, R. (2007). Hirsch-type indices and library management: The case of Tongji University Library. In
*11th International Conference of the International Society for Scientrometrics and Informetrics*, June 25–27, (pp. 514–522) Madrid, Spain.Google Scholar - Meho, L. I., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and Google Scholar.
*Journal of the American Society for Information Science and Technology,**58*(13), 2105–2125.CrossRefGoogle Scholar - Moed, H. F., Bar-Ilan, J., & Halevi, G. (2016). A new methodology for comparing Google Scholar and scopus.
*Journal of Informetrics,**10*(2), 533–551.CrossRefGoogle Scholar - Noruzi, A. (2005). Google Scholar: The new generation of citation indexes.
*Libri*,*55*(4), 170–180.Google Scholar - Panaretos, J., & Malesios, C. (2009). Assessing scientific research performance and impact with single indices.
*Scientometrics,**81*(3), 635–670.CrossRefGoogle Scholar - Rosenstreich, D., & Wooliscroft, B. (2009). Measuring the impact of accounting journals using Google Scholar and the g-index.
*The British Accounting Review,**41*(4), 227–239.CrossRefGoogle Scholar - Schreiber, M. (2008). An empirical investigation of the g-index for 26 physicists in comparison with the h-index, the A-index, and the R-index.
*Journal of the American Society for Information Science and Technology,**59*(9), 1513–1522.CrossRefGoogle Scholar - Schreiber, M. (2010). Twenty Hirsch index variants and other indicators giving more or less preference to highly cited papers.
*Annalen der Physik,**522*(8), 536–554.CrossRefGoogle Scholar - Sidiropoulos, A., Katsaros, D., & Manolopoulos, Y. (2007). Generalized Hirsch h-index for disclosing latent facts in citation networks.
*Scientometrics,**72*(2), 253–280.CrossRefGoogle Scholar - Sun, J., Ma, J., Cheng, X., Liu, Z. & Cao, X. (2013). Finding an expert: A model recommendation system. In
*Thirty Fourth International Conference on Information Systems*, Milan.Google Scholar - Tol, R. (2009). The h-index and its alternatives: An application to the 100 most prolific economists.
*Scientometrics,**80*(2), 317–324.CrossRefGoogle Scholar - Van Raan, A. F. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups.
*Scientometrics,**67*(3), 491–502.CrossRefGoogle Scholar - Wildgaard, L., Schneider, J. W., & Larsen, B. (2014). A review of the characteristics of 108 author-level bibliometric indicators.
*Scientometrics,**10191,*125–158.CrossRefGoogle Scholar - Wu, Q. (2010). The w-index: A measure to assess scientific impact by focusing on widely cited papers.
*Journal of the American Society for Information Science and Technology,**61*(3), 609–614.Google Scholar - Wu, Q., & Zhang, P. (2017). Some indices violating the basic domination relation.
*Scientometrics,**113*(1), 495–500.CrossRefGoogle Scholar - Yan, Z., Wu, Q., & Li, X. (2016). Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators.
*Scientometrics,**109*(3), 1815–1833.CrossRefGoogle Scholar