# Evaluation of *h*-index and its citation intensity based variants in the field of mathematics

## Abstract

Assessing and evaluating the academic impact and its results produced by researchers is necessary to promote the academic progress. A diverse and varied set of parameters have been proposed by the scientific community to find the most influential researchers, including citation count, the total number of publications, hybrid approaches, *h*-index, extensions and variants of *h*-index. Current state-of-the-art depicts that there is no standard benchmark available to determine the optimum parameter to find the most influential author of a specific domain. Furthermore, it has been observed that such indices are assessed on a small dataset and ingenious scenarios. The small dataset can never truly help to analyze the nature of these indices and it is very difficult to determine the significance and influence of every index over the others. Hence, it’s necessary to assess them on a large dataset. The following research would help in scrutinizing the *h*-index along with its citation intensity based variants to rank the authors by using a large dataset of Mathematics domain that consist of 57,533 authors and 62,033 total numbers of publications. These indices make use of the total published papers, citation count, along with the *h*-index and the five of its citation intensity based variants. The esteemed awards that are won nationally and internationally in the field of mathematics serve as a benchmark. This study would deal and help to recognize the most influential authors by concluding the results gained after evaluation of these indices. For this purpose, firstly, we calculated the correlation among different indices. The strong correlation was found between the *h*-index and its five citation intensity based variants. The occurrence of the award winners is examined according to the rank lists. *H*-index brought around 30.88% awardees in the top 10% of the ranked list. In a bird’s eye view, no index could succeed in elevating a 50% of award winners in the top-ranking. Our benchmark dataset is composed of 68 awardees. In the ranking lists, the maximum number of awardees belongs to American Mathematics Society (AMS) which are 29.

## Keywords

Authors ranking Citation count Citation intensity*H*-index

*H*-index variants Mathematics Subject Classification

## Notes

### Funding

Funding was provided by Capital University of Science and Technology.

## References

- Adler, R., Ewing, J., & Taylor, P. (2009). Citation statistics: A report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS).
*Statistical Science,**24*(1), 1–14.MathSciNetCrossRefzbMATHGoogle Scholar - Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2009). h-index: A review focused on its variants, computation, and standardization for different scientific fields.
*Journal of Informetrics,**3*(4), 273–289.CrossRefGoogle Scholar - Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2010). hg-index: A new index to characterize the scientific output of researchers based on the h-and g-indices.
*Scientometrics,**82*(2), 391–400.CrossRefGoogle Scholar - Aoun, S. G., Bendok, B. R., Rahme, R. J., Dacey, R. G., & Batjer, H. H. (2013). Standardizing the evaluation of scientific and academic performance in neurosurgery—Critical review of the “h” index and its variants.
*World Neurosurgery,**80*(5), e85–e90.CrossRefGoogle Scholar - Ayaz, S., & Afzal, M. T. (2016). Identification of conversion factor for completing-h index for the field of mathematics.
*Scientometrics,**109*(3), 1511–1524.CrossRefGoogle Scholar - Balog, K., Azzopardi, L., & De Rijke, M. (2006, August). Formal models for expert finding in enterprise corpora. In
*Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval*(pp. 43–50). ACM.Google Scholar - Beel, J., & Gipp, B. (2009, April). Google Scholar’s ranking algorithm: The impact of citation counts (an empirical study). In
*Third international conference on research challenges in information science, 2009. RCIS 2009*(pp. 439–446). IEEE.Google Scholar - Behrens, H., & Luksch, P. (2011). Mathematics 1868–2008: A bibliometric analysis.
*Scientometrics,**86*(1), 179–194.CrossRefzbMATHGoogle Scholar - Bogers, T., & Van den Bosch, A. (2008, October). Recommending scientific articles using CiteULike. In
*Proceedings of the 2008 ACM conference on recommender systems*(pp. 287–290). ACM.Google Scholar - Bornmann, L., & Daniel, H. D. (2005). Committee peer review at an international research foundation: Predictive validity and fairness of selection decisions on post-graduate fellowship applications.
*Research Evaluation*,*14*(1), 15–20.CrossRefGoogle Scholar - Bornmann, L., Mutz, R., & Daniel, H. D. (2008). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine.
*Journal of the Association for Information Science and Technology,**59*(5), 830–837.Google Scholar - Bornmann, L., Mutz, R., Daniel, H. D., Wallon, G., & Ledin, A. (2009). Are there really two types of h index variants? A validation study by using molecular life sciences data.
*Research Evaluation,**18*(3), 185–190.CrossRefGoogle Scholar - Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H. D. (2011). A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants.
*Journal of Informetrics,**5*(3), 346–359.CrossRefGoogle Scholar - Bosman, J., Mourik, I. V., Rasch, M., Sieverts, E., &Verhoeff, H. (2006).
*Scopus reviewed and compared: The coverage and functionality of the citation database Scopus, including comparisons with Web of Science and Google Scholar*. Report, Utrecht University Library.Google Scholar - Bui, D. L., Nguyen, T. T., & Ha, Q. T. (2014). Measuring the influence of bloggers in their community based on the H-index family. In T. van Do, H. Thi, & N. Nguyen (Eds.),
*Advanced computational methods for knowledge engineering*(pp. 313–324). Cham: Springer.CrossRefGoogle Scholar - Burrell, Q. (2007). Hirsch index or Hirsch rate? Some thoughts arising from Liang’s data.
*Scientometrics,**73*(1), 19–28.CrossRefGoogle Scholar - Cabrerizo, F. J., Alonso, S., Herrera-Viedma, E., & Herrera, F. (2010). q
^{2}-Index: Quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core.*Journal of Informetrics,**4*(1), 23–28.CrossRefGoogle Scholar - Cameron, D. H. L., Aleman-Meza, B., Decker, S., &Arpinar, I. B. (2007).
*SEMEF: A taxonomy*-*based discovery of experts, expertise and collaboration networks*. Doctoral dissertation, University of Georgia.Google Scholar - Corder, G. W., & Foreman, D. I. (2009). Comparing variables of ordinal or dichotomous scales: Spearman rank‐order, point‐biserial, and biserial correlations. In
*Nonparametric statistics for non*-*statisticians: A step*-*by*-*step approach*(pp. 122–154). Wiley.Google Scholar - De Winter, J. C., Zadpoor, A. A., & Dodou, D. (2014). The expansion of Google Scholar versus Web of Science: A longitudinal study.
*Scientometrics,**98*(2), 1547–1565.CrossRefGoogle Scholar - Dienes, K. R. (2015). Completing h.
*Journal of Informetrics,**9*(2), 385–397.MathSciNetCrossRefGoogle Scholar - Dorta-Gonzalez, P., & Dorta-González, M. I. (2013). Impact maturity times and citation time windows: The 2-year maximum journal impact factor.
*Journal of Informetrics,**7*(3), 593–602.CrossRefGoogle Scholar - Fukuzawa, N. (2014). An empirical analysis of the relationship between individual characteristics and research productivity.
*Scientometrics,**99*(3), 785–809.CrossRefGoogle Scholar - Ghani, R., Qayyum, F., Afzal, M. T., & Maurer, H. (2019). Comprehensive evaluation of
*h*-index and its extensions in the domain of mathematics.*Scientometrics*. https://doi.org/10.1007/s11192-019-03007-0.Google Scholar - Harzing, A. W. (2010). Citation analysis across disciplines: The impact of different data sources and citation metrics. https://harzing.com/publications/white-papers/citation-analysis-across-disciplines. Accessed 13 Jan 2019.
- Harzing, A. W. (2013). A preliminary test of Google Scholar as a source for citation data: A longitudinal study of Nobel prize winners.
*Scientometrics,**94*(3), 1057–1075.CrossRefGoogle Scholar - Harzing, A. W., & Alakangas, S. (2016). Google Scholar, Scopus and the Web of Science: A longitudinal and cross-disciplinary comparison.
*Scientometrics,**106*(2), 787–804.CrossRefGoogle Scholar - Harzing, A. W. K., & Van der Wal, R. (2008). Google Scholar as a new source for citation analysis.
*Ethics in Science and Environmental Politics,**8*(1), 61–73.CrossRefGoogle Scholar - Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output.
*Proceedings of the National Academy of Sciences of the United States of America,**102*(46), 16569.CrossRefzbMATHGoogle Scholar - James, D. W. (2014). Completing Hirsch’s h-index measuring scholarly impact. In Scholardox E4.Google Scholar
- Jiang, X., Sun, X., & Zhuge, H. (2013). Graph-based algorithms for ranking researchers: Not all swans are white!
*Scientometrics,**96*(3), 743–759.CrossRefGoogle Scholar - Jin, B. (2006). H-index: An evaluation indicator proposed by the scientist.
*Science Focus,**1*(1), 8–9.Google Scholar - Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R-and AR-indices: Complementing the h-index.
*Chinese Science Bulletin,**52*(6), 855–863.CrossRefGoogle Scholar - Katsaros, D., Akritidis, L., & Bozanis, P. (2009). The f index: Quantifying the impact of coterminal citations on scientists’ ranking.
*Journal of the Association for Information Science and Technology,**60*(5), 1051–1056.Google Scholar - Liang, R., & Jiang, X. (2016, February).Scientific ranking over heterogeneous academic hypernetwork. In
*AAAI*(pp. 20–26).Google Scholar - Liu, Y., Rao, I. R., & Rousseau, R. (2009). Empirical series of journal h-indices: The JCR category Horticulture as a case study.
*Scientometrics*,*80*(1), 59–74.CrossRefGoogle Scholar - Liu, Y., & Rousseau, R. (2007). Hirsch-type indices and library management: The case of Tongji University Library. In
*11th International conference of the International Society for Scientrometrics and Informetrics*, June 25–27, 2007, Madrid, Spain (pp. 514–522).Google Scholar - Mazloumian, A., Helbing, D., Lozano, S., Light, R. P., & Börner, K. (2013). Global multi-level analysis of the ‘Scientific Food Web’.
*Scientific reports,**3,*1167.CrossRefGoogle Scholar - Mingers, J. (2009). Measuring the research contribution of management academics using the Hirsch-index.
*Journal of the Operational Research Society,**60*(9), 1143–1153.CrossRefGoogle Scholar - Moreira, C., Calado, P., & Martins, B. (2015). Learning to rank academic experts in the DBLP dataset.
*Expert Systems,**32*(4), 477–493.CrossRefGoogle Scholar - Moreira, C., & Wichert, A. (2013). Finding academic experts on a multisensor approach using Shannon’s entropy.
*Expert Systems with Applications*,*40*(14), 5740–5754.CrossRefGoogle Scholar - Neuhaus, C., Neuhaus, E., Asher, A., & Wrede, C. (2006). The depth and breadth of Google Scholar: An empirical study.
*Portal Libraries and the Academy,**6*(2), 127–141.CrossRefGoogle Scholar - Panaretos, J., & Malesios, C. (2009). Assessing scientific research performance and impact with single indices.
*Scientometrics,**81*(3), 635.CrossRefGoogle Scholar - Raheel, M., Ayaz, S., & Afzal, M. T. (2018). Evaluation of h-index, its variants, and extensions based on publication age & citation intensity in civil engineering.
*Scientometrics,**114,*1107–1127.CrossRefGoogle Scholar - Rousseau, R. (2006). New developments related to the Hirsch index. Preprint, http://eprints.rclis.org/7616/. Accessed 13 Jan 2019.
- Sangwal, K. (2012). On the age-independent publication index.
*Scientometrics,**91*(3), 1053–1058.CrossRefGoogle Scholar - Schreiber, M. (2008a). An empirical investigation of the g-index for 26 physicists in comparison with the h-index, the A-index, and the R-index.
*Journal of the American Society for Information Science and Technology*,*59*(9), 1513–1522.CrossRefGoogle Scholar - Schreiber, M. (2008b). To share the fame in a fair way, hm modifies h for multi-authored manuscripts.
*New Journal of Physics*,*10*(4), 040201.MathSciNetCrossRefGoogle Scholar - Schreiber, M. (2010). Twenty Hirsch index variants and other indicators giving more or less preference to highly cited papers.
*Annalen der Physik*,*522*(8), 536–554.CrossRefGoogle Scholar - Sidiropoulos, A., Katsaros, D., & Manolopoulos, Y. (2007). Generalized Hirsch h-index for disclosing latent facts in citation networks.
*Scientometrics,**72*(2), 253–280.CrossRefGoogle Scholar - Sidiropoulos, A., & Manolopoulos, Y. (2005). A citation-based system to assist prize awarding.
*ACM SIGMOD Record,**34*(4), 54–60.CrossRefGoogle Scholar - Sidiropoulos, A., & Manolopoulos, Y. (2006). Generalized comparison of graph-based ranking algorithms for publications and authors.
*Journal of Systems and Software,**79*(12), 1679–1700.CrossRefGoogle Scholar - Smolinsky, L., & Lercher, A. (2012). Citation rates in mathematics: A study of variation by subdiscipline.
*Scientometrics,**91*(3), 911–924.CrossRefGoogle Scholar - Teixeira, A. A., & Mota, L. (2012). A bibliometric portrait of the evolution, scientific roots and influence of the literature on university-industry links.
*Scientometrics,**93*(3), 719–743.CrossRefGoogle Scholar - Wells, J. D. (2014). Scholardox E4 (2014) Completing Hirsch’s h-index measuring scholarly impact By James D. Wells (University of Michigan, Ann Arbor) April 13, 2014.Google Scholar
- Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S. H., Jones, R., et al. (2015).
*The metric tide: Report of the independent review of the role of metrics in research assessment and management.*Technical report.Google Scholar - Wu, Q. (2010). The w-index: A measure to assess scientific impact by focusing on widely cited papers.
*Journal of the Association for Information Science and Technology,**61*(3), 609–614.Google Scholar - Yan, E., Ding, Y., & Sugimoto, C. R. (2011). P-Rank: An indicator measuring prestige in heterogeneous scholarly networks.
*Journal of the American Society for Information Science and Technology,**62*(3), 467–477.Google Scholar