Evaluation of h-index and its citation intensity based variants in the field of mathematics

  • Qurat-ul Ain
  • Hira Riaz
  • Muhammad Tanvir AfzalEmail author


Assessing and evaluating the academic impact and its results produced by researchers is necessary to promote the academic progress. A diverse and varied set of parameters have been proposed by the scientific community to find the most influential researchers, including citation count, the total number of publications, hybrid approaches, h-index, extensions and variants of h-index. Current state-of-the-art depicts that there is no standard benchmark available to determine the optimum parameter to find the most influential author of a specific domain. Furthermore, it has been observed that such indices are assessed on a small dataset and ingenious scenarios. The small dataset can never truly help to analyze the nature of these indices and it is very difficult to determine the significance and influence of every index over the others. Hence, it’s necessary to assess them on a large dataset. The following research would help in scrutinizing the h-index along with its citation intensity based variants to rank the authors by using a large dataset of Mathematics domain that consist of 57,533 authors and 62,033 total numbers of publications. These indices make use of the total published papers, citation count, along with the h-index and the five of its citation intensity based variants. The esteemed awards that are won nationally and internationally in the field of mathematics serve as a benchmark. This study would deal and help to recognize the most influential authors by concluding the results gained after evaluation of these indices. For this purpose, firstly, we calculated the correlation among different indices. The strong correlation was found between the h-index and its five citation intensity based variants. The occurrence of the award winners is examined according to the rank lists. H-index brought around 30.88% awardees in the top 10% of the ranked list. In a bird’s eye view, no index could succeed in elevating a 50% of award winners in the top-ranking. Our benchmark dataset is composed of 68 awardees. In the ranking lists, the maximum number of awardees belongs to American Mathematics Society (AMS) which are 29.


Authors ranking Citation count Citation intensity H-index H-index variants Mathematics Subject Classification 



Funding was provided by Capital University of Science and Technology.


  1. Adler, R., Ewing, J., & Taylor, P. (2009). Citation statistics: A report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS). Statistical Science, 24(1), 1–14.MathSciNetCrossRefzbMATHGoogle Scholar
  2. Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2009). h-index: A review focused on its variants, computation, and standardization for different scientific fields. Journal of Informetrics, 3(4), 273–289.CrossRefGoogle Scholar
  3. Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2010). hg-index: A new index to characterize the scientific output of researchers based on the h-and g-indices. Scientometrics, 82(2), 391–400.CrossRefGoogle Scholar
  4. Aoun, S. G., Bendok, B. R., Rahme, R. J., Dacey, R. G., & Batjer, H. H. (2013). Standardizing the evaluation of scientific and academic performance in neurosurgery—Critical review of the “h” index and its variants. World Neurosurgery, 80(5), e85–e90.CrossRefGoogle Scholar
  5. Ayaz, S., & Afzal, M. T. (2016). Identification of conversion factor for completing-h index for the field of mathematics. Scientometrics, 109(3), 1511–1524.CrossRefGoogle Scholar
  6. Balog, K., Azzopardi, L., & De Rijke, M. (2006, August). Formal models for expert finding in enterprise corpora. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 43–50). ACM.Google Scholar
  7. Beel, J., & Gipp, B. (2009, April). Google Scholar’s ranking algorithm: The impact of citation counts (an empirical study). In Third international conference on research challenges in information science, 2009. RCIS 2009 (pp. 439–446). IEEE.Google Scholar
  8. Behrens, H., & Luksch, P. (2011). Mathematics 1868–2008: A bibliometric analysis. Scientometrics, 86(1), 179–194.CrossRefzbMATHGoogle Scholar
  9. Bogers, T., & Van den Bosch, A. (2008, October). Recommending scientific articles using CiteULike. In Proceedings of the 2008 ACM conference on recommender systems (pp. 287–290). ACM.Google Scholar
  10. Bornmann, L., & Daniel, H. D. (2005). Committee peer review at an international research foundation: Predictive validity and fairness of selection decisions on post-graduate fellowship applications. Research Evaluation, 14(1), 15–20.CrossRefGoogle Scholar
  11. Bornmann, L., Mutz, R., & Daniel, H. D. (2008). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the Association for Information Science and Technology, 59(5), 830–837.Google Scholar
  12. Bornmann, L., Mutz, R., Daniel, H. D., Wallon, G., & Ledin, A. (2009). Are there really two types of h index variants? A validation study by using molecular life sciences data. Research Evaluation, 18(3), 185–190.CrossRefGoogle Scholar
  13. Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H. D. (2011). A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. Journal of Informetrics, 5(3), 346–359.CrossRefGoogle Scholar
  14. Bosman, J., Mourik, I. V., Rasch, M., Sieverts, E., &Verhoeff, H. (2006). Scopus reviewed and compared: The coverage and functionality of the citation database Scopus, including comparisons with Web of Science and Google Scholar. Report, Utrecht University Library.Google Scholar
  15. Bui, D. L., Nguyen, T. T., & Ha, Q. T. (2014). Measuring the influence of bloggers in their community based on the H-index family. In T. van Do, H. Thi, & N. Nguyen (Eds.), Advanced computational methods for knowledge engineering (pp. 313–324). Cham: Springer.CrossRefGoogle Scholar
  16. Burrell, Q. (2007). Hirsch index or Hirsch rate? Some thoughts arising from Liang’s data. Scientometrics, 73(1), 19–28.CrossRefGoogle Scholar
  17. Cabrerizo, F. J., Alonso, S., Herrera-Viedma, E., & Herrera, F. (2010). q2-Index: Quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core. Journal of Informetrics, 4(1), 23–28.CrossRefGoogle Scholar
  18. Cameron, D. H. L., Aleman-Meza, B., Decker, S., &Arpinar, I. B. (2007). SEMEF: A taxonomy-based discovery of experts, expertise and collaboration networks. Doctoral dissertation, University of Georgia.Google Scholar
  19. Corder, G. W., & Foreman, D. I. (2009). Comparing variables of ordinal or dichotomous scales: Spearman rank‐order, point‐biserial, and biserial correlations. In Nonparametric statistics for non-statisticians: A step-by-step approach (pp. 122–154). Wiley.Google Scholar
  20. De Winter, J. C., Zadpoor, A. A., & Dodou, D. (2014). The expansion of Google Scholar versus Web of Science: A longitudinal study. Scientometrics, 98(2), 1547–1565.CrossRefGoogle Scholar
  21. Dienes, K. R. (2015). Completing h. Journal of Informetrics, 9(2), 385–397.MathSciNetCrossRefGoogle Scholar
  22. Dorta-Gonzalez, P., & Dorta-González, M. I. (2013). Impact maturity times and citation time windows: The 2-year maximum journal impact factor. Journal of Informetrics, 7(3), 593–602.CrossRefGoogle Scholar
  23. Egghe, L. (2006). An improvement of the h-index: The g-index. ISSI.Google Scholar
  24. Fukuzawa, N. (2014). An empirical analysis of the relationship between individual characteristics and research productivity. Scientometrics, 99(3), 785–809.CrossRefGoogle Scholar
  25. Ghani, R., Qayyum, F., Afzal, M. T., & Maurer, H. (2019). Comprehensive evaluation of h-index and its extensions in the domain of mathematics. Scientometrics. Scholar
  26. Harzing, A. W. (2010). Citation analysis across disciplines: The impact of different data sources and citation metrics. Accessed 13 Jan 2019.
  27. Harzing, A. W. (2013). A preliminary test of Google Scholar as a source for citation data: A longitudinal study of Nobel prize winners. Scientometrics, 94(3), 1057–1075.CrossRefGoogle Scholar
  28. Harzing, A. W., & Alakangas, S. (2016). Google Scholar, Scopus and the Web of Science: A longitudinal and cross-disciplinary comparison. Scientometrics, 106(2), 787–804.CrossRefGoogle Scholar
  29. Harzing, A. W. K., & Van der Wal, R. (2008). Google Scholar as a new source for citation analysis. Ethics in Science and Environmental Politics, 8(1), 61–73.CrossRefGoogle Scholar
  30. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569.CrossRefzbMATHGoogle Scholar
  31. James, D. W. (2014). Completing Hirsch’s h-index measuring scholarly impact. In Scholardox E4.Google Scholar
  32. Jiang, X., Sun, X., & Zhuge, H. (2013). Graph-based algorithms for ranking researchers: Not all swans are white! Scientometrics, 96(3), 743–759.CrossRefGoogle Scholar
  33. Jin, B. (2006). H-index: An evaluation indicator proposed by the scientist. Science Focus, 1(1), 8–9.Google Scholar
  34. Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R-and AR-indices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855–863.CrossRefGoogle Scholar
  35. Katsaros, D., Akritidis, L., & Bozanis, P. (2009). The f index: Quantifying the impact of coterminal citations on scientists’ ranking. Journal of the Association for Information Science and Technology, 60(5), 1051–1056.Google Scholar
  36. Liang, R., & Jiang, X. (2016, February).Scientific ranking over heterogeneous academic hypernetwork. In AAAI (pp. 20–26).Google Scholar
  37. Liu, Y., Rao, I. R., & Rousseau, R. (2009). Empirical series of journal h-indices: The JCR category Horticulture as a case study. Scientometrics, 80(1), 59–74.CrossRefGoogle Scholar
  38. Liu, Y., & Rousseau, R. (2007). Hirsch-type indices and library management: The case of Tongji University Library. In 11th International conference of the International Society for Scientrometrics and Informetrics, June 25–27, 2007, Madrid, Spain (pp. 514–522).Google Scholar
  39. Mazloumian, A., Helbing, D., Lozano, S., Light, R. P., & Börner, K. (2013). Global multi-level analysis of the ‘Scientific Food Web’. Scientific reports, 3, 1167.CrossRefGoogle Scholar
  40. Mingers, J. (2009). Measuring the research contribution of management academics using the Hirsch-index. Journal of the Operational Research Society, 60(9), 1143–1153.CrossRefGoogle Scholar
  41. Moreira, C., Calado, P., & Martins, B. (2015). Learning to rank academic experts in the DBLP dataset. Expert Systems, 32(4), 477–493.CrossRefGoogle Scholar
  42. Moreira, C., & Wichert, A. (2013). Finding academic experts on a multisensor approach using Shannon’s entropy. Expert Systems with Applications, 40(14), 5740–5754.CrossRefGoogle Scholar
  43. Neuhaus, C., Neuhaus, E., Asher, A., & Wrede, C. (2006). The depth and breadth of Google Scholar: An empirical study. Portal Libraries and the Academy, 6(2), 127–141.CrossRefGoogle Scholar
  44. Panaretos, J., & Malesios, C. (2009). Assessing scientific research performance and impact with single indices. Scientometrics, 81(3), 635.CrossRefGoogle Scholar
  45. Raheel, M., Ayaz, S., & Afzal, M. T. (2018). Evaluation of h-index, its variants, and extensions based on publication age & citation intensity in civil engineering. Scientometrics, 114, 1107–1127.CrossRefGoogle Scholar
  46. Rousseau, R. (2006). New developments related to the Hirsch index. Preprint, Accessed 13 Jan 2019.
  47. Sangwal, K. (2012). On the age-independent publication index. Scientometrics, 91(3), 1053–1058.CrossRefGoogle Scholar
  48. Schreiber, M. (2008a). An empirical investigation of the g-index for 26 physicists in comparison with the h-index, the A-index, and the R-index. Journal of the American Society for Information Science and Technology, 59(9), 1513–1522.CrossRefGoogle Scholar
  49. Schreiber, M. (2008b). To share the fame in a fair way, hm modifies h for multi-authored manuscripts. New Journal of Physics, 10(4), 040201.MathSciNetCrossRefGoogle Scholar
  50. Schreiber, M. (2010). Twenty Hirsch index variants and other indicators giving more or less preference to highly cited papers. Annalen der Physik, 522(8), 536–554.CrossRefGoogle Scholar
  51. Sidiropoulos, A., Katsaros, D., & Manolopoulos, Y. (2007). Generalized Hirsch h-index for disclosing latent facts in citation networks. Scientometrics, 72(2), 253–280.CrossRefGoogle Scholar
  52. Sidiropoulos, A., & Manolopoulos, Y. (2005). A citation-based system to assist prize awarding. ACM SIGMOD Record, 34(4), 54–60.CrossRefGoogle Scholar
  53. Sidiropoulos, A., & Manolopoulos, Y. (2006). Generalized comparison of graph-based ranking algorithms for publications and authors. Journal of Systems and Software, 79(12), 1679–1700.CrossRefGoogle Scholar
  54. Smolinsky, L., & Lercher, A. (2012). Citation rates in mathematics: A study of variation by subdiscipline. Scientometrics, 91(3), 911–924.CrossRefGoogle Scholar
  55. Teixeira, A. A., & Mota, L. (2012). A bibliometric portrait of the evolution, scientific roots and influence of the literature on university-industry links. Scientometrics, 93(3), 719–743.CrossRefGoogle Scholar
  56. Wells, J. D. (2014). Scholardox E4 (2014) Completing Hirsch’s h-index measuring scholarly impact By James D. Wells (University of Michigan, Ann Arbor) April 13, 2014.Google Scholar
  57. Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S. H., Jones, R., et al. (2015). The metric tide: Report of the independent review of the role of metrics in research assessment and management. Technical report.Google Scholar
  58. Wu, Q. (2010). The w-index: A measure to assess scientific impact by focusing on widely cited papers. Journal of the Association for Information Science and Technology, 61(3), 609–614.Google Scholar
  59. Yan, E., Ding, Y., & Sugimoto, C. R. (2011). P-Rank: An indicator measuring prestige in heterogeneous scholarly networks. Journal of the American Society for Information Science and Technology, 62(3), 467–477.Google Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceCapital University of Science and TechnologyIslamabadPakistan

Personalised recommendations