Innovative Higher Education

, Volume 44, Issue 1, pp 37–51 | Cite as

A Nationwide Study of Research Publication Impact of Faculty in U.S. Higher Education Doctoral Programs

  • Richard ScruggsEmail author
  • Paul A. McDermott
  • Xin Qiao


Research impact is very important in academia. This study explored the research impact of faculty in doctoral higher education programs through the use of Hirsch’s h index as measured by Google Scholar results. Characteristics of the h index in this field are discussed, and norms are offered for professors of different ranks. We also explore relationships between gender, experience, and U.S. News and World Report ranking and the index. We find that gender has no significant relationship to faculty index in this field, but faculty experience and school rankings do have a relationship. Our findings support the use of the h index in assessing research impact in the higher education field, and they may be of interest to persons beyond this field as we consider the manner in which we assess faculty research.


Higher education Doctoral education H index Research impact 


  1. Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2009). H-index: A review focused in its variants, computation and standardization for different scientific fields. Journal of Informetrics, 3, 273–289. CrossRefGoogle Scholar
  2. Batista, P. D., Campiteli, M. G., & Kinouchi, O. (2006). Is it possible to compare researchers with different scientific interests? Scientometrics, 68 (1), 179–189.CrossRefGoogle Scholar
  3. Bormann, L., & Daniel, H.-D. (2009). The state of h index research. EMBO Reports, 10, 2–6. CrossRefGoogle Scholar
  4. Bray, N. J., & Major, C. H. (2011). Status of journals in the field of higher education. The Journal of Higher Education, 82, 479–503. CrossRefGoogle Scholar
  5. Budd, J. M., & Magnuson, L. (2010). Higher Education Literature Revisited: Citation Patterns Examined. Research in Higher Education, 51(3), 294–304CrossRefGoogle Scholar
  6. Burrows, R. (2012). Living with the h-index? Metric assemblages in the contemporary academy. The Sociological Review, 60, 355–372. CrossRefGoogle Scholar
  7. Calma, A., & Davies, M. (2015). Studies in Higher Education 1976–2013: A retrospective using citation network analysis. Studies in Higher Education, 40, 4–21. CrossRefGoogle Scholar
  8. Calma, A., & Davies, M. (2017). Geographies of influence: A citation network analysis of Higher Education 1972–2014. Scientometrics, 110(3), 1579–1599. CrossRefGoogle Scholar
  9. de Bellis, N. (2009). Bibliometrics and citation analysis: From the science citation index to cybermetrics. Lanham, MD: Scarecrow Press.Google Scholar
  10. de Winter, J. C. F., Zadpoor, A. A., & Dodou, D. (2014). The expansion of Google scholar versus web of science: A longitudinal study. Scientometrics, 98, 1547–1565. CrossRefGoogle Scholar
  11. Delgado López-Cózar, E., Robinson-García, N., & Torres-Salinas, D. (2014). The Google scholar experiment: How to index false papers and manipulate bibliometric indicators. Journal of the Association for Information Science and Technology, 65, 446–454. CrossRefGoogle Scholar
  12. Dill, D. D., & Morrison, J. L. (1985). Ed.D. and Ph.D. research training in the field of higher education: A survey and a proposal. The Review of Higher Education, 8, 169–186.CrossRefGoogle Scholar
  13. Doğan, G., Şencan, İ., & Tonta, Y. (2016). Does dirty data affect Google scholar citations? Proceedings of the Association for Information Science and Technology, 53, 1–4.
  14. Earp, V. J. (2010). A bibliometric snapshot of The Journal of Higher Education and its impact on the field. Behavioral & Social Sciences Librarian, 29, 283–295. CrossRefGoogle Scholar
  15. Freeman, S., Hagedorn, L. S., Goodchild, L. F., & Wright, D. A. (Eds.) (2014). Advancing Higher Education as a field of study: In quest of doctoral degree guidelines -- Commemorating 120 years of excellence. Sterling, VA: Stylus.Google Scholar
  16. Freeman, S., & Kochan, F. (2014). Toward a theoretical framework for the doctorate in Higher Education administration. In S. Freeman, L.S. Hagedorn, L.F. Goodchild, & D.A. Wright (Eds.), Advancing Higher Education as a field of study: In quest of doctoral degree guidelines – Commemorating 120 years of excellence (pp. 145–168). Sterling, VA: Stylus.Google Scholar
  17. Garfield, E. (2006). Citation indexes for science. A new dimension in documentation through association of ideas. International Journal of Epidemiology, 35, 1123–1127. CrossRefGoogle Scholar
  18. Geraci, L., Balsis, S., & Busch, A. J. B. (2015). Gender and the h index in psychology. Scientometrics, 105, 2023–2034. CrossRefGoogle Scholar
  19. Goodchild, L.F. (2014). Higher Education as a field of study: Its history, degree programs, associations, and national guidelines. In S. Freeman, L.S. Hagedorn, L.F. Goodchild, & D.A. Wright (Eds.), Advancing Higher Education as a field of study: In quest of doctoral degree guidelines – Commemorating 120 years of excellence (pp. 13–50). Sterling, VA: Stylus.Google Scholar
  20. Greenbank, P. (2006). The academic’s role: The need for a re-evaluation? Teaching in Higher Education, 11, 107–112. CrossRefGoogle Scholar
  21. Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S. (2015). The role of Google scholar in evidence reviews and its applicability to grey literature searching. PLoS One, 10, e0138237. CrossRefGoogle Scholar
  22. Hao, L., & Naiman, D. (2007). Quantile regression. Thousand Oaks, CA: SAGE.CrossRefGoogle Scholar
  23. Harzing, A.W. (2007) Publish or Perish. Available from
  24. Harzing, A. W., & Alakangas, S. (2016). Google scholar, Scopus and the Web of Science: A longitudinal and cross-disciplinary comparison. Scientometrics, 106, 787–804. CrossRefGoogle Scholar
  25. Harzing, A. W., Alakangas, S., & Adams, D. (2014). hIa: An individual annual h-index to accommodate disciplinary and career length differences. Scientometrics, 99, 811–821. CrossRefGoogle Scholar
  26. Hendrickson, R.M. (2014). The core knowledge of Higher Education. In S. Freeman, L.S. Hagedorn, L.F. Goodchild, & D.A. Wright (Eds.), Advancing Higher Education as a field of study: In quest of doctoral degree guidelines – Commemorating 120 years of excellence (pp.229–240). Sterling, VA: Stylus.Google Scholar
  27. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102, 16569–16572. CrossRefGoogle Scholar
  28. Holden, G., Rosenberg, G., & Barker, K. (2005). Bibliometrics: A potential decision making aid in hiring, reappointment, tenure and promotion decisions. Social Work in Health Care, 41(3–4), 67–92. CrossRefGoogle Scholar
  29. Jacsò, P. (2010). Pragmatic issues in calculating and comparing the quantity and quality of research through rating and ranking of researchers based on peer reviews and bibliometric indicators from Web of Science, Scopus and Google scholar. Online Information Review, 34, 972–982. CrossRefGoogle Scholar
  30. Johnson, M. R., Wagner, N. J., & Reusch, J. (2016). Publication trends in top-tier journals in higher education. Journal of Applied Research in Higher Education, 8, 439–454. CrossRefGoogle Scholar
  31. Kaur, J., Radicchi, F., & Menczer, F. (2013). Universality of scholarly impact metrics. Journal of Informetrics, 7, 924–932. CrossRefGoogle Scholar
  32. Kezar, A. J. (2000). Higher education research at the millennium: Still trees without fruit? The Review of Higher Education, 23, 443–468. CrossRefGoogle Scholar
  33. Kousha, K., Thelwall, M., & Rezaie, S. (2011). Assessing the citation impact of books: The role of Google books, Google scholar, and Scopus. Journal of the American Society for Information Science and Technology, 62, 2147–2164. CrossRefGoogle Scholar
  34. Kretschmer, H., & Kretschmer, T. (2013). Gender bias and explanation models for the phenomenon of women’s discriminations in research careers. Scientometrics, 97, 25–36. CrossRefGoogle Scholar
  35. Martínez-Lebrón, C. (2016). Ph.D. and Ed.D. degrees in higher education programs: A mixed methods study. Terre Haute, IN: Indiana State University. Retrieved from
  36. Meho, L. I., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google scholar. Journal of the American Society for Information Science and Technology, 58, 2105–2125. CrossRefGoogle Scholar
  37. Morse, R., & Hines, K. (2018, March 19). Methodology: 2019 best education schools rankings. Retrieved from
  38. National Science Foundation, National Center for Science and Engineering Statistics (2018). Doctorate Recipients from U.S. Universities: 2016. Special Report NSF 18–304. Alexandria, VA. Retrieved from
  39. Norris, M., & Oppenheim, C. (2010). Peer review and the h-index: Two studies. Journal of Informetrics, 4, 221–232. CrossRefGoogle Scholar
  40. Ortega, J. L., & Aguillo, I. F. (2014). Microsoft academic search and Google scholar citations: Comparative analysis of author profiles. Journal of the Association for Information Science and Technology, 65, 1149–1156. CrossRefGoogle Scholar
  41. Petscher, Y., & Logan, J. A. R. (2014). Quantile regression in the study of developmental sciences. Child Development, 85, 861–881. CrossRefGoogle Scholar
  42. Prins, A. A. M., Costas, R., van Leeuwen, T. N., & Wouters, P. F. (2016). Using Google scholar in research evaluation of humanities and social science programs: A comparison with Web of Science data. Research Evaluation, 25, 264–270. CrossRefGoogle Scholar
  43. Rumbley, L., Altbach, P. G., Stanfield, D. A., Boston College, & Center for International Higher Education (2014). Higher education: A worldwide inventory of research centers, academic programs, and journals and publications. Chestnut Hill, MA: Boston College.Google Scholar
  44. Saunders, D. B., Kolek, E. A., Williams, E. A., & Wells, R. S. (2016). Who is shaping the field? Doctoral education, knowledge creation and postsecondary education research in the United States. Higher Education Research & Development, 35, 1039–1052. CrossRefGoogle Scholar
  45. Schreiber, M. (2008). An empirical investigation of theg-index for 26 physicists in comparison with theh-index, theA-index, and theR-index. Journal of the American Society for Information Science and Technology, 59(9), 1513–1522.CrossRefGoogle Scholar
  46. Shulman, L. S., Golde, C. M., Bueschel, A. C., & Garabedian, K. J. (2006). Reclaiming education’s doctorates: A critique and a proposal. Educational Researcher, 35, 25–32. CrossRefGoogle Scholar
  47. Symonds, M. R. E., Gemmell, N. J., Braisher, T. L., Gorringe, K. L., & Elgar, M. A. (2006). Gender differences in publication output: Towards an unbiased metric of research performance. PLoS One, 1(1), e127. CrossRefGoogle Scholar
  48. Tight, M. (2008). Higher education research as tribe, territory and/or community: A co-citation analysis. Higher Education, 55, 593–605. CrossRefGoogle Scholar
  49. Tight, M. (2014). Working in separate silos? What citation patterns reveal about higher education research internationally. Higher Education, 68, 379–395. CrossRefGoogle Scholar
  50. U.S. News and World Report (2016). America's best graduate schools, 2017 edition. Washington, DC: Author.Google Scholar
  51. U.S. News and World Report (2017). America's best graduate schools, 2018th edition. Washington, DC: Author.Google Scholar
  52. van Raan, A. F. J. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67, 491–502. CrossRefGoogle Scholar
  53. Wainer, J., & Vieira, P. (2013). Correlations between bibliometrics and peer evaluation for all disciplines: The evaluation of Brazilian scientists. Scientometrics, 96, 395–410. CrossRefGoogle Scholar
  54. Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10, 365–391. CrossRefGoogle Scholar
  55. Waltman, L., & van Eck, N. J. (2012). The inconsistency of the h-index. Journal of the American Society for Information Science and Technology, 63, 406–415. CrossRefGoogle Scholar
  56. Waltman, L., van Eck, N. J., van Leeuwen, T. N., Visser, M. S., & van Raan, A. F. J. (2011). On the correlation between bibliometric indicators and peer review: Reply to Opthof and Leydesdorff. Scientometrics, 88, 1017–1022. CrossRefGoogle Scholar
  57. Watkins, M. W., & Chan-Park, C. Y. (2015). The research impact of school psychology faculty. Journal of School Psychology, 53, 231–241. CrossRefGoogle Scholar
  58. Weingart, P. (2005). Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics, 62, 117–131. CrossRefGoogle Scholar
  59. Wildgaard, L. E. (2015). Measure up!: The extent author-level bibliometric indicators are appropriate measures of individual researcher performance. (unpublished doctoral dissertation). Københavns Universitet, Copenhagen, Denmark.Google Scholar
  60. Wilkins, S., & Huisman, J. (2015). Stakeholder perspectives on citation and peer-based rankings of higher education journals. Tertiary Education and Management, 21, 1–15. CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2018

Authors and Affiliations

  1. 1.Graduate School of EducationUniversity of PennsylvaniaPhiladelphiaUSA
  2. 2.College of EducationUniversity of MarylandCollege ParkUSA

Personalised recommendations