Abstract
Research impact is very important in academia. This study explored the research impact of faculty in doctoral higher education programs through the use of Hirsch’s h index as measured by Google Scholar results. Characteristics of the h index in this field are discussed, and norms are offered for professors of different ranks. We also explore relationships between gender, experience, and U.S. News and World Report ranking and the index. We find that gender has no significant relationship to faculty index in this field, but faculty experience and school rankings do have a relationship. Our findings support the use of the h index in assessing research impact in the higher education field, and they may be of interest to persons beyond this field as we consider the manner in which we assess faculty research.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2009). H-index: A review focused in its variants, computation and standardization for different scientific fields. Journal of Informetrics, 3, 273–289. https://doi.org/10.1016/j.joi.2009.04.001
Batista, P. D., Campiteli, M. G., & Kinouchi, O. (2006). Is it possible to compare researchers with different scientific interests? Scientometrics, 68 (1), 179–189.
Bormann, L., & Daniel, H.-D. (2009). The state of h index research. EMBO Reports, 10, 2–6. https://doi.org/10.1038/embor.2008.233
Bray, N. J., & Major, C. H. (2011). Status of journals in the field of higher education. The Journal of Higher Education, 82, 479–503. https://doi.org/10.1353/jhe.2011.0020
Budd, J. M., & Magnuson, L. (2010). Higher Education Literature Revisited: Citation Patterns Examined. Research in Higher Education, 51(3), 294–304
Burrows, R. (2012). Living with the h-index? Metric assemblages in the contemporary academy. The Sociological Review, 60, 355–372. https://doi.org/10.1111/j.1467-954X.2012.02077.x
Calma, A., & Davies, M. (2015). Studies in Higher Education 1976–2013: A retrospective using citation network analysis. Studies in Higher Education, 40, 4–21. https://doi.org/10.1080/03075079.2014.977858
Calma, A., & Davies, M. (2017). Geographies of influence: A citation network analysis of Higher Education 1972–2014. Scientometrics, 110(3), 1579–1599. https://doi.org/10.1007/s11192-016-2228-3
de Bellis, N. (2009). Bibliometrics and citation analysis: From the science citation index to cybermetrics. Lanham, MD: Scarecrow Press.
de Winter, J. C. F., Zadpoor, A. A., & Dodou, D. (2014). The expansion of Google scholar versus web of science: A longitudinal study. Scientometrics, 98, 1547–1565. https://doi.org/10.1007/s11192-013-1089-2
Delgado López-Cózar, E., Robinson-García, N., & Torres-Salinas, D. (2014). The Google scholar experiment: How to index false papers and manipulate bibliometric indicators. Journal of the Association for Information Science and Technology, 65, 446–454. https://doi.org/10.1002/asi.23056
Dill, D. D., & Morrison, J. L. (1985). Ed.D. and Ph.D. research training in the field of higher education: A survey and a proposal. The Review of Higher Education, 8, 169–186.
Doğan, G., Şencan, İ., & Tonta, Y. (2016). Does dirty data affect Google scholar citations? Proceedings of the Association for Information Science and Technology, 53, 1–4. https://doi.org/10.1002/pra2.2016.14505301098
Earp, V. J. (2010). A bibliometric snapshot of The Journal of Higher Education and its impact on the field. Behavioral & Social Sciences Librarian, 29, 283–295. https://doi.org/10.1080/01639269.2010.521034
Freeman, S., Hagedorn, L. S., Goodchild, L. F., & Wright, D. A. (Eds.) (2014). Advancing Higher Education as a field of study: In quest of doctoral degree guidelines -- Commemorating 120 years of excellence. Sterling, VA: Stylus.
Freeman, S., & Kochan, F. (2014). Toward a theoretical framework for the doctorate in Higher Education administration. In S. Freeman, L.S. Hagedorn, L.F. Goodchild, & D.A. Wright (Eds.), Advancing Higher Education as a field of study: In quest of doctoral degree guidelines – Commemorating 120 years of excellence (pp. 145–168). Sterling, VA: Stylus.
Garfield, E. (2006). Citation indexes for science. A new dimension in documentation through association of ideas. International Journal of Epidemiology, 35, 1123–1127. https://doi.org/10.1093/ije/dyl189
Geraci, L., Balsis, S., & Busch, A. J. B. (2015). Gender and the h index in psychology. Scientometrics, 105, 2023–2034. https://doi.org/10.1007/s11192-015-1757-5
Goodchild, L.F. (2014). Higher Education as a field of study: Its history, degree programs, associations, and national guidelines. In S. Freeman, L.S. Hagedorn, L.F. Goodchild, & D.A. Wright (Eds.), Advancing Higher Education as a field of study: In quest of doctoral degree guidelines – Commemorating 120 years of excellence (pp. 13–50). Sterling, VA: Stylus.
Greenbank, P. (2006). The academic’s role: The need for a re-evaluation? Teaching in Higher Education, 11, 107–112. https://doi.org/10.1080/13562510500400248
Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S. (2015). The role of Google scholar in evidence reviews and its applicability to grey literature searching. PLoS One, 10, e0138237. https://doi.org/10.1371/journal.pone.0138237
Hao, L., & Naiman, D. (2007). Quantile regression. Thousand Oaks, CA: SAGE.
Harzing, A.W. (2007) Publish or Perish. Available from http://www.harzing.com/pop.htm
Harzing, A. W., & Alakangas, S. (2016). Google scholar, Scopus and the Web of Science: A longitudinal and cross-disciplinary comparison. Scientometrics, 106, 787–804. https://doi.org/10.1007/s11192-015-1798-9
Harzing, A. W., Alakangas, S., & Adams, D. (2014). hIa: An individual annual h-index to accommodate disciplinary and career length differences. Scientometrics, 99, 811–821. https://doi.org/10.1007/s11192-013-1208-0
Hendrickson, R.M. (2014). The core knowledge of Higher Education. In S. Freeman, L.S. Hagedorn, L.F. Goodchild, & D.A. Wright (Eds.), Advancing Higher Education as a field of study: In quest of doctoral degree guidelines – Commemorating 120 years of excellence (pp.229–240). Sterling, VA: Stylus.
Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102, 16569–16572. https://doi.org/10.1073/pnas.0507655102
Holden, G., Rosenberg, G., & Barker, K. (2005). Bibliometrics: A potential decision making aid in hiring, reappointment, tenure and promotion decisions. Social Work in Health Care, 41(3–4), 67–92. https://doi.org/10.1300/J010v41n03_03
Jacsò, P. (2010). Pragmatic issues in calculating and comparing the quantity and quality of research through rating and ranking of researchers based on peer reviews and bibliometric indicators from Web of Science, Scopus and Google scholar. Online Information Review, 34, 972–982. https://doi.org/10.1108/14684521011099432
Johnson, M. R., Wagner, N. J., & Reusch, J. (2016). Publication trends in top-tier journals in higher education. Journal of Applied Research in Higher Education, 8, 439–454. https://doi.org/10.1108/JARHE-01-2015-0003
Kaur, J., Radicchi, F., & Menczer, F. (2013). Universality of scholarly impact metrics. Journal of Informetrics, 7, 924–932. https://doi.org/10.1016/j.joi.2013.09.002
Kezar, A. J. (2000). Higher education research at the millennium: Still trees without fruit? The Review of Higher Education, 23, 443–468. https://doi.org/10.1353/rhe.2000.0018
Kousha, K., Thelwall, M., & Rezaie, S. (2011). Assessing the citation impact of books: The role of Google books, Google scholar, and Scopus. Journal of the American Society for Information Science and Technology, 62, 2147–2164. https://doi.org/10.1002/asi.21608
Kretschmer, H., & Kretschmer, T. (2013). Gender bias and explanation models for the phenomenon of women’s discriminations in research careers. Scientometrics, 97, 25–36. https://doi.org/10.1007/s11192-013-1023-7
Martínez-Lebrón, C. (2016). Ph.D. and Ed.D. degrees in higher education programs: A mixed methods study. Terre Haute, IN: Indiana State University. Retrieved from https://search.proquest.com/openview/5e36f0cf423f2618df23d28413f7e665/1
Meho, L. I., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google scholar. Journal of the American Society for Information Science and Technology, 58, 2105–2125. https://doi.org/10.1002/asi.20677
Morse, R., & Hines, K. (2018, March 19). Methodology: 2019 best education schools rankings. Retrieved from https://www.usnews.com/education/best-graduate-schools/articles/education-schools-methodology
National Science Foundation, National Center for Science and Engineering Statistics (2018). Doctorate Recipients from U.S. Universities: 2016. Special Report NSF 18–304. Alexandria, VA. Retrieved from www.nsf.gov/statistics/2018/nsf18304/
Norris, M., & Oppenheim, C. (2010). Peer review and the h-index: Two studies. Journal of Informetrics, 4, 221–232. https://doi.org/10.1016/j.joi.2009.11.001
Ortega, J. L., & Aguillo, I. F. (2014). Microsoft academic search and Google scholar citations: Comparative analysis of author profiles. Journal of the Association for Information Science and Technology, 65, 1149–1156. https://doi.org/10.1002/asi.23036
Petscher, Y., & Logan, J. A. R. (2014). Quantile regression in the study of developmental sciences. Child Development, 85, 861–881. https://doi.org/10.1111/cdev.12190
Prins, A. A. M., Costas, R., van Leeuwen, T. N., & Wouters, P. F. (2016). Using Google scholar in research evaluation of humanities and social science programs: A comparison with Web of Science data. Research Evaluation, 25, 264–270. https://doi.org/10.1093/reseval/rvv049
Rumbley, L., Altbach, P. G., Stanfield, D. A., Boston College, & Center for International Higher Education (2014). Higher education: A worldwide inventory of research centers, academic programs, and journals and publications. Chestnut Hill, MA: Boston College.
Saunders, D. B., Kolek, E. A., Williams, E. A., & Wells, R. S. (2016). Who is shaping the field? Doctoral education, knowledge creation and postsecondary education research in the United States. Higher Education Research & Development, 35, 1039–1052. https://doi.org/10.1080/07294360.2016.1139552
Schreiber, M. (2008). An empirical investigation of theg-index for 26 physicists in comparison with theh-index, theA-index, and theR-index. Journal of the American Society for Information Science and Technology, 59(9), 1513–1522.
Shulman, L. S., Golde, C. M., Bueschel, A. C., & Garabedian, K. J. (2006). Reclaiming education’s doctorates: A critique and a proposal. Educational Researcher, 35, 25–32. https://doi.org/10.3102/0013189X035003025
Symonds, M. R. E., Gemmell, N. J., Braisher, T. L., Gorringe, K. L., & Elgar, M. A. (2006). Gender differences in publication output: Towards an unbiased metric of research performance. PLoS One, 1(1), e127. https://doi.org/10.1371/journal.pone.0000127
Tight, M. (2008). Higher education research as tribe, territory and/or community: A co-citation analysis. Higher Education, 55, 593–605. https://doi.org/10.1007/s10734-007-9077-1
Tight, M. (2014). Working in separate silos? What citation patterns reveal about higher education research internationally. Higher Education, 68, 379–395. https://doi.org/10.1007/s10734-014-9718-0
U.S. News and World Report (2016). America's best graduate schools, 2017 edition. Washington, DC: Author.
U.S. News and World Report (2017). America's best graduate schools, 2018th edition. Washington, DC: Author.
van Raan, A. F. J. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67, 491–502. https://doi.org/10.1556/Scient.67.2006.3.10
Wainer, J., & Vieira, P. (2013). Correlations between bibliometrics and peer evaluation for all disciplines: The evaluation of Brazilian scientists. Scientometrics, 96, 395–410. https://doi.org/10.1007/s11192-013-0969-9
Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10, 365–391. https://doi.org/10.1016/j.joi.2016.02.007
Waltman, L., & van Eck, N. J. (2012). The inconsistency of the h-index. Journal of the American Society for Information Science and Technology, 63, 406–415. https://doi.org/10.1002/asi.21678
Waltman, L., van Eck, N. J., van Leeuwen, T. N., Visser, M. S., & van Raan, A. F. J. (2011). On the correlation between bibliometric indicators and peer review: Reply to Opthof and Leydesdorff. Scientometrics, 88, 1017–1022. https://doi.org/10.1007/s11192-011-0425-7
Watkins, M. W., & Chan-Park, C. Y. (2015). The research impact of school psychology faculty. Journal of School Psychology, 53, 231–241. https://doi.org/10.1016/j.jsp.2015.03.003
Weingart, P. (2005). Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics, 62, 117–131. https://doi.org/10.1007/s11192-005-0007-7
Wildgaard, L. E. (2015). Measure up!: The extent author-level bibliometric indicators are appropriate measures of individual researcher performance. (unpublished doctoral dissertation). Københavns Universitet, Copenhagen, Denmark.
Wilkins, S., & Huisman, J. (2015). Stakeholder perspectives on citation and peer-based rankings of higher education journals. Tertiary Education and Management, 21, 1–15. https://doi.org/10.1080/13583883.2014.987313
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Scruggs, R., McDermott, P.A. & Qiao, X. A Nationwide Study of Research Publication Impact of Faculty in U.S. Higher Education Doctoral Programs. Innov High Educ 44, 37–51 (2019). https://doi.org/10.1007/s10755-018-9447-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10755-018-9447-x