Advertisement

Research in Higher Education

, Volume 44, Issue 2, pp 121–148 | Cite as

Using Publications Counts to Measure an Institution's Research Productivity

  • Robert K. Toutkoushian
  • Stephen R. Porter
  • Cherry Danielson
  • Paula R. Hollis
Article

Abstract

Studies that attempt to evaluate and rank institutions often overlook the research activities of an institution. In this article, we show how readily available data from the Institute of Scientific Inquiry (ISI) may be used to estimate the number of scholarly articles written by an institution's faculty. We show how institutions are ranked according to total publications and the ratio of publications to full-time faculty, how these measures vary by type of institution, and how they are correlated with other selected measures of research resources and institutional quality.

research performance indicators productivity 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

REFERENCES

  1. Adams, A., and Krislov, J. (1978). Evaluating the quality of American universities: A new approach. Res. Higher Educ. 8: 97–109.Google Scholar
  2. Baird, L. (1986). What characterizes a productive research department? Res. Higher Educ. 25: 211–225.Google Scholar
  3. Baird, L. (1991). Publication productivity in doctoral research departments: Interdisciplinary and intradisciplinary factors. Res. Higher Educ. 32: 303–318.Google Scholar
  4. Banta, T., and Borden, V. (1994). Performance indicators for accountability and improvement. In: Borden, V., and Banda, T. (eds.), Using Performance Indicators to Guide Strategic Decision Making, New Directions for Institutional Research, no. 82, Jossey-Bass, San Francisco, pp. 95–106.Google Scholar
  5. Bell, J., and Seater, J. (1978). Publishing performance: Departmental and individual. Econ. Inq. 16: 599–615.Google Scholar
  6. Bellas, M., and Toutkoushian, R. (1999). Faculty time allocation and research productivity: Gender, race and family effects. Rev. Higher Educ. 22: 367–390.Google Scholar
  7. Borokhovich, K., Bricker, R., Brunarski, K., and Simkins, B. (1995). Finance research productivity and influence. J. Finance 50: 1691–1717.Google Scholar
  8. Bowker, A. (1965). Quality and quantity in higher education. J. Am. Stat. Assoc. 60(309): 1–15.Google Scholar
  9. Braxton, J., and Bayer, A. (1986). Assessing faculty scholarly performance. In: Cresswell, J. W. (ed.), Measuring Faculty Research Performance, New Directions for Institutional Research, no. 50, Jossey-Bass, San Francisco, pp. 25–42.Google Scholar
  10. Braxton, J. (1996). Contrasting perspectives on the relationship between teaching and research. In: Braxton, J. (ed.), Faculty Teaching and Research: Is There a Conflict? New Directions for Institutional Research, no. 90, Jossey-Bass, San Francisco, pp. 5–14.Google Scholar
  11. Bray, N., Braxton, J., and Smart, J. (1996). Research activity and the support of undergraduate education. In: Braxton, J. (ed.), Faculty Teaching and Research: Is There a Conflict? New Directions for Institutional Research, no. 90, Jossey-Bass, San Francisco, pp. 23–30.Google Scholar
  12. Brinkman, P. (1981). Factors affecting instructional costs at major research universities. J. Higher Educ. 52: 265–279.Google Scholar
  13. Brown, D. (1967). The Mobile Professors, The American Council on Education, Washington, DC.Google Scholar
  14. Buchmueller, T., Dominitz, J., and Hansen, W. (1999). Graduate training and the early career productivity of Ph.D. economists. Econ. Educ. Rev. 18: 65–77.Google Scholar
  15. Cohn, E., Rhine, S., and Santos, M. (1989). Institutions of higher education as multiproduct firms: Economies of scale and scope. Rev. Educ. Stat. 71: 284–290.Google Scholar
  16. Creamer, E. (1998). Assessing faculty publication productivity: Issues of equity. ASHEERIC Higher Education Report Volume 26, No. 2, The George Washington University, Graduate School of Education and Human Development, Washington, DC.Google Scholar
  17. Davis, P., and Papanek, G. (1984). Faculty ratings of major economics departments by citations. Am. Econ. Rev. 74: 225–230.Google Scholar
  18. de Groot, H., McMahon, W., and Volkwein, J. (1991). The cost structure of American research universities. Rev. Econ. Stat. 73: 424–431.Google Scholar
  19. Diamond, A. (1986). What is a citation worth? J. Hum. Resour. 21: 200–215.Google Scholar
  20. Diamond, N., and Graham, H. (2000). How should we rate research universities? Change 32: 20–33.Google Scholar
  21. Dolence, M., and Norris, D. (1994). Using key performance indicators to drive strategic decision making. In: Borden, V., and Banda, T. (eds.), Using Performance Indicators to Guide Strategic Decision Making, New Directions for Institutional Research, no. 82, Jossey-Bass, San Francisco, pp. 63–80.Google Scholar
  22. Dundar, H., and Lewis, D. R. (1998). Determinants of research productivity in higher education.Res. Higher Educ. 39: 607–631.Google Scholar
  23. Ehrenberg, R., and Hurst, P. (1996). The 1995 NRC ratings of doctoral programs: A hedonic model. Change 28(3): 46–50.Google Scholar
  24. Feldman, K. (1987). Research productivity and scholarly accomplishment of college teachers as related to their instructional effectiveness: A review and exploration. Res. Higher Educ. 26: 227–298.Google Scholar
  25. Fox, M. (1992). Research, teaching and publication productivity: Mutuality versus competition in academia. Sociol. Educ. 65: 293–305.Google Scholar
  26. Gaither, G., Nedwek, B., and Neal, J. (1994). Measuring up: the Promises and pitfalls of performance indicators in higher education. ASHE-ERIC Higher Education Report No. 5, The George Washington University, Graduate School of Education and Human Development, Washington, DC.Google Scholar
  27. Gander, J. (1999). Faculty gender effects on academic research and teaching. Res. Higher Educ. 40: 171–184.Google Scholar
  28. Garfield, E. (1979). Citation Indexing: Its Theory and Applications in Science, Technology, and Humanities, Free Press, New York.Google Scholar
  29. Gavlick, M. (1996). Triangulating the relationships among publication productivity, teaching effectiveness, and student achievement. In: Braxton, J. (ed.), Faculty Teaching and Research: Is There a Conflict? New Directions for Institutional Research, no. 90, Jossey-Bass, San Francisco, pp. 49–56.Google Scholar
  30. Gerrity, D., and McKenzie, R. (1978). The ranking of southern economics departments: New criterion and further evidence. Southern Econ. J. 45: 608–614.Google Scholar
  31. Golden, J., and Carstensen, F. V. (1992). Academic research productivity, department size and organization: further results, comment. Econ. Educ. Rev. 11: 153–160.Google Scholar
  32. Graham, A., and Thompson, N. (2001). Broken ranks. The Washington Monthly, September, pp. 1–8. Available: http://www.washingtonmonthly.com/features/2001/0109.graham.thompson.hmlGoogle Scholar
  33. Graves, P., Marchand, J., and Thompson, R. (1982). Economics departmental rankings: Research incentives, constraints, and efficiency. Am. Econ. Rev. 72: 1131–1141.Google Scholar
  34. House, D., and Yeager, J. (1978). The distribution of publication success within and among top economics departments: A disaggregative view of recent evidence. Econ. Inq. 16: 593–598.Google Scholar
  35. Hughes, R. (1925). A Study of Graduate Schools of America, The Miami University Press, Oxford, OH.Google Scholar
  36. Hughes, R. (1934). Report of the Committee on Graduate Instruction. Educ. Record 15: 192–234.Google Scholar
  37. Johnes, J., and Johnes, G. (1995). Research funding and performance in U.K. university departments of economics: A frontier analysis. Econ. Educ. Rev. 14: 301–314.Google Scholar
  38. Jones, L., Lindzey, G., and Coggeshall, P. (eds.) (1982). An Assessment of Research-Doctorate Programs in the United States: Mathematical and Physical Sciences, The National Academy Press, Washington, DC.Google Scholar
  39. Jordan, J., Meador, M., and Walters, S. (1988). Effects of department size and organization on the research productivity of academic economists. Econ. Educ. Rev. 7: 251–255.Google Scholar
  40. Keith, B. (1999). University eminence and the contextual effect of institutional performance indicators. Paper presented at the 1999 meetings of the Association of Institutional Research, Seattle, WA.Google Scholar
  41. Keniston, H. (1959). Graduate Study in Research in the Arts and Sciences at the University of Pennsylvania, The University of Pennsylvania Press, Philadelphia.Google Scholar
  42. Koshal, R., and Koshal, M. (1999). Economies of scale and scope in higher education: A case of comprehensive universities. Econ. Educ. Rev. 18: 269–277.Google Scholar
  43. Lindsey, D. (1989). Using citation counts as a measure of quality in science. Scientometrics 15: 189–203.Google Scholar
  44. Lombardi, J., Craig, D., Capaldi, E., Gater, D., and Mendonca, S. (2001). The Top American Research Universities, The Center at the University of Florida, Gainesville.Google Scholar
  45. Long, J., and McGinnis, R. (1981). Organizational context and scientific productivity. Am. Sociol. Rev. 46: 422–442.Google Scholar
  46. Manis, J. (1951). Some academic influences upon publication productivity. Soc. Forces 29: 267–272.Google Scholar
  47. Miller, J., and Tollison, R. (1975). Rates of publications per faculty member in fortyfive 'rated' economics departments. Econ. Inq. 13: 122–123.Google Scholar
  48. Monks, J., and Ehrenberg, R. (1999). U.S. News and World Report's college rankings: Why do they matter? Change 31: 42–51.Google Scholar
  49. Nedwek, B. (1996). Public policy and public trust: The use and misuse of performance indicators in higher education. In: Smart, J. C. (ed.), Higher Education: Handbook of Theory and Research (Vol. 11), Agathon Press, New York, pp. 47–89.Google Scholar
  50. Noser, T., Manakyan, H., and Tanner, J. (1996). Research productivity and perceived teaching effectiveness: a survey of economics faculty. Res. Higher Educ. 37: 299–321.Google Scholar
  51. Patrick, W., and Stanley, E. (1998). Teaching and research quality indicators and the shaping of higher education. Res. Higher Educ. 39: 19–41.Google Scholar
  52. Perry, R., Clifton, R., Menec, V., Struthers, C., and Menges, R. (2000). Faculty in transition: A longitudinal analysis of perceived control and type of institution in the research productivity of newly hired faculty. Res. Higher Educ. 41: 165–194.Google Scholar
  53. Porter, S., and Umbach, P. (2001). Analyzing faculty workload using multilevel modeling. Res. Higher Educ. 42: 171–196.Google Scholar
  54. Ransom, M., and Megdal, S. (1993). Sex differences in the academic labor market in the affirmative action era. Econ. Educ. Rev. 12: 21–43.Google Scholar
  55. Rodenhouse, M. (ed.) (2001). 2001 Higher Education Directory, Higher Education Publications, Inc., Falls Church, VA.Google Scholar
  56. South Carolina Commission on Higher Education (2002). Performance funding guidelines and updates. Available: http://www.che400.state.sc.us/web /Perform/ReportCards/ Report_Frames.htm Southwick, L. (1969). Cost trends in land grant colleges and universities. Appl. Econ. 1: 167–182.Google Scholar
  57. Tan, D. (1986). The assessment of quality in higher education: A critical review. Res. Higher Educ. 24: 223–265.Google Scholar
  58. Taylor, B., and Massy, W. (1996). Strategic Indicators for Higher Education: Vital Benchmarks and Information to Help You Evaluate and Improve Your Institution's Performance, Peterson's, Princeton, NJ.Google Scholar
  59. Tien, F., and Blackburn, R. (1996). Faculty rank system, research motivation, and faculty research productivity. J. Higher Educ. 67: 2–22.Google Scholar
  60. Toutkoushian, R. (1994). Using citations to measure sex discrimination in faculty salaries. Rev. Higher Educ. 18: 61–82.Google Scholar
  61. Toutkoushian, R. (1999). The value of cost functions for policymaking and institutional research. Res. Higher Educ. 40: 1–15.Google Scholar
  62. Toutkoushian, R., and Danielson, C. (2003). Using performance indicators to evaluate decentralized budgeting systems and institutional performance. In: Priest, D. M., Becker, W. E., Hossler, D., and St. John, E. P. (eds.), Incentive-Based Budgeting Systems in Public Universities, Edward Elgar Publishing Inc., Northampton, MA, pp. 205–226.Google Scholar
  63. Toutkoushian, R., Dundar, H., and Becker, W. (1998). The National Research Council graduate program ratings: What are they measuring? Rev. Higher Educ. 21: 427–443.Google Scholar
  64. U.S. Department of Education (1997). Federal Support for Education: Fiscal years 1980 to 1997, NCES 97-383, by C. M. Hoffman, National Center for Education Statistics, Washington, DC.Google Scholar
  65. U.S. Department of Education (2001a). Digest of Education Statistics, 2000, NCES 2001-034, by T. D. Snyder and C. M. Hoffman, National Center for Education Statistics, Washington, DC.Google Scholar
  66. U.S. Department of Education (2001b). Background Characteristics, Work Activities, and Compensation of Faculty and Instructional Staff in Postsecondary Institutions: Fall 1998, NCES 2001-152, by L. J. Zimbler, National Center for Education Statistics, Washington, DC.Google Scholar
  67. Wanner, R., Lewis, L., and Gregorio, D. (1981). Research productivity in academia: A comparative study of the sciences, social sciences, and humanities. Sociol. Educ. 54: 238–253.Google Scholar
  68. Webster, D. (1983). America's highest ranking graduate schools, 1925-1982. Change 15(4): 14–24.Google Scholar
  69. Webster, D., and Skinner, T. (1996). Rating PhD programs: What the NRC report says... and doesn't say. Change 28(3): 22–44.Google Scholar
  70. Zheng, H., and Stewart, A. (2002). Assessing the performance of public research universities using NSF/NCES data and data envelopment analysis technique. AIR Prof. File 83: 1–21.xGoogle Scholar

Copyright information

© Human Sciences Press, Inc. 2003

Authors and Affiliations

  • Robert K. Toutkoushian
    • 1
  • Stephen R. Porter
    • 2
  • Cherry Danielson
    • 3
  • Paula R. Hollis
    • 3
  1. 1.Office of Policy AnalysisUniversity System of New HampshireUSA
  2. 2.Office of Institutional ResearchWesleyan UniversityUSA
  3. 3.Office of Policy AnalysisUniversity System of New HampshireUSA

Personalised recommendations