Advertisement

Research in Higher Education

, Volume 26, Issue 1, pp 71–84 | Cite as

Faculty evaluation: Reliability of peer assessments of research, teaching, and service

  • Lawrence S. Root
Article

Abstract

In this paper, assessments of faculty performance for the determination of salary increases are analyzed to estimate interrater reliability. Using the independent ratings by six elected members of the faculty, correlations between the ratings are calculated and estimates of the reliability of the composite (group) ratings are generated. Average intercorrelations are found to range from 0.603 for teaching, to 0.850 for research. The average intercorrelation for the overall faculty ratings is 0.794. Using these correlations, the reliability of the six-person group (the composite reliability) is estimated to be over 0.900 for each of the three areas and 0.959 for the overall faculty rating. Furthermore, little correlation is found between the ratings of performance levels of individual faculty members in the three areas of research, teaching, and service. The high intercorrelations and, consequently, the high composite reliabilities suggest that a reduction in the number of raters would have relatively small effects on reliability. The findings are discussed in terms of their relationship to issues of validity as well as to other questions of faculty assessment.

Keywords

Faculty Member Faculty Performance Faculty Rating Performance Level Education Research 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ceci, S. J., and Peters, D. P. (1982). Peer review: A study of reliability.Change 14(6): 44–48.Google Scholar
  2. Centra, J. A. (1979).Determining Faculty Effectiveness. San Francisco: Jossey-Bass.Google Scholar
  3. Cohen, P. A. (1983). Comment on ‘A selective review of the validity of student ratings of teaching.’Journal of Higher Education 54: 448–458.Google Scholar
  4. Cohen, P. A., and McKeachie, W. J. (1980). The role of colleagues in the evaluation of college teaching.Improving College and University Teaching 28: 147–154.Google Scholar
  5. Dowell, D. A., and Neal, J. A. (1982). A selective review of the validity of student ratings of teaching.Journal of Higher Education 53: 51–62.Google Scholar
  6. Doyle, K. O., Jr. (1983).Evaluating Teaching. Lexington, MA: Lexington Books.Google Scholar
  7. Eble, K. E. (1972).The Recognition and Evaluation of Teaching. Washington, D.C.: American Association of University Professors.Google Scholar
  8. Feldman, K. A. (1977). Consistency and variability among college students in rating their teachers and courses: a review and analysis.Research in Higher Education 6: 223–274.Google Scholar
  9. French-Lazovik, G. (1981). Peer review: documentary evidence in the evaluation of teaching. In J. Millman (Ed.),Handbook of Teacher Evaluation pp. 73–89. Beverly Hills, CA: Sage.Google Scholar
  10. Greene, R. J., and Wallace, M. J. (1984). Is there/can there be merit in merit programs?ACA 1984 Conference Proceedings, pp. 12–19. Scottsdale, AR: American Compensation Association.Google Scholar
  11. Guilford, J. P. (1954).Psychometric Methods. New York: McGraw-Hill.Google Scholar
  12. Hildebrand, M., Wilson, R. C., and Dienst, E. R. (1971).Evaluating University Teaching. Berkeley, CA: University of California Center for Research and Development in Higher Education.Google Scholar
  13. Johnson, M., and Kasten, K. (1983). Meritorious work and faculty rewards: An empirical test of the relationship.Research in Higher Education 19: 49–71.Google Scholar
  14. Kasten, K. (1984). Tenure and merit pay as rewards for research, teaching, and service at a research university.Journal of Higher Education 55: 500–514.Google Scholar
  15. Katz, D. A. (1973). Faculty salaries, promotions, and productivity at a large university.American Economic Review 63: 469–477.Google Scholar
  16. Keaveny, T. J., and Allen, R. E. (1983). The implications of an across-the-board salary increase.Research in Higher Education 19: 11–24.Google Scholar
  17. Kulik, J. A., and McKeachie, W. J. (1975). The evaluation of teachers in higher education. In F. N. Kerlinger (Ed.),Review of Research in Higher Education, Vol. 3, pp. 210–240. Itasca, IL: Peacock.Google Scholar
  18. Lee, B. A. (1985). Federal court involvement in academic personnel decisions.Journal of Higher Education 56: 38–54.Google Scholar
  19. Machalak, S. J., Jr., and Friedrich, R. J. (1981). Research productivity and teaching effectiveness at a small liberal arts college.Journal of Higher Education 52: 578–597.Google Scholar
  20. Marsh, H. W. (1984). Students' evaluations of university teaching: Dimensionality, reliability, validity, potential biases, and utility.Journal of Educational Psychology 76: 707–754.Google Scholar
  21. McKeachie, W. J. (1979). Financial incentives are ineffective for faculty. In Lewis, D. R. and Becker, W. E. (Eds.),Academic Reward in Higher Education pp. 3–20. Cambridge, MA: Ballinger.Google Scholar
  22. Miller, A. C., and Serzan, S. L. (1984). Criteria for identifying a refereed journal.Journal of Higher Education 55: 673–699.Google Scholar
  23. Miller, D. A. (1978). Criteria for appointment, promotion, and retention of faculty in graduate social work programs.Journal of Education for Social Work 14(2): 74–81.Google Scholar
  24. Powell, R. W. (1977). Grades, learning, and student evaluation of instruction.Research in Higher Education 7: 193–205.Google Scholar
  25. Rawls, J. (1971).A Theory of Justice. Cambridge, MA: Harvard University Press.Google Scholar
  26. Ryan, J. J., Anderson, J. A., and Birchler, A. B. (1980). Student evaluations: The faculty responds.Research in Higher Education 12: 312–333.Google Scholar
  27. Tuckman, H. P., Gapinski, J. H., and Hagemann, R. P. (1977). Faculty skills and salary structure in academe: A market perspective.American Economic Review 67: 692–702.Google Scholar

Copyright information

© Agathon Press, Inc. 1987

Authors and Affiliations

  • Lawrence S. Root
    • 1
  1. 1.School of Social WorkThe University of MichiganAnn Arbor

Personalised recommendations