Skip to main content
SpringerLink
Log in

The Relative Merits of Different Types of Overall Evaluations of Teaching Effectiveness

  • Published:
Research in Higher Education Aims and scope Submit manuscript

Abstract

An unresolved issue in student evaluations of teaching effectiveness (SETE) literature is what type of overall evaluation of teaching effectiveness should be used in personnel decisions. The objective of this study is to compare the merits of: (a) an overall evaluation made by students, (b) a weighted average overall evaluation with the weights determined by students, (c) a weighted average overall evaluation with the weights being determined by the individual instructors teaching their respective classes, (d) an unweighted average overall evaluation, and (e) a second-order factor that proxies for an overall evaluation. Our results indicate that: (a) all of these overall evaluations were very highly intercorrelated, (b) the unweighted and weighted average overall evaluations measured virtually the same thing, and (c) the second-order factor that served as an overall evaluation was most highly correlated with the other overall evaluations and had the advantage of being most understandable to the faculty.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

REFERENCES

  • Abrami, P. C. (1985). Dimensions of effective college instruction. Review of Higher Education 8(3): 211-228.

    Google Scholar 

  • Abrami, P. C. (1989). How should we use student ratings to evaluate teaching? Research in Higher Education 30: 221-227.

    Google Scholar 

  • Abrami, P. C., and d'Apollonia, S. (1990). The dimensionality of ratings and their use in personnel decisions. In: Theall, M., and Franklin, J. (eds), New Directions for Teaching and Learning Student Ratings of Instruction: Issues for Improving Practice (No. 43), Jossey-Bass, San Francisco, pp. 97-111.

    Google Scholar 

  • Abrami, P. C., and d'Apollonia, S. (1991). Multidimensional students' evaluations of teaching effectiveness-generalizability of “N = 1” research: Comment on Marsh (1991). Journal of Educational Psychology 83(3): 411-415.

    Google Scholar 

  • Abrami, P. C., and d'Apollonia, S. (1999). Current concerns are past concerns. American Psychologist 54(7): 519-520.

    Google Scholar 

  • Burdsal, C. A., and Bardo, J. W. (1986). Measuring student's perceptions of teaching: Dimensions of evaluation. Educational and Psychological Measurement 46: 63-79.

    Google Scholar 

  • Cashin, W. E., and Downey, R. G. (1992). Using global student rating items for summative evaluation. Journal of Educational Psychology 84(4): 563-572.

    Google Scholar 

  • d'Apollonia, S. and Abrami, P. C. (1997). Navigating student ratings of instruction. Journal of Educational Psychology 52(11): 1198-1208.

    Google Scholar 

  • Feldman, K. A. (1988). Effective college teaching from the students' and faculty view: Matched or mismatched priorities? Research in Higher Education 28: 291-344.

    Google Scholar 

  • Frey, P. W. (1973). Student ratings of teaching: Validity of several rating factors. Science 182: 83-85.

    Google Scholar 

  • Frey, P. W. (1974, February). The ongoing debate: Student evaluation of teaching. Change 6(1): 47-49.

    Google Scholar 

  • Frey, P. W. (1978). A two-dimensional analysis of student ratings of instruction. Research in Higher Education 9: 69-91.

    Google Scholar 

  • Gillmore, G. M., and Greenwald, A. G., (1999). Using statistical adjustment to reduce bias in student ratings. American Psychologist 54(7): 518-519.

    Google Scholar 

  • Greenwald, A. G. (1997). Validity concerns and usefulness of student ratings of instruction. Journal of Educational Psychology 52(11): 1182-1186.

    Google Scholar 

  • Greenwald, A. G., and Gillmore, G. M. (1997). Grading leniency is a removable contaminant of student ratings. Journal of Educational Psychology 52(11): 1209-1217.

    Google Scholar 

  • Harrison, P. D., Ryan, J. M., and Moore, P.S. (1996). College students' self-insight and common implicit theories in ratings of teaching effectiveness. Journal of Educational Psychology 88(4): 775-782.

    Google Scholar 

  • L'Hommedieu, R., Menges, R. J., and Brinko, K. T. (1990). Methodological explanations for the modest effects of feedback. Journal of Educational Psychology 82: 232-241.

    Google Scholar 

  • Jackson, D. L., Teal, C. R., Raines, S. J., Nansel, T. R., Force, R. C., and Burdsal, C. A. (1999). The dimensions of students' perceptions of teaching effectiveness. Educational and Psychological Measurement 59(4): 580-596.

    Google Scholar 

  • Marsh, H. W. (1977). The validity of students' evaluations: Classroom evaluations of instructors independently nominated as best and worst teachers by graduating seniors. American Educational Research Journal 14(4): 441-447.

    Google Scholar 

  • Marsh, H. W. (1982). SEEQ: A reliable, valid, and useful instrument for collecting students' evaluations of university teaching. British Journal of Educational Psychology 52(1): 77-95.

    Google Scholar 

  • Marsh, H. W. (1983). Multidimensional ratings of teaching effectiveness by students for different academic settings and their relationship to student/course/instructor characteristics. Journal of Educational Psychology 75(1): 150-166.

    Google Scholar 

  • Marsh, H. W. (1984). Students' evaluations of university teaching dimensionality, reliability, validity, potential biases, and utility. Journal of Educational Psychology 76(5): 707-754.

    Google Scholar 

  • Marsh, H. W. (1986). Global self-esteem: Its relation to specific facets of self-concept and their importance. Journal of Personality and Social Psychology 51: 1224-1236.

    Google Scholar 

  • Marsh, H. W. (1991). A multidimensional perspective on students' evaluations of teaching effectiveness: A reply to Abrami and d'Apollonia. Journal of Educational Psychology 83(3): 416-421.

    Google Scholar 

  • Marsh, H. W. (1994). Weighting for the right criteria to validate student evaluations of teaching in the IDEA system. Journal of Educational Psychology 86(4): 631-648.

    Google Scholar 

  • Marsh, H. W., and Dunkin, M. (1992). Students' evaluations of university teaching: A multidimensional perspective. Higher Education: Handbook on Theory and Research (Vol. 8), Agathon, New York.

    Google Scholar 

  • Marsh, H. W., Fleiner, H., and Thomas, C. S. (1975). Validity and usefulness of student evaluations of instructional quality. Journal of Educational Psychology 67(6): 833-839.

    Google Scholar 

  • Marsh, H. W., and Hocevar, D. (1991). The multidimensionality of students' evaluations of teaching effectiveness: The generality of factor structures across academic discipline, instructor level, and course level. Teaching & Teacher Education 7(1): 9-18.

    Google Scholar 

  • Marsh, H. W., and Overall, J. U. (1980). Validity of students' evaluations of teaching effectiveness: Cognitive and affective criteria. Journal of Educational Psychology 72: 468-475.

    Google Scholar 

  • Marsh, H. W., Overall, J. U., and Kesler, S. P. (1979). Class size, students' evaluations, and instructional effectiveness. American Educational Research Journal 16(1): 57-70.

    Google Scholar 

  • Marsh, H. W., and Roche, L. A. (1992). The use of student evaluations of university instructors in different settings. Australian Journal of Education 36: 278-300.

    Google Scholar 

  • Marsh, H. W., and Roche, L. A. (1993). The use of students' evaluations and an individually structured intervention to enhance university teaching effectiveness. American Educational Research Journal 30(1): 217-251.

    Google Scholar 

  • Marsh, H. W., and Roche, L. A. (1997). Making students' evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. Journal of Educational Psychology 52(11): 1187-1197.

    Google Scholar 

  • Marsh, H. W., and Roche, L. A. (1999). Rely upon SET research. American Psychologist 54(7): 517-518

    Google Scholar 

  • McKeachie, W. J. (1997). Student ratings: The validity of use. Journal of Educational Psychology 52(11): 1218-1225.

    Google Scholar 

  • Ryan, J. M., and Harrison, P. D. (1995). The relationship between individual instructional characteristics and the overall assessment of teaching effectiveness across different instructional contexts. Research in Higher Education 36: 213-228.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Business, Wichita State University, 1845 Fairmont, Wichita, KS, 67260–0087

    Paul D. Harrison

  2. Wichita State University, Wichita, KS

    Deanna K. Douglas & Charles A. Burdsal

Authors
  1. Paul D. Harrison
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Deanna K. Douglas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Charles A. Burdsal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul D. Harrison.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Harrison, P.D., Douglas, D.K. & Burdsal, C.A. The Relative Merits of Different Types of Overall Evaluations of Teaching Effectiveness. Research in Higher Education 45, 311–323 (2004). https://doi.org/10.1023/B:RIHE.0000019592.78752.da

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:RIHE.0000019592.78752.da

Search

Navigation