The Dimensionality of Student Ratings of Instruction: What We Know and What We Do Not*

  • Philip C. Abrami
  • Sylvia d’Apollonia
  • Steven Rosenfield

Keywords

Student ratings effective teaching Scholarship of teaching and learning post-secondary education meta-analysis 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abrami, P.C. (1984, February). Using meta-analytic techniques to review the instructional evaluation literature. Postsecondary Education Newsletter 6: 8.Google Scholar
  2. Abrami, P.C. (1985). Dimensions of effective college instruction. Review of Higher Education 8: 211–228.Google Scholar
  3. Abrami, P.C. (1988). SEEQ and ye shall find: A review of Marsh’s ‘‘Students’ evaluation of university teaching.’’ Instructional Evaluation 9(2): 19–27.Google Scholar
  4. Abrami, P.C. (1989a). SEEQing the truth about student ratings of instruction. Educational Researcher 43(1): 43–45.CrossRefGoogle Scholar
  5. Abrami, P.C. (1989b). How should we use student ratings to evaluate teaching? Research in Higher Education 30: 221–227.CrossRefGoogle Scholar
  6. Abrami, P.C., Chambers, B., Poulsen, C., DeSimone, C., d’Apollonia, S., and Howden, J. (1995). Classroom Connections: Understanding and Using Cooperative Learning. Toronto, Ontario: Harcourt Brace.Google Scholar
  7. Abrami, P.C., Cohen, P.A., and d’Apollonia, S. (1988). Implementation problems in meta-analysis. Review of Educational Research 58: 151–179.CrossRefGoogle Scholar
  8. Abrami, P.C., and d’Apollonia, S. (1987, April). A Conceptual Critique of Meta-analysis: The Literature on Student Ratings of Instruction. Paper presented at the annual meeting of the American Educational Research Association, Washington, DC.Google Scholar
  9. Abrami, P.C., and d’Apollonia, S. (1988, April). The Literature on Student Ratings of Instruction: A Conceptual Solution to Some Implementation Problems of Meta-analysis. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.Google Scholar
  10. Abrami, P.C., and d’Apollonia, S. (1990). The dimensionality of ratings and their use in personnel decisions. In M. Theall and J. Franklin (eds.) Student Ratings of Instruction: Issues for Improving Practice. New Directions for Teaching and Learning. Number 43, pp. 97–111. San Francisco: Jossey-Bass.Google Scholar
  11. Abrami, P.C., and d’Apollonia, S. (1991). Multidimensional students’ evaluations of teaching effectiveness-Generalizability of ‘‘N=1’’ research: Comment on Marsh (1991). Journal of Educational Psychology 83: 411–415.CrossRefGoogle Scholar
  12. Abrami, P.C., d’Apollonia, S., and Cohen, P.A. (1990). The validity of student ratings of instruction: What we know and what we do not. Journal of Educational Psychology 82: 219–231.CrossRefGoogle Scholar
  13. Abrami, P.C., Leventhal, L., and Perry, R.P. (1982). Educational seduction. Review of Educational Research 52: 446–464.CrossRefGoogle Scholar
  14. Abrami, P.C., and Mizener, D.A. (1985). Student/instructor attitude similarity, student ratings, and course performance. Journal of Educational Psychology 77: 693–702.CrossRefGoogle Scholar
  15. Anderson, N.H. (1968). Likableness ratings of 555 personality-trait words. Journal of Personality and Social Psychology 9: 272–279.CrossRefGoogle Scholar
  16. Borich, G.D. (1977). The Appraisal of Teaching: Concepts and Process. Reading, MA: Addison-WesleyGoogle Scholar
  17. Bushman, B.J., Cooper, H.M., and Lemke, K.M. (1991). Meta-analysis of factor analyses: An illustration using the Buss-Durke Hostility Inventory. Personality and Social Psychology Bulletin 17: 344–349.CrossRefGoogle Scholar
  18. Bloom, B.S., Engelhart, M.D., Frost, E.J., Hill, W.H., and Krathwohl, D.R. (1956). Taxonomy of Educational Objectives. Handbook I: Cognitive Domain. New York: David McKay.Google Scholar
  19. Campbell, D.T., and Stanley, J.C. (1966). Experimental and Quasi-experimental Designs for Research. Boston: Houghton-Mifflin.Google Scholar
  20. Cashin, W.E., and Downey, R.G. (1992). Using global student rating items for summative evaluations. Journal of Educational Psychology 84: 563–572.CrossRefGoogle Scholar
  21. Cohen, P.A. (1981). Student ratings of instruction and student achievement: A meta-analysis of multisection validity studies. Review of Educational Research 51: 281–309.CrossRefGoogle Scholar
  22. Cohen, P.A. (1982). Validity of student ratings in psychology courses: A meta-analysis of multisection validity studies. Teaching of Psychology 9: 78–82.CrossRefGoogle Scholar
  23. Cohen, P.A. (1983). Comment on a selective review of the validity of student ratings of teaching. Journal of Higher Education 54: 448–458.CrossRefGoogle Scholar
  24. Cohen, P.A. (1986, April). An Updated and Expanded Meta-analysis of Multisection Student Rating Validity Studies. Paper presented at the annual meeting of the American Educational Research Association, San Francisco. CA.Google Scholar
  25. Cohen, P.A. (1987, April). A Critical Analysis and Reanalysis of the Multisection Validity Meta-analysis. Paper presented at the annual meeting of the American Educational Research Association, Washington, DC.Google Scholar
  26. Cooper, J., Prescott, S., Cook, L., Smith, L., Mueck, R., and Cuseo, J. (1990). Cooperative learning and College Instruction: Effective Use of Student Learning Teams. Long Beach, CA: California State University Foundation on behalf of California State University Institute for Teaching and Learning, Office of the Chancellor.Google Scholar
  27. d’Apollonia, S. and Abrami, P.C. (1987, April). An Empirical Critique of Meta-analysis: The Literature on Student Ratings of Instruction. Paper presented at the annual meeting of the American Educational Research Association, Washington, DC.Google Scholar
  28. d’Apollonia, S. and Abrami, P.C. (1988, April). The Literature on Student Ratings of Instruction: Yet Another Meta-analysis. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.Google Scholar
  29. d’Apollonia, S., Abrami, P., and Rosenfield, S. (1993, April). The Dimensionality of Student Ratings of Instruction: A Meta-analysis of the Factor Studies. Paper presented at the annual meeting of the American Educational Research Association, Atlanta, GA.Google Scholar
  30. d’Apollonia, S., Abrami, P., and Rosenfield, S. (in preparation). A multivariate meta-analysis of the multisection validity studies.Google Scholar
  31. Doyle, K.O., Jr. (1981). Validity and perplexity: An incomplete list of disturbing issues. Instructional Evaluation, 6(1): 23–25.Google Scholar
  32. Doyle, K.O., and Crichton, L.I. (1978). Student, peer, and self-evaluations of college instructors. Journal of Educational Psychology 5: 815–826.CrossRefGoogle Scholar
  33. Feldman, K.A. (1976). The superior college teacher from the student’s view. Research in Higher Education 5: 243–288.CrossRefGoogle Scholar
  34. Feldman, K.A. (1977). Consistency and variability among college students in rating their teachers and courses: A review and analysis. Research in Higher Education 6: 223–274.CrossRefGoogle Scholar
  35. Feldman, K.A. (1983). Seniority and experience of college teachers as related to evaluations they receive from students. Research in Higher Education 18: 3–214.CrossRefGoogle Scholar
  36. Feldman, K.A. (1984). Class size and college students’ evaluations of teachers and courses: A closer look. Research in Higher Education 21: 45–116.CrossRefGoogle Scholar
  37. Feldman, K.A. (1988). Effective college teaching from the students’ and faculty’s view: Matched or mismatched priorities? Research in Higher Education 28: 291–344.CrossRefGoogle Scholar
  38. Feldman, K.A. (1989a). The association between student ratings of specific instructional dimensions and student achievement: Refining and extending the synthesis of data from multisection validity studies. Research in Higher Education 30: 583–645.CrossRefGoogle Scholar
  39. Feldman, K.A. (1989b). Instructional effectiveness of college teachers as judged by teachers themselves, current and former students, colleagues, administrators, and external (neutral) observers. Research in Higher Education 30: 137–194.CrossRefGoogle Scholar
  40. Feldman, K.A. (1990). An afterword for ‘‘The association between student ratings of specific instructional dimensions and student achievement: Refining and extending the syn<??>hesis of data from multisection validity studies.’’ Research in Higher Education 31: 315–318.CrossRefGoogle Scholar
  41. Fernald, P.S. (1990). Students’ ratings of instruction: Standardized and customized. Teaching of Psychology 17: 105–109.CrossRefGoogle Scholar
  42. Franklin, J., and Theall, M. (1989, April). Rating the Readers: Knowledge, Attitude, and Practice of Users of Student Ratings of Instruction. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco.Google Scholar
  43. Frey, P.W. (1978). A two dimensional analysis of student ratings of instruction. Research in Higher Education 9: 69–91.CrossRefGoogle Scholar
  44. Gaski, J.F. (1987). On ‘‘Construct validity of measures of college teaching effectiveness.’’ Journal of Educational Psychology 79: 326–330.CrossRefGoogle Scholar
  45. Gorsuch, R.I. (1983). Factor Analysis. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  46. Hedges, L.V., and Olkin, I. (1985). Statistical Methods for Meta-analysis. Orlando, FL: Academic Press.Google Scholar
  47. Hill, P.W. (1984). Testing hierarchy in educational taxonomies: A theoretical and empirical investigation. Evaluation Education 8: 181–278.Google Scholar
  48. Howard, G.S., Conway, C.G., and Maxwell, S.E. (1985). Construct validity of measures of college teaching effectiveness. Journal of Educational Psychology 77: 187–196.CrossRefGoogle Scholar
  49. Isaacson, R.L., McKeachie, W.J., Milholland, J.E., Lin, Y.G., Hofeller, M., Baerwaldt, J.W., and Zinn, K.L. (1964). Dimensions of student evaluations of teaching. Journal of Educational Psychology 55: 344–351.CrossRefGoogle Scholar
  50. Johnson, D.W., Johnson, R.T., and Smith, K.A. (1991). Active Learning: Cooperation in the College Classroom. Edina, MN: Interaction Book Co.Google Scholar
  51. Kaiser, H.F., Hunka, S., and Bianchini, J.C. (1969). Relating Factors Between Studies Based Upon Different Individuals. In H.J. Eysenck (ed.), Personality Structure and Measurement. San Diego, CA: Knapp.Google Scholar
  52. Kulik, J.A., and McKeachie, W.J. (1975). The evaluation of teachers in higher education. Review of Research in Education 3: 210–240.CrossRefGoogle Scholar
  53. Linn, R.L., Centra, J.A., and Tucker, L. (1975). Between, within, and total group factor analysis of student ratings of instruction. Multivariate Behavioral Research 10: 277–288.CrossRefGoogle Scholar
  54. Marsh, H.W. (1987). Students’ evaluations of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research 11: 253–388.CrossRefGoogle Scholar
  55. Marsh, H.W. (1991). Multidimensional students’ evaluations of teaching effectiveness: A test of alternative higher-order structures. Journal of Educational Psychology 83: 285–296.CrossRefGoogle Scholar
  56. Marsh, H.W. (1991). A multidimensional perspective on students’ evaluations of teaching effectiveness: Reply to Abrami and d’Apollonia. Journal of Educational Psychology 83: 416–421.CrossRefGoogle Scholar
  57. Marsh, H.W., and Dunkin, M.J. (1992). Students’ evaluations of university teaching: A multidimensional perspective. In J. Smart (ed.), Higher Education: Handbook of Theory and Research, Vol. VIII. New York: Agathon Press.Google Scholar
  58. Marsh, H.W., and Hocevar, D. (1984). The factorial invariance of student evaluations of college teaching. American Educational Research Journal 21: 341–366.CrossRefGoogle Scholar
  59. Marsh, H.W., and Hocevar, D. (1991). The multidimensionality of students’ evaluations of teaching effectiveness: The generality of factor structures across academic discipline, instructor level, and course level. Teaching and Teacher Education 7: 9–18.CrossRefGoogle Scholar
  60. Maxwell, S.E., and Howard, G.S. (1987). On the underdetermination of theory by evidence. Journal of Educational Psychology 79: 331–332.CrossRefGoogle Scholar
  61. McCallum, L.W. (1984). A meta-analysis of course evaluation data and its use in the tenure decision. Research in Higher Education 21: 150–158.CrossRefGoogle Scholar
  62. Murray, H.G. (1991). Effective Teaching Behaviors in the College Classroom. In J. Smart (ed.), Higher Education, Handbook of Theory and Research (Vol. 6) New York: Agathon.Google Scholar
  63. Murray, H.G., Rushton, J.P., and Paunonen, S.V. (1990). Teacher personality traits and student instructional ratings in six types of university courses. Journal of Educational Psychology 82: 250–261.CrossRefGoogle Scholar
  64. Osgood, C.E., Suci, G.J., and Tannenbaum, P.H. (1957). The Measurement of Meaning. Urbana, IL: University of Illinois Press.Google Scholar
  65. Rosenfield, S., d’Apollonia, S., and Abrami, P.C. (1993, April). The Dimensionality of Student Ratings of Instruction: Aggregating Factor Studies. Paper presented at the annual meeting of the American Educational Research Association, Atlanta, GA.Google Scholar
  66. Seldin, P. (1991). The Teaching Portfolio. Bolton, MA: Anker Publishing Co.Google Scholar
  67. Shore, B.M., Foster, S.F., Knapper, C.K., Nadeau, C.G., Neill, N., and Sim, V.W. (1986). The Teaching Dossier: A Guide to Its Preparation and Use. Ottawa: Canadian Association of University Teachers.Google Scholar
  68. Smith, R.A., and Cranton, P.A. (1992). Students’ perceptions of teaching skills and overall effectiveness across instructional settings. Research in Higher Education 33: 747.CrossRefGoogle Scholar
  69. Sullivan, A.M., and Skanes, G.R. (1974). Validity of student evaluations of teaching and the characteristics of successful instructors. Journal of Educational Psychology 66: 584–590.CrossRefGoogle Scholar
  70. Tabachnick, B.G., and Fidell, L.S. (1983). Using Multivariate Statistics. New York: Harper and Row.Google Scholar
  71. Thomson, B. (1989). Meta-analysis of factor structure studies: A case study example with Bem’s Androgyny measure. Journal of Experimental Education 57: 182, 197.Google Scholar
  72. Wherry, R.L. (1951). The Control of Bias in Ratings: Factor Analysis of Rating Item Content. Columbus: The Ohio State University Research Foundation, United States Army, AGO, Personnel Research Branch, PRB Report No. 919.Google Scholar
  73. Widlak, F.W., McDaniel, E.D., and Feldhusen, J.F. (1973). Factor Analyses of an Instructor Rating Scale. Paper presented at the annual meeting of the American Educational Research Association, New Orleans. (ERIC Document Reproduction Service ED 079324).Google Scholar

Copyright information

© Springer 2007

Authors and Affiliations

  • Philip C. Abrami
    • 1
  • Sylvia d’Apollonia
    • 2
  • Steven Rosenfield
    • 3
  1. 1.Concordia UniversityMontrealCanada
  2. 2.Vanier CollegeMontrealCanada
  3. 3.American Comedy InstituteSan FranciscoUSA

Personalised recommendations