Regular Feedback from Student Ratings of Instruction: Do College Teachers Improve their Ratings in the Long Run?
The authors examined whether feedback from student ratings of instruction not augmented with consultation helps college teachers to improve their student ratings on a long-term basis. The study reported was conducted in an institution where no previous teaching-effectiveness evaluations had taken place. At the end of each of four consecutive semesters, student ratings were assessed and teachers were provided with feedback. Data from 3122 questionnaires evaluating 12 teachers were analyzed using polynomial and piecewise random coefficient models. Results revealed that student ratings increased from the no-feedback baseline semester to the second semester and then gradually decreased from the second to the fourth semester, although feedback was provided after each semester. The findings suggest that student ratings not augmented with consultation are far less effective than typically assumed when considered from a long-term perspective.
Keywordsfeedback long-term effects student ratings teaching effectiveness
Unable to display preview. Download preview PDF.
We would like to thank Jessica Ippolito, Anette Kluge, Jan Schilling, and two anonymous reviewers for their helpful comments on an earlier version of this article, and Paul D. Bliese for answering questions on random coefficient modeling and data aggregation. Further thanks go to Susannah Goss for improving the language of this article.
- Akaike H. (1973). Information theory as an extension of the maximum likelihood principle. In: Petrov B.N., Csaki F. (eds). Second international symposium on information theory. Akademiai Kiado, Budapest, Hungary, pp. 267–281Google Scholar
- Bliese P.D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In: Klein K.J., Kozlowski S.W.J. (eds). Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions. Jossey-Bass, San Francisco, CA, pp. 349–381Google Scholar
- Bliese P.D., Ployhart R.E. (2002). Growth modeling using random coefficient models: Model building, testing, and illustrations. Organizational Research Methods 5:362–387Google Scholar
- Campbell D.T., Stanley J.C. (1963). Experimental and quasi-experimental designs for research. Rand McNally, ChicagoGoogle Scholar
- Cohen J., Cohen P., West S.G., Aiken L.S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed). Erlbaum, Mahwah, NJGoogle Scholar
- Diehl, J.M. (2002) VBVOR–VBREF. Fragebögen zur studentischen Evaluation von Hochschulveranstaltungen – Manual. [VBVOR – VBREF. Questionnaires for students’ evaluations of college courses–Manual]. Retrieved on March 17, 2005 from http://www.psychol.uni-giessen.de/dl/det/diehl/2368/
- Diehl J.M. (2003). Normierung zweier Fragebögen zur studentischen Beurteilung von Vorlesungen und Seminaren [Student evaluations of lectures and seminars: Norms for two recently developed questionnaires]. Psychologie in Erziehung und Unterricht 50:27–42Google Scholar
- Diehl J.M., Kohr H.-U. (1977). Entwicklung eines Fragebogens zur Beurteilung von Hochschulveranstaltungen im Fach Psychologie [Development of a psychology course evaluation questionnaire]. Psychologie in Erziehung und Unterricht 24:61–75Google Scholar
- Franklin J., Theall M. (2002). Faculty thinking about the design and evaluation of instruction. In: Hativa N., Goodyear P. (eds). Teacher thinking, beliefs and knowledge in higher education. Kluwer, Dordrecht, The NetherlandsGoogle Scholar
- Longford N. (1993). Random coefficient models. Oxford University Press, OxfordGoogle Scholar
- Marsh H.W. (1994). Comments to: “Review of the dimensionality of student ratings of instruction: I. Introductory remarks. II. Aggregation of factor studies. III. A meta-analysis of the factor studies”. Instructional Evaluation and Faculty Development 14:13–19Google Scholar
- Pinheiro J.C., Bates D.M. (2000). Mixed-effects models in S and S-PLUS. Springer, New YorkGoogle Scholar
- R Development Core Team (2005). R: A language and environment for statistical computing. R Foundation for Statistical Computing, ViennaGoogle Scholar
- Raudenbush S.W., Bryk A.S. (2002). Hierarchical linear models: Applications and data analysis methods, 2nd edn. Sage, Thousand Oaks, CAGoogle Scholar
- Rousseau D. (1985). Issues of level in organizational research: Multi-level and cross-level perspectives. In: Cummings L.L., Staw B.M. (eds). Research in organizational behavior, vol. 7. JAI Press, Greenwich, CT, pp. 1–37Google Scholar
- Schwarz G. (1978). Estimating the dimension of a model. Annals of Statistics 6:461–464Google Scholar
- Wood R.E., Locke E.A. (1990). Goal setting and strategy effects on complex tasks. In: Cummings L.L., Staw B.M. (eds). Research in organizational behavior, vol. 12. JAI Press, Greenwich, CT, pp. 73–109Google Scholar