Abstract
Lecturer promotion and tenure decisions are critical both for university management and for the affected lecturers. Therefore, they should be made cautiously and based on reliable information. Student evaluations of teaching quality are among the most used and analysed sources of such information. However, to date little attention has been paid in how to process them in order to be able to estimate their reliability. Within this paper we present an approach that provides estimates of such reliability in terms of confidence intervals. This approach, based on Bayesian inference, also provides a means for improving reliability even for lecturers having a low number of student evaluations. Such improvement is achieved by using past information in every year’s evaluations. Results of applying the proposed procedure to university-wide data corresponding to two consecutive years are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abrami, P., d’Apollonia, S., & Rosenfield, S. (2007). The dimensionality of student ratings of instruction: What we know and what we do not. In R. P. Perry & J. C. Smart (Eds.), The scholarship of teaching and learning in higher education: An evidence-based perspective (pp. 385–456). New York: Springer.
Aleamoni, L. M. (1999). Student rating myths versus research facts from 1924 to 1998. Journal of Personnel Evaluation in Education, 13(2), 153–166.
ANECA. (2006). DOCENTIA: Modelo de evaluación. Programa de apoyo para la evaluación de la actividad docente del profesorado universitario, Agencia Nacional de Evaluación de la Calidad y Acreditación, http://www.aneca.es/Programas/DOCENTIA.
Baldwin, T., & Blattner, N. (2003). Guarding against potential bias in student evaluations: What every faculty member needs to know. College Teaching, 51(1), 27–32.
Bedggood, R. E., & Donovan, J. D. (2012). University performance evaluations: What are we really measuring? Studies in Higher Education, 37(7), 825–842.
Cashin, W. E. (1996). Developing an effective faculty evaluation system. IDEA paper # 33, The IDEA Center, Kansas University, http://eric.ed.gov/?id=ED395536.
Dommeyer, C. J., Baum, P., Hanna, R. W., & Chapman, K. S. (2004). Gathering faculty teaching evaluations by in-class and online surveys: Their effects on response rates and evaluations. Assessment & Evaluation in Higher Education, 29(5), 611–623.
Feldman, K. A. (1977). Consistency and variability among college students in rating their teachers and courses: A review and analysis. Research in Higher Education, 6(3), 223–274.
Feldman, K. A. (1988). Effective college teaching from the students’ and faculty’s view: Matched or mismatched priorities? Research in Higher Education, 28(4), 291–329.
Franklin, J., & Theall, M. (1989). Who reads ratings: Knowledge, attitude, and practice of users of student ratings of instruction. In Annual Meeting of the American Edcucation Research Association, San Francisco (USA), pp. 1–27. http://eric.ed.gov/?id=ED306241.
Ginns, P., Prosser, M., & Barrie, S. (2007). Students’ perceptions of teaching quality in higher education: The perspective of currently enrolled students. Studies in Higher Education, 32(5), 603–615.
Goldstein, G. S., & Benassi, V. A. (2006). Students’ and instructors’ beliefs about excellent lecturers and discussion leaders. Research in Higher Education, 47(6), 685–707.
Kember, D., & Wong, A. (2000). Implications for evaluation from a study of students’ perceptions of good and poor teaching. Higher Education, 40(1), 69–97.
Marsh, H. W. (1984). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential baises, and utility. Journal of Educational Psychology, 76(5), 707–754.
McKeachie, W. J. (1997). Student ratings: The validity of use. The American Psychologist, 52(11), 1218–1225.
O’Hagan, A., & Forster, J. (1994). Bayesian inference. In Kendall’s advanced theory of statistics, Vol. 2B, Arnold.
Peña, D. (2002). Análisis de datos multivariantes. Madrid: McGraw-Hill.
Pratt, D. D. (1997). Reconceptualizing the evaluation of teaching in higher education. Higher Education, 34(1), 23–44.
Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1992). Numerical recipes in C: The art of scientific computing (2nd ed.). Cambridge: Cambridge University Press.
Schmelkin, L. P., Spencer, K. J., & Gellman, E. S. (1997). Faculty perspectives on course and teacher evaluations. Research in Higher Education, 38(5), 575–592.
Shevlin, M., Banyard, P., Davies, M., & Griffiths, M. (2000). The validity of student evaluation of teaching in higher education: Love me, love my lectures? Assessment & Evaluation in Higher Education, 25(4), 397–405.
Thune, C. (2005). Standards and guidelines for quality assurance in the European Higher Education Area. Report, European Association for Quality Assurance in the European Higher Education. http://www.enqa.eu/index.php/home/esg.
Wachtel, H. K. (1998). Student evaluation of college teaching effectiveness: A brief review. Assessment & Evaluation in Higher Education, 23(2), 191–212.
Wetzstein, M. E., Broder, J. M., & Wilson, G. (1984). Bayesian inference and student evaluations of teachers and courses. Journal of Economic Education, 15(1), 40–45.
Young, G. A., & Smith, R. L. (2005). Essentials of statistical inference. Cambridge: Cambridge University Press.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fraile, R., Bosch-Morell, F. Considering teaching history and calculating confidence intervals in student evaluations of teaching quality. High Educ 70, 55–72 (2015). https://doi.org/10.1007/s10734-014-9823-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10734-014-9823-0