Abstract
Purpose: There is a paucity of evaluation forms specifically developed and validated for outpatient settings. The purpose of this study was to develop and validate an instrument specifically for evaluating outpatient teaching, to provide reliable and valid ratings for individual and group feedback to faculty, and to identify outstanding teachers in that setting. Method: By literature reviews and pilot studies at the Faculties of Health Sciences, McMaster University (Canada) and Aga Khan University (AKU-Pakistan), a 15-item instrument, Student Evaluation of Teaching in Outpatient Clinics (SETOC), was created with five subscales: “Establishing Learning-Milieu, Clinical-Teaching, General-Teaching, Clinical-Competence, and Global-Rating.” Seven-point Likert-type rating scales were used. Students also nominated three “best” outpatient teachers. Participants: 87 faculty members (79%) rated by all 224 third to fifth-year students (clerks) at outpatient departments of the AKU hospital over a one-year period. Analyses: Repeated measures generalizability studies, correlations, concurrent validity of SETOC scores with best teacher nominations. Results: Inter-rater G-coefficient and internal consistency of SETOC student ratings were 0.92 and 0.98. Average inter-item and inter-subscale correlations were 0.79 and 0.86. Comparing SETOC scores against “Best Teacher” nominations, sensitivity, specificity, positive and negative predictive values were greater than 0.84.Student ratings ranged from unsatisfactory (fourteen instructors) to outstanding (four instructors). Mean-scores for Learning-Milieu, Clinical-Teaching and General-Teaching were lower than those for Clinical-Competence and Global-Rating (p=0.000 for all). Conclusions: The SETOC elicited reliable and valid student ratings that can provide specific feedback to individual faculty with weak or outstanding teaching skills, and identify overall group shortcomings for faculty development.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Berk R.A. (1979). The construction of rating instruments for faculty evaluation. Journal of Higher Education 50(5): 650–669
Braskamp, L.A. & Ory, J.C. (1994). New perspective on faculty assessment and development. Chapter 2 In Assessing Faculty Work. Enhancing Individual and Institutional Performance, pp. 12’26. San Francisco, California: Jossey-Bass Inc
Cashin W.E. (1995). Student ratings of teaching: the research revisited. Center for Faculty Evaluation and Development 32: 1–9
Costin F., Greenough W.T. and Menges R.J (1971). Student ratings of college teaching: reliability, validity, and usefulness. Review of Educational Research 41(5): 511–531
d’Apollonia S. and Abrami P.C (1997). Navigating student ratings of instruction. American Psychologist 52(11): 1198–1208
Guyatt G.H., Nishikawa J., Willan A., McIlroy W., Cook D., Gibson J., Kerigan A. and Neville A (1993). A measurement process for evaluating clinical teachers in internal medicine. Canadian Medical Association Journal 149(8): 1097–1102
Irby D.M. (1978). Clinical teacher effectiveness in medicine. Journal of Medical Education 53: 808–815
Irby D.M. (1983). Evaluating instruction in medical education. Journal of Medical Education 58: 844–849
Irby D.M. (1995). Teaching and learning in ambulatory care settings: a thematic review of literature. Academic Medicine 70(10): 898–931
Irby D.M., Gillmore G.M. and Ramsey P.G (1987). Factors affecting ratings of clinical teachers by medical students and residents. Journal of Medical Education 62: 1–7
James P.A., Kreiter C.D., Shipengrover J., Crosson J., Heaton C. and Kernan J (2001). Students on the clinical race track — where they ran and how they ran. a generalizability study of a standardized rating form used to evaluate instructional quality in clinical ambulatory sites. Academic Medicine 76(10): S33–S35
Litzelman D.K., Stratos G.A., Marriott D.J. and Skeff K.M (1998). Factorial validation of a widely disseminated educational framework for evaluating clinical teachers. Academic Medicine 73(6): 688–695
Lovejoy F.H. and Clark M.B (1995). A promotion ladder for teachers at Harvard Medical School: experience and challenges. Academic Medicine 70: 1079–1086
Marsh H.W. (1984). Students’ evaluations of university teaching: multidimensionality, reliability, validity, potential biases and utility. Journal of Educational Psychology 76: 707–754
McKeachie W.J. (1997). Student ratings. The validity of use. American Psychologist 52(11): 1218–1225
Norman, G.R. & Streiner, D.L. (1994). Repeated Measures ANOVA. In Biostatistics — The Bare Essentials, pp. 129–142. St Louis Missouri: Mosby Year Book Inc
Nunnally, J.C. (1970). Reliability of Measurements. Chapter 5 In Introduction to Psychological Measurement pp. 107–131. McGraw-Hill
Skeff K.M. (1988). Enhancing teaching effectiveness and vitality in the ambulatory setting. Journal of General and Internal Medicine 3: S26–S33
Streiner, D.L. & Norman, G.R. (1995). Reliability. Chapter 8 in Health Measurement Scales. A Practical Guide to their Development and Use, 2nd ed., pp. 104–127. Oxford University Press
Stritter F.T., Hain J.D. and Grimes D.A (1975). Clinical teaching re-examined. Journal of Medical Education 50: 876–882
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zuberi, R.W., Bordage, G. & Norman, G.R. Validation of the SETOC Instrument — Student Evaluation of Teaching in Outpatient Clinics. Adv Health Sci Educ Theory Pract 12, 55–69 (2007). https://doi.org/10.1007/s10459-005-2328-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-005-2328-y