Advances in Health Sciences Education

, Volume 7, Issue 2, pp 99–116

Medical Students' Ratings of Faculty Teaching in a Multi-Instructor Setting: An Examination of Monotonic Response Patterns

  • Terry D. Stratton
  • Donald B. Witzke
  • Robert J. Jacob
  • Marlene J. Sauer
  • Amy Murphy-Spencer
Article

DOI: 10.1023/A:1015735813922

Cite this article as:
Stratton, T.D., Witzke, D.B., Jacob, R.J. et al. Adv Health Sci Educ Theory Pract (2002) 7: 99. doi:10.1023/A:1015735813922

Abstract

Realizing that the psychometric properties of a measure may be highly variable is especially relevant in a multi-instructor context, since an implicit assumption is that student ratings are equally reliable and valid for all faculty ratees. As a possible indicator of nonattending (i.e. invalid) responses, the authors examined the effects of monotonic response patterns on the reliabilities of students' ratings of faculty teaching – including how an alternative presentation format may reduce the prevalence of this behavior. Second-year medical and dental students(n = 130) enrolled in a required basic science course during the 1998–99 academic year were randomly assigned to one of two groups – each of which evaluated the teaching of 6 different faculty across 6 distinct dimensions (i.e. overall quality, organization, preparation, stimulation, respectfulness, and helpfulness). Using a `split ballot' design, two versions of the conceptually equivalent faculty evaluation form were distributed at random to students in each group. Form A contained the `traditional' items-within-faculty format, while Form B listed faculty-within-item.

The number of monotonic forms (i.e. the identical rating of all 6 items) varied measurably across faculty ratees, as did the respective effects on scale reliabilities. Alpha was especially inflated where a sizeable proportion of monotonic patterns were located on response categories that were either very high (> +1.28 zm deviations) or very low (< −1.28 zm deviations) compared to the group mean. Lastly, the prevalence of monotonic response patterns was significantly(p = ≤ 0.01) less when a faculty-within-item format is used (Form B).These findings suggest that monotonic response patterns differentially impact the reliabilities and, hence, the validity of students' ratings of individual faculty in a multi-instructor context.

effects faculty evaluation monotonic response patterns multi-instructor course nonattending behaviors response validity 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Terry D. Stratton
    • 1
  • Donald B. Witzke
    • 2
  • Robert J. Jacob
    • 3
  • Marlene J. Sauer
    • 4
  • Amy Murphy-Spencer
    • 5
  1. 1.Office of Academic Affairs, Division of Testing and EvaluationUniversity of Kentucky College of MedicineLexingtonU.S.A
  2. 2.Department of Pathology and Laboratory MedicineUniversity of Kentucky College of MedicineLexingtonU.S.A
  3. 3.Department of Microbiology and Immunology and Division of Oral Health SciencesUniversity of Kentucky, Colleges of Medicine and DentistryLexingtonU.S.A
  4. 4.Office of Academic AffairsUniversity of Kentucky College of MedicineLexingtonU.S.A
  5. 5.Office of Academic Affairs, Division of Testing and EvaluationUniversity of Kentucky College of MedicineLexingtonU.S.A

Personalised recommendations