Factors affecting the reliability of ratings of students’ clinical skills in a medicine clerkship
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.Get Access
Objective:To determine the overall reliability and factors that might affect the reliability of ratings of students’ clinical skills in a medicine clerkship.
Design:A nine-item instrument was used to evaluate students’ clinical skills. Raters were also asked to provide a grade of each student’s overall clinical performance. Generalizability studies were performed to estimate the reliability of the ratings. The effects of rater experience and clerkship setting were investigated by regression analysis.
Setting:Teaching hospitals and community-based sites in three Northwestern states.
Participants:All students (328) who had completed the 12-week clerkship in internal medicine at one medical school during the academic years 1987–1989. Raters included attending physicians, chief residents, and other residents.
Results:Seven observations were needed to provide a reliable rating of the overall clinical grade. More observations were needed to obtain reliable ratings for individual items, ranging from seven observations needed for the rating of data gathering skills to 27 observations needed for the rating of interpersonal relationships with patients. Rater experience and clerkship setting (i.e., teaching hospitals vs. community-based clinics) were found, in general, not to affect significantly the ratings received by students.
Conclusions:Reliable ratings of students’ overall clinical skills, including overall clinical grades, can be achieved by collecting a minimum of seven observations. More observations are needed to measure reliably the interpersonal aspects of clinical performance. These findings support the use of performance ratings to evaluate clinical skills and knowledge of students in clerkship settings.
- Magarian, GJ, Mazur, DJ (1990) Evaluation of students in medicine clerkships. Acad. Med 65: pp. 341-5 CrossRef
- Tonesk, X, Buchanan, RG (1987) An AAMC pilot study by 10 medical schools of clinical evaluation of students. J Med Educ 62: pp. 707-18
- Hunt, DD, Carline, JD, Tonesk, X (1989) Types of problem students encountered by clinical teachers on clerkships. J Med Educ 23: pp. 14-8
- Harden, RM, Gleeson, FA (1979) Assessment of clinical competence using an Objective Structured Clinical Examination (OSCE). Med Educ 13: pp. 41-54
- Ainsworth MA, Rogers LP, Markus JF, Dorsey NK, Blackwell TA, Petrusa ER. Standardized patient encounters: a method for teaching and evaluation. JAMA. 191;266:1390–6.
- Maxim, BR, Dielman, TE (1987) Dimensionality, internal consistency and interrater reliability of clinical performance ratings. Med Educ 21: pp. 130-7 CrossRef
- Hull, AL (1982) Medical student performance: a comparison of house officer and attending staff as evaluators. Eval Health Prof 5: pp. 87-94 CrossRef
- Carline, JD, Cook, CE, Lennard, ES, Coluccio, GM, Norman, NL (1986) Resident and faculty differences in student evaluations: implications for changes in a clerkship evaluation system. Surgery 100: pp. 88-94
- Ramsey, PG, Shannon, NF, Fleming, L (1986) Use of objective examinations in medicine clerkships. Am J Med 81: pp. 669-74 CrossRef
- Shavelson, RJ, Webb, NM (1991) Generalizability theory: a primer. Sage Publications, Newbury Park, CA
- Winer, BJ (1971) Statistics: principles in experimental design. McGraw-Hill, New York
- O’Brien, PC, Shampo, MA (1988) Statistical considerations for performing multiple tests in a single experiment. Mayo Clin Proc 63: pp. 816-20
- Factors affecting the reliability of ratings of students’ clinical skills in a medicine clerkship
Journal of General Internal Medicine
Volume 7, Issue 5 , pp 506-510
- Cover Date
- Print ISSN
- Online ISSN
- Additional Links
- medical students
- clinical clerkship
- educational measurement
- Industry Sectors
- Author Affiliations
- 1. Department of Medical Education, SC-45, University of Washington School of Medicine, 98195, Seattle, WA
- 2. the Department of Medicine, University of Washington School of Medicine, Seattle, Washington