Advances in Health Sciences Education

, Volume 8, Issue 1, pp 27–47

Quality Assurance Methods for Performance-Based Assessments

  • John R. Boulet
  • Danette W. McKinley
  • Gerald P. Whelan
  • Ronald K. Hambleton
Article

DOI: 10.1023/A:1022639521218

Cite this article as:
Boulet, J.R., McKinley, D.W., Whelan, G.P. et al. Adv Health Sci Educ Theory Pract (2003) 8: 27. doi:10.1023/A:1022639521218

Abstract

Performance assessments are subject to many potential error sources. For performance-based assessments, including standardized patient (SP) examinations, these error sources, if left unchecked, can compromise the validity and reliability of scores. Quality assurance (QA) measures, both quantitative and qualitative, can be used to ensure that candidate scores are accurate and reasonably free from measurement error. The purpose of this paper is to outline several QA strategies that can be used to identify potential content- and score-related problems with SP assessments. These approaches include case analyses and various comparisons of primary and observer scores. Specific examples from the ECFMG Clinical Skills Assessment(CSA®) are used to educate the reader concerning appropriate statistical methods and legitimate data interpretations. The results presented in this investigation highlight the need for well-defined training regimes, regular feedback to those involved in rating/scoring performances, and detailed statistical analyses of all scores.

clinical skillsperformance assessmentpsychometricsquality assurancestandardized patients

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • John R. Boulet
    • 1
  • Danette W. McKinley
    • 1
  • Gerald P. Whelan
    • 1
  • Ronald K. Hambleton
    • 2
  1. 1.Research and EvaluationEducational Commission for Foreign Medical Graduates (ECFMG)PhiladelphiaUSA
  2. 2.University of MassachusettsUSA