Checklist Content on a Standardized Patient Assessment: An Ex Post Facto Review
- 282 Downloads
While checklists are often used to score standardized patient based clinical assessments, little research has focused on issues related to their development or the level of agreement with respect to the importance of specific items. Five physicians independently reviewed checklists from 11 simulation scenarios that were part of the former Educational Commission for Foreign Medical Graduate’s Clinical Skills Assessment and classified the clinical appropriateness of each of the checklist items. Approximately 78% of the original checklist items were judged to be needed, or indicated, given the presenting complaint and the purpose of the assessment. Rater agreement was relatively poor with pairwise associations (Kappa coefficient) ranging from 0.09 to 0.29. However, when only consensus indicated items were included, there was little change in examinee scores, including their reliability over encounters. Although most checklist items in this sample were judged to be appropriate, some could potentially be eliminated, thereby minimizing the scoring burden placed on the standardized patients. Periodic review of checklist items, concentrating on their clinical importance, is warranted.
Keywordschecklist inter-rater agreement reliability standardized patient validity
Unable to display preview. Download preview PDF.
- De Champlain A.F., Macmillan M.K., King A.M., Klass D.J., Margolis M.J. (1999). Assessing the impacts of intra-site and inter-site checklist recording discrepancies on the reliability of scores obtained in a nationally administered standardized patient examination. Academic Medicine 74:S52–S54CrossRefGoogle Scholar
- Federation of State Medical Boards & National Board of Medical Examiners (2003). 2004 USMLE Step 2 CS Content Description and General Information Booklet Philadelphia: FSMB and NBMEGoogle Scholar