Competency Assessment in Simulation-Based Training: Educational Framework and Optimal Strategies

  • Etoi A. GarrisonEmail author
  • Jessica L. Pippen
Part of the Comprehensive Healthcare Simulation book series (CHS)


Simulation-based training can be used to evaluate communication and teamwork competencies among interdisciplinary teams and promote healthcare quality and safety (Rosen MA et al., Sim Health Care 3:33–41, 2008; Guise J et al., Sim Healthcare. 3:217–223, 2008; Goffman D et al., Semin perinatol. 37:140–142, 2013). Simulation has been used as an instructional design method to teach medical students, residents, and subspecialty fellows (Medical simulation in medical education: results of an AAMC survey. / simulation in medical education an aamc survey.pdfS; Deering S et al., Semin Perinatol. 37:143–14, 2013). It is also recognized as a method of evaluating professional competence for practicing physician (Sidi A et al., J Grad Med Educ 489–494, 2014; Chiu M et al., Can J Anesth. 63:1357–1363, 2016). Simulation-based training is most effective when it is designed to successfully capture the knowledge, skills, and clinical reasoning that underlie expected behavior within the framework of the competencies assessed. Accurate and objective evaluation of trainee performance depends on the assessment method and the assessment tools selected. In this chapter, we will review a commonly recommended framework for evaluating the quality of an assessment tool. We end the chapter with examples of checklists and global rating scales, two common assessment tools that can be used during simulation-based training to determine if the expected clinical and procedural performance outcomes for learners have been achieved.


Medical education Patient simulation Assessment Validity Reliability 


  1. 1.
    Rosen MA, Salas E, Wilson KA, et al. Measuring team performance in simulation-based training; adopting best practices for healthcare. Sim Health Care. 2008;3:33–41.CrossRefGoogle Scholar
  2. 2.
    Guise J, Deering SH, Kanki BG, et al. Validation of a tool to measure and promote clinical teamwork. Sim Healthcare. 2008;3:217–23.CrossRefGoogle Scholar
  3. 3.
    Goffman D, Lee C, Bernstein PS. Simulation in maternal–fetal medicine: making a case for the need. Semin perinatol. 2013;37:140–2.CrossRefGoogle Scholar
  4. 4.
    Medical simulation in medical education: results of an AAMC survey. simulation in medical education an aamc survey.pdfS.
  5. 5.
    Deering S, Auguste T, Lockrow E. Obstetric simulation for medical student, resident, and fellow education. Semin Perinatol. 2013;37:143–14.CrossRefGoogle Scholar
  6. 6.
    Sidi A, Gravenstein N, Lampotang S. Construct validity and generalizability of simulation-based objective structured clinical examination scenarios. J Grad Med Educ. 2014;6(3):489–94.CrossRefGoogle Scholar
  7. 7.
    Chiu M, Tarshis J, Antoniou A. Simulation-based assessment of anesthesiology residents’ competence: development and implementation of the Canadian National Anesthesiology Simulation Curriculum (CanNASC). Can J Anesth. 2016;63:1357–63.CrossRefGoogle Scholar
  8. 8.
  9. 9.
    Scalese, RJ, Hatala R. Competency assessment. The comprehensive textbook of healthcare simulation. Chapter 11. Springer Science Media: New York 2013. 135–160.Google Scholar
  10. 10.
    Cook DA, Hatala R. Validation of educational assessments: a primer for simulation and beyond. Adv Simul. 2016;1:31.CrossRefGoogle Scholar
  11. 11.
    McEvoy MD, Hand WR, Furse CM, et al. Validity and reliability assessment of detailed scoring checklists for use during perioperative emergency simulation training. Simul Healthc : J Soc Simul Healthc. 2014;9(5):295–303.CrossRefGoogle Scholar
  12. 12.
  13. 13.
    Andreatta PB, Gruppen LD. Conceptualising and classifying validity evidence for simulation. Med Educ. 2009;43:128–03.Google Scholar
  14. 14.
    Andreatta PB, Marzano DA, Curran DS. Validity: what does it mean for competency based assessment in obstetrics and gynecology? Am J Obstet Gynecol. 2011;204:384 e1–6.CrossRefGoogle Scholar
  15. 15.
    Downing S. Validity: on the meaningful interpretation of assessment data. Med Educ. 2003;37:830–7.CrossRefGoogle Scholar
  16. 16.
    Messick S. Validity of psychological assessment: validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. Am Psychologist. 1995;50:741–9.CrossRefGoogle Scholar
  17. 17.
    Cook DA, Zendeja B, Hamstra SJ, Hatala R, Brydges R. What counts as validity evidence? Examples and prevalence in a systemic review of simulation based assessment. Adv Health Sci Educ. 2014;19:233–50.CrossRefGoogle Scholar
  18. 18.
    Messick S. Standards of validity and the validity of standards in performance assessment. Educ Measure Issues Prac. 1995;14:50–8.Google Scholar
  19. 19.
    American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Validity. Standards for educational and psychological testing. Washington, DC: American Educational Research Association. 2014. pp. 11–31.Google Scholar
  20. 20.
    Cook DA, Brydges R, Zendejas B, et al. Technology-enhanced simulation to assess health professionals: A systematic review of validity evidence, research methods, and reporting quality. Acad Med:872–83. Scholar
  21. 21.
    Ilgen JS, Ma IWY, Hatala R, Cook DA. A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment. Med Educ. 2015;49:161–73.CrossRefGoogle Scholar
  22. 22.
    Norman GR, Van der Vleuten CP, De Graaff E. Pitfalls in the pursuit of objectivity: issues of validity, efficiency and acceptability. Med Educ. 1991;25:119–26.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Obstetrics and GynecologyVanderbilt University Medical CenterNashvilleUSA
  2. 2.Department of Obstetrics and GynecologyThe Ohio State University Wexner Medical CenterColumbusUSA

Personalised recommendations