Simulation for Licensure and Certification

Chapter

Abstract

In the past two decades, there is a growing recognition of the power of simulation-based assessment (SBA) to serve as a complimentary method to the traditional examination methods used for licensure and certification in medical education.

This chapter deals with the role SBA can and should play in regulatory-driven assessment programs. We introduce the main challenges that accompany SBA programs and suggest ways to overcome these challenges. The challenges are discussed in reference to a few key questions regarding SBA: Does SBA really simulate reality? Can SBA really measure what we want it to measure? Can examinees in SBA show their real ability? and Does SBA worth the huge investment? In the answers suggested, the keys to successful SBA are presented, with a focus on the unique multi-professional collaboration that is required for sound SBA: the collaboration between content experts, simulation experts, and measurement experts—psychometricians. Finally, the chapter describes the Israeli experience in SBA at MSR—the Israel Center for Medical Simulation—and highlights the authors’ insights from 10 years of experience in high-stakes SBA programs, such as the Israeli Board Examination in Anesthesiology, Paramedics Certification Exam, National Registration Exam for Nurse Specialists, and the “MOR” assessment center—selection of candidates to medical schools.

Keywords

Licensure Certification High-stakes assessment Patients’ safety Simulation-based assessment 

References

  1. 1.
    Boulet JR. Summative assessment in medicine: the promise of simulation for high-stakes evaluation. Acad Emerg Med. 2008;15:1017–124.PubMedCrossRefGoogle Scholar
  2. 2.
    Gaba DM. Do as we say, not as you do: using simulation to investigate clinical behavior in action. Simul Healthc. 2009;4:67–9.PubMedCrossRefGoogle Scholar
  3. 3.
    Scalese RJ, Issenberg SB. Simulation-based assessment. In: Holmboe ES, Hawkins RE, editors. A practical approach to the evaluation of clinical competence. Philadelphia: Mosby/Elsevier; 2008.Google Scholar
  4. 4.
    Holmboe E, Rizzolo MA, Sachdeva A, Rosenberg M, Ziv A. Simulation based assessment and the regulation of healthcare professionals. Simul Healthc. 2011;6:s58–62.PubMedCrossRefGoogle Scholar
  5. 5.
    MSR Israel Center for Medical Simulation. Available at: http://www.msr.org.il/. Accessed Oct 2006.
  6. 6.
    Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27:10–28.PubMedCrossRefGoogle Scholar
  7. 7.
    Medical Council of Canada. Medical Council of Canada qualifying Examination Part II (MCCQE Part II). 2008. Medical Council of Canada.Google Scholar
  8. 8.
    Whelan G. High–Stakes medical performance testing: the Clinical Skills Assessment Program. JAMA. 2000;283:1748.PubMedCrossRefGoogle Scholar
  9. 9.
    Federation of State Medical Board, Inc. and National Board of Medical Examiners. United States Medical Licensing Examination: Step 2 clinical skills (cs) content description and general information. 2008. Federation of State Medical Board, Inc. and National Board of Medical Examiners.Google Scholar
  10. 10.
    American Board of Surgery. ABS to Require ACLS, ATLS and FLS for General Surgery Certification. Available at: http://home.absurgery.org/default.jsp?news_newreqs. Accessed 27 May 2011.
  11. 11.
    Soper NJ, Fried GM. The fundamentals of laparoscopic surgery: its time has come. Bulletin of the American College of Surgeons. 2008. Available at: http://www.flsprogram.org/wp-content/uploads/2010/10/FLSprogramSoperFried.pdf. Accessed 7 Mar 2011.
  12. 12.
    Petrusa ER. Clinical performance assessment. In: Norman GR, van der Vleuten CPM, Newble DI, editors. International Handbook of Research in Medical Education. Dordrecht: Kluwer Academic Publications; 2002.Google Scholar
  13. 13.
    Tombeson P, Fox RA, Dacre JA. Defining the content for the objective structured clinical examination component of the Professional and Linguistic Assessment Board examination: development of a blueprint. Med Educ. 2000;34:566–72.CrossRefGoogle Scholar
  14. 14.
    Boyd MA, Gerrow JD, Duquette P. Rethinking the OSCE as a tool for national competency evaluation. Eur J Dent Educ. 2004;8:95.CrossRefGoogle Scholar
  15. 15.
    Boulet JR, Smee SM, Dillon GF, Gimplr JR. The use of standardized patient assessments for certification and licensure decisions. Simul Healthc. 2009;4:35–42.PubMedCrossRefGoogle Scholar
  16. 16.
    Ordre des Infirmieres et Infirmiers Quebec. Available at: http://www.oiiq.org/. Accessed 7 Mar 2011.
  17. 17.
    College and Association of Registered Nurses of Alberta. Available at: http://www.nurses.ab.ca/carna/index.aspx?WebStructureID_3422. Accessed 7 Mar 2011.
  18. 18.
    Ziv A, Rubin O, Moshinsky A, et al. MOR: a simulation-based assessment centre for evaluating the personal and interpersonal qualities of medical school candidates. Med Educ. 2008;42:991–8.PubMedCrossRefGoogle Scholar
  19. 19.
    Harris S, Owen C. Discerning quality: using the multiple mini-interview in student selection for the Australian National University Medical School. Med Educ. 2007;41(3):234–41.PubMedCrossRefGoogle Scholar
  20. 20.
    O’Brien A, Harvey J, Shannon M, Lewis K, Valencia O. A comparison of multiple mini-interviews and structured interviews in a UK setting. Med Teach. 2011;33(5):397–402.PubMedCrossRefGoogle Scholar
  21. 21.
    Eva KW, Rosenfeld J, Reiter HI. Norman GR An admissions OSCE: the multiple mini-interview. Med Educ. 2004;38(3):314–26.PubMedCrossRefGoogle Scholar
  22. 22.
    Issenberg SB, Scalese RJ. Simulation in health care education. Perspect Biol Med. 2008;51:31–46.PubMedCrossRefGoogle Scholar
  23. 23.
    Boulet JR, van Zanten M, de Champlain A, Hawkins RE, Peitzman SJ. Checklist content on a standardized patient assessment: an ex post facto review. Adv Health Sci Educ. 2008;13:59–69.CrossRefGoogle Scholar
  24. 24.
    Baker DP, Salas E, King H, Battles J, Barach P. The role of teamwork in the professional education of physicians: current status and assessment recommendations. Jt Comm J Qual Patient Saf. 2005;31:185–202.PubMedGoogle Scholar
  25. 25.
    van Zanten M, Boulet JR, McKinley DW, de Champlain A, Jobe AC. Assessing the communication and interpersonal skills of graduates of international medical schools as part of the United States Medical Licensing Exam (USMLE) Step 2 Clinical Skills (CS) Exam. Acad Med. 2007;82(10 Suppl l):S65–8.PubMedCrossRefGoogle Scholar
  26. 26.
    Weller JM, Bloch M, Young S, et al. Evaluation of high fidelity patient simulator in assessment of performance of anesthetists. Br J Anaesth. 2003;90:43–7.PubMedCrossRefGoogle Scholar
  27. 27.
    Morgan PJ, Cleave-Hogg D, Guest CB. A comparison of global ratings and checklist scores from an undergraduate assessment using anesthesia simulator. Acad Med. 2001;76:1053–5.PubMedCrossRefGoogle Scholar
  28. 28.
    Downing SM. Validity: on the meaningful interpretation of assessment data. Med Educ. 2003;37:830–7.PubMedCrossRefGoogle Scholar
  29. 29.
    Newble D. Techniques for measuring clinical competence: objective structured clinical examinations. Med Educ. 2004;38:199–203.PubMedCrossRefGoogle Scholar
  30. 30.
    Berkenstadt H, Ziv A, Gafni N, Sidi A. The validation process of incorporating simulation-based accreditation into the anesthesiology Israeli national board exams. Isr Med Assoc J. 2006;8: 728–33.PubMedGoogle Scholar
  31. 31.
    Murray DJ, Boulet JR, Avidan M, et al. Performance of residents and anesthesiologists in a simulation based skill assessment. Anesthesiology. 2007;107:705–13.PubMedCrossRefGoogle Scholar
  32. 32.
    Girzadas Jr DV, Clay L, Caris J, Rzechula K, Harwood R. High fidelity simulation can discriminate between novice and experienced residents when assessing competency in patient care. Med Teach. 2007;29:452–6.CrossRefGoogle Scholar
  33. 33.
    Rosenthal R, Gantert WA, Hamel C, et al. Assessment of construct validity of a virtual reality laparoscopy simulator. J Laparoendosc Adv Surg Tech A. 2007;7:407–13.CrossRefGoogle Scholar
  34. 34.
    Tamblyn R, Abrahamowicz M, Dauphinee D, et al. Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities. JAMA. 2007;298:993–1001.PubMedCrossRefGoogle Scholar
  35. 35.
    Hatala R, Issenberg SB, Kassen B, Cole G, Bacchus CM, Scalese RJ. Assessing cardiac physical examination skills using simulation technology and real patients: a comparison study. Med Educ. 2008;42:628–36.PubMedCrossRefGoogle Scholar
  36. 36.
    Ziv A, Erez D, Munz Y, Vardi A, Barsuk D, Levine I, et al. The Israel Center for Medical Simulation: a paradigm for cultural change in medical education. Acad Med. 2006;81(12):1091–7.PubMedCrossRefGoogle Scholar
  37. 37.
    National Institute for Testing and Evaluation. Available at: http://www.nite.org.il/. Accessed Oct 2006.
  38. 38.
    Berkenstadt H, Ziv A, Gafni N, Sidi A. Incorporating simulation-based objective structured clinical examination into the Israeli National Board Examination in Anesthesiology. Anesth Analg. 2006;102(3):853–8.PubMedCrossRefGoogle Scholar
  39. 39.
    Ziv A, Rubin O, Sidi A, Berkenstadt H. Credentialing and certifying with simulation. Anesthesiol Clin. 2007;25(2):261–9.PubMedCrossRefGoogle Scholar
  40. 40.
    Ben-Menachem E, Ezri T, Ziv A, Sidi A, Berkenstadt H. Identifying and managing technical faults in the anesthesia machine: lessons learned from the Israeli Board of Anesthesiologists. Anesth Analg. 2011;112(4):864–6. Epub 2011 Feb 2.PubMedCrossRefGoogle Scholar
  41. 41.
    Gafni N, Moshinsky A, Eisenberg O, Zeigler D, Ziv A. Reliability estimates: behavioural stations and questionnaires in medical school admissions. Med Educ. 2012;46(3):277–88.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Amitai Ziv
    • 1
    • 2
  • Haim Berkenstadt
    • 3
    • 4
  • Orit Eisenberg
    • 5
    • 6
  1. 1.Medical Education, Sackler School of MedicineTel Aviv UniversityTel AvivIsrael
  2. 2.Patient Safety and Risk ManagementIsrael Center for Medical Simulation (MSR), Chaim Sheba Medical CenterTel HashomerIsrael
  3. 3.Department of Anesthesiology and Intensive Care, Sackler Faculty of MedicineTel Aviv UniversityTel AvivIsrael
  4. 4.Department of AnesthesiologySheba Medical CenterRamat GanIsrael
  5. 5.MAROM—Assessment and Admissions Centers, National Institute for Testing and EvaluationJerusalemIsrael
  6. 6.Assessment UnitThe Israel Center for Medical SimulationTel HashomerIsrael

Personalised recommendations