Skip to main content
SpringerLink
Log in

Competency Assessment

  • Chapter
  • First Online:
Comprehensive Healthcare Simulation: Anesthesiology

Part of the book series: Comprehensive Healthcare Simulation ((CHS))

  • 851 Accesses

Abstract

The paradigm shift within medical education to one that is outcome based has made competency assessment critically important. Currently, residents are expected to meet and exceed specific core competencies prior to completion of training. Furthermore, attending physicians have a litany of challenges in maintaining their training after graduation. Given the multitude of practitioners requiring assessment in some phase of training, it is apparent the number of challenges that exist in creating a framework for competency assessment that will work for everyone. One such tool that can be utilized to create a fully manipulatable assessment that will work for varying levels of practitioners is human simulation, coupled with a clear guide or protocol detailing what observable behaviors are expected. A low- or high-fidelity simulated environment, using a myriad of personnel and devices, can ensure patient safety and at the same time allows for manipulation of variables to guarantee consistent delivery of scenarios to those being assessed. Competency assessment in anesthesiology can and should be used to ensure adequate introduction and preparation to monitor the development and progression of both technical and nontechnical skills, as well as for attending physicians to ensure proper maintenance of licensure to retraining of physicians to reenter practice.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Goldberg A, et al. High-stakes simulation-based assessment for retraining and returning physicians to practice. Int Anesthesiol Clin. 2015;53(4):70–80.

    Article  PubMed  Google Scholar 

  2. McGaghie WC, et al. A critical review of simulation-based medical education research: 2003–2009. Med Educ. 2010;44(1):50–63.

    Article  PubMed  Google Scholar 

  3. Issenberg SB, et al. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27(1):10–28.

    Article  PubMed  Google Scholar 

  4. Weller JM, et al. Psychometric characteristics of simulation-based assessment in anaesthesia and accuracy of self-assessed scores. Anaesthesia. 2005;60(3):245–50.

    Article  CAS  PubMed  Google Scholar 

  5. Cook DA. When I say... validity. Med Educ. 2014;48(10):948–9.

    Article  PubMed  Google Scholar 

  6. Cook DA, et al. A contemporary approach to validity arguments: a practical guide to Kane’s framework. Med Educ. 2015;49(6):560–75.

    Article  PubMed  Google Scholar 

  7. Kane MT. Validating the interpretations and uses of test scores. J Educ Meas. 2013;50(1):1–73.

    Article  Google Scholar 

  8. Gingerich A, Regehr G, Eva KW. Rater-based assessments as social judgments: rethinking the etiology of rater errors. Acad Med. 2011;86(10 Suppl):S1–7.

    Article  PubMed  Google Scholar 

  9. Yeates P, et al. Seeing the same thing differently: mechanisms that contribute to assessor differences in directly-observed performance assessments. Adv Health Sci Educ Theory Pract. 2013;18(3):325–41.

    Article  PubMed  Google Scholar 

  10. Accreditation Council for Graduate Medical Education. Next Accreditation System (NAS) and Milestones – a resident perspective. 2014 [cited 2016 09/25]. Available from: http://www.acgme.org/portals/0/pdfs/resident-services/9nasresidentsmay2014.pdf.

  11. The Accreditation Council for Graduate Medical Education and The American Board of Anesthesiology. The Anesthesiology Milestone Project. 2015 [cited 2016 09/20]. Available from: https://www.acgme.org/Portals/0/PDFs/Milestones/AnesthesiologyMilestones.pdf.

  12. Accreditation Council for Graduate Medical Education and The American Board of Medical Specialties. ACGME competencies: suggested best methods for evaluation (Toolbox of assessment methods). 2000 [cited 2016 09/26]. Available from: https://www.partners.org/Assets/Documents/Graduate-Medical-Education/ToolTable.pdf.

  13. Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9 Suppl):S63–7.

    Article  CAS  PubMed  Google Scholar 

  14. Pangaro L. A new vocabulary and other innovations for improving descriptive in-training evaluations. Acad Med. 1999;74(11):1203–7.

    Article  CAS  PubMed  Google Scholar 

  15. Dreyfus, S.E. A five-stage model of the mental activities involved in directed skill acquisition. 1980 [cited 2016 09/28]. Monograph available online. Available from: http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA084551&Location=U2&doc=GetTRDoc.pdf.

  16. Eraut M. Developing professional knowledge and competence. London: RoutledgeFalmer; 1994. p. 124.

    Google Scholar 

  17. Barrows HS, Abrahamson S. The programmed patient: a technique for appraising student performance in clinical neurology. J Med Educ. 1964;39:802–5.

    CAS  PubMed  Google Scholar 

  18. Abrahamson S, Denson JS, Wolf RM. Effectiveness of a simulator in training anesthesiology residents. J Med Educ. 1969;44(6):515–9.

    CAS  PubMed  Google Scholar 

  19. Kane MT. An argument-based approach to validity. Psychol Bull. 1992;112(3):527–35.

    Article  Google Scholar 

  20. Downing SM, Haladyna TM. Validity and its threats. In: Downing SM, Yudkowsky R, editors. Assessment in health professions education. New York: Routledge; 2009. p. 21–56.

    Chapter  Google Scholar 

  21. Goldmann K, Steinfeldt T. Acquisition of basic fiberoptic intubation skills with a virtual reality airway simulator. J Clin Anesth. 2006;18(3):173–8.

    Article  PubMed  Google Scholar 

  22. Maran NJ, Glavin RJ. Low- to high-fidelity simulation – a continuum of medical education? Med Educ. 2003;37(Suppl 1):22–8.

    Article  PubMed  Google Scholar 

  23. Demaria S Jr, et al. Adding emotional stressors to training in simulated cardiopulmonary arrest enhances participant performance. Med Educ. 2010;44(10):1006–15.

    Article  PubMed  Google Scholar 

  24. USMLE. Bulletin of Information. 2016 [cited 2016 09/30]. Available from: http://www.usmle.org/pdfs/bulletin/2016bulletin.pdf.

  25. Baker KS, Cormican D, Seidman PA. Summer anesthesiology externship: demonstrating the ability of early clinical involvement to educate and increase specialty interest. J Educ Perioper Med. 2012;14(4):E063.

    PubMed  PubMed Central  Google Scholar 

  26. Fletcher GC, et al. The role of non-technical skills in anaesthesia: a review of current literature. Br J Anaesth. 2002;88(3):418–29.

    Article  CAS  PubMed  Google Scholar 

  27. Glavin RJ. Excellence in anesthesiology: the role of nontechnical skills. Anesthesiology. 2009;110(2):201–3.

    PubMed  Google Scholar 

  28. Flynn FM, Sandaker K, Ballangrud R. Aiming for excellence – a simulation-based study on adapting and testing an instrument for developing non-technical skills in Norwegian student nurse anaesthetists. Nurse Educ Pract. 2016;22:37–46.

    Article  PubMed  Google Scholar 

  29. Lyk-Jensen HT, et al. Assessing nurse anaesthetists’ non-technical skills in the operating room. Acta Anaesthesiol Scand. 2014;58(7):794–801.

    Article  CAS  PubMed  Google Scholar 

  30. Wunder LL, et al. Objective structured clinical examination as an educational initiative for summative simulation competency evaluation of first-year student registered nurse anesthetists’ clinical skills. AANA J. 2014;82(6):419–25.

    PubMed  Google Scholar 

  31. Wunder LL. Effect of a nontechnical skills intervention on first-year student registered nurse anesthetists’ skills during crisis simulation. AANA J. 2016;84(1):46–51.

    PubMed  Google Scholar 

  32. Accreditation Council for Graduate Medical Education. Specialty-specific Duty Hour Definitions. 2012 [cited 2016 09/29]. Available from: http://www.acgme.org/Portals/0/PDFs/DH_Definitions.pdf.

  33. Mudumbai SC, et al. External validation of simulation-based assessments with other performance measures of third-year anesthesiology residents. Simul Healthc. 2012;7(2):73–80.

    Article  PubMed  Google Scholar 

  34. Berkenstadt H, et al. Deficits in the provision of cardiopulmonary resuscitation during simulated obstetric crises: results from the Israeli Board of Anesthesiologists. Anesth Analg. 2012;115(5):1122–6.

    Article  PubMed  Google Scholar 

  35. Ben-Menachem E, et al. Objective structured clinical examination-based assessment of regional anesthesia skills: the Israeli National Board Examination in anesthesiology experience. Anesth Analg. 2011;112(1):242–5.

    Article  PubMed  Google Scholar 

  36. Flin R, et al. Anaesthetists’ non-technical skills. Br J Anaesth. 2010;105(1):38–44.

    Article  CAS  PubMed  Google Scholar 

  37. Fehr JJ, et al. Simulation-based assessment of pediatric anesthesia skills. Anesthesiology. 2011;115(6):1308–15.

    PubMed  Google Scholar 

  38. Blum RH, et al. A method for measuring the effectiveness of simulation-based team training for improving communication skills. Anesth Analg. 2005;100(5):1375–80, table of contents

    Article  PubMed  Google Scholar 

  39. Malec JF, et al. The mayo high performance teamwork scale: reliability and validity for evaluating key crew resource management skills. Simul Healthc. 2007;2(1):4–10.

    Article  PubMed  Google Scholar 

  40. Blum RH, et al. Simulation-based assessment to identify critical gaps in safe anesthesia resident performance. Anesthesiology. 2014;120(1):129–41.

    Article  PubMed  Google Scholar 

  41. Goldberg A, et al. Learning through simulated independent practice leads to better future performance in a simulated crisis than learning through simulated supervised practice. Br J Anaesth. 2015;114(5):794–800.

    Article  CAS  PubMed  Google Scholar 

  42. Wayne DB, et al. A longitudinal study of internal medicine residents’ retention of advanced cardiac life support skills. Acad Med. 2006;81(10 Suppl):S9–S12.

    Article  PubMed  Google Scholar 

  43. Wayne DB, et al. Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice. J Gen Intern Med. 2006;21(3):251–6.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Wayne DB, et al. Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: a case-control study. Chest. 2008;133(1):56–61.

    Article  PubMed  Google Scholar 

  45. Rodgers DL, Securro S Jr, Pauley RD. The effect of high-fidelity simulation on educational outcomes in an advanced cardiovascular life support course. Simul Healthc. 2009;4(4):200–6.

    Article  PubMed  Google Scholar 

  46. The American Board of Anesthesiology. The American Board of Anesthesiology: APPLIED (Staged Exams). 2016 [cited 2016 09/30]. Available from: http://www.theaba.org/Exams/APPLIED-(Staged-Exam)/About-APPLIED-(Staged-Exam).

  47. The American Board of Anesthesiology. Applied Examination. [cited 2016 12/13/2016]. Available from: http://www.theaba.org/PDFs/APPLIED-Exam/APPLIED-OSCE-ContentOutline.

  48. The American Board of Anesthesiology. MOCA Part 4 Requirements. [cited 2016]. Available from: http://www.theaba.org/PDFs/MOCA/MOCA-2-0-Part-4-Requirements.

  49. Steadman RH, et al. Practice improvements based on participation in simulation for the maintenance of certification in anesthesiology program. Anesthesiology. 2015;122(5):1154–69.

    Article  PubMed  Google Scholar 

  50. Levine AI, et al. Simulation-based Maintenance of Certification in Anesthesiology (MOCA) course optimization: use of multi-modality educational activities. J Clin Anesth. 2012;24(1):68–74.

    Article  PubMed  Google Scholar 

  51. Grace ES, et al. Physicians reentering clinical practice: characteristics and clinical abilities. J Contin Educ Heal Prof. 2011;31(1):49–55.

    Article  Google Scholar 

  52. Jewett EA, Brotherton SE, Ruch-Ross H. A national survey of ‘inactive’ physicians in the United States of America: enticements to reentry. Hum Resour Health. 2011;9:7.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Gallagher CJ, Tan JM. The current status of simulation in the maintenance of certification in anesthesia. Int Anesthesiol Clin. 2010;48(3):83–99.

    Article  PubMed  Google Scholar 

  54. DeMaria S Jr, Levine AI, Bryson EO. The use of multi-modality simulation in the retraining of the physician for medical licensure. J Clin Anesth. 2010;22(4):294–9.

    Article  PubMed  Google Scholar 

  55. Levine AI, et al. Role of simulation in U.S. physician licensure and certification. Mt Sinai J Med. 2012;79(1):140–53.

    Article  PubMed  Google Scholar 

  56. Steadman RH, Huang YM. Simulation for quality assurance in training, credentialing and maintenance of certification. Best Pract Res Clin Anaesthesiol. 2012;26(1):3–15.

    Article  PubMed  Google Scholar 

  57. DeMaria S Jr, et al. Simulation-based assessment and retraining for the anesthesiologist seeking reentry to clinical practice: a case series. Anesthesiology. 2013;119(1):206–17.

    Article  PubMed  Google Scholar 

  58. Ilgen JS, et al. A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment. Med Educ. 2015;49(2):161–73.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Anesthesiology, Perioperative and Pain Medicine, Icahn School of Medicine at Mount Sinai, New York, NY, USA

    Anjan Shah & Samuel DeMaria

  2. Department of Anesthesiology, James J Peters VA Medical Center, New York, NY, USA

    Andrew Goldberg

Authors
  1. Anjan Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Samuel DeMaria
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrew Goldberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Samuel DeMaria .

Editor information

Editors and Affiliations

  1. Department of Anesthesiology, Perioperative and Pain Medicine, Mount Sinai St. Luke’s and West Hospitals, Mount Sinai St. Luke’s & Roosevelt Hospital, Icahn School of Medicine at Mount Sinai, New York, NY, USA

    Bryan Mahoney

  2. Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, MA, USA

    Rebecca D. Minehart

  3. Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, MA, USA

    May C. M. Pian-Smith

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Shah, A., DeMaria, S., Goldberg, A. (2020). Competency Assessment. In: Mahoney, B., Minehart, R., Pian-Smith, M. (eds) Comprehensive Healthcare Simulation: Anesthesiology . Comprehensive Healthcare Simulation. Springer, Cham. https://doi.org/10.1007/978-3-030-26849-7_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-26849-7_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-26848-0

  • Online ISBN: 978-3-030-26849-7

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation