Skip to main content

Advertisement

SpringerLink
Log in

Are we talking about practice? A randomized study comparing simulation-based deliberate practice and mastery learning to self-guided practice

  • Original Research
  • Published:
Canadian Journal of Emergency Medicine Aims and scope Submit manuscript

Abstract

Objectives

Simulation-based technical skills training is now ubiquitous in medicine, particularly for high acuity, low occurrence (HALO) procedures. Mastery learning and deliberate practice (ML + DP) are potentially valuable educational methods, however, they are resource intensive. We sought to compare the effect of deliberate practice and mastery learning versus self-guided practice on skill performance of the rare, life-saving procedure, a bougie-assisted cricothyroidotomy (BAC).

Methods

We conducted a multi-center, randomized study at five North American emergency medicine (EM) residency programs. We randomly assigned 176 EM residents to either the ML + DP or self-guided practice groups. Three blinded airway experts independently evaluated BAC skill performance by video review before (pre-test), after (post-test) and 6–12 months (retention) after the training session. The primary outcome was post-test skill performance using a global rating score (GRS). Secondary outcomes included performance time and skill performance at the retention test.

Results

Immediately following training, GRS scores were significantly higher as mean performance improved from pre-test, (22, 95% CI = 21–23) to post-test (27, 95% CI = 26–28), (p < 0.001) for all participants. However, there was no difference between the groups on GRS scores (p = 0.2) at the post-test or at the retention test (p = 0.2). At the retention test, participants in the ML + DP group had faster performance times (66 s, 95% CI = 57–74) compared to the self-guided group (77 s, 95% CI = 67–86), (p < 0.01).

Conclusions

There was no significant difference in skill performance between groups. Residents who received deliberate practice and mastery learning demonstrated an improvement in skill performance time.

Abstrait

Objectifs

La formation aux compétences techniques fondée sur la simulation est maintenant omniprésente en médecine, en particulier pour les procédures de grande acuité et de faible occurrence (HALO). L’apprentissage de la maîtrise et la pratique délibérée (ML+DP) sont des méthodes éducatives potentiellement précieuses, mais elles exigent beaucoup de ressources. Nous avons cherché à comparer l’effet de la pratique délibérée et de l’apprentissage de la maîtrise par rapport à la pratique autoguidée sur le rendement des compétences de la rare intervention de sauvetage, une cricothyroïdotomie à la bougie.

Méthodes

Nous avons mené une étude multicentrique randomisée dans le cadre de cinq programmes nord-américains de résidence en médecine d’urgence. Nous avons affecté au hasard 176 résidents en SE aux groupes de ML+DP ou de pratique autoguidée. Trois experts des voies respiratoires aveuglés ont évalué de façon indépendante la performance des compétences en BAC par examen vidéo avant (pré-test), après (post-test) et 6 à 12 mois (rétention) après la séance de formation. Le principal résultat a été le rendement des compétences après le test au moyen d’une cote globale (SRC). Les résultats secondaires comprenaient le temps consacré au rendement et le rendement des compétences au test de rétention.

Résultats

Immédiatement après la formation, les résultats des SRC étaient beaucoup plus élevés, car le rendement moyen s’est amélioré entre le prétest (22, IC à 95 % = 21 à 23) et le post-test (27, IC à 95 % = 26 à 28), (p < 0,001) pour tous les participants. Cependant, il n’y avait aucune différence entre les groupes sur les scores GRS (p = 0,2) au post-test ou au test de rétention (p = 0,2). Au test de rétention, les participants du groupe ML+DP avaient des temps de performance plus rapides (66 secondes, IC à 95 % = 57 à 74) que ceux du groupe autoguidé (77 secondes, IC à 95 % = 67 à 86) (p < 0,01).

Conclusions

Il n’y avait pas de différence significative dans le rendement des compétences entre les groupes. Les résidents qui ont bénéficié d’une pratique délibérée et d’un apprentissage de la maîtrise ont démontré une amélioration du temps consacré aux compétences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

The data that support the findings of this study are available on request from the corresponding author.

References

  1. Issenberg S, Mcgaghie W, Petrusa E, Gordon D, Scalese R. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27:10–28.

    Article  PubMed  Google Scholar 

  2. Cook DA, Brydges R, Zendejas B, Hamstra SJ, Hatala R. Mastery learning for health professionals using technology-enhanced simulation: a systematic review and meta-analysis. Acad Med. 2013;88:1178–86.

    Article  PubMed  Google Scholar 

  3. Rodriguez-Paz JM, Kennedy M, Salas E, et al. Beyond “see one, do one, teach one”: toward a different training paradigm. Qual Saf Health Care. 2009;18:63–8.

    CAS  PubMed  Google Scholar 

  4. Cocks M, Moulton CA, Luu S, Cil T. What surgeons can learn from athletes: mental practice in sports and surgery. J Surg Educ. 2014;71:262–9.

    Article  PubMed  Google Scholar 

  5. Ericsson KA. Deliberate practice and acquisition of expert performance: a general overview. Acad Emerg Med. 2008;15:988–94.

    Article  PubMed  Google Scholar 

  6. Schumacher DJ, Englander R, Carraccio C. Developing the master learner: applying learning theory to the learner, the teacher, and the learning environment. Acad Med. 2013;88:1635–45.

    Article  PubMed  Google Scholar 

  7. Walsh CM, Ling SC, Wang CS, Carnahan H. Concurrent versus terminal feedback: it may be better to wait. Acad Med. 2009;84:S54–7.

    Article  PubMed  Google Scholar 

  8. Cheng A, Eppich W, Grant V, Sherbino J, Zendejas B, Cook DA. Debriefing for technology-enhanced simulation: a systematic review and meta-analysis. Med Educ. 2014;48:657–66.

    Article  PubMed  Google Scholar 

  9. Eppich WJ, Hunt EA, Duval-Arnould JM, Siddall VJ, Cheng A. Structuring feedback and debriefing to achieve mastery learning goals. Acad Med. 2015;90:1501–8.

    Article  PubMed  Google Scholar 

  10. Donoghue A, Navarro K, Diederich E, Auerbach M, Cheng A. Deliberate practice and mastery learning in resuscitation education: a scoping review. Resusc Plus. 2021;6: 100137.

    Article  PubMed  PubMed Central  Google Scholar 

  11. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011;86:706–11.

    Article  PubMed  PubMed Central  Google Scholar 

  12. McGaghie WC. When I say … mastery learning. Med Educ. 2015;49:558–9.

    Article  PubMed  Google Scholar 

  13. Udani AD, Harrison TK, Mariano ER, et al. Comparative-effectiveness of simulation-based deliberate practice versus self-guided practice on resident anesthesiologists’ acquisition of ultrasound-guided regional anesthesia skills. Reg Anesth Pain Med. 2016;41:151–7.

    Article  CAS  PubMed  Google Scholar 

  14. Friedman Z, You-Ten KE, Bould MD, Naik V. Teaching lifesaving procedures: the impact of model fidelity on acquisition and transfer of cricothyrotomy skills to performance on cadavers. Anesth Analg. 2008;107:1663–9.

    Article  PubMed  Google Scholar 

  15. Braude D, Webb H, Stafford J, et al. The bougie-aided cricothyrotomy. Air Med J. 2009;28:191–4.

    Article  PubMed  Google Scholar 

  16. Frerk C, Mitchell VS, McNarry AF, et al. Difficult Airway Society 2015 guidelines for management of unanticipated difficult intubation in adults. Br J Anaesth. 2015;115:827–48.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Hill C, Reardon R, Joing S, Falvey D, Miner J. Cricothyrotomy technique using gum elastic bougie is faster than standard technique: a study of emergency medicine residents and medical students in an animal lab. Acad Emerg Med. 2010;17:666–9.

    Article  PubMed  Google Scholar 

  18. Petrosoniak A, Lu M, Gray S, et al. Perfecting practice: a protocol for assessing simulation-based mastery learning and deliberate practice versus self-guided practice for bougie-assisted cricothyroidotomy performance. BMC Med Educ. 2019;19:100.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Dharamsi A, Gray S, Hicks C, Sherbino J, McGowan M, Petrosoniak A. Bougie-assisted cricothyroidotomy: Delphi-derived essential steps for the novice learner. CJEM. 2019;21:283–90.

    Article  PubMed  Google Scholar 

  20. Boet S, Borges BC, Naik VN, et al. Complex procedural skills are retained for a minimum of 1 yr after a single high-fidelity simulation training session. Br J Anaesth. 2011;107:533–9.

    Article  CAS  PubMed  Google Scholar 

  21. Naik VN, Matsumoto ED, Houston PL, et al. Fiberoptic orotracheal intubation on anesthetized patients: do manipulation skills learned on a simple model transfer into the operating room? Anesthesiology. 2001;95:343–8.

    Article  CAS  PubMed  Google Scholar 

  22. Dawe SR, Windsor JA, Broeders JA, Cregan PC, Hewett PJ, Maddern GJ. A systematic review of surgical skills transfer after simulation-based training: laparoscopic cholecystectomy and endoscopy. Ann Surg. 2014;259:236–48.

    Article  PubMed  Google Scholar 

  23. Ilgen JS, Ma IW, Hatala R, Cook DA. A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment. Med Educ. 2015;49:161–73.

    Article  PubMed  Google Scholar 

  24. Monteiro S, Sibbald M. Choosing wisely-Clinician educators’ guide to high-value simulation-based education. Arch Med Health Sci. 2020;8:302.

    Article  Google Scholar 

Download references

Acknowledgements

The authors wish to thank the staff at the Allan Waters Family Simulation Centre (Nazanin Khodadoust, Kari White, Kristen Daly, Susan Zelko, Hentley Small and Ashley Rosen) for their contributions to the simulation design, and Laura Pozzobon and Emma O’Neil for their administrative support. The authors also wish to thank the simulation teams and emergency medicine residency programs at each participating centre: Queen’s University (Jessica Montagner), McMaster University (Bingxian Wang, Lisa Da Maia, Melissa Hymer), Yale University (Marc Auerbach) and the University of Ottawa (Kymber Tran-Ai, Lisa Thurgur, Maggie Kisilewicz) for their support of this study.

Funding

This work is supported by a medical education grant from Physicians Services Incorporated (Ontario, Canada), Grant no [5498-New Investigator]. The funding sources had no role in the design of this study and they will not have any role in the study’s implementation, data analysis or dissemination of study results.

Author information

Authors and Affiliations

  1. Division of Emergency Medicine, Department of Medicine, University of Toronto, St. Michael’s Hospital, Toronto, ON, Canada

    Andrew Petrosoniak, Sara Gray, Christopher Hicks, George Mastoras & Melissa McGowan

  2. Interdepartmental Division of Critical Care, University of Toronto, Toronto, ON, Canada

    Sara Gray

  3. McMaster Education Research, Innovation and Theory (MERIT) Program, McMaster University, Hamilton, ON, Canada

    Jonathan Sherbino & Sandra Monteiro

  4. Division of Education and Innovation, Department of Medicine, McMaster University, Hamilton, ON, Canada

    Jonathan Sherbino & Sandra Monteiro

  5. Division of Emergency Medicine, Department of Medicine, McMaster University, Hamilton, ON, Canada

    Jonathan Sherbino & Julian Owen

  6. College of Medicine-Jacksonville, University of Florida, Gainesville, FL, USA

    Thomas Beardsley

  7. Department of Emergency Medicine, Yale School of Medicine, New Haven, CT, USA

    James Bonz & Ambrose H. Wong

  8. Department of Emergency Medicine, University of Ottawa, Ottawa, ON, Canada

    Andrew K. Hall

  9. Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada

    Andrew K. Hall

  10. Division of Emergency Medicine, Department of Medicine, Western University, London, ON, Canada

    Julie Kim

Authors
  1. Andrew Petrosoniak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jonathan Sherbino
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Beardsley
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. James Bonz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sara Gray
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Andrew K. Hall
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Christopher Hicks
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Julie Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. George Mastoras
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Melissa McGowan
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Julian Owen
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Ambrose H. Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Sandra Monteiro
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All the authors made substantial contributions to the conception, design and analysis of the study, the drafting and revisions of the manuscript and agree to accountability for all aspects of the manuscript. All the authors approved the manuscript and this submission.

Corresponding author

Correspondence to Andrew Petrosoniak.

Ethics declarations

Conflict of interest

AP and CH are co-founders of Advanced Performance Healthcare Design, no other conflicts to declare; JS, TB, JB, SG, AKH, JK, JO, MM and AWH, no conflicts to declare; GM has a financial relationship and leadership role with Rocket Doctor, no other conflicts to declare; SM worked as statistical consultant for Aquifer and received salary support from Touchstone Institute, no other conflicts to declare.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Petrosoniak, A., Sherbino, J., Beardsley, T. et al. Are we talking about practice? A randomized study comparing simulation-based deliberate practice and mastery learning to self-guided practice. Can J Emerg Med 25, 667–675 (2023). https://doi.org/10.1007/s43678-023-00531-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s43678-023-00531-0

Keywords

Mots clés

Search

Navigation