Skip to main content
SpringerLink
Log in

The effect of structured self-assessment in virtual reality simulation training of mastoidectomy

  • Otology
  • Published:
European Archives of Oto-Rhino-Laryngology Aims and scope Submit manuscript

Abstract

Purpose

Virtual reality (VR) simulation surgical skills training is well established, but self-directed practice is often associated with a learning curve plateau. In this study, we investigate the effects of structured self-assessment as a means to improve performance in mastoidectomy training.

Methods

The study was a prospective, educational study. Two cohorts of novices (medical students) were recruited for practice of anatomical mastoidectomy in a training program with five distributed training blocks. Fifteen participants performed structured self-assessment after each procedure (intervention cohort). A reference cohort of another 14 participants served as controls. Performances were assessed by two blinded raters using a modified Welling Scale and simulator-recorded metrics.

Results

The self-assessment cohort performed superiorly to the reference cohort (mean difference of final product score 0.87 points, p = 0.001) and substantially reduced the number of repetitions needed. The self-assessment cohort also had more passing performances for the combined metrics-based score reflecting increased efficiency. Finally, the self-assessment cohort made fewer collisions compared with the reference cohort especially with the chorda tympani, the facial nerve, the incus, and the malleus.

Conclusions

VR simulation training of surgical skills benefits from having learners perform structured self-assessment following each procedure as this increases performance, accelerates the learning curve thereby reducing time needed for training, and induces a safer performance with fewer collisions with critical structures. Structured self-assessment was in itself not sufficient to counter the learning curve plateau and for continued skills development additional supports for deliberate practice are needed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Frithioff A, Sørensen MS, Andersen SAW (2018) European status on temporal bone training: a questionnaire study. Eur Arch Otorhinolaryngol 275(2):357–363. https://doi.org/10.1007/s00405-017-4824-0

    Article  PubMed  Google Scholar 

  2. Fennessy BG, O’Sullivan P (2009) Establishing a temporal bone laboratory: considerations for ENT specialist training. Ir J Med Sci 178(4):393–395. https://doi.org/10.1007/s11845-009-0373-x

    Article  CAS  PubMed  Google Scholar 

  3. Andersen SAW, Foghsgaard S, Konge L, Caye-Thomasen P, Sørensen MS (2016) The effect of self-directed virtual reality simulation on dissection training performance in mastoidectomy. Laryngoscope 126(8):1883–1888. https://doi.org/10.1002/lary.25710

    Article  Google Scholar 

  4. Wiet GJ, Stredney D, Kerwin T et al (2012) Virtual temporal bone dissection system: OSU virtual temporal bone system. Laryngoscope 122(S1):S1–S12. https://doi.org/10.1002/lary.22499

    Article  PubMed  PubMed Central  Google Scholar 

  5. Zhao YC, Kennedy G, Yukawa K, Pyman B, Stephen O'Leary (2011) Can virtual reality simulator be used as a training aid to improve cadaver temporal bone dissection? Results of a randomized blinded control trial. Laryngoscope 121(4):831–837.https://doi.org/10.1002/lary.21287

    Article  PubMed  Google Scholar 

  6. Andersen SAW, Foghsgaard S, Caye-Thomasen P, Sørensen MS (2018) The effect of a distributed virtual reality simulation training program on dissection mastoidectomy performance. Otol Neurotol 39(10):1277–1284. https://doi.org/10.1097/MAO.0000000000002031

    Article  Google Scholar 

  7. Andersen SA, Konge L, Caye-Thomasen P, Sørensen MS (2015) Learning curves of virtual mastoidectomy in distributed and massed practice. JAMA Otolaryngol Head Neck Surg141(10):913–918. https://doi.org/10.1001/jamaoto.2015.1563

  8. Brydges R, Nair P, Ma I, Shanks D, Hatala R (2012) Directed self-regulated learning versus instructor-regulated learning in simulation training. Med Educ 46(7):648–656. https://doi.org/10.1111/j.1365-2923.2012.04268.x

    Article  PubMed  Google Scholar 

  9. Brydges R, Manzone J, Shanks D et al (2015) Self-regulated learning in simulation-based training: a systematic review and meta-analysis. Med Educ 49(4):368–378. https://doi.org/10.1111/medu.12649

    Article  PubMed  Google Scholar 

  10. Nash R, Sykes R, Majithia A, Arora A, Singh A, Khemani S (2012) Objective assessment of learning curves for the Voxel-Man TempoSurg temporal bone surgery computer simulator. J Laryngol Otol 126(07):663–669. https://doi.org/10.1017/S0022215112000734

    Article  CAS  PubMed  Google Scholar 

  11. Ericsson KA (2004) Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 79(Supplement):S70–S81

    Article  Google Scholar 

  12. Jowett N, LeBlanc V, Xeroulis G, MacRae H, Dubrowski A (2007) Surgical skill acquisition with self-directed practice using computer-based video training. Am J Surg 193:237–242. https://doi.org/10.1016/j.amjsurg.2006.11.003

    Article  PubMed  Google Scholar 

  13. Andersen SA, Konge L, Mikkelsen PT, Cayé-Thomasen P, Sørensen MS (2017) Mapping the plateau of novices in virtual reality simulation training of mastoidectomy. Laryngoscope 127(4):907–914. https://doi.org/10.1002/lary.26000

    Article  Google Scholar 

  14. Brydges R, Dubrowski A, Regehr G (2010) A new concept of unsupervised learning: directed self-guided learning in the health professions. Acad Med 85:S49–S55. https://doi.org/10.1097/ACM.0b013e3181ed4c96

    Article  PubMed  Google Scholar 

  15. MacDonald J, Williams RG, Rogers DA (2003) Self-assessment in simulation-based surgical skills training. Am J Surg 185:319–322. https://doi.org/10.1016/s0002-9610(02)01420-4

    Article  PubMed  Google Scholar 

  16. Brydges R, Carnahan H, Safir O, Dubrowski A (2009) How effective is self-guided learning of clinical technical skills? It’s all about process. Med Educ 43:507–515. https://doi.org/10.1111/j.1365-2923.2009.03329.x

    Article  PubMed  Google Scholar 

  17. Andersen SA, Cayé-Thomasen P, Sørensen MS (2015) Mastoidectomy performance assessment of virtual simulation training using final-product analysis. Laryngoscope 125(2):431–435. https://doi.org/10.1002/lary.24838

    Article  Google Scholar 

  18. Trier P, Noe KØ, Sørensen MS, Mosegaard J (2008) The visible ear surgery simulator. Stud Health Technol Infor 132:523–525

  19. Sørensen MS, Mosegaard J, Trier P (2009) The visible ear simulator: a public PC application for GPU-accelerated haptic 3D simulation of ear surgery based on the visible ear data. Otol Neurotol 30(4):484–487. https://doi.org/10.1097/MAO.0b013e3181a5299b

    Article  Google Scholar 

  20. The visible ear simulator. Available from https://ves.alexandra.dk. Accessed 6 Sep 2019.

  21. Andersen SA, Mikkelsen PT, Sørensen MS (2019) Expert sampling of VR simulator metrics for automated assessment of mastoidectomy performance. Laryngoscope 129(9):2170–2177. https://doi.org/10.1002/lary.27798

    Article  Google Scholar 

  22. Sewell C, Morris D, Blevins NH, Barbagli F, Salisbury K (2007) Evaluating drilling and suctioning technique in a mastoidectomy simulator. Stud Health Technol Inform 125:427–432

    PubMed  Google Scholar 

  23. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L (2006) Accuracy of physician self-assessment compared with observed measures of competence. JAMA 296(9):1094–1102. https://doi.org/10.1001/jama.296.9.1094

    Article  CAS  PubMed  Google Scholar 

  24. Eva KW, Regehr G (2005) Self-assessment in the health professions: a reformulation and research agenda. Acad Med 80: S46–S54

    Article  Google Scholar 

  25. Jethwa AR, Perdoni CJ, Kelly EA, Yueh B, Levine SC, Adams ME (2018) Randomized controlled pilot study of video self-assessment for resident mastoidectomy training. OTO Open 2(2): 2473974X1877041-X1877045. https://doi.org/10.1177/2473974X18770417

    Article  Google Scholar 

  26. Reznick RK (1993) Teaching and testing technical skills. Am J Surg 165(3):358–361. https://doi.org/10.1016/s0002-9610(05)80843-8

    Article  CAS  PubMed  Google Scholar 

  27. Barry Issenberg S, Mcgaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ (2005) Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 27(1):10–28. https://doi.org/10.1080/01421590500046924

    Article  PubMed  Google Scholar 

  28. Piromchai P, Ioannou I, Wijewickrema S et al (2017) Effects of anatomical variation on trainee performance in a virtual reality temporal bone surgery simulator. J Laryngol Otol 131(S1):S29–S35. https://doi.org/10.1017/S0022215116009233

    Article  CAS  PubMed  Google Scholar 

  29. Al-Shahrestani F, Sørensen MS, Andersen SAW (2019) Performance metrics in mastoidectomy training: a systematic review. Eur Arch Otorhinolaryngol 276(3):657–664. https://doi.org/10.1007/s00405-018-05265-9

    Article  PubMed  Google Scholar 

  30. Rutledge C, Walsh CM, Swinger N et al (2018) Gamification in action: theoretical and practical considerations for medical educators. Acad Med 93(7):1014–1020. https://doi.org/10.1097/ACM.0000000000002183

    Article  PubMed  Google Scholar 

Download references

Funding

The general development of the Visible Ear Simulator software has been financially supported by the Oticon Foundation. The foundation did not play a role in the design or conduct of the study.

Author information

Authors and Affiliations

  1. Department of Otorhinolaryngology-Head and Neck Surgery, Rigshospitalet, Blegdamsvej 9, DK-2100, Copenhagen, Denmark

    Steven Arild Wuyts Andersen, Mads Guldager & Mads Sølvsten Sørensen

  2. Copenhagen Academy for Medical Education and Simulation (CAMES), Centre for HR, the Capital Region of Denmark, Copenhagen, Denmark

    Steven Arild Wuyts Andersen & Mads Guldager

  3. The Alexandra Institute, Aarhus, Denmark

    Peter Trier Mikkelsen

Authors
  1. Steven Arild Wuyts Andersen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mads Guldager
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter Trier Mikkelsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mads Sølvsten Sørensen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Steven Arild Wuyts Andersen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 1789 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Andersen, S.A.W., Guldager, M., Mikkelsen, P.T. et al. The effect of structured self-assessment in virtual reality simulation training of mastoidectomy. Eur Arch Otorhinolaryngol 276, 3345–3352 (2019). https://doi.org/10.1007/s00405-019-05648-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00405-019-05648-6

Keywords

Search

Navigation