Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

The effect of structured self-assessment in virtual reality simulation training of mastoidectomy

  • 108 Accesses

  • 1 Citations

Abstract

Purpose

Virtual reality (VR) simulation surgical skills training is well established, but self-directed practice is often associated with a learning curve plateau. In this study, we investigate the effects of structured self-assessment as a means to improve performance in mastoidectomy training.

Methods

The study was a prospective, educational study. Two cohorts of novices (medical students) were recruited for practice of anatomical mastoidectomy in a training program with five distributed training blocks. Fifteen participants performed structured self-assessment after each procedure (intervention cohort). A reference cohort of another 14 participants served as controls. Performances were assessed by two blinded raters using a modified Welling Scale and simulator-recorded metrics.

Results

The self-assessment cohort performed superiorly to the reference cohort (mean difference of final product score 0.87 points, p = 0.001) and substantially reduced the number of repetitions needed. The self-assessment cohort also had more passing performances for the combined metrics-based score reflecting increased efficiency. Finally, the self-assessment cohort made fewer collisions compared with the reference cohort especially with the chorda tympani, the facial nerve, the incus, and the malleus.

Conclusions

VR simulation training of surgical skills benefits from having learners perform structured self-assessment following each procedure as this increases performance, accelerates the learning curve thereby reducing time needed for training, and induces a safer performance with fewer collisions with critical structures. Structured self-assessment was in itself not sufficient to counter the learning curve plateau and for continued skills development additional supports for deliberate practice are needed.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3

References

  1. 1.

    Frithioff A, Sørensen MS, Andersen SAW (2018) European status on temporal bone training: a questionnaire study. Eur Arch Otorhinolaryngol 275(2):357–363. https://doi.org/10.1007/s00405-017-4824-0

  2. 2.

    Fennessy BG, O’Sullivan P (2009) Establishing a temporal bone laboratory: considerations for ENT specialist training. Ir J Med Sci 178(4):393–395. https://doi.org/10.1007/s11845-009-0373-x

  3. 3.

    Andersen SAW, Foghsgaard S, Konge L, Caye-Thomasen P, Sørensen MS (2016) The effect of self-directed virtual reality simulation on dissection training performance in mastoidectomy. Laryngoscope 126(8):1883–1888. https://doi.org/10.1002/lary.25710

  4. 4.

    Wiet GJ, Stredney D, Kerwin T et al (2012) Virtual temporal bone dissection system: OSU virtual temporal bone system. Laryngoscope 122(S1):S1–S12. https://doi.org/10.1002/lary.22499

  5. 5.

    Zhao YC, Kennedy G, Yukawa K, Pyman B, Stephen O'Leary (2011) Can virtual reality simulator be used as a training aid to improve cadaver temporal bone dissection? Results of a randomized blinded control trial. Laryngoscope 121(4):831–837.https://doi.org/10.1002/lary.21287

  6. 6.

    Andersen SAW, Foghsgaard S, Caye-Thomasen P, Sørensen MS (2018) The effect of a distributed virtual reality simulation training program on dissection mastoidectomy performance. Otol Neurotol 39(10):1277–1284. https://doi.org/10.1097/MAO.0000000000002031

  7. 7.

    Andersen SA, Konge L, Caye-Thomasen P, Sørensen MS (2015) Learning curves of virtual mastoidectomy in distributed and massed practice. JAMA Otolaryngol Head Neck Surg141(10):913–918. https://doi.org/10.1001/jamaoto.2015.1563

  8. 8.

    Brydges R, Nair P, Ma I, Shanks D, Hatala R (2012) Directed self-regulated learning versus instructor-regulated learning in simulation training. Med Educ 46(7):648–656. https://doi.org/10.1111/j.1365-2923.2012.04268.x

  9. 9.

    Brydges R, Manzone J, Shanks D et al (2015) Self-regulated learning in simulation-based training: a systematic review and meta-analysis. Med Educ 49(4):368–378. https://doi.org/10.1111/medu.12649

  10. 10.

    Nash R, Sykes R, Majithia A, Arora A, Singh A, Khemani S (2012) Objective assessment of learning curves for the Voxel-Man TempoSurg temporal bone surgery computer simulator. J Laryngol Otol 126(07):663–669. https://doi.org/10.1017/S0022215112000734

  11. 11.

    Ericsson KA (2004) Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 79(Supplement):S70–S81

  12. 12.

    Jowett N, LeBlanc V, Xeroulis G, MacRae H, Dubrowski A (2007) Surgical skill acquisition with self-directed practice using computer-based video training. Am J Surg 193:237–242. https://doi.org/10.1016/j.amjsurg.2006.11.003

  13. 13.

    Andersen SA, Konge L, Mikkelsen PT, Cayé-Thomasen P, Sørensen MS (2017) Mapping the plateau of novices in virtual reality simulation training of mastoidectomy. Laryngoscope 127(4):907–914. https://doi.org/10.1002/lary.26000

  14. 14.

    Brydges R, Dubrowski A, Regehr G (2010) A new concept of unsupervised learning: directed self-guided learning in the health professions. Acad Med 85:S49–S55. https://doi.org/10.1097/ACM.0b013e3181ed4c96

  15. 15.

    MacDonald J, Williams RG, Rogers DA (2003) Self-assessment in simulation-based surgical skills training. Am J Surg 185:319–322. https://doi.org/10.1016/s0002-9610(02)01420-4

  16. 16.

    Brydges R, Carnahan H, Safir O, Dubrowski A (2009) How effective is self-guided learning of clinical technical skills? It’s all about process. Med Educ 43:507–515. https://doi.org/10.1111/j.1365-2923.2009.03329.x

  17. 17.

    Andersen SA, Cayé-Thomasen P, Sørensen MS (2015) Mastoidectomy performance assessment of virtual simulation training using final-product analysis. Laryngoscope 125(2):431–435. https://doi.org/10.1002/lary.24838

  18. 18.

    Trier P, Noe KØ, Sørensen MS, Mosegaard J (2008) The visible ear surgery simulator. Stud Health Technol Infor 132:523–525

  19. 19.

    Sørensen MS, Mosegaard J, Trier P (2009) The visible ear simulator: a public PC application for GPU-accelerated haptic 3D simulation of ear surgery based on the visible ear data. Otol Neurotol 30(4):484–487. https://doi.org/10.1097/MAO.0b013e3181a5299b

  20. 20.

    The visible ear simulator. Available from https://ves.alexandra.dk. Accessed 6 Sep 2019.

  21. 21.

    Andersen SA, Mikkelsen PT, Sørensen MS (2019) Expert sampling of VR simulator metrics for automated assessment of mastoidectomy performance. Laryngoscope 129(9):2170–2177. https://doi.org/10.1002/lary.27798

  22. 22.

    Sewell C, Morris D, Blevins NH, Barbagli F, Salisbury K (2007) Evaluating drilling and suctioning technique in a mastoidectomy simulator. Stud Health Technol Inform 125:427–432

  23. 23.

    Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L (2006) Accuracy of physician self-assessment compared with observed measures of competence. JAMA 296(9):1094–1102. https://doi.org/10.1001/jama.296.9.1094

  24. 24.

    Eva KW, Regehr G (2005) Self-assessment in the health professions: a reformulation and research agenda. Acad Med 80: S46–S54

  25. 25.

    Jethwa AR, Perdoni CJ, Kelly EA, Yueh B, Levine SC, Adams ME (2018) Randomized controlled pilot study of video self-assessment for resident mastoidectomy training. OTO Open 2(2): 2473974X1877041-X1877045. https://doi.org/10.1177/2473974X18770417

  26. 26.

    Reznick RK (1993) Teaching and testing technical skills. Am J Surg 165(3):358–361. https://doi.org/10.1016/s0002-9610(05)80843-8

  27. 27.

    Barry Issenberg S, Mcgaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ (2005) Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 27(1):10–28. https://doi.org/10.1080/01421590500046924

  28. 28.

    Piromchai P, Ioannou I, Wijewickrema S et al (2017) Effects of anatomical variation on trainee performance in a virtual reality temporal bone surgery simulator. J Laryngol Otol 131(S1):S29–S35. https://doi.org/10.1017/S0022215116009233

  29. 29.

    Al-Shahrestani F, Sørensen MS, Andersen SAW (2019) Performance metrics in mastoidectomy training: a systematic review. Eur Arch Otorhinolaryngol 276(3):657–664. https://doi.org/10.1007/s00405-018-05265-9

  30. 30.

    Rutledge C, Walsh CM, Swinger N et al (2018) Gamification in action: theoretical and practical considerations for medical educators. Acad Med 93(7):1014–1020. https://doi.org/10.1097/ACM.0000000000002183

Download references

Funding

The general development of the Visible Ear Simulator software has been financially supported by the Oticon Foundation. The foundation did not play a role in the design or conduct of the study.

Author information

Correspondence to Steven Arild Wuyts Andersen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 1789 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Andersen, S.A.W., Guldager, M., Mikkelsen, P.T. et al. The effect of structured self-assessment in virtual reality simulation training of mastoidectomy. Eur Arch Otorhinolaryngol 276, 3345–3352 (2019). https://doi.org/10.1007/s00405-019-05648-6

Download citation

Keywords

  • Self-assessment
  • Temporal bone surgery
  • Mastoidectomy
  • Virtual reality surgical simulation
  • Tutoring
  • Simulation-based training