Virtual reality (VR) simulation surgical skills training is well established, but self-directed practice is often associated with a learning curve plateau. In this study, we investigate the effects of structured self-assessment as a means to improve performance in mastoidectomy training.
The study was a prospective, educational study. Two cohorts of novices (medical students) were recruited for practice of anatomical mastoidectomy in a training program with five distributed training blocks. Fifteen participants performed structured self-assessment after each procedure (intervention cohort). A reference cohort of another 14 participants served as controls. Performances were assessed by two blinded raters using a modified Welling Scale and simulator-recorded metrics.
The self-assessment cohort performed superiorly to the reference cohort (mean difference of final product score 0.87 points, p = 0.001) and substantially reduced the number of repetitions needed. The self-assessment cohort also had more passing performances for the combined metrics-based score reflecting increased efficiency. Finally, the self-assessment cohort made fewer collisions compared with the reference cohort especially with the chorda tympani, the facial nerve, the incus, and the malleus.
VR simulation training of surgical skills benefits from having learners perform structured self-assessment following each procedure as this increases performance, accelerates the learning curve thereby reducing time needed for training, and induces a safer performance with fewer collisions with critical structures. Structured self-assessment was in itself not sufficient to counter the learning curve plateau and for continued skills development additional supports for deliberate practice are needed.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Frithioff A, Sørensen MS, Andersen SAW (2018) European status on temporal bone training: a questionnaire study. Eur Arch Otorhinolaryngol 275(2):357–363. https://doi.org/10.1007/s00405-017-4824-0
Fennessy BG, O’Sullivan P (2009) Establishing a temporal bone laboratory: considerations for ENT specialist training. Ir J Med Sci 178(4):393–395. https://doi.org/10.1007/s11845-009-0373-x
Andersen SAW, Foghsgaard S, Konge L, Caye-Thomasen P, Sørensen MS (2016) The effect of self-directed virtual reality simulation on dissection training performance in mastoidectomy. Laryngoscope 126(8):1883–1888. https://doi.org/10.1002/lary.25710
Wiet GJ, Stredney D, Kerwin T et al (2012) Virtual temporal bone dissection system: OSU virtual temporal bone system. Laryngoscope 122(S1):S1–S12. https://doi.org/10.1002/lary.22499
Zhao YC, Kennedy G, Yukawa K, Pyman B, Stephen O'Leary (2011) Can virtual reality simulator be used as a training aid to improve cadaver temporal bone dissection? Results of a randomized blinded control trial. Laryngoscope 121(4):831–837.https://doi.org/10.1002/lary.21287
Andersen SAW, Foghsgaard S, Caye-Thomasen P, Sørensen MS (2018) The effect of a distributed virtual reality simulation training program on dissection mastoidectomy performance. Otol Neurotol 39(10):1277–1284. https://doi.org/10.1097/MAO.0000000000002031
Andersen SA, Konge L, Caye-Thomasen P, Sørensen MS (2015) Learning curves of virtual mastoidectomy in distributed and massed practice. JAMA Otolaryngol Head Neck Surg141(10):913–918. https://doi.org/10.1001/jamaoto.2015.1563
Brydges R, Nair P, Ma I, Shanks D, Hatala R (2012) Directed self-regulated learning versus instructor-regulated learning in simulation training. Med Educ 46(7):648–656. https://doi.org/10.1111/j.1365-2923.2012.04268.x
Brydges R, Manzone J, Shanks D et al (2015) Self-regulated learning in simulation-based training: a systematic review and meta-analysis. Med Educ 49(4):368–378. https://doi.org/10.1111/medu.12649
Nash R, Sykes R, Majithia A, Arora A, Singh A, Khemani S (2012) Objective assessment of learning curves for the Voxel-Man TempoSurg temporal bone surgery computer simulator. J Laryngol Otol 126(07):663–669. https://doi.org/10.1017/S0022215112000734
Ericsson KA (2004) Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 79(Supplement):S70–S81
Jowett N, LeBlanc V, Xeroulis G, MacRae H, Dubrowski A (2007) Surgical skill acquisition with self-directed practice using computer-based video training. Am J Surg 193:237–242. https://doi.org/10.1016/j.amjsurg.2006.11.003
Andersen SA, Konge L, Mikkelsen PT, Cayé-Thomasen P, Sørensen MS (2017) Mapping the plateau of novices in virtual reality simulation training of mastoidectomy. Laryngoscope 127(4):907–914. https://doi.org/10.1002/lary.26000
Brydges R, Dubrowski A, Regehr G (2010) A new concept of unsupervised learning: directed self-guided learning in the health professions. Acad Med 85:S49–S55. https://doi.org/10.1097/ACM.0b013e3181ed4c96
MacDonald J, Williams RG, Rogers DA (2003) Self-assessment in simulation-based surgical skills training. Am J Surg 185:319–322. https://doi.org/10.1016/s0002-9610(02)01420-4
Brydges R, Carnahan H, Safir O, Dubrowski A (2009) How effective is self-guided learning of clinical technical skills? It’s all about process. Med Educ 43:507–515. https://doi.org/10.1111/j.1365-2923.2009.03329.x
Andersen SA, Cayé-Thomasen P, Sørensen MS (2015) Mastoidectomy performance assessment of virtual simulation training using final-product analysis. Laryngoscope 125(2):431–435. https://doi.org/10.1002/lary.24838
Trier P, Noe KØ, Sørensen MS, Mosegaard J (2008) The visible ear surgery simulator. Stud Health Technol Infor 132:523–525
Sørensen MS, Mosegaard J, Trier P (2009) The visible ear simulator: a public PC application for GPU-accelerated haptic 3D simulation of ear surgery based on the visible ear data. Otol Neurotol 30(4):484–487. https://doi.org/10.1097/MAO.0b013e3181a5299b
The visible ear simulator. Available from https://ves.alexandra.dk. Accessed 6 Sep 2019.
Andersen SA, Mikkelsen PT, Sørensen MS (2019) Expert sampling of VR simulator metrics for automated assessment of mastoidectomy performance. Laryngoscope 129(9):2170–2177. https://doi.org/10.1002/lary.27798
Sewell C, Morris D, Blevins NH, Barbagli F, Salisbury K (2007) Evaluating drilling and suctioning technique in a mastoidectomy simulator. Stud Health Technol Inform 125:427–432
Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L (2006) Accuracy of physician self-assessment compared with observed measures of competence. JAMA 296(9):1094–1102. https://doi.org/10.1001/jama.296.9.1094
Eva KW, Regehr G (2005) Self-assessment in the health professions: a reformulation and research agenda. Acad Med 80: S46–S54
Jethwa AR, Perdoni CJ, Kelly EA, Yueh B, Levine SC, Adams ME (2018) Randomized controlled pilot study of video self-assessment for resident mastoidectomy training. OTO Open 2(2): 2473974X1877041-X1877045. https://doi.org/10.1177/2473974X18770417
Reznick RK (1993) Teaching and testing technical skills. Am J Surg 165(3):358–361. https://doi.org/10.1016/s0002-9610(05)80843-8
Barry Issenberg S, Mcgaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ (2005) Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 27(1):10–28. https://doi.org/10.1080/01421590500046924
Piromchai P, Ioannou I, Wijewickrema S et al (2017) Effects of anatomical variation on trainee performance in a virtual reality temporal bone surgery simulator. J Laryngol Otol 131(S1):S29–S35. https://doi.org/10.1017/S0022215116009233
Al-Shahrestani F, Sørensen MS, Andersen SAW (2019) Performance metrics in mastoidectomy training: a systematic review. Eur Arch Otorhinolaryngol 276(3):657–664. https://doi.org/10.1007/s00405-018-05265-9
Rutledge C, Walsh CM, Swinger N et al (2018) Gamification in action: theoretical and practical considerations for medical educators. Acad Med 93(7):1014–1020. https://doi.org/10.1097/ACM.0000000000002183
The general development of the Visible Ear Simulator software has been financially supported by the Oticon Foundation. The foundation did not play a role in the design or conduct of the study.
Conflict of interest
The authors declare that they have no conflict of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Andersen, S.A.W., Guldager, M., Mikkelsen, P.T. et al. The effect of structured self-assessment in virtual reality simulation training of mastoidectomy. Eur Arch Otorhinolaryngol 276, 3345–3352 (2019). https://doi.org/10.1007/s00405-019-05648-6
- Temporal bone surgery
- Virtual reality surgical simulation
- Simulation-based training