Operator experience determines performance in a simulated computer-based brain tumor resection task



Develop measures to differentiate between experienced and inexperienced neurosurgeons in a virtual reality brain surgery simulator environment.


Medical students (\(n=71\)) and neurosurgery residents (\(n=12\)) completed four simulated Glioblastoma multiforme resections. Simulated surgeries took place over four days with intermittent spacing in between (average time between surgeries of 4.77 \(\pm \) 0.73 days). The volume of tumor removed (cc), volume of healthy brain removed (cc), and instrument path length (mm) were recorded. Additionally, surgical effectiveness (% tumor removed divided by % healthy brain removed) and efficiency (% tumor removed divided by instrument movement in mm) were calculated. Performance was compared (1) between groups, and (2) for each participant over time to assess the learning curve. In addition, the effect of real-time instruction (“coaching”) was assessed with a randomly selected group of medical students.


Neurosurgery residents removed less healthy brain, were more effective in removing tumor and sparing healthy brain tissue, required less instrument movement, and were more efficient in removing tumor tissue than medical students. Medical students approached the resident level of performance over serial sessions. Coached medical students showed more conservative surgical behavior, removing both less tumor and less healthy brain. In sum, neurosurgery residents removed more tumor, removed less healthy brain, and required less instrument movement than medical students. Coaching modified medical student performance.


Virtual Reality brain surgery can differentiate operators based on both recent and long-term experience and may be useful in the acquisition and assessment of neurosurgical skills. Coaching alters the learning curve of naïve inexperienced individuals.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 99

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6


  1. 1.

    Aarabi B, Hesdorffer DC, Ahn ES, Aresco C, Scalea TM, Eisenberg HM (2006) Outcome following decompressive craniectomy for malignant swelling due to severe head injury. J Neurosurg 104:469–479. doi:10.3171/jns.2006.104.4.469

  2. 2.

    Robison RA, Liu CY, Apuzzo MLJ (2011) Man, mind, and machine: the past and future of virtual reality simulation in neurologic surgery. World Neurosurg 76:419–430. doi:10.1016/j.wneu.2011.07.008

  3. 3.

    Schmitt PJ, Agarwal N, Prestigiacomo CJ (2012) From planes to brains: parallels between military development of virtual reality environments and virtual neurological surgery. World Neurosurg 78:214–219. doi:10.1016/j.wneu.2012.06.014

  4. 4.

    Choudhury N, Gélinas-Phaneuf N, Delorme S, Del Maestro R (2012) Fundamentals of neurosurgery: virtual reality tasks for training and evaluation of technical skills. World Neurosurg 9–19. doi:10.1016/j.wneu.2012.08.022

  5. 5.

    Kockro RA (2013) Neurosurgery simulators-beyond the experiment. World Neurosurg 80:e101–e102. doi:10.1016/j.wneu.2013.02.017

  6. 6.

    Aoun SG, Bendok BR, Rahme RJ, Dacey RG, Batjer HH (2013) Standardizing the evaluation of scientific and academic performance in neurosurgery-critical review of the “h” Index and its variants. World Neurosurg 80:e85–e90. doi:10.1016/j.wneu.2012.01.052

  7. 7.

    Munz Y, Almoudaris AM, Moorthy K, Dosis A, Liddle AD, Darzi AW (2007) Curriculum-based solo virtual reality training for laparoscopic intracorporeal knot tying: objective assessment of the transfer of skill from virtual reality to reality. Am J Surg 193:774–783. doi:10.1016/j.amjsurg.2007.01.022

  8. 8.

    Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK, Hanna GB, Jakimowicz JJ (2005) Consensus guidelines for validation of virtual reality surgical simulators. Surg Endosc 19:1523–1532. doi:10.1007/s00464-005-0384-2

  9. 9.

    Seixas-Mikelus SA, Kesavadas T, Srimathveeravalli G, Chandrasekhar R, Wilding GE, Guru KA (2010) Face validation of a novel robotic surgical simulator. Urology 76:357–360. doi:10.1016/j.urology.2009.11.069

  10. 10.

    Kenney PA, Wszolek MF, Gould JJ, Libertino JA, Moinzadeh A (2009) Face, content, and construct validity of dV-trainer, a novel virtual reality simulator for robotic surgery. Urology 73:1288–1292. doi:10.1016/j.urology.2008.12.044

  11. 11.

    Lee JY, Mucksavage P, Kerbl DC, Hunyh VB, Etafy M, McDougall EM (2012) Validation study of a virtual reality robotic simulator-role as an assessment tool? J Urol 187:998–1002. doi:10.1016/j.juro.2011.10.160

  12. 12.

    Gélinas-Phaneuf N, Choudhury N, Al-Habib AR, Cabral A, Nadeau E, Mora V, Pazos V, Debergue P, DiRaddo R, Del Maestro RF (2013) Assessing performance in brain tumor resection using a novel virtual reality simulator. Int J Comput Assist Radiol Surg. doi:10.1007/s11548-013-0905-8

  13. 13.

    Azarnoush H, Alzhrani G, Winkler-Schwartz A, Alotaibi F, Gelinas-Phaneuf N, Pazos V, Choudhury N, Fares J, DiRaddo R, Del Maestro RF (2014) Neurosurgical virtual reality simulation metrics to assess psychomotor skills during brain tumor resection. Int J Comput Assist Radiol Surg. doi:10.1007/s11548-014-1091-z

  14. 14.

    Paschold M, Schröder M, Kauff DW, Gorbauch T, Herzer M, Lang H, Kneist W (2011) Virtual reality laparoscopy: which potential trainee starts with a higher proficiency level? Int J Comput Assist Radiol Surg 6:653–662. doi:10.1007/s11548-010-0542-4

  15. 15.

    Selvander M, Åsman P (2012) Virtual reality cataract surgery training: learning curves and concurrent validity. Acta Ophthalmol 90:412–417. doi:10.1111/j.1755-3768.2010.02028.x

  16. 16.

    Iwata N, Fujiwara M, Kodera Y, Tanaka C, Ohashi N, Nakayama G, Koike M, Nakao A (2011) Construct validity of the LapVR virtual-reality surgical simulator. Surg Endosc 25:423–428. doi:10.1007/s00464-010-1184-x

  17. 17.

    Ohgaki H, Kleihues P (2005) Population-based studies on incidence, survival rates, and genetic alterations in astrocytic and oligodendroglial gliomas. J Neuropathol Exp Neurol 64:479–489

  18. 18.

    Gélinas-Phaneuf N, Del Maestro RF (2013) Surgical expertise in neurosurgery: integrating theory into practice. Neurosurgery 73(Suppl 1):30–38. doi:10.1227/NEU.0000000000000115

  19. 19.

    Delorme S, Laroche D, DiRaddo R, Del Maestro RF (2012) NeuroTouch: a physics-based virtual simulator for cranial microneurosurgery training. Neurosurgery 71:32–42. doi:10.1227/NEU.0b013e318249c744

  20. 20.

    Cole SJ, Mackenzie H, Ha J, Hanna GB, Miskovic D (2014) Randomized controlled trial on the effect of coaching in simulated laparoscopic training. Surg Endosc 28:979–986. doi:10.1007/s00464-013-3265-0

  21. 21.

    Teitelbaum EN, Soper NJ, Santos BF, Rooney DM, Patel P, Nagle AP, Hungness ES (2014) A simulator-based resident curriculum for laparoscopic common bile duct exploration. Surgery 156:880–887, 890–893. doi:10.1016/j.surg.2014.06.020

  22. 22.

    Rosseau G, Bailes J, del Maestro R, Choudhury N, Comas O, Debergue P, De Luca G, Hovdebo J, Jiang D, Laroche D, Neubauer A, Pazos V, Thibault F, DiRaddo R (2013) The development of a virtual simulator for training neurosurgeons to perform and perfect endoscopic endonasal transsphenoidal surgery. Neurosurgery 73(Suppl 1):85–93. doi:10.1227/NEU.0000000000000112

  23. 23.

    Neubauer A, Wolfsberger S (2013) Virtual endoscopy in neurosurgery: a review. Neurosurgery 72(Suppl 1):97–106. doi:10.1227/NEU.0b013e31827393c9

  24. 24.

    Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA, Ramel S, Smith CD, Arvidsson D (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193:797–804. doi:10.1016/j.amjsurg.2006.06.050

  25. 25.

    Badurdeen S, Abdul-Samad O, Story G, Wilson C, Down S, Harris A (2010) Nintendo Wii video-gaming ability predicts laparoscopic skill. Surg Endosc 24:1824–1828. doi:10.1007/s00464-009-0862-z

  26. 26.

    Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J (2003) Impact of hand dominance, gender, and experience with computer games on performance in virtual reality laparoscopy. Surg Endosc 17:1082–1085. doi:10.1007/s00464-002-9176-0

  27. 27.

    Kolozsvari NO, Andalib A, Kaneva P, Cao J, Vassiliou MC, Fried GM, Feldman LS (2011) Sex is not everything: the role of gender in early performance of a fundamental laparoscopic skill. Surg Endosc 25:1037–1042. doi:10.1007/s00464-010-1311-8

  28. 28.

    Schlickum MK, Hedman L, Enochsson L, Kjellin A, Fellander-Tsai L (2009) Systematic video game training in surgical novices improves performance in virtual reality endoscopic surgical simulators: a prospective randomized study. World J Surg 33:2360–2367. doi:10.1007/s00268-009-0151-y

  29. 29.

    Boyle E, Kennedy A-M, Traynor O, Hill ADK (2011) Training surgical skills using nonsurgical tasks-can Nintendo \(\text{ Wii }^{\text{ TM }}\) improve surgical performance? J Surg Educ 68:148–154. doi:10.1016/j.jsurg.2010.11.005

Download references


We would like to thank the NRC for logistical support, Steven Philemond for coaching participants in the study, and Dr. Errol Gordon and Dr. E. J. Fernandez for their assistance with experimental design, the medical students and neurosurgery residents of the Icahn School of Medicine at Mount Sinai for their participation in our study, and all those who contributed to the study or manuscript preparation. Zachary Lorsch and Michael Chary are supported by the Medical Scientist Training Program of the Icahn School of Medicine at Mount Sinai (NIH Grant # T32 GM 007280).

Conflict of interest

Terrell Holloway, Zachary Lorsch, Michael Chary, Stanislaw Sobotka, Maximilian Moore, Anthony Costa, and Rolando Del Maestro report no Conflicts of Interest. As stated in the disclosures, Dr. Joshua Bederson, is part of the Neurotouch Consortium advisory board without compensation.

Ethical standard All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent Informed consent was obtained from all individual participants included in the study. The institutional review board (IRB) of Mount Sinai Hospital determined that this study was exempt from IRB review.

Author information

Correspondence to Joshua Bederson.

Additional information

Terrell Holloway and Zachary S Lorsch have contributed equally to this work.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Video demonstration of simulated GBM resection using NeuroTouch surgical simulator. Outline of tools used %during virtual GBM resection; suction utilized by the aspirator in right hand (RH) and cauterization utilized by the %bipolar in left hand (LH)

Video demonstration of simulated GBM resection using NeuroTouch surgical simulator. Outline of tools used %during virtual GBM resection; suction utilized by the aspirator in right hand (RH) and cauterization utilized by the %bipolar in left hand (LH)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Holloway, T., Lorsch, Z.S., Chary, M.A. et al. Operator experience determines performance in a simulated computer-based brain tumor resection task. Int J CARS 10, 1853–1862 (2015) doi:10.1007/s11548-015-1160-y

Download citation


  • Simulator
  • Virtual reality
  • Training
  • NeuroTouch
  • Coaching
  • Glioblastoma multiforme tumor