Assessing performance in brain tumor resection using a novel virtual reality simulator

  • Nicholas Gélinas-PhaneufEmail author
  • Nusrat Choudhury
  • Ahmed R. Al-Habib
  • Anne Cabral
  • Etienne Nadeau
  • Vincent Mora
  • Valerie Pazos
  • Patricia Debergue
  • Robert DiRaddo
  • Rolando F. Del MaestroEmail author
Original Article



NeuroTouch is a virtual reality (VR) simulator developed for neurosurgical skill training. Validation demonstrating that the system is useful and reliable is required for formal adoption into training curriculums. Face and content validity have been demonstrated for some neurosurgical simulators, but construct validity remains difficult to establish. A pilot validation study was conducted for a NeuroTouch training exercise.


Participants completed the internal resection of a simulated convexity meningioma and filled out questionnaires to provide feedback on the experience. Performance metrics included volume of tissues removed, tool path lengths, duration of excessive forces applied and efficient use of the aspirator. Results were analyzed according to participants’ level of training, gender, handedness, surgical experience in meningioma removal and hours/week playing musical instruments or video games.


Seventy-two participants (10 medical students, 18 junior residents and 44 senior residents) were enrolled. Analyses demonstrated statistically significant increase in tumor removed and efficiency of ultrasonic aspirator use between medical students and residents, but not between junior and senior residents. After covariate adjustment for the number of meningioma cases operated on, multivariate analysis of the level of training became nonsignificant. Participants judged the exercise appropriate and realistic, desiring use of the system in current training programs.


We have conducted a pilot validation study for the NeuroTouch tumor resection scenario and demonstrated for the first time, face, content and construct validity of a VR neurosurgical simulation exercise. Future full-scale studies will be conducted in noncompetitive settings and incorporate expert participants.


NeuroTouch Virtual reality simulation Haptic feedback Brain tumor resection Neurosurgical oncology  Performance metrics 



American Association of Neurological Surgeons


National Research Council


Virtual reality


Postgraduate year



We would like to thank all the individuals at the National Research Council Canada (NRC) and all the collaborators at Canadian and international Universities involved in this project. The VR surgical simulation program has been funded by the NRC Genomics and Health Initiative. This work was also supported by the Montréal English School Board, the Franco Di Giovanni, B-Strong, the Tony Colannino Foundations and the Montreal Neurological Institute and Hospital. Dr. Gelinas-Phaneuf was funded by a generous contribution from the Harold and Audrey Fisher Brain Tumour Research Award. Dr. Del Maestro holds the William Feindel Chair of Neuro-Oncology at the Montreal Neurological Institute.

Conflict of Interest

The authors declare no conflict of interest.

Supplementary material

11548_2013_905_MOESM1_ESM.mpg (14.8 mb)
ESM 1 (MPG 15,201 kb)


  1. 1.
    Ferroli P, Tringali G, Acerbi F, Schiariti M, Broggi M, Aquino D, Broggi G (2013) Advanced 3-dimensional planning in neurosurgery. Neurosurgery 72(Suppl 1):54–62. doi: 10.1227/NEU.0b013e3182748ee8 PubMedCrossRefGoogle Scholar
  2. 2.
    Mitha AP, Almekhlafi MA, Janjua MJ, Albuquerque FC, McDougall CG (2013) Simulation and augmented reality in endovascular neurosurgery: lessons from aviation. Neurosurgery 72(Suppl 1):107–114. doi: 10.1227/NEU.0b013e31827981fd PubMedCrossRefGoogle Scholar
  3. 3.
    Chan S, Conti F, Salisbury K, Blevins NH (2013) Virtual reality simulation in neurosurgery: technologies and evolution. Neurosurgery 72(Suppl 1):154–164. doi: 10.1227/NEU.0b013e3182750d26 PubMedCrossRefGoogle Scholar
  4. 4.
    Alaraj A, Lemole MG, Finkle JH, Yudkowsky R, Wallace A, Luciano C, Banerjee PP, Rizzi SH, Charbel FT (2011) Virtual reality training in neurosurgery: review of current status and future applications. Surg Neurol Int 2(1):52. doi: 10.4103/2152-7806.80117 PubMedCentralPubMedCrossRefGoogle Scholar
  5. 5.
    Malone HR, Syed ON, Downes MS, D’Ambrosio AL, Quest DO, Kaiser MG (2010) Simulation in neurosurgery: a review of computer-based simulation environments and their surgical applications. Neurosurgery 67(4):1105–1116. doi: 10.1227/NEU.0b013e3181ee46d0 PubMedCrossRefGoogle Scholar
  6. 6.
    Neubauer A, Wolfsberger S (2013) Virtual endoscopy in neurosurgery: a review. Neurosurgery 72(Suppl 1):97–106. doi: 10.1227/NEU.0b013e31827393c9 PubMedCrossRefGoogle Scholar
  7. 7.
    Choudhury N, Gelinas-Phaneuf N, Delorme S, Del Maestro RF (2012) Fundamentals of neurosurgery: virtual reality tasks for training and evaluation of technical skills. World Neurosurg. doi: 10.1016/j.wneu.2012.08.022
  8. 8.
    Delorme S, Laroche D, Diraddo R, Del Maestro RF (2012) NeuroTouch: a physics-based virtual simulator for cranial microneurosurgery training. Neurosurgery 71(1 Suppl Operative):ons32–ons42. doi: 10.1227/NEU.0b013e318249c744
  9. 9.
    Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK, Hanna GB, Jakimowicz JJ (2005) Consensus guidelines for validation of virtual reality surgical simulators. Surg Endosc 19(12):1523–1532. doi: 10.1007/s00464-005-0384-2 PubMedCrossRefGoogle Scholar
  10. 10.
    Gallagher AG, Ritter EM, Champion H, Higgins G, Fried MP, Moses G, Smith CD, Satava RM (2005) Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg 241(2):364–372PubMedCrossRefGoogle Scholar
  11. 11.
    Lewis TM, Aggarwal R, Kwasnicki RM, Rajaretnam N, Moorthy K, Ahmed A, Darzi A (2012) Can virtual reality simulation be used for advanced bariatric surgical training? Surgery 151(6):779–784. doi: 10.1016/j.surg.2012.03.014 PubMedCrossRefGoogle Scholar
  12. 12.
    Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA 3rd, Ramel S, Smith CD, Arvidsson D (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193(6):797–804. doi: 10.1016/j.amjsurg.2006.06.050 PubMedCrossRefGoogle Scholar
  13. 13.
    Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236(4):458–463; discussion 463–454. doi: 10.1097/01.SLA.0000028969.51489.B4 Google Scholar
  14. 14.
    Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, Schroeder TV, Ottesen BS (2009) Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 338:b1802. doi: 10.1136/bmj.b1802 PubMedCentralPubMedCrossRefGoogle Scholar
  15. 15.
    Perrenot C, Perez M, Tran N, Jehl JP, Felblinger J, Bresler L, Hubert J (2012) The virtual reality simulator dV-Trainer®is a valid assessment tool for robotic surgical skills. Surg Endosc 26(9):2587–2593. doi: 10.1007/s00464-012-2237-0 Google Scholar
  16. 16.
    Bajka M, Tuchschmid S, Fink D, Székely G, Harders M (2010) Establishing construct validity of a virtual-reality training simulator for hysteroscopy via a multimetric scoring system. Surg Endosc 24(1):79–88PubMedCrossRefGoogle Scholar
  17. 17.
    Spiotta AM, Rasmussen PA, Masaryk TJ, Benzel EC, Schlenk R (2012) Simulated diagnostic cerebral angiography in neurosurgical training: a pilot program. J Neurointerv Surg. doi: 10.1136/neurintsurg-2012-010319
  18. 18.
    Chaer RA, Derubertis BG, Lin SC, Bush HL, Karwowski JK, Birk D, Morrissey NJ, Faries PL, McKinsey JF, Kent KC (2006) Simulation improves resident performance in catheter-based intervention: results of a randomized, controlled study. Ann Surg 244(3):343–352PubMedGoogle Scholar
  19. 19.
    Marcus H, Vakharia V, Kirkman MA, Murphy M, Nandi D (2013) Practice makes perfect? The role of simulation-based deliberate practice and script-based mental rehearsal in the acquisition and maintenance of operative neurosurgical skills. Neurosurgery 72(Suppl 1):124–130. doi: 10.1227/NEU.0b013e318270d010 PubMedCrossRefGoogle Scholar
  20. 20.
    Alaraj A, Charbel FT, Birk D, Tobin M, Luciano C, Banerjee PP, Rizzi S, Sorenson J, Foley K, Slavin K, Roitberg B (2013) Role of cranial and spinal virtual and augmented reality simulation using immersive touch modules in neurosurgical training. Neurosurgery 72(Suppl 1):115–123. doi: 10.1227/NEU.0b013e3182753093 PubMedCentralPubMedCrossRefGoogle Scholar
  21. 21.
    Luciano CJ, Banerjee PP, Sorenson JM, Foley KT, Ansari SA, Rizzi S, Germanwala AV, Kranzler L, Chittiboina P, Roitberg BZ (2013) Percutaneous spinal fixation simulation with virtual reality and haptics. Neurosurgery 72(Suppl 1):89–96. doi: 10.1227/NEU.0b013e3182750a8d PubMedCrossRefGoogle Scholar
  22. 22.
    Banerjee PP, Luciano CJ, Lemole GM Jr, Charbel FT, Oh MY (2007) Accuracy of ventriculostomy catheter placement using a head- and hand-tracked high-resolution virtual reality simulator with haptic feedback. J Neurosurg 107(3):515–521. doi: 10.3171/JNS-07/09/0515 PubMedCrossRefGoogle Scholar
  23. 23.
    Luciano CJ, Banerjee PP, Bellotte B, Oh GM, Lemole Jr M, Charbel FT, Roitberg B (2011) Learning retention of thoracic pedicle screw placement using a high-resolution augmented reality simulator with haptic feedback. Neurosurgery 69 (1 Suppl Operative):ons14–ons19; discussion ons19. doi: 10.1227/NEU.0b013e31821954ed
  24. 24.
    Neubauer A, Brooks R, Brouwer I, Debergue P, Laroche D (2012) Haptic collision handling for simulation of transnasal surgery. Comput Animat Virtual Worlds. doi: 10.1002/cav.1489
  25. 25.
    Borgeat L, Massicotte P, Poirier G, Godin G (2011) Layered surface fluid simulation for surgical training. Med Image Comput Comput Assist Interv MICCAI 14(Pt 1):323–330Google Scholar
  26. 26.
    Delorme S, Cabral A, Ayres F, Jiang D (2011) Modeling the thermal effect of the bipolar electrocautery for neurosurgery simulation. Stud Health Technol Inform 163:166–172PubMedGoogle Scholar
  27. 27.
    Jiang D, Choudhury N, Mora V, Delorme S (2010) Characterization of suction and CUSA interaction with brain tissue. In: Paper presented at the proceedings of the 5th international conference on Biomedical simulation, Phoenix, AZ, USAGoogle Scholar
  28. 28.
    Mora V, Jiang D, Brooks R, Delorme S (2009) A computer model of soft tissue interaction with a surgical aspirator. Med Image Comput Comput Assist Interv MICCAI 12(Pt 1):51–58Google Scholar
  29. 29.
    Lemole M, Banerjee PP, Luciano C, Charbel F, Oh M (2009) Virtual ventriculostomy with ‘shifted ventricle’: neurosurgery resident surgical skill assessment using a high-fidelity haptic/graphic virtual reality simulator. Neurol Res 31(4):430–431. doi: 10.1179/174313208X353695 PubMedCrossRefGoogle Scholar
  30. 30.
    Iwata N, Fujiwara M, Kodera Y, Tanaka C, Ohashi N, Nakayama G, Koike M, Nakao A (2011) Construct validity of the LapVR virtual-reality surgical simulator. Surg Endosc 25(2):423–428. doi: 10.1007/s00464-010-1184-x Google Scholar
  31. 31.
    Bajka M, Tuchschmid S, Fink D, Székely G, Harders M (2010) Establishing construct validity of a virtual-reality training simulator for hysteroscopy via a multimetric scoring system. Surg Endosc 24(1):79–88PubMedCrossRefGoogle Scholar
  32. 32.
    Pellen MGC, Horgan LF, Barton JR, Attwood SE (2009) Construct validity of the ProMIS laparoscopic simulator. Surg Endosc 23(1):130–139PubMedCrossRefGoogle Scholar
  33. 33.
    Selvander M, Asman P (2010) Virtual reality cataract surgery training: learning curves and concurrent validity. Acta Ophthalmol. doi: 10.1111/j.1755-3768.2010.02028.x
  34. 34.
    Fried MP, Sadoughi B, Weghorst SJ, Zeltsan M, Cuellar H, Uribe JI, Sasaki CT, Ross DA, Jacobs JB, Lebowitz RA, Satava RM (2007) Construct validity of the endoscopic sinus surgery simulator: II. Assessment of discriminant validity and expert benchmarking. Arch Otolaryngol Head Neck Surg 133(4):350–357. doi: 10.1001/archotol.133.4.350 PubMedCrossRefGoogle Scholar
  35. 35.
    Paschold M, Schroder M, Kauff DW, Gorbauch T, Herzer M, Lang H, Kneist W (2011) Virtual reality laparoscopy: which potential trainee starts with a higher proficiency level? Int J Comput Assist Radiol Surg 6(5):653–662. doi: 10.1007/s11548-010-0542-4 PubMedCrossRefGoogle Scholar
  36. 36.
    Kolozsvari NO, Andalib A, Kaneva P, Cao J, Vassiliou MC, Fried GM, Feldman LS (2011) Sex is not everything: the role of gender in early performance of a fundamental laparoscopic skill. Surg Endosc 25(4):1037–1042. doi: 10.1007/s00464-010-1311-8 PubMedCrossRefGoogle Scholar
  37. 37.
    Van Herzeele I, O’Donoghue KG, Aggarwal R, Vermassen F, Darzi A, Cheshire NJ (2010) Visuospatial and psychomotor aptitude predicts endovascular performance of inexperienced individuals on a virtual reality simulator. J Vasc Surg Off Publ Soc Vasc Surg Int Soc Cardiovasc Surg N Am Chapter 51(4):1035–1042. doi: 10.1016/j.jvs.2009.11.059
  38. 38.
    Risucci D, Geiss A, Gellman L, Pinard B, Rosser J (2001) Surgeon-specific factors in the acquisition of laparoscopic surgical skills. Am J Surg 181(4):289–293PubMedCrossRefGoogle Scholar
  39. 39.
    Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J (2003) Impact of hand dominance, gender, and experience with computer games on performance in virtual reality laparoscopy. Surg Endosc 17(7):1082–1085. doi: 10.1007/s00464-002-9176-0 PubMedCrossRefGoogle Scholar
  40. 40.
    Enochsson L, Isaksson B, Tour R, Kjellin A, Hedman L, Wredmark T, Tsai-Fellander L (2004) Visuospatial skills and computer game experience influence the performance of virtual endoscopy. J Gastrointest Surg Off J Soc Surg Aliment Tract 8 (7):876–882; discussion 882. doi: 10.1016/j.gassur.2004.06.015 Google Scholar
  41. 41.
    Kennedy AM, Boyle EM, Traynor O, Walsh T, Hill AD (2011) Video gaming enhances psychomotor skills but not visuospatial and perceptual abilities in surgical trainees. J Surg Educ 68(5):414–420. doi: 10.1016/j.jsurg.2011.03.009 Google Scholar
  42. 42.
    Fanning J, Fenton B, Johnson C, Johnson J, Rehman S (2011) Comparison of teenaged video gamers vs PGY-I residents in obstetrics and gynecology on a laparoscopic simulator. J Minim Invasive Gynecol 18(2):169–172. doi: 10.1016/j.jmig.2010.11.002 PubMedCrossRefGoogle Scholar
  43. 43.
    Boyle E, Kennedy AM, Traynor O, Hill AD (2011) Training surgical skills using nonsurgical tasks-can Nintendo Wii\(^{\text{ TM }}\) improve surgical performance? J Surg Educ 68(2):148–154. doi:  10.1016/j.jsurg.2010.11.005 PubMedCrossRefGoogle Scholar
  44. 44.
    Badurdeen S, Abdul-Samad O, Story G, Wilson C, Down S, Harris A (2010) Nintendo Wii video-gaming ability predicts laparoscopic skill. Surg Endosc 24(8):1824–1828. doi: 10.1007/s00464-009-0862-z PubMedCrossRefGoogle Scholar
  45. 45.
    Schlickum MK, Hedman L, Enochsson L, Kjellin A, Fellander-Tsai L (2009) Systematic video game training in surgical novices improves performance in virtual reality endoscopic surgical simulators: a prospective randomized study. World J Surg 33(11):2360–2367. doi: 10.1007/s00268-009-0151-y PubMedCrossRefGoogle Scholar
  46. 46.
    Shane MD, Pettitt BJ, Morgenthal CB, Smith CD (2008) Should surgical novices trade their retractors for joysticks? Videogame experience decreases the time needed to acquire surgical skills. Surg Endosc 22(5):1294–1297. doi: 10.1007/s00464-007-9614-0 PubMedCrossRefGoogle Scholar
  47. 47.
    Lynch J, Aughwane P, Hammond TM (2010) Video games and surgical ability: a literature review. J Surg Educ 67(3):184–189. doi: 10.1016/j.jsurg.2010.02.010 PubMedCrossRefGoogle Scholar
  48. 48.
    Madan AK, Harper JL, Frantzides CT, Tichansky DS (2008) Nonsurgical skills do not predict baseline scores in inanimate box or virtual-reality trainers. Surg Endosc 22(7):1686–1689. doi: 10.1007/s00464-007-9691-0 PubMedCrossRefGoogle Scholar
  49. 49.
    Glaser AY, Hall CB, Uribe SJ, Fried MP (2005) The effects of previously acquired skills on sinus surgery simulator performance. Otolaryngol Head Neck Surg Off J Am Acad Otolaryngol Head Neck Surg 133(4):525–530. doi: 10.1016/j.otohns.2005.06.022 CrossRefGoogle Scholar
  50. 50.
    Selden NR, Barbaro N, Origitano TC, Burchiel KJ (2011) Fundamental skills for entering neurosurgery residents: report of a Pacific region “boot camp” pilot course, 2009. Neurosurgery 68(3):759–764; discussion 764. doi: 10.1227/NEU.0b013e3182077969 Google Scholar
  51. 51.
    Selden NR, Origitano TC, Burchiel KJ, Getch CC, Anderson VC, McCartney S, Abdulrauf SI, Barrow DL, Ehni BL, Grady MS, Hadjipanayis CG, Heilman CB, Popp AJ, Sawaya R, Schuster JM, Wu JK, Barbaro NM (2012) A national fundamentals curriculum for neurosurgery PGY1 residents: the 2010 Society of Neurological Surgeons boot camp courses. Neurosurgery 70 (4):971–981; discussion 981. doi: 10.1227/NEU.0b013e31823d7a45 Google Scholar
  52. 52.
    Laurence BG (2007) Who holds physicians accountable? Trans Am Clin Climatol Assoc 118:57–68PubMedCentralPubMedGoogle Scholar

Copyright information

© CARS 2013

Authors and Affiliations

  • Nicholas Gélinas-Phaneuf
    • 1
    Email author
  • Nusrat Choudhury
    • 2
  • Ahmed R. Al-Habib
    • 3
  • Anne Cabral
    • 2
  • Etienne Nadeau
    • 2
  • Vincent Mora
    • 2
  • Valerie Pazos
    • 2
  • Patricia Debergue
    • 2
  • Robert DiRaddo
    • 2
  • Rolando F. Del Maestro
    • 1
    Email author
  1. 1.Department of Neurosurgery and Neurology, Neurosurgical Simulation Research Center, Montreal Neurological Institute and HospitalMcGill UniversityMontrealCanada
  2. 2.National Research Council CanadaBouchervilleCanada
  3. 3.Royal College of Surgeons in IrelandDublinIreland

Personalised recommendations