Neurosurgical virtual reality simulation metrics to assess psychomotor skills during brain tumor resection

  • Hamed AzarnoushEmail author
  • Gmaan Alzhrani
  • Alexander Winkler-Schwartz
  • Fahad Alotaibi
  • Nicholas Gelinas-Phaneuf
  • Valérie Pazos
  • Nusrat Choudhury
  • Jawad Fares
  • Robert DiRaddo
  • Rolando F. Del Maestro
Original Article



   Virtual reality simulator technology together with novel metrics could advance our understanding of expert neurosurgical performance and modify and improve resident training and assessment. This pilot study introduces innovative metrics that can be measured by the state-of-the-art simulator to assess performance. Such metrics cannot be measured in an operating room and have not been used previously to assess performance.


   Three sets of performance metrics were assessed utilizing the NeuroTouch platform in six scenarios with simulated brain tumors having different visual and tactile characteristics. Tier 1 metrics included percentage of brain tumor resected and volume of simulated “normal” brain tissue removed. Tier 2 metrics included instrument tip path length, time taken to resect the brain tumor, pedal activation frequency, and sum of applied forces. Tier 3 metrics included sum of forces applied to different tumor regions and the force bandwidth derived from the force histogram.


   The results outlined are from a novice resident in the second year of training and an expert neurosurgeon. The three tiers of metrics obtained from the NeuroTouch simulator do encompass the wide variability of technical performance observed during novice/expert resections of simulated brain tumors and can be employed to quantify the safety, quality, and efficiency of technical performance during simulated brain tumor resection. Tier 3 metrics derived from force pyramids and force histograms may be particularly useful in assessing simulated brain tumor resections.


   Our pilot study demonstrates that the safety, quality, and efficiency of novice and expert operators can be measured using metrics derived from the NeuroTouch platform, helping to understand how specific operator performance is dependent on both psychomotor ability and cognitive input during multiple virtual reality brain tumor resections.


Performance metrics Virtual reality neurosurgical simulation Haptic feedback Brain tumor resection Neurosurgical oncology NeuroTouch 



National Research Council


Virtual reality


Postgraduate year


Cubic centimeters





This work was supported by The Di Giovanni Foundation, the Montreal English School Board, the B-Strong Foundation, the Colannini Foundation, and the Montreal Neurological Institute and Hospital. Dr. R. F. Del Maestro is the William Feindel Emeritus Professor in Neuro-Oncology at McGill University. Dr. Azarnoush held the Postdoctoral Neuro-Oncology Fellowship from the Montreal Neurological Institute and Hospital. Dr. Gélinas-Phaneuf was funded by a generous contribution from the Harold and Audrey Fisher Brain Tumour Research Award.

Supplementary material

Supplementary material 1 (wmv 4794 KB)

Supplementary material 2 (avi 7285 KB)

11548_2014_1091_MOESM3_ESM.avi (5 mb)
Supplementary material 3 (avi 5134 KB)
11548_2014_1091_MOESM4_ESM.doc (10 kb)
Supplementary material 4 (doc 10 KB)


  1. 1.
    Brydges R, Kurahashi A, Brümmer V, Satterthwaite L, Classen R, Dubrowski A (2008) Developing criteria for proficiency-based training of surgical technical skills using simulation: changes in performances as a function of training year. J Am Coll Surg 206:205–211. doi: 10.1016/j.jamcollsurg.2007.07.045 CrossRefPubMedGoogle Scholar
  2. 2.
    Aggarwal R, Grantcharov TR, Eriksen JR, Blirup D, Kristiansen VB, Funch-Jensen P, Darzi A (2006) An evidence-based virtual reality training program for novice laparoscopic surgeons. Ann Surg 244:310–314. doi: 10.1097/01.sla.0000218094.92650.44 CrossRefPubMedCentralPubMedGoogle Scholar
  3. 3.
    Ericsson KA, Charness N (1994) Expert performance. Am Psychol 49:725–747CrossRefGoogle Scholar
  4. 4.
    Tsuda S, Scott D, Doyle J, Jones DB (2009) Surgical skills training and simulation. Curr Probl Surg 46:271–370. doi: 10.1067/j.cpsurg.2008.12.003 CrossRefPubMedGoogle Scholar
  5. 5.
    Jaffer A, Bednarz B, Challacombe B, Sriprasad S (2009) The assessment of surgical competency in the UK. Int J Surg 7:12–15. doi: 10.1016/j.ijsu.2008.10.006 CrossRefPubMedGoogle Scholar
  6. 6.
    Reznick RK, MacRae H (2006) Teaching surgical skills—changes in the wind. N Eng J Med 355:2664–2669. doi: 10.1056/NEJMra054785 CrossRefGoogle Scholar
  7. 7.
    Brightwell A, Grant J (2013) Competency-based training: who benefits? Postgrad Med J 89:107–110. doi: 10.1136/postgradmedj-2012-130881 CrossRefPubMedGoogle Scholar
  8. 8.
    Bhatti NI, Cummings CW (2007) Competency in surgical residency training: defining and raising the bar. Acad Med 82:569–573CrossRefPubMedGoogle Scholar
  9. 9.
    Gélinas-Phaneuf N, Del Maestro RF (2013) Surgical expertise in neurosurgery integrating theory into practice. Neurosurgery 73(Suppl. 1):30–38. doi: 10.1227/NEU.0000000000000115 CrossRefPubMedGoogle Scholar
  10. 10.
    Gaba DM, DeAnda A (1998) A comprehensive anesthesia simulation environment: re-creating the operating room for research and training. Anesthesiology 69:387–394CrossRefGoogle Scholar
  11. 11.
    Chung J, Sackier J (1998) A method of objectively evaluating improvements in laparoscopic skills. Surg Endosc 12:1111–1116CrossRefPubMedGoogle Scholar
  12. 12.
    Derossis AM, Bothwell J, Sigman HH, Fried GM (1998) The effect of practice on performance in a laparoscopic simulator. Surg Endosc 12:1117–1120CrossRefPubMedGoogle Scholar
  13. 13.
    Delp SL, Loan JP, Hoy MG, Zajac FE, Topp EL, Rosen JM (1990) An interactive graphics-based model of the lower extremity to study orthopaedic surgical procedures. IEEE Trans Biomed Eng 37:757–767CrossRefPubMedGoogle Scholar
  14. 14.
    Satava RM (1993) Virtual reality surgical simulator. Surg Endosc 7:203–205CrossRefPubMedGoogle Scholar
  15. 15.
    Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA 3rd, Ramel S, Smith CD, Arvidsson D (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193:797–804. doi: 10.1016/j.amjsurg.2006.06.050 CrossRefPubMedGoogle Scholar
  16. 16.
    Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM, Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236:458–463. doi: 10.1097/01.SLA.0000028969.51489 CrossRefPubMedCentralPubMedGoogle Scholar
  17. 17.
    Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, Schroeder TV, Ottesen BS (2009) Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 338:b1802. doi: 10.1136/bmj.b1802 CrossRefPubMedCentralPubMedGoogle Scholar
  18. 18.
    Phillips NI, John NW (2000) Web-based surgical simulation for ventricular catheterization. Neurosurgery 46:933–937PubMedGoogle Scholar
  19. 19.
    Ferroli P, Tringali G, Acerbi F, Schiariti M, Broggi M, Aquino D, Broggi G (2013) Advanced 3-dimensional planning in neurosurgery. Neurosurgery 72(Suppl 1):54–62. doi: 10.1227/NEU.0b013e3182748ee8 CrossRefPubMedGoogle Scholar
  20. 20.
    Neubauer A, Wolfsberger S (2013) Virtual endoscopy in neurosurgery: a review. Neurosurgery 72(Suppl 1):97–106. doi: 10.1227/NEU.0b013e31827393c9 CrossRefPubMedGoogle Scholar
  21. 21.
    Mitha AP, Almekhlafi MA, Janjua MJ, Albuquerque FC, McDougall CG (2013) Simulation and augmented reality in endovascular neurosurgery: lessons from aviation. Neurosurgery 72(Suppl 1):107–114. doi: 10.1227/NEU.0b013e31827981fd CrossRefPubMedGoogle Scholar
  22. 22.
    Rousseau G, Bailes J, del Maestro RF, Cabral A, Choudhury N, Comas O, Debergue P, De Luca G, Hovdebo J, Jiang D, Laroche D, Neubauer A, Pazos V, Thibault T, DiRaddo R (2013) The development of a virtual simulator for training of neurosurgeons to perform and to perfect endoscopic endonasal transphenoidal surgery. Neurosurgery 73(Suppl. 1):85–93. doi: 10.1227/NEU.0000000000000112 CrossRefGoogle Scholar
  23. 23.
    Kockro RA, Serra L, Tseng-Tsai Y, Yih-Yian Chan CS, Gim-Guan C, Lee E, Hoe LY, Hern N, Nowinski WL (2000) Planning and simulation of neurosurgery in a virtual reality environment. Neurosurgery 46:118–137CrossRefPubMedGoogle Scholar
  24. 24.
    Luciano C, Banerjee P, Lemole G, Charbel F (2006) Second generation haptic ventriculostomy simulator using the immersive touch system. Stud Health Technol Inform 119:343–348 Google Scholar
  25. 25.
    Banerjee PP, Luciano CJ, Lemole GM Jr, Charbel FT, Oh MY (2007) Accuracy of ventriculostomy catheter placement using a head- and hand-tracked high-resolution virtual reality simulator with haptic feedback. J Neurosurg 107:515–521. doi: 10.3171/JNS-07/09/0515 CrossRefPubMedGoogle Scholar
  26. 26.
    Luciano CJ, Banerjee PP, Bellotte B, Oh GM, Lemole M, Jr., Charbel FT, Roitberg B (2011) Learning retention of thoracic pedicle screw placement using a high-resolution augmented reality simulator with haptic feedback. Neurosurgery 69(1 Suppl Operative):ons14–19. doi: 10.1227/NEU.0b013e31821954ed
  27. 27.
    Luciano CJ, Banerjee PP, Florea L, Dawe G (2005) Design of the ImmersiveTouch\(^{TM}\): a high-performance haptic augmented virtual reality system. In: 11th International conference on human-computer interaction, Las Vegas, NVGoogle Scholar
  28. 28.
    Delorme S, Laroche D, Diraddo R, F. Del Maestro R (2012) NeuroTouch: a Physics-based virtual simulator for cranial microneurosurgery training. Neurosurgery 71(1 Suppl Operative):ons 32–42. doi: 10.1227/NEU.0b013e318249c744
  29. 29.
    Choudhury N, Gelinas-Phaneuf N, Delorme S, Del Maestro R (2013) Fundamentals of Neurosurgery: virtual reality tasks for training and evaluation of technical skills. World Neurosurg 80:E9–E19. doi: 10.1016/j.wneu.2012.08.022
  30. 30.
    Gélinas-Phaneuf N, Choudhury N, Al-Habib AR, Cabral A, Nadeau E, Mora V, Debergue Pazos VP, Diraddo R, Del Maestro RF (2013) Assessing performance in brain tumor resection using a novel virtual reality simulator. Int J CARS. doi: 10.1007/s11548-013-0905-8
  31. 31.
    Wheatstone C (1838) Contributions to the physiology of vision.-Part the first. On some remarkable, and hitherto unobserved, phenomena of binocular vision. Philos Trans R Soc Lond 128:371–394CrossRefGoogle Scholar
  32. 32.
    Kohn LT, Corrigan J, Donaldson MS (2000) To err is human: building a safer health care system. National Academy Press, WashingtonGoogle Scholar
  33. 33.
    Gray JD (1996) Global rating scale in residency education. Acad Med 71:555–563CrossRefGoogle Scholar
  34. 34.
    Turnbull J, Gray J, MacFadyen J (1998) Improving in training evaluation programs. J Gen Intern Med 13:317–323. doi: 10.1046/j.1525-1497.1998.00097.x CrossRefPubMedCentralPubMedGoogle Scholar
  35. 35.
    Katz R, Hoznek A, Salomon L, Antiphon P, de la Taille A, Abbou CC (2005) Skill assessment of urological laparoscopic surgeons: can criterion levels of surgical performance be determined using the pelvic box trainer? Eur Urol 47:482–487. doi: 10.1016/j.eururo.2004.12.007 CrossRefPubMedGoogle Scholar
  36. 36.
    de Vries EN, Ramrattan MA, Smorenburg SM, Gouma DJ, Boermeester MA (2008) The incidence and nature of in-hospital adverse events: a systemic review. Qual Saf Health Care 17:216–23. doi: 10.1136/qshc.2007.023622 CrossRefPubMedCentralPubMedGoogle Scholar
  37. 37.
    Gawande AA, Thomas EJ, Zinner MJ, Brennan TA (1999) The incidence and nature of surgical adverse events in Colorado and Utah in 1992. Surgery 126:66–75CrossRefPubMedGoogle Scholar
  38. 38.
    Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJ (2013) Surgical skills and complication rates after bariatric surgery. N Engl J Med 369:1434–1442. doi: 10.1056/NEJMsa1300625 CrossRefPubMedGoogle Scholar
  39. 39.
    Kockro RA (2013) Neurosurgery simulators-beyond the experiment. World Neurosurg. 80:E101–E102. doi: 10.1016/j.wneu.2013.02.017

Copyright information

© CARS 2014

Authors and Affiliations

  • Hamed Azarnoush
    • 1
    • 3
    Email author
  • Gmaan Alzhrani
    • 1
    • 4
  • Alexander Winkler-Schwartz
    • 1
  • Fahad Alotaibi
    • 1
    • 4
  • Nicholas Gelinas-Phaneuf
    • 1
  • Valérie Pazos
    • 2
  • Nusrat Choudhury
    • 2
  • Jawad Fares
    • 1
  • Robert DiRaddo
    • 2
  • Rolando F. Del Maestro
    • 1
  1. 1.Neurosurgical Simulation Research Center, Department of Neurosurgery and Neurology, 2nd Floor, Montreal Neurological Institute and HospitalMcGill UniversityMontrealCanada
  2. 2.National Research Council CanadaBouchervilleCanada
  3. 3.Department of Biomedical EngineeringTehran PolytechnicTehranIran
  4. 4.National Neuroscience InstituteKing Fahad Medical CityRiyadhSaudi Arabia

Personalised recommendations