Physical and virtual surgical simulators are increasingly being used in training technical surgical skills. However, metrics such as completion time or subjective performance checklists often show poor correlation to transfer of skills into clinical settings. We hypothesize that non-invasive brain imaging can objectively differentiate and classify surgical skill transfer, with higher accuracy than established metrics, for subjects based on motor skill levels.
18 medical students at University at Buffalo were randomly assigned into control, physical surgical trainer, or virtual trainer groups. Training groups practiced a surgical technical task on respective simulators for 12 consecutive days. To measure skill transfer post-training, all subjects performed the technical task in an ex-vivo environment. Cortical activation was measured using functional near-infrared spectroscopy (fNIRS) in the prefrontal cortex, primary motor cortex, and supplementary motor area, due to their direct impact on motor skill learning.
Classification between simulator trained and untrained subjects based on traditional metrics is poor, where misclassification errors range from 20 to 41%. Conversely, fNIRS metrics can successfully classify physical or virtual trained subjects from untrained subjects with misclassification errors of 2.2% and 8.9%, respectively. More importantly, untrained subjects are successfully classified from physical or virtual simulator trained subjects with misclassification errors of 2.7% and 9.1%, respectively.
fNIRS metrics are significantly more accurate than current established metrics in classifying different levels of surgical motor skill transfer. Our approach brings robustness, objectivity, and accuracy in validating the effectiveness of future surgical trainers in translating surgical skills to clinically relevant environments.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Dawe SR, Windsor JA, Broeders JAJL, Cregan PC, Hewett PJ, Maddern GJ (2014) A systematic review of surgical skills transfer after simulation-based training. Ann Surg 259:236–248. https://doi.org/10.1097/SLA.0000000000000245
Fraser SA, Klassen DR, Feldman LS, Ghitulescu GA, Stanbridge D, Fried GM (2003) Evaluating laparoscopic skills: Setting the pass/fail score for the MISTELS system. Surg Endosc 17:964–967. https://doi.org/10.1007/s00464-002-8828-4
Fraser SA, Feldman LS, Stanbridge D, Fried GM (2005) Characterizing the learning curve for a basic laparoscopic drill. Surg Endosc 19:1572–1578. https://doi.org/10.1007/s00464-005-0150-5
Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG (2004) Proving the value of simulation in laparoscopic surgery. Ann Surg 240:518–525. https://doi.org/10.1097/01.SLA.0000136941.46529.56
McCluney AL, Vassiliou MC, Kaneva PA, Cao J, Stanbridge DD, Feldman LS, Fried GM (2007) FLS simulator performance predicts intraoperative laparoscopic skill. Surg Endosc 21:1991–1995. https://doi.org/10.1007/s00464-007-9451-1
Scott DJ, Ritter EM, Tesfay ST, Pimentel EA, Nagji A, Fried GM (2008) Certification pass rate of 100% for fundamentals of laparoscopic surgery skills after proficiency-based training. Surg Endosc 22:1887–1893. https://doi.org/10.1007/s00464-008-9745-y
Satava RM (2010) Emerging trends that herald the future of surgical simulation. Surg Clin North Am 90:623–633. https://doi.org/10.1016/j.suc.2010.02.002
Fried GM (2008) FLS assessment of competency using simulated laparoscopic tasks. J Gastrointest Surg 12:210–212. https://doi.org/10.1007/s11605-007-0355-0
Poulose BK, Vassiliou MC, Dunkin BJ, Mellinger JD, Fanelli RD, Martinez JM, Hazey JW, Sillin LF, Delaney CP, Velanovich V, Fried GM, Korndorffer JR, Marks JM (2014) Fundamentals of endoscopic surgery cognitive examination: development and validity evidence. Surg Endosc 28:631–638. https://doi.org/10.1007/s00464-013-3220-0
Vassiliou MC, Dunkin BJ, Fried GM, Mellinger JD, Trus T, Kaneva P, Lyons C, Korndorffer JR, Ujiki M, Velanovich V, Kochman ML, Tsuda S, Martinez J, Scott DJ, Korus G, Park A, Marks JM (2014) Fundamentals of endoscopic surgery: creation and validation of the hands-on test. Surg Endosc 28:704–711. https://doi.org/10.1007/s00464-013-3298-4
McDougall EM (2007) Validation of surgical simulators. J Endourol 21:244–247. https://doi.org/10.1089/end.2007.9985
Scott DJ, Pugh CM, Ritter EM, Jacobs LM, Pellegrini CA, Sachdeva AK (2011) New directions in simulation-based surgical education and training: validation and transfer of surgical skills, use of nonsurgeons as faculty, use of simulation to screen and select surgery residents, and long-term follow-up of learners. Surgery 149:735–744. https://doi.org/10.1016/j.surg.2010.11.010
Aggarwal R, Grantcharov T, Moorthy K, Milland T, Papasavas P, Dosis A, Bello F, Darzi A (2007) An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room. Ann Surg 245:992–999. https://doi.org/10.1097/01.sla.0000262780.17950.e5
Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondré K, Stanbridge D, Fried GM (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:107–113. https://doi.org/10.1016/j.amjsurg.2005.04.004
Doyle JD, Webber EM, Sidhu RS (2007) A universal global rating scale for the evaluation of technical skills in the operating room. Am J Surg 193:551–555. https://doi.org/10.1016/j.amjsurg.2007.02.003
Dawe SR, Pena GN, Windsor JA, Broeders JAJL, Cregan PC, Hewett PJ, Maddern GJ (2014) Systematic review of skills transfer after surgical simulation-based training. Br J Surg 101:1063–1076. https://doi.org/10.1002/bjs.9482
Moorthy K, Munz Y (2003) Objective assessment of technical skills in surgery. Br Med J 327:1032–1037. https://doi.org/10.1136/bmj.327.7422.1032
Hogle NJ, Chang L, Strong VEM, Welcome AOU, Sinaan M, Bailey R, Fowler DL (2009) Validation of laparoscopic surgical skills training outside the operating room: a long road. Surg Endosc 23:1476–1482. https://doi.org/10.1007/s00464-009-0379-5
Bosecker C, Dipietro L, Volpe B, Krebs HI (2010) Kinematic robot-based evaluation scales and clinical counterparts to measure upper limb motor performance in patients with chronic stroke. Neurorehabil Neural Repair 24:62–69. https://doi.org/10.1177/1545968309343214
Buckley CE, Kavanagh DO, Traynor O, Neary PC (2014) Is the skillset obtained in surgical simulation transferable to the operating theatre? Am J Surg 207:146–157. https://doi.org/10.1016/j.amjsurg.2013.06.017
Dai X, Zhang T, Yang H, Tang J, Carney PR, Jiang H (2017) Fast non-invasive functional diffuse optical tomography for brain imaging. J Biophotonics. https://doi.org/10.1002/jbio.201600267
Leff DR, Orihuela-Espina F, Elwell CE, Athanasiou T, Delpy DT, Darzi AW, Yang G-Z (2011) Assessment of the cerebral cortex during motor task behaviours in adults: a systematic review of functional near infrared spectroscopy (fNIRS) studies. Neuroimage 54:2922–2936. https://doi.org/10.1016/j.neuroimage.2010.10.058
Leff DR, Elwell CE, Orihuela-Espina F, Atallah L, Delpy DT, Darzi AW, Yang GZ (2008) Changes in prefrontal cortical behaviour depend upon familiarity on a bimanual co-ordination task: an fNIRS study. Neuroimage 39:805–813. https://doi.org/10.1016/j.neuroimage.2007.09.032
James DRC, Orihuela-Espina F, Leff DR, Sodergren MH, Athanasiou T, Darzi AW, Yang G-Z (2011) The ergonomics of natural orifice translumenal endoscopic surgery (NOTES) navigation in terms of performance, stress, and cognitive behavior. Surgery 149:525–533. https://doi.org/10.1016/j.surg.2010.11.019
Ohuchida K, Kenmotsu H, Yamamoto A, Sawada K, Hayami T, Morooka K, Takasugi S, Konishi K, Ieiri S, Tanoue K, Iwamoto Y, Tanaka M, Hashizume M (2009) The frontal cortex is activated during learning of endoscopic procedures. Surg Endosc 23:2296–2301. https://doi.org/10.1007/s00464-008-0316-z
Shewokis PA, Shariff FU, Liu Y, Ayaz H, Castellanos A, Lind DS (2017) Acquisition, retention and transfer of simulated laparoscopic tasks using fNIR and a contextual interference paradigm. Am J Surg 213:336–345. https://doi.org/10.1016/j.amjsurg.2016.11.043
Andreu-Perez J, Leff DR, Shetty K, Darzi A, Yang G-Z (2016) Disparity in frontal lobe connectivity on a complex bimanual motor task aids in classification of operator skill level. Brain Connect 6:375–388. https://doi.org/10.1089/brain.2015.0350
Modi HN, Singh H, Yang G-Z, Darzi A, Leff DR, Mansur A, Yang AE, Darzi G-Z, Leff A DR (2017) A decade of imaging surgeons’ brain function (part I): terminology, techniques, and clinical translation. Surgery 28:2189–2198. https://doi.org/10.1016/j.surg.2017.05.021
Nemani A, Kruger U, Intes X, De S (2017) Increased sensitivity in discriminating surgical motor skills using prefrontal cortex activation over established metrics. In: Optics in the life sciences congress. OSA, San Diego, p 11
Soper NJ, Fried GM (2008) The fundamentals of laparoscopic surgery: its time has come. Bull Am Coll Surg 93:30–32
Peters JH, Fried GM, Swanstrom LL, Soper NJ, Sillin LF, Schirmer B, Hoffman K, the SAGES FLS Committee the SF (2004) Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery 135:21–27. https://doi.org/10.1016/S0039-6060(03)00156-9
Zhang L, Sankaranarayanan G, Arikatla VS, Ahn W, Grosdemouge C, Rideout JM, Epstein SK, De S, Schwaitzberg SD, Jones DB, Cao CGL (2013) Characterizing the learning curve of the VBLaST-PT(©) (Virtual basic laparoscopic skill trainer). Surg Endosc 27:3603–3615. https://doi.org/10.1007/s00464-013-2932-5
Maciel A, Liu Y, Ahn W, Singh TP, Dunnican W, De S (2008) Development of the VBLaST: a virtual basic laparoscopic skill trainer. Int J Med Robot Comput Assist Surg 4:131–138. https://doi.org/10.1002/rcs.185
Arikatla VS, Sankaranarayanan G, Ahn W, Chellali A, De S, Caroline GL, Hwabejire J, DeMoya M, Schwaitzberg S, Jones DB (2013) Face and construct validation of a virtual peg transfer simulator. Surg Endosc 27:1721–1729. https://doi.org/10.1007/s00464-012-2664-y
Sankaranarayanan G, Lin H, Arikatla VS, Mulcare M, Zhang L, Derevianko A, Lim R, Fobert D, Cao C, Schwaitzberg SD, Jones DB, De S (2010) Preliminary face and construct validation study of a virtual basic laparoscopic skill trainer. J Laparoendosc Adv Surg Tech 20:153–157. https://doi.org/10.1089/lap.2009.0030
Chellali A, Ahn W, Sankaranarayanan G, Flinn JT, Schwaitzberg SD, Jones DB, De S, Cao CGL (2015) Preliminary evaluation of the pattern cutting and the ligating loop virtual laparoscopic trainers. Surg Endosc 29:815–821. https://doi.org/10.1007/s00464-014-3764-7
Nemani A, Ahn W, Cooper C, Schwaitzberg S, De S Convergent validation and transfer of learning studies of a virtual reality-based pattern cutting simulator. Surg Endosc. https://doi.org/10.1007/s00464-017-5802-8
Hikosaka O, Nakamura K, Sakai K, Nakahara H (2002) Central mechanisms of motor skill learning. Curr Opin Neurobiol 12:217–222. https://doi.org/10.1016/S0959-4388(02)00307-0
Wolpert DM, Diedrichsen J, Flanagan JR (2011) Principles of sensorimotor learning. Nat Rev Neurosci 12:739–751. https://doi.org/10.1038/nrn3112
Swinnen SP (2002) Intermanual coordination: from behavioural principles to neural-network interactions. Nat Rev Neurosci 3:348–359. https://doi.org/10.1038/nrn807
Swinnen SP, Gooijers J (2015) Bimanual Coordination. In: Brain mapp, 1st edn. Elsevier, San Diego, pp 475–482
Swinnen SP, Wenderoth N (2004) Two hands, one brain: cognitive neuroscience of bimanual skill. Trends Cogn Sci 8:18–25
Nemani A, Intes X, De S (2014) Monte Carlo based simulation of sensitivity curvature for evaluating optimal probe geometry. In: Biomedical optics 2014. OSA, Washington, D.C., p BM3A.36
Cope M, Delpy DT (1988) System for long-term measurement of cerebral blood and tissue oxygenation on newborn infants by near infra-red transillumination. Med Biol Eng Comput 26:289–294. https://doi.org/10.1007/BF02447083
Delpy DT, Cope M, Zee P, van der Arridge S, Wray S, Wyatt J (1988) Estimation of optical pathlength through tissue from direct time of flight measurement. Phys Med Biol 33:1433–1442. https://doi.org/10.1088/0031-9155/33/12/008
Huppert TJ, Diamond SG, Franceschini MA, Boas DA (2009) Homer: a review of time-series analysis methods for near-infrared spectroscopy of the brain. Appl Opt 48:280–298. https://doi.org/10.1364/AO.48.00D280
Franceschini MA, Joseph DK, Huppert TJ, Diamond SG, Boas DA (2006) Diffuse optical imaging of the whole head. J Biomed Opt 11:1–22. https://doi.org/10.1117/1.2363365
Zhang Y, Brooks DH, Franceschini MA, Boas DA (2005) Eigenvector-based spatial filtering for reduction of physiological interference in diffuse optical imaging. J Biomed Opt 10:1–11. https://doi.org/10.1117/1.1852552
Gagnon L, Yücel MA, Boas DA, Cooper RJ (2014) Further improvement in reducing superficial contamination in NIRS using double short separation measurements. Neuroimage 85:127–135. https://doi.org/10.1016/j.neuroimage.2013.01.073
Tabachnick BG, Fidell LS (2007) Using multivariate statistics, 5th edn. Allyn & Bacon, Needham Heights
Silverman BW (1986) Density estimation for statistics and data analysis. CRC press, Boca Raton
Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA, Ramel S, Smith CD, Arvidsson D (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193:797–804. https://doi.org/10.1016/j.amjsurg.2006.06.050
Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236:458–463. https://doi.org/10.1097/01.SLA.0000028969.51489.B4
Ahlberg G, Hultcrantz R, Jaramillo E, Lindblom A, Arvidsson D (2005) Virtual reality colonoscopy simulation: a compulsory practice for the future colonoscopist? Endoscopy 37:1198–1204. https://doi.org/10.1055/s-2005-921049
Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P (2004) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91:146–150. https://doi.org/10.1002/bjs.4407
Sakai K, Hikosaka O, Miyauchi S, Takino R, Sasaki Y, Pütz B (1998) Transition of brain activation from frontal to parietal areas in visuomotor sequence learning. J Neurosci 18:1827–1840
Honda M (1998) Dynamic cortical involvement in implicit and explicit motor sequence learning: a PET study. Brain 121:2159–2173. https://doi.org/10.1093/brain/121.11.2159
Nakamura K, Sakai K, Hikosaka O (1998) Neuronal activity in medial frontal cortex during learning of sequential procedures. J Neurophysiol 80:2671–2687
Klintsova AY, Greenough WT (1999) Synaptic plasticity in cortical systems. Curr Opin Neurobiol 9:203–208
Wolpert DM, Ghahramani Z, Flanagan JR (2001) Perspectives and problems in motor learning. Trends Cogn Sci 5:487–494. https://doi.org/10.1016/S1364-6613(00)01773-3
Serrien DJ, Ivry RB, Swinnen SP (2006) Dynamics of hemispheric specialization and integration in the context of motor control. Nat Rev Neurosci 7:160–166. https://doi.org/10.1038/nrn1849
Gagnon L, Cooper RJ, Yücel MA, Perdue KL, Greve DN, Boas DA (2012) Short separation channel location impacts the performance of short channel regression in NIRS. Neuroimage 59:2518–2528. https://doi.org/10.1016/j.neuroimage.2011.08.095
The authors would like to thank the medical student subjects and their dedication for this study. The authors would also like to thank the anatomical gift program and the gross anatomy lab at University at Buffalo for their support regarding the ex-vivo cadaveric samples. We would also like to thank Arthur “Buzz” DiMartino and his team at TechEn in graciously providing us support with the CW6 spectrometer.
This work is supported by NIBIB 1R01EB014305, NHBLI 1R01HL119248, and NCI 1R01CA197491 grants that were awarded to Suvranu De.
Drs. Arun Nemani, Uwe Kruger, Meryem Yucel, Clairice Cooper, Steven Schwaitzberg, Xavier Intes, and Suvranu De have no conflicts of interest or financial ties to disclose.
Presented at ACS Clinical Congress 2017, San Diego, CA.
About this article
Cite this article
Nemani, A., Kruger, U., Cooper, C.A. et al. Objective assessment of surgical skill transfer using non-invasive brain imaging. Surg Endosc 33, 2485–2494 (2019). https://doi.org/10.1007/s00464-018-6535-z
- Surgical skill assessment
- Surgical skill transfer
- Brain imaging
- Surgical simulators
- Surgical training
- Functional near-infrared spectroscopy