Skip to main content

Advertisement

SpringerLink
Log in

Neurosurgical skills analysis by machine learning models: systematic review

  • Review
  • Published:
Neurosurgical Review Aims and scope Submit manuscript

Abstract

Machine learning (ML) models are being actively used in modern medicine, including neurosurgery. This study aimed to summarize the current applications of ML in the analysis and assessment of neurosurgical skills. We conducted this systematic review in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. We searched the PubMed and Google Scholar databases for eligible studies published until November 15, 2022, and used the Medical Education Research Study Quality Instrument (MERSQI) to assess the quality of the included articles. Of the 261 studies identified, we included 17 in the final analysis. Studies were most commonly related to oncological, spinal, and vascular neurosurgery using microsurgical and endoscopic techniques. Machine learning-evaluated tasks included subpial brain tumor resection, anterior cervical discectomy and fusion, hemostasis of the lacerated internal carotid artery, brain vessel dissection and suturing, glove microsuturing, lumbar hemilaminectomy, and bone drilling. The data sources included files extracted from VR simulators and microscopic and endoscopic videos. The ML application was aimed at classifying participants into several expertise levels, analysis of differences between experts and novices, surgical instrument recognition, division of operation into phases, and prediction of blood loss. In two articles, ML models were compared with those of human experts. The machines outperformed humans in all tasks. The most popular algorithms used to classify surgeons by skill level were the support vector machine and k-nearest neighbors, and their accuracy exceeded 90%. The “you only look once” detector and RetinaNet usually solved the problem of detecting surgical instruments — their accuracy was approximately 70%. The experts differed by more confident contact with tissues, higher bimanuality, smaller distance between the instrument tips, and relaxed and focused state of the mind. The average MERSQI score was 13.9 (from 18). There is growing interest in the use of ML in neurosurgical training. Most studies have focused on the evaluation of microsurgical skills in oncological neurosurgery and on the use of virtual simulators; however, other subspecialties, skills, and simulators are being investigated. Machine learning models effectively solve different neurosurgical tasks related to skill classification, object detection, and outcome prediction. Properly trained ML models outperform human efficacy. Further research on ML application in neurosurgery is needed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Availability of data and materials

Not applicable.

References

  1. Sharma R, Suri A (2022) Microsurgical suturing assessment scores: a systematic review. Neurosurg Rev 45:119–124. https://doi.org/10.1007/s10143-021-01569-3

    Article  PubMed  Google Scholar 

  2. Stogowski P, Fliciński F, Białek J, Dąbrowski F, Piotrowski M, Mazurek T (2021) Microsurgical Anastomosis Rating Scale (MARS10): a final product scoring system for initial microsurgical training. Plast Surg (Oakville, Ont) 29:243–249. https://doi.org/10.1177/2292550320969649

    Article  Google Scholar 

  3. Manjul S, Bettag M, Roy TS, Lalwani SAT (2016) Simulation based skills training in neurosurgery and contemporary surgical practices. Ann Natl Acad Med Sci 52:56–75. https://doi.org/10.1055/s-0040-1712607

    Article  Google Scholar 

  4. Aghazadeh MA, Jayaratna IS, Hung AJ, Pan MM, Desai MM, Gill IS, Goh AC (2015) External validation of Global Evaluative Assessment of Robotic Skills (GEARS). Surg Endosc 29:3261–3266. https://doi.org/10.1007/s00464-015-4070-8

    Article  PubMed  Google Scholar 

  5. Liu M, Purohit S, Mazanetz J, Allen W, Kreaden US, Curet M (2018) Assessment of Robotic Console Skills (ARCS): construct validity of a novel global rating scale for technical skills in robotically assisted surgery. Surg Endosc 32:526–535. https://doi.org/10.1007/s00464-017-5694-7

    Article  PubMed  Google Scholar 

  6. Wagner MW, Namdar K, Biswas A, Monah S, Khalvati F, Ertl-Wagner BB (2021) Radiomics, machine learning, and artificial intelligence-what the neuroradiologist needs to know. Neuroradiology 63:1957–1967. https://doi.org/10.1007/s00234-021-02813-9

    Article  PubMed  PubMed Central  Google Scholar 

  7. Javidan AP, Li A, Lee MH, Forbes TL, Naji F (2022) A systematic review and bibliometric analysis of applications of artificial intelligence and machine learning in vascular surgery. Ann Vasc Surg 85:395–405. https://doi.org/10.1016/j.avsg.2022.03.019

    Article  PubMed  Google Scholar 

  8. Lalehzarian SP, Gowd AK, Liu JN (2021) Machine learning in orthopaedic surgery. World J Orthop 12:685–699. https://doi.org/10.5312/wjo.v12.i9.685

    Article  PubMed  PubMed Central  Google Scholar 

  9. Sakamoto T, Goto T, Fujiogi M, Kawarai Lefor A (2022) Machine learning in gastrointestinal surgery. Surg Today 52:995–1007. https://doi.org/10.1007/s00595-021-02380-9

    Article  PubMed  CAS  Google Scholar 

  10. Kaka H, Zhang E, Khan N (2021) Artificial intelligence and deep learning in neuroradiology: exploring the new frontier. Can Assoc Radiol J = J l’Association Can des Radiol 72:35–44. https://doi.org/10.1177/0846537120954293

    Article  Google Scholar 

  11. Witten AJ, Patel N, Cohen-Gadol A (2022) Image segmentation of operative neuroanatomy into tissue categories using a machine learning construct and its role in neurosurgical training. Oper Neurosurg (Hagerstown, Md) 23:279–286. https://doi.org/10.1227/ons.0000000000000322

    Article  Google Scholar 

  12. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 339:b2700. https://doi.org/10.1136/bmj.b2700

    Article  PubMed  PubMed Central  Google Scholar 

  13. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, Henry DA (2017) AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 358:j4008. https://doi.org/10.1136/bmj.j4008

    Article  PubMed  PubMed Central  Google Scholar 

  14. Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH (2017) Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev 6:245. https://doi.org/10.1186/s13643-017-0644-y

    Article  PubMed  PubMed Central  Google Scholar 

  15. Cook DA, Reed DA (2015) Appraising the quality of medical education research methods: the medical education research study quality instrument and the Newcastle-Ottawa scale-education. Acad Med 90:1067–1076. https://doi.org/10.1097/ACM.0000000000000786

    Article  PubMed  Google Scholar 

  16. Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM (2007) Association between funding and quality of published medical education research. JAMA 298:1002–1009. https://doi.org/10.1001/jama.298.9.1002

    Article  PubMed  CAS  Google Scholar 

  17. Kumar S (2019) Evidence in surgery—levels and significance. Indian J Surg 81:307–316. https://doi.org/10.1007/s12262-019-01939-8

    Article  Google Scholar 

  18. Bissonnette V, Mirchi N, Ledwos N, Alsidieri G, Winkler-Schwartz A, Del Maestro RF (2019) Artificial intelligence distinguishes surgical training levels in a virtual reality spinal task. J Bone Joint Surg Am 101:e127. https://doi.org/10.2106/JBJS.18.01197

    Article  PubMed  Google Scholar 

  19. Winkler-Schwartz A, Yilmaz R, Mirchi N, Bissonnette V, Ledwos N, Siyar S, Azarnoush H, Karlik B, Del Maestro R (2019) Machine learning identification of surgical and operative factors associated with surgical expertise in virtual reality simulation. JAMA Netw open 2:e198363. https://doi.org/10.1001/jamanetworkopen.2019.8363

    Article  PubMed  Google Scholar 

  20. Mirchi N, Bissonnette V, Ledwos N, Winkler-Schwartz A, Yilmaz R, Karlik B, Del Maestro RF (2020) Artificial neural networks to assess virtual reality anterior cervical discectomy performance. Oper Neurosurg 19:65–75

    Article  Google Scholar 

  21. Mirchi N, Bissonnette V, Yilmaz R, Ledwos N, Winkler-Schwartz A, Del Maestro RF (2020) The virtual operative assistant: an explainable artificial intelligence tool for simulation-based training in surgery and medicine. PLoS One 15:e0229596. https://doi.org/10.1371/journal.pone.0229596

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  22. Siyar S, Azarnoush H, Rashidi S, Winkler-Schwartz A, Bissonnette V, Ponnudurai N, Del Maestro RF (2020) Machine learning distinguishes neurosurgical skill levels in a virtual reality tumor resection task. Med Biol Eng Comput 58:1357–1367. https://doi.org/10.1007/s11517-020-02155-3

    Article  PubMed  Google Scholar 

  23. Alkadri S, Ledwos N, Mirchi N, Reich A, Yilmaz R, Driscoll M, Del Maestro RF (2021) Utilizing a multilayer perceptron artificial neural network to assess a virtual reality surgical procedure. Comput Biol Med 136:104770. https://doi.org/10.1016/j.compbiomed.2021.104770

    Article  PubMed  Google Scholar 

  24. Davids J, Makariou S-G, Ashrafian H, Darzi A, Marcus HJ, Giannarou S (2021) Automated vision-based microsurgical skill analysis in neurosurgery using deep learning: development and preclinical validation. World Neurosurg 149:e669–e686. https://doi.org/10.1016/j.wneu.2021.01.117

    Article  PubMed  Google Scholar 

  25. Khan DZ, Luengo I, Barbarisi S, Addis C, Culshaw L, Dorward NL, Haikka P, Jain A, Kerr K, Koh CH (2021) Automated operative workflow analysis of endoscopic pituitary surgery using machine learning: development and preclinical evaluation (IDEAL stage 0). J Neurosurg 1:1–8

    Google Scholar 

  26. Fazlollahi AM, Bakhaidar M, Alsayegh A, Yilmaz R, Winkler-Schwartz A, Mirchi N, Langleben I, Ledwos N, Sabbagh AJ, Bajunaid K, Harley JM, Del Maestro RF (2022) Effect of artificial intelligence tutoring vs expert instruction on learning simulated surgical skills among medical students: a randomized clinical trial. JAMA Netw open 5:e2149008. https://doi.org/10.1001/jamanetworkopen.2021.49008

    Article  PubMed  PubMed Central  Google Scholar 

  27. Koskinen J, Torkamani-Azar M, Hussein A, Huotarinen A, Bednarik R (2022) Automated tool detection with deep learning for monitoring kinematics and eye-hand coordination in microsurgery. Comput Biol Med 141:105121. https://doi.org/10.1016/j.compbiomed.2021.105121

    Article  PubMed  Google Scholar 

  28. Ledwos N, Mirchi N, Yilmaz R, Winkler-Schwartz A, Sawni A, Fazlollahi AM, Bissonnette V, Bajunaid K, Sabbagh AJ, Del Maestro RF (2022) Assessment of learning curves on a simulated neurosurgical task using metrics selected by artificial intelligence. J Neurosurg 1–12. https://doi.org/10.3171/2021.12.JNS211563

  29. Natheir S, Christie S, Yilmaz R, Winkler-Schwartz A, Bajunaid K, Sabbagh AJ, Werthner P, Fares J, Azarnoush H, Del Maestro R (2023) Utilizing artificial intelligence and electroencephalography to assess expertise on a simulated neurosurgical task. Comput Biol Med 152:106286. https://doi.org/10.1016/j.compbiomed.2022.106286

    Article  PubMed  Google Scholar 

  30. Pangal DJ, Kugener G, Zhu Y, Sinha A, Unadkat V, Cote DJ, Strickland B, Rutkowski M, Hung A, Anandkumar A, Han XY, Papyan V, Wrobel B, Zada G, Donoho DA (2022) Expert surgeons and deep learning models can predict the outcome of surgical hemorrhage from 1 min of video. Sci Rep 12:8137. https://doi.org/10.1038/s41598-022-11549-2

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  31. Reich A, Mirchi N, Yilmaz R, Ledwos N, Bissonnette V, Tran DH, Winkler-Schwartz A, Karlik B, Del Maestro RF (2022) Artificial neural network approach to competency-based training using a virtual reality neurosurgical simulation. Oper Neurosurg (Hagerstown, Md) 23:31–39. https://doi.org/10.1227/ons.0000000000000173

    Article  Google Scholar 

  32. Unadkat V, Pangal DJ, Kugener G, Roshannai A, Chan J, Zhu Y, Markarian N, Zada G, Donoho DA (2022) Code-free machine learning for object detection in surgical video: a benchmarking, feasibility, and cost study. Neurosurg Focus 52:E11

    Article  PubMed  Google Scholar 

  33. Yilmaz R, Winkler-Schwartz A, Mirchi N, Reich A, Christie S, Tran DH, Ledwos N, Fazlollahi AM, Santaguida C, Sabbagh AJ, Bajunaid K, Del Maestro R (2022) Continuous monitoring of surgical bimanual expertise using deep neural networks in virtual reality simulation. NPJ Digit Med 5:54. https://doi.org/10.1038/s41746-022-00596-8

    Article  PubMed  PubMed Central  Google Scholar 

  34. Kugener G, Pangal DJ, Cardinal T, Collet C, Lechtholz-Zey E, Lasky S, Sundaram S, Markarian N, Zhu Y, Roshannai A (2022) Utility of the simulated outcomes following carotid artery laceration video data set for machine learning applications. JAMA Netw open 5:e223177–e223177

    Article  PubMed  PubMed Central  Google Scholar 

  35. Bhandari M, Zeffiro T, Reddiboina M (2020) Artificial intelligence and robotic surgery: current perspective and future directions. Curr Opin Urol 30:48–54. https://doi.org/10.1097/MOU.0000000000000692

    Article  PubMed  Google Scholar 

  36. Howard J (2019) Artificial intelligence: implications for the future of work. Am J Ind Med 62:917–926. https://doi.org/10.1002/ajim.23037

    Article  PubMed  Google Scholar 

  37. Turing AM (1950) I.—Computing machinery and intelligence. Mind LIX:433–460. Springer. https://doi.org/10.1093/mind/LIX.236.433

    Book  Google Scholar 

  38. McCarthy J, Minsky ML, Rochester N, Shannon CE (2006) A proposal for the dartmouth summer research project on artificial intelligence, august 31, 1955. AI Mag 27:12

    Google Scholar 

  39. ROSENBLATT F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65:386–408. doi: https://doi.org/10.1037/h0042519

  40. Samuel AL (1988) In: Levy DNL (ed) Some studies in machine learning using the game of checkers. II—Recent Progress BT - Computer Games I. Springer New York, New York, NY, pp 366–400

    Google Scholar 

  41. Schilling AT, Shah PP, Feghali J, Jimenez AE, Azad TD (2022) A brief history of machine learning in neurosurgery. Acta Neurochir Suppl 134:245–250. https://doi.org/10.1007/978-3-030-85292-4_27

    Article  PubMed  Google Scholar 

  42. Crossnohere NL, Elsaid M, Paskett J, Bose-Brill S, Bridges JFP (2022) Guidelines for artificial intelligence in medicine: literature review and content analysis of frameworks. J Med Internet Res 24:e36823. https://doi.org/10.2196/36823

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

This work was supported by the Russian Science Foundation grant № 22-75-10117.

Author information

Authors and Affiliations

  1. Burdenko Neurosurgery Center, Moscow, Russia

    Oleg Titov, Andrey Bykanov & David Pitskhelauri

  2. OPEN BRAIN, Laboratory of Neurosurgical Innovations, Moscow, Russia

    Oleg Titov

Authors
  1. Oleg Titov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrey Bykanov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Pitskhelauri
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

O.T.: study design, material preparation, data collection and analysis, and manuscript writing

A.B.: study design, data collection and analysis, and manuscript writing

D.P.: study design, data analysis, and general supervision

All authors read and approved the final manuscript.

Corresponding author

Correspondence to Oleg Titov.

Ethics declarations

Ethical approval

This is a review study. No ethical approval is required.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

No portion of the paper has been presented or published previously.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Titov, O., Bykanov, A. & Pitskhelauri, D. Neurosurgical skills analysis by machine learning models: systematic review. Neurosurg Rev 46, 121 (2023). https://doi.org/10.1007/s10143-023-02028-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10143-023-02028-x

Keywords

Search

Navigation