Abstract
Advanced minimally invasive neurosurgery navigation relies mainly on Magnetic Resonance Imaging (MRI) guidance. MRI guidance, however, only provides pre-operative information in the majority of the cases. Once the surgery begins, the value of this guidance diminishes to some extent because of the anatomical changes due to surgery. Guidance with live image feedback coming directly from the surgical device, e.g., endoscope, can complement MRI-based navigation or be an alternative if MRI guidance is not feasible. With this motivation, we present a method for live image-only guidance leveraging a large data set of annotated neurosurgical videos. First, we report the performance of a deep learning-based object detection method, YOLO, on detecting anatomical structures in neurosurgical images. Second, we present a method for generating neurosurgical roadmaps using unsupervised embedding without assuming exact anatomical matches between patients, presence of an extensive anatomical atlas, or the need for simultaneous localization and mapping. A generated roadmap encodes the common anatomical paths taken in surgeries in the training set. At inference, the roadmap can be used to map a surgeon’s current location using live image feedback on the path to provide guidance by being able to predict which structures should appear going forward or backward, much like a mapping application. Even though the embedding is not supervised by position information, we show that it is correlated to the location inside the brain and on the surgical path. We trained and evaluated the proposed method with a data set of 166 transsphenoidal adenomectomy procedures.
Partially supported by the EANS 2021 Leica Research Grant.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Berkmann, S., Schlaffer, S., Nimsky, C., Fahlbusch, R., Buchfelder, M.: Intraoperative high-field MRI for transsphenoidal reoperations of nonfunctioning pituitary adenoma. J. Neurosurg. 121(5), 1166–1175 (2014)
Burkhardt, J.K., et al.: High-frequency intra-operative ultrasound-guided surgery of superficial intra-cerebral lesions via a single-burr-hole approach. Ultrasound Med. Biol. 40(7), 1469–1475 (2014)
De Witt Hamer, P.C., Robles, S.G., Zwinderman, A.H., Duffau, H., Berger, M.S.: Impact of intraoperative stimulation brain mapping on glioma surgery outcome: a meta-analysis. J. Clin. Oncol.: Official J. Am. Soc. Clin. Oncol. 30(20), 2559–2565 (2012)
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., Muller, P.-A.: Evaluating surgical skills from kinematic data using convolutional neural networks. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 214–221. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_25
Grasa, O.G., Bernal, E., Casado, S., Gil, I., Montiel, J.M.: Visual SLAM for handheld monocular endoscope. IEEE Trans. Med. Imaging 33(1), 135–146 (2014)
Grasa, O.G., Civera, J., Montiel, J.M.: EKF monocular SLAM with relocalization for laparoscopic sequences. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 4816–4821 (2011)
Hadjipanayis, C.G., Widhalm, G., Stummer, W.: What is the surgical benefit of utilizing 5-aminolevulinic acid for fluorescence-guided surgery of malignant gliomas? Neurosurgery 77(5), 663–673 (2015)
Härtl, R., Lam, K.S., Wang, J., Korge, A., Kandziora, F., Audigé, L.: Worldwide survey on the use of navigation in spine surgery. World Neurosurg. 79(1), 162–172 (2013)
Hervey-Jumper, S.L., et al.: Awake craniotomy to maximize glioma resection: methods and technical nuances over a 27-year period. J. Neurosurg. 123(2), 325–339 (2015)
Iversen, D.H., Wein, W., Lindseth, F., Unsgård, G., Reinertsen, I.: Automatic intraoperative correction of brain shift for accurate neuronavigation. World Neurosurg. 120, e1071–e1078 (2018)
Leonard, S., et al.: Evaluation and stability analysis of video-based navigation system for functional endoscopic sinus surgery on in vivo clinical data. IEEE Trans. Med. Imaging 37(10), 2185–2195 (2018)
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: 7th International Conference on Learning Representations, ICLR 2019 (2017)
Luongo, F., Hakim, R., Nguyen, J.H., Anandkumar, A., Hung, A.J.: Deep learning-based computer vision to recognize and classify suturing gestures in robot-assisted surgery. Surgery 169(5), 1240–1244 (2021)
Mahmoud, N., et al.: ORBSLAM-based endoscope tracking and 3D reconstruction. In: Peters, T., et al. (eds.) CARE 2016. LNCS, vol. 10170, pp. 72–83. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54057-3_7
Orringer, D.A., Golby, A., Jolesz, F.: Neuronavigation in the surgical management of brain tumors: current and future trends. Expert Rev. Med. Dev. 9(5), 491–500 (2012)
Ozyoruk, K.B., et al.: EndoSLAM dataset and an unsupervised monocular visual odometry and depth estimation approach for endoscopic videos. Med. Image Anal. 71, 102058 (2021)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2015)
Rivas-Blanco, I., Perez-Del-Pulgar, C.J., Garcia-Morales, I., Munoz, V.F., Rivas-Blanco, I.: A review on deep learning in minimally invasive surgery. IEEE Access 9, 48658–48678 (2021)
Sanai, N., Mirzadeh, Z., Berger, M.S.: Functional outcome after language mapping for glioma resection. N. Engl. J. Med. 358(1), 18–27 (2008)
Sarikaya, D., Corso, J.J., Guru, K.A.: Detection and localization of robotic tools in robot-assisted surgery videos using deep neural networks for region proposal and detection. IEEE Trans. Med. Imaging 36(7), 1542–1549 (2017)
Staartjes, V.E., et al.: Machine learning in neurosurgery: a global survey. Acta Neurochir. 162(12), 3081–3091 (2020)
Staartjes, V.E., Volokitin, A., Regli, L., Konukoglu, E., Serra, C.: Machine vision for real-time intraoperative anatomic guidance: a proof-of-concept study in endoscopic pituitary surgery. Oper. Neurosurg. (Hagerstown, Md.) 21(4), 242–247 (2021)
Stienen, M.N., Fierstra, J., Pangalu, A., Regli, L., Bozinov, O.: The Zurich checklist for safety in the intraoperative magnetic resonance imaging suite: technical note. Oper. Neurosurg. (Hagerstown, Md.) 16(6), 756–765 (2019)
Stummer, W., Stepp, H., Wiestler, O.D., Pichlmeier, U.: Randomized, prospective double-blinded study comparing 3 different doses of 5-aminolevulinic acid for fluorescence-guided resections of malignant gliomas. Neurosurgery 81(2), 230–239 (2017)
Thananjeyan, B., Garg, A., Krishnan, S., Chen, C., Miller, L., Goldberg, K.: Multilateral surgical pattern cutting in 2D orthotropic gauze with deep reinforcement learning policies for tensioning (2017)
Ulrich, N.H., Burkhardt, J.K., Serra, C., Bernays, R.L., Bozinov, O.: Resection of pediatric intracerebral tumors with the aid of intraoperative real-time 3-D ultrasound. Child’s Nervous Syst.: ChNS: Official J. Int. Soc. Pediatr. Neurosurg. 28(1), 101–109 (2012)
Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors (2022)
Wang, S., Raju, A., Huang, J.: Deep learning based multi-label classification for surgical tool presence detection in laparoscopic videos, pp. 620–623 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sarwin, G. et al. (2023). Live Image-Based Neurosurgical Guidance and Roadmap Generation Using Unsupervised Embedding. In: Frangi, A., de Bruijne, M., Wassermann, D., Navab, N. (eds) Information Processing in Medical Imaging. IPMI 2023. Lecture Notes in Computer Science, vol 13939. Springer, Cham. https://doi.org/10.1007/978-3-031-34048-2_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-34048-2_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-34047-5
Online ISBN: 978-3-031-34048-2
eBook Packages: Computer ScienceComputer Science (R0)