Skip to main content

Advertisement

SpringerLink
Log in

Real-time vessel segmentation and reconstruction for virtual fixtures for an active handheld microneurosurgical instrument

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Complications related to vascular damage such as intra-operative bleeding may be avoided during neurosurgical procedures such as petroclival meningioma surgery. To address this and improve the patient’s safety, we designed a real-time blood vessel avoidance strategy that enables operation on deformable tissue during petroclival meningioma surgery using Micron, a handheld surgical robotic tool.

Methods

We integrated real-time intra-operative blood vessel segmentation of brain vasculature using deep learning, with a 3D reconstruction algorithm to obtain the vessel point cloud in real time. We then implemented a virtual-fixture-based strategy that prevented Micron’s tooltip from entering a forbidden region around the vessel, thus avoiding damage to it.

Results

We achieved a median Dice similarity coefficient of 0.97, 0.86, 0.87 and 0.77 on datasets of phantom blood vessels, petrosal vein, internal carotid artery and superficial vessels, respectively. We conducted trials with deformable clay vessel phantoms, keeping the forbidden region 400 \(\mu \)m outside and 400 \(\mu \)m inside the vessel. Micron’s tip entered the forbidden region with a median penetration of just 8.84 \(\mu \)m and 9.63 \(\mu \)m, compared to 148.74 \(\mu \)m and 117.17 \(\mu \)m without our strategy, for the former and latter trials, respectively.

Conclusion

Real-time control of Micron was achieved at 33.3 fps. We achieved improvements in real-time segmentation of brain vasculature from intra-operative images and showed that our approach works even on non-stationary vessel phantoms. The results suggest that by enabling precise, real-time control, we are one step closer to using Micron in real neurosurgical procedures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Availability of data and material

Data are available at https://drive.google.com/drive/folders/1zEoZAq3yviJ6XRumxCUr-RX4bNZz89Zq?usp=sharing.

Code Availability Statement

Code is available at https://github.com/aravindvenu7/MicronProject.

References

  1. Al-Mefty O, Fox JL Sr, Smith RR (1988) Petrosal approach for petroclival meningiomas. Neurosurgery 22(3):510–517

    Article  CAS  Google Scholar 

  2. Attia M, Hossny M, Nahavandi S, Asadi H (2017) Surgical tool segmentation using a hybrid deep cnn-rnn auto encoder-decoder. In: 2017 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, pp 3373–3378

  3. Ghosh T, Li L, Chakareski J (2018) Effective deep learning for semantic segmentation based bleeding zone detection in capsule endoscopy images. In: 2018 25th IEEE international conference on image processing (ICIP). IEEE, pp 3034–3038

  4. Gonenc B, Balicki MA, Gehlbach P, Riviere CN, Taylor RH, Iordachita I (2012) Preliminary evaluation of a micro-force sensing handheld robot for vitreoretinal surgery. In: 2012 IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 4125–4130

  5. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: Bengio Y, LeCun Y (eds) 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015. Conference Track Proceedings (2015)

  6. Lang J (2012) Clinical anatomy of the head: neurocranium\(\cdot \) Orbit\(\cdot \) Craniocervical Regions. Springer Science & Business Media

  7. MacLachlan RA, Becker BC, Tabarés JC, Podnar GW, Lobes LA Jr, Riviere CN (2011) Micron: an actively stabilized handheld tool for microsurgery. IEEE Trans Robotics 28(1):195–212

    Article  Google Scholar 

  8. MacLachlan RA, Riviere CN (2008) High-speed microscale optical tracking using digital frequency-domain multiplexing. IEEE Trans Instrum Measur 58(6):1991–2001

    Article  Google Scholar 

  9. Moccia S, Foti S, Routray A, Prudente F, Perin A, Sekula R, Mattos L, Balzer J, Fellows-Mayle W, De EM, Riviere C (2018) Toward improving safety in neurosurgery with an active handheld instrument. Ann Biomed Eng 46(10):1450–1464

    Article  Google Scholar 

  10. Moccia S, Romeo L, Migliorelli L, Frontoni E, Zingaretti P (2020) Supervised cnn strategies for optical image segmentation and classification in interventional medicine. In: Deep learners and deep learner descriptors for medical applications. Springer, pp 213–236

  11. Niu PP, Yu Y, Zhou HW, Liu Y, Luo Y, Guo ZN, Jin H, Yang Y (2016) Vessel wall differences between middle cerebral artery and basilar artery plaque s on magnetic resonance imaging. Sci Rep 6(1):1–7

    Article  CAS  Google Scholar 

  12. Poudel RP, Bonde U, Liwicki S, Zach C (2018) Contextnet: exploring context and detail for semantic segmentation in real-time. In: 29th British machine vision conference

  13. Poudel RP, Liwicki S, Cipolla R (2019) Fast-scnn: fast semantic segmentation network. In: 30th British machine vision conference

  14. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

  15. Sabaté S, Mases A, Guilera N, Canet J, Castillo J, Orrego C, Sabaté A, Fita G, Parramón F, Paniagua P, Rodriguez A (2011) Incidence and predictors of major perioperative adverse cardiac and cerebrovascular events in non-cardiac surgery. Br J Anaes 107(6):879–890

    Article  Google Scholar 

  16. Zhao H, Qi X, Shen X, Shi J, Jia J (2018) Icnet for real-time semantic segmentation on high-resolution images. In: Proceedings of the European conference on computer vision (ECCV), pp 405–420

Download references

Funding

This work was partially funded by the US National Institutes of Health (grant nos. R01EB000526 and R01EB024564).

Author information

Authors and Affiliations

  1. Department of Computer Science, BITS Pilani, Pilani, Goa, India

    Aravind Venugopal

  2. The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy

    Sara Moccia

  3. Department of Excellence in Robotics and AI, Scuola Superiore Sant’Anna, Pisa, Italy

    Sara Moccia

  4. Department of Computer Science, University College London, London, UK

    Simone Foti

  5. Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA

    Arpita Routray, Robert A. MacLachlan & Cameron N. Riviere

  6. Istituto Neurologico Carlo Besta, Milan, Italy

    Alessandro Perin

  7. Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy

    Leonardo S. Mattos

  8. Allegheny Health Network, Pittsburgh, PA, USA

    Alexander K. Yu & Jody Leonardo

  9. Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy

    Elena De Momi

Authors
  1. Aravind Venugopal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sara Moccia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Simone Foti
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Arpita Routray
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Robert A. MacLachlan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alessandro Perin
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Leonardo S. Mattos
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Alexander K. Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jody Leonardo
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Elena De Momi
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Cameron N. Riviere
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cameron N. Riviere.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Ethics approval

The collection of data was in accordance with the 1964 Declaration of Helsinki, revised in 2000.

Consent for publication

Written consent was obtained from all authors included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Venugopal, A., Moccia, S., Foti, S. et al. Real-time vessel segmentation and reconstruction for virtual fixtures for an active handheld microneurosurgical instrument. Int J CARS 17, 1069–1077 (2022). https://doi.org/10.1007/s11548-022-02584-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-022-02584-5

Keywords

Search

Navigation