Abstract
Purpose
In brain tumor surgery, tissue shift (called brain shift) can move the surgical target and invalidate the surgical plan. A cost-effective and flexible tool, intra-operative ultrasound (iUS) with robust image registration algorithms can effectively track brain shift to ensure surgical outcomes and safety.
Methods
We proposed to employ a Siamese neural network, which was first trained using natural images and fine-tuned with domain-specific data to automatically detect matching anatomical landmarks in iUS scans at different surgical stages. An efficient 2.5D approach and an iterative re-weighted least squares algorithm are utilized to perform landmark-based registration for brain shift correction. The proposed method is validated and compared against the state-of-the-art methods using the public BITE and RESECT datasets.
Results
Registration of pre-resection iUS scans to during- and post-resection iUS images were executed. The results with the proposed method shows a significant improvement from the initial misalignment (\(p<0.001\)) and the method is comparable to the state-of-the-art methods validated on the same datasets.
Conclusions
We have proposed a robust technique to efficiently detect matching landmarks in iUS and perform brain shift correction with excellent performance. It has the potential to improve the accuracy and safety of neurosurgery.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Bucholz RD, Smith KR, Laycock KA, McDurmont LL (2001) Three-dimensional localization: from image-guided surgery to information-guided therapy. Methods 25(2):186–200
Xiao Y, Eikenes L, Reinertsen I, Rivaz H (2018) Nonlinear deformation of tractography in ultrasound-guided low-grade gliomas resection. Int J Comput Assist Radiol Surg 13(3):457–467
Rivaz H, Collins DL (2015) Near real-time robust non-rigid registration of volumetric ultrasound images for neurosurgery. Ultrasound Med Biol 41(2):574–587
Masoumi N, Xiao Y, Rivaz H (2019) Arena: Inter-modality affine registration using evolutionary strategy. Int J Comput Assist Radiol Surg 14(3):441–450
Marko NF, Weil RJ, Schroeder JL, Lang FF, Suki D, Sawaya RE (2014) Extent of resection of glioblastoma revisited: personalized survival modeling facilitates more accurate survival prediction and supports a maximum-safe-resection approach to surgery. J Clin Oncol 32(8):774
Lu X, Zhang S, Yang W, Chen Y (2010) Sift and shape information incorporated into fluid model for non-rigid registration of ultrasound images. Comput Methods Programs Biomed 100(2):123–131
Urschler M, Bauer J, Ditt H, Bischof H (2006) Sift and shape context for feature-based nonlinear registration of thoracic CT images. In: International workshop on computer vision approaches to medical image analysis. Springer, pp 73–84
Schneider RJ, Perrin DP, Vasilyev NV, Marx GR, Pedro J, Howe RD (2012) Real-time image-based rigid registration of three-dimensional ultrasound. Med Image Anal 16(2):402–414
Machado I, Toews M, Luo J, Unadkat P, Essayed W, George E, Teodoro P, Carvalho H, Martins J, Golland P et al (2018) Non-rigid registration of 3d ultrasound for neurosurgery using automatic feature detection and matching. Int J Comput Assist Radiol Surg 13(10):1525–1538
Tuysuzoglu A, Tan J, Eissa K, Kiraly AP, Diallo M, Kamen A (2018) Deep adversarial context-aware landmark detection for ultrasound imaging. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 151–158
Zhang J, Liu M, Shen D (2017) Detecting anatomical landmarks from limited medical imaging data using two-stage task-oriented deep neural networks. IEEE Trans Image Process 26(10):4753–4764
Gomariz A, Li W, Ozkan E, Tanner C, Goksel O (2019) Siamese networks with location prior for landmark tracking in liver ultrasound sequences. In: 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019). IEEE, pp 1757–1760
Grewal M, Deist TM, Wiersma J, Bosman PA, Alderliesten T (2020) An end-to-end deep learning approach for landmark detection and matching in medical images. In: Medical imaging 2020: image processing, vol 11313. International Society for Optics and Photonics, p 1131328
Pirhadi A, Rivaz H, Ahmad MO, Xiao Y (2021) Robust ultrasound-to-ultrasound registration for intra-operative brain shift correction with a Siamese neural network. In: International workshop on advances in simplifying medical ultrasound. Springer, pp 85–95
Xiao Y, Fortin M, Unsgård G, Rivaz H, Reinertsen I (2017) Re trospective evaluation of cerebral tumors (resect): A clinical database of pre-operative MRI and intra-operative ultrasound in low-grade glioma surgeries. Med Phys 44(7):3875–3882
Mercier L, Del Maestro RF, Petrecca K, Araujo D, Haegelen C, Collins DL (2012) Online database of clinical MR and ultrasound images of brain tumors. Med Phys 39(6Part1):3253–3261
Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PH (2016) Fully-convolutional Siamese networks for object tracking. In: European conference on computer vision. Springer, pp 850–865
Koch G, Zemel R, Salakhutdinov R (2015) Siamese neural networks for one-shot image recognition. In: ICML deep learning workshop. vol. 2. Lille
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105
Heinrich MP, Hansen L (2020) Highly accurate and memory efficient unsupervised learning-based discrete CT registration using 2.5D displacement search. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 190–200
Maknojia S, Tam F, Das S, Schweizer T, Graham SJ (2019) Visualization of brain shift corrected functional magnetic resonance imaging data for intraoperative brain mapping. World Neurosurg X 2:100021
Holland PW, Welsch RE (1977) Robust regression using iteratively reweighted least-squares. Commun Stat Theory Methods 6(9):813–827
Rivaz H, Boctor EM, Choti MA, Hager GD (2010) Real-time regularized ultrasound elastography. IEEE Trans Med Imaging 30(4):928–945
Yu X, Wang J, Hong QQ, Teku R, Wang SH, Zhang YD (2022) Transfer learning for medical images analyses: a survey. Neurocomputing 489:230–254
Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vision 115(3):211–252
Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J (2016) Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Med Imaging 35(5):1299–1312
Canalini L, Klein J, Miller D, Kikinis R (2020) Enhanced registration of ultrasound volumes by segmentation of resection cavity in neurosurgical procedures. Int J Comput Assist Radiol Surg 15(12):1963–1974
Luo J, Ma G, Frisken S, Juvekar P, Haouchine N, Xu Z, Xiao Y, Golby A, Codd P, Sugiyama M, et al. (2020) Do public datasets assure unbiased comparisons for registration evaluation? arXiv preprint arXiv:2003.09483
Acknowledgements
Funding was provided by Natural Science and Engineering Research Council of Canada
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
A.Pirhadi, H. Rivaz, M.O. Ahmad, and Y.Xiao declare no conflict of interest. The study has been approved and performed in accordance with ethical standards. This work was supported by NSERC.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Pirhadi, A., Salari, S., Ahmad, M.O. et al. Robust landmark-based brain shift correction with a Siamese neural network in ultrasound-guided brain tumor resection. Int J CARS 18, 501–508 (2023). https://doi.org/10.1007/s11548-022-02770-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-022-02770-5