Abstract
In brain tumor resection, soft tissue shift (called brain shift) can displace the surgical target and render the surgical plan invalid. Intra-operative ultrasound (iUS) with robust image registration algorithms can effectively correct brain shift to ensure quality of resection and patient safety. Herein, we proposed a novel technique to automatically align iUS scans acquired before and after tumor resection, in order to confirm removal of cancerous tissues while minimizing resection of healthy tissue. More specifically, we employed a Siamese network to locate matching anatomical landmarks within iUS scans. Selected landmarks were used to search for the best affine transformation to align iUS obtained at different surgical stages. The proposed method was validated with the publicly available REtroSpective Evaluation of Cerebral Tumors (RESECT) database. After image alignment, the mean target registration error (mTRE) was effectively reduced from 3.55 ± 1.76 mm to 1.26 ± 0.48 mm in before and after resection and from 3.49 ± 1.56 mm to 1.16 ± 0.49 mm in before and during resection. In general, the results are comparable to the state-of-the-art techniques, validated on the same database, and our technique demonstrated excellent performance in iUS-based brain shift correction for optimal therapeutic outcomes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Unsgaard, G., et al.: Intra-operative 3d ultrasound in neurosurgery. Acta Neurochir. 148(3), 235–253 (2006)
Xiao, Y., Eikenes, L., Reinertsen, I., Rivaz, H.: Nonlinear deformation of tractography in ultrasound-guided low-grade gliomas resection. Int. J. Comput. Assist. Radiol. Surg. 13(3), 457–467 (2018)
Wein, W.: Brain-shift correction with image-based registration and landmark accuracy evaluation. In: Stoyanov, D., et al. (eds.) POCUS/BIVPCS/CuRIOUS/CPM 2018. LNCS, vol. 11042, pp. 146–151. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01045-4_17
Heinrich, M.P.: Intra-operative ultrasound to MRI fusion with a public multimodal discrete registration tool. In: Stoyanov, D., et al. (eds.) POCUS/BIVPCS/CuRIOUS/CPM 2018. LNCS, vol. 11042, pp. 159–164. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01045-4_19
Masoumi, N., Xiao, Y., Rivaz, H.: ARENA: inter-modality affine registration using evolutionary strategy. Int. J. Comput. Assist. Radiol. Surg. 14(3), 441–450 (2019)
Marko, N.F., Weil, R.J., Schroeder, J.L., Lang, F.F., Suki, D., Sawaya, R.E.: Extent of resection of glioblastoma revisited: personalized survival modeling facilitates more accurate survival prediction and supports a maximum-safe-resection approach to surgery. J. Clin. Oncol. 32(8), 774 (2014)
Xiao, Y., et al.: Evaluation of MRI to ultrasound registration methods for brain shift correction: the curious2018 challenge. IEEE Trans. Med. Imaging 39(3), 777–786 (2020)
Lu, X., Zhang, S., Yang, W., Chen, Y.: Sift and shape information incorporated into fluid model for non-rigid registration of ultrasound images. Comput. Methods Programs Biomed. 100(2), 123–131 (2010)
Urschler, M., Bauer, J., Ditt, H., Bischof, H.: SIFT and shape context for feature-based nonlinear registration of thoracic CT images. In: Beichel, R.R., Sonka, M. (eds.) CVAMIA 2006. LNCS, vol. 4241, pp. 73–84. Springer, Heidelberg (2006). https://doi.org/10.1007/11889762_7
Machado, I., et al.: Non-rigid registration of 3d ultrasound for neurosurgery using automatic feature detection and matching. Int. J. Comput. Assist. Radiol. Surg. 13(10), 1525–1538 (2018)
Canalini, L., Klein, J., Miller, D., Kikinis, R.: Enhanced registration of ultrasound volumes by segmentation of resection cavity in neurosurgical procedures. Int. J. Comput. Assist. Radiol. Surg. 15(12), 1963–1974 (2020). https://doi.org/10.1007/s11548-020-02273-1
Guo, Q., Feng, W., Zhou, C., Huang, R., Wan, L., Wang, S.: Learning dynamic Siamese network for visual object tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1763–1771 (2017)
He, A., Luo, C., Tian, X., Zeng, W.: A twofold Siamese network for real-time object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4834–4843 (2018)
Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., Torr, P.H.S.: Fully-convolutional Siamese networks for object tracking. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 850–865. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48881-3_56
Gomariz, A., Li, W., Ozkan, E., Tanner, C., Goksel, O.: Siamese networks with location prior for landmark tracking in liver ultrasound sequences. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 1757–1760. IEEE (2019)
Koch, G., Zemel, R., Salakhutdinov, R.: Siamese neural networks for one-shot image recognition. In: ICML Deep Learning Workshop, vol. 2. Lille (2015)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Adv. Neural. Inf. Process. Syst. 25, 1097–1105 (2012)
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015)
Pytorch-SiamFC. https://github.com/rafellerc/Pytorch-SiamFC. Accessed 29 June 2021
Xiao, Y., Fortin, M., Unsgård, G., Rivaz, H., Reinertsen, I.: Retrospective evaluation of cerebral tumors (RESECT): a clinical database of pre-operative MRI and intra-operative ultrasound in low-grade glioma surgeries. Med. Phys. 44(7), 3875–3882 (2017)
Heinrich, M.P., Hansen, L.: Highly accurate and memory efficient unsupervised learning-based discrete CT registration using 2.5D displacement search. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12263, pp. 190–200. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59716-0_19
Holland, P.W., Welsch, R.E.: Robust regression using iteratively reweighted least-squares. Commun. Stat. Theory Methods 6(9), 813–827 (1977)
Rivaz, H., Boctor, E.M., Choti, M.A., Hager, G.D.: Real-time regularized ultrasound elastography. IEEE Trans. Med. Imaging 30(4), 928–945 (2010)
Canalini, L., Klein, J., Miller, D., Kikinis, R.: Segmentation-based registration of ultrasound volumes for glioma resection in image-guided neurosurgery. Int. J. Comput. Assist. Radiol. Surg. 14(10), 1697–1713 (2019). https://doi.org/10.1007/s11548-019-02045-6
Luo, J., et al.: Do public datasets assure unbiased comparisons for registration evaluation? arXiv preprint arXiv:2003.09483 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Pirhadi, A., Rivaz, H., Ahmad, M.O., Xiao, Y. (2021). Robust Ultrasound-to-Ultrasound Registration for Intra-operative Brain Shift Correction with a Siamese Neural Network. In: Noble, J.A., Aylward, S., Grimwood, A., Min, Z., Lee, SL., Hu, Y. (eds) Simplifying Medical Ultrasound. ASMUS 2021. Lecture Notes in Computer Science(), vol 12967. Springer, Cham. https://doi.org/10.1007/978-3-030-87583-1_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-87583-1_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87582-4
Online ISBN: 978-3-030-87583-1
eBook Packages: Computer ScienceComputer Science (R0)
