The brain undergoes significant structural change over the course of neurosurgery, including highly nonlinear deformation and resection. It can be informative to recover the spatial mapping between structures identified in preoperative surgical planning and the intraoperative state of the brain. We present a novel feature-based method for achieving robust, fully automatic deformable registration of intraoperative neurosurgical ultrasound images.
A sparse set of local image feature correspondences is first estimated between ultrasound image pairs, after which rigid, affine and thin-plate spline models are used to estimate dense mappings throughout the image. Correspondences are derived from 3D features, distinctive generic image patterns that are automatically extracted from 3D ultrasound images and characterized in terms of their geometry (i.e., location, scale, and orientation) and a descriptor of local image appearance. Feature correspondences between ultrasound images are achieved based on a nearest-neighbor descriptor matching and probabilistic voting model similar to the Hough transform.
Experiments demonstrate our method on intraoperative ultrasound images acquired before and after opening of the dura mater, during resection and after resection in nine clinical cases. A total of 1620 automatically extracted 3D feature correspondences were manually validated by eleven experts and used to guide the registration. Then, using manually labeled corresponding landmarks in the pre- and post-resection ultrasound images, we show that our feature-based registration reduces the mean target registration error from an initial value of 3.3 to 1.5 mm.
This result demonstrates that the 3D features promise to offer a robust and accurate solution for 3D ultrasound registration and to correct for brain shift in image-guided neurosurgery.
This is a preview of subscription content, access via your institution.
Bucholz RD, Smith KR, Laycock KA, McDurmont LL (2001) Three-dimensional localization: from image-guided surgery to information-guided therapy. Methods 25(2):186–200
Hill DG, Maurer CR, Maciunas RJ, Barwise JA, Fitzpatrick JM, Wang MY (1998) Measurement of intraoperative brain surface deformation under a craniotomy. Neurosurgery 43(3):514–528
Roberts D, Hartov A, Kennedy F, Miga M, Paulsen K (1998) Intraoperative brain shift and deformation: a quantitative analysis of cortical displacement in 28 cases. Neurosurgery 43:749–758
Letteboer MMJ, Willems PW, Viergever MA, Niessen WJ (2005) Brain shift estimation in image-guided neurosurgery using 3-D ultrasound. IEEE Trans Biomed Eng 52(2):268–276
Audette MA, Siddiqi K, Ferrie FP, Peters TM (2003) An integrated range-sensing, segmentation and registration framework for the characterization of intra-surgical brain deformations in image-guided surgery. Comput Vis Image Underst 89(2–3):226–251
Marko NF, Weil RJ, Schroeder JL, Lang FF, Suki D, Sawaya RE (2014) Extent of resection of glioblastoma revisited: personalized survival modeling facilitates more accurate survival prediction and supports a maximum-safe-resection approach to surgery. J Clin Oncol 32(8):774
Coburger J, Merkel A, Scherer M, Schwartz F, Gessler F, Roder C, Jungk C (2015) Low-grade glioma surgery in intraoperative magnetic resonance imaging: results of a multicenter retrospective assessment of the German Study Group for Intraoperative Magnetic Resonance Imaging. Neurosurgery 78(6):775–786
Brown TJ, Brennan MC, Li M, Church EW, Brandmeir NJ, Rakszawski KL, Glantz M (2016) Association of the extent of resection with survival in glioblastoma: a systematic review and meta-analysis. JAMA Oncol 2(11):1460–1469
Almeida JP, Chaichana KL, Rincon-Torroella J, Quinones-Hinojosa A (2015) The value of extent of resection of glioblastomas: clinical evidence and current approach. Curr Neurol Neurosci Rep 15(2):517
Hatiboglu MA, Weinberg JS, Suki D, Rao G, Prabhu SS, Shah K, Jackson E, Sawaya R (2009) Impact of intra-operative high-field magnetic resonance imaging guidance on glioma surgery: a prospective volumetric study. Neurosurgery 64:1073–1081
Claus EB, Horlacher A, Hsu L, Schwartz RB, Dello-Iacono D, Talos F, Jolesz FA, Black PM (2005) Survival rates in patients with low-grade glioma after intra-operative magnetic resonance image guidance. Cancer 103:1227–1233
Nabavi A, Black PM, Gering DT, Westin CF, Mehta V, Pergolizzi RS Jr, Ferrant M, Warfield SK, Hata N, Schwartz RB, Wells WM, Kikinis R, Jolesz F (2001) Serial intra-operative magnetic resonance imaging of brain shift. Neurosurgery 48:787–797
Kuhnt D, Bauer MH, Nimsky C (2012) Brain shift compensation and neurosurgical image fusion using intraoperative MRI: current status and future challenges. Crit Rev Biomed Eng 40:175–185
Tirakotai D, Miller S, Heinze L, Benes L, Bertalanffy H, Sure U (2006) A novel platform for image-guided ultrasound. Neurosurgery 58:710–718
Zhou H, Rivaz H (2016) Registration of pre-and postresection ultrasound volumes with noncorresponding regions in neurosurgery. IEEE J Biomed Health Inform 20(5):1240–1249
Rivaz H, Collins DL (2015) Near real-time robust non-rigid registration of volumetric ultrasound images for neurosurgery. Ultrasound Med Biol 41(2):574–587
Mercier L, Araujo D, Haegelen C, Del Maestro RF, Petrecca K, Collins DL (2013) Registering pre- and post-resection 3-dimensional ultrasound for improved visualization of residual brain tumor. Ultrasound Med Biol 39(1):16–29
Gerard IJ, Kersten-Oertel M, Petrecca K, Sirhan D, Hall JA, Collins DL (2017) Brain shift in neuronavigation of brain tumors: a review. Med Image Anal 35:403–420
Pheiffer TS, Thompson RC, Rucker DC, Simpson AL, Miga MI (2014) Model-based correction of tissue compression for tracked ultrasound in soft tissue image-guided surgery. Ultrasound Med Biol 40(4):788–803
Morin F, Chabanas M, Courtecuisse H, Payan Y (2017) Biomechanical modeling of brain soft tissues for medical applications. Biomechanics of living organs. Academic Press, Cambridge, pp 127–146
Morin F, Courtecuisse H, Reinertsen I, Lann FL, Palombi O, Payan Y, Chabanas M (2017) Brain-shift compensation using intraoperative ultrasound and constraint-based biomechanical simulation. Med Image Anal 40:133–153 ISSN 1361-8415
Luo M, Frisken SF, Weis JA, Clements LW, Unadkat P, Thompson RC, Golby AJ, Miga MI (2017) Validation of model-based brain shift correction in neurosurgery via intraoperative magnetic resonance imaging: preliminary results. In: Proceedings of SPIE 10135, medical imaging: image-guided procedures, robotic interventions, and modeling
Rivaz H, Collins DL (2015) Deformable registration of preoperative MR, pre-resection ultrasound, and post-resection ultrasound images of neurosurgery. Int J Comput Assist Radiol Surg 10(7):1017–1028
Blumenthal T, Hartov A, Lunn K, Kennedy FE, Roberts DW, Paulsen KD (2005) Quantifying brain shift during neurosurgery using spatially tracked ultrasound. In: Proceedings of SPIE 5744, medical imaging: visualization, image-guided procedures, and display
Mercier L, Fonov V, Haegelen C, Del Maestro RF, Petrecca K, Collins DL (2012) Comparing two approaches to rigid registration of three-dimensional ultrasound and magnetic resonance images for neurosurgery. Int J Comput Assist Radiol Surg 7(1):125–136
Poon T, Rohling R (2006) Three-dimensional extended field-of-view ultrasound. Ultrasound Med Biol 32:357–369
Schers J, Troccaz J, Daanen V, Fouard C, Plaskos C, Kilian P (2007) 3D/4D ultrasound registration of bone. In: Proceedings of the IEEE ultrasonics symposium, pp 2519–2522
Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE TPAMI 24(5):603–619
Grimson WEL, Lozano-Perez T (1987) Localizing overlapping parts by searching the interpretive tree. IEEE TPAMI 9(4):469–482
Beis JS, Lowe DG (1997) Shape indexing using approximate nearest-neighbour search in high-dimensional spaces. In: CVPR, pp 1000–1006
Biederman I (1987) Recognition-by-components: a theory of human image understanding. Psychol Rev 2(94):115–147
Pratikakis I (2003) Robust multiscale deformable registration of 3D ultrasound images. Int J Image Graph 3:547–565
Cen F, Jiang Y, Zhang Z, Tsui HT, Lau TK, Xie H (2004) Robust registration of 3D-ultrasound images based on gabor filter and mean-shift. In: Method, pp 304–316
Schneider RJ, Perrin DP, Vasilyev NV, Marx GR, Pedro J, Howe RD (2012) Real-time image-based rigid registration of three-dimensional ultrasound. Med Image Anal 16:402–414
Toews M, Wells WM III (2013) Efficient and robust model-to-image registration using 3D scale-invariant features. Med Image Anal 17(3):271–282
Ni D, Qu Y, Yang X, Chui YP, Wong TT, Ho SS, Heng PA (2008) Volumetric ultrasound panorama based on 3D SIFT. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 52-60
Bersvendsen J, Toews M, Danudibroto A, Wells WM, Urheim S, Estépar RSJ, Samset E (2016) Robust spatio-temporal registration of 4D cardiac ultrasound sequences. In: Medical imaging 2016: ultrasonic imaging and tomography, vol. 9790. International society for optics and photonics, p 97900F
Kadir T, Brady M (2001) Saliency, scale and image description. Int J Comput Vis 45(2):83–105
Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. IEEE Trans Pattern Anal Mach Intell 27(10):1615–1630
Toews M, Wells WM (2009) Sift-rank: ordinal description for invariant feature correspondence. In: IEEE conference on computer vision and pattern recognition, 2009. CVPR 2009, pp 172–177
Jenkinson M, Smith S (2001) A global optimisation method for robust affine registration of brain images. Med Image Anal 5(2):143–156
Bookstein FL (1989) Principal warps: thin-plate splines and the decomposition of deformations. IEEE Trans Pattern Anal Mach Intell 11(6):567–585
Rohr K, Stiehl HS, Sprengel R, Beil W, Buzug TM, Weese J, Kuhn MH (1996) Point-based elastic registration of medical image data using approximating thin-plate splines. In: Visualization in biomedical computing. Springer, Berlin, pp 297–306
Tempany C, Jayender J, Kapur T, Bueno R, Golby A, Agar N, Jolesz FA (2015) Multimodal imaging for improved diagnosis and treatment of cancers. Cancer 121(6):817–827
Strong EB, Rafii A, Holhweg-Majert B, Fuller SC, Metzger MC (2008) Comparison of 3 optical navigation systems for computer-aided maxillofacial surgery. Arch Otolaryngol Head Neck Surg 134(10):1080–1084
Tokuda J, Fischer GS, Papademetris X, Yaniv Z, Ibanez L, Cheng P, Liu H, Blevins J, Arata J, Golby AJ, Kapur T, Pieper S, Burdette EC, Fichtinger G, Tempany CM, Hata N (2009) OpenIGTLink: an open network protocol for image-guided therapy environment. Int J Med Robot 5(4):423–34
Lasso A, Heffter T, Rankin A, Pinter C, Ungi T, Fichtinger G (2014) PLUS: open-source toolkit for ultrasound-guided intervention systems. IEEE Trans Biomed Eng 61(10):2527–2537
Gobbi DG, Peters TM (2002) Interactive intra-operative 3D ultrasound reconstruction and visualization. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 156–163
Boisvert J, Gobbi D, Vikal S, Rohling R, Fichtinger G, Abolmaesumi P (2008) An open-source solution for interactive acquisition, processing and transfer of interventional ultrasound images. In: The MIDAS journal-systems and architectures for computer assisted interventions, p 70
Selbekk T, Jakola AS, Solheim O, Johansen TF, Lindseth F, Reinertsen I, Unsgård G (2013) Ultrasound imaging in neurosurgery: approaches to minimize surgically induced image artefacts for improved resection control. Acta Neurochir 155(6):973–980. https://doi.org/10.1007/s00701-013-1647-7
Kikinis R, Pieper SD, Vosburgh K (2014) 3D Slicer: a platform for subject-specific image analysis, visualization, and clinical support. Intraoperative Imaging Image Guided Ther 3(19):277–289. ISBN: 978-1-4614-7656-6
Jannin P, Fitzpatrick JM, Hawkes D, Pennec X, Shahidi R, Vannier M (2002) Validation of medical image processing in image-guided therapy. IEEE Trans Med Imaging 21(12):1445–9
Mercier L, Del Maestro RF, Petrecca K, Araujo D, Haegelen C, Collins DL (2012) Online database of clinical MR and ultrasound images of brain tumors. Med Phys 39(6Part1):3253–3261
Xiao Y, Fortin M, Unsgård G, Rivaz H, Reinertsen I (2017) REtroSpective Evaluation of Cerebral Tumors (RESECT): a clinical database of pre-operative MRI and intra-operative ultrasound in low-grade glioma surgeries. Med Phys 44(7):3875–3882
This study is supported by the National Institute of Health Grants P41-EB015898-09, P41-EB015902 and R01-NS049251 and the Natural Sciences and Engineering Research Council of Canada, Discovery grant. The authors would like to acknowledge the financial support from the Portuguese Foundation for Science and Technology under the references PD/BD/105869/2014 and IDMEC/LAETA UID/EMS/50022/2013.
Conflict of interest
The authors declare that they have no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study.
About this article
Cite this article
Machado, I., Toews, M., Luo, J. et al. Non-rigid registration of 3D ultrasound for neurosurgery using automatic feature detection and matching. Int J CARS 13, 1525–1538 (2018). https://doi.org/10.1007/s11548-018-1786-7
- Brain shift
- Intraoperative ultrasound
- Image-guided neurosurgery
- Image registration
- 3D scale-invariant features