Abstract
Non-rigid registration is a key component in soft-tissue navigation. We focus on laparoscopic liver surgery, where we register the organ model obtained from a preoperative CT scan to the intraoperative partial organ surface, reconstructed from the laparoscopic video. This is a challenging task due to sparse and noisy intraoperative data, real-time requirements and many unknowns - such as tissue properties and boundary conditions. Furthermore, establishing correspondences between pre- and intraoperative data can be extremely difficult since the liver usually lacks distinct surface features and the used imaging modalities suffer from very different types of noise. In this work, we train a convolutional neural network to perform both the search for surface correspondences as well as the non-rigid registration in one step. The network is trained on physically accurate biomechanical simulations of randomly generated, deforming organ-like structures. This enables the network to immediately generalize to a new patient organ without the need to re-train. We add various amounts of noise to the intraoperative surfaces during training, making the network robust to noisy intraoperative data. During inference, the network outputs the displacement field which matches the preoperative volume to the partial intraoperative surface. In multiple experiments, we show that the network translates well to real data while maintaining a high inference speed. Our code is made available online.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bilic, P., Christ, P.F., Vorontsov, E., Chlebus, G., Chen, H., Dou, Q., et al.: The liver tumor segmentation benchmark (LiTS). ArXiv abs/1901.04056 (2019)
Brunet, J.-N., Mendizabal, A., Petit, A., Golse, N., Vibert, E., Cotin, S.: Physics-based deep neural network for augmented reality during liver surgery. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11768, pp. 137–145. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32254-0_16
Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: representing model uncertainty in deep learning. In: Proceedings of the 33rd International Conference on International Conference on Machine Learning. vol. 48 (2016)
Geuzaine, C., Remacle, J.F.: Gmsh: A 3-d finite element mesh generator with built-in pre- and post-processing facilities. Int. J. Numer. Meth. Eng. 79(11), 1309–1331 (2009)
Griffiths, D., Boehm, J.: A review on deep learning techniques for 3d sensed data classification. Remote Sensing 11(12), 1499 (2019)
Heiselman, J., Clements, L., Collins, J., Weis, J., Simpson, A., Geevarghese, S.: Characterization and correction of intraoperative soft tissue deformation in image-guided laparoscopic liver surgery. J. Med. Imaging 5(2), 021203 (2017)
Koo, B., Özgür, E., Le Roy, B., Buc, E., Bartoli, A.: Deformable registration of a preoperative 3d liver volume to a laparoscopy image using contour and shading cues. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 326–334. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_38
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (ICLR) (2017)
Malinen, M., Råback, P.: Elmer Finite Element Solver for Multiphysics and Multiscale Problems. Multiscale Modelling Methods for Applications in Materials Science, Forschungszentrum Jülich (2013)
Mendizabal, A., Márquez-Neila, P., Cotin, S.: Simulation of hyperelastic materials in real-time using deep learning. Med. Image Anal. 59, 101569 (2019)
Mendizabal, A., Tagliabue, E., Brunet, J.N., Dallálba, D., Fiorini, P., Cotin, S.: Physics-based deep neural network for real-time lesion tracking in ultrasound-guided breast biopsy. In: Computational Biomechanics for Medicine XIV. Shenzhen, China (2019)
Mur-Artal, R., Tardós, J.D.: ORB-SLAM2: an open-source SLAM system for monocular, stereo and RGB-D cameras. IEEE Trans. Robot. 33(5), 1255–1262 (2017)
Özgür, E., Koo, B., Le Roy, B., Buc, E., Bartoli, A.: Preoperative liver registration for augmented monocular laparoscopy using backward-forward biomechanical simulation. Int. J. Comput. Assist. Radiol. Surg. 13, 1629–1640 (2018)
Pellicer-Valero, O.J., Rupérez, M.J., Martínez-Sanchis, S., Martín-Guerrero, J.D.: Real-time biomechanical modeling of the liver using machine learning models trained on finite element method simulations. Expert Syst. Appl. 143, 113083 (2020)
Pfeiffer, M., et al.: Generating large labeled data sets for laparoscopic image processing tasks using unpaired image-to-image translation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11768, pp. 119–127. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32254-0_14
Pfeiffer, M., Riediger, C., Weitz, J., Speidel, S.: Learning soft tissue behavior of organs for surgical navigation with convolutional neural networks. Int. J. Comput. Assist. Radiol. Surg. 14(7), 1147–1155 (2019). https://doi.org/10.1007/s11548-019-01965-7
Plantefeve, R., Peterlik, I., Haouchine, N., Cotin, S.: Patient-specific biomechanical modeling for guidance during minimally-invasive hepatic surgery. Ann. Biomed. Eng. 143, 113083 (2015)
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: deep learning on point sets for 3d classification and segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Smith, L.N., Topin, N.: Super-convergence: very fast training of residual networks using large learning rates. CoRR abs/1708.07120 (2017)
Suwelack, S., et al.: Physics-based shape matching for intraoperative image guidance. Med. phys. 41, (2014)
Wang, H., Guo, J., Yan, D.-M., Quan, W., Zhang, X.: Learning 3D Keypoint descriptors for non-rigid shape matching. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11212, pp. 3–20. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01237-3_1
Yang, G., Manela, J., Happold, M., Ramanan, D.: Hierarchical deep stereo matching on high-resolution images. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Pfeiffer, M. et al. (2020). Non-Rigid Volume to Surface Registration Using a Data-Driven Biomechanical Model. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12264. Springer, Cham. https://doi.org/10.1007/978-3-030-59719-1_70
Download citation
DOI: https://doi.org/10.1007/978-3-030-59719-1_70
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59718-4
Online ISBN: 978-3-030-59719-1
eBook Packages: Computer ScienceComputer Science (R0)
