Skip to main content
SpringerLink
Log in

Highly Accurate and Memory Efficient Unsupervised Learning-Based Discrete CT Registration Using 2.5D Displacement Search

  • Conference paper
  • First Online:
Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 (MICCAI 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12263))

Abstract

Learning-based registration, in particular unsupervised approaches that use a deep network to predict a displacement field that minimise a conventional similarity metric, has gained huge interest within the last two years. It has, however, not yet reached the high accuracy of specialised conventional algorithms for estimating large 3D deformations. Employing a dense set of discrete displacements (in a so-called correlation layer) has shown great success in learning 2D optical flow estimation, cf. FlowNet and PWC-Net, but comes at excessive memory requirements when extended to 3D medical registration. We propose a highly accurate unsupervised learning framework for 3D abdominal CT registration that uses a discrete displacement layer and a contrast-invariant metric (MIND descriptors) that is evaluated in a probabilistic fashion. We realise a substantial reduction in memory and computational demand by iteratively subdividing the 3D search space into orthogonal planes. In our experimental validation on inter-subject deformable 3D registration, we demonstrate substantial improvements in accuracy (at least \(\approx \)10% points Dice) compared to widely used conventional methods (ANTs SyN, NiftyReg, IRTK) and state-of-the-art U-Net based learning methods (VoxelMorph). We reduce the search space 5-fold, speed-up the run-time twice and are on-par in terms of accuracy with a fully 3D discrete network.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/multimodallearning/pdd2.5/.

  2. 2.

    https://github.com/mattiaspaul/deedsBCV.

  3. 3.

    https://learn2reg.grand-challenge.org/Dataset/ (Task 3).

References

  1. Anonymous: Tackling the problem of large deformations in deep learning based medical image registration using displacement embeddings. Medical Imaging with Deep Learning, pp. 1–5 (2020, under reviewed). https://openreview.net/pdf?id=kPBUZluVq

  2. Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: Voxelmorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imag. (2019)

    Google Scholar 

  3. Blendowski, M., Nickisch, H., Heinrich, M.P.: How to learn from unlabeled volume data: self-supervised 3D context feature learning. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 649–657. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_72

    Chapter  Google Scholar 

  4. Budelmann, D., König, L., Papenberg, N., Lellmann, J.: Fully-deformable 3D image registration in two seconds. In: Handels, H., Deserno, T., Maier, A., Maier-Hein, K., Palm, C., Tolxdorff, T. (eds.) BVM, pp. 302–307. Springer, Heidelberg (2019). https://doi.org/10.1007/978-3-658-25326-4_67

    Chapter  Google Scholar 

  5. Chen, W., Liu, B., Peng, S., Sun, J., Qiao, X.: S3D-UNet: separable 3D U-Net for brain tumor segmentation. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11384, pp. 358–368. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11726-9_32

    Chapter  Google Scholar 

  6. Dalca, A.V., Balakrishnan, G., Guttag, J., Sabuncu, M.R.: Unsupervised learning for fast probabilistic diffeomorphic registration. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 729–738. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_82

    Chapter  Google Scholar 

  7. Dosovitskiy, A., et al.: Flownet: learning optical flow with convolutional networks. In: Proceedings of ICCV, pp. 2758–2766 (2015)

    Google Scholar 

  8. Eppenhof, K.A., Lafarge, M.W., Veta, M., Pluim, J.P.: Progressively trained convolutional neural networks for deformable image registration. IEEE Trans. Med. Imag. (2019)

    Google Scholar 

  9. Eppenhof, K.A., Pluim, J.P.: Pulmonary CT registration through supervised learning with convolutional neural networks. IEEE Trans. Med. Imag. 38(5), 1097–1105 (2018)

    Article  Google Scholar 

  10. Glocker, B., Komodakis, N., Tziritas, G., Navab, N., Paragios, N.: Dense image registration through MRFs and efficient linear programming. Med. Image Anal. 12(6), 731–741 (2008)

    Article  Google Scholar 

  11. Heinrich, M.P.: Closing the gap between deep and conventional image registration using probabilistic dense displacement networks. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 50–58. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_6

    Chapter  Google Scholar 

  12. Heinrich, M.P., Jenkinson, M., Brady, M., Schnabel, J.A.: MRF-based deformable registration and ventilation estimation of lung CT. IEEE Trans. Med. Imag. 32(7), 1239–1248 (2013)

    Article  Google Scholar 

  13. Heinrich, M.P., Oktay, O., Bouteldja, N.: OBELISK-Net: fewer layers to solve 3D multi-organ segmentation with sparse deformable convolutions. Med. Image Anal. 54, 1–9 (2019)

    Article  Google Scholar 

  14. Heinrich, M.P., Jenkinson, M., Papież, B.W., Brady, S.M., Schnabel, J.A.: Towards realtime multimodal fusion for image-guided interventions using self-similarities. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8149, pp. 187–194. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40811-3_24

    Chapter  Google Scholar 

  15. Hering, A., van Ginneken, B., Heldmann, S.: mlVIRNET: multilevel variational image registration network. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 257–265. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_29

    Chapter  Google Scholar 

  16. Hering, A., Kuckertz, S., Heldmann, S., Heinrich, M.P.: Memory-efficient 2.5 D convolutional transformer networks for multi-modal deformable registration with weak label supervision applied to whole-heart CT and MRI scans. Int. J. Comput. Assist. Radiol. Surg. 14(11), 1901–1912 (2019)

    Google Scholar 

  17. Krebs, J., Delingette, H., Mailhé, B., Ayache, N., Mansi, T.: Learning a probabilistic model for diffeomorphic registration. IEEE Trans. Med. Imag. 38(9), 2165–2176 (2019)

    Article  Google Scholar 

  18. Liu, C., Yuen, J., Torralba, A.: SIFT flow: dense correspondence across scenes and its applications. IEEE Trans. Patt. Anal. Mach. Intell. 33(5), 978–994 (2010)

    Article  Google Scholar 

  19. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

  20. Modat, M., et al.: Fast free-form deformation using graphics processing units. Comput. Methods Programs Biomed. 98(3), 278–284 (2010)

    Article  Google Scholar 

  21. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  22. Rühaak, J., et al.: Estimation of large motion in lung CT by integrating regularized keypoint correspondences into dense deformable registration. IEEE Trans. Med. Imag. 36(8), 1746–1757 (2017)

    Article  Google Scholar 

  23. Setio, A.A.A., et al.: Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans. Med. Imag. 35(5), 1160–1169 (2016)

    Article  Google Scholar 

  24. Shekhovtsov, A., Kovtun, I., Hlaváč, V.: Efficient MRF deformation model for non-rigid image matching. Comput. Vis. Image Und. 112(1), 91–99 (2008)

    Article  Google Scholar 

  25. Shin, H.C., et al.: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imag. 35(5), 1285–1298 (2016)

    Article  Google Scholar 

  26. Sun, D., Yang, X., Liu, M.Y., Kautz, J.: PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In: Proceedings of CVPR, pp. 8934–8943 (2018)

    Google Scholar 

  27. Veksler, O.: Stereo correspondence by dynamic programming on a tree. In: Proceedings of CVPR, vol. 2, pp. 384–390. IEEE (2005)

    Google Scholar 

  28. de Vos, B.D., Berendsen, F.F., Viergever, M.A., Sokooti, H., Staring, M., Išgum, I.: A deep learning framework for unsupervised affine and deformable image registration. Med. Image Anal. 52, 128–143 (2019)

    Article  Google Scholar 

  29. Warfield, S.K., Zou, K.H., Wells, W.M.: Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation. IEEE Trans. Med. Imag. 23(7), 903–921 (2004)

    Article  Google Scholar 

  30. Xu, Z., et al.: Evaluation of 6 registration methods for the human abdomen on clinically acquired CT. IEEE Trans. Biomed. Eng. 63(8), 1563–1572 (2016)

    Article  Google Scholar 

  31. Zheng, S., et al.: Conditional random fields as recurrent neural networks. In: Proceedings of ICCV, pp. 1529–1537 (2015)

    Google Scholar 

  32. Zikic, D., et al.: Linear intensity-based image registration by Markov random fields and discrete optimization. Med. Image Anal. 14(4), 550–562 (2010)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Medical Informatics, Universität zu Lübeck, Lübeck, Germany

    Mattias P. Heinrich & Lasse Hansen

Authors
  1. Mattias P. Heinrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lasse Hansen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mattias P. Heinrich .

Editor information

Editors and Affiliations

  1. University of Toronto, Toronto, ON, Canada

    Anne L. Martel

  2. The University of British Columbia, Vancouver, BC, Canada

    Purang Abolmaesumi

  3. University College London, London, UK

    Danail Stoyanov

  4. École Centrale de Nantes, Nantes, France

    Diana Mateus

  5. EURECOM, Biot, France

    Maria A. Zuluaga

  6. Chinese Academy of Sciences, Beijing, China

    S. Kevin Zhou

  7. Sorbonne University, Paris, France

    Daniel Racoceanu

  8. The Hebrew University of Jerusalem, Jerusalem, Israel

    Leo Joskowicz

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Heinrich, M.P., Hansen, L. (2020). Highly Accurate and Memory Efficient Unsupervised Learning-Based Discrete CT Registration Using 2.5D Displacement Search. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12263. Springer, Cham. https://doi.org/10.1007/978-3-030-59716-0_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-59716-0_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-59715-3

  • Online ISBN: 978-3-030-59716-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation