Abstract
Diffeomorphic registration is widely used in medical image processing with the invertible and one-to-one mapping between images. Recent progress has been made to diffeomorphic registration by utilizing a convolutional neural network for efficient and end-to-end inference of registration fields from an image pair. However, existing deep learning-based registration models neglect to employ attention mechanisms to handle the long-range cross-image relevance in embedding learning, limiting such approaches to identify the semantically meaningful correspondence of anatomical structures. In this paper, we propose a novel dual transformer network (DTN) for diffeomorphic registration, consisting of a learnable volumetric embedding module, a dual cross-image relevance learning module for feature enhancement, and a registration field inference module. The self-attention mechanisms of DTN explicitly model both the inter- and intra-image relevances in the embedding from both the separate and concatenated volumetric images, facilitating semantical correspondence of anatomical structures in diffeomorphic registration. Extensive quantitative and qualitative evaluations demonstrate that the DTN performs favorably against state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ashburner, J.: A fast diffeomorphic image registration algorithm. NeuroImage 38, 95–113 (2007)
Avants, B., Epstein, C., Grossman, M., Gee, J.: Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med. Image Anal. 12(1), 26–41 (2008)
Avants, B., Tustison, N., Song, G., Cook, P., Klein, A., Gee, J.: A reproducible evaluation of ants similarity metric performance in brain image registration. NeuroImage 54, 2033–2044 (2011)
Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: An unsupervised learning model for deformable medical image registration. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 9252–9260 (2018)
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. ArXiv abs/2005.12872 (2020)
Chen, H., et al.: Pre-trained image processing transformer. ArXiv abs/2012.00364 (2020)
Chen, J., et al.: TransuNet: transformers make strong encoders for medical image segmentation. ArXiv abs/2102.04306 (2021)
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 424–432 (2016)
Dalca, A.V., Balakrishnan, G., Guttag, J., Sabuncu, M.R.: Unsupervised learning for fast probabilistic diffeomorphic registration. In: Frangi, A., Schnabel, J., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 729–738. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_82
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. ArXiv abs/2010.11929 (2020)
Fischl, B.: Freesurfer. NeuroImage 62, 774–781 (2012)
Haskins, G., Kruger, U., Yan, P.: Deep learning in medical image registration: a survey. Mach. Vis. Appl. 31, 1–18 (2020)
Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. In: NeurIPS, pp. 2017–2025 (2015)
Leow, A., et al.: Inverse consistent mapping in 3D deformable image registration: its construction and statistical properties. Inf. Process. Med. Imaging 19, 493–503 (2005)
Liao, R., et al.: An artificial agent for robust image registration. In: AAAI (2017)
Marcus, D., Wang, T.H., Parker, J., Csernansky, J., Morris, J., Buckner, R.: Open access series of imaging studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. J. Cogn. Neurosci. 19, 1498–1507 (2007)
Miao, S., et al.: Dilated FCN for multi-agent 2D/3D medical image registration. ArXiv abs/1712.01651 (2018)
Mok, T.C.W., Chung, A.C.S.: Fast symmetric diffeomorphic image registration with convolutional neural networks. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4643–4652 (2020)
Petit, O., Thome, N., Rambour, C., Soler, L.: U-net transformer: self and cross attention for medical image segmentation. ArXiv abs/2103.06104 (2021)
Schlemper, J., et al.: Attention gated networks: learning to leverage salient regions in medical images. Med. Image Anal. 53, 197–207 (2019)
Sotiras, A., Davatzikos, C., Paragios, N.: Deformable medical image registration: a survey. IEEE Trans. Med. Imaging 32(7), 1153–1190 (2013)
Tan, H.H., Bansal, M.: LXMERT: learning cross-modality encoder representations from transformers. In: EMNLP/IJCNLP (2019)
Valanarasu, J.M.J., Oza, P., Hacihaliloglu, I., Patel, V.: Medical transformer: gated axial-attention for medical image segmentation (2021)
Vaswani, A., et al.: Attention is all you need. ArXiv abs/1706.03762 (2017)
Wang, Q., et al.: Learning deep transformer models for machine translation. ArXiv abs/1906.01787 (2019)
Zhang, J.: Inverse-consistent deep networks for unsupervised deformable image registration. ArXiv abs/1809.03443 (2018)
Zhang, Y., Pei, Y., Guo, Y., Ma, G., Xu, T., Zha, H.: Fully convolutional network for consistent voxel-wise correspondence. In: AAAI (2020)
Acknowledgments
This work was supported in part by National Natural Science Foundation of China under Grant 61876008 and 82071172, Beijing Natural Science Foundation under Grant 7192227, and Research Center of Engineering and Technology for Digital Dentistry, Ministry of Health.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, Y., Pei, Y., Zha, H. (2021). Learning Dual Transformer Network for Diffeomorphic Registration. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12904. Springer, Cham. https://doi.org/10.1007/978-3-030-87202-1_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-87202-1_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87201-4
Online ISBN: 978-3-030-87202-1
eBook Packages: Computer ScienceComputer Science (R0)
