Advertisement

Deep Learning Based Inter-modality Image Registration Supervised by Intra-modality Similarity

  • Xiaohuan Cao
  • Jianhuan Yang
  • Li Wang
  • Zhong Xue
  • Qian Wang
  • Dinggang Shen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11046)

Abstract

Non-rigid inter-modality registration can facilitate accurate information fusion from different modalities, but it is challenging due to the very different image appearances across modalities. In this paper, we propose to train a non-rigid inter-modality image registration network, which can directly predict the transformation field from the input multimodal images, such as CT and MRI. In particular, the training of our inter-modality registration network is supervised by intra-modality similarity metric based on the available paired data, which is derived from a pre-aligned CT and MRI dataset. Specifically, in the training stage, to register the input CT and MR images, their similarity is evaluated on the warped MR image and the MR image that is paired with the input CT. So that, the intra-modality similarity metric can be directly applied to measure whether the input CT and MR images are well registered. Moreover, we use the idea of dual-modality fashion, in which we measure the similarity on both CT modality and MR modality. In this way, the complementary anatomies in both modalities can be jointly considered to more accurately train the inter-modality registration network. In the testing stage, the trained inter-modality registration network can be directly applied to register the new multimodal images without any paired data. Experimental results have shown that, the proposed method can achieve promising accuracy and efficiency for the challenging non-rigid inter-modality registration task and also outperforms the state-of-the-art approaches.

References

  1. 1.
    Cao, X., et al.: Region-adaptive deformable registration of ct/mri pelvic images via learning-based image synthesis. IEEE Trans. Image Process. 27(7), 3500–3512 (2018)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Pluim, J.P., Maintz, J.A., Viergever, M.A.: Mutual-information-based registration of medical images: a survey. IEEE Trans. Med. Imaging 22(8), 986–1004 (2003)CrossRefGoogle Scholar
  3. 3.
    Cao, X. et al.: Deformable image registration using cue-aware deep regression network. IEEE Trans. Biomed. Eng. (2018)Google Scholar
  4. 4.
    de Vos, B.D., et al.: End-to-end unsupervised deformable image registration with a convolutional neural network. In: Cardoso, M.J., Arbel, T., Carneiro, G., Syeda-Mahmood, T., Tavares, J.M.R.S., Moradi, M., Bradley, A., Greenspan, H., Papa, J.P., Madabhushi, A., Nascimento, J.C., Cardoso, J.S., Belagiannis, V., Lu, Z. (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 204–212. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-67558-9_24CrossRefGoogle Scholar
  5. 5.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab, Nassir, Hornegger, Joachim, Wells, William M., Frangi, Alejandro F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  6. 6.
    Jaderberg, M., Simonyan, K., Zisserman, A.: Spatial transformer networks. In: Advances in Neural Information Processing Systems (2015)Google Scholar
  7. 7.
    Avants, B.B., et al.: Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med. Image Anal. 12(1), 26–41 (2008)CrossRefGoogle Scholar
  8. 8.
    Vercauteren, T., Pennec, X., Perchant, A., Ayache, N.: Non-parametric Diffeomorphic Image Registration with the Demons Algorithm. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007. LNCS, vol. 4792, pp. 319–326. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-75759-7_39CrossRefGoogle Scholar
  9. 9.
    Jenkinson, M., Smith, S.: A global optimisation method for robust affine registration of brain images. Med. Image Anal. 5(2), 143–156 (2001)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Xiaohuan Cao
    • 1
    • 2
  • Jianhuan Yang
    • 1
  • Li Wang
    • 2
  • Zhong Xue
    • 3
  • Qian Wang
    • 4
  • Dinggang Shen
    • 2
  1. 1.School of AutomationNorthwestern Polytechnical UniversityXi’anChina
  2. 2.Department of Radiology and BRICUniversity of North Carolina at Chapel HillChapel HillUSA
  3. 3.Shanghai United Imaging Intelligence Co., Ltd.ShanghaiChina
  4. 4.School of Biomedical EngineeringInstitute for Medical Imaging Technology, Shanghai Jiao Tong UniversityShanghaiChina

Personalised recommendations