Skip to main content

Data-Driven Multi-modal Partial Medical Image Preregistration by Template Space Patch Mapping

  • 4395 Accesses

Part of the Lecture Notes in Computer Science book series (LNCS,volume 13436)


Image registration is an essential part of Medical Image Analysis. Traditional local search methods (e.g., Mean Square Errors (MSE) and Normalized Mutual Information (NMI)) achieve accurate registration but require good initialization. However, finding a good initialization is difficult in partial image matching. Recent deep learning methods such as images-to-transformation directly solve the registration problem but need images of mostly same sizes and already roughly aligned. This work presents a learning-based method to provide good initialization for partial image registration. A light and efficient network learns the mapping from a small patch of an image to a position in the template space for each modality. After computing such mapping for a set of patches, we compute a rigid transformation matrix that maps the patches to the corresponding target positions. We tested our method to register a 3DRA image of a partial brain to a CT image of a whole brain. The result shows that MSE registration with our initialization significantly outperformed baselines including naive initialization and recent deep learning methods without template. You can access our source code in


  • Multi-modal
  • Partial image
  • Preregistration
  • Patch mapping

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-031-16446-0_25
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   99.00
Price excludes VAT (USA)
  • ISBN: 978-3-031-16446-0
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   129.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.


  1. Bashiri, F.S., Baghaie, A., Rostami, R., Yu, Z., D’Souza, R.M.: Multi-modal medical image registration with full or partial data: a manifold learning approach. J. Imaging 5(1), 5 (2019)

    CrossRef  Google Scholar 

  2. Cheng, X., Zhang, L., Zheng, Y.: Deep similarity learning for multimodal medical images. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 6(3), 248–252 (2018)

    CrossRef  Google Scholar 

  3. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    CrossRef  MathSciNet  Google Scholar 

  4. Guo, H., Kruger, M., Xu, S., Wood, B.J., Yan, P.: Deep adaptive registration of multi-modal prostate images. Comput. Med. Imaging Graph. 84, 101769 (2020)

    CrossRef  Google Scholar 

  5. Heautot, J., et al.: Analysis of cerebrovascular diseases by a new 3-dimensional computerised x-ray angiography system. Neuroradiology 40(4), 203–209 (1998)

    CrossRef  Google Scholar 

  6. Hochmuth, A., Spetzger, U., Schumacher, M.: Comparison of three-dimensional rotational angiography with digital subtraction angiography in the assessment of ruptured cerebral aneurysms. Am. J. Neuroradiol. 23(7), 1199–1205 (2002)

    Google Scholar 

  7. Kin, T., et al.: A new strategic neurosurgical planning tool for brainstem cavernous malformations using interactive computer graphics with multimodal fusion images. J. Neurosurg. 117(1), 78–88 (2012)

    CrossRef  Google Scholar 

  8. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  9. Kumamaru, K.K., Hoppel, B.E., Mather, R.T., Rybicki, F.J.: CT angiography: current technology and clinical use. Radiol. Clin. 48(2), 213–235 (2010)

    CrossRef  Google Scholar 

  10. Liao, H., Lin, W.A., Zhang, J., Zhang, J., Luo, J., Zhou, S.K.: Multiview 2D/3D rigid registration via a point-of-interest network for tracking and triangulation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12638–12647 (2019)

    Google Scholar 

  11. Lukiyanov, V.: Pytorch implementation of SDAE (stacked denoising autoencoder) (2018).

  12. Müller, M., Heidelberger, B., Teschner, M., Gross, M.: Meshless deformations based on shape matching. ACM Trans. Graph. (TOG) 24(3), 471–478 (2005)

    CrossRef  Google Scholar 

  13. Napel, S., et al.: Ct angiography with spiral CT and maximum intensity projection. Radiology 185(2), 607–610 (1992)

    CrossRef  Google Scholar 

  14. Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  15. Pluim, J., Maintz, J., Viergever, M.: Mutual-information-based registration of medical images: a survey. IEEE Trans. Med. Imaging 22(8), 986–1004 (2003).

    CrossRef  Google Scholar 

  16. Raabe, A., Beck, J., Rohde, S., Berkefeld, J., Seifert, V.: Three-dimensional rotational angiography guidance for aneurysm surgery. J. Neurosurg. 105(3), 406–411 (2006)

    CrossRef  Google Scholar 

  17. Sandkühler, R., Jud, C., Andermatt, S., Cattin, P.C.: Airlab: autograd image registration laboratory. arXiv preprint arXiv:1806.09907 (2018)

  18. Sloan, J.M., Goatman, K.A., Siebert, J.P.: Learning rigid image registration - utilizing convolutional neural networks for medical image registration. In: Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies - BIOIMAGING, pp. 89–99. INSTICC, SciTePress (2018).

  19. Song, X., et al.: Cross-modal attention for MRI and ultrasound volume registration. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12904, pp. 66–75. Springer, Cham (2021).

    CrossRef  Google Scholar 

  20. Stalling, D., Westerhoff, M., Hege, H.C., et al.: Amira: a highly interactive system for visual data analysis. In: The Visualization Handbook, vol. 38, pp. 749–67 (2005)

    Google Scholar 

  21. Studholme, C., Hill, D., Hawkes, D.: An overlap invariant entropy measure of 3D medical image alignment. Pattern Recognit. 32(1), 71–86 (1999)

    CrossRef  Google Scholar 

  22. Yan, P., Xu, S., Rastinehad, A.R., Wood, B.J.: Adversarial image registration with application for MR and TRUS image fusion. In: Shi, Y., Suk, H.-I., Liu, M. (eds.) MLMI 2018. LNCS, vol. 11046, pp. 197–204. Springer, Cham (2018).

    CrossRef  Google Scholar 

  23. Yao, Z., et al.: A supervised network for fast image-guided radiotherapy (IGRT) registration. J. Med. Syst. 43(7), 1–8 (2019)

    CrossRef  Google Scholar 

  24. Zheng, J., Miao, S., Liao, R.: Learning CNNs with pairwise domain adaption for real-time 6DoF ultrasound transducer detection and tracking from X-Ray images. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 646–654. Springer, Cham (2017).

    CrossRef  Google Scholar 

  25. Zheng, J., Miao, S., Wang, Z.J., Liao, R.: Pairwise domain adaptation module for CNN-based 2-D/3-D registration. J. Med. Imaging 5(2), 021204 (2018)

    CrossRef  Google Scholar 

Download references


This research was supported by AMED under Grant Number JP18he1602001, Japan and JST CREST under Grant Number JPMJCR17A1, Japan.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Ding Xia .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 13818 KB)

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Xia, D., Yang, X., van Kaick, O., Kin, T., Igarashi, T. (2022). Data-Driven Multi-modal Partial Medical Image Preregistration by Template Space Patch Mapping. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13436. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-16445-3

  • Online ISBN: 978-3-031-16446-0

  • eBook Packages: Computer ScienceComputer Science (R0)