Abstract
Image registration is an essential part of Medical Image Analysis. Traditional local search methods (e.g., Mean Square Errors (MSE) and Normalized Mutual Information (NMI)) achieve accurate registration but require good initialization. However, finding a good initialization is difficult in partial image matching. Recent deep learning methods such as images-to-transformation directly solve the registration problem but need images of mostly same sizes and already roughly aligned. This work presents a learning-based method to provide good initialization for partial image registration. A light and efficient network learns the mapping from a small patch of an image to a position in the template space for each modality. After computing such mapping for a set of patches, we compute a rigid transformation matrix that maps the patches to the corresponding target positions. We tested our method to register a 3DRA image of a partial brain to a CT image of a whole brain. The result shows that MSE registration with our initialization significantly outperformed baselines including naive initialization and recent deep learning methods without template. You can access our source code in https://github.com/ApisXia/PartialMedPreregistration.
Keywords
- Multi-modal
- Partial image
- Preregistration
- Patch mapping
This is a preview of subscription content, access via your institution.
Buying options
References
Bashiri, F.S., Baghaie, A., Rostami, R., Yu, Z., D’Souza, R.M.: Multi-modal medical image registration with full or partial data: a manifold learning approach. J. Imaging 5(1), 5 (2019)
Cheng, X., Zhang, L., Zheng, Y.: Deep similarity learning for multimodal medical images. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 6(3), 248–252 (2018)
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Guo, H., Kruger, M., Xu, S., Wood, B.J., Yan, P.: Deep adaptive registration of multi-modal prostate images. Comput. Med. Imaging Graph. 84, 101769 (2020)
Heautot, J., et al.: Analysis of cerebrovascular diseases by a new 3-dimensional computerised x-ray angiography system. Neuroradiology 40(4), 203–209 (1998)
Hochmuth, A., Spetzger, U., Schumacher, M.: Comparison of three-dimensional rotational angiography with digital subtraction angiography in the assessment of ruptured cerebral aneurysms. Am. J. Neuroradiol. 23(7), 1199–1205 (2002)
Kin, T., et al.: A new strategic neurosurgical planning tool for brainstem cavernous malformations using interactive computer graphics with multimodal fusion images. J. Neurosurg. 117(1), 78–88 (2012)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Kumamaru, K.K., Hoppel, B.E., Mather, R.T., Rybicki, F.J.: CT angiography: current technology and clinical use. Radiol. Clin. 48(2), 213–235 (2010)
Liao, H., Lin, W.A., Zhang, J., Zhang, J., Luo, J., Zhou, S.K.: Multiview 2D/3D rigid registration via a point-of-interest network for tracking and triangulation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12638–12647 (2019)
Lukiyanov, V.: Pytorch implementation of SDAE (stacked denoising autoencoder) (2018). https://github.com/vlukiyanov/pt-sdae
Müller, M., Heidelberger, B., Teschner, M., Gross, M.: Meshless deformations based on shape matching. ACM Trans. Graph. (TOG) 24(3), 471–478 (2005)
Napel, S., et al.: Ct angiography with spiral CT and maximum intensity projection. Radiology 185(2), 607–610 (1992)
Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Pluim, J., Maintz, J., Viergever, M.: Mutual-information-based registration of medical images: a survey. IEEE Trans. Med. Imaging 22(8), 986–1004 (2003). https://doi.org/10.1109/TMI.2003.815867
Raabe, A., Beck, J., Rohde, S., Berkefeld, J., Seifert, V.: Three-dimensional rotational angiography guidance for aneurysm surgery. J. Neurosurg. 105(3), 406–411 (2006)
Sandkühler, R., Jud, C., Andermatt, S., Cattin, P.C.: Airlab: autograd image registration laboratory. arXiv preprint arXiv:1806.09907 (2018)
Sloan, J.M., Goatman, K.A., Siebert, J.P.: Learning rigid image registration - utilizing convolutional neural networks for medical image registration. In: Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies - BIOIMAGING, pp. 89–99. INSTICC, SciTePress (2018). https://doi.org/10.5220/0006543700890099
Song, X., et al.: Cross-modal attention for MRI and ultrasound volume registration. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12904, pp. 66–75. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87202-1_7
Stalling, D., Westerhoff, M., Hege, H.C., et al.: Amira: a highly interactive system for visual data analysis. In: The Visualization Handbook, vol. 38, pp. 749–67 (2005)
Studholme, C., Hill, D., Hawkes, D.: An overlap invariant entropy measure of 3D medical image alignment. Pattern Recognit. 32(1), 71–86 (1999)
Yan, P., Xu, S., Rastinehad, A.R., Wood, B.J.: Adversarial image registration with application for MR and TRUS image fusion. In: Shi, Y., Suk, H.-I., Liu, M. (eds.) MLMI 2018. LNCS, vol. 11046, pp. 197–204. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00919-9_23
Yao, Z., et al.: A supervised network for fast image-guided radiotherapy (IGRT) registration. J. Med. Syst. 43(7), 1–8 (2019)
Zheng, J., Miao, S., Liao, R.: Learning CNNs with pairwise domain adaption for real-time 6DoF ultrasound transducer detection and tracking from X-Ray images. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 646–654. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66185-8_73
Zheng, J., Miao, S., Wang, Z.J., Liao, R.: Pairwise domain adaptation module for CNN-based 2-D/3-D registration. J. Med. Imaging 5(2), 021204 (2018)
Acknowledgement
This research was supported by AMED under Grant Number JP18he1602001, Japan and JST CREST under Grant Number JPMJCR17A1, Japan.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Xia, D., Yang, X., van Kaick, O., Kin, T., Igarashi, T. (2022). Data-Driven Multi-modal Partial Medical Image Preregistration by Template Space Patch Mapping. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13436. Springer, Cham. https://doi.org/10.1007/978-3-031-16446-0_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-16446-0_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16445-3
Online ISBN: 978-3-031-16446-0
eBook Packages: Computer ScienceComputer Science (R0)
-
Published in cooperation with
http://miccai.org/
