Abstract
Multi-domain abdominal image alignment is a valuable and challenging task for clinical research. Normally, with the assistance of the image synthesizer, the register can perform well. However, the deviation of the synthesizer is likely to be the bottleneck of the register performance. In this case, using the registration training information to guide the synthesizer optimization is meaningful. Therefore, we propose the Joint Network of Registration and Synthesis (RSNet). The network calculates the loss caused only by the synthetic deviation according to the registration training, which effectively improves the performance of the synthesizer. Meanwhile, the network constructs an adaptive weighting factor based on the similarity measure of the synthetic image, which significantly generalizes the performance of the register. The real-world datasets are collected and processed with the help of a cooperative hospital. Our experiments demonstrate that RSNet can achieve state-of-the-art performance.
This work is supported in part by the National Natural Science Foundation of China (61802347, 61972347, 61773348, U20A20171), and the Natural Science Foundation of Zhejiang Province (LGF20H180002, LY21F020027, LSD19H180003).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Liu, F., Cai, J., Huo, Y., Cheng, C., et al.: JSSR: A joint synthesis, segmentation, and registration system for 3D multi-domain image alignment of large-scale pathological CT scans (ECCV), July 2020
Huijskens, S.C., et al.: Abdominal organ position variation in children during image-guided radiotherapy. In: Radiat Oncol 13, vol. 173 (2018)
Zhang, Y., Jiang, F., Shen, R.: Region-based face alignment with convolution neural network cascade. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, E.S. (eds.) ICONIP 2017. LNCS, vol. 10636, pp. 300–309. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70090-8_31
Jaderberg, M., Simonyan, K., et al.: Spatial transformer networks. In: Advances in Neural Information Processing Systems, pp. 2017–2025 (2015)
Cao, X., Yang, J., Gao, Y., Guo, Y., Wu, G., Shen, D.: Dual-core steered non-rigid registration for multi-domain images via bi-directional image synthesis. In: Medical Image Analysis, vol. 41, pp. 18–31 (2017)
Fan, J., Cao, X., Wang, Q., et al.: Adversarial learning for mono- or multi-domain registration. Med. Image Anal. 58, 101545 (2019)
Dubost, F., Bruijne, M.D., Nardin, M., et al.: Multi-atlas image registration of clinical data with automated quality assessment using ventricle segmentation. Med. Image Anal. 63, 101698 (2020)
Blendowski, M., Hansen L., Heinrich, M.P.: Weakly-supervised learning of multi-domain features for regularised iterative descent in 3D image registration. Med. Image Anal. 67, 101822 (2021)
Zhu, J., Park, T., Isola, P., Efros, A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International Conference on Computer Vision, pp. 2242–2251 (2017)
Liu, M., Thomas, B., Jan, K.: Unsupervised image-to-image translation networks. In: Annual Conference on Neural Information Processing Systems, pp. 701–709 (2017)
Wei, D., Ahmad, S., Huo, J., et al.: SLIR: synthesis, localization, inpainting, and registration for image-guided thermal ablation of liver tumors. Med. Image Anal. 65, 101763 (2020)
Qin, C., Shi, B., Liao, R., Mansi, T., Rueckert, D., Kamen, A.: Unsupervised deformable registration for multi-modal images via disentangled representations. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 249–261. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_19
Xu, Z., et al.: Adversarial uni- and multi-modal stream networks for multimodal image registration. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12263, pp. 222–232. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59716-0_22
Zhou, B., Augenfeld, Z., Chapiro, J., Zhou, S.K., Liu, C., Duncan, J.S.: Anatomy-guided multidomain registration by learning segmentation without ground truth: application to intraprocedural CBCT/MR liver segmentation and registration. Med. Image Anal. 71, 102041 (2021)
Arar, M., Ginger, Y., et al.: Unsupervised multi-domain image registration via geometry preserving image-to-image translation (CVPR), March 2020
Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 38(8), 1788–1800 (2019)
Goodfellow, I., et al.: Generative adversarial nets. In: Proceedings of the International Conference on Neural Information Processing Systems, Montr šŠal, Canada, pp. 2672–2680 (2014)
Heusel, M., Ramsauer, H., et al.: GANs trained by a two time-scale update rule converge to a local Nash equilibrium (ANIPS), June 2017
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)
Kim, B., Kim, D.H., Park, S.H., Kim, J., Lee, J., Ye, J.C.: CycleMorph: cycle consistent unsupervised deformable image registration. Med. Image Anal. 71, 102036 (2021)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, Y., Lu, Z., Yang, XH., Hu, H., Guan, Q., Chen, F. (2021). Multi-domain Abdomen Image Alignment Based on Joint Network of Registration and Synthesis. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13110. Springer, Cham. https://doi.org/10.1007/978-3-030-92238-2_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-92238-2_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92237-5
Online ISBN: 978-3-030-92238-2
eBook Packages: Computer ScienceComputer Science (R0)