Skip to main content
SpringerLink
Log in

Multi-domain abdomen image alignment based on multi-scale diffeomorphic jointed network

  • Published:
Optoelectronics Letters Aims and scope Submit manuscript

Abstract

Recently, the generative adversarial network (GAN) has been extensively applied to the cross-modality conversion of medical images and has shown outstanding performance than other image conversion algorithms. Hence, we propose a novel GAN-based multi-domain registration method named multiscale diffeomorphic jointed network of registration and synthesis (MDJRS-Net). The deviation of the generator of the GAN-based approach affects the alignment phase, so a joint training strategy is introduced to improve the performance of the generator, which feedbacks the structural loss contained in the deformation field. Meanwhile, the nature of diffeomorphism can enable the network to generate deformation fields with more anatomical properties. The average dice score (Dice) is improved by 1.95% for the computer tomography venous (CTV) to magnetic resonance imaging (MRI) registration task and by 1.92% for the CTV to computer tomography plain (CTP) task compared with the other methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. LIU F, CAI J, HUO Y, et al. JSSR: a joint synthesis, segmentation, and registration system for 3D multi-modal image alignment of large-scale pathological CT scans[C]//European Conference on Computer Vision 2020, August 23–28, 2020, Glasgow, UK. Berlin, Heidelberg: Springer-Verlag, 2020: 257–274.

    Google Scholar 

  2. HUIJSKENS S C, VAN DIJK I W E M, VISSER J, et al. Abdominal organ position variation in children during image-guided radiotherapy[J]. Radiation oncology, 2018, 13(1): 1–9.

    Article  Google Scholar 

  3. ZHANG Y, JIANG F, SHEN R. Region-based face alignment with convolution neural network cascade[C]//24th International Conference on Neural Information Processing, November 14–18, 2017, Guangzhou, China. Berlin, Heidelberg: Springer-Verlag, 2017: 300–309.

    Google Scholar 

  4. JADERBERG M, SIMONYAN K, ZISSERMAN A. Spatial transformer networks[J]. Advances in neural information processing systems, 2015, 28: 2017–2025.

    Google Scholar 

  5. CAO X, YANG J, GAO Y, et al. Dual-core steered non-rigid registration for multi-modal images via bi-directional image synthesis[J]. Medical image analysis, 2017, 41: 18–31.

    Article  Google Scholar 

  6. FAN J, CAO X, WANG Q, et al. Adversarial learning for mono- or multi-domain registration[J]. Medical image analysis, 2019, 58: 101545.

    Article  Google Scholar 

  7. DUBOST F, DE BRUIJNE M, NARDIN M, et al. Multi-atlas image registration of clinical data with automated quality assessment using ventricle segmentation[J]. Medical image analysis, 2020, 63: 101698.

    Article  Google Scholar 

  8. BLENDOWSKI M, HANSEN L, HEINRICH M P. Weakly-supervised learning of multi-modal features for regularised iterative descent in 3D image registration[J]. Medical image analysis, 2021, 67: 101822.

    Article  Google Scholar 

  9. TANG T, GUAN Q, WU Y. Support vector machine incremental learning triggered by wrongly predicted samples[J]. Optoelectronics letters, 2018, 14(3): 232–235.

    Article  ADS  Google Scholar 

  10. VERCAUTEREN T, PENNEC X, PERCHANT A, et al. Diffeomorphic demons: efficient non-parametric image registration[J]. Neuroimage, 2009, 45(1): S61–S72.

    Article  Google Scholar 

  11. BEG M F, MILLER M I, TROUVÉ A, et al. Computing large deformation metric mappings via geodesic flows of diffeomorphisms[J]. International journal of computer vision, 2005, 61(2): 139–157.

    Article  Google Scholar 

  12. GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[J]. Advances in neural information processing systems, 2014, 27: 2672–2680.

    Google Scholar 

  13. ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision, October 22–29, 2017, Venice, Italy. New York: IEEE, 2017: 2223–2232.

    Google Scholar 

  14. LIU M Y, BREUEL T, KAUTZ J. Unsupervised image-to-image translation networks[J]. Advances in neural information processing systems, 2017, 30: 700–708.

    Google Scholar 

  15. QIN C, SHI B, LIAO R, et al. Unsupervised deformable registration for multi-modal images via disentangled representations[C]//International Conference on Information Processing in Medical Imaging, June 2, 2019, Hong Kong, China. Berlin, Heidelberg: Springer-Verlag, 2019: 249–261.

    Google Scholar 

  16. WEI D, AHMAD S, HUO J, et al. SLIR: synthesis, localization, inpainting, and registration for image-guided thermal ablation of liver tumors[J]. Medical image analysis, 2020, 65: 101763.

    Article  Google Scholar 

  17. XU Z, LUO J, YAN J, et al. Adversarial uni-and multi-modal stream networks for multimodal image registration[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention, October 4–8, 2020, Lima, Peru. Berlin, Heidelberg: Springer-Verlag, 2020: 222–232.

    Google Scholar 

  18. ZHOU B, AUGENFELD Z, CHAPIRO J, et al. Anatomy-guided multimodal registration by learning segmentation without ground truth: application to intra-procedural CBCT/MR liver segmentation and registration[J]. Medical image analysis, 2021, 71: 102041.

    Article  Google Scholar 

  19. BALAKRISHNAN G, ZHAO A, SABUNCU M R, et al. Voxelmorph: a learning framework for deformable medical image registration[J]. IEEE transactions on medical imaging, 2019, 38(8): 1788–1800.

    Article  Google Scholar 

  20. ARAR M, GINGER Y, DANON D, et al. Unsupervised multi-modal image registration via geometry preserving image-to-image translation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 16–18, 2020, Seattle, WA, USA. New York: IEEE, 2020: 13410–13419.

    Google Scholar 

  21. ZHAO S, DONG Y, CHANG E I, et al. Recursive cascaded networks for unsupervised medical image registration[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, October 27-November 2, 2019, Seoul, Korea (South). New York: IEEE, 2019: 10600–10610.

    Google Scholar 

  22. KIM B, KIM D H, PARK S H, et al. Cyclemorph: cycle consistent unsupervised deformable image registration[J]. Medical image analysis, 2021, 71: 102036.

    Article  Google Scholar 

  23. HEUSEL M, RAMSAUER H, UNTERTHINER T, et al. GANs trained by a two time-scale update rule converge to a local nash equilibrium[J]. Advances in neural information processing systems, 2017, 30: 6626–6637.

    Google Scholar 

  24. CHEN Y, LU Z, YANG X H, et al. Multi-domain abdomen image alignment based on joint network of registration and synthesis[C]//International Conference on Neural Information Processing, December 8–12, 2021, Bali, Indonesia. Berlin, Heidelberg: Springer-Verlag, 2021: 334–344.

    Google Scholar 

  25. SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 16–July 1, 2016, Las Vegas, USA. New York: IEEE, 2016: 2818–2826.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, 310014, China

    Zhengwei Lu, Yong Wang, Qiu Guan, Yizhou Chen, Dongchun Liu & Xinli Xu

Authors
  1. Zhengwei Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiu Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yizhou Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dongchun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xinli Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiu Guan.

Additional information

This work has been supported by the National Natural Science Foundation of China (Nos.U20A20171, 61802347, 61972347, and 61773348), and the Science Foundation of Zhejiang Province (Nos.LY21F020027, LGF20H180002, and LSD19H180003).

Statements and Declarations

The authors declare that there are no conflicts of interest related to this article.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, Z., Wang, Y., Guan, Q. et al. Multi-domain abdomen image alignment based on multi-scale diffeomorphic jointed network. Optoelectron. Lett. 18, 628–634 (2022). https://doi.org/10.1007/s11801-022-2052-3

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11801-022-2052-3

Document code

Search

Navigation