Skip to main content
SpringerLink
Log in

Multimodal medical volumetric image fusion using 3-D Shearlet transform and T-S fuzzy reasoning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Multimodal medical volumetric image fusion is a hot topic in medical image processing. Currently, most multi-scale based medical image fusion methods are put forward in two-dimensional space. However, they often fail to deal with multimodal medical volumetric image fusion due to the inaccurate image representation caused by ignoring the correlation between adjacent slices and the inappropriate design of fusion rule based on individual feature. To overcome the above drawbacks, a novel multimodal fusion method using 3-D Shearlet transform and T-S fuzzy reasoning is proposed, named as 3DSTSF. Firstly, the low frequency subbands and high frequency subbands of multimodal medical volumetric images are obtained by using the 3-D Shearlet transform. For comprehensive interpretation of source image, a contextual hidden Markov model is established for 3-D Shearlet transform high frequency subbands to model multiple dependency relationship among coefficients. Then, a fuzzy reasoning rule based on contextual hidden Markov model statistical characteristics, interval type-2 fuzzy entropy and region energy of high-frequency coefficients is designed to fuse high frequency subbands, which can accurately describe volumetric images and avoid introducing false information. Besides, a novel and simple local energy based fusion rule is performed on low frequency subbands to ensure the visual quality of fused image. Finally, the fused medical volumetric image is reconstructed by the inverse 3-D Shearlet transform. A series of experimental results demonstrate the superiority of 3DSTSF.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Bhatnagar G, Wu QM, Liu Z (2013) Directive contrast based multimodal medical image fusion in NSCT domain. IEEE Trans Multimedia 15(5):1014–1024

    Article  Google Scholar 

  2. Crouse MS, Baraniuk RG (1997) Contextual hidden Markov models for wavelet-domain signal processing. In: Conference Record of the Thirty-First Asilomar Conference on Signals, Systems and Computers, pp 95–100

    Google Scholar 

  3. Crouse MS, Nowak RD, Baraniuk RG (1998) Wavelet-based statistical signal processing using hidden Markov models. IEEE Trans Signal Process 46(4):886–902

    Article  MathSciNet  Google Scholar 

  4. Deng T, Wang Z, Wang P (2012) Study on fuzzy entropy of type-2 fuzzy sets. Control and Decision 27(3):408–412

    MathSciNet  MATH  Google Scholar 

  5. Du S, Liu C, Huang D (2015) A shearlet-based separation method of 3d engineering surface using high definition metrology. Precis Eng 40:55–73

    Article  Google Scholar 

  6. Fu K, Fan D-P, Ji G-P, Zhao Q, Shen J, Zhu C (2020) Siamese network for RGB-D salient object detection and beyond. IEEE Trans Pattern Anal Mach Intell 14(8):1–18

    Google Scholar 

  7. Haribabu M, Bindu CH, Prasad KS (2012) Multimodal medical image fusion of MRI-PET using wavelet transform. In: International Conference on Advances in Mobile Network, Communication and its Applications, pp 127–130. https://doi.org/10.1109/MNCApps.2012.33

    Chapter  Google Scholar 

  8. Huang F, Zhang X, Xu J, Zhao Z, Li Z (2019) Multi-modal learning of social image representation by exploiting social relations. IEEE Trans Cybernetics:1–12

  9. Johnson KA, Becker JA (1999) The whole brain altas. Bmj 319(7223):1507

    Article  Google Scholar 

  10. Li S, Kang X, Hu J (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875

    Article  Google Scholar 

  11. Li XS, Zhou FQ, Tan HS, Zhang W, Zhao C (2021) Multimodal medical image fusion based on joint bilateral filter and local gradient energy. Inf Sci 569:302–325

    Article  MathSciNet  Google Scholar 

  12. Li XS, Zhou FQ, Tan HS (2021) Joint image fusion and denoising via three-layer decomposition and sparse representation. Knowl-Based Syst 224:107087

    Article  Google Scholar 

  13. Ma J, Ma Y, Li C (2019) Infrared and visible image fusion methods and applications: a survey. Information Fusion 45:153–178

    Article  Google Scholar 

  14. Marchi VA, Rojas FR, Louzada F (2012) The chi-plot and its asymptotic confidence interval for analyzing bivariate dependence: an application to the average intelligence and atheism rates across nations data. Journal of Data Science 10(4):711–722

    Article  MathSciNet  Google Scholar 

  15. Melin P, Gonzalez CI, Castro JR, Mendoza O, Castillo O (2014) Edge-detection method for image processing based on generalized type-2 fuzzy logic. IEEE Trans Fuzzy Syst 22(6):1515–1525

    Article  Google Scholar 

  16. Po DY, Do MN (2006) Directional multiscale modeling of images using the contourlet transform. IEEE Trans Image Process 15(6):1610–1620

    Article  MathSciNet  Google Scholar 

  17. Sahu A, Bhateja V, Krishn A (2014) Medical image fusion with Laplacian pyramids. In: IEEE International Conference on Medical Imaging, m-Health and Emerging Communication Systems (MedCom), pp 448–453

    Google Scholar 

  18. Shen R, Cheng I, Basu A (2013) Cross-scale coefficient selection for volumetric medical image fusion. IEEE Trans Biomed Eng 60(4):1069–1079

    Article  Google Scholar 

  19. Srivastava R, Prakash O, Khare A (2016) Local energy-based multimodal medical image fusion in curvelet domain. IET Comput Vis 10(6):513–527

    Article  Google Scholar 

  20. Takagi T, Sugeno M (1985) Fuzzy identification of systems and its applications to modeling and control. IEEE Transactions on Systems Ystems, Man, and Cybernetics SMC-15(1):116–132

    Article  MATH  Google Scholar 

  21. Wang ZB, Ma Y (2008) Medical image fusion using m-PCNN. Information Fusion 9(2):176–185

    Article  MathSciNet  Google Scholar 

  22. Wang L, Li B, Tian L (2014) Multimodal medical volumetric data fusion using 3-D discrete shearlet transform and global-to-local rule. IEEE Trans Biomed Eng 61(1):197–206

    Article  Google Scholar 

  23. Wang S, Li J, Zhang C (2017) A robust algorithm of encrypted medical volume data retrieval based on 3D DWT and 3D DFT. In: IEEE international conference on software engineering research, management and applications (SERA), pp 143–149

    Google Scholar 

  24. Wang Z, Li X, Duan H, Su Y, Zhang X, Guan X (2021) Medical image fusion based on convolutional neural networks and non-subsampled contourlet transform. Expert Syst Appl 171:114574

    Article  Google Scholar 

  25. Wu D (2012) Twelve considerations in choosing between Gaussian and trapezoidal membership functions in interval type-2 fuzzy logic controllers. In: IEEE International Conference on Fuzzy Systems, pp 1–8

    Google Scholar 

  26. Xu H, Ma J, Jiang J, Guo X, Ling H (2020) U2Fusion: A Unified Unsupervised Image Fusion Network. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, pp 1–17

    Google Scholar 

  27. Yang Y, Lu H, Huang S, Tu W (2020) Remote sensing image fusion based on fuzzy logic and salience measure. IEEE Geosci Remote Sens Lett 17(11):1943–1947

    Article  Google Scholar 

  28. Yu L, Xun C, Ward RK (2016) Image fusion with convolutional sparse representation. IEEE Signal Processing Letters 99:1–1

    Google Scholar 

  29. Yu J, Chen L, Zhou S, Wang L, Huang S (2020) Adaptive image denoising for speckle noise images based on fuzzy logic. Int J Imaging Syst Technol 30(4):1132–1142

    Article  Google Scholar 

  30. Zhang H, Luo X, Wu X (2014) Statistical Modeling of Multi-modal Medical Image Fusion Method Using C-CHMM and M-PCNN. In: International Conference on Pattern Recognition, pp 1067–1072

    Google Scholar 

  31. Zhang Y, Liu Y, Sun P, Yan H, Zhao X, Zhang L (2020) IFCNN: a general image fusion framework based on convolutional neural network. Information Fusion 54:99–118

    Article  Google Scholar 

  32. Zhang Z, Xi X, Luo X et al (2020) Multimodal image fusion based on global-regional-local rule in NSST domain. Multimed Tools Appl 5:1–27

    Google Scholar 

  33. Zhou T, Huazhu F, Chen G, Shen J, Shao L (2020) Hi-net: hybrid-fusion network for multi-modal MR image synthesis. IEEE Trans Med Imaging 39(9):2772–2781

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 61772237, 61906087, in part by the Six Talent Climax Foundation of Jiangsu under Grant XYDXX-030 and Translational Medicine Special Project of Wuxi Health and Safety Commission under Grant ZZ002 and Natural Science Foundation of Jiangsu Province under Grant BK20180692. Thanks are due to Miss Anqi Wang of Jiangnan university for assistance with the experiments.

Code availability

https://github.com/qqchong/Multimodal-medical-volumetric-image-fusion-using-3-D-Shearlet-transform-and-T-S-Fuzzy-Reasoning

Author information

Authors and Affiliations

  1. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, 214122, China

    Xiaoqing Luo, Xinxing Xi, Yuting Jiang & Xiao-Jun Wu

  2. School of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou, 215009, China

    Zhancheng Zhang

  3. Affiliated Hospital, Dept Thorac Surg, Jiangnan University, Wuxi, 214122, China

    Qingjun You

  4. College of Electrical Engineering and Control Science, Nanjing Tech University, Nanjing, 211816, China

    Jing Dong

Authors
  1. Xiaoqing Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinxing Xi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhancheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuting Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qingjun You
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jing Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xiao-Jun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhancheng Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Luo, X., Xi, X., Zhang, Z. et al. Multimodal medical volumetric image fusion using 3-D Shearlet transform and T-S fuzzy reasoning. Multimed Tools Appl 82, 22577–22612 (2023). https://doi.org/10.1007/s11042-022-14266-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14266-0

Keywords

Search

Navigation