Skip to main content
SpringerLink
Log in

AQWT: adaptive quaternion wavelet transform and hybrid heuristic improvement for multi-modal medical image fusion model

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In healthcare, a fusion of multi-modal medical images is very difficult and also an important work for the planning of surgical concepts and the diagnosis of precision. The findings of deep learning-assisted for the fusion of images are present hotspot issues. The fusion of medical images includes the difficulties in little samples and the deficiency in a united end-to-end system for input of distinct modal sets. The edges of the resultant images are lost because of the high-frequency systems fusion conditions, and also, the fused image has less contrast because of the utilization of the average rule in the low-frequency systems. Hence, this task focuses on fusing multi-modal medical images employing heuristic-assisted models. The required multi-modal images are collected for the benchmark dataset and fed into the decomposing stage. Here the implemented Adaptive Quaternion Wavelet Transform (AQWT) is employed, where the parameter in the model is optimized by an enhanced Fitness-aided Election Based and Lemur Optimizer (FEBLO) model. After decomposing, the multi-modal images split into two resultant images such as low and high-frequency multi-modal images. The acquired low and high multi-modal images forwarded to the Max rule and the fuzzy logic techniques accordingly for the image fusion. Finally, the results of the two technique images are fused together and then forwarded to the inverse AQWT model for image reconstruction. The findings are contrasted with multiple conventional models to assure the developed model's efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data availability

The data underlying this article are available in Dataset: http://www.med.harvard.edu/aanlib/home.html.

References

  1. Xiao, A., Shen, B., Shi, X., Zhang, Z., Zhang, Z., Tian, J., Ji, N., Hu, Z.: Intraoperative glioma grading using neural architecture search and multi-modal imaging. IEEE Trans. Med. Imaging 41(10), 2570–2581 (2022)

    Article  PubMed  Google Scholar 

  2. Zhang, Y., Zhang, H., Xiao, L., Bai, Y., Calhoun, V.D., Wang, Y.-P.: Multi-modal imaging genetics data fusion via a hypergraph-based manifold regularization: application to schizophrenia study. IEEE Trans. Med. Imaging 41(9), 2263–2272 (2022)

    Article  PubMed  PubMed Central  Google Scholar 

  3. Fang, F., Yao, Y., Zhou, T., Xie, G., Lu, J.: Self-supervised multi-modal hybrid fusion network for brain tumor segmentation. IEEE J. Biomed. Health Inform. 26(11), 5310–5320 (2022)

    Article  PubMed  Google Scholar 

  4. Dolz, J., Gopinath, K., Yuan, J., Lombaert, H., Desrosiers, C., Ben Ayed, I.: HyperDense-Net: a hyper-densely connected cnn for multi-modal image segmentation. IEEE Trans. Med. Imaging 38(5), 1116–1126 (2019)

    Article  PubMed  Google Scholar 

  5. Ning, Z., Xiao, Q., Feng, Q., Chen, W., Zhang, Y.: Relation-induced multi-modal shared representation learning for Alzheimer’s disease diagnosis. IEEE Trans. Med. Imaging 40(6), 1632–1645 (2021)

    Article  PubMed  Google Scholar 

  6. Li, X., Zhao, J.: A novel multi-modal medical image fusion algorithm. J. Ambient Intell. Humaniz. Comput. 12, 1995–2002 (2021)

    Article  Google Scholar 

  7. Vijendran, A.S., Ramasamy, K.: Optimal segmentation and fusion of multi-modal brain images using clustering based deep learning algorithm. Meas. Sens. 27, 100691 (2023)

    Article  Google Scholar 

  8. Xiang, Z., Zhuo, Q., Zhao, C., Deng, X., Zhu, T., Wang, T., Jiang, W., Lei, B.: Self-supervised multi-modal fusion network for multi-modal thyroid ultrasound image diagnosis. Comput. Biol. Med. 150, 106164 (2022)

    Article  PubMed  Google Scholar 

  9. Wang, L., Liu, Y., Mi, J., Zhang, J.: MSE-fusion: weakly supervised medical image fusion with modal synthesis and enhancement. Eng. Appl. Artif. Intell. 119, 105744 (2023)

    Article  Google Scholar 

  10. Wang, R., Fang, N., He, Y., Li, Y., Cao, W., Wang, H.: Multi-modal medical image fusion based on geometric algebra discrete cosine transform. Adv. Appl. Clifford Algebras 32, 19 (2022)

    Article  MathSciNet  Google Scholar 

  11. Nawaz, Q., Xiao, B., Hamid, I., Jiao, D.: Multi-modal color medical image fusion using quaternion discrete fourier transform. Sens. Imaging 17, 1–15 (2016)

    Article  ADS  Google Scholar 

  12. Lei Zhang, Yu., Zhang, F.Y., Wang, X., Ji, L.: Multi-modal image fusion with the hybrid ℓ0ℓ1 layer decomposing and multi-directional filter banks. Multimed. Tools Appl. 81, 21369–21384 (2022)

    Article  Google Scholar 

  13. Nagaraja Kumar, N., Jayachandra Prasad, T., Satya Prasad, K.: An Intelligent multimodal medical image fusion model based on improved fast discrete curvelet transform and Type-2 fuzzy entropy. Int. J. Fuzzy Syst. 25, 96–117 (2023)

    Article  Google Scholar 

  14. Rizzo, G., Cattaneo, G.M., Castellone, P., Castiglioni, I., Ceresoli, G.L., Messa, C., Landoni, C., Gilardi, M.C., Arienti, R., Cerutti, S., Fazio, F.: Multi-modal medical image integration to optimize radiotherapy planning in lung cancer treatment. Ann. Biomed. Eng. 32, 1399–1408 (2004)

    Article  CAS  PubMed  Google Scholar 

  15. Simanek, J., Kubelka, V., Reinstein, M.: Improving multi-modal data fusion by anomaly detection. Auton. Robots 39, 139–154 (2015)

    Article  Google Scholar 

  16. Xia, K.-J., Yin, H.-S., Wang, J.-Q.: A novel improved deep convolutional neural network model for medical image fusion. Clust. Comput. 22, 1515–1527 (2019)

    Article  Google Scholar 

  17. Gupta, M., Kumar, N., Gupta, N., Zaguia, A.: Fusion of multi-modality biomedical images using deep neural networks. Soft Comput. 26, 8025–8036 (2022)

    Article  Google Scholar 

  18. Zhou, T., Fu, H., Chen, G., Shen, J., Shao, L.: Hi-Net: hybrid-fusion network for multi-modal MR image synthesis. IEEE Trans. Med. Imaging 39(9), 2772–2781 (2020)

    Article  PubMed  Google Scholar 

  19. Li, W., Jia, L., Du, J.: Multi-modal sensor medical image fusion based on multiple salient features with guided image filter. IEEE Access 7, 173019–173033 (2019)

    Article  Google Scholar 

  20. Xiang, L., Chen, Y., Chang, W., Zhan, Y., Lin, W., Wang, Q., Shen, D.: Deep-learning-based multi-modal fusion for fast MR reconstruction. IEEE Trans. Biomed. Eng. 66(7), 2105–2114 (2019)

    Article  ADS  Google Scholar 

  21. Diwakar, M., Shankar, A., Chakraborty, C., Singh, P., Arunkumar, G.: Multi-modal medical image fusion in NSST domain for internet of medical things. Multimed. Tools Appl. 81, 37477–37497 (2022)

    Article  Google Scholar 

  22. Zhou, Q., Ye, S., Wen, M., Huang, Z., Ding, M., Zhang, X.: Multi-modal medical image fusion based on densely-connected high-resolution CNN and hybrid transformer. Neural Comput. Appl. 34, 21741–21761 (2022)

    Article  Google Scholar 

  23. Dinh, P.-H.: Multi-modal medical image fusion based on equilibrium optimizer algorithm and local energy functions. Appl. Intell. 51, 8416–8431 (2021)

    Article  Google Scholar 

  24. Li, B., Peng, H., Wang, J.: A novel fusion method based on dynamic threshold neural P systems and nonsubsampled contourlet transform for multi-modality medical images. Signal Process. 178, 107793 (2021)

    Article  Google Scholar 

  25. Wang, L., Li, B., Tian, L.-F.: EGGDD: an explicit dependency model for multi-modal medical image fusion in shift-invariant shearlet transform domain. Elsevier Inf Fusion 19, 29–37 (2014)

    Article  Google Scholar 

  26. Trojovský, P., Dehghani, M.: A new optimization algorithm based on mimicking the voting process for leader selection. PeerJ. Comput. Sci. 8, e976 (2022)

    Article  PubMed  PubMed Central  Google Scholar 

  27. Abasi, A.K., Makhadmeh, S.N., Al-Betar, M.A., Alomari, O.A., Awadallah, M.A., Alyasseri, Z.A.A., Doush, I.A., Elnagar, A., Alkhammash, E.H., Hadjouni, M.: Lemurs optimizer: a new metaheuristic algorithm for global optimization. Appl. Sci. 12(19), 2076–3417 (2022)

    Article  Google Scholar 

  28. Zhancheng, Z., Xiaoqing, L., Mengyu, X., Zhiwen, W., Kai, L.: Medical image fusion based on quaternion wavelet transform. J. Algorithms Comput. Technol. (2020). https://doi.org/10.1177/1748302620931297

    Article  Google Scholar 

  29. Reena Benjamin, J., Jayasree, T.: Improved medical image fusion based on cascaded PCA and shift invariant wavelet transforms. Int. J. Comput. Assist. Radiol. Surg. 13, 229–240 (2018)

    Article  CAS  PubMed  Google Scholar 

  30. Seng, C.H., Bouzerdoum, A., Tivive, F.H C., Amin, M. G.: Fuzzy logic-based image fusion for multi-view through-the-Wall Radar. In: 2010 International Conference on Digital Image Computing: Techniques and Applications, pp. 423–428 (2010)

  31. Liu, R., Wang, T., Zhou, J., Hao, X., Xu, Y., Qiu, J.: Improved African vulture optimization algorithm based on quasi-oppositional differential evolution operator. IEEE Access 10, 95197–95218 (2022)

    Article  Google Scholar 

  32. Shaheen, A.M., Elattar, E.E., El-Sehiemy, R.A., Elsayed, A.M.: An improved sunflower optimization algorithm-based monte carlo simulation for efficiency improvement of radial distribution systems considering wind power uncertainty. IEEE Access 9, 2332–2344 (2021)

    Article  Google Scholar 

  33. Celik, T., Tjahjadi, T.: Image resolution enhancement using dual-tree complex wavelet transform. IEEE Geosci. Remote Sens. Lett. 7(3), 554–557 (2010)

    Article  ADS  Google Scholar 

  34. Shen, F., Wang, Y., Liu, C.: Change detection in SAR images based on improved non-subsampled shearlet transform and multi-scale feature fusion CNN. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 14, 12174–12186 (2021)

    Article  ADS  Google Scholar 

  35. Lee, T.-Y., Shen, H.-W.: Efficient local statistical analysis via integral histograms with discrete wavelet transform. IEEE Trans. Vis. Comput. Graph. 19(12), 2693–2702 (2013)

    Article  PubMed  Google Scholar 

  36. Nair, R.R., Singh, T., Basavapattana, A., Pawar, M.M.: Multi-layer, multi-modal medical image intelligent fusion. Multimed. Tools Appli. 81, 42821–42847 (2022)

    Article  Google Scholar 

  37. Nair, R.R., Singh, T., Sankar, R., Gunndu, K.: Multi-modal medical image fusion using LMF-GAN - a maximum parameter infusion technique. Soft Comput. Intell. Syst. Tech. Appl. 41(5), 5375–5386 (2021)

    Google Scholar 

  38. Nair, R.R., Singh, T.: An optimal registration on shearlet domain with novel weighted energy fusion for multi-modal medical images. Optik 225, 165742 (2021)

    Article  ADS  Google Scholar 

  39. Liu, H., Li, S., Zhu, J., Deng, K., Liu, M., Nie, L.: DDIFN: a dual-discriminator multi-modal medical image fusion network. ACM Trans. Multimed. Comput. Commun. Appl. 19(4), 1–17 (2023)

    Google Scholar 

  40. Zhang, Y., Nie, R., Cao, J., Ma, C.: Self-supervised fusion for multi-modal medical images via contrastive auto-encoding and convolutional information exchange. IEEE Comput. Intell. Mag. 18(1), 68–80 (2023)

    Article  Google Scholar 

Download references

Funding

This research did not receive any specific funding.

Author information

Authors and Affiliations

  1. Department of ECE, Jawaharlal Nehru Technological University Ananthapur, Ananthapuramu, Andhra Pradesh, India

    B. Suresh Babu (Research Scholar)

  2. Assistant Professor, Department of ECE, Srinivasa Ramanujan Institute of Technology Ananthapuramu, Rotarypuram, Anantapur, Andhra Pradesh, India

    B. Suresh Babu (Research Scholar)

  3. Department of ECE, KSRM College of Engineering, Affiliated to Jawaharlal Nehru Technological University Ananthapur, Ananthapuramu, Andhra Pradesh, India

    M. Venkatanarayana (Professor)

Authors
  1. B. Suresh Babu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Venkatanarayana
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors have made substantial contributions to conception and design, revising the manuscript, and the final approval of the version to be published. Also, all authors agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Corresponding author

Correspondence to B. Suresh Babu.

Ethics declarations

Competing interests

The authors declare no competing interests.

Conflict of interest

The authors declare no conflict of interest.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 966 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Babu, B.S., Venkatanarayana, M. AQWT: adaptive quaternion wavelet transform and hybrid heuristic improvement for multi-modal medical image fusion model. SIViP 18, 1041–1051 (2024). https://doi.org/10.1007/s11760-023-02760-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02760-3

Keywords

Search

Navigation