Skip to main content
SpringerLink
Log in

Fusion of multi-modality biomedical images using deep neural networks

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

With the recent advancement in the medical diagnostic tools, multi-modality medical images are extensively utilized as a lifesaving tool. An efficient fusion of medical images can improve the performance of various medical diagnostic tools. But, gathering of all modalities for a given patient is defined as an ill-posed problem as medical images suffer from poor visibility and frequent patient dropout. Therefore, in this paper, an efficient multi-modality image fusion model is proposed to fuse multi-modality medical images. To tune the hyper-parameters of the proposed model, a multi-objective differential evolution is used. The fusion factor and edge strength metrics are utilized to form a multi-objective fitness function. Performance of the proposed model is compared with nine competitive models over fifteen benchmark images. Performance analyses reveal that the proposed model outperforms the competitive fusion models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

References

  • Chen J, Zhang L, Lu L, Li Q, Hu M, Yang X (2020) A novel medical image fusion method based on rolling guidance filtering. Internet Things 14:100172

    Article  Google Scholar 

  • Corbat L, Nauval M, Henriet J, Lapayre J-C (2020) A fusion method based on deep learning and case-based reasoning which improves the resulting medical image segmentations. Expert Syst Appl 147:113200

    Article  Google Scholar 

  • Daniel E, Anitha J, Kamaleshwaran K, Rani I (2017) Optimum spectrum mask based medical image fusion using gray wolf optimization. Biomed Signal Process Control 34:36–43

    Article  Google Scholar 

  • De Luca G (2022) A survey of nisq era hybrid quantum-classical machine learning research. J Artif Intell Technol 2(1):9–15

    Google Scholar 

  • Du J, Li W, Lu K, Xiao B (2016) An overview of multi-modal medical image fusion. Neurocomputing 215:3–20

    Article  Google Scholar 

  • Du J, Li W, Tan H (2020) Three-layer medical image fusion with tensor-based features. Inf Sci 525:93–108

    Article  MathSciNet  MATH  Google Scholar 

  • El-Hoseny HM, El-Rahman WA, El-Rabaie E-SM, El-Samie FEA, Faragallah OS (2018) An efficient dt-cwt medical image fusion system based on modified central force optimization and histogram matching. Infrared Phys Technol 94:223–231

    Article  Google Scholar 

  • Fan Q, Yan X (2018) Multi-objective modified differential evolution algorithm with archive-base mutation for solving multi-objective xylene oxidation rocess. J Intell Manuf 29(1):35–49

    Article  Google Scholar 

  • Hu Q, Hu S, Zhang F (2020) Multi-modality medical image fusion based on separable dictionary learning and gabor filtering. Signal Process: Image Commun 83:115758

    Google Scholar 

  • James AP, Dasarathy BV (2014) Medical image fusion: a survey of the state of the art. Inf Fus 19:4–19

    Article  Google Scholar 

  • Jiang W, Yang X, Wu W, Liu K, Ahmad A, Sangaiah AK, Jeon G (2018) Medical images fusion by using weighted least squares filter and sparse representation. Comput Electr Eng 67:252–266

    Article  Google Scholar 

  • Jin X, Chen G, Hou J, Jiang Q, Zhou D, Yao S (2018) Multimodal sensor medical image fusion based on nonsubsampled shearlet transform and s-pcnns in hsv space. Signal Process 153:379–395

    Article  Google Scholar 

  • Karim AM, Güzel MS, Tolun MR, Kaya H, Çelebi FV (2019) A new framework using deep auto-encoder and energy spectral density for medical waveform data classification and processing. Biocybernet Biomed Eng 39(1):148–159

    Article  Google Scholar 

  • Karthik P, Sekhar K (2021) Resource scheduling approach in cloud testing as a service using deep reinforcement learning algorithms. CAAI Trans Intell Technol 6(2):147–154

    Article  Google Scholar 

  • Kaur M, Singh D (2021) Multi-modality medical image fusion technique using multi-objective differential evolution based deep neural networks. J Ambient Intell Humaniz Comput 12(2):2483–2493

    Article  Google Scholar 

  • Kaur M, Singh D, Kumar V, Gupta B, Abd El-Latif AA (2021) Secure and energy efficient-based e-health care framework for green internet of things. IEEE Trans Green Commun Netw 5(3):1223–1231

    Article  Google Scholar 

  • Kaushik H, Singh D, Kaur M, Alshazly H, Zaguia A, Hamam H (2021) Diabetic retinopathy diagnosis from fundus images using stacked generalization of deep models. IEEE Access 9:108276–108292

    Article  Google Scholar 

  • Kaushik R, Jain S, Jain S, Dash T (2021) Performance evaluation of deep neural networks for forecasting time-series with multiple structural breaks and high volatility. CAAI Trans Intell Technol 6(3):265–280. https://doi.org/10.1049/cit2.12002

    Article  Google Scholar 

  • Li Z-K, Tan J-R, Feng Y-X, Fang H (2008) Multi-objective particle swarm optimization algorithm based on crowding distance sorting and its application. Comput Integr Manuf Syst 7:1329–1336

    Google Scholar 

  • Li H, He X, Tao D, Tang Y, Wang R (2018) Joint medical image fusion, denoising and enhancement via discriminative low-rank sparse dictionaries learning. Pattern Recogn 79:130–146

    Article  Google Scholar 

  • Liu J, Liu Z, Sun C, Zhuang J (2022) A data transmission approach based on ant colony optimization and threshold proxy re-encryption in wsns. J Artif Intell Technol 2(1):23–31

    Google Scholar 

  • Manchanda M, Sharma R (2018) An improved multimodal medical image fusion algorithm based on fuzzy transform. J Vis Commun Image Represent 51:76–94

    Article  Google Scholar 

  • Maqsood S, Javed U (2020) Multi-modal medical image fusion based on two-scale image decomposition and sparse representation. Biomed Signal Process Control 57:101810

    Article  Google Scholar 

  • Padmavathi K, Asha C, Maya VK (2020) A novel medical image fusion by combining tv-l1 decomposed textures based on adaptive weighting scheme. Eng Sci Technol, Int J 23(1):225–239

    Google Scholar 

  • Polinati S, Dhuli R (2020) Multimodal medical image fusion using empirical wavelet decomposition and local energy maxima. Optik 205:163947

    Article  Google Scholar 

  • Prakash O, Park CM, Khare A, Jeon M, Gwak J (2019) Multiscale fusion of multimodal medical images using lifting scheme based biorthogonal wavelet transform. Optik 182:995–1014

    Article  Google Scholar 

  • Prakash O, Park CM, Khare A, Jeon M, Gwak J (2019) Multiscale fusion of multimodal medical images using lifting scheme based biorthogonal wavelet transform. Optik 182:995–1014

    Article  Google Scholar 

  • Rajalingam B, Priya R, Bhavani R (2019) Hybrid multimodal medical image fusion using combination of transform techniques for disease analysis. Procedia Comput Sci 152:150–157 (international Conference on Pervasive Computing Advances and Applications- PerCAA 2019)

  • Ravi P, Krishnan J (2018) Image enhancement with medical image fusion using multiresolution discrete cosine transform. Materials Today: Proceedings 5(1, Part 1), 1936 – 1942. international Conference on Processing of Materials, Minerals and Energy (July 29th - 30th) 2016, Ongole, Andhra Pradesh, India

  • Rezaeipanah A, Mojarad M (2021) Modeling the scheduling problem in cellular manufacturing systems using genetic algorithm as an efficient meta-heuristic approach. J Artif Intell Technol 1(4):228–234

    Google Scholar 

  • Sheng Guan J, Kang S bo, Sun Y (2019) Medical image fusion algorithm based on multi-resolution analysis coupling approximate spare representation. Futur Gener Comput Syst 98:201–207

    Article  Google Scholar 

  • Singh S, Anand R (2018) Ripplet domain fusion approach for ct and mr medical image information. Biomed Signal Process Control 46:281–292

    Article  Google Scholar 

  • Singh D, Kumar V, Kaur M, Jabarulla MY, Lee H-N (2021) Screening of covid-19 suspected subjects using multi-crossover genetic algorithm based dense convolutional neural network. IEEE Access 9:142566–142580

  • Singh D, Kaur M, Jabarulla MY, Kumar V, Lee H-N (2022) Evolving fusion-based visibility restoration model for hazy remote sensing images using dynamic differential evolution. IEEE Trans Geosci Remote Sens. https://doi.org/10.1109/TGRS.2022.3155765

  • Ullah H, Ullah B, Wu L, Abdalla FY, Ren G, Zhao Y (2020) Multi-modality medical images fusion based on local-features fuzzy sets and novel sum-modified-laplacian in non-subsampled shearlet transform domain. Biomed Signal Process Control 57:101724

    Article  Google Scholar 

  • Wang X, Zheng Z, He Y, Yan F, Zeng Z, Yang Y (2020) Progressive local filter pruning for image retrieval acceleration, arXiv preprint arXiv:2001.08878

  • Wani A, Khaliq R (2021) Sdn-based intrusion detection system for iot using deep learning classifier (idsiot-sdl). CAAI Trans Intell Technol 6(3):281–290

    Article  Google Scholar 

  • Xia K-J, Yin H-S, Wang J-Q (2019) A novel improved deep convolutional neural network model for medical image fusion. Clust Comput 22(1):1515–1527

    Article  MathSciNet  Google Scholar 

  • Xu L, Si Y, Jiang S, Sun Y, Ebrahimian H (2020) Medical image fusion using a modified shark smell optimization algorithm and hybrid wavelet-homomorphic filter. Biomed Signal Process Control 59:101885

    Article  Google Scholar 

  • Xydeas C, Petrovic V (2000) Objective image fusion performance measure. Electron Lett 36(4):308–309

    Article  Google Scholar 

  • Zhou T, Fu H, Chen G, Shen J, Shao L (2020) Hi-net: hybrid-fusion network for multi-modal mr image synthesis. IEEE Trans Med Imaging 39(9):2772–27781

    Article  Google Scholar 

  • Zhu Z, Chai Y, Yin H, Li Y, Liu Z (2016) A novel dictionary learning approach for multi-modality medical image fusion. Neurocomputing 214:471–482

    Article  Google Scholar 

  • Zong J. jing, Qiu T. shuang (2017) Biomedical signal processing and control. Med Image Fus Based Sparse Represent Classif Image Patches 34:195-205

    Google Scholar 

Download references

Funding

The authors have not disclosed any funding.

Author information

Authors and Affiliations

  1. Department of Computer Science & Engineering, Moradabad Institute of Technology, Moradabad, 244001, India

    Manish Gupta & Neha Gupta

  2. Department of Computer Science & Engineering, Maharaja Surajmal Institute of Technology, Janakpuri, New Delhi, 110058, India

    Naresh Kumar

  3. Computer Sciences Department, College of CIT, Taif University, P.O. Box 11099, Taif, 21944, Saudi Arabia

    Atef Zaguia

Authors
  1. Manish Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Naresh Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Neha Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Atef Zaguia
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MG contributed to conceptualization, methodology and software. NK performed data curation and writing—original draft preparation. NG contributed to visualization, investigation, supervision, software and validation. AZ performed writing—reviewing and editing.

Corresponding author

Correspondence to Naresh Kumar.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest regarding the publication of this paper.

Additional information

Communicated by Irfan Uddin.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gupta, M., Kumar, N., Gupta, N. et al. Fusion of multi-modality biomedical images using deep neural networks. Soft Comput 26, 8025–8036 (2022). https://doi.org/10.1007/s00500-022-07047-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-022-07047-2

Keywords

Search

Navigation