Abstract
An image fusion based on multimodal medical images renders a considerable enhancement in the quality of fused images. An effective image fusion technique produces output images by preserving all the viable and prominent information gathered from the source images without any introduction of flaws or unnecessary distortions. This review paper intends to bring out the process of image fusion, its utilization in the medical domain, merits, and demerits and reviews the perspective of multimodal medical image fusion. It also discusses the involvement of various medical entities like medical resonance imaging (MRI), positron emission tomography (PET), and computed tomography (CT). The usefulness of such modalities is presented, suggesting plausible hybrid modality combinations which could greatly enhance image fusion. This review also discusses innovative dispositions in the medical image fusion techniques for the achievement of incisively desired, quality images focused on fusion with wavelet transform and use of independent component analysis (ICA) and principal component analysis (PCA) techniques for the purpose denoising and data dimension reductions. Additionally, the future-prospects of an ideal technique for medical image fusion through the utilization of various medical modalities have been also discussed in this review paper.
Similar content being viewed by others
References
Aggarwal JK, editor (2013) Multisensor fusion for computer vision. Springer Science & Business Media 99
Prakash O, Khare A (2015) CT and MR images fusion based on stationary wavelet transform by modulus maxima. In: Computational vision and robotics. Springer, New Delhi, pp 199–204
Mitchell HB (2010) Image fusion: theories, techniques and applications. Springer Sci Bus Media:9–17. https://doi.org/10.1007/978-3-642-11216-4_2
Amini N, Fatemizadeh E, Behnam H (2014) MRI-PET image fusion based on NSCT transform using local energy and local variance fusion rules. J Med Eng Technol 38(4):211–219
Yang B, Jing ZL, Zhao HT (2010) Review of pixel-level image fusion. J Shanghai Jiaotong Univ (Science) 15(1):6–12. https://doi.org/10.1007/s12204-010-7186-y
Singh R, Khare A (2014) Redundant discrete wavelet transform based medical image fusion. In: Advances in signal processing and intelligent recognition systems. Springer, Cham, pp 505–515. https://doi.org/10.1007/978-3-319-04960-1_44
Kaur R, Kaur EG (2015) Medical image fusion using redundant wavelet based ICA co-variance analysis. Int J Eng Comp Sci 4(08):28. https://doi.org/10.18535/ijecs/v4i8.45
Liu X, Mei W, Du H (2018) Detail-enhanced multimodality medical image fusion based on gradient minimization smoothing filter and shearing filter. Med Biol Eng Comput 56(9):1565–1578. https://doi.org/10.1007/s11517-018-1796-1
Bhateja V, Patel H, Krishn A, Sahu A, Lay-Ekuakille A (2015) Multimodal medical image sensor fusion framework using cascade of wavelet and contourlet transform domains. IEEE Sensors J 15(12):6783–6790. https://doi.org/10.1109/JSEN.2015.2465935
Bhatnagar G, Wu QJ, Liu Z (2015) A new contrast based multimodal medical image fusion framework. Neurocomputing 157:143–152. https://doi.org/10.1016/j.neucom.2015.01.025
Vadhi R, Kilari V, Samayamantula S (2012) Uniform based approach for image fusion. In: Mathew J, Patra P, Pradhan D K, Kuttyamma A J (eds) Eco-friendly computing and communication systems. ICECCS 2012. Communications in Computer and Information Science, springer, Berlin, Heidelberg;305. https://doi.org/10.1007/978-3-642-32112-2_23
Kusuma J, Murthy KN (2015) Fusion of medical image by using STSVD—a survey. Int J Eng Res Gen Sci 3:571–577. https://doi.org/10.1007/978-981-10-3156-4_7
Bindu CH, Prasad KS (2018) Automatic region segmentation and variance based multimodal medical image fusion. In: Cognitive science and health bioinformatics. Springer, Singapore, pp 57–63. https://doi.org/10.1007/978-981-10-6653-5_5
Pohl C, Nazirun NN, Tamin SS Multimodal medical image fusion in cardiovascular applications. In Medical Imaging Technology, Springer, Singapore. 2015;91–109. https://doi.org/10.1007/978-981-287-540-2_4
Wu D, Yang A, Zhu L, Zhang C (2014) Survey of multi-sensor image fusion. In: In International Conference on Life System Modeling and Simulation and International Conference on Intelligent Computing for Sustainable Energy and Environment. Springer, Berlin, pp 358–367. https://doi.org/10.1007/978-3-662-45283-7_37
Salimi-Khorshidi G, Douaud G, Beckmann CF, Glasser MF, Griffanti L, Smith SM (2014) Automatic denoising of functional MRI data: combining independent component analysis and hierarchical fusion of classifiers. Neuroimage. 90:449–468
Cui Z, Zhang G, Wu J. International Joint Conference on Medical image fusion based on wavelet transform and independent component analysis. In Artificial Intelligence, 2009. JCAI'09. 2009;480–483
Egfin Nirmala D, Paul ABS, Vaidehi V (2013) Improving independent component analysis using support vector machines for multimodal image fusion. J Comput Sci 9:1117–1132
Desale RP, Verma SV. Study and analysis of PCA, DCT & DWT based image fusion techniques. In 2013 International Conference on Signal Processing Image Processing & Pattern Recognition (ICSIPR), 2013;66–69
Kaur N, Bahl M, Kaur H (2014) Introduce review on: Image Fusion Using Wavelet and Curvelet Transform (IJCSIT). Int J Comp Sci Inform Technol 5(2):2467–2470
Krishn A, Bhateja V, Sahu A. Medical image fusion using a combination of PCA and wavelet analysis. In 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI), 2014;986–991
Pradhan S, Patra D, Singh A. Image registration of medical images using ripplet transform. In Proceedings of International Conference on Computer Vision and Image Processing, Springer, Singapore 2017:487–494. DOI: https://doi.org/10.1007/978-981-10-2107-7_44
Oliveira FP, Tavares JMR (2014) Medical image registration: a review. Comp Methods Biomech Biomed Eng 17(2):73–93
Chavan S, Pawar A, Talbar S. Multimodality medical image fusion using rotated wavelet transform. In International Conference on Communication and Signal Processing 2016 (ICCASP 2016) 2016
Mohammed HA, Hassan MA (2016) The image registration techniques for medical imaging (MRI-CT). Am J Biomed Eng 6(2):53–58
Dasarathy BV (2012) Information fusion in the realm of medical applications-a bibliographic glimpse at its growing appeal. Inform Fusion 13(1):1–9. https://doi.org/10.1016/j.inffus.2011.06.003
Srivastava A, Bhateja V, Moin A. Combination of PCA and contourlets for multispectral image fusion. In Proceedings of the international conference on data engineering and communication technology Springer, Singapore 2017;577–585
Mishra HO, Bhatnagar S, Shukla A, Tiwari A (2014) Medical image fusion based on wavelet transform. Int J Sci Eng Res 5(2):772–778
Wang N, Ma Y, Zhan K, Yuan M. (2013) Multimodal medical image fusion framework based on simplified PCNN in nonsubsampled contourlet transform domain. J Multimed;8(3)
Twycross J, Aickelin U (2010) Information fusion in the immune system. Information Fusion 11(1):35–44. https://doi.org/10.1016/j.inffus.2009.04.008
Lei JB, Yin JB, Shen HB. Feature fusion and selection for recognizing cancer-related mutations from common polymorphisms. In IEEE 2010 Chinese Conference on Pattern Recognition (CCPR) 2010;1–5. https://doi.org/10.1109/CCPR.2010.5659154
Darwish SM (2013) Multi-level fuzzy contourlet-based image fusion for medical applications. IET Image Process 7(7):694–700
James AP, Dasarathy BV (2014) Medical image fusion: a survey of the state of the art. Information Fusion 19:4–19. https://doi.org/10.1016/j.inffus.2013.12.002
Li Y, Verma R (2011) Multichannel image registration by feature-based information fusion. IEEE Trans Med Imaging 30(3):707–720. https://doi.org/10.1109/TMI.2010.2093908
Umaamaheshvari A, Thanushkodi K (2013) Medical image watermarking using multi ridgelet and Fast ICA. Int J Comp Appl Eng Sci 3(1):9
Soomro TA, Khan TM, Khan MA, Gao J, Paul M, Zheng L (2018) Impact of ICA-based image enhancement technique on retinal blood vessels segmentation. IEEE Access 6:3524–3538
Rani K, Sharma R (2013) Study of different image fusion algorithm. Int J Emerg Technol Adv Eng 3(5):288–291
Guo Q, Dong F, Sun S et al (2013) Image denoising algorithm based on contourlet transform for optical coherence tomography heart tube image [J]. Image Proc 7(5):442–450
Solanki Chetan K, Patel NM. (2011) Pixel based and wavelet based image fusion methods with their comparative study. In National conference on recent trends in engineering & technology Vol. 13
W. Hao-quan, X. Hao, IEEE, International Symposium on multi-mode medical image fusion algorithm based on principal component analysis, in: Computer Network and Multimedia Technology, CNMT 2009. 2009;1–4
Al-Azzawi N, Abdullah WA. (2009) In Proceedings of the Annual International Conference of the IEEE on Medical Image Fusion Schemes using Contourlet Transform and PCA Based 5813–5816
Gong S, Liu C, Ji Y, Zhong B, Li Y, Dong H (2019) Image fusion. In: In Advanced Image and Video Processing Using MATLAB. Springer, Cham, pp 233–269
He C, Liu Q, Li H, Wang H (2010) Multimodal medical image fusion based on IHS and PCA. Proc Eng 7:280–285. https://doi.org/10.1016/j.proeng.2010.11.045
Nawaz Q, Xiao B, Hamid I, Jiao D (2016) Multi-modal color medical image fusion using quaternion discrete Fourier transform. Sens Imaging 17(1):7
Calhoun VD, Adali T. ICA for fusion of brain imaging data. In Signal Processing Techniques for Knowledge Extraction and Information Fusion, Springer, Boston, MA 2008;221–240. https://doi.org/10.1007/978-0-387-74367-7_12
Rajalingam B, Priya R (2017) Multimodality medical image fusion based on hybrid fusion techniques. Int J Eng Manuf Sci 7(1):22–29
Liu Z, Feng Y, Zhang Y, Li X (2016) A fusion algorithm for infrared and visible images based on RDU-PCNN and ICA-bases in NSST domain. Infrared Phys Technol 79:183–190
Budhiraja S (2016) Multimodal medical image fusion based on guided filtered multi-scale decomposition [J]. Int J Biomed Eng Technol 20(4):285
Li H, He X, Tao D, Tang Y, Wang R (2018) Joint medical image fusion, denoising and enhancement via discriminative low-rank sparse dictionaries learning. Pattern Recogn 79:130–146. https://doi.org/10.1016/j.patcog.2018.02.005
Adali T, Anderson M, Fu GS (2014) Diversity in independent component and vector analyses: identifiability, algorithms, and applications in medical imaging. IEEE Signal Process Mag 31(3):18–33
Lu H, Zhang L, Serikawa S (2012) Maximum local energy: an effective approach for multisensor image fusion in beyond wavelet transform domain. Comput Math Appl 64(5):996–1003. https://doi.org/10.1016/j.camwa.2012.03.017
Mehra I, Nishchal NK (2014) Image fusion using wavelet transform and its application to asymmetric cryptosystem and hiding. Opt Express 22(5):5474–5482
Pavithra C, Bhargavi DS (2013) Fusion of two images based on wavelet transform. Int J Innov Res Sci Eng Technol 2(5):1814–1819
Tian J, Chen L (2012) Adaptive multi-focus image fusion using a wavelet-based statistical sharpness measure. Signal Process 92(9):2137–2146
Grace SR, Sheela MI. (2014) A study on image fusion techniques of complementary medical images. International Journal of Advanced Research in Computer Science 5(6)
Günzel K, Cash H, Buckendahl J, Königbauer M, Asbach P, Haas M, Neymeyer J, Hinz S, Miller K, Kempkensteffen C (2017) The addition of a sagittal image fusion improves the prostate cancer detection in a sensor-based MRI/ultrasound fusion guided targeted biopsy. BMC Urol 17(1):7
Geng P, Sun X, Liu J (2017) Adopting quaternion wavelet transform to fuse multi-modal medical images. J Med Biol Eng 37(2):230–239. https://doi.org/10.1007/s40846-016-0200-6
Ganasala P, Kumar V (2014) Multimodality medical image fusion based on new features in NSST domain. Biomed Eng Lett 4(4):414–424
Cha DI, Lee MW, Kim AY, Kang TW, Oh YT, Jeong JY, Chang JW, Ryu J, Lee KJ, Kim J, Bang WC (2017) Automatic image fusion of real-time ultrasound with computed tomography images: a prospective comparison between two auto-registration methods. Acta Radiol 58(11):1349–1357
Palkar B, Mishra D. Fusion of multimodal lumbar spine images using Kekre’s wavelet transform. In Ambient Communications and Computer Systems Springer, Singapore 2018;659–669
Sandhya S, Kumar MS, Karthikeyan L (2019) A hybrid fusion of multimodal medical images for the enhancement of visual quality in medical diagnosis. In: In Computer aided intervention and diagnostics in clinical and medical images. Springer, Cham, pp 61–70
Serikawa S, Lu H, Li Y, Zhang L, Yang S, Yamawaki A, Nakashima S, Kitazono Y. (2012) Multimodal medical image fusion in extended contourlet transform domain. In Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, Springer, Berlin, Heidelberg. 2013;215–226
Liu Z, Yin H, Chai Y, Yang SX (2014) A novel approach for multimodal medical image fusion. Expert Syst Appl 41(16):7425–7435. https://doi.org/10.1016/j.eswa.2014.05.043
Gambhir D, Manchanda M (2018) Wave-atom transform-based multimodal medical image fusion. SIViP:1–9
Singh V, Verma NK, Islam ZU, Cui Y (2019) Feature learning using stacked autoencoder for shared and multimodal fusion of medical images. In: In Computational intelligence: theories, applications and future directions, 1st edn. Springer, Singapore, pp 53–66
Aktar MN, Lambert AJ, Pickering M (2018) An automatic fusion algorithm for multi-modal medical images. Comp Methods Biomech Biomed Eng: Imaging & Visualization 6(5):584–598
Miao QG, Shi C, Xu PF, Yang M, Shi YB (2011) A novel algorithm of image fusion using shearlets. Opt Commun 284(6):1540–1547
Singh R, Khare A (2013) Multiscale medical image fusion in wavelet domain. Sci World J 2013:10
Singh AK, Kumar B, Dave M, Mohan A (2015) Multiple watermarking on medical images using selective discrete wavelet transform coefficients. J Med Imag Health Inform 5(3):607–614
Yang Y, Park DS, Huang S, Rao N (2010) Medical image fusion via an effective wavelet-based approach. EURASIP J Adv Signal Process 2010:44. https://doi.org/10.1155/2010/579341
Benjamin JR, Jayasree T (2018) Improved medical image fusion based on cascaded PCA and shift invariant wavelet transforms. Int J Comput Assist Radiol Surg 13(2):229–240. https://doi.org/10.1007/s11548-017-1692-4
Sui J, Adali T, Pearlson G, Yang H, Sponheim SR, White T, Calhoun VD (2010) A CCA+ ICA based model for multi-task brain imaging data fusion and its application to schizophrenia. Neuroimage. 51(1):123–134. https://doi.org/10.1016/j.neuroimage.2010.01.069
Krishn A, Bhateja V, Sahu A (2015) PCA based medical image fusion in ridgelet domain. In: In Proceedings of the 3rd International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA). Springer, Cham, pp 475–482
Zhou Y, Mayyas A, Omar MA (2011) Principal component analysis-based image fusion routine with application to automotive stamping split detection. Res Nondestruct Eval 22(2):76–91. https://doi.org/10.1080/09349847.2011.553348
Mamatha S, Gayatri L (2012) An image fusion using wavelet and curvelet transforms. Global J Adv Eng Technol 1(2):69–73
Singh R, Srivastava R, Prakash O, Khare A. (2012) Mixed scheme based multimodal medical image fusion using Daubechies Complex Wavelet Transform. In 2012 International Conference on Informatics, Electronics & Vision (ICIEV) 304–309
Nithya R, Elayaraja S (2015) Medical image fusion schemes using contourlet transform and PCA bases. Asian J Electr Sci 4(1):27–33
Asaithambi N, Kayalvizhi R, Selvi W. 3D multimodal medical image fusion and evaluation of diseases. In Proceedings of the International Conference on Soft Computing Systems Springer, New Delhi 2016;415–425
Nemec SF, Peloschek P, Schmook MT, Krestan CR, Hauff W, Matula C, Czerny C (2010) CT–MR image data fusion for computer-assisted navigated surgery of orbital tumors. Eur J Radiol 73(2):224–229. https://doi.org/10.1016/j.ejrad.2008.11.003
El-Gamal FE, Elmogy M, Atwan A (2016) Current trends in medical image registration and fusion. Egypt Inform J 17(1):99–124
Wang L, Dong X, Cheng X, Lin S (2018) An improved coupled dictionary and multi-norm constraint fusion method for CT/MR medical images. Multimed Tools Appl:1–7
Mitra S, Shankar BU (2015) Medical image analysis for cancer management in natural computing framework. Inf Sci 306:111–131
Aklan B, Paulus DH, Wenkel E, Braun H, Navalpakkam BK, Ziegler S, Geppert C, Sigmund EE, Melsaether A, Quick HH (2013) Toward simultaneous PET/MR breast imaging: systematic evaluation and integration of a radiofrequency breast coil. Med Phys 40(2):024301
Duarte GM, Cabello C, Torresan RZ, Alvarenga M, Telles GH, Bianchessi ST, Caserta N, Segala SR, de Lima MD, de Camargo Etchebehere EC, Camargo EE (2007) Fusion of magnetic resonance and scintimammography images for breast cancer evaluation: a pilot study. Ann Surg Oncol 14(10):2903–2910. https://doi.org/10.1245/s10434-007-9476-7
Goshtasby AA (2012) Image registration: principles, tools and methods. Springer Sci Bus Media. https://doi.org/10.1007/978-1-4471-2458-0
Koley S, Galande A, Kelkar B, Sadhu AK, Sarkar D, Chakraborty C (2016) Multispectral MRI image fusion for enhanced visualization of meningioma brain tumors and edema using contourlet transform and fuzzy statistics. J Med Biol Eng 36(4):470–484
Bhateja V, Moin A, Srivastava A, Bao LN, Lay-Ekuakille A, Le DN (2016) Multispectral medical image fusion in contourlet domain for computer based diagnosis of Alzheimer’s disease. Rev Sci Instrum 87(7):074303
Malviya A, Bhirud SG (2009) Image fusion of digital images. Int J Recent Trends Eng 2(3):146
Deserno TM, Aach T, Amunts K, Hillen W, Kuhlen T, Scholl I. Advances in medical image processing 2011. DOI https://doi.org/10.1007/s00450-010-0142-0, 26, 1, 3
Chandana M, Amutha S, Kumar N (2011) A hybrid multi-focus medical image fusion based on wavelet transform. Int J Res Rev Comp Sci 2(4):948
Das S, Chowdhury M, Kundu MK (2011) Medical image fusion based on ripplet transform type-I. Prog Electromagn Res 30:355–370
Adu J, Xie S, Gan J (2016) Image fusion based on visual salient features and the cross-contrast. J Vis Commun Image Represent 40:218–224
Zhang L, Dong W, Zhang D, Shi G (2010) Two-stage image denoising by principal component analysis with local pixel grouping. Pattern Recogn 43(4):1531–1549
Akshata M, Aparna BV, Donthi S, Jain N, Chakrasali S (2016) A comparative study between contourlet and wavelet transform for medical image registration and fusion. Int J Comp Sci Netw Secur (IJCSNS) 16(5):102
Vijayarajan R, Muttan S (2015) Discrete wavelet transform based principal component averaging fusion for medical images. AEU-Int J Electr Commun 69(6):896–902
Liu W, Huang J, Zhao Y Image fusion based on PCA and undecimated discrete wavelet transform. In: In International Conference on Neural Information Processing 2006 Oct 3. Springer, Berlin, pp 481–488
Naidu VP, Raol JR (2008) Pixel-level image fusion using wavelets and principal component analysis. Def Sci J 58(3):338
Srikanth J, Sujatha CN. Image fusion based on wavelet transform for medical diagnosis. Int. Journal of Eng Research and Applications 2013 ;3(6):252–256
Mishra HO, Bhatnagar S. (2014) MRI and CT image fusion based on wavelet transform. International Journal of Information and Computation Technology. ISSN. 0974-2239
Wei H, Viallon M, Delattre BM, Moulin K, Yang F, Croisille P, Zhu Y (2015) Free-breathing diffusion tensor imaging and tractography of the human heart in healthy volunteers using wavelet-based image fusion. IEEE Trans Med Imaging 34(1):306–316
Murthy KN, Kusuma J. (2017) Fusion of medical image using ST-SVD. In Proceedings of the 5th International Conference on Frontiers in Intelligent Computing: Theory and Applications Springer, Singapore. 69–79
Biswas B, Chakrabarti A, Dey KN. (2015) Spine medical image fusion using wiener filter in shearlet domain. In 2015 2nd International Conference on Recent Trends in Information Systems (ReTIS) 387–392
Nithya R, Elayaraja S (2015) Medical image fusion scheme using contourlet transform and PCA bases. Asian J Electr Sci 4(1):27–33. https://doi.org/10.1504/IJBET.2016.076604
Pritika BS (2016) Multimodal medical image fusion based on guided filtered multi-scale decomposition. Int J Biomed Eng Technol 20(4):285–301
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yadav, S.P., Yadav, S. Image fusion using hybrid methods in multimodality medical images. Med Biol Eng Comput 58, 669–687 (2020). https://doi.org/10.1007/s11517-020-02136-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11517-020-02136-6