Skip to main content
SpringerLink
Log in

Multi-modal Medical Image Fusion Based on Geometric Algebra Discrete Cosine Transform

  • Published:
Advances in Applied Clifford Algebras Aims and scope Submit manuscript

Abstract

Multi-modal medical image fusion refers to the combination of patient area images obtained under diverse or identical imaging modalities, which improves the clinical applicability and provides more specific disease information for diagnosis. However, most of the existing image fusion algorithms usually divided color images into three channels of R, G, B for processing separately, which ignores the correlation between the channels and easily causes image information loss and blurring. This paper proposes a multi-modal color medical image fusion algorithm based on geometric algebra discrete cosine transform (GA-DCT). The GA-DCT algorithm combines the character of GA, which represents the multi-vector signal as a whole, can improve the quality of the fusion image and avoid a large number of complex operations related to encoding and decoding. Firstly, the source images are divided into several image blocks and expressed in GA multi-vector form; Secondly, we extend the traditional DCT to GA space and propose GA-DCT; Thirdly, we use GA-DCT to decompose the image to obtain AC and DC coefficients and finally a fusion algorithm are used to fuse the images. The experimental results show that the proposed algorithm can get clear and comprehensive fusion image, which also has great advantages under different compression ratios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Abdollahzadeh, M., Malekzadeh, T., Seyedarabi, H.: Multi-focus image fusion for visual sensor networks. In: 2016 24th Iranian Conference on Electrical Engineering (ICEE), pp. 1673–1677 (2016)

  2. Aishwarya, N., Abirami, S., Amutha, R.: Multifocus image fusion using discrete wavelet transform and sparse representation. In: 2016 International Conference on Wireless Communications. Signal Processing and Networking (WiSPNET). IEEE, Mar (2016)

  3. Amin-Naji, M., Aghagolzadeh, A.: MultiFocus image fusion in DCT domain using variance and energy of Laplacian and correlation coefficient for visual sensor networks. J. Artif. Intel. Data Min. 6(2), 233–250 (2018)

    Google Scholar 

  4. Amirjanov, A., Dimililer, K.: Image compression system with an optimisation of compression ratio. IET Image Process. 13(11), 1960–1969 (2019)

    Article  Google Scholar 

  5. Ardeshir Goshtasby, A.: Image fusion: advances in the state of the art. Inf. Fusion 8(2), 114–118 (2007)

    Article  Google Scholar 

  6. Arif, M., Wang, G.: Fast curvelet transform through genetic algorithm for multimodal medical image fusion. Soft Comput. 24(3), 1815–1836 (2020)

    Article  Google Scholar 

  7. Asokan, A., Anitha, J.: 2D discrete cosine transform for fusion of multitemporal satellite images. In: 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS), vol. 10, pp. 5–10 (2018)

  8. Batard, T., Berthier, M., Jean, C.S.: Clifford-Fourier transform for color image processing. Geom. Algebra Comput. 135–162 (2010)

  9. Beaulieu, M., Foucher, S., Gagnon, L.: Multi-spectral image resolution refinement using stationary wavelet transform. Remote Sens. 4032–4034 (2003)

  10. Bhatnagar, G., Wu, Q.M.J., Liu, Z.: Directive contrast based multimodal medical image fusion in NSCT domain. IEEE Trans. 15, 1014–1024 (2013)

    Google Scholar 

  11. Brain images database of Harvard Medical School. (online). http://www.med.harvard.edu/AANLIB/home.html.

  12. Buchholz, S., Sommer, G.: On Clifford neurons and Clifford multi-layer perceptrons. Neural Netw. 21(7), 925–935 (2008)

    Article  Google Scholar 

  13. Byrnes, J.: Computational Noncommutative Algebra and Applications. Springer, Dordrecht (2004)

    Google Scholar 

  14. Cao, W., F, Lyu., He, Z., Cao, G., He, Z.: Multimodal medical image registration based on feature spheres in geometric algebra. IEEE Access 21164–21172 (2018)

  15. Chapman, A., Vishne, U.: Clifford algebras of binary homogeneous forms. J. Algebra 366, 94–111 (2012)

    Article  MathSciNet  Google Scholar 

  16. Do Minh, N., Vetterli, M.: The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans. Image Process. 14(12), 2091–2106 (2005)

    Article  ADS  Google Scholar 

  17. Ebling, J., Scheuermann, G.: Clifford Fourier transform on vector fields. IEEE Trans. Vis. Comput. Graph. 11(4), 469–479 (2005)

    Article  Google Scholar 

  18. Felsberg, M.: Low-level image processing with the structure multivector. Appl. Geom. Algebra Comput. Sci. Eng. (2002)

  19. Filippo, N., Andrea, G., Stefano, B.: Remote sensing image fusion using the curvelet transform. Inf. Fusion 8(2), 143–156 (2007)

    Article  Google Scholar 

  20. Franchini, S., Gentile, A., Sorbello, F., Vassallo, G., Vitabile, S.: ConformalALU: a conformal geometric algebra coprocessor for medical image processing. IEEE Trans. Comput. 64(4), 955–970 (2015)

    Article  MathSciNet  Google Scholar 

  21. Gawari, N., Lalitha, S.: Comparative analysis of PCA, DCT and DWT based image fusion techniques. Int. J. Emerg. Res. Manag. Technol. 3(5) (2013)

  22. Guo, K., Labate, D.: Optimally sparse multidimensional representation using shearlets. SIAM J. Math. Anal. 39(1), 298–318 (2007)

    Article  MathSciNet  Google Scholar 

  23. Guruprasad, S., Kurian, M.Z., Suma, H.N.: A medical multi-modality image fusion of CT/PET with PCA, DWT methods. J. Dent. Mater. Tech. 4(2), 677–681 (2013)

    Google Scholar 

  24. Haghighat, M.B.A., Aghagolzadeh, A., Seyedarabi, H.: Multi-focus image fusion for visual sensor networks in DCT domain. Comput. Electr. Eng. 37(5), 789–797 (2011)

    Article  Google Scholar 

  25. He, G., Xing, S., He, X.: Image fusion method based on simultaneous sparse representation with non-subsampled contourlet transform. IET Comput. Vis. 13(2), 240–248 (2019)

    Article  Google Scholar 

  26. Hestenes, D.: New foundations for classical mechanics. Am. J. Phys. 58(7), 703–704 (1990)

    Article  ADS  MathSciNet  Google Scholar 

  27. Hestenes, D., Sobczyk, G.: Clifford algebra to geometric calculus a unified language for mathematics and physics by D. Hestenes and G. Sobczyk. Bull. Lond. Math. Soc. 17(3), 289–290 (1984)

    Google Scholar 

  28. Hore, A., Ziou, D.: Image quality metrics: PSNR vs. SSIM. In: 2010 20th International Conference on Pattern Recognition, pp. 2366–2369 (2010)

  29. Jia, Q., Wan, X., Hei, B.: A new disparity map quality assessment based on structural similarity for remotely sensed image pairs. Remote Sens. Lett. 11(7), 659–666 (2020)

    Article  Google Scholar 

  30. Lewis, J.J., O’Callaghan, R.J., Nikolov, S.G.: Pixel- and region-based image fusion with complex wavelets. Inf. Fusion 8(2), 119–130 (2005)

    Article  Google Scholar 

  31. Li, J., Dai, W.: Image quality assessment based on the correlation coefficient and the 2-D discrete wavelet transform. In: IEEE International Conference on Automation and Logistics (2009)

  32. Li, H., Hestenes, D., Rockwood, A.: Generalized Homogeneous Coordinates for Computational Geometry, pp. 27–60. Springer, Berlin (2001)

    MATH  Google Scholar 

  33. Li, Y., Liu, W., Li, X.: GA-SIFT: a new scale invariant feature transform for multispectral image using geometric algebra. Inf. Sci. 281, 559–572 (2014)

    Article  MathSciNet  Google Scholar 

  34. Li, H., Li, L., Zhang, J.X.: Multi-focus image fusion based on sparse feature matrix decomposition and morphological filtering. Opt. Commun. 342, 1–11 (2015)

    Article  ADS  Google Scholar 

  35. Liu, F., Chen, L., Lu, L.: Medical image fusion method by using Laplacian pyramid and convolutional sparse representation. Concurr. Comput. Pract. Exp. 32(17) (2020)

  36. Lopes, W.B., Lopes, C.G.: Geometric-algebra adaptive filters. IEEE Trans. Signal Process. 3649–3662 (2019)

  37. Marshall, S., Matsopoulos, G., Brunt, J.: Multiresolution morphological fusion of MR and CT images of the human brain. In: IEE Colloquium on Multiresolution Modelling and Analysis in Image Processing and Computer Vision (1995)

  38. Naeem, E.A., Abd Elnaby, M.M., El-Sayed, H.S.: Wavelet fusion for encrypting images with a few details. Comput. Electr. Eng. 54, 450–470 (2016)

    Article  Google Scholar 

  39. Naji, M.A., Aghagolzadeh, A.: Multi-focus image fusion in DCT domain based on correlation coefficient. In: 2015 2nd International Conference on Knowledge-Based Engineering and Innovation (KBEI), pp. 632–639 (2015)

  40. Naji, M.A., Noiey, P.R., Aghagolzadeh, A.: Multi-focus image fusion using singular value decomposition in DCT domain. In: 2017 10th Iranian Conference on Machine Vision and Image Processing (MVIP) (2017)

  41. Okarma, K., Fastowicz, J.: Improved quality assessment of colour surfaces for additive manufacturing based on image entropy. Pattern Anal. Appl. 23, 1035–1047 (2020)

    Article  Google Scholar 

  42. Pham, M.T., Yoshikawa, T., Furuhashi, T., Tachibana, K.: Robust feature extractions from geometric data using geometric algebra. In: IEEE International Conference on Systems, pp. 529–533 (2009)

  43. Pohl, C., Genderen, J.L.: Image fusion in remote sensing by multi-objective deep learning. Int. J. Remote Sens. 41(24), 9507–9524 (1998)

    Google Scholar 

  44. Pu, T., Ni, G.: Contrast-based image fusion using the discrete wavelet transform. Opt. Eng. 39(8), 2075–2082 (2000)

    Article  ADS  Google Scholar 

  45. Qu, G., Zhang, D., Yan, P.: Information measure for performance of image fusion. Electron. Lett. 38(7), 313–315 (2002)

    Article  ADS  Google Scholar 

  46. Rajarshi, K., Himabindu, C.: DWT based medical image fusion with maximum local extrema. In: 2016 International Conference on Computer Communication and Informatics (ICCCI) (2016)

  47. Raut, G.N., Paikrao, P.L., Chaudhari, D.S. A study of quality assessment techniques for fused images. Int. J. Innov. Technol. Explor. Eng. 290–294 (2013)

  48. Saurabh, S., Aglika, G., George, B.: Infrared and visible image fusion for face recognition. SPIE Defense Commer. Sens. 585–596 (2004)

  49. Shi, L., Funt, B.: Quaternion color texture segmentation. Comput. Vis. Image Underst. 107(1), 88–96 (2006)

    Google Scholar 

  50. Sommer, G.: Geometric Computing with Clifford Algebras. Springer, Berlin (2001)

    Book  Google Scholar 

  51. Su, H., Bo, Z.: Conformal geometric algebra based band selection and classification for hyperspectral imagery. In: Proc. 8th Workshop Hyperspectral Image Signal Process, pp. 1–4 (2016)

  52. Tang, J.: A contrast based image fusion technique in the DCT domain. Digital Signal Process. 14(3), 218–226 (2004)

    Article  Google Scholar 

  53. Vijayarajan, R., Muttan, S.: Discrete wavelet transform based principal component averaging fusion for medical images. AEU Int. J. Electron. Commun. 69(6), 896–902 (2015)

    Article  Google Scholar 

  54. Wang, R., Shen, M., Cao, W.: Joint sparse representation model for multi-channel image based on reduced geometric algebra. IEEE Access 24213–24223 (2018)

  55. Wang, R., Shen, M., Cao, W.: Multivector sparse representation for multispectral images using geometric algebra. IEEE Access 12755–12767 (2019)

  56. Wang, R., Zhang, W., Shi, Y., Wang, X., Cao, W.: GA-ORB: a new efficient feature extraction algorithm for multispectral images based on geometric algebra. IEEE Access 71235–71244 (2019)

  57. Wang, R., Zhou, Y.-X., Jin, Y.-L.: Sparse fast Clifford Fourier transform. Front. Inf. Technol. Electron. Eng. 18(8), 1131–1141 (2017)

    Article  Google Scholar 

  58. Xin, W., You-Li, W., Fu, L.: New multi-source image sequence fusion algorithm based on SIDWT. In: 2013 Seventh International Conference on Image and Graphics (2013)

  59. Yang, Y.: A novel DWT based multi-focus image fusion method. Procedia Eng. 24, 177–181 (2011)

    Article  Google Scholar 

  60. Yang, L., Guo, B.L., Ni, W.: Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform. Neurocomputing 72(1), 203–211 (2008)

    Article  Google Scholar 

  61. Zhang, Q., Guo, B.L.: Multifocus image fusion using the nonsubsampled contourlet transform. Signal Process 89(7), 1334–1346 (2009)

    Article  ADS  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (NSFC) under Grant nos. 61771299, 61771322 and Shen zhen foundation for basic researchJCYJ20190808160815125.

Author information

Authors and Affiliations

  1. School of Communication and Information Engineering, Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Joint International Research Laboratory of Specialty Fiber Optics and Advanced Communication, Shanghai Institute for Advanced Communication and Data Science Shanghai University, Shanghai, China

    Rui Wang, Nian Fang, Yinmei He & Yanping Li

  2. College of Electronics and Information Engineering, Shenzhen University, Shenzhen, China

    Wenming Cao

  3. Department of General Surgery, Shanghai General Hospital of Shanghai Jiaotong University, Shanghai, China

    Haiquan Wang

Authors
  1. Rui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nian Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yinmei He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanping Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wenming Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Haiquan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenming Cao.

Additional information

Communicated by Hongbo Li.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, R., Fang, N., He, Y. et al. Multi-modal Medical Image Fusion Based on Geometric Algebra Discrete Cosine Transform. Adv. Appl. Clifford Algebras 32, 19 (2022). https://doi.org/10.1007/s00006-021-01197-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00006-021-01197-6

Keywords

Mathematics Subject Classification

Search

Navigation