Skip to main content
SpringerLink
Log in

A Multi-module Medical Image Fusion Method Based on Non-subsampled Shear Wave Transformation and Convolutional Neural Network

  • Original Paper
  • Published:
Sensing and Imaging Aims and scope Submit manuscript

Abstract

In order to solve the fusion limitation of single module medical image, a multi-module medical image fusion method based on the non-subsampled shear wave transform (NSST) and convolutional neural network (CNN) is proposed. Firstly, NSST is used to decompose the source image into multi-scale and multi-direction layers to obtain the sub-band coefficients of low frequency and high frequency. Secondly, the low-frequency sub-band coefficients are derived from region energy and variance, and the fusion method based on region feature weight is adopted. The high frequency inner layer sub-band coefficient is firstly calculated by CNN, and then selected by average gradient weighting. The high frequency outer layer sub-band coefficient adopts the larger region absolute value fusion rule. Finally, the image is reconstructed by inverse NSST. The experiment results show that the proposed method achieves better fusion effect on both subjective and objective evaluation indexes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Teng, L., Li, H., & Karim, S. (2019). DMCNN: a deep multiscale convolutional neural network model for medical image segmentation. Journal of Healthcare Engineering. https://doi.org/10.1155/2019/8597606.

    Article  Google Scholar 

  2. Yin, S., & Zhang, Ye. (2019). Singular value decomposition-based anisotropic diffusion for fusion of infrared and visible images. International Journal of Image and Data Fusion, 10(2), 146–163.

    Article  Google Scholar 

  3. Yin, S., Li, H., Teng, L., Jiang, M., & Karim, S. (2020). An optimised multi-scale fusion method for airport detection in large-scale optical remote sensing images. International Journal of Image and Data Fusion, 11(2), 201–214. https://doi.org/10.1080/19479832.2020.1727573.

    Article  Google Scholar 

  4. Wang, S. Y., Chen, X. X., Li, Y., et al. (2016). Application of multimodality imaging fusion technology in diagnosis and treatment of malignant tumors under the precision medicine plan. Chinese Medical Journal, 129(024), 2991–2997.

    Article  Google Scholar 

  5. Haneishi, H., Yamaguchi, T., Nakamura, R., et al. (2013). Research status in the fusion and enrichment of medical imaging for high quality diagnosis and treatment (FERMI) project. Journal of Medical Imaging & Health Informatics, 3(1), 51–58.

    Article  Google Scholar 

  6. Jianbo, C., & University, C. J. (2016). The model and characteristics of student affairs management in Hongkong Universities. Journal of Zhejiang Sci-Tech University (Social Sciences Edition), 36(3), 313–316.

  7. Haijiang, W., Jingpu, W., Fuqi, Y., et al. Multi-band contourlet transform for adaptive remote sensing image denoising. The Computer Journal (7):7.

  8. He, G., Xing, S., He, X., et al. (2019). Image fusion method based on simultaneous sparse representation with non-subsampled contourlet transform. IET Computer Vision, 13(2), 240–248.

    Article  Google Scholar 

  9. Wang, X. L., & Chen, C. X. (2016). Image fusion for synthetic aperture radar and multispectral images based on sub-band-modulated non-subsampled contourlet transform and pulse coupled neural network methods. Journal of Photographic Science, 64(2), 87–93.

  10. Luping, X., Guorong, G., & Dongzhu, F. (2013). Multi-focus image fusion based on non-subsampled shearlet transform. Iet Image Processing, 7(6), 633–639.

    Article  Google Scholar 

  11. Ganasala, P., & Kumar, V. (2014). Multimodality medical image fusion based on new features in NSST domain. Biomedical Engineering Letters, 4(4), 414–424.

    Article  Google Scholar 

  12. Liu, Y., Chen, X., Peng, H., et al. (2017). Multi-focus image fusion with a deep convolutional neural network. Information Fusion, 36, 191–207.

    Article  Google Scholar 

  13. Starovoitov, V., Makarau, A., Zakharov, I., et al. (2015). Multispectral image enhancement based on fusion and super-resolution. European Signal Processing Conference. IEEE, 2015.

  14. Xing, C., Wang, M., Dong, C., et al. (2020). Using Taylor expansion and convolutional sparse representation for image fusion. Neurocomputing, 402, 437–455.

    Article  Google Scholar 

  15. Giuseppe, M., Davide, C., Luisa, V., et al. (2016). Pansharpening by convolutional neural networks. Remote Sensing, 8(7), 594.

    Article  Google Scholar 

  16. Parvathy, V. S., Pothiraj, S., & Sampson, J. (2020). A novel approach in multimodality medical image fusion using optimal shearlet and deep learning. International Journal of Imaging Systems and Technology, 30(4).

  17. Kaur, J., & Shekhar, C. (2020). Multimodal medical image fusion using deep learning. Advances in Computational Techniques for Biomedical Image Analysis, pp 35–56. https://doi.org/10.1016/B978-0-12-820024-7.00002-5

  18. Guo, Z., Li, X., Huang, H., et al. (2019). Deep learning-based image segmentation on multimodal medical imaging. IEEE Transactions on Radiation and Plasma Medical Sciences, 3(2), 162–169.

  19. Yin, S., & Li, H. (2020). Hot region selection based on selective search and modified fuzzy C-means in remote sensing images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13, 5862–5871.

  20. Yin, S., Li, H., Liu, D., & Karim, S. (2020). Active contour modal based on density-oriented BIRCH clustering method for medical image segmentation . Multimedia Tools and Applications, 79, 31049–31068.

  21. Wang, X., Yin, S., Li, H., & Liu, J. (2020). GKFC-CNN: modified Gaussian Kernel fuzzy C-means and convolutional neural network for apple segmentation and recognition. Journal of Applied Science and Engineering, 23(3), 555–561.

    Google Scholar 

  22. Yu, J., Li, H., & Liu, D. (2020). Modified immune evolutionary algorithm for IoT big data clustering and feature extraction under cloud computing environment . Journal of Healthcare Engineering, 2020(6):1–11.

  23. Teng, L., Li, H., & Karim, S. (2019). DMCNN: A deep multiscale convolutional neural network model for medical image segmentation. Journal of Healthcare Engineering, 2019, 8597606. https://doi.org/10.1155/2019/8597606

  24. Yin, S., Zhang, Y., & Karim, S. (2019). Region search based on hybrid convolutional neural network in optical remote sensing images. International Journal of Distributed Sensor Networks. https://doi.org/10.1177/1550147719852036

  25. Xia, K. J., Yin, H. S., & Wang, J. Q. (2019). A novel improved deep convolutional neural network model for medical image fusion. Cluster Computing, 22, 1515–1527.

  26. Asha, C., Lal, S., & Gurupur, V. P., et al. (2019). Multi-modal medical image fusion with adaptive weighted combination of NSST bands using Chaotic Grey Wolf Optimization. IEEE Access, pp 40782–40796.

  27. Guan, J. S., Kang, S. B., & Sun, Y. (2019). Medical image fusion algorithm based on multi-resolution analysis coupling approximate spare representation. Future Generation Computer Systems, 98, 201–207.

    Article  Google Scholar 

  28. Zhang, Y., Li, D., Zhang, R., et al. (2020). Sparse features with fast guided filtering for medical image fusion. Journal of Medical Imaging and Health Informatics, 10(5), 1195–1204.

  29. Liu, F., Chen, L., Lu, L., et al. (2019). Medical image fusion method by using Laplacian pyramid and convolutional sparse representation. Concurrency and Computation: Practice and Experience, 32(17), e5632. https://doi.org/10.1002/cpe.5632

  30. Huang, J., Le, Z., Ma, Y., et al. (2020). MGMDcGAN: Medical image fusion using multi-generator multi-discriminator conditional generative adversarial network. IEEE Access, PP(99):1–1.

  31. Jin X, Jiang Q, Chu X, et al. (2019). Brain medical image fusion using L2-norm-based features and fuzzy-weighted measurements in 2D Littlewood-Paley EWT domain. IEEE Transactions on Instrumentation and Measurement, PP(99):1–1.

  32. Liu, Z., Blasch, E., Xue, Z., et al. (2011). Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative study. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(1), 94–109.

    Article  Google Scholar 

  33. Petrovi, V. (2007). Subjective tests for image fusion evaluation and objective metric validation. Information Fusion, 8(2), 208–216.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Zhengzhou University of Science and Technology, Zhengzhou, 450000, China

    Mingju Zhao & Yuping Peng

Authors
  1. Mingju Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuping Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuping Peng.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, M., Peng, Y. A Multi-module Medical Image Fusion Method Based on Non-subsampled Shear Wave Transformation and Convolutional Neural Network. Sens Imaging 22, 9 (2021). https://doi.org/10.1007/s11220-021-00330-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11220-021-00330-w

Keywords

Search

Navigation