Skip to main content
SpringerLink
Log in

Advanced Image Processing Techniques for Ultrasound Images using Multiscale Self Attention CNN

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

The aim of this research is to enhance the quality of prenatal ultrasound images by addressing common artifacts such as missing or damaged areas, speckle noise, and other types of distortions that can impede accurate diagnosis. The proposed approach involves a novel preprocessing pipeline for prenatal 5th-month ultrasound scan images, which includes three main steps. First, Multiscale Self Attention convolutional neural network (CNN) is used for image inpainting and augmentation to fill missing or damaged areas and generate augmented images for training DL models. Second, Anisotropic Diffusion Filtering is used for speckle noise reduction, and the filter parameters are adapted to local noise characteristics using memory-based speckle statistics. Third, the CNN is trained to estimate local statistics of the speckle noise and adapt filtering parameters accordingly to capture local and global image features. The effectiveness of the proposed approach is evaluated on a prenatal 5th-month ultrasound scan dataset. The results demonstrate that the proposed preprocessing steps significantly improve the quality of ultrasound images and lead to better performance of DL models. The proposed preprocessing pipeline using Multiscale Self Attention CNN for image inpainting and augmentation, followed by Anisotropic Diffusion Filtering and memory-based speckle statistics for speckle noise reduction, can significantly enhance the quality of prenatal ultrasound images and enhance the accuracy of diagnostic models. The approach has potential for broader use in medical imaging applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Drukker L, Bradburn E, Rodriguez GB, Roberts NW, Impey L, Papageorghiou AT (2021) How often do we identify fetal abnormalities during routine third-trimester ultrasound? A systematic review and meta-analysis. BJOG Int J Obstetr Gynaecol 128(2):259–269

    Article  Google Scholar 

  2. Prieto JC, Shah H, Rosenbaum AJ, Jiang X, Musonda P, Price JT, Stringer EM, Vwalika B, Stamilio DM, Stringer JS (2021). An automated framework for image classification and segmentation of fetal ultrasound images for gestational age estimation. In: medical imaging 2021: image processing, 11596: 453–462. SPIE

  3. Best RG (2022) Prenatal screening for neural tube defects and aneuploidy. In: Emery and Rimoin's principles and practice of medical genetics and genomics (pp 9–34). Academic Press

  4. D’Asta M, La Ferrera N, Gulino FA, Ettore C, Ettore G (2022) Is It possible to diagnose preoperatively a tubal ectopic hydatidiform molar pregnancy? Description of a case report and review of the literature of the last ten years. J Clin Med 11(19):5783

    Article  Google Scholar 

  5. Peixoto AO, Costa RM, Uzun R, Fraga ADM, Ribeiro JD, Marson FL (2021) Applicability of lung ultrasound in COVID-19 diagnosis and evaluation of the disease progression: a systematic review. Pulmonology 27(6):529–562

    Article  Google Scholar 

  6. Sudharson S, Kokil P (2021) Computer-aided diagnosis system for the classification of multi-class kidney abnormalities in the noisy ultrasound images. Comput Methods Programs Biomed 205:106071

    Article  Google Scholar 

  7. Raheem A (2021) Effects of artifacts on the diagnosis of ultrasound image. Medico-Legal Update. https://doi.org/10.37506/mlu.v21i4.3152

    Article  Google Scholar 

  8. Frenzel F, Kubale R, Massmann A, Raczeck P, Jagoda P, Schlueter C, Stroeder J, Buecker A, Minko P (2021) Artifacts in contrast-enhanced ultrasound during follow-up after endovascular aortic repair: impact on Endoleak detection in comparison with computed tomography angiography. Ultrasound Med Biol 47(3):488–498

    Article  Google Scholar 

  9. Fu Z, Zhang J, Lu Y, Wang S, Mo X, He Y, Wang C, Chen H (2021) Clinical applications of superb microvascular imaging in the superficial tissues and organs: a systematic review. Acad Radiol 28(5):694–703

    Article  Google Scholar 

  10. Rafailidis V, Huang DY, Yusuf GT, Sidhu PS (2020) General principles and overview of vascular contrast-enhanced ultrasonography. Ultrasonography 39(1):22

    Article  Google Scholar 

  11. Quer G, Arnaout R, Henne M, Arnaout R (2021) Machine learning and the future of cardiovascular care: JACC state-of-the-art review. J Am Coll Cardiol 77(3):300–313

    Article  Google Scholar 

  12. Das PK, Meher S, Panda R, Abraham A (2021) An efficient blood-cell segmentation for the detection of hematological disorders. IEEE Trans Cybern 52(10):10615–10626

    Article  Google Scholar 

  13. Vimala BB, Srinivasan S, Mathivanan SK, Muthukumaran V, Babu JC, Herencsar N, Vilcekova L (2023) Image noise removal in ultrasound breast images based on hybrid deep learning technique. Sensors 23(3):1167

    Article  Google Scholar 

  14. Yancheng LI, Zeng X, Dong Q, Wang X (2023) RED-MAM: a residual encoder-decoder network based on multi-attention fusion for ultrasound image denoising. Biomed Signal Process Control 79:104062

    Article  Google Scholar 

  15. Ilesanmi AE, Idowu OP, Chaumrattanakul U, Makhanov SS (2021) Multiscale hybrid algorithm for pre-processing of ultrasound images. Biomed Signal Process Control 66:102396

    Article  Google Scholar 

  16. Jain L, Singh P (2022) A novel wavelet thresholding rule for speckle reduction from ultrasound images. J King Saud Univ-Comput Inf Sci 34(7):4461–4471

    Google Scholar 

  17. Karaoğlu O, Bilge HŞ, Uluer İ (2022) Removal of speckle noises from ultrasound images using five different deep learning networks. Eng Sci Technol Int J 29:101030

    Google Scholar 

  18. Mikolaj K, Lin M, Bashir Z, Svendsen MBS, Tolsgaard M, Nymark A, Feragen A (2023) Removing confounding information from fetal ultrasound images. arXiv preprint arXiv:2303.13918.

  19. Luo D, Wen H, Peng G, Lin Y, Liang M, Liao Y, Qin Y, Zeng Q, Dang J, Li S (2021) A prenatal ultrasound scanning approach: one-touch technique in second and third trimesters. Ultrasound Med Biol 47(8):2258–2265

    Article  Google Scholar 

  20. Monkam P, Lu W, Jin S, Shan W, Wu J, Zhou X, Tang B, Zhao H, Zhang H, Ding X, Chen H (2023) US-Net: a lightweight network for simultaneous speckle suppression and texture enhancement in ultrasound images. Comput Biol Med 152:106385

    Article  Google Scholar 

  21. Mousania Y, Karimi S, Farmani A (2023) Optical remote sensing, brightness preserving and contrast enhancement of medical images using histogram equalization with minimum cross-entropy-Otsu algorithm. Opt Quant Electron 55(2):1–22

    Article  Google Scholar 

  22. Largo R (2022) Fetal-ultrasound-brain. Kaggle. Retrieved 2023, from https://www.kaggle.com/datasets/rahimalargo/fetalultrasoundbrain

  23. Arican ME, Kara O, Bredell G, Konukoglu E (2022) Isnas-dip: Image-specific neural architecture search for deep image prior. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp 1960–1968)

  24. Angah O, Chen AY (2020) Removal of occluding construction workers in job site image data using U-Net based context encoders. Autom Constr 119:103332

    Article  Google Scholar 

  25. Cai W, Wei Z (2020) PiiGAN: generative adversarial networks for pluralistic image inpainting. IEEE Access 8:48451–48463

    Article  Google Scholar 

  26. Sinha AK, Moorthi SM, Dhar D (2022) NL-FFC: non-local fast fourier convolution for image super resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp 467–476)

  27. Wang G, Jiao Y, Xu Q, Wang Y, Yang C (2021) Deep generative learning via schrödinger bridge. In: International conference on machine learning (pp 10794–10804). PMLR

  28. Yu W, Du J, Liu R, Li Y, Zhu Y (2022) Interactive image inpainting using semantic guidance. In: 2022 26th international conference on pattern recognition (ICPR) (pp 168–174). IEEE

  29. Mingote V, Miguel A, Ribas D, Ortega A, Lleida E (2020) Knowledge distillation and random erasing data augmentation for text-dependent speaker verification. In: ICASSP 2020–2020 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp 6824–6828). IEEE

  30. Ma Y, Xu X, Li Y (2020) LungRN+ NL: An improved adventitious lung sound classification using non-local block resnet neural network with mixup data augmentation. In: Interspeech (pp 2902–2906)

  31. Harris E, Marcu A, Painter M, Niranjan M, Prügel-Bennett A, Hare J (2020) Fmix: Enhancing mixed sample data augmentation. arXiv preprint arXiv:2002.12047.

  32. Yang S, Xiao W, Zhang M, Guo S, Zhao J, Shen F (2022) Image data augmentation for deep learning: A survey. arXiv preprint arXiv:2204.08610.

  33. Faryna K, van der Laak J, Litjens G (2021) Tailoring automated data augmentation to H&E-stained histopathology. In: Medical imaging with deep learning

  34. Yamashita R, Long J, Banda S, Shen J, Rubin DL (2021) Learning domain-agnostic visual representation for computational pathology using medically-irrelevant style transfer augmentation. IEEE Trans Med Imaging 40(12):3945–3954

    Article  Google Scholar 

  35. Singh P, Shree R (2020) A new homomorphic and method noise thresholding based despeckling of SAR image using anisotropic diffusion. J King Saud Univ-Comput Inf Sci 32(1):137–148

    Google Scholar 

  36. Li H, Duan XL (2022) SAR ship image speckle noise suppression algorithm based on adaptive bilateral filter. Wirel Commun Mobile Comput. https://doi.org/10.1155/2022/9392648

    Article  Google Scholar 

  37. Guo F, Tang H, Liu W (2023) Non-local means de-speckling based on multi-directional local plane inclination angle. Remote Sens 15(4):1029

    Article  Google Scholar 

  38. Khan SI, Choubey SB, Choubey A, Bhatt A, Naishadhkumar PV, Basha MM (2022) Automated glaucoma detection from fundus images using wavelet-based denoising and machine learning. Concurr Eng 30(1):103–115

    Article  Google Scholar 

  39. Guntuboyina A, Lieu D, Chatterjee S, Sen B (2020) Adaptive risk bounds in univariate total variation denoising and trend filtering. Ann Statist. https://doi.org/10.1214/18-AOS1799

    Article  MathSciNet  Google Scholar 

  40. Tian Q, Li Z, Fan Q, Polimeni JR, Bilgic B, Salat DH, Huang SY (2022) SDnDTI: self-supervised deep learning-based denoising for diffusion tensor MRI. Neuroimage 253:119033

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computing Technologies, School of Computing, SRM Institute of Science and Technology, Kattankulathur, India

    D. Vetriselvi & R. Thenmozhi

Authors
  1. D. Vetriselvi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. Thenmozhi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors agreed on the content of the study. VD and RT collected all the data for analysis. VD agreed on the methodology. VD and RT completed the analysis based on agreed steps. Results and conclusions are discussed and written together. All authors read and approved the final manuscripts.

Corresponding author

Correspondence to D. Vetriselvi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Human and Animal Rights

This article does not contain any studies with human or animal subjects performed by any of the authors.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vetriselvi, D., Thenmozhi, R. Advanced Image Processing Techniques for Ultrasound Images using Multiscale Self Attention CNN. Neural Process Lett 55, 11945–11973 (2023). https://doi.org/10.1007/s11063-023-11404-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11404-z

Keywords

Search

Navigation