Skip to main content

Advertisement

SpringerLink
Log in

Visual attention condenser model for multiple disease detection from heterogeneous medical image modalities

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The World Health Organization (WHO) has identified breast cancer and tuberculosis (TB) as major global health issues. While breast cancer is a top killer of women, TB is an infectious disease caused by a single bacterium with a high mortality rate. Since both TB and breast cancer are curable, early screening ensures treatment. Medical imaging modalities, such as chest X-ray radiography and ultrasound, are widely used for diagnosing TB and breast cancer. Artificial intelligence (AI) techniques are applied to supplement the screening process for effective and early treatment due to the global shortage of radiologists and oncologists. These techniques fast-track the screening process leading to early detection and treatment. Deep learning (DL) is the most used technique producing outstanding results. Despite the success of these DL models in the automatic detection of TB and breast cancer, the suggested models are task-specific, meaning they are disease-oriented. Again, the complexity and weight of the DL applications make it difficult to apply the models on edge devices. Motivated by this, a Multi Disease Visual Attention Condenser Network (MD-VACNet) got proposed for multiple disease identification from different medical image modalities. The network architecture got designed automatically through a machine-driven design exploration with generative synthesis. The proposed MD-VACNet is a lightweight stand-alone visual recognition deep neural network based on VAC with a self-attention mechanism to run on edge devices. In the experiment, TB was identified based on chest X-ray images and breast cancer was based on ultrasound images. The suggested model achieved a 98.99% accuracy score, a 99.85% sensitivity score, and a 98.20% specificity score on the x-ray radiographs for TB diagnosis. The model also produced a cutting-edge performance on breast cancer classification into benign and malignant, with accuracy, sensitivity and specificity scores of 98.47%, 98.42%, and 98.31%, respectively. Regarding model architectural complexity, MD-VACNet is simple and lightweight for edge device implementation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

The chest x-ray images containing tuberculosis and the breast cancer ultrasound image datasets that support the experiment of this study were accessed from the Kaggle repository https://www.kaggle.com/search?q=NLM+dataset and https://www.kaggle.com/datasets/tawsifurrahman/tuberculosis-tb-chest-xray-dataset.

References

  1. World Health Organization (2020) Global tuberculosis report, Geneva

  2. Byra M (2021) Breast mass classification with transfer learning based on scaling of deep representations. Biomed Sign Process Control 69:102828. https://doi.org/10.1016/j.bspc.2021.102828

    Article  Google Scholar 

  3. Zahoor S, Shoaib U, Lali IU (2022) Breast cancer mammograms classification using deep neural network and entropy-controlled whale optimization algorithm”. Diagnostics 12:557. https://doi.org/10.3390/diagnostics12020557

    Article  PubMed  PubMed Central  Google Scholar 

  4. Puttagunta MK, Ravi S (2021) Detection of tuberculosis based on deep learning based methods. J Phys Conf Ser 1767(1):012004. https://doi.org/10.1088/1742-6596/1767/1/012004

    Article  Google Scholar 

  5. Ayaz M, Shaukat F, Raja G (2021) Ensemble learning based automatic detection of tuberculosis in chest X-ray images using hybrid feature descriptors. Phys Eng Sci Med 44(1):183–194

    Article  PubMed  PubMed Central  Google Scholar 

  6. Iqbal A, Usman M, Ahmed Z (2022) An efficient deep learning-based framework for tuberculosis detection using chest X-ray images”. Tuberculosis 136:102234. https://doi.org/10.1016/j.tube.2022.102234

    Article  PubMed  Google Scholar 

  7. Duwairi R, Melhem A (2023) A deep learning-based framework for automatic detection of drug resistance in tuberculosis patients. Egypt Inform J 24(1):139–148

    Article  Google Scholar 

  8. Huang PW, Ouyang H, Hsu BY, Chang YR, Lin YC et al (2023) Deep-learning based breast cancer detection for cross-staining histopathology images. Heliyon 9(2):e13171. https://doi.org/10.1016/j.heliyon.2023.e13171

    Article  PubMed  PubMed Central  Google Scholar 

  9. Sheeba A, Kumar PS, Ramamoorthy M, Sasikala S (2023) Microscopic image analysis in breast cancer detection using ensemble deep learning architectures integrated with web of things. Biomed Signal Process Control 79(2):104048. https://doi.org/10.1016/j.bspc.2022.104048

    Article  Google Scholar 

  10. Sahu A, Das PK, Meher S (2023) High accuracy hybrid CNN classifiers for breast cancer detection using mammogram and ultrasound datasets. Biomed Signal Process Control 80(1):104292. https://doi.org/10.1016/j.bspc.2022.104292

    Article  Google Scholar 

  11. Mukherjee P, Roy CK, Roy SK (2022) OCFormer: One-class transformer network for image classification. Xiv:2204.11449v1 [cs.CV]. https://doi.org/10.48550/arXiv.2204.11449

  12. Kotei E, Thirunavukarasu R (2022) Ensemble technique coupled with deep transfer learning framework for automatic detection of tuberculosis from chest X-ray radiographs”. Healthcare 10:2335. https://doi.org/10.3390/healthcare10112335

    Article  PubMed  PubMed Central  Google Scholar 

  13. Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S (2020) End-to-end object detection with transformers. arXiv:2005.12872v3 [cs.CV]. https://doi.org/10.48550/arXiv.2005.12872

  14. Wang Y, Zhang X, Yang T, Sun J (2022) Anchor DETR: query design for transformer-based detector. In: Proc. association for the advancement of artificial intelligence, AAAI, California, pp 2567–2575. https://doi.org/10.48550/arXiv.2109.07107

  15. Chen X, Sun S, Bai N, Han K, Liu Q et al (2021) A deep learning-based auto-segmentation system for organs-at-risk on whole-body computed tomography images for radiation therapy. Radiother Oncol 160:175–184. https://doi.org/10.1016/j.radonc.2021.04.019

    Article  PubMed  Google Scholar 

  16. Su Y, Liu Q, Xie W, Hu P (2022) YOLO-LOGO: a transformer-based YOLO segmentation model for breast mass detection and segmentation in digital mammograms. Comput Methods Prog Biomed 221:106903. https://doi.org/10.1016/j.cmpb.2022.106903

    Article  Google Scholar 

  17. Lecun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  ADS  CAS  PubMed  Google Scholar 

  18. Bhaskaran KL, Osei RS, Kotei E, Agbezuge EY, Ankora C et al (2022) A survey on big data in pharmacology, toxicology and pharmaceutics. Big Data Cogn Comput 6(4):161. https://doi.org/10.3390/bdcc6040161

    Article  Google Scholar 

  19. Kotei E, Thirunavukarasu R (2022) Computational techniques for the automated detection of mycobacterium tuberculosis from digitized sputum smear microscopic images: A systematic review. Progress Biophys Mol Biol 171:4–16. https://doi.org/10.1016/j.pbiomolbio.2022.03.004

    Article  Google Scholar 

  20. Thirunavukarasu R, Doss GP, Gnanasambandan R, Gopikrishnan M, Palanisamy V (2022) Towards computational solutions for precision medicine based big data healthcare system using deep learning models: a review. Comput Biol Med 149:106020. https://doi.org/10.1016/j.compbiomed.2022.106020

    Article  PubMed  Google Scholar 

  21. Kotei E, Thirunavukarasu R (2023) A systematic review of transformer-based pre-trained language models through self-supervised learning. Information 14(3):187. https://doi.org/10.3390/info14030187

    Article  Google Scholar 

  22. Duong LT, Le NH, Tran TB, Ngo VM, Nguyen PT (2021) Detection of tuberculosis from chest Xray images:boosting the performance with vision transformer and transfer learning. Expert Syst Appl 184:115519. https://doi.org/10.1016/j.eswa.2021.115519

    Article  Google Scholar 

  23. Sandler M, Howard A, Zhu M, Zhmoginov A (2018) MobileNetV2: inverted residuals and linear bottlenecks. In: Proc. IEEE/CVF conference on computer vision and pattern recognition, CVPR, Salt Lake City, UT, pp 4510–4520. https://doi.org/10.1109/CVPR.2018.00474

  24. Azizi S, Mustafa B, Ryan F, Beaver Z, Freyberg J et al (2021) Big self-supervised models advance medical image classification. In: Proc. international conference on computer vision, ICCV, Montreal, QC, Canada, pp 3458–3468. https://doi.org/10.1109/ICCV48922.2021.00346

  25. Rajaraman S, Zamzmi G, Folio LR, Antani S (2022) Detecting tuberculosis-consistent findings in lateral chest x-rays using an ensemble of CNNs and vision transformers. Front Gen 13:1–13. https://doi.org/10.3389/fgene.2022.864724

    Article  Google Scholar 

  26. Dai Y, Gao Y, Liu F (2021) Transmed: Transformers advance multi-modal medical image classification. Diagnostics 11(8):1–15

    Article  Google Scholar 

  27. Wong A, Famouri M, Shafiee MJ (2020) AttendNets: tiny deep image recognition neural networks for the edge via visual attention condensers. arXiv:2009.14385v1 [cs.CV]. https://doi.org/10.48550/arXiv.2009.14385

  28. Momeny M, Neshat AA, Gholizadeh A, Jafarnezhad A, Rahmanzadeh E et al (2022) Greedy Autoaugment for classification of mycobacterium tuberculosis image via generalized deep CNN using mixed pooling based on minimum square rough entropy. Comput Biol Med 141:105175. https://doi.org/10.1016/j.compbiomed.2021.105175

    Article  PubMed  Google Scholar 

  29. Aljaddouh B, Malathi D (2022) Trends of using machine learning for detection and classification of respiratory diseases: Investigation and analysis. Mater Today Proc 62:4651–4658. https://doi.org/10.1016/j.matpr.2022.03.120

    Article  CAS  Google Scholar 

  30. Apostolopoulos ID, Mpesiana TA (2020) Covid-19: automatic detection from X-ray images utilizing transfer learning with convolutional neural networks. Phys Eng Sci Med 43(2):635–640

    Article  PubMed  PubMed Central  Google Scholar 

  31. Hooda R, Sofat S, Kaur S, Mittal A, Meriaudeau F (2017) Deep-learning: a potential method for tuberculosis detection using chest radiography. In: Proc. IEEE international conference on signal and image processing applications, ICSIPA, Kuching, Malaysia, pp 497–502. https://doi.org/10.1109/ICSIPA.2017.8120663

  32. Jaeger S, Candemir S, Antani S, Wáng YX, Lu P-X et al (2014) Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. Quant Imag Med Surg 4(6):475–7

    Google Scholar 

  33. Akbar S, GhaniHaider N, Tariq H (2019) Tuberculosis diagnosis using x-ray images. Int J Adv Res 7(4):689–696

    Article  Google Scholar 

  34. Guo R, Passi K, Jain CK (2020) Tuberculosis diagnostics and localization in chest x-rays via deep learning models. Front Artif Intell 3:583427. https://doi.org/10.3389/frai.2020.583427

    Article  PubMed  PubMed Central  Google Scholar 

  35. Abideen Z, Ghafoor M, Munir K, Saqib M, Ullah A et al (2020) Uncertainty assisted robust tuberculosis identification with bayesian convolutional neural networks. IEEE Access 8:22812–22825. https://doi.org/10.1109/ACCESS.2020.2970023

    Article  Google Scholar 

  36. Chouhan V, Singh SK, Khamparia A, Gupta D, Tiwari P et al (2020) A novel transfer learning based approach for pneumonia detection in chest X-ray images. Appl Sci 10(2):559. https://doi.org/10.3390/app10020559

    Article  Google Scholar 

  37. Rahman T, Khandakar A, Kadir MA, Islam KR, Islam KF et al (2020) Reliable tuberculosis detection using chest X-ray with deep learning, segmentation and visualization. IEEE Access 8:191586–191601. https://doi.org/10.1109/ACCESS.2020.3031384

    Article  Google Scholar 

  38. Sahlol AT, Elaziz MA, Jamal AT, Damaševičius R, HassanOF, (2020) A novel method for detection of tuberculosis in chest radiographs using artificial ecosystem-based optimisation of deep neural network features. Symmetry (Basel) 12(7):1146. https://doi.org/10.3390/sym12071146

    Article  ADS  Google Scholar 

  39. Kaggle (2018) RSNA Pneumonia detection challenge 2020. https://www.kaggle.com/datasets/sovitrath/rsnapneumonia-detection-2018

  40. Spanhol FA, Oliveira LS, Petitjean C, Heutte L (2016) A dataset for breast cancer histopathological image classification. IEEE Trans Biomed Eng 63(7):1455–1462

    Article  PubMed  Google Scholar 

  41. Benhammou Y, Tabik S, Achchab B, Herrera F (2018) A first study exploring the performance of the state-of-the art CNN model in the problem of breast cancer. In: Proc. ACM international conference on learning and optimization algorithms: theory and applications, LOPAL, Rabat, Morocco, pp 1–6. https://doi.org/10.1145/3230905.3230940

  42. Silva LF, Saade DCM, Sequeiros GO, Silva AC, Paiva AC et al (2014) A new database for breast research with infrared image. J Med Imag Health Inform 4(1):92–100

    Article  Google Scholar 

  43. Roslidar R, Saddami K, Arnia F, Syukri M, Munadi K (2019) A study of fine-tuning CNN models based on thermal imaging for breast cancer classification. In: Proc IEEE international conference on cybernetics and computational intelligence, CYBERNETICSCOM, Banda Aceh, Indonesia, pp 77–81. https://doi.org/10.1109/CYBERNETICSCOM.2019.8875661

  44. Khan MHM, Jahangeer NB, Dullull W, Nathire S, Gao X et al (2021) Multi- class classification of breast cancer abnormalities using Deep Convolutional Neural Network (CNN). PLoS One 16:1–15. https://doi.org/10.1371/journal.pone.0256500

    Article  CAS  Google Scholar 

  45. Sawyer-Lee R, Gimenez F, Hoogi A, Rubin D (2016) Curated breast imaging subset of digital database for screening mammography (CBIS-DDSM) [Data set]. The Cancer Imaging Archive. https://doi.org/10.7937/K9/TCIA.2016.7O02S9CY

  46. Liu H, Cui G, Luo Y, Guo Y, Zhao L et al (2022) Artificial intelligence-based breast cancer diagnosis using ultrasound images and grid-based deep feature generator. Int J Gen Med 15:2271–2282. https://doi.org/10.2147/IJGM.S347491

    Article  PubMed  PubMed Central  Google Scholar 

  47. Al-Dhabyani W, Gomaa M, Khaled H, Fahmy A (2020) Dataset of breast ultrasound images. Data Br 28:104863. https://doi.org/10.1016/j.dib.2019.104863

    Article  Google Scholar 

  48. Lambert Z, Petitjean C, Dubray B, Kuan S (2020) SegTHOR: Segmentation of thoracic organs at risk in CT images. In: Proc. tenth international conference on image processing theory, tools and applications, IPTA, Paris, France, pp 1–6. https://doi.org/10.1109/IPTA50016.2020.9286453

  49. Kaggle (2020) Tuberculosis (TB) Chest X-ray Database. https://www.kaggle.com/datasets/tawsifurrahman/tuberculosis-tb-chest-xray-dataset

  50. Bello I, Zoph B, Le Q, Vaswani A, Shlens J (2019) Attention augmented convolutional networks. In: Proc. IEEE/CVF international conference on computer vision, ICCV, Seoul, Korea (South), pp 3285–3294. https://doi.org/10.1109/ICCV.2019.00338

  51. Hu J, Shen L, Albanie S, Sun G, Wu E (2020) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42(8):2011–2023

    Article  PubMed  Google Scholar 

  52. Wong A, Shafiee MJ, Chwyl B, Li F (2018) FermiNets: learning generative machines to generate efficient neural networks via generative synthesis. arXiv:1809.05989v2 [cs.NE]. https://doi.org/10.48550/arXiv.1809.05989

  53. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W et al (2017) MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861v1 [cs.CV]. https://doi.org/10.48550/arXiv.1704.04861

  54. Zoph B, Vasudevan V, Shlens J, Le QV (2018) Learning transferable architectures for scalable image recognition. In: Proc. IEEE/CVF conference on computer vision and pattern recognition, CVPR, Salt Lake City, UT, pp 8697–8710. https://doi.org/10.1109/CVPR.2018.00907

  55. Tan M, Le QV (2019) EfficientNet: rethinking model scaling for convolutional neural networks. arXiv:1905.11946v5 [cs.LG]. https://doi.org/10.48550/arXiv.1905.11946

Download references

Author information

Authors and Affiliations

  1. School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, 632014, India

    Evans Kotei & Ramkumar Thirunavukarasu

Authors
  1. Evans Kotei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ramkumar Thirunavukarasu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ramkumar Thirunavukarasu.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflicts of interest to report regarding the present study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kotei, E., Thirunavukarasu, R. Visual attention condenser model for multiple disease detection from heterogeneous medical image modalities. Multimed Tools Appl 83, 30563–30585 (2024). https://doi.org/10.1007/s11042-023-16625-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16625-x

Keywords

Search

Navigation