Skip to main content
SpringerLink
Log in

CheXNet and feature pyramid network: a fusion deep learning architecture for multilabel chest X-Ray clinical diagnoses classification

  • Original Paper
  • Published:
The International Journal of Cardiovascular Imaging Aims and scope Submit manuscript

Abstract

The existing multilabel X-Ray image learning tasks generally contain much information on pathology co-occurrence and interdependency, which is very important for clinical diagnosis. However, the challenging part of this subject is to accurately diagnose multiple diseases that occurred in a single X-Ray image since multiple levels of features are generated in the images, and create different features as in single label detection. Various works were developed to address this challenge with proposed deep learning architectures to improve classification performance and enrich diagnosis results with multi-probability disease detection. The objective is to create an accurate result and a faster inference system to support a quick diagnosis in the medical system. To contribute to this state-of-the-art, we designed a fusion architecture, CheXNet and Feature Pyramid Network (FPN), to classify and discriminate multiple thoracic diseases from chest X-Rays. This concept enables the model to extract while creating a pyramid of feature maps with different spatial resolutions that capture low-level and high-level semantic information to encounter multiple features. The model’s effectiveness is evaluated using the NIH ChestXray14 dataset, with the Area Under Curve (AUC) and accuracy metrics used to compare the results against other cutting-edge approaches. The overall results demonstrate that our method outperforms other approaches and has become promising for multilabel disease classification in chest X-Rays, with potential applications in clinical practice. The result demonstrated that we achieved an average AUC of 0.846 and an accuracy of 0.914. Further, our proposed architecture diagnoses images in 0.013 s, faster than the latest approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The publicly accessible ChestXRay14 database is utilized in this work [17].

References

  1. Ypsilantis P-P, Montana G Learning what to look in chest X-Rays with a recurrent visual attention model, no. Nips, pp. 1–5, 2017, [Online]. Available: http://arxiv.org/abs/1701.06452

  2. Padma T, Usha Kumari C (2020) Deep learning based chest X-Ray image as a Diagnostic Tool for COVID-19. Proc - Int Conf Smart Electron Commun ICOSEC 2020 no Icosec:589–592. https://doi.org/10.1109/ICOSEC49089.2020.9215257

    Article  Google Scholar 

  3. Pham HH, Le TT, Tran DQ, Ngo DT, Nguyen HQ (2021) Interpreting chest X-Rays via CNNs that exploit hierarchical Disease dependencies and uncertainty labels. Neurocomputing 437:186–194. https://doi.org/10.1016/j.neucom.2020.03.127

    Article  Google Scholar 

  4. Urinbayev K, Orazbek Y, Nurambek Y, Mirzakhmetov A, Varol HA (2020) End-to-end deep diagnosis of X-Ray images. Proc Annu Int Conf IEEE Eng Med Biol Soc EMBS 2020–July:2182–2185. https://doi.org/10.1109/EMBC44109.2020.9175208

    Article  Google Scholar 

  5. Sharma H, Jain JS, Bansal P, Gupta S (2020) Feature extraction and classification of chest X-Ray images using CNN to detect Pneumonia. Proc Conflu 2020–10th Int Conf Cloud Comput Data Sci Eng 227–231. https://doi.org/10.1109/Confluence47617.2020.9057809

  6. de Groot PM, Carter BW, Abbott GF, Wu CC (2015) Pitfalls in chest Radiographic Interpretation: blind spots. Semin Roentgenol 50(3):197–209. https://doi.org/10.1053/j.ro.2015.01.008

    Article  PubMed  Google Scholar 

  7. Rimmer A (2017) Radiologist shortage leaves patient care at risk, warns royal college. BMJ 359:j4683. https://doi.org/10.1136/bmj.j4683

    Article  PubMed  Google Scholar 

  8. Yarmus L, Feller-Kopman D (2012) Pneumothorax in the critically ill patient. Chest 141(4):1098–1105. https://doi.org/10.1378/chest.11-1691

    Article  PubMed  Google Scholar 

  9. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst, vol. 25,

  10. Szegedy C, Toshev A, Erhan D (2013) Deep Neural Networks for Object Detection, Adv. Neural Inf. Process. Syst, vol. 26, [Online]. Available: https://proceedings.neurips.cc/paper/2013/file/f7cade80b7cc92b991cf4d2806d6bd78-Paper.pdf

  11. Budd S, Robinson EC, Kainz B (2021) A survey on active learning and human-in-the-loop deep learning for medical image analysis. Med Image Anal 71:102062. https://doi.org/10.1016/j.media.2021.102062

    Article  PubMed  Google Scholar 

  12. Najafabadi MM, Villanustre F, Khoshgoftaar TM, Seliya N, Wald R, Muharemagic E (2015) Deep learning applications and challenges in big data analytics. J Big Data 2(1):1–21. https://doi.org/10.1186/s40537-014-0007-7

    Article  Google Scholar 

  13. Ravi D et al (2017) Deep Learning for Health Informatics. IEEE J Biomed Heal Informatics 21(1):4–21. https://doi.org/10.1109/JBHI.2016.2636665

    Article  Google Scholar 

  14. Karimi D, Dou H, Warfield SK, Gholipour A (2020) Deep learning with noisy labels: exploring techniques and remedies in medical image analysis. Med Image Anal 65. https://doi.org/10.1016/j.media.2020.101759

  15. Guan Q, Huang Y (2020) Multi-label chest X-Ray image classification via category-wise residual attention learning. Pattern Recognit Lett 130:259–266. https://doi.org/10.1016/j.patrec.2018.10.027

    Article  Google Scholar 

  16. Sridhar SS, Mootha S, Seetharaman R, Rayan AXA (2020) A study on co-occurrence of various Lung Diseases and COVID-19 by observing chest X-Ray similarity using deep convolutional neural networks. 4th Int Conf Intell Comput Data Sci ICDS 2020. https://doi.org/10.1109/ICDS50568.2020.9268711

    Article  Google Scholar 

  17. Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2017) ChestX-Ray8: Hospital-scale chest X-Ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases, Proc. – 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR vol. 2017-Janua, pp. 3462–3471, 2017, https://doi.org/10.1109/CVPR.2017.369

  18. Chang IY, Huang TY (2022) Deep learning-based classification for lung opacities in chest X-Ray radiographs through batch control and sensitivity regulation. Sci Rep 12(1):1–8. https://doi.org/10.1038/s41598-022-22506-4

    Article  CAS  Google Scholar 

  19. Ge Z, Mahapatra D, Chang X, Chen Z, Chi L, Lu H (2020) Improving multi-label chest X-Ray Disease diagnosis by exploiting Disease and health labels dependencies. Multimed Tools Appl 79:21–22. https://doi.org/10.1007/s11042-019-08260-2

    Article  Google Scholar 

  20. Jin Y, Lu H, Zhu W, Huo W (2022) Deep learning based classification of multi-label chest X-Ray images via dual-weighted metric loss, Comput. Biol. Med, vol. 157, no. November p. 106683, 2023, https://doi.org/10.1016/j.compbiomed.2023.106683

  21. Albahli S, Rauf HT, Algosaibi A, Balas VE (2021) AI-driven deep CNN approach for multilabel pathology classification using chest X-Rays. PeerJ Comput Sci 7:1–17. https://doi.org/10.7717/peerj-cs.495

    Article  Google Scholar 

  22. Souid A, Sakli N, Sakli H (2021) Classification and predictions of lung Diseases from chest x- rays using mobilenet v2. Appl Sci 11(6). https://doi.org/10.3390/app11062751

  23. Xu J, Li H, Li X (2021) MS-ANet: deep learning for automated multi-label thoracic Disease detection and classification. PeerJ Comput Sci 7:1–12. https://doi.org/10.7717/PEERJ-CS.541

    Article  CAS  Google Scholar 

  24. Rajpurkar P et al (2017) CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning, pp. 3–9, [Online]. Available: http://arxiv.org/abs/1711.05225

  25. Yao L, Poblenz E, Dagunts D, Covington B, Bernard D, Lyman K (2017) Learning to diagnose from scratch by exploiting dependencies among labels, pp. 1–12, [Online]. Available: http://arxiv.org/abs/1710.10501

  26. Zhao G, Ge W, Yu Y (2021) GraphFPN: Graph Feature Pyramid Network for Object Detection, Proc. IEEE Int. Conf. Comput. Vis, pp. 2743–2752, https://doi.org/10.1109/ICCV48922.2021.00276

  27. Shorten C, Khoshgoftaar TM (2019) A survey on Image Data Augmentation for Deep Learning. J Big Data 6(1). https://doi.org/10.1186/s40537-019-0197-0

  28. Lin T-Y, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature Pyramid Networks for Object Detection, Proc. IEEE Conf. Comput. Vis. pattern Recognit

  29. Ruby U, Theerthagiri P, Jacob J, Yendapalli V (2020) Binary cross entropy with deep learning technique for image classification. Int J Adv Trends Comput Sci Eng 9(4):5393–5397. https://doi.org/10.30534/ijatcse/2020/175942020

    Article  Google Scholar 

  30. Venkatesan R, Er MJ (2014) Multi-label classification method based on extreme learning machines, 13th Int. Conf. Control Autom. Robot. Vision, ICARCV 2014, pp. 619–624, 2014, https://doi.org/10.1109/ICARCV.2014.7064375

  31. Salunkhe UR, Mali SN (2016) Classifier Ensemble Design for Imbalanced Data Classification: A Hybrid Approach, Procedia Comput. Sci, 85, no. Cms:725–732. https://doi.org/10.1016/j.procs.2016.05.259

  32. Chen B, Li J, Lu G, Zhang D (2020) Lesion location attention guided Network for Multi-label thoracic Disease classification in chest X-Rays. IEEE J Biomed Heal Informatics 24(7):2016–2027. https://doi.org/10.1109/JBHI.2019.2952597

    Article  Google Scholar 

  33. Chen B, Li J, Lu G, Yu H, Zhang D (2020) Label co-occurrence learning with graph convolutional networks for multi-label chest X-Ray image classification. IEEE J Biomed Heal Informatics 24(8):2292–2302. https://doi.org/10.1109/JBHI.2020.2967084

    Article  Google Scholar 

  34. Chen B, Zhang Z, Li Y, Lu G, Zhang D (2022) Multi-label chest X-Ray image classification via Semantic Similarity Graph Embedding. IEEE Trans Circuits Syst Video Technol 32(4):2455–2468. https://doi.org/10.1109/TCSVT.2021.3079900

    Article  Google Scholar 

  35. Bradley AP (1997) The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognit 30(7):1145–1159. https://doi.org/10.1016/S0031-3203(96)00142-2

    Article  Google Scholar 

  36. Bhusal D, Panday DSP (2022) Multi-Label Classification of Thoracic Diseases using Dense Convolutional Network on Chest Radiographs, pp. 1–11, [Online]. Available: http://arxiv.org/abs/2202.03583

  37. Wang H, Xia Y, ChestNet (2018) : A Deep Neural Network for Classification of Thoracic Diseases on Chest Radiography, pp. 1–8, [Online]. Available: http://arxiv.org/abs/1807.03058

  38. Chen B, Li J, Guo X, Lu G (2019) DualCheXNet: dual asymmetric feature learning for thoracic Disease classification in chest X-Rays. Biomed Signal Process Control 53:101554. https://doi.org/10.1016/j.bspc.2019.04.031

    Article  Google Scholar 

  39. Xu X, Guo Q, Guo J, Yi Z (2018) DeepCXray: automatically diagnosing Diseases on chest X-Rays using deep neural networks. IEEE Access 6:66972–66982. https://doi.org/10.1109/ACCESS.2018.2875406

    Article  Google Scholar 

Download references

Funding

This study was supported by a collaborative research project between the National Taiwan University Hospital (NTUH) and the National Taiwan University of Science and Technology (NTUST) under the grant number NSTC-111-2218-E-002-043-.

Author information

Authors and Affiliations

  1. Department of Electronic and Computer Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan

    Uswatun Hasanah, Cries Avian, Muhamad Faisal, Setya Widyawan Prakosa & Jenq-Shiou Leu

  2. Department of Electrical Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan

    Nabil Bachroin

  3. Department of Bioinformatics, International Institute for Life Science, Jakarta, Indonesia

    Jeremie Theddy Darmawan

  4. Department of Internal Medicine, National Taiwan University Hospital, Taipei, Taiwan

    Chia-Ti Tsai

Authors
  1. Uswatun Hasanah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cries Avian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeremie Theddy Darmawan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nabil Bachroin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Muhamad Faisal
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Setya Widyawan Prakosa
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jenq-Shiou Leu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Chia-Ti Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

U.H. and C.A. wrote the main manuscript text. N.B., and J.T.D. prepared the models and results, C.A., S.W.P., and M.F. were responsible for reviewing and editing the manuscript. J.S.L. and C.T.T were responsible for supervising the research. All authors reviewed the manuscript.

Corresponding author

Correspondence to Jenq-Shiou Leu.

Ethics declarations

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hasanah, U., Avian, C., Darmawan, J.T. et al. CheXNet and feature pyramid network: a fusion deep learning architecture for multilabel chest X-Ray clinical diagnoses classification. Int J Cardiovasc Imaging 40, 709–722 (2024). https://doi.org/10.1007/s10554-023-03039-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10554-023-03039-x

Keywords

Search

Navigation