Skip to main content
SpringerLink
Log in

Multi-label neural architecture search for chest radiography image classification

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Chest radiography remain the global standard for diagnosing pulmonary diseases. Despite numerous research efforts, medical professionals still face challenges in rapidly and accurately analyzing multiple diseases on a single chest radiography. Moreover, traditional deep learning methods suffer from complexities in design and prolonged processing times. To address these issues, we propose a multi-label neural architecture search (MLNAS) approach. Primarily intended for multi-label chest radiography image classification, MLNAS employs automated modeling, data augmentation, and threshold calculation strategies to improve the accuracy of chest radiography image classification and enhance result interpretation. Furthermore, MLNAS demonstrates the potential for application in other multi-label medical image classification domains. Experimental results indicate that MLNAS achieves state-of-the-art prediction accuracy for 9 out of 14 lung diseases. This novel approach presents a new solution for computer-aided diagnosis of chest X-rays.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Algorithm 2
Algorithm 3
Fig. 2
Algorithm 4
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availibility

The data and material that has been used in this article is cited with references at their respective place.

References

  1. Ahmadi, Z., Kramer, S.: A label compression method for online multi-label classification. Pattern Recogn. Lett. 111, 64–71 (2018). https://doi.org/10.1016/j.patrec.2018.04.015

    Article  ADS  Google Scholar 

  2. Annarumma, M., Withey, S.J., Bakewell, R.J., et al.: Automated triaging of adult chest radiographs with deep artificial neural networks. Radiology 291(1), 196–202 (2019). https://doi.org/10.1148/radiol.2018180921

    Article  PubMed  Google Scholar 

  3. Baemani, M.J., Monadjemi, A., Moallem, P.: Detection of respiratory abnormalities using artificial neural networks. J. Comput. Sci. 4(8), 663 (2008)

    Article  Google Scholar 

  4. Baker, B., Gupta, O., Naik, N., et al.: Designing neural network architectures using reinforcement learning (2016). arXiv preprint arXiv:1611.02167

  5. Boutell, M.R., Luo, J., Shen, X., et al.: Learning multi-label scene classification. Pattern Recogn. 37(9), 1757–1771 (2004). https://doi.org/10.1016/j.patcog.2004.03.009

    Article  Google Scholar 

  6. Charte, F., Rivera, A.J., del Jesus, M.J., et al.: Mlsmote: approaching imbalanced multilabel learning through synthetic instance generation. Knowl.-Based Syst. 89, 385–397 (2015). https://doi.org/10.1016/j.knosys.2015.07.019

    Article  Google Scholar 

  7. Chawla, N.V., Bowyer, K.W., Hall, L.O., et al.: Smote: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002). https://doi.org/10.1613/jair.953

    Article  Google Scholar 

  8. Chen, X., Xie, L., Wu, J., et al.: Progressive differentiable architecture search: Bridging the depth gap between search and evaluation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 1294–1303 (2019a)

  9. Chen, ZM., Wei, XS., Wang, P., et al.: Multi-label image recognition with graph convolutional networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019b)

  10. Dey, R., Lu, Z., Hong, Y.: Diagnostic classification of lung nodules using 3d neural networks. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 774–778 (2018). https://doi.org/10.1109/ISBI.2018.8363687

  11. Elisseeff, A., Weston, J.: A kernel method for multi-labelled classification. Adv. Neural Inf. Process. Syst. 14 (2001)

  12. Gao, M., Bagci, U., Lu, L., et al.: Holistic classification of ct attenuation patterns for interstitial lung diseases via deep convolutional neural networks. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 6(1), 1–6 (2018). https://doi.org/10.1080/21681163.2015.1124249

    Article  PubMed  Google Scholar 

  13. Grauman, K., Darrell, T.: The pyramid match kernel: Discriminative classification with sets of image features. In: Tenth IEEE International Conference on Computer Vision (ICCV’05) vol. 12, pp. 1458–1465 (2005)

  14. Guan, Q., Huang, Y.: Multi-label chest x-ray image classification via category-wise residual attention learning. Pattern Recogn. Lett. 130, 259–266 (2020). https://doi.org/10.1016/j.patrec.2018.10.027

    Article  ADS  Google Scholar 

  15. He, Y., Yang, D., Roth, H., et al.: Dints: Differentiable neural network topology search for 3d medical image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5841–5850 (2021)

  16. Hermoza, R., Maicas, G., Nascimento, J.C., et al.: Region proposals for saliency map refinement for weakly-supervised disease localisation and classification. Med. Image Comput. Comput. Assist. Interv. MICCAI 2020, 539–549 (2020). https://doi.org/10.1007/978-3-030-59725-2_52

    Article  Google Scholar 

  17. Hua, Y., Mou, L., Zhu, X.X.: Recurrently exploring class-wise attention in a hybrid convolutional and bidirectional lstm network for multi-label aerial image classification. ISPRS J. Photogramm. Remote. Sens. 149, 188–199 (2019). https://doi.org/10.1016/j.isprsjprs.2019.01.015

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  18. Iandola, F., Moskewicz, M., Karayev, S., et al.: Densenet: implementing efficient convnet descriptor pyramids (2014). arXiv preprint arXiv:1404.1869

  19. Jin, J., Nakayama, H.: Annotation order matters: Recurrent image annotator for arbitrary length image tagging. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 2452–2457 (2016)

  20. Jozefowicz, R., Zaremba, W., Sutskever, I.: An empirical exploration of recurrent network architectures. In: International Conference on Machine Learning, pp. 2342–2350 (2015)

  21. Kelly, B.: The chest radiograph. Ulst. Med. J. 81(3), 143 (2012)

    Google Scholar 

  22. Khobragade, S., Tiwari, A., Patil, C., et al.: Automatic detection of major lung diseases using chest radiographs and classification by feed-forward artificial neural network. In: 2016 IEEE 1st International Conference on Power Electronics, Intelligent Control and Energy Systems (ICPEICES), IEEE, pp. 1–5 (2016)

  23. Kim, E., Kim, S., Seo, M., et al.: Xprotonet: diagnosis in chest radiography with global and local explanations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15719–15728 (2021)

  24. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. (2012). https://doi.org/10.1145/3065386

    Article  Google Scholar 

  25. Li, Z., Wang, C., Han, M., et al.: Thoracic disease identification and localization with limited supervision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8290–8299 (2018a)

  26. Li, Z., Wang, C., Han, M., et al.: Thoracic disease identification and localization with limited supervision. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8290–8299 (2018b)

  27. Li, L., Cao, P., Yang, J., et al.: Modeling global and local label correlation with graph convolutional networks for multi-label chest x-ray image classification. Med. Biol. Eng. Comput. 60(9), 2567–2588 (2022). https://doi.org/10.1007/s11517-022-02604-1

    Article  PubMed  Google Scholar 

  28. Liu, C., Chen, LC., Schroff, F., et al.: Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 82–92 (2019a)

  29. Liu, H., Simonyan, K., Yang, Y.: DARTS: Differentiable architecture search. In: International Conference on Learning Representations (2019b)

  30. Liu, J., Zhao, G., Fei, Y., et al.: Align, attend and locate: Chest x-ray diagnosis via contrast induced attention network with limited supervision. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10632–10641 (2019c)

  31. Loza Mencia, E., Furnkranz, J .: Pairwise learning of multilabel classifications with perceptrons. In: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pp. 2899–2906 (2008). https://doi.org/10.1109/IJCNN.2008.4634206

  32. Lyu, F., Wu, Q., Hu, F., et al.: Attend and imagine: multi-label image classification with visual attention and recurrent neural networks. IEEE Trans. Multimedia 21(8), 1971–1981 (2019). https://doi.org/10.1109/TMM.2019.2894964

    Article  Google Scholar 

  33. Ma, C., Wang, H., Hoi, SC.: Multi-label thoracic disease image classification with cross-attention networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 730–738 (2019a)

  34. Ma, C., Wang, H., Hoi, S.C.H.: Multi-label thoracic disease image classification with cross-attention networks. Med. Image Comput. Comput. Assist. Interv. MICCAI 2019, 730–738 (2019). https://doi.org/10.1007/978-3-030-32226-7_81

    Article  Google Scholar 

  35. Miranda, E., Aryuni, M., Irwansyah, E.: A survey of medical image classification techniques. In: 2016 International Conference on Information Management and Technology (ICIMTech), pp. 56–61 (2016). https://doi.org/10.1109/ICIMTech.2016.7930302

  36. Oliveira, H., dos Santos, J.: Deep transfer learning for segmentation of anatomical structures in chest radiographs. In: 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 204–211 (2018). https://doi.org/10.1109/SIBGRAPI.2018.00033

  37. O’Neill, D., Xue, B., Zhang, M.: Evolutionary neural architecture search for high-dimensional skip-connection structures on densenet style networks. IEEE Trans. Evol. Comput. 25(6), 1118–1132 (2021). https://doi.org/10.1109/TEVC.2021.3083315

    Article  Google Scholar 

  38. Pauletto, L., Amini, M.R., Babbar, R., et al.: Neural Architecture Search for extreme multi-label classification: an evolutionary approach. In: The Fourth International Workshop on Automation in Machine Learning (AutoML 2020) (2020)

  39. Pham, H., Guan, M., Zoph, B., et al.: Efficient neural architecture search via parameters sharing. In: International Conference on Machine Learning, pp. 4095–4104 (2018)

  40. Real, E., Moore, S., Selle, A., et al.: Large-scale evolution of image classifiers. In: International Conference on Machine Learning, pp. 2902–2911 (2017)

  41. Real, E., Aggarwal, A., Huang, Y., et al.: Regularized evolution for image classifier architecture search. Proc. AAAI Conf. Artif. Intell. 33(01), 4780–4789 (2019). https://doi.org/10.1609/aaai.v33i01.33014780

    Article  Google Scholar 

  42. Ruuskanen, O., Lahti, E., Jennings, L.C., et al.: Viral pneumonia. Lancet 377(9773), 1264–1275 (2011). https://doi.org/10.1016/S0140-6736(10)61459-6

    Article  PubMed  PubMed Central  Google Scholar 

  43. Shen, Y., Gao, M.: Dynamic routing on deep neural network for thoracic disease classification and sensitive area localization. In: Machine Learning in Medical Imaging, pp. 389–397 (2018). https://doi.org/10.1007/978-3-030-00919-9_45

  44. Sim, Y., Chung, M.J., Kotter, E., et al.: Deep convolutional neural network-based software improves radiologist detection of malignant lung nodules on chest radiographs. Radiology 294(1), 199–209 (2020). https://doi.org/10.1148/radiol.2019182465

    Article  PubMed  Google Scholar 

  45. Tai, Y.: A deep learning based workflow for detection of lung nodules with chest radiograph (2021). arXiv preprint arXiv:2112.10184

  46. Tang, Y., Wang, X., Harrison, A.P., et al.: Attention-guided curriculum learning for weakly supervised classification and localization of thoracic diseases on chest radiographs. In: Shi, Y., Suk, H.I., Liu, M. (eds.) Machine Learning in Medical Imaging, pp. 249–258. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00919-9_29

    Chapter  Google Scholar 

  47. Tarekegn, A.N., Giacobini, M., Michalak, K.: A review of methods for imbalanced multi-label classification. Pattern Recogn. 118, 107965 (2021). https://doi.org/10.1016/j.patcog.2021.107965

    Article  Google Scholar 

  48. Targ, S., Almeida, D., Lyman, K.: Resnet in resnet: generalizing residual architectures (2016). arXiv preprint arXiv:1603.08029

  49. Tsoumakas, G., Vlahavas, I.: Random k-labelsets: an ensemble method for multilabel classification. Mach. Learn. ECML 2007, 406–417 (2007)

    Google Scholar 

  50. Wang, J., Yang, Y., Mao, J., et al.: Cnn-rnn: a unified framework for multi-label image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

  51. Wang, X., Peng, Y., Lu, L., et al.: Chestx-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2097–2106 (2017)

  52. Wang, W., Liang, D., Chen, Q., et al.: Medical image classification using deep learning. Healthc. Paradigms Appl, Deep Learn (2020). https://doi.org/10.1007/978-3-030-32606-7_3

    Book  Google Scholar 

  53. Xie, L., Yuille, A.: Genetic cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1379–1388 (2017)

  54. Xu, Y., Xie, L., Zhang, X., et al.: Pc-darts: Partial channel connections for memory-efficient architecture search. In: International Conference on Learning Representations (2020)

  55. Yang, Z., Wang, Y., Chen, X., et al.: Cars: continuous evolution for efficient neural architecture search. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

  56. Yang, Y., Wei, J., Yu, Z., et al.: A trustworthy neural architecture search framework for pneumonia image classification utilizing blockchain technology. J. Supercomput. (2023). https://doi.org/10.1007/s11227-023-05541-4

    Article  PubMed  PubMed Central  Google Scholar 

  57. Yao, L., Prosky, J., Poblenz, E., et al.: Weakly supervised medical diagnosis and localization from multiple resolutions (2018). arXiv preprint arXiv:1803.07703

  58. Zhang, ML., Zhang, K.: Multi-label learning by exploiting label dependency. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining—KDD ’10. Association for Computing Machinery, New York, NY, USA, KDD ’10, pp. 999–1008 (2010). https://doi.org/10.1145/1835804.1835930

  59. Zhang, M.L., Zhou, Z.H.: A k-nearest neighbor based algorithm for multi-label classification. In: 2005 IEEE International Conference on Granular Computing, vol. 2, pp. 718–721 (2005). https://doi.org/10.1109/GRC.2005.1547385

  60. Zhang, M.L., Zhou, Z.H.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26(8), 1819–1837 (2014). https://doi.org/10.1109/TKDE.2013.39

    Article  Google Scholar 

  61. Zhang, X., Hou, P., Zhang, X., et al.: Neural architecture search with random labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10907–10916 (2021)

  62. Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning (2016). arXiv preprint arXiv:1611.01578

  63. Zoph, B., Vasudevan, V., Shlens, J., et al.: Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8697–8710 (2018)

Download references

Funding

This work was supported by the Natural Science Foundation of Gansu Province, Grant number no. 22JR11RA042. We would like to thank the Editor-in-Chief for handling our paper. We would also like to thank the reviewers for their time and effort in providing constructive suggestions for our paper.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Lanzhou University, Lanzhou, 730000, Gansu, China

    Yi Yang, Jiaxuan Wei, Zhixuan Yu & Ruisheng Zhang

  2. Information Center, The First Hospital of Lanzhou University, Lanzhou, 730000, Gansu, China

    Zhixuan Yu

Authors
  1. Yi Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiaxuan Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhixuan Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ruisheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yi Yang: development or design of methodology, designing computer programs, implementation of the computer code and supporting algorithms and preparation, creation and presentation of the published work, specifically writing the initial draft. Jiaxuan Wei and Zhixuan Yu: preparation, creation and presentation of the published work by those from the original research group, specifically critical review, commentary or revision. Ruisheng Zhang: oversight and leadership responsibility for the research activity planning and execution, including mentorship external to the core team.

Corresponding authors

Correspondence to Jiaxuan Wei or Ruisheng Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethics approval

This work does not involve any work related to ethics.

Consent to publish

All authors consent to publication.

Additional information

Communicated by F. Wu.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, Y., Wei, J., Yu, Z. et al. Multi-label neural architecture search for chest radiography image classification. Multimedia Systems 30, 8 (2024). https://doi.org/10.1007/s00530-023-01215-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00530-023-01215-6

Keywords

Search

Navigation