Skip to main content

How Does Pruning Impact Long-Tailed Multi-label Medical Image Classifiers?

  • 1889 Accesses

Part of the Lecture Notes in Computer Science book series (LNCS,volume 14224)


Pruning has emerged as a powerful technique for compressing deep neural networks, reducing memory usage and inference time without significantly affecting overall performance. However, the nuanced ways in which pruning impacts model behavior are not well understood, particularly for long-tailed, multi-label datasets commonly found in clinical settings. This knowledge gap could have dangerous implications when deploying a pruned model for diagnosis, where unexpected model behavior could impact patient well-being. To fill this gap, we perform the first analysis of pruning’s effect on neural networks trained to diagnose thorax diseases from chest X-rays (CXRs). On two large CXR datasets, we examine which diseases are most affected by pruning and characterize class “forgettability” based on disease frequency and co-occurrence behavior. Further, we identify individual CXRs where uncompressed and heavily pruned models disagree, known as pruning-identified exemplars (PIEs), and conduct a human reader study to evaluate their unifying qualities. We find that radiologists perceive PIEs as having more label noise, lower image quality, and higher diagnosis difficulty. This work represents a first step toward understanding the impact of pruning on model behavior in deep long-tailed, multi-label medical image classification. All code, model weights, and data access instructions can be found at


  • Pruning
  • Chest X-Ray
  • Imbalance
  • Long-Tailed Learning

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. 1.

    NIH ChestXRay14 can be found here, and MIMIC-CXR can be found here.


  1. Chen, L., Zhao, L., Chen, C.Y.C.: Enhancing adversarial defense for medical image analysis systems with pruning and attention mechanism. Med. Phys. 48(10), 6198–6212 (2021)

    CrossRef  Google Scholar 

  2. Davis, J., Goadrich, M.: The relationship between precision-recall and roc curves. In: ICML, pp. 233–240 (2006)

    Google Scholar 

  3. Dinsdale, N.K., Jenkinson, M., Namburete, A.I.: Stamp: simultaneous training and model pruning for low data regimes in medical image segmentation. Med. Image Anal. 81, 102583 (2022)

    CrossRef  Google Scholar 

  4. Fernandes, F.E., Yen, G.G.: Automatic searching and pruning of deep neural networks for medical imaging diagnostic. IEEE Trans. Neural Netw. Learn. Syst. 32(12), 5664–5674 (2020)

    CrossRef  Google Scholar 

  5. Fernández, A., García, S., Galar, M., Prati, R.C., Krawczyk, B., Herrera, F.: Learning from Imbalanced Data Sets. Springer, Cham (2018).

    CrossRef  Google Scholar 

  6. Frankle, J., Carbin, M.: The lottery ticket hypothesis: finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635 (2018)

  7. Hajabdollahi, M., Esfandiarpoor, R., Najarian, K., Karimi, N., Samavi, S., Soroushmehr, S.R.: Hierarchical pruning for simplification of convolutional neural networks in diabetic retinopathy classification. In: IEEE Engineering in Medicine and Biology Society (EMBC), pp. 970–973. IEEE (2019)

    Google Scholar 

  8. Han, Y., Holste, G., Ding, Y., Tewfik, A., Peng, Y., Wang, Z.: Radiomics-Guided Global-Local Transformer For Weakly Supervised Pathology Localization in Chest X-rays. IEEE Trans. Med, Imaging PP (Oct (2022)

    Google Scholar 

  9. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 770–778 (2016)

    Google Scholar 

  10. Hesamian, M.H., Jia, W., He, X., Kennedy, P.: Deep learning techniques for medical image segmentation: achievements and challenges. J. Digit. Imaging 32, 582–596 (2019)

    CrossRef  Google Scholar 

  11. Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)

  12. Holste, G., et al.: Long-tailed classification of thorax diseases on chest x-ray: A new benchmark study. In: Data Augmentation, Labelling, and Imperfections: Second MICCAI Workshop, pp. 22–32. Springer (2022).

  13. Hooker, S., Courville, A., Clark, G., Dauphin, Y., Frome, A.: What do compressed deep neural networks forget? arXiv preprint arXiv:1911.05248 (2019)

  14. Jacob, B., et al.: Quantization and training of neural networks for efficient integer-arithmetic-only inference. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2704–2713 (2018)

    Google Scholar 

  15. Jaiswal, A., Chen, T., Rousseau, J.F., Peng, Y., Ding, Y., Wang, Z.: Attend who is weak: Pruning-assisted medical image localization under sophisticated and implicit imbalances. In: WACV, pp. 4987–4996 (2023)

    Google Scholar 

  16. Jeong, T., Bollavaram, M., Delaye, E., Sirasao, A.: Neural network pruning for biomedical image segmentation. In: Medical Imaging 2021: Image-Guided Procedures, Robotic Interventions, and Modeling. vol. 11598, pp. 415–425. SPIE (2021)

    Google Scholar 

  17. Jiang, Z., Chen, T., Mortazavi, B.J., Wang, Z.: Self-damaging contrastive learning. In: International Conference on Machine Learning, pp. 4927–4939. PMLR (2021)

    Google Scholar 

  18. Johnson, A.E., et al.: MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports. Scientific Data (2019)

    Google Scholar 

  19. Kong, H., Lee, G.H., Kim, S., Lee, S.W.: Pruning-guided curriculum learning for semi-supervised semantic segmentation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5914–5923 (2023)

    Google Scholar 

  20. Kurtic, E., Alistarh, D.: Gmp*: Well-tuned global magnitude pruning can outperform most bert-pruning methods. arXiv preprint arXiv:2210.06384 (2022)

  21. LeCun, Y., Denker, J., Solla, S.: Optimal brain damage. Adv. Neural Inform. Process. Syst. 2 (1989)

    Google Scholar 

  22. Lee, N., Ajanthan, T., Torr, P.H.: Snip: single-shot network pruning based on connection sensitivity. arXiv preprint arXiv:1810.02340 (2018)

  23. Lin, M., et al.: Automated diagnosing primary open-angle glaucoma from fundus image by simulating human’s grading with deep learning. Sci. Rep. 12(1), 14080 (2022)

    CrossRef  MathSciNet  Google Scholar 

  24. Lin, X., Yu, L., Cheng, K.T., Yan, Z.: The lighter the better: rethinking transformers in medical image segmentation through adaptive pruning. arXiv preprint arXiv:2206.14413 (2022)

  25. Mahbod, A., Entezari, R., Ellinger, I., Saukh, O.: Deep neural network pruning for nuclei instance segmentation in hematoxylin and eosin-stained histological images. In: Wu, S., Shabestari, B., Xing, L. (eds.) Applications of Medical Artificial Intelligence: First International Workshop, AMAI 2022, Held in Conjunction with MICCAI 2022, Singapore, September 18, 2022, Proceedings, pp. 108–117. Springer, Cham (2022).

    CrossRef  Google Scholar 

  26. Rajpurkar, P., et al.: Chexnet: radiologist-level pneumonia detection on chest -X-rays with deep learning. arXiv preprint arXiv:1711.05225 (2017)

  27. Rethmeier, N., Augenstein, I.: Long-tail zero and few-shot learning via contrastive pretraining on and for small data. In: Computer Sciences & Mathematics Forum. vol. 3, p. 10. MDPI (2022)

    Google Scholar 

  28. Seyyed-Kalantari, L., Liu, G., McDermott, M., Chen, I.Y., Ghassemi, M.: Chexclusion: fairness gaps in deep chest x-ray classifiers. In: BIOCOMPUTING 2021: proceedings of the Pacific symposium, pp. 232–243. World Scientific (2020)

    Google Scholar 

  29. Valverde, J.M., Shatillo, A., Tohka, J.: Sauron u-net: Simple automated redundancy elimination in medical image segmentation via filter pruning. arXiv preprint arXiv:2209.13590 (2022)

  30. Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: ChestX-Ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 3462–3471 (2017)

    Google Scholar 

  31. Wu, Y., Zeng, D., Xu, X., Shi, Y., Hu, J.: FairPrune: achieving fairness through pruning for dermatological disease diagnosis. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part I, pp. 743–753. Springer, Cham (2022).

    CrossRef  Google Scholar 

  32. Yang, B., et al.: Network pruning for OCT image classification. In: Fu, H., Garvin, M.K., MacGillivray, T., Xu, Y., Zheng, Y. (eds.) Ophthalmic Medical Image Analysis: 6th International Workshop, OMIA 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 17, Proceedings, pp. 121–129. Springer, Cham (2019).

    CrossRef  Google Scholar 

  33. Zhou, S.K., et al.: A review of deep learning in medical imaging: imaging traits, technology trends, case studies with progress highlights, and future promises. Proc. IEEE 109(5), 820–838 (2021)

    CrossRef  Google Scholar 

  34. Zhu, M., Gupta, S.: To prune, or not to prune: exploring the efficacy of pruning for model compression. arXiv preprint arXiv:1710.01878 (2017)

Download references


This project was supported by the Intramural Research Programs of the National Institutes of Health, Clinical Center. It also was supported by the National Library of Medicine under Award No. 4R00LM013001, NSF CAREER Award No. 2145640, Cornell Multi-Investigator Seed Grant (Peng and Shih), and Amazon Research Award.

Author information

Authors and Affiliations


Corresponding authors

Correspondence to Yifan Peng or Zhangyang Wang .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 162 KB)

Rights and permissions

Reprints and Permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Holste, G. et al. (2023). How Does Pruning Impact Long-Tailed Multi-label Medical Image Classifiers?. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14224. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43903-2

  • Online ISBN: 978-3-031-43904-9

  • eBook Packages: Computer ScienceComputer Science (R0)