Abstract
Pruning has emerged as a powerful technique for compressing deep neural networks, reducing memory usage and inference time without significantly affecting overall performance. However, the nuanced ways in which pruning impacts model behavior are not well understood, particularly for long-tailed, multi-label datasets commonly found in clinical settings. This knowledge gap could have dangerous implications when deploying a pruned model for diagnosis, where unexpected model behavior could impact patient well-being. To fill this gap, we perform the first analysis of pruning’s effect on neural networks trained to diagnose thorax diseases from chest X-rays (CXRs). On two large CXR datasets, we examine which diseases are most affected by pruning and characterize class “forgettability” based on disease frequency and co-occurrence behavior. Further, we identify individual CXRs where uncompressed and heavily pruned models disagree, known as pruning-identified exemplars (PIEs), and conduct a human reader study to evaluate their unifying qualities. We find that radiologists perceive PIEs as having more label noise, lower image quality, and higher diagnosis difficulty. This work represents a first step toward understanding the impact of pruning on model behavior in deep long-tailed, multi-label medical image classification. All code, model weights, and data access instructions can be found at https://github.com/VITA-Group/PruneCXR.
Keywords
- Pruning
- Chest X-Ray
- Imbalance
- Long-Tailed Learning
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Chen, L., Zhao, L., Chen, C.Y.C.: Enhancing adversarial defense for medical image analysis systems with pruning and attention mechanism. Med. Phys. 48(10), 6198–6212 (2021)
Davis, J., Goadrich, M.: The relationship between precision-recall and roc curves. In: ICML, pp. 233–240 (2006)
Dinsdale, N.K., Jenkinson, M., Namburete, A.I.: Stamp: simultaneous training and model pruning for low data regimes in medical image segmentation. Med. Image Anal. 81, 102583 (2022)
Fernandes, F.E., Yen, G.G.: Automatic searching and pruning of deep neural networks for medical imaging diagnostic. IEEE Trans. Neural Netw. Learn. Syst. 32(12), 5664–5674 (2020)
Fernández, A., García, S., Galar, M., Prati, R.C., Krawczyk, B., Herrera, F.: Learning from Imbalanced Data Sets. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98074-4
Frankle, J., Carbin, M.: The lottery ticket hypothesis: finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635 (2018)
Hajabdollahi, M., Esfandiarpoor, R., Najarian, K., Karimi, N., Samavi, S., Soroushmehr, S.R.: Hierarchical pruning for simplification of convolutional neural networks in diabetic retinopathy classification. In: IEEE Engineering in Medicine and Biology Society (EMBC), pp. 970–973. IEEE (2019)
Han, Y., Holste, G., Ding, Y., Tewfik, A., Peng, Y., Wang, Z.: Radiomics-Guided Global-Local Transformer For Weakly Supervised Pathology Localization in Chest X-rays. IEEE Trans. Med, Imaging PP (Oct (2022)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 770–778 (2016)
Hesamian, M.H., Jia, W., He, X., Kennedy, P.: Deep learning techniques for medical image segmentation: achievements and challenges. J. Digit. Imaging 32, 582–596 (2019)
Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)
Holste, G., et al.: Long-tailed classification of thorax diseases on chest x-ray: A new benchmark study. In: Data Augmentation, Labelling, and Imperfections: Second MICCAI Workshop, pp. 22–32. Springer (2022). https://doi.org/10.1007/978-3-031-17027-0_3
Hooker, S., Courville, A., Clark, G., Dauphin, Y., Frome, A.: What do compressed deep neural networks forget? arXiv preprint arXiv:1911.05248 (2019)
Jacob, B., et al.: Quantization and training of neural networks for efficient integer-arithmetic-only inference. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2704–2713 (2018)
Jaiswal, A., Chen, T., Rousseau, J.F., Peng, Y., Ding, Y., Wang, Z.: Attend who is weak: Pruning-assisted medical image localization under sophisticated and implicit imbalances. In: WACV, pp. 4987–4996 (2023)
Jeong, T., Bollavaram, M., Delaye, E., Sirasao, A.: Neural network pruning for biomedical image segmentation. In: Medical Imaging 2021: Image-Guided Procedures, Robotic Interventions, and Modeling. vol. 11598, pp. 415–425. SPIE (2021)
Jiang, Z., Chen, T., Mortazavi, B.J., Wang, Z.: Self-damaging contrastive learning. In: International Conference on Machine Learning, pp. 4927–4939. PMLR (2021)
Johnson, A.E., et al.: MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports. Scientific Data (2019)
Kong, H., Lee, G.H., Kim, S., Lee, S.W.: Pruning-guided curriculum learning for semi-supervised semantic segmentation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5914–5923 (2023)
Kurtic, E., Alistarh, D.: Gmp*: Well-tuned global magnitude pruning can outperform most bert-pruning methods. arXiv preprint arXiv:2210.06384 (2022)
LeCun, Y., Denker, J., Solla, S.: Optimal brain damage. Adv. Neural Inform. Process. Syst. 2 (1989)
Lee, N., Ajanthan, T., Torr, P.H.: Snip: single-shot network pruning based on connection sensitivity. arXiv preprint arXiv:1810.02340 (2018)
Lin, M., et al.: Automated diagnosing primary open-angle glaucoma from fundus image by simulating human’s grading with deep learning. Sci. Rep. 12(1), 14080 (2022)
Lin, X., Yu, L., Cheng, K.T., Yan, Z.: The lighter the better: rethinking transformers in medical image segmentation through adaptive pruning. arXiv preprint arXiv:2206.14413 (2022)
Mahbod, A., Entezari, R., Ellinger, I., Saukh, O.: Deep neural network pruning for nuclei instance segmentation in hematoxylin and eosin-stained histological images. In: Wu, S., Shabestari, B., Xing, L. (eds.) Applications of Medical Artificial Intelligence: First International Workshop, AMAI 2022, Held in Conjunction with MICCAI 2022, Singapore, September 18, 2022, Proceedings, pp. 108–117. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-17721-7_12
Rajpurkar, P., et al.: Chexnet: radiologist-level pneumonia detection on chest -X-rays with deep learning. arXiv preprint arXiv:1711.05225 (2017)
Rethmeier, N., Augenstein, I.: Long-tail zero and few-shot learning via contrastive pretraining on and for small data. In: Computer Sciences & Mathematics Forum. vol. 3, p. 10. MDPI (2022)
Seyyed-Kalantari, L., Liu, G., McDermott, M., Chen, I.Y., Ghassemi, M.: Chexclusion: fairness gaps in deep chest x-ray classifiers. In: BIOCOMPUTING 2021: proceedings of the Pacific symposium, pp. 232–243. World Scientific (2020)
Valverde, J.M., Shatillo, A., Tohka, J.: Sauron u-net: Simple automated redundancy elimination in medical image segmentation via filter pruning. arXiv preprint arXiv:2209.13590 (2022)
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: ChestX-Ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 3462–3471 (2017)
Wu, Y., Zeng, D., Xu, X., Shi, Y., Hu, J.: FairPrune: achieving fairness through pruning for dermatological disease diagnosis. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part I, pp. 743–753. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16431-6_70
Yang, B., et al.: Network pruning for OCT image classification. In: Fu, H., Garvin, M.K., MacGillivray, T., Xu, Y., Zheng, Y. (eds.) Ophthalmic Medical Image Analysis: 6th International Workshop, OMIA 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 17, Proceedings, pp. 121–129. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32956-3_15
Zhou, S.K., et al.: A review of deep learning in medical imaging: imaging traits, technology trends, case studies with progress highlights, and future promises. Proc. IEEE 109(5), 820–838 (2021)
Zhu, M., Gupta, S.: To prune, or not to prune: exploring the efficacy of pruning for model compression. arXiv preprint arXiv:1710.01878 (2017)
Acknowledgments
This project was supported by the Intramural Research Programs of the National Institutes of Health, Clinical Center. It also was supported by the National Library of Medicine under Award No. 4R00LM013001, NSF CAREER Award No. 2145640, Cornell Multi-Investigator Seed Grant (Peng and Shih), and Amazon Research Award.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Holste, G. et al. (2023). How Does Pruning Impact Long-Tailed Multi-label Medical Image Classifiers?. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14224. Springer, Cham. https://doi.org/10.1007/978-3-031-43904-9_64
Download citation
DOI: https://doi.org/10.1007/978-3-031-43904-9_64
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43903-2
Online ISBN: 978-3-031-43904-9
eBook Packages: Computer ScienceComputer Science (R0)
-
Published in cooperation with
http://miccai.org/
