Abstract
Pulmonary embolism (PE) represents a thrombus (“blood clot”), usually originating from a lower extremity vein, that travels to the blood vessels in the lung, causing vascular obstruction and in some patients, death. This disorder is commonly diagnosed using CT pulmonary angiography (CTPA). Deep learning holds great promise for the computer-aided CTPA diagnosis (CAD) of PE. However, numerous competing methods for a given task in the deep learning literature exist, causing great confusion regarding the development of a CAD PE system. To address this confusion, we present a comprehensive analysis of competing deep learning methods applicable to PE diagnosis using CTPA at the both image and exam levels. At the image level, we compare convolutional neural networks (CNNs) with vision transformers, and contrast self-supervised learning (SSL) with supervised learning, followed by an evaluation of transfer learning compared with training from scratch. At the exam level, we focus on comparing conventional classification (CC) with multiple instance learning (MIL). Our extensive experiments consistently show: (1) transfer learning consistently boosts performance despite differences between natural images and CT scans, (2) transfer learning with SSL surpasses its supervised counterparts; (3) CNNs outperform vision transformers, which otherwise show satisfactory performance; and (4) CC is, surprisingly, superior to MIL. Compared with the state of the art, our optimal approach provides an AUC gain of 0.2% and 1.05% for image-level and exam-level, respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We pre-trained SeXception on ImageNet and the others are taken from PyTorch.
References
U.S. Department of Health and Human Services Food and Drug Administration: The Surgeon General’s Call to Action to Prevent Deep Vein Thrombosis and Pulmonary Embolism (2008)
Stein, P.D., et al.: Multidetector computed tomography for acute pulmonary embolism. N. Engl. J. Med. 354(22), 2317–2327 (2006)
Lucassen, W.A.M., et al.: Concerns in using multi-detector computed tomography for diagnosing pulmonary embolism in daily practice. A cross-sectional analysis using expert opinion as reference standard. Thromb. Res. 131(2), 145–149 (2013)
Masutani, Y., MacMahon, H., Doi, K.: Computerized detection of pulmonary embolism in spiral CT angiography based on volumetric image analysis. IEEE TMI 21(12), 1517–1523 (2002)
Liang, J., Bi, J.: Computer aided detection of pulmonary embolism with tobogganing and mutiple instance classification in CT pulmonary angiography. In: Karssemeijer, N., Lelieveldt, B. (eds.) IPMI 2007. LNCS, vol. 4584, pp. 630–641. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-73273-0_52
Zhou, C., et al.: Computer-aided detection of pulmonary embolism in computed tomographic pulmonary angiography (CTPA): performance evaluation with independent data sets. Med. Phys. 36(8), 3385–3396 (2009)
Tajbakhsh, N., Gotway, M.B., Liang, J.: Computer-aided pulmonary embolism detection using a novel vessel-aligned multi-planar image representation and convolutional neural networks. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9350, pp. 62–69. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24571-3_8
Rajan, D., et al.: PI-PE: a pipeline for pulmonary embolism detection using sparsely annotated 3D CT images. In: Proceedings of the Machine Learning for Health NeurIPS Workshop, pp. 220–232. PMLR, 13 December 2020
Huang, S.-C., et al.: PENet-a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging (2020)
Zhou, Z., et al.: Models genesis: generic autodidactic models for 3D medical image analysis. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11767, pp. 384–393. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32251-9_42
Zhou, Z.: Towards annotation-efficient deep learning for computer-aided diagnosis. PhD thesis, Arizona State University (2021)
Zhou, Z., Shin, J.Y., Gurudu, S.R., Gotway, M.B., Liang, J.: Active, continual fine tuning of convolutional neural networks for reducing annotation efforts. Med. Image Anal., 101997 (2021)
Zhou, Z., Shin, J., Zhang, L., Gurudu, S., Gotway, M., Liang, J.: Fine-tuning convolutional neural networks for biomedical image analysis: actively and incrementally. In: CVPR, pp. 7340–7349 (2017)
Colak, E., et al.: The RSNA pulmonary embolism CT dataset. Radiol. Artif. Intell. 3(2) (2021)
Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)
Deng, S., et al.: Deep learning in digital pathology image analysis: a survey. Frontiers Med. 14(4), 470–487 (2020). https://doi.org/10.1007/s11684-020-0782-9
Devlin, J., et al.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Brown, T.B., et al.: Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)
Dosovitskiy, A., et al.: An image is worth 16 \(\times \) 16 words: transformers for image recognition at scale. In: ICLR (2021)
Han, K., et al.: Transformer in transformer (2021)
Touvron, H., et al.: Training data-efficient image transformers & distillation through attention. arXiv preprint arXiv:2012.12877 (2020)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Haghighi, F., Taher, M.R.H., Zhou, Z., Gotway, M.B., Liang, J.: Transferable visual words: exploiting the semantics of anatomical patterns for self-supervised learning (2021)
Tajbakhsh, N., et al.: Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE TMI 35(5), 1299–1312 (2016)
Shin, H.-C., et al.: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE TMI 35(5), 1285–1298 (2016)
Jing, L., Tian, Y.: Self-supervised visual feature learning with deep neural networks: a survey. TPAMI, 1 (2020)
Haghighi, F., Hosseinzadeh Taher, M.R., Zhou, Z., Gotway, M.B., Liang, J.: Learning semantics-enriched representation via self-discovery, self-classification, and self-restoration. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 137–147. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_14
Ilse, M., Tomczak, J.M., M.: Welling. Attention-based deep multiple instance learning. arXiv preprint arXiv:1802.04712, 2018
RSNA STR Pulmonary Embolism Detection (2020). https://www.kaggle.com/c/rsna-str-pulmonary-embolism-detection/overview. Accessed 21 June 2021
RSNA STR Pulmonary Embolism Detection (2020). https://www.kaggle.com/c/rsna-str-pulmonary-embolism-detection/discussion/194145. Accessed 21 June 2021
Devlin, J., et al.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the NAACL: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186 (2019)
Asano, Y.M., et al.: Self-labelling via simultaneous clustering and representation learning. arXiv preprint arXiv:1911.05371 (2019)
Caron, M., Bojanowski, P., Joulin, A., Douze, M.: Deep clustering for unsupervised learning of visual features. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 139–156. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01264-9_9
Zbontar, J., et al.: Barlow twins: self-supervised learning via redundancy reduction. arXiv preprint arXiv:2103.03230 (2021)
Hu, D., et al.: How well self-supervised pre-training performs with streaming data? arXiv preprint arXiv:2104.12081 (2021)
Carbonneau, M.-A., et al.: Multiple instance learning: a survey of problem characteristics and applications. arXiv preprint arXiv:1612.03365 (2016)
Gildenblat, J., contributors: Pytorch library for cam methods. https://github.com/jacobgil/pytorch-grad-cam (2021)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: CVPR, pp. 7132–7141 (2018)
Acknowledgments
This research has been supported partially by ASU and Mayo Clinic through a Seed Grant and an Innovation Grant, and partially by the NIH under Award Number R01HL128785. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. This work has utilized the GPUs provided partially by the ASU Research Computing and partially by the Extreme Science and Engineering Discovery Environment (XSEDE) funded by the National Science Foundation (NSF) under grant number ACI-1548562. We thank Ruibin Feng for helping us some experiments. The content of this paper is covered by patents pending.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Islam, N.U., Gehlot, S., Zhou, Z., Gotway, M.B., Liang, J. (2021). Seeking an Optimal Approach for Computer-Aided Pulmonary Embolism Detection. In: Lian, C., Cao, X., Rekik, I., Xu, X., Yan, P. (eds) Machine Learning in Medical Imaging. MLMI 2021. Lecture Notes in Computer Science(), vol 12966. Springer, Cham. https://doi.org/10.1007/978-3-030-87589-3_71
Download citation
DOI: https://doi.org/10.1007/978-3-030-87589-3_71
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87588-6
Online ISBN: 978-3-030-87589-3
eBook Packages: Computer ScienceComputer Science (R0)
