Abstract
With more than 60,000 deaths annually in the United States, Pulmonary Embolism (PE) is among the most fatal cardiovascular diseases. It is caused by an artery blockage in the lung; confirming its presence is time-consuming and is prone to over-diagnosis. The utilization of automated PE detection systems is critical for diagnostic accuracy and efficiency. In this study we propose a two-stage attention-based CNN-LSTM network for predicting PE, its associated type (chronic, acute) and corresponding location (leftsided, rightsided or central) on computed tomography (CT) examinations. We trained our model on the largest available public Computed Tomography Pulmonary Angiogram PE dataset (RSNA-STR Pulmonary Embolism CT (RSPECT) Dataset, N = 7279 CT studies) and tested it on an in-house curated dataset of N = 106 studies. Our framework mirrors the radiologic diagnostic process via a multi-slice approach so that the accuracy and pathologic sequela of true pulmonary emboli may be meticulously assessed, enabling physicians to better appraise the morbidity of a PE when present. Our proposed method outperformed a baseline CNN classifier and a single-stage CNN-LSTM network, achieving an AUC of 0.95 on the test set for detecting the presence of PE in the study.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
RSNA-STR pulmonary embolism CT (RSPECT) dataset, copyright RSNA. https://www.rsna.org/education/ai-resources-and-training/ai-image-challenge/rsna-pe-detection-challenge-2020
Anam, C., Budi, W., Haryanto, F., Fujibuchi, T., Dougherty, G.: A novel multiple-windows blending of CT images in red-green-blue (RGB) color space: phantoms study. Sci. Vis. 11(5) (2019)
Bejnordi, B.E., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318(22), 2199–2210 (2017)
Colak, E., et al.: The RSNA pulmonary embolism CT dataset. Radiol.: Artif. Intell. 3(2), e200254 (2021)
Coudray, N., et al.: Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24(10), 1559–1567 (2018)
Friedman, T., Winokur, R.S., Quencer, K.B., Madoff, D.C.: Patient assessment: clinical presentation, imaging diagnosis, risk stratification, and the role of pulmonary embolism response team. In: Seminars in Interventional Radiology, vol. 35, pp. 116–121. Thieme Medical Publishers (2018)
Ghaye, B., Ghuysen, A., Bruyere, P.J., D’Orio, V., Dondelinger, R.F.: Can CT pulmonary angiography allow assessment of severity and prognosis in patients presenting with pulmonary embolism? What the radiologist needs to know. Radiographics 26(1), 23–39 (2006)
Huang, S.C., et al.: PENet-a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging. NPJ Digit. Med. 3(1), 1–9 (2020)
Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015)
Ilse, M., Tomczak, J., Welling, M.: Attention-based deep multiple instance learning. In: International Conference on Machine Learning, pp. 2127–2136. PMLR (2018)
Khorrami, M., et al.: Changes in CT radiomic features associated with lymphocyte distribution predict overall survival and response to immunotherapy in non-small cell lung cancer. Cancer Immunol. Res. 8(1), 108–119 (2020)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Lankeit, M.: Always think of the right ventricle, even in “low-risk” pulmonary embolism (2017)
Liang, J., Bi, J.: Computer aided detection of pulmonary embolism with tobogganing and mutiple instance classification in CT pulmonary angiography. In: Karssemeijer, N., Lelieveldt, B. (eds.) IPMI 2007. LNCS, vol. 4584, pp. 630–641. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-73273-0_52
Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(11) (2008)
Masutani, Y., MacMahon, H., Doi, K.: Computerized detection of pulmonary embolism in spiral CT angiography based on volumetric image analysis. IEEE Trans. Med. Imaging 21(12), 1517–1523 (2002)
Özkan, H., Osman, O., Şahin, S., Boz, A.F.: A novel method for pulmonary embolism detection in CTA images. Comput. Methods Programs Biomed. 113(3), 757–766 (2014)
Park, S.C., Chapman, B.E., Zheng, B.: A multistage approach to improve performance of computer-aided detection of pulmonary embolisms depicted on CT images: preliminary investigation. IEEE Trans. Biomed. Eng. 58(6), 1519–1527 (2010)
Rajan, D., Beymer, D., Abedin, S., Dehghan, E.: Pi-PE: a pipeline for pulmonary embolism detection using sparsely annotated 3D CT images. In: Machine Learning for Health Workshop, pp. 220–232. PMLR (2020)
Shi, L., Rajan, D., Abedin, S., Yellapragada, M.S., Beymer, D., Dehghan, E.: Automatic diagnosis of pulmonary embolism using an attention-guided framework: a large-scale study. In: Medical Imaging with Deep Learning, pp. 743–754. PMLR (2020)
Singh, G., et al.: Radiomics and radiogenomics in gliomas: a contemporary update. Br. J. Cancer 1–17 (2021)
Sirinukunwattana, K., Raza, S.E.A., Tsang, Y.W., Snead, D.R., Cree, I.A., Rajpoot, N.M.: Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans. Med. Imaging 35(5), 1196–1206 (2016)
Tan, M., Le, Q.: Efficientnet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114. PMLR (2019)
Wittram, C., Maher, M.M., Yoo, A.J., Kalra, M.K., Shepard, J.A.O., McLoud, T.C.: CT angiography of pulmonary embolism: diagnostic criteria and causes of misdiagnosis. Radiographics 24(5), 1219–1238 (2004)
Yang, X., Lin, Y., Su, J., Wang, X., Li, X., Lin, J., Cheng, K.T.: A two-stage convolutional neural network for pulmonary embolism detection from CTPA images. IEEE Access 7, 84849–84857 (2019)
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)
Zhou, C., et al.: Preliminary investigation of computer-aided detection of pulmonary embolism in threedimensional computed tomography pulmonary angiography images. Acad. Radiol. 12(6), 782 (2005)
Acknowledgment
Dimitris Samaras was partially supported by the Partner University Fund, the SUNY2020 Infrastructure Transportation Security Center, and a gift from Adobe.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Suman, S. et al. (2021). Attention Based CNN-LSTM Network for Pulmonary Embolism Prediction on Chest Computed Tomography Pulmonary Angiograms. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12907. Springer, Cham. https://doi.org/10.1007/978-3-030-87234-2_34
Download citation
DOI: https://doi.org/10.1007/978-3-030-87234-2_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87233-5
Online ISBN: 978-3-030-87234-2
eBook Packages: Computer ScienceComputer Science (R0)
