Skip to main content
SpringerLink
Log in

PE-Ynet: a novel attention-based multi-task model for pulmonary embolism detection using CT pulmonary angiography (CTPA) scan images

  • Scientific Paper
  • Published:
Physical and Engineering Sciences in Medicine Aims and scope Submit manuscript

Abstract

Pulmonary Embolism (PE) has diverse manifestations with different etiologies such as venous thromboembolism, septic embolism, and paradoxical embolism. In this study, a novel attention-based multi-task model is proposed for PE segmentation and detection from Computed Tomography Pulmonary Angiography (CTPA) images. A Y-Net architecture is used to implement this model, which facilitates segmentation and classification jointly, improving performance and efficiency. It is leveraged with Multi Head Attention (MHA), which allows the model to focus on important regions of the image while suppressing irrelevant information, improving the accuracy of the segmentation and detection tasks. The proposed PE-YNet model is tested with two public datasets, achieving a maximum mean detection and segmentation accuracy of 99.89% and 99.83%, respectively, on the CAD-PE challenge dataset. Similarly, it also achieves a detection accuracy of 99.75% and a segmentation accuracy of 99.81% on the FUMPE dataset. Additionally, sensitivity analysis also shows a high sensitivity of 0.9885 for the localization error ɛ = 0 for the CAD-PE dataset, demonstrating the model’s robustness against false predictions compared to state-of-the-art models. Further, this model also exhibits lower inference time, size, and memory usage compared to representative models. An automated PE-YNet tool can assist physicians with PE diagnosis, treatment, and prognosis monitoring in the clinical management of CoVID-19.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Poyiadji N, Cormier P, Patel PY, Hadied MO, Bhargava P, Khanna K, Song T (2020) Acute pulmonary embolism and CoVID-19. Radiology 297(3):E335–E338

    Article  PubMed  Google Scholar 

  2. Bompard F, Monnier H, Saab I, Tordjman M, Abdoul H, Fournier L, Revel MP (2020) Pulmonary embolism in patients with CoVID-19 pneumonia. Eur Respir J. https://doi.org/10.1183/13993003.01365-2020

    Article  PubMed  PubMed Central  Google Scholar 

  3. Konstantinides SV, Meyer G, Becattini C, Bueno H, Geersing GJ, Harjola VP, Zamorano JL (2020) Eur heart J 41(4):543–603

    Article  PubMed  Google Scholar 

  4. Murugappan M, Prakash NB, Jeya R, Mohanarathinam A, Hemalakshmi GR, Mahmud M (2022) A novel few-shot classification framework for diabetic retinopathy detection and grading. Measurement 200:111485

    Article  Google Scholar 

  5. Prakash NB, Murugappan M, Hemalakshmi GR, Jayalakshmi M, Mahmud M (2021) Deep transfer learning for COVID-19 detection and infection localization with superpixel based segmentation. Sustain Cities Soc 75:103252

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Murugappan M, Bourisly AK, Krishnan PT, Maruthapillai V, Muthusamy H (2021) Artificial intelligence based covid-19 detection using medical imaging methods: a review. Comput Model Imag SARS-CoV-and COVID-19. https://doi.org/10.1201/9781003142584-1-1

    Article  Google Scholar 

  7. Soffer S, Klang E, Shimon O, Barash Y, Cahan N, Greenspana H, Konen E (2021) Deep learning for pulmonary embolism detection on computed tomography pulmonary angiogram: a systematic review and meta-analysis. Sci Rep 11(1):1–8

    Article  Google Scholar 

  8. Casey K, Iteen A, Nicolini R, Auten J (2020) CoVID-19 pneumonia with hemoptysis: acute segmental pulmonary emboli associated with novel coronavirus infection. Am J Emerg Med 38(7):1544-e1

    Article  PubMed Central  Google Scholar 

  9. Danzi GB, Loffi M, Galeazzi G, Gherbesi E (2020) Acute pulmonary embolism and CoVID-19 pneumonia: a random association? Eur Heart J 41(19):1858–1858

    Article  CAS  PubMed  Google Scholar 

  10. Kwee RM, Adams HJ, Kwee TC (2021) Pulmonary embolism in patients with CoVID-19 and value of D-dimer assessment: a meta-analysis. Eur Radiol. https://doi.org/10.1007/s00330-021-08003-8

    Article  PubMed  PubMed Central  Google Scholar 

  11. García-Ortega A, Oscullo G, Calvillo P, López-Reyes R, Méndez R, Gómez-Olivas JD, Martínez-García MÁ (2021) Incidence, risk factors, and thrombotic load of pulmonary embolism in patients hospitalized for CoVID-19 infection. J Infect 82(2):261–269

    Article  PubMed  PubMed Central  Google Scholar 

  12. Bouma H, Sonnemans JJ, Vilanova A, Gerritsen FA (2009) Automatic detection of pulmonary embolism in CTPA images. IEEE Trans Med Imaging 28(8):1223–1230

    Article  PubMed  Google Scholar 

  13. Pioped Investigators (1990) Value of the ventilation/perfusion scan in acute pulmonary embolism. Results of the prospective investigation of pulmonary embolism diagnosis (PIOPED). JAMA 263(20):2753–2759

    Article  Google Scholar 

  14. Tajbakhsh N, Gotway MB, Liang J (2015) Computer-aided pulmonary embolism detection using a novel vessel-aligned multi-planar image representation and convolutional neural networks. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, pp 62–69

    Google Scholar 

  15. Rajan D, Beymer D, Abedin S, Dehghan E (2019) Pi-PE: A Pipeline for Pulmonary Embolism Detection using Sparsely Annotated 3D CT Images. arXiv preprint arXiv:1910.02175.

  16. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, pp 234–241

    Google Scholar 

  17. Xingjian SHI, Chen Z, Wang H, Yeung DY, Wong WK, Woo WC (2015) Convolutional LSTM network: a machine learning approach for precipitation nowcasting. Adv Neural Info Process Syst 28:802–810

    Google Scholar 

  18. Yang X, Lin Y, Su J, Wang X, Li X, Lin J, Cheng KT (2019) A two-stage convolutional neural network for pulmonary embolism detection from ctpa images. IEEE Access 7:84849–84857

    Article  Google Scholar 

  19. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 770–778

  20. CAD-PE Challenge. [Online]. Available: http://www.cad-pe.org

  21. Lin Y, Su J, Wang X, Li X, Liu J, Cheng KT, Yang X (2019) Automated pulmonary embolism detection from CTPA images using an end-to-end convolutional neural network. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, pp 280–288

    Google Scholar 

  22. Tajbakhsh N, Shin JY, Gotway MB, Liang J (2019) Computer-aided detection and visualization of pulmonary embolism using a novel, compact, and discriminative image representation. Med Image Anal 58:101541

    Article  PubMed  PubMed Central  Google Scholar 

  23. Cano-Espinosa C, Cazorla M, González G (2020) Computer aided detection of pulmonary embolism using multi-slice multi-axial segmentation. Appl Sci 10(8):2945

    Article  CAS  Google Scholar 

  24. Huang SC, Kothari T, Banerjee I, Chute C, Ball RL, Borus N (2020) PENet—a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging. npj Digital Med 3(1):1–9

    Google Scholar 

  25. Iandola F N, Han S, Moskewicz M W, Ashraf K, Dally W J, Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size. arXiv preprint arXiv:1602.07360

  26. Huang SC, Pareek A, Zamanian R, Banerjee I, Lungren MP (2020) Multimodal fusion with deep neural networks for leveraging CT imaging and electronic health record: a case-study in pulmonary embolism detection. Sci Rep 10(1):1–9

    Article  Google Scholar 

  27. Weifang L, Liu M, Xiaojuan G, Peiyao Z, Zhang L, Rongguo Z, Sheng X (2020) Evaluation of acute pulmonary embolism and clot burden on CTPA with deep learning. Eur Radiol 30(6):3567–3575

    Article  Google Scholar 

  28. Weikert T, Winkel DJ, Bremerich J, Stieltjes B, Parmar V, Sauter AW, Sommer G (2020) Automated detection of pulmonary embolism in CT pulmonary angiograms using an AI-powered algorithm. Eur Radiol 30(12):6545–6553

    Article  PubMed  Google Scholar 

  29. Kiourt C, Feretzakis G, Dalamarinis K, Kalles D, Pantos G, Papadopoulos I & Sakagianni A (2021) Pulmonary embolism identification in computerized tomography pulmonary angiography scans with deep learning technologies in CoVID-19 patients. arXiv preprint arXiv:2105.11187.

  30. Raj ANJ, Zhu H, Khan A, Zhuang Z, Yang Z, Mahesh VG, Karthik G (2021) ADID-UNET—a segmentation model for CoVID-19 infection from lung CT scans. Peer J Comput Sci 7:e349

    Article  Google Scholar 

  31. Yuan H, Shao Y, Liu Z, Wang H (2021) An improved faster R-CNN for pulmonary embolism detection from CTPA images. IEEE Access 9:105382–105392

    Article  Google Scholar 

  32. González C, Ranem A, Pinto dos Santos D et al (2023) Lifelong nnU-Net: a framework for standardized medical continual learning. Sci Rep 13:9381. https://doi.org/10.1038/s41598-023-34484-2

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Guo J, Liu X, Chen Y, Zhang S, Tao G, Yu H, Wang N (2022) AANet: artery-aware network for pulmonary embolism detection in CTPA images. Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings Part I. Springer Nature Switzerland, Cham, pp 473–483

    Google Scholar 

  34. Masoudi M, Pourreza HR, Saadatmand-Tarzjan M, Eftekhari N, Zargar FS, Rad MP (2018) A new dataset of computed-tomography angiography images for computer-aided detection of pulmonary embolism. Scientific data 5:180180

    Article  PubMed  PubMed Central  Google Scholar 

  35. Mehta S, Mercan E, Bartlett J, Weaver D, Elmore JG, Shapiro L (2018) Y-Net: joint segmentation and classification for diagnosis of breast biopsy images. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, pp 893–901

    Google Scholar 

  36. Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.

  37. Cai Y and Wang Y (2022) MA-Unet: an improved version of Unet based on multi-scale and attention mechanism for medical image segmentation, Proc. SPIE 12167, Third International Conference on Electronics and Communication; Network and Computer Technology (ECNCT 2021), 121670X (7 March 2022) https://doi.org/10.1117/12.2628519

  38. Selvaraju R R, Cogswell M, Das A, Vedantam R, Parikh D and Batra D (2017). Grad-cam: visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626.

Download references

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. School of Computing Science and Engineering, Vellore Institute of Technology, Bhopal, Madhya Pradesh, India

    G. R. Hemalakshmi

  2. Intelligent Signal Processing (ISP) Research Lab, Department of Electronics and Communication Engineering, Kuwait College of Science and Technology, Block 4, 13133, Doha, Kuwait

    M. Murugappan

  3. Department of Electronics and Communication Engineering, School of Engineering, Vels Institute of Sciences, Technology, and Advanced Studies, Chennai, Tamil Nadu, India

    M. Murugappan

  4. Center of Excellence for Unmanned Aerial Systems (CoEUAS), Universiti Malaysia Perlis, 02600, Arau, Perlis, Malaysia

    M. Murugappan

  5. Biomedical Equipment Technology, College of Applied Medical Sciences, Majmaah University, Al Majma’ah, Saudi Arabia

    Mohamed Yacin Sikkandar

  6. Department of Biomedical Engineering, Mepco Schlenk Engineering College, Sivakasi, India

    D. Santhi

  7. Department of Electrical and Electronics Engineering, National Engineering College, Kovilpatti, India

    N. B. Prakash

  8. Department of Electronics and Communication Engineering, Karpagam Academy of Higher Education, Coimbatore, Tamil Nadu, 641021, India

    A. Mohanarathinam

Authors
  1. G. R. Hemalakshmi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Murugappan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed Yacin Sikkandar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. D. Santhi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. N. B. Prakash
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. A. Mohanarathinam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Murugappan.

Ethics declarations

Conflict of interest

All the authors declare that they have no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hemalakshmi, G.R., Murugappan, M., Sikkandar, M.Y. et al. PE-Ynet: a novel attention-based multi-task model for pulmonary embolism detection using CT pulmonary angiography (CTPA) scan images. Phys Eng Sci Med (2024). https://doi.org/10.1007/s13246-024-01410-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13246-024-01410-3

Keywords

Search

Navigation