Skip to main content
SpringerLink
Log in

Automated abdominal CT contrast phase detection using an interpretable and open-source artificial intelligence algorithm

  • Technical Developments
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

To develop and validate an open-source artificial intelligence (AI) algorithm to accurately detect contrast phases in abdominal CT scans.

Materials and methods

Retrospective study aimed to develop an AI algorithm trained on 739 abdominal CT exams from 2016 to 2021, from 200 unique patients, covering 1545 axial series. We performed segmentation of five key anatomic structures—aorta, portal vein, inferior vena cava, renal parenchyma, and renal pelvis—using TotalSegmentator, a deep learning-based tool for multi-organ segmentation, and a rule-based approach to extract the renal pelvis. Radiomics features were extracted from the anatomical structures for use in a gradient-boosting classifier to identify four contrast phases: non-contrast, arterial, venous, and delayed. Internal and external validation was performed using the F1 score and other classification metrics, on the external dataset “VinDr-Multiphase CT”.

Results

The training dataset consisted of 172 patients (mean age, 70 years ± 8, 22% women), and the internal test set included 28 patients (mean age, 68 years ± 8, 14% women). In internal validation, the classifier achieved an accuracy of 92.3%, with an average F1 score of 90.7%. During external validation, the algorithm maintained an accuracy of 90.1%, with an average F1 score of 82.6%. Shapley feature attribution analysis indicated that renal and vascular radiodensity values were the most important for phase classification.

Conclusion

An open-source and interpretable AI algorithm accurately detects contrast phases in abdominal CT scans, with high accuracy and F1 scores in internal and external validation, confirming its generalization capability.

Clinical relevance statement

Contrast phase detection in abdominal CT scans is a critical step for downstream AI applications, deploying algorithms in the clinical setting, and for quantifying imaging biomarkers, ultimately allowing for better diagnostics and increased access to diagnostic imaging.

Key Points

  • Digital Imaging and Communications in Medicine labels are inaccurate for determining the abdominal CT scan phase.

  • AI provides great help in accurately discriminating the contrast phase.

  • Accurate contrast phase determination aids downstream AI applications and biomarker quantification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Abbreviations

AI:

Artificial intelligence

AUPRC:

Area under precision-recall curve

AUROC:

Area under the receiver operating characteristic curve

CT:

Computed tomography

DICOM:

Digital Imaging and Communications in Medicine

IVC:

Inferior vena cava

SHAP:

Shapley additive explanations

XGBoost:

Extreme gradient boosting

References

  1. Kammerer S, Höink AJ, Wessling J et al (2015) Abdominal and pelvic CT: Is positive enteric contrast still necessary? Results of a retrospective observational study. Eur Radiol 25:669–678

    Article  CAS  PubMed  Google Scholar 

  2. Shirkhoda A (1991) Diagnostic pitfalls in abdominal CT. Radiographics 11:969–1002

    Article  CAS  PubMed  Google Scholar 

  3. Radiya K, Joakimsen HL, Mikalsen KØ, Aahlin EK, Lindsetmo R-O, Mortensen KE (2023) Performance and clinical applicability of machine learning in liver computed tomography imaging: a systematic review. Eur Radiol. https://doi.org/10.1007/s00330-023-09609-w

  4. Rocha BA, Ferreira LC, Vianna LGR et al (2022) Contrast phase recognition in liver computer tomography using deep learning. Sci Rep 12:20315

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Schieda N, Nguyen K, Thornhill RE, McInnes MDF, Wu M, James N (2020) Importance of phase enhancement for machine learning classification of solid renal masses using texture analysis features at multi-phasic CT. Abdom Radiol (NY) 45:2786–2796

    Article  PubMed  Google Scholar 

  6. Han S, Hwang SI, Lee HJ (2019) The classification of renal cancer in 3-phase CT images using a deep learning method. J Digit Imaging 32:638–643

    Article  PubMed  PubMed Central  Google Scholar 

  7. Boutin RD, Kaptuch JM, Bateni CP, Chalfant JS, Yao L (2016) Influence of IV contrast administration on CT measures of muscle and bone attenuation: implications for sarcopenia and osteoporosis evaluation. AJR Am J Roentgenol 207:1046–1054

    Article  PubMed  Google Scholar 

  8. Pickhardt PJ, Lauder T, Pooler BD et al (2016) Effect of IV contrast on lumbar trabecular attenuation at routine abdominal CT: correlation with DXA and implications for opportunistic osteoporosis screening. Osteoporos Int 27:147–152

    Article  CAS  PubMed  Google Scholar 

  9. Rühling S, Navarro F, Sekuboyina A et al (2022) Automated detection of the contrast phase in MDCT by an artificial neural network improves the accuracy of opportunistic bone mineral density measurements. Eur Radiol 32:1465–1474

    Article  PubMed  Google Scholar 

  10. Jang S, Graffy PM, Ziemlewicz TJ, Lee SJ, Summers RM, Pickhardt PJ (2019) Opportunistic osteoporosis screening at routine abdominal and thoracic CT: normative L1 trabecular attenuation values in more than 20 000 adults. Radiology 291:360–367

    Article  PubMed  Google Scholar 

  11. Ye Z, Qian JM, Hosny A et al (2022) Deep learning-based detection of intravenous contrast enhancement on CT scans. Radiol Artif Intell 4:e210285

    Article  PubMed  PubMed Central  Google Scholar 

  12. Na S, Sung YS, Ko Y et al (2022) Development and validation of an ensemble artificial intelligence model for comprehensive imaging quality check to classify body parts and contrast enhancement. BMC Med Imaging 22:87

    Article  PubMed  PubMed Central  Google Scholar 

  13. Muhamedrahimov R, Bar A, Laserson J, Akselrod-Ballin A, Elnekave E (2022) Using machine learning to identify intravenous contrast phases on computed tomography. Comput Methods Programs Biomed 215:106603

    Article  PubMed  Google Scholar 

  14. Blankemeier L, Yao L, Long J et al (2024) Skeletal muscle area on CT: determination of an optimal height scaling power and testing for mortality risk prediction. AJR Am J Roentgenol 222:e2329889

    Article  PubMed  Google Scholar 

  15. Chartrand G, Cheng PM, Vorontsov E et al (2017) Deep learning: a primer for radiologists. Radiographics 37:2113–2131

    Article  PubMed  Google Scholar 

  16. Fuentes-Orrego JM, Pinho D, Kulkarni NM, Agrawal M, Ghoshhajra BB, Sahani DV (2014) New and evolving concepts in CT for abdominal vascular imaging. Radiographics 34:1363–1384

    Article  PubMed  Google Scholar 

  17. Blankemeier L, Desai A, Chaves JMZ et al (2023) Comp2Comp: open-source body composition assessment on computed tomography. Preprint at https://doi.org/10.48550/ARXIV.2302.06568

  18. Gauriau R, Bridge C, Chen L et al (2020) Using DICOM metadata for radiological image series categorization: a feasibility study on large clinical brain MRI datasets. J Digit Imaging 33:747–762

    Article  PubMed  PubMed Central  Google Scholar 

  19. Reis EP, De Paiva JPQ, Da Silva MCB et al (2022) BRAX, Brazilian labeled chest x-ray dataset. Sci Data 9:487

    Article  PubMed  PubMed Central  Google Scholar 

  20. Wasserthal J, Breit H-C, Meyer MT et al (2023) TotalSegmentator: robust segmentation of 104 anatomic structures in CT images. Radiol Artif Intell 5:e230024

    Article  PubMed  PubMed Central  Google Scholar 

  21. Chowdhary CL, Acharjya DP (2020) Segmentation and feature extraction in medical imaging: a systematic review. Procedia Comput Sci 167:26–36

    Article  Google Scholar 

  22. Van Timmeren JE, Cester D, Tanadini-Lang S, Alkadhi H, Baessler B (2020) Radiomics in medical imaging—“how-to” guide and critical reflection. Insights Imaging 11:91

    Article  PubMed  PubMed Central  Google Scholar 

  23. Chen T, Guestrin C (2016) XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, San Francisco, 13–17

  24. Dao BT, Nguyen TV, Pham HH, Nguyen HQ (2022) Phase recognition in contrast‐enhanced CT scans based on deep learning and random sampling. Med Phys 49:4518–4528

    Article  PubMed  Google Scholar 

  25. Clopper CJ, Pearson ES (1934) The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 26:404–413

    Article  Google Scholar 

Download references

Acknowledgements

We would like to acknowledge the team behind the TotalSegmentator open-source project, for their work on Abdominal CT segmentations. We would like to acknowledge funding from the NIH, Stanford Precision Health and Integrated Diagnostics Seed Grant, Stanford Human-Centered AI, and Center for AI in Medicine and Image Seed Grant.

Funding

This study has received funding from NIH NHLBI R01 HL167974, Stanford Precision Health and Integrated Diagnostics Seed Grant; Stanford Human-Centered AI, and Center for AI in Medicine and Image Seed Grant.

Author information

Authors and Affiliations

  1. Department of Radiology, Stanford University, Stanford, CA, USA

    Eduardo Pontes Reis, Juan Manuel Zambrano Chaves, Malte Engmann Kjeldskov Jensen, Sally Yao, Marc H. Willis, Scott Adams, Robert D. Boutin & Akshay S. Chaudhari

  2. Center for Artificial Intelligence in Medicine & Imaging (AIMI), Stanford University, Stanford, CA, USA

    Eduardo Pontes Reis

  3. Hospital Israelita Albert Einstein, Sao Paulo, Brazil

    Eduardo Pontes Reis, Cesar Augusto Madid Truyts & Edson Amaro Jr

  4. Department of Electrical Engineering, Stanford University, Stanford, CA, USA

    Louis Blankemeier

  5. Department of Biomedical Data Science, Stanford University, Stanford, CA, USA

    Juan Manuel Zambrano Chaves & Akshay S. Chaudhari

Authors
  1. Eduardo Pontes Reis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Louis Blankemeier
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Juan Manuel Zambrano Chaves
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Malte Engmann Kjeldskov Jensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sally Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Cesar Augusto Madid Truyts
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Marc H. Willis
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Scott Adams
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Edson Amaro Jr
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Robert D. Boutin
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Akshay S. Chaudhari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eduardo Pontes Reis.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Eduardo Pontes Reis.

Conflict of interest

The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board (Stanford University School of Medicine) approval was obtained.

Study subjects or cohorts overlap

The study subjects or cohorts have not been previously reported.

Methodology

  • Retrospective

  • Diagnostic study

  • Performed at one institution

Additional information

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Reis, E.P., Blankemeier, L., Zambrano Chaves, J.M. et al. Automated abdominal CT contrast phase detection using an interpretable and open-source artificial intelligence algorithm. Eur Radiol (2024). https://doi.org/10.1007/s00330-024-10769-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00330-024-10769-6

Keywords

Search

Navigation