Abstract
Coronary computed angiography (CCTA) with non-invasive fractional flow reserve (FFR) calculates lesion-specific ischemia when compared with invasive FFR and can be considered for patients with stable chest pain and intermediate-grade stenoses according to recent guidelines. The objective of this study was to compare a new CCTA-based artificial-intelligence deep-learning model for FFR prediction (FFRAI) to computational fluid dynamics CT-derived FFR (FFRCT) in patients with intermediate-grade coronary stenoses with FFR as reference standard. The FFRAI model was trained with curved multiplanar-reconstruction CCTA images of 500 stenotic vessels in 413 patients, using FFR measurements as the ground truth. We included 37 patients with 39 intermediate-grade stenoses on CCTA and invasive coronary angiography, and with FFRCT and FFR measurements in this retrospective proof of concept study. FFRAI was compared with FFRCT regarding the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and diagnostic accuracy for predicting FFR ≤ 0.80. Sensitivity, specificity, PPV, NPV, and diagnostic accuracy of FFRAI in predicting FFR ≤ 0.80 were 91% (10/11), 82% (23/28), 67% (10/15), 96% (23/24), and 85% (33/39), respectively. Corresponding values for FFRCT were 82% (9/11), 75% (21/28), 56% (9/16), 91% (21/23), and 77% (30/39), respectively. Diagnostic accuracy did not differ significantly between FFRAI and FFRCT (p = 0.12). FFRAI performed similarly to FFRCT for predicting intermediate-grade coronary stenoses with FFR ≤ 0.80. These findings suggest FFRAI as a potential non-invasive imaging tool for guiding therapeutic management in these stenoses.
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Avoid common mistakes on your manuscript.
Introduction
Coronary computed tomography angiography (CCTA) is the first-line radiological investigation for patients with stable chest pain, according to guidelines issued by the European Society of Cardiology in 2019, [1] the American Heart Association (AHA) and American College of Cardiology in 2022. [2] However, coronary stenosis morphology correlates poorly with functional impact, notably in the intermediate-grade range. [3] Consequently, fractional flow reserve (FFR) measurement during invasive coronary angiography (ICA) is recommended in 2022 guidelines on coronary artery revascularization. [4] However, invasive FFR measurement is not widely performed in clinical practice given the increased risk of ICA complications and additional cost. [4] According to recent guidelines, computational fluid dynamics (CFD) based FFR calculations on CCTA (FFRCT)(HeartFlow Analysis, HeartFlow, Mountain View, CA) can be considered for patients with stable, recent-onset chest pain and intermediate grade stenosis. [2]
In contrast to CFD models, CorEx (Spimed-AI, Paris, France) is a new semi-automated program which uses a supervised artificial-intelligence (AI) CCTA-based deep-learning model (DLM) to predict FFR. This model has already been validated for CAD-RADS classification. [5] Several other AI DLMs have been designed for FFR prediction showing variable results (diagnostic accuracies between 66% and 79%), including a wide range of stenoses and without comparison with FFRCT. [6, 7] Therefore, the objective of this retrospective study was to compare a new FFRAI model to FFRCT for predicting FFR ≤ 0.80 in intermediate-grade stenoses.
Methods
Patients and study design
The study protocol was approved by our institutional review board. In compliance with Belgian law on retrospective analyses of de-identified health data, patient informed consent was not required.
We identified the study participants retrospectively by searching our institutional database for patients with intermediate-grade coronary stenoses assessed by both CCTA and ICA as previously reported. [8] The study population was not part of the training set to create FFRAI. The patients were included in a previous study designed to compare FFRCT and cardiac magnetic resonance imaging FFR estimates to FFR values. [8] Intermediate-grade stenoses were defined as 40–70% diameter reduction on quantitative coronary angiography. [4] CCTA, FFRCT computation, and ICA with pressure wire-derived FFR measurements were performed as previously described. [8] The same CCTA data were used to determine FFRAI and compared to the FFR values obtained during ICA.
Deep-learning FFRAI model
To develop the CorEx AI model (Spimed-AI), we designed a DLM architecture derived from the convolutional neural network (CNN) InceptionV3 (Google, Mountain View, CA). The model has two components, one specifically designed to extract relevant features from images and the other to classify the extracted features. This second component is composed of symmetric and asymmetric building blocks containing convolutions followed by maximum-pooling and concatenation.
To maximize performance, the model was fed with two inputs: the nine cMPR image and a vector of size 99 composed of the output of 11 other classifications from each of the nine cMPRs. The final output uses a softmax activation with a binary FFR classification as ≤ 0.80 or > 0.80, as recommended in the 2018 ESC/EACTS and 2021 ACC/AHA/SCAI Guidelines on myocardial revascularization. [4, 9]
For training, the model weights were initialized randomly according to the normal distribution, and biases were zero-initialized. Various random data augmentations were performed on the input data, such as vertical flip, horizontal flip, translations, scaling, rotation, additive Gaussian noise, additive Laplace noise, additive Poisson noise, brightness shift, and motion blur simulation. For each training sequence, only those weights maximizing the validation Matthews correlation coefficient (MCC) over all epochs were kept.
We created a training dataset comprised of 4500 curved cMPR images from 500 stenotic vessels in 413 patients who had at least one coronary stenosis, with the corresponding FFR values. For each stenotic artery, cMPR images were extracted at 40° intervals over the 360° circumference around the coronary centerline, yielding nine images in all. The centerlines were corrected manually if necessary (notably for calcified arteries) by a cardiac radiologist with 25 years of experience (JFP). Patients with stents and/or coronary-artery bypass grafting were not included in the training set. CCTA images were acquired using several different CT machines including a dual-source machine (Flash Definition, Siemens Healthineers, Erlangen, Germany), a 64-slice machine (Discovery CT 750 HD, GE, Milwaukee, WI), and a 320-detector machine (Aquilion One Genesis Edition, Canon Medical, Tochigi, Japan). Each image was annotated and classified considering the whole artery based on whether FFR was ≤ 0.80 or > 0.80, with cutoffs according to recent guidelines. [4, 9] The FFR measurements were performed by four different cardiology teams. Of the 500 stenotic coronary arteries in the training set, 125 (25%) had FFR values ≤ 0.80.
Image analysis
Integration of FFR, FFRCT, and FFRAI
FFRAI predicts the FFR value based on the entire artery as opposed to only the stenotic segment. The model uses the cMPR images of the entire artery for the prediction. To enable direct comparison of the FFRCT value to the FFR measurement distal to the stenosis of interest, the 3D FFRCT view was used to match the stenotic sites on the CCTA and ICA, as previously reported (Fig. 1). [8]
A 62-year-old patient presenting with stable chest pain. Curved multiplanar CCTA reconstructions of the left anterior descending artery show an intermediate-grade stenosis of the proximal segment (white arrow in A), confirmed by invasive coronary angiography (white arrow in B). Invasive FFR measurement was 0.61, FFRCT analysis was 0.66 (C) and FFRAI prediction was ≤ 0.80 (D). CCTA: Coronary computed tomography angiography; LAD: left anterior descending artery; FFR: Fractional flow reserve measured during invasive coronary angiography; FFRCT: fractional flow reserve (FFR) estimated by computational fluid dynamics software applied to CCTA images; FFRAI: FFR estimated by a deep learning model applied to the same CCTA images
FFRAI postprocessing and analysis
All CCTA images were sent to an Aquarius Intuition 3D-workstation (V4.5, TeraRecon, Durham, NC), which automatically segmented each contrast-enhanced coronary artery. If required, the centerlines were corrected manually by the same experienced cardiac radiologist (JFP), who was blinded to the FFR and FFRCT values. As for the validation set, nine cMPR images at 40° intervals around the centerline were acquired and uploaded to CorEx. For each of the nine images, the AI model predicted whether the FFR was ≤ 0.80 or > 0.80. The mean of the nine predictions was computed and used to classify each artery as having a predicted FFR ≤ 0.80 or > 0.80.
Statistical analysis
The statistical analyses were performed using SPSS® version 29.0 (IBM, Armonk, NY). Normally distributed continuous variables were expressed as mean ± standard deviation. For the FFRCT and FFRAI values, sensitivity, specificity, positive predictive value (PPV), and negative predictive values (NPV) were computed with their 95% confidence intervals (95%CIs), using the FFR value as the reference standard. Diagnostic accuracy was computed as the sum of true positives (TP) and true negatives (TN) over the sum of TP, TN, false positives, and false negatives. Diagnostic accuracy of the CCTA and AI methods was compared by applying the chi-square test.
Proportion of agreement was computed between FFRAI and FFR, between FFRCT and FFR and between FFRAI and FFRCT. Values of p < 0.05 were taken to indicate statistically significant differences.
Results
We retrospectively included 37 patients, 25 men and 12 women, with 39 intermediate-grade coronary stenoses. Mean age was 61 ± 9 years overall, 59 ± 9 years in the men, and 67 ± 8 years in the women. Patient selection flowchart and the main patient characteristics have been reported elsewhere [8]. Of the 39 stenoses, 23 were on left anterior descending arteries, 11 on right coronary arteries, and five on left circumflex arteries. No adverse events on ICA or contrast administration occurred.
The mean FFR value was 0.85 ± 0.10 (range, 0.60–0.99). Of the 39 stenoses, 11 (28%) had FFR values ≤ 0.80 (range, 0.60–0.79).
Both FFRAI and FFRCT were determined successfully for each stenosis. Table 1 shows the contingency table for each method. The stenosis that was a false negative by FFRAI had an FFR value of 0.78. The FFR values for the two false negatives by FFRCT were 0.79 and 0.74, respectively.
Table 2 reports the sensitivity, specificity, PPV, NPV, and diagnostic accuracy for FFRAI and FFRCT, using FFR as the reference standard. Accuracy in predicting FFR ≤ 0.80 was not significantly different between the two methods (p = 0.12). Proportion of agreement was 85% (33/39) between FFRAI and FFR, 77% (30/39) between FFRCT and FFR, and 77% (30/39) between FFRAI and FFRCT.
Discussion
In this proof-of-concept study, the AI DLM was as accurate as computational fluid dynamics analysis in predicting whether FFR was ≤ 0.80 or > 0.80 in intermediate-grade coronary artery stenoses. [10,11,12]
The 85% diagnostic accuracy of our model was higher than the 76% value obtained using a 3D, fully automatic, multiple-neural-network model based on 131 datasets. [7] Another AI approach using a support vector machine with 2D MPR images obtained from centerlines in 137 patients (192 coronary arteries) had 66–79% accuracy depending on whether prediction targeted FFR values ≤ 0.70 to ≥ 0.90.13 However, this study was not confined to arteries with intermediate-grade stenoses. Intermediate-grade stenoses are challenging, as only one third of these lesions are ischemic and require revascularization. [4] The inclusion of a broader range of stenosis severities may have increased the diagnostic accuracy of our AI model. A strength of our study is the large training set comprising 4500 labeled images from 500 stenotic arteries in 413 patients. The previously reported models used smaller training datasets, which may have influenced the diagnostic performance. [7, 13] Our model was trained on segmented cMPR images, with expert corrections, which may facilitate the learning efficiency compared to models using directly axial images. When designing AI models to perform medical tasks, great care is required given the multiple potential sources of bias, and extensive studies must then be performed to assess general applicability. [14, 15] AI models for FFRAI prediction can be expected to improve over time as increasingly large training datasets are used.
Our study is the first to compare an AI model with FFRCT in intermediate-grade coronary stenoses. FFRCT was developed over a decade ago as a non-invasive method for estimating FFR. [16] A review of randomized controlled trials in patients with intermediate-grade stenoses showed that FFRCT had high sensitivity and NPV for predicting FFR ≤ 0.80, similar to the values shown in our study. [17] One of the reviewed trials demonstrated that FFRCT was nearly as sensitive and considerably more specific (79% vs. 34%) compared to morphological evaluation on CCTA images. [17, 18] In several previous studies with larger sample sizes than ours, the diagnostic performance characteristics of FFRCT were in line with those found in our cohort. [11, 12, 16] Proportion of agreement was high between all methods and highest between FFRAI and FFR. All three false negative cases, one for FFRAI and two for FFRCT were within or around the FFR ‘grey zone’ of 0.75–0.80. [19] For such values, the appropriateness of revascularization as opposed to medical treatment alone is difficult to assess, [20] as the optimal FFR cutoff for determining revascularization remains debated. [21, 22] The clinical benefits of revascularization are not expected to be as marked for lesions with borderline FFR values, compared to stenoses of greater severity. In predominantly single-vessel disease, FFR cutoffs of 0.72 to 0.75 have been suggested for predicting myocardial ischemia. [23]
FFRCT determination has gained a place as a non-invasive tool for decreasing the number of unnecessary ICAs, thereby diminishing risks to patients and costs to healthcare systems.
AI for FFR prediction has several advantages over morphological CCTA analysis. AI models for automatically and accurately classifying stenoses as < 50% vs. ≥50% are available [5, 24, 25] and could be used for selecting those patients who require additional functional evaluation. Our model uses cMPR reconstructions for FFR prediction enabling instant FFRAI evaluation on CCTA images into the daily clinical workflow.
The main limitations of our study are the retrospective data collection, single-center recruitment, and small sample size. However, we included only patients with intermediate-grade stenoses, that is, posing the greatest management challenges. The results of AI model requires confirmation in a multicenter prospective study with a larger sample size. Second, the model uses cMPR images with manual centerline adjustment when needed. The accuracy of centerline adjustment may vary across operators, notably with poor-quality images or heavily calcified stenoses. An AI model capable of performing automatic contouring and segmentation, would eliminate this drawback. Finally, our AI model uses a dichotomous outcome, predicting FFR as ≤ 0.80 or > 0.80, a continuous outcome would be more clinically relevant especially in borderline lesions. A global-artery evaluation by FFRAI might help to assess extensive lesions but would require a dedicated AI model.
In conclusion, FFRAI had similar diagnostic accuracy as FFRCT for predicting intermediate-grade stenoses with FFR ≤ 0.80. This method may be an accurate non-invasive imaging modality for guiding the therapeutic management of patients with intermediate-grade stenoses. Further assessment of these preliminary findings in a larger, prospective population is required.
Data availability
No datasets were generated or analysed during the current study.
References
Knuuti J, Wijns W, Saraste A et al (2020) 2019 ESC guidelines for the diagnosis and management of chronic coronary syndromes. Eur Heart J 41:407–477
Gulati M, Levy PD, Mukherjee D et al (2022) 2021 AHA/ACC/ASE/CHEST/SAEM/SCCT/SCMR Guideline for the evaluation and diagnosis of chest pain: a report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice guidelines. J Cardiovasc Comput Tomogr 16:54–122
Ghekiere O, Dewilde W, Bellekens M et al (2015) Diagnostic performance of quantitative coronary computed tomography angiography and quantitative coronary angiography to predict hemodynamic significance of intermediate-grade stenoses. Int J Cardiovasc Imaging 31:1651–1661
Lawton JS, Tamis-Holland JE, Bangalore S et al (2022) 2021 ACC/AHA/SCAI Guideline for coronary artery revascularization: a report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice guidelines. Circulation 145:E18–E114
Paul JF, Rohnean A, Giroussens H et al (2022) Evaluation of a deep learning model on coronary CT angiography for automatic stenosis detection. Diagn Interv Imaging 103:316–323
Zreik M, Van Hamersvelt RW, Wolterink JM, Leiner T et al (2019) A recurrent CNN for automatic detection and classification of coronary artery plaque and stenosis in coronary CT angiography. IEEE Trans Med Imaging 38:1588–1598
Kumamaru KK, Fujimoto S, Otsuka Y et al (2020) Diagnostic accuracy of 3D deep-learning-based fully automated estimation of patient-level minimum fractional flow reserve from coronary computed tomography angiography. Eur Heart J Cardiovasc Imaging 21:437–445
Ghekiere O, Bielen J, Leipsic J et al (2019) Correlation of FFR-derived from CT and stress perfusion CMR with invasive FFR in intermediate-grade coronary artery stenosis. Int J Cardiovasc Imaging 35:559–568
Neumann FJ, Sousa-Uva M, Ahlsson A et al (2019) 2018 ESC/EACTS guidelines on myocardial revascularization. Eur Heart J 40:87–165
Tesche C, De Cecco CN, Albrecht MH et al (2017) Coronary CT angiography-derived fractional flow reserve. Radiology 285:17–33
Kruk M, Wardziak Ł, Demkow M et al (2016) Workstation-based calculation of CTA-Based FFR for Intermediate Stenosis. JACC Cardiovasc Imaging 9:690–699
Nakazato R, Park HB, Berman DS et al (2013) Noninvasive fractional flow reserve derived from computed tomography angiography for coronary lesions of intermediate stenosis severity: results from the DeFACTO study. Circ Cardiovasc Imaging 6:881–889
Zreik M, Van Hamersvelt RW, Khalili N et al (2020) Deep learning analysis of coronary arteries in cardiac CT angiography for detection of patients requiring invasive coronary angiography. IEEE Trans Med Imaging 39:1545–1557
Futoma J, Simons M, Panch T et al (2020) The myth of generalisability in clinical research and machine learning in health care. Lancet Digit Health 2:e489
Liao J, Huang L, Qu M et al (2022) Artificial intelligence in coronary ct angiography: current status and future prospects. Front Cardiovasc Med 9:896366
Min JK, Leipsic J, Pencina MJ et al (2012) Diagnostic accuracy of fractional flow reserve from anatomic CT angiography. JAMA 308:1237–1245
Raja J, Seitz MP, Yedlapati N, Khouzam RN (2021) Can computed fractional Flow Reserve Coronary CT Angiography (FFRCT) offer an Accurate Noninvasive comparison to invasive coronary angiography (ICA)? The noninvasive CATH. A Comprehensive Review. Curr Probl Cardiol. ;46
Nørgaard BL, Leipsic J, Gaur S et al (2014) Diagnostic performance of noninvasive fractional flow reserve derived from coronary computed tomography angiography in suspected coronary artery disease: the NXT trial (analysis of coronary blood flow using ct angiography: next steps). J Am Coll Cardiol 63:1145–1155
Kang DY, Ahn JM, Lee CH et al (2018) Deferred vs. performed revascularization for coronary stenosis with grey-zone fractional flow reserve values: data from the IRIS-FFR registry. Eur Heart J 39:1610–1619
Johnson NP, Tóth GG, Lai D et al (2014) Prognostic value of fractional flow reserve: linking physiologic severity to clinical outcomes. J Am Coll Cardiol 64:1641–1654
Zimmermann FM, Ferrara A, Johnson NP et al (2015) Deferral vs. performance of percutaneous coronary intervention of functionally non-significant coronary stenosis: 15-year follow-up of the DEFER trial. Eur Heart J 36:3182–3188
De Bruyne B, Pijls NHJ, Kalesan B et al (2012) Fractional flow reserve-guided PCI versus medical therapy in stable coronary disease. New Engl J Med 367:991–1001
Berry C, Corcoran D, Hennigan B et al (2015) Fractional flow reserve-guided management in stable coronary disease and acute myocardial infarction: recent developments. Eur Heart J 36:3155–3164c
Lin A, Manral N, McElhinney P et al (2022) Deep learning-enabled coronary CT angiography for plaque and stenosis quantification and cardiac risk prediction: an international multicentre study. Lancet Digit Health 4:e256–e265
Choi AD, Marques H, Kumar V et al (2021) CT evaluation by artificial intelligence for atherosclerosis, stenosis and vascular morphology (CLARIFY): A multi-center, international study. J Cardiovasc Comput Tomogr 15:470–476
Acknowledgements
This research is part of the Limburg Clinical Research Center (LCRC) Uhasselt-ZOL-Jessa, supported by Hasselt University, Jessa Hospital and Ziekenhuis Oost-Limburg.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Olivier Ghekiere, Benjamin Peters and Wouter Franssen. The first draft of the manuscript was written by Benjamin Peters and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
Jean-François Paul is a co-founder of Spimed-AI. None of the other authors has any conflicts of interest to declare. This research did not receive financial support from any funding agencies in the public, commercial, or not-for-profit sectors.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Peters, B., Paul, JF., Symons, R. et al. Invasive fractional-flow-reserve prediction by coronary CT angiography using artificial intelligence vs. computational fluid dynamics software in intermediate-grade stenosis. Int J Cardiovasc Imaging (2024). https://doi.org/10.1007/s10554-024-03173-0
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10554-024-03173-0