Introduction

Coronary computed tomography angiography (CCTA) is the first-line radiological investigation for patients with stable chest pain, according to guidelines issued by the European Society of Cardiology in 2019, [1] the American Heart Association (AHA) and American College of Cardiology in 2022. [2] However, coronary stenosis morphology correlates poorly with functional impact, notably in the intermediate-grade range. [3] Consequently, fractional flow reserve (FFR) measurement during invasive coronary angiography (ICA) is recommended in 2022 guidelines on coronary artery revascularization. [4] However, invasive FFR measurement is not widely performed in clinical practice given the increased risk of ICA complications and additional cost. [4] According to recent guidelines, computational fluid dynamics (CFD) based FFR calculations on CCTA (FFRCT)(HeartFlow Analysis, HeartFlow, Mountain View, CA) can be considered for patients with stable, recent-onset chest pain and intermediate grade stenosis. [2]

In contrast to CFD models, CorEx (Spimed-AI, Paris, France) is a new semi-automated program which uses a supervised artificial-intelligence (AI) CCTA-based deep-learning model (DLM) to predict FFR. This model has already been validated for CAD-RADS classification. [5] Several other AI DLMs have been designed for FFR prediction showing variable results (diagnostic accuracies between 66% and 79%), including a wide range of stenoses and without comparison with FFRCT. [6, 7] Therefore, the objective of this retrospective study was to compare a new FFRAI model to FFRCT for predicting FFR ≤ 0.80 in intermediate-grade stenoses.

Methods

Patients and study design

The study protocol was approved by our institutional review board. In compliance with Belgian law on retrospective analyses of de-identified health data, patient informed consent was not required.

We identified the study participants retrospectively by searching our institutional database for patients with intermediate-grade coronary stenoses assessed by both CCTA and ICA as previously reported. [8] The study population was not part of the training set to create FFRAI. The patients were included in a previous study designed to compare FFRCT and cardiac magnetic resonance imaging FFR estimates to FFR values. [8] Intermediate-grade stenoses were defined as 40–70% diameter reduction on quantitative coronary angiography. [4] CCTA, FFRCT computation, and ICA with pressure wire-derived FFR measurements were performed as previously described. [8] The same CCTA data were used to determine FFRAI and compared to the FFR values obtained during ICA.

Deep-learning FFRAI model

To develop the CorEx AI model (Spimed-AI), we designed a DLM architecture derived from the convolutional neural network (CNN) InceptionV3 (Google, Mountain View, CA). The model has two components, one specifically designed to extract relevant features from images and the other to classify the extracted features. This second component is composed of symmetric and asymmetric building blocks containing convolutions followed by maximum-pooling and concatenation.

To maximize performance, the model was fed with two inputs: the nine cMPR image and a vector of size 99 composed of the output of 11 other classifications from each of the nine cMPRs. The final output uses a softmax activation with a binary FFR classification as ≤ 0.80 or > 0.80, as recommended in the 2018 ESC/EACTS and 2021 ACC/AHA/SCAI Guidelines on myocardial revascularization. [4, 9]

For training, the model weights were initialized randomly according to the normal distribution, and biases were zero-initialized. Various random data augmentations were performed on the input data, such as vertical flip, horizontal flip, translations, scaling, rotation, additive Gaussian noise, additive Laplace noise, additive Poisson noise, brightness shift, and motion blur simulation. For each training sequence, only those weights maximizing the validation Matthews correlation coefficient (MCC) over all epochs were kept.

We created a training dataset comprised of 4500 curved cMPR images from 500 stenotic vessels in 413 patients who had at least one coronary stenosis, with the corresponding FFR values. For each stenotic artery, cMPR images were extracted at 40° intervals over the 360° circumference around the coronary centerline, yielding nine images in all. The centerlines were corrected manually if necessary (notably for calcified arteries) by a cardiac radiologist with 25 years of experience (JFP). Patients with stents and/or coronary-artery bypass grafting were not included in the training set. CCTA images were acquired using several different CT machines including a dual-source machine (Flash Definition, Siemens Healthineers, Erlangen, Germany), a 64-slice machine (Discovery CT 750 HD, GE, Milwaukee, WI), and a 320-detector machine (Aquilion One Genesis Edition, Canon Medical, Tochigi, Japan). Each image was annotated and classified considering the whole artery based on whether FFR was ≤ 0.80 or > 0.80, with cutoffs according to recent guidelines. [4, 9] The FFR measurements were performed by four different cardiology teams. Of the 500 stenotic coronary arteries in the training set, 125 (25%) had FFR values ≤ 0.80.

Image analysis

Integration of FFR, FFRCT, and FFRAI

FFRAI predicts the FFR value based on the entire artery as opposed to only the stenotic segment. The model uses the cMPR images of the entire artery for the prediction. To enable direct comparison of the FFRCT value to the FFR measurement distal to the stenosis of interest, the 3D FFRCT view was used to match the stenotic sites on the CCTA and ICA, as previously reported (Fig. 1). [8]

Fig. 1
figure 1

A 62-year-old patient presenting with stable chest pain. Curved multiplanar CCTA reconstructions of the left anterior descending artery show an intermediate-grade stenosis of the proximal segment (white arrow in A), confirmed by invasive coronary angiography (white arrow in B). Invasive FFR measurement was 0.61, FFRCT analysis was 0.66 (C) and FFRAI prediction was ≤ 0.80 (D). CCTA: Coronary computed tomography angiography; LAD: left anterior descending artery; FFR: Fractional flow reserve measured during invasive coronary angiography; FFRCT: fractional flow reserve (FFR) estimated by computational fluid dynamics software applied to CCTA images; FFRAI: FFR estimated by a deep learning model applied to the same CCTA images

Full size image

FFRAI postprocessing and analysis

All CCTA images were sent to an Aquarius Intuition 3D-workstation (V4.5, TeraRecon, Durham, NC), which automatically segmented each contrast-enhanced coronary artery. If required, the centerlines were corrected manually by the same experienced cardiac radiologist (JFP), who was blinded to the FFR and FFRCT values. As for the validation set, nine cMPR images at 40° intervals around the centerline were acquired and uploaded to CorEx. For each of the nine images, the AI model predicted whether the FFR was ≤ 0.80 or > 0.80. The mean of the nine predictions was computed and used to classify each artery as having a predicted FFR ≤ 0.80 or > 0.80.

Statistical analysis

The statistical analyses were performed using SPSS® version 29.0 (IBM, Armonk, NY). Normally distributed continuous variables were expressed as mean ± standard deviation. For the FFRCT and FFRAI values, sensitivity, specificity, positive predictive value (PPV), and negative predictive values (NPV) were computed with their 95% confidence intervals (95%CIs), using the FFR value as the reference standard. Diagnostic accuracy was computed as the sum of true positives (TP) and true negatives (TN) over the sum of TP, TN, false positives, and false negatives. Diagnostic accuracy of the CCTA and AI methods was compared by applying the chi-square test.

Proportion of agreement was computed between FFRAI and FFR, between FFRCT and FFR and between FFRAI and FFRCT. Values of p < 0.05 were taken to indicate statistically significant differences.

Results

We retrospectively included 37 patients, 25 men and 12 women, with 39 intermediate-grade coronary stenoses. Mean age was 61 ± 9 years overall, 59 ± 9 years in the men, and 67 ± 8 years in the women. Patient selection flowchart and the main patient characteristics have been reported elsewhere [8]. Of the 39 stenoses, 23 were on left anterior descending arteries, 11 on right coronary arteries, and five on left circumflex arteries. No adverse events on ICA or contrast administration occurred.

The mean FFR value was 0.85 ± 0.10 (range, 0.60–0.99). Of the 39 stenoses, 11 (28%) had FFR values ≤ 0.80 (range, 0.60–0.79).

Both FFRAI and FFRCT were determined successfully for each stenosis. Table 1 shows the contingency table for each method. The stenosis that was a false negative by FFRAI had an FFR value of 0.78. The FFR values for the two false negatives by FFRCT were 0.79 and 0.74, respectively.

Table 1 Contingency tables for FFR prediction by FFRAI and FFRCT
Full size table

Table 2 reports the sensitivity, specificity, PPV, NPV, and diagnostic accuracy for FFRAI and FFRCT, using FFR as the reference standard. Accuracy in predicting FFR ≤ 0.80 was not significantly different between the two methods (p = 0.12). Proportion of agreement was 85% (33/39) between FFRAI and FFR, 77% (30/39) between FFRCT and FFR, and 77% (30/39) between FFRAI and FFRCT.

Table 2 Diagnostic performance of FFRAI and FFRCT compared to fractional flow reserve measured during invasive coronary angiography
Full size table

Discussion

In this proof-of-concept study, the AI DLM was as accurate as computational fluid dynamics analysis in predicting whether FFR was ≤ 0.80 or > 0.80 in intermediate-grade coronary artery stenoses. [10,11,12]

The 85% diagnostic accuracy of our model was higher than the 76% value obtained using a 3D, fully automatic, multiple-neural-network model based on 131 datasets. [7] Another AI approach using a support vector machine with 2D MPR images obtained from centerlines in 137 patients (192 coronary arteries) had 66–79% accuracy depending on whether prediction targeted FFR values ≤ 0.70 to ≥ 0.90.13 However, this study was not confined to arteries with intermediate-grade stenoses. Intermediate-grade stenoses are challenging, as only one third of these lesions are ischemic and require revascularization. [4] The inclusion of a broader range of stenosis severities may have increased the diagnostic accuracy of our AI model. A strength of our study is the large training set comprising 4500 labeled images from 500 stenotic arteries in 413 patients. The previously reported models used smaller training datasets, which may have influenced the diagnostic performance. [7, 13] Our model was trained on segmented cMPR images, with expert corrections, which may facilitate the learning efficiency compared to models using directly axial images. When designing AI models to perform medical tasks, great care is required given the multiple potential sources of bias, and extensive studies must then be performed to assess general applicability. [14, 15] AI models for FFRAI prediction can be expected to improve over time as increasingly large training datasets are used.

Our study is the first to compare an AI model with FFRCT in intermediate-grade coronary stenoses. FFRCT was developed over a decade ago as a non-invasive method for estimating FFR. [16] A review of randomized controlled trials in patients with intermediate-grade stenoses showed that FFRCT had high sensitivity and NPV for predicting FFR ≤ 0.80, similar to the values shown in our study. [17] One of the reviewed trials demonstrated that FFRCT was nearly as sensitive and considerably more specific (79% vs. 34%) compared to morphological evaluation on CCTA images. [17, 18] In several previous studies with larger sample sizes than ours, the diagnostic performance characteristics of FFRCT were in line with those found in our cohort. [11, 12, 16] Proportion of agreement was high between all methods and highest between FFRAI and FFR. All three false negative cases, one for FFRAI and two for FFRCT were within or around the FFR ‘grey zone’ of 0.75–0.80. [19] For such values, the appropriateness of revascularization as opposed to medical treatment alone is difficult to assess, [20] as the optimal FFR cutoff for determining revascularization remains debated. [21, 22] The clinical benefits of revascularization are not expected to be as marked for lesions with borderline FFR values, compared to stenoses of greater severity. In predominantly single-vessel disease, FFR cutoffs of 0.72 to 0.75 have been suggested for predicting myocardial ischemia. [23]

FFRCT determination has gained a place as a non-invasive tool for decreasing the number of unnecessary ICAs, thereby diminishing risks to patients and costs to healthcare systems.

AI for FFR prediction has several advantages over morphological CCTA analysis. AI models for automatically and accurately classifying stenoses as < 50% vs. ≥50% are available [5, 24, 25] and could be used for selecting those patients who require additional functional evaluation. Our model uses cMPR reconstructions for FFR prediction enabling instant FFRAI evaluation on CCTA images into the daily clinical workflow.

The main limitations of our study are the retrospective data collection, single-center recruitment, and small sample size. However, we included only patients with intermediate-grade stenoses, that is, posing the greatest management challenges. The results of AI model requires confirmation in a multicenter prospective study with a larger sample size. Second, the model uses cMPR images with manual centerline adjustment when needed. The accuracy of centerline adjustment may vary across operators, notably with poor-quality images or heavily calcified stenoses. An AI model capable of performing automatic contouring and segmentation, would eliminate this drawback. Finally, our AI model uses a dichotomous outcome, predicting FFR as ≤ 0.80 or > 0.80, a continuous outcome would be more clinically relevant especially in borderline lesions. A global-artery evaluation by FFRAI might help to assess extensive lesions but would require a dedicated AI model.

In conclusion, FFRAI had similar diagnostic accuracy as FFRCT for predicting intermediate-grade stenoses with FFR ≤ 0.80. This method may be an accurate non-invasive imaging modality for guiding the therapeutic management of patients with intermediate-grade stenoses. Further assessment of these preliminary findings in a larger, prospective population is required.