For many years, the efficacy of percutaneous coronary intervention (PCI) in the treatment of coronary artery disease (CAD) in chronic coronary syndrome was beyond dispute. However, several studies including DEFER have recently challenged this perception by showing futility or even harm of PCI in patients with CAD without hemodynamically significant stenosis. [1, 2] Current studies like ISCHEMIA even question the benefit of PCI in ischemia inducing stable CAD, except for the purpose of symptom relief in patients with angina. [3]

With regard to the ESC guidelines of 2019, non-invasive tests are therefore preferred over invasive coronary angiography (ICA) as the initial diagnostic modality for chronic coronary syndromes. [4] In the recent decade, coronary computed tomography angiography (cCTA) has established itself as a powerful tool for the non-invasive assessment of coronary stenosis, particularly in patients with low-to-intermediate pretest probability. [5, 6] Despite its high negative predictive value, it has long been recognized that cCTA correlates comparatively poorly with the actual hemodynamic significance of stenosis, and that cCTA overestimates stenosis severity. [7,8,9] Therefore ESC guidelines recommend further functional testing after cCTA unless the stenosis is severe (> 90% diameter stenosis, except left main stem > 50%). [4]

There exists a large variety of cardiac imaging techniques for this further testing. Stress perfusion cardiovascular magnetic resonance imaging (CMR) is an established functional non-invasive method to guide coronary revascularization. It enables the detection of myocardial ischemia based on the first-pass kinetics of gadolinium contrast agent after adenosin-induced vasodilatation. The MR-INFORM trial demonstrated prospectively the non-inferiority of stress perfusion CMR compared to invasive fractional flow reserve (FFR) measurements with respect to major adverse cardiac events [10]. Despite its high diagnostic power reported in a number of studies, CMR also has limitations that prevent it from becoming the standard method of ischemia assessment. [11,12,13,14]

CT derived fractional flow reserve (CT-FFR) is a new and promising non-invasive approach that combines morphological and functional information about coronary lesions in a single diagnostic tool. CT-FFR uses a patient's standard cCTA study along with a fluid dynamics model to calculate a value that can be interpreted similar than an invasive FFR measurement. Several studies and meta-analyses have shown that CT-FFR achieves better diagnostic accuracy than cCTA alone when compared to invasive FFR. [15,16,17,18,19] A more recent CT-FFR technology is based on a machine-learning algorithm (CT-FFRML) that enables on-site use and reduces computation time. [20, 21] Furthermore, in contrast to stress perfusion CMR, CT-FFRML does neither require an additional image acquisition nor the application of a stress agent. Therefore, it is associated with lower costs, less procedural time, and fewer contraindications. [13]

The aim of this study was to compare the diagnostic performance of the novel functional non-invasive diagnostic tool CT-FFRML to standard cCTA in the detection of myocardial ischemia using stress perfusion CMR as reference standard.


Patient population and study design

Our study was designed as an observational, retrospective, single-center study (The Radiology Center, Sinsheim-Eberbach-Erbach-Walldorf-Heidelberg, Germany). The database query and patient inclusion were performed in a consecutive order. Each of our 141 included patients underwent a clinically indicated cCTA between January 2017 and September 2020, revealing at least one coronary artery stenosis with unclear hemodynamic relevance. In accordance with current ESC guidelines for chronic coronary syndromes, this finding was followed by a functional imaging test for myocardial ischemia. Therefore, our study population received a stress perfusion CMR within two months of the initial cCTA. In summary, our inclusion criteria comprise a clinically indicated cCTA showing one or more stenotic arteries with unclear hemodynamic relevance followed by a stress perfusion CMR without exceeding the interprocedural time of two months.

We retrospectively analyzed cCTA images with on-site CT-FFRML, if patients did not meet one of the following exclusion criteria: software problems with patients’ data, inadequate cCTA image quality, coronary anomalies, severe left main disease, severe stenosis at coronary ostia, coronary artery bypass grafts, aneurysms, chronic total occlusion or previous PCI in a vessel of interest (Fig. 1). Patient baseline characteristics and cardiovascular risk factors were obtained from medical records. The study was approved by the local ethics committee (S-108/2020 and 2020-882R) and was conducted in accordance with the Declaration of Helsinki.

Fig. 1
figure 1

Flow chart. Development of study population. CT-FFRML = fractional flow reserve derived from coronary computed tomography angiography based on machine learning algorithm, cCTA = coronary computed tomography angiography, CMR = Cardiovascular magnetic resonance imaging, CABG = coronary artery bypass graft

Acquisition and analysis of cCTA datasets

Acquisition of the cCTA datasets were performed using a 2 × 192-slice dual-source CT System (Somatom FORCE, Siemens Healthineers, Forchheim, Germany). First, each patient underwent a low-dose calcium scoring scan in order to obtain the Agatston score equivalent. [22] Prior to this scan, nitroglycerin and beta blocker were administered sublingually and intravenously, respectively, with a target heart rate of < 65/min. For the contrast agent enhanced cCTA scan, patients received 80 ml of iodinated contrast agent (Iomeron 400; Bracco Imaging S.p.A., Milan, Italy) in an antecubital vein using a power injector (Stellant D; Medrad, Warrendale, PA, USA) followed by administration of a saline chaser. Experienced radiologists and cardiologists subsequently analyzed the cCTA studies using a dedicated workstation (Syngo VB30A; Siemens Healthineers, Forchheim, Germany). Following Society of Cardiovascular Computed Tomography (SCCT) guidelines, the coronary tree was sectioned into 18 segments and CAD-RADS™ was determined. [23, 24] Additionally, two experienced radiologists and one cardiologist blinded to the clinical representation of the patients graded each segment with respect to the structure of the plaque (calcified, soft, mixed, high-risk) and severity of the putative stenosis (cannot be assessed, minimal 1–24%, mild 25–49%, moderate 50–69%, severe 70–94%, subtotal 95–99%) in sense of consensus interpretation. CAD-RADS™ ≥ 3 (≥ moderate 50–69%) as well as a segment severity grade of ≥ 50% define a positive cCTA result in this study. In case of any conflict consensus reading was performed.

Acquisition and analysis of stress perfusion CMR dataset

Electrocardiogram-gated cine images were acquired using a segmented steady-state free precession (true-FISP) sequence. 4-, 2-, and 3-chamber views as well as sequential short-axis views (apical, midventricular, and basal) were acquired during end-expiratory breath-holds for anatomical and functional measurements on a 1.5 T magnetic resonance imaging (MRI) scanner (Aera, Siemens Healtineers, Forchheim, Germany). Electrocardiographic rhythm and blood pressure were monitored continuously. For perfusion imaging, intravenous adenosin was infused at 140 µg/kg/min for at least 3 min, with up-titration to 210 µg/kg/min, if there was failure to achieve the pre-specified physiological target i.e. patients’ hemodynamic responses were considered adequate if they demonstrated an increase in heart rate ≥ 10 bpm. Following this, a bolus of the 0.05 mmol/kg body weight gadobutrol (Gadovist, BayerVital, Germany) was administered intravenously. 3-short-axis slices of the left ventricle were acquired per cardiac cycle during free-breathing. Late gadolinium enhancement (LGE) imaging was performed ten minutes after injection of gadolinium contrast in continuous short-axis, 4-chamber 3-chamber and 2-chamber long-axis views using a phase sensitive inversion recovery (PSIR) 2D sequence. All images were analyzed by cardiologists and radiologists with consensus reading using the multi-modality 3D-enabled workstation (Syngo VB30A; Siemens Healthineers, Forchheim, Germany).

Analysis of machine-learning computed tomographic-based fractional flow reserve

For calculation of the fractional flow reserve derived from the cCTA dataset of our patients, we used an on-site CT-FFRML prototype installed on a conventional workstation (Syngo VB30A; Siemens Healthineers Forchheim, Germany). This prototype is based on a machine-learning algorithm (Siemens cFFR, version 3.2; Siemens Healthineers, Forchheim, Germany), which is not yet commercially available. First, the software proposes manually adjustable centerlines that run through the lumen of each detected coronary artery. Then, it calculates the boundaries of the vessels and excludes coronary plaques. After acceptance by the user a 3-dimensional color-coded mesh of the coronary tree and aortic root is generated (Figs. 2, 3). An estimated CT-FFR value is provided for every spot on the mesh. Similar to interpreting invasive FFR results, a coronary stenosis with a CT-FFR value less than or equal to the cut-off value 0.80 is considered to be hemodynamically significant. [25] CT-FFRML measurements were performed directly distal of a stenosis. No CT-FFRML value was determined for vessels without at least one stenosis with unclear hemodynamic significance in cCTA.

Fig. 2
figure 2

80-year-old male patient with pre-test probability of CAD of 37% (CAD consortium (%)- basic model). a cCTA demonstrates a high-risk plaque in the main stem of the left coronary artery and in the proximal left anterior descending coronary artery (arrow). b CT-FFRML software provides a color-coded 3-dimensional mesh of the coronary tree and calculated a value of 0.98 in an area distal of the stenosis (arrow). c Stress perfusion CMR in three short axis slices shows no perfusion deficit and thus no myocardial ischemia. CAD = coronary artery disease; cCTA = coronary CT angiography; CT-FFRML = Fractional flow reserve derived from coronary computed tomography angiography based on machine learning algorithm; CMR = cardiovascular magnetic resonance imaging

Fig. 3
figure 3

53-year-old male patient with suspected CAD and arterial hypertension. a cCTA illustrates a moderately graded stenosis (50–69%) with unclear hemodynamic relevance caused by mixed structured plaques in the mid left anterior descending coronary artery (arrow). b Color-coded 3-dimensional mesh created by CT-FFRML software shows a flow-limiting stenosis with a measured value of 0.77 (arrow). c a midventricular short axis stress perfusion CMR image demonstrates a significant perfusion deficit in the left anterior descending coronary artery territory, which correlates with the myocardium subtended by the hemodynamically significant lesion identified on CT-FFRML; CAD = coronary artery disease; cCTA = coronary CT angiography; CT-FFRML = Fractional flow reserve derived from coronary computed tomography angiography based on machine learning algorithm; CMR = cardiovascular magnetic resonance imaging

Statistical analysis

Categorial data is given as counts and proportions and continuous data as mean ± standard deviation. Diagnostic test evaluation was performed using Receiver Operating Characteristic Curve analyses, including Area Under the Curve calculations. Logistic regression models were applied to assess the diagnostic power of the CT-derived parameters. A p-value < 0.05 was regarded as statistically significant. The sample size calculation was performed for the AUC analysis of the CT-FFRML. We hypothesized an AUC of 0.75 for CT-FFRML and defined the null hypothesis value as 0.5. As CMR perfusion was the reference standard, the ratio of negative to positive cases was 124/17. With an alpha value of 0.05 and a beta value of 0.2, resulting in a power of 0.8, the needed sample size was 100 cases (12 positive). An interobserver variability was calculated using Cohen’s Kappa statistic. Statistical analyses were conducted using dedicated statistical software (Medcalc 19.6, MedCalc Software Ltd, Ostend, Belgium).


Between January 2017 and September 2020, 4507 patients received a clinically indicated cCTA examination. Stress perfusion CMR was performed in 164 patients showing a CAD with at least one stenosis of unclear hemodynamic significance.17 cases were excluded, because the interprocedural time of 2 months between cCTA and CMR was exceeded. Furthermore, coronary anomalies, CABG (coronary artery bypass graft), inadequate image quality or software problems prevented successful CT-FFRML analysis in a total of 6 cases. Our final study population included 141 patients. After the exclusion of 3 vessels due to insufficient image quality and 13 vessels due to stents, cCTA and stress perfusion CMR images of 407 vessels (96%) were included in the final analysis. CT-FFRML was calculated in 269 vessels (64%) having stenosis with unclear hemodynamic relevance as prespecified in the study design (Fig. 1).

Baseline characteristics of the included patients are presented in Table 1. In our study population, we calculated the mean pretest probability of having CAD to be 20% ± 12% in the CAD consortium-basic model and the mean number of cardiovascular risk factors to be 2.0 ± 1.1. [26]

Table 1 Baseline characteristics

Among all 141 patients, 129 (91%) patients had at least one diameter stenosis ≥ 50% in cCTA (positive cCTA result). Only 12% (16 of 129) of the cCTA positive cases showed a positive stress perfusion CMR demonstrating the hemodynamic relevance of the assessed stenosis. Hence, the functional CMR examination could prevent invasive coronary angiography in 88% of these patients (113 of 129). In 79% (102 of 129) of all cCTA positive patients, CT-FFRML alone could have revealed lesions without ischemia (true negative results) and thus, would have rendered further downstream testing as stress perfusion CMR unnecessary. This effect becomes more pronounced if only intermediately-graded stenosis (50–69%) in cCTA were considered (101 patients); this yields a percentage of 6% (6 of 101) positive stress perfusion CMR results, and 12% (12 of 101) positive CT-FFRML results. cCTA, CT-FFRML and stress perfusion CMR findings are presented in Table 2. We could calculate an interobserver kappa coefficient of 0.94 for our cCTA analyses which demonstrates a high interobserver agreement.

Table 2 Findings of cCTA, CT-FFRML and stress perfusion CMR

Our univariate logistic regression analysis shows a highly significant relationship between CT-FFRML and stress perfusion CMR, with Chi2 = 47 (p < 0.0001). However, the relationship between cCTA in patients with stenosis ≥ 70% and stress perfusion CMR is significant as well, with Chi2 = 15 (p < 0.0001). In contrast, cCTA with ≥ 50% stenosis could not reach the level of significance upon Chi2-testing, and thus could not predict hemodynamic significance of stenosis assessed by stress perfusion CMR.

Using stress perfusion CMR as the reference standard, we calculated for CT-FFRML a per patient (n = 141) sensitivity of 88%, specificity of 90%, positive predictive value of 56%, negative predictive value of 98% and accuracy of 90%. The accuracy of cCTA alone to predict hemodynamically significant stenosis was 19% for cCTA (≥ 50%) and 82% for cCTA (≥ 70%) when compared to stress perfusion CMR. In addition to notably higher accuracies, the positive predictive values from cCTA to CT-FFRML were markedly improved as well (Table 3). As illustrated in Fig. 4, the diagnostic accuracy measured by the AUC of CT-FFRML (0.89) surpassed those of the Agatston score (0.70) as well as the degree of cCTA stenosis ≥ 70% (0.74) in detection of ischemia-inducing lesions with respect to the reference standard of stress perfusion CMR. Namely, there was a significant (p < 0.05) increase from 0.74 to 0.89 between cCTA ≥ 70% and CT-FFRML.

Table 3 Diagnostic performance of CT-FFRML, cCTA (≥ 50% stenosis) and cCTA (≥ 70% stenosis) using stress perfusion cardiovascular magnetic resonance imaging as reference standard
Fig. 4
figure 4

ROC of Agatston score, cCTA (stenosis ≥ 70%) and CT-FFRML with stress perfusion CMR as reference standard. The AUC for detection of ischemia inducing stenosis by CT-FFRML was 0.89. Agatston score and cCTA (stenosis ≥ 70%) provide AUC values of 0.70 and 0.74 (n = 138). CAD = coronary artery disease; cCTA = coronary CT angiography; CT-FFRML = Fractional flow reserve derived from coronary computed tomography angiography based on machine learning algorithm; CMR = cardiovascular magnetic resonance imaging


In this study, we conducted one of the first investigations into the diagnostic power of the recently introduced CT-FFRML software for the assessment of the hemodynamic relevance of coronary stenosis in comparison to the established diagnostic modality of stress perfusion CMR. We also examined, if there is an additional value that CT-FFRML provides over the standard cCTA diagnostic approach. Taken together, our data herein suggest high diagnostic power and notable potential in the clinical utility of CT-FFRML in the management of chronic CAD.

Although the Food and Drug Administration granted approval of the CT-FFRML in 2015, the software is currently only available for research purposes. To date, the only commercially available alternative is the fluid dynamics-based CT-FFR algorithm from HeartFlow® (HeartFlow Inc., Redwood, CA, USA) which is an off-site software. An increasing interest in CT-FFR technology has spurred more investigations comparing CT-FFR to invasive FFR as DISCOVER-FLOW, DeFACTO and NXT. These studies yielded values for sensitivity, specificity, accuracy and AUC of 84–89%, 61–86%, 73–84% and 0.79–0.93, respectively. [16,17,18] Not only CT-FFR but also other functional diagnostic tools for ischemia detection were compared to the reference standard invasive FFR. These included single-photon emission computed tomography (SPECT), stress echocardiography and stress perfusion CMR with reported sensitivities of 0.70 (0.59–0.80), 0.77 (0.61–0.88), 0.90 (0.75–0.97) and specificities of 0.78 (0.68–0.87), 0.75 (0.63–0.85), 0.94 (0.79–0.99). [11] These diagnostic values, as well as the MR-INFORM noninferiority clinical-effectiveness trial, implicate an outstanding diagnostic performance of stress perfusion CMR when compared to other functional diagnostic modalities. [10]

In our study, we performed a comparison between CT-FFRML and stress perfusion CMR, which served as reference standard. We calculated sensitivity, specificity, accuracy and AUC values for CT-FFRML of 88%, 90%, 90% and 0.89. Our results thus corroborate findings of previous studies. Although CT-FFRML has demonstrated to be a reliable diagnostic tool in detection of ischemia inducing coronary stenosis, there are still some limitations. First, there was a notable number of patients (7–12%) in previous studies in which cCTA data were not evaluable by CT-FFR. [27] Of note, the proportion of excluded patients (4%) in this study was below average. This could be attributed to the improved image quality attained by the use of a state-of-the-art dual-source CT scanner of the third generation. [28]

Although CT technology has undergone considerable improvements, patient radiation exposure is unavoidable. Furthermore, there are still more false negative cases in CT-FFRML than in stress perfusion CMR, when comparing the sensitivities of previous studies referred to invasive FFR. [11, 12, 20] In our study population of 141 patients, there were 2 cases with a negative CT-FFRML but positive stress perfusion CMR. In clinical practice, these false negative results could have had led to wrong diagnosis and subsequent therapy for the respective patients. Finally, standard cCTA as well as CT-FFRML unlike stress perfusion CMR are not able to detect microvascular disease. However, computed tomography-based diagnostic techniques have many advantages over stress perfusion CMR. Stress agents like adenosin are not required for imaging, which implies that patients are not exposed to potential adverse effects. Additionally, stress perfusion CMR is very time-consuming, whereas cCTA is a fast imaging method. Renker et al. gave a calculation time for on-site CT-FFR of 37.5 ± 13.8 min. [29] But for CT-FFRML an average duration of only 11 ± 2 min per analysis was demonstrated. [30] In addition, stress perfusion CMR is an expensive imaging tool with limited availability compared to cCTA. [13] Finally, there is substantial number of contraindications for CMR including older non MRI safe implanted pacemakers and other electronical implants or magnetizable objects that do rarely affect cCTA or CT-FFRML.

In summary, cCTA alone or in combination with CT-FFRML exhibits better clinical practicability compared to stress perfusion CMR for the detection of significant coronary stenosis. Besides its feasibility, the 5-year follow-up of SCOT-HEART illustrated that cCTA is able to reduce mortality from CAD and nonfatal myocardial infarction, compared to a standard of care group. These benefits were attributed to a refinement in healthy lifestyle and preventive therapies, but not by altering PCI strategy. [31] The two landmark trials SCOT-HEART and PROMISE suggest a notable benefit of cCTA as initial diagnostic approach for patients with stable CAD [31, 32]. However, they also point to an improvement in patient selection for ICA or PCI when adding a functional element to an anatomical diagnostic strategy [33]

We can confirm this conclusion through our study results. As we demonstrated, in 88% of patients showing obstructive CAD (≥ 50% stenosis), stress perfusion CMR showed no significant ischemia in these subjects. If there had not been subsequent stress perfusion CMR after the initial cCTA, this may have resulted in unnecessary ICA or even PCI in all of these cases. Not only stress perfusion CMR, but also CT-FFRML proved to be able to reduce unnecessary invasive diagnostic examinations. This discriminatory benefit through CT-FFRML has also been reported in previous studies. [19, 34] In addition, the accuracy of 19% for cCTA (≥ 50% stenosis) and of 82% for cCTA (≥ 70% stenosis) reflect the limited capability of cCTA to assess the hemodynamic relevance of coronary stenosis. Due to a small number (n = 12) of cCTA negative patients in our study population, accuracy for cCTA (≥ 50% stenosis) lies beyond the average of previous studies. Nevertheless, DISCOVER-FLOW, DeFACTO and NXT, also provided low accuracy for cCTA (≥ 50% stenosis) of 53–64% in the prediction of hemodynamic relevance. CT-FFRML leads to a remarkable improvement in accuracy (up to 90%) in our study.

The calculated AUC depicts a moderate diagnostic value in detection of ischemia inducing coronary stenosis of Agatston score (0.70). This result is consistent with AUC of Agatston score referred to invasive FFR. [34] Superiority of the recent non-invasive diagnostic tool CT-FFRML to standard cCTA method related to the established stress perfusion CMR becomes particularly clear when considering the significant difference in already high AUC values (0.89 against 0.74).

All in all, our study is a further contribution to the long development of the non invasive diagnostic possibilities of chronic coronary heart disease. The combination of non invasive Agatston scoring, cCTA, CT-FFRML might prevent unnecessary invasive coronary angiographies in accordance to the current ESC guidelines when compared to the golden standard stress perfusion CMR. Recent studies demonstrated a significant correlation of CT-FFRML and morphological CT derived plaque parameters when compared with the reference standard RFR to detected hemodynamically significant coronary stenosis [35]. Furthermore CT-FFRML provided a good correlation with invasive iFR® measurements without vasodilator induced hyperemia [30]. Supplementary CT-FFRML might provide valuable guidance in necessary coronary interventions by a dedicated detection of ischemic myocardium especially in case of multiple equivalent coronary stenosis. The aim is to of find the most uncomplicated, precise and safe method for diagnosing hemodynamically relevant coronary stenoses in each and every individual patient.


When evaluating this study, some limitations should be considered. In first place, representativeness of our results for the general population is limited by our retrospective single-center study design and the relatively low number of included patients. Thus, small differences for instance the difference between the CTA and the Agatston score, which did not reach statistical significance in our population might, have in larger one. In consequence additional studies are needed to further investigate our findings.

Second, we did not investigate reproducibility of CT-FFRML analyses for this study. However, all MRI and CT-FFRML-measurements were performed as consensus readings of at least one experienced cardiologist and one experienced radiologist. Furthermore, previous studies from our own group and others yield high interobserver correlation, also among operators of variable expertise [30, 35, 36].

Third, due to ESC guidelines and ethical aspects patients without stenosis of unclear hemodynamical significance did not undergo stress perfusion CMR. This led to a lack of stress perfusion CMR images of patients providing stenosis graded < 50% in cCTA. Accordingly, the informative value of our calculated accuracy for cCTA ≥ 50% stenosis is limited. Furthermore, the comparison to actual gold standard invasive FFR would have been very interesting.


Our study is one of the first to show the high diagnostic accuracy of CT-FFRML compared to stress perfusion CMR. Furthermore, the diagnostic values herein demonstrate that CT-FFRML is superior to cCTA alone. Moreover, as our data shows, CT-FFRML is able to reduce unnecessary ICAs when it is added to the standard CT diagnostic algorithm.