Introduction

Fractional flow reserve (FFR) is regarded as the gold standard for the assessment of intermediate coronary artery stenoses (diameter stenosis (DS) 30–90%). An FFR ≤ 0.80 is regarded to represent functionally significant stenosis where revascularization is recommended. However, invasive pressure wire assessment requires a wire to be advanced through the lesion to the distal vessel. This comes with procedural risks along with increased time and cost. Adenosine is the most commonly used hyperaemic agent for invasive FFR assessment which itself is associated with the small risk of dyspnea, arrhythmia and cost. Therefore, although most guidelines have recommended FFR evaluation of intermediate coronary stenosis lesions to guide revascularization, only 15% of these lesions are evaluated by FFR in daily clinical practice [1,2,3,4].

Over the last few years, several angiography-derived virtual FFR assessment technologies have been developed. These are based on vessel geometry derived from 3 dimensional-quantitative coronary angiographies (3D-QCA), processed either by computational fluid dynamics (CFD) methods or using a mathematical formula which calculates the pressure drop across the lesion [5]. At present, three of these technologies are commercially available (Quantitative Flow Ratio (QFR), Medis medical imaging system bv., Netherland; FFRangio, Cathworks Ltd, Israel and vessel fractional flow reserve (vFFR), Pie Medical Imaging, Netherland) which have been validated against invasive FFR. All three technologies have shown a strong correlation with FFR measurements and good accuracy in predicting functionally significant coronary stenosis [6,7,8,9,10,11,12].

As angiography-derived computational FFR is based on 3D-QCA reconstruction of the interrogated vessel, good quality angiographic projections are required for accurate assessment of functional coronary stenosis. Over the last decade, several radiation reduction techniques have been implemented to reduce radiation exposure in the cardiac catheterization suite. This can either be done via reducing the X-ray frame rate, pulse or a combination of both [13, 14]. However, radiation reduction comes at a cost on angiographic image quality with a decreased signal to noise ratio. [15]. The acquired angiographic images may provide sufficient diagnostic information to guide percutaneous coronary intervention (PCI) but whether angiographic derived FFR based on these images allow accurate estimation of functional stenosis is unknown. This is important for the retrospective analysis of angiograms, which have been obtained in clinical practice with contemporary radiation reduction parameters.

In this study, we focused on the feasibility and diagnostic performance of two commercially available technologies, QFR and vFFR against invasive FFR when the coronary angiography has been performed using radiation-save mode (7.5 frames per second (fps) and 7 pulses per second (pps)), and try to develop a reasonable algorithm to select angiograms with adequate image quality under such mode for further angiography-derived computational FFR analysis.

Methods

Study design and patient population

The present study is a single centre, observational, retrospective study in which the diagnostic performance of both QFR and vFFR were evaluated and compared with invasive FFR. Patients (age > 18 years old) with at least one native coronary artery with intermediate stenosis (diameter stenosis (DS) 30–90%) that was assessed with invasive FFR between March and December 2018 at the Barts Heart Centre, London, United Kingdom, were enrolled. Patient/vessel exclusion criteria included left ventricular ejection fraction (LVEF) < 40% and invasive FFR performed in the culprit vessel of a patient presenting with acute coronary syndrome (ACS). All angiography cine images were acquired at 7.5fps and 7pps radiation-save mode. Angiography exclusion criteria comprised (1) aorto-ostial lesions (> 30% stenosis within ≤ 3 mm from the aorta); (2) previous coronary artery bypass grafts (3) coronary aneurysm of the assessed vessel; (4) bifurcation lesions with severe stenosis at the ostium of branch vessel (Medina 1,0,1 or 1,1,1); (5) inaccurate FFR measurement such as significant drift (> 0.02) and evidence of insufficient maximal hyperemia; (6) lack of 2 angiographic projections which are ≥ 30° for satisfactory QFR and vFFR analysis.

Evaluation of angiography images for analysis

All angiographic sequences included in the study were evaluated according to two different methods (described below) and excluded if their quality was deemed inadequate. For each vessel, two projections which are at least ≥ 30° apart were selected for QFR and vFFR analysis. Following that, two methods were used to examine the quality of angiography acquisitions. The first is the conventional manual (M group) method which relies on the operator’s expertise in analyzing the suitability of the images. Two experts (CJ and AR) independently assessed all the angiography images and excluded vessels with severe overlapping, tortuosity, foreshortening and poor vessel opacification. In the event of disagreement, a third expert’s (CB) opinion was sought for consensus. The same set of images was also analyzed using a new algorithm (A group) that included the following variables: vessel overlapping, foreshortening and image quality (Fig. 1). For each interrogated angiographic projection cine, these variables were classified into different severity grades with different weights. The weights of each interrogated angiographic cines were added, and the sums of the paired sets of projections were added to a final score (Table 1). The algorithm was first tested on a validation group (n = 51) to select a cut-off score beyond which the correlation of QFR/vFFR with invasive FFR could be improved. The algorithm was then applied to all the subjects with the selected cut-off, to exclude vessels with poor angiographic image quality.

Fig. 1
figure 1

Typical sample images of different grades of each variable enrolled in the algorithmic excluding method (interrogated vessels are labeled at the right-up corner, image quality not acceptable is not shown due to lack of sample). Specific definition of each grade could be found in the supplements

Full size image
Table 1 Weights of each factor enrolled for algorithmic image quality scoring
Full size table

Coronary angiography and invasive pressure wire assessment

Coronary angiography was performed according to standard local practice. FFR measurement was taken during maximum hyperemia where the lowest value and drift during wire pullback were recorded. The aortic root pressure at the beginning of coronary angiography was recorded and used for vFFR analysis.

QFR and vFFR analysis

QFR analysis was performed offline by using QAngio XA3D/QFR solution (Medis medical imaging system bv., Leiden, the Netherlands), while vFFR was calculated by CAAS workstation 8.0 (Pie Medical Imaging, Maastricht, the Netherlands) according to their well-establish protocols. To minimize the differences between the two systems, the same projection and end-diastolic frame (agreed by two experts) was selected and used for analysis in both. The distal reference point for vessel reconstruction during QFR/vFFR analysis was matched to the pressure wire sensor on coronary angiography while the proximal point was set at the visually normal proximal reference segment. For QFR, both fixed QFR (fQFR) and contrast QFR (cQFR) assessment were calculated. All the measurements were performed by 2 independent experts blinded to FFR results. vFFR analysis was performed at least 4 weeks following QFR analysis to minimize bias.

Reproducibility

The inter- and intra- observer reliability were tested to ensure reproducibility. 51 consecutive vessels were analyzed by 2 independent experts (CJ and AR), blinded to each other to test the inter-observer reliability. For the intra-observer reliability test, expert 1 (CJ) re-analyzed the same 51 vessels at 4 weeks after the first analysis. The intra- and inter-observer variability results are not statistically different as shown in Fig. 2.

Fig. 2
figure 2

Bland–Altman plots of intra and inter observer reliability. For intra observer reliability, the mean difference between the 2 groups of measurements by observer 1 was 0.016 ± 0.060 (p = 0.066) for fQFR, 0.009 ± 0.053 (p = 0.230) for cQFR, and 0.008 ± 0.040 (p = 0.175) for vFFR, respectively. Inter-observer difference was also non-significant, with 0.001 ± 0.036 (p = 0.847) for fQFR, − 0.001 ± 0.049 (p = 0.910) for cQFR, and − 0.005 ± 0.037 (p = 0.393) for vFFR

Full size image

Statistical analysis

Continuous data, if normally distributed were expressed as mean ± SD, otherwise as median (interquartile range (IQR)). Categorical data were expressed as percentages. Data were analyzed on a per vessel bias for QFR or vFFR analysis. For mean comparison, a paired t-test was used. Bland–Altman plot analysis was used for evaluating the agreement between FFR and QFR/vFFR measurements, as well as for the intra- and inter-observer reliability. Pearson correlation was used to estimate the correlation between FFR and QFR/vFFR. Receiver operating characteristic (ROC) curve analysis was used to evaluate the diagnostic efficiency of QFR/vFFR with FFR ≤ 0.80 used as the gold standard to indicate functional significance. Sensitivity, specificity, accuracy, positive predicting value (PPV), negative predicting value (NPV), positive likelihood ratio ((+) LR) and negative likelihood ratio ((−) LR) were derived from ROC curve analysis. Paired ROC comparison was performed according to the DeLong method within the same group. For different exclusion method groups, a z-test was used for correlation and ROC curve comparison. All the analysis were performed using MedCalc Statistical Software version 18.2.1 (MedCalc Software bvba, Ostend, Belgium) and R software version 3.5.1 (R Foundation for Statistical Computing, Vienna, Austria). A p-value of < 0.05 was assumed to be statistically significant.

Results

Baseline clinical and lesion characteristics

A total of 134 vessels from 102 patients were screened, from which 22 vessels from 16 patients were excluded due to inaccurate FFR measurement or unsuitable coronary anatomy. Finally, 112 vessels from 86 patients were enrolled. The study baseline characteristics are summarized in Table 2. The mean age of the study population was 61.2 ± 14.3 years old, 31.4% were female, and 48.8% were diabetic.

Table 2 Baseline characteristics of study population (N = 86)
Full size table

Angiography image quality evaluation

In M group, 3 vessels were excluded due to severe overlapping (n = 1), poor filling of contrast (n = 1) and severe foreshortening (n = 1). In the validation cohort (n = 51) of A group, an increasing correlation between QFR/vFFR and FFR assessment was observed when excluding vessels using an algorithmic cut-off score of > 8, > 7, > 6, and > 5 points, respectively (Table 3). The algorithm was then applied to all the study subjects, the mean score of all the studied vessels was 4.32 ± 2.79, ranging from 0 to 9 points. The cut-off > 6 was chosen as a consideration of the best correlation with FFR for both QFR and vFFR in the validation cohort and maximum sample size saving. The A group finally enrolled 82 vessels from 82 patients. All the 3 vessels excluded in the M group were also excluded by the algorithmic method (Fig. 3).

Table 3 Correlation between QFR/vFFR and FFR by excluding cases with different cut-off score in validation cohort
Full size table
Fig. 3
figure 3

Study flowchart

Full size image

Agreement and correlation between QFR/vFFR and FFR

There was no significant difference between the mean value of QFR and FFR measurements in both M and A group (mean difference with FFR: 0.004 ± 0.078, p = 0.5730 for fQFR, 0.002 ± 0.091, p = 0.8333 for cQFR in the M group, and 0.001 ± 0.061, p = 0.8703 for fQFR, − 0.006 ± 0.062, p = 0.4132 for cQFR in the A group). The mean value of vFFR measurements in the M group was significantly higher than FFR (mean difference: − 0.018 ± 0.084, p = 0.0232). However, in the A group, no statistically significant difference was observed between the two measurements (mean difference: − 0.010 ± 0.073, p = 0.1999). A good correlation was observed between QFR and FFR in both M and A group with the coefficient of 0.6242 (p < 0.0001) for fQFR, 0.5888 (p < 0.0001) for cQFR in the M group, and 0.7055 (p < 0.0001) for fQFR, 0.6793 (p < 0.0001) for cQFR in the A group. The correlation coefficient between vFFR and FFR was moderate in the M group (r = 0.4089, p < 0.0001), which was similar to cQFR (p = 0.0787) but significantly lower than fQFR (p = 0.0303), however, improved in the A group (r = 0.5664, p < 0.0001). Although the correlation coefficients between QFR/vFFR and FFR were higher in the A group than M group, there was no significant difference between the 2 exclusion methods (z = − 0.9844, − 1.0226, − 1.3989, p = 0.3249, 0.3065, 0.1619, for fQFR, cQFR and vFFR, respectively) (Fig. 4a, b).

Fig. 4
figure 4

Agreement and correlation between QFR/vFFR and FFR, and ROC curves of identifying FFR ≤ 0.80. For both a and b panel, upper part: Bland–Altman plots of differences against the means; lower part: correlation between QFR/vFFR and FFR. For c and d panel, ROC curves of QFR/vFFR and their comparison within same group

Full size image

Diagnostic performance of QFR/vFFR on functional significant stenosis

FFR ≤ 0.80 was used as the gold standard to define functional significance. In the M group, both fQFR and cQFR showed good accuracy at detecting functional stenosis, while vFFR revealed moderate accuracy (AUCfQFR: 0.852 (95% CI 0.722–0.913), AUCcQFR: 0.858 (95% CI 0.778–0.917), AUCvFFR: 0.682 (95% CI 0.586–0.768)). AUCvFFR was significantly lower than both cQFR and fQFR (AUCfQFR vs AUCvFFR: z = 2.732, p = 0.0063; AUCcQFR vs AUCvFFR: z = 2.782, p = 0.0054). In the A group, QFR showed high diagnostical performance (AUCfQFR: 0.898 (95% CI 0.811–0.945), AUCcQFR: 0.892 (95% CI 0.803–0.949)), while vFFR also showed good performance (AUCvFFR: 0.843 (95% CI 0.746–0.914)). No significant difference was observed within all the three assessments in this group according to the ROC curve comparisons (AUCfQFR vs AUCcQFR: z = 0.322, p = 0.7478; AUCfQFR vs AUCvFFR: z = 0.873, p = 0.3825; AUCcQFR vs AUCvFFR: z = 0.741, p = 0.4585). vFFR had a better diagnostic performance in the A group than that in the M group (z = − 2.055, p = 0.0399), while QFR remained similar (fQFR: z = -0.794, p = 0.4271; cQFR: z = − 0.540, p = 0.5892) (Fig. 4c, d). Table 4 list the diagnostical performance of 3 different evaluation methods and the p values between the 2 different groups.

Table 4 Diagnostical performance of QFR/vFFR in different exclusion method groups
Full size table

Discussion

Our study confirmed the feasibility of two commercially available angiography derived FFR solutions (QFR and vFFR) for assessing the functional significance of intermediate coronary stenosis using angiographic images performed under radiation save mode (7.5 fps and 7pps). Under such conditions, QFR showed good agreement and correlation with the invasive pressure wire derived FFR by both conventional manual exclusion and our newly developed algorithm of excluding poor quality images. The results from our study are in line and confirm the good diagnostical performance of previous QFR studies with high sensitivity, specificity and accuracy when compared with invasive FFR [6, 11, 12]. The manual exclusion method which relies on the expert’s visual interpretation of suitability of angiographic images revealed that vFFR showed a moderate correlation with FFR and diagnostical performance of detecting functional stenosis. However, this significantly improved when the new algorithm was used, with which the diagnostic accuracy, agreement and correlation of vFFR was similar to QFR when compared to invasive FFR.

Radiation exposure during coronary angiography and PCI is a major concern for patients and interventional cardiologists. The main principle of radiation exposure should be “as low as reasonably achievable”. This is to reduce the effects of radiation on superficial skin reactions and cancer risks to patients and catheterization suite staffs [16]. Modern X-ray scanners deliver X-ray energy in pulses, therefore reducing the pulse rate from the conventional 15 pps to a lower value is a good way of reducing radiation exposure [13]. Another method of reducing radiation exposure is to reduce the conventional 15fps frame rate that is routinely used during fluoroscopic and cine-acquisition [14]. Although reduction of pulse rates from 15 to 10pps has little impact on image quality [13], extremely lower pps (< 10pps) could result in a decrease in perceived resolution and contrast [16]. Also, the reduction of frame rates from 15fps to 7.5fps has been identified to reduce the image quality [14]. A combination of a reduction in both fps and pps is commonly used for reducing radiation, however, with a cost of reducing angiographic image quality, at both contrast and resolution level [17].

All the angiography derived FFR assessments require 3D-QCA reconstruction derived from at least 2 angiographic projections. The current recommendation for both QFR and vFFR is to use angiographic images acquired with 15 fps, without specific clarification on pulse rate setting. This is to ensure good accuracy of vessel contour detection, which provides the required geometry of a vessel for 3-dimensional reconstruction. Therefore, this would limit the application to prospectively acquired angiograms and restrict the use in a retrospective analysis of angiograms performed with contemporary radiation protection parameters. To our knowledge, this is the first study that has examined and confirmed the efficacy of angiography derived FFR software using angiography images acquired under 7.5fps and 7pps radiation save mode.

So far, all the validation studies of angiography derived FFR against invasive FFR have focused on the manual exclusion of poor-quality angiograms prior to analysis. The exclusion rates varied, from 0.3 to 8% for QFR [6, 11, 12, 18] and up to 13% of the screened cases for vFFR [7], depending on the study design and the requirements of angiography image acquisition. Often, the excluded cases are labelled as ‘poor image quality’ rather than a specific exclusion reasoning or definition, not only in validation studies of QFR/vFFR, but also other angiography derived FFR validation trials [10, 19]. In our current study, the exclusion rate in the M group should theoretically be higher than those studies using normal frame and pulse rate settings because of the relatively lower image quality, but in fact, was actually similar (2.7%). This may because that the observers have been used to these low dose images’ characteristics and therefore subjectively think that they’re suitable for analysis, which indicates the manual exclusion method may not be very reliable. Thus, a more objective and reliable exclusion method needs to be considered.

Our new algorithmic exclusion method, for the first time, provides a new quantitative method of screening angiographic images that are suitable for angiography derived FFR measurement. By using a cut-off score of 6 points, this algorithm helps to improve the performance of vFFR, both in agreement with FFR and in detecting FFR ≤ 0.80. The algorithm also had slight improvement when applied to the QFR analysis but without statistical significance. Although the exclusion rate is relatively high, our algorithm provided a well-defined methodology to select angiograms with adequate quality for angiography derived FFR assessment, especially when the angiograms have been acquired under radiation save mode and when vFFR was used for analysis. Cases excluded by the algorithm should be considered to use invasive FFR to detect functional stenosis for accuracy, and avoiding assessment with vFFR.

Currently, no head-to-head comparisons have been performed between angiography derived FFR software packages. Although the correlation coefficient and AUC value in the FAST study of vFFR was higher than those in FAVOR series studies of QFR, a recent meta-analysis showed that the computational approaches and mathematical formula based techniques did not influence the diagnostic accuracy of angiography derived FFR [5]. Thus, the performance of vFFR could theoretically be similar to QFR. The application of our algorithm to select angiograms with adequate quality is useful where vFFR showed similar performance at detecting functional ischemia with QFR. However, vFFR was inferior to QFR when the angiographic images were manually selected.

One of the reasons of the poorer performance of vFFR in manual exclusion group may be due to its vessel contour detecting algorithm. Figure 5 shows a vessel analyzed in both software packages. Vessel contours were semi-automatically detected with minimal modifications. The QFR contour detection algorithm appears to be much smoother than vFFR. The automated detection can be modified manually in vFFR but this can lead to errors and higher observer variability. Table 5 lists the 3D-QCA results provided by both QFR and vFFR software package using their default lesion detections on the interrogated vessels. The minimal lumen diameter, minimal lumen area and reference diameters generated by QFR software package in the M group were all significantly lower than those by vFFR software package, while the diameter stenosis percentage was significantly higher. In the A group, the minimal lumen diameter, reference diameters generated by QFR were still lower than those by vFFR, the diameter stenosis percentage remained higher, however, there was no significant difference at minimal lumen area generated by the two different packages. This may indicate the reason of the statistically higher vFFR assessment in the M group while QFR revealed similar with FFR, as vFFR provided bigger mean minimal lumen area, while in the A group, similar minimal lumen area was provided by both QFR and vFFR packages.

Fig. 5
figure 5

Vessel contour detection with necessary manual modifications. Same vessel (an intermediate ramus) was assessed by both QFR and vFFR software package. A Initial frame without any projections. A1 Semi-automatic detection of vessel contours with necessary manual modifications by QFR software package. A2 3D reconstruction of the vessel by QFR software package. B1 semi-automatic detection of vessel contours with necessary manual modifications by vFFR software package, the contours are not so smooth and jagged. B2 Same vessel contours after being modified with “Hard correction” tool provided by the software package, the vessel contours become smoother. B3 3D reconstruction of the vessel by vFFR software package, the reconstructed vessel wall is not as smooth as that in QFR software

Full size image
Table 5 The 3D reconstruction projections of both QFR and vFFR in M and A group
Full size table

Apart from the potential poorer image quality, angiographic images acquired with low frame rate influences the frame counting during QFR analysis. Frame counting is essential for cQFR which has shown to be superior to fQFR or adenosine-flow QFR (aQFR) [6]. In our current study, cQFR revealed no additional advantage compared with fQFR. In the cQFR processing, the shortest thrombolysis in myocardial infarction (TIMI) frame count is essential for calculating the contrast medium transport time in the interrogated vessel, which is one of the key factors to calculate hyperemic flow velocity (HFV) [6, 20]. Hence, lower frame rate results in a more extensive time interval, which decrease the precision of contrast medium transport time, therefore influences the precision of HFV, and in turn the accuracy of cQFR. The lowest frame rate used in QFR validation studies was 12.5fps and is currently the recommended minimum by the software [12]. Unlike cQFR, fQFR uses a fixed empiric HFV of 0.35 m/s, therefore lower frame rate does not affect the calculation of fQFR. Our current study revealed that both fQFR and cQFR had a similar performance at detecting functional stenoses, this may allow time to be saved by focusing on fQFR when low frame rate radiation save mode angiograms are used for analysis.

Limitations

This study is a retrospective study, 22 cases were excluded due to lack of 2 suitable projections for QFR/vFFR analysis and inaccurate FFR measurement, this may cause bias. A prospective multi-center study with a larger patient cohort is needed to validate our algorithm and assess the effects of angiography derived FFR. Also, this study only evaluated the performance of QFR/vFFR under 7.5fps/7pps angiograms, other radiation saving mode (e.g. 7.5fps/15pps or 15fps/7pps, etc.) angiograms should be further analyzed to draw a clear picture of the performance of QFR/vFFR under such conditions.

Conclusion

Angiography derived FFR assessment of intermediate coronary stenosis is feasible under radiation save mode—7.5fps and 7pps coronary angiography. In these cases, the newly developed algorithm helps to exclude vessels that may not be suitable for angiography derived FFR analysis, thus improving the diagnostic accuracy of these commercially available software packages. With 7.5fps angiography, cQFR reveals no additional advantage compared to fQFR, so fQFR could be the first choice for evaluating such cases.