Introduction

Perfusion cardiovascular magnetic resonance (CMR) is an emerging technique for the detection of coronary artery disease (CAD). The technique is attractive because of its non-invasive nature and safe characteristics, and might potentially play a major role in future diagnosis and risk stratification guidelines for patients with suspected CAD. Several small studies have evaluated the diagnostic performance of stress perfusion CMR and some of those have been included in a previous meta-analysis [1]. In the current study we provide a comprehensive and contemporary meta-analysis of its diagnostic accuracy compared with an invasive coronary angiography (CA) used as a reference standard.

Methods

Search strategy

Using the combined medical subject headings (MeSH) of magnetic resonance, perfusion, and coronary angiography, with the exploded terms coronary artery disease; the MEDLINE, Cochrane Library and BioMedCentral databases were searched independently by two investigators (MH, GF) for all publications, in any language, before July 2009. In addition, the published reference lists of these articles were systematically searched.

Study eligibility

The search results were collated by the same two investigators (MH, GF), and duplicate or overlapping papers removed. Studies were eligible if: [1] stress perfusion CMR was used as a diagnostic test for significant obstructive CAD; [2] conventional invasive CA was used as the reference standard for diagnosing significant obstructive CAD, defined as a ≥50% diameter stenosis; and [3] the absolute numbers of true positive (TP), false positive (FP), true negative (TN), and false negative (FN) were reported, or could be derived. Studies were excluded if they were performed with a 0.5 or 1 Tesla MR scanner, if they included less than 10 patients, and if only abstracts from scientific meetings were published as the data provided may either be not sufficiently detailed or finalized. Any disagreements on eligibility were resolved by discussion and consensus between the two investigators.

Data extraction and quality assessment

Data extraction was performed independently by the two investigators (MH, GF) for each study. The following fields were recorded: study population size; gender distribution; mean age and standard deviation; number of patients with documented CAD; prevalence of CAD; relative timing of the two imaging procedures; the degree of blinding in interpretation of test results (both to the patient's clinical context and the results of the other imaging modality); type and brand of MR machine used; the type of perfusion stressor (adenosine, nicorandil, dipyridamole), and the number of side effects; the dose and injection rate of Gadolinium administrated; and the modality of MR image analysis (visual, or semi-quantitative). Any discrepancies were resolved by discussion and consensus between the two investigators. Where available, data was recorded separately at the level of coronary territories and coronary arteries. The study quality conformed to the Quality Assessment of Studies of Diagnostic Accuracy included in Systematic Reviews guidelines [2]. In one study, for which patients were evaluated both with 1.5 and 3T CMR, we used 1.5 T data in the meta-analysis. For the studies where analysis was performed with both 50% and 70% coronary stenosis definitions, we included results with the 70% definition in the pooled reported sensitivity and specificity.

Data synthesis and statistical analysis

Data analysis was performed at the level of the patient, the coronary territory and the coronary artery. Sensitivity and specificity were calculated using the TP, TN, FP, and FN rates [3, 4]. From these were calculated the likelihood ratios, which express how much the odds of significant obstructive CAD change in the presence of either an abnormal stress perfusion CMR (positive likelihood ratio: PLR = sensitivity/(1- specificity)), or a normal stress perfusion CMR (negative likelihood ratio: NLR = (1- sensitivity)/specificity). Finally, the ratio of the PLR to the NLR was used to calculate the diagnostic odds ratio (DOR), which estimates how much greater the odds of having significant obstructive CAD are for patients with a positive test result compared with a negative one.

All these measures of diagnostic accuracy were calculated for each individual study and reported as point estimates with 95% confidence intervals. They were then combined using a random-effects model and each point estimate weighted by the inverse of the sum of its variance and the between-study variance. We also assessed between-study statistical heterogeneity using the Cochran Q chi-square tests (cut off for statistical significance P ≤ .10). Since diagnostic parameters are, by definition, interdependent, independent weighting may sometimes give spurious results and provide biased estimates; to overcome the interdependence problem, we computed the weighted symmetric summary receiver operating characteristic curve, with pertinent areas under the curve, using the Moses-Shapiro-Littenberg method [5–7]. All statistical calculations were performed with SPSS 14.0 (SPSS, Chicago, IL) and Meta-DiSc [8], and significance testing was at the two-tailed 0.05 level [9].

Results

Database and literature searches retrieved 263 citations, amongst which 55 relevant publications were identified (Figure 1). Further scrutiny led 20 papers to be rejected either because of overlapping data, or exclusion criteria were met (employed 0.5 or 1 T CMR, or inclusion criteria were absent (impossible to find or calculate absolute figures from presented data). Therefore, 35 studies were finally included in the meta-analysis [10–44], all of which had been published between 2000 and 2009. Study and population characteristics are summarized in Table 1, and the results of the pooled analyses are summarized in Table 2. Dose of contrast Gadolinium administrated range from 0.025 to 0.15 mmol/kg, with an injection rate varying from 3 to 10 mL/s. Quality assessments for all included studies are shown in Table 3. The 35 papers eligible for the analyses comprised 2,456 patients, and of the 2,154 patients for whom gender and the age were specified, 1,481 were males (68.7%) and the mean age was 61.3 years.

Figure 1
figure 1

Flow diagram of the reviewing process.

Full size image
Table 1 Characteristics of included studies
Full size table
Table 2 Pooled summary results
Full size table
Table 3 Quality assessment (QUADAS)
Full size table

Diagnostic performance of stress perfusion CMR: Patient-based analysis

Overall per-patient analysis results pooled from 26 studies (2,125 patients) demonstrated a sensitivity of 89% (95% CI: 88-91%), a specificity of 80% (95% CI: 78-83%), a PLR of 4.18 (3.31-5.27), a NLR of 0.15 (95% CI: 0.11-0.20), a DOR of 33.65 (95% CI: 22.09-51.27), and an AUC of 0.92 (Figures 2, 3, 4, 5, 6). Statistical heterogeneity was observed for all relevant diagnostic performance measures. The per-patient prevalence of CAD was 57% (1,205 of 2,125 patients).

Figure 2
figure 2

Forest plot of patient-level sensitivity of stress perfusion CMR, compared with coronary angiography..

Full size image
Figure 3
figure 3

Forest plot of patient-level specificity of stress perfusion CMR, compared with coronary angiography.

Full size image
Figure 4
figure 4

Forest plot of patient-level positive likelihood ratio of stress perfusion CMR, compared with coronary angiography.

Full size image
Figure 5
figure 5

Forest plot of patient-level negative likelihood ratio of stress perfusion CMR, compared with coronary angiography.

Full size image
Figure 6
figure 6

Plot of symmetric summary receiver operating curve characteristic of stress perfusion CMR, compared with coronary angiography. The receiver operator characteristic curve provides a graphical display of diagnostic accuracy by plotting 1-specificity in the horizontal axis and sensitivity in the vertical axis. The pertinent area under the curve (AUC) and the Q* statistic (the point where sensitivity and specificity are maximized), both with standard errors (SE), are also included.

Full size image

With adenosine as the stressor (20 studies, 1,658 patients) the results were: a sensitivity of 90% (88-92%), a specificity of 81% (78-84%), a PLR of 4.47 (3.39-5.88), a NLR of 0.14 (0.11-0.18), a DOR of 37.17 (25.16-54.91), and an AUC of 0.93. Statistical heterogeneity was observed for all relevant diagnostic performance measures. With dipyridamole as the stressor (5 studies, 417 patients), the results were: a sensitivity of 86% (80-90%), a specificity of 77% (71-82%), a PLR of 2.97 (2.16-4.09), a NLR of 0.20 (0.09-0.45), a DOR of 17.03 (5.56-52.18), and an AUC of 0.84. Statistical heterogeneity was observed for all relevant diagnostic performance measures, except specificity and positive likelihood ratio. ROC curves for stress perfusion CMR performed with adenosine or dipyridamole are shown in Figure 7.

Figure 7
figure 7

Plots of symmetric summary receiver operating curve characteristic of stress perfusion CMR, compared with coronary angiography for adenosine and dipyridamole stressors.

Full size image

A sensitivity analysis was carried out based on the equipment used (3 Tesla, and 1.5 Tesla MRI). For 3 Tesla (4 studies, 282 patients), results were: a sensitivity of 92% (87-95%), a specificity of 78% (69-85%), a PLR of 3.96 (2.78-5.63), a NLR of 0.12 (0.07-0.20), and a DOR of 35.74 (17.13-74.53). For 1.5 Tesla (23 studies, 1,904 patients), results were: a sensitivity 89% (87-91%), a specificity of 80% (78-83%), a PLR of 4.26 (3.26-5.55), a NLR of 0.15 (0.11-0.20), and a DOR of 34.25 (21.26-55.17).

Diagnostic performance of stress perfusion CMR: Coronary territory and coronary artery-based analysis

Per-territory results, pooled from 17 studies corresponding to 2,709 coronary territories, demonstrated a sensitivity of 82% (79-84%), a specificity of 84% (82-85%), a PLR of 4.90 (3.66-6.55), a NLR of 0.23 (0.20-0.27), and a DOR of 23.23 (18.33-29.45). At the territory level heterogeneity was significant for all relevant diagnostic performance measures except sensitivity, negative likelihood ratio and diagnostic odd ratios.

Per-artery analysis pooled 8 datasets and demonstrated for left anterior descending artery (LAD), circumflex artery (CX) and right coronary artery (RCA), respectively, sensitivities of 83%, 76% and 78% and specificities of 83%, 87%, and 87%. Statistical heterogeneity was observed for all the performance measurements except sensitivity and negative likelihood ratio for LAD and CX, and diagnostic odds ratio for CX.

Discussion

This meta-analysis showed stress perfusion CMR to have a high sensitivity (89%) and a moderate specificity (80%) at patient level for the diagnosis of significant obstructive CAD in patients with high prevalence of CAD (57%). We included twelve more studies (on stress perfusion CMR) than the previous meta-analysis by Nandalur et al. [1], which showed a similar diagnostic performance with a pooled sensitivity and specificity of respectively 90% and 81% from 14 perfusion studies. A high false positive rate could have driven the relatively low specificity, and may be due to perfusion defects caused by: [1] dark rim artefacts, the hypo-intensities along the endocardial border of the left ventricular myocardium seen during first-pass transit of a MR contrast medium, thought to be due to a combination of the gadolinium bolus, motion and resolution [45]; [2] the presence of microvascular disease; and [3] spontaneous or therapeutic re-opening of a coronary artery supplying an area of myocardial infarction that has persistent microvascular obstruction [28, 32].

Alternatively, because CA detects luminal morphology rather than the functional significance of a stenosis, a false positive CMR results may in fact represent a 'false negative' angiogram in the context of angiographically 'invisible' small vessel disease capable of inducing subendocardial ischemia [40]. This potential source of error could be minimised if the hemodynamic significance of an epicardial coronary artery stenosis were to be determined by the measurement of the fractional flow reserve (FFR) during CA. If validated, this may represent a better reference standard than CA alone. However, although three studies found there to be a good correlation between the performance of stress perfusion CMR and CA with FFR measurement [31, 33, 35], sufficient data was not present to evaluate its accuracy in this study.

Another point to outline is that for some studies [11, 17, 19], different decision thresholds to diagnose perfusion CMR as abnormal were appraised: for these studies, the reported sensitivity and specificity could be considered as optimistic because the end points was chosen retrospectively.

In addition, there was a large range of contrast doses used in the individual studies, with the dose of gadolinium administered in the included studies varying by 6-fold, with dose ranging from 0.025 to 0.15 mmole/kg. Although currently there is no consensus regarding the optimal dose and injection rates for perfusion CMR, two multicenter dose-ranging studies have evaluated the impact of contrast dose on the performance of perfusion CMR using a visual analysis [46, 47]. In the first, Wolff et al. considered a low dose of 0.05 mmol/kg to be at least as efficacious as any higher dose, and hypothesized that higher doses preformed less well because of the increased likelihood and intensity of artefacts at these doses [46]. However, in the MR-Impact study, Schwitter et al. found better results were obtained using 0.1 mmol/kg [47].

In this meta-analysis, 18 studies were based on stress perfusion CMR alone [10–13, 15, 17, 19–21, 24, 26, 31–33, 35, 37, 39, 44], whilst the other 17 included a multi-component examination (cine and/or late gadolinium enhancement (LGE) and/or coronary angiography and/or stress tagging) [14, 16, 18, 22, 23, 25, 27–30, 34, 36, 38, 40–43]. In their studies, Plein [22], Cury [28] and Klem [29] evaluated the differences in accuracy based on the sequences evaluated and found that all studies increased accuracy when using a combined analysis. In his study, Klem reported increased specificity (moving from 58% to 87%) when using an algorithm interpretation (including perfusion, cine and LGE).

Having access to data from different sequences (cine, perfusion, and LGE) is especially useful when one component shows a borderline result or is affected by image artefacts. Most of the authors have argued that rest perfusion is an important component because, in combination with late enhancement CMR, it can help distinguish true defects from artefacts on the stress perfusion images.

The fact that the meta-analysis demonstrated a low NLR for stress perfusion CMR suggests that a negative test result may in fact be more clinically useful. This is in keeping with several reports, in different clinical settings, of improved prognosis associated with a normal adenosine stress perfusion CMR scan [48–50]. This meta-analysis also demonstrated adenosine to be superior to dipyridamole as the vasodilating stressor agent. Adenosine may also be safer, with minor side effects of flushing and headache being reported to occur more frequently that any severe adverse effects [51]. Its shorter half life (< 10 s) is an added advantage. Moreover, adenosine has documented safety in the context of non-ST elevation acute coronary syndromes (in a study of 72 patients only one demonstrated intolerance), and in recent ST elevation myocardial infarction [14, 22, 34].

From this analysis, visual assessment of stress perfusion CMR provided a higher sensitivity but a lower specificity than semi-quantitative assessment. Currently there is no consensus on the superiority of visual over semi-quantitative assessment, or on which method of semi-quantitative assessment should be used. However, the drawbacks of semi-quantitative assessment are that it is more time-consuming, hence not ideal for day-to-day clinical purposes, and the lack of any homogeneous post-processing protocols. Therefore, visual assessment is currently the method most often used in routine clinical practice.

Only 4 studies were performed using 3T CMR, which provides improved resolution [32, 38, 39, 43]. Enhanced sensitivity has been reported [32] and attributed to the higher signal-to-noise and contrast-to-noise ratios permitting improved detection of endocardial perfusion defects. Although most authors argue that the increased prevalence of dark rim artefacts at these higher field strengths (ranging from 8 up to 82%) does not hamper myocardial perfusion analysis [32, 39, 43], Gebker disagrees and suggests they could limit specificity by increasing false positive rates [38]. In this analysis, 3T CMR was also found to have a decreased specificity, indicating that higher false positive rates may be a real problem. Further studies will be necessary if this controversy is to be resolved.

The results of the per- territory-based analysis showed the anticipated decrease in sensitivity and increase in specificity seen when moving from the level of the patient to that of the coronary territory. Among the 8 studies that performed a coronary-artery level analysis, stress perfusion CMR had a higher sensitivity for detection of significant coronary disease in the LAD artery, compared with the CX and RCA. A possible explanation for this finding may have been the use of a surface radiofrequency coil, which led to lower signal intensities in the more distant inferior and lateral segments.

Study limitations

Although conventional CA is the established technique for diagnosing significant CAD in routine clinical practice, it remains an imperfect reference standard due to its inability to evaluate the hemodynamic significance of a stenosis.

Substantial inter-study heterogeneity in multiple performance characteristics were observed. Therefore, the pooled performance indices and their interpretation have to be treated with a degree of caution, even though the random-effects model used throughout the analysis should have compensated for this. The observed heterogeneity may have been due to variations in: (i) the image acquisition technique (MR scanner manufacturer, 1.5T or 3T field strengths, pulse sequence, number of slices, contrast dose and rate of infusion); (ii) the interpretation method (visual or semi-quantitative, post-processing techniques); (iii) the patient selection criteria (exclusion or inclusion of patients with prior myocardial infarction, patient populations with differing prevalence of CAD); and (iv) in the definition of significant obstructive CAD (50% or 70%). We noticed, as expected, that studies which performed analysis for 50% and for 70% coronary artery stenosis thresholds, reported an increased sensitivity and a decreased specificity when moving thresholds from 50% to 70% [29, 33, 36].

These general limitations of stress perfusion CMR could be addressed in future multi-centre studies if standardized imaging protocols, post-processing techniques and patient selection criteria are employed.

Conclusion

Stress Perfusion CMR has a high sensitivity and moderate specificity for the diagnosis of significant obstructive CAD compared with CA in patients with a high prevalence of the disease.

Future technical developments that increase spatial and temporal resolution whilst reducing artefacts may further improve the diagnostic performance of stress perfusion CMR, and in particular improve its specificity [32]. Currently, however, the low NLR makes stress perfusion CMR particularly accurate and useful in ruling out significant CAD.