Background

Globally, about 1 billion people are overweight or obese, which will cause an increase of epidemic proportions in the number of persons with type 2 diabetes and cardiovascular disease. In 2011, 366 million people had been diagnosed with diabetes, and the number is expected to rise to 552 million by 2030. Most people with diabetes live in the developing world, and this area will see the greatest increase over the next 19 years [1].

Type 2 diabetes mellitus (T2DM) is characterised by high blood glucose levels that cause eye, kidney, and nerve complications and an increased risk for cardiovascular disease. Most of the morbidity and mortality associated with diabetes is due to cardiovascular disease; diabetes is considered by some as a coronary heart disease risk–equivalent condition [2]. Although this concept is controversial [3], there is evidence that the risk equivalence might hold at least for women [4].

Controversy has surrounded the cardiovascular safety of some non-insulin blood glucose–lowering drugs (NIBGLD), particularly sulfonylureas and thiazolidinediones. Rosiglitazone was removed from the market in Europe and its use highly restricted in the United States of America (US) after review of safety evaluations from clinical trials suggested that its use increased the incidence of acute myocardial infarction (AMI) [5]. This and the premature termination of the ACCORD trial [6], after observing an increase in cardiovascular mortality in the group treated more aggressively, prompted the US Food and Drug Administration (FDA) to revise its policy for approving new drugs for T2DM; a two-step approach ensuring cardiovascular safety is now required. Evidence has led some to question rosiglitazone’s adverse cardiovascular safety profile [7–9]. Using observational data, compared with metformin, sulfonylureas have been associated with a 21 % increase in the risk of hospitalisation for AMI, stroke, or death [10].

Meta-analyses of clinical trials and observational studies reviewing the risk of AMI and/or stroke associated with the use of oral blood glucose–lowering drugs for T2DM have been published [11–18]. However, previous meta-analyses of observational studies have not investigated the heterogeneity present in results and methods of individual studies [11, 15, 16]. Heterogeneity, analysis, important for meta-analyses of observational studies [19], might explain inconsistencies among results of individual studies.

This research was part of the Safety Evaluation of Adverse Reactions in Diabetes (SAFEGUARD) project. This report summarises the results of a systematic review and meta-analysis of published observational studies on the risk of AMI and stroke in patients with T2DM receiving treatment with NIBGLD, analysis of the heterogeneity in study characteristics and results, and evaluation of study quality as an explanatory factor for statistical heterogeneity. We also provide results of drug comparisons not reported in previous meta-analyses. With the recent introduction in clinical practice of new classes of NIBGLD such as dipeptidyl peptidase-4 inhibitors, glucagon-like peptide-1 receptor agonists, and sodium-glucose cotransporter-2 inhibitors, it becomes crucial to evaluate the cardiovascular safety of older NIBGLD (e.g., glitazones, metformin and sufonylureas) before their use in clinical practice may change with the introduction of the newer agents, and newer clinical guidelines.

Methods

We conducted a systematic literature search in Medline, Embase, and the Cochrane Library. The search was conducted on November 11, 2011 (see Additional file 1), and was limited to observational studies on humans (systematic reviews, meta-analyses, original articles), with no publication date or language restrictions for blood glucose–lowering drugs (except insulin); outcomes were AMI, stroke, and others reported elsewhere with details on the literature search, study selection, and data abstraction [20]. Studies including transient ischaemic attack in the definition of stroke were excluded. Studies including stable angina in the definition of AMI were excluded. We conducted an updated literature search in September 2014, and the impact of additional studies that were identified is reported in the discussion section.

We assessed the quality of each study included in the systematic review using two tools, the Newcastle-Ottawa Scale [21] and the RTI item bank (RTI-IB) on risk of bias and precision [22]. We used a version of the RTI-IB adapted to the research question with 31 items. It was applied independently by two researchers who discussed disagreements until reaching consensus, when necessary. Possible responses for most items denoted high, unclear, or low risk of bias. Additional details on the tool and how the quality assessment was performed are reported elsewhere [23].

Quantitative analysis was conducted using Review Manager software version 5.2.3 [24]. Additional details can be found elsewhere [20]. For each comparison with at least three independent point estimates available, summary relative risks (sRRs) and 95 % CIs for AMI and stroke were estimated using both fixed- and random-effects models. Forest plots were constructed based on random-effects models, and between-study heterogeneity was assessed by graphical inspection and with the Higgins I 2 statistic, which describes the percentage of between-study variability in effect estimates attributable to true heterogeneity rather than chance. The Cochran’s χ2 test of homogeneity and Tau2, estimating the between-study variance, are also presented. For studies providing estimates for both monotherapy and combined therapy, we selected the combined-therapy estimate because in clinical practice most patients with T2DM require combined therapy and because it would include more patients (more precision). To explore sources of heterogeneity, we implemented subgroup analyses designed a priori [20], followed by sensitivity analyses in which we excluded studies with high/unclear risk of bias for more than 30 % of the items in the RTI-IB. An additional sensitivity analysis examined the potential impact of the rosiglitazone controversy on the meta-analysis results by grouping the studies according to whether the study period finished before or on 2007, started after 2007, or included 2007, the year in which the first studies showing an increased cardiovascular risk associated with the use of rosiglitazone were published. Publication bias was examined by visual evaluation of funnel plots. The present report follows the MOOSE (Meta-analysis Of Observational Studies in Epidemiology) checklist [25] (see Additional file 1: Table 10e). The present systematic review of published observational studies does not require ethics approval.

Results and discussion

Results

Figure 1 displays the study selection process. From the 44 studies selected for the systematic review, we identified 23 studies evaluating the risk of AMI and 8 studies evaluating the risk of stroke. Of the 23 studies on AMI, 17 contributed to the meta-analysis [26–42], with five drug comparisons: rosiglitazone versus pioglitazone (n = 11 studies), rosiglitazone versus metformin (n = 7), pioglitazone versus metformin (n = 4), rosiglitazone versus sulfonylureas (n = 5), and sulfonylureas versus metformin (n = 4). Of the 8 studies on stroke, 3 contributed to the meta-analysis for the comparison of rosiglitazone versus pioglitazone [26, 30, 41]. Table 1 displays the list of studies excluded from the meta-analyses for AMI and stroke and the reasons for exclusion.

Fig. 1
figure 1

Flow diagram of study identification and selection process. Note: No additional study was identified by checking reference lists of included studies. Some studies contributed to more than one drug-drug comparison

Full size image
Table 1 Reasons for exclusion of studies
Full size table

Table 2 displays the main characteristics of the 17 studies evaluating the risk of AMI included in the present meta-analysis. Of the 17 studies, 1 was a case–control study and the rest were cohort studies (3 included a nested case–control analysis). The majority of the studies defined the outcome based on hospitalisations. Most studies included the specific diagnosis code for AMI (i.e., ICD-9 code 410 or ICD-10 codes I21 and I22) in their algorithms to identify cases of AMI; Tzoulaki et al. [38] used Read codes, and Sauer et al. [37] used the Minnesota Heart Study criteria to validate cases of AMI in a field study. Six studies included only first-ever AMI; 11 studies included first-ever AMI plus previous AMI. The number of AMI events in each study ranged from 39 to 15,917. The majority of studies included a broad age range, while a few were restricted to those aged 65 years or older. The definition of exposure varied across studies: 10 studies included only new users of the drugs studied, and the remaining 7 studies included a combination of new and prevalent users. Thirteen studies included patients from North America, 2 studies included European patients, 1 study was conducted in Israel, and 1 in Taiwan. Most of the US studies used claims databases.

Table 2 Main characteristics of studies included in the meta-analysis
Full size table

Table 2 displays the main characteristics of the 3 studies evaluating the risk of stroke. All were cohort studies comparing rosiglitazone versus pioglitazone [26, 30, 41]. The number of events ranged from 32 to 1,869. Two studies included only elderly subjects that were new users of study medications, while the other study included all subjects aged older than 17 years and both new and prevalent users.

In Figs. 2 and 3, the left panel displays the random-effects forest plot for the comparisons included in our meta-analysis. Summary results of study quality assessment with the RTI-IB are provided in the right panel. Detailed results of the Newcastle-Ottawa Scale and the RTI-IB for the 31 studies included in the systematic review are in the Additional file 1. An overview of the experience with and a comparison of the two tools is reported elsewhere [23].

Fig. 2
figure 2

Relative risk of acute myocardial infarction and stroke in rosiglitazone users compared with pioglitazone users. AMI, acute myocardial infarction; IV, inverse variance. Red bars, percentage of items in the RTI item bank indicating high risk of bias; yellow bars, percentage of items at unclear risk of bias. Items with low risk of bias not shown. Denominators indicate the number of items evaluated for each study.

Full size image
Fig. 3
figure 3

Relative risk of acute myocardial infarction in users of blood glucose–lowering medications. IV, inverse variance. Red bars, percentage of items in the RTI item bank indicating high risk of bias; yellow bars, percentage of items at unclear risk of bias. Items with low risk of bias not shown. Denominators indicate the number of items evaluated for each study. Rosiglitazone versus metformin: Tzoulaki et al. [38] contributed the relative risk reported for combination therapy; for Loebstein et al. [35], the combination-therapy estimate was not estimable. Sulfonylureas versus metformin: Tzoulaki et al. [38] contributed the estimate for second-generation sulfonylureas

Full size image

Risk of AMI: rosiglitazone versus pioglitazone

The overall sRR (95 % CI) was 1.13 (1.04-1.24), and there was evidence of substantial heterogeneity (I 2 = 55 %) (Fig. 2). With the RTI-IB, for AMI, 2 studies had a high/unclear risk of bias for 30 % or more of the items assessed [27, 29]; 2 more studies had a high/unclear risk of bias for 25 to 29 % of the items assessed [26, 40]. For stroke, 1 study had high/unclear risk of bias for 25 % or more of the items assessed [26].

The sensitivity analyses showed that heterogeneity was reduced when restricting the analysis to new users of rosiglitazone and pioglitazone (I 2 = 37 %) (Table 3). Even larger reductions in heterogeneity were obtained in the sensitivity analysis excluding the two studies at high/unclear risk of bias for 30 % or more of the items assessed (I 2 = 16 %). Heterogeneity was also markedly reduced in the subgroup sensitivity analyses. The sRR did not change much in those analyses and were around 1.10 in most cases, but the 95 % CIs were more conservative; when the two high/unclear-bias studies were removed from the first-ever and prior AMI subgroup and the new users subgroup, the random-effects CIs included the null effect (Table 3).

Table 3 Risk of AMI in rosiglitazone users compared with pioglitazone users, overall and subgroup analyses
Full size table

Of the 11 studies included in the AMI meta-analysis, three reported estimates for monotherapy drug use: one study reported estimates for rosiglitazone and pioglitazone as add-on therapy for metformin users [28], and the rest reported estimates of comparisons in which both rosiglitazone and pioglitazone groups allowed combined treatment with other oral blood glucose–lowering drugs. The monotherapy groups included fewer patients than the combination-therapy groups because very rarely in clinical practice are glitazones prescribed without adjuvant therapy.

Risk of stroke: rosiglitazone versus pioglitazone

The sRR (95 % CI) was 1.18 (1.02–1.36); there was no strong evidence of heterogeneity (I 2 = 42 %). The meta-analysis evaluating the risk of stroke associated with the use of rosiglitazone compared with pioglitazone included 3 cohort studies of combination therapy (Fig. 2). Two studies used Medicare data and studied patients aged 65 years and older; 1 included patients aged 18 years or older. None reported information on dose or duration.

Risk of AMI: rosiglitazone versus metformin

The overall sRR (95 % CI) was 1.42 (1.03–1.98) (Fig. 3), but, there was strong evidence of heterogeneity (I 2 = 86 %). With the RTI-IB quality assessment, 2 studies had a high/unclear risk of bias for 30 % or more of the items assessed [27, 32]; and 1 more study had a high/unclear risk of bias for 25 % or more of the items assessed [36].

Subgroup analyses showed that heterogeneity was reduced when restricting the analysis to new users of rosiglitazone and metformin (I 2 = 55 %) (Table 4). Even larger reductions in heterogeneity were obtained in the sensitivity analyses excluding the two studies at high/unclear risk of bias for 30 % or more of the items assessed (I 2 = 0 %). The sRR decreased in the subgroup analysis (new users) to 1.29 and to 1.13 in the overall sensitivity analysis; the null effect was included in the subgroup analyses random-effects 95 % CI and the overall sensitivity analysis random- and fixed-effects 95 % CIs.

Table 4 Risk of AMI in rosiglitazone users compared with metformin users, overall and subgroup analyses
Full size table

All 7 studies included in the meta-analysis used a cohort design, but 1 study performed a nested case–control analysis. Only 1 study was restricted to first-ever events. Of the 7 studies, 5 included only new users of the drugs studied, 4 reported estimates only for monotherapy drug use, 1 reported estimates only of comparisons in which the rosiglitazone and pioglitazone groups allowed combination treatment with other oral blood glucose–lowering drugs, and 2 provided estimates for both monotherapy and combination therapy. Only 1 study provided results by exposure duration.

Risk of AMI: pioglitazone versus metformin

The sRR (95 % CI) was 1.02 (0.75–1.38), and there was no evidence of significant heterogeneity in a meta-analysis of 3 studies (I 2 = 17 %). Four studies were available for this comparison (Fig. 3); however, the 95 % CI reported by Dormuth et al. [28] did not result in an estimable standard error and therefore the study could not be included in the meta-analysis.

Risk of AMI: rosiglitazone versus sulfonylureas

The sRR (95 % CI) was 0.99 (0.78–1.25), with strong evidence of heterogeneity for this comparison (I 2 = 70 %) (Fig. 3 and Additional file 1: Table 8e). Five studies were available for meta-analysis. In the sensitivity analysis that excluded the high/unclear-bias studies [27, 32], heterogeneity was markedly reduced (I 2 = 0 %) and the sRR (95 % CI) decreased to 0.82 (0.69–0.98).

Risk of AMI: sulfonylureas versus metformin

The sRR (95 % CI) was 1.24 (1.14–1.34) and evidence of heterogeneity was moderate (I 2 = 41 %). Four studies were available for this comparison (Fig. 3, Additional file 1: Table 9e); all analysed sulphonylureas as a group and some also reported on individual agents. Individual agents in the evaluated studies were either first- or second-generation sulphonylureas. For our analysis, we included the reported RR for second-generation sulfonylureas as they were the most frequently used. Three of the four studies [28, 31, 38] provided specific data for the comparison of glyburide versus metformin. When combining those studies (Additional file 1: Table 10e), effect estimates were homogeneous across studies and there was no evidence of heterogeneity (I 2 = 0 %). The sRR (95 % CI) was 1.22 (1.14–1.31). Two of the four studies were included in the meta-analysis for only this comparison [31, 37]. One was a field case–control study with an imprecise RR estimate and 30 % or more of the RTI-IB items assessed at high/unclear risk of bias. The other was a nested case–control analysis that included prevalent and new users. Three studies included only first-ever AMI.

No impact on the meta-analysis results was found after grouping the studies according to whether the study period finished before or on 2007, started after 2007, or the study period included 2007 (data not shown).

Publication bias

Examination of funnel plots did not suggest publication bias, although the number of studies was small for some comparisons. The funnel plots for studies evaluating the risk of AMI in rosiglitazone users compared with pioglitazone users (n = 11), in rosiglitazone users compared with metformin users (n = 7), and in rosiglitazone users compared with sulfonylurea users (n = 5) are displayed in Additional file 1: Figures 1e, 2e, and 3e.

Discussion

This systematic review of published observational studies on the risk of AMI associated with glitazones, metformin and sufonylureas in patients with T2DM confirmed that studies in this field are very heterogeneous in exposure definition, comparison drugs, potential for biases secondary to design characteristics, and study results. Therefore, summarising the scientific evidence is challenging. The lack of a common reference medication for evaluation of all potential exposures across studies limited direct comparison of effect estimates. Of the 31 studies included in our systematic literature review, only 20 could contribute to the meta-analysis, 17 for AMI and 3 for stroke.

Our summary effect estimates are compatible with a small increase (around 10%) in the risk of AMI among patients with T2DM using rosiglitazone compared with the risk in those using pioglitazone. However, the degree of heterogeneity present and results of the quality assessment and sensitivity analyses indicate that caution should be used when interpreting this result. Residual confounding might be present in most observational studies; therefore, small increases in risk are difficult to interpret.

Studies in the stroke risk meta-analysis (n = 3) had more homogeneous point estimates and quality assessments than studies in the AMI meta-analysis (n = 11). Considering that the lower limit of the 95 % CI is very close to 1 and only 3 studies were available for meta-analysis, the observed increase of 18 % in the risk of stroke among patients using rosiglitazone compared with those using pioglitazone should be considered cautiously.

Similar words of caution apply to the comparisons of rosiglitazone versus metformin and rosiglitazone versus sulfonylureas. Although the sRRs indicate a 40 % increase in risk of AMI for the first comparison and no increase in risk for the second comparison, there was strong evidence of heterogeneity for both comparisons, and the sensitivity analyses produced effect estimates closer to the null for the first comparison and further from the null for the second comparison (from sRR of 0.99 to 0.92 in the subgroup analysis and to 0.82 in the sensitivity analysis).

Results reported from the individual studies included in other pioglitazone-versus-metformin and sulfonylureas-versus-metformin comparisons produced more homogeneous results but involved few studies. The sRR for pioglitazone versus metformin suggested no difference in AMI risk for the two drugs, but the other comparison indicated a 24 % higher risk of AMI for sulfonylureas users than for metformin users.

In September 2014, we updated the literature search using the original search terms in PubMed. Two new studies were considered eligible and were reviewed [43, 44]; they compared risk of AMI and of stroke for rosiglitazone and pioglitazone. The updated effect estimates adding these 2 studies remained virtually unchanged: sRR, 1.12 (95 % CI, 1.03–1.21) for AMI and 1.17 (95 % CI, 1.07, 1.27) for stroke.

The results of our meta-analysis are consistent with results of two previous meta-analyses of observational studies evaluating the risk of AMI comparing rosiglitazone and pioglitazone. The meta-analysis of 7 studies conducted by Chen et al. [16] estimated a sRR of 1.17 (95 % CI, 1.04–1.32); I 2 = 70 %. Loke et al. [11] conducted a meta-analysis of 13 studies including patients with T2DM, comparing rosiglitazone and pioglitazone safety for AMI, congestive heart failure, and mortality. For AMI, the odds ratio was 1.11 (95 % CI, 1.04–1.18) ; I 2 = 16 %. These estimated relative risks are similar to our estimate. Differences in the studies included/excluded between those two meta-analyses and our meta-analysis might explain the differences in statistical heterogeneity.

Three meta-analyses of randomised controlled trials (RCTs) have been published evaluating the safety of rosiglitazone for AMI and stroke and found that rosiglitazone increased the risk of AMI about 40 % compared to control therapies or placebo [12–14]. The results of those meta-analysis were not without controversy [45], and there is evidence that they are sensitive to the statistical method used for pooling and the inclusion/exclusion of different studies [46]. However, the available evidence prompted the European Medicines Agency to withdraw rosiglitazone from the market and the FDA to place severe restrictions on rosiglitazone use [5]. In contrast, meta-analyses indicate a reduced risk of stroke (about 20 %) for rosiglitazone compared to control therapies. Although the 95 % CIs of those estimates include the null effect, the inconsistency of the findings for AMI and stroke is difficult to explain [46]. An independent re-analysis of the RECORD trial data confirmed the original results [9], that rosiglitazone did not increase cardiovascular risk compared to a combination of metformin and sulfonylurea [47]; this has prompted the FDA to ease some restrictions on rosiglitazone use. Another re-analysis of clinical trial data has further questioned the adverse cardiovascular safety profile of rosiglitazone, reporting a hazard ratio of 0.77 for AMI (95 % CI, 0.54–1.10) and of 0.36 (95 % CI, 0.16–0.86) for stroke when comparing rosiglitazone with non-thiazolidinedione use [7].

We did not identify any meta-analysis of RCTs directly comparing rosiglitazone and pioglitazone with AMI as the safety outcome of interest. Two meta-analyses of RCTs comparing pioglitazone with control therapies or placebo found reductions of around 20 % in the risk of AMI or stroke, favouring pioglitazone, with 95 % CIs in both cases including the null effect [48, 49].

For AMI and stroke we could not identify any published meta-analyses of observational studies evaluating the other drug comparisons included in this meta-analysis, i.e., rosiglitazone or pioglitazone versus metformin, rosiglitazone versus sulfonylureas, and sulfonylureas versus metformin. Therefore, the current meta-analysis seems to be the first one to review the results of studies including all those comparisons.

This systematic review and meta-analysis has several strengths. We included in the meta-analysis only studies with a clear definition of the reported comparisons. Our detailed evaluation of the quality of each reviewed study and the sensitivity analyses helped us interpret meta-analysis results of heterogeneous studies combined for the purpose of evaluating the risk of AMI and stroke. Our analyses included drug comparisons relevant to clinicians and not included in previous meta-analyses.

As is true for every meta-analysis of observational studies, the main limitation of this meta-analysis is the heterogeneity in design and conduct of the primary studies. Key drivers of this heterogeneity were the complex array of treatment options, varying severity of diabetes, and varying outcome definitions. The studies that combined medications (e.g., “any other treatment”) as the reference group are of particular concern for this and future meta-analyses for two main reasons. First, “any other treatment” represents different treatments depending on the study period and population, which decreases the applicability and comparability of results. Second, results relative to such reference treatment may not be useful for clinical decision makers who need to choose between specific therapeutic alternatives. The results of this study show that methodological limitations present in the original studies have an important role in explaining the statistical heterogeneity found when combining the individual study results in the meta-analysis. When excluding those studies at high/unclear risk of bias for 30 % or more of the items assessed by the RTI-IB, heterogeneity was largely reduced in most of the comparisons analysed. Overall results in those comparisons were sensitive to the exclusion of the studies more prone to bias, which indicates lack of robustness in results when including such studies in the meta-analysis.

Few of the studies included accounted for severity of diabetes; therefore, confounding by indication could be present in the majority of within-study comparisons for studies in this meta-analysis. Residual confounding might be present in studies that failed to systematically record or adjust for lifestyle factors. Few studies adjusted for socioeconomic status, education, physical activity, or BMI, which can all be associated with both treatment selection and the development of outcomes. These and other methodological limitations of the majority of the studies included in this meta-analysis have been reviewed in recent publications, which support the overall qualitative findings of this study [50, 51]. Since the magnitude of the increased risks was rather small for most of the comparisons, small residual confounding, if present, could explain the small increases in the risk estimates.

Evaluation of the risk of stroke was limited due to the small number of published studies and inclusion of only users of thiazolidinediones. Other limitations of this meta-analysis had to do with the fact that dose and duration effects could not be evaluated for any comparison due to the scarcity of data in the published studies and the fact that the newest oral drugs for diabetes could not be evaluated due to lack of studies.

Conclusions

In conclusion, sulfonylureas seem to increase the risk of AMI by 24 % compared with metformin. With 3 small studies and a corresponding lack of precision, results of this meta-analysis suggest no difference in the risk of AMI for pioglitazone compared with metformin. The presence of heterogeneity in the meta-analysis results precludes any conclusions on the risk of AMI for the other comparisons evaluated. The quality assessment with the RTI-IB and the sensitivity analyses indicate that statistical heterogeneity might be attributable to the studies that were at the highest risk of bias according to the RTI-IB. Future studies should consider the methodological pitfalls identified in the existing body of evidence. Results from ongoing large multidatabase studies, carefully planned and conducted, are awaited and will help to elucidate the risk of AMI and stroke associated with the use of NIBGLDs.

Competing interests

RTI Health Solutions employees work on projects funded by pharmaceutical companies including manufacturers of treatments for patients with diabetes. As employees of RTI Health Solutions, Manel Pladevall, Susana Perez-Gutthann, and Cristina Varas-Lorenzo also participate in advisory boards funded by pharmaceutical companies.The authors declare that they have no competing interests

Authors' contribution

MP, NRG, CVL, AM, and SPG participated in development of the literature search strategy; MP, NRG, and AM abstracted and compiled the data; NRG, BC, AM, MP, and CVL performed the analyses; all authors oversaw design of the study and facilitated interpretation of the findings. MP drafted the manuscript; all coauthors reviewed and revised it critically for important intellectual content and read and approved the final manuscript. All authors had full access to all of the data abstracted from published studies included in this systematic review and take responsibility for the integrity of summarising the data and the accuracy of the meta-analysis.All authors read and approved the final manuscript