Introduction

It is thought that antigen-based therapy might inhibit autoimmune diseases via active or passive tolerance, possibly with little adverse impact on the immune system itself [1]. GAD is a major target of the autoimmune response that occurs in type 1 diabetes mellitus. Although treatment with GAD in murine models of type 1 diabetes has provided encouraging results [2, 3], clinical randomised controlled trials (RCTs) in human participants have so far given conflicting results regarding efficacy (Table 1).

Table 1 GAD RCTs
Full size table

Based on a PubMed search of the literature, the earliest published Phase 2 or Phase 3 RCT of GAD vaccination was Ludvigsson et al (2008) [4], in which 70 patients with recent-onset type 1 diabetes were randomised (1:1) to receive either 20 μg GAD with alum (GAD) or a placebo of alum in a two-dose regimen at days 1 and 30 post-randomisation. This was followed in 2011 by a TrialNet study led by Wherrett et al [5] in which 145 participants were enrolled into a three-arm RCT (1:1:1) and given one of the following three treatments: three doses of GAD given at weeks 0, 4 and 12 after randomisation; two doses of GAD at baseline and week 4 and one dose of placebo at week 12; or three doses of placebo. Finally, another RCT by Ludvigsson et al appeared in 2012 [6], which enrolled 334 participants into three treatment groups that received: 20 μg GAD given in four doses (days 1, 30, 90 and 270); 20 μg GAD given in two doses (days 1 and 30) with placebo given on the other days; or placebo given on all dosing days for the placebo group.

The primary findings from these three studies were markedly different. In Ludvigsson (2008) [4], a statistically significant (p = 0.045) difference was found in the change in fasting C-peptide concentration at 30 months, with GAD-vaccinated individuals showing a smaller loss than the placebo group. In addition, a statistically significant reduction in C-peptide loss, measured by the change in stimulated C-peptide area over a 2 h test (AUC) was observed at 15 months (p = 0.01) and 30 months (p = 0.04). However, the primary endpoint, the 15-month change in fasting C-peptide concentration, failed to reach statistical significance (p = 0.28). Contrary to these findings, both Wherrett (2011) and Ludvigsson (2012) reported non-significant findings for their primary endpoint – the change in stimulated C-peptide AUC at 12 and 15 months, respectively.

Further consideration of the studies (Table 1) reveals differences in their design and analysis. Both Ludvigsson studies [4, 6] had a restricted age range (10–18 years) compared with Wherrett’s study, which enrolled participants as young as 3 and as old as 45 years. Ludvigsson (2008) [4] enrolled participants up 18 months after diagnosis, whereas the other two studies enrolled individuals within 3 months. Finally, not only did the primary endpoints differ (a 15 month change in fasting C-peptide level in Ludvigsson [2008], a 12 month change in stimulated 2 h C-peptide AUC in Wherrett [2011] and a 15 month AUC change in Ludvigsson [2012]), but the analyses were also heterogeneous. In Ludvigsson (2008), the arithmetic difference of C-peptide (both fasting and AUC) at 15 months and baseline were compared after adjusting for baseline C-peptide; in Wherrett (2011), the geometric mean of the AUC + 1 (i.e. the number 1 was added to the AUC value) was compared after adjusting for the baseline natural logarithm of AUC + 1, sex and age; and in Ludvigsson (2012), the natural logarithm of the proportion change from baseline was compared after adjusting for baseline log AUC and country of the clinical site.

In this study, we conducted a meta-analysis using individual-level data from these clinical RCTs in order to determine the probability that GAD has any positive biological effect, defining a ‘positive biological effect’ as a reduction of C-peptide loss 12–15 months after diagnosis of type 1 diabetes compared with placebo. This ‘posterior probability of positive effect’ was estimated using Bayesian methods [7]. Our goal in conducting this meta-analysis was to see whether a clearer answer to the question of whether GAD65 has an effect on C-peptide could be reached by combining the data from the RCTs.

Methods

As described above, we used individual-level data from three major GAD trials (Table 1) which, as of 1 June 2015, represented the entirety of published clinical RCTs of GAD in relation to a reduction in the loss of endogenous insulin production in patients with recent-onset type 1 diabetes. Two of the studies [4, 6] were sponsored by Diamyd, the company producing the vaccine. The third trial was conducted by TrialNet independently of Diamyd, with the same study drug donated by the company.

We used Bayes methods to compute ‘posterior probabilities’ of biological effect (reduction of insulin secretion loss compared with placebo) using data combined from the three studies. As described below, the adjective ‘posterior’ is used to denote that this probability is computed after the data have been observed.

The Bayes method is similar to computation of the predictive value for a diagnostic test, which can itself be thought of as a ‘posterior probability‘. For example, in diagnostic testing, one begins with an initial (i.e. prior to diagnostic testing) assessment of the probability that the patient has a disease (e.g. disease prevalence) and then updates that assessment using the data from a test to yield an improved probability assessment of disease (the ‘posterior probability’, which is synonymous with predictive value). Bayes methods simply supply the calculus needed to compute the posterior probability of a treatment or biological effect using the data from a scientific study. Further information about Bayes methods in clinical research can be found elsewhere [8–10] . Although Bayesian approaches to the design of clinical trials have been used in type 2 diabetes [11], as of the time of our review they had not appeared in the literature on type 1 diabetes. However, Bayesian methods in type 1 diabetes outside of RCTs can be found: for example, Irie et al [12] employed a Bayesian method for genome-wide gene expression data of CD4+ T cells; a Bayesian method for normalisation of quantitative PCR data in profiling urinary microRNA in the nephropathy of type 1 diabetes by Argyropoulos et al [13, 14] has been published; and as recently as 2015, Bayesian methods were applied to improve glucose sensor models [15].

To make the interpretation of ‘biological effect’ similar across the studies, our endpoint was the natural logarithm of 2 h stimulated C-peptide AUC after 1 year expressed as a proportion of baseline. Interpretation of the posterior probability of a biological effect (PPBE) was then similar for each study as the comparison was the difference in mean log e (proportion of baseline) in the GAD-treated group minus the mean for the placebo group. As a positive treatment effect means a reduction in loss, positive values for this difference meant that GAD treatment had a positive biological effect. Thus, the probability that the effect of treatment with GAD had a positive biological effect equals the posterior probability that the coefficient of GAD treatment is greater than zero when the placebo group is considered as the referent.

Our meta-analysis combined individual-level data from the three studies related to the endpoint as well as age, sex and log baseline C-peptide. The availability of other data was variable among the studies so was not included in forming our meta-analysis database.

As a ‘prior’ distribution for the biological effect of GAD, we assumed equipoise, that is, we assumed that a negative biological effect was as likely as a positive biological effect. We then used the combined data to ‘update’ the equipoise prior for the combined data. In addition, we applied the Bayes method to each of the same statistical models used in each of the three studies and conducted a sensitivity analysis by examining the effects of changing our assumptions on the prior.

The model fit for meta-analysis was: change in log (C-peptide AUC) = Intercept + Study + Baseline log (C-peptide AUC) + Age + Sex + Treatment, where ‘log’ refers to the natural logarithm. This was a generalised linear model with normally distributed errors and statistical estimation accomplished using the Bayes statement in the SAS V9.4 (SAS Institute, Cary, NC, USA) GENMOD procedure. The model included the source of the data (the study) as a fixed classificatory effect in order to examine possible inherent differences between the studies that could not be accounted for by the variables included in the meta-analysis database. Age (years) was included as a continuous variable. Sex (male/female) had ‘male’ as the referent category. Treatment was included as a classification variable with 0, 2, 3 and 4 doses of GAD. Conjugate normal (0.1 × 106) priors were assumed for all regression coefficients in the generalised model with identity link function and normal distribution of errors. An improper prior was assumed for the dispersion parameter of the errors. Maximum likelihood estimates were used as the initial values of the chain. The Bayes estimation of posterior distributions used Gibbs sampling with a burn-in size of 2000 followed by 10,000 sets of sampled parameter estimates from the posterior. Evaluation of convergence and model run were assessed with Geweke diagnostics and plots of parameter values across the 10,000 runs and autocorrelation plots. These results are available from the corresponding author. Posterior estimates were output and graphed using GraphPad Prism V.6.00 for Windows (GraphPad Software, La Jolla, CA, USA; www.graphpad.com). For sensitivity analysis, we considered Jeffery’s and uniform distributions as alternatives for the priors.

Results

Histograms of the results from Bayes analysis of the effect of two-dose GAD treatment relative to placebo are displayed in Fig. 1a–c for each study individually and in Fig. 1d for the studies combined. Histogram bars are centred at 5% units representing a 5% increase (if positive) or decrease (if negative) in C-peptide change in treated participants compared with placebo. The width of each bar represents a range of biological effect (change) that was ±2.5% of the bar centre. The height of each bar corresponds to the probability of that range of effect. The sum of the areas of the bars above zero gives the PPBE.

Fig. 1
figure 1

PPBE from two-dose GAD vaccine. Histograms of the results from Bayes analysis of the effect of two-dose GAD treatment relative to placebo. Histogram bars are centred at 5% units representing a 5% increase (if positive) or decrease (if negative) in C-peptide change in treated individuals compared with placebo. The width of each bar represents a range of biological effect (change) of ±2.5% of the bar centre. The height of each bar corresponds to the probability of that range of effect. The sum of the areas of the bars above zero gives the PPBE: (a) Ludvigsson (2008) [4], 99.8%; (b) Wherrett (2011) [5], 32.1%; (c) Ludvigsson (2012) [6], 96.6%; (d) meta-analysis, 98.0%

Full size image

Combining the studies, our meta-analysis estimates a 98% probability that two-dose GAD has a positive biological effect (Fig. 1d). The most likely (modal) values are between 15% and 20%. The meta-analysis also yields high probabilities for the biological effect of three-dose (85.71%) and four-dose (95.59%) GAD treatment regimens (Table 2).

Table 2 Posterior PPBE
Full size table

Examination of the posterior probability histograms from each of the three studies (Fig. 1a–c) highlights the heterogeneity between the studies. While each of the Ludvigsson studies [4, 6] provided a high PPBE for the two-dose GAD treatment, the Wherrett study provided a much lower PPBE. We used post hoc methods to identify potential reasons for the differences between the studies.

Table 3 reports the results from statistical comparisons between the studies of the covariates employed in the models. The patients in Wherrett (2011) [5] were significantly (p < 0.05) older, and the baseline log AUC was higher in Ludvigsson (2008) [4] than in the other studies. Some literature suggests that older individuals with type 1 diabetes have reduced C-peptide loss compared with younger individuals, which may lead to the smaller apparent effects of GAD that were seen in Wherrett (2011). On the other hand, subgroups reported in Wherrett (2011) did not show a statistically significant difference in C-peptide change when comparing younger with older participants. There were also some data to suggest that higher baseline values of C-peptide AUC are associated with increased loss due to a ‘basement’ restriction, so we would expect more of a loss in Ludvigsson (2008), as was indeed observed. Of course, the inclusion of these covariates in the analysis of the combined datasets should control for these differences when computing posterior probabilities of biological effect, so we felt that other differences needed to be considered. However, we were prevented from doing this because no other variables were consistently available in each of the datasets provided to us.

Table 3 Statistical comparison of covariates between studies
Full size table

We were nonetheless able to conduct a post hoc comparison of the percentage of ‘placebo responders’ between the studies. A placebo responder is defined here as a participant treated with placebo who showed a maintained or increased C-peptide AUC from baseline. We decided to compare this variable as it measures, to some extent, a potential bias arising from the mechanisms of patient selection between the studies.

The results, presented in Table 4, suggest that the Wherrett (2011) study [5] had more than double the rate of placebo responders found in the other two studies and thus suggests that the patients were overall more likely to retain or have reduced C-peptide loss overall. This possible explanation for the difference in treatment effect between the studies is further supported if we limit the Wherrett (2011) study to the same age range (i.e. 10–18 years), which gives an increased placebo responder rate of 16.7% compared with 6.1% and 5.6% in Ludvigsson (2008) [4] and Ludvigsson (2012) [6], respectively. Additional evidence of the potential for clinically different study patients is presented in Fig. 2, which compares the median (95% CI) percentage C-peptide loss from baseline in the placebo-treated participants from each of the three studies after adjusting for baseline AUC and age. Although no statistically significant differences were found, placebo-treated individuals in Wherrett (2011) empirically had a higher median, suggesting the possibility of greater ‘natural’ C-peptide retention by this set of patients than in either of the other two studies.

Table 4 Comparison of placebo responders
Full size table
Fig. 2
figure 2

Median C-peptide loss in individuals in the placebo groups. Median (95% CI) of the C-peptide AUC change at approximately 1 year from baseline. The change is expressed as the percentage loss from baseline. Estimates have been adjusted for age and baseline differences between the three studies. DW2011, Wherrett (2011) [5]; JL2008, Ludvigsson (2008) [4]; JL2012, Ludvigsson (2012) [6]

Full size image

Tables 5 and 6 report the results of sensitivity analyses. Table 5 reports PPBE estimates using the endpoints and models from the source publications, and shows that our findings are robust against the methodological heterogeneity of these studies. Table 6 shows that altering the form assumed for the prior distributions had a minimal effect on PPBE estimates and did not alter the conclusions of our study.

Table 5 Summary of results from sensitivity analyses: posterior PPBE using models from each source publication (i.e. differing outcome and explanatory variables)
Full size table
Table 6 Summary of results from sensitivity analyses: posterior PPBE and changing prior distribution
Full size table

Discussion

In this study, we sought to answer the question ‘Should development of the GAD vaccine be continued or discontinued?’ The currently published RCT literature provides conflicting information and the studies are not by themselves sufficient to answer our question. In addition, from the decision-maker’s position, an answer to the question ultimately requires an assessment of the probability of whether or not the GAD vaccine is biologically effective. For this, we used Bayesian methods to determine the posterior probability of biological effect. Such a strategy has been advocated as an aid to the interpretation of RCTs [7]. We used a natural starting point for this prior assessment, equipoise, in which we assigned a 50% chance of a positive biological effect. We then used data to update the prior using Bayesian methods. From this resultant ‘posterior’ distribution, we were then able to compute the PPBE, thus providing the information needed for answering our question.

Our analysis of the combined data estimates there to be a high (approximately 98%) probability that two-dose GAD vaccine has a positive biological effect, i.e. a reduction in C-peptide loss compared with placebo. The most likely (modal) value of the biological effect is a 15–20% reduction in loss relative to placebo. Greenbaum et al [16] estimated a C-peptide AUC loss of −0.0245 pmol/ml per month, which translates to an expected loss of −0.294 pmol/ml at 1 year for untreated newly diagnosed patients. We therefore estimate that, most probably, two doses of 20 μg GAD with alum GAD vaccine reduces C-peptide AUC loss by on average 0.044–0.059 pmol/ml after 1 year. That is, GAD vaccine is likely to reduce the 1 year loss to somewhere between −0.250 and −0.235 pmol/year. Similar levels of biological effect for three- and four-dose GAD were estimated. However, these two estimates were based on data coming only from single studies: three-dose treatment in Wherrett (2011) [5] and four-dose treatment in Ludvigsson (2012) [6]. We thus conclude that, most likely, the dosing regimens considered to date confer rather small and similar clinical benefits. On the other hand, a clinical benefit has to be put in relation to the feasibility and tolerability of the treatment. Thus, for GAD therapy, which is extremely easy for the patient, and has no known treatment-related adverse events, a clinical benefit of 15–20% may be as relevant and valuable as a 25–30% efficacy using a treatment modality that is more difficult to administer and/or has larger risks and more common and serious adverse events.

Of course, one must also consider the impact of GAD therapy on other endpoints (e.g. HbA1c concentration, insulin usage, glycaemic variation, hypoglycaemia, etc.) to fully support a clinical benefit for it. However, some experts argue that C-peptide retention is a clinically important endpoint in its own right, pointing to research suggesting that residual C-peptide confers clinical benefits including better long-term glycaemic control [17–19]. Nonetheless, our study was designed to answer the question of whether or not GAD therapy should be developed further, so we await optimisation of this vaccine before a robust appraisal of its full clinical benefit can be attempted.

Another limitation of our study is that our meta-analysis had to limit explanatory variables to those that were common to all studies. For example, disease duration was collected by each study but only analysed as a covariate in Ludvigsson (2008) [4], in which it was found to be not statistically significant. Future studies of GAD should consider a comprehensive and common ‘core’ set of covariates for consistency and to be used in meta-analyses.

The potential of a differential selection of participants needs to be investigated further and emphasises the need for improved selection methods in clinical trials of type 1 diabetes. To this end, biomarkers that are able to select for rapid C-peptide loss need to be developed and adopted in type 1 diabetes clinical trials.

We conclude that it is highly likely that GAD vaccine has a positive biological effect. However, this might be rather small under the dose and treatment regimens considered so far in published trials. In sum, our synthesis of published RCT evidence encourages continued investigation of the GAD vaccine, specifically into ways, for example giving it as part of combination therapies that will increase its effectiveness.