Advertisement

Japanese Journal of Statistics and Data Science

, Volume 2, Issue 2, pp 347–373 | Cite as

Bivariate beta-binomial model using Gaussian copula for bivariate meta-analysis of two binary outcomes with low incidence

  • Yusuke YamaguchiEmail author
  • Kazushi Maruo
Original Paper
  • 659 Downloads

Abstract

In meta-analysis of rare-event outcomes, an additional statistical consideration is necessary due to the occurrence of studies with no event. The traditional approaches of adding a correction factor or omitting these studies are known to result in misleading conclusions. Furthermore, studies involved in the meta-analysis often report results for more than one outcome. Bivariate meta-analysis is known as a promising approach for jointly combining two outcomes whilst incorporating correlations between outcomes. However, there has not been sufficient discussion on a bivariate extension in the context of meta-analysis for rare-event outcomes. We consider a joint synthesis of two binary outcomes with low incidence, and propose a novel bivariate meta-analysis method using copula. The method assumes marginal beta-binomial distributions for the two outcomes, and links these margins by a bivariate copula which identifies an overall dependence structure between outcomes. A simulation study suggested that the method could provide a robust estimation for the incidence of rare events and have potential benefits of bivariate meta-analysis such as an improvement of precision of pooled estimates. We illustrated the method through an application to a meta-analysis of 48 studies that evaluated a potential risk of rosiglitazone on myocardial infection and cardiovascular death.

Keywords

Bivariate meta-analysis Beta-binominal model Gaussian copula Binary outcome Rare event 

1 Introduction

A strength of meta-analysis is its capability of accumulating evidence from several studies to detect benefits or risks of a therapeutic treatment that are difficult to be assessed in a single study alone. In particular, if interest lies in safety assessments of a treatment regarding serious adverse events with very low incidence (e.g., less than 1%), the meta-analysis enables the estimation and quantification of the risk of such rare adverse events in situations where individual studies do not have large enough sample sizes for reliable evaluations (Loke et al. 2011; Berlin et al. 2013; Shuster and Walker 2016). Bennetts et al. (2017) summarized several publications and guidelines on the safety meta-analytic approaches, and advocated that the meta-analysis of safety data is now becoming a requirement in more drug development programs and beyond.

In the meta-analysis of rare-event outcomes, an additional statistical consideration is necessary due to the occurrence of studies with no event. It is not unusual for practitioners to face a situation that some studies involved in the meta-analysis have zero events in one or more treatment groups. The traditional approaches of adding a correction factor or omitting these studies are known to result in misleading conclusions (Sweeting et al. 2004; Bradburn et al. 2007; Friedrich et al. 2007; Cheng et al. 2015). For the handling of studies with no event, a variety of alternative approaches have been discussed broadly, including arcsine difference (Rucker et al. 2009), Poisson–gamma model (Cai et al. 2010), hypergeometric–normal model (Stijnen et al. 2010), bivariate binomial–normal model (Chu et al. 2012; Stijnen et al. 2010), bivariate beta-binomial model (Chu et al. 2012), unweighted and study-size weighted methods (Shuster et al. 2012), beta-binomial regression model (Kuss 2015), and Poisson regression model (Bohning et al. 2015; Spittal et al. 2015). Among them, we focus on the beta-binomial model, since it has attractive features such as capabilities of including information from studies having no event without any corrections, leading to a closed-form expression of likelihood function and providing a simple interpretation of model parameters. An extensive simulation study by Kuss (2015) has also revealed that the beta-binomial model could outperform the other approaches in terms of estimation bias and accuracy for the treatment effects (odds ratio, relative risk, and risk difference) in the meta-analysis of rare-event outcomes.

Furthermore, studies involved in the meta-analysis often report results for more than one outcome. For example, Nissen and Wolski (2007) performed a meta-analysis of 48 studies to evaluate a potential risk of rosiglitazone on myocardial infection and cardiovascular death. Hammad et al. (2006) performed a meta-analysis of 24 studies to investigate a relationship between antidepressant drugs and suicidality in pediatric patients, where two different endpoints were specified for measuring suicidal behavior and ideation. Bivariate meta-analysis is known as a promising approach for jointly combining two outcomes whilst incorporating correlations between outcomes, which has potential benefits to improve the precision of pooled estimates and to reduce an impact of outcome reporting bias compared with separate univariate meta-analyses of each outcome (Riley et al. 2007, 2008, 2009; Jackson et al. 2011; Kirkham et al. 2012; Mavridis and Salanti 2013). However, there has not been sufficient discussion on a bivariate extension in the context of meta-analysis for rare-event outcomes. Indeed, most of the methods developed for performing meta-analysis of rare-event outcomes were limited to applications to the univariate outcome.

In this article, we consider a joint synthesis of two binary outcomes with low incidence, and propose a novel bivariate meta-analysis method using copula. The method assumes marginal beta-binomial distributions for the two outcomes, and links these margins by a bivariate copula which identifies an overall dependence structure between outcomes. This is fundamentally built on a framework of bivariate copulas for discrete data (Joe 2014). The use of copula allows for a flexible modeling of dependences, and several researchers have discussed the copula modeling in the context of meta-analysis of diagnostic test accuracy studies (Kuss et al. 2014; Hoyer and Kuss 2015; Nikoloulopoulos 2015a, b; Chen et al. 2016). They aimed at modeling the relationship between disease groups rather than the relationship between outcomes. Our contribution is to apply the copula to the joint synthesis of two binary rare-event outcomes.

In Sect. 2, we introduce a motivating example of rosiglitazone data, which will be used for illustrating the method. In Sect. 3, we describe the bivariate beta-binomial model using the copula. We begin with brief introductions to the beta-binomial distribution and the bivariate copula, and then provide detailed explanations of the proposed method. In Sect. 4, we perform a simulation study to examine a relative performance of the method in comparison to the existing approaches. We show that the method is capable of providing a robust estimation for the incidence of rare events and maintaining potential benefits of bivariate meta-analysis such as an improvement of precision of pooled estimates. In Sect. 5, we illustrate the method through an application to the rosiglitazone data. Finally, in Sect. 6, we conclude this article with some discussion.

2 Motivating example

Nissen and Wolski (2007) have performed a meta-analysis to evaluate a potential risk of rosiglitazone on myocardial infection (MI) and cardiovascular death (CVD) for type 2 diabetes patients. 48 studies satisfying their criteria (e.g., Phase II–IV, 24 weeks, or longer duration and randomized control groups) were involved in the analysis, where 10 studies had no MI events and 25 studies had no CVD events. Because the incidences of MI and CVD events were potentially very low and an unignorable number of studies had no events in control and/or rosiglitazone groups, several researchers have reanalysed the rosiglitazone data where a conflict between conclusions reached from different analysis methods has been controversial (Shuster et al. 2007, 2012; Tian et al. 2009; Shuster 2010; Chu et al. 2012; Lane 2011; Shuster and Walker 2016). These previous investigations have addressed the MI and the CVD events as univariate outcomes and performed meta-analyses of them separately; however, the two outcomes could be correlated through studies. For example, a patient with the MI could be more likely to have the CVD in comparison to a subject without the MI. In addition, due to the difference of patient population across studies, some studies could have higher true incidences of both MI and CVD than the others; for instance, when studies allowed enrolling more severe patients than the others, they might have higher potential risks in both of the two outcomes. Figure 1a-1 and a-2 shows the bubble plots of correlations between the incidences of the MI and the CVD events in the control and the rosiglitazone group, respectively. Figure 1b also shows a bubble plot of correlation between the log odds ratios of the MI and the CVD events. When calculating the incidences and the log odds ratio, continuity correction factors of 0.5 were added to studies with zero events. Each circle represents a single study and the circle size is proportional to the sample size in each group or study. The three bubble plots indicate a positive relationship between the MI and the CVD events. In Sect. 5, we will consider a joint synthesis of the MI and the CVD events whilst incorporating correlations between the two outcomes. The rosiglitazone data will be used to illustrate and critically assess the method developed; those interested in more clinical conclusions are referred elsewhere (Nissen and Wolski 2007, 2010).
Fig. 1

Correlation between: (a-1) incidences of MI and CVD events in the control group, (a-2) incidences of MI and CVD events in the rosiglitazone group, and (b) log odds ratios of MI and CVD events

3 Methods

Consider a joint synthesis of two binary outcomes by performing meta-analysis of N studies with two groups (control and treatment group). Let nij denote the number of subjects in the ith study (i = 1, …, N) and the group j which takes a value of j = 0 for the control group and j = 1 for the treatment group. Let yijk denote the number of events for the outcome k (k = 1, 2) in the ith study and group j. Then, yijk is assumed to follow a binomial distribution with an event probability pijk, i.e., yijk ~ Binomial(nijpijk) that assumes different true event probabilities underlying different studies. This assumption reflects a fact that the true event probability might be similar across studies, but is unlikely to be identical because of difference in the characteristics of included subjects and the study design, among other reasons. We are now interested in the situation of rare-event outcomes; that is, the pijk can take very small values such as less than 0.01.

3.1 Beta-binomial distribution

Assume that the event probability pijk follows a beta distribution with parameters αjk and βjk, i.e., pijk ~ Beta(αjkβjk). This is genuinely a form of random-effects meta-analysis models that allow for unexplained heterogeneity across studies. Letting fbin and fbeta represent probability mass functions (pmf) of the binomial distribution and a probability density function of the beta distribution, respectively, a pmf of the beta-binomial distribution, fBB, is given by the following:
$$ \begin{aligned} & f_{\text{BB}} \left( {y_{ijk} ,n_{ij} ;\alpha_{jk} ,\beta_{jk} } \right) \\ & = \int {f_{\text{bin}} } \left( {y_{ijk} ,n_{ij} ;p_{ijk} } \right)f_{\text{beta}} \left( {p_{ijk} ;\alpha_{jk} ,\beta_{jk} } \right){\text{d}}p_{ijk} \\ & = \left( {\begin{array}{*{20}c} {y_{ijk} } \\ {n_{ij} } \\ \end{array} } \right)\frac{{B\left( {y_{ijk} + \alpha_{jk} ,n_{ij} - y_{ijk} + \beta_{jk} } \right)}}{{B\left( {\alpha_{jk} ,\beta_{jk} } \right)}}, \\ \end{aligned} $$
(1)
for i = 1, …, N, j = 0, 1 and k = 1, 2, where αjk > 0, βjk > 0 and B(·) is a beta function. Then, its cumulative distribution function (cdf), FBB, is given by the following:
$$ F_{\text{BB}} \left( {y_{ijk} ,n_{ij} ;\alpha_{jk} ,\beta_{jk} } \right) = \mathop \sum \limits_{y = 0}^{{y_{ijk} }} f_{\text{BB}} \left( {y,n_{ij} ;\alpha_{jk} ,\beta_{jk} } \right), $$
(2)
for i = 1, …, N, j = 0, 1 and k = 1, 2. Note that the pmf of the beta-binomial distribution (1) is expressed as a closed form, which leads to an advantage of avoiding a numerical approximation of integrals needed in the other random-effects modeling such as bivariate generalized linear mixed effects models (Chen et al. 2016). In addition, the beta-binomial distribution is capable of including the information from studies having zero events in both groups, without using any continuity corrections.

3.2 Bivariate copula

The bivariate copula is defined as a bivariate distribution function with uniform margins on [0, 1]. Letting F12 denotes a joint cdf of random variables x1 and x2, Sklar’s theorem (Sklar 1973) implies that there is a bivariate copula C, such that
$$ F_{12} (x_{1} ,x_{2} ) = C(F_{1} (x_{1} ),F_{2} (x_{2} )), $$
(3)
where F1 and F2 are cdfs of the marginal distributions for x1 and x2, respectively. When x1 and x2 are discrete variables, the bivariate copula C is unique on the set of Range(F1) × Range(F2) but not elsewhere (Joe 2014). Then, letting u1 = F1(x1) and u2 = F2(x2), Eq. (3) can be written as follows:
$$ C(u_{1} ,u_{2} ) = F_{12} (F_{1}^{ - 1} (u_{1} ),F_{2}^{ - 1} (u_{2} )), $$
for 0 ≤ u1 ≤ 1 and 0 ≤ u2 ≤ 1. An advantage of copulas in the dependence modeling is to allow the dependence structure to be considered separately from the univariate margins.

A variety of copula families can be used to identify the dependence between variables (Joe 2014). We here focus on a typical bivariate copula family of Gaussian copula. The Gaussian copula interpolates from the Fréchet lower (perfect negative dependence) to the Fréchet upper (perfect positive dependence) bound, and thus, this allows for both situations where positive and negative dependences between the two outcomes are expected (Nikoloulopoulos 2015a).

Bivariate Gaussian copula: The cdf is defined by the following:
$$ C\left( {u_{1} ,u_{2} ;\theta } \right) = \Phi_{2} \left( {\Phi^{ - 1} \left( {u_{1} } \right),\Phi^{ - 1} \left( {u_{2} } \right);\theta } \right), $$
for − 1 < θ < 1, where Φ2 is the cdf of bivariate normal distribution of which the two marginals have mean 0, variance 1, and correlation θ, and Φ−1 is the inverse cdf of standard normal distribution. The dependences between variables are positive for 0 < θ < 1, negative for − 1 < θ < 0, and independent for θ = 0. Song (2000) showed that the Gaussian copula parameter θ was equal to the Pearson correlation of the two normal scores Φ−1(F1(x1)) and Φ−1(F2(x2)), measuring the dependence between x1 and x2 based on a monotone non-linear transformation. When x1 and x2 are continuous variables, θ is very close to the Spearman correlation of the original margins. This is not the case when x1 and x2 are discrete variables, while some supportive evidence for the extension to the discrete margins can be found in Song (2000) and Song et al. (2009).

3.3 Bivariate beta-binomial model using copula and its estimation

The joint pmf of two discrete variables comes from rectangle probability (Joe 2014). Given the cdfs of the marginal beta-binomial distributions (2) for each outcome and the specified bivariate copula, we have a joint pmf of yij1 and yij2 as follows:
$$ f_{{{\text{JBB}}}} \left( {y_{{ij1}} ,y_{{ij2}} ,n_{{ij}} ;\alpha _{{j1}} ,\beta _{{j1}} ,\alpha _{{j2}} ,\beta _{{j2}} ,\theta _{j} } \right) = \left\{ {\begin{array}{*{20}l} {C\left( {F_{{{\text{BB}}}} \left( {y_{{ij1}} ,n_{{ij}} ;\alpha _{{j1}} ,\beta _{{j1}} } \right),F_{{{\text{BB}}}} \left( {y_{{ij2}} ,n_{{ij}} ;\alpha _{{j2}} ,\beta _{{j2}} } \right);\theta _{j} } \right),} \hfill & {{\text{for}}\quad y_{{ij1}} = 0,y_{{ij2}} = 0} \hfill \\\\ \begin{gathered} C\left( {F_{{{\text{BB}}}} \left( {y_{{ij1}} ,n_{{ij}} ;\alpha _{{j1}} ,\beta _{{j1}} } \right),F_{{{\text{BB}}}} \left( {y_{{ij2}} ,n_{{ij}} ;\alpha _{{j2}} ,\beta _{{j2}} } \right);\theta _{j} } \right) \hfill \\ - C\left( {F_{{{\text{BB}}}} \left( {y_{{ij1}} - 1,n_{{ij}} ;\alpha _{{j1}} ,\beta _{{j1}} } \right),F_{{{\text{BB}}}} \left( {y_{{ij2}} ,n_{{ij}} ;\alpha _{{j2}} ,\beta _{{j2}} } \right);\theta _{j} } \right), \hfill \\ \end{gathered} \hfill & {{\text{for}}\quad y_{{ij1}} >0,y_{{ij2}} = 0} \hfill \\ \\ \begin{gathered} C\left( {F_{{{\text{BB}}}} \left( {y_{{ij1}} ,n_{{ij}} ;\alpha _{{j1}} ,\beta _{{j1}} } \right),F_{{{\text{BB}}}} \left( {y_{{ij2}} ,n_{{ij}} ;\alpha _{{j2}} ,\beta _{{j2}} } \right);\theta _{j} } \right) \hfill \\ - C\left( {F_{{{\text{BB}}}} \left( {y_{{ij1}} ,n_{{ij}} ;\alpha _{{j1}} ,\beta _{{j1}} } \right),F_{{{\text{BB}}}} \left( {y_{{ij2}} - 1,n_{{ij}} ;\alpha _{{j2}} ,\beta _{{j2}} } \right);\theta _{j} } \right), \hfill \\ \end{gathered} \hfill & {{\text{for}}\quad y_{{ij1}} = 0,y_{{ij2}} > 0} \hfill \\ \\ \begin{gathered} C\left( {F_{{{\text{BB}}}} \left( {y_{{ij1}} ,n_{{ij}} ;\alpha _{{j1}} ,\beta _{{j1}} } \right),F_{{{\text{BB}}}} \left( {y_{{ij2}} ,n_{{ij}} ;\alpha _{{j2}} ,\beta _{{j2}} } \right);\theta _{j} } \right) \hfill \\ - C\left( {F_{{{\text{BB}}}} \left( {y_{{ij1}} - 1,n_{{ij}} ;\alpha _{{j1}} ,\beta _{{j1}} } \right),F_{{{\text{BB}}}} \left( {y_{{ij2}} ,n_{{ij}} ;\alpha _{{j2}} ,\beta _{{j2}} } \right);\theta _{j} } \right) \hfill \\ - C\left( {F_{{{\text{BB}}}} \left( {y_{{ij1}} ,n_{{ij}} ;\alpha _{{j1}} ,\beta _{{j1}} } \right),F_{{{\text{BB}}}} \left( {y_{{ij2}} - 1,n_{{ij}} ;\alpha _{{j2}} ,\beta _{{j2}} } \right);\theta _{j} } \right) \hfill \\ + C\left( {F_{{{\text{BB}}}} \left( {y_{{ij1}} - 1,n_{{ij}} ;\alpha _{{j1}} ,\beta _{{j1}} } \right),F_{{{\text{BB}}}} \left( {y_{{ij2}} - 1,n_{{ij}} ;\alpha _{{j2}} ,\beta _{{j2}} } \right);\theta _{j} } \right), \hfill \end{gathered} & {{\text{for}}\quad y_{{ij1}} > 0,y_{{ij2}} > 0} \hfill \\ \end{array} } \right. \\ $$
for i = 1, …, N and j = 0, 1. Since the beta-binomial distribution is expressed by the closed form, the copula functions are also straightforward enough to be computed without any asymptotic assumptions and numerical approximations. Meta-analytic approaches using the copulas have been considered in the context of meta-analysis of diagnostic test accuracy studies. Kuss (2015) handled the observed variables that have the beta-binomial distributions as being continuous, and constructed a random-effects model under the theory for copula models with continuous margins. Nikoloulopoulos (2015a) has proposed a comprehensive framework of copula mixed models where the copula was used for identifying the correlation between the beta random-effects variables. The method developed here is differentiated from their approaches in the sense of: (a) using the theory for copula models with discrete margins directly; (b) identifying the correlations between outcomes rather than the correlations between groups (disease group in the case of diagnostic test accuracy studies).
Once we have the joint pmf of yij1 and yij2, a log likelihood function is simply derived by the following:
$$ l\left( {\alpha_{01} ,\beta_{01} ,\alpha_{02} ,\beta_{02} ,\theta_{0} ,\alpha_{11} ,\beta_{11} ,\alpha_{12} ,\beta_{12} ,\theta_{1} } \right) = \mathop \sum \limits_{i = 1}^{N} \mathop \sum \limits_{j = 0}^{1} { \log }\left( {f_{\text{JBB}} \left( {y_{ij1} ,y_{ij2} ,n_{ij} ;\alpha_{j1} ,\beta_{j1} ,\alpha_{j2} ,\beta_{j2} ,\theta_{j} } \right)} \right). $$
(4)

For convenience of estimation and interpretation, we consider the parameterisations of μjk = αjk/(αjk + βjk) and γjk = 1/(αjk + βjk + 1). The μjk is mean of the beta random-effects distribution of Beta(αjkβjk), and is further parametrized by logit(μ0k) = φk and logit(μ1k) = φk + δk. The δk is a parameter representing a log odds ratio between groups for the outcome k, which can be interpreted as an overall (summary) treatment effect. The γjk represents the variability (heterogeneity) across studies in the group j for the outcome k. The correlation between outcomes is identified by the bivariate copula and its parameter θj, and thus, the model allows the two outcomes to be correlated by groups. Then, all the parameters to be estimated consist of (φ1δ1γ01γ11φ2δ2γ02γ12θ0θ1). Maximization of the log likelihood function (4) is performed by numerical optimisation methods; in particular, we use a quasi-Newton method with BFGS (Broyden–Fletcher–Goldfarb–Shanno) algorithm for obtaining maximum-likelihood estimates \( (\hat{\varphi }_{1} ,\hat{\delta }_{1} ,\hat{\gamma }_{01} ,\hat{\gamma }_{11} ,\hat{\varphi }_{2} ,\hat{\delta }_{2} ,\hat{\gamma }_{02} ,\hat{\gamma }_{12} ,\hat{\theta }_{0} ,\hat{\theta }_{1} ) \) and their variance estimates.

Furthermore, using delta method, event probabilities for the outcome k can be estimated by the following:
$$ \begin{aligned} E\left( {p_{i0k} } \right) \approx &\, \hat{\mu }_{0k} = \frac{1}{{1 + \exp \left( { - \hat{\varphi }_{k} } \right)}}, \\ V\left( {p_{i0k} } \right) \approx & \left\{ {\frac{{\exp \left( { - \hat{\varphi }_{k} } \right)}}{{\left( {1 + \exp \left( { - \hat{\varphi }_{k} } \right)} \right)^{2} }}} \right\}^{2} V\left( {\hat{\varphi }_{k} } \right), \\ \end{aligned} $$
and
$$ \begin{aligned} E\left( {p_{i1k} } \right) \approx &\, \hat{\mu }_{1k} = \frac{1}{{1 + { \exp }( - \hat{\varphi }_{k} - \hat{\delta }_{k} )}},\\ V\left( {p_{i1k} } \right) \approx &\, \left\{ {\frac{{\exp \left( { - \hat{\varphi }_{k} - \hat{\delta }_{k} } \right)}}{{\left( {1 + \exp \left( { - \hat{\varphi }_{k} - \hat{\delta }_{k} } \right)} \right)^{2} }}} \right\}^{2} (V\left( {\hat{\varphi }_{k} } \right) + V(\hat{\delta}_{k} ) + 2Cov(\hat{\varphi }_{k} ,\hat{\delta }_{k} )), \\ \end{aligned} $$
for the control and the treatment group, respectively.
A situation that some studies report only one outcome is likely to be more common. The method is straightforward to be applied to this situation. Letting Λ12, Λ1, and Λ2 denote a set of studies reporting both outcomes, only the outcome 1 and only the outcome 2, respectively, the log likelihood function (4) can be replaced by the following:
$$ \begin{aligned} & l\left( {\alpha_{01} ,\beta_{01} ,\alpha_{02} ,\beta_{02} ,\theta_{0} ,\alpha_{11} ,\beta_{11} ,\alpha_{12} ,\beta_{12} ,\theta_{1} } \right) \\ &\quad = \mathop \sum \limits_{{i \in \varLambda_{12} }} \mathop \sum \limits_{j = 0}^{1} \log \left( {f_{\text{JBB}} \left( {y_{ij1} ,y_{ij2} ,n_{ij} ;\alpha_{j1} ,\beta_{j1} ,\alpha_{j2} ,\beta_{j2} ,\theta_{j} } \right)} \right) \\ &\qquad + \mathop \sum \limits_{{i \in \varLambda_{1} }} \mathop \sum \limits_{j = 0}^{1} \log \left( {f_{\text{BB}} \left( {y_{ij1} ,n_{ij} ;\alpha_{j1} ,\beta_{j1} } \right)} \right) + \mathop \sum \limits_{{i \in \varLambda_{2} }} \mathop \sum \limits_{j = 0}^{1} \log \left( {f_{\text{BB}} \left( {y_{ij2} ,n_{ij} ;\alpha_{j2} ,\beta_{j2} } \right)} \right), \\ \end{aligned} $$
for i = 1, …, N and j = 0, 1, where |Λ12 + Λ1 + Λ2| = N.

4 Simulation study

We conducted a simulation study to examine a relative performance of the method in comparison to the existing approaches under a variety of meta-analytic situations. Suppose that, in each study involved in a meta-analysis, subjects randomly assigned to a control or a treatment group were assessed for two binary outcomes whose potential incidences were assumed to be very low. Since the method allows for the joint synthesis of the two binary outcomes, we considered situations where some studies reported only one outcome.

4.1 Designs

An overview of the simulation study is as follows:
  1. Step 1.
    Choose a scenario of the following parameters:
    • Number of studies reporting both outcomes

    • Event probabilities in each group and their odds ratio

    • Correlation between outcomes.

     
  2. Step 2.

    Generate a set of meta-analysis data under the scenario given in Step 1.

     
  3. Step 3.

    Perform meta-analyses using five methods.

     
  4. Step 4.

    Repeat Step 2–3 5000 times, and summarize results obtained in Step 3.

     

We below describe each step in detail.

Step 1–2: Generating meta-analysis data sets

We used the bivariate beta-binomial model as a true model for generating meta-analysis data, where the correlation between outcomes was identified using the bivariate Gaussian copula; that is, for each study in a meta-analysis, we drew samples from
$$ (y_{ij1} ,y_{ij2} ) \sim f_{\text{JBB}} \left( {Y_{1} ,Y_{2} ,n_{ij} ;\alpha_{j1} ,\beta_{j1} ,\alpha_{j2} ,\beta_{j2} ,\theta_{j} } \right), $$
(5)
for i = 1, …, N and j = 0, 1. Parameters included in the true model (5) were set using the parameterisations of αjk = μjk(1 − γjk)/γjk and βjk = (1 − μjk)(1 − γjk)/γjk. As described in Sect. 3.3, μ0k and μ1k represent the overall event probabilities in the control and the treatment group for the outcome k, respectively. Also, γ0k and γ1k represent the variability across studies in the control and the treatment group for the outcome k, respectively. Here, a wide range of scenarios for true parameter values were considered to cover realistic situations, which are summarized in Table 1.
Table 1

Scenario of true parameter values in the simulation study

Parameter

No. of scenarios

Value

Number of studies reporting both outcomes

S1.1

N = 48, |Λ12| = 48, |Λ1| = 0, |Λ2| = 0

 

S1.2

N = 48, |Λ12| = 32, |Λ1| = 16, |Λ2| = 0

 

S1.3

N = 48, |Λ12| = 24, |Λ1| = 24, |Λ2| = 0

 

S1.4

N = 48, |Λ12| = 16, |Λ1| = 32, |Λ2| = 0

Number of subjects per group in a single study

\( n_{i0} \sim {\text{LN}}({\text{meanlog}} = { \log }(300),{\text{sdlog}} = 0.5) \), ni1 = ni0

Event probabilities in each group and their odds ratio

S2.1

Outcome 1: \( \left( {\mu_{01} ,\mu_{11} ,OR_{1} } \right) = (0.003, 0.003, 1.0) \)

Outcome 2: \( \left( {\mu_{02} ,\mu_{12} ,OR_{2} } \right) = (0.003, 0.003, 1.0) \)

 

S2.2

Outcome 1: \( \left( {\mu_{01} ,\mu_{11} ,OR_{1} } \right) = (0.003, 0.003, 1.0) \)

Outcome 2: \( \left( {\mu_{02} ,\mu_{12} ,OR_{2} } \right) = (0.003, 0.005, 1.8) \)

 

S2.3

Outcome 1: \( \left( {\mu_{01} ,\mu_{11} ,OR_{1} } \right) = (0.003, 0.005, 1.8) \)

Outcome 2: \( \left( {\mu_{02} ,\mu_{12} ,OR_{2} } \right) = (0.003, 0.003, 1.0) \)

 

S2.4

Outcome 1: \( \left( {\mu_{01} ,\mu_{11} ,OR_{1} } \right) = (0.003, 0.005, 1.8) \)

Outcome 2: \( \left( {\mu_{02} ,\mu_{12} ,OR_{2} } \right) = (0.003, 0.005, 1.8) \)

 

S2.5

Outcome 1: \( \left( {\mu_{01} ,\mu_{11} ,OR_{1} } \right) = (0.006, 0.006, 1.0) \)

Outcome 2: \( \left( {\mu_{02} ,\mu_{12} ,OR_{2} } \right) = (0.006, 0.006, 1.0) \)

 

S2.6

Outcome 1: \( \left( {\mu_{01} ,\mu_{11} ,OR_{1} } \right) = (0.006, 0.006, 1.0) \)

Outcome 2: \( \left( {\mu_{02} ,\mu_{12} ,OR_{2} } \right) = (0.006, 0.010, 1.8) \)

 

S2.7

Outcome 1: \( \left( {\mu_{01} ,\mu_{11} ,OR_{1} } \right) = (0.006, 0.010, 1.8) \)

Outcome 2: \( \left( {\mu_{02} ,\mu_{12} ,OR_{2} } \right) = (0.006, 0.006, 1.0) \)

 

S2.8

Outcome 1: \( \left( {\mu_{01} ,\mu_{11} ,OR_{1} } \right) = (0.006, 0.010, 1.8) \)

Outcome 2: \( \left( {\mu_{02} ,\mu_{12} ,OR_{2} } \right) = (0.006, 0.010, 1.8) \)

Correlation between outcomes

S3.1

θ1 = θ2 = 0.40

 

S3.2

θ1 = θ2 = 0.80

Variability across studies

γ01 = γ11 = γ02 = γ12 = 0.001

We gave 8 scenarios of event probabilities in each group and their odds ratio. The true event probability in the control group, μ0k, was set by 0.003 (from scenario S2.1 to S2.4) or 0.006 (from scenario S2.5 to S2.8), which came from the incidences of the MI and the CVD events in the rosiglitazone data. Specifically, the incidences of the MI and the CVD events in the control group were 0.007 and 0.005 on average over 48 studies, respectively (continuity correction factor of 0.5 was added to studies with zero events in either group). The true odds ratio between groups was set by 1.0 or 1.8, where the former signified a null hypothesis of no treatment effect. The true event probability in the treatment group, μ1k, was calculated using the true event probability in the control group and the true odds ratio between groups. The parameter representing the variability across studies, γjk, was fixed as γ01 = γ02 = γ11 = γ12 = 0.001. The correlation parameters in the Gaussian copula, θ0 and θ1, were set by 0.4 (scenario S3.1) or 0.8 (scenario S3.2), reflecting the situations of a moderate or a high correlation between outcomes in each group.

Total number of studies involved in a meta-analysis was fixed as N = 48, which was the same total number of studies in the rosiglitazone data. We gave 4 scenarios of the number of studies reporting both outcomes:
  • Scenario S1.1: All studies reported both of the outcome 1 and 2; i.e., |Λ12| = 48, |Λ1| = 0, and |Λ2| = 0

  • Scenario S1.2: 32 studies (2/3 of the total number of studies) reported both of the outcome 1 and 2, and 16 studies (1/3 of the total number of studies) reported only the outcome 1; i.e., |Λ12| = 32, |Λ1| = 16, and |Λ2| = 0.

  • Scenario S1.3: 24 studies (1/2 of the total number of studies) reported both of the outcome 1 and 2, and 24 studies (1/2 of the total number of studies) reported only the outcome 1; i.e., |Λ12| = 24, |Λ1| = 24, and |Λ2| = 0.

  • Scenario S1.4: 16 studies (1/3 of the total number of studies) reported both of the outcome 1 and 2, and 32 studies (2/3 of the total number of studies) reported only the outcome 1; i.e., |Λ12| = 16, |Λ1| = 32, and |Λ2| = 0.

These scenarios reflect a common situation that some studies report only one outcome. The number of subjects per group in a single study was drawn from a log-normal distribution with log-mean of log(300) and scale of 0.5, and the two groups were assumed to have the same number of subjects; i.e., ni0 ~ LN(meanlog = log(300), sdlog = 0.5) with rounding to the nearest integer and ni0 = ni1. With this log-normal distribution, Q1/median/Q3 of the number of subjects per group was 214/300/420. This setting also came from the rosiglitazone data, where the average numbers of subjects were 270.0 in the control group and 351.2 in the rosiglitazone group.

Given the number of subjects per group, the event probabilities in each group and their odds ratio, the correlation between outcomes, and the variability across studies, we first generated bivariate binomial data of the two outcomes individually for the 48 studies. And then, we randomly selected studies reporting both outcomes in accordance with the corresponding scenario.

Step 3–4: Performing meta-analyses and summarizing their results

In Step 3, we performed meta-analyses using the following five methods:
  1. (i)

    Yusuf–Peto method (Yusuf et al. 1985): This is currently known as a standard method for performing meta-analysis of the rare-event outcomes. Indeed, the Cochrane collaboration (Higgins and Green 2011) recommended using the Yusuf–Peto method at event rates below 0.01. They advocated that this was the least biased and the most powerful method, provided that there was no substantial imbalance between treatment and control group sizes within studies and treatment effects were not exceptionally large. This finding has been supported by a simulation study conducted in Sweeting et al. (2004). We applied the Yusef–Peto method to the two outcomes separately.

     
  2. (ii)

    Two separate univariate meta-analyses using normal random-effects models: The normal random-effects model is the most common practice in the meta-analysis. The model assumes an asymptotic normality of treatment effect estimates from each study conditioned on different true treatment effects underlying different studies, where the true treatment effects are also assumed to follow a normal distribution. The log odds ratio is here considered as the treatment effect of interest. We applied the normal random-effects model to the two outcomes separately.

     
  3. (iii)

    Bivariate meta-analysis using a bivariate normal random-effects model: The bivariate normal random-effects model can be used for jointly combining the two outcomes whilst incorporating within-study correlations and a between-study correlation. We here supposed a situation that the within-study correlations were unknown, and considered using an alternative model proposed by Riley et al. (2008). This alternative model first assumes marginal normal distributions for the two outcomes separately, and then, a single overall correlation is introduced for linking the margins under bivariate normality assumption. We applied the bivariate normal random-effects model (alternative model) to the two outcomes.

     
  4. (iv)

    Two separate univariate meta-analyses using beta-binomial models: The beta-binomial distribution (1) has a potential benefit of naturally including information from studies having no event without any corrections. We applied the beta-binomial model to the two outcomes separately.

     
  5. (v)

    Bivariate meta-analysis using a bivariate beta-binomial model with bivariate Gaussian copula: We applied the bivariate beta-binomial model described in Sect. 3 to the two outcomes.

     

In the method (i), studies having zero events in both groups were removed from the analysis. In the methods (ii) and (iii), a continuity correction factor of 0.5 was added to studies with zero events in either group.

In Step 4, we calculated the following quantities for comparing the methods described above: (a) bias and root-mean-square error (RMSE) of pooled estimates, and coverage probability of 95% confidence interval of the log odds ratio for the two outcomes, and (b) type I error rate and empirical power on odds ratio for the two outcomes.

4.1.1 Misspecification of copula

To assess robustness of the bivariate beta-binomial model with the Gaussian copula against misspecification of the correlation structure, the simulation study was repeated under the situations that the Gaussian copula was not a correct model for the dependence. We considered three typical bivariate Archimedean copulas as true correlation structures between outcomes: Clayton (1978), Cook and Johnson (1981), Oakes (1982), Gumbel (1960), Barnett (1980), and Frank (1979) copula. Their copula parameters were provided, so that Kendall’s tau becomes 0.60. Note that the Gaussian copula parameter is 0.81 for the Kendall’s tau of 0.60 (see also Nelsen (2006) and Joe (2014) for further details of each copula).

4.2 Results

Results of the simulation studies are shown in Figs. 2, 3, 4, 5, and 6, where the scenarios of the number of studies reporting both outcomes (S1.1–S1.4) and the correlation between outcomes (S3.1–S3.2) are depicted on top of the panel. The horizontal axis represents the scenarios of the event probabilities in each group and their odds ratio. We below refer to the methods as follows: Yusuf–Peto method (YP), two separate univariate meta-analyses using normal random-effects models (UN), bivariate meta-analysis using a bivariate normal random-effects model (BN), two separate univariate meta-analyses using beta-binomial models (UBB), and bivariate meta-analysis using a bivariate beta-binomial model with bivariate Gaussian copula (BBB).
Fig. 2

Bias (top), root-mean-square error (middle), and coverage probability of 95% confidence interval (bottom) of log odds ratio for the outcome 1

Fig. 3

Bias (top), root-mean-square error (middle), and coverage probability of 95% confidence interval (bottom) of log odds ratio for the outcome 2

Fig. 4

Median of length of 95% confidence interval of log odds ratio for the outcome 2

Fig. 5

Type I error rate (top) and empirical power (bottom) on odds ratio for the outcome 1

Fig. 6

Type I error rate (top) and empirical power (bottom) on odds ratio for the outcome 2

4.2.1 Bias and RMSE of pooled estimates, and coverage probability of the log odds ratio

Figures 2, 3 show the biases and the RMSEs of pooled estimates, and the coverage probabilities of the log odds ratio for the outcomes 1 and 2, respectively. The UN and the BN provided negatively biased estimates in almost all the scenarios for both outcomes. The negative biases from the UN and the BN were substantial, especially in the scenarios that the true event probabilities in the treatment group were higher than those in the control group; i.e., scenarios S2.3, S2.4, S2.7, and S2.8 for the outcome 1, and scenarios S2.2, S2.4, S2.6, and S2.8 for the outcome 2. These biases indicate a well-known concern associated with the continuity correction. When the true event probabilities in the treatment group were higher than those in the control group, zero events were more likely to occur in the control group. Thus, adding the continuity correction factor of 0.5 to studies with zero events obviously pulled the pooled estimates in the negative direction. Although the RMSEs from the UN and the BN were smaller than those from the other methods, the coverage probabilities from the UN and the BN were seriously below or above nominal level of 0.95 in almost all the scenarios for both outcomes. These findings were not depending on the scenarios of the correlation between outcomes and the number of studies reporting both outcomes.

The pooled estimates from the YP were unbiased for both outcomes, except for the scenario that the true event probabilities in the treatment group were higher than those in the control group; i.e., scenario S2.3, S2.4, S2.7, and S2.8 for the outcome 1, and scenario S2.2, S2.4, S2.6, and S2.8 for the outcome 2, where slight negative biases were found. The coverage probabilities from the YP were slightly below the nominal level of 0.95 for the outcome 2, especially in the scenario that the number of studies reporting the outcome 2 was the smallest (scenario S1.4).

The UBB and the BBB provided quite similar results for the outcome 1. We could find no difference between the UBB and the BBB in Fig. 1. The two methods worked well in all the scenarios for the outcome 1. Indeed, their pooled estimates were unbiased, and their coverage probabilities were well controlled around the nominal level of 0.95. For the outcome 2, the pooled estimates from the UBB and the BBB were again unbiased, while we could find a reduction of RMSE from the BBB. In the scenarios of the high correlation (scenario S3.2), the BBB provided smaller RMSEs for the outcome 2 than the UBB as decreasing the number of studies reporting the outcome 2 (scenario S1.2, S1.3, and S1.4), indicating the potential benefit of incorporating the correlation between outcomes. The gain in the RMSE from the BBB was remarkable in the scenario that the number of studies reporting the outcome 2 was the smallest (scenario S1.4). The same trend of reducing the RMSEs was found in results from the UN and the BN for the outcome 2. The coverage probabilities from the UBB and the BBB for the outcome 2 were close to the nominal level of 0.95 in almost all the scenarios. In the scenario that the number of studies reporting the outcome 2 was the smallest (scenario S1.4), the coverage probabilities from the BBB were slightly below the nominal level of 0.95 (around 1%). This is explained by seeing the length of the 95% confidence interval. Figure 4 shows the medians of the length of the 95% confidence interval for the scenarios that the numbers of studies reporting the outcome 2 were 24 and 16 (scenario S1.3 and S1.4, respectively). The medians of the length of 95% confidence interval from the BBB were shorter than those from the UBB in the scenario that the number of studies reporting the outcome 2 was the smallest and the correlation was high. In such situations, the confidence interval of the BBB might be slightly too short to keep the nominal level.

4.2.2 Type I error rate and empirical power on the odds ratio

Figures 5, 6 show the type I error rates and the empirical powers on the odds ratio for the outcome 1 and 2, respectively. The type I error rates from the UN and the BN were substantially below the nominal level of 0.05 for both outcomes in all the scenarios. This again indicates the negative influence of adding the continuity correction factor to studies with zero events. The empirical powers from the two methods were also much lower than the other methods, especially in the scenario of the lower event probabilities in the control group; i.e., scenario S2.3 and S2.4 for the outcome 1, and scenario S2.2 and S2.4 for the outcome 2.

The type I error rates from the YP were slightly above the nominal level of 0.05 for both outcomes (around 1–2%). This inflation would come from the model assumption of the YP. The YP genuinely assumes a common treatment effect across studies (the so-called fixed effect), which might lead to slight underestimation of variance of the pooled estimates for both outcomes.

The UBB and the BBB provided well-controlled type I error rates for both outcomes. For the outcome 2, in the scenario that the number of studies reporting the outcome 2 was the smallest (scenario S1.4), the type I error rate from the BBB was slightly above the nominal level of 0.05 (around 1%). The empirical powers from the UBB and the BBB were quite similar for the outcome 1. In the scenario of the high correlation (scenario S3.2) for the outcome 2, the empirical powers from the BBB were higher than those from the UBB as decreasing the number of studies reporting the outcome 2 (scenario S1.2, S1.3, and S1.4), which were remarkable in the scenario that the number of studies reporting the outcome 2 was the smallest (scenario S1.4). This was suggested that the reduction of RMSE in the BBB could yield the gain in the empirical power. The same trend of increasing the empirical power was also found in results from the UN and the BN for the outcome 2.

4.2.3 Robustness against misspecification of copula

Figures 7, 8 show the biases and the RMSEs of pooled estimates, and the coverage probabilities of the log odds ratio for the outcome 1 and 2, respectively, where results for the three scenarios of true correlation structures were depicted in each panel: Clayton (left), Gumbel (central), and Frank (right) copula. The figures include results only for the scenarios that the number of studies reporting the outcome 2 was relatively small (scenario S1.3 and S1.4), because these cases could be more sensitive to the misspecification of copula.
Fig. 7

Misspecification of copula: bias (top), root-mean-square error (middle), and coverage probability of 95% confidence interval (bottom) of log odds ratio for the outcome 1

Fig. 8

Misspecification of copula: bias (top), root-mean-square error (middle), and coverage probability of 95% confidence interval (bottom) of log odds ratio for the outcome 2

Substantial negative biases from the UN and the BN were again observed in almost all the scenarios for both outcomes, as was the case in the simulation study using the Gaussian copula as the true correlation structure. Results from the YP were also consistent with those in the above simulation study.

The UBB and the BBB provided quite similar results for the outcome 1. We could find no difference between the UBB and the BBB for every scenario of the true correlation structure in Fig. 7. Their pooled estimates were unbiased and their coverage probabilities were well controlled around the nominal level of 0.95, indicating that the BBB would be robust against the misspecification of copula in the estimation of the log odds ratio for the outcome 1. For the outcome 2, the pooled estimates from the UBB and the BBB were again unbiased in almost all the scenarios. In particular, despite the presence of misspecification of copula, the BBB could be expected to provide unbiased estimation of the log odds ratio for the outcome 2. The BBB provided smaller RMSEs for the outcome 2 than the UBB for every scenario of the true correlation structure; especially in the scenarios of the Gumbel copula, gains in the reduction of RMSEs were the largest. The coverage probabilities from the UBB and the BBB for the outcome 2 were close to the nominal level of 0.95 in almost all the scenarios. However, in the scenario that the number of studies reporting the outcome 2 was the smallest (scenario S1.4), the coverage probabilities from the BBB were again slightly below the nominal level of 0.95. This trend was more apparent in the scenarios of the Clayton copula (around 3%) than when the Gaussian copula was correctly specified.

In summary, even when the Gaussian copula was not a correct model for the dependence, the BBB could be robust enough to provide unbiased estimation of the log odds ratio for both outcomes. However, it should be also noted that, in some cases (e.g., when assuming the Clayton copula as the true correlation structure), the BBB would provide the underestimation of standard error of the log odds ratio for the outcome 2. Then, the confidence interval from the BBB could become too short to keep the nominal level.

5 Application to rosiglitazone data

Consider now application to the rosiglitazone data. Table 2 shows the results of application to the rosiglitazone data by the five methods: (i) Yusuf–Peto method (YP), (ii) two separate univariate meta-analyses using normal random-effects models (UN), (iii) bivariate meta-analysis using a bivariate normal random-effects model (BN), (iv) two separate univariate meta-analyses using beta-binomial models (UBB), and (v) bivariate meta-analysis using a bivariate beta-binomial model with bivariate Gaussian copula (BBB) (see Sect. 4.1 for details of each meta-analysis method). In the method (i), studies having zero events in both (rosiglitazone and control) groups were removed from the analysis. In the methods (ii) and (iii), a continuity correction factor of 0.5 was added to studies with zero events in either group. Note that the incidences of the MI and the CVD events were estimated only by the UBB and the BBB. The variability across studies shows the estimates of root of between-study variance for the UN and the BN, and estimates of γjk for the UBB and the BBB. The correlation between the MI and the CVD events was also provided from the two bivariate methods: the BN provided an overall correlation between the log odds ratios of the MI and the CVD events, and the BBB provided correlations between the incidences of the MI and the CVD events through the Gaussian copula.
Table 2

Results of application to the rosiglitazone data

 

YP

UN

BN

UBB

BBB

Myocardial infection (MI) event

     

Incidence in control group (SE) [95% CI]

NA

NA

NA

0.00380 (0.00088)

[0.00207, 0.00553]

0.00384 (0.00096)

[0.00195, 0.00573]

Incidence in rosiglitazone group (SE) [95% CI]

NA

NA

NA

0.00491 (0.00090)

[0.00315, 0.00668]

0.00492 (0.00090)

[0.00316, 0.00668]

Odds ratio between groups [95% CI]

1.428 [1.031, 1.979]

1.232 [0.909, 1.670]

1.353 [1.037, 1.764]

1.295 [0.723, 2.318]

1.283 [0.697, 2.361]

Variability across studies*

NA

0.0077

0.0025

0.0039, 0.0034

0.0052, 0.0035

Cardiovascular death (CVD) event

     

Incidence in control group (SE) [95% CI]

NA

NA

NA

0.00133 (0.00043)

[0.00049, 0.00216]

0.00129 (0.00044)

[0.00044, 0.00215]

Incidence in rosiglitazone group (SE) [95% CI]

NA

NA

NA

0.00228 (0.00048)

[0.00134, 0.00322]

0.00235 (0.00049)

[0.00139, 0.00331]

Odds ratio between groups [95% CI]

1.640 [0.980, 2.744]

1.097 [0.725, 1.657]

1.225 [0.853, 1.758]

1.724 [0.809, 3.676]

1.816 [0.834, 3.956]

Variability across studies*

NA

0.0095

0.0105

0.0009, 0.0010

0.0018, 0.0010

Correlation between MI and CVD event†

NA

NA

0.806

NA

0.774, 0.547

NA: not applicable, SE: standard error, CI: confidence interval

*UN and BN provided the estimates of root of between-study variance. UBB and BBB provided the estimates of the parameter gamma for control and rosiglitazone groups, shown in this order

†BN provided an overall correlation between log odds ratios of MI and CVD events. BBB provided correlations through Gaussian copula for control and rosiglitazone groups, shown in this order

The pooled estimates of odds ratio for the MI events were similar among the five methods. The UBB and the BBB provided wider range of 95% confidence intervals. There was no considerable heterogeneity across studies for the MI event, according to the results from the four methods except for the YP. The incidences of the MI events from the UBB and the BBB were similar for both groups. The pooled estimates from the UN and the BN were much smaller than those from the other methods. This would be because adding the continuity correction factor of 0.5 to studies with zero events pulled the pooled estimates in the negative direction. Indeed, more studies were subjected to zero CVD events in comparison to the MI events. Considering the negative biases from the UN and the BN observed in the simulation study, the YP, the UBB, and the BBB were expected to provide more robust assessments of the risk of rosiglitazone on the CVD events. There were no considerable heterogeneity across studies for the CVD events. The incidences of the CVD events from the UBB and the BBB were similar for both groups. The correlation between the MI and the CVD events was high enough to be considered. In the BBB, the correlation in the control group was higher than that in the rosiglitazone group.

5.1 Illustration with studies reporting MI and/or CVD

To imitate situations that some studies reported only one outcome (either MI or CVD events), we generated 6 scenarios in terms of the number of studies reporting the MI and/or the CVD events. In particular, we assumed the scenarios of: (number of studies reporting MI events, number of studies reporting CVD events) = {(48, 36), (48, 24), (48, 16), (36, 48), (24, 48), (16, 48)}. We first randomly selected studies reporting the MI and/or the CVD events in accordance with the corresponding scenario, and then performed meta-analyses using the five methods (i)–(v) described above. The implementation steps were repeated 1000 times. In each scenario, we took medians of 1000 pooled estimates of the log odds ratio and their 1000 standard errors obtained from each method. The incidences of the MI and the CVD events estimated from the UBB and the BBB were also summarized in the same way.

Figures 9, 10 show the medians of pooled estimates and standard errors for the log odds ratio of the MI and the CVD events, respectively. The squares depicted on the left side of each panel represent the results using complete data from 48 studies, which are identical with those in Table 2. The horizontal axis represents the scenarios of the number of studies reporting the MI and/or CVD events. The univariate approaches of the YP, the UN, and the UBB provided constant results for the MI events in the scenarios that 48 studies reported the MI events, and for the CVD events in the scenarios that 48 studies reported the CVD events. When comparing the medians of standard errors from the UBB and the BBB for the MI (Fig. 9) and the CVD events (Fig. 10), the BBB provided smaller standard errors than the UBB as decreasing the number of studies reporting the MI and the CVD events, respectively, indicating the potential benefit of incorporating the correlation between outcomes.
Fig. 9

Pooled estimates of log odds ratio for MI event and their standard errors

Fig. 10

Pooled estimates of log odds ratio for CVD event and their standard errors

Figures 11, 12 show the medians of pooled estimates and standard errors for the incidences of the MI and the CVD events, respectively. The squares depicted on the left side of each panel again represent the results using complete data from 48 studies. We could find no considerable difference between the pooled estimates from the UBB and the BBB, while the BBB provided smaller standard errors than the UBB in both groups for the MI (Fig. 11) and the CVD events (Fig. 12) as decreasing the number of studies reporting the MI and the CVD events, respectively. The reduction of standard error in the BBB was larger for the MI events.
Fig. 11

Pooled estimates of incidence in each group for MI event and their standard errors

Fig. 12

Pooled estimates of incidence in each group for CVD event and their standard errors

6 Discussion

We considered the joint synthesis of two binary outcomes with low incidence, and proposed a novel bivariate meta-analysis method using copula. The meta-analysis of rare-event outcomes requires an additional statistical consideration due to the occurrence of studies with no event. While a variety of approaches have been developed for dealing with this matter, our proposed method would be another option for performing meta-analysis of two rare-event outcomes efficiently. The simulation study suggested that the method could be highly robust against the situations including studies with zero events. This advantage came from the use of beta-binomial margins, which have been already recognized as a promising alternative for the rare-event outcome. Another advantage of the method indicated from the simulation study was the gain in precision of the pooled estimates. In the situations that some studies reported only one outcome, the method gave smaller RMSEs and higher empirical powers in comparison to the existing univariate approaches such as the Yusuf–Peto method and the beta-binomial model. These gains were remarkable in the situation with the higher correlation, and thus, the method could be expected to provide efficient estimation of treatment effects for the two outcomes whilst incorporating the correlation between outcomes. These findings were illustrated through an application to the rosiglitazone data. In addition, even though any realistic situations were not necessarily covered, the simulation study suggested that the method could be robust enough to provide unbiased estimation of the treatment effects in the presence of misspecification of copula. The robustness will need to be enhanced by further examinations.

Our simulation study was designed mainly for examining the benefits of using the bivariate concept in comparison to the univariate beta-binomial model which was known as one of the best choices in the meta-analysis of rare-event outcomes. On the one hand, some other forms of bivariate beta-binomial models (Kuss et al. 2014; Nikoloulopoulos 2015a, b; Chen et al. 2016) would also be available. A comparison between various forms of bivariate beta-binomial models needs to be further addressed in a future research. Note that the existing bivariate approaches were developed for applications to the meta-analysis of diagnostic test accuracy studies; hence, the correlation should be treated and interpreted in different manners for the meta-analysis of rare-event outcomes.

The interpretation of the correlation parameter θj is not straightforward. Note that, as is the case in bivariate meta-analyses of continuous outcomes, two different sources of correlations, i.e., within-study correlations and a between-study correlation, could contribute to the dependence between the two binary outcomes. The within-study correlation, measuring the dependence between yij1 and yij2, occurs, because the two outcomes are assessed using the same set of patients in each study, while the between-study correlation allows the studies’ true underlying effects to be correlated (i.e., the dependence between pij1 and pij2) due to the characteristics of the two events (Riley et al. 2007; Riley 2009; Jackson et al. 2011). The proposed method first assumes marginal beta-binomial distributions for the two outcomes separately, and then introduces a single overall correlation parameter θj for linking the two margins. This procedure leads to a kind of amalgam of the within-study correlations and the between-study correlation, which is the same concept as the alternative model proposed by Riley et al. (2008). Therefore, the estimated Gaussian copula parameter value should be interpreted with caution, and some possible situations such as discrepant within-study correlations across studies (i.e., the case that there exists the between-study heterogeneity of correlation) cannot be directly handled by the proposed method. Further investigations will be necessary for confirming the robustness of the proposed method against these situations.

There are some limitations of the proposed method and further research is required. We, in this article, used the Gaussian copula for identifying the correlation structure between the two outcomes; however, a variety of bivariate copula families are now available, and thus, the choice of bivariate copulas and its optimisation might lead to the better option depending on the situation faced by users. Indeed, our simulation study indicated that, in some cases with the presence of misspecification of copula, the confidence interval of the treatment effect from the bivariate beta-binomial model with the Gaussian copula could be too short to keep the nominal level. Furthermore, the more investigations regarding heterogeneity across studies would be necessary in the context of meta-analysis for the rare-event outcomes. A quantification of heterogeneity under the beta-binomial model and a model extension for incorporating study-level covariates could be essential in practical use of the method. Outcome reporting bias is another important issue to be discussed in the bivariate meta-analysis. It should be cared that the outcome reporting bias for safety outcomes might differ from efficacy outcomes, which have a possible tendency of not reporting clinically relevant adverse events with a few observations (Bennetts et al. 2017). Further research of extending the method to allow for a mixed treatment comparison would be welcome.

Notes

Compliance with ethical standards

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

References

  1. Barnett, V. (1980). Some bivariate uniform distributions. Communications in Statistics—Theory and Methods, 9, 453–461.MathSciNetzbMATHGoogle Scholar
  2. Bennetts, M., Whalen, E., Ahadiehc, S., & Cappelleri, J. C. (2017). An appraisal of meta-analysis guidelines: How do they relate to safety outcomes? Research Synthesis Methods, 8, 64–78.Google Scholar
  3. Berlin, J. A., Crowe, B. J., Whalen, E., Xia, H. A., Koro, C. E., & Kuebler, J. (2013). Meta-analysis of clinical trial safety data in a drug development program: Answer to frequently asked questions. Clinical Trials, 10, 20–31.Google Scholar
  4. Bohning, D., Mylona, K., & Kimber, A. (2015). Meta-analysis of clinical trials with rare events. Biometrical Journal, 4, 633–648.MathSciNetzbMATHGoogle Scholar
  5. Bradburn, M. J., Deeks, J. J., Berlin, J. A., & Localio, A. R. (2007). Much ado about nothing: A comparison of the performance of meta-analytical methods with rare events. Statistics in Medicine, 26, 53–77.MathSciNetGoogle Scholar
  6. Cai, T., Parast, L., & Ryan, L. (2010). Meta-analysis for rare events. Statistics in Medicine, 29, 2078–2089.MathSciNetGoogle Scholar
  7. Chen, Y., Hong, C., Ning, Y., & Su, X. (2016). Meta-analysis of studies with bivariate binary outcomes: A marginal beta-binomial model approach. Statistics in Medicine, 35, 21–40.MathSciNetGoogle Scholar
  8. Cheng, J., Pullenayegum, E., Marshall, J. K., Iorio, A., & Thabane, L. (2015). Impact of including or excluding both-armed zero-event studies on using standard meta-analysis methods for rare event outcome: A simulation study. British Medical Journal Open, 6, e010983.  https://doi.org/10.1136/bmjopen-2015-010983.CrossRefGoogle Scholar
  9. Chu, H., Nie, L., Chen, Y., Huang, Y., & Sun, W. (2012). Bivariate random effects models for meta-analysis of comparative studies with binary outcomes: Methods for the absolute risk difference and relative risk. Statistical Methods in Medical Research, 21, 621–633.MathSciNetGoogle Scholar
  10. Clayton, D. (1978). A model for association in bivariate life tables and its application in epidemiological studies of family tendency in chronic disease incidence. Biometrika, 65, 141–151.MathSciNetzbMATHGoogle Scholar
  11. Cook, R. D., & Johnson, M. E. (1981). A family of distributions for modeling non-elliptically symmetric multivariate data. Journal of the Royal Statistical Society: Series B, 43, 210–218.zbMATHGoogle Scholar
  12. Frank, M. J. (1979). On the simultaneous associativity of F(x, y) and x + y-F(x, y). Aequationes mathematicae, 21, 194–226.MathSciNetzbMATHGoogle Scholar
  13. Friedrich, J. O., Adhikari, N. K. J., & Beyene, J. (2007). Inclusion of zero total event trials in meta-analyses maintains analytic consistency and incorporates all available data. BMC Medical Research Methodology, 7, 5.Google Scholar
  14. Gumbel, E. J. (1960). Bivariate exponential distributions. Journal of the American Statistical Association, 55, 698–707.MathSciNetzbMATHGoogle Scholar
  15. Hammad, T. A., Laughren, T., & Racoosin, J. (2006). Suicidality in pediatric patients treated with antidepressant drugs. Archives of General Psychiatry, 63, 332–339.Google Scholar
  16. Higgins, J. P. T., & Green, S. (editors). (2011). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration.Google Scholar
  17. Hoyer, A., & Kuss, O. (2015). Meta-analysis of diagnostic tests accounting for disease prevalence: A new model using trivariate copulas. Statistics in Medicine, 34, 1912–1924.MathSciNetGoogle Scholar
  18. Jackson, D., Riley, R. D., & White, I. R. (2011). Multivariate meta-analysis: Potential and promise. Statistics in Medicine, 30, 2481–2498.MathSciNetGoogle Scholar
  19. Joe, H. (2014). Dependence modeling with copulas. London: Chapman and Hall/CRC.zbMATHGoogle Scholar
  20. Kirkham, J. J., Riley, R. D., & Williamson, P. R. (2012). A multivariate meta-analysis approach for reducing the impact of outcome reporting bias in systematic reviews. Statistics in Medicine, 31, 2179–2195.MathSciNetGoogle Scholar
  21. Kuss, O. (2015). Statistical methods for meta-analyses including information from studies without any events-add nothing to nothing and succeed nevertheless. Statistics in Medicine, 34, 1097–1116.MathSciNetGoogle Scholar
  22. Kuss, O., Hoyer, A., & Solms, A. (2014). Meta-analysis for diagnostic accuracy studies: A new statistical model using beta-binomial distributions and bivariate copulas. Statistics in Medicine, 33, 17–30.MathSciNetGoogle Scholar
  23. Lane, P. W. (2011). Meta-analysis of incidence of rare events. Statistical Methods in Medical Research, 22, 117–132.MathSciNetGoogle Scholar
  24. Loke, Y. K., Price, D., & Herxheimer, A. (2011). Chapter 14: adverse effects. In Higgins, J.P.T., Green, S. (eds.), Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. The Cochrane Collaboration. www.handbook.cochrane.org. Updated March, 2011.
  25. Mavridis, D., & Salanti, G. (2013). A practical introduction to multivariate meta-analysis. Statistical Methods in Medical Research, 22, 133–158.MathSciNetGoogle Scholar
  26. Nelsen, R. B. (2006). An introduction to copulas (2nd ed.). New York: Springer.zbMATHGoogle Scholar
  27. Nikoloulopoulos, A. K. (2015a). A mixed effect model for bivariate meta-analysis of diagnostic test accuracy studies using a copula representation of the random effects distribution. Statistics in Medicine, 34, 3842–3865.MathSciNetGoogle Scholar
  28. Nikoloulopoulos, A. K. (2015b). A vine copula mixed effect model for trivariate meta-analysis of diagnostic test accuracy studies accounting for disease prevalence. Statistical Methods in Medical Research.  https://doi.org/10.1177/0962280215596769.MathSciNetCrossRefGoogle Scholar
  29. Nissen, S. E., & Wolski, K. (2007). Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. New England Journal of Medicine, 356, 2457–2471.Google Scholar
  30. Nissen, S. E., & Wolski, K. (2010). Rosiglitazone revisited: An updated meta-analysis for myocardial infarction and cardiovascular mortality. Archives of Internal Medicine, 170, 1191–1201.Google Scholar
  31. Oakes, D. (1982). A model for association in bivariate survival data. Journal of the Royal Statistical Society: Series B, 44, 414–422.MathSciNetzbMATHGoogle Scholar
  32. Riley, R. D. (2009). Multivariate meta-analysis: The effect of ignoring within-study correlation. Journal of the Royal Statistical Society: Series A, 172, 789–811.MathSciNetGoogle Scholar
  33. Riley, R. D., Abrams, K. R., Lambert, P. C., Sutton, A. J., & Thompson, J. R. (2007). An evaluation of bivariate random-effects meta-analysis for the joint synthesis of two correlated outcomes. Statistics in Medicine, 26, 78–97.MathSciNetGoogle Scholar
  34. Riley, R. D., Thompson, J. R., & Abrams, K. R. (2008). An alternative model for bivariate random-effects meta-analysis when the within-study correlations are unknown. Biostatistics, 9, 172–186.zbMATHGoogle Scholar
  35. Rucker, G., Schwarzer, G., Carpenter, J. R., & Olkin, I. (2009). Why add anything to nothing? The arcsine difference as a measure of treatment effect in meta-analysis with zero cells. Statistics in Medicine, 28, 721–738.MathSciNetGoogle Scholar
  36. Shuster, J. J. (2010). Empirical vs natural weighting in random effects meta-analysis. Statistics in Medicine, 29, 1259–1265.MathSciNetGoogle Scholar
  37. Shuster, J. J., Guo, J. D., & Skyler, J. S. (2012). Meta-analysis of safety for low event-rate binomial trials. Research Synthesis Methods, 3, 30–50.Google Scholar
  38. Shuster, J. J., Jones, L. S., & Salmon, D. A. (2007). Fixed vs random effects meta-analysis in rare event studies: The rosiglitazone link with myocardial infarction and cardiac death. Statistics in Medicine, 26, 4375–4385.MathSciNetGoogle Scholar
  39. Shuster, J. J., & Walker, M. A. (2016). Low-event-rate meta-analyses of clinical trials: Implementing good practices. Statistics in Medicine, 35, 2467–2478.MathSciNetGoogle Scholar
  40. Sklar, A. (1973). Random variables, joint distributions, and copulas. Kybernetica, 9, 449–460.MathSciNetzbMATHGoogle Scholar
  41. Song, P. X.-K. (2000). Multivariate dispersion models generated from Gaussian copula. Scandinavian Journal of Statistics, 27, 305–320.MathSciNetzbMATHGoogle Scholar
  42. Song, P. X.-K., Li, M., & Yuan, Y. (2009). Joint regression analysis of correlated data using Gaussian copulas. Biometrics, 65, 60–68.MathSciNetzbMATHGoogle Scholar
  43. Spittal, M. J., Pirkis, J., & Gurrin, L. C. (2015). Meta-analysis of incidence rate data in the presence of zero events. BMC Medical Research Methodology.  https://doi.org/10.1186/s12874-015-0031-0.CrossRefGoogle Scholar
  44. Stijnen, T., Hamza, T. H., & Ozdemir, P. (2010). Random effects meta-analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data. Statistics in Medicine, 29, 3046–3067.MathSciNetGoogle Scholar
  45. Sweeting, M., Sutton, A. J., & Lambert, P. C. (2004). What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Statistics in Medicine, 23, 1351–1375.Google Scholar
  46. Tian, L., Cai, T., Pfeffer, M. A., Piankov, N., Cremieux, P. Y., & Wei, L. J. (2009). Exact and efficient inference procedure for meta-analysis and its application to the analysis of independent 2 × 2 tables with all available data but without artificial continuity correction. Biostatistics, 10, 275–281.Google Scholar
  47. Yusuf, S., Peto, R., Lewis, J., Collins, R., & Sleight, P. (1985). Beta blockade during and after myocardial infarction: An overview of the randomized trials. Progress in Cardiovascular Disease, 27, 335–371.Google Scholar

Copyright information

© Japanese Federation of Statistical Science Associations 2019

Authors and Affiliations

  1. 1.Japan-Asia Data Science, DevelopmentAstellas Pharma Inc.TokyoJapan
  2. 2.Faculty of MedicineUniversity of TsukubaTsukubaJapan

Personalised recommendations