Bayesian model averaging for evaluation of candidate gene effects

Abstract

Statistical assessment of candidate gene effects can be viewed as a problem of variable selection and model comparison. Given a certain number of genes to be considered, many possible models may fit to the data well, each including a specific set of gene effects and possibly their interactions. The question arises as to which of these models is most plausible. Inference about candidate gene effects based on a specific model ignores uncertainty about model choice. Here, a Bayesian model averaging approach is proposed for evaluation of candidate gene effects. The method is implemented through simultaneous sampling of multiple models. By averaging over a set of competing models, the Bayesian model averaging approach incorporates model uncertainty into inferences about candidate gene effects. Features of the method are demonstrated using a simulated data set with ten candidate genes under consideration.

Appendix: Preliminary model selection using the Occam’s Window method

The procedure described here is adapted from the Occam’s Window (OW) method of Madigan and Raftery (1994), which averages over a set of data-supported models. There are two principles underlying OW. First, a model that predicts the data far worse than the model producing the best predictions should no longer be considered. That is, models not belonging to set

$$ A^{\prime} = \left\{ {M_{k} :{\frac{{\max_{l} \left\{ {p\left( {M_{l} |{\mathbf{y}}} \right)} \right\}}}{{p\left( {M_{k} |{\mathbf{y}}} \right)}}} \le c} \right\} $$

(17)

should be excluded, where c is a chosen value (e.g., c = 20, by analogy with the popular 0.05 cutoff for P-values). The second principle (optional) appeals to the Occam’s razor, which excludes complex models (M _k ) receiving less support from the data than their simpler counterparts (M _l ), i.e., by excluding models in set:

$$ B = \left\{ {M_{k} :M_{l} \in A^{\prime}\;{\text{and}}\;M_{l} \in M_{k} \;{\text{and}}\;{\frac{{p\left( {M_{l} |{\mathbf{y}}} \right)}}{{p\left( {M_{k} |{\mathbf{y}}} \right)}}} > 1} \right\} $$

(18)

Then, Bayesian model averaging (2) is replaced by

$$ p\left( {\Updelta |{\mathbf{y}}} \right) = \sum\limits_{{M_{k} \in A}} {p\left( {\Updelta |M_{k} ,{\mathbf{y}}} \right)p\left( {M_{k} |{\mathbf{y}}} \right)} $$

(19)

where A contains models in set A′ but not in set B, and all probabilities are implicitly conditional on the set of models in A. So, the BMA problem is reduced to finding set A.

Rejection based on OW is based on the interpretation of the ratio of posterior probabilities of two competing models. Let $r = {\frac{{p\left( {M_{0} |{\mathbf{y}}} \right)}}{{P\left( {M_{1} |{\mathbf{y}}} \right)}}}$, where M ₀ and M ₁ denote the “smaller” and “larger” models, respectively, and $p\left( {M_{k} |y} \right)$ is the marginal (with respect to the parameters) posterior probability that k is the true model (k = 0, 1). Let O _L and O _R define the left and right boundaries of OW, respectively. If evidence supports M ₀ (i.e., r > O _R ), then M ₁ is rejected. Rejecting M ₀ occurs if r < O _L (Optionally, rejecting “smaller” model M ₀ may require stronger evidence if choosing O _L  < O ⁻¹ _R ). Once M ₀ is rejected, all its “submodels” are rejected as well. If O _L  ≤ r ≤ O _R , then both models stay.

Computing model posterior probability (3) requires evaluating integral (4), which may be analytically difficult or even impossible in many practical situations. Laplace’s approximation (Tierney and Kadane 1986) can be used to compute the marginal likelihood, say, of model k:

$$ p\left( {{\mathbf{y}}|M_{k} } \right) \approx \left( {2\pi } \right)^{{p_{k/2} }} \left| {{\mathbf{H}}_{{\tilde{{\varvec{\theta}}}_{k} }}^{ - 1} } \right|^{1/2} p\left( {{\mathbf{y}}|\tilde{{\varvec{\theta}}}_{k} ,M = k} \right)p\left( {\tilde{{\varvec{\theta}}}_{k} |M = k} \right) $$

(20)

where p _k is the dimension of θ _k , $\tilde{{\varvec{\theta}}}_{k}$ is the posterior mode of θ _k and ${\mathbf{H}}_{{\tilde{{\varvec{\theta}}}_{k} }}^{ - 1}$ is minus the inverse Hessian of $h\left( {{\varvec{\theta}}_{k} } \right) = \log \left\{ {p\left( {{\mathbf{y}}|{\varvec{\theta}}_{k} ,M = k} \right)p\left( {{\varvec{\theta}}_{k} |M = k} \right)} \right\}$, evaluated at ${\varvec{\theta}}_{k} = \tilde{{\varvec{\theta}}}_{k}$. A computationally convenient variant to approximation (20) uses the maximum likelihood estimator $\hat{{\varvec{\theta}}}_{k}$, instead of the posterior mode $\tilde{{\varvec{\theta}}}_{k}$ (Sorensen and Gianola 2002). In particular, if observations are i.i.d, then,

$$ p\left( {{\mathbf{y}}|M_{k} } \right) \approx \left( {2\pi } \right)^{{p_{k/2} }} \left( n \right)^{{p_{k/2} }} \left| {{\mathbf{H}}_{{1,\hat{{\varvec{\theta}}}_{k} }}^{ - 1} } \right|^{1/2} p\left( {{\mathbf{y}}|\hat{{\varvec{\theta}}}_{k} ,M = k} \right)p\left( {\hat{{\varvec{\theta}}}_{k} |M = k} \right) $$

(21)

where ${\mathbf{H}}_{{1,\hat{{\varvec{\theta}}}_{k} }}$ is the observed information matrix calculated from a single observation, evaluated at the maximum likelihood estimates of θ _k . Suppose that the prior conveys some sort of “minimal” information represented by ${\varvec{\theta}}_{k} |M = k\sim N\left( {\hat{{\varvec{\theta}}}_{k} ,H_{{1,\hat{{\varvec{\theta}}}}}^{ - 1} } \right)$. This is a unit information prior centered at the maximum likelihood estimator and having a precision equivalent to that brought up by a sample of size n = 1. Following Sorensen and Gianola (2002), it can be shown that

$$ \log r = \log \left[ {{\frac{{p\left( {M_{0} |{\mathbf{y}}} \right)}}{{P\left( {M_{1} |{\mathbf{y}}} \right)}}}} \right] = \log \left[ {{\frac{{p\left( {{\mathbf{y}}|\hat{{\varvec{\theta}}}_{0} ,M_{0} } \right)}}{{p\left( {{\mathbf{y}}|\hat{{\varvec{\theta}}}_{1} ,M_{1} } \right)}}}} \right] - {\frac{{\left( {p_{0} - p_{1} } \right)}}{2}}\log n $$

(22)

This is one half of the Bayesian Information Criterion (BIC, Schwarz 1978). Note that evaluating (20) or (21) requires specification of the number of parameters, which is not always obvious with correlated random effects (e.g., animal or sire effects) in the model. However, as shown in (22), what matters is the difference in number of parameters between the two models. In a CG study, all models share the same set of infinitesimal additive effects, and computing (22) is straightforward. Nevertheless, (22) is not readily applicable to situations when the number of random effects varies among models. A review of approaches for computing marginal likelihoods for selection of models involving random effects is in Sinharay and Stein (2005).

Model selection search can proceed in two directions, e.g., moving either from larger to smaller models (“down” algorithm) or from smaller to larger models (“up” algorithm). Let A and C be dynamically changing subsets of model space M, which contain “acceptable” models and “candidate” models (i.e., those currently under consideration), respectively. Both algorithms start with A = Ø and C = {set of starting models} and proceed until set C is empty. Upon completion, set A contains a set of potentially acceptable models. Finally, all models meeting either of the following two criteria are removed:

$$ \left\{ {M_{k} :M_{l} \in A^{\prime}\;{\text{and}}\;M_{l} \in M_{k} \;{\text{and}}\;\log \left( {{\frac{{p\left( {M_{l} |{\mathbf{y}}} \right)}}{{p\left( {M_{k} |{\mathbf{y}}} \right)}}}} \right) > O_{R} } \right\} $$

(23)

$$ \left\{ {M_{k} :\log \left( {{\frac{{\max_{l} \left\{ {p\left( {M_{l} |{\mathbf{y}}} \right)} \right\}}}{{p\left( {M_{k} |{\mathbf{y}}} \right)}}}} \right) > \log (c)} \right\} $$

(24)

This reduces considerably the number of models, and retains a set of data-supported models for the BMA analysis.

To illustrate the process, consider model selection involving the choice of three regression variables x ₁, x ₂, and x ₃, which may represent three candidate gene effects. Let sample size be n = 1,200, and x ₂ be the only regression variable that truly affects observation y. Without considering their interactions, there are eight possible models, including the null model, as follows:

$$ \begin{array}{*{20}c} {M1:y = \mu + x_{1} + x_{2} + x_{3} + e;} \hfill & {M2:y = \mu + x_{2} + x_{3} + e;} \hfill \\ {M3:y = \mu + x_{1} + x_{3} + e;} \hfill & {M4:y = \mu + x_{1} + x_{2} + e;} \hfill \\ {M5:y = \mu + x_{1} + e;} \hfill & {M6:y = \mu + x_{2} + e;} \hfill \\ {M7:y = \mu + x_{3} + e;} \hfill & {M8:y = \mu + e.} \hfill \\ \end{array} $$

Let log (O _L ) = log (1/20) ≈ −1.30 and log (O _R ) = log (20) ≈ 1.30 define the left and right boundaries of the OW, respectively.

Suppose that the “down” algorithm initializes with A = Ø and for simplicity C = {M1}. To start, pick a model, i.e., M1, from set C and add it to set A. Model selection search proceeds by comparing M1 with one of its submodels formed by removing one regression variable each time (leading to M2, M3, and M4, respectively). If M1 is rejected, the submodel replaces M1; otherwise, it “survives” as an acceptable model. First, select a submodel of M1, say M2 (which is M1 without x ₁). Because the influence of x ₁ on y is immaterial, models M1 and M2 would be expected to give similar likelihoods but the latter model has one less parameter. Using (22), this leads to $\log \left( {{\frac{{p\left( {M2|y} \right)}}{{p\left( {M1|y} \right)}}}} \right) \approx 0 - {\frac{{\left( { - 1} \right)}}{2}}\log \left( {120} \right) \approx 1.54 > \log \left( {O_{R} } \right).$ Thus, model selection rejects M1, and M2 replaces M1 in set C. Continue the same process with the other two submodels M3 and M4. Assume that M3 is rejected and M4 is accepted, because the latter contains the influential variable x ₂. Then, set C now consists of two models, M2 and M4.

Next, M2 is compared with all its submodels with one less regression variable (i.e., M6 and M7). First, select M7, which is M2 without x ₂. Assume that $\log \left[ {{\frac{{p\left( {{\mathbf{y}}|\hat{{\varvec{\theta}}}_{7} ,M7} \right)}}{{p\left( {{\mathbf{y}}|\hat{{\varvec{\theta}}}_{2} ,M2} \right)}}}} \right] = - 3.2$. Then, $\log \left[ {{\frac{{p\left( {M7|{\mathbf{y}}} \right)}}{{P\left( {M2|{\mathbf{y}}} \right)}}}} \right] \approx - 3.20 + 1.54 = - 1.66 < \log (O_{L} )$. Thus, M7 is rejected, and so is its submodel M8. Model selection continues to compare it with the other submodel M6 (i.e., formed by removing x ₃ from M2). Because x ₃ has no significant effect on y, we would expect that the likelihoods of the two models are approximately equal, such that $\log \left( {{\frac{{p\left( {M6|y} \right)}}{{p\left( {M2|y} \right)}}}} \right) \approx 0 - {\frac{{\left( { - 1} \right)}}{2}}\log \left( {120} \right) \approx 1.54 > \log \left( {O_{R} } \right)$. Thus, model selection rejects M2, and M6 replaces M2 in set C (now, set C consists of M4 and M6).

Like with M2, repeat the same calculation for comparing M4 and all its submodels (i.e., M5 and M6). For simplicity, we assume that M5 is rejected and M6 is accepted, because the latter contains the influential variable x ₃. Because M6 is already in set C, there is no need to add it there.

By the same reasoning, the model selection retains M6 when compared to the null model M8, so M6 enters set A. Now, there is no more “candidate” model in set C, and the “down” algorithm ends. The outcome is set A which contains only one acceptable model. In real situations, however, more models are expected in set A.

The “up” algorithm starts, e.g., with A = Ø and, for example, C = {M6}, where set C contains model(s) output from the “down” algorithm. To proceed, the model selection compares M6 with one of its super-models each with one more regression variable. If there is decisive evidence for the super-model, it replaces the smaller model, otherwise the smaller model remains. Adding x ₁ and x ₃ to M6 leads to super-models M4 and M2, respectively. In this setting, model selection rejects both super-models and retains the smaller model (M6), because both variables do not influence y. Upon completion, set A contains the acceptable model, i.e., M6. Because there is only one model in set A, evaluations based on (23) and (24) are no longer necessary, and set A is final.