Bayesian Model Selection in Factor Analytic Models

Abstract

Factor analytic models are widely used in social science applications to study latent traits, such as intelligence, creativity, stress, and depression, that cannot be accurately measured with a single variable. In recent years, there has been a rise in the popularity of factor models due to their flexibility in characterizing multivari-ate data. For example, latent factor regression models have been used as a dimensionality reduction tool for modeling of sparse covariance structures in genomic applications (West, 2003; Carvalho et al. 2008). In addition, structural equation models and other generalizations of factor analysis are widely useful in epidemi-ologic studies involving complex health outcomes and exposures (Sanchez et al., 2005). Improvements in Bayesian computation permit the routine implementation of latent factor models via Markov chain Monte Carlo (MCMC) algorithms, and a very broad class of models can be fitted easily using the freely available software package WinBUGS. The literature on methods for fitting and inferences in latent factor models is vast (for recent books, see Loehlin, 2004; Thompson, 2004).

Appendix: Full Conditional Distributions for the Gibbs Sampler

Suppose we have a model with k factors, conditional distributions for the PX Gibbs Sampler are presented below:

$$\begin{array}{*{20}c} {y_{ij} = {\mathbf{z'}}_{ij} {\mathbf{\lambda }}_j^* + \varepsilon _{ij},} & {\varepsilon _{ij} \sim N\left({0,\sigma _j^2 } \right),} \\ \end{array} $$

where ${\text{z}}_{ij} = \left({\eta _{i1}^*, \ldots,\eta _{ik_j }^* } \right)^\prime,{\lambda }_j^* = \left({\lambda _{j1}^*, \ldots,\lambda _{jk_j }^* } \right)^\prime$ denotes the free elements of row j of Λ^*, and k _j = min(j, k) is the number of free elements. Let $\pi \left({{\lambda }_j^* } \right) = {\text{N}}_{k_j } \left({{\lambda }_{0j}^*,\Sigma _{0{\lambda }_j^* } } \right)$ denote the prior for ${\lambda }_j^*,$ the full conditional posterior distributions are as follows:

$$\pi \left({{\lambda }_j^* |\eta ^*,{\psi,}\sum {\text{,y}}} \right) = {\text{N}}_{kj} \left({\left({\sum\nolimits _{0{\lambda }_j^* }^{ - 1} + \sigma _j^{ - 2} {\mathbf{Z'}}_j {\mathbf{Z}}_j } \right)^{ - 1} \left({\sum\nolimits _{0{\lambda }_j^* }^{ - 1} {\lambda }_{0j}^* + \sigma _j^{ - 2} {\mathbf{Z'}}_j {\mathbf{Y}}_j } \right),\left({\sum\nolimits _{0{\lambda }_j^* }^{ - 1} + \sigma _j^{ - 2} {\mathbf{Z'}}_j {\mathbf{Z}}_j } \right)^{ - 1} } \right),$$

where Z _j = (z _1j ,…,z _nj )′ and Y _j = (y _1j ,…, y _nj )′. In addition, we have

$$\begin{array}{l} {\pi \left({\eta _i^* |\Lambda ^*,\sum,{\Psi,\text{y}}} \right) = {\text{N}}_k \left({\left({{\Psi }^{ - 1} + \Lambda ^{*\prime} \sum ^{ - 1} \Lambda ^* } \right)^{ - 1} \Lambda ^{*\prime} \sum ^{ - 1} {\mathbf{y}}_i,\left({{\mathbf{\Psi }}^{ - 1} + \Lambda ^{*\prime} \sum ^{ - 1} \Lambda ^* } \right)^{ - 1} } \right),} \\ {\pi \left({\Psi _l^{ - 1} |\eta ^*,{\mathbf{\Lambda }}^*,\sum,{\mathbf{y}}} \right) = \mathcal{G}\left({a_l + \frac{n} {2},b_l + \frac{1} {2}\sum\limits_{i = 1}^n {\eta _{il}^{*2} } } \right),} \\ {\pi \left({\sigma _j^{ - 1} |\eta ^*,{\mathbf{\Lambda }}^*,{\Psi },{\mathbf{y}}} \right) = \mathcal{G}\left({c_j + \frac{n} {2},b_j + \frac{1} {2}\sum\limits_{i = 1}^n {\left({y_{ij} - {\mathbf{z'}}_{ij} {\mathbf{\lambda }}_j^* } \right)^2 } } \right),} \\ \end{array} $$

where g(a _l , b _l ) is the prior for $\psi _l^{ - 1},$ for l = 1,…,k, and g(c _j , d _j ) is the prior for $\sigma _j^{ - 2},$ for j = 1,…,p.