Multilevel Heterogeneous Factor Analysis and Application to Ecological Momentary Assessment

Abstract

Ansari et al. (Psychometrika 67:49–77, 2002) applied a multilevel heterogeneous model for confirmatory factor analysis to repeated measurements on individuals. While the mean and factor loadings in this model vary across individuals, its factor structure is invariant. Allowing the individual-level residuals to be correlated is an important means to alleviate the restriction imposed by configural invariance. We relax the diagonality assumption of residual covariance matrix and estimate it using a formal Bayesian Lasso method. The approach improves goodness of fit and avoids ad hoc one-at-a-time manipulation of entries in the covariance matrix via modification indexes. We illustrate the approach using simulation studies and real data from an ecological momentary assessment.

Appendix: Conditional Distributions for the Block Gibbs Sampler

• 1. The full conditional distribution for the latent factor scores $\varvec{\omega }_{1gi}$ is multivariate normal and is given by

  $$\begin{aligned} \varvec{\omega }_{1gi} |\mathbf{y}_{gi}, \varvec{\mu }_g, \varvec{\theta }\sim N[\varvec{\omega }_{1gi}^*, \mathbf{V}_{\omega _{1gi}}], \end{aligned}$$

  (15)

  where $\mathbf{V}_{\omega _{1gi}} = (\varvec{\Lambda }_1^T \varvec{\Psi }_{1g}^{-1} \varvec{\Lambda }_1 + \varvec{\Phi }_{1g}^{-1})^{-1}$ and $\varvec{\omega }_{1gi}^* = \mathbf{V}_{\omega _{1gi}} \varvec{\Lambda }_1^T \varvec{\Psi }_{1g}^{-1} (\mathbf{y}_{gi} - \varvec{\mu }_g)$.

• 2. The full conditional distribution for the individual-specific intercept $\varvec{\mu }_g$ is a multivariate normal distribution given by

  $$\begin{aligned} \varvec{\mu }_g|\mathbf{Y}_{g}, \varvec{\Omega }_{1g}, \varvec{\Omega }_2, \varvec{\theta }\sim N[\varvec{\mu }_g^*, \mathbf{V}_{\mu _g}], \end{aligned}$$

  (16)

  where $\mathbf{V}_{\mu _g} = (N_g \varvec{\Psi }_{1g}^{-1} + \varvec{\Psi }_2^{-1})^{-1}$ and $\varvec{\mu }_g^* = \mathbf{V}_{\mu _g}[\varvec{\Psi }_2^{-1}(\varvec{\mu }+\varvec{\Lambda }_2\varvec{\omega }_{2g}) + \sum _{i=1}^{N_g}\varvec{\Psi }_g^{-1}(\mathbf{y}_{1gi}-\varvec{\Lambda }_1\varvec{\omega }_{1gi})]$.

• 3. Let $\varvec{\Lambda }_{1k}^T$ be the kth row of $\varvec{\Lambda }_1$, $\varvec{\Lambda }_{1(-k)}$ be submatrix of $\varvec{\Lambda }_1$ with the kth row deleted, $\mathbf{Y}_{gk}$ be the submatrix of $\mathbf{Y}_g$ only with the kth row and $\mathbf{Y}_{g(-k)}$ be the submatrix of $\mathbf{Y}_g$ with the kth row deleted. Note that $\varvec{\Psi }_{1g}$ is not diagonal, and without a loss of generality, we partition and rearrange the columns of $\varvec{\Psi }_{1g}$ as follows:

  $$\begin{aligned}&\varvec{\Psi }_{1g}=\left( \begin{array}{cccc} \varvec{\Psi }_{1(-kk)} &{}\quad \varvec{\psi }_{1k} \\ \varvec{\psi }_{1k}^T &{}\quad \psi _{1kk} \end{array}\right) , \end{aligned}$$

  where $\psi _{1kk}$ is the kth diagonal element of $\varvec{\Psi }_{1g}$, $\varvec{\psi }_{1k} = (\psi _{1k1}, \cdots , \psi _{1k,k-1}, \psi _{1k,k+1}, \cdots , \psi _{1kp})^T$ is the vector of all off-diagonal elements of the kth column and $\varvec{\Psi }_{1(-kk)}$ is the $(p-1) \times (p-1)$ matrix resulting from deleting the kth row and kth column from $\varvec{\Psi }_{1g}$. We suppress the subscript g in the definitions of $\varvec{\Psi }_{1(-kk)}$, $\varvec{\psi }_{1k}$ and $\psi _{1kk}$. It can be shown that: for $k=1, 2, \ldots , p$,

  $$\begin{aligned} \varvec{\Lambda }_{1k}|\mathbf{Y}, \varvec{\Omega }_1, \varvec{\mu }_g, \varvec{\theta }\sim N[\varvec{\Lambda }_{1k}^*, \mathbf{V}_{\Lambda _{1k}}], \end{aligned}$$

  (17)

  where

  $$\begin{aligned} \mathbf{V}_{\Lambda _{1k}}= & {} \left( \sum _{g=1}^G \varvec{\Psi }_{1gk}^{*-1}\varvec{\Omega }_{1g}\varvec{\Omega }_{1g}^T + \mathbf{H}_{01k}^{-1}\right) ^{-1}, \\ \varvec{\Lambda }_{1k}^*= & {} \mathbf{V}_{\Lambda _{1k}}\left( \sum _{g=1}^G \varvec{\Psi }_{1gk}^{*-1} \varvec{\Omega }_{1g} \mathbf{Y}_{gk}^{*T} +\mathbf{H}_{01k}^{-1}\varvec{\Lambda }_{01k}\right) , \\ \varvec{\Psi }_{1gk}^*= & {} \psi _{1kk}-\varvec{\psi }_{1k}^T\varvec{\Psi }_{1(-kk)}^{-1}\varvec{\psi }_{1k}, \end{aligned}$$

  and $\mathbf{Y}_{gk}^*$ is the matrix with element

  $$\begin{aligned} y_{gik}^*=y_{gik}-\mu _{gk}-\varvec{\psi }_{1k}^T\varvec{\Psi }_{1(-kk)}^{-1}(\mathbf{y}_{gi(-k)}-\varvec{\mu }_{g(-k)}-\varvec{\Lambda }_{1(-k)}\varvec{\omega }_{1gi}), \end{aligned}$$

  $\mathbf{y}_{gi(-k)}$ is the vector of $\mathbf{y}_{gi}$ with the kth element deleted and $\varvec{\mu }_{g(-k)}$ is the vector of $\varvec{\mu }_g$ with the kth element deleted.

• 4. For $g=1, 2, \ldots , G$, let ${{\tilde{\mathbf{S}}}_g}=\sum _{i=1}^{N_g} (\mathbf{y}_{gi} - \varvec{\mu }_g - \varvec{\Lambda }_1\varvec{\omega }_{1gi})(\mathbf{y}_{gi} - \varvec{\mu }_g - \varvec{\Lambda }_1\varvec{\omega }_{1gi})^T$. Following Wang (2012), an efficient block Gibbs sampling is updated by ${\varvec{\Sigma }}_g$ one column at a time after appropriate reparametrization. Denote the running index for observed measurements by k. For $k=1, 2, \ldots , p$, partition and rearrange the columns of ${\varvec{\Sigma }}_g$ and ${{\tilde{\mathbf{S}}}_g}$ as follows:

  $$\begin{aligned}&{\varvec{\Sigma }}_g=\left( \begin{array}{cccc} {\varvec{\Sigma }}_{(-kk)} &{}\quad \varvec{\sigma }_k \\ \varvec{\sigma }_k^T &{}\quad \sigma _{kk} \\ \end{array}\right) ,~~~ {{\tilde{\mathbf{S}}}_g}=\left( \begin{array}{cccc} {{\tilde{\mathbf{S}}}}_{(-kk)} &{}\quad {{\tilde{\mathbf{s}}}}_k \\ {{\tilde{\mathbf{s}}}}_k^T &{}\quad {{\tilde{s}}}_{kk} \\ \end{array}\right) , \end{aligned}$$

  where $\sigma _{kk}$ is the kth diagonal element of ${\varvec{\Sigma }}_g$, $\varvec{\sigma }_k = (\sigma _{k1}, \ldots , \sigma _{k,k-1}, \sigma _{k,k+1}, \ldots , \sigma _{kp})^T$ is the vector of all off-diagonal elements of the kth column and ${\varvec{\Sigma }}_{(-kk)}$ is the $(p-1) \times (p-1)$ matrix resulting from deleting the kth row and kth column from ${\varvec{\Sigma }}_g$. Similarly, ${{\tilde{s}}}_{kk}$ is the kth diagonal element of ${{\tilde{\mathbf{S}}}_g}$, ${{\tilde{\mathbf{s}}}}_k$ is the vector of all off-diagonal elements of the kth column of ${{\tilde{\mathbf{S}}}_g}$, and ${{\tilde{\mathbf{S}}}}_{(-kk)}$ is the matrix with the kth row and kth column of ${{\tilde{\mathbf{S}}}_g}$ deleted. Note that for notational simplicity, we also suppress the subscript g in the above definitions. Let ${\varvec{\beta }}_g = \varvec{\sigma }_k$ and $\gamma _g = \sigma _{kk}-\varvec{\sigma }_k^T{\varvec{\Sigma }}_{(-kk)}^{-1}\varvec{\sigma }_k$. It can be shown that:

  $$\begin{aligned} {\varvec{\beta }}_g|{\varvec{\Sigma }}_{(-kk)}, \mathbf{Y}, \varvec{\Omega }_1, \varvec{\theta }, \varvec{\tau }_g, \lambda _g\sim & {} N[-\mathbf{V}_{\beta _g}{{\tilde{\mathbf{s}}}}_k, \mathbf{V}_{\beta _g}], \end{aligned}$$

  (18)
  $$\begin{aligned} \gamma _g|{\varvec{\Sigma }}_{(-kk)}, \mathbf{Y}, \varvec{\Omega }_1, \varvec{\theta }, \varvec{\tau }_g, \lambda _g\sim & {} Gamma\left( \frac{N_g}{2}+1, \frac{{{\tilde{s}}}_{kk}+ \lambda _g}{2}\right) , \end{aligned}$$

  (19)

  where $\mathbf{V}_{\beta _g} = \left[ ({\tilde{s}}_{kk}+\lambda _g){\varvec{\Sigma }}_{(-kk)}^{-1} + \mathbf{M}_{\varvec{\tau }_g}^{-1} \right] ^{-1}$ and $\mathbf{M}_{\varvec{\tau }_g}$ is the diagonal matrix with diagonal elements $\tau _{g,k1}, \ldots , \tau _{g,k(k-1)}, \tau _{g,k(k+1)}, \ldots , \tau _{g,kp}$. After simulating observations from the above conditional distributions, we can obtain $\varvec{\sigma }_k = {\varvec{\beta }}_g$, $\varvec{\sigma }_k^T = {\varvec{\beta }}_g^T$ and $\sigma _{kk} = \gamma _g + \varvec{\sigma }_k^T{\varvec{\Sigma }}_{(-kk)}^{-1}\varvec{\sigma }_k$; then, the last column and row of ${\varvec{\Sigma }}_g$ are updated at a time. At the end, $\varvec{\Psi }_{1g}={\varvec{\Sigma }}_g^{-1}$ is computed. The conditional distribution of $\varvec{\tau }_g$ can be expressed as follows: for $i<j$,

  $$\begin{aligned} \frac{1}{\tau _{g,ij}}|\mathbf{Y}, \varvec{\Omega }_1, \varvec{\theta }, {\varvec{\Sigma }}_g, \lambda _g \sim IG\left( \sqrt{\frac{\lambda _g^2}{\sigma _{g,ij}^2}}, \lambda _g^2\right) , \end{aligned}$$

  (20)

  where IG(a, b) indicates the inverse Gaussian distribution with mean a and shape parameter b. Additionally, it can be shown that the conditional distribution of $\lambda _g$ follows:

  $$\begin{aligned} \lambda _g|\mathbf{Y}, \varvec{\Omega }_1, \varvec{\theta }, {\varvec{\Sigma }}_g, \varvec{\tau }_g \sim Gamma\left( \alpha _{g}+\frac{p(p+1)}{2}, \beta _{g}+\frac{1}{2}\sum _{i=1}^p \sum _{j=1}^p |\sigma _{g,ij}|\right) . \end{aligned}$$

  (21)

• 5. The full conditional distribution for the latent factor scores $\varvec{\omega }_{2g}$ is multivariate normal and is given by

  $$\begin{aligned} \varvec{\omega }_{2g} |\varvec{\mu }_{g}, \varvec{\theta }\sim N[\varvec{\omega }_{2g}^*, \mathbf{V}_{\omega _{2g}}], \end{aligned}$$

  (22)

  where $\mathbf{V}_{\omega _{2g}} = (\varvec{\Lambda }_2^T \varvec{\Psi }_{2}^{-1} \varvec{\Lambda }_2 + \varvec{\Phi }_{2}^{-1})^{-1}$ and $\varvec{\omega }_{2g}^* = \mathbf{V}_{\omega _{2g}} \varvec{\Lambda }_2^T \varvec{\Psi }_{2}^{-1} (\varvec{\mu }_{g} - \varvec{\mu })$.

• 6. The full conditional distribution of $\varvec{\mu }$ can be written as a multivariate normal distribution given by

  $$\begin{aligned} \varvec{\mu }|\mathbf{U}, \varvec{\Omega }_2, \varvec{\theta }\sim N[\varvec{\mu }^*, \mathbf{V}_{\mu }], \end{aligned}$$

  (23)

  where $\mathbf{U}= (\varvec{\mu }_1, \ldots , \varvec{\mu }_G)$, $\mathbf{V}_{\mu } = (G \varvec{\Psi }_2^{-1} + \mathbf{H}_{02}^{-1})^{-1}$, $\varvec{\mu }^* = \mathbf{V}_{\mu }[\mathbf{H}_{02}^{-1}\varvec{\mu }_{02}+\sum _{g=1}^G \varvec{\Psi }_2^{-1} (\varvec{\mu }_g-\varvec{\Lambda }_2\varvec{\omega }_{2g})]$.

• 7. The full conditional distribution of $\varvec{\Phi }_2$ can be written as an inverse Wishart distribution given by

  $$\begin{aligned} \varvec{\Phi }_2 |\mathbf{U}, \varvec{\Omega }_2, \varvec{\theta }\sim IW\left( \sum _{g=1}^G \varvec{\omega }_{2g}\varvec{\omega }_{2g}^T + \mathbf{R}_{02}^{-1}, \rho _{01}+G\right) , \end{aligned}$$

  (24)

  where $IW(\cdot , \cdot )$ denotes the inverse Wishart distribution.

• 8. The full conditional distribution of $\psi _{2k}^{-1}$ can be written as gamma distribution given by: for $k=1, 2, \ldots , p$,

  $$\begin{aligned} \psi _{2k}^{-1} |\mathbf{U}, \varvec{\Omega }_2, \varvec{\theta }\sim Gamma\left( \frac{G}{2}+\alpha _{02k}, \frac{1}{2}\sum _{g=1}^G(\mu _{gk}-\mu _k-\varvec{\Lambda }_{2k}^T\varvec{\omega }_{2g})^2+\beta _{02k}\right) ,\qquad \quad \end{aligned}$$

  (25)

  where $\mu _{gk}$ and $\mu _k$ are the kth element of $\varvec{\mu }_g$ and $\varvec{\mu }$, respectively, and $\varvec{\Lambda }_{2k}^T$ is the kth row of $\varvec{\Lambda }_2$.

• 9. The full conditional distribution of $\varvec{\Lambda }_{2k}$ can be written as a multivariate normal distribution given by

  $$\begin{aligned} \varvec{\Lambda }_{2k} |\mathbf{U}, \varvec{\Omega }_2, \varvec{\theta }\sim N[\varvec{\Lambda }_{2k}^*, \mathbf{V}_{\Lambda _{2k}}], \end{aligned}$$

  (26)

  where $\mathbf{V}_{\Lambda _{2k}} = (\psi _{2k}^{-1}\varvec{\Omega }_{2}\varvec{\Omega }_{2}^T + \mathbf{H}_{02k}^{-1})^{-1}$, $\varvec{\Lambda }_{2k}^* = \mathbf{V}_{\Lambda _{2k}}(\psi _{2k}^{-1} \varvec{\Omega }_{2} \mathbf{U}_{k}^* +\mathbf{H}_{02k}^{-1}\varvec{\Lambda }_{02k})$ and $\mathbf{U}_{k}^{*T}=(\mu _{1k}-\mu _k, \mu _{2k}-\mu _k, \ldots , \mu _{Gk}-\mu _k)$.

• 10. The full conditional distribution for $\mathbf{R}$ can be expressed as an inverse Wishart distribution as follows:

  $$\begin{aligned} \mathbf{R}|\varvec{\Phi }_{11}, \varvec{\Phi }_{12}, \ldots , \varvec{\Phi }_{1G}, \varvec{\theta }\sim IW\left( \sum _{g=1}^G \varvec{\Phi }_{1g}^{-1} + \mathbf{R}_0 ^{-1}, \rho _0+G\rho \right) . \end{aligned}$$

  (27)

• 11. According to the model and the prior distribution defined, respectively, in Eqs. (5) and (12), the full conditional distribution of $\rho '$ is as follows:

  $$\begin{aligned} p(\rho '|\varvec{\Phi }_{11}, \ldots , \varvec{\Phi }_{1G}, \varvec{\theta }) \propto \prod _{g=1}^G p(\varvec{\Phi }_{1g}|\rho ', \mathbf{R}) \times p(\rho '), \end{aligned}$$

  (28)

  where the distribution of $\varvec{\Phi }_{1g}$ is Wishart, and the prior distribution of $\rho '$ is truncated univariate normal; therefore, this full conditional distribution is nonstandard, and the MH algorithm is employed to sample from this nonstandard distribution. Given the current value $\rho '^{(l)}$, we simulate a new candidate $\rho '^*$ from proposal distribution $N(\rho '^{(l)}, \varphi )$. $\rho '^*$ is then accepted as new observation $\rho '^{(l+1)}$ with the following probability

  $$\begin{aligned} \text {min}\left\{ 1, \frac{\prod _{g=1}^G p(\varvec{\Phi }_{1g}|\rho '^*, \mathbf{R}) \times p(\rho '^*)}{\prod _{g=1}^G p(\varvec{\Phi }_{1g}|\rho '^{(l)}, \mathbf{R}) \times p(\rho '^{(l)})}\right\} , \end{aligned}$$

  where tuning parameter $\varphi$ is selected such that the average acceptance rate is around 0.25 (Gelman et al. 1996).

• 12. The full conditional distribution of $\varvec{\Phi }_{1g}$ can be written as an inverse Wishart distribution given by: for $g=1, 2, \ldots , G$,

  $$\begin{aligned} \varvec{\Phi }_{1g} |\varvec{\Omega }_1, \varvec{\theta }\sim IW\left( \sum _{i=1}^{N_g} \varvec{\omega }_{1gi}\varvec{\omega }_{1gi}^T + \mathbf{R}^{-1}, \rho +N_g\right) . \end{aligned}$$

  (29)