Clustering non-linear interactions in factor analysis

Abstract

Factor analysis is a powerful tool for dimensionality reduction in multivariate studies. This study extends the factor model with non-linear interactions. The main contribution of our work is to present two approaches to cluster the non-linear interactions and thus develop new models that are not restricted to the extreme scenarios where all non-null interactions are different or all are the same. The first strategy to handle the clusters involves a finite mixture of degenerate components. The second option is specified via the Dirichlet process. A comprehensive simulation study is developed to explore the performance of the proposals. A sensitivity analysis is carried out to evaluate advantages of estimating a smoothness parameter defined in a covariance function of the Gaussian process establishing the non-linearity of the interactions. In terms of application, the methodology is illustrated with the analysis of gene expression levels related to four breast cancer data sets. The genes belonging to disjoint genome regions, with copy number alteration, are connected to the main factors and their non-linear interactions are estimated and clustered. The mutual investigation and comparison of these four breast cancer data sets is rarely found in the literature.

Appendices

Appendix A: Full posterior conditional distributions

$(F^{*}_{r}\mid \alpha , \lambda , \sigma ^{2}, z, X) \sim N_{n}(M_{F^{*}_{r}}, V_{F^{*}_{r}})$   where

$$\begin{aligned} V_{F^{*}_{r}}= & {} \left[ \displaystyle {\left( \sum _{i=1}^{m} \frac{z_{ir}}{\sigma ^{2}_{i}} \right) } I_{n} + K^{-1}(\lambda ,\phi ) \right] ^{-1} \; \hbox {and}\\ M_{F^{*}_{r}}= & {} V_{F^{*}_{r}} \left[ \displaystyle { \sum _{i=1}^{m} \frac{z_{ir}}{\sigma ^{2}_{i}}} \left( X^{\top }_{i\bullet } -\lambda ^{\top } \alpha ^{\top }_{i\bullet } \right) \right] . \end{aligned}$$

For the probability $\rho ^{*}_{ir}$ in (10), consider $Q_{0}=\mathbf {0}$ and   $Q_{r} = \displaystyle {-\frac{1}{2\sigma ^{2}_{i}} \left[ F^{*}_{r} F^{*\top }_{r} -2F^{*}_{r} \left( X^{\top }_{i\bullet } -\lambda ^{\top } \alpha ^{\top }_{i\bullet }\right) \right] }$,   with   $r = 0, 1, \ldots , R$.

$(\sigma ^{2}_{i} \mid \alpha _{i \bullet }, \lambda , F_{i \bullet }, X_{i \bullet }) \sim \text{ IG }(A,B)$ where $A = a + n/2$ and

$$\begin{aligned} B=\displaystyle {\frac{1}{2}\left[ X_{i\bullet }X^{\top }_{i\bullet }-2\alpha _{i\bullet }\lambda (X^{\top }_{i\bullet }-F^{\top }_{i\bullet }) - 2F_{i\bullet }X^{\top }_{i\bullet } + F_{i\bullet }F^{\top }_{i\bullet } + \alpha _{i\bullet }\lambda \lambda ^{\top }\alpha ^{\top }_{i\bullet }\right] + b}. \end{aligned}$$

In order to update $q_{il}^{*}$ and $\alpha _{il}$, consider the $N(M_{\alpha _{il}},V_{\alpha _{il}})$ such that $V_{\alpha _{il}} = \left[ \displaystyle {\frac{1}{w} } +\displaystyle { \frac{1}{\sigma ^{2}_{i}} }\sum _{j=1}^{n}\lambda ^{2}_{lj}\right] ^{-1}$ and $M_{\alpha _{il}} = V_{\alpha _{il}}\left[ \displaystyle {\frac{1}{\sigma ^{2}_{i}} }\sum _{j=1}^{n}\lambda _{lj}\left( X_{ij}-F_{ij}-\sum _{l^{*}\ne l}\alpha _{il^{*}}\lambda _{l^{*}j} \right) \right]$.

The construction of posterior weights via stick-breaking process takes into account:   $(\mathcal {V}_{ir} \mid F, z, \rho , \lambda , \phi ) \sim \text{ Beta }(z_{ir} + 1, \sum _{s=r+1}^{R}z_{is} + \tau )$.

The full conditional distribution of $\phi$ is:

$$\begin{aligned} p(\phi \mid \alpha , \lambda , F, \sigma ^2, z, X)&\propto p(X \mid \alpha ,\lambda , F, \sigma ^{2})~ p(F \mid \lambda , z)~ p(\phi ) \\&\propto \left\{ \prod _{r=0}^{R}\prod _{i=1}^{m}\left[ N_{n}(F^{\top }_{i\bullet }\mid M_{F^{\top }_{i\bullet }}, V_{F^{\top }_{i\bullet }}) \right] ^{z_{ir}}\right\} p(\phi ), \end{aligned}$$

with $p(\phi )$ being the density of the U(0.1, 0.5). We have $M_{F_{i\bullet }^\top } = M_{F_r^*}$ and $V_{F^{\top }_{i\bullet }} = V_{F^*_r}$ when $F_{i\bullet }^\top = F_{r}^*$.

The full conditional distribution of $\lambda _{\bullet j}$ is given by:

$$\begin{aligned} p(\lambda _{\bullet j} \mid \alpha , \lambda _{-\left\{ \bullet j\right\} }, F, \sigma ^2_{i}, X)&\propto p(X \mid \alpha ,\lambda , F, \sigma ^{2})~ p(F^{*}_{1},F^{*}_{2}, \ldots , F^{*}_{R} \mid \lambda , z_{i})~ p(\lambda _{\bullet j}) \\&\propto N_{L}(\lambda _{\bullet j} \mid M_{\lambda _j},V_{\lambda _j})\left| K(\lambda , \phi )\right| ^{-\sum ^{R}_{r=1} z_{ir}/2} \\&\quad \times \exp \left\{ -\frac{1}{2}\sum ^{R}_{r=1}z_{ir}F^{*}_{r}K(\lambda ,\phi )^{-1}F^{*\top }_{r}\right\} , \end{aligned}$$

where $V_{\lambda _j}=\left[ \alpha ^{\top }D^{-1}\alpha + I_{L}\right] ^{-1}$   and   $M_{\lambda _j}=V_{\lambda _j}\left[ \alpha ^{\top }D^{-1}(X_{\bullet j}-F_{\bullet j})\right]$. The term $\lambda _{-\left\{ \bullet j\right\} }$ indicates the matrix $\lambda$ without the j-th column.

Appendix B: Short description of some goodness-of-fit measurements

Let $\theta$ be a generic vector of unknown parameters associated to the model with likelihood $p(Y|\theta )$. In this case, $Y = \{Y_1, Y_2, \cdots , Y_n\}$ represents the set of observed data and n is the sample size. Supposed that an MCMC algorithm was applied to sample from the target posterior distribution $p(\theta |Y)$. As a result, $\theta ^{(s)}$ is the value generated in the s-th MCMC iteration after the burn-in period, for $s = 1, \ldots , S$. Assume that $\bar{\theta }$ is the posterior mean of $\theta$. Three measurements, considered in this paper to compare models in terms of goodness-of-fit, are summarized as follows:

• The DIC is a widely used criterion for model selection in the Bayesian context. According to [27] this quantity is calculated by $2\bar{D}(\theta )-D(\bar{\theta })$, where $\bar{D}(\theta ) = -2 \sum _{s=1}^{S} \ln [p(Y|\theta ^{(s)})]/S$ and $D(\bar{\theta }) = -2\ln [p(Y|\bar{\theta })]$.

• The WAIC criterion is obtained through the following difference $\hat{\text{ lppd }} - \hat{p}_{\tiny \text{ WAIC }}$. The first term is the estimated log pointwise predictive density given by $\hat{\text{ lppd }} = \sum _{i=1}^{n} \ln [\sum _{s=1}^{S} p(Y_i|\theta ^{(s)})/S]$. The second term is the estimated effective number of parameters obtained through the formulation $\hat{p}_{\tiny \text{ WAIC }} = \sum _{i=1}^{n} V_{s=1}^{S}[\ln p(Y_i|\theta ^{(s)})]$, where $V_{s=1}^{S}[a^{(s)}] = \sum _{s=1}^{S} (a^{(s)}-\bar{a})^2/(S-1)$ and $\bar{a} = \sum _{s=1}^{S} a^{(s)}/S$. Consider [29] for more details.

• The LPML is a model selection criterion based on the so called conditional predictive ordinate (CPO). For the i-th observation, we calculate $\hat{\text{ CPO }}_i = S [\sum _{s=1}^{S} 1/ p(Y_i|\theta ^{(s)})]^{-1}$. The target result is given by $\sum _{i=1}^{n} \ln \hat{\text{ CPO }}_i$. See [8] for additional details.