Some Mathematical Properties of the Matrix Decomposition Solution in Factor Analysis

Abstract

A new factor analysis (FA) procedure has recently been proposed which can be called matrix decomposition FA (MDFA). All FA model parameters (common and unique factors, loadings, and unique variances) are treated as fixed unknown matrices. Then, the MDFA model simply becomes a specific data matrix decomposition. The MDFA parameters are found by minimizing the discrepancy between the data and the MDFA model. Several algorithms have been developed and some properties have been discussed in the literature (notably by Stegeman in Comput Stat Data Anal 99:189–203, 2016), but, as a whole, MDFA has not been studied fully yet. A number of new properties are discovered in this paper, and some existing ones are derived more explicitly. The properties provided concern the uniqueness of results, covariances among common factors, unique factors, and residuals, and assessment of the degree of indeterminacy of common and unique factor scores. The properties are illustrated using a real data example.

Appendix

Here, we describe the procedure for generating (34) to have ${\hat{\mathbf{Z}}}= \mathbf{Z}^{*}+\mathbf{Z}_{\bot }^{*}$, i.e., (13), for the matrix $\mathbf{Z}^{*}$ (33) given after Steps 1–4 in Sect. 2.

Let ${\mathbf{J_{1}}} = \mathbf{I}_{n}-n^{-1}{} \mathbf{1}_{n}{} \mathbf{1}_{{{n}}}{}^{\prime }, {\mathbf{J_K}}=\mathbf{I}_{n} - \mathbf{KK}^{\prime }$, and ${\mathbf{J_L}}= \mathbf{I}_{m+p} - \mathbf{LL}^{\prime }$. Using those matrices with arbitrary $n \times m$ matrix G and $(m+p) \times m$ matrix H, we consider the following SVDs:

$$\begin{aligned} \mathbf{J}_{\mathbf{1}}{} \mathbf{J}_{\mathbf{K}}{} \mathbf{G} = {\varvec{\Gamma }}_{\mathbf{G}}{\varvec{\Theta }} _{\mathbf{G}}{\varvec{\Xi }}_{\mathbf{G}}^{\prime }\quad \hbox {and}\quad \mathbf{J}_{\mathbf{L}}{} \mathbf{H} ={\varvec{\Gamma }} _{\mathbf{H}}{\varvec{\Theta }}_{\mathbf{H}}{\varvec{\Xi }}_{\mathbf{H}}^{\prime } \end{aligned}$$

(A1)

with $\Theta _{\mathbf{G}}$ and $\Theta _{\mathbf{H}}$ being diagonal and

$$\begin{aligned} {\varvec{\Gamma }}_{\mathbf{G}}^{\prime }{\varvec{\Gamma }}_{\mathbf{G}} ={\varvec{\Gamma }} _{\mathbf{H}}^{\prime }{\varvec{\Gamma }}_{\mathbf{H}} = {\varvec{\Xi }}_{\mathbf{G}}^{\prime }{\varvec{\Xi }}_{\mathbf{G}} = {\varvec{\Xi }}_{\mathbf{H}}^{\prime }{\varvec{\Xi }}_{\mathbf{H}} = \mathbf{I}_{m}. \end{aligned}$$

(A2)

Those SVDs allow us to set $\mathbf{K}_{\bot } = {\varvec{\Gamma }}_{\mathbf{G}}$ and $\mathbf{L}_{\bot } = {\varvec{\Gamma }}_{\mathbf{H}}$ to generate (34), which is summed to (33) so that (13) is given by

$$\begin{aligned} {\hat{\mathbf{Z}}} = n^{1/2}{\tilde{\mathbf{K}}}{\tilde{\mathbf{L}}}=n^{1/2}\mathbf{KL}^{\prime }+n^{1/2}{\varvec{\Gamma }}_{\mathbf{G}}{\varvec{\Gamma }} _{\mathbf{H}}^{\prime } = \mathbf{XS}_{\mathrm{XX}}^{-1}{} \mathbf{S}_{\mathrm{XZ}}+n^{1/2}{\varvec{\Gamma }} _{\mathbf{G}}{\varvec{\Gamma }}_{\mathbf{H}}^{\prime }. \end{aligned}$$

(A3)

It is because (A1) leads to ${\varvec{\Gamma }}_{\mathbf{G}} = \mathbf{J}_{1}{} \mathbf{J}_{K}\mathbf{G}\Xi _{\mathbf{G}}{\varvec{\Theta }}_{\mathbf{G}}^{-1}$ and ${\varvec{\Gamma }}_{\mathbf{H}} = \mathbf{J}_\mathbf{L}{} \mathbf{H}\Xi _{\mathbf{H}}{\varvec{\Theta }}_{\mathbf{H}}^{-1}$, which imply that

$$\begin{aligned} \mathbf{1}_{n}^{\prime }{\varvec{\Gamma }}_{\mathbf{G}} = \mathbf{0}_{m}, \mathbf{L}^{\prime }{\varvec{\Gamma }}_{\mathbf{H}}={_{p}{} \mathbf{O}}_{m},\ \text { and }{} \mathbf{K}^{\prime }{\varvec{\Gamma }}_{\mathbf{G}}={_{p}{} \mathbf{O}}_{m}. \end{aligned}$$

(A4)

Here, the last identity is derived as $\mathbf{K}^{\prime }{\varvec{\Gamma }} _{\mathbf{G}} = \mathbf{K}^{\prime }{} \mathbf{J}_\mathbf{1}{} \mathbf{J}_{\mathbf{K}}\mathbf{{G}}{\varvec{\Xi }}_{\mathbf{G}}{\varvec{\Theta }}_{\mathbf{G}}^{-1}= {\mathbf{K}}^{\prime }{} \mathbf{J}_{\mathrm{K}}{\mathbf{G}}{\varvec{\Xi }}_{{ \mathbf{G}}}{\varvec{\Theta }}_{\mathbf{G}}^{-1 }={_{p}{} \mathbf{O}_{m}}$ using $\mathbf{J}_{\mathbf{1}}{} \mathbf{{K}} = \mathbf{K}$ which follows from $\mathbf{1}_{n}^{\prime }{} \mathbf{K} = {\mathbf{0}_{p}}^{\prime }$ (Adachi, 2012). The equations $\mathbf{{K}}^{\prime }{} \mathbf{K} = \mathbf{L}^{\prime }{} \mathbf{L} = \mathbf{I}_{p}$ following from SVD (14) and the above (A2), (A4), and $\mathbf{1}_{n}^{\prime }{} \mathbf{K} = \mathbf{0}_{{p}^{\prime }}$ allow us to find that ${\tilde{\mathbf{K}}} = [\mathbf{K}, \mathbf{K}_{\bot } ] = [\mathbf{K}, {\varvec{\Gamma }}_{\mathbf{G}}$] and ${\tilde{\mathbf{L}}}'= [\mathbf{L}, \mathbf{L}_{\bot }] = [\mathbf{L},{\varvec{\Gamma }}_{\mathbf{H}}$] satisfy ${{\tilde{\mathbf{K}}}'\tilde{\mathbf{K}}}={{\tilde{\mathbf{L}}}'\tilde{\mathbf{L}}}={\tilde{\mathbf{L}}}{\tilde{\mathbf{L}}}'= \mathbf{I}_{p+m}$ and $\mathbf{1}_{n}^{\prime } {\hat{\mathbf{Z}}} = \mathbf{1}_{n}^{\prime }[\mathbf{K}$, ${\varvec{\Gamma }}_{\mathbf{G}}][\mathbf{L}$, ${\varvec{\Gamma }}_{\mathbf{H}}]^{\prime } = \mathbf{0}_{p+m}$: (A3) is the optimal with satisfying (4) and (5).

      Thus, we can obtain a factor score matrix Z with the following procedure:

Step 1.:

Generate G and H randomly

Step 2.:

Perform SVDs in (A1)

Step 3.:

Obtain ${\hat{\mathbf{Z}}}$ with (A3)

Step 4.:

Replace F in ${\hat{\mathbf{Z}}}= [{\hat{\mathbf{F}}},{\hat{\mathbf{U}}}$] by rotated ${\hat{\mathbf{F}}}{} \mathbf{T}$.

Here, each element of G and H is sampled randomly from the uniform distribution ranging from $-\,1$ to 1, and the matrix T is the orthonormal matrix for the varimax rotation giving the MDFA loadings in Table 2. The replication of the steps gives a number of ${\hat{\mathbf{Z}}}$.