Exploratory Procedure for Component-Based Structural Equation Modeling for Simple Structure by Simultaneous Rotation

Abstract

Generalized structured component analysis (GSCA) is a structural equation modeling (SEM) procedure that constructs components by weighted sums of observed variables and confirmatorily examines their regressional relationship. The research proposes an exploratory version of GSCA, called exploratory GSCA (EGSCA). EGSCA is analogous to exploratory SEM (ESEM) developed as an exploratory factor-based SEM procedure, which seeks the relationships between the observed variables and the components by orthogonal rotation of the parameter matrices. The indeterminacy of orthogonal rotation in GSCA is first shown as a theoretical support of the proposed method. The whole EGSCA procedure is then presented, together with a new rotational algorithm specialized to EGSCA, which aims at simultaneous simplification of all parameter matrices. Two numerical simulation studies revealed that EGSCA with the following rotation successfully recovered the true values of the parameter matrices and was superior to the existing GSCA procedure. EGSCA was applied to two real datasets, and the model suggested by the EGSCA’s result was shown to be better than the model proposed by previous research, which demonstrates the effectiveness of EGSCA in model exploration.

Appendices

Appendix A

In this appendix, we consider the minimization problem

$$\begin{aligned} \phi= & {} ||\textbf{Z}_1 - \textbf{Z}_1\textbf{W}_1\textbf{C}_1||^2 + ||\textbf{Z}_2 - \textbf{Z}_2\textbf{W}_2\textbf{C}_2||^2 \nonumber \\{} & {} + ||\textbf{Z}_1\textbf{W}_1 - \textbf{Z}_2\textbf{W}_2\textbf{B}_2||^2 + ||\textbf{Z}_2\textbf{W}_2 - \textbf{Z}_1\textbf{W}_1\textbf{B}_1||^2 \end{aligned}$$

(38)

over $\textbf{W}_1$,$\textbf{W}_2$,$\textbf{C}_1$,$\textbf{C}_2$,$\textbf{B}_1$, and $\textbf{B}_2$, subject to the constraint

$$\begin{aligned} \textbf{W}_1^{\prime }{} \textbf{Z}_1^{\prime }{} \textbf{Z}_1\textbf{W}_1 = \textbf{I}_{D_1},\ \textbf{W}_2^{\prime }{} \textbf{Z}_2^{\prime }{} \textbf{Z}_2\textbf{W}_2 = \textbf{I}_{D_2}. \end{aligned}$$

(39)

The loss function in (38) is equivalent to the minimization of (10) with $\tilde{\textbf{B}}_1$ and $\tilde{\textbf{B}}_2$ being the matrices filled with zeros, as assumed in the EGSCA procedure. The component score is constrained to be orthogonal within the variable set in order to avoid the standardization issue after the rotation of the parameter matrices. The estimated parameter matrices are used for an initial solution for the EGSCA procedure. All elements in the parameter matrices are treated as free parameters to be estimated, while some are fixed in the conventional GSCA procedure.

The following iterative algorithm is used for minimizing (38).

• 1. Initialize $\textbf{W}_2$, $\textbf{C}_1$, $\textbf{C}_2$, $\textbf{B}_1$, and $\textbf{B}_2$.

• 2. Update $\textbf{W}_1$ by the one minimizing $\phi$ with other parameter matrices kept fixed.

• 3. Update $\textbf{W}_2$ by the one minimizing $\phi$ with other parameter matrices kept fixed.

• 4. Update $\textbf{C}_1$ and $\textbf{C}_2$ by the ones minimizing $\phi$ with other parameter matrices kept fixed.

• 5. Update $\textbf{B}_1$ and $\textbf{B}_2$ by the ones minimizing $\phi$ with other parameter matrices kept fixed.

• 6. Terminate the algorithm if the decrement of $\phi$ value is less than $\epsilon$, otherwise go back to Step 2.

The loss function is guaranteed to decrease at Steps 2–5, and the algorithm starts from $M_{ST}$ initial values to avoid local minimum. $M_{ST} = 100$ and $\epsilon = 1.0 \times 10^{-6}$ were used for all the simulation studies and the applications.

The update formulae for the Steps 2–5 are presented in the following.

First, consider to minimize $\phi$ over $\textbf{W}_1$ subject to (39). $\phi$ is expanded as

$$\begin{aligned} \phi= & {} -2\textrm{tr}\textbf{Z}_1^{\prime }{} \textbf{Z}_1\textbf{W}_1\textbf{C}_1 + \textrm{tr}\textbf{C}_1^{\prime }{} \textbf{C}_1 -2\textrm{tr}\textbf{Z}_2^{\prime }{} \textbf{Z}_2\textbf{W}_2\textbf{C}_2 + \textrm{tr}\textbf{C}_2^{\prime }{} \textbf{C}_2 \nonumber \\&\ {}&-2\textrm{tr}\textbf{W}_1^{\prime }{} \textbf{Z}_1^{\prime }{} \textbf{Z}_2\textbf{W}_2\textbf{B}_2 + \textrm{tr}\textbf{B}_2^{\prime }{} \textbf{B}_2 -2\textrm{tr}\textbf{W}_2^{\prime }{} \textbf{Z}_2^{\prime }{} \textbf{Z}_1\textbf{W}_1\textbf{B}_1 + \textrm{tr}\textbf{B}_1^{\prime }{} \textbf{B}_1 + c \end{aligned}$$

(40)

where c denotes the constant irrelevant to the parameter matrices. (40) indicates that the minimization of $\phi$ over $\textbf{W}_1$ is equivalent to maximize

$$\begin{aligned} \phi _{\textbf{W}_1}= & {} \textrm{tr}\textbf{Z}_1^{\prime }{} \textbf{Z}_1\textbf{W}_1\textbf{C}_1 + \textbf{W}_1^{\prime }{} \textbf{Z}_1^{\prime }{} \textbf{Z}_2\textbf{W}_2\textbf{B}_2 + \textrm{tr}\textbf{W}_2^{\prime }{} \textbf{Z}_2^{\prime }{} \textbf{Z}_1\textbf{W}_1\textbf{B}_1 \nonumber \\= & {} \textrm{tr}\textbf{W}_1^{\prime }{} \textbf{M}_1 \end{aligned}$$

(41)

where $\textbf{M}_1 = \textbf{Z}_1^{\prime }{} \textbf{Z}_1\textbf{C}_1^{\prime } + \textbf{Z}_1^{\prime }{} \textbf{Z}_2\textbf{W}_2(\textbf{B}_1^{\prime } + \textbf{B}_2)$ with the dimensionality of $J_1 \times D_1$. Here, using $\tilde{\textbf{W}}_1 = (\textbf{Z}_1^{\prime }{} \textbf{Z}_1)^{1/2}{} \textbf{W}_1$, the first constraint in (39) can be rewritten as

$$\begin{aligned} \tilde{\textbf{W}}_1^{\prime }\tilde{\textbf{W}}_1 = \textbf{I}_{D_1} \end{aligned}$$

(42)

and $\phi _{\textbf{W}_1}$ becomes

$$\begin{aligned} \phi _{\textbf{W}_1} = \textrm{tr}\tilde{\textbf{W}}_1^{\prime }(\textbf{Z}_1^{\prime }{} \textbf{Z})^{-1/2}{} \textbf{M}_1 \end{aligned}$$

(43)

Thus, the minimization of $\phi$ over $\textbf{W}_1^{\prime }\textbf{Z}_1^{\prime }{} \textbf{Z}_1\textbf{W}_1 = \textbf{I}_{D_1}$ is equivalent to maximizing $\phi _{\textbf{W}_1}$ over the column-orthonormal matrix $\tilde{\textbf{W}}_1$. $\phi _{\textbf{W}_1}$ is maximized as

$$\begin{aligned} \phi _{\textbf{W}_1} = \textrm{tr}\tilde{\textbf{W}}_1^{\prime }{} \textbf{K}_{1}\mathbf{\Lambda }_{1}{} \textbf{L}_{1}^{\prime } \le \textrm{tr}{\varvec{\Lambda }}_{1} \end{aligned}$$

(44)

using the singular value decomposition

$$\begin{aligned} (\textbf{Z}_1^{\prime }{} \textbf{Z}_1)^{-1/2}{} \textbf{M}_1 = \textbf{K}_{1}{\varvec{\Lambda }}_{1}{} \textbf{L}_{1}^{\prime } \end{aligned}$$

(45)

where $\textbf{K}_{1}$ and $\textbf{L}_{1}$ are the matrices of the left and right singular vectors of $(\textbf{Z}_1^{\prime }{} \textbf{Z})^{-1/2}\textbf{M}_1$, respectively, and ${\varvec{\Lambda }}_{1}$ is the diagonal matrix of the singular value arranged in descending order (Ten Berge, 1993). The equality in (45) holds when $\tilde{\textbf{W}}_1 = \textbf{K}_{1}{} \textbf{L}_{1}^{\prime }$ which leads

$$\begin{aligned} \textbf{W}_1 = (\textbf{Z}_1^{\prime }{} \textbf{Z}_1)^{-1/2}{} \textbf{K}_{1}{} \textbf{L}_{1}^{\prime } \end{aligned}$$

(46)

as the update formula for $\textbf{W}_1$ in Step 2.

The update formula for the subsequent step is similarly given by

$$\begin{aligned} \textbf{W}_2 = (\textbf{Z}_2^{\prime }{} \textbf{Z}_2)^{-1/2}{} \textbf{K}_{2}{} \textbf{L}_{2}^{\prime } \end{aligned}$$

(47)

where the columns of $\textbf{K}_2$ and $\textbf{L}_2$ are the left and right singular vectors of $(\textbf{Z}_2^{\prime }{} \textbf{Z}_2)^{-1/2}(\textbf{Z}_2^{\prime }{} \textbf{Z}_2\textbf{C}_2^{\prime } + \textbf{Z}_2^{\prime }{} \textbf{Z}_1\textbf{W}_1(\textbf{B}_1 + \textbf{B}_2^{\prime }))$, respectively.

The $\textbf{C}_1$ and $\textbf{C}_2$ minimizing $\phi$ is simply obtained by the multivariate regression;

$$\begin{aligned} \textbf{C}_1 = \textbf{W}_1^{\prime }{} \textbf{Z}_1^{\prime }{} \textbf{Z}_1,\ \textbf{C}_2 = \textbf{W}_2^{\prime }{} \textbf{Z}_2^{\prime }{} \textbf{Z}_1. \end{aligned}$$

(48)

The update formulae in Step 5 are also given by

$$\begin{aligned} \textbf{B}_1 = \textbf{W}_1^{\prime }{} \textbf{Z}_1^{\prime }{} \textbf{Z}_2\textbf{W}_2,\ \textbf{B}_2 = \textbf{W}_2^{\prime }{} \textbf{Z}_2^{\prime }{} \textbf{Z}_1\textbf{W}_1. \end{aligned}$$

(49)

The second example in the fourth section fixes $\textbf{B}_2$ as $_{D_2}{} \textbf{O}_{D_1}$, and it is accomplished by setting $\textbf{B}_2 = \ _{D_2}{} \textbf{O}_{D_1}$ in Step 1, and suppressing the update of $\textbf{B}_2$ in Step 5.

Appendix B

Table 10 Covariance matrix of error terms in numerical simulation.