Consistent Partial Least Squares for Nonlinear Structural Equation Models

Abstract

Partial Least Squares as applied to models with latent variables, measured indirectly by indicators, is well-known to be inconsistent. The linear compounds of indicators that PLS substitutes for the latent variables do not obey the equations that the latter satisfy. We propose simple, non-iterative corrections leading to consistent and asymptotically normal (CAN)-estimators for the loadings and for the correlations between the latent variables. Moreover, we show how to obtain CAN-estimators for the parameters of structural recursive systems of equations, containing linear and interaction terms, without the need to specify a particular joint distribution. If quadratic and higher order terms are included, the approach will produce CAN-estimators as well when predictor variables and error terms are jointly normal. We compare the adjusted PLS, denoted by PLSc, with Latent Moderated Structural Equations (LMS), using Monte Carlo studies and an empirical application.

Appendices

Appendix A

Here we derive Equation (23). The other moment equations are derived in a similar, usually even simpler, way.

Equation (23) says

$$ E\overline{\eta}_{i}^{2}\overline{\eta }_{j}^{2}=Q_{i}^{2}Q_{j}^{2} \cdot\bigl( E\eta_{i}^{2}\eta_{j}^{2}-1 \bigr) +1. $$

Recall that $\overline{\eta}_{i}=Q_{i}\eta_{i}+\delta_{i}$ where the δ’s are mutually independent, and independent of the η’s. The Q’s are real numbers from [−1,1]. The η’s and $\overline {\eta}$’s have zero mean and unit variance. The δ’s have a zero mean as well. Note that $E\delta_{i}^{2}=1-Q_{i}^{2}$. We have

$$\begin{aligned} E\overline{\eta}_{i}^{2}\overline{\eta }_{j}^{2} =&E \bigl[ ( Q_{i}\eta _{i}+\delta_{i} ) ^{2}\cdot( Q_{j} \eta_{j}+\delta_{j} ) ^{2} \bigr] \\ =&E \bigl[ \bigl( Q_{i}^{2}\eta_{i}^{2}+ \delta_{i}^{2}+2Q_{i}\eta_{i}\delta _{i} \bigr) \cdot\bigl( Q_{j}^{2}\eta _{j}^{2}+\delta_{j}^{2}+2Q_{j} \eta_{j}\delta_{j} \bigr) \bigr] . \end{aligned}$$

The expected value consists of three parts. The first part equals

$$\begin{aligned} E \bigl[ Q_{i}^{2}\eta_{i}^{2}\cdot \bigl( Q_{j}^{2}\eta_{j}^{2}+\delta _{j}^{2}+2Q_{j}\eta_{j}\delta _{j} \bigr) \bigr] =&Q_{i}^{2}Q_{j}^{2}E\eta _{i}^{2}\eta_{j}^{2}+Q_{i}^{2}E \eta_{i}^{2}\delta_{j}^{2}+2Q_{i}^{2}Q_{j}E \eta_{i}^{2}\eta_{j}\delta_{j} \\ =&Q_{i}^{2}Q_{j}^{2}E\eta _{i}^{2}\eta_{j}^{2}+Q_{i}^{2} \cdot1\cdot\bigl( 1-Q_{j}^{2} \bigr) +0. \end{aligned}$$

The second part equals

$$\begin{aligned} E \bigl[ \delta_{i}^{2}\cdot\bigl( Q_{j}^{2} \eta_{j}^{2}+\delta_{j}^{2}+2Q_{j} \eta_{j}\delta_{j} \bigr) \bigr] =&Q_{j}^{2}E\delta_{i}^{2}\eta _{j}^{2} +E\delta_{i}^{2}\delta _{j}^{2} +2Q_{j}E\delta_{i}^{2} \eta_{j}\delta_{j} \\ =&Q_{j}^{2} \bigl( 1-Q_{i}^{2} \bigr) \cdot1+ \bigl( 1-Q_{i}^{2} \bigr) \bigl( 1-Q_{j}^{2} \bigr) +0. \end{aligned}$$

And the last part is

$$\begin{aligned} E \bigl[ 2Q_{i}\eta_{i}\delta_{i}\cdot \bigl( Q_{j}^{2}\eta_{j}^{2}+\delta _{j}^{2}+2Q_{j}\eta_{j}\delta _{j} \bigr) \bigr] =&2Q_{i}E \bigl[ \delta_{i}\cdot\bigl( \eta _{i} \bigl( Q_{j}^{2}\eta_{j}^{2}+ \delta_{j}^{2}+2Q_{j}\eta_{j}\delta _{j} \bigr) \bigr) \bigr] =0. \end{aligned}$$

Collecting terms yields

$$\begin{aligned} E\overline{\eta}_{i}^{2}\overline{\eta }_{j}^{2} =&Q_{i}^{2}Q_{j}^{2}E \eta_{i}^{2}\eta_{j}^{2}+ Q_{i}^{2} \bigl( 1-Q_{j}^{2} \bigr) +Q_{j}^{2} \bigl( 1-Q_{i}^{2} \bigr) + \bigl( 1-Q_{i}^{2} \bigr) \bigl( 1-Q_{j}^{2} \bigr) \\ =&Q_{i}^{2}Q_{j}^{2}E\eta _{i}^{2}\eta_{j}^{2}+1-Q_{i}^{2}Q_{j}^{2} \end{aligned}$$

as claimed.

Appendix B. Stein’s Identity

The normal distribution has many interesting (and characterizing) properties. One of them is the property known as Stein’s identity. This tells us that when X is a p-dimensional normal vector with mean μ and covariance matrix Ω, then f(X), where f(.) is a real valued smooth function, satisfies (cf. Liu, 1994):

$$ \operatorname{cov} \bigl( X,f ( X ) \bigr) =\varOmega E \bigl[ \nabla f ( X ) \bigr] . $$

(B.1)

Here ∇f(.) is f(.)’s gradient. So when Ω is invertible, the regression of f(X) on X equals

$$ f ( X ) =Ef ( X ) +E \bigl[ \nabla f ( X ) \bigr] ^{\intercal} \cdot( X-\mu) +\text{regression residual}. $$

(B.2)

Consequently, the regression coefficients are the average first partial derivatives. We add here an extension of Stein’s identity that covers the quadratic case:

$$\begin{aligned} f ( X ) =&Ef ( X ) +E \bigl[ \nabla f ( X ) \bigr] ^{\intercal} \cdot( X-\mu) \\ &{}+1/2\cdot \operatorname{trace} \bigl[ E \bigl( H ( X ) \bigr) \cdot\bigl( ( X-\mu) (X-\mu) ^{\intercal}-\varOmega\bigr) \bigr] +\nu \end{aligned}$$

(B.3)

where H(.) is the Hessian of f(.) and ν is the regression residual. So the regression coefficients for the interaction and quadratic terms (apart from the multiple ${\frac{1}{2}}$) are average second order partial derivatives. They are estimated consistently when a quadratic specification is used instead of the true nonlinear function. (There does not appear to be a comparably neat expression for higher order partial derivatives.)

We will prove (B.3), assuming the existence of partial derivatives and boundedness of their expected absolute values.

Write $X=\mu+\varOmega^{{\frac{1}{2}}}Z$, where Z is p-dimensional standard normal. Consider first, for a real function g(.) from the same class as f(.), the regression of g(Z)−Eg(Z) on Z and the squares and cross-products of its components. The covariance matrix of the regressors is diagonal, with ones everywhere, except at the entries corresponding with the squares where we have 2. So the regression coefficient of Z _i equals EZ _i g(Z)=E∇g(Z) _i by Stein’s identity. For twice the coefficient of $Z_{i}^{2}-1$ we get

$$\begin{aligned} E \bigl( Z_{i}^{2}-1 \bigr) g ( Z ) =&EZ_{i}^{2}g ( Z ) -Eg ( Z ) \end{aligned}$$

(B.4)

$$\begin{aligned} =& EZ_{i} \bigl( Z_{i}g ( Z ) \bigr) -Eg(Z) = E \bigl( g ( Z)+Z_{i}\nabla g ( Z ) _{i} \bigr) -Eg ( Z ) \end{aligned}$$

(B.5)

$$\begin{aligned} =& E \bigl( Z_{i}\nabla g ( Z ) _{i} \bigr) = EH_{ii} ( Z ) . \end{aligned}$$

(B.6)

On the second line we applied Stein’s identity to Z _i g(Z) and on the third line to ∇g(Z) _i . One obtains similarly for the coefficient of a cross-product (i≠j):

$$ EZ_{i} \bigl( Z_{j}g ( Z ) \bigr) =E \bigl( Z_{j}\nabla g ( Z ) _{i} \bigr) =EH_{ij} ( Z ) . $$

(B.7)

Collecting terms yields (with H subscripted by g for identification):

$$ g(Z)=Eg(Z)+E\nabla g(Z)^{\intercal}\cdot Z+1/2\cdot\text{trace} \bigl[ EH_{g}(Z)\cdot\bigl( ZZ^{\intercal}-I \bigr) \bigr] +\nu. $$

(B.8)

Finally take a smooth function f(.) of X, and define $g ( Z ) :=f ( \mu+\varOmega^{{\frac{1}{2}}}Z )$. A substitution of $Z=\varOmega^{-{\frac{1}{2}}} ( X-\mu )$, $\nabla g ( Z ) =\varOmega ^{{\frac{1}{2}}}\nabla f(X)$, and $H_{g}(Z)=\varOmega^{{\frac {1}{2}}}H_{f}(X)\varOmega^{{\frac{1}{2}}}$ into (B.8) yields the desired expression for general f(X).