Model-Implied Instrumental Variable—Generalized Method of Moments (MIIV-GMM) Estimators for Latent Variable Models

Abstract

The common maximum likelihood (ML) estimator for structural equation models (SEMs) has optimal asymptotic properties under ideal conditions (e.g., correct structure, no excess kurtosis, etc.) that are rarely met in practice. This paper proposes model-implied instrumental variable – generalized method of moments (MIIV-GMM) estimators for latent variable SEMs that are more robust than ML to violations of both the model structure and distributional assumptions. Under less demanding assumptions, the MIIV-GMM estimators are consistent, asymptotically unbiased, asymptotically normal, and have an asymptotic covariance matrix. They are “distribution-free,” robust to heteroscedasticity, and have overidentification goodness-of-fit J-tests with asymptotic chi-square distributions. In addition, MIIV-GMM estimators are “scalable” in that they can estimate and test the full model or any subset of equations, and hence allow better pinpointing of those parts of the model that fit and do not fit the data. An empirical example illustrates MIIV-GMM estimators. Two simulation studies explore their finite sample properties and find that they perform well across a range of sample sizes.

Appendices

Appendix A. Notation Example

In the section on “From Latent to Observed Variables,” we introduced the notation of y ^∗=ZA+u. To illustrate this notation, suppose that the first latent variable equation for the ith case is

$$ \eta _{1i}=\alpha _{\eta 1}+\beta _{12}\eta _{2i}+\gamma _{11}\xi _{1i}+\gamma _{12} \xi _{2i}+\zeta _{1i} $$

(A.1)

with y _1i , y _2i ,x _1i , and x _2i as the scaling indicators for η _1i ,η _2i ,ξ _1i , and ξ _2i , respectively. By replacing each latent variable by its scaling indicator minus its error [e.g., η _1i =(y _1i −ϵ _1i )], the observed variable counterpart to this first latent variable equation is

(A.2)

The Z ₁ for this first equation is

$$ \mathbf{Z}_{1}=\left [ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} 1 & y_{21} & x_{11} & x_{21} \\ 1 & y_{22} & x_{12} & x_{22} \\ \vdots & \vdots & \vdots & \vdots \\ 1 & y_{2N} & x_{1N} & x_{2N}\end{array} \right ] $$

(A.3)

and u ₁ is the vector

$$ \mathbf{u}_{1}=\left [ \begin{array}{c} u_{11} \\ u_{12} \\ \vdots \\ u_{1N}\end{array} \right ] =\left [ \begin{array}{c} -\beta _{12}\epsilon _{21}-\gamma _{11}\delta _{11}-\gamma _{12}\delta _{21}+\epsilon _{11}+\zeta _{11} \\ -\beta _{12}\epsilon _{22}-\gamma _{11}\delta _{12}-\gamma _{12}\delta _{22}+\epsilon _{12}+\zeta _{12} \\ \vdots \\ -\beta _{12}\epsilon _{2N}-\gamma _{11}\delta _{1N}-\gamma _{12}\delta _{2N}+\epsilon _{1N}+\zeta _{1N}\end{array} \right ] . $$

(A.4)

The Z ₂,…,Z _p+q−n are constructed in an analogous fashion.

In the full system of equations, the coefficient vector A contains all of the intercepts, factor loadings and other coefficients in the model. It is the partitioned vector

$$ \mathbf{A=}\left [ \begin{array}{c} \mathbf{A}_{1} \\ \mathbf{A}_{2} \\ \vdots \\ \mathbf{A}_{p+q-n}\end{array} \right ], $$

(A.5)

where A _j contains the intercept and coefficients for the jth equation in the model.

Continuing with the previous example of the equation for y _1i where y _1i depends on y _2i , x _1i , and x _2i ,

$$ \mathbf{A}_{1}=\left [ \begin{array}{c} \alpha _{\eta _{1}} \\ \beta _{12} \\ \gamma _{11} \\ \gamma _{12}\end{array} \right ]. $$

(A.6)

The other A _j vectors are formed in a similar way.

Each of the regression coefficients in the original LISREL-type model appears only once in A. Variance and covariance parameters are not estimated by MIIV-GMM. Also, A has a lot of structure, with zeroes corresponding to the lack of a direct effect of latent or observed variables. Thus, there is a one-to-one correspondence between entries of A, on one hand, and entries of the collection B,Γ,Λ _x , and Λ _y , on the other.

Appendix B. Selection of MIIVs

The basic process for selecting MIIVs starts with all observed variables in the model and eliminates as potential MIIVs any variables that are directly or indirectly influenced by the errors or unique factors that are part of or that are correlated with the composite disturbance for a given equation. The remaining variables are the MIIVs for the given equation. More specifically, finding the MIIVs involves the following steps:

• 1. Make a list of all observed variables in the model since these are the potential MIIVs;

• 2. Make a list of all errors or unique factors (ϵs, δs, or ζs) that are included in the composite disturbance term that is part of the equation of interest;

• 3. Eliminate any observed variable that is directly or indirectly influenced by the errors or unique factors noted in Step 2;

• 4. Eliminate any observed variable that is directly or indirectly influenced by an error or unique factor that is correlated with the errors or unique factors noted in Step 2;

• 5. The remaining observed variables are the MIIVs for the given equation.

This procedure can be implemented in several ways. One is by visual inspection of the path diagram of the model. Another is by looking at the reduced-form model for each observed variable and determining whether the disturbances or errors in question have an effect. Finally, Bollen and Bauer (2004) provide a SAS macro to implement this check, and Bauldry (forthcoming), an implementation of the same algorithm in Stata. In virtually all identified SEMs that we have examined, there are sufficient MIIVs to estimate all equations in the model. There is no need to search for additional observed variables once a researcher starts with an (over)identified model. This is different than the usual IV approach where a researcher searches for auxiliary IVs that were not part of the original structure (Bollen 2012).

Appendix C. GMM Theory and Technical Aspects

This Appendix outlines the general theory of the generalized method of moments estimates. It restates the results given in the original development of Hansen (1982), as well as in comprehensive reviews such as Hall (2005) and Newey and McFadden (1986).

Given the p-variate data vector z and model parameters θ, the generalized method of moments works with q-variate vector of functions g(z,θ) that combine the data and parameters in such a way that in the population,

$$ \mathrm{E}\bigl[ g(\mathbf{z},\boldsymbol{\theta}_0) \bigr] = 0 $$

(C.1)

for the unique “true” value θ ₀. Equations (C.1) are typically referred to as “moment conditions” in economics, or “estimating equations” in statistics (van der Vaart 1998; Godambe & Thompson, 1978).

The GMM proceeds as follows. First, the sample analogues to the estimating equations are formed:

$$ g_N(\boldsymbol{\theta}) = \frac{1}{N} \sum _{i=1}^N g(\mathbf{z}_i,\boldsymbol{ \theta}). $$

(C.2)

Second, these estimating equations are collected together into a quadratic form:

$$ Q_N(\boldsymbol{\theta};\mathbf{Z},W) = g_N'( \boldsymbol{\theta}) \mathbf{W}_N g_N(\boldsymbol{ \theta}), $$

(C.3)

where W _N is a conforming q×q weight matrix, possibly obtained from the data. Third, this quadratic form is minimized with respect to θ to obtain the parameter estimates:

$$ \widehat{\boldsymbol{\theta}}_N = \arg \min_{\boldsymbol{\theta}} Q_N(\boldsymbol{\theta};\mathbf{Z},{\mathbf{W}}_N). $$

(C.4)

Thus, a GMM estimator $\widehat{\boldsymbol{\theta}}_{N}$ is defined by a combination of the estimating equations g(z,θ) and the weight matrix W. Conceptually, both of them are at researcher discretion. However, generally the estimating equations are strongly determined by the model of interest (including our case of the model-implied instrumental variables), and some choices of the weight matrix W are obviously better than others, as explained below.

In the MIIV-GMM methodology discussed in this paper, the estimating equations are given by Equation (16):

$$g(\mathbf{z}_i,\boldsymbol{\theta}) = \mathbf{V}_{i}^{\prime }\bigl(\mathbf{y}_{i}^{\ast }-\mathbf{Z}_{i}\mathbf{A}\bigr). $$

After algebraic simplifications, these estimating equations reduce to linear combinations of cross-products ξ _i ζ _i , ξ _i ϵ _i , ξ _i δ _i , ζ _i ϵ _i , ζ _i δ _i and ϵ _i δ _i . Since these pairs of variables are assumed to be uncorrelated, the estimating equations have indeed zero expectations, as required by (C.1).

The desirable properties of the GMM estimates include consistency, asymptotic normality and, with an optimal choice of the weight matrix W _N , asymptotic efficiency. Also, specification tests of whether the assumptions (C.1) are supported by the data are available. All of these results are asymptotic, and their justification requires certain regularity conditions.

Consistency of the GMM estimates (Newey & McFadden, 1986, Theorem 2.6, p. 2132; Hall 2005, Theorem 3.1, p. 68) is obtained under the following conditions:

• 1. z _i ∼ i.i.d.;

• 2. ${\mathbf{W}}_{N} \stackrel{p}{\rightarrow} \mathbf{W}$ (Hall 2005, Assumption 3.7);

• 3. W is positive semidefinite (Hall 2005, Assumption 3.7);

• 4. WE[g(z,θ)]=0 iff θ=θ ₀ (Hall 2005, Assumptions 3.3 and 3.4);

• 5. $\boldsymbol{\theta}_{0} \in \operatorname{int}\boldsymbol{\Theta} \in R^{p}$ (Hall 2005, Assumption 3.5);

• 6. Θ is compact (Hall 2005, Assumption 3.8);

• 7. g(z,θ) is continuous at each θ with probability 1 (Hall 2005, Assumption 3.2);

• 8. E[sup _θ ∥g(z,θ)∥]<∞ (Hall 2005, Assumptions 3.2 and 3.10).

Instead of Condition 1, Hall (2005) uses a weaker conditions of strict stationarity and ergodicity of the data (Assumption 3.1), in which case i is the time index. Hall (2005) Assumption 3.1 also allows for heteroscedasticity of the measurement errors and unique variances.

Let us apply these conditions to the MIIV-GMM framework. The conditions on the weight matrix are satisfied for all the matrices we consider in this paper. The fourth condition is satisfied when the model is identified. Continuity of the estimating equations is trivial, as they are linear in the parameters. Finally, the last condition on E[sup _θ ∥g(z,θ)∥] is satisfied under the fourth-order cross-moments condition given in the section on the model and assumptions.

Asymptotic normality additionally requires the following conditions (Newey & McFadden, 1986, Theorem 3.4, p. 2148; Hall 2005, Theorem 3.2, p. 71):

• 9. g(z,θ) is continuously differentiable in the neighborhood of θ ₀ with probability approaching 1 (Hall 2005, Assumptions 3.5 and 3.12);

• 10. E[g(z,θ ₀)]=0 and E[∥g(z,θ ₀)∥²]<∞ (Hall 2005, Assumption 3.11);

• 11. E[sup _θ ∥∇ _θ g(z,θ)∥]<∞ (Hall 2005, Assumption 3.2);

• 12. $\operatorname{rank}E[ \nabla_{\boldsymbol{\theta}} g(\mathbf{z},\boldsymbol{\theta}_{0}) ] = p = \operatorname{dim}\boldsymbol{\Theta}$ (Hall 2005, Assumption 3.6);

• 13. G′WG is non-singular for G=E[∇ _θ g(z,θ)];

• 14. $\sup_{\boldsymbol{\theta}} \| \frac{1}{n} \sum_{i} \nabla_{\boldsymbol{\theta}} g(\mathbf{z}_{i},\boldsymbol{\theta}) - \mathrm{E}[\nabla_{\boldsymbol{\theta}} g(\mathbf{z},\boldsymbol{\theta})] \| \stackrel{p}{\rightarrow} \mathbf{0}$ (Hall 2005, Assumption 2.13).

Let us apply these conditions to the MIIV-GMM framework. Smoothness of g(z,θ) is trivial since g(z,θ) is linear in θ. Finite second moment of g(z,θ ₀) is ensured by the fourth-order cross-moments condition given in the section on the model and assumptions. For the estimating equations given by (16), the gradients with respect to the parameters A are given by

$$ \nabla_{\boldsymbol{\theta}} g(\mathbf{z},\boldsymbol{\theta}) = - \mathbf{V}_{i}^{\prime } \mathbf{Z}_{i}. $$

(C.5)

Finiteness of its absolute value follows from the finiteness of the second moment of the data. The condition on the matrix G′WG is one of the conditions for the estimator (18) to be properly defined. The condition on the rank of the moment derivative matrix is similar to the condition of nondegenerate Jacobian in the likelihood context, and is satisfied whenever there are no perfectly collinear dependent variables in the model. The last condition is satisfied for i.i.d. data by virtue of the central limit theorem (CLT) for the derivatives of the moment conditions, since the first terms are of order O _p (n ^−1/2).

Under the same set of conditions, the asymptotic variance estimator with GMM estimates plugged in for the population parameters is consistent for the target variance (Newey & McFadden, 1986, Theorem 4.5, p. 2160; Hall 2005, Section 3.5.1) when the data are i.i.d. For dependent or heteroscedastic data, one additionally needs (Hall 2005, Section 3.5.3):

• 15. sup _θ E∥∂ ² g(z _i ,θ)/∂ θ _j ∂ θ _k ∥<∞ for all j,k in a neighborhood of θ ₀.

As is easily seen, Equations (C.5) do not involve the parameters explicitly or implicitly, so this condition is easily satisfied for MIIV-GMM. Alternatively, if heteroscedasticity is a function of observed or unobserved variables present in the model, as is the case in the simulations, the expected values operators in the population conditions (C.1) and (15) will include integration over the variables that cause heteroscedasticity. After this integration is performed, the data form a skewed and kurtotic distribution. In our first simulation, this is demonstrated by the highly significant results of Mardia’s test.

Asymptotic efficiency is achieved with an optimal choice of the weight matrix W _N . Namely, W _N needs to converge to the asymptotic variance of the estimating equations g(z,θ ₀). The result is given in Theorem 5.2 of Newey and McFadden (1986, p. 2165) and Theorem 3.4 of Hall (2005, p. 88), and does not require any additional assumptions beyond those necessary for asymptotic normality of the estimates.

A distinction should be made of the use of the term “moments” in the three literatures related to the current paper. In the statistics literature, a “moment” is universally understood as the expected value of a power (most typically, a positive integer power) of a random variable X, possibly centered, i.e., E[X ^k] or E[(X−μ)^k] where μ=E[X], or their sample analogues. In the covariance modeling approach to structural equation modeling, “moments” refer to covariances σ _jk =E[(X _j −μ _j )(X _k −μ _k )]. A further distinction is made of the sample, population, and implied moments. In the econometrics literature, the term “moment” is used more loosely to indicate any relation between (vector-valued) data X and (vector-valued) parameter θ such that E[g(X,θ)]=0. This generalization covers the standard uses: (i) E[X−μ]=0 for the population mean; (ii) E[X ²−μ ²−σ ²]=0 for the population variance; (iii) E[(X _j −μ _j )(X _k −μ _k )−σ _jk (θ)]=0 for the covariance structure models. It also allows for other uses, such as the normal equations in regression, E[x _j (y−x′β)]=0, or the instrumental variables orthogonality conditions, E[z _k (y−x′β)]=0. At this level of generality, the econometric “moments” have the same meaning as “estimating equations” in statistics, the point we made in Sections 2 and 5.1. As the impetus for this paper comes from bringing the econometric ideas into latent variable modeling, we use the term “moment” in the latter, econometric, sense to denote the functions of data and parameters. For the MIIV-GMM application, the moments we use in estimation are given by Equation (15), in which the parameters are implicitly present in the composite error u.

Another way to look at these terminology distinctions is to observe that the covariance structure methods, such as MLE, ADF and other least square methods, make multiple steps from (i) setting up the model, as is done in Section 2, to (ii) deriving the implied second moments to (iii) minimizing the discrepancy between the sample and the implied moments to (iv) forming the variances of the sample moments to (v) utilizing the delta method to derive the standard errors of the parameter estimates. MIIV-GMM is, in fact, more straightforward, as it uses the equations from the latent variable and the measurement model with some minimal transformations, and obtains the standard errors explicitly with analytically available formulae that do not involve any derivatives. Given its greater simplicity, it is not surprising that the method works quite well in small samples even despite severe nonnormality of the data.