Using SAS PROC CALIS to fit Level-1 error covariance structures of latent growth models

Abstract

In the present article, we demonstrates the use of SAS PROC CALIS to fit various types of Level-1 error covariance structures of latent growth models (LGM). Advantages of the SEM approach, on which PROC CALIS is based, include the capabilities of modeling the change over time for latent constructs, measured by multiple indicators; embedding LGM into a larger latent variable model; incorporating measurement models for latent predictors; and better assessing model fit and the flexibility in specifying error covariance structures. The strength of PROC CALIS is always accompanied with technical coding work, which needs to be specifically addressed. We provide a tutorial on the SAS syntax for modeling the growth of a manifest variable and the growth of a latent construct, focusing the documentation on the specification of Level-1 error covariance structures. Illustrations are conducted with the data generated from two given latent growth models. The coding provided is helpful when the growth model has been well determined and the Level-1 error covariance structure is to be identified.

The latent growth model (LGM) plays an important role in repeated measure analysis over a limited number of occasions in large samples (e.g., Meredith & Tisak, 1990; Muthén & Khoo, 1998; Preacher, Wichman, MacCallum, & Briggs, 2008, p. 12). The model can not only characterize intraindividual (within subjects) change over time, but also examine interindividual (between subjects) difference by means of random growth coefficients, and is a typical application of hierarchical linear modeling (HLM). The within-subjects errors over time and the between-subjects errors are conventionally referred to as “Level-1” and “Level-2” errors, respectively. LGM can also be handled by using structural equation modeling (SEM) (e.g., Bauer, 2003; Bollen & Curran, 2006; Chan, 1998; Curran, 2003; Duncan, Duncan, & Hops, 1996; Mehta & Neal 2005; Meredith & Tisak, 1990; Willet & Sayer, 1994). SEM and HLM stem from different statistical theory, and each has developed its own terminology and standard ways of framing research questions. However, there exists much overlap between the two methodologies under some circumstances. Typically, when a two-level data structure arises from the repeated observations of a variable over time for a set of individuals (so that time is hierarchically nested within each individual), SEM is analytically equivalent to HLM (e.g., Bauer, 2003; Bovaird, 2007; Curran, 2003; MacCallum, Kim, Malarkey, & Kiecolt-Glaser, 1997; Raudenbush, 2001; Rovine & Molenaar, 2000; Willett & Sayer, 1994). The SEM approach provides advantages over the HLM approach in examining model fit; modeling the change over time for latent constructs, with the curve-of-factors model; embedding LGM into a larger latent variable model, with the factor-of-curves model; and incorporating measurement models for latent predictors (e.g., Bauer, 2003; Bollen & Curran, 2006, Chap. 7, 8; Bovaird, 2007; Chan, 1998; Curran, 2003; Duncan, Duncan, & Strycker, 2006, Chap. 4; MacCallum et al., 1997; Raudenbush, 2001; Rovine & Molenaar, 2000). However, the SEM approach suffers from a tedious and error-prone data management task. Many steps are needed to properly structure the data, and the SEM code quickly becomes unwieldy. In contrast, the HLM approach allows for simpler model specification, is computationally more efficient, and can easily be expanded to higher level growth models for manifest variables (Curran, 2003; Wu, West, & Taylor, 2009). A detailed comparison between HLM and SEM can be seen in Bauer (2003) and Curran (2003).

Specialized software for SEM such as EQS (Bentler & Wu, 2005), LISREL (Jöreskog & Sörbom, 2001), Mplus (Muthén & Muthén, 2007), Mx (Neale, Boker, Xie, & Maes, 2003), and SAS PROC CALIS (SAS Institute Inc., 2010) are readily available. HLM (Raudenbush, Bryk, & Congdon, 2005), MLwiN (Rasbash et al., 2000), and SAS PROC MIXED (SAS Institute Inc., 2010) are typical software for HLM. Because of the isomorphism between SEM and HLM for the same growth model, parameter estimates with SEM and those with HLM should be equivalent. Any minor variations can be attributed to different computational methods used (standard maximum likelihood [ML] estimation or full information maximum likelihood [FIML] estimation for SEM, and restricted maximum-likelihood estimation for HLM). Relevant discussions have been given in Bauer (2003), Bovaird (2007), Curran (2003), and Mehta and Neale (2005).

Level-1 errors could be autocorrelated. Autocorrelations, considered to be nuisance parameters, might result from carryover effects, memory effects, practice effects, or other unmodeled associations, and might not be present when a more complex model or a more appropriate time structure is used (Grimm & Widaman, 2010; Sivo & Fan, 2008). For example, the growth curve ARMA (p, q) model has been proposed to absorb error autocorrelations (e.g., Sivo, Fan, & Witta, 2005; Sivo & Fan, 2008). When Level-1 errors are autocorrelated, misspecification of their covariance structures has a substantial impact on the inference for model parameters (Ferron, Dailey, & Yi, 2002; Kwok, West, & Green, 2007; Murphy & Pituch, 2009). However, correct covariance structure is difficult to specify by theory (Kwok et al., 2007, p. 588). Therefore, a specification search becomes needed. Littell, Milliken, Stroup, Wolfinger, & Schabenberger (2006, Chap. 5) illustrated two types of tools with SAS PROC MIXED to help select a covariance structure. First are graphical tools to visualize correlation patterns among residuals. Second are information criteria measuring the relative fit of competing covariance structures. AIC (Akaike, 1974) and BIC (Schwarz, 1978) are commonly used descriptive measures. The model that minimizes AIC or BIC is preferred. Before using these methods, researchers should first rule out covariance structures that are obviously inconsistent with the characteristics of the data. On the other hand, although linear growth curve models are often fitted because of their ease in estimation, theory may suggest that more complex growth models be used, since they can better capture developmental patterns. Correctly specifying the growth model might lead to a simple covariance structure (Grimm & Widaman, 2010). Moreover, when the growth model is misspecified, statistical inference during the search process can be misleading (Yuan & Bentler, 2004). Therefore, the growth model should be well determined before searching for an “optimal” covariance structure for Level-1 errors.

A variety of processes underlying Level-1 errors may be specified (e.g., Newsom, 2002; Singer & Willett, 2003, Chap. 7; Wolfinger, 1996). SAS PROC MIXED contains more than 30 different types of Level-1 preprogrammed error processes. However, some important processes are unavailable, and any modification of existing processes is not allowed. In contrast, there exists much flexibility in PROC CALIS when specifying error covariance structures. For example, the second-order autoregressive process, not available in PROC MIXED, can be handled with PROC CALIS. The strength of PROC CALIS is always accompanied with technical coding work, which needs to be specifically addressed, and is the focus of this article. In addition to PROC CALIS, any comparable SEM software could be used.

There seems to be no commonly acceptable criteria for assessing model fit according to the indices such as AIC and BIC resulting from PROC MIXED. In contrast, there is some agreement on the cut-off criteria of conventional fit indices based on the likelihood ratio test in SEM, such as RMSEA, CFI, and NNFI (TLI) (e.g., Hu & Bentler, 1999). However, since in SEM-based LGM the factor loadings are usually fixed at time points rather than freely estimated, and the fit of the model to the mean structure should be reflected as well, assessment of model fit by using conventional SEM-based fit indices should be cautious (Mehta & Neale, 2005; Wu et al., 2009). When every individual is observed at the same fixed set of time points (called “balanced”) with no missing values (called “complete”), ML estimation is used; otherwise, FIML estimation is used (Wu et al., 2009). With FIML estimation, the model-implied means and covariances are computed for each individual, and the maximum likelihood chi-square fit function is obtained by summing −2 log likelihood across all of the individual data vectors (Bovaird, 2007). For balanced and complete data, FIML simplifies to ML, and, in this case, RMSEA, CFI, and NNFI among the SEM-based fit indices have shown good potential performance in evaluating the fit of LGM (Wu & West, 2010; Wu et al., 2009). For unbalanced designs or missing data, conventional guidelines for adequate fit with these indices may be misleading (Wu et al., 2009).

During the search process, we need instruments for the implementation of fitting various types of error covariance structures. The primary motivation to use PROC CALIS is to take advantage of its flexibility in specifying Level-1 error covariance structures and its capability to deal with growth modeling for both manifest variables and latent constructs. PROC CALIS performs better than PROC MIXED, but more sophisticated coding work is required. The purpose of the present article is to address this issue by giving a tutorial on the syntax using PROC CALIS to fit many types of Level-1 error covariance structures in LGM for a manifest variable as well as for a latent construct. Illustrations will be conducted with the data generated from two given latent growth models. SAS is a general-purpose and publicly available software. Its ability to do data management and analysis within a single package will make the instruments we provide attractive to many researchers.

Latent growth models

In this section, we briefly introduce the LGM with a variety of Level-1 error covariance structures through a typical example depicted in Fig. 1. In the figure, y ₁ – y ₄ denote the repeated measures of y on four occasions and X is a Level-2 predictor. ${\eta_{{{\alpha_i}}}}$ is the unobserved intercept representing the initial status for individual i, and ${\eta_{{{\beta_i}}}}$ the unobserved slope showing the individual’s linear rate of change per unit increase in time. ${\eta_{{{\alpha_i}}}}$ and ${\eta_{{{\beta_i}}}}$ are both latent factors. The Level-1 model can be written as

$$ {\mathbf{y}} = {\mathbf{\Lambda }}_y^{*}\eta + \varepsilon $$

(1)

where $y = [{y_1}{ }{y_2}{ }{y_3}{ }{y_4}]\prime$, ${\mathbf{\Lambda}} ^{{*\prime }}_{y} = {\left[ {\begin{array}{*{20}c} {1} & {1} & {1} & {1} \\ {{\lambda _{1} }} & {{\lambda _{2} }} & {{\lambda _{3} }} & {{\lambda _{4} }} \\ \end{array} } \right]}$, $\eta = [{\eta_{\alpha }}{ }{\eta_{\beta }}]\prime$, and $\varepsilon = [{\varepsilon_1}{ }{\varepsilon_2}{ }{\varepsilon_3}{ }{\varepsilon_4}]\prime$. λ _t is the measurement time points (t = 1, 2, 3, 4) and ε denotes Level-1 errors. The solid line with four arrowheads presented in Fig. 1 indicate that ε _t are pairwise correlated. The factor loading associated with initial status are all fixed at 1, whereas those associated with the slope are set at the value λ _t to reflect the particular time point t for individual i. A common coding of λ _t for different time points is to set λ ₁ = 0 for baseline and ${\lambda_t} = t - 1$ for the follow-ups. For this model, subject i’s growth trajectory is a straight line, ${\eta_{{{\alpha_i}}}} + { }{\lambda_t}{\eta_{{{\beta_i}}}}$, λ _t = 0, 1, 2, 3. (For simplicity, subscript i is omitted for the rest part of this article.) The loading matrix ${\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}$ containing fixed values has a superscript * to distinguish from the traditional notation used for the unknown loadings in confirmatory factor analysis (CFA). The model is a restricted CFA model.

Fig. 1

Linear latent growth model with four repeated measures and a predictor X (adapted from Bollen and Curran [2006, p. 128] and Preachers et al. [2008, p. 29])

The Level-2 model can be written as

$$ \eta = {{\mathbf{\Gamma }}_0} + {{\mathbf{\Gamma }}_{{\mathbf{x}}}}{\mathbf{x}} + {\zeta_{\eta }} ,$$

(2)

where ${{\mathbf{\Gamma }}_0} = [{\gamma_{{00}}}{ }{\gamma_{{01}}}]\prime$, ${{\mathbf{\Gamma }}_{{\mathbf{x}}}} = [{\gamma_{{10}}}{ }{\gamma_{{11}}}]\prime$, ${\mathbf{x}} = \left[ X \right]$, and ${\zeta_{\eta }} = [{\zeta_{{{\eta_{\alpha }}}}}{ }{\zeta_{{{\eta_{\beta }}}}}]$. Growth factors η _α and η _β (a random intercept and a random slope) are both predicted by a time invariant subject-level covariate X. γ ₀₀ and γ ₁₀ denote, respectively, the intercept and slope of the regression of η _α on X; γ ₀₁ and γ ₁₁ are those of η _β on X; and ${\zeta_{{{\eta_{\alpha }}}}}$ and ${\zeta_{{{\eta_{\beta }}}}}$are Level-2 errors. Two or more time invariant predictors of change may be included. Since it is not our focus, for simplicity, we consider only one predictor here. ζ _η and ε are assumed to be uncorrelated. The models can be rewritten in combined form as

$$ y = {\mathbf{\Lambda }}_y^{*}({{\mathbf{\Gamma }}_0} + {{\mathbf{\Gamma }}_{{\mathbf{x}}}}{\mathbf{x}}) + {\mathbf{\Lambda }}_y^{*}{{\mathbf{\zeta_{\eta }}}} + {\mathbf{\varepsilon}} $$

(3)

based on which the model-implied mean vector μ and the model-implied covariance matrix Σ of the manifest variables y ₁–y ₄ and X can be expressed as functions of the model parameters as follows (Bollen & Curran, 2006, p. 134–135):

$$ {\mathbf{\mu}} = \left[ {\begin{array}{*{20}{c}} {{\mu_y}} \\ {{{\mathbf{\mu}}_{{\mathbf{x}}}}} \\ \end{array} } \right] = \left[ {\begin{array}{*{20}{c}} {{\mathbf{\Lambda }}_y^{*}({{\mathbf{\Gamma }}_0} + {{\mathbf{\Gamma }}_{{\mathbf{x}}}}{{\mathbf{\mu}}_{{\mathbf{x}}}})} \\ {{{\mathbf{\mu}}_{{\mathbf{x}}}}} \\ \end{array} } \right] $$

(4)

$$ {\mathbf{\Sigma }} = \left[ {\begin{array}{*{20}{c}} {{\mathbf{\Lambda }}_y^{*}({{\mathbf{\Gamma }}_{{\mathbf{x}}}}{{\mathbf{\Sigma }}_{{{\mathbf{xx}}}}}{\mathbf{\Gamma }}_{{\mathbf{x}}}^{\prime } + {{\mathbf{\Psi }}_{{{\zeta_{\eta }}}}}){\mathbf{\Lambda }}{{_y^{*}}^{\prime }} + {{\mathbf{\Theta }}_{\varepsilon }}} & {{\mathbf{\Lambda }}_y^{*}{{\mathbf{\Gamma }}_{{\mathbf{x}}}}{{\mathbf{\Sigma }}_{{{\mathbf{xx}}}}}} \\ {{{\mathbf{\Sigma }}_{{{\mathbf{xx}}}}}{\mathbf{\Gamma }}_{{\mathbf{x}}}^{\prime }{\mathbf{\Lambda }}{{_y^{*}}^{\prime }}} & {{{\mathbf{\Sigma }}_{{{\mathbf{xx}}}}}} \\ \end{array} } \right] ,$$

(5)

where Θ _ε and ${{\mathbf{\Psi }}_{{{\zeta_{\eta }}}}}$ denote the variance-covariance matrices of ε and ζ _η , respectively, and μ _x and Σ _xx denote, respectively, the mean vector and the variance–covariance matrix of predictors (${\mu_{{\mathbf{x}}}} = {\mu_X}$ and ${{\mathbf{\Sigma }}_{{{\mathbf{xx}}}}} = \sigma_X^2$ for this model, since there is only one predictor).

The Level-1 errors, ε ₁, ε ₂, ε ₃, and ε ₄, are assumed to be normally distributed with zero means. The general error covariance matrix (ECM) is unstructured and is given by

$$ {{\mathbf{\Theta }}_{\varepsilon }} = \left[ {\begin{array}{*{20}{c}} {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} \\ {{\sigma_{{{\varepsilon_2}{\varepsilon_1}}}}} & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} \\ {{\sigma_{{{\varepsilon_3}{\varepsilon_1}}}}} & {{\sigma_{{{\varepsilon_3}{\varepsilon_2}}}}} & {\sigma_{{{\varepsilon_3}}}^2} & {} \\ {{\sigma_{{{\varepsilon_4}{\varepsilon_1}}}}} & {{\sigma_{{{\varepsilon_4}{\varepsilon_2}}}}} & {{\sigma_{{{\varepsilon_4}{\varepsilon_3}}}}} & {\sigma_{{{\varepsilon_4}}}^2} \\ \end{array} } \right] .$$

(6)

The corresponding option given in SAS PROC MIXED is TYPE = UN. Other types of ECM, with fewer parameters may be desirable. The Level-2 errors ${\zeta_{{{\eta_{\alpha }}}}}$ and ${\zeta_{{{\eta_{\beta }}}}}$ are assumed to be normally distributed with zero means. Their covariance matrix is usually specified as unstructured (Murphy & Pituch, 2009):

$$ {{\mathbf{\Psi }}_{{{\zeta_{\eta }}}}} = \left[ {\begin{array}{*{20}{c}} {\sigma_{{{\zeta_{{{\eta_{\alpha }}}}}}}^2} & {{\sigma_{{{\zeta_{{{\eta_{\alpha }}}}}{\zeta_{{{\eta_{\beta }}}}}}}}} \\ {{\sigma_{{{\zeta_{{{\eta_{\alpha }}}}}{\zeta_{{{\eta_{\beta }}}}}}}}} & {\sigma_{{{\zeta_{{{\eta_{\beta }}}}}}}^2} \\ \end{array} } \right] $$

(7)

Types of the Level-1 Error Covariance Structure and SAS Statements

Any type of the Level-1 ECM (Θ _ε ) except unstructured can be expressed as a set of linear and/or nonlinear constraints on the parameters involving the covariance structure. SAS PROC MIXED provides a REPEATED statement, in which many types of the Level-1 error covariance structure can be specified through the TYPE = option (e.g., Singer, 1998). However, some important processes, such as higher order autoregressive and moving average processes are not included. Moreover, PROC MIXED cannot handle LGM for constructs.¹ To improve, use PROC CALIS. The STD, COV, and PARAMETERS statements in PROC CALIS can be used together to specify any type of ECM. The STD statement defines variances to estimate for exogenous and error variables. The COV statement defines covariances to estimate for exogenous and error variables. The PARAMETERS statement defines additional parameters that are not specified in the models, and it uses both the original and additional parameters for modeling ECM. In other words, each specific type of ECM is composed of functions of the original and additional parameters. The SAS statements in PROC CALIS for fitting different types of the Level-1 error covariance structures, including AR(1) (the first-order autoregressive), MA(1) (the first-order moving average), ARMA(1,1) (the first-order autoregressive moving average), AR(2) (the second-order autoregressive), MA(2) (the second-order moving average), ARH(1) (heterogeneous AR(1)), TOEPH (heterogeneous Toeplitz), and UN (unstructured)—with four equally spaced occasions—are summarized in Table 1. AR(1), MA(1), ARMA(1,1), AR(2), and MA(2) are members of the ARMA family. Documentation for LGM with ARMA(1,1), TOEPH, and AR(2) for Level-1 errors is given as follows:

Table 1 SAS statements in PROC CALIS for specifying different types of the Level-1 error covariance structure with four occasions

Example 1: ARMA(1,1)

The ARMA(1,1) process is defined as ${\varepsilon_t} = {\phi_1}{\varepsilon_{{t - 1}}} + {\nu_t} - {\theta_1}{\nu_{{t - 1}}}$, where ${\phi_1}$ denotes the autoregressive parameter, θ ₁ the moving average parameter, and v _t an i.i.d. disturbance process (Box, Jenkins, & Reinsel, 1994, p. 77). Its interpretation is that the Level-1 error at time t can be predicted by the Level-1 error at time t–1 and the independent disturbance at time t–1. The resulting ECM is given by

$$ {{\mathbf{\Theta }}_{\varepsilon }} = \sigma_{\varepsilon }^2\left[ {\begin{array}{*{20}{c}} 1 & {} & {} & {} \\ {{\rho_1}} & 1 & {} & {} \\ {{\rho_2}} & {{\rho_1}} & 1 & {} \\ {{\rho_3}} & {{\rho_2}} & {{\rho_1}} & 1 \\ \end{array} } \right] ,$$

(8)

where $\sigma_{\varepsilon }^2$ denotes the common variance of ε _t ; t = 1, 2, 3, 4; and ρ _k denotes their autocorrelation coefficient at lag k, given by ${\rho_1} = \frac{{({\phi_1} - {\theta_1})(1 - {\phi_1}{\theta_1})}}{{(1 - 2{\phi_1}{\theta_1} + \theta_1^2)}}$, ${\rho_k} = {\phi_1}{\rho_{{k - 1}}},$ k = 2, 3; with the constraints of $|{\phi_1}| < 1$ and $|{\theta_1}| < 1$. Program 1 in Appendix A demonstrates how to use PROC CALIS for modeling LGM with the ARMA(1,1) covariance structure for Level-1 errors and the unstructured covariance for Level-2 errors for four equally spaced time points. The UCOV and AUG options are specified to analyze the mean structures in an uncorrected covariance matrix. The data set to be analyzed is augmented by an intercept variable INTERCEPT that has constant values equal to 1. The LINEQS statement given below is used to specify the Level-1 model (the restricted CFA model) shown in Equation 1 and the Level-2 model shown in Equation 2.

$$ \begin{array}{*{20}{c}} {\text{LINEQS}} \hfill \\ {\,\,\,{\text{Y1}} = {\text{1 F}}\_{\text{Alpha}} + 0{\text{ F}}\_{\text{Beta}} + {\text{E1}},} \hfill \\ {\,\,\,{\text{Y2}} = {\text{1 F}}\_{\text{Alpha}} + {\text{1 F}}\_{\text{Beta}} + {\text{E2}},} \hfill \\ {\,\,\,{\text{Y3}} = {\text{1 F}}\_{\text{Alpha}} + {\text{2 F}}\_{\text{Beta}} + {\text{E3}},} \hfill \\ {\,\,\,{\text{Y4}} = {\text{1 F}}\_{\text{Alpha}} + {\text{3 F}}\_{\text{Beta}} + {\text{E4}},} \hfill \\ {\,\,\,{\text{F}}\_{\text{Alpha}} = {\text{GA}}00{\text{ INTERCEPT}} + {\text{GA}}0{\text{1 X}} + {\text{D}}0,} \hfill \\ {\,\,\,{\text{F}}\_{\text{Beta}} = {\text{GA1}}0{\text{ INTERCEPT}} + {\text{GA11 X}} + {\text{D1}};} \hfill \\ \end{array} $$

where F_ALPHA and F_BETA represent latent factors ${\eta_{{{\alpha_i}}}}$ and ${\eta_{{{\beta_i}}}}$. Factor loadings are fixed values (in ${\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}$). Level-1 errors ε ₁ – ε ₄ are named E1–E4, and Level-2 errors ${\zeta_{{\eta_{\alpha }}}}$ and ${\zeta_{{\eta_{\beta }}}}$ are named D0 and D1. GA00, GA01, GA10, and GA11 represent estimates of growth parameters γ ₀₀, γ ₀₁, γ ₁₀, and γ ₁₁.

By Equation 8, Level-1 error variances are equal, their autocovariances at lag 1 are equal, and their autocovariances at lag 2 are equal as well. Level-2 error variances/covariances are unstructured, as shown in Equation 7. Therefore, the STD and COV statements are given as follows:

$$ \begin{array}{*{20}{c}} {\text{STD}} \hfill \\ {\,\,\,{\text{E1}} = {\text{VARE}},{\text{ E2}} = {\text{VARE}},{\text{ E3}} = {\text{VARE}},{\text{ E4}} = {\text{VARE}},{\text{ D}}0 = {\text{VARD}}0,{\text{ D1}} = {\text{VARD1}};} \hfill \\ {\text{COV}} \hfill \\ {\quad {\text{E1 E2}} = {\text{COV}}\_{\text{lag1}},{\text{ E2 E3}} = {\text{COV}}\_{\text{lag1}},{\text{ E3 E4}} = {\text{COV}}\_{\text{lag1}},} \hfill \\ {\quad {\text{E1 E3}} = {\text{COV}}\_{\text{lag2}},{\text{ E2 E4}} = {\text{COV}}\_{\text{lag2}},} \hfill \\ {\quad {\text{E1 E4}} = {\text{COV}}\_{\text{lag3}},} \hfill \\ {\quad {\text{D}}0{\text{ D1}} = {\text{COVD}}0{\text{D1}};} \hfill \\ \end{array} $$

in which VARE represents the estimate of the common variance $\sigma_{\varepsilon }^2$ of the four Level-1 errors, and VARD0 and VARD1 represent the estimates of the variances, $\sigma_{{{\zeta_{{\eta_{\alpha }}}}}}^2$ and $\sigma_{{{\zeta_{{\eta_{\beta }}}}}}^2$, of the two Level-2 errors. COV_lag1 and COV_lag2 represent, respectively, the common Level-1 error autocovariance estimates at lag 1 and lag 2. COV_lag3 is the estimate of the error autocovariance at lag 3. CD0D1 is the estimate of ${\sigma_{{{\zeta_{{\eta_{\alpha }}}}{\zeta_{{\eta_{\beta }}}}}}}$, the covariance of ${\zeta_{{\eta_{\alpha }}}}$ and ${\zeta_{{\eta_{\beta }}}}$.

Since there exist extra parameters in ECM, they need to be defined, and the work can be achieved by using the PARAMETERS statement given by

$$ \begin{array}{*{20}{c}} {\text{PARAMETERS}} \hfill \\ {\quad {\text{PHI1 RHO1}};} \hfill \\ {\,\,\,{\text{COV}}\_{\text{lag1}} = {\text{RHO1}}*{\text{VARE}};} \hfill \\ {\quad {\text{COV}}\_{\text{lag2}} = {\text{PHI1}}*{\text{ COV}}\_{\text{lag1}};{ }/*{\text{ i}}.{\text{e}}.,{\text{ COV}}\_{\text{lag2}} = {\text{PHI1}}*{\text{RHO1}}*{\text{ VARE}};{ }*/} \hfill \\ {\,\,\,{\text{COV}}\_{\text{lag3}} = {\text{PHI1}}*{\text{ COV}}\_{\text{lag2}};{ }/*{\text{ i}}.{\text{e}}.,{\text{ COV}}\_{\text{lag3}} = \left( {{\text{PHI1}}**{2}} \right)*{\text{RHO1}}*{\text{VARE}};{ }*/} \hfill \\ \end{array} $$

in which PHI1 and RHO1 represent the estimates of ρ ₁ and ${\phi_1}$, defined through their relationships with the autocovariances shown in Equation 8. COV_lag1 = RHO1*VARE corresponds to the requirement that the common autocovariance at lag 1 be equal to $\sigma_{\varepsilon }^2{\rho_1}$. The syntax corresponding to the requirements for the autocovariances at lag 2 (=$\sigma_{\varepsilon }^2{\phi_1}{\rho_1}$) and lag 3 (=$\sigma_{\varepsilon }^2{\phi_1}{\rho_2} = \sigma_{\varepsilon }^2\phi_1^2{\rho_1}$) is given in a similar way.

The constraint of $|{\phi_1}| < 1$ is specified by the following BOUNDS statement:

$$ \begin{array}{*{20}{c}} {\text{BOUNDS}} \hfill \\ {\,\quad - {1}.{ } < {\text{ PHI1}} < {1}.} \hfill \\ \end{array} $$

Example 2: TOEPH

The ECM resulting from heterogeneous Toeplitz is given by

$$ {{\mathbf{\Theta }}_{\varepsilon }} = \left[ \begin{gathered} \sigma_{{{\varepsilon_1}}}^2 \hfill \\ \begin{array}{*{20}{c}} {{\sigma_{{{\varepsilon_2}}}}{\sigma_{{{\varepsilon_1}}}}{\rho_1}} \hfill & {\sigma_{{{\varepsilon_2}}}^2} \hfill \\ \end{array} \hfill \\ \begin{array}{*{20}{c}} {{\sigma_{{{\varepsilon_3}}}}{\sigma_{{{\varepsilon_1}}}}{\rho_2}} \hfill & {{\sigma_{{{\varepsilon_3}}}}{\sigma_{{{\varepsilon_2}}}}{\rho_1}} \hfill & {\sigma_{{{\varepsilon_3}}}^2} \hfill \\ \end{array} \hfill \\ \begin{array}{*{20}{c}} {{\sigma_{{{\varepsilon_4}}}}{\sigma_{{{\varepsilon_1}}}}{\rho_3}} \hfill & {{\sigma_{{{\varepsilon_4}}}}{\sigma_{{{\varepsilon_2}}}}{\rho_2}} \hfill & {{\sigma_{{{\varepsilon_4}}}}{\sigma_{{{\varepsilon_3}}}}{\rho_1}} \hfill & {\sigma_{{{\varepsilon_4}}}^2} \hfill \\ \end{array} \hfill \\ \end{gathered} \right] $$

(9)

where $\sigma_{\varepsilon_t}$ denotes the standard deviation for ε _t , t = 1, 2, 3, 4; and ρ _k the autocorrelation at lag k; k = 1, 2, 3. The Level-1 error variances are unequal, but the autocorrelations at the same lag are equal. The STD and COV statements are given as follows:

$$ \begin{array}{*{20}{c}} {\text{STD}} \hfill \\ {\quad {\text{E1}} = {\text{VARE1}},{\text{ E2}} = {\text{VARE2}},{\text{ E3}} = {\text{VARE3}},{\text{ E4}} = {\text{VARE4}},} \hfill \\ {\quad {\text{D}}0 = {\text{VARD}}0,{\text{ D1}} = {\text{VARD1}};} \hfill \\ {\text{COV}} \hfill \\ {\quad {\text{E1 E2}} = {\text{COVE1E2}},{\text{ E1 E3}} = {\text{COVE1E3}},{\text{ E1 E4}} = {\text{COVE1E4}},} \hfill \\ {\quad {\text{E2 E3}} = {\text{COVE2E3}},{\text{ E2 E4}} = {\text{COVE2E4}},{\text{ E3 E4}} = {\text{COVE3E4}},} \hfill \\ {\quad {\text{D}}0{\text{ D1}} = {\text{COVD}}0{\text{D1}};} \hfill \\ \end{array} $$

in which VARE1–VARE4 represent the estimates of the four Level-1 error variances, and VARD0 and VARD1 represent those of the two Level-2 error variances. COVE1E2–COVE3E4 represent the corresponding Level-1 error autocovariance estimates, and COVD0D1 represents the Level-2 error autocovariance estimate. Since the error covariances $\sigma_{{{\varepsilon_t}{\varepsilon_{{t'}}}}}$ of ε _t and ε _t’ are given by $\sigma_{{{\varepsilon_t}{\varepsilon_{{t'}}}}} = \sigma_{{{\varepsilon_t}}}\sigma_{{{\varepsilon_{{t'}}}}}\rho_{{{\varepsilon_t}{\varepsilon_{{t'}}}}}$ and the autocorrelations at the same lag are constrained to be equal, the following PARAMETERS statement needs to be added:

$$ \begin{array}{*{20}{c}} {\text{PARAMETERS}} \hfill \\ {\quad {\text{RHO1 RHO2 RHO3}};} \hfill \\ {\,\,\,{\text{COVE1E2}} = {\text{SQRT}}\left( {\text{VARE1}} \right)*{\text{SQRT}}\left( {\text{VARE2}} \right)*{\text{RHO1}};} \hfill \\ {\,\,\,{\text{COVE2E3}} = {\text{SQRT}}\left( {\text{VARE2}} \right)*{\text{SQRT}}\left( {\text{VARE3}} \right)*{\text{RHO1}};} \hfill \\ {\,\,\,{\text{COVE3E4}} = {\text{SQRT}}\left( {\text{VARE3}} \right)*{\text{SQRT}}\left( {\text{VARE4}} \right)*{\text{RHO1}};} \hfill \\ {\,\,\,{\text{COVE1E3}} = {\text{SQRT}}\left( {\text{VARE1}} \right)*{\text{SQRT}}\left( {\text{VARE3}} \right)*{\text{RHO2}};} \hfill \\ {\,\,\,{\text{COVE2E4}} = {\text{SQRT}}\left( {\text{VARE2}} \right)*{\text{SQRT}}\left( {\text{VARE4}} \right)*{\text{RHO2}};} \hfill \\ {\,\,\,{\text{COVE1E4}} = {\text{SQRT}}\left( {\text{VARE1}} \right)*{\text{SQRT}}\left( {\text{VARE4}} \right)*{\text{RHO3}};} \hfill \\ \end{array} $$

where RHO1, RHO2, and RHO3 are estimates of ρ ₁, ρ ₂, and ρ ₃. The LINEQS statement used for this example is the same as that given in Example 1.

Example 3: AR(2)

It is not possible to model AR(2) for Level-1 errors by using PROC MIXED, but the task can be done by using PROC CALIS, with the statements shown in Table 1. The AR(2) process, given by ${\varepsilon_t} = {\phi_1}{\varepsilon_{{t - 1}}} + {\phi_2}{\varepsilon_{{t - 2}}} + {\nu_t}$, where ${\phi_1}$ and ${\phi_2}$ are autoregressive parameters and v _t an i.i.d. process (Box et al., 1994, p. 54), leads to the following Level-1 ECM:

$$ \sigma_{\varepsilon }^2\left[ {\begin{array}{*{20}{c}} 1 & {} & {} & {} \\ {{\rho_1}} & 1 & {} & {} \\ {{\rho_2}} & {{\rho_1}} & 1 & {} \\ {{\rho_3}} & {{\rho_2}} & {{\rho_1}} & 1 \\ \end{array} } \right] $$

(10)

where $\sigma_{\varepsilon }^2$ denotes the common variance of ε _t , t = 1, 2, 3, 4, and ρ _k denotes their autocorrelation at lag k, given by ${\rho_0} = 1,$ ${\rho_1} = {\phi_1}/(1 - {\phi_2})$, and ${\rho_k} = {\phi_1}{\rho_{{k - 1}}} + {\phi_2}{\rho_{{k - 2}}},{ }k = 2,{ }3,$ with the constraints of $|{\phi_2}| < 1$, ${\phi_2} + {\phi_1} < 1$, and ${\phi_2} - {\phi_1} < 1$. It follows that the autocovariances at lags 1, 2, and 3, denoted respectively by σ ₁, σ ₂, and σ ₃, are given by ${\sigma_1} = {\rho_1}\sigma_{\varepsilon }^2$, ${\sigma_2} = {\rho_2}\sigma_{\varepsilon }^2 = {\phi_1}{\rho_1}\sigma_{\varepsilon }^2 + {\phi_2}\sigma_{\varepsilon }^2 = {\phi_1}{\sigma_1} + {\phi_2}\sigma_{\varepsilon }^2,$ and ${\sigma_3} = {\rho_3}\sigma_{\varepsilon }^2 = {\phi_1}{\rho_2}\sigma_{\varepsilon }^2 + {\phi_2}{\rho_1}\sigma_{\varepsilon }^2 = {\phi_1}{\sigma_2}$ ${\phi_2}{\sigma_1}$. Note that the last two constraints are specified by using the LINCON statement. Relevant SAS statements are given as follows:

$$ \begin{array}{*{20}{c}} {\text{STD}} \hfill \\ {\,\,\,{\text{E1}} - {\text{E4}} = {4}*{\text{VARE}},{ }/*{\text{ i}}.{\text{e}}.,{\text{ E1}} = {\text{VARE}},{\text{ E2}} = {\text{VARE}},{\text{ E3}} = {\text{VARE}},{\text{ E4}} = {\text{VARE }}*/} \hfill \\ {\quad {\text{D}}0 = {\text{VARD}}0,{\text{ D1}} = {\text{VARD1}};} \hfill \\ {\text{COV}} \hfill \\ {\quad {\text{E1 E2}} = {\text{COV}}\_{\text{lag1}},{\text{ E2 E3}} = {\text{COV}}\_{\text{lag1}},{\text{ E3 E4}} = {\text{COV}}\_{\text{lag1}},} \hfill \\ {\quad {\text{E1 E3}} = {\text{COV}}\_{\text{lag2}},{\text{ E2 E4}} = {\text{COV}}\_{\text{lag2}},{\text{ E1 E4}} = {\text{COV}}\_{\text{lag3}},} \hfill \\ {\quad {\text{D}}0{\text{ D1}} = {\text{CD}}0{\text{D1}};} \hfill \\ {{\text{PARAMETERS PHI1 PHI2}};} \hfill \\ {\,\,\,{\text{RHO1}} = {\text{PHI1}}/({1}--{\text{PHI2}});} \hfill \\ {\quad {\text{COV}}\_{\text{lag1}} = {\text{RHO1}}*{\text{VARE}};} \hfill \\ {\,\,\,{\text{COV}}\_{\text{lag2}} = {\text{PHI1}}*{\text{COV}}\_{\text{lag1}} + {\text{ PHI2 }}*{\text{VARE}};} \hfill \\ {\quad {\text{COV}}\_{\text{lag3}} = {\text{PHI1}}*{\text{COV}}\_{\text{lag2}} + {\text{PHI2}}*{\text{COV}}\_{\text{lag1}};} \hfill \\ {\text{LINCON}} \hfill \\ {\quad {\text{PHI2}} + {\text{PHI1}} < {1}.,{\text{PHI2}}-{\text{PHI1}} < {1}.;} \hfill \\ {\text{BOUNDS}} \hfill \\ {\quad - {1}.{ } < {\text{PHI2}} < {1}.;} \hfill \\ \end{array} $$

In addition to those presented in Table 1, more Level-1 error covariance structures for equally spaced data, including ARMA(p, q) [autoregressive moving average of order (p, q)], CS (compound symmetry), TOEP(q) (Toeplitz with q bands, q = 1,…, 4, in which the first q bands of the matrix are to be estimated, setting all higher bands equal to zero), CSH (heterogeneous CS), TOEPH(q) (heterogeneous Toeplitz with q bands, q = 1, …, 4), and UN(q) (UN with q bands, q = 1, …, 4), are summarized in Appendix B. In particular, TOEP(1) indicates i.i.d. Level-1 errors. SAS statements in PROC CALIS for each of them can be obtained in a similar way as shown in Table 1.

The Level-1 error covariance structures displayed in Table 1 and Appendix B are frequently seen in the LGM literature (e.g., Beck & Katz, 1995; Blozis, Harring, & Mels, 2008; Dawson, Gennings, & Carter, 1997; Eyduran & Akbas, 2010; Ferron et al., 2002; Goldstein, Healy, & Rasbash, 1994; Heitjan & Sharma, 1997; Keselman, Algina, Kowalchuk, & Wolfinger, 1998; Kowalchuk & Keselman, 2001; Kwok et al., 2007; Littell, Henry, & Ammerman,1998; Littell, Rendergast, & Natarajan, 2000; Mansour, Nordheim, & Rutledge, 1985; Murphy & Pituch, 2009; Orhan, Eyduran, & Akbas, 2010; Rovine & Molennaar, 1998; 2000; Singer & Willett, 2003; Chap. 7; Velicer & Fava, 2003; West & Hepworth, 1991; Willett & Sayer, 1994; Wolfinger, 1993, 1996; Wulff & Robinson, 2009). The SAS statements provided can facilitate the implementation of their specification.

Illustration

An illustration is given based on the data set generated from the linear growth model shown in Fig. 1 with the ARH(1) Level-1 error covariance structure and the UN Level-2 error covariance structure. Population parameters are given in Table 2. The sample size of 300 was used (Muthén & Muthén, 2002). The RANDNORMAL function in SAS PROC IML was used to generate multivariate normal data based on the population model-implied mean vector μ, shown in Equation 4, and the population model-implied variance–covariance matrixΣ, shown in Equation 5, of y and x . The population mean vector and covariance matrix, as well as sample mean and covariance, are reported in Table 2.

Table 2 Population parameters of the model in Fig. 1 with the Level-1 error covariance structure of ARH(1) and the sample covariance matrix of y ₁–y ₄ and X resulting from a data set of size 300 generated from the model

The parameter estimates resulting from fitting ARH(1) with PROC CALIS (the SEM approach) and PROC MIXED (the HLM approach), given in Table 3, are very close and verify each other. Furthermore, the fit results from PROC CALIS (chi-square = 11.076 with df = 6, p = .086; CFI = .998; NNFI = .996; RMSEA = .05) indicate good model fit.

Table 3 Summary of the results by fitting ARH(1) for Level-1 errors and UN for Level-2 errors based on the sample covariance matrix shown in Table 2 by using PROC CALIS and PROC MIXED

Second-order latent growth models

A second-order latent growth model can be a curve-of-factors model or a factor-of-curves model (e.g., Duncan, Duncan, & Strycker, 2006, Chap. 4; Hancock, Kuo, & Lawrence, 2001). The curve-of-factors model is used to investigate the growth trajectory of a construct over time. It incorporates the multiple indicators (items) representing the latent construct observed at different time points into the model. Repeated latent constructs are termed the first-order factors, and growth factors (i.e., random intercept and slope) are termed the second-order factors. The factor-of-curves model includes higher order common factors for random intercepts and random slopes associated with manifest variables used in LGM. In this model, growth factors are the first-order factors and the underlying common intercept and common slope are the second-order factors, accounting for common developmental patterns. Both the curve-of-factors model and the factor-of-curves model can be well handled by using PROC CALIS.

In this section, the second-order demonstration is given for the curve-of-factors model. The model has several advantages (Blozis, 2006; Preacher et al., 2008; Sayer & Cumsille, 2001). First, the model explicitly recognizes the presence of measurement errors in repeated measures and captures the growth of repeated constructs adjusted for the presence of these errors. Second, the model allows the separation of variation due to departure from the trajectory (temporal instability) and unique variation due to measurement error (unreliability). Third, the model permits the test of longitudinal factorial invariance.

For example, let latent construct F be measured by three indicators, observed at four occasions, denoted by y _1t –y _3t , t = 1, 2, 3, 4. The latent constructs F ₁–F ₄ at the four occasions are the first-order factors, and the growth factors, denoted by η _α and η _β , are the second-order factors. Let ξ, measured by indicators x ₁ – x ₃, be a time-invariant latent predictor for the growth factors. The second-order curve-of-factors, LGM, is pictorially presented in Fig. 2, and can be expressed in matrix form as

$$ \begin{gathered} {\mathbf{y}} = {{\mathbf{\Lambda }}_{{\mathbf{y}}}}F + \varepsilon, \hfill \\ {\mathbf{x}} = {{\mathbf{\Lambda }}_{{\mathbf{x}}}}\xi + \delta, \hfill \\ F = {\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}\eta + {{\mathbf{\zeta }}_F}, \hfill \\ \eta = {{\mathbf{\Gamma }}_0} + {{\mathbf{\Gamma }}_{\xi }}\xi + {{\mathbf{\zeta }}_{\eta }}, \hfill \\ \end{gathered} $$

(11)

where $y = [{y_{{11}}}{ }{y_{{21}}}{ }{y_{{31}}}{ }{y_{{12}}}{ }{y_{{22}}}{ }{y_{{32}}}{ }{y_{{13}}}{ }{y_{{23}}}{ }{y_{{33}}}{ }{y_{{14}}}{ }{y_{{24}}}{ }{y_{{34}}}{]}\prime$, $x = [{x_1}{ }{x_2}{ }{x_3}{]}\prime$, $F = [{F_1}{ }{F_2}{ }{F_3}{ }{F_4}]\prime$, $\eta = [{\eta_{\alpha }}{ }{\eta_{\beta }}]\prime$, $\varepsilon = [\begin{array}{*{20}{c}} {{\varepsilon_{{11}}}} & {{ }{\varepsilon_{{21}}}} & {{ }{\varepsilon_{{31}}}} & {{ }{\varepsilon_{{12}}}} & {{ }{\varepsilon_{{22}}}} & {{ }{\varepsilon_{{32}}}} & {{ }{\varepsilon_{{13}}}} & {{ }{\varepsilon_{{23}}}} & {{ }{\varepsilon_{{33}}}} & {{ }{\varepsilon_{{14}}}} & {{ }{\varepsilon_{{24}}}} & {{ }{\varepsilon_{{34}}}} \\ \end{array} ]\prime$, $\delta = [{\delta_1}{ }{\delta_2}{ }{\delta_3}{]}\prime$, ${{\mathbf{\zeta }}_F} = [{\zeta_{{{F_1}}}}{ }{\zeta_{{{F_2}}}}{ }{\zeta_{{{F_3}}}}{ }{\zeta_{{{F_4}}}}]\prime$, and ${{\mathbf{\zeta }}_{\eta }} = [{\zeta_{{{\eta_{\alpha }}}}}{ }{\zeta_{{{\eta_{\beta }}}}}]\prime$. Λ _y and Λ _x in the measurement model denote the loading matrices showing the relations of indicators to their underlying constructs. One of the indicators for each construct is selected as the reference indicator, and its loading is fixed to 1 at each time point for scaling purpose (Blozis, 2006; Chan, 1998; Sayer & Cumsille, 2001). ${\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}$ denotes the loading matrix (with fixed values) of F on η. Γ ₀ and Γ _ξ denote, respectively, the vector of intercepts and slopes of the regressions of the growth factors η on the latent predictor ξ. ε and $\delta$ denote, respectively, the measurement errors for F and ξ. ζ _F and ζ _η denote, respectively, the errors reflecting the departure of the repeated latent constructs from the trajectory and the errors associated with the random intercept and slope. ε and ζ _F are Level-1 errors, and $\delta$ and ζ _η are Level-2 errors. The assumptions include the following: (a) ε, ζ _F , $\delta$, and ζ _η are uncorrelated; (b) ${\zeta_{{{F_1}}}}$, ${ }{\zeta_{{{F_2}}}}$, ${\zeta_{{{F_3}}}}$, and ${\zeta_{{{F_4}}}}$ are uncorrelated; (c) the measurement errors associated with different indicators are uncorrelated. However, those associated with the same indicator at different points in time are allowed to covary; (d) ${\zeta_{{{\eta_{\alpha }}}}}$ and ${ }{\zeta_{{{\eta_{\beta }}}}}$ are correlated (see, e.g., Blozis, 2006; Bollen & Curran, 2006, p. 249; Preacher et al., 2008, p. 63; Sayer & Cumsille, 2001). The correlated measurement errors are depicted in Fig. 2 by the linkage of three solid lines with four arrowheads, one line for each indicator. On the basis of the aforementioned assumptions, the structures of ${{\mathbf{\Psi }}_{{{\zeta_F}}}}$ and ${{\mathbf{\Theta }}_{\delta }}$ are both TOEPH(1), the structure of ${{\mathbf{\Psi }}_{{{\zeta_{\eta }}}}}$ is UN, and the covariance structure of the correlated measurement errors needs to be identified.

Fig. 2

A second-order linear latent growth model with one time-invariant latent predictor and four repeated latent constructs, each measured by three indicators (adapted from Chan [1998] and Preachers et al. [2008, p. 63])

Weak factorial invariance is usually assumed in the second-order LGM to allow meaningful interpretations of growth trajectories. Weak factorial invariance requires the equality of the loadings in the measurement model for the same indicator across time (Blozis, 2006; Bollen & Curran, 2006, p. 255; Chan, 1998; Hancock et al., 2001; Preacher et al., 2008, p. 63; Sayer & Cumsille, 2001).

Program 2 in Appendix A demonstrates using PROC CALIS to fit a second-order linear trajectory model for four equally spaced time points, in which AR(1) is specified for three series:ε _1t , ε _2t , and ε _3t , t = 1, 2, 3, 4; TOEPH(1) is specified for ζ _F and $\delta$, and UN is specified for ζ _η . The LINEQS statement, based on Equation 11, is given below. It is an extended version from that in Program 1 by incorporating the measurement models for F and the latent predictor ξ.

$$ \begin{array}{*{20}{c}} {\text{LINEQS}} \hfill \\ {\,\,\,{\text{Y11}} = {1} {\text{F1}} + {\text{EY11}},{\text{ Y21}} = {\text{LY21F1 F1}} + {\text{EY21}},{\text{ Y31}} = {\text{LY31F1 F1}} + {\text{EY31}},} \hfill \\ {\,\,\,{\text{Y12}} = {1} {\text{F2}} + {\text{EY12}},{\text{ Y22}} = {\text{LY22F2 F2}} + {\text{EY22}},{\text{ Y32}} = {\text{LY32F2 F2}} + {\text{EY32}},} \hfill \\ {\,\,\,{\text{Y13}} = {1} {\text{F3}} + {\text{EY13}},{\text{ Y23}} = {\text{LY23F3 F3}} + {\text{EY23}},{\text{ Y33}} = {\text{LY33F3 F3}} + {\text{EY33}},} \hfill \\ {\,\,\,{\text{Y14}} = {1} {\text{F4}} + {\text{EY14}},{\text{ Y24}} = {\text{LY24F4 F4}} + {\text{EY24}},{\text{ Y34}} = {\text{LY34F4 F4}} + {\text{EY34}},} \hfill \\ {\,\,\,{\text{X1}} = {1} {\text{F7}} + {\text{EX1}},{\text{ X2}} = {\text{LX2F7 F7}} + {\text{EX2}},{\text{ X3}} = {\text{LX3F7 F7}} + {\text{EX3}},} \hfill \\ {\,\,\,{\text{F1}} = {\text{1 F}}\_{\text{Alpha}} + 0{\text{ F}}\_{\text{Beta}} + {\text{EZF1}},} \hfill \\ {\quad {\text{F2}} = {\text{1 F}}\_{\text{Alpha}} + {\text{1 F}}\_{\text{Beta}} + {\text{EZF2}},} \hfill \\ {\,\,\,{\text{F3}} = {\text{1 F}}\_{\text{Alpha}} + {\text{2 F}}\_{\text{Beta}} + {\text{EZF3}},} \hfill \\ {\,\,\,{\text{F4}} = {\text{1 F}}\_{\text{Alpha}} + {\text{3 F}}\_{\text{Beta}} + {\text{EZF4}},} \hfill \\ {\,\,{\text{F}}\_{\text{Alpha}} = {\text{GA}}00{\text{ INTERCEPT}} + {\text{GA1}}0{\text{ F7}} + {\text{EZF5}},} \hfill \\ {\,\,\,{\text{F}}\_{\text{Beta}} = {\text{GA}}0{\text{1 INTERCEPT}} + {\text{GA11 F7}} + {\text{EZF6}},} \hfill \\ {\,\,\,{\text{F7}} = {\text{F7}}\_{\text{int INTERCEPT}} + {\text{EZF7}};} \hfill \\ \end{array} $$

where F1−F4 are the first-order factors at the four occasions, and F_ALPHA and F_BETA represent the second-order latent factors, ${\eta_{{{\alpha_i}}}}$ and ${\eta_{{{\beta_i}}}}$. Yjt denotes the observed score on the jth indicator for F at occasion t, j = 1, 2, 3; t = 1, 2, 3, 4. Xj (j = 1, 2, 3) denotes the observed score on the jth indicator for construct ξ, named F7. The loadings of Y1t on Ft (t = 1, 2, 3, 4) are fixed to 1. LYjtFt represents the estimate of the first-order loading of Yjt on Ft, j = 2, 3; t = 1, 2, 3, 4. EYjt denotes the corresponding measurement error. Similarly, the loadings of X1, X2, and X3 on F7 are 1, LX2F7, and LX3F7, respectively. EX1−EX3 are the corresponding measurement errors. Second-order factor loadings are fixed values (in${\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}$). Level-1 errors ${\zeta_{{{F_1}}}}{, }{\zeta_{{{F_2}}}}{, }{\zeta_{{{F_3}}}}$, and ${\zeta_{{{F_4}}}}$ are named EZF1–EZF4, and Level-2 errors ${\zeta_{{\eta_{\alpha }}}}$ and ${\zeta_{{\eta_{\beta }}}}$ are named EZF5 and EZF6. F7_int denotes the mean of F7, and EZF7 is the deviation of F7 from its mean.

The parameters in AR(1) for ε _jt , j = 1, 2, 3, include error variance $\sigma_{{{\varepsilon_j}}}^2$ and the autocorrelation at lag 1 ${\phi_{{1{\varepsilon_j}}}}$. The resulting ECM for ε is given by

$$ \begin{gathered} {{\mathbf{\Theta }}_{\varepsilon }} = Cov[\begin{array}{*{20}{c}} {{\varepsilon_{{11}}}} & {{ }{\varepsilon_{{21}}}} & {{ }{\varepsilon_{{31}}}} & {{ }{\varepsilon_{{12}}}} & {{ }{\varepsilon_{{22}}}} & {{ }{\varepsilon_{{32}}}} & {{ }{\varepsilon_{{13}}}} & {{ }{\varepsilon_{{23}}}} & {{ }{\varepsilon_{{33}}}} & {{ }{\varepsilon_{{14}}}{ }} & {{\varepsilon_{{24}}}} & {{ }{\varepsilon_{{34}}}} \\ \end{array} ]\prime \hfill \\ { } = \left[ {\begin{array}{*{20}{c}} {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} & {} & {} & {} & {} & {} & {} & {} & {} \\ 0 & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} & {} & {} & {} & {} & {} & {} & {} & {} \\ 0 & 0 & {\sigma_{{{\varepsilon_3}}}^2} & {} & {} & {} & {} & {} & {} & {} & {} & {} \\ {{\phi_{{1{\varepsilon_1}}}}\sigma_{{{\varepsilon_1}}}^2} & 0 & 0 & {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} & {} & {} & {} & {} & {} \\ 0 & {{\phi_{{1{\varepsilon_2}}}}\sigma_{{{\varepsilon_2}}}^2} & 0 & 0 & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} & {} & {} & {} & {} & {} \\ 0 & 0 & {{\phi_{{1{\varepsilon_3}}}}\sigma_{{{\varepsilon_3}}}^2} & 0 & 0 & {\sigma_{{{\varepsilon_3}}}^2} & {} & {} & {} & {} & {} & {} \\ {\phi_{{1{\varepsilon_1}}}^2\sigma_{{{\varepsilon_1}}}^2} & 0 & 0 & {{\phi_{{1{\varepsilon_1}}}}\sigma_{{{\varepsilon_1}}}^2} & 0 & 0 & {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} & {} & {} \\ 0 & {\phi_{{1{\varepsilon_2}}}^2\sigma_{{{\varepsilon_2}}}^2} & 0 & 0 & {{\phi_{{1{\varepsilon_2}}}}\sigma_{{{\varepsilon_2}}}^2} & 0 & 0 & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} & {} & {} \\ 0 & 0 & {\phi_{{1{\varepsilon_3}}}^2\sigma_{{{\varepsilon_3}}}^2} & 0 & 0 & {{\phi_{{1{\varepsilon_3}}}}\sigma_{{{\varepsilon_3}}}^2} & 0 & 0 & {\sigma_{{{\varepsilon_3}}}^2} & {} & {} & {} \\ {\phi_{{1{\varepsilon_1}}}^3\sigma_{{{\varepsilon_1}}}^2} & 0 & 0 & {\phi_{{1{\varepsilon_1}}}^2\sigma_{{{\varepsilon_1}}}^2} & 0 & 0 & {{\phi_{{1{\varepsilon_1}}}}\sigma_{{{\varepsilon_1}}}^2} & 0 & 0 & {\sigma_{{{\varepsilon_1}}}^2} & {} & {} \\ 0 & {\phi_{{1{\varepsilon_2}}}^3\sigma_{{{\varepsilon_2}}}^2} & 0 & 0 & {\phi_{{1{\varepsilon_2}}}^2\sigma_{{{\varepsilon_2}}}^2} & 0 & 0 & {{\phi_{{1{\varepsilon_2}}}}\sigma_{{{\varepsilon_2}}}^2} & 0 & 0 & {\sigma_{{{\varepsilon_2}}}^2} & {} \\ 0 & 0 & {\phi_{{1{\varepsilon_3}}}^3\sigma_{{{\varepsilon_3}}}^2} & 0 & 0 & {\phi_{{1{\varepsilon_3}}}^2\sigma_{{{\varepsilon_3}}}^2} & 0 & 0 & {{\phi_{{1{\varepsilon_3}}}}\sigma_{{{\varepsilon_3}}}^2} & 0 & 0 & {\sigma_{{{\varepsilon_3}}}^2} \\ \end{array} } \right] \hfill \\ \end{gathered} $$

(12)

which can be reexpressed as follows to facilitate readability:

$$ \begin{gathered} {{\mathbf{\Theta }}_{\varepsilon }} = Cov[\begin{array}{*{20}{c}} {{\varepsilon_{{11}}}} & {{\varepsilon_{{12}}}} & {{ }{\varepsilon_{{13}}}} & {{ }{\varepsilon_{{14}}}} & {{ }{\varepsilon_{{21}}}} & {{ }{\varepsilon_{{22}}}} & {{ }{\varepsilon_{{23}}}} & {{ }{\varepsilon_{{24}}}} & {{ }{\varepsilon_{{31}}}} & {{ }{\varepsilon_{{32}}}} & {{ }{\varepsilon_{{33}}}} & {{ }{\varepsilon_{{34}}}} \\ \end{array} ]\prime \hfill \\ { } = \left[ {\begin{array}{*{20}{c}} {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} & {} & {} & {} & {} & {} & {} & {} & {} \\ {{\phi_{{1{\varepsilon_1}}}}\sigma_{{{\varepsilon_1}}}^2} & {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} & {} & {} & {} & {} & {} & {} & {} \\ {\phi_{{1{\varepsilon_1}}}^2\sigma_{{{\varepsilon_1}}}^2} & {{\phi_{{1{\varepsilon_1}}}}\sigma_{{{\varepsilon_1}}}^2} & {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} & {} & {} & {} & {} & {} & {} \\ {\phi_{{1{\varepsilon_1}}}^3\sigma_{{{\varepsilon_1}}}^2} & {\phi_{{1{\varepsilon_1}}}^2\sigma_{{{\varepsilon_1}}}^2} & {{\phi_{{1{\varepsilon_1}}}}\sigma_{{{\varepsilon_1}}}^2} & {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} & {} & {} & {} & {} & {} \\ 0 & 0 & 0 & 0 & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} & {} & {} & {} & {} & {} \\ 0 & 0 & 0 & 0 & {{\phi_{{1{\varepsilon_2}}}}\sigma_{{{\varepsilon_2}}}^2} & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} & {} & {} & {} & {} \\ 0 & 0 & 0 & 0 & {\phi_{{1{\varepsilon_2}}}^2\sigma_{{{\varepsilon_2}}}^2} & {{\phi_{{1{\varepsilon_2}}}}\sigma_{{{\varepsilon_2}}}^2} & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} & {} & {} & {} \\ 0 & 0 & 0 & 0 & {\phi_{{1{\varepsilon_2}}}^3\sigma_{{{\varepsilon_2}}}^2} & {\phi_{{1{\varepsilon_2}}}^2\sigma_{{{\varepsilon_2}}}^2} & {{\phi_{{1{\varepsilon_2}}}}\sigma_{{{\varepsilon_2}}}^2} & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} & {} & {} \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & {\sigma_{{{\varepsilon_3}}}^2} & {} & {} & {} \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & {{\phi_{{1{\varepsilon_3}}}}\sigma_{{{\varepsilon_3}}}^2} & {\sigma_{{{\varepsilon_3}}}^2} & {} & {} \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & {\phi_{{1{\varepsilon_3}}}^2\sigma_{{{\varepsilon_2}}}^2} & {{\phi_{{1{\varepsilon_3}}}}\sigma_{{{\varepsilon_3}}}^2} & {\sigma_{{{\varepsilon_3}}}^2} & {} \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & {\phi_{{1{\varepsilon_3}}}^3\sigma_{{{\varepsilon_3}}}^2} & {\phi_{{1{\varepsilon_3}}}^2\sigma_{{{\varepsilon_2}}}^2} & {{\phi_{{1{\varepsilon_3}}}}\sigma_{{{\varepsilon_3}}}^2} & {\sigma_{{{\varepsilon_3}}}^2} \\ \end{array} } \right] \hfill \\ \end{gathered} $$

(13)

where $\phi_{{1{\varepsilon_j}}}^k$is the autocorrelation at lag k forε _jt ; k = 1, 2, 3; t = 1, 2, 3, 4; j = 1, 2, 3, with the constraints of $|{\phi_{{1{\varepsilon_j}}}}| < 1$. For each indicator, their error variances at different time points are equal, their error autocovariances at lag 1 are equal, and their error autocovariances at lag 2 are equal as well. Therefore, the STD and COV statements are given as follows:

$$ \begin{gathered} \begin{array}{*{20}{c}} {\text{STD}} \hfill \\ {\,\,\,{\text{EY11}} - {\text{EY14}} = {4}*{\text{VARE1}},{\text{ EY21}} - {\text{EY24}} = {4}*{\text{VARE2}},{\text{ EY31}} - {\text{EY34}} = {4}*{\text{VARE3}},} \hfill \\ {\,\,\,{\text{EX1}} = {\text{VAREX1}},{\text{ EX2}} = {\text{VAREX2}},{\text{ EX3}} = {\text{VAREX3}},} \hfill \\ {\,\,\,{\text{EZF1}} = {\text{VARZF1}},{\text{ EZF2}} = {\text{VARZF2}},{\text{ EZF3}} = {\text{VARZF3}},{\text{ EZF4}} = {\text{VARZF4}},} \hfill \\ {\quad {\text{EZF5}} = {\text{VARE}}\_{\text{Intercept}},{\text{ EZF6}} = {\text{VARE}}\_{\text{Slope}},{\text{ EZF7}} = {\text{VARZF7}};} \hfill \\ \end{array} \hfill \\ \begin{array}{*{20}{c}} {\text{COV}} \hfill \\ {\quad \quad /*{\text{ for the Level}} - {\text{1 measurement errors associated withindicator1 }}*/} \hfill \\ {\,\,\,{\text{EY11 EY12}} = {\text{COV1}}\_{\text{lag1}},{\text{ EY12 EY13}} = {\text{COV1}}\_{\text{lag1}},{\text{ EY13 EY14}} = {\text{COV1}}\_{\text{lag1}},} \hfill \\ {\,\,\,{\text{EY11 EY13}} = {\text{COV1}}\_{\text{lag2}},{\text{ EY12 EY14}} = {\text{COV1}}\_{\text{lag2}},{\text{ EY11 EY14}} = {\text{COV1}}\_{\text{lag3}},} \hfill \\ {\quad \quad /*{\text{ for the Level}} - {\text{1 measurement errors associated withindicator2 }}*/} \hfill \\ {\,\,\,{\text{EY21 EY22}} = {\text{COV2}}\_{\text{lag1}},{\text{ EY22 EY23}} = {\text{COV2}}\_{\text{lag1}},{\text{ EY23 EY24}} = {\text{COV2}}\_{\text{lag1}},} \hfill \\ {\,\,\,{\text{EY21 EY23}} = {\text{COV2}}\_{\text{lag2}},{\text{ EY22 EY24}} = {\text{COV2}}\_{\text{lag2}},{\text{ EY21 EY24}} = {\text{COV2}}\_{\text{lag3}},} \hfill \\ {\quad \quad /*{\text{ for the Level}} - {\text{1 measurement errors associated withindicator3 }}*/} \hfill \\ {\quad {\text{EY31 EY32}} = {\text{COV3}}\_{\text{lag1}},{\text{ EY32 EY33}} = {\text{COV3}}\_{\text{lag1}},{\text{ EY33 EY34}} = {\text{COV3}}\_{\text{lag1}},} \hfill \\ {\,\,\,{\text{EY31 EY33}} = {\text{COV3}}\_{\text{lag2}},{\text{ EY32 EY34}} = {\text{COV3}}\_{\text{lag2}},{\text{ EY31 EY34}} = {\text{COV3}}\_{\text{lag3}},} \hfill \\ {\quad \quad /*{\text{ for the Level}} - {\text{2 errors associated with growth factors }}*/} \hfill \\ {\quad {\text{EZF5 EZF6}} = {\text{CZF5ZF6}};} \hfill \\ \end{array} \hfill \\ \end{gathered} $$

in which VARE1, VARE2, and VARE3 represent, respectively, the estimates of the common variances $\sigma_{{{\varepsilon_1}}}^2$, $\sigma_{{{\varepsilon_2}}}^2$, and $\sigma_{{{\varepsilon_3}}}^2$. VAREX1−VAREX3 represent the estimates of variances of ${\delta_1} - {\delta_3}$. VARZF1−VARZF4 represent the estimates of variances of ${\zeta_{{{F_1}}}} - {\zeta_{{{F_4}}}}$. VARE_Intercept, VARE_Slope, and CZF5ZF6 represent, the estimates of variances and covariance of the second-order factor errors ${\zeta_{{\eta_{\alpha }}}}$ and ${\zeta_{{\eta_{\beta }}}}$. VARZF7 represents the estimate of variance of the latent predictor ξ. COV1_lag1, COV1_lag2, and COV1_lag3 represent, respectively, the estimates of common autocovariance at lags 1, 2, 3 for ε _1t . Similarly, COV2_lag1, COV2_lag2, and COV2_lag3 represent those for ε _2t , and COV3_lag1, COV3_lag2, and COV3_lag3 represent those for ε _3t .

The following PARAMETERS statement is needed to bring three additional parameters, ${\phi_{{1{\varepsilon_1}}}}$, ${\phi_{{1{\varepsilon_2}}}}$, and ${\phi_{{1{\varepsilon_3}}}}$, based on Equation 13:

$$ \begin{array}{*{20}{c}} {{\text{PARAMETERSPHI1 PHI2 PHI3}};} \hfill \\ {\quad \quad /*{\text{ for the Level}} - {\text{1 measurement errors associated withindicator1 }}*/} \hfill \\ {\quad {\text{COV1}}\_{\text{lag1}} = {\text{PHI1}}*{\text{VARE1}};{\text{ COV1}}\_{\text{lag2}} = ({\text{PHI1}}**{2})*{\text{VARE1}};} \hfill \\ {\quad {\text{COV1}}\_{\text{lag3}} = ({\text{PHI1}}**{3})*{\text{VARE1}};} \hfill \\ {\quad \quad /*{\text{ for the Level}} - {\text{1 measurement errors associated withindicator2 }}*/} \hfill \\ {\quad {\text{COV2}}\_{\text{lag1}} = {\text{PHI2}}*{\text{VARE2}};{\text{ COV2}}\_{\text{lag2}} = ({\text{PHI2}}**{2})*{\text{VARE2}};} \hfill \\ {\quad {\text{COV2}}\_{\text{lag3}} = ({\text{PHI2}}**{3})*{\text{VARE2}};} \hfill \\ {\quad \quad /*{\text{ for the Level}} - {\text{1 measurement errors associated withindicator3 }}*/} \hfill \\ {\quad {\text{COV3}}\_{\text{lag1}} = {\text{PHI3}}*{\text{VARE3}};{\text{ COV3}}\_{\text{lag2}} = ({\text{PHI3}}**{2})*{\text{VARE3}};} \hfill \\ {\,\,\,{\text{COV3}}\_{\text{lag3}} = ({\text{PHI3}}**{3})*{\text{VARE3}};} \hfill \\ \end{array} $$

in which PHI1, PHI2, and PHI3 represent the estimates of $\phi_{{1{\varepsilon_1}}}$, $\phi_{{1{\varepsilon_2}}}$, and $\phi_{{1{\varepsilon_3}}}$. COV1_lag1 = PHI1*VARE1 corresponds to the requirement that the common autocovariance at lag 1 for ε _1t be equal to ${\phi_{{1{\varepsilon_1}}}}\sigma_{{{\varepsilon_1}}}^2$. COV1_lag2 = (PHI1**2)*VARE1 corresponds to the requirement that the common autocovariance at lag 2 be equal to $\phi_{{1{\varepsilon_1}}}^2\sigma_{{{\varepsilon_1}}}^2$. COV1_lag3 = (PHI1**3)*VARE1 corresponds to the requirement that the autocovariance at lag 3 be equal to $\phi_{{1{\varepsilon_1}}}^3\sigma_{{{\varepsilon_1}}}^2$. The relevant statements for ε _2t and ε _3t are given similarly.

The constraints of $|{\phi_{{1{\varepsilon_1}}}}| < 1$, $|{\phi_{{1{\varepsilon_2}}}}| < 1$,$|{\phi_{{1{\varepsilon_3}}}}| < 1$ are specified by the following BOUNDS statement:

$$ \begin{array}{*{20}{c}} {\text{BOUNDS}} \hfill \\ {\quad - {1}. < {\text{PHI1}} < {1}.,--{1}. < {\text{PHI2}} < {1}.,--{1}. < {\text{PHI3}} < {1}.;} \hfill \\ \end{array} $$

Under the assumption of weak factorial invariance, the LINCON statement should be added to equalize the loadings for the same indicator across occasions as follows:

$$ \begin{array}{*{20}{c}} {\text{LINCON}} \hfill \\ {\quad {\text{LY21F1}} = {\text{LY22F2}},{\text{ LY21F1}} = {\text{LY23F3}},{\text{ LY21F3}} = {\text{LY24F4}},} \hfill \\ {\quad {\text{LY31F1}} = {\text{LY32F2}},{\text{ LY31F1}} = {\text{LY33F3}},{\text{ LY31F3}} = {\text{LY34F4}};} \hfill \\ \end{array} $$

Illustration

Another illustration is given with another data set of size 300 generated from the second-order LGM in Fig. 2. The population parameters with the AR(1) covariance structure for Level-1 error processes ε _1t , ε _2t , and ε _3t and the sample covariance matrix of y and x resulting from the simulated data set are presented in Table 4. The RANDNORMAL function in PROC IML was used again to generate multivariate normal data based on the population model-implied mean vector and variance–covariance matrix of y and x in Fig. 2 (see Appendix C for the derivation). The parameter estimates by fitting AR(1) for ε _1t , ε _2t , and ε _3t are summarized in Table 5. The resulting parameter estimates are all close to the corresponding population values specified in Table 4, and the model fit is excellent (chi-square = 90.49 with df = 109, p = .9009; CFI =1.0; NNFI =1.0; RMSEA < .0001).

Table 4 Population parameters of the model in Fig. 2 with the Level-1 error covariance structure of AR(1) and the sample covariance matrix resulting from a data set of size 300 generated from the model

Table 5 Summary of the results by fitting AR(1) for Level-1 errors ε _1t , ε _2t , and ε _3t and UN for Level-2 errors based on the sample covariance matrix shown in Table 2 by using PROC CALIS

Conclusion

In the present article, we presented a systematic coding for various Level-1 error covaraince structures in LGM by using SAS PROC CALIS. The joint use of the STD, COV, PARAMETERS, LINCON, and BOUNDS statements in PROC CALIS can be extended for other types of ECM in a similar way to meet analysts’ needs. The advantages to use PROC CALIS include its flexibility in specifying ECM and its capabilities to better assess model fit for balanced complete data and to deal with latent constructs. A tutorial on the syntax has been provided for manifest variables and latent constructs. It is our hope that the coding provided will help applied researchers with LGM studies.

Although our demonstration is based on linear growth models, SAS statements in PROC CALIS for specifying Level-1 ECM are applicable for quadratic and polynomial growth models. Theory may suggest appropriate growth models. As was mentioned previously, misspecification of the growth model can lead to incorrect selection of the error covariance structure. The coding provided is useful when the growth model has been well determined and the Level-1 error covariance structure is to be identified. Although the guidelines given in Littell et al. (2006, Chap. 5) for selecting appropriate Level-1 error covariance structures are helpful, more research work on this issue is needed. How to fit Level-1 error covariance structures based on multivariate error processes deserves future research as well.

Appendices

APPENDIX A

Sample SAS programs for LGM by using PROC CALIS

Program 1

A SAS program for the LGM shown in Figure 1 by fitting ARMA(1,1) for Level-1 error covariance structure

Program 2

A SAS program for the second-order LGM shown in Figure 2 by fitting AR(1) for the Level-1 error covariance structure associated with each indicator

APPENDIX B

More types of Level-1 error covariance structures with four equally spaced occasions

Structure (Θ ε )	ECM
ARMA(p, q):	$\sigma_{\varepsilon }^2\left[ {\begin{array}{*{20}{c}} 1 & {} & {} & {} \\ {{\rho_1}} & 1 & {} & {} \\ {{\rho_2}} & {{\rho_1}} & 1 & {} \\ {{\rho_3}} & {{\rho_2}} & {{\rho_1}} & 1 \\ \end{array} } \right]$
$\begin{gathered} {\varepsilon_t} = {\phi_1}{\varepsilon_{{t - 1}}} + \cdots + {\phi_p}{\varepsilon_{{t - p}}} + \hfill \\ { }{v_t} - {\theta_1}{v_{{t - 1}}} - \cdots - {\theta_q}{v_{{t - q}}} \hfill \\ \end{gathered}$
ARMA(1,2):
${\varepsilon_t} = {\phi_1}{\varepsilon_{{t - 1}}} + {v_t} - {\theta_1}{v_{{t - 1}}} - {\theta_2}{v_{{t - 2}}},$
$|{\phi_1}| < 1,\;|{\theta_2}| < 1,{ }{\theta_1} + {\theta_2} < 1,{ }{\theta_2} - {\theta_1} < 1;$
${\rho_0} = 1,\;{\rho_1} = \frac{{( - {\theta_1} + {\theta_1}{\theta_2}) + [{\phi_1}(1 + \theta_2^2) + {\phi_1}(\theta_1^2 - {\theta_2}) - \phi_1^2{\theta_1}(1 - {\theta_2}) - \phi_1^3{\theta_2}]}}{{(1 + \theta_1^2 + \theta_2^2) - [2{\phi_1}({\theta_1} - {\theta_1}{\theta_2} + {\phi_1}{\theta_2})]}},$
${\rho_2} = \frac{{ - {\theta_2} - [{\phi_1}{\theta_1}(1 - {\theta_2}) + \phi_1^2(1 + \theta_1^2 + \theta_2^2) - \phi_1^3{\theta_1}(1 - {\theta_2}) - \phi_1^4{\theta_2}]}}{{(1 + \theta_1^2 + \theta_2^2) - [2{\phi_1}({\theta_1} - {\theta_1}{\theta_2} + {\phi_1}{\theta_2})]}},$
${\rho_k} = {\phi_1}{\rho_{{k - 1}}}{, }k > 2.$
ARMA(2,1):
${\varepsilon_t} = {\phi_1}{\varepsilon_{{t - 1}}} + {\phi_2}{\varepsilon_{{t - 2}}} + {v_t} - {\theta_1}{v_{{t - 1}}},$
$|{\phi_2}| < 1,{ }{\phi_2} + {\phi_1} < 1,{ }{\phi_2} - {\phi_1} < 1,\;|{\theta_1}| < 1;$
${\rho_0} = 1,{\rho_1} = [({\phi_1} - {\theta_1})(1 - {\phi_1}{\theta_1}) + \phi_2^2{\theta_1}]/[(1 - {\phi_2})(1 + \theta_1^2) - 2{\phi_1}{\theta_1}],$
${\rho_k} = ({\phi_1}{\rho_{{k - 1}}} + {\phi_2}{\rho_{{k - 2}}}),{ }k > 1.$
CS:	$\left[ {\begin{array}{*{20}{c}} {\sigma_{\varepsilon }^2} & {} & {} & {} \\ {{\sigma_1}} & {{ }\sigma_{\varepsilon }^2} & {} & {} \\ {{\sigma_1}} & {{ }{\sigma_1}} & {{ }\sigma_{\varepsilon }^2} & {} \\ {{\sigma_1}} & {{ }{\sigma_1}} & {{ }{\sigma_1}} & {{ }\sigma_{\varepsilon }^2} \\ \end{array} } \right]$
$\begin{gathered} \sigma_{\varepsilon }^2[\rho 1(t \ne t\prime ) + 1(t = t\prime )], \hfill \\ {\rho_k} = \rho = {\sigma_1}/\sigma_{\varepsilon }^2{, }k > 0 \hfill \\ \end{gathered}$
TOEP(q) (Toeplitz with q bands, q = 1,…,4):	$\begin{gathered} {\text{ TOEP(1) TOEP(2) }} \hfill \\ \left[ {\begin{array}{*{20}{c}} {\sigma_{\varepsilon }^2} & {} & {} & {} \\ 0 & {\sigma_{\varepsilon }^2} & {} & {} \\ 0 & 0 & {\sigma_{\varepsilon }^2} & {} \\ 0 & 0 & 0 & {\sigma_{\varepsilon }^2} \\ \end{array} } \right]{, }\left[ {\begin{array}{*{20}{c}} {\sigma_{\varepsilon }^2} & {} & {} & {} \\ {{\sigma_1}} & {\sigma_{\varepsilon }^2} & {} & {} \\ 0 & {{\sigma_1}} & {\sigma_{\varepsilon }^2} & {} \\ 0 & 0 & {{\sigma_1}} & {\sigma_{\varepsilon }^2} \\ \end{array} } \right], \hfill \\ {\text{ TOEP(3) TOEP(4) }} \hfill \\ \left[ {\begin{array}{*{20}{c}} {\sigma_{\varepsilon }^2} & {} & {} & {} \\ {{\sigma_1}} & {\sigma_{\varepsilon }^2} & {} & {} \\ {{\sigma_2}} & {{\sigma_1}} & {\sigma_{\varepsilon }^2} & {} \\ 0 & {{\sigma_2}} & {{\sigma_1}} & {\sigma_{\varepsilon }^2} \\ \end{array} } \right]{, }\left[ {\begin{array}{*{20}{c}} {\sigma_{\varepsilon }^2} & {} & {} & {} \\ {{\sigma_1}} & {\sigma_{\varepsilon }^2} & {} & {} \\ {{\sigma_2}} & {{\sigma_1}} & {\sigma_{\varepsilon }^2} & {} \\ {{\sigma_3}} & {{\sigma_2}} & {{\sigma_1}} & {\sigma_{\varepsilon }^2} \\ \end{array} } \right] \hfill \\ \end{gathered}$
$\begin{gathered} {\sigma_{{|t - t'|}}}1(|t - t\prime | < q), \hfill \\ {\sigma_0} = \sigma_{\varepsilon }^2{,}{\sigma_k} = {\rho_k}\sigma_{\varepsilon }^2{, }k > 0{ } \hfill \\ \end{gathered}$
CSH (heterogeneous CS):	$\left[ {\begin{array}{*{20}{c}} {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} \\ {{\sigma_{{{\varepsilon_2}}}}{\sigma_{{{\varepsilon_1}}}}\rho } & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} \\ {{\sigma_{{{\varepsilon_3}}}}{\sigma_{{{\varepsilon_1}}}}\rho } & {{ }{\sigma_{{{\varepsilon_3}}}}{\sigma_{{{\varepsilon_2}}}}\rho } & {\sigma_{{{\varepsilon_3}}}^2} & {} \\ {{\sigma_{{{\varepsilon_4}}}}{\sigma_{{{\varepsilon_1}}}}\rho } & {{ }{\sigma_{{{\varepsilon_4}}}}{\sigma_{{{\varepsilon_2}}}}\rho } & {{ }{\sigma_{{{\varepsilon_4}}}}{\sigma_{{{\varepsilon_3}}}}\rho { }} & {\sigma_{{{\varepsilon_4}}}^2} \\ \end{array} } \right]$
$\begin{gathered} {\sigma_{{{\varepsilon_t}}}}{\sigma_{{{\varepsilon_{{t\prime }}}}}}[\rho 1(t \ne t\prime ) + 1(t = t\prime )], \hfill \\ {\rho_k} = \rho {, }k > 0 \hfill \\ \end{gathered}$
TOEPH(q) (heterogeneous Toeplitz with q bands, q = 1, …,4):	$\begin{gathered} {\text{ TOEPH(1) TOEPH(2) }} \hfill \\ { }\left[ {\begin{array}{*{20}{c}} {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} \\ 0 & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} \\ 0 & 0 & {\sigma_{{{\varepsilon_3}}}^2} & {} \\ 0 & 0 & 0 & {\sigma_{{{\varepsilon_4}}}^2} \\ \end{array} } \right]{, }\left[ {\begin{array}{*{20}{c}} {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} \\ {{\sigma_{{{\varepsilon_2}}}}{\sigma_{{{\varepsilon_1}}}}{\rho_1}} & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} \\ 0 & {{ }{\sigma_{{{\varepsilon_3}}}}{\sigma_{{{\varepsilon_2}}}}{\rho_1}} & {\sigma_{{{\varepsilon_3}}}^2} & {} \\ 0 & 0 & {{ }{\sigma_{{{\varepsilon_4}}}}{\sigma_{{{\varepsilon_3}}}}{\rho_1}{ }} & {\sigma_{{{\varepsilon_4}}}^2} \\ \end{array} } \right], \hfill \\ \hfill \\ \end{gathered}$
${\sigma_{{{\varepsilon_t}}}}{\sigma_{{{\varepsilon_{{t\prime }}}}}}{\rho_{{|t - t\prime |}}}1(|t - t\prime | < q)$
	$\begin{gathered} {\text{ TOEPH(3) }} \hfill \\ { }\left[ {\begin{array}{*{20}{c}} {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} \\ {{\sigma_{{{\varepsilon_2}}}}{\sigma_{{{\varepsilon_1}}}}{\rho_1}} & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} \\ {{\sigma_{{{\varepsilon_3}}}}{\sigma_{{{\varepsilon_1}}}}{\rho_2}} & {{ }{\sigma_{{{\varepsilon_3}}}}{\sigma_{{{\varepsilon_2}}}}{\rho_1}} & {\sigma_{{{\varepsilon_3}}}^2} & {} \\ 0 & {{ }{\sigma_{{{\varepsilon_4}}}}{\sigma_{{{\varepsilon_2}}}}{\rho_2}} & {{ }{\sigma_{{{\varepsilon_4}}}}{\sigma_{{{\varepsilon_3}}}}{\rho_1}} & {{ }\sigma_{{{\varepsilon_4}}}^2} \\ \end{array} } \right]{,} \hfill \\ \hfill \\ {\text{ TOEPH(4)}} \hfill \\ { }\left[ {\begin{array}{*{20}{c}} {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} \\ {{\sigma_{{{\varepsilon_2}}}}{\sigma_{{{\varepsilon_1}}}}{\rho_1}} & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} \\ {{\sigma_{{{\varepsilon_3}}}}{\sigma_{{{\varepsilon_1}}}}{\rho_2}} & {{ }{\sigma_{{{\varepsilon_3}}}}{\sigma_{{{\varepsilon_2}}}}{\rho_1}} & {\sigma_{{{\varepsilon_3}}}^2} & {} \\ {{\sigma_{{{\varepsilon_4}}}}{\sigma_{{{\varepsilon_1}}}}{\rho_3}} & {{ }{\sigma_{{{\varepsilon_4}}}}{\sigma_{{{\varepsilon_2}}}}{\rho_2}} & {{ }{\sigma_{{{\varepsilon_4}}}}{\sigma_{{{\varepsilon_3}}}}{\rho_1}} & {{ }\sigma_{{{\varepsilon_4}}}^2} \\ \end{array} } \right]. \hfill \\ \hfill \\ \end{gathered}$
UN(q) (UN with q bands, q = 1, …,4):	$\begin{gathered} {\text{ UN(1) UN(2)}} \hfill \\ { }\left[ {\begin{array}{*{20}{c}} {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} \\ 0 & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} \\ 0 & 0 & {\sigma_{{{\varepsilon_3}}}^2} & {} \\ 0 & 0 & 0 & {\sigma_{{{\varepsilon_4}}}^2} \\ \end{array} } \right]{, }\left[ {\begin{array}{*{20}{c}} {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} \\ {{\sigma_{{{\varepsilon_2}{\varepsilon_1}}}}} & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} \\ 0 & {{\sigma_{{{\varepsilon_3}{\varepsilon_2}}}}} & {\sigma_{{{\varepsilon_3}}}^2} & {} \\ 0 & 0 & {{\sigma_{{{\varepsilon_4}{\varepsilon_3}}}}} & {\sigma_{{{\varepsilon_4}}}^2} \\ \end{array} } \right], \hfill \\ \hfill \\ {\text{ UN(3) UN(4)}} \hfill \\ { }\left[ {\begin{array}{*{20}{c}} {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} \\ {{\sigma_{{{\varepsilon_2}{\varepsilon_1}}}}} & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} \\ {{\sigma_{{{\varepsilon_3}{\varepsilon_1}}}}} & {{\sigma_{{{\varepsilon_3}{\varepsilon_2}}}}} & {\sigma_{{{\varepsilon_3}}}^2} & {} \\ 0 & {{\sigma_{{{\varepsilon_4}{\varepsilon_2}}}}} & {{\sigma_{{{\varepsilon_4}{\varepsilon_3}}}}} & {\sigma_{{{\varepsilon_4}}}^2} \\ \end{array} } \right]{, }\left[ {\begin{array}{*{20}{c}} {\sigma_{{{\varepsilon_1}}}^2} & {} & {} & {} \\ {{\sigma_{{{\varepsilon_2}{\varepsilon_1}}}}} & {\sigma_{{{\varepsilon_2}}}^2} & {} & {} \\ {{\sigma_{{{\varepsilon_3}{\varepsilon_1}}}}} & {{\sigma_{{{\varepsilon_3}{\varepsilon_2}}}}} & {\sigma_{{{\varepsilon_3}}}^2} & {} \\ {{\sigma_{{{\varepsilon_4}{\varepsilon_1}}}}} & {{\sigma_{{{\varepsilon_4}{\varepsilon_2}}}}} & {{\sigma_{{{\varepsilon_4}{\varepsilon_3}}}}} & {\sigma_{{{\varepsilon_4}}}^2} \\ \end{array} } \right] \hfill \\ \hfill \\ \end{gathered}$
${\sigma_{{{\varepsilon_t}{\varepsilon_{{t\prime }}}}}}1(|t - t\prime | < q)$

1(A) equals 1 when A is true. For example, $1(|t - t\prime | < q)$= 1 when $|t - t\prime | < q$ and 0 otherwise, $q \geqslant 1$. ρ _k denotes the autocorrelation coefficient at lag k. ρ ₀ = 1. TOEP(4) = TOEP; TOEPH(4) = TOEPH; UN(4) = UN; TOEPH(1) = UN(1)

APPENDIX C

Derivation of the population model-implied mean vector μ and variance–covariance matrix Σ of y and x for the second-order LGM (Fig. 2)

Based on Equation 11, we have

$F = {\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}({{\mathbf{\Gamma }}_0} + {{\mathbf{\Gamma }}_{\xi }}\xi + {{\mathbf{\zeta }}_{\eta }}) + {{\mathbf{\zeta }}_F} = {\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Gamma }}_0} + {\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Gamma }}_{\xi }}\xi + {\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\zeta }}_{\eta }} + {{\mathbf{\zeta }}_F},$

$\begin{gathered} {\mathbf{y}} = {{\mathbf{\Lambda }}_{{\mathbf{y}}}}F + \varepsilon \hfill \\ { } = {{\mathbf{\Lambda }}_{{\mathbf{y}}}}({\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Gamma }}_0} + {\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Gamma }}_{\xi }}\xi + {\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\zeta }}_{\eta }} + {{\mathbf{\zeta }}_F}) + \varepsilon \hfill \\ { } = {{\mathbf{\Lambda }}_{{\mathbf{y}}}}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Gamma }}_0} + {{\mathbf{\Lambda }}_{{\mathbf{y}}}}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Gamma }}_{\xi }}\xi + {{\mathbf{\Lambda }}_{{\mathbf{y}}}}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\zeta }}_{\eta }} + {{\mathbf{\Lambda }}_{{\mathbf{y}}}}{{\mathbf{\zeta }}_F} + \varepsilon {,} \hfill \\ \end{gathered}$

${\mathbf{x}} = {{\mathbf{\Lambda }}_{{\mathbf{x}}}}{\mathbf{\xi }} + \delta .$

Therefore

$\begin{array}{*{20}{c}} {{\mu_{{\mathbf{y}}}} = E(y) = {{\mathbf{\Lambda }}_{{\mathbf{y}}}}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Gamma }}_0} + {{\mathbf{\Lambda }}_{{\mathbf{y}}}}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Gamma }}_{\xi }}{\mu_{\xi }}} \hfill \\ {{\mu_{{\mathbf{x}}}} = E(x) = {{\mathbf{\Lambda }}_{{\mathbf{x}}}}{\mu_{\xi }}} \hfill \\ \end{array},$

that is,

$\mu = \left[ {\begin{array}{*{20}{c}} {{\mu_y}} \\ {{\mu_{{\mathbf{x}}}}} \\ \end{array} } \right] = \left[ {\begin{array}{*{20}{c}} {{{\mathbf{\Lambda }}_{{\mathbf{y}}}}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Gamma }}_0} + {{\mathbf{\Lambda }}_{{\mathbf{y}}}}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Gamma }}_{\xi }}{\mu_{\xi }}} \\ {{{\mathbf{\Lambda }}_{{\mathbf{x}}}}{\mu_{\xi }}} \\ \end{array} } \right]$

and

$\begin{array}{*{20}{c}} {{{\mathbf{\Sigma }}_{{{\mathbf{yy}}}}} = Cov({\mathbf{y}}) = {{\mathbf{\Lambda }}_{{\mathbf{y}}}}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Gamma }}_{\xi }}{{\mathbf{\Phi }}_{\xi }}{\mathbf{\Gamma }}_{\xi }^{\prime }{\mathbf{\Lambda }}{{_{{\mathbf{y}}}^{*}}^{\prime }}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{\prime } + {{\mathbf{\Lambda }}_{{\mathbf{y}}}}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Psi }}_{{{\zeta_{\eta }}}}}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}\prime {\mathbf{\Lambda }}_{{\mathbf{y}}}^{\prime } + {{\mathbf{\Lambda }}_{{\mathbf{y}}}}{{\mathbf{\Psi }}_{{{\zeta_F}}}}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{\prime } + {{\mathbf{\Theta }}_{\varepsilon }}} \hfill \\ {\quad \quad \quad \quad \quad = {{\mathbf{\Lambda }}_{{\mathbf{y}}}}({\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Gamma }}_{\xi }}{{\mathbf{\Phi }}_{\xi }}{\mathbf{\Gamma }}_{\xi }^{\prime }{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}\prime + {\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Psi }}_{{{\zeta_{\eta }}}}}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}\prime + {{\mathbf{\Psi }}_{{{\zeta_F}}}}){\mathbf{\Lambda }}_{{\mathbf{y}}}^{\prime } + {{\mathbf{\Theta }}_{\varepsilon }}{, }} \hfill \\ {{{\mathbf{\Sigma }}_{{{\mathbf{xx}}}}} = Cov({\mathbf{x}}) = {{\mathbf{\Lambda }}_{{\mathbf{x}}}}{{\mathbf{\Phi }}_{\xi }}{\mathbf{\Lambda }}_{{\mathbf{x}}}^{\prime } + {{\mathbf{\Theta }}_{\delta }}} \hfill \\ {{{\mathbf{\Sigma }}_{{{\mathbf{yx}}}}} = Cov({\mathbf{y}},x) = {{\mathbf{\Lambda }}_{{\mathbf{y}}}}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Gamma }}_{\xi }}{{\mathbf{\Phi }}_{\xi }}{\mathbf{\Lambda }}_{{\mathbf{x}}}^{\prime },} \hfill \\ {{{\mathbf{\Sigma }}_{{{\mathbf{xy}}}}} = {{\mathbf{\Lambda }}_{{\mathbf{x}}}}{\mathbf{\Phi }}_{\xi }^{\prime }{\mathbf{\Gamma }}_{\xi }^{\prime }{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}\prime {\mathbf{\Lambda }}\prime } \hfill \\ \end{array}$

that is

${\mathbf{\Sigma }} = \left[ {\begin{array}{*{20}{c}} {{{\mathbf{\Sigma }}_{{{\mathbf{yy}}}}}} & {{{\mathbf{\Sigma }}_{{{\mathbf{yx}}}}}} \\ {{{\mathbf{\Sigma }}_{{{\mathbf{xy}}}}}} & {{{\mathbf{\Sigma }}_{{{\mathbf{xx}}}}}} \\ \end{array} } \right] = \left[ {\begin{array}{*{20}{c}} {{{\mathbf{\Lambda }}_{{\mathbf{y}}}}({\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Gamma }}_{\xi }}{{\mathbf{\Phi }}_{\xi }}{\mathbf{\Gamma }}_{\xi }^{\prime }{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}\prime + {\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Psi }}_{{{\zeta_{\eta }}}}}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}\prime + {{\mathbf{\Psi }}_{{{\zeta_F}}}}){\mathbf{\Lambda }}_{{\mathbf{y}}}^{\prime } + {{\mathbf{\Theta }}_{\varepsilon }}} & {{{\mathbf{\Lambda }}_{{\mathbf{y}}}}{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}{{\mathbf{\Gamma }}_{\xi }}{{\mathbf{\Phi }}_{\xi }}{\mathbf{\Lambda }}_{{\mathbf{x}}}^{\prime }} \\ {{{\mathbf{\Lambda }}_{{\mathbf{x}}}}{\mathbf{\Phi }}_{\xi }^{\prime }{\mathbf{\Gamma }}_{\xi }^{\prime }{\mathbf{\Lambda }}_{{\mathbf{y}}}^{*}\prime {\mathbf{\Lambda }}_{{\mathbf{y}}}^{\prime }} & {{{\mathbf{\Lambda }}_{{\mathbf{x}}}}{{\mathbf{\Phi }}_{\xi }}{\mathbf{\Lambda }}_{{\mathbf{x}}}^{\prime } + {{\mathbf{\Theta }}_{\delta }}} \\ \end{array} } \right].$