Skip to main content

Multilevel structural equation modeling-based quasi-experimental synthetic cohort design

Abstract

This paper provides a theoretical foundation to examine the effectiveness of post-hoc adjustment approaches such as propensity score matching in reducing the selection bias of synthetic cohort design (SCD) for causal inference and program evaluation. Compared with the Solomon four-group design, the SCD often encounters selection bias due to the imbalance of covariates between the two cohorts. The efficiency of SCD is ensured by the historical equivalence of groups (HEoG) assumption, indicating the comparability between the two cohorts. The multilevel structural equation modeling framework is used to define the HEoG assumption. According to the mathematical proof, HEoG ensures that the use of SCD results in an unbiased estimator of the schooling effect. Practical considerations and suggestions for future research and use of SCD are discussed.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. The comparability can be statistically tested through the multivariate group comparison approach (Tatsuoka 1971). The comparability of the two groups is revealed by the discriminant function (DF) of X (Tatsuoka 1971). X includes q column vectors such as level-1 (student-level) covariates and their interaction terms and level-2 (class- or school-level) covariates and their interaction terms. It is denoted as \(X=(x_{1}, \ldots , x_{q})\). The DF is a linear combination of X. For example, the first DF of Xs can be written as \(DF_{1}=v_{11}x_{1}+ v_{12}x_{2}+\cdots + v_{1p}x_{q}\). Vector \(V_{1}=(v_{11}, \ldots , v_{1p})\) is the first eigenvector of \({\Sigma ^W}^{-1}\Sigma ^B\). \(\Sigma ^W\) and \(\Sigma ^B\) are the within-group and between-group variance–covariance matrices of Xs, respectively. Notice that the within-group variance–covariance matrix \(\Sigma ^W\) should be computed by taking account of the hierarchical structure of the data (see Schmidt and Houang 1986). If \({(\Sigma ^W)}^{-1}\Sigma ^B\) has n non-zero eigenvalues, then we can define n DFs—namely, \(DF_{1}, DF_{2}, \ldots , DF_{n}\). Using DF simplifies group comparability testing when the number of covariates is large. Following the descriptive discriminant analysis (DDA, Huberty and Olejnik 2006), the group comparability testing can determine if Cohort 2 at time 0 is comparable to Cohort 1 at time 1 regarding covariates X. Here, a two-step testing approach can be conducted. First, one computes the latent roots of \({(\Sigma ^W)}^{-1}\Sigma ^B\) to construct DFs and test whether the two groups are unanimously comparable or not. Second, if they are not unanimously comparable, the univariate group comparison can reveal the non-comparability. Thus, a set of covariates will be identified. The two groups should be non-comparable on each of the covariates in terms of their means. The set of covariates can then be used as matching variables.

  2. Selection bias, also called “sample selection bias”, refers to the bias that is due to the use of non-random samples in estimating relationships among variables of interests. It can occur in two situations: (1) self-selection by objects being studied, and (2) non-random sample selection by data analysts or researchers. The use of selection-biased samples results in a biased estimator of the effect of an intervention that should have been selected randomly. The intervention can refer to “treatment of migration, manpower training, or unionism” (Heckman 1979, p. 154).

  3. \(\alpha\) represents the base-line value due to history or prior learning, which is identical in both treatment and control groups after randomization. It was not specified in Solomon (1949); however, it is important in this study for three reasons. First, it is a quantity that relates to or indicates the initial comparability of the groups. Second, it is involved the process of computing treatment effects (see subsection of SCD in this study). Third, more importantly, it will be a critical criterion to match the groups when randomization is unavailable. Note that when randomization is unavailable, another useful method for the creation of comparable groups is matching (Solomon 1949).

  4. For example, the interaction effect of pre-test and treatment is a function of the four quantities. The quantity in experimental group 1 is \(Q_{E_1} =f(\alpha +\delta +\gamma +\tau +I)\). The quantity in experimental group 2 is \(Q_{E_2} =f(\alpha +\gamma +\delta )\). The quantity in control group 1 is \(Q_{C_1} =f(\alpha +\tau +\gamma )\), and the quantity in control group 2 is \(Q_{C_2} =f(\alpha +\gamma )\). Interaction effect, denoted as I, is computed as \(Q_{E_1}-Q_{E_2} -Q_{C_1} +Q_{C_2}\).

  5. The mean of the sample distribution of \(\hat{\delta }_{C2T1-C1T1}\) is \(\delta _{C2T1-C1T1}\). At the population level, \(\delta _{C2T1-C1T1}\) is an estimator of \(\delta _{C2T0-C2T1}\). This way, bias can be defined as \(BIAS(\delta _{C2T1-C1T1})=\delta _{C2T1-C1T1}-\delta _{C2T0-C2T1}\).

  6. In the two-parameter logistic (2PL) uni/multidimensional IRT model (Lord and Novick 1968; Reckase 2009), measurement equations for pre- and post-test are

    $$\begin{aligned} \log \left[ {\frac{{\rm {prb}}(Y_{0}^{E_1} =1)}{1-{\rm {prb}}(Y_{0}^{E_1} =1)}} \right] =a_0 (\eta _{0}-b_{0}); \end{aligned}$$
    (5)

    and

    $$\begin{aligned} \log \left[ {\frac{{\rm {prb}}(Y_{1}^{E_1} =1)}{1-{\rm {prb}}(Y_{1}^{E_1} =1)}} \right] =a_1 (\eta _{1} -b_{1}), \end{aligned}$$
    (6)

    respectively. \(b_{0}\) and \(b_{1}\) are the item difficulty parameter vectors. \(a_1\) and \(a_0\) are the discrimination parameter vectors. Multidimensional IRT parameter settings and dimension specification are on p. 71 and p. 93 of Reckase (2009).

  7. This equation specifies a general case. For the purpose of simplicity, \(\mathcal {V}\) can be set as 1 across all four groups. \(\tau\) and \(\gamma\) are speculated in the structural model because they reflect changes associated with latent mathematics proficiency. The latent changes will further reveal their effects through the measurement equation.

  8. For the purpose of simplicity, the superscripts (group indices) are dropped; however, Table 2 clearly displays each group in a separate row. Adding subscripts would be redundant. Also, after covariates have been included, the errors terms are now denoted by \(\xi\) rather than \(\varepsilon\).

  9. In education studies, this effect is often referred to as the “schooling effect”.

References

  • Battistin E, Chesher A (2014) Treatment effect estimation with covariate measurement error. J Econom 178(2):707–715

    MathSciNet  MATH  Article  Google Scholar 

  • Berger V (2005) Selection bias and covariate imbalances in randomized clinical trials. Wiley, New York

    Book  Google Scholar 

  • Biemer PP, Groves RM, Lyberg LE (2004) Measurement errors in surveys. Wiley, Hoboken

    Book  Google Scholar 

  • Bloom H (2005) Learning More from Social Experiments: Evolving Analytic Approaches. Russell Sage Foundation, New York

    Google Scholar 

  • Bloom HS, Richburg-Hayes L, Black AR (2007) Using covariates to improve precision for studies that randomize schools to evaluate educational interventions. Educ Eval Policy Anal 29(1):30–59

    Article  Google Scholar 

  • Bollen KA (1989) Structural equations with latent variables. Wiley, New York

    MATH  Book  Google Scholar 

  • Burstein L (1992) The IEA study of mathematics III: student growth and classroom processes. Pergamon Press, Oxford

    Google Scholar 

  • Campbell RT, Hudson CM (1985) Synthetic cohorts from panel surveys. Res Aging 7(1):81–93

    Article  Google Scholar 

  • Campbell DT, Stanley JC (1963) Experimental and quasi-experimental designs for research. Rand McNally College Publishing Company, Skokie

    Google Scholar 

  • Carroll RJ, Ruppert D, Stefanski LA, Crainiceanu CM (2006) Measurement error in nonlinear models: a modern perspective. CRC Press, Boca Raton

    MATH  Book  Google Scholar 

  • Cheung GW, Rensvold RB (2002) Evaluating goodness-of-fit indexes for testing measurement invariance. Struct Equ Model Multidiscip J 9(2):233–255

    MathSciNet  Article  Google Scholar 

  • Cochran WG (1957) Analysis of covariance: its nature and uses. Biometrics 13(3):261–281

    MathSciNet  Article  Google Scholar 

  • Cochran WG (1968) Errors of measurement in statistics. Technometrics 10(4):637–666

    MATH  Article  Google Scholar 

  • Cochran WG, Chambers SP (1965) The planning of observational studies of human populations. J R Stat Soc Ser A (General) 128(2):234–266

    Article  Google Scholar 

  • Cochran WG, Rubin DB (1973) Controlling bias in observational studies: a review. Sankhy Indian J Stat Ser A 35:417–446

    MATH  Google Scholar 

  • Cox DR, Reid N (2000) The theory of the design of experiments. CRC Press, Boca Raton

    MATH  Google Scholar 

  • Elder GH (1998) The life course as developmental theory. Child Dev 69(1):1–12

    Article  Google Scholar 

  • Freedman LS, Green SB, Byar DP (1990) Assessing the gain in efficiency due to matching in a community intervention study. Stat Med 9(8):943–952

    Article  Google Scholar 

  • Fuller WA (1987) Measurement error models. Wiley, New York

    MATH  Book  Google Scholar 

  • Fuller WA (1995) Estimation in the presence of measurement error. Int Stat Rev (Revue Internationale de Statistique) 63(2):121–141

    MATH  Article  Google Scholar 

  • Hansen MH, Hurwitz WN, Bershad MA (1961) Measurement errors in censuses and surveys. Bull Inst Int Stat 38(2):359–374

    Google Scholar 

  • Haviland AM, Nagin DS (2005) Causal inferences with group based trajectory models. Psychometrika 70(3):557–578

    MathSciNet  MATH  Article  Google Scholar 

  • Heckman JJ (1979) Sample selection bias as a specification error. Econom J Econom Soc 47(1):153–161

    MathSciNet  MATH  Google Scholar 

  • Hedges LV (2007) Correcting a significance test for clustering. J Educ Behav Stat 32(2):151–179

    Article  Google Scholar 

  • Heimberg RG, Stein MB, Hiripi E, Kessler RC (2000) Trends in the prevalence of social phobia in the united states: a synthetic cohort analysis of changes over four decades. Eur Psychiatry 15(1):29–37

    Article  Google Scholar 

  • Hong G, Raudenbush SW (2006) Evaluating kindergarten retention policy. J Am Stat Assoc 101(475):901–910

    MATH  Article  Google Scholar 

  • Huberty CJ, Olejnik S (2006) Applied MANOVA and discriminant analysis. Wiley, New York

    MATH  Book  Google Scholar 

  • International Association for the Evaluation of Educational Achievement (1977) The second international mathematics study. IEA, Amsterdam

    Google Scholar 

  • Jakubowski M et al (2015) Latent variables and propensity score matching: a simulation study with application to data from the programme for international student assessment in poland. Empir Econ 48(3):1287–1325

    MathSciNet  Article  Google Scholar 

  • Jöreskog KG, Sörbom D (1996) LISREL 8: user’s reference guide. Scientific software International, Chicago

    Google Scholar 

  • Kaplan D (1999) An extension of the propensity score adjustment method for the analysis of group differences in mimic models. Multivar Behav Res 34(4):467–492

    Article  Google Scholar 

  • Kaplan D (2008) Structural equation modeling: foundations and extensions. Sage Publications, Thousand Oaks

    MATH  Google Scholar 

  • Kessler RC, Stein MB, Berglund P (1998) Social phobia subtypes in the national comorbidity survey. Am J Psychiatry 155(5):613–619

    Article  Google Scholar 

  • Lee SY (2007) Structural equation modeling: a Bayesian approach. Wiley, New York

    Book  Google Scholar 

  • Leon AC, Hedeker D (2005) A mixed-effects quintile-stratified propensity adjustment for effectiveness analyses of ordered categorical doses. Stat Med 24(4):647–658

    MathSciNet  Article  Google Scholar 

  • Li YP, Propert KJ, Rosenbaum PR (2001) Balanced risk set matching. J Am Stat Assoc 96(455):870–882

    MathSciNet  MATH  Article  Google Scholar 

  • Lord FM (1980) Applications of item response theory to practical testing problems. Lawrence Erlbaum Associates, Mahwah

    Google Scholar 

  • Lord FM, Novick MR (1968) Statistical theories of mental test scores. Addison-Wesley Publishing Company, Boston

    MATH  Google Scholar 

  • Mahalanobis PC (1946) A sample survey of after-effects of the bengal famine of 1943. Sankhy Indian J Stat 7(4):337–400

    Google Scholar 

  • McCaffrey DF, Lockwood JR, Setodji CM (2013) Inverse probability weighting with error-prone covariates. Biometrika 100(3):671–680

    MathSciNet  MATH  Article  Google Scholar 

  • Muthén LK, Muthén BO (1998–2012) Mplus user’s guide. Muthén & Muthén

  • Muthén BO (1994) Multilevel covariance structure analysis. Sociol Methods Res 22(3):376–398

    Article  Google Scholar 

  • Muthén BO, KG Jöreskog (1983) Selectivity problems in quasi-experimental studies. Eval Rev 7(2):139–174

    Article  Google Scholar 

  • Raab GM, Butcher I (2001) Balance in cluster randomized trials. Stat Med 20(3):351–365

    Article  Google Scholar 

  • Raudenbush SW, Liu XF (2001) Effects of study duration, frequency of observation, and sample size on power in studies of group differences in polynomial change. Psychol Methods 6(4):387–401

    Article  Google Scholar 

  • Reckase M (2009) Multidimensional item response theory, vol 150. Springer, Berlin

    MATH  Book  Google Scholar 

  • Rosenbaum PR (1986) Dropping out of high school in the united states: an observational study. J Educ Behav Stat 11(3):207–224

    Article  Google Scholar 

  • Rosenbaum PR, Rubin DB (1983) The central role of the propensity score in observational studies for causal effects. Biometrika 70(1):41–55

    MathSciNet  MATH  Article  Google Scholar 

  • Rubin DB (1973) Matching to remove bias in observational studies. Biometrics 29(1):159–183

    Article  Google Scholar 

  • Rubin DB (1978) Bayesian inference for causal effects: the role of randomization. Ann Stat 6(1):34–58

    MathSciNet  MATH  Article  Google Scholar 

  • Schafer JL, Graham JW (2002) Missing data: our view of the state of the art. Psychol Methods 7(2):147 (ISSN 1939-1463)

    Article  Google Scholar 

  • Schmidt WH, Burstein L (1992) Concomitants of growth in mathematics achievement during the population a school year. In: Burstein L (ed) The IEA study of mathematics III: student growth and classroom processes. Pergamon Press, Oxford, pp 309–327

    Chapter  Google Scholar 

  • Schmidt WH, Houang TR (1986) Ein vergleich von drei analyseverfahren fur hierarchist strukturierte daten. In: Saldern MV (ed) Mehrebenenanalyse. PVU, Weinheim, pp 71–81

    Google Scholar 

  • Solomon RL (1949) An extension of control group design. Psychol Bull 46(2):137–150

    Article  Google Scholar 

  • Song M, Herman R (2010) Critical issues and common pitfalls in designing and conducting impact studies in education. Educ Eval Policy Anal 32(3):351–371

    Article  Google Scholar 

  • Spiegelman D, Schneeweiss S, McDermott A (1997) Measurement error correction for logistic regression models with an “alloyed gold standard”. Am J Epidemiol 145(2):184

    Article  Google Scholar 

  • Steiner PM, Cook TD, Shadish WR (2011) On the importance of reliable covariate measurement in selection bias adjustments using propensity scores. J Educ Behav Stat 36(2):213–236

    Article  Google Scholar 

  • Stuart EA, Rubin DB (2008) Matching with multiple control groups with adjustment for group differences. J Educ Behav Stat 33(3):279–306

    Article  Google Scholar 

  • Tatsuoka MM (1971) Multivariate analysis: techniques for educational and psychological research. Wiley, New York

    MATH  Google Scholar 

  • Wang Q (2015) Propensity score matching on multilevel data. In: Pan W, Bai H (eds) Propensity score analysis: fundamentals and developments. Guilford, New York, pp 217–235

    Google Scholar 

  • Wang Q, Maier K, Houang R (2017a) Omitted variables, R 2, and bias reduction in matching hierarchical data: a monte carlo study. J Stat Adv Theory Appl 17(1):43–81

    Article  Google Scholar 

  • Wang Q, Houang R, Maier K (2017b) Bias reduction rates for latent variable matching versus matching through surrogate variables with measurement errors. Interdiscip Educ Psychol 1(1):9

    Article  Google Scholar 

  • Webb-Vargas Y, Rudolph KE, Lenis D, Murakami P, Stuart EA (2015) An imputation-based solution to using mismeasured covariates in propensity score analysis. Stat Methods Med Res 26(4):1824–1837. https://doi.org/10.1177/0962280215588771

    MathSciNet  Article  Google Scholar 

  • Wiley DE, Wolfe RG (1992) Major survey design issues for the IEA third international mathematics and science study. Prospects 22(3):297–304

    Article  Google Scholar 

  • Wolfe RG (1987) Second international mathematics study: training manual for use of the databank of the longitudinal, classroom process surveys for population a in the IEA second international mathematics study. Contractor’s report. Center for Education Statistics, Washington, DC

    Google Scholar 

  • Wooldridge JM (2002) Econometric analysis of cross section and panel data. MIT Press, Cambridge

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Qiu Wang.

Additional information

Communicated by Hsiu-Ting Yu

This paper is based on work supported by the National Science Foundation under Grant no. DUE-0831581. Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Appendix 1: Variance–covariance decomposition of the extended Solomon four-group design (SFGD) based on two-level SEM framework

Appendix 1: Variance–covariance decomposition of the extended Solomon four-group design (SFGD) based on two-level SEM framework

Appendix 1.1: Variance–covariance matrix of SFGD experimental group 1

The following are the variance–covariance matrix of Y and X in the pre- and post-test design—SFGD’s experimental

$$\begin{aligned} \begin{array}{l} {\mathbb{COV}}(Y,X)=\left[ {{\begin{array}{ll} {{\mathbb{V}}(Y)} & \\ {{\mathbb{COV}}(Y,X)} & {\mathbb {V}(X)} \\ \end{array} }} \right] =\left[ {{\begin{array}{llll} {{\mathbb{V}}(Y_0 )} & & & \\ {{\mathbb{COV}}(Y_0 ,Y_1 )} & {{\mathbb{V}}(Y_1 )} & & \\ {{\mathbb{COV}}(Y_0 ,X_0 )} & {{\mathbb{COV}}(Y_1 ,X_0 )} &{{\mathbb{V}}(X_0 )} & \\ {{\mathbb{COV}}(Y_0 ,X_1 )} & {{\mathbb{COV}}(Y_1 ,X_1 )} & {{\mathbb{COV}}(X_0 ,X_1 )} & {{\mathbb{V}}(X_1 )} \\ \end{array} }} \right] \\ =\left[ {{\begin{array}{llll} {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^B \Pi '_0 +\Psi _{u_0 }^B ){\Lambda _0} ^{'}+\Theta _{\varepsilon _0 }^B } & & &\\ {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^B \Pi '_0 +\Psi _{u_0 }^B ){{\Lambda } }'_1 {\mathcal{V}}'} & {{\Lambda _1} (\Pi _1 {\mathcal{K}} \Psi _{\xi _0 }^B {\mathcal{K}}' \Pi '_1+\Psi _{u_1 }^B ){\Lambda _1} ^{'}+\Theta _{\varepsilon _1 }^B } & & \\ {{\Lambda _0} \Pi _0 \Psi _{\xi _0 }^B {\Gamma }'_0 } & {{\Lambda _1} \Pi _1 {\mathcal{K}} \Psi _{\xi _0 }^W {\Gamma }'_0 } & {\Gamma _0 \Psi _{\xi _0 }^B {\Gamma }'_0 +\Theta _{\zeta _0 }^B } & \\ {{\Lambda _0} \Pi _0 {\mathcal{K}} \Psi _{\xi _0 }^B {\Gamma }'_1 } & {{\Lambda _1} \Pi _1 {\mathcal{K}} \Psi _{\xi _0 }^B {\mathcal{K} }'{\Gamma }'_1 } &{\Gamma _0 \Psi _{\xi _0 }^B {\Gamma }'_1 {\mathcal{K} }'} &{\Gamma _1 {\mathcal{K}} \Psi _{\xi _0 }^B {\mathcal{K} }^{'} {\Gamma }'_1 +\Theta _{\zeta _1 }^B } \\ \end{array} }} \right] \\ +\, \left[ {{\begin{array}{llll} {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^W \Pi _0 ^{'}+\Psi _{u_0 }^W ){\Lambda _0} ^{'}+\Theta _{\varepsilon _0 }^W } &&&\\ {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^W \Pi'_0+\Psi _{u_0 }^W ){{\Lambda } }'_1 {\mathcal{V}}'} & {{\Lambda _1} (\Pi _1 {\mathcal{K}} \Psi _{\xi _0 }^W {\mathcal{K} }' \Pi _1 ^{'}+\Psi _{u_1 }^W ){\Lambda _1} ^{'}+\Theta _{\varepsilon _1 }^W } &&\\ {{\Lambda _0} \Pi _0 \Psi _{\xi _0 }^W {\Gamma }'_0 } & {{\Lambda _1} \Pi _1 {\mathcal{K}} \Psi _{\xi _0 }^W {\Gamma }'_0 } & {\Gamma _0 \Psi _{\xi _0 }^W {\Gamma }'_0 +\Theta _{\zeta _0 }^W } & \\ {{\Lambda _0} \Pi _0 {\mathcal{K}} \Psi _{\xi _0 }^W {\Gamma }'_1 } & {{\Lambda _1} \Pi _1 {\mathcal{K}} \Psi _{\xi _0 }^W {\mathcal{K}}'{\Gamma }'_1 } &{\Gamma _0 \Psi _{\xi _0 }^W {\Gamma }'_1 {\mathcal{K} }'} & {\Gamma _1 {\mathcal{K}} \Psi _{\xi _0 }^W {\mathcal{K} }' {\Gamma }'_1 +\Theta _{\zeta _1 }^W } \\ \end{array} }} \right] \\ \end{array} \end{aligned}$$

Appendix 1.2: Variance–covariance matrix of SFGD experimental group 2

$$ \begin{aligned} \begin{array}{ll} {\mathbb{COV}}(Y,X)= & \left[ {{\begin{array}{ll} {\mathbb{V}(Y)} & \\ {\mathbb{COV}(Y,X)} & {\mathbb{V}(X)} \\ \end{array} }} \right] \end{array}\\= & \left[ \begin{array}{llll} {\mathbb{V}(Y_0 )} & & & \\ {\mathbb{COV}(Y_0 ,Y_1 )} & {\mathbb{V}(Y_1 )} & & \\ {\mathbb{COV}(Y_0 ,X_0 )} & {\mathbb{COV}(Y_1 ,X_0 )} & {\mathbb{V}(X_0 )} & \\ {\mathbb{COV}(Y_0 ,X_1 )} & {\mathbb{COV}(Y_1 ,X_1 )} & {\mathbb{COV}(X_0 ,X_1 )} & {\mathbb{V}(X_1 )} \\ \end{array} \right]\\ = & \left[ {{\begin{array}{llll} {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^B \Pi _0 ^{'}+\Psi _{u_0 }^B ){\Lambda _0} ^{'}+\Theta _{\varepsilon _0 }^B } & & & \\ {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^B \Pi _0 ^{'}+\Psi _{u_0 }^B ){{\Lambda } }'_1 } & {{\Lambda _1} (\Pi _1 {\mathcal {K}} \Psi _{\xi _0 }^B {{\mathcal {K}} }' \Pi _1 ^{'}+\Psi _{u_1 }^B ){\Lambda _1} ^{'}+\Theta _{\varepsilon _1 }^B } & & \\ {{\Lambda _0} \Pi _0 \Psi _{\xi _0 }^B {\Gamma }'_0 } & {{\Lambda _1} \Pi _1 {\mathcal {K}} \Psi _{\xi _0 }^W {\Gamma }'_0 } & {\Gamma _0 \Psi _{\xi _0 }^B {\Gamma }'_0 +\Theta _{\zeta _0 }^B } & \\ {{\Lambda _0} \Pi _0 {\mathcal {K}} \Psi _{\xi _0 }^B {\Gamma }'_1 } & {{\Lambda _1} \Pi _1 {\mathcal {K}} \Psi _{\xi _0 }^B {{\mathcal {K}} }'{\Gamma }'_1 } & {\Gamma _0 \Psi _{\xi _0 }^B {\Gamma }'_1 {{\mathcal {K}} }'} & {\Gamma _1 {\mathcal {K}} \Psi _{\xi _0 }^B {{\mathcal {K}} }' {\Gamma }'_1 +\Theta _{\zeta _1 }^B } \\ \end{array} }} \right] \\&+ \left[ {{\begin{array}{llll} {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^W \Pi _0 ^{'}+\Psi _{u_0 }^W ){\Lambda _0} ^{'}+\Theta _{\varepsilon _0 }^W } & & & \\ {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^W \Pi _0 ^{'}+\Psi _{u_0 }^W ){{\Lambda } }'_1 } & {{\Lambda _1} (\Pi _1 {\mathcal {K}} \Psi _{\xi _0 }^W {{\mathcal {K}} }' \Pi _1 ^{'}+\Psi _{u_1 }^W ){\Lambda _1} ^{'}+\Theta _{\varepsilon _1 }^W } & & \\ {{\Lambda _0} \Pi _0 \Psi _{\xi _0 }^W {\Gamma }'_0 } & {{\Lambda _1} \Pi _1 {\mathcal {K}} \Psi _{\xi _0 }^W {\Gamma }'_0 } & {\Gamma _0 \Psi _{\xi _0 }^W {\Gamma }'_0 +\Theta _{\zeta _0 }^W } & \\ {{\Lambda _0} \Pi _0 {\mathcal {K}} \Psi _{\xi _0 }^W {\Gamma }'_1 } & {{\Lambda _1} \Pi _1 {\mathcal {K}} \Psi _{\xi _0 }^W {{\mathcal {K}} }'{\Gamma }'_1 } & {\Gamma _0 \Psi _{\xi _0 }^W {\Gamma }'_1 {{\mathcal {K}} }'} & {\Gamma _1 {\mathcal {K}} \Psi _{\xi _0 }^W {{\mathcal {K}} }' {\Gamma }'_1 +\Theta _{\zeta _1 }^W } \\ \end{array} }} \right] \\ \end{aligned}$$

Appendix 1.3: Variance–covariance matrix of SFGD control group 1

$$\begin{aligned} \mathbb {COV}(Y,X)= & {} \left[ {{\begin{array}{ll} {\mathbb {V}(Y)} &{} \\ {\mathbb {COV}(Y,X)} &{} {\mathbb {V}(X)} \\ \end{array} }} \right] \\= & {} \left[ {{\begin{array}{llll} &{} &{} &{} \\ &{} {\mathbb {V}(Y_1 )} &{} &{} \\ &{} &{} &{} \\ &{} {\mathbb {COV}(Y_1 ,X_1 )} &{} &{} {\mathbb {V}(X_1 )} \\ \end{array} }} \right] \\= & {} \left[ {{\begin{array}{llll} &{} &{} &{} \\ &{} {{\Lambda _1} (\Pi _1 \Psi _{\xi _1 }^B \Pi _1 ^{'}+\Psi _{u_1 }^B ){\Lambda _1} ^{'}+\Theta _{\varepsilon _1 }^B } &{} &{} \\ &{} &{} &{} \\ &{} {{\Lambda _1} \Pi _1 \Psi _{\xi _1 }^B {\Gamma }'_1 } &{} &{} {\Gamma _1 \Psi _{\xi _1 }^B {\Gamma }'_1 +\Theta _{\zeta _1 }^B } \\ \end{array} }} \right] \\\\&+ \left[ {{\begin{array}{llll} &{} &{} &{} \\ &{} {{\Lambda _1} (\Pi _1 \Psi _{\xi _1 }^W \Pi _1 ^{'}+\Psi _{u_1 }^W ){\Lambda _1} ^{'}+\Theta _{\varepsilon _1 }^W } &{} &{} \\ &{} &{} &{} \\ &{} {{\Lambda _1} \Pi _1 \Psi _{\xi _1 }^W {\Gamma }'_1 } &{} &{} {\Gamma _1 \Psi _{\xi _1 }^W {\Gamma }'_1 +\Theta _{\zeta _1 }^W } \\ \end{array} }} \right] \\ \end{aligned}$$

Appendix 1.4: Variance–covariance matrix of SFGD control group 2

$$\begin{aligned} \mathbb {COV}(Y,X)= & {} \left[ {{\begin{array}{ll} {\mathbb {V}(Y)} &{} \\ {\mathbb {COV}(Y,X)} &{} {\mathbb {V}(X)} \\ \end{array} }} \right] \\= & {} \left[ {{\begin{array}{llll} &{} &{} &{} \\ &{} {\mathbb {V}(Y_1 )} &{} &{} \\ &{} &{} &{} \\ &{} {\mathbb {COV}(Y_1 ,X_1 )} &{} &{} {\mathbb {V}(X_1 )} \\ \end{array} }} \right] \\= & {} \left[ {{\begin{array}{llll} &{} &{} &{} \\ &{} {{\Lambda _1} (\Pi _1 \Psi _{\xi _1 }^B \Pi _1 ^{'}+\Psi _{u_1 }^B ){\Lambda _1} ^{'}+\Theta _{\varepsilon _1 }^B } &{} &{} \\ &{} &{} &{} \\ &{} {{\Lambda _1} \Pi _1 \Psi _{\xi _1 }^B {\Gamma }'_1 } &{} &{} {\Gamma _1 \Psi _{\xi _1 }^B {\Gamma }'_1 +\Theta _{\zeta _1 }^B } \\ \end{array} }} \right] \\&+ \left[ {{\begin{array}{llll} &{} &{} &{} \\ &{} {{\Lambda _1} (\Pi _1 \Psi _{\xi _1 }^W \Pi _1 ^{'}+\Psi _{u_1 }^W ){\Lambda _1} ^{'}+\Theta _{\varepsilon _1 }^W } &{} &{} \\ &{} &{} &{} \\ &{} {{\Lambda _1} \Pi _1 \Psi _{\xi _1 }^W {\Gamma }'_1 } &{} &{} {\Gamma _1 \Psi _{\xi _1 }^W {\Gamma }'_1 +\Theta _{\zeta _1 }^W } \\ \end{array} }} \right] \\ \end{aligned}$$

Appendix 1.5: Detailed variance–covariance decomposition

This appendix discusses the detailed variance–covariance decomposition of the SFGD’s experimental group 1. The measurement model is defined in Eq. (14), where \(\varepsilon \sim N(0,\Theta _{\varepsilon })\). \(\varepsilon\) is independent of \(\eta\), \(\xi\) and \(\zeta\). \(\zeta \sim N(0,\Theta _{\zeta })\) is independent of \(\eta\), \(\xi\) and \(\varepsilon\). The latent variable \(\xi\) is hierarchically structured and includes both within-cluster and between-cluster components. The latent variable \(\eta\) of Y is hierarchically structured (Muthén 1994, p. 379). This is because \(\eta\) has a functional relationship with \(\xi\) in the structural model.

If a is the intercept vector and \(\Pi\) the loading matrix, then the structural model (Jöreskog and Sörbom 1996) is written as Eq. (15), where \(u\sim N(0,\Theta _u)\) is independent of \(\xi\), \(\varepsilon\) and \(\zeta\). The independence assumption will be used in the computation of the covariance of Y and X. This model is a two-level factor analysis model (Muthén and Muthén 1998–2012). Correspondingly, based on the SEM framework, data variation can be decomposed into within- and between-cluster components (Muthén 1994, p. 380).

Appendix 1.5.1: Decomposition of variation of X

The variation of the latent variable \(\xi\) can be decomposed as

$$\begin{aligned} \mathbb {V}(\xi )=\Psi _\xi =\Psi _\xi ^B +\Psi _\xi ^W. \end{aligned}$$
(30)

The variation of X’s residual can be decomposed into between- and within-cluster components. That is,

$$\begin{aligned} \mathbb {V}(\zeta )=\Theta _{\zeta }^B +\Theta _{\zeta }^W. \end{aligned}$$
(31)

The variation of outcome X is decomposed as

$$\begin{aligned} \mathbb {V}(X)=\Phi _X^B +\Phi _X^W, \end{aligned}$$
(32)

with

$$\begin{aligned} \Phi _X^B =\Gamma \Psi _\xi ^B \Gamma ^{'}+\Theta _{\zeta }^B, \end{aligned}$$
(33)

and

$$\begin{aligned} \Phi _X^W =\Gamma \Psi _\xi ^W \Gamma ^{'}+\Theta _{\zeta }^W. \end{aligned}$$
(34)

Appendix 1.5.2: Decomposition of latent variable \(\eta\)

The variation of \(\eta\) can be decomposed using the structural model. First, the residual variance is decomposed as

$$\begin{aligned} {\mathbb {V}(u)=\Theta _u^B +\Theta _u^W}. \end{aligned}$$
(35)

The variation of \(\eta\) is decomposed as

$$\begin{aligned} \mathbb {V}(\eta )=\Psi _\eta =\Psi _\eta ^B +\Psi _\eta ^W, \end{aligned}$$
(36)

with

$$\begin{aligned} \Psi _\eta ^B =\Pi \Psi _\xi ^B \Pi ^{'}+{\Psi _u^B}, \end{aligned}$$
(37)

and

$$\begin{aligned} \Psi _\eta ^W =\Pi \Psi _\xi ^W \Pi ^{'}+{\Psi _u^W}. \end{aligned}$$
(38)

Appendix 1.5.3: Decomposition of variation of Y

The variation of Y’s residual can also be decomposed into between- and within-cluster components,

$$\begin{aligned} \mathbb {V}(\varepsilon )=\Theta _{\varepsilon }^B +\Theta _{\varepsilon }^W. \end{aligned}$$
(39)

Now the variation of variable Y is decomposed as

$$\begin{aligned} \mathbb {V}(Y)=\Phi _Y^B +\Phi _Y^W, \end{aligned}$$
(40)

with

$$\begin{aligned} \Phi _Y^B ={\Lambda } \Psi _\eta ^B {\Lambda } ^{'}+\Theta _{\varepsilon }^B, \end{aligned}$$
(41)

and

$$\begin{aligned} \Phi _Y^W ={\Lambda } \Psi _\eta ^W {\Lambda } ^{'}+\Theta _{\varepsilon }^W. \end{aligned}$$
(42)

Appendix 1.5.4: Covariance of Y and X

Based on independence assumptions in the measurement and structural models above, the covariance of Y and X is computed as:

$$\begin{aligned} {\rm COV}[Y,X]= & {} {\rm COV}[{\Lambda }{(a+\Pi \xi +u)}+\varepsilon ,v+\Gamma \xi +\zeta ] \nonumber \\= & {} {\Lambda \Pi \Gamma }Var(\xi )\nonumber \\= & {} {{\Lambda }\Pi \Gamma }\Psi _\xi ^B+{{\Lambda } \Pi \Gamma }\Psi _\xi ^W. \end{aligned}$$
(43)

Thus, the whole data variance–covariance matrix is shown below:

$$\begin{aligned} \mathbb {COV}(Y,X)= & {} \left[ {{\begin{array}{ll} {\mathbb {V}(Y)} &{} \\ {{\rm COV}[Y,X]} &{} {\mathbb {V}(X)} \\ \end{array} }} \right] =\left[ {{\begin{array}{ll} {\Phi _Y^B +\Phi _Y^W } &{} \\ {{{\Lambda } \Pi \Gamma }\Psi _\xi ^B +{{\Lambda } \Pi \Gamma }\Psi _\xi ^W } &{} {\Phi _X^B +\Phi _X^W } \\ \end{array} }} \right] \\= & {} \left[ {{\begin{array}{ll} {\Phi _Y^B } &{} \\ {{{\Lambda } \Pi \Gamma }\Psi _\xi ^B } &{} {\Phi _X^B } \\ \end{array} }} \right] +\left[ {{\begin{array}{ll} {\Phi _Y^W } &{} \\ {{\Lambda \Pi \Gamma }\Psi _\xi ^W } &{} {\Phi _X^W } \\ \end{array} }} \right] \\= & {} \left[ {{\begin{array}{ll} {{\Lambda } \Psi _\eta ^B {\Lambda } ^{'}+\Theta _{\varepsilon }^B } &{} \\ {{\Lambda \Pi \Gamma }\Psi _\xi ^B } &{} {\Gamma \Psi _\xi ^B \Gamma ^{'}+\Theta _{\zeta }^B } \\ \end{array} }} \right] +\left[ {{\begin{array}{ll} {{\Lambda } \Psi _\eta ^W {\Lambda } ^{'}+\Theta _{\varepsilon }^W } &{} \\ {{\Lambda \Pi \Gamma }\Psi _\xi ^W } &{} {\Gamma \Psi _\xi ^W \Gamma ^{'}+\Theta _{\zeta }^W } \\ \end{array} }} \right] \\= & {} \left[ {{\begin{array}{ll} {{{\Lambda } (\Pi }\Psi _\xi ^B {\Pi ^{'}}+\Psi _E^B ){\Lambda } ^{'}+\Theta _{\varepsilon }^B } &{} \\ {{\Lambda \Pi \Gamma }\Psi _\xi ^B } &{} {\Gamma \Psi _\xi ^B \Gamma ^{'}+\Theta _{\zeta }^B } \\ \end{array} }} \right] \\&+ \left[ {{\begin{array}{ll} {{{\Lambda } (\Pi }\Psi _\xi ^W {\Pi ^{'}}+\Psi _E^W ){\Lambda } ^{'}+\Theta _{\varepsilon }^W } &{} \\ {{\Lambda \Pi \Gamma }\Psi _\xi ^W } &{} {\Gamma \Psi _\xi ^W \Gamma ^{'}+\Theta _{\zeta }^W } \\ \end{array} }} \right] \end{aligned}$$

Appendix 1.5.5: Variance–covariance decomposition across times

This is the temporal decomposition of variance–covariance in Appendix 1.1.

Note the measurement model in Eq. (14), \(\varepsilon \sim N(0,\Theta _{\varepsilon })\). \(\varepsilon\) is independent of \(\eta\), \(\xi\) and \(\zeta\). \(\zeta \sim N(0,\Theta _{\zeta })\) is independent of \(\eta\), \(\xi\) and \(\varepsilon\). Rewrite each component in the model into two parts. One part represents the measure collected at time 0 and the other in time 1. For the first equation, \(Y=\left( {{\begin{array}{c} {Y_0 }\\ {Y_1 }\\ \end{array} }} \right)\), \(\delta =\left( {{\begin{array}{c} {\delta _0 }\\ {\delta _1 }\\ \end{array} }} \right)\), \({\Lambda } =\left( {{\begin{array}{c} {{\begin{array}{c} {{\Lambda _0} } \\ 0 \\ \end{array} }} \quad {{\begin{array}{c} 0 \\ {{\Lambda _1} }\\ \end{array} }} \\ \end{array} }} \right)\), \(\eta =\left( {{\begin{array}{c} {\eta _0 } \\ {\eta _1 } \\ \end{array} }} \right)\), \(\varepsilon =\left( {{\begin{array}{c} {\varepsilon _0 }\\ {\varepsilon _1 }\\ \end{array} }} \right)\). Variation of variable \(Y_t\) is decomposed as

$$\begin{aligned} {\mathbb {V}}(Y_t)=\Phi _{Y_t }^B +\Phi _{Y_t }^W, \end{aligned}$$
(44)

with

$$\begin{aligned} \Phi _{Y_t }^B ={\Lambda }_t \Psi _{\eta _t }^B {\Lambda }_t ^{'}+\Theta _{\varepsilon _{t} }^B, \end{aligned}$$
(45)

and

$$\begin{aligned} \Phi _{Y_t}^W ={\Lambda }_t \Psi _{\eta _t}^W {\Lambda }_t^{'}+\Theta _{\varepsilon _{t} }^W, \end{aligned}$$
(46)

for t = 0, 1.

Similarly, write

$$\begin{aligned} X=\left( {{\begin{array}{c} {X_0 } \\ {X_1 } \\ \end{array} }} \right) , {v}=\left( {{\begin{array}{c} {v_0 } \\ {v_1 } \\ \end{array} }} \right) , \Gamma =\left( {{\begin{array}{c} {{\begin{array}{*{20}c} {\Gamma _0 } \\ 0 \\ \end{array} }} \quad {{\begin{array}{c} 0 \\ {\Gamma _1 } \\ \end{array} }} \\ \end{array} }} \right) , \xi =\left( {{\begin{array}{c} {\xi _0 } \\ {\xi _1 } \\ \end{array} }} \right) , \zeta =\left( {{\begin{array}{c} {\zeta _0 } \\ {\zeta _1 } \\ \end{array} }} \right) . \end{aligned}$$

Correspondingly, the variation of outcome \(X_{t}\) is decomposed as

$$\begin{aligned} {\mathbb {V}}(X_t)=\Phi _{X_t }^B +\Phi _{X_t }^W, \end{aligned}$$
(47)

with

$$\begin{aligned} \Phi _{X_t }^B =\Gamma _t \Psi _{\xi _t }^B \Gamma _t^{'}+\Theta _{e_{2i}}^B, \end{aligned}$$
(48)

and

$$\begin{aligned} \Phi _{X_t }^W =\Gamma _t \Psi _{\xi _t }^W \Gamma _t ^{'}+\Theta _{e_{2i} }^W, \end{aligned}$$
(49)

for t = 0,1.

The latent variables \(\xi _0\), \(\xi _1\), \(\eta _0\), and \(\eta _1\) are hierarchically structured and include within-cluster and between-cluster components (Muthén 1994, p. 379). This is because \(\eta\) has a functional relationship with \(\xi\) in the structural model of Eq. (20), where \(U\sim N(0,\Theta _U)\) is independent of \(\xi\), \(\varepsilon\) and \(\zeta\). Thus, it results in

$$\begin{aligned} {\mathbb {V}}(\eta _0 )=\Psi _{\eta _0 } =\Psi _{\eta _0 }^B +\Psi _{\eta _0 }^W, \end{aligned}$$
(50)

with

$$\begin{aligned} \Psi _{\eta _0 }^B =\Pi _0 \Psi _{\xi _0 }^B \Pi _0 ^{'}+\Psi _{u_0 }^B, \end{aligned}$$
(51)

and

$$\begin{aligned} \Psi _{\eta _0 }^W =\Pi _0 \Psi _{\xi _0 }^W \Pi _0 ^{'}+\Psi _{u_0 }^W. \end{aligned}$$
(52)

Now, write the variance–covariance matrix as

$$\begin{aligned} {\mathbb {COV}}(Y,X)= & {} \left[ {{\begin{array}{ll} {\mathbb {V}(Y)} &{} \\ {\mathbb {COV}(Y,X)} &{} {\mathbb {V}}(X) \\ \end{array}}} \right] \\= & {} \left[ {{\begin{array}{llll} {\mathbb {V}}(Y_0 ) &{} &{} &{} \\ {\mathbb {COV}(Y_0 ,Y_1 )} &{} {\mathbb {V}}(Y_1 ) &{} &{} \\ {\mathbb {COV}}(Y_0 ,X_0 ) &{} {\mathbb {COV}}(Y_1 ,X_0 ) &{} {\mathbb {V}}(X_0 ) &{} \\ {\mathbb {COV}}(Y_0 ,X_1 ) &{} {\mathbb {COV}}(Y_1 ,X_1 ) &{} {\mathbb {COV}}(X_0 ,X_1 ) &{} {\mathbb {V}}(X_1 ) \\ \end{array} }}\right]\, . \end{aligned}$$

Appendix 1.5.6: Compute \({\mathbb {V}}(Y)\)

In order to determine \({\mathbb {COV}}(Y_0, Y_1)\), the structural relationship between \(\eta _1\), and \(\eta _1\) is displayed by Eq. (17), where slope \({\mathcal {V}}\) represents schooling effect and \(\gamma\) represents maturation effect; \(\tau\) represents the pre-test effect at time 1 (Solomon 1949).

Thus, \({\mathbb {COV}}(Y_0 ,Y_1 )={\mathbb {COV}}[\delta _1 +{\Lambda _0} \eta _0 +\varepsilon _0,\delta _2 +{\Lambda _1} \eta _1 +\varepsilon _1 )= {\mathbb {COV}}[{\Lambda _0} \eta _0 ,{\Lambda _1} (\tau +\gamma +{\mathcal {V}} \eta _0 )]={\Lambda _0} {\mathbb {V}}(\eta _0 ){{\Lambda } }'_1 {\mathcal {V} }',\) with \({\mathbb {COV}}[\varepsilon _0 ,\varepsilon _1 )=0\).

Plug in those components and write

$$\begin{aligned} {\mathbb {V}}(Y)= & {} {\mathbb {COV}}(Y,Y)\\ {}= & {} \left[ {{\begin{array}{ll} {\mathbb {V}}(Y_0 ) &{} \\ {\mathbb {COV}}(Y_0 ,Y_1 ) &{} {\mathbb {V}}(Y_1 ) \\ \end{array} }} \right] \\= & {} \left[ {{\begin{array}{ll} {\Phi _{Y_0 }^B +\Phi _{Y_0 }^W } &{} \\ {\mathbb {COV}}(Y_0 ,Y_1 ) &{} {\Phi _{Y_1 }^B +\Phi _{Y_1 }^W } \\ \end{array} }} \right] \\= & {} \left[ {{\begin{array}{lll} {{\Lambda _0} \Psi _{\eta _0 }^B {\Lambda _0} ^{'}+\Theta _{\varepsilon _0 }^B } &{} \\ {{\Lambda _0} \Psi _{\eta _0 }^B {{\Lambda } }'_1 {\mathcal {V}}'} &{} {{\Lambda _1} \Psi _{\eta _1 }^B {\Lambda _1} ^{'}+\Theta _{\varepsilon _1 }^B } \\ \end{array} }} \right] \\&+ \left[ {{\begin{array}{ll} {{\Lambda _0} \Psi _{\eta _0 }^W {\Lambda _0} ^{'}+\Theta _{\varepsilon _0 }^W } &{} \\ {{\Lambda _0} \Psi _{\eta _0 }^W {{\Lambda } }'_1 {\mathcal {V}}'} &{} {{\Lambda _1} \Psi _{\eta _1 }^W {\Lambda _1} ^{'}+\Theta _{\varepsilon _1 }^W } \\ \end{array} }} \right] \\= & {} \left[ {{\begin{array}{ll} {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^B \Pi _0 ^{'}+\Psi _{u_0 }^B ){\Lambda _0} ^{'}+\Theta _{\varepsilon _0 }^B } &{} \\ {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^B \Pi _0 ^{'}+\Psi _{u_0 }^B ){{\Lambda } }'_1 {\mathcal {V} }' } &{} {{\Lambda _1} (\Pi _1 \Psi _{\xi _1 }^B \Pi _1 ^{'}+\Psi _{u_1 }^B ){\Lambda _1} ^{'}+\Theta _{\varepsilon _1 }^B } \\ \end{array} }} \right] \\&+ \left[ {{\begin{array}{ll} {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^W \Pi _0 ^{'}+\Psi _{u_0 }^W ){\Lambda _0} ^{'}+\Theta _{\varepsilon _0 }^W } &{} \\ {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^W \Pi _0 ^{'}+\Psi _{u_0 }^W ){{\Lambda } }'_1 {{\mathcal {V}} }'} &{} {{\Lambda _1} (\Pi _1 \Psi _{\xi _1 }^W \Pi _1 ^{'}+\Psi _{u_1 }^W ){\Lambda _1} ^{'}+\Theta _{\varepsilon _1 }^W } \\ \end{array} }} \right] \\= & {} \left[ {{\begin{array}{ll} {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^B \Pi _0 ^{'}+\Psi _{u_0 }^B ){\Lambda _0} ^{'}+\Theta _{\varepsilon _0 }^B } &{} \\ {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^B \Pi _0 ^{'}+\Psi _{u_0 }^B ){{\Lambda } }'_1 {{\mathcal {V}} }' } &{} {{\Lambda _1} (\Pi _1 {\mathcal {K}} \Psi _{\xi _0 }^B {{\mathcal {K}} }'\Pi _1 ^{'}+\Psi _{u_1 }^B ){\Lambda _1} ^{'}+\Theta _{\varepsilon _1 }^B } \\ \end{array} }} \right] \\&+ \left[ {{\begin{array}{ll} {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^W \Pi _0 ^{'}+\Psi _{u_0 }^W ){\Lambda _0} ^{'}+\Theta _{\varepsilon _0 }^W } &{} \\ {{\Lambda _0} (\Pi _0 \Psi _{\xi _0 }^W \Pi _0 ^{'}+\Psi _{u_0 }^W ){{\Lambda } }'_1 {{\mathcal {V}} }' } &{} {{\Lambda _1} (\Pi _1 {\mathcal {K}} \Psi _{\xi _0 }^W {{\mathcal {K}} }'\Pi _1 ^{'}+\Psi _{u_1 }^W ){\Lambda _1} ^{'}+\Theta _{\varepsilon _1 }^W } \\ \end{array} }} \right] \\ \end{aligned}$$

Appendix 1.5.7: Compute \({\mathbb {V}}(X)\)

In order to determine \({\mathbb {COV}}(X_0 ,X_1 )\), the structural relationship between \(\xi _0\), and \(\xi _1\) is displayed by Eq. (19).

Thus, \({\mathbb {COV}}(X_0 ,X_1 )={\mathbb {COV}}[v_0 +\Gamma _0 \xi _0 +\zeta _0 ,v_1 +\Gamma _1 \xi _1 +\zeta _1 )={\mathbb {COV}}[\Gamma _0 \xi _0 ,\Gamma _1 (\beta +{\mathcal {K}} \xi _0 )]=\Gamma _0 {\mathbb {V}}(\xi _0 ){\Gamma }'_1 {{{\mathcal {K}}} }',\) with \({\mathbb {COV}}[\zeta _0 ,\zeta _1 )=0\), and

$$\begin{aligned} {\mathbb {V}}(\xi _0 )=\Psi _{\xi _0 }=\Psi _{\xi _0 }^B +\Psi _{\xi _0 }^W. \end{aligned}$$
(53)

Thus,

$$\begin{aligned} {\mathbb {V}}(X)= & {} {\mathbb {COV}}(X,X)\\= & {} \left[ {{\begin{array}{ll} {\mathbb {V}}(X_0 ) &{} \\ {\mathbb {COV}}(X_0 ,X_1 ) &{} {\mathbb {V}}(X_1) \\ \end{array} }} \right] \\= & {} \left[ {{\begin{array}{ll} {\Phi _{X_0 }^B +\Phi _{X_0 }^W } &{} \\ {\mathbb {COV}}(X_0 ,X_1 ) &{} {\Phi _{X_1 }^B +\Phi _{X_1 }^W } \\ \end{array} }} \right] \\= & {} \left[ {{\begin{array}{ll} {\Gamma _0 \Psi _{\xi _0 }^B {\Gamma }'_0 +\Theta _{\zeta _0 }^B } &{} \\ {\Gamma _0 \Psi _{\xi _0 }^B {\Gamma }'_1 {{\mathcal {K}} }'} &{} {\Gamma _1 \Psi _{\xi _1 }^B {\Gamma }'_1 +\Theta _{\zeta _1 }^B } \\ \end{array} }} \right] \\&+\left[ {{\begin{array}{*{20}c} {\Gamma _0 \Psi _{\xi _0 }^W {\Gamma }'_0 +\Theta _{\zeta _0 }^W } &{} \\ {\Gamma _0 \Psi _{\xi _0 }^W {\Gamma }'_1 {{\mathcal {K}} }'} &{} {\Gamma _1 \Psi _{\xi _1 }^W {\Gamma }'_1 +\Theta _{\zeta _1 }^W } \\ \end{array} }} \right] \\&=\left[ {{\begin{array}{ll} {\Gamma _0 \Psi _{\xi _0 }^B {\Gamma }'_0 +\Theta _{\zeta _0 }^B } &{} \\ {\Gamma _0 \Psi _{\xi _0 }^B {\Gamma }'_1 {{\mathcal {K}} }'} &{} {\Gamma _1 {\mathcal {K}} \Psi _{\xi _0 }^B {{\mathcal {K}} }'{\Gamma }'_1 +\Theta _{\zeta _1 }^B } \\ \end{array} }} \right] \\&+\left[ {{\begin{array}{ll} {\Gamma _0 \Psi _{\xi _0 }^W {\Gamma }'_0 +\Theta _{\zeta _0 }^W } &{} \\ {\Gamma _0 \Psi _{\xi _0 }^W {\Gamma }'_1 {{\mathcal {K}} }'} &{} {\Gamma _1 {\mathcal {K}} \Psi _{\xi _0 }^W {{\mathcal {K}} }'{\Gamma }'_1 +\Theta _{\zeta _1 }^W } \\ \end{array} }} \right] . \end{aligned}$$

Appendix 1.5.8: Compute \({\mathbb {COV}}(Y_t,X_{t^{'}})\)

Components of \({\mathbb {COV}}(Y_t ,X_{t^{'}})\)—for \(t,t^{'} = 0,1\)—are computed as

$$\begin{aligned} {\mathbb {COV}}(Y_t ,X_{t^{'}} )={\mathbb {COV}}[{\Lambda } _t (a_t +\Pi _t \xi _t +u_t )+\varepsilon _{t} ,v_t^{'} +\Gamma _t^{'} \xi _t^{'} +\zeta _{t^{'}} ]={\Lambda }_t \Pi _t{\mathbb {COV}}(\xi _t ,\xi _t^{'} ){\Gamma }'_{t^{'}}. \end{aligned}$$
(54)

Thus, the four components are computed as

$$\begin{aligned} {\mathbb {COV}}(Y_0 ,X_0 )={\Lambda _0} \Pi _0 (\Psi _{\xi _0 }^B +\Psi _{\xi _0 }^W ){\Gamma }'_0, \end{aligned}$$
(55)
$$\begin{aligned} {\mathbb {COV}}(Y_0 ,X_1 )={\Lambda _0} \Pi _0 {\mathcal {K}} (\Psi _{\xi _0 }^B +\Psi _{\xi _0 }^W ){\Gamma }'_1, \end{aligned}$$
(56)
$$\begin{aligned} {\mathbb {COV}}(Y_1 ,X_0 )={\Lambda _1} \Pi _1 {\mathcal {K}} (\Psi _{\xi _0 }^B +\Psi _{\xi _0 }^W ){\Gamma }'_0, \end{aligned}$$
(57)

and

$$\begin{aligned} {\mathbb {COV}}(Y_1 ,X_1 )={\Lambda _1} \Pi _1 {\mathcal {K}} (\Psi _{\xi _0 }^B +\Psi _{\xi _0 }^W ){{\mathcal {K}} }'{\Gamma }'_1. \end{aligned}$$
(58)

These procedures derive all components displayed in the matrices of Appendix 1.1. Other variance–covariance matrices in Appendices 1.21.4 can be derived with similar procedures.

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wang, Q., Houang, R.T. & Maier, K. Multilevel structural equation modeling-based quasi-experimental synthetic cohort design. Behaviormetrika 45, 261–294 (2018). https://doi.org/10.1007/s41237-018-0053-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41237-018-0053-0

Keywords

  • Propensity score matching
  • Solomon four-group design
  • Multilevel analysis
  • Quasi-longitudinal design
  • Causal inference
  • Multilevel structural equation modeling
  • Matching
  • Synthetic cohort design

Mathematics Subject Classification

  • 62-P25
  • 62B15
  • 62-07