Equivalence of two least-squares estimators for indirect effects

Abstract

In social and behavioral sciences, the mediation test based on the indirect effect is an important topic. There are many methods to assess intervening variable effects. In this paper, we focus on the difference method and the product method in mediation models. Firstly, we analyze the regression functions in the simple mediation model, and provide an expectation-consistent condition. We further show that the difference estimator and the product estimator are numerically equivalent based on the least-squares regression regardless of the error distribution. Secondly, we generalize the equivalence result to the three-path model and the multiple mediators model, and prove a general equivalence result in a class of restricted linear mediation models. Thirdly, we investigate the empirical distributions of the indirect effect estimators in the simple mediation model by simulations, and show that the indirect effect estimators are normally distributed as long as one multiplicand of the product estimator is large. Finally, we introduce some popular R packages for mediation analysis and also provide some useful suggestions on how to correctly conduct mediation analysis.

Appendices

Appendix : Appendix A: Proof of Theorem 2

Proof

We first consider the simple case where β₀ = β₁ = β₂ = 0. The least-squares estimators for the simplified models are

$$ \begin{array}{@{}rcl@{}} \hat{c}&=&\arg\underset{c}{\min}\sum\limits_{i=1}^{n}(Y_{i}-cX_{i})^{2}=\frac{X^{T}Y}{X^{T}X}, \\ \hat{a}&=&\arg\underset{a}{\min}\sum\limits_{i=1}^{n}(M_{i}-aX_{i})^{2}=\frac{X^{T}M}{X^{T}X}, \\ (\hat{c}^{\prime}, \hat{b})^{T}&=&\arg{\min_{c^{\prime}, b}}\sum\limits_{i=1}^{n}(Y_{i}-c^{\prime}X_{i}-bM_{i})^{2}\\&=&\frac{Y\left( \begin{array}{c} M^{T}MX^{T}-X^{T}MM^{T} \\ X^{T}XM^{T}-X^{T}MX^{T} \end{array}\right)}{X^{T}XM^{T}M-X^{T}MX^{T}M}. \end{array} $$

where X = (X₁,…,X_n)^T, M = (M₁,…,M_n)^T, and Y = (Y₁,…,Y_n)^T. By the above least-squares estimators, the difference estimator is

$$ \hat{c}-\hat{c}^{\prime} =\frac{X^{T}M(X^{T}XM^{T}-X^{T}MX^{T})Y}{X^{T}X(X^{T}XM^{T}M-X^{T}MX^{T}M)}, $$

and the product estimator is

$$ \hat{a}\hat{b}=\frac{X^{T}M(X^{T}XM^{T}-X^{T}MX^{T})Y}{X^{T}X(X^{T}XM^{T}M-X^{T}MX^{T}M)}. $$

This shows that $\hat {c}-\hat {c}^{\prime }=\hat {a}\hat {b}$. That is, the difference estimator is equivalent to the product estimator for the linear regression models with zero intercept.

The proof can readily be generalized to the models with non-zero intercept by replacing X and M by their demeaned couterparts and so is omitted. □

Appendix : Appendix B: Proof of Theorems 3

Proof

By substituting Eqs. 13 and 14 into Eq. 15, it follows that

$$ \begin{array}{@{}rcl@{}} Y_{i}&=&\beta_{3}+c^{\prime}X_{i}+b_{1}M_{1, i}\\ &&+b_{2}(\beta_{2}+dM_{1, i}+a_{2}X_{i}+\epsilon_{2, i})+\epsilon_{3, i} \\ &=&(\beta_{3}+b_{2}\beta_{2})+(c^{\prime}+a_{2}b_{2})X_{i}+(b_{1}+b_{2}d)M_{1, i}\\ &&+(b_{2}\epsilon_{2, i}+\epsilon_{3, i}) \\ &=&(\beta_{3}+b_{2}\beta_{2})+(c^{\prime}+a_{2}b_{2})X_{i}\\ &&+(b_{1}+b_{2}d)(\beta_{1}+a_{1}X_{i}+\epsilon_{1, i}) +(b_{2}\epsilon_{2, i}+\epsilon_{3, i}) \\ &=&(\beta_{3}+b_{2}\beta_{2}+(b_{1}+b_{2}d)\beta_{1})\\ &&+(c^{\prime}+a_{2}b_{2}+a_{1}b_{1}+a_{1}db_{2})X_{i}+\epsilon_{i}, \end{array} $$

(20)

where 𝜖_i = (b₁ + b₂d)𝜖_1,i + b₂𝜖_2,i + 𝜖_3,i. Since 𝜖_j,i for j = 1, 2, 3 are zero-mean distributed, 𝜖_i is also zero-mean distributed with E[𝜖_i|X_i] = 0. Taking expectation of Eqs. 12 and 20, we have

$$ \begin{array}{@{}rcl@{}} E[Y_{i}|X_{i}]&=&\beta_{0}+cX_{i}, \\ E[Y_{i}|X_{i}]&=&(\beta_{3}+b_{2}\beta_{2}+(b_{1}+b_{2}d)\beta_{1})\\ &&+(c^{\prime}+a_{1}b_{1}+a_{2}b_{2}+a_{1}db_{2})X_{i}. \end{array} $$

This leads to β₀ = β₃ + b₂β₂ + (b₁ + b₂d)β₁ and $c-c^{\prime }=a_{1}b_{1}+a_{2}b_{2}+a_{1}d b_{2}$.

For the simplified models with β₀ = β₁ = β₂ = β₃ = 0, the least-squares estimators are

$$ \begin{array}{@{}rcl@{}} \tilde{c}&=&\arg\underset{c}{\min}(Y_{i}-cX_{i})^{2}=\frac{A_{7}}{A_{1}}, \\ \tilde{a}_{1}&=&\arg\underset{a}{\min}(M_{1, i}-a_{1}X_{i})^{2}=\frac{A_{2}}{A_{1}},\\ (\tilde{a}_{2}, \tilde{d})^{T}&=&\arg\underset{a_{2}, d}{\min}(M_{2, i}-a_{2}X_{i}-dM_{1, i})^{2}=\frac{\left( \begin{array}{c} A_{3}A_{4}-A_{2}A_{10} \\ A_{1}A_{10}-A_{2}A_{3} \end{array}\right)}{A_{1}A_{4}-A_{2}A_{2}}, \\ (\tilde{c}^{\prime}, \tilde{b}_{1}, \tilde{b}_{2})^{T}&=&\arg\underset{c^{\prime}, b_{1}, b_{2}}{\min}(Y_{i}-c^{\prime}X_{i} -b_{1}M_{1, i}-b_{2}M_{2, i})^{2}\\ &=&\frac{\left( \begin{array}{ccc} (A_{4}A_{6}-A_{5}A_{5})A_{7}+(A_{3}A_{5}-A_{2}A_{6})A_{8}+(A_{2}A_{5}-A_{3}A_{4})A_{9} \\ (A_{3}A_{5}-A_{2}A_{6})A_{7}+(A_{1}A_{6}-A_{3}A_{3})A_{8}+(A_{2}A_{3}-A_{1}A_{5})A_{9} \\ (A_{2}A_{5}-A_{3}A_{4})A_{7}+(A_{2}A_{3}-A_{1}A_{5})A_{8}+(A_{1}A_{4}-A_{2}A_{2})A_{9} \end{array}\right)}{A_{1}(A_{4}A_{6}-A_{5}A_{5})+A_{2}(A_{3}A_{5}-A_{2}A_{6})+A_{3}(A_{2}A_{5}-A_{3}A_{4})}. \end{array} $$

where A₁ = X^TX, A₂ = X^TM₁, A₃ = X^TM₂, $A_{4}={M_{1}^{T}}M_{1}$, $A_{5}={M_{1}^{T}}M_{2}$, $A_{6}={M_{2}^{T}}M_{2}$, A₇ = X^TY, $A_{8}={M_{1}^{T}}Y$, $A_{9}={M_{2}^{T}}Y$, and $A_{10}={M_{1}^{T}}M_{2}$. By the above least-squares estimators, it is easy to verify that $\tilde {c}-\tilde {c}^{\prime }=\tilde {a}_{1}\tilde {b}_{1}+\tilde {a}_{2}\tilde {b}_{2}+\tilde {a}_{1}\tilde {d}\tilde {b}_{2}$. That is, the difference estimator is equivalent to the product estimator for the linear regression models with zero intercept.

The proof can readily be generalized to the models with non-zero intercept and so is omitted. □

Appendix : Appendix C: Proof of Theorem 6

Proof

Suppose there are k mediators:

$$ \begin{array}{@{}rcl@{}} M_{j}&=&\beta_{j}+{R}_{j}^{T}a_{j}+\epsilon_{j}, \quad j=1, \ldots, k, \\ Y&=&\beta_{k+1}+Xc^{\prime}+{R}_{Y}^{T}xb+\epsilon_{k+1}, \end{array} $$

where R_j contains the non-constant regressors in the equation for M_j, which may include X and/or other M_j’s, and R_Y contains M_j’s that appear in the equation for Y. After substituting the equations for M_j,j = 1,…,k, into the equation for Y, suppose we have

$$ Y=\alpha_{0}+(c^{\prime}+\alpha_{1})X+\epsilon_{0}\equiv\alpha_{0}+cX+\epsilon_{0}, $$

where (α₀,α₁) are functions of the coefficients in the equations for $\{M_{j}\}_{j=1}^{k}$ and Y, i.e.,

$$ \begin{array}{@{}rcl@{}} \alpha_{0}&=&f(\beta_{1}, \ldots, \beta_{k+1}; a_{1}, \ldots, a_{k}, b), \\ \alpha_{1}&=&f(a_{1}, \ldots, a_{k}, b), \end{array} $$

α₁ does not depend on β₁,…,β_k+ 1 because it measures the sensitivity of Y to X while β₁,…,β_k+ 1 does not contain such information, $c^{\prime }$ is the coefficient of X in the equation for Y, and 𝜖₀ is a linear combination of the error terms in these (k + 1) equations, so it satisfies E[𝜖₀|X] = 0.

Because all the coefficients are estimated by least-squares regression, they employ the moment conditions

$$ \begin{array}{@{}rcl@{}} \mathrm{E}\left[\left( \begin{array}{l} 1 \\ R_{j} \end{array}\right)(M_{j}-\beta_{j}-{R_{j}^{T}}a_{j})\right]&=0, \\ \mathrm{E}\left[\left( \begin{array}{l} 1 \\ X \\ R_{Y} \end{array}\right)(Y-\beta_{k+1}-Xc^{\prime}-{R_{Y}^{T}}b)\right]&=0. \end{array} $$

If these moment conditions imply

$$ \mathrm{E}\left[\left( \begin{array}{l} 1 \\ X \end{array}\right)(Y-\alpha_{0}-cX)\right]=0, $$

then our result follows since we just replace E[⋅] by $1/n{\sum }_{i=1}^{n}$ in the least-squares estimation. However, this indeed holds because 𝜖₀ is a linear function of $\{\epsilon _{j}\}_{j=1}^{k+1}$ so that

$$ \mathrm{E}\left[\left( \begin{array}{l} 1 \\ X \end{array}\right)\epsilon_{j}\right]=0 $$

implies

$$ \mathrm{E}\left[\left( \begin{array}{l} 1 \\ X \end{array}\right)\epsilon_{0}\right]=0. $$

Here, note that X must be a regressor in the M_j equation, otherwise $\mathrm {E}\left [\left (\begin {array}{l} 1 \\ X \end {array}\right )\epsilon _{j}\right ]=0$ cannot hold such that $\mathrm {E}\left [\left (\begin {array}{l} 1 \\ X \end {array}\right )\epsilon _{0}\right ]=0$ cannot hold and the equivalence result fails. □