Abstract
In observational data, covariance-based measures of dependence are of limited use for detecting reverse-causation (using y → x instead of x → y when quantifying the causal effect) and confounding biases (ignoring common causes). However, within the last decades, methodological progress has been made and, under certain conditions, scholars are now able to make evidence-based statements about causal mechanisms of variables from observational data alone. One such condition is to consider higher than second variable moments. This article introduces principles of causal structure learning and direction dependence modeling in linear models focusing on third moments and presents measures for causal structure learning and causal effect identification under hidden confounding. Results of a simulation study suggest that third moment-based causal effect estimators tend to show smaller biases than ordinary least squares estimators. Thus, third moment-based causal inference methods constitute a valuable add-on to existing approaches to causal inference.
Similar content being viewed by others
References
Aguinis H, Gottfredson RK, Joo H (2013) Best-practice recommendations for defining, identifying, and handling outliers. Organ Res Methods 16(2):270–301. https://doi.org/10.1177/1094428112470848
Blanca MJ, Arnau J, López-Montiel D, Bono R, Bendayan R (2013) Skewness and kurtosis in real data samples. Methodol Eur J Res Methods Behav Social Sci 9(2):78–84. https://doi.org/10.1027/1614-2241/a000057
Cai R, Xie F, Chen W, Hao Z (2017) An efficient kurtosis-based causal discovery method for linear non-Gaussian acyclic data. In: IEEE/ACM 25th International Symposium on Quality of Service, p 1–6
Cain MK, Zhang Z, Yuan K-H (2017) Univariate and multivariate skewness and kurtosis for measuring nonnormality: prevalence, influence and estimation. Behav Res Methods 49(5):1716–1735. https://doi.org/10.3758/s13428-016-0814-1
Chen Z, Chan L (2013) Causality in linear non-Gaussian acyclic models in the presence of latent Gaussian confounders. Neural Comput 25(6):1605–1641. https://doi.org/10.1162/NECO_a_00444
Chen W, Drton M, Wang YS (2019) On causal discovery with an equal-variance assumption. Biometrika 106(4):973–980. https://doi.org/10.1093/biomet/asz049
Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Lawrence Erlbaum Associates, Mahwah
Cragg JG (1997) Using higher moments to estimate the simple errors-in-variables model. Rand J Econ 28:S71–S91. https://doi.org/10.2307/3087456
Curran PJ, West SG, Finch JF (1996) The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychol Methods 1(1):16–29. https://doi.org/10.1037/1082-989X.1.1.16
Darmois G (1953) Analyse générale des liaisons stochastiques: Etude particulière de l’analyse factorielle linéaire [general analysis of stochastic links]. Revue De L’institut International De Statistique/rev Int Stat Inst 21(1/2):2–8. https://doi.org/10.2307/1401511
DeCarlo LT (1997) On the meaning and use of kurtosis. Psychol Methods 2(3):292–307. https://doi.org/10.1037/1082-989X.2.3.292
Dodge Y, Rousson V (1999) The complications of the fourth central moment. Am Stat 53(3):267. https://doi.org/10.2307/2686108
Dodge Y, Rousson V (2000) Direction dependence in a regression line. Commun Stat Theory Methods 29(9–10):1957–1972. https://doi.org/10.1080/03610920008832589
Dodge Y, Rousson V (2001) On asymmetric properties of the correlation coeffcient in the regression setting. Am Stat 55(1):51–54. https://doi.org/10.1198/000313001300339932
Dodge Y, Rousson V (2016) Statistical inference for direction of dependence in linear models. In: Wiedermann W, von Eye A (eds) Statistics and causality: methods for applied empirical research. Wiley, pp 45–62
Dodge Y, Yadegari I (2010) On direction of dependence. Metrika 72(1):139–150. https://doi.org/10.1007/s00184-009-0273-0
Elwert F, Winship C (2014) Endogenous selection bias: the problem of conditioning on a collider variable. Ann Rev Sociol 40(1):31–53. https://doi.org/10.1146/annurev-soc-071913-043455
Flora DB, Curran PJ (2004) An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychol Methods 9(4):466–491. https://doi.org/10.1037/1082-989X.9.4.466
Frisch R, Waugh FV (1933) Partial time regressions as compared with individual trends. Econometrica. https://doi.org/10.2307/1907330
Fuller WA (1987) Measurement error models. Wiley
Geary RC (1949) Determination of linear relations between systematic parts of variables with errors of observation the variances of which are unknown. Econometrica 17(1):30–58. https://doi.org/10.2307/1912132
Gillard J (2014) Method of moments estimation in linear regression with errors in both variables. Commun Stat Theor Methods 43(15):3208–3222. https://doi.org/10.1080/03610926.2012.698785
Greenland S, Pearl J, Robins JM (1999) Causal diagrams for epidemiologic research. Epidemiology 10(1):37–48. https://doi.org/10.1097/00001648-199901000-00008
Greenland S, Robins JM (1986) Identifiability, exchangeability, and epidemiological confounding. Int J Epidemiol 15(3):413–419. https://doi.org/10.1093/ije/15.3.413
Gretton A, Fukumizu K, Teo CH, Song L, Schölkopf B, Smola AJ (2008) A kernel statistical test of independence. Adv Neural Inf Process Syst 20:585–592
Hernandez-Lobato D, Morales-Mombiela P, Lopez-Paz D, Suarez A (2016) Non-linear causal inference using gaussianity measures. J Mach Learn Res 17:1–39
Hoyer PO, Shimizu S, Kerminen AJ, Palviainen M (2008) Estimation of causal effects using linear non-Gaussian causal models with hidden variables. Int J Approx Reason 49(2):362–378. https://doi.org/10.1016/j.ijar.2008.02.006
Hyvärinen A, Smith SM (2013) Pairwise likelihood ratios for estimation of non-Gaussian structural equation models. J Mach Learn Res 14:111–152
Hyvärinen A, Karhunen J, Oja E (2001) Independent component analysis. Wiley
Hyvärinen A, Zhang K, Shimizu S, Hoyer PO (2010) Estimation of a structural vector autoregression model using non-Gaussianity. J Mach Learn Res 11:1709–1731
Kendall MG, Stuart A (1979) The advanced theory of statistics: inference and relationship, 2nd edn. Chares Griffin & Company, London
Li X, Wiedermann W (2020) Conditional direction dependence analysis: evaluating the causal direction of effects in linear models with interaction terms. Multivar Behav Res 55(5):786–810. https://doi.org/10.1080/00273171.2019.1687276
Lovell MC (1963) Seasonal adjustment of economic time series and multiple regression analysis. J Am Stat Assoc 58(304):993–1010. https://doi.org/10.1080/01621459.1963.10480682
Lovell MC (2008) A simple proof of the FWL theorem. J Econ Educ 39(1):88–91. https://doi.org/10.3200/JECE.39.1.88-91
Maeda TN, Shimizu S (2020) Causal discovery of linear non-Gaussian acyclic models in the presence of latent confounders. https://arxiv.org/abs/2001.04197. Accessed 15 Aug 2021
Marszalek JM, Barber C, Kohlhart J, Holmes CB (2011) Sample size in psychological research over the past 30 years. Percept Mot Skills 112(2):331–348. https://doi.org/10.2466/03.11.PMS.112.2.331-348
Micceri T (1989) The unicorn, the normal curve, and other improbable creatures. Psychol Bull 105(1):156–166. https://doi.org/10.1037/0033-2909.105.1.156
Nigg JT, Stadler DD, Von Eye A, Wiedermann W (2020) Determining causality in relation to early risk factors of ADHD. In: Wiedermann W, Kim D, Sungur EA, von Eye A (eds) Direction dependence in statistical modeling: methods of analysis. Wiley, pp 295–324
Ozaki K, Ando J (2009) Direction of causation between shared and non-shared environmental factors. Behav Genet 39(3):321–336. https://doi.org/10.1007/s10519-009-9257-0
Pal M (1980) Consistent moment estimators of regression coefficients in the presence of errors in variables. J Econom 14(3):349–364. https://doi.org/10.1016/0304-4076(80)90032-9
Park G, Kim Y (2020) Identifiability of Gaussian linear structural equation models with homogeneous and heterogeneous error variances. J Korean Stat Soc 49(1):276–292. https://doi.org/10.1007/s42952-019-00019-7
Pearl J (1995) Causal diagrams for empirical research. Biometrika 82(4):669–688. https://doi.org/10.1093/biomet/82.4.669
Pearl J (2009) Causality: models, reasoning, and inference, 2nd edn. Cambridge University Press, Cambridge
Peters J, Bühlmann P (2014) Identifiability of Gaussian structural equation models with equal error variances. Biometrika 101(1):219–228. https://doi.org/10.1093/biomet/ast043
Peters J, Mooij D, Janzing D, Schölkopf B (2014) Causal discovery with continuous additive noise models. J Mach Learn Res 15:2009–2053
Peters J, Janzing D, Schölkopf B (2017) Elements of causal inference: Foundations and learning algorithms. MIT Press, Cambridge
R Core Team (2021) R: a language and environment for statistical computing. R foundation for statistical computing. http://www.R-project.org/. Accessed 1 Apr 2021
Reichenbach H (1956) The direction of time. Los Angeles University Press
Reiersøl O (1950) Identifiability of a linear relation between variables which are subject to error. Econometrica 18(4):375–389. https://doi.org/10.2307/1907835
Rousseeuw PJ, Leroy AM (2003) Robust regression and outlier detection. Wiley & Sons
Rubin DB (1974) Estimating causal effects of treatments in randomized and nonrandomized studies. J Educ Psychol 66(5):688–701. https://doi.org/10.1037/h0037350
Ruppert D (1987) What is kurtosis? An influence function approach. Am Stat 41(1):1. https://doi.org/10.2307/2684309
Scott EL (1950) Note on consistent estimates of the linear structural relation between two variables. Ann Math Stat 21(2):284–288. https://doi.org/10.1214/aoms/1177729846
Shimizu S (2019) Non-Gaussian methods for causal structure learning. Prev Sci 20(3):431–441. https://doi.org/10.1007/s11121-018-0901-x
Shimizu S, Hoyer PO, Hyvärinen A, Kerminen A (2006) A linear non-Gaussian acyclic model for causal discovery. J Mach Learn Res 7:2003–2030
Shimizu S, Inazumi T, Sogawa Y, Hyvärinen A, Kawahara Y, Washio T, Hoyer PO, Bollen K (2011) DirectLiNGAM: a direct method for learning a linear non-Gaussian structural equation model. J Mach Learn Res 12:1225–1248
Shmueli G (2010) To explain or to predict? Stat Sci 25(3):289–310. https://doi.org/10.1214/10-STS330
Skitovich WP (1953) On a property of the normal distribution. Doklady Akademii Nauk SSSR [reports of the Academy of Sciences USSR] 89:217–219
Spiegelman C (1979) On estimating the slope of a straight line when both variables are subject to error. Ann Stat 7(1):201–206. https://doi.org/10.1214/aos/1176344565
Stelzl I (1986) Changing the causal hypothesis without changing the fit: some rules for generating equivalent path models. Multivar Behav Res 21:309–331. https://doi.org/10.1207/s15327906mbr2103-3
Stigler SM (2010) The changing history of robustness. Am Stat 64(4):277–281. https://doi.org/10.1198/tast.2010.10159
Székely GJ, Rizzo ML, Bakirov NK (2007) Measuring and testing dependence by correlation of distances. Ann Stat 35(6):2769–2794. https://doi.org/10.1214/009053607000000505
von Eye A, DeShon RP (2012) Directional dependence in developmental research. Int J Behav Dev 36(4):303–312. https://doi.org/10.1177/0165025412439968
Westfall PH (2014) Kurtosis as peakedness, 1905–2014. R.I.P. Am Stat 68(3):191–195. https://doi.org/10.1080/00031305.2014.917055
Wiedermann W (2018) A note on fourth moment-based direction dependence measures when regression errors are non normal. Commun Stat Theor Methods 47(21):5255–5264. https://doi.org/10.1080/03610926.2017.1388403
Wiedermann W (2020) Asymmetry properties of the partial correlation coefficient: foundations for covariate adjustment in distribution-based direction dependence analysis. In: Wiedermann W, Kim D, Sungur EA, von Eye A (eds) Direction dependence in statistical modeling: methods of analysis. Wiley, pp 81–110
Wiedermann W, Hagmann M (2016) Asymmetric properties of the Pearson correlation coefficient: correlation as the negative association between linear regression residuals. Commun Stat Theor Methods 45(21):6263–6283. https://doi.org/10.1080/03610926.2014.960582
Wiedermann W, Li X (2018) Direction dependence analysis: a framework to test the direction of effects in linear models with an implementation in SPSS. Behav Res Methods 50(4):1581–1601. https://doi.org/10.3758/s13428-018-1031-x
Wiedermann W, Sebastian J (2020a) Direction dependence analysis in the presence of confounders: applications to linear mediation models using observational data. Multivar Behav Res 55(4):495–515. https://doi.org/10.1080/00273171.2018.1528542
Wiedermann W, Sebastian J (2020b) Sensitivity analysis and extensions of testing the causal direction of dependence: A rejoinder to Thoemmes (2019). Multivar Behav Res 55(4):523–530. https://doi.org/10.1080/00273171.2019.1659127
Wiedermann W, von Eye A (2015a) Direction of effects in mediation analysis. Psychol Methods 20(2):221–244. https://doi.org/10.1037/met0000027
Wiedermann W, von Eye A (2015b) Direction-dependence analysis: a confirmatory approach for testing directional theories. Int J Behav Dev 39(6):570–580. https://doi.org/10.1177/0165025415582056
Wiedermann W, von Eye A (2020) Reciprocal relations in categorical variables. Psychol Methods 25(6):708–725. https://doi.org/10.1037/met0000257
Wiedermann W, Merkle EC, Eye A (2018) Direction of dependence in measurement error models. Br J Math Stat Psychol 71(1):117–145. https://doi.org/10.1111/bmsp.12111
Wiedermann W, Kim D, Sungur EA, von Eye A (2020) Direction dependence in statistical models: methods of analysis. Wiley
Wright DB, Herrington JA (2011) Problematic standard errors and confidence intervals for skewness and kurtosis. Behav Res Methods 43(1):8–17. https://doi.org/10.3758/s13428-010-0044-x
Zhang K, Gong M, Ramsey J, Batmanghelich K, Spirtes P, Glymour C (2018) Causal discovery with linear non-Gaussian models under measurement error: Structural identifiability results. In: Proc. Conference on Uncertainty in Artificial Intelligence (UAI’18)
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The author states that there is no conflict of interest.
Additional information
Communicated by Shohei Shimizu.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Wiedermann, W. Third moment-based causal inference. Behaviormetrika 49, 303–328 (2022). https://doi.org/10.1007/s41237-021-00154-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41237-021-00154-8