For and Against Methodologies: Some Perspectives on Recent Causal and Statistical Inference Debates

Abstract

I present an overview of two methods controversies that are central to analysis and inference: That surrounding causal modeling as reflected in the “causal inference” movement, and that surrounding null bias in statistical methods as applied to causal questions. Human factors have expanded what might otherwise have been narrow technical discussions into broad philosophical debates. There seem to be misconceptions about the requirements and capabilities of formal methods, especially in notions that certain assumptions or models (such as potential-outcome models) are necessary or sufficient for valid inference. I argue that, once these misconceptions are removed, most elements of the opposing views can be reconciled. The chief problem of causal inference then becomes one of how to teach sound use of formal methods (such as causal modeling, statistical inference, and sensitivity analysis), and how to apply them without generating the overconfidence and misinterpretations that have ruined so many statistical practices.

This is a preview of subscription content, access via your institution.

Notes

  1. 1.

    The same comment may well apply to physical sciences, but those are far outside my scope.

  2. 2.

    In these papers, “identification” has its strict statistical meaning of estimability rather than its more recent epidemiologic meaning of qualitative identification, as in Schwartz et al. (2016) and VanderWeele (2016).

  3. 3.

    Holland called this RPOA “Rubin’s causal model,” even though such models had long been in use in experimental analysis (e.g., Neyman 1923; Welch 1937; Wilk 1955). Rubin’s seminal contributions extended the models to statistical analysis of nonexperiments (Rubin 1990).

  4. 4.

    Apart from invertible equation systems, which do not exist in realistic models for health and social phenomena.

  5. 5.

    In a common notation, knowing the treatment indicator X is positively associated with the outcome indicator Y0 under X=0 yet unassociated with the observed outcome Yobs leads us to infer that Y1 > Y0 for some observed unit.

  6. 6.

    In terms of potential outcomes Yx indexed by drug dose x, we would infer that the unobserved variable Y40 is sometimes greater than Y0 even though Pr(X=40) = 0.

  7. 7.

    In notation: With Yx the potential outcome and Yobs its measurement, we can have Yobs ≠ Yx yet still infer that Y1 ≠ Y0 for some unit.

  8. 8.

    Adding confusion, the term ‘consistency' is already well established for unrelated concepts such as estimator convergence and freedom from contradiction.

  9. 9.

    Even more startling is that temporality (cause preceding effect) is not necessary in some counterfactual accounts of causation (Price 1996).

  10. 10.

    In logic this syntactical structure is called a theory, and the interpretations that follow that structure are called models of the theory. I instead call this structure a model, which I think more in line with common usage in statistics and applied sciences.

  11. 11.

    This fact is one way of seeing why quantum physics has defied classical causal explanations: Robins et al. (2015) show that potential-outcome models obey Bell’s inequality, whose observed violations conflict with local definiteness (local realism, local hidden variables) and local causal diagrams (Gill 2014).

  12. 12.

    Notably, similar concerns about untestable mathematical theory arise in hard sciences like physics (Ellis and Silk 2014).

  13. 13.

    When in addition to the causal graph we can assume faithfulness (open paths imply association), the number of logically possible structures is reduced drastically – to the point that a certain limited type of conditional causal identification can be enabled (Spirtes et al. 2001; Robins et al. 2003).

  14. 14.

    One measure of the evidence against a model (whether causal or not) supplied by the P-value p from a test of its fit is the binary information or surprisal −log2(p).

  15. 15.

    The retrolental-fibroplasia controversy provides one such case study (Greenland 1991).

References

  1. Aalen OO, Røysland K, Gran JM, Kouyos R, Lange T. Can we believe the DAGs? A comment on the relationship between causal DAGs and mechanisms. Stat Meth Med Res. 2016;25(5):2294–314.

    CAS  Article  Google Scholar 

  2. Altman DG, Machin D, Bryant TN, Gardner MJ, editors. Statistics with Confidence. 2nd ed. London: BMJ Books; 2000.

    Google Scholar 

  3. Baggerly K, Gunsalus CK. Penalty too light. Cancer Letter. 2015;41(42):1–9.

    Google Scholar 

  4. Bancroft TW, Han CP. Inference based on conditional specification. Int Stat Rev. 1977;45:117–28.

    Google Scholar 

  5. Belluz J, Plumer B, Resnick B. The 7 biggest problems facing science, according to 270 scientists. Vox, September 7, 2016, accessed Oct. 16, 2016 from http://www.vox.com/2016/7/14/12016710/science-challeges-research-funding-peer-review-process

  6. Box GEP. Sampling and Bayes inference in scientific modeling and robustness. J R Stat Soc Ser A. 1980;143:383–430.

    Article  Google Scholar 

  7. Box GEP. Comment. Statist Sci. 1990;5:448–9.

    Google Scholar 

  8. Breslow NE. Are statistical contributions to medicine undervalued? Biometrics. 2003;59(1):1–8.

    PubMed  Article  Google Scholar 

  9. Broadbent A, Vandenbroucke JP, Pearce N. Authors’ Reply to: VanderWeele et al., Chiolero, and Schooling et al. (letter). Int J Epidemiol 2016; in press.

  10. Bross IDJ. Pertinency of an extraneous variable. J Chronic Dis. 1967;20:487–95.

    CAS  PubMed  Article  Google Scholar 

  11. Chiolero A. Counterfactual and interventionist approach to cure risk factor epidemiology (letter). Int J Epidemiol 2016; in press.

  12. Cornfield J, Haenszel WH, Hammond EC, Lilienfeld AM, Shimkin MB, Wynder EL. Smoking and lung cancer: recent evidence and a discussion of some questions. J Natl Cancer Inst. 1959;22:173–203.

    CAS  PubMed  Google Scholar 

  13. Cox DR. Some problems connected with statistical inference. Ann Math Stat. 1958;29:357–72.

    Article  Google Scholar 

  14. Daniel RM, De Stavola BL, Vansteelandt S. The formal approach to quantitative causal inference in epidemiology: misguided or misrepresented? Int J Epidemiol 2016; in press.

  15. Dawid AP. The well-calibrated Bayesian (with discussion). JASA. 1982;77:604–13.

    Google Scholar 

  16. Dawid AP. Causal inference without counterfactuals (with discussion). JASA. 2000;95:407–48.

    Article  Google Scholar 

  17. Dawid AP. Beware of the DAG! In: D Janzing IG, Schoelkopf B (eds). Proceedings of the NIPS 2008 Workshop on Causality. Journal of Machine Learning Research Workshop and Conference Proceedings, Whistler, Canada, 2008, 59–86.

  18. Discacciati A, Orsini N, Greenland S. Approximate Bayesian logistic regression via penalized likelihood by data augmentation. Stata Journal. 2015;15(3):712–36.

    Google Scholar 

  19. Dunning T. Improving causal inference: Strengths and limitations of natural experiments. Political Research Quarterly. 2008;61(2):282–93.

    Article  Google Scholar 

  20. Ellis G, Silk J. Defend the integrity of physics. Nature. 2014;516:321–3.

    CAS  PubMed  Article  Google Scholar 

  21. Farsides T, Sparks P. Opinion: Buried in bullshit. The Psychologist. 2016;29:368–71.

    Google Scholar 

  22. Feyerabend P. Against Method. New York: New Left Books, 1975; 3rd ed. New York: Verso, 1993.

  23. Feyerabend P. Killing Time. Chicago: U Chicago Press; 1995.

    Google Scholar 

  24. Flanders WD, Johnson CY, Howards PP, Greenland S. Dependence of confounding on the target population: A modification of causal graphs to account for coaction. Ann Epidemiol. 2011;21:698–705.

    PubMed  Article  Google Scholar 

  25. Fleiss JL. Significance tests have a role in epidemiologic research: reactions to A. M. Walker. Am J Public Health. 1986a;76:559–60.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  26. Fleiss JL. Dr. Fleiss responds. Am J Public Health. 1986b;76:1033–4.

    PubMed Central  Article  Google Scholar 

  27. Freedman DA. As others see us: a case study in path analysis (with discussion). J Educ Stat. 1987;12:101–223.

    Google Scholar 

  28. Freedman DA, Navidi W, Peters SC. On the impact of variable selection in fitting regression equations. In: Dijlestra TK, editor. On model uncertainty and its statistical implications. Berlin: Springer-Verlag; 1988. p. 1–16.

    Google Scholar 

  29. Galea S. An argument for a consequentialist epidemiology. Am J Epidemiol. 2013;178:1185–91.

    PubMed  Article  Google Scholar 

  30. Gelman A. Causality and statistical learning. Am J Sociol. 2011;117:955–66.

    Article  Google Scholar 

  31. Gelman A. P-values and statistical practice. Epidemiology. 2013;24:69–72.

    PubMed  Article  Google Scholar 

  32. Gelman A, Loken E. The statistical crisis in science: Data-dependent analysis—a “garden of forking paths”—explains why many statistically significant comparisons don’t hold up. Am Sci 2014a;102:460–465. Erratum at http://andrewgelman.com/2014/10/14/didnt-say-part-2/, accessed 25 Oct 2016.

  33. Gelman A, Loken E. The AAA tranche of subprime science. Chance. 2014;27(1):51–7.

    Article  Google Scholar 

  34. Gelman A, Shalizi CS. Philosophy and the practice of Bayesian statistics (with discussion). Br J Math Stat Psych. 2013;66:8–80.

    Article  Google Scholar 

  35. Gelman A, Stern HS. The difference between “significant” and “not significant” is not itself statistically significant. Am Stat. 2006;60:328–31.

    Article  Google Scholar 

  36. George SL, Buyse M. Data fraud in clinical trials. Clin. Invest (London). 2015;5(2):161–73.

    CAS  Article  Google Scholar 

  37. Gigerenzer G. We need statistical thinking, not statistical rituals. Behavioral & Brain Sciences. 1998;21:199–200.

    Article  Google Scholar 

  38. Gigerenzer G. Mindless statistics. J Socioecon. 2004;33:567–606.

    Google Scholar 

  39. Gigerenzer G, Marewski JN. Surrogate science: the idol of a universal method for scientific inference. J Manag. 2015;41:421–40.

    Google Scholar 

  40. Gill RD. Statistics, causality and Bell’s theorem. Statistical Science. 2014;29(4):512–28.

    Article  Google Scholar 

  41. Gilovich T, Griffin D, Kahneman D. Heuristics and biases: the psychology of intuitive judgment. New York: Cambridge University Press; 2002.

    Google Scholar 

  42. Glass TA, Goodman SN, Hernán MA, Samet JM. Causal inference in public health. Annu Rev Public Health. 2013;34:61–75.

    PubMed  PubMed Central  Article  Google Scholar 

  43. Glymour C. Comment: Statistics and metaphysics. JASA. 1986;81:964–6.

    Google Scholar 

  44. Glymour C, Glymour MR. Commentary: race and sex are causes. Epidemiology. 2014;25:488–90.

    PubMed  Article  Google Scholar 

  45. Glymour MM, Greenland S. Causal diagrams. Ch. 12 in: Rothman KJ, Greenland S, Lash TL. Modern Epidemiology, 3rd ed. Philadelphia: Lippincott 2008; 183-209.

  46. Goodman SN. P Values, hypothesis tests, and likelihood: implications for epidemiology of a neglected historical debate. Am J Epidemiol. 1993;137:485–96.

    CAS  PubMed  Article  Google Scholar 

  47. Goodman SN. Introduction to Bayesian methods I: measuring the strength of evidence. Clin Trials. 2005;2:282–90.

    PubMed  Article  Google Scholar 

  48. Goodman SN, Royall R. Evidence and scientific research. Am J Public Health. 1988;78:1568–74.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  49. Greenhouse SW. Some epidemiologic issues for the 1980s. Am J Epidemiol. 1980;112(2):269–73.

    CAS  PubMed  Article  Google Scholar 

  50. Greenland S. The author replies to Newman and Browner. Am J Epidemiol. 1988;128:1182–4.

    Article  Google Scholar 

  51. Greenland S. Randomization, statistics, and causal inference. Epidemiol. 1990;1:421–9.

    CAS  Article  Google Scholar 

  52. Greenland S. Science versus advocacy: The challenge of Dr. Feinstein. Epidemiol. 1991;2:72–79.

    Article  Google Scholar 

  53. Greenland S. The sensitivity of a sensitivity analysis (invited paper). In: 1997 Proceedings of the Biometrics Section. Alexandria, VA: American Statistical Association 1998; 19-21.

  54. Greenland S. Epidemiologic measures and policy formulation: Lessons from potential outcomes (with discussion). Emerging Themes in Epidemiology (online). 2005;2:1–4.

    Article  Google Scholar 

  55. Greenland S. Introduction to Bayesian statistics. Ch. 18 in: Rothman KJ, Greenland S, Lash TL. Modern Epidemiology, 3rd ed. Philadelphia: Lippincott 2008; 328-44.

  56. Greenland S. Weaknesses of certain Bayesian methods for meta-analysis: The case of vitamin E and mortality (invited commentary). Clinical Trials. 2009a;6:42–6.

    PubMed  Article  Google Scholar 

  57. Greenland S. Relaxation penalties and priors for plausible modeling of nonidentified bias sources. Statistical Science. 2009b;24:195–210.

    Article  Google Scholar 

  58. Greenland S. Overthrowing the tyranny of null hypotheses hidden in causal diagrams. Ch. 22 in: Dechter R, Geffner H, Halpern JY, eds. Heuristics, Probabilities, and Causality: A Tribute to Judea Pearl. London: College Press, 2010a, 365–382. Available at http://intersci.ss.uci.edu/wiki/pdf/Pearl/22_Greenland.pdf

  59. Greenland S. Comment: The need for syncretism in applied statistics. Statist Sci. 2010b;25:158–61.

    Article  Google Scholar 

  60. Greenland S. Null misinterpretation in statistical testing and its impact on health risk assessment. Preventive Medicine. 2011;53:225–8.

    PubMed  Article  Google Scholar 

  61. Greenland S. Causal inference as a prediction problem: Assumptions, identification, and evidence synthesis. Ch. 5 in: Berzuini C, Dawid AP, Bernardinelli L (eds.). Causal Inference: Statistical Perspectives and Applications. Chichester: Wiley 2012a, 43-58.

  62. Greenland S. Nonsignificance plus high power does not imply support for the null over the alternative. Annals of Epidemiology. 2012a;22:364–8.

    PubMed  Article  Google Scholar 

  63. Greenland S. Transparency and disclosure, neutrality and balance: shared values or just shared words? Journal of Epidemiology and Community Health. 2012b;66:967–70.

    PubMed  Article  Google Scholar 

  64. Greenland S. The ASA guidelines and null bias in current teaching and practice. Am Statist 2016; 70: suppl. 10 at http://www.tandfonline.com/doi/suppl/10.1080/00031305.2016.1154108

  65. Greenland S. A serious misinterpretation of a consistent inverse association of statin use with glioma across 3 case-control studies. Eur J Epidemiol 2017a;32: in press.

  66. Greenland S. The biases of bias analyses will not help validity or reproducibility. Am J Epidemiol 2017b; to appear.

  67. Greenland S, Brumback BA. An overview of relations among causal modeling methods. Int J Epidemiol. 2002;31:1030–7.

    PubMed  Article  Google Scholar 

  68. Greenland S, Lash TL. Bias analysis. Ch. 19 in: Rothman KJ, Greenland S, Lash TL. Modern Epidemiology, 3rd ed. Philadelphia: Lippincott 2008; 345-80.

  69. Greenland S, Maclure M, Schlesselman JJ, Poole C, Morgenstern H. Standardized regression coefficients: a further critique and review of some alternatives. Epidemiology. 1991;2:387–92.

    CAS  PubMed  Article  Google Scholar 

  70. Greenland S, Mansournia MA. Limitations of individual causal models, causal graphs, and ignorability assumptions, as illustrated by random confounding and design unfaithfulness. Eur J Epidemiol. 2015a;30:1101–10.

    PubMed  Article  Google Scholar 

  71. Greenland S, Mansournia MA. Penalization, bias reduction, and default priors in logistic and related categorical and survival regressions. Stat Med 2015b;34:3133–43.

  72. Greenland S, Neutra RR. Control of confounding in the assessment of medical technology. Int J Epidemiol. 1980;9:361–7.

    CAS  PubMed  Article  Google Scholar 

  73. Greenland S, Pearl J, Robins JM. Causal diagrams for epidemiologic research. Epidemiology. 1999;10:37–48.

    CAS  PubMed  Article  Google Scholar 

  74. Greenland S, Poole C. Living with statistics in observational research. Epidemiology. 2013;24:73–8.

    PubMed  Article  Google Scholar 

  75. Greenland S, Robins JM. Identifiability, exchangeability and epidemiological confounding. Int J Epidemiol. 1986;15:413–9.

    CAS  PubMed  Article  Google Scholar 

  76. Greenland S, Senn SJ, Rothman KJ, Carlin JC, Poole C, Goodman SN, Altman DG. Statistical tests, confidence intervals, and power: A guide to misinterpretations. Eur J Epidemiol 2016; 31:337-50. https://dx.doi.org/10.1007%2Fs10654-016-0149-3

  77. Greenwood M. Is the statistical method of any value in medical research? Lancet. 1924;204:153–8.

    Article  Google Scholar 

  78. Gustafson P. Bayesian inference for partially identified models. Int J Biostatist. 2010;6(2):17.

    Article  Google Scholar 

  79. Gustafson P, Greenland S. The performance of random coefficient regression in accounting for residual confounding. Biometrics. 2006;62:760–8.

    PubMed  Article  Google Scholar 

  80. Gustafson P, Greenland S. Interval estimation for messy observational data. Stat Sci. 2009;24:328–42.

    Article  Google Scholar 

  81. Gustafson P, McCandless L. Priors, parameters, and probability: A Bayesian perspective on sensitivity analysis. Epidemiol. 2014;26:910–2.

    Article  Google Scholar 

  82. Hall JB. An address on yesterday, to-day, and to-morrow. Lancet. 1924;204:151–3.

    Article  Google Scholar 

  83. Hernán MA. Invited commentary: hypothetical interventions to define causal effects afterthought or prerequisite? Am J Epidemiol. 2005;162:618–20.

    PubMed  Article  Google Scholar 

  84. Hernán MA. Does water kill? A call for less casual causal inferences. Ann Epidemiol. 2016;26:683–4.

    Article  Google Scholar 

  85. Hernán MA, Clayton D, Keiding N. The Simpson’s paradox unraveled. Int J Epidemiol. 2011;40:780–5.

    PubMed  PubMed Central  Article  Google Scholar 

  86. Hernán MA, Robins JM. Causal inference. New York, Chapman & Hall, 2017, to appear.

  87. Hernán MA, Taubman SL. Does obesity shorten life? The importance of well-defined interventions to answer causal questions. Int J Obes. 2008;32(suppl 3):S8–14.

    Article  Google Scholar 

  88. Hill AB. The environment and disease: association or causation? Proc R Soc Med. 1965;58:295–300.

    CAS  PubMed  PubMed Central  Google Scholar 

  89. Höfler M. The Bradford Hill considerations on causality: a counterfactual perspective. Emerg Themes Epidemiol. 2005;2(1):11.

    PubMed  PubMed Central  Article  Google Scholar 

  90. Holland PW. Statistics and causal inference (with discussion). J Am Stat Assoc. 1986;81:945–70.

    Article  Google Scholar 

  91. Hume, D. An Enquiry Concerning Human Understanding. Reprint of 1748 original by Oxford University Press, New York, 1999.

  92. Ioannidis JPA. Why most discovered true associations are inflated. Epidemiol. 2008;19:640–8.

    Article  Google Scholar 

  93. Kaufman JS. Race: ritual, regression, and reality. Epidemiol. 2014;25:485–7.

    Article  Google Scholar 

  94. Kaufman JS. There is no virtue in vagueness. Ann Epidemiol. 2016;26:683–4.

    PubMed  Article  Google Scholar 

  95. Keyes K, Galea S. What matters most: quantifying an epidemiology of consequence. Ann Epidemiol. 2015;25(5):305–11.

    PubMed  PubMed Central  Article  Google Scholar 

  96. King G, Zeng L. When can history be our guide? The pitfalls of counterfactual inference. Int Stud Q. 2007;51:183–210.

    Article  Google Scholar 

  97. Krieger N, Davey Smith G. The tale wagged by the DAG: broadening the scope of causal inference and explanation for Epidemiology. Int J Epidemiol 2016; in press.

  98. Lachenbruch PA, Clark VA, Cumberland WG, Chang PC, Afifi AA, Flack VF, Elashoff RM. Letter to the Editor. AJPH. 1987;77(2):237.

    Google Scholar 

  99. Lash TL. Heuristic thinking and inference from observational epidemiology. Epidemiology. 2007;18:67–72.

    PubMed  Article  Google Scholar 

  100. Lash TL, Fox MP, Fink AK. Applying Quantitative Bias Analysis to Epidemiologic Data. Boston: Springer; 2009.

    Google Scholar 

  101. Leamer EE. Specification Searches. New York: Wiley; 1978.

    Google Scholar 

  102. Leamer EE. Sensitivity analyses would help. Am Econ Rev. 1985;75:308–13.

    Google Scholar 

  103. Lewis D. Causation J. Philos 1973;70:556–567. Reprinted with postscript in: Lewis D. Philosophical papers. New York: Oxford University Press, 1986.

  104. Little RJA. Calibrated Bayes: A Bayes/frequentist roadmap. Am Statist. 2006;60:1–11.

    Article  Google Scholar 

  105. Maclure M, Schneeweiss S. Causation of bias: The Episcope. Epidemiol. 2001;12:114–22.

    CAS  Google Scholar 

  106. MacMahon B, Pugh TF. Causes and entities of disease. In: Clark DW, MacMahon B, eds. Preventive medicine. Boston: Little, Brown, 1967.

  107. Maldonado G. Toward a clearer understanding of causal concepts in epidemiology. Ann Epidemiol. 2013;23:743–9.

    PubMed  Article  Google Scholar 

  108. Maldonado G. The role of counterfactual theory in causal reasoning. Ann Epidemiol. 2016;26:681–2.

    PubMed  Article  Google Scholar 

  109. Maldonado G, Greenland S. Response: Defining and estimating causal effects. Int J Epidemiol. 2002;31:434–8.

    Article  Google Scholar 

  110. Mansournia MA, Higgins JPT, Sterne JAC, Hernán MA. Biases in randomized trials-A conversation between trialists and epidemiologists. Epidemiol. 2017;28:54–9.

    Article  Google Scholar 

  111. Mill JS. A System of Logic. Reprint by Longmans, Green, London: Ratiocinative and Inductive; 1843. p. 1956.

    Google Scholar 

  112. Morabia A. Has epidemiology become infatuated with methods? Annu Rev Public Health. 2015;36:69–88.

    PubMed  Article  Google Scholar 

  113. Naimi AI, Kaufman JS, MacLehose RF. Mediation misgivings: ambiguous clinical and public health interpretations of natural direct and indirect effects. In J Epidemiol. 2014;43:1656–61.

    Google Scholar 

  114. Naimi AI. The counterfactual implications of fundamental cause theory. Curr Epidemiol Rep. 2016;3:92–7.

    Article  Google Scholar 

  115. Neyman J. On the application of probability theory to agricultural experiments. Essay on principles. Section 9, 1923 (in Polish; translation in Statistical Science 1990, 465–472).

  116. Neyman J. Frequentist probability and frequentist statistics. Synthese. 1977;36:97–131.

    Article  Google Scholar 

  117. Newman TB, Browner WS. Re: “Interpretation and choice of effect measures in epidemiologic analyses” (letter). Am J Epidemiol. 1988;12:1181–2.

    Article  Google Scholar 

  118. Pearce N, Vandenbroucke JP. Commentary: Causation, mediation and explanation. Int J Epidemiol 2017;46: in press.

  119. Pearl J. Causal diagrams for empirical research. Biometrika. 1995;82:669–710.

    Article  Google Scholar 

  120. Pearl J. Causality: models, reasoning and inference. 2nd ed. Cambridge, UK: Cambridge University Press; 2009.

    Google Scholar 

  121. Pearl J. On the consistency rule in causal inference: Axiom, definition, assumption, or theorem? Am J Epidemiol. 2010;21(6):872–5.

    Article  Google Scholar 

  122. Pearl J. Causes of effects and effects of causes. Soc Meth Res. 2015;44(1):149–64.

    Article  Google Scholar 

  123. Phillips CV. Quantifying and reporting uncertainty from systematic errors. Epidemiology. 2003;14:459–66.

    PubMed  Google Scholar 

  124. Phillips CV, Goodman KJ. The missed lessons of Sir Austin Bradford Hill. Epidemiol Perspect Innov. 2004;1:3. doi:10.1186/1742-5573-1-3.

    PubMed  PubMed Central  Article  Google Scholar 

  125. Phillips CV, Goodman KJ. Causal criteria and counterfactuals: nothing more (or less) than scientific common sense. Emerg Themes Epidemiol. 2006;3:5. doi:10.1186/1742-7622-3-5.

    PubMed  PubMed Central  Article  Google Scholar 

  126. Price H. Time’s Arrow and Archimedes’ Point. New York: Oxford, 1996.

  127. Poole C. Beyond the confidence interval. Am J Public Health. 1987;77:195–9.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  128. Poole C. Low P-values or narrow confidence intervals: Which are more durable? Epidemiol. 2001;12:291–4.

    CAS  Article  Google Scholar 

  129. Poole C, Greenland S. How a court accepted a possible explanation: A comment on Gastwirth, Krieger, and Rosenbaum. Am Statist. 1997;51:112–4.

    Google Scholar 

  130. Porta M, Bolúmar F. Caution: work in progress. Eur J Epidemiol. 2016;31:535–9.

    PubMed  Article  Google Scholar 

  131. Porta M, Vineis P, Bolúmar F. The current deconstruction of paradoxes: one sign of the ongoing methodological “revolution”. Eur J Epidemiol. 2015;30:1079–87.

    PubMed  Article  Google Scholar 

  132. Rhodes E. Replication: Is the glass half full, half empty, or irrelevant? The Psychologist, 7th March 2016.

  133. Richardson T, Robins JM. Single world intervention graphs (SWIGs): a unification of the counterfactual and graphical approaches to causality. Working Paper 128. Center for the Statistics and the Social Sciences, University of Washington, Seattle, 2013.

  134. Robins JM. A graphical approach to the identification and estimation of causal parameters in mortality studies with sustained exposure periods. J Chron Dis. 1987;40(supplement 2):139S–61S.

    PubMed  Article  Google Scholar 

  135. Robins JM, Greenland S. Identifiability and exchangeability for direct and indirect effects. Epidemiology. 1992;3:143–55.

    CAS  PubMed  Article  Google Scholar 

  136. Robins JM, Greenland S. Comment. J Am Stat Assoc. 2000;95:431–5.

    Article  Google Scholar 

  137. Robins JM, Richardson TS. Alternative graphical causal models and the identification of direct effects. Ch. 6 in Shrout P, Keyes K, Ornstein K, eds. Causality and Psychopathology: Finding the Determinants of Disorders and their Cures. New York: Oxford, 2011, 1-52.

  138. Robins JM, Vander Weele TJ, Gill RD. A proof of Bell’s inequality in quantum mechanics using causal interactions. Scand J Statistics. 2015;42:329–35.

    Article  Google Scholar 

  139. Robins JM, Weissman M. Counterfactual causation and streetlamps: What is to be done? Int J Epidemiol 2016; in press.

  140. Robins JM, Wasserman L. On the impossibility of inferring causation from association without background knowledge (with discussion). In: Glymour C, Cooper G, editors. Computation, Causation, and Discovery. Cambridge, MA: MIT Press; 1999. p. 305–42.

    Google Scholar 

  141. Robins JM, Scheines R, Spirtes P, Wasserman L. Uniform consistency in causal inference. Biometrika. 2003;90:491–515.

    Article  Google Scholar 

  142. Romer P. The trouble with macroeconomics. Am Economist 2016;to appear.

  143. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70:41–55.

    Article  Google Scholar 

  144. Rothman KJ. Causes. Am J Epidemiol. 1976;104:587–92.

    CAS  PubMed  Article  Google Scholar 

  145. Rothman KJ. A show of confidence. NEJM. 1978;299:1362–3.

    CAS  PubMed  Article  Google Scholar 

  146. Rothman KJ. Significance questing. Ann Intern Med. 1986;105:445–7.

    CAS  PubMed  Article  Google Scholar 

  147. Rouen TA. Letter to the Editor. AJPH. 1987;77(2):237.

    Google Scholar 

  148. Rubin DB. Estimating causal effects of treatments in randomized and nonrandomized studies. J Educ Psychol. 1974;66:688–701.

    Article  Google Scholar 

  149. Rubin DB. Comment: Neyman (1923) and causal inference in experiments and observational studies. Stat Sci. 1990;5:472–80.

    Google Scholar 

  150. Rubin DB. Practical implications of modes of statistical inference for causal effects, and the critical role of the assignment mechanism. Biometrics. 1991;47:1213–34.

    CAS  PubMed  Article  Google Scholar 

  151. Schooling C, Chow C, Au Yeung S. Causality and causal inference in epidemiology: we need also to address causes of effects (letter). Int J Epidemiol 2016; in press.

  152. Schwartz S, Gatto NM, Campbell UB. Causal identification: A charge of epidemiology in danger of marginalization. Ann Epidemiol. 2016;26:669–73.

    PubMed  Article  Google Scholar 

  153. Schwartz S, Gatto NM, Campbell UB. Heeding the call for less casual causal inferences: The utility of realized (quantitative) causal effects. Ann Epidemiol 2017;27: in press.

  154. Seliger C, Meier CR, Becker C, Jick SS, Bogdahn U. Hau1 P, Leitzmann MF. Statin use and risk of glioma: population-based case-control analysis. Eur J Epidemiol. 2016;31:947–51.

    CAS  PubMed  Article  Google Scholar 

  155. Sellke TM, Bayarri MJ, Berger JO. Calibration of p values for testing precise null hypotheses. Am Stat. 2001;55:62–71.

    Article  Google Scholar 

  156. Senn SJ. Two cheers for P-values. J Epidemiol Biostat. 2001;6(2):193–204.

    CAS  PubMed  Article  Google Scholar 

  157. Senn SJ. Letter to the Editor re: Goodman 1992. Stat Med. 2002;21:2437–44.

    PubMed  Article  Google Scholar 

  158. Shafer G. Comment: Estimating causal effects. Int J Epidemiol. 2002;31:434–5.

    PubMed  Article  Google Scholar 

  159. Simon HA, Rescher N. Cause and counterfactual. Philosophy of Science. 1966;33(4):323–40.

    Article  Google Scholar 

  160. Spirtes P, Glymour C, Scheines R. Causation, prediction, and search. Cambridge MA: MIT Press; 2001.

    Google Scholar 

  161. Stallones RA. To advance epidemiology. Ann Rev Public Health. 1980;1:69–82.

    CAS  Article  Google Scholar 

  162. Stalnaker RC. A theory of conditionals. In: Studies in Logical Theory, ed. Rescher N, 98-112. Oxford: Blackwell, 1968. Repr. in Causation and Conditionals, ed. E. Sosa, 165-79. Oxford: Oxford University Press, 1975.

  163. Stolley PD. When genius errs: R.A. Fisher and the lung cancer controversy. Am J Epidemiol. 1991;133(5):416–25.

    CAS  PubMed  Article  Google Scholar 

  164. Student (Gossett, WS). The probable error of a mean. Biometrika 1908;VI:1–25.

  165. Sullivan S, Greenland S. Bayesian regression in SAS software. Int J Epidemiol 2013;42:308-317. Erratum. Int J Epidemiol. 2014;43:1667–8.

    Article  Google Scholar 

  166. Susser M. Judgment and causal inference. Am J Epidemiol. 1977;105:1–15.

    CAS  PubMed  Article  Google Scholar 

  167. Susser M. What is a cause and how do we know one? A grammar for pragmatic epidemiology. Am J Epidemiol. 1991;133:635–48.

    CAS  PubMed  Article  Google Scholar 

  168. Taleb NN. The Black Swan: The Impact of the Highly Improbable, 2nd ed. Random House 2010.

  169. Tukey JW. Causation, regression, and path analysis. In: Kempthorne O, ed. Statistics and Mathematics in Biology. Ames: Iowa State Press, 1954; Ch. 3.

  170. Tukey JW. The future of data analysis. Ann Math Stat. 1962;33:1–67.

    Article  Google Scholar 

  171. Vandenbroucke JP. Commentary: ‘Smoking and lung cancer’ the embryogenesis of modern epidemiology. Int J Epidemiol. 2009;38:1193–6.

    PubMed  Article  Google Scholar 

  172. Vandenbroucke JP, Broadbent A, Pearce N. Causality and causal inference in epidemiology: the need for a pluralistic approach. Int J Epidemiol 2016; in press.

  173. VanderWeele TJ. Explanation in causal inference: methods for mediation and interaction. New York, NY: Oxford University Press; 2015.

    Google Scholar 

  174. VanderWeele TJ. On causes, causal inference, and potential outcomes. Int J Epidemiol 2016a; in press.

  175. VanderWeele TJ. The role of potential outcomes thinking in assessing mediation and interaction. Int J Epidemiol 2016b; in press.

  176. VanderWeele TJ. Discussion of “Causal inference using invariant prediction: identification and confidence intervals” by Peters, Bühlmann and Meinshausen. J Roy Stat Soc B. 2016;78:995.

    Google Scholar 

  177. VanderWeele TJ, Hernán MA. Causal effects and natural laws: Towards a conceptualization of causal counterfactuals for nonmanipulable exposures, with application to the effects of race and sex. Ch. 9 in: Berzuini C, Dawid AP, Bernardinelli L (eds.). Causal Inference: Statistical Perspectives and Applications. Chichester: Wiley 2012, 101-13.

  178. VanderWeele TJ, Hernán MA, Tchetgen Tchetgen EJ, Robins JM. Re: Causality and causal inference in epidemiology: the need for a pluralistic approach (letter). Int J Epidemiol 2016; in press.

  179. VanderWeele TJ, Robins JM. Stochastic counterfactuals and stochastic sufficient causes. Statistica Sinica. 2012;22:279–92.

    Article  Google Scholar 

  180. VanderWeele TJ, Robinson WR. On causal interpretation of race in regressions adjusting for confounding and mediating variables. Epidemiol. 2014;25:473–84.

    Article  Google Scholar 

  181. Wagenmakers E-J. A practical solution to the pervasive problem of p values. Psychon Bull Rev. 2007;14:779–804.

    PubMed  Article  Google Scholar 

  182. Walker AM. Reporting the results of epidemiologic studies. Am J Public Health. 1986a;76:556–8.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  183. Walker AM. Significance tests represent consensus and standard practice. Am J Public Health. 1986b;76:1033.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  184. Wasserstein RL, Lazar NA. The ASA’s statement on p-values: context, process, and purpose. Am Statist. 2016;70(2):129–33.

    Article  Google Scholar 

  185. Anonymous. A new low in drug research: 21 fabricated studies. Wall Street Journal Mar. 11, 2009.

  186. Welch BL. On the z-test in randomized blocks and Latin squares. Biometrika. 1937;29:21–52.

    Article  Google Scholar 

  187. Wilk MB. The randomization analysis of a generalized randomized block design. Biometrika. 1955;42:70–9.

    Google Scholar 

  188. Wright S. Correlation and causation. Journal of Agricultural Research. 1921;20:557–85.

    Google Scholar 

  189. Yates F. The influence of statistical methods for research workers on the development of the science of statistics. J Am Stat Assoc. 1951;46:19–34.

    Google Scholar 

Download references

Acknowledgements

I am deeply indebted to many colleagues for extensive comments and correspondence on the initial draft of this paper, including Alex Broadbent, Jan Vandenbroucke, Neil Pearce, Ashley Naimi, Jay Kaufman, Sharon Schwartz, Nicolle Gatto, Ulka Campbell, George Maldonado, Alfredo Morabia, James Robins, and Tyler VanderWeele. Any errors that remain are solely my responsibility.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Sander Greenland.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Greenland, S. For and Against Methodologies: Some Perspectives on Recent Causal and Statistical Inference Debates. Eur J Epidemiol 32, 3–20 (2017). https://doi.org/10.1007/s10654-017-0230-6

Download citation

Key Words

  • Bias
  • Causal inference
  • Causation
  • Counterfactuals
  • Potential outcomes
  • Effect estimation
  • Hypothesis testing
  • Intervention analysis
  • Modeling
  • Significance testing
  • Research synthesis
  • Statistical inference