Abstract
In their recent book, Is Inequality Bad for Our Health?, Daniels, Kennedy, and Kawachi claim that to “act justly in health policy, we must have knowledge about the causal pathways through which socioeconomic (and other) inequalities work to produce differential health outcomes.” One of the central problems with this approach is its dependency on “knowledge about the causal pathways.” A widely held belief is that the randomized clinical trial (RCT) is, and ought to be the “gold standard” of evaluating the causal efficacy of interventions. However, often the only data available are non-experimental, observational data. For such data, the necessary randomization is missing. Because the randomization is missing, it seems to follow that it is not possible to make epistemically warranted claims about the causal pathways. Although we are not sanguine about the difficulty in using observational data to make warranted causal claims, we are not as pessimistic as those who believe that the only warranted causal claims are claims based on data from (idealized) RCTs. We argue that careful, thoughtful study design, informed by expert knowledge, that incorporates propensity score matching methods in conjunction with instrumental variable analyses, provides the possibility of warranted causal claims using observational data.
Similar content being viewed by others
References
Angrist J.D., Imbens G.W., Rubin D.B. (1996). Identification of effects using instrumental variables. Journal of the American Statistical Association 91(434): 444–455
Angrist J.D., Krueger A.B. (2001). Instrumental variables and the search for identification: From supply and demand to natural experiments. Journal of Economic Perspectives 15(4): 69–85
Austin P.C., Grootendorst P., Anderson G.M. (2007). A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: A Monte Carlo study. Statistics in Medicine 26(4): 734–753
Bellman R. (1961). Adaptive control processes. Princeton University Press, Princeton, NJ
Berk R.A. (2004). Regression analysis: A constructive critique. Sage Publications, Thousand Oaks, CA
Bond S.J., White I.R., Walker A.S. (2007). Instrumental variables and interactions in the causal analysis of a complex clinical trial. Statistics in Medicine 26(7): 1473–1496
Clogg C.C., Haritou A. (1997). The regression method of causal inference and a dilemma confronting this method. In: McKim V., Turner S. (eds) Causality in crisis? Statistical methods and the search for causal knowledge in the social sciences. University of Notre Dame, Press, Notre Dame IN, pp 83–112
D’Agostino R.B., Jr. (1998). Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Statistics in Medicine 17(19): 2265–2281
D’Agostino R.B., Jr., Rubin D.B. (2000). Estimating and using propensity scores with partially missing data. Journal of the American Statistical Association 95(451): 749–759
Daniels D., Kennedy B., Kawachi I. (2000). Justice is good for our health. In: Cohen J., Rogers J. (eds) Is inequality bad for our health?. Beacon Press, Boston, MA, pp 3–33
Davidson R., MacKinnon J.G. (1993). Estimation and inference in econometrics. Oxford University Press, New York, NY
DiPrete T.A., Gangl M. (2004). Assessing bias in the estimation of causal effects: Rosenbaum bounds on matching estimators and instrumental variables estimation with imperfect instruments. Sociological Methodology 34: 271–310
Freedman D.A. (1999). From association to causation: Some remarks on the history of statistics. Statistical Science 14(3): 243–258
Freedman D.A. (2005). Statistical models: Theory and practice. Cambridge University Press, Cambridge
Glymour M.M. (2006). Natural experiments and instrumental variable analyses in social epidemiology. In: Oakes J.M., Kaufman J. (eds) Methods in social epidemiology. Jossey-Bass, San Francisco, CA, pp 429–460
Greenland S. (1990). Randomization, statistics, and causal inference. Epidemiology 1: 421–429
Greenland S. (2000). An introduction to instrumental variables for epidemiologists. International Journal of Epidemiology 29: 722–729
Greenland S., Robins J.M. (1986). Identifiability, exchangeability, and epidemiological confounding. International Journal of Epidemiology 15(3): 413–419
Haukoos J.S., Newgard C.D. (2007). Advanced statistics: Missing data in clinical research – Part 1: An introduction and conceptual framework. Academic Emergency Medicine 14(7): 662–668
Heckman J.J. (1997). Instrumental variables: A study of implicit behavioral assumptions used in making program evaluations. The Journal of Human Resources 32(3): 441–462
Heckman, J. J. (2005). The scientific model of causality. In R. Stolzenberg (Ed.), Sociological methodology (Vol. 35, pp. 1–97). Oxford: Basil Blackwell (for the American Sociological Association).
Hernán M.A. (2004). A definition of causal effect for epidemiological research. Journal of Epidemiology and Community Health 58: 265–271
Hernán, M.A., Robins J.M. (2006). Instruments for causal inference: An epidemiologist’s dream? Epidemiology 17(4): 360–372
Hintikka J. (1975). The intentions of intentionality and other new models for modalities. D. Reidel Publishing Co., Dordrecht
Holland P. (1986). Statistics and causal inference. Journal of the American Statistical Association 81(396): 945–960
Humphreys P. (1986). Causation in the social sciences: An overview. Synthese 68: 1–12
Imbens G.W., Angrist J.D. (1994). Identification and estimation of local average treatment effects. Econometrica 62(2): 467–475
Imbens G.W., Rosenbaum P.R. (2005). Robust, accurate confidence intervals with a weak instrument: Quarter of birth and education. Journal of the Royal Statistical Society, Series A 168(part 1): 109–126
Kaufman J.S., Kaufman S., Poole C. (2003). Causal inference from randomized trials in social epidemiology. Social Science and Medicine 57: 2397–2409
Linden A., Adams J.L. (2006). Evaluating disease management programme effectiveness: An introduction to instrumental variables. Journal of Evaluation in Clinical Practice 12(2): 148–154
Little R.J., Rubin D.B. (2000). Causal effects in clinical and epidemiological studies via potential outcomes: Concepts and approaches. Annual Review of Public Health 21: 121–145
Luellen J.K., Shadish W.R., Clark M.H. (2005). Propensity scores: An introduction and experimental test. Evaluation Review 29(6): 530–558
Maldonado G., Greenland S. (2002). Estimating causal effects. International Journal of Epidemiology 31: 422–429
Manski, C. F. (1993). Identification problems in the social sciences. In P. Marsden (Ed.), Social methodology (Vol. 23, pp. 1–56). Oxford: Basil Blackwell (for the American Sociological Association).
Manski C.F. (1995). Identification problems in the social sciences. Harvard University Press, Cambridge, MA
Moffitt R. (2005). Remarks on the analysis of causal relationships in population research. Demography 42(1): 91–108
Newgard C.D., Haukoos J.S. (2007). Advanced statistics: Missing data in clinical research – Part 2: Multiple imputation. Academic Emergency Medicine 14(7): 669–678
Newgard C.D., Hedges J.R., Arthur M., Mullins R.J. (2004). Advanced statistics: The propensity score – A method for estimating treatment effect in observational research. Academic Emergency Medicine 11(9): 953–961
Newhouse J.P., McClellan M. (1998). Econometrics in outcomes research: The use of instrumental variables. Annual Review of Public Health 19: 17–34
Newman S.C. (2004). Commonalities in the classical, collapsibility and counterfactual concepts of confounding. Journal of Clinical Epidemiology 57: 325–329
Oakes M.J. (2004). The (Mis)estimation of neighborhood effects: Causal inference for a practicable social epidemiology. Social Science and Medicine 58(10): 1929–1952
Oakes M.J., Johnson P.J. (2006). Propensity score matching for social epidemiology. In: Oakes J.M., Kaufman J. (eds) Methods in social epidemiology. Jossey-Bass, San Francisco, CA, pp 370–392
Pearl J. (2000). Causality: Models, reasoning, and inference. Cambridge University Press, Cambridge
Pearl J. (2001). Causal inference in health sciences: A conceptual introduction. Health Services and Outcomes Research Methodology 2: 189–220
Randall Jr. J.H. (1940). The making of the modern mind – Revised edition. Houghton Mifflin Company, Boston, MA
Reiter J. (2000). Using statistics to determine causal relationships. The American Mathematical Monthly 107(1): 24–32
Robins J.M., Scheines R., Spirtes P., Wasserman L. (2003). Uniform consistency in causal inference. Biometrika 90(3): 491–515
Rosenbaum P.R. (2002). Observational studies (2nd ed). Springer, New York, NY
Rosenbaum P.R. (2004). Matching in observational studies. In: Gelman A., Meng X.-L. (eds) Applied Bayesian modeling and causal inference from incomplete-data perspectives. Wiley and Sons, Ltd, West Sussex, pp 15–24
Rosenbaum P.R., Rubin D.B. (1983). The central role of propensity scores in observational studies for causal effects. Biometrika 70(1): 41–55
Rosenbaum P.R., Rubin D.B. (1984). On the nature and discovery of structure: Comment. Journal of the American Statistical Association 79(385): 26–28
Rosenbaum P.R., Rubin D.B. (1985a). Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. The American Statistician 39(1): 33–38
Rosenbaum P.R., Rubin D.B. (1985b). The bias due to incomplete matching. Biometrics 41(1): 103–116
Rothman K. J. (2002). Epidemiology: An introduction. Oxford University Press, Oxford
Rubin D.B. (1986). Statistics and causal inference: Comment: Which ifs have causal answers. Journal of the American Statistical Association 81(396): 961–962
Rubin D.B. (1997). Estimating causal effects from large data sets using propensity scores. Annals of Internal Medicine 127(8): 757–763
Rubin D.B. (2004). On principles for modeling propensity scores in medical research. Pharmacoepidemiology and Drug Safety 13(12): 855–857
Rubin D.B. (2007). The design versus the analysis of observational studies for causal effects: Parallels with the design of randomized trials. Statistics in Medicine 26(1): 20–36
Rubin D.B., Thomas N. (1996). Matching using estimated propensity scores: Relating theory to practice. Biometrics 52(1): 249–264
Shadish W.R., Cook T.D., Campbell D.T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin Company, Boston, MA
Smith H.L. (1997). Matching with multiple controls to estimate treatment effects in observational studies. Sociological Methodology 27: 325–353
Smith H.L. (2003). Some thoughts on causation as it relates to demography and population studies. Population and Development Review 29(3): 459–469
Smith J.A., Todd P.E. (2001). Reconciling conflicting evidence on the performance of propensity-score matching methods. American Economics Review 91(2): 112–118
Sobel, M. E. (2005). Discussion: The scientific model of causality. In R. Stolzenberg (Ed.), Social methodology (Vol. 35, pp. 99–133). Oxford: Basil Blackwell (for the American Sociological Association).
Urbach P. (1985). Randomization and the design of experiments. Philosophy of Science 52(2): 256–273
Weitzen S., Lapane K.L., Toledano A.Y., Hume A.L., Mor V. (2005). Weaknesses of goodness-of-fit tests for evaluating propensity score models: The case of the omitted confounder. Pharmacoepidemiology and Drug Safety 14(4): 227–238
Winship C., Morgan S.L. (1999). The estimation of causal effects from observational data. Annual Review of Sociology 25: 659–706
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ward, A., Johnson, P.J. Addressing confounding errors when using non-experimental, observational data to make causal claims. Synthese 163, 419–432 (2008). https://doi.org/10.1007/s11229-007-9292-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11229-007-9292-4