An intuitive review of methods for observational studies of comparative effectiveness
- 635 Downloads
I use diagrams to illustrate the sources of potential selection bias in observational studies of comparative effectiveness. I adapt these diagrams for three hypothetical scenarios that clarify the strengths and weaknesses of two prominent methods used to account for potential selection bias: propensity scores and instrumental variables. After reviewing the fundamentals of how to apply each method, including new developments that make implementation easier, I refer to some recent studies that illustrate how choice of method can affect estimates. I conclude by emphasizing that many studies with apparently rich sources of data are nevertheless unlikely to produce unbiased estimates and that conceptual modeling can help identify these problems in advance.
KeywordsComparative effectiveness Observational studies Selection bias Propensity scores Instrumental variables
This research was supported by Grant Number IAD 06-112 from the Health Services Research and Development Service of the U.S. Department of Veterans Affairs. All opinions expressed in this paper are those of the author and do not necessarily reflect the official position of the U.S. Department of Veterans Affairs or of Boston University. The author wishes to thank Matt Maciejewski, Paul Hebert, Ann Hendricks, Austin Frakt, and an anonymous reviewer for helpful comments.
- ALLHAT Officers and Coordinators for the ALLHAT Collaborative Research Group: Major outcomes in high-risk hypertensive patients randomized to angiotensin-converting enzyme inhibitor or calcium channel blocker vs. diuretic. JAMA 288(23), 2981–2997 (2002). doi: 10.1001/jama.288.23.2981 CrossRefGoogle Scholar
- Baum, C.F., Shaffer, M.E., Stillman, S.: Instrumental variables and GMM: estimation and testing. Stat. J. 3(1), 1–31 (2003)Google Scholar
- Clancy, C.: Health issues and opportunities at AHRQ. Testimony before the House Subcommittee on Labor-HHS-Education appropriations, Washington DC, March 5, 2008. http://www.ahrq.gov/news/test30508.htm (2008). Accessed 7 April 2008
- Congressional Budget Offices: Research on the comparative effectiveness of medical treatments: issues and options for an expanded federal role. Congress of the United States, Pub. No. 2975, December 2007Google Scholar
- Congressional Research Service: Comparative clinical effectiveness and cost-effectiveness research: background, history, and overview. CRS Report for Congress, October 15, 2007Google Scholar
- Davidson, R., MacKinnon, J.G.: Estimation and Inference in Econometrics. Oxford University Press, New York (1993)Google Scholar
- Dixon, K.: US may compare medical products; companies wary. Reuters, March 31 (2008)Google Scholar
- Institute of Medicine: Learning what works best: the nation’s need for evidence on comparative effectiveness in health care. http://www.iom.edu/ebm-effectiveness (2007) Accessed 19 May 2008
- Stukel, T.A., Fisher, E.S., Wennberg, D.E., Alter, D.A., Gottlieb, D.J., Vermeulen, M.J.: Analysis of observational studies in the presence of treatment selection bias: effects of invasive cardiac management on AMI survival using propensity score and instrumental variable methods. JAMA 297(3), 278–285 (2007). doi: 10.1001/jama.297.3.278 PubMedCrossRefGoogle Scholar