Identifying Heterogeneous Treatment Effects of Drug Policy in Quasi-experimental Settings
Purpose of Review
We sought to describe the difference-in-differences study design and how they can be applied to identify the average treatment effect. We then extend this approach to identify heterogeneity in treatment effects based on (1) an individuals’ baseline risk of an event using risk scores, (2) the outcome distribution using quantile regression, and (3) prior trajectories of outcomes using group-based trajectory models.
The methods for the identification of heterogeneous treatment effect have developed in ways that can provide researchers and policymakers a more nuanced understanding of treatment effects.
Recent analytic advances found in other fields should be adopted and tested by pharmacoepidemiology and drug policy researcher to better understand the effects of new policies and interventions.
KeywordsHeterogeneity Heterogeneous treatment effects Policy analysis Difference-in-difference Quantile regression Risk score Group-based trajectory model Propensity score
Compliance With Ethical Standards
Conflict of Interest
Aaron N. Winn is supported by the National Center for Research Resources, the National Center for Advancing Translational Sciences, and the office of the Director, National Institutes of Health, through Grant Number KL2TR001438. Matthew L. Maciejewski reports grants from VA HSR&D (RCS 10–391), grants from VA HSR&D (CIN 13–410), grants from VA HSR&D (CRE 12–306), and grants from NIDA (R01 DA040056), outside the submitted work. Dr. Maciejewski owns Amgen stock due to his spouse’s employment. Stacie B. Dusetzina declares no potential conflicts of interest.
Human and Animal Rights and Informed Consent
This article does not contain any studies with human or animal subjects performed by any of the authors.
Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance
- 1.Kravitz RL, Duan N, Braslow J. Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Q. 2004;82(4):661–87.Google Scholar
- 2.Rothwell PM, Warlow CP. Prediction of benefit from carotid endarterectomy in individual patients: a risk-modelling study. European Carotid Surgery Trialists’ Collaborative Group. Lancet. 1999;353(9170):2105–10.Google Scholar
- 3.•• Le Cook B, Manning WG. Thinking beyond the mean: a practical guide for using quantile regression methods for health services research. Shanghai Arch Psychiatry. 2013;25(1):55–9 Excellent article that describes how to use quantile regression. Google Scholar
- 4.Haviland A, Nagin DS, Rosenbaum PR, Tremblay RE. Combining group-based trajectory modeling and propensity score matching for causal inferences in nonexperimental longitudinal data. Dev Psychol. 2008;44(2):422–36.Google Scholar
- 5.•• Haviland A, Nagin DS, Rosenbaum PR. Combining propensity score matching and group-based trajectory analysis in an observational study. Psychol Methods. 2007;12(3):247–67 First article that lays out how to combine group based trajectory models to identify heterogeneous treatment effects. Google Scholar
- 6.Winn AN, Fergestrom NM, Neuner JM. Using group-based trajectory models and propensity score weighting to detect heterogeneous treatment effects: the case study of generic hormonal therapy for women with breast cancer. Med Care. 2019;57(1):85–93.Google Scholar
- 7.Angrist J, Pischke J. Mostly harmless econometrics: an empiricist’s companion. Princeton, NJ: Princeton University Press; 2008 2008.Google Scholar
- 8.Bertrand M, Duflo E, Mullainathan S. How much should we trust difference-in-differences estimates? Q J Econ. 2004;119(1):249–75.Google Scholar
- 9.•• Daw JR, Hatfield LA. Matching and regression to the mean in difference-in-differences analysis. Health Serv Res. 2018;53(6):4138–56 Provides an excellent overview on the use of difference-in-difference design and how concerns about how to appropriate adjust for non-parallel trends. Google Scholar
- 10.Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002;27(4):299–309.Google Scholar
- 11.•• Ryan AM, Kontopantelis E, Linden A, Burgess JF, Jr. Now trending: coping with non-parallel trends in difference-in-differences analysis. Statistical methods in medical research. 2018:962280218814570. Provides an excellent overview on the use of difference-in-difference design and if a researcher can use matching to overcome non-parallel trends. Google Scholar
- 12.Daw JR, Hatfield LA. Matching in difference-in-differences: between a rock and a hard place. Health Serv Res. 2018;53(6):4111–7.Google Scholar
- 13.• Ryan AM, Burgess JF Jr, Dimick JB. Why we should not be indifferent to specification choices for difference-in-differences. Health Serv Res. 2015;50(4):1211–35 Details approaches to ensure that difference-in-difference design do not suffer from bias and fixes to common problems. Google Scholar
- 14.•• Norton EC, Dowd BE, Maciejewski ML. Marginal effects-quantifying the effect of changes in risk factors in logistic regression models. JAMA. 2019;321(13):1304–5 Provides guidance on how to use understand interaction terms in non-linear models. Google Scholar
- 15.Ai C, Norton EC. Interaction terms in logit and probit models. Econ Lett. 2003;80(1):123–9.Google Scholar
- 16.•• Kent DM, Nelson J, Dahabreh IJ, Rothwell PM, Altman DG, Hayward RA. Risk and treatment effect heterogeneity: re-analysis of individual participant data from 32 large clinical trials. Int J Epidemiol. 2016;45(6):2075–88 Superb article that details how important baseline risk in when understanding the harms and benefits of treatments. Google Scholar
- 17.Hayward RA, Kent DM, Vijan S, Hofer TP. Reporting clinical trial results to inform providers, payers, and consumers. Health Aff (Millwood). 2005;24(6):1571–81.Google Scholar
- 18.Kent DM, Hayward RA, Griffith JL, Vijan S, Beshansky JR, Califf RM, et al. An independently derived and validated predictive model for selecting patients with myocardial infarction who are likely to benefit from tissue plasminogen activator compared with streptokinase. Am J Med. 2002;113(2):104–11.Google Scholar
- 19.•• Lipkovich I, Dmitrienko A. B. R. D’ Agostino S. Tutorial in biostatistics: data-driven subgroup identification and analysis in clinical trials. Stat Med. 2017;36(1):136–96 Excellent guide to researcher that would like to identify subgroups using dava-driven approaches. Google Scholar
- 20.Green KM, Stuart EA. Examining moderation analyses in propensity score methods: application to depression and substance use. J Consult Clin Psychol. 2014;82(5):773–83.Google Scholar
- 21.Chin AL, Bentley JP, Pollom EL. Impact of state parity laws on copayments for and adherence to oral endocrine therapy for breast cancer. Cancer. 2018;125(3):374–81.Google Scholar
- 22.Dusetzina SB, Huskamp HA, Winn AN, Basch E, Keating NL. Out-of-pocket and health care spending changes for patients using orally administered anticancer therapy after adoption of state parity laws. JAMA Oncol. 2018;4(6):e173598.Google Scholar
- 23.Heckman J, Smith JC, Clements N. Making the most out of programme evaluations and social experiments: accounting for heterogeneity in programme impacts. Rev Econ Stud. 1997;64(4):487–535.Google Scholar
- 24.Sergio F. Efficient semiparametric estimation of quantile treatment effects. Econometrica. 2007;75(1):259–76.Google Scholar
- 25.Nagin DS, Odgers CL. Group-based trajectory modeling in clinical research. Annu Rev Clin Psychol. 2010;6:109–38.Google Scholar
- 26.Austin PC, Stuart EA. Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Stat Med. 2015;34(28):3661–79.Google Scholar