Abstract
This article compared standard regression (logistic), propensity score weighting, propensity score matching, and difference-in-difference (DID) methods in determining the impact of second-generation antidepressant (AD) use on mania-related visits among adult patients with bipolar disorder. Using a large managed care claims database, a logistic regression was developed as a standard approach to predict the likelihood of having mania-related visits after receiving various types of treatments (AD monotherapy, mood stabilizer (MS) monotherapy, and AD-MS combination therapy) controlling for individual baseline characteristics. The propensity score method predicted the propensity to be with one treatment type versus another in the first-stage. Both weighting and greedy matching approaches were applied in the second-stage outcome model. For the DID method, a logistic regression was applied to predict the differential likelihood of having mania-related visits in post-baseline versus baseline periods on different treatments. Both full sample and propensity score-matched sample were applied for the DID method. Except DID with full sample, the results from all other methods suggested no higher likelihood of mania-related visits for second-generation AD-related therapies compared to MS monotherapy. We concluded that standard regression, propensity scoring, and DID methods may produce inconsistent outcomes in a logistic regression framework, when patient baseline characteristics are different between comparison groups and/or not all potential confounders can be correctly measured and fully controlled. Researchers need to be cautious of the basic assumptions and sensitivities of various methods before making a final conclusion. The DID method may be considered in outcome studies when pre-and-post data are available.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abadie, A.: Semiparametric difference-in-differences estimators. Rev. Econ. Stud. 72, 1–19 (2005)
Abadie, A., Imbens, G.W.: Large sample properties of matching estimators for average treatment effects. Econometrica 74, 235–267 (2006)
American Psychiatric Association Practice guideline for the treatment of patients with bipolar disorder (revision). Am. J. Psychiatry 159, 1–50 (2002)
Andersen, R.M.: Revisiting the behavioral model and access to medical care: Does it matter? J. Health Soc. Behav. 36, 1–10 (1995)
Angrist, J.D., Imbens, G.W., Rubin, D.B.: Identification of causal effects using instrumental variables. J. Am. Stat. Assoc. 91, 444–455 (1996)
Athey, S., Imbens, G.W.: Identification and inference in nonlinear difference-in-differences models. Econometrica 74, 431–497 (2006)
Baser, O.: Too much ado about propensity score models? Comparing methods of propensity score matching. Value Health 9, 377–385 (2006)
Becker, S.O., Ichino, A.: Estimation of average treatment effects based on propensity scores. Stata J. 2, 358–377 (2002)
Cassidy, F., Ahearn, E., Carroll, B.J.: Elevated frequency of diabetes mellitus in hospitalized manic-depressive patients. Am. J. Psych. 156, 1417–1420 (1999)
Charlson, M.E., Pompei, P., Ales, K.L., McKenzie, C.R.: A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation. J. Chron. Dis. 40, 373–383 (1987)
Chow, G.C.: Tests of equality between sets of coefficients in two linear regressions. Econometrica 28, 591–605 (1960)
D’Hoore, W., Bouckaert, A., Tilquin, C.: Practical considerations on the use of the Charlson comorbidity index with administrative data bases. J. Clin. Epidemiol. 49, 1429–1433 (1996)
Dehejia, R.H., Wahba, S.: Propensity score-matching methods for nonexperimental causal studies. Rev. Econ. Stat. 84, 151–161 (2002)
Fu, A.Z., Christensen, D.B., Hansen, R.A., Liu, G.G.: Second-generation antidepressant discontinuation on depressive relapse in adult patients with bipolar depression: A retrospective analysis. Clin. Ther. 28, 979–989 (2006)
Ghaemi, S.N., Hsu, D.J., Soldani, F., Goodwin, F.K.: Antidepressants in bipolar disorder: the case for caution. Bipolar Disord. 5, 421–433 (2003)
Gu, X.S., Rosenbaum, P.R.: Comparison of multivariate matching methods: structures, distances and algorithms. J. Comput. Graph. Stat. 2, 405–420 (1993)
Hansen, B.B.: ull matching in an observational study of coaching for the SAT. J. Am. Stat. Assoc. 99, 609–618 (2004)
Hirano, K., Imbens, G.W.: Estimation of causal effects using propensity score weighting: An application to data on right heart catheterization. Health Serv. Outcomes Res. Methodol. 2, 259–278 (2001)
Hirschfeld, R.M., Calabrese, J.R., Weissman, M.M., Reed, M., Davis, M.A., Frye, M.A., Keck, P.E. Jr, Lewis, L., McElroy, S.L., McNulty, J.P., Wagner, K.D.: Screening for bipolar disorder in the community. J. Clinl. Psychiatry 64, 53–59 (2003)
Kim, C.D., Lesage, A.D., Seguin, M., Chawky, N., Vanier, C., Lipp, O., Turecki, G.: Seasonal differences in psychopathology of male suicide completers. Compr. Psychiatry 45, 333–339 (2004)
Little, R.J.A. and Rubin, D.B.: Statistical analysis with missing data, Hoboken NJ, USA, John Wiley & Sons Inc (2002)
Lunceford, J.K., Davidian, M.: Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Stat. Med. 23, 2937–2960 (2004)
McClellan, M., McNeil, B., Newhouse, J.: Does more intensive treatment of acute myocardial infarction in the elderly reduce mortality? Analysis using instrumental variables. JAMA 272, 859–866 (1994)
Moller, H.J., Grunze, H.: Have some guidelines for the treatment of acute bipolar depression gone too far in the restriction of antidepressants? Eur Arch Psychiatry Clin Neurosci 250, 57–68 (2000)
Parsons, L.S.: Reducing Bias in a Propensity Score Matched-Pair Sample Using Greedy Matching Techniques, SAS Users Group International (SUGI) 26th conference, paper 214–26 (2001)
Rosenbaum, P.R., Rubin, D.B.: The central role of the propensity score in observational studies for causal effects. Biometrika 70, 41–55 (1983)
Rosenbaum, P.R., Rubin, D.B.: Reducing bias in observational studies using subclassification on the propensity score. J. Am. Stat. Assoc. 79, 516–524 (1984)
Rubin, D.B.: Estimating causal effects from large data sets using propensity scores. Ann. Intern. Med. 127(8 Pt 2), 757–763 (1997)
Shin, K., Schaffer, A., Levitt, A.J., Boyle, M.H.: Seasonality in a community sample of bipolar, unipolar and control subjects, J. Affect Disord. 86, 19–25 (2005)
Smith, H.L.: Matching With Multiple Controls to Estimate Treatment Effects in Observational Studies. Sociol. Methodol. 27, 325–353 (1997)
Suppes, T., Leverich, G.S., Keck, P.E., Nolen, W.A., Denicoff, K.D., Altshuler, L.L., McElroy, S.L., Rush, A.J., Kupka, R., Frye, M.A., Bickel, M., Post, R.M.: The Stanley Foundation Bipolar Treatment Outcome Network. II, Demographics and illness characteristics of the first 261 patients. J. Affect Disord. 67, 45–59 (2001)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fu, A.Z., Dow, W.H. & Liu, G.G. Propensity score and difference-in-difference methods: a study of second-generation antidepressant use in patients with bipolar disorder. Health Serv Outcomes Res Method 7, 23–38 (2007). https://doi.org/10.1007/s10742-006-0016-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10742-006-0016-x