Skip to main content
SpringerLink
Log in

Evaluation of Different Approaches for Confounding in Nonrandomised Observational Data: A Case-Study of Antipsychotics Treatment

  • Original Paper
  • Published:
Community Mental Health Journal Aims and scope Submit manuscript

Abstract

Although randomised controlled trials are regarded as the gold standard for treatments efficacy, evidence from observational studies remains relevant. To address the problem of possible confounding in these studies, investigators must employ analysis methods that adjust for confounders and lead to an unbiased estimation of the treatment effect. In this paper, the authors describe two relevant statistical methods. The first method represents the classical approach consisting of a multiple regression model including the effects of treatment and covariates. This approach considers the relation between prognostic factors and the outcome variable as a relevant criterion for adjustment. The second method is based on the propensity score, and focuses on the relation between prognostic factors and treatment assignment. These approaches were applied to a cohort of 183 French schizophrenic patients who were followed for a 2-year period (from 1998 to 2000). The probability of relapse according to antipsychotic treatment exposure was modelled using Cox regression models with the two statistical methods. Goodness-of-fit criteria were used to compare the modelling approaches. This study demonstrates that the propensity score, a predicted probability, has an important balancing property that underscores its value in strengthening the results of nonrandomised observational studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Adamina, M., et al. (2006). Propensity scores and the surgeon. British Journal of Surgery, 93(4), 389–394.

    Article  CAS  PubMed  Google Scholar 

  • Addington, D., Addington, J., & Atkinson, M. (1996). A psychometric comparison of the Calgary Depression Scale for Schizophrenia and the Hamilton Depression Rating Scale. Schizophrenia Research, 19(2–3), 205–212.

    Article  CAS  PubMed  Google Scholar 

  • Addington, D., Addington, J., & Maticka-Tyndale, E. (1993). Assessing depression in schizophrenia: The Calgary Depression Scale. British Journal of Psychiatry. Supplement, 22, 39–44.

    PubMed  Google Scholar 

  • Addington, D., Addington, J., & Maticka-Tyndale, E. (1994). Specificity of the Calgary Depression Scale for schizophrenics. Schizophrenia Research, 11(3), 239–244.

    Article  CAS  PubMed  Google Scholar 

  • Addington, D., Addington, J., & Schissel, B. (1990). A depression rating scale for schizophrenics. Schizophrenia Research, 3(4), 247–251.

    Article  CAS  PubMed  Google Scholar 

  • Addington, D., et al. (1992). Reliability and validity of a depression rating scale for schizophrenics. Schizophrenia Research, 6(3), 201–208.

    Article  CAS  PubMed  Google Scholar 

  • American Psychiatric Association. (1994). DSM-IV. Diagnostic and statistical manual of mental disorders. Washington, DC: American Psychiatric Association.

  • Austin, P. C. (2007). The performance of different propensity score methods for estimating marginal odds ratios. Statistics in Medicine, 26(16), 3078–3094.

    Article  PubMed  Google Scholar 

  • Austin, P. C. (2009). The relative ability of different propensity score methods to balance measured covariates between treated and untreated subjects in observational studies. Medical Decision Making, 29, 661–677.

    Google Scholar 

  • Austin, P. C., et al. (2007). Conditioning on the propensity score can result in biased estimation of common measures of treatment effect: A Monte Carlo study. Statistics in Medicine, 26(4), 754–768.

    Article  PubMed  Google Scholar 

  • Barnes, T. R. (1989). A rating scale for drug-induced akathisia. British Journal of Psychiatry, 154, 672–676.

    Article  CAS  PubMed  Google Scholar 

  • Baser, O. (2006). Too much ado about propensity score models? Comparing methods of propensity score matching. Value Health, 9(6), 377–385.

    Article  PubMed  Google Scholar 

  • Bebbington, P. E., et al. (2005). The European Schizophrenia Cohort (EuroSC): A naturalistic prognostic and economic study. Social Psychiatry and Psychiatric Epidemiology, 40(9), 707–717.

    Article  PubMed  Google Scholar 

  • Bender, R., & Lange, S. (2001). Adjusting for multiple testing—When and how? Journal of Clinical Epidemiology, 54(4), 343–349.

    Article  CAS  PubMed  Google Scholar 

  • Bora, E., Yucel, M., & Pantelis, C. (2009). Cognitive functioning in schizophrenia, schizoaffective disorder and affective psychoses: Meta-analytic study. British Journal of Psychiatry, 195(6), 475–482.

    Article  PubMed  Google Scholar 

  • Cascio, M. T., et al. (2012). Gender and duration of untreated psychosis: A systematic review and meta-analysis. Early Intervention in Psychiatry, 6(2), 115–127.

    Article  PubMed  Google Scholar 

  • Cepeda, M. S., et al. (2003). Comparison of logistic regression versus propensity score when the number of events is low and there are multiple confounders. American Journal of Epidemiology, 158(3), 280–287.

    Article  PubMed  Google Scholar 

  • Cleophas, T. J., & Zwinderman, A. H. (2007). Clinical trials: How to assess confounding and why so. Current Clinical Pharmacology, 2(2), 129–133.

    Article  PubMed  Google Scholar 

  • CNIL. (2004). Act no 78-17 of 6 January 1978 on data processing, data files and individual liberties (amended by the Act of 6 August 2004 relating to the protection of individuals with regard to the processing of personal data), J.o.d.l.R.F.A.N.e. Sénat, Editor, France.

  • Cook, E. F., & Goldman, L. (1989). Performance of tests of significance based on stratification by a multivariate confounder score or by a propensity score. Journal of Clinical Epidemiology, 42(4), 317–324.

    Article  CAS  PubMed  Google Scholar 

  • Correll, C. U., & Gallego, J. A. (2012). Antipsychotic polypharmacy: A comprehensive evaluation of relevant correlates of a long-standing clinical practice. Psychiatric Clinics of North America, 35(3), 661–681.

    Article  PubMed Central  PubMed  Google Scholar 

  • Correll, C. U., et al. (2007). Does antipsychotic polypharmacy increase the risk for metabolic syndrome? Schizophrenia Research, 89(1–3), 91–100.

    Article  PubMed Central  PubMed  Google Scholar 

  • Correll, C. U., et al. (2009). Antipsychotic combinations vs monotherapy in schizophrenia: A meta-analysis of randomized controlled trials. Schizophrenia Bulletin, 35(2), 443–457.

    Article  PubMed Central  PubMed  Google Scholar 

  • D’Agostino, R. B., Jr. (1998). Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Statistics in Medicine, 17(19), 2265–2281.

    Article  PubMed  Google Scholar 

  • Dausch, B. M., Miklowitz, D. J., & Richards, J. A. (1996). Global assessment of relational functioning scale (GARF): II. Reliability and validity in a sample of families of bipolar patients. Family Process, 35(2), 175–189.

    Article  CAS  PubMed  Google Scholar 

  • de Leon, J. (2008). Beyond the “hype” on the association between metabolic syndrome and atypical antipsychotics: The confounding effects of cohort, typical antipsychotics, severe mental illness, comedications, and comorbid substance use. Journal of Clinical Psychopharmacology, 28(2), 125–131.

    Article  PubMed  Google Scholar 

  • Doan, B. Q., et al. (2006). Covariate-based linkage analysis: Application of a propensity score as the single covariate consistently improves power to detect linkage. European Journal of Human Genetics, 14(9), 1018–1026.

    Article  CAS  PubMed  Google Scholar 

  • Gail, M., Wieand, S., & Piantadosi, S. (1984). Biased estimates of treatment effect in randomized experiment with nonlinear regressions and omitted covariates. Biometrika, 7, 431–444.

    Article  Google Scholar 

  • Gilmer, T. P., et al. (2007). Antipsychotic polypharmacy trends among Medicaid beneficiaries with schizophrenia in San Diego County, 1999–2004. Psychiatric Services (Washington, D. C.), 58(7), 1007–1010.

    Article  Google Scholar 

  • Goghari, V. M., et al. (2012) A 20-year multi-follow-up of hallucinations in schizophrenia, other psychotic, and mood disorders. Psychological Medicine, 43(6), 1151–1160.

    Google Scholar 

  • Goldman, H. H., Skodol, A. E., & Lave, T. R. (1992). Revising axis V for DSM-IV: A review of measures of social functioning. American Journal of Psychiatry, 149(9), 1148–1156.

    CAS  PubMed  Google Scholar 

  • Groenwold, R. H., et al. (2011). Selection of confounding variables should not be based on observed associations with exposure. European Journal of Epidemiology, 26, 589–593.

    Article  PubMed Central  PubMed  Google Scholar 

  • Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29–36.

    Article  CAS  PubMed  Google Scholar 

  • Joukamaa, M., et al. (2006). Schizophrenia, neuroleptic medication and mortality. British Journal of Psychiatry, 188, 122–127.

    Article  PubMed  Google Scholar 

  • Kay, S. R., Fiszbein, A., & Opler, L. A. (1987). The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin, 13(2), 261–276.

    Article  CAS  PubMed  Google Scholar 

  • Kay, S. R., Opler, L. A., & Lindenmayer, J. P. (1989). The Positive and Negative Syndrome Scale (PANSS): Rationale and standardisation. British Journal of Psychiatry. Supplement, 7, 59–67.

    PubMed  Google Scholar 

  • Khan, A. (2010). Antipsychotic polypharmacy is not associated with increased mortality from natural causes in patients with schizophrenia compared to monotherapy. Evidence Based Mental Health, 13(3), 90.

    Google Scholar 

  • Kirkbride, J. B., et al. (2012). Incidence of schizophrenia and other psychoses in England, 1950–2009: A systematic review and meta-analyses. PLoS ONE, 7(3), e31660.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  • Klungel, O. H., et al. (2004). Methods to assess intended effects of drug treatment in observational studies are reviewed. Journal of Clinical Epidemiology, 57(12), 1223–1231.

    Article  PubMed  Google Scholar 

  • Kovess, V., et al. (1995). Has the sectorization of psychiatric services in France really been effective? Social Psychiatry and Psychiatric Epidemiology, 30(3), 132–138.

    Article  CAS  PubMed  Google Scholar 

  • Kreyenbuhl, J., et al. (2007). Adding or switching antipsychotic medications in treatment-refractory schizophrenia. Psychiatric Services (Washington, D. C.), 58(7), 983–990.

    Article  Google Scholar 

  • Kwiatkowski, F., et al. (2007). Propensity score: Interest and limits. Bulletin du Cancer, 94(7), 680–686.

    PubMed  Google Scholar 

  • Lader, M. (1995). What is relapse in schizophrenia? International Clinical Psychopharmacology, 9(Suppl 5), 5–9.

    PubMed  Google Scholar 

  • Lehman, A. F., et al. (2004a). Practice guideline for the treatment of patients with schizophrenia, second edition. American Journal of Psychiatry, 161(2 Suppl), 1–56.

    PubMed  Google Scholar 

  • Lehman, A. F., et al. (2004b). The Schizophrenia Patient Outcomes Research Team (PORT): Updated treatment recommendations 2003. Schizophrenia Bulletin, 30(2), 193–217.

    Article  PubMed  Google Scholar 

  • Leucht, S., et al. (2008). Methodological issues in current antipsychotic drug trials. Schizophrenia Bulletin, 34(2), 275–285.

    Article  PubMed Central  PubMed  Google Scholar 

  • Martens, E. P., et al. (2008). Comparing treatment effects after adjustment with multivariable Cox proportional hazards regression and propensity score methods. Pharmacoepidemiology and Drug Safety, 17(1), 1–8.

    Article  PubMed  Google Scholar 

  • Misawa, F., et al. (2011). Is antipsychotic polypharmacy associated with metabolic syndrome even after adjustment for lifestyle effects? A cross-sectional study. BMC Psychiatry, 11, 118.

    Article  PubMed Central  PubMed  Google Scholar 

  • Montout, C., et al. (2002). Neuroleptics and mortality in schizophrenia: Prospective analysis of deaths in a French cohort of schizophrenic patients. Schizophrenia Research, 57(2–3), 147–156.

    Article  PubMed  Google Scholar 

  • Moore, T. A., et al. (2007). The Texas Medication Algorithm Project antipsychotic algorithm for schizophrenia: 2006 update. Journal of Clinical Psychiatry, 68(11), 1751–1762.

    Article  CAS  PubMed  Google Scholar 

  • Morgan, M. G., et al. (2003). Prospective analysis of premature mortality in schizophrenia in relation to health service engagement: A 7.5-year study within an epidemiologically complete, homogeneous population in rural Ireland. Psychiatry Research, 117(2), 127–135.

    Article  PubMed  Google Scholar 

  • NICE. (2002). Schizophrenia. Core interventions in the treatment and management of schizophrenia in primary and secondary care. London: National Institute for Clinical Excellence.

    Google Scholar 

  • Riou Franca, L., et al. (2006). Cost-effectiveness of drotrecogin alfa (activated) in the treatment of severe sepsis with multiple organ failure. International Journal of Technology Assessment in Health Care, 22(1), 101–108.

    PubMed  Google Scholar 

  • Rosembaum, P., & Rubin, D. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–45.

    Article  Google Scholar 

  • Rubin, D. B. (1997). Estimating causal effects from large data sets using propensity scores. Annals of Internal Medicine, 127(8 Pt 2), 757–763.

    Article  CAS  PubMed  Google Scholar 

  • Ruta, D., et al. (1993). The SF 36 health survey questionnaire. A valid measure of health status. BMJ, 307(6901), 448–449.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  • Schisterman, E. F., Cole, S. R., & Platt, R. W. (2009). Overadjustment bias and unnecessary adjustment in epidemiologic studies. Epidemiology, 20(4), 488–495.

    Article  PubMed Central  PubMed  Google Scholar 

  • Schneeweiss, S., & Avorn, J. (2009). Antipsychotic agents and sudden cardiac death—How should we manage the risk? New England Journal of Medicine, 360(3), 294–296.

    Article  CAS  PubMed  Google Scholar 

  • Sebastian, C. S., Glazer, W., & Buckley, P. F. (2004). Naturalistic studies of second generation antipsychotics in the treatment of schizophrenia. Current Medicinal Chemistry, 11(3), 329–342.

    Article  CAS  PubMed  Google Scholar 

  • Shah, B. R., et al. (2005). Propensity score methods gave similar results to traditional regression modeling in observational studies: A systematic review. Journal of Clinical Epidemiology, 58(6), 550–559.

    Article  PubMed  Google Scholar 

  • Stukel, T. A., et al. (2007). Analysis of observational studies in the presence of treatment selection bias: Effects of invasive cardiac management on AMI survival using propensity score and instrumental variable methods. JAMA, 297(3), 278–285.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  • Sturmer, T., et al. (2005). Analytic strategies to adjust confounding using exposure propensity scores and disease risk scores: Nonsteroidal antiinflammatory drugs and short-term mortality in the elderly. American Journal of Epidemiology, 161(9), 891–898.

    Article  PubMed Central  PubMed  Google Scholar 

  • Sturmer, T., et al. (2006). A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. Journal of Clinical Epidemiology, 59(5), 437–447.

    Article  PubMed Central  PubMed  Google Scholar 

  • Tiihonen, J., et al. (2012). Polypharmacy with antipsychotics, antidepressants, or benzodiazepines and mortality in schizophrenia. Archives of General Psychiatry, 69(5), 476–483.

    Article  CAS  PubMed  Google Scholar 

  • VanderWeele, T. J. (2009). On the relative nature of overadjustment and unnecessary adjustment. Epidemiology, 20(4), 496–499.

    Article  PubMed  Google Scholar 

  • Waddington, J. L., Youssef, H. A., & Kinsella, A. (1998). Mortality in schizophrenia. Antipsychotic polypharmacy and absence of adjunctive anticholinergics over the course of a 10-year prospective study. British Journal of Psychiatry, 173, 325–329.

    Article  CAS  PubMed  Google Scholar 

  • Weiden, P., et al. (1994). Rating of medication influences (ROMI) scale in schizophrenia. Schizophrenia Bulletin, 20(2), 297–310.

    Article  CAS  PubMed  Google Scholar 

  • Weinmann, S., Read, J., & Aderhold, V. (2009). Influence of antipsychotics on mortality in schizophrenia: Systematic review. Schizophrenia Research, 113(1), 1–11.

    Article  PubMed  Google Scholar 

  • Weitzen, S., et al. (2005). Weaknesses of goodness-of-fit tests for evaluating propensity score models: The case of the omitted confounder. Pharmacoepidemiology and Drug Safety, 14(4), 227–238.

    Article  PubMed  Google Scholar 

  • WMA. (2008). Declaration of Helsinki, ethical principles for medical research involving human subjects. G.A. World Medical Association, Editor. Seoul.

Download references

Conflict of interest

The authors declare that they have no competing interests.

Author information

Authors and Affiliations

  1. INSERM, U669, National Institute of Health and Medical Research, Paris, France

    E. Sarlon

  2. UMR-S0669, University of Paris-Sud and University of Paris Descartes, Paris, France

    E. Sarlon

  3. Department of Public Health, GHPSO, Oise, France

    E. Sarlon

  4. Creativ-Ceutical, Paris, France

    A. Millier, S. Aballéa & M. Toumi

  5. Decision Sciences and Health Policy, University Claude Bernard Lyon I, Lyon, France

    M. Toumi

  6. DIM, Groupe Hospitalier Public du Sud de l’Oise, Boulevard Laennec, 60109, Creil Cedex, France

    E. Sarlon

Authors
  1. E. Sarlon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Millier
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. Aballéa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. Toumi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to E. Sarlon.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sarlon, E., Millier, A., Aballéa, S. et al. Evaluation of Different Approaches for Confounding in Nonrandomised Observational Data: A Case-Study of Antipsychotics Treatment. Community Ment Health J 50, 711–720 (2014). https://doi.org/10.1007/s10597-014-9723-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10597-014-9723-x

Keywords

Search

Navigation