Do effectiveness (“real world”) studies on antipsychotics tell us the real truth?



In recent years, so-called “effectiveness studies” have gained increasing importance in the context of evidence-based medicine. These studies supposedly follow less restrictive methodological standards than phase III studies in terms of patient selection, co-medication and other design issues, and their results should therefore be better generalisable than those of phase III studies. Effectiveness studies, like other types of phase IV studies, can therefore contribute to the knowledge about antipsychotics or other psychopharmaceuticals and supply relevant information in addition to that gained from phase III trials. However, the less restrictive design and inherent methodological problems of phase IV studies mean that their results cannot falsify the results of phase III studies. The greater complexity of phase IV studies, for example the greater variance caused by the different kinds of confounders, means that their results have to be interpreted with great care and especially with a high degree of awareness of problematic design issues, such as insensitive primary outcome criteria, biased randomisation, unblinded treatment conditions, inclusion of chronic refractory patients, etc. Some recently published effectiveness studies on antipsychotic treatment of schizophrenia will be discussed under these methodological aspects. The main conclusions of these trials will be questioned on the basis of their severe methodological pitfalls.


effectiveness studies real world studies antipsychotics SGAs FGAs 


  1. 1.
    Adam D, Kasper S, Möller HJ, Singer EA (2005) Placebo-controlled trials in major depression are necessary and ethically justifiable: how to improve the communication between researchers and ethical committees. Eur Arch Psychiatry Clin Neurosci 255:258–260PubMedCrossRefGoogle Scholar
  2. 2.
    Awad AG, Voruganti LN (2004) New antipsychotics, compliance, quality of life, and subjective tolerability—are patients better off? Can J Psychiatry 49:297–302PubMedGoogle Scholar
  3. 3.
    Baldwin D, Broich K, Fritze J, Kasper S, Westenberg H, Möller HJ (2003) Placebo-controlled studies in depression: necessary, ethical and feasible. Eur Arch Psychiatry Clin Neurosci 253:22–28PubMedCrossRefGoogle Scholar
  4. 4.
    Barak Y, Shamir E, Mirecki I, Weizman R, Aizenberg D (2004) Switching elderly chronic psychotic patients to olanzapine. Int J Neuropsychopharmacol 7:165–169PubMedCrossRefGoogle Scholar
  5. 5.
    Bender S, Dittmann-Balcar A, Schall U, Wolstein J, Klimke A, Riedel M, Vorbach EU, Kuhn KU, Lambert M, Dittmann RW et al (2006) Influence of atypical neuroleptics on executive functioning in patients with schizophrenia: a randomized, double-blind comparison of olanzapine vs. clozapine. Int J Neuropsychopharmacol 9:135–145PubMedCrossRefGoogle Scholar
  6. 6.
    Buckley PF (2007) Receptor-binding profiles of antipsychotics: clinical strategies when switching between agents. J Clin Psychiatry 68(suppl 6):5–9Google Scholar
  7. 7.
    Buckley PF, Harvey PD, Bowie CR, Loebel A (2007) The relationship between symptomatic remission and neuropsychological improvement in schizophrenia patients switched to treatment with ziprasidone. Schizophr Res 94:99–106PubMedCrossRefGoogle Scholar
  8. 8.
    Caley CF, Weber SS (1995) Sulpride: an antipsychotic with selective dopaminergic antagonist properties. Ann Pharmacother 29:152–160PubMedGoogle Scholar
  9. 9.
    Cramer JA, Rosenheck R, Xu W, Thomas J, Henderson W, Charney DS (2000) Quality of life in schizophrenia: a comparison of instruments. Department of Veterans Affairs Cooperative Study Group on clozapine in refractory schizophrenia. Schizophr Bull 26:659–666PubMedGoogle Scholar
  10. 10.
    Dossenbach MR, Folnegovic-Smalc V, Hotujac L, Uglesic B, Tollefson GD, Grundy SL, Friedel P, Jakovljevic MM (2004) Double-blind, randomized comparison of olanzapine versus fluphenazine in the long-term treatment of schizophrenia. Prog Neuropsychopharmacol Biol Psychiatry 28:311–318PubMedCrossRefGoogle Scholar
  11. 11.
    Essock SM, Covell NH, Davis SM, Stroup TS, Rosenheck RA, Lieberman JA (2006) Effectiveness of switching antipsychotic medications. Am J Psychiatry 163:2090–2095PubMedCrossRefGoogle Scholar
  12. 12.
    Falkai P, Wobrock T, Lieberman J, Glenthoj B, Gattaz WF, Moller HJ (2005) World Federation of Societies of Biological Psychiatry (WFSBP) guidelines for biological treatment of schizophrenia, Part 1: acute treatment of schizophrenia. World J Biol Psychiatry 6:132–191PubMedCrossRefGoogle Scholar
  13. 13.
    Falkai P, Wobrock T, Lieberman J, Glenthoj B, Gattaz WF, Moller HJ (2006) World Federation of Societies of Biological Psychiatry (WFSBP) guidelines for biological treatment of schizophrenia, part 2: long-term treatment of schizophrenia. World J Biol Psychiatry 7:5–40PubMedCrossRefGoogle Scholar
  14. 14.
    Franz M, Lis S, Pluddemann K, Gallhofer B (1997) Conventional versus atypical neuroleptics: subjective quality of life in schizophrenic patients. Br J Psychiatry 170:422–425PubMedCrossRefGoogle Scholar
  15. 15.
    Glick ID (2006) Understanding the results of CATIE in the context of the field. CNS Spectr 11:40–47PubMedGoogle Scholar
  16. 16.
    Hegerl U, Juckel G, Muller-Schubert A, Pietzcker A, Gaebel W (1995) Schizophrenics with small P300: a subgroup with a neurodevelopmental disturbance and a high risk for tardive dyskinesia? Acta Psychiatr Scand 91:120–125PubMedCrossRefGoogle Scholar
  17. 17.
    Heinrichs DW, Hanlon TE, Carpenter WT Jr (1984) The Quality of life scale: an instrument for rating the schizophrenic deficit syndrome. Schizophr Bull 10:388–398PubMedGoogle Scholar
  18. 18.
    Heres S, Davis J, Maino K, Jetzinger E, Kissling W, Leucht S (2006) Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics. Am J Psychiatry 163:185–194PubMedCrossRefGoogle Scholar
  19. 19.
    Hofer A, Hummer M, Huber R, Kurz M, Walch T, Fleischhacker WW (2000) Selection bias in clinical trials with antipsychotics. J Clin Psychopharmacol 20:699–702PubMedCrossRefGoogle Scholar
  20. 20.
    Hogarty GE, Goldberg SC (1973) Drug and sociotherapy in the aftercare of schizophrenic patients. One-year relapse rates. Arch Gen Psychiatry 28:54–64PubMedGoogle Scholar
  21. 21.
    Hugenholtz GW, Heerdink ER, Stolker JJ, Meijer WE, Egberts AC, Nolen WA (2006) Haloperidol dose when used as active comparator in randomized controlled trials with atypical antipsychotics in schizophrenia: comparison with officially recommended doses. J Clin Psychiatry 67:897–903PubMedGoogle Scholar
  22. 22.
    Jones PB, Barnes TR, Davies L, Dunn G, Lloyd H, Hayhurst KP, Murray RM, Markwick A, Lewis SW (2006) Randomized controlled trial of the effect on quality of life of second- vs first-generation antipsychotic drugs in schizophrenia: cost utility of the Latest Antipsychotic Drugs in Schizophrenia Study (CUtLASS 1). Arch Gen Psychiatry 63:1079–1087PubMedCrossRefGoogle Scholar
  23. 23.
    Kane JM (2006) Commentary on the clinical antipsychotic trials of intervention effectiveness (CATIE). J Clin Psychiatry 67:831–832PubMedGoogle Scholar
  24. 24.
    Karow A, Naber D (2002) Subjective well-being and quality of life under atypical antipsychotic treatment. Psychopharmacology (Berl) 162:3–10CrossRefGoogle Scholar
  25. 25.
    Kasper S, Winkler D (2006) Addressing the limitations of the CATIE study. World J Biol Psychiatry 7:126–127PubMedCrossRefGoogle Scholar
  26. 26.
    Kupfer DJ, Sartorius N (2003) The usefulness and use of second-generation antipsychotic medication—an update. Curr Opin Psychiatry 16(suppl 1):S1–S44Google Scholar
  27. 27.
    Lambert M, Naber D (2004) Current issues in schizophrenia: overview of patient acceptability, functioning capacity and quality of life. CNS Drugs 18(suppl 2):5–17Google Scholar
  28. 28.
    Lavin MR, Rifkin A (1991) Prophylactic antiparkinson drug use: II. Withdrawal after long-term maintenance therapy. J Clin Pharmacol 31:769–777PubMedGoogle Scholar
  29. 29.
    Lehman AF, Kreyenbuhl J, Buchanan RW, Dickerson FB, Dixon LB, Goldberg R, Green-Paden LD, Tenhula WN, Boerescu D, Tek C et al (2004) The Schizophrenia Patient Outcomes Research Team (PORT): updated treatment recommendations 2003. Schizophr Bull 30:193–217PubMedGoogle Scholar
  30. 30.
    Lehman AF, Lieberman JA, Dixon LB, McGlashan TH, Miller AL, Perkins DO, Kreyenbuhl J (2004) Practice guideline for the treatment of patients with schizophrenia, 2nd edn. Am J Psychiatry 161:1–56PubMedCrossRefGoogle Scholar
  31. 31.
    Leucht S, Pitschel-Walz G, Abraham D, Kissling W (1999) Efficacy and extrapyramidal side-effects of the new antipsychotics olanzapine, quetiapine, risperidone, and sertindole compared to conventional antipsychotics and placebo. A meta-analysis of randomized controlled trials. Schizophr Res 35:51–68PubMedCrossRefGoogle Scholar
  32. 32.
    Lewis SW, Barnes TR, Davies L, Murray RM, Dunn G, Hayhurst KP, Markwick A, Lloyd H, Jones PB (2006) Randomized controlled trial of effect of prescription of clozapine versus other second-generation antipsychotic drugs in resistant schizophrenia. Schizophr Bull 32:715–723PubMedCrossRefGoogle Scholar
  33. 33.
    Lieberman JA (2006) Comparative effectiveness of antipsychotic drugs. A commentary on: Cost Utility Of The Latest Antipsychotic Drugs In Schizophrenia Study (CUtLASS 1) and Clinical Antipsychotic Trials Of Intervention Effectiveness (CATIE). Arch Gen Psychiatry 63:1069–1072PubMedCrossRefGoogle Scholar
  34. 34.
    Lieberman JA, Alvir J, Geisler S, Ramos-Lorenzi J, Woerner M, Novacenko H, Cooper T, Kane JM (1994) Methylphenidate response, psychopathology and tardive dyskinesia as predictors of relapse in schizophrenia. Neuropsychopharmacology 11:107–118PubMedGoogle Scholar
  35. 35.
    Lieberman JA, Stroup TS, McEvoy JP, Swartz MS, Rosenheck RA, Perkins DO, Keefe RS, Davis SM, Davis CE, Lebowitz BD et al (2005) Effectiveness of antipsychotic drugs in patients with chronic schizophrenia. N Engl J Med 353:1209–1223PubMedCrossRefGoogle Scholar
  36. 36.
    Lieberman JA, Tollefson G, Tohen M, Green AI, Gur RE, Kahn R, McEvoy J, Perkins D, Sharma T, Zipursky R et al (2003) Comparative efficacy and safety of atypical and conventional antipsychotic drugs in first-episode psychosis: a randomized, double-blind trial of olanzapine versus haloperidol. Am J Psychiatry 160:1396–1404PubMedCrossRefGoogle Scholar
  37. 37.
    Maier W, Moller HJ (2007) Meta-analyses: a tool for maximizing therapy study evidence?. Nervenarzt 78:1028–1036PubMedCrossRefGoogle Scholar
  38. 38.
    Manos N, Lavrentiadis G, Gkiouzepas J (1986) Evaluation of the need for prophylactic antiparkinsonian medication in psychotic patients treated with neuroleptics. J Clin Psychiatry 47:114–116PubMedGoogle Scholar
  39. 39.
    McCue RE, Waheed R, Urcuyo L, Orendain G, Joseph MD, Charles R, Hasan SM (2006) Comparative effectiveness of second-generation antipsychotics and haloperidol in acute schizophrenia. Br J Psychiatry 189: 433–440Google Scholar
  40. 40.
    McEvoy JP, Lieberman JA, Stroup TS, Davis SM, Meltzer HY, Rosenheck RA, Swartz MS, Perkins DO, Keefe RS, Davis CE et al (2006) Effectiveness of clozapine versus olanzapine, quetiapine, and risperidone in patients with chronic schizophrenia who did not respond to prior atypical antipsychotic treatment. Am J Psychiatry 163:600–610PubMedCrossRefGoogle Scholar
  41. 41.
    Meltzer HY, Bobo WV (2006) Interpreting the efficacy findings in the CATIE study: what clinicians should know. CNS Spectr 11:14–24PubMedGoogle Scholar
  42. 42.
    Miller DD, McEvoy JP, Davis SM, Caroff SN, Saltz BL, Chakos MH, Swartz MS, Keefe RS, Rosenheck RA, Stroup TS et al (2005) Clinical correlates of tardive dyskinesia in schizophrenia: baseline data from the CATIE schizophrenia trial. Schizophr Res 80:33–43PubMedCrossRefGoogle Scholar
  43. 43.
    Möller HJ (2000) Rating depressed patients: observer- vs self-assessment. Eur Psychiatry 15:160–172PubMedCrossRefGoogle Scholar
  44. 44.
    Möller HJ (2003) Amisulpride: limbic specificity and the mechanism of antipsychotic atypicality. Prog Neuropsychopharmacol Biol Psychiatry 27:1101–1111PubMedCrossRefGoogle Scholar
  45. 45.
    Möller HJ (2003) Plazebo-kontrollierte Studien zum Wirkungsnachweis von Antidepressiva sind notwendig! Psychopharmakotherapie 10:85–86Google Scholar
  46. 46.
    Möller HJ (2004) Sind placebokontrollierte Studien zum Wirksamkeitsbeweis von Antidepressiva notwendig? Nervenarzt 5:421–424CrossRefGoogle Scholar
  47. 47.
    Möller HJ (2005) Are the new antipsychotics no better than the classical neuroleptics? The problematic answer from the CATIE study. Eur Arch Psychiatry Clin Neurosci 255:371–372PubMedCrossRefGoogle Scholar
  48. 48.
    Möller HJ (2007) “Effectiveness”-Studien: methodische Mängel und negative Konsequenzen für die Versorgung. Psychopharmakotherapie 14:45–46Google Scholar
  49. 49.
    Möller HJ (2008) Standardised rating scales in psychiatry: methodological basis, possibilities, limitations and descriptions of important rating scales. World J Biol Psychiatry (in press)Google Scholar
  50. 50.
    Moller HJ, Maier W (2007) Problems of evidence-based medicine in psychopharmacotherapy: problems of evidence grading and of the evidence basis for complex clinical decision making. Nervenarzt 78:1014–1027CrossRefGoogle Scholar
  51. 51.
    Mudge MAC, Davey PJ, Coleman KA, Montgomery W, Croker VS, Mullen K, Castle DJ (2005) A comparison of olanzapine versus risperidone for the treatment of schizophrenia: a meta-analysis of randomised clinical trials. Int J Psychiatry Clin Prac 9:3–15CrossRefGoogle Scholar
  52. 52.
    Mulert C, Juckel G, Giegling I, Pogarell O, Leicht G, Karch S, Mavrogiorgou P, Moller HJ, Hegerl U, Rujescu D (2006) A Ser9Gly polymorphism in the dopamine D3 receptor gene (DRD3) and event-related P300 potentials. Neuropsychopharmacology 31:1335–1344PubMedGoogle Scholar
  53. 53.
    Murray RM, O’Callaghan E, Castle DJ, Lewis SW (1992) A neurodevelopmental approach to the classification of schizophrenia. Schizophr Bull 18:319–332PubMedGoogle Scholar
  54. 54.
    Murray RM, Van Os J (1998) Predictors of outcome in schizophrenia. J Clin Psychopharmacol 18:2S–4SPubMedCrossRefGoogle Scholar
  55. 55.
    Naber D (2005) Subjective effects of antipsychotic treatment. Acta Psychiatr Scand 111:81–83PubMedCrossRefGoogle Scholar
  56. 56.
    Oosthuizen P, Emsley R, Jadri TH, Keyter N (2004) A randomized, controlled comparison of the efficacy and tolerability of low and high doses of haloperidol in the treatment of first-episode psychosis. Int J Neuropsychopharmacol 7:125–131PubMedCrossRefGoogle Scholar
  57. 57.
    Oosthuizen PP, Emsley RA, Maritz JS, Turner JA, Keyter N (2003) Incidence of tardive dyskinesia in first-episode psychosis patients treated with low-dose haloperidol. J Clin Psychiatry 64:1075–1080PubMedCrossRefGoogle Scholar
  58. 58.
    Paykel ES, Norton KRW (1986) Self-report and clinical interview in the assessment of depression. In: Sartorius N, Ban TA (eds) Assessment of depression. Springer, New York, pp 356–366Google Scholar
  59. 59.
    Phillips GA, Van Brunt DL, Roychowdhury SM, Xu W, Naber D (2006) The relationship between quality of life and clinical efficacy from a randomized trial comparing olanzapine and ziprasidone. J Clin Psychiatry 67:1397–1403PubMedCrossRefGoogle Scholar
  60. 60.
    Pukrop R, Möller HJ, Steinmeyer EM (2000) Quality of life in psychiatry: a systematic contribution to construct validation and the development of the integrative assessment tool “modular system for quality of life”. Eur Arch Psychiatry Clin Neurosci 250:120–132PubMedCrossRefGoogle Scholar
  61. 61.
    Pukrop R, Schlaak V, Möller-Leimkühler AM, Albus M, Czernik A, Klosterkötter J, Möller HJ (2003) Reliability and validity of quality of life assessed by the Short-Form 36 and the modular system of quality of life in patients with schizophrenia and patients with depression. Psych Res 119:63–79CrossRefGoogle Scholar
  62. 62.
    Pyne JM, Sullivan G, Kaplan R, Williams DK (2003) Comparing the sensitivity of generic effectiveness measures with symptom improvement in persons with schizophrenia. Med Care 41:208–217PubMedCrossRefGoogle Scholar
  63. 63.
    Ragins M (2005) Should the CATIE study be a wake-up call? Psychiatr Serv 56:1489PubMedCrossRefGoogle Scholar
  64. 64.
    Rosenheck R, Perlick D, Bingham S, Liu-Mares W, Collins J, Warren S, Leslie D, Allan E, Campbell EC, Caroff S et al (2003) Effectiveness and cost of olanzapine and haloperidol in the treatment of schizophrenia: a randomized controlled trial. JAMA 290:2693–2702PubMedCrossRefGoogle Scholar
  65. 65.
    Rosenheck RA (2006) Outcomes, costs, and policy caution. A commentary on: the Cost Utility of the Latest Antipsychotic Drugs in Schizophrenia Study (CUtLASS 1). Arch Gen Psychiatry 63:1074–1076PubMedCrossRefGoogle Scholar
  66. 66.
    Schatzberg AF, Nemeroff CB (2000) The American psychiatric publishing textbook of psychopharmacology, 3rd edn. American Psychiatric Publishing, Washington DCGoogle Scholar
  67. 67.
    Soares BG, Fenton M, Chue P (2000) Sulpiride for schizophrenia. Cochrane Database Syst Rev: CD001162Google Scholar
  68. 68.
    Soares KV, McGrath JJ (2000) Anticholinergic medication for neuroleptic-induced tardive dyskinesia. Cochrane Database Syst Rev: CD000204Google Scholar
  69. 69.
    Sramek JJ, Simpson GM, Morrison RL, Heiser JF (1986) Anticholinergic agents for prophylaxis of neuroleptic-induced dystonic reactions: a prospective study. J Clin Psychiatry 47:305–309PubMedGoogle Scholar
  70. 70.
    Stern TA, Anderson WH (1979) Benztropine prophylaxis of dystonic reactions. Psychopharmacology (Berl) 61:261–262CrossRefGoogle Scholar
  71. 71.
    Stroup TS, Lieberman JA, McEvoy JP, Swartz MS, Davis SM, Rosenheck RA, Perkins DO, Keefe RS, Davis CE, Severe J et al (2006) Effectiveness of olanzapine, quetiapine, risperidone, and ziprasidone in patients with chronic schizophrenia following discontinuation of a previous atypical antipsychotic. Am J Psychiatry 163:611–622PubMedCrossRefGoogle Scholar
  72. 72.
    Stroup TS, McEvoy JP, Swartz MS, Byerly MJ, Glick ID, Canive JM, McGee MF, Simpson GM, Stevens MC, Lieberman JA (2003) The National Institute of Mental Health Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) project: schizophrenia trial design and protocol development. Schizophr Bull 29:15–31PubMedGoogle Scholar
  73. 73.
    Strous RD, Kupchik M, Roitman S, Schwartz S, Gonen N, Mester R, Weizman A, Spivak B (2006) Comparison between risperidone, olanzapine, and clozapine in the management of chronic schizophrenia: a naturalistic prospective 12-week observational study. Hum Psychopharmacol 21:235–243PubMedCrossRefGoogle Scholar
  74. 74.
    Tandon R, Möller HJ, Belmaker RH, Gattaz WF, Lopez-Ibor JJ, Okasha A, Burrows G, Stein DJ, Olie JP, Nutt D, et al. (2008) World Psychiatric Association Pharmacopsychiatry section statement on comparative effectiveness of antipsychotics in the treatment of schizophrenia. Schizophr Res (in press)Google Scholar
  75. 75.
    Tiihonen J, Walhbeck K, Lonnqvist J, Klaukka T, Ioannidis JP, Volavka J, Haukka J (2006) Effectiveness of antipsychotic treatments in a nationwide cohort of patients in community care after first hospitalisation due to schizophrenia and schizoaffective disorder: observational follow-up study. BMJ 333:224PubMedCrossRefGoogle Scholar
  76. 76.
    Tunis SL, Johnstone BM, Gibson PJ, Loosbrock DL, Dulisse BK (1999) Changes in perceived health and functioning as a cost-effectiveness measure for olanzapine versus haloperidol treatment of schizophrenia. J Clin Psychiatry 60 (suppl 19):38–45Google Scholar
  77. 77.
    Tunis SR, Stryer DB, Clancy CM (2003) Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA 290:1624–1632PubMedCrossRefGoogle Scholar
  78. 78.
    Ware JE, Jr., Sherbourne CD (1992) The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care 30:473–483PubMedCrossRefGoogle Scholar
  79. 79.
    Ware JE, Snow KK, Kosinski M, Gandek B (1993) SF-36 Health Survey Manual and Interpretation Guide. The Health Institute, New England Medical Center, BostonGoogle Scholar
  80. 80.
    Weiden PJ (2006) Switching antipsychotics: an updated review with a focus on quetiapine. J Psychopharmacol 20:104–118PubMedCrossRefGoogle Scholar
  81. 81.
    Weiden PJ (2007) Discontinuing and switching antipsychotic medications: understanding the CATIE schizophrenia trial. J Clin Psychiatry 68(suppl 1):12–9Google Scholar
  82. 82.
    Weiden PJ, Simpson GM, Potkin SG, O’Sullivan RL (2003) Effectiveness of switching to ziprasidone for stable but symptomatic outpatients with schizophrenia. J Clin Psychiatry 64:580–588PubMedGoogle Scholar

Copyright information

© Springer 2008

Authors and Affiliations

  1. 1.Department of PsychiatryLudwig-Maximilians-University MünchenMunichGermany

Personalised recommendations