Challenges to evidence-based medicine:
- 169 Downloads
The practice of evidence-based medicine depends on the availability of clinically relevant research, yet questions have been raised about the generalizability of findings from randomized controlled trials (RCTs).
The aim of this study was to quantify differences between RCT patients and treatments and those in day-to-day clinical practice.
Data from published reports of two key RCTs underlying recent treatment advances in psychiatry were compared with data on routine psychiatric practice collected through a Practice Research Network (PRN).
Hospital inpatient units (RCT) and the full range of psychiatric practice settings in the United States (PRN).
Adults with bipolar I disorder and adults with schizophrenia.
Demographic (age, gender, race), clinical (principal diagnoses, comorbid conditions, psychosocial functioning, and histories of hospitalization), and treatment (medication name and dosage) characteristics.
PRN patients had more comorbid conditions and were more likely to be white, female, and older than RCT patients. In all, 38% of PRN patients with schizophrenia and 55% of PRN patients with bipolar I disorder would have been ineligible for the corresponding RCT. Most PRN patients receiving an RCT study medication were also receiving other medications not allowed by the RCT protocol.
Findings support the assertion that RCT patients and treatments are not typical of those in clinical practice, and most patients in clinical practice are receiving treatments that do not have direct empirical support. Research is needed to determine the extent to which RCT findings should be used to guide routine clinical decisions.
Key wordsevidence-based medicine randomized controlled trials generalizability schizophrenia bipolar disorder
Unable to display preview. Download preview PDF.
- 2.Sackett DL, Haynes RB,Guyatt GH,Tugwell P (1991) Clinical Epidemiology: A Basic Science for Clinical Medicine. Boston: Little, Brown and Company, p 281Google Scholar
- 4.Zarin DA, West JC, Pincus HA, McIntyre JS (1995) The American Psychiatric Association Practice Research Network. In: Sederer L, Dickey B, eds. Outcomes Assessment in Clinical Practice. Baltimore, MD: William and Wilkins, pp 146–155Google Scholar
- 5.Begg CB (1987) Cancer clinical trials in the USA: Patient eligibility, generalizability of results and technology transfer. Bull Cancer 74:197–203Google Scholar
- 6.Chlebowski RT, Butler J, Nelson A, Lillington L (1993) Breast cancer chemoprevention. Tamoxifen: current issues and future prospective. Cancer 72(Suppl 3):1032–1037Google Scholar
- 7.Tannock IF (1992) Assessment of study design in clinical trials for bladder cancer. Urol Clin North Am 19:655–662Google Scholar
- 8.Lee KL, Califf RM, Simes J, Van de Werf F, Topol EJ (1994) Holding GUSTO up to the light. Global Utilization of Streptokinase and Tissue Plasminogen Activator for Occluded Coronary Arteries. Ann Intern Med 120:876–881Google Scholar
- 10.Egglin TK, Horowitz RI (1996) The case for better research standards in peripheral thrombolysis: poor quality of randomized trials during the past decade. Acad Radiol 3:1–9Google Scholar
- 12.Saver JL (1995) Coping with an embarrassment of riches. How stroke centers may participate in multiple, concurrent clinical stroke trials. Stroke 26:1289–1292Google Scholar
- 13.Schneider LS, Olin JT, Lyness SA, Chui HC (1997) Eligibility of Alzheimer’s disease clinic patients for clinical trials. J Am Geriatr Soc 45:923–928Google Scholar
- 14.Cunningham WE, Bozzette SA, Hays RD, Kanouse DE, Shapiro MF (1995) Comparison of health-related quality of life in clinical trials and nonclinical trial human immunodeficiency virus-infected cohorts. Med Care 33(Suppl 4):AS15–AS25Google Scholar
- 17.Licht RW, Gouliaev G,Vestergaard P, Frydenberg M (1997) Generalisability of results from randomised drug trials. Br J Psychiatry 170:264–267Google Scholar
- 19.Miller CA, Hooper CL, Bakish D (1997) A comparison of patients with major depressive disorder recruited through newspaper advertising versus consultation referrals for clinical drug trials. Psychopharmacol Bull 33:69–73Google Scholar
- 20.Amori G, Lenox RH (1989) Do volunteer subjects bias clinical trials? J Clin Psychopharmacol 9:321–327Google Scholar
- 21.Taylor KM, Feldstein ML, Skeel RT, Pandya KJ, Ng P, Carbone PP (1994) Fundamental dilemmas of the randomized clinical trial process: results of a survey of the 1,737 Eastern Cooperative Oncology Group Investigators. Clin Oncol 12:1796–1805Google Scholar
- 22.Antman K, Amato D, Wood W, Carson J, Suit H, Proppe K, Carey R, Greenberger J, Wilson R, Frei E 3rd (1985) Selection bias in clinical trials. Clin Oncol 3:1142–1147Google Scholar
- 23.Skerritt U, Pitt B, Armstrong S, O’Brien A (1996) Recruiting patients for drug trials: a difficult task. Psychiatr Bull 20:708–710Google Scholar
- 25.Smith P, Arnesen H (1998) Non-respondents in a post-myocardial infarction trial: characteristics and reasons for refusal. Acta Med Scand 223:537–542Google Scholar
- 26.Torgerson DJ, Sibbald B (1998) Understanding controlled trials: what is a patient preference trial? BMJ 316:360Google Scholar
- 28.Liberati A, Himel HN, Chalmers TC (1986) A quality assessment of randomized control trials of primary treatment of breast cancer. J Clin Oncol 4:942–951Google Scholar
- 32.Scott JE, Dixon LB (1995) Psychological interventions for schizophrenia. Schizophr Bull 21:621–630Google Scholar
- 33.May WW (1998) Findings from Project MATCH: fact or artifact? Behav Health Management 18:38–39Google Scholar
- 34.Fortmann SP, Killen JD (1994) Who shall quit? Comparison of volunteer and population-based recruitment in two minimalcontact smoking cessation studies. Am J Epidemiol 140:39–51Google Scholar
- 35.Rapaport MH, Frevert T, Babior S, Zisook S, Judd LL (1995) A comparison of demographic variables, symptom profiles, and measurements of functioning in symptomatic volunteers and an outpatient clinical population. Psychopharmacol Bull 31:111–114Google Scholar
- 37.Essock SM, Hargreaves WA, Covell NH, Goethe J (1996) Clozapine’s effectiveness for patients in state hospitals: results from a randomized trial. Psychopharmacol Bull 32(4):683–697Google Scholar
- 38.Zarin DA, Pincus HA, West JC, McIntyre JS (1997) Practice-based research in psychiatry. Am J Psychiatry 154:1199–1208Google Scholar
- 39.American Psychiatric Association (1994) Practice guideline for the treatment of patients with bipolar disorder.Am J Psychiatry 151(Suppl):1–36 Google Scholar
- 40.American Psychiatric Association (1997) Practice guideline for the treatment of patients with schizophrenia. Am J Psychiatry 154(Suppl):1–63Google Scholar
- 41.Zarin DA, Seigle L, Pincus HA, MyIntyre JS (1997) Evidence-based practice guidelines. Psychopharmacol Bull 33:641–646Google Scholar
- 43.Pope HG Jr, McElroy SL, Keck PE Jr, Hudson JI (1991) Valproate in the treatment of acute mania: a placebo controlled study. Arch Gen Psychiatry 48:62–68Google Scholar
- 45.Zarin DA, Pincus HA, Peterson BD, West JC, Suarez AP, Marcus SC, McIntyre JS (1998) Characterizing psychiatry with findings from the 1996 National Survey of Psychiatric Practice. Am J Psychiatry 155:397–404Google Scholar
- 48.USP DI (1998) Volume I: Drug Information for the Health Care Professional. 18th edition. Rockville, MD: The United States Pharmacopeial Convention, IncGoogle Scholar
- 49.American Psychiatric Association (1994) Diagnostic and Statistical Manual of Mental Disorders, 4th ed. Washington, D. C.: American Psychiatric AssociationGoogle Scholar
- 50.Shah BV, Barnwell BG, Bieler GS (1997) SUDAAN User’s Manual, Release 7.5. Research Triangle Park, NC: Research Triangle InstituteGoogle Scholar
- 51.Olfson M, Klerman GL, Pincus HA (1993) The roles of psychiatrists in organized outpatient mental health settings. Am J Psychiatry 150:625–631Google Scholar
- 52.Lehman AF, Thompson JW, Dixon LB, Scott JE (1995) Schizophrenia: treatment outcomes research—editor’s introduction. Schizophr Bull 21:561–566Google Scholar