Generalizing from the results of randomized studies of treatment: Can non-randomized studies be of help?

  • Noel S. WeissEmail author

Different patients who have the same illness may receive different forms of treatment. By documenting their treatment and the progression and complications of their illness, an attempt can be made to draw inferences regarding the relative impact of these different treatments. However, the interpretation of such comparisons can be compromised by underlying differences in the likelihood of the various outcome events among the patient treatment groups. Differences of opinion exist with respect to the frequency and magnitude of the confounding that may arise from these underlying differences. Benson and Hartz [1] concluded that “misuse of observational studies [of the efficacy of therapy] does not often occur in the recent literature”, and that nonrandomized studies of therapy “usually do provide valid information.” On the other hand, Pocock and Elbourne [2] state that greater reliance on the results of nonrandomized studies of therapeutic efficacy could lead to “considerable dangers to...



  1. 1.
    Benson K, Hartz AJ. A comparison of observational studies and randomized controlled trials. N Engl J Med. 2000;342:1878–86.CrossRefGoogle Scholar
  2. 2.
    Pocock SJ, Elbourne DR. Randomized trials or observational tribulations? N Engl J Med. 2000;342:1907–9.CrossRefGoogle Scholar
  3. 3.
    Goldberg IJ. To drink or not to drink? N Engl J Med. 2003;348:163–4.CrossRefGoogle Scholar
  4. 4.
    Weiss NS, Clinical epidemiology: the study of the occurrence of illness. Oxford, New York; 2006.Google Scholar
  5. 5.
    Elting LS. Author reply. Cancer. 2007;109:342.CrossRefGoogle Scholar
  6. 6.
    Weiss NS, Koepsell TD, Psaty BM. Generalizability of the results of randomized trials. Arch Intern Med. 2008;168:133–5.CrossRefGoogle Scholar
  7. 7.
    Pitt B, Zannad F, Remme WJ, et al. The effect of spironolactone on morbidity and mortality in patients with severe heart failure. N Engl J Med. 1999;341:709–17.CrossRefGoogle Scholar
  8. 8.
    Juurlink DN, Mamdani M, Kopp A, et al. Drug–drug interactions among elderly patients hospitalized for drug toxicity. JAMA. 2003;289:1652–8.CrossRefGoogle Scholar
  9. 9.
    Grant RM, Lama JR, Anderson PI, et al. Pre-exposure prophylaxis for HIV prevention in men who have sex with men. N Engl J Med. 2010;363:2587–99.CrossRefGoogle Scholar
  10. 10.
    McCormack S, Dunn DT, Sesai M, et al. Pre-exposure prophylaxis to prevent the acquisition of HIV-1 infection (PROUD): effectiveness results from the pilot phase of a pragmatic open-label randomized trial. Lancet. 2016;387:53–60.CrossRefGoogle Scholar
  11. 11.
    Elting LS, Cooksley C, Bekele BN, et al. Generalizability of cancer clinical trial results: prognostic differences between participants and nonparticipants. Cancer. 2006;106:2452–8.CrossRefGoogle Scholar
  12. 12.
    Van de Water W, Kiderlen M, Bastiaannet F, et al. External validity of a trial comprised of elderly patients with hormone receptor-positive breast cancer. JNCI. 2014;106:1–8.CrossRefGoogle Scholar
  13. 13.
    Atrial Fibrillation Investigators. Risk factors for stroke and efficacy of antithrombotic therapy in atrial fibrillation: analysis of pooled data from five randomized controlled trials. Arch Intern Med. 1994;154:1449–57.CrossRefGoogle Scholar
  14. 14.
    Stern S, Altkorn D, Levinson W. Anticoagulation for chronic atrial fibrillation. JAMA. 2000;283:2901–3.CrossRefGoogle Scholar
  15. 15.
    Horwitz RI, Singer BH. Why evidence-based medicine failed in patient care and medicine-based evidence will succeed. J Clin Epidemiol. 2017;84:14–7.CrossRefGoogle Scholar
  16. 16.
    Lauer MS, Collins FS. Using science to improve the nation’s health system: NIH’s commitment to comparative effectiveness research. JAMA. 2010;303:2182–3.CrossRefGoogle Scholar
  17. 17.
    Goldman B. Treatments that work for people just like you. Winter: Stanford Medicine; 2016. p. 32.Google Scholar
  18. 18.
    Weiss NS. The new world of data linkages in clinical epidemiology: Are we being brave or foolhardy? Epidemiology. 2011;22:292–4.CrossRefGoogle Scholar
  19. 19.
    Baxter NN, Goldwasser MA, Paszat LF, et al. Association of colonoscopy and death from colorectal cancer. Ann Intern Med. 2009;150:1–8.CrossRefGoogle Scholar
  20. 20.
    Weiss NS, Dhillon PK, Etzioni R. Case-control studies of the efficacy of cancer screening: overcoming bias from non-random patterns of screening. Epidemiology. 2004;15:409–13.CrossRefGoogle Scholar
  21. 21.
    Ray WA, Griffin MR, Schaffner W, et al. Psychotropic drug use and risk of fracture. N Engl Med. 1987;316:363–9.CrossRefGoogle Scholar
  22. 22.
    Gerstman BB, Friesman JP, Hine LK. Use of subsequent anticoagulants to increase the predictive value of Medicaid deep venous thromboembolism diagnosis. Epidemiology. 1990;1:122–7.CrossRefGoogle Scholar
  23. 23.
    Jackson LA, Jackson ML, Nelson JC, et al. Evidence of bias in estimates of influenza vaccine effectiveness in seniors. Int J Epidemiol. 2006;35:337–44.CrossRefGoogle Scholar
  24. 24.
    Pignon JP, Tribodet H, Scagliotti GV, et al. Lung adjuvant cisplatin evaluation: a pooled analysis by the LACE Collaborative Group. J Clin Oncol. 2008;26:3552–9.CrossRefGoogle Scholar
  25. 25.
    Ganti AK, Williams CD, Gajra A, et al. Effect of age on the efficacy of adjuvant chemotherapy for resected non-small cell lung cancer. Cancer. 2015;121:2578–85.CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.University of WashingtonSeattleUSA

Personalised recommendations