Healthy User and Related Biases in Observational Studies of Preventive Interventions: A Primer for Physicians

Abstract

The current emphasis on comparative effectiveness research will provide practicing physicians with increasing volumes of observational evidence about preventive care. However, numerous highly publicized observational studies of the effect of prevention on health outcomes have reported exaggerated relationships that were later contradicted by randomized controlled trials. A growing body of research has identified sources of bias in observational studies that are related to patient behaviors or underlying patient characteristics, known as the healthy user effect, the healthy adherer effect, confounding by functional status or cognitive impairment, and confounding by selective prescribing. In this manuscript we briefly review observational studies of prevention that have appeared to reach incorrect conclusions. We then describe potential sources of bias in these studies and discuss study designs, analytical methods, and sensitivity analyses that may mitigate bias or increase confidence in the results reported. More careful consideration of these sources of bias and study designs by providers can enhance evidence-based decision-making.

This is a preview of subscription content, access via your institution.

References

  1. 1.

    Varas-Lorenzo C, Garcia-Rodriguez LA, Perez-Gutthann S, Duque-Oliart A. Hormone replacement therapy and incidence of acute myocardial infarction. A population-based nested case-control study. Circulation. 2000;101(22):2572–8.

    PubMed  CAS  Google Scholar 

  2. 2.

    Stampfer MJ, Willett WC, Colditz GA, Rosner B, Speizer FE, Hennekens CH. A prospective study of postmenopausal estrogen therapy and coronary heart disease. N Engl J Med. 1985;313(17):1044–9.

    PubMed  Article  CAS  Google Scholar 

  3. 3.

    Grady D, Rubin SM, Petitti DB, et al. Hormone therapy to prevent disease and prolong life in postmenopausal women. Ann Intern Med. 1992;117(12):1016–37.

    PubMed  CAS  Google Scholar 

  4. 4.

    Hulley S, Grady D, Bush T, et al. Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/progestin Replacement Study (HERS) Research Group. Jama. 1998;280(7):605–13.

    PubMed  Article  CAS  Google Scholar 

  5. 5.

    Rossouw JE, Anderson GL, Prentice RL, et al. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results From the Women's Health Initiative randomized controlled trial. Jama. 2002;288(3):321–33.

    PubMed  Article  CAS  Google Scholar 

  6. 6.

    Hennekens CH, Buring JE, Manson JE, et al. Lack of effect of long-term supplementation with beta carotene on the incidence of malignant neoplasms and cardiovascular disease. N Engl J Med. 1996;334(18):1145–9.

    PubMed  Article  CAS  Google Scholar 

  7. 7.

    Yusuf S, Dagenais G, Pogue J, Bosch J, Sleight P. Vitamin E supplementation and cardiovascular events in high-risk patients. The Heart Outcomes Prevention Evaluation Study Investigators. N Engl J Med. 2000;342(3):154–60.

    PubMed  Article  CAS  Google Scholar 

  8. 8.

    Virtamo J, Rapola JM, Ripatti S, et al. Effect of vitamin E and beta carotene on the incidence of primary nonfatal myocardial infarction and fatal coronary heart disease. Arch Intern Med. 1998;158(6):668–75.

    PubMed  Article  CAS  Google Scholar 

  9. 9.

    Egger M, Schneider M, Schneider M, Davey Smith G. Spurious precision? Meta-analysis of observational studies. Br Med J. 1998;316(7125):140–4.

    CAS  Google Scholar 

  10. 10.

    Toh S, Hernandez-Diaz S. Statins and fracture risk. A systematic review. Pharmacoepidemiol Drug Saf. 2007;16(6):627–40.

    PubMed  Article  CAS  Google Scholar 

  11. 11.

    Haley RW, Dietschy JM. Is there a connection between the concentration of cholesterol circulating in plasma and the rate of neuritic plaque formation in Alzheimer disease? Arch Neurol. 2000;57(10):1410–2.

    PubMed  Article  CAS  Google Scholar 

  12. 12.

    Majumdar SR, McAlister FA, Eurich DT, Padwal RS, Marrie TJ. Statins and outcomes in patients admitted to hospital with community acquired pneumonia: population based prospective cohort study. Br Med J. 2006;333(7576):999.

    Article  Google Scholar 

  13. 13.

    Setoguchi S, Glynn RJ, Avorn J, Mogun H, Schneeweiss S. Statins and the risk of lung, breast, and colorectal cancer in the elderly. Circulation. 2007;115(1):27–33.

    PubMed  Article  CAS  Google Scholar 

  14. 14.

    Jefferson T, Rivetti D, Rivetti A, Rudin M, Di Pietrantonj C, Demicheli V. Efficacy and effectiveness of influenza vaccines in elderly people: a systematic review. Lancet. 2005;366(9492):1165–74.

    PubMed  Article  CAS  Google Scholar 

  15. 15.

    Nichol KL, Nordin JD, Nelson DB, Mullooly JP, Hak E. Effectiveness of influenza vaccine in the community-dwelling elderly. N Engl J Med. 2007;357(14):1373–81.

    PubMed  Article  CAS  Google Scholar 

  16. 16.

    Jackson LA, Jackson ML, Nelson JC, Neuzil KM, Weiss NS. Evidence of bias in estimates of influenza vaccine effectiveness in seniors. Int J Epidemiol. 2006;35(2):337–44.

    PubMed  Article  Google Scholar 

  17. 17.

    Jackson LA, Nelson JC, Benson P, et al. Functional status is a confounder of the association of influenza vaccine and risk of all cause mortality in seniors. Int J Epidemiol. 2006;35(2):345–52.

    PubMed  Article  Google Scholar 

  18. 18.

    Matthews KA, Kuller LH, Wing RR, Meilahn EN, Plantinga P. Prior to use of estrogen replacement therapy, are users healthier than nonusers? Am J Epidemiol. 1996 May 15;143(10):971-8

  19. 19.

    Brett KM, Madans JH. Use of postmenopausal hormone replacement therapy: estimates from a nationally representative cohort study. Am J Epidemiol. 1997;145(6):536–45.

    PubMed  CAS  Google Scholar 

  20. 20.

    Barrett-Connor E. Postmenopausal estrogen and prevention bias. Ann Intern Med. 1991;115(6):455–6.

    PubMed  CAS  Google Scholar 

  21. 21.

    Posthuma WF, Westendorp RG, Vandenbroucke JP. Cardioprotective effect of hormone replacement therapy in postmenopausal women: is the evidence biased? Br Med J. 1994;308(6939):1268–9.

    CAS  Google Scholar 

  22. 22.

    Curtis JR, Xi J, Westfall AO, et al. Improving the prediction of medication compliance: the example of bisphosphonates for osteoporosis. Med Care. 2009;47(3):334–41.

    PubMed  Article  Google Scholar 

  23. 23.

    Brookhart MA, Patrick AR, Dormuth C, et al. Adherence to lipid-lowering therapy and the use of preventive health services: an investigation of the healthy user effect. Am J Epidemiol. 2007;166(3):348–54.

    PubMed  Article  Google Scholar 

  24. 24.

    Dormuth CR, Patrick AR, Shrank WH, et al. Statin adherence and risk of accidents: a cautionary tale. Circulation. 2009;119(15):2051–7.

    PubMed  Article  CAS  Google Scholar 

  25. 25.

    Granger BB, Swedberg K, Ekman I, et al. Adherence to candesartan and placebo and outcomes in chronic heart failure in the CHARM programme: double-blind, randomised, controlled clinical trial. Lancet. 2005;366(9502):2005–11.

    PubMed  Article  CAS  Google Scholar 

  26. 26.

    Simpson SH, Eurich DT, Majumdar SR, et al. A meta-analysis of the association between adherence to drug therapy and mortality. Br Med J. 2006;333(7557):15.

    Article  Google Scholar 

  27. 27.

    Chan L, Doctor JN, MacLehose RF, et al. Do Medicare patients with disabilities receive preventive services? A population-based study. Arch Phys Med Rehabil. 1999;80(6):642–6.

    PubMed  Article  CAS  Google Scholar 

  28. 28.

    Glynn RJ, Knight EL, Levin R, Avorn J. Paradoxical relations of drug treatment with mortality in older persons. Epidemiology. 2001;12(6):682–9.

    PubMed  Article  CAS  Google Scholar 

  29. 29.

    Glynn RJ, Schneeweiss S, Wang PS, Levin R, Avorn J. Selective prescribing led to overestimation of the benefits of lipid-lowering drugs. J Clin Epidemiol. 2006;59(8):819–28.

    PubMed  Article  Google Scholar 

  30. 30.

    Bratzler DW, Houck PM, Jiang H, et al. Failure to vaccinate Medicare inpatients: a missed opportunity. Arch Intern Med. 2002;162(20):2349–56.

    PubMed  Article  Google Scholar 

  31. 31.

    Fedson DS, Wajda A, et al. Disparity between influenza vaccination rates and risks for influenza-associated hospital discharge and death in Manitoba in 1982–1983. Ann Intern Med. 1992;116(7):550–5.

    PubMed  CAS  Google Scholar 

  32. 32.

    Schneeweiss S, Seeger JD, Maclure M, Wang PS, Avorn J, Glynn RJ. Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data. Am J Epidemiol. 2001;154(9):854–64.

    PubMed  Article  CAS  Google Scholar 

  33. 33.

    Schneeweiss S, Rassen JA, Glynn RJ, Avorn J, Mogun H, Brookhart MA. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology. 2009;20(4):512–22.

    PubMed  Article  Google Scholar 

  34. 34.

    Rasmussen JN, Chong A, Alter DA. Relationship between adherence to evidence-based pharmacotherapy and long-term mortality after acute myocardial infarction. Jama. 2007;297(2):177–86.

    PubMed  Article  CAS  Google Scholar 

  35. 35.

    Schneeweiss S, Seeger JD, Landon J, Walker AM. Aprotinin during coronary-artery bypass grafting and risk of death. N Engl J Med. 2008;358(8):771–83.

    PubMed  Article  CAS  Google Scholar 

  36. 36.

    Brookhart MA, Wang PS, Solomon DH, Schneeweiss S. Evaluating short-term drug effects using a physician-specific prescribing preference as an instrumental variable. Epidemiology. 2006;17(3):268–75.

    PubMed  Article  Google Scholar 

  37. 37.

    Simonsen L, Reichert TA, Viboud C, Blackwelder WC, Taylor RJ, Miller MA. Impact of influenza vaccination on seasonal mortality in the US elderly population. Arch Intern Med. 2005;165(3):265–72.

    PubMed  Article  Google Scholar 

  38. 38.

    Ray WA. Evaluating medication effects outside of clinical trials: new-user designs. Am J Epidemiol. 2003;158(9):915–20.

    PubMed  Article  Google Scholar 

Download references

Acknowledgments

This work is supported by a career development award from the National Heart, Lung and Blood Institute (K23HL090505-01) for Dr. Shrank. Dr. Brookhart is supported by a career development award from the National Institute of Health (AG-027400).

Conflict of Interest

None disclosed.

Author information

Affiliations

Authors

Corresponding author

Correspondence to William H. Shrank MD, MSHS.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Shrank, W.H., Patrick, A.R. & Alan Brookhart, M. Healthy User and Related Biases in Observational Studies of Preventive Interventions: A Primer for Physicians. J GEN INTERN MED 26, 546–550 (2011). https://doi.org/10.1007/s11606-010-1609-1

Download citation

Keywords

  • Statin
  • Hormone Replacement Therapy
  • Influenza Vaccination
  • Preventive Therapy
  • Comparative Effectiveness Research