The current emphasis on comparative effectiveness research will provide practicing physicians with increasing volumes of observational evidence about preventive care. However, numerous highly publicized observational studies of the effect of prevention on health outcomes have reported exaggerated relationships that were later contradicted by randomized controlled trials. A growing body of research has identified sources of bias in observational studies that are related to patient behaviors or underlying patient characteristics, known as the healthy user effect, the healthy adherer effect, confounding by functional status or cognitive impairment, and confounding by selective prescribing. In this manuscript we briefly review observational studies of prevention that have appeared to reach incorrect conclusions. We then describe potential sources of bias in these studies and discuss study designs, analytical methods, and sensitivity analyses that may mitigate bias or increase confidence in the results reported. More careful consideration of these sources of bias and study designs by providers can enhance evidence-based decision-making.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Varas-Lorenzo C, Garcia-Rodriguez LA, Perez-Gutthann S, Duque-Oliart A. Hormone replacement therapy and incidence of acute myocardial infarction. A population-based nested case-control study. Circulation. 2000;101(22):2572–8.
Stampfer MJ, Willett WC, Colditz GA, Rosner B, Speizer FE, Hennekens CH. A prospective study of postmenopausal estrogen therapy and coronary heart disease. N Engl J Med. 1985;313(17):1044–9.
Grady D, Rubin SM, Petitti DB, et al. Hormone therapy to prevent disease and prolong life in postmenopausal women. Ann Intern Med. 1992;117(12):1016–37.
Hulley S, Grady D, Bush T, et al. Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/progestin Replacement Study (HERS) Research Group. Jama. 1998;280(7):605–13.
Rossouw JE, Anderson GL, Prentice RL, et al. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results From the Women's Health Initiative randomized controlled trial. Jama. 2002;288(3):321–33.
Hennekens CH, Buring JE, Manson JE, et al. Lack of effect of long-term supplementation with beta carotene on the incidence of malignant neoplasms and cardiovascular disease. N Engl J Med. 1996;334(18):1145–9.
Yusuf S, Dagenais G, Pogue J, Bosch J, Sleight P. Vitamin E supplementation and cardiovascular events in high-risk patients. The Heart Outcomes Prevention Evaluation Study Investigators. N Engl J Med. 2000;342(3):154–60.
Virtamo J, Rapola JM, Ripatti S, et al. Effect of vitamin E and beta carotene on the incidence of primary nonfatal myocardial infarction and fatal coronary heart disease. Arch Intern Med. 1998;158(6):668–75.
Egger M, Schneider M, Schneider M, Davey Smith G. Spurious precision? Meta-analysis of observational studies. Br Med J. 1998;316(7125):140–4.
Toh S, Hernandez-Diaz S. Statins and fracture risk. A systematic review. Pharmacoepidemiol Drug Saf. 2007;16(6):627–40.
Haley RW, Dietschy JM. Is there a connection between the concentration of cholesterol circulating in plasma and the rate of neuritic plaque formation in Alzheimer disease? Arch Neurol. 2000;57(10):1410–2.
Majumdar SR, McAlister FA, Eurich DT, Padwal RS, Marrie TJ. Statins and outcomes in patients admitted to hospital with community acquired pneumonia: population based prospective cohort study. Br Med J. 2006;333(7576):999.
Setoguchi S, Glynn RJ, Avorn J, Mogun H, Schneeweiss S. Statins and the risk of lung, breast, and colorectal cancer in the elderly. Circulation. 2007;115(1):27–33.
Jefferson T, Rivetti D, Rivetti A, Rudin M, Di Pietrantonj C, Demicheli V. Efficacy and effectiveness of influenza vaccines in elderly people: a systematic review. Lancet. 2005;366(9492):1165–74.
Nichol KL, Nordin JD, Nelson DB, Mullooly JP, Hak E. Effectiveness of influenza vaccine in the community-dwelling elderly. N Engl J Med. 2007;357(14):1373–81.
Jackson LA, Jackson ML, Nelson JC, Neuzil KM, Weiss NS. Evidence of bias in estimates of influenza vaccine effectiveness in seniors. Int J Epidemiol. 2006;35(2):337–44.
Jackson LA, Nelson JC, Benson P, et al. Functional status is a confounder of the association of influenza vaccine and risk of all cause mortality in seniors. Int J Epidemiol. 2006;35(2):345–52.
Matthews KA, Kuller LH, Wing RR, Meilahn EN, Plantinga P. Prior to use of estrogen replacement therapy, are users healthier than nonusers? Am J Epidemiol. 1996 May 15;143(10):971-8
Brett KM, Madans JH. Use of postmenopausal hormone replacement therapy: estimates from a nationally representative cohort study. Am J Epidemiol. 1997;145(6):536–45.
Barrett-Connor E. Postmenopausal estrogen and prevention bias. Ann Intern Med. 1991;115(6):455–6.
Posthuma WF, Westendorp RG, Vandenbroucke JP. Cardioprotective effect of hormone replacement therapy in postmenopausal women: is the evidence biased? Br Med J. 1994;308(6939):1268–9.
Curtis JR, Xi J, Westfall AO, et al. Improving the prediction of medication compliance: the example of bisphosphonates for osteoporosis. Med Care. 2009;47(3):334–41.
Brookhart MA, Patrick AR, Dormuth C, et al. Adherence to lipid-lowering therapy and the use of preventive health services: an investigation of the healthy user effect. Am J Epidemiol. 2007;166(3):348–54.
Dormuth CR, Patrick AR, Shrank WH, et al. Statin adherence and risk of accidents: a cautionary tale. Circulation. 2009;119(15):2051–7.
Granger BB, Swedberg K, Ekman I, et al. Adherence to candesartan and placebo and outcomes in chronic heart failure in the CHARM programme: double-blind, randomised, controlled clinical trial. Lancet. 2005;366(9502):2005–11.
Simpson SH, Eurich DT, Majumdar SR, et al. A meta-analysis of the association between adherence to drug therapy and mortality. Br Med J. 2006;333(7557):15.
Chan L, Doctor JN, MacLehose RF, et al. Do Medicare patients with disabilities receive preventive services? A population-based study. Arch Phys Med Rehabil. 1999;80(6):642–6.
Glynn RJ, Knight EL, Levin R, Avorn J. Paradoxical relations of drug treatment with mortality in older persons. Epidemiology. 2001;12(6):682–9.
Glynn RJ, Schneeweiss S, Wang PS, Levin R, Avorn J. Selective prescribing led to overestimation of the benefits of lipid-lowering drugs. J Clin Epidemiol. 2006;59(8):819–28.
Bratzler DW, Houck PM, Jiang H, et al. Failure to vaccinate Medicare inpatients: a missed opportunity. Arch Intern Med. 2002;162(20):2349–56.
Fedson DS, Wajda A, et al. Disparity between influenza vaccination rates and risks for influenza-associated hospital discharge and death in Manitoba in 1982–1983. Ann Intern Med. 1992;116(7):550–5.
Schneeweiss S, Seeger JD, Maclure M, Wang PS, Avorn J, Glynn RJ. Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data. Am J Epidemiol. 2001;154(9):854–64.
Schneeweiss S, Rassen JA, Glynn RJ, Avorn J, Mogun H, Brookhart MA. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology. 2009;20(4):512–22.
Rasmussen JN, Chong A, Alter DA. Relationship between adherence to evidence-based pharmacotherapy and long-term mortality after acute myocardial infarction. Jama. 2007;297(2):177–86.
Schneeweiss S, Seeger JD, Landon J, Walker AM. Aprotinin during coronary-artery bypass grafting and risk of death. N Engl J Med. 2008;358(8):771–83.
Brookhart MA, Wang PS, Solomon DH, Schneeweiss S. Evaluating short-term drug effects using a physician-specific prescribing preference as an instrumental variable. Epidemiology. 2006;17(3):268–75.
Simonsen L, Reichert TA, Viboud C, Blackwelder WC, Taylor RJ, Miller MA. Impact of influenza vaccination on seasonal mortality in the US elderly population. Arch Intern Med. 2005;165(3):265–72.
Ray WA. Evaluating medication effects outside of clinical trials: new-user designs. Am J Epidemiol. 2003;158(9):915–20.
This work is supported by a career development award from the National Heart, Lung and Blood Institute (K23HL090505-01) for Dr. Shrank. Dr. Brookhart is supported by a career development award from the National Institute of Health (AG-027400).
Conflict of Interest
About this article
Cite this article
Shrank, W.H., Patrick, A.R. & Alan Brookhart, M. Healthy User and Related Biases in Observational Studies of Preventive Interventions: A Primer for Physicians. J GEN INTERN MED 26, 546–550 (2011). https://doi.org/10.1007/s11606-010-1609-1
- Hormone Replacement Therapy
- Influenza Vaccination
- Preventive Therapy
- Comparative Effectiveness Research