Journal of General Internal Medicine

, Volume 32, Issue 7, pp 790–795 | Cite as

Using Active Choice Within the Electronic Health Record to Increase Influenza Vaccination Rates

  • Mitesh S. Patel
  • Kevin G. Volpp
  • Dylan S. Small
  • Craig Wynne
  • Jingsan Zhu
  • Lin Yang
  • Steven HoneywellJr.
  • Susan C. Day
Original Research



Despite the benefits of influenza vaccination, each year more than half of adults in the United States do not receive it.


To evaluate the association between an active choice intervention in the electronic health record (EHR) and changes in influenza vaccination rates.


Observational study.


Adults eligible for influenza vaccination with a clinic visit at one of three internal medicine practices at the University of Pennsylvania Health System between September 2010 and March 2013.


The EHR confirmed patient eligibility during the clinic visit and, upon accessing the patient chart, prompted the physician and their medical assistant to actively choose to “accept” or “cancel” an order for the influenza vaccine.

Main Measures

Change in influenza vaccination order rates at the intervention practice compared to two control practices for the 2012–2013 flu season, comparing trends during the prior two flu seasons adjusting for time trends and patient and clinic visit characteristics.

Key Results

The sample (n = 45,926 patients) was 62.9% female, 35.9% white, and 54.4% black, with a mean age of 50.2 years. Trends were similar between practices during the 2 years in the pre-intervention period. Vaccination rates increased in both groups in the post-intervention year, but the intervention practice using active choice had a significantly greater increase than the control (adjusted difference-in-difference: 6.6 percentage points; 95% CI, 5.1–8.1; P < 0.001), representing a 37.3% relative increase compared to the pre-intervention period. More than 99.9% (9938/9941) of orders placed during the study period resulted in vaccination.


Active choice through the EHR was associated with a significant increase in influenza vaccination rates.


active choice choice architecture nudge physician behavior behavioral economics electronic health record influenza vaccination 



This study was funded by a grant from the Leonard Davis Institute of Health Economics at the University of Pennsylvania to Dr. Patel and a grant from the National Institute on Aging (P30AG034546) through the LDI Center for Health Incentives and Behavioral Economics to Dr. Volpp. Dr. Patel had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Compliance with Ethical Standards

Conflict of Interest

Dr. Patel is a principal at Catalyst Health, a behavior change and technology consulting firm. Dr. Volpp is a principal at VAL Health, a behavioral economics consulting firm. Dr. Volpp has also received consulting income from CVS Caremark and research funding from Humana, CVS Caremark, Discovery (South Africa), Hawaii Medical Services Association, and Merck, none of which is related to the work described in this manuscript. All other authors declare no conflict of interest.


  1. 1.
    Thompson WW, Shay DK, Weintraub E, Brammer L, Bridges CB, Cox NJ, Fukuda K. Influenza-associated hospitalizations in the United States. JAMA. 2004;292(11):1333–40.CrossRefPubMedGoogle Scholar
  2. 2.
    Disease Burden of Influenza. Available online at: Accessed September 15, 2016.
  3. 3.
    Bridges CB, Thompson WW, Meltzer MI, Reeve GR, Talamonti WJ, Cox NJ, Lilac HA, Hall H, Klimov A, Fukuda K. Effectiveness and cost-benefit of influenza vaccination of healthy working adults: a randomized controlled trial. JAMA. 2000;284(13):1655–63.CrossRefPubMedGoogle Scholar
  4. 4.
    Nichol KL, Wuorenma J, von Sternberg T. Benefits of influenza vaccination for low-, intermediate-, and high-risk senior citizens. Arch Intern Med. 1998;158(16):1769–76.CrossRefPubMedGoogle Scholar
  5. 5.
    FluVaxView. Centers for Disease Control and Prevention. Available online at: Accessed September 15, 2016.
  6. 6.
    Fewer Than Half of Americans Report Having Gotten A Flu Vaccine this Season. Centers for Disease Control and Prevention. Available online at: Accessed September 15, 2016.
  7. 7.
    Use and Characteristics of Electronic Health Record Systems Among Office-based Physician Practices: United States, 2001–2013. National Center for Health Statistics. Centers for Disease Control and Prevention. Available online at: Accessed September 15, 2016.
  8. 8.
    Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998;280(15):1339–46.CrossRefPubMedGoogle Scholar
  9. 9.
    Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, Sam J, Haynes RB. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223–38.CrossRefPubMedGoogle Scholar
  10. 10.
    Souza NM, Sebaldt RJ, Mackay JA, Prorok JC, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB, CCDSS Systematic Review Team. Computerized clinical decision support systems for primary preventive care: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci. 2011;6:87.CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, Samsa G, Hasselblad V, Williams JW, Musty MD, Wing L, Kendrick AS, Sanders GD, Lobach D. Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012;157(1):29–43.CrossRefPubMedGoogle Scholar
  12. 12.
    Patel MS, Volpp KG. Leveraging insights from behavioral economics to increase the value of health-care service provision. J Gen Intern Med. 2012;27:1544–7.CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Patel MS, Day S, Small DS, Howell JT, Lautenbach GL, Nierman EH, Volpp KG. Using default options within the electronic health record to increase the prescribing of generic-equivalent medications: a quasi-experimental study. Ann Intern Med. 2014;161:S44–52.CrossRefPubMedGoogle Scholar
  14. 14.
    Patel MS, Day SC, Halpern SD, Hanson CW, Martinez JR, Honeywell S, Volpp KG. Change in generic medication prescription rates after health system-wide redesign of default options within the electronic health record. JAMA Intern Med. 2016;176(8):847–8.CrossRefPubMedGoogle Scholar
  15. 15.
    Meeker D, Linder JA, Fox CR, Friedberg MW, Persell SD, Goldstein NJ, Knight TK, Hay JW, Doctor JN. Effect of behavioral interventions on inappropriate antibiotic prescribing among primary care practices: a randomized clinical trial. JAMA. 2016;315(9):562–70.CrossRefPubMedGoogle Scholar
  16. 16.
    Patel MS, Volpp KG, Small DS, Wynn C, Zhu J, Yang L, Honeywell S, Day SC. Using active choice within the electronic health record to increase physician ordering and patient completion of high-value cancer screening tests. Healthcare J Deliv Sci Innov. 2016;4:340–5.Google Scholar
  17. 17.
    Campbell DT, Stanley JC. Experimental and Quasi-Experimental Designs for Research. Dallas, TX: Houghton Mifflin Co; 1963.Google Scholar
  18. 18.
    Dimick JB, Ryan AM. Methods for evaluating changes in health care policy: the difference-in-differences approach. JAMA. 2014;312(22):2401–2.CrossRefPubMedGoogle Scholar
  19. 19.
    Rosenbaum PR. Stability in the absence of treatment. J Am Stat Assoc. 2001;96:210–9.CrossRefGoogle Scholar
  20. 20.
    Shadish WR, Cook TD, Campbell DT. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton-Mifflin; 2002.Google Scholar
  21. 21.
    Patel MS, Patel N, Small DS, Rosin R, Rohrbach JI, Stromberg N, Hanson CW, Asch DA. Change in length of stay and readmissions among hospitalized medical patients after inpatient medicine service adoption of mobile secure text messaging. J Gen Intern Med. 2016;31(8):863–70.CrossRefPubMedGoogle Scholar
  22. 22.
    Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83.CrossRefPubMedGoogle Scholar
  23. 23.
    Rogers WH. Regression standard errors in clustered samples. Stata Tech Bull Rep. 1993;3:88–94.Google Scholar
  24. 24.
    Williams RL. A note on robust variance estimation for cluster-correlated data. Biometrics. 2000;56:645–6.CrossRefPubMedGoogle Scholar
  25. 25.
    Newman SC. Biostatistical Methods in Epidemiology. New York: Wiley; 2001:136.Google Scholar
  26. 26.
    Agresti A. An Introduction to Categorical Data Analysis. New York: Wiley; 2007:108–9.CrossRefGoogle Scholar
  27. 27.
    Efron B, Tibshirani RJ. An Introduction to the Bootstrap. New York, NY: Chapman & Hall; 1993.CrossRefGoogle Scholar
  28. 28.
    Davison AC, Hinkley DV. Bootstrap Methods and Their Application. Cambridge, England: Cambridge University Press; 1997.CrossRefGoogle Scholar
  29. 29.
    Centers for Disease Control and Prevention. Past Flu Seasons. Available online at: Accessed January 15, 2017.
  30. 30.
    Thomas RE, Lorenzetti DL. Interventions to increase influenza vaccination rates of those 60 years and older in the community. Cochrane Database Syst Rev. 2014;7:CDOO5188.Google Scholar
  31. 31.
    Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009;124(1):159–69.CrossRefPubMedGoogle Scholar
  32. 32.
    Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med. 2011;8:e1000387.CrossRefPubMedPubMedCentralGoogle Scholar
  33. 33.
    van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc. 2006;13:138–47.CrossRefPubMedPubMedCentralGoogle Scholar
  34. 34.
    Avery AJ, Savelyich BS, Sheikh A, Cantrill J, Morris CJ, Fernando B, et al. Identifying and establishing consensus on the most important safety features of GP computer systems: e-Delphi study. Inform Prim Care. 2005;13:3–12.PubMedGoogle Scholar
  35. 35.
    Phansalkar S, van der Sijs H, Tucker AD, Desai AA, Bell DS, Teich JM, Middleton B, Bates DW. Drug–drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc. 2013;20(3):489–93.CrossRefPubMedGoogle Scholar
  36. 36.
    Ledwich LJ, Harrington TM, Ayoub WT, Sartorius JA, Newman ED. Improving influenza and pneumococcal vaccination in rheumatology patients taking immunosuppressants using an electronic health record best practice alert. Arthritis Rheum. 2009;61(11):1505–10.CrossRefPubMedGoogle Scholar
  37. 37.
    Dexter PR, Perkins SM, Maharry KS, Jones K, McDonald CJ. Inpatient computer-based standing orders vs physician reminders to increase influenza and pneumococcal vaccination rates: a randomized trial. JAMA. 2004;292(19):2366–71.CrossRefPubMedGoogle Scholar
  38. 38.
    Chapman GB, Li M, Colby H, Yoon H. Opting in vs opting out of influenza vaccination. JAMA. 2010;304(1):43–4.CrossRefPubMedGoogle Scholar
  39. 39.
    Gerber JS, Prasad PA, Fiks AG, Localio AR, Grundmeier RW, Bell LM, Wasserman RC, Keren R, Zaoutis TE. Effect of an outpatient antimicrobial stewardship intervention on broad-spectrum antibiotic prescribing by primary care pediatricians: a randomized trial. JAMA. 2013;309(22):2345–52.CrossRefPubMedGoogle Scholar
  40. 40.
    Navathe AS, Emanuel EJ. Physician peer comparisons as a nonfinancial strategy to improve the value of care. JAMA. 2016;316(17):1759–60.CrossRefPubMedGoogle Scholar

Copyright information

© Society of General Internal Medicine 2017

Authors and Affiliations

  • Mitesh S. Patel
    • 1
    • 2
    • 3
    • 4
    • 5
  • Kevin G. Volpp
    • 1
    • 2
    • 3
    • 4
    • 5
  • Dylan S. Small
    • 2
  • Craig Wynne
    • 1
  • Jingsan Zhu
    • 1
    • 4
  • Lin Yang
    • 1
  • Steven HoneywellJr.
    • 1
  • Susan C. Day
    • 1
  1. 1.Perelman School of MedicineUniversity of PennsylvaniaPhiladelphiaUSA
  2. 2.The Wharton SchoolUniversity of PennsylvaniaPhiladelphiaUSA
  3. 3.Crescenz Veterans Affairs Medical CenterPhiladelphiaUSA
  4. 4.Center for Health Incentives and Behavioral Economics, Leonard Davis Institute of Health EconomicsUniversity of PennsylvaniaPhiladelphiaUSA
  5. 5.Penn Medicine Center for Health Care InnovationUniversity of Pennsylvania Health SystemPhiladelphiaUSA

Personalised recommendations