Journal of General Internal Medicine

, Volume 33, Issue 5, pp 659–667 | Cite as

Improving Rates of Outpatient Influenza Vaccination Through EHR Portal Messages and Interactive Automated Calls: A Randomized Controlled Trial

  • Sarah L. CutronaEmail author
  • Jessica G. Golden
  • Sarah L. Goff
  • Jessica Ogarek
  • Bruce Barton
  • Lloyd Fisher
  • Peggy Preusse
  • Devi Sundaresan
  • Lawrence Garber
  • Kathleen M. Mazor
Original Research



Patient reminders for influenza vaccination, delivered via electronic health record (EHR) patient portal messages and interactive voice response (IVR) calls, offer an innovative approach to improving patient care.


To test the effectiveness of portal and IVR outreach in improving rates of influenza vaccination.


Randomized controlled trial of EHR portal messages and IVR calls promoting influenza vaccination.


Adults with no documented influenza vaccination 2 months after the start of influenza season (2014–2015).


Using a factorial design, we assigned 20,000 patients who were active portal users to one of four study arms: (a) receipt of a portal message promoting influenza vaccines, (b) receipt of IVR call with similar content, (c) both a and b, or (d) neither (usual care). We randomized 10,000 non-portal users to receipt of IVR call or usual care. In all intervention arms, information on pneumococcal vaccination was included if the targeted patient was overdue for pneumococcal vaccine.

Main Measures

EHR-documented influenza vaccination during the 2014–2015 influenza season, measured April 2015.

Key Results

Among portal users, 14.0% (702) of those receiving both portal messages and calls, 13.4% (669) of message recipients, 12.8% (642) of call recipients, and 11.6% (582) of those with usual care received vaccines. On multivariable analysis of portal users, those receiving portal messages alone (OR 1.20, 95% CI 1.06–1.35) or IVR calls alone (OR 1.15 95% CI 1.02–1.30) were more likely than usual care recipients to be vaccinated. Those receiving both messages and calls were also more likely than the usual care group to be vaccinated (ad hoc analysis, using a Bonferroni correction: OR 1.29, 97.5% CI 1.13, 1.48). Among non-portal users, 8.5% of call recipients and 8.6% of usual care recipients received influenza vaccines (p = NS). Pneumococcal vaccination rates showed no significant improvement.


Our outreach achieved a small but significant improvement in influenza vaccination rates.

Registration: Identifier NCT02266277 (


electronic health records influenza vaccination patient care 


Influenza infections contribute to increased healthcare costs and loss of productivity, and can lead to serious complications and even death.1 An estimated 5–20% of the US population contracts influenza every year, and over 200,000 people are hospitalized due to influenza-related complications.2 Estimates of annual influenza- and pneumonia-associated deaths (2009–2014) range from 50,000 to almost 57,000.38 According to estimates from the Centers for Disease Control and Prevention (CDC), during the 2013–2014 influenza season, vaccination resulted in approximately 7.2 million fewer illnesses and 90,068 fewer hospitalizations.9 Despite widespread publicity promoting influenza vaccination, vaccines are underutilized.1014 In 2013, national vaccine coverage for influenza was 41.0%, while the Healthy People 2020 target was 70%.15

Patient outreach interventions have been shown to improve rates of influenza and pneumococcal vaccination. Interventions tested16, 17 include letters, postcards, calls, text messages,1821 emails,22 social media,18, 2224 and personally controlled patient health records without links to an electronic health record (EHR).25 EHR-linked software applications have also been used successfully to promote influenza and pneumococcal vaccination.26, 27

There are compelling reasons to use EHR-tethered portals to promote influenza and pneumococcal vaccination. For vaccine outreach, data recorded in the EHR through routine care delivery can inform real-time identification of unvaccinated populations. Portal-based outreach can be more cost-effective than phone calls or mailings and easier to implement than a new application; simple messages can be sent out by office staff without informatics expertise. Studies show that patient portals can enhance patient empowerment, improve medication adherence, reduce office visits, increase self-management of disease and disease awareness, increase the use of preventive medicine, and increase the inclusion of patients in decision-making.2830 To date, no randomized trials have tested the impact of patient-directed portal messages on influenza and pneumococcal vaccination rates.

We conducted a randomized controlled trial aimed at improving influenza vaccination rates among eligible adults in a large multi-specialty group practice in central Massachusetts. We used EHR patient portal messages and interactive voice response (IVR) calls to promote influenza vaccine completion and to solicit patient self-reports on vaccines received outside the clinic and on barriers to vaccination. In all intervention arms, information on pneumococcal vaccines was included if the patient was eligible and overdue for pneumococcal vaccination.

Our primary objective was to increase completion rates for influenza vaccination among unvaccinated adults in the group practice. Our secondary objective was to increase completion rates for pneumococcal vaccination in patients who were also eligible and overdue for pneumococcal vaccination.

Additional objectives were to (a) improve documentation of influenza vaccinations administered outside the practice by inviting patient self-report (improving accuracy of existing decision support tools), (b) deliver targeted vaccine information related to patients’ concerns, and (c) examine process measures (e.g., rates of portal message opening and IVR call answering).


Study Design

This non-blinded randomized controlled trial was conducted at a large multi-specialty medical group in central Massachusetts. Study design information has been published previously.31

Using a factorial design, we assigned 20,000 patients who were active portal users to one of four study arms (5000/arm): (a) receipt of a portal message promoting influenza vaccines, (b) receipt of IVR call with similar content, (c) both a and b, or (d) neither (usual care). We assigned 10,000 patients (5000/arm) who were not active portal users to (e) receipt of IVR call or (f) no call (usual care).

The study was reviewed and approved in 2014 by the Reliant Medical Group Institutional Review Board (IRB). This IRB later disbanded and in 2015, oversight was transferred to the University of Massachusetts Medical School IRB. A waiver of informed consent for patient outreach was approved by these IRBs. Patients were not compensated for participation.

Study Population

Patients were eligible if, on the date of randomization, they were (1) active with a medical group primary care provider (PCP) and (2) aged ≥18 years. We defined “active with a PCP” as having an assigned provider and a recent visit or telephone encounter (internal medicine, family medicine, or pediatrics). Criteria for “recent” varied by age. Since older patients visit providers more frequently, we required a visit or call within the prior 18 months for patients aged ≥65 years. For adults aged 18–64, we required a visit or call within the prior 3 years.

Exclusion criteria included EHR documentation of influenza vaccine allergy, patient’s name on do-not-call list (or no available phone number), or EHR documentation of influenza vaccination receipt after April 2014. Twenty patients who participated in qualitative interviews (which informed the development of outreach materials) were also excluded.

Within the above-described population, patients were eligible for pneumococcal vaccine outreach if, due to age or medical conditions, they were eligible for pneumococcal vaccination and were neither up to date nor allergic.

Eligibility for portal user arms included active portal use, defined as having an activated EHR portal with a login at least once in the prior year.


This study consisted of portal-based or IVR outreach, beginning November 10, 2014. Qualitative interviews with patients, physicians, nurses, and staff informed the development of outreach material.31

Electronic Patient Portal Intervention

A secure portal message was sent via tethered patient portal to patients randomized to portal arms (Appendix A online). Messages appeared in letter format; the signature line contained the name of the patient’s PCP. Messages were delivered through standard channels. A generic message (without personal health information or reference to vaccines) was delivered to the patient’s email account, prompting login to the secure portal via a hyperlink. Once logged in, patients clicked on a message labeled “Brief Flu Questionnaire” to view.

Characteristics unique to the portal message (as compared to IVR) included access to direct online scheduling of influenza vaccination appointments. Information about CDC vaccine website(s) appeared within the body of the message as a hyperlink (conveyed verbally via IVR). Opportunities to report community-administered influenza vaccinations, barrier questions, and targeted information dispelling misconceptions matched the IVR call content. Among these patients, those also overdue for pneumococcal vaccination received outreach with additional messaging encouraging them to speak with their healthcare provider about pneumococcal vaccines, as well as a CDC hyperlink for more information.

In order to target unvaccinated patients, we intervened 2 months into influenza season, a practice supported by prior research.32 Messages were sent to 500–1500 patients daily over 9–10 days, beginning 1 week post-randomization.

Interactive Voice Response (IVR) Call Intervention

IVR calls appeared on caller ID as originating from the medical group. Combining speech recognition with branching logic, calls elicited patient self-report of influenza vaccinations completed outside the medical group (Appendix B online). For patients reporting no vaccine, calls asked about barriers and responded with brief, targeted education. As with portal messages, information on pneumococcal vaccines was given to eligible, overdue patients.

Calls, initiated 1 week post-randomization, began by confirming that the call had reached the targeted patient; if voicemail or another household member was reached, a message requested callback, providing an inbound number. The last outbound calls were placed on December 1, 2014. The inbound line was maintained throughout the duration of outgoing calls and for 2 weeks thereafter; callers from the phone number of record heard the entire call script, beginning with questions confirming their identity.

Study Outcomes

Primary Outcome: Receipt of Influenza Vaccine

Our primary outcome was EHR-documented influenza vaccine administered during the 2014–2015 influenza season. Along with vaccines administered by medical group staff, we included routinely collected patient reports (e.g., patient reported vaccine directly to PCP or staff, with manual entry of information into EHR). We excluded community-administered vaccinations self-reported solely through the portal questionnaire or IVR. The usual care group lacked this opportunity for self-report; thus, excluding these self-reported vaccines was intended to avoid introducing bias through differential capture of outcomes.31

Additional Outcomes of Interest

We tracked self-report (via portal or IVR) of influenza vaccines administered outside the medical group, reported on or before April 1, 2015.

We also tracked patient-reported barriers to influenza vaccine, captured via portal questionnaire and IVR recording.

We performed a secondary analysis of EHR-documented pneumococcal vaccine completion, measured on April 1, 2015, evaluated among the subset of patients who were eligible and overdue for pneumococcal vaccination.

Process Measures

For portal message recipients, process measures included the percentage of recipients who (a) completed one or more logins to the portal between message delivery and April 1, 2015, (b) opened their message, and (c) completed their questionnaire. For call recipients, process measures included the percentage of recipients who (a) answered the call and (b) responded to the survey.

Sample Size

With our proposed sample size and using a factorial design as described, power calculations based on estimates of baseline vaccination rates indicated that 4286 participants per arm would give 80% power to detect a 3% improvement in influenza vaccination rates between groups (α = 0.05; 2-sided). Using computer-generated random number assignments, we selected 20,000 portal users and 10,000 non portal users (total of 30,000 patients) from the eligible population.


Using a factorial design (Fig. 1), we used computerized randomization to assign 5000 patients to each of four arms (portal users) and, separately, to each of two arms (non-portal users), resulting in a total of six arms, each with 5000 patients.
Figure 1

CONSORT diagram.


Primary Outcome: Receipt of Influenza Vaccine

To determine the impact of our interventions on influenza vaccine completion, we calculated frequencies and performed intention-to-treat bivariate analyses of randomized patients (30,000 patients), assessing whether vaccine completion was associated with group assignment. Due to differential estimated rates of EHR-recorded vaccination at baseline between portal users (35.9%) and non-users (25%), analyses in these groups were conducted separately.

We then performed multivariable logistic regression analyses. We created dummy variables for assignment to the portal message arm (among portal users) and for assignment to the IVR arm (among both portal users and, separately, non-portal users). Including these dummy variables and adjusting for demographic covariates, we modeled the odds of receiving an influenza vaccine.

Two comparisons, (a) receipt of both (IVR and portal message) vs. usual care and (b) receipt of both vs. IVR call, were not pre-specified in our published protocol and are considered a secondary analysis. Because of the possibility of an inflated alpha level for these ad hoc comparisons, when calculating the confidence intervals, we reduced the alpha level to 0.025 using a Bonferroni correction, yielding 97.5% confidence intervals.

Additional Outcomes of Interest

We calculated the percentage of patients who used the portal or IVR for self-report of influenza vaccine completion outside the medical group.

We also calculated the number of patients who used the portal questionnaire or IVR survey to report influenza vaccination barriers. For patients providing free-text responses to influenza vaccine concerns or barriers, we coded responses (allowing up to three codes per response), and grouped coded responses by theme, presenting the number of patients reporting each grouped code.

Using as a denominator those patients identified as overdue for pneumococcal vaccine, we calculated frequencies and performed bivariate and multivariable logistic regression analyses, examining the association between randomization group and completion of pneumococcal vaccine. We analyzed portal users and non-portal users separately.

Process Measures

We performed a descriptive analysis of process measures.


Treatment group characteristics are shown in Table 1.
Table 1

Baseline Characteristics of Participants


Total sample

Portal users*

Non-portal users

No. (%) N = 30,000

Message & IVR call

Message only

IVR call only

Usual care

IVR call only

Usual care

No. (%)

No. (%)

No. (%)

No. (%)

No. (%)

No. (%)




17,616 (58.72) 12,384 (41.28)

3111 (62.22) 1889 (37.78)

3179 (63.58) 1821 (36.42)

3166 (63.32) 1834 (36.68)

3132 (62.64) 1868 (37.36)

2497 (49.94) 2503 (50.06)

2531 (50.62) 2469 (49.38)

Age (years)






9289 (30.96) 8496 (28.32) 8488 (28.29) 2267 (7.56) 1487 (4.96)

1416 (28.32) 1461 (29.22) 1567 (31.34) 399 (7.98) 157 (3.14)

1337 (26.74) 1482 (29.64) 1597 (31.94) 409 (8.18) 175 (3.50)

1380 (27.60) 1505 (30.10) 1518 (30.36) 412 (8.24) 185 (3.70)

1306 (26.12) 1427 (28.54) 1627 (32.54) 422 (8.44)218 (4.36)

1931 (38.62) 1273 (25.46) 1116 (22.32) 319 (6.38) 361 (7.22)

1919 (38.38) 1321 (26.42) 1063 (21.26) 306 (6.12) 391 (7.82)







23,356 (77.85) 1025 (3.42) 1312 (4.37) 622 (2.08) 3685 (12.28)

4031 (80.62) 150 (3.00) 252 (5.04) 86 (1.72) 481 (9.62)

4105 (82.10) 145 (2.90) 212 (4.24) 72 (1.44) 466 (9.32)

4022 (80.44) 153 (3.06) 248 (4.96) 81 (1.62) 496 (9.92)

4125 (82.50) 113 (2.26) 211 (4.22) 84 (1.68) 467 (9.34)

3519 (70.38) 234 (4.68) 189 (3.78) 138 (2.76) 920 (18.40)

3554 (71.08) 230 (4.60) 200 (4.00) 161 (3.22) 855 (17.10)

Visited clinic during flu season

 Had office visit

5072 (16.91)

916 (18.32)

919 (18.38)

911 (18.22)

984 (19.68)

694 (13.88)

738 (14.76)

 Did not have office visit

24,928 (83.09)

4084 (81.68)

4081 (81.62)

4089 (81.78)

4106 (82.12)

4306 (86.12)

4262 (85.24)

Received flu vaccine between randomization and first contact§








Overdue for pneumococcal vaccine at time of randomization

7787 (25.96)

1122 (22.44)

1164 (23.28)

1176 (23.52)

1183 (23.66)

1580 (31.60)

1562 (31.24)

*Logged on in past year

Either no portal or has portal with no login over past year

Since start of current flu season (9/1/2014–10/31/2014)

§Randomization occurred 11/3/2014; first contact was the date the portal message was delivered or the IVR call placed (for participants receiving both, earlier date was used)

Primary Outcome: Receipt of Influenza Vaccine

Among portal users, 14.0% (702/5000) of those receiving both portal messages and IVR calls, 13.4% (669/5000) of those receiving messages only, 12.8% (642/5000) of those receiving calls only, and 11.6% (582/5000) of the usual care group received EHR-documented influenza vaccines. Among non-portal users, 8.5% of call recipients and 8.6% of usual care recipients received vaccines (p = NS).

On bivariate analysis of portal users (Table 2), those receiving portal messages alone (OR 1.17, CI 1.04–1.32) were more likely than the usual care patients to receive influenza vaccinations. Bivariate analysis of non-portal users found no statistically significant differences between intervention and usual care.
Table 2

Likelihood of Receiving an Influenza Vaccine in the 2014–2015 Influenza Season among Patients Who Use and Who Do Not Use Tethered EHR Patient Portals







95% CI


95% CI

EHR patient portal users

 IVR call arm vs. usual care






 Portal message arm vs. usual care






 Arm receiving both (IVR + portal message) vs. portal message arm






 Arm receiving both (IVR + portal message) vs. IVR call arm






 Arm receiving both (IVR + portal message) vs. usual care






Non-users of EHR patient portals

 IVR call arm vs. usual care






*Adjusted for age, sex, and race

97.5% CI. Two comparisons—receipt of both (IVR and portal message) vs. usual care and receipt of both vs. IVR call—were not pre-specified in our published protocol. Because of the possibility of inflated alpha level for these ad hoc comparisons, when calculating the confidence intervals, we reduced the alpha level to 0.025 using a Bonferroni correction, yielding 97.5% confidence intervals

On multivariable analysis of portal users adjusting for age, race, and sex, those receiving portal messages alone (OR 1.20, CI 1.06–1.35) were more likely to receive influenza vaccinations than those with usual care; call recipients were also more likely than usual care recipients to receive influenza vaccinations (OR 1.15 CI 1.02–1.30). Multivariable analysis of non-portal users revealed no significant differences.

For our ad-hoc comparisons, those receiving both (IVR calls and portal messages) were significantly more likely than those with usual care to receive influenza vaccinations on bivariate and multivariable analysis (Table 2).

Additional Outcomes of Interest

A total of 8311 patients responded to portal questionnaires and IVR surveys (1537 portal questionnaires, 6774 IVR surveys), reporting 2591 influenza vaccines completed outside the clinic and not previously recorded in the EHR; we also gained insight into barriers for unvaccinated patients (Fig. 2).
Figure 2

Barriers to influenza vaccination reported by MyChart message recipients and IVR call recipients who reported not yet having received an annual influenza vaccine.

Among portal users overdue for pneumococcal vaccine, 13.6% (153/1122) of those receiving both portal messages and calls, 13.3% (155/1164) of those receiving portal messages only, 15.0% (176/1176) of those receiving IVR calls only, and 15.1% (178/1183) of the usual care group received pneumococcal vaccines. Among non-portal users, 10.2% (161/1580) of those receiving IVR calls and 9.9% (154/1562) of the usual care group received pneumococcal vaccines.

There were no significant differences between groups on bivariate or multivariable analysis.

Process Measures

Among those randomized to receive both calls and messages, messages were opened by 53.2%; questionnaires were answered by 14.8% (Table 3). Among those receiving messages alone, 54.4% opened messages and 16.0% responded. Of portal users who logged in during flu season, 70.4% (5381/7643) opened our message; 28.6% (1537/5381) of message openers responded.
Table 3

Process Measures for Portal Message Recipients

Randomization arm

Message & IVR call (N = 5000)

Message only (N = 5000)

Total (N = 10,000)







Message sent (11/10/14–11/20/14)







Logged into patient portal on or after message delivery and before April 1, 2015







Messages opened







Opened by December 31, 2014







Opened January 1, 2015–March 31, 2015







Responded to questionnaire







Among those randomized to receive both calls and messages, 61.2% of patients were reached by IVR (patient answered or called back); in the call-only group, 61.1% of patients were reached (Table 4). Among non-portal users, IVR reached 51.1% of the call group, with 78.3% (6774/8656) of those reached responding to at least the first question.
Table 4

Process Measures for IVR Call Recipients

Randomization arm

Portal users

Non-portal users

Total (N = 15,000)

Message & IVR call (N = 5000)

IVR call only (N = 5000)

IVR call only (N = 5000)








Call attempted (11/10/14–12/1/14)








Target Reached (11/10/14–12/15/14)

Inbound & outbound calls








Responded to inquiry: “Have you received the flu vaccine on or after August 1, 2014?”








Responded to inquiry: “Are you planning to get a flu vaccine?”









Our study demonstrated a small but statistically significant improvement in completion of influenza vaccination among portal users receiving a portal message, an IVR call, or both. Achieved across a broad sample of our medical group’s non-adherent patients, and using easily reproducible outreach methods, this finding is clinically relevant. Adding IVR to portal messages was slightly but not significantly better than messages alone, and IVR calls among non-portal users did not improve vaccination rates. Our findings support ongoing use of portal messages, but may not support continued dedication of resources to IVR calls for influenza vaccine reminders, particularly among non-portal users.

Though the effect of our intervention was small, a comparable increase in influenza vaccinations among the over 25,500 portal users eligible for our November 2014 outreach (Fig. 1) would yield between 460 and 610 completed vaccinations. Once developed, scaling up portal message delivery (and repeating outreach yearly) can be implemented with minimal additional costs (several hours for a staff member to send batched portal messages). IVR calls require some additional ongoing costs but could be creatively combined with existing outreach (e.g., IVR appointment reminders). Our medical group plans to continue yearly portal outreach for influenza vaccination and has already adapted these messages in response to physician requests, targeting pediatric populations at high risk for influenza. Our intervention’s small effect size is matched with considerable reach and sustainability, increasing its potential impact.

IVR alone worked better with portal users than non-portal users, perhaps due to a higher level of healthcare engagement among portal users. With higher baseline vaccination rates and a number of portal respondents indicating intent to get vaccinated (Fig. 2), an IVR call acting as reminder and facilitator (e.g., list of upcoming clinics) might have adequately addressed barriers for a higher percentage of people in this population.

Previous studies using EHR-linked software applications for patient outreach have yielded mixed results. One study, in contrast to our findings, found that access to an EHR-linked interactive health record led to improvement in rates of pneumococcal but not influenza vaccines.26 Another trial describing an interactive EHR-linked “eJournal” found significant improvement27 among those receiving vaccine-related information. Only one-fifth of those approached for the eJournal trial consented to enrollment; in contrast, our population was more representative but possibly less engaged at baseline.27

This outreach represents a successful brief patient engagement effort. We attained greater than 50% open rates for portal messages, and over two-thirds of those who logged in during flu season opened our message. We also gained actionable information from over 1000 patients describing why they were unvaccinated. More than half of those responding to the portal barrier questions had not discussed the vaccine with a healthcare provider. Commonly cited barriers included fear of vaccine side effects and the belief that they would not get the flu. Provider discussion and educational materials have the potential to address these barriers.

Patient engagement with the portal component of our study was comparable to previous portal outreach studies. In one study, among patients randomized to active reminders about multiple health maintenance services, nearly 65% of patients logged into the portal after receiving the first of several messages.33 Multiple health maintenance messages delivered via a secure personal health record (PHR) “ejournal” yielded higher rates (71–75% opening) for influenza and pneumococcal vaccine messages. Future studies could explore ways to improve engagement through adjustments in outreach timing or message content.

Our study has limitations. We do not know whether these findings are generalizable to patients in other geographic regions or those without access to primary care or EHR patient portals. In 2014, there were multiple changing recommendations regarding the pneumococcal vaccine and changes in coverage by payers; thus pneumococcal vaccine findings are difficult to interpret. We intentionally timed our outreach to target a population with high rates of non-adherence based on medical group data from the previous 3 years showing that over two-thirds of influenza vaccinations are completed prior to November. The overall vaccination rate in our study, therefore, is low, potentially limiting generalizability. A follow-up study by our team is exploring the impact of earlier outreach.

In conclusion, outreach via portal message and IVR to a non-adherent primary care population achieved a small but statistically significant improvement in rates of influenza vaccination among portal users. Through existing portal functions embedded within a widely implemented EHR vendor, our intervention design lends itself to sustained use across diverse settings and with limited resource expenditure.



This study was funded by Pfizer Independent Grants for Learning and Change. The authors designed and implemented the study with full oversight and responsibility for data collection, analysis, and manuscript preparation. Our study team acknowledges the contributions made by Eliza, an HMS Company, toward the design and implementation of the IVR calls.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Supplementary material

11606_2017_4266_MOESM1_ESM.docx (22 kb)
ESM 1 (DOCX 22 kb)


  1. 1.
    Molinari NA, Ortega-Sanchez IR, Messonnier ML, Thompson WW, Wortley PM, Weintraub E, et al. The annual impact of seasonal influenza in the US: measuring disease burden and costs. Vaccine. 2007;25(27):5086-96. Scholar
  2. 2.
    NIH Fact Sheets - Influenza. In: Diseases NIH-NIAID, editor. Research Portfolio Online Reporting Tools (RePORT)2011. Accessed on Jan 9, 2018 at
  3. 3.
    National Center for Health Statistics (US. "Health, United States, 2015: with special feature on racial and ethnic health disparities." (2016 May). Report No.: 2016-1232.Google Scholar
  4. 4.
    Kochanek KD, Murphy SL, Xu J. Deaths: Final Data for 2011. National vital statistics reports : from the Centers for Disease Control and Prevention, National Center for Health Statistics, National Vital Statistics System. 2015;63(3):1-120.Google Scholar
  5. 5.
    Kochanek KD, Xu J, Murphy SL, Minino AM, Kung HC. Deaths: final data for 2009. National vital statistics reports : from the Centers for Disease Control and Prevention, National Center for Health Statistics, National Vital Statistics System. 2011;60(3):1-116.Google Scholar
  6. 6.
    Murphy SL, Kochanek KD, Xu J, Heron M. Deaths: Final Data for 2012. National vital statistics reports : from the Centers for Disease Control and Prevention, National Center for Health Statistics, National Vital Statistics System. 2015;63(9):1-117.Google Scholar
  7. 7.
    Murphy SL, Xu J, Kochanek KD. Deaths: final data for 2010. National vital statistics reports : from the Centers for Disease Control and Prevention, National Center for Health Statistics, National Vital Statistics System. 2013;61(4):1-117.Google Scholar
  8. 8.
    Xu J, Murphy SL, Kochanek KD, Bastian BA. Deaths: Final Data for 2013. National vital statistics reports : from the Centers for Disease Control and Prevention, National Center for Health Statistics, National Vital Statistics System. 2016;64(2):1-119.Google Scholar
  9. 9.
    Reed C, Kim IK, Singleton JA, Chaves SS, Flannery B, Finelli L, et al. Estimated influenza illnesses and hospitalizations averted by vaccination--United States, 2013-14 influenza season. MMWR. Morbidity and mortality weekly report. 2014;63(49):1151–54.Google Scholar
  10. 10.
    A pathway to leadership for adult immunization: recommendations of the National Vaccine Advisory Committee: approved by the National Vaccine Advisory Committee on June 14, 2011. Public health reports (Washington, DC : 1974). 2012 Jan-Feb;127 Suppl 1:1–42.Google Scholar
  11. 11.
    Appel A. Improving adult immunization rates: overcoming barriers. American family physician. 2011;84(9):977-8.PubMedGoogle Scholar
  12. 12.
    Bean-Mayberry B, Yano EM, Mor MK, Bayliss NK, Xu X, Fine MJ. Does sex influence immunization status for influenza and pneumonia in older veterans? Journal of the American Geriatrics Society. 2009;57(8):1427-32. Scholar
  13. 13.
    Nichol KL, Lind A, Margolis KL, Murdoch M, McFadden R, Hauge M, et al. The effectiveness of vaccination against influenza in healthy, working adults. The New England journal of medicine. 1995;333(14):889-93. doi: Scholar
  14. 14.
    Vaughn JA, Miller RA. Update on immunizations in adults. American family physician. 2011;84(9):1015-20.PubMedGoogle Scholar
  15. 15.
    National Center for Health Statistics (US. 2015). Health, United States, 2014: with special feature on adults aged 55–64. Accessed on Jan 9, 2018 at
  16. 16.
    Moniz MH, Hasley S, Meyn LA, Beigi RH. Improving influenza vaccination rates in pregnancy through text messaging: a randomized controlled trial. Obstetrics and gynecology. 2013;121(4):734-40. Scholar
  17. 17.
    Odone A, Ferrari A, Spagnoli F, Visciarelli S, Shefer A, Pasquarella C, et al. Effectiveness of interventions that apply new media to improve vaccine uptake and vaccine coverage. Human vaccines & immunotherapeutics. 2015;11(1):72-82. Scholar
  18. 18.
    Herrett E, Williamson E, van Staa T, Ranopa M, Free C, Chadborn T, et al. Text messaging reminders for influenza vaccine in primary care: a cluster randomised controlled trial (TXT4FLUJAB). BMJ open. 2016;6(2):e010069. Scholar
  19. 19.
    Stockwell MS, Kharbanda EO, Martinez RA, Vargas CY, Vawdrey DK, Camargo S. Effect of a text messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent population: a randomized controlled trial. Jama. 2012;307(16):1702-8. Scholar
  20. 20.
    Jordan ET, Bushar JA, Kendrick JS, Johnson P, Wang J. Encouraging Influenza Vaccination Among Text4baby Pregnant Women and Mothers. American journal of preventive medicine. 2015;49(4):563-72. Scholar
  21. 21.
    Stockwell MS, Westhoff C, Kharbanda EO, Vargas CY, Camargo S, Vawdrey DK, et al. Influenza vaccine text message reminders for urban, low-income pregnant women: a randomized controlled trial. American journal of public health. 2014;104 Suppl 1:e7-12. Scholar
  22. 22.
    Ghadieh AS, Hamadeh GN, Mahmassani DM, Lakkis NA. The effect of various types of patients' reminders on the uptake of pneumococcal vaccine in adults: A randomized controlled trial. Vaccine. 2015;33(43):5868-72. Scholar
  23. 23.
    Thomas RE, Russell ML, Lorenzetti DL. Systematic review of interventions to increase influenza vaccination rates of those 60 years and older. Vaccine. 2010;28(7):1684-701. Scholar
  24. 24.
    Jacobson Vann JC, Szilagyi P. Patient reminder and patient recall systems to improve immunization rates. The Cochrane database of systematic reviews. 2005 Jul 20(3):Cd003941. Scholar
  25. 25.
    Lau AY, Sintchenko V, Crimmins J, Magrabi F, Gallego B, Coiera E. Impact of a web-based personally controlled health management system on influenza vaccination and health services utilization rates: a randomized controlled trial. Journal of the American Medical Informatics Association : JAMIA. 2012;19(5):719-27. Scholar
  26. 26.
    Krist AH, Woolf SH, Rothemich SF, Johnson RE, Peele JE, Cunningham TD, et al. Interactive preventive health record to enhance delivery of recommended care: a randomized trial. Annals of family medicine. 2012;10(4):312-9. Scholar
  27. 27.
    Wright A, Poon EG, Wald J, Feblowitz J, Pang JE, Schnipper JL, et al. Randomized controlled trial of health maintenance reminders provided directly to patients through an electronic PHR. Journal of general internal medicine. 2012;27(1):85-92. Scholar
  28. 28. What is a patient portal? Available at: Accessed November 20, 2017.
  29. 29.
    Kruse CS, Argueta DA, Lopez L, Nair A. Patient and provider attitudes toward the use of patient portals for the management of chronic disease: a systematic review. Journal of medical Internet research. 2015;17(2):e40. Scholar
  30. 30.
    Kruse CS, Bolton K, Freriks G. The effect of patient portals on quality outcomes and its implications to meaningful use: a systematic review. Journal of medical Internet research. 2015;17(2):e44. Scholar
  31. 31.
    Cutrona SL, Sreedhara M, Goff SL, Fisher LD, Preusse P, Jackson M, et al. Improving Rates of Influenza Vaccination Through Electronic Health Record Portal Messages, Interactive Voice Recognition Calls and Patient-Enabled Electronic Health Record Updates: Protocol for a Randomized Controlled Trial. JMIR research protocols. 2016;5(2):e56. Scholar
  32. 32.
    Minor DS, Eubanks JT, Butler KR, Jr., Wofford MR, Penman AD, Replogle WH. Improving influenza vaccination rates by targeting individuals not seeking early seasonal vaccination. The American journal of medicine. 2010;123(11):1031-5. Scholar
  33. 33.
    Fischer GS, Hess R, Landeen BM, Weimer M, Zieth CR, Dong X, et al. Electronic reminders to patients within an interactive patient health record. Telemedicine journal and e-health : the official journal of the American Telemedicine Association. 2013;19(6):497-500. Scholar

Copyright information

© Society of General Internal Medicine (outside the USA) 2018

Authors and Affiliations

  • Sarah L. Cutrona
    • 1
    • 2
    Email author
  • Jessica G. Golden
    • 3
  • Sarah L. Goff
    • 4
    • 5
  • Jessica Ogarek
    • 6
  • Bruce Barton
    • 2
  • Lloyd Fisher
    • 2
    • 7
  • Peggy Preusse
    • 7
  • Devi Sundaresan
    • 7
  • Lawrence Garber
    • 3
    • 7
  • Kathleen M. Mazor
    • 2
    • 3
  1. 1.Veterans Health AdministrationBedfordUSA
  2. 2.University of Massachusetts Medical SchoolWorcesterUSA
  3. 3.Meyers Primary Care InstituteWorcesterUSA
  4. 4.Baystate Medical CenterSpringfieldUSA
  5. 5.Tufts University School of MedicineBostonUSA
  6. 6.Brown UniversityProvidenceUSA
  7. 7.Reliant Medical GroupWorcesterUSA

Personalised recommendations