INTRODUCTION

Influenza infections contribute to increased healthcare costs and loss of productivity, and can lead to serious complications and even death.1 An estimated 5–20% of the US population contracts influenza every year, and over 200,000 people are hospitalized due to influenza-related complications.2 Estimates of annual influenza- and pneumonia-associated deaths (2009–2014) range from 50,000 to almost 57,000.3,4,5,6,7,–8 According to estimates from the Centers for Disease Control and Prevention (CDC), during the 2013–2014 influenza season, vaccination resulted in approximately 7.2 million fewer illnesses and 90,068 fewer hospitalizations.9 Despite widespread publicity promoting influenza vaccination, vaccines are underutilized.10,11,12,13,–14 In 2013, national vaccine coverage for influenza was 41.0%, while the Healthy People 2020 target was 70%.15

Patient outreach interventions have been shown to improve rates of influenza and pneumococcal vaccination. Interventions tested16, 17 include letters, postcards, calls, text messages,18,19,20,–21 emails,22 social media,18, 22,23,–24 and personally controlled patient health records without links to an electronic health record (EHR).25 EHR-linked software applications have also been used successfully to promote influenza and pneumococcal vaccination.26, 27

There are compelling reasons to use EHR-tethered portals to promote influenza and pneumococcal vaccination. For vaccine outreach, data recorded in the EHR through routine care delivery can inform real-time identification of unvaccinated populations. Portal-based outreach can be more cost-effective than phone calls or mailings and easier to implement than a new application; simple messages can be sent out by office staff without informatics expertise. Studies show that patient portals can enhance patient empowerment, improve medication adherence, reduce office visits, increase self-management of disease and disease awareness, increase the use of preventive medicine, and increase the inclusion of patients in decision-making.28,29,–30 To date, no randomized trials have tested the impact of patient-directed portal messages on influenza and pneumococcal vaccination rates.

We conducted a randomized controlled trial aimed at improving influenza vaccination rates among eligible adults in a large multi-specialty group practice in central Massachusetts. We used EHR patient portal messages and interactive voice response (IVR) calls to promote influenza vaccine completion and to solicit patient self-reports on vaccines received outside the clinic and on barriers to vaccination. In all intervention arms, information on pneumococcal vaccines was included if the patient was eligible and overdue for pneumococcal vaccination.

Our primary objective was to increase completion rates for influenza vaccination among unvaccinated adults in the group practice. Our secondary objective was to increase completion rates for pneumococcal vaccination in patients who were also eligible and overdue for pneumococcal vaccination.

Additional objectives were to (a) improve documentation of influenza vaccinations administered outside the practice by inviting patient self-report (improving accuracy of existing decision support tools), (b) deliver targeted vaccine information related to patients’ concerns, and (c) examine process measures (e.g., rates of portal message opening and IVR call answering).

METHODS

Study Design

This non-blinded randomized controlled trial was conducted at a large multi-specialty medical group in central Massachusetts. Study design information has been published previously.31

Using a factorial design, we assigned 20,000 patients who were active portal users to one of four study arms (5000/arm): (a) receipt of a portal message promoting influenza vaccines, (b) receipt of IVR call with similar content, (c) both a and b, or (d) neither (usual care). We assigned 10,000 patients (5000/arm) who were not active portal users to (e) receipt of IVR call or (f) no call (usual care).

The study was reviewed and approved in 2014 by the Reliant Medical Group Institutional Review Board (IRB). This IRB later disbanded and in 2015, oversight was transferred to the University of Massachusetts Medical School IRB. A waiver of informed consent for patient outreach was approved by these IRBs. Patients were not compensated for participation.

Study Population

Patients were eligible if, on the date of randomization, they were (1) active with a medical group primary care provider (PCP) and (2) aged ≥18 years. We defined “active with a PCP” as having an assigned provider and a recent visit or telephone encounter (internal medicine, family medicine, or pediatrics). Criteria for “recent” varied by age. Since older patients visit providers more frequently, we required a visit or call within the prior 18 months for patients aged ≥65 years. For adults aged 18–64, we required a visit or call within the prior 3 years.

Exclusion criteria included EHR documentation of influenza vaccine allergy, patient’s name on do-not-call list (or no available phone number), or EHR documentation of influenza vaccination receipt after April 2014. Twenty patients who participated in qualitative interviews (which informed the development of outreach materials) were also excluded.

Within the above-described population, patients were eligible for pneumococcal vaccine outreach if, due to age or medical conditions, they were eligible for pneumococcal vaccination and were neither up to date nor allergic.

Eligibility for portal user arms included active portal use, defined as having an activated EHR portal with a login at least once in the prior year.

Intervention

This study consisted of portal-based or IVR outreach, beginning November 10, 2014. Qualitative interviews with patients, physicians, nurses, and staff informed the development of outreach material.31

Electronic Patient Portal Intervention

A secure portal message was sent via tethered patient portal to patients randomized to portal arms (Appendix A online). Messages appeared in letter format; the signature line contained the name of the patient’s PCP. Messages were delivered through standard channels. A generic message (without personal health information or reference to vaccines) was delivered to the patient’s email account, prompting login to the secure portal via a hyperlink. Once logged in, patients clicked on a message labeled “Brief Flu Questionnaire” to view.

Characteristics unique to the portal message (as compared to IVR) included access to direct online scheduling of influenza vaccination appointments. Information about CDC vaccine website(s) appeared within the body of the message as a hyperlink (conveyed verbally via IVR). Opportunities to report community-administered influenza vaccinations, barrier questions, and targeted information dispelling misconceptions matched the IVR call content. Among these patients, those also overdue for pneumococcal vaccination received outreach with additional messaging encouraging them to speak with their healthcare provider about pneumococcal vaccines, as well as a CDC hyperlink for more information.

In order to target unvaccinated patients, we intervened 2 months into influenza season, a practice supported by prior research.32 Messages were sent to 500–1500 patients daily over 9–10 days, beginning 1 week post-randomization.

Interactive Voice Response (IVR) Call Intervention

IVR calls appeared on caller ID as originating from the medical group. Combining speech recognition with branching logic, calls elicited patient self-report of influenza vaccinations completed outside the medical group (Appendix B online). For patients reporting no vaccine, calls asked about barriers and responded with brief, targeted education. As with portal messages, information on pneumococcal vaccines was given to eligible, overdue patients.

Calls, initiated 1 week post-randomization, began by confirming that the call had reached the targeted patient; if voicemail or another household member was reached, a message requested callback, providing an inbound number. The last outbound calls were placed on December 1, 2014. The inbound line was maintained throughout the duration of outgoing calls and for 2 weeks thereafter; callers from the phone number of record heard the entire call script, beginning with questions confirming their identity.

Study Outcomes

Primary Outcome: Receipt of Influenza Vaccine

Our primary outcome was EHR-documented influenza vaccine administered during the 2014–2015 influenza season. Along with vaccines administered by medical group staff, we included routinely collected patient reports (e.g., patient reported vaccine directly to PCP or staff, with manual entry of information into EHR). We excluded community-administered vaccinations self-reported solely through the portal questionnaire or IVR. The usual care group lacked this opportunity for self-report; thus, excluding these self-reported vaccines was intended to avoid introducing bias through differential capture of outcomes.31

Additional Outcomes of Interest

We tracked self-report (via portal or IVR) of influenza vaccines administered outside the medical group, reported on or before April 1, 2015.

We also tracked patient-reported barriers to influenza vaccine, captured via portal questionnaire and IVR recording.

We performed a secondary analysis of EHR-documented pneumococcal vaccine completion, measured on April 1, 2015, evaluated among the subset of patients who were eligible and overdue for pneumococcal vaccination.

Process Measures

For portal message recipients, process measures included the percentage of recipients who (a) completed one or more logins to the portal between message delivery and April 1, 2015, (b) opened their message, and (c) completed their questionnaire. For call recipients, process measures included the percentage of recipients who (a) answered the call and (b) responded to the survey.

Sample Size

With our proposed sample size and using a factorial design as described, power calculations based on estimates of baseline vaccination rates indicated that 4286 participants per arm would give 80% power to detect a 3% improvement in influenza vaccination rates between groups (α = 0.05; 2-sided). Using computer-generated random number assignments, we selected 20,000 portal users and 10,000 non portal users (total of 30,000 patients) from the eligible population.

Randomization

Using a factorial design (Fig. 1), we used computerized randomization to assign 5000 patients to each of four arms (portal users) and, separately, to each of two arms (non-portal users), resulting in a total of six arms, each with 5000 patients.

Figure 1
figure 1

CONSORT diagram.

Full size image

Analyses

Primary Outcome: Receipt of Influenza Vaccine

To determine the impact of our interventions on influenza vaccine completion, we calculated frequencies and performed intention-to-treat bivariate analyses of randomized patients (30,000 patients), assessing whether vaccine completion was associated with group assignment. Due to differential estimated rates of EHR-recorded vaccination at baseline between portal users (35.9%) and non-users (25%), analyses in these groups were conducted separately.

We then performed multivariable logistic regression analyses. We created dummy variables for assignment to the portal message arm (among portal users) and for assignment to the IVR arm (among both portal users and, separately, non-portal users). Including these dummy variables and adjusting for demographic covariates, we modeled the odds of receiving an influenza vaccine.

Two comparisons, (a) receipt of both (IVR and portal message) vs. usual care and (b) receipt of both vs. IVR call, were not pre-specified in our published protocol and are considered a secondary analysis. Because of the possibility of an inflated alpha level for these ad hoc comparisons, when calculating the confidence intervals, we reduced the alpha level to 0.025 using a Bonferroni correction, yielding 97.5% confidence intervals.

Additional Outcomes of Interest

We calculated the percentage of patients who used the portal or IVR for self-report of influenza vaccine completion outside the medical group.

We also calculated the number of patients who used the portal questionnaire or IVR survey to report influenza vaccination barriers. For patients providing free-text responses to influenza vaccine concerns or barriers, we coded responses (allowing up to three codes per response), and grouped coded responses by theme, presenting the number of patients reporting each grouped code.

Using as a denominator those patients identified as overdue for pneumococcal vaccine, we calculated frequencies and performed bivariate and multivariable logistic regression analyses, examining the association between randomization group and completion of pneumococcal vaccine. We analyzed portal users and non-portal users separately.

Process Measures

We performed a descriptive analysis of process measures.

RESULTS

Treatment group characteristics are shown in Table 1.

Table 1 Baseline Characteristics of Participants
Full size table

Primary Outcome: Receipt of Influenza Vaccine

Among portal users, 14.0% (702/5000) of those receiving both portal messages and IVR calls, 13.4% (669/5000) of those receiving messages only, 12.8% (642/5000) of those receiving calls only, and 11.6% (582/5000) of the usual care group received EHR-documented influenza vaccines. Among non-portal users, 8.5% of call recipients and 8.6% of usual care recipients received vaccines (p = NS).

On bivariate analysis of portal users (Table 2), those receiving portal messages alone (OR 1.17, CI 1.04–1.32) were more likely than the usual care patients to receive influenza vaccinations. Bivariate analysis of non-portal users found no statistically significant differences between intervention and usual care.

Table 2 Likelihood of Receiving an Influenza Vaccine in the 2014–2015 Influenza Season among Patients Who Use and Who Do Not Use Tethered EHR Patient Portals
Full size table

On multivariable analysis of portal users adjusting for age, race, and sex, those receiving portal messages alone (OR 1.20, CI 1.06–1.35) were more likely to receive influenza vaccinations than those with usual care; call recipients were also more likely than usual care recipients to receive influenza vaccinations (OR 1.15 CI 1.02–1.30). Multivariable analysis of non-portal users revealed no significant differences.

For our ad-hoc comparisons, those receiving both (IVR calls and portal messages) were significantly more likely than those with usual care to receive influenza vaccinations on bivariate and multivariable analysis (Table 2).

Additional Outcomes of Interest

A total of 8311 patients responded to portal questionnaires and IVR surveys (1537 portal questionnaires, 6774 IVR surveys), reporting 2591 influenza vaccines completed outside the clinic and not previously recorded in the EHR; we also gained insight into barriers for unvaccinated patients (Fig. 2).

Figure 2
figure 2

Barriers to influenza vaccination reported by MyChart message recipients and IVR call recipients who reported not yet having received an annual influenza vaccine.

Full size image

Among portal users overdue for pneumococcal vaccine, 13.6% (153/1122) of those receiving both portal messages and calls, 13.3% (155/1164) of those receiving portal messages only, 15.0% (176/1176) of those receiving IVR calls only, and 15.1% (178/1183) of the usual care group received pneumococcal vaccines. Among non-portal users, 10.2% (161/1580) of those receiving IVR calls and 9.9% (154/1562) of the usual care group received pneumococcal vaccines.

There were no significant differences between groups on bivariate or multivariable analysis.

Process Measures

Among those randomized to receive both calls and messages, messages were opened by 53.2%; questionnaires were answered by 14.8% (Table 3). Among those receiving messages alone, 54.4% opened messages and 16.0% responded. Of portal users who logged in during flu season, 70.4% (5381/7643) opened our message; 28.6% (1537/5381) of message openers responded.

Table 3 Process Measures for Portal Message Recipients
Full size table

Among those randomized to receive both calls and messages, 61.2% of patients were reached by IVR (patient answered or called back); in the call-only group, 61.1% of patients were reached (Table 4). Among non-portal users, IVR reached 51.1% of the call group, with 78.3% (6774/8656) of those reached responding to at least the first question.

Table 4 Process Measures for IVR Call Recipients
Full size table

DISCUSSION

Our study demonstrated a small but statistically significant improvement in completion of influenza vaccination among portal users receiving a portal message, an IVR call, or both. Achieved across a broad sample of our medical group’s non-adherent patients, and using easily reproducible outreach methods, this finding is clinically relevant. Adding IVR to portal messages was slightly but not significantly better than messages alone, and IVR calls among non-portal users did not improve vaccination rates. Our findings support ongoing use of portal messages, but may not support continued dedication of resources to IVR calls for influenza vaccine reminders, particularly among non-portal users.

Though the effect of our intervention was small, a comparable increase in influenza vaccinations among the over 25,500 portal users eligible for our November 2014 outreach (Fig. 1) would yield between 460 and 610 completed vaccinations. Once developed, scaling up portal message delivery (and repeating outreach yearly) can be implemented with minimal additional costs (several hours for a staff member to send batched portal messages). IVR calls require some additional ongoing costs but could be creatively combined with existing outreach (e.g., IVR appointment reminders). Our medical group plans to continue yearly portal outreach for influenza vaccination and has already adapted these messages in response to physician requests, targeting pediatric populations at high risk for influenza. Our intervention’s small effect size is matched with considerable reach and sustainability, increasing its potential impact.

IVR alone worked better with portal users than non-portal users, perhaps due to a higher level of healthcare engagement among portal users. With higher baseline vaccination rates and a number of portal respondents indicating intent to get vaccinated (Fig. 2), an IVR call acting as reminder and facilitator (e.g., list of upcoming clinics) might have adequately addressed barriers for a higher percentage of people in this population.

Previous studies using EHR-linked software applications for patient outreach have yielded mixed results. One study, in contrast to our findings, found that access to an EHR-linked interactive health record led to improvement in rates of pneumococcal but not influenza vaccines.26 Another trial describing an interactive EHR-linked “eJournal” found significant improvement27 among those receiving vaccine-related information. Only one-fifth of those approached for the eJournal trial consented to enrollment; in contrast, our population was more representative but possibly less engaged at baseline.27

This outreach represents a successful brief patient engagement effort. We attained greater than 50% open rates for portal messages, and over two-thirds of those who logged in during flu season opened our message. We also gained actionable information from over 1000 patients describing why they were unvaccinated. More than half of those responding to the portal barrier questions had not discussed the vaccine with a healthcare provider. Commonly cited barriers included fear of vaccine side effects and the belief that they would not get the flu. Provider discussion and educational materials have the potential to address these barriers.

Patient engagement with the portal component of our study was comparable to previous portal outreach studies. In one study, among patients randomized to active reminders about multiple health maintenance services, nearly 65% of patients logged into the portal after receiving the first of several messages.33 Multiple health maintenance messages delivered via a secure personal health record (PHR) “ejournal” yielded higher rates (71–75% opening) for influenza and pneumococcal vaccine messages. Future studies could explore ways to improve engagement through adjustments in outreach timing or message content.

Our study has limitations. We do not know whether these findings are generalizable to patients in other geographic regions or those without access to primary care or EHR patient portals. In 2014, there were multiple changing recommendations regarding the pneumococcal vaccine and changes in coverage by payers; thus pneumococcal vaccine findings are difficult to interpret. We intentionally timed our outreach to target a population with high rates of non-adherence based on medical group data from the previous 3 years showing that over two-thirds of influenza vaccinations are completed prior to November. The overall vaccination rate in our study, therefore, is low, potentially limiting generalizability. A follow-up study by our team is exploring the impact of earlier outreach.

In conclusion, outreach via portal message and IVR to a non-adherent primary care population achieved a small but statistically significant improvement in rates of influenza vaccination among portal users. Through existing portal functions embedded within a widely implemented EHR vendor, our intervention design lends itself to sustained use across diverse settings and with limited resource expenditure.