INTRODUCTION

“No-shows,” which are appointments neither attended nor canceled, are a persistent problem in all healthcare systems. They compromise patient access, lengthen wait times, increase health care costs and inefficiencies, and worsen clinical care.1,2,3,4 The Office of Inspector General estimated that in 2008, 5.7 million veterans and their dependents used the Veterans Health Administration and over 12% of 26.5 million scheduled healthcare appointments were no-shows, which cost the Department of Veterans Affairs (VA) $564 million that year.5, 6 The VA is the largest integrated healthcare system in the USA, with their care population increasing to 6.6 million veterans and their dependents in 2014.6 Despite numerous no-show mitigation efforts by VA in the last decade, rates of missed visits have continued to be elevated. From October 1, 2018, to September 30, 2019, the national no-show rate in VA was 12.5%, resulting in a staggering 8.7 million missed outpatient clinic appointments.7 In many specialty care clinics, no-show rates are even higher, including mental health, where the no-show rate was 17.7% that same year.

Attending appointments can be viewed as a health behavior. No-shows have many causes, including factors related to both the patient and to the healthcare system.8, 9 Forgetting is a common reason for missing an appointment, and appointment reminders are a common strategy used to reduce no-shows. VA uses multiple modalities for appointment reminders, including letters, postcards, telephone calls, and text messages. A 2015 systematic review concluded that the evidence does not demonstrate that any particular reminder modality is more effective than another.10

Due to the size of VA, reminders are used on an extremely large scale. For instance, the Western States Network Consortium Regional Reproduction Center in Sacramento, California prints and mails approximately one million postcard reminders each month (approximately 55,000 per day), and this only covers 36 of 170 VA medical centers and their associated satellite clinics.11 In 2018, VA launched automated text message appointment reminders (a program called “VEText” https://www.va.gov/HEALTH/VEText.asp), which in its first year sent over 134 million text message reminders to over 7 million Veterans.12 Perhaps most common of all, however, is the traditional printed appointment reminder letter. These reminder letters are printed and mailed (or directly handed if scheduled in-person) at individual VA hospitals and clinics.

While there have been dozens of randomized controlled trials testing the effectiveness of various appointment reminder systems and modes of delivery, there has been exceedingly little qualitative research done on appointment reminders.13 Moreover, most research and implementation efforts on no-shows have not examined patients’ behaviors related to appointment reminders and attendance, instead focusing on their demographic and other characteristics.9 Qualitative research is well-suited to understanding patient perspectives, experiences, and behaviors related to appointments. The overarching aim of this manuscript was to gain a patient-informed understanding of veterans’ experiences with the usability of appointment reminders within VA.

METHODS

Recruitment and Participants

The data in this study were collected as part of an ongoing randomized controlled pragmatic clinical trial14 to examine the effectiveness of adding brief motivational messages to appointment reminder letters to act as a nudge15, 16 to reduce no-shows. As part of the intervention development process for the pragmatic trial, the study team obtained informed consent and conducted interviews with patients from the VA Portland Health Care System, which provided institutional review board approval for the study.

To identify potential interview participants, we used the VA Corporate Data Warehouse (CDW) between December 2018 and February 2019 to identify patients who had at least one upcoming outpatient appointment in either primary care or mental health scheduled in at one of the eight clinic sites within VA Portland Health Care System, which included both large, hospital-based clinics and smaller satellite clinics (called CBOCs). We then used purposive sampling17 to recruit participants with diversity across several key characteristics of interest (i.e., gender, military service era, and no-show history). We conducted recruitment in five waves and interviewed a total of 27 patients. Between each wave, we adjusted our sampling to maintain or increase participant diversity and iterated intervention content, which was then presented to participants’ in the subsequent wave. The portion of the interview guide exploring how veterans’ experience appointment reminders within VA remained the same throughout each wave.

Procedures

One of the co-authors, an experienced qualitative researcher (AT), conducted all interviews using a semi-structured interview guide that (1) addressed feedback on draft wording for the nudges; (2) reviewed status quo and draft intervention appointment reminder letters; and (3) asked about experiences with attending, missing, and rescheduling appointments.

Interviews were conducted in-person between March and July 2019. All but two interviews were audio recorded and transcribed; due to human error, the original audio recordings were not saved for these interviews. In one case the interviewer produced field notes summarizing the interview content immediately after the interview. In the second case, the mistake was discovered during the interview and the participant summarized their earlier responses, as part of the recorded interview. Interviews ranged from 26 to 70 min (mean of 48 min) in length. Participants were compensated $20.

Data Analysis

We conducted both deductive and inductive content analysis,18 in order to identify unexpected themes as well as themes grounded in a priori categories. First, between each interview wave we conducted rapid qualitative analysis, a team-based iterative data analysis process to review data, make decisions, and collect more data for review to gain an understanding of the participant’s perspective.19 During our rapid analysis, one co-author (EM) listened to audio recordings of interviews by wave and took notes on the valence and content of reactions to specific nudges and letter design. This information was presented to the research team for analysis between waves of interviews, after which decisions were incorporated into the next wave of interviews for future feedback. This process allowed the research team to make preliminary decisions regarding the design of appointment letter messaging based on participant feedback, followed by testing updated content in future interviews.

Informed by the rapid analysis process, a preliminary codebook was created based on themes identified during that initial review. The full research team collectively reviewed several transcripts and revised the codebook to add additional codes grounded in participants’ observations. Codes included semantic and latent interpretations of patients’ statements, meaning codes allowed not just for direct statements but interpretation by coders, who reached consensus on such interpretations. A co-author with experience in qualitative data analysis (WS) was the primary coder and trained a second coder (KT); coders used Atlas.ti Version 8 for coding and data management. Coders met regularly with the study team to discuss impressions and maintain consistency in coding. At coding completion, the research team examined code reports and developed a three-part analytic framework for the data:

  1. I.

    Limitations related to usability of current appointment reminders (limitations)

  2. II.

    Strategies used by patients that maximize usability of appointment reminders (strategies)

  3. III.

    Recommendations for healthcare systems to enhance appointment reminders (recommendations)

Using this analytic framework, we then examined output from codes to sort data into analytic categories, discussed emergent themes within those categories, and selected quotations that exemplified each theme.

RESULTS

Summary descriptive characteristics of the 27 participants are provided in Table 1. Ages ranged from 34 to 75 years-old, 67% were male, 78% white, and 52% were Vietnam-era veterans. Patients were recruited evenly from mental health and primary care clinics. Participants had zero to 56 appointments (mean of 14) scheduled in the preceding 2 years. Among patients with prior scheduled appointments, four participants had a low no-show rate (1–10%), seven had a moderate no-show rate (11–20%), and seven had a high no-show rate (21%+). We assigned participants pseudonyms in our presentation of results to maintain a sense of their humanity while maintaining privacy.20 Table 2 contains a description of each individual participant’s characteristics to provide context for their comments below.

Table 1 Summary Descriptive Characteristics of Participants (N = 27)
Full size table
Table 2 Descriptive Characteristics for Individual Participants (N = 27)
Full size table

We grouped our findings within our analytic framework of (I) limitations, (II) strategies, and (III) recommendations. Four limitations highlighted by patients related to usability of current appointment reminders. Four strategies used by patients who were successful at keeping appointments involved actions when receiving appointment reminders. And six recommendations for healthcare systems to increase the usability of appointment reminders could enhance their content, timing, and mode of delivery. A summary of quotes that support each appointment reminders limitation and recommendation is contained in Table 3.

  1. I.

    Limitations related to usability of current appointment reminders

Table 3 Summary of Limitations of Appointment Reminders and Related Recommendations with Examples of Supporting Quotes
Full size table

Veterans implied, and sometimes directly stated, that repetitive information, frustrating telephone systems, cryptic clinic information, and “reminder fatigue” all contributed to missing appointments.

  1. 1.

    Excessive information within reminders

Many patients felt that content contained within appointment reminders was excessive or repetitive. Reminder letters, in particular, tended to be perceived as lengthy or containing “a lot of fine print” (Gloria) for busy patients with limited attention spans. As a result, patients often skimmed—rather than read— reminder letters, with a focus on attending to appointment date, time, and location.

  1. 2.

    Frustrating telephone systems when calling in response to an appointment reminder

When a patient needed to call in response to an appointment reminder—for instance, to reschedule—the experience was highly frustrating to some patients. They strongly preferred having a direct telephone number, or at least an extension, specific to the clinic. Dealing with the telephone tree system to cancel or reschedule was viewed as so time-consuming that, at times, no-showing was preferable.

  1. 3.

    Missing or cryptic information about clinic logistics

Patients almost universally complained about the inclusion of indecipherable and impenetrable clinic names in mailed appointment reminders. Appointment reminders included clinic names that appeared as a code, rather than simply stating where and with whom the appointment is. One patient succinctly noted: “I’m not in the military anymore. I don’t speak abbreviations” (Veronica). Confusion from this was most apparent for patients with appointments at the larger medical center sites where there were multiple departments and buildings.

  1. 4.

    Reminder fatigue

Patients described receiving multiple appointment reminders from up to five different sources or modalities (i.e., letter, postcard, telephone call, text message, email/secure message). Patients who are regular users of the VA receive a monthly barrage of information which can lead them to discard mailed appointment reminders. Patients who received reminders as text messages commented on the additional redundancy of multiple texts for the same appointment. Making matters even worse, reminders did not take in account whether a patient has already received or responded to a reminder. Taken together, these circumstances led to patients feeling that repeated appointment reminders were excessive and superfluous.

  1. II.

    Strategies used by patients that maximize usability of appointment reminders

Veterans who were successful at keeping appointments often reported using specific strategies, techniques, and tools when receiving appointment reminders. We identified four strategies: (1) using a calendar; (2) heightening visibility; (3) piggybacking; and (4) combining strategies.

  1. 1.

    Using a calendar

The most common strategy, by far, was transferring information from the appointment reminder to an electronic or paper calendar. One patient commented, “I read them usually as soon as I get it. Then I put it on my calendar… I have a calendar on the wall” (Carl).

  1. 2.

    Heightening visibility

Another common strategy was to heighten the visibility of an appointment reminder. Patients frequently highlighted information on appointment reminder letters or put appointment reminder information in a predictable location encountered on a daily basis. Examples of locations included a white board (Kyle), the refrigerator (Patrice), a doorway (Zoe), and the front seat of the car (Shannon).

  1. 3.

    Piggybacking

A smaller number of patients used “piggybacking,” a strategy in which a person links or associates a new behavior with another established behavior (i.e., habit).21 Shannon, for instance, described putting appointment reminders by the coffee pot since she uses it every morning.

  1. 4.

    Combining strategies

The final technique we identified was combining strategies. Doug explained, “I write all my appointments down. Then after… I put that in my file basket… And if it’s important… after I write it in, I’ll highlight it. So it just kind of sticks out. Because I’m color blind, too.” Larry added, “…You’ve got to have a book like mine, and I’ve still got a calendar and then I get letters, and between the three, I’m fine.”

  1. III.

    Recommendations for healthcare systems to enhance appointment reminders

Six recommendations based on patients’ experiences with current appointment reminders were developed, of which four relate to content, one timing, and one mode of delivery (see Fig. 1). The following quotations contextualize each recommendation.

Figure 1
figure 1

Summary of patient-centered recommendations for enhancing appointment reminders.

Full size image
  1. 1.

    Mix up their content and format

Veterans noted that they are likely to tune out, or even ignore, appointment reminders that appear the same, and that varying appointment reminders might prompt them to examine the content more closely. This effect might be achieved by adding new content on a rotating basis, altering the length, or otherwise changing the appearance or format of reminders.

  1. 2.

    Keep them short and simple

Rather than having to read through a lengthy letter and hunt for appointment information, some patients often advocated for simplicity. In terms of key content, Tony suggested focusing on important information such as appointment location, time, date, and how to cancel. However, not everyone agreed with this approach: “If it was longer, I would read it more in depth, because I would be like—that doesn’t look normal…. I would be like—whoa this looks different than the last one.” (Leann).

  1. 3.

    Add a personal touch

While patients were familiar with receiving very direct communication from the military, most preferred appointment reminders that were more personable and “human” (Patrice). They generally liked reminders that struck the tone of a request rather than a demand. Selectively (not excessively) using words such as “please” and “you” would accomplish this without sounding like “begging” (Carl).

  1. 4.

    Include specifics on clinic location and contact information

Including detail about the location and contact information for a clinic was viewed as vital. Some appointment reminders (particularly for new patient appointments) could be improved by including information such as a map, the precise location of a clinic within a hospital, and a specific phone extension for a clinic.

  1. 5.

    Time reminders based on the mode of delivery

Patients often described a preference for appointment reminder letters and postcards to be delivered further in advance of an appointment—typically about 1 week—than other types of reminders. This was because these reminders were often most useful to help confirm details of an appointment and trigger any advance planning necessary for the appointment. In contrast, text messages—typically received 1 or 2 days before the appointment—often served more literally as a reminder for an appointment.

  1. 6.

    Hand over control of reminders to patients

Patients want a say in the types of reminders they receive. They described different functions that reminders serve depending on how they are reminded, and they valued receiving more than one type of appointment reminder (e.g., text message and mailed letter). Multiple reminder modalities may be especially beneficial in subgroups of patients at higher risk of no-showing.

However, patients also emphasized the danger of reminder fatigue. Having a clear opt-out or opt-in system would allow patients to decide what types of reminders work well for them. Leann noted, “I wish I could opt out of the letter. Because I think it’s a waste of paper, personally, for me. Paper, postage, for the government to spend….” A related preference was for improved responsiveness to patient feedback, especially with respect to text message reminders that employed two-way communication. Connor suggested not sending additional reminders once a patient has confirmed an appointment: “Or you might even have the ability—another option is, no further texts. I’m coming, no further texts….” Conversely, an additional reminder could be provided when a patient has not confirmed an appointment.

DISCUSSION

Our findings revealed that current appointment reminders are limited in terms of their usability for patients. Perhaps as a consequence of this, patients have devised strategies to make the best of these reminders. There are certain patient behaviors that can overcome some of these limitations, and we have described several of these patient behaviors in our results related to strategies to optimize the usability of reminders. In addition to patient behaviors, there are also actions healthcare systems can take to enhance the usability of reminders. Informed by patients’ experiences, we identified six recommendations that build off the existing framework for appointment reminders while further optimizing their effectiveness. These recommendations include four suggestions related to reminder content, one related to timing, and one related to delivery mode (Fig. 1).

When considering the complexity of large integrated healthcare systems, compared with interventions that require developing new systems and processes, many of our recommendations are relatively amenable to implementation, such as changing the content contained in existing reminders. In addition, our findings suggest other opportunities to enhance reminder systems worth exploring include simplifying telephone systems to make it easier to reach scheduling staff and offering skill trainings for patients with a history of repeated no-shows to more effectively manage their appointments.

Several of our findings are strengthened by their alignment with existing behavior change theory and practice. For instance, the strategy of heightening the visibility of an appointment reminder is similar to the concept of salience, the degree to which something garners a person’s attention.22 Also, patients who put their appointment reminders in a place they encounter every day are tapping into the behavior change strategy known as piggybacking.21 Some of the recommendations, such as “keep it simple” and “mix it up,” parallel suggestions in the realm of “sticky” ideas and learning.23

Our study has several limitations. First, we only interviewed 27 veterans in the VA’s integrated healthcare system in one region of the USA, and thus our findings may not be generalizable to a more general population. Second, our results focused on patients’ experiences with outpatient clinic appointments for which they receive printed appointment reminder letters. Reminders designed for other types of healthcare services—particularly distinctive, “one-time” services such as a surgery or diagnostic procedure—may require separate consideration. Additionally, implementation of our recommendations requires developing a nuanced understanding of the barriers and facilitators to changing appointment reminders, particularly when considering the complexity of large integrated healthcare systems. Finally, randomized controlled trials (RCTs) are needed to definitively determine whether these recommendations translate into reduced improved outcomes such as fewer no-shows. Some of our recommendations (e.g., timing of reminder phone calls) have been examined in RCTs.24

Notwithstanding these limitations, it is clear that a tremendous amount of effort goes into distributing appointment reminders, and our results suggest that poorly received reminders may actually be counterproductive. Applying user feedback into reminder design seems a promising, low-cost start towards increasing patient satisfaction. Implementation studies of enhanced appointment reminders would seem an important next step in research, particularly considering how commonly employed appointment reminders are in healthcare systems.25 The value of improving patient satisfaction is worth underscoring in an environment where patients—in VA as well as other healthcare systems—have choices for where they get their care.

In conclusion, this study builds upon the existing literature by describing four limitations of current appointment reminders and providing six patient-informed recommendations for improving appointment reminders. As healthcare systems struggle to optimize clinic access and minimize no-shows, these insights can help design improved systems of care that are more efficient and meet the needs that patients themselves identify.