Introduction

Before the Covid-19 pandemic, the two most recent mass vaccination campaigns in the United States of America occurred during the 1976 swine flu outbreak and 2009 swine flu, and the operation of these initiatives relied on manual tasks and paper [1,2,3,4,5,6,7].

Immunization Information Systems (IIS) have been under development since 1997 by the Centers for Disease Control and Prevention (CDC), the National Vaccination Advisory Committee, and immunization program grantees to collect and consolidate population-based, data on vaccine administration for a geopolitical area and sent to the CDC [8]. Half of the vaccination campaigns in 2009 recorded vaccinations into the IIS [9,10,11].

In light of the pressing requirements of the current pandemic, clinical care sites exchange data with IIS through electronic interfaces established with their electronic health record (EHR) [12, 13] Organizations and companies have stepped up to provide solutions where public health systems have lacked the required functionality. Vaxigo (AZOVA, Inc., Alpine, Utah) provides mass vaccination services for schools and businesses, as well as a vaccination location finder for private citizens, leveraging especially their relationships with pharmacies and pharmacy networks. The Multi-state Partnership for Prevention, a non-profit organization, offers its PrepMod pandemic response management system to automate the registration, scheduling, and reporting needs of mass vaccination clinics. Still, other companies like Vacmobile (Vacmobile Corporation, Alpharetta, Georgia) and Healthvana (Healthvana, Inc., Los Angeles, California) offer patient-facing immunization tracking personal health records (PHR) applications that are pivoting to support Covid-19 immunization needs alongside existing routine vaccination services. In many cases, these systems can report doses administered to IIS and retrieve patient immunization histories.

To respond to the vaccination efforts, the California Department of Public Health (CDPH) created My Turn [14], a mobile cloud computing backend and Web 2.0 web-based patient portal to allow patients to determine eligibility, register, and schedule an appointment on their own [15, 16] and report data to the IIS [17]. My Turn was implemented by Accenture (Accenture plc, Dublin, Republic of Ireland) with cloud-based software solutions provided by Salesforce (Salesforce.com, Inc., San Francisco, California) and Skedulo (Skedulo Holdings, Inc., San Francisco, California) [18]. Despite the use of the most current technology, vaccine administrators and their assistants still used manual tasks and paper as part of the vaccination process. Some of these manual tasks could be replaced by leveraging the existing My Turn platform and reducing the use of paper in light of the ongoing need for vaccine boosters and labor shortages in healthcare.

Objective

In this article, we describe the use of My Turn at two points of dispensing in California and comment on process improvements to tasks of six identified processes of vaccination—registration, scheduling, administration, documentation, follow-up, and digital vaccine record–to minimize the number of manual tasks of vaccine administrators and their assistants.

These manual tasks are more accurately performed and better suited to the technology of My Turn. Vaccine administrators and their assistants are highly trained for clinical decision-making and their time and attention are better spent on tasks that genuinely require their expertise.

Methods

We identified the manual tasks of six identified processes of vaccination–registration, scheduling, administration, documentation, follow-up, and digital vaccine record. To analyze these processes, we reviewed publicly available documents of My Turn primarily from the CDPH Immunization Branch [19] and EZIZ, the website for the California Vaccines for Children Program, and videos on YouTube [20].

To examine documentation–the Covid-19 vaccination record cards provided by the Centers for Disease Control and Prevention (CDC) provided to patients–we searched Instagram using the hashtags “#mosconecenter” and “#oaklandcoliseum” and Google using the search criteria “vaccine cards”. The eligibility criteria included posts on the front of cards from patients who received either the first or second dose of one of the three Covid-19 vaccines with Emergency Use Authorization (EUA): Pfizer-BioNTech, Moderna, or Janssen. To meet the criteria, the record cards required each of the elements: last name, first name, date of birth, manufacturer, lot number, date (of vaccination), and healthcare professional or clinic site. Markings, such as from a pen, stamp, or printing from a label, for each element must be visible but not necessarily legible. Some patients intentionally obscured these elements to protect their privacy. Three of the patients were contacted on Instagram to further discuss their experience with vaccination, but none of them responded. None of the patients were paid or compensated with in-kind donations for sharing their health records on the Internet. This study examined publicly available information, and no personally identifiable information about subjects or interventions was included in this paper. No written informed consent, therefore, was obtained.

Results

At the Moscone Center, 329,608 vaccines doses were given [21,22,23,24,25]. At CVC, more than 500,000 vaccine doses were administered [26,27,28].

Our analysis of My Turn relied primarily on YouTube videos from the CDPH, and three documents from EZIZ. Nineteen cards from George R. Moscone Convention Center (Moscone Center) in San Francisco and twenty-six cards from Oakland Coliseum Community Vaccination Clinic (CVC) were reviewed on Instagram and Google. Four patient record cards from each site met eligibility criteria and were included for analysis of documentation. Cards were eliminated for missing elements or markings that could not be visualized. None of the three contacted patients were among the four included for analysis because some or all of the markings on their cards could not be seen.

Registration

At the two mass vaccination points of dispensing (POD), patients registered on their own by visiting My Turn website (https://myturn.ca.gov/) before receiving their vaccination. Alternatively, patients called to speak with a vaccine administrator assistant, who would use My Turn [29]. Patients who wanted to receive their vaccine at CVC could walk in to register [30, 31]. Figure 1 shows the sign-up page of My Turn.

Fig. 1
figure 1

My Turn sign-up page [32]

Full size image

The patient provided information such as name, date of birth, gender, email address, home number, among others during the registration [33] and reviewed questions that could be a contraindication to receive a vaccine [32, 34]. Figure 2 shows a checklist of My Turn.

Fig. 2
figure 2

My Turn checklist [32]

Full size image

Scheduling

The checklist used by My Turn was designed to suggest rescheduling an appointment if there were contraindications [32]. If there were none, the patient chose to schedule one or both appointments. The patient chooses the vaccine manufacturer, and My Turn automatically defaulted to the manufacturer’s recommended time for the second dose. My Turn did not allow the booking of a second appointment for the Janssen vaccine, which only required a single dose [35]. The patient was sent a text message and email from My Turn, and Fig. 3 shows a sample text message sent to the patient after registering and scheduling an appointment [34, 36].

Fig. 3
figure 3

Sample text message sent to the patient after registering and scheduling an appointment [32]

Full size image

Administration

My Turn used myCAvax for accessing the patient’s record [37]. When the patient arrived at the POD such as Moscone Center and CVC, vaccine administrator assistants used myCAvax to search for the patient’s appointment by typing their appointment number or patient name [32]. The patient was then verified, screened, and checked in with myCAvax [36].

For verification, patients were required to produce documentation with their names, and health insurance was reviewed [32, 38]. Vaccine administrator assistants reviewed a checklist for changes to the patient’s health that may require postponing the appointment, and the patient was asked for verbal consent to receive the vaccine [32]. Once each of these required elements was reviewed, vaccine administrators or their assistants toggled a button on the screen to check in the patient [32]. Figure 4 shows the verbal consent form of myCAvax.

Fig. 4
figure 4

myCAvax verbal consent [32]

Full size image

The vaccine details collected by myCAvax include the items and methods and actions taken by the vaccine administrators or their assistants in Fig. 5.

Fig. 5
figure 5

Screenshot of documentation into myCAvax [32]

Full size image

After the vaccination administration at Moscone Center, the vaccine administrators or their assistants wrote on paper the time the patient could leave after being monitored for adverse effects [39].

Documentation

Based on the review of four front sides of record cards at each POD, vaccine administrators or their assistants used a pen to write the last name, first name, and date of birth of the patient onto the record card supplied by the CDC. The manufacturer, lot number, and clinic site were preprinted on a sticker that was affixed to the card. For CVC, the date of vaccination was included on the same preprinted sticker. At Moscone Center, the date of vaccination was imprinted with a rubber stamp. For the second dose for one card at CVC, the manufacturer, lot number, date of vaccination, and clinic site were not printed on the sticker and filled in with a pen.

Follow-up

The follow-up appointment could be booked at the same time as the registration in the online patient portal, and the patient received another text message and email from My Turn. There were several reasons why the patient could not sign-up for the appointment at the same time as registration including no availability and appointments booked at walk-in and single-dose POD [32].

The reverse side of the record card was reserved to write the date and time for the second dose of the vaccine if needed.

Digital vaccine record

After the vaccination, the patient could optionally sign-up to receive a Digital Covid-19 Vaccine Record, which reproduces the data of the record card provided by the CDC on a mobile device such as a smartphone [19] and is shown in Fig. 6.

Fig. 6
figure 6

Sign-up page for the Digital COVID-19 Vaccine Record [40]

Full size image

Discussion

Process improvements to reduce manual tasks of six identified processes of vaccination—registration, scheduling, administration, documentation, follow-up, and digital vaccine record–and paper. My Turn was built de novo using mobile cloud computing and a Web 2.0 web-based patient portal without relying on existing health records (EHR) and had most of the necessary information to eliminate several manual tasks and paper.

Registration

The registration of patients with a Web 2.0 web-based patient portal relies on patient-supplied content [8]. There were several benefits of this system [8]. First, because the health records of patients in America are fragmented [8], placing the burden on the patient to input information could reduce the tasks and save the time for vaccine administrators and their assistants. Second, vaccine administrators and their assistants have many tasks to perform including verifying the information provided by the patient. Third, once verified, the data entered into My Turn could be seamlessly reused for other processes throughout the vaccination process.

Scheduling

A web-based patient portal also supported patients who have been prioritized to receive the vaccine. Because the initial supply of the vaccine was less than the demand [41], only patients who have been authorized to receive the vaccine were allowed to register. For instance, they were given an access code to be used in My Turn and prevent unauthorized patients from using the site. Some access codes, however, were shared by text message, social media, and other means and allowed anyone with the code to book an appointment. The codes were not unique and did not expire after being used and permitted multiple appointments to be created from one code [42]. In this case, each access code should expire or only allow one patient to use it.

Finally, the web-based patient portal may have reduced crowding at POD and iatrogenic spread of Covid-19 where communally used items such as paper and pens may be shared [43]. It may have also reduced medical emergencies such as anxiety and syncope from waiting in line to register for an appointment [2, 44].

My Turn screened patients with an algorithm to review eligibility, contraindications, and precautions. Any reason why the patient should not proceed any further in the process could be flagged for review by a health care professional [43]. Otherwise, the patient could schedule the necessary appointments. My Turn also allowed two appointments to be scheduled at registration for vaccines that required two doses and automatically sent reminders by text message [45,46,47] and email [48]. This process likely increased the rates of dose-series completion as it has for influenza, increased vaccination efficacy, and minimized adverse drug events. The most commonly reported errors in a systematic review of vaccinations were the wrong vaccine administered and off-schedule administration types [49].

Administration

The process of checking in the patient required the vaccine administrator assistant to search for the patient by either appointment number or name. This required typing in a search field of myCAvax. The patient may not be found or the incorrect patient selected if details are entered incorrectly, or a patient with the same or similar name may be entered inadvertently. My Turn has the capability of generating a QR Code that can be sent by email as shown in Fig. 7 and be scanned from a mobile device to open the patient record unambiguously. Prior studies have demonstrated a decrease in documentation from 86 to 26 s, or 60 s, while concurrently increasing accuracy from 95 to 100% [50].

Fig. 7
figure 7

QR code sent by email to the patient [32]

Full size image

Consent is taken verbally at the end of the registration appointment and marked when the vaccine administrator assistant checks a box in myCAvax [32, 51]. This information could be provided during the registration process when the patient has the opportunity to consider the reasons to receive the vaccine and confirmed again to ensure that the patient understands the risks and benefits of the vaccine. If there is no change, no further actions need to be taken by the vaccine administrator assistant. To be sure, there is no Federal requirement for informed consent for immunization, but state and local regulations may differ [52].

Scanning the barcode of the patient’s QR code on a mobile device can accurately record or confirm the necessary information such as name. Similarly, presenting the barcode of the vaccine administrators or their assistant can enter their names and credentials.

Moreover, data collected by the patient at registration can be automatically entered into the EHR such as My Turn. The time and date of the vaccine can be automatically entered by Network Time Protocol (NTP) server, which is a source of time [53].

The CDC mandated that two-dimensional (2D) barcodes that contain the National Drug Code (NDC), lot number, and a placeholder expiration date of 12/31/9999 be placed on the carton [41]. This barcode can be read with a scanner to rapidly, accurately, and automatically document the vaccine administration including the lot number, manufacturer, and expiration date in the EHR and data supplied to the patient [41].

One of the twenty-one elements–the site of the inoculation–can be automatically entered by defaulting the value to LA, or left arm. The left arm is the most common site for vaccinations [54]. The record can be manually corrected by the vaccine administrator assistant if another location is used. Even if this information is recorded inaccurately, this type of error is unlikely to result in demonstrable harm to the patient, and expediting patient receipt of a Covid-19 vaccine may be worth these drawbacks. The alternative method used at Moscone Center and CVC by manually entering the value for this field is also prone to error if the wrong value is selected.

The patient could be informed to leave by sending a text message or email in a manner similar to the appointment reminder. The paper that was given at Moscone is designed to serve as a reminder rather than a way to prevent the patient from leaving prematurely and a text message or email could serve this same purpose.

Documentation

To increase efficiency during documentation, twenty of the twenty-one data elements required can be entered automatically. Figure 5 showed the method that data was entered into myCAvax. Figure 8 summarizes the elements for documenting vaccines used by the two PODs with suggested methods of documentation.

Fig. 8
figure 8

Required vaccination administration record elements, their data source, POD, and suggested methods for documenting [41]

Full size image

Follow-up

Accuracy was also likely enhanced because My Turn automatically calculated the day that the patient should return for the second dose for each vaccine manufacturer. The time, however, defaulted to 12 am when most PODs may not be open and selecting the time may have led to errors. Appointment errors were more likely for patients who chose a single dose or walked in a POD that may not have offered an appointment for their second dose of the vaccine made by Pfizer-BioNTech or Moderna [36].

To complete the documentation, the CDC required that patients receive record cards [55]. At Moscone Center and CVC, they were filled out by various means including by hand with a pen, stamp, or pre-printed label affixed to the record card and by. In California, the Digital Covid-19 Vaccine Record was implemented [56].

Digital vaccine record

Data collected by My Turn has the necessary data to generate the Digital Covid-19 Vaccine Record and could be emailed or texted, and stored on a mobile device [57]. Specifically, the data on the Digital Covid-19 Vaccine Record includes the patient’s name, date of birth, vaccine manufacturer, and dates of vaccination. It also consists of a QR code that can be read by a SMART Health Card reader such as a SMART Health Card Verifier App (The Commons Project Foundation, New York, New York) operating on a mobile device. Once scanned, the app can display the information that can be read by the operator and verify that it is valid and issued by a trusted source [58]. Guests at bars, restaurants, clubs, gyms, and large indoor events in San Francisco, at the time of this writing, require proof of vaccination, and the Digital Covid-19 Vaccine Record shown on a mobile device is one form of proof [59].

Limitations

One limitation of My Turn is cybersecurity concerns with the use of mobile cloud computing since data, particularly large population-wide medical information, can be accessed remotely at any time [60]. CDPH and its vendors reportedly use best practices for security and privacy [61,62,63] and may be vulnerable to breaches, surveillance, and misuse [64].

Conclusion

The use of mobile cloud computing and Web 2.0 can be used to assist future releases.