Introduction

Inflammatory bowel disease (IBD) is a chronic idiopathic inflammatory disease of the gastrointestinal tract. Recent advances in the treatment of IBD include the use of immune modifiers and monoclonal antibodies such as tumor necrosis factor (TNF) alpha inhibitors, anti-integrin agents, interleukin-12/23 inhibitors and janus kinase (JAK) inhibitors, presently being introduced early in the disease course, or as combination therapy with immunomodulators in subsets of patients [1,2,3,4]. These agents achieve higher rates of clinical remission and mucosal healing than conventional therapy [1,2,3,4]. However, these therapies increase the risk of infections, including some vaccine-preventable diseases (VPD) [5,6,7,8,9].

Patients with IBD, regardless of immunosuppression, are at increased risk for VPD including influenza, pneumonia and herpes zoster when compared to the general population [5, 7, 10,11,12,13]. To decrease the risk of these VPDs, preventative care guidelines emphasize the importance of obtaining a vaccine history and immunizing at-risk patients with IBD [14]. It has been shown that patients with IBD are less likely to be up to date with recommended vaccinations compared to patients without IBD [15,16,17,18,19]. Gastroenterologists should share the responsibility for providing or recommending administration of age-appropriate vaccination as recommended in the adult immunization schedule by the Advisory Committee on Immunization Practices (ACIP) [20].

As vaccination status is often difficult to determine, many clinicians use self-report when determining vaccination status or assessing vaccine history in elderly or hospitalized patients [21, 22]. When previously evaluated in general populations, self-report assessment of influenza vaccination is highly sensitive and specific for patients’ true vaccination statuses [23,24,25]. Pneumococcal vaccination self-report had slightly lower sensitivity and specificity compared to influenza vaccination but has also been found to be accurate for general populations [23,24,25]. Survey-based studies conducted among patients with IBD used self-report to assess vaccination rates as well as effectiveness in preventing illness, but in these studies, actual vaccination status was not confirmed [12, 26,27,28].

In the USA, influenza and other vaccines are available at several sites, including clinics, pharmacies, public health departments and places of employment. Self-report may be the simplest solution to obtain information on a patient’s vaccination status. Although several studies of self-report have been conducted, no other prior studies have evaluated the accuracy of self-reported vaccination status for patients with IBD. Accurate information on vaccination history is important to determine appropriate clinical vaccination status and needs, as well as to minimize exposure misclassification and bias when measuring vaccine effectiveness. The primary aim of our study was to determine the accuracy of self-reported influenza vaccination status in a cohort of patients with IBD. A secondary outcome was evaluating receipt of any pneumococcal vaccine, and calculating sensitivity and specificity of self-report of pneumococcal vaccination status. An exploratory aim was evaluating the accuracy of self-reported history of pediatric live vaccines, which were varicella and measles, mumps and rubella (MMR).

Materials and Methods

Study Setting

We prospectively enrolled a cohort that consisted of a convenience sample of 200 consecutive individuals seen in our outpatient gastroenterology office between January 28th, 2019, and August 26th, 2019, at the University of Wisconsin Digestive Health Center, in Madison, WI. The study met the requirements for quality improvement as determined by the University of Wisconsin-Madison and was therefore deemed exempt from Institutional Review Board review.

Study Population and Design

All patients with IBD (Crohn’s disease, ulcerative colitis, indeterminate colitis) presenting for clinic appointments were eligible to participate. We excluded patients who were < 18 years old, who did not have an active Wisconsin Immunization Registry (WIR) record, or who were unable to answer the immunization questions. All participants completed a four question survey (“Appendix 1”). Influenza immunization status was determined in all patients, pneumococcal immunization status was determined only in those meeting criteria for pneumococcal immunizations, and childhood immunization status was only determined in those who completed the MMR or varicella vaccine series.

Pneumococcal vaccine status was simplified to include any pneumococcal vaccine (pneumococcal conjugate vaccine (PCV13) or pneumococcal polysaccharide vaccine (PPSV23)) by self-report and any pneumococcal vaccine recorded in WIR. Only participants who were age 65 years or older, prescribed an anti-integrin inhibitor (vedolizumab), or on maintenance therapy with an immunosuppressive medication at the time of the study were included for question 2 about pneumococcal vaccination. Immunosuppressive medications included steroid therapy (prednisone or budesonide), immunomodulators (azathioprine, 6-mercaptopurine, methotrexate), anti-TNF therapy (adalimumab, golimumab, certolizumab and infliximab), anti-IL12/23 (ustekinumab) or tofacitinib [14]. Only participants with a WIR record of receiving the MMR or varicella vaccination in childhood were included in the results for questions 3 and 4. MMR and varicella vaccine status was recorded as yes or no and as present or absent in the WIR. Patient demographics, IBD history and treatment history were collected from the electronic medical record (EMR).

Wisconsin Immunization Registry (WIR)

The WIR is a state-wide computerized Internet database that was developed to record and track immunization records of Wisconsin residents. The WIR is provided by the Department of Health and Family Services of the State of Wisconsin and has been available since May 2000 [29]. Immunization history was backloaded from January 1995. Immunizations provided by both public and private providers in Wisconsin are uploaded into the registry, and 98.5% of Wisconsin adults having an active WIR [30]. Studies have demonstrated that the WIR captures 97% of vaccines administered in Wisconsin [31]. The WIR is directly incorporated into the University of Wisconsin Hospital and Clinics EMR.

Outcomes

The primary outcome of this study was to determine the accuracy of self-reported influenza vaccination status. Secondary outcomes were the accuracy of self-reported pneumococcal vaccination. An exploratory aim was evaluating the accuracy of receiving MMR or varicella vaccine series among participants documented to have received these vaccinations.

Statistical Analyses

Using the WIR as the gold standard, we calculated the sensitivity, specificity, positive predictive value and negative predictive value of self-reported vaccination status. Descriptive statistics were used to describe patient demographic characteristics, and the sensitivity and specificity were calculated using tabular forms or data listings and frequency tables. Statistical analysis was performed using STATA software version 15 (Stata Company, College Station, TX).

Results

Total enrollment in the study was 200 persons, mean age 41 ± 15.5 years (Table 1), all with WIR documentation present. Almost two-thirds of the sample population had Crohn’s disease. The majority of the cohort (77%) was on immunosuppressive therapy, with 46 patients prescribed only mesalamine or no maintenance therapy. In the study population, 74.5% of patients with IBD received the influenza vaccine based on WIR data. Overall, the accuracy of self-reported influenza vaccination was 96.5% (Table 2). The area under the receiver operating curve showed excellent discrimination at 94% for self-report of influenza vaccination (“Appendix 2”).

Table 1 Baseline demographics
Full size table
Table 2 Validity of self-reported influenza and pneumococcal vaccination status
Full size table

Analysis of pneumococcal vaccination status was limited to 154 immunosuppressed patients, as well as 5 patients who were not immunosuppressed, but were 65 years or older. Within this cohort, 3 patients did not enter a response regarding pneumococcal vaccination status, and these responses were not used for results calculations. Overall, 156 patients were included in the analysis. Of patients that answered the question regarding pneumococcal vaccination, and who were eligible to receive the vaccine, 81.4% were vaccinated. Of patients that had received a pneumococcal vaccination, 13 patients had received PCV13, 10 had received PPSV23, and 98 had received both vaccinations. Respondents were 84.0% accurate in their response regarding pneumococcal vaccination status. Among this cohort, self-report of pneumococcal vaccination status had a sensitivity of 83.5% and was 86.2% specific (Table 2). Mis-reporting pneumococcal vaccination status occurred in 25 respondents. Of those respondents, the average age was 37.7 years old, 100% were Caucasian, and 72% were male. 92% were diagnosed with Crohn’s disease, and 64% had primary care providers outside of the UW Health system.

Analysis of survey data for MMR and varicella vaccines was limited to patients with documented MMR or varicella vaccination administration present within the WIR. The assessment of MMR included a total of 85 respondents. Of these patients, 5 were noted to have a response of “Unsure”, which was counted as incorrect. Of the remaining 80, all respondents correctly identified vaccination status, leading to a calculated sensitivity of 94.1%. Varicella vaccination status assessment was managed similarly, with results limited to patients with documented vaccination against varicella present within the WIR. Thirty-three patients were within this cohort. Although 2 responded “unsure”, the remaining 31 correctly reported their vaccination statuses. Sensitivity was also calculated at 93.9%.

Discussion

To our knowledge, this is the first study to evaluate the accuracy of self-reported vaccination status among patients with IBD verified with immunization records. We found that the self-report of vaccination status among patients with IBD was a highly sensitive and specific representation of true vaccination status. When previously evaluated in general populations, self-report vaccination survey assessment has been shown to range in sensitivity for influenza vaccination status between 93.0 and 100%, and specificity between 38 and 90.3%, for a patient’s true vaccination status [23,24,25, 32, 33]. Pneumococcal vaccination in general populations had slightly lower sensitivity, ranging from 64.4 to 97%, with specificity ranging between 46 and 92.1% [23,24,25]. Many centers use self-report when assessing vaccination status and vaccination effectiveness. Determination of vaccination status is important to limit under-immunization, as well as reduce episodes of unnecessary re-vaccination in patients. Repeated doses of influenza vaccine in the same season are well tolerated, with pain at injection site the most frequent side effect in about 9% of studied participants [34, 35], however, it is unnecessary and increases healthcare costs. In a cohort study of patients receiving the pneumococcal polysaccharide vaccination, 1% of first-time vaccinees had severe arm restriction where they were unable to raise their arm above their heads compared to 5% of re-vaccinees [36]. In a US trial of re-vaccination, conducted in patients receiving pneumococcal polysaccharide vaccinations, 10% of patients developed severe erythema, similar in effect to cellulitis of the arm [37]. Although pneumococcal self-report was less reliable than influenza, as patients with IBD are candidates for both PCV13 and PPSC23, an immunization registry is an important tool to determine which pneumococcal vaccine was administered, as well as monitoring immunosuppressed patients who will need PPSV23 reimmunization after 5 years. Evaluation of mis-reported pneumococcal vaccination status suggested that male patients with Crohn’s disease may be more likely to incorrectly identify vaccination status, but further analysis is necessary. Our results support that patients with IBD have similar sensitivity and specificity of self-report to previously evaluated populations. Providers caring for patients with IBD should use medical records and immunization registries if they are available, bit in their absence, immunization self-report is reliable.

Our IBD population overall had high rates of immunization with 74.5% of study participants up to date with recommended influenza vaccination, and nearly 80% of participants had received a pneumococcal vaccine. This finding is much higher compared to our previous reports and published literature, including Xu et al., that found that the studied patients with IBD had influenza vaccination rates of 48.4%, with 75.3% of participants receiving a pneumococcal vaccination [17, 19]. We attribute these data to our proactive approach to vaccinating patients with IBD managed in our practice with standard protocols to administer indicated vaccines during patient rooming, and high rates of communication of the importance of vaccination through our EMR patient portal. With 85.5% of our participants active on an online health system communication portal, they may have received immunization reminders, which has been shown to increase vaccine uptake in patients with IBD [38, 39]. Uncertainty regarding who is responsible for providing necessary vaccinations, which patients require vaccination, and patient and provider misconceptions regarding safety of vaccinations in immunosuppressed patients may contribute to suboptimal immunization rates [27, 40, 41]. Guidelines emphasize the importance of a multidisciplinary team approach, including primary care and gastroenterology providers, to improve vaccination coverage in patients with IBD [14, 42]. Provider confidence in the accuracy of self-reported vaccination status will hopefully improve immunization rates in patients with IBD.

This study was possible only with the existence of a reliable immunization information systems (IIS), the WIR, which was used as the gold standard and allowed for validation of patient responses. The Wisconsin adult population participation in the state’s IIS is at 98.5%, compared to 56% nationally per the CDC [30]. The entries into the WIR have been validated [31]. IIS cover the USA, but may be infrequently used by providers, with reasons cited as unaware that system existed within their state, lack of interoperability between EHR and IIS, and use of IIS is not mandated or standard of practice [43]. IIS use has demonstrated the ability to improve vaccination rates within the general population [44].

Overall, the strengths of this study include a large sample size, and the use of a gold standard immunization registry to confirm patient responses. There are several limitations of our study. A vast majority of the population sampled self-identified as Caucasian. Although likely representative of the local community, this may limit the generalizability of our results. Little other demographic and socioeconomic information was collected, as the goal of this study was to represent real-world practices. Integrated workflows present within the clinical system at UW Health allow medical assistants to administer vaccinations in clinic if desired by patients. If patients have hesitancy regarding immunization, physicians then can discuss concerns with patients. Although this likely leads to increased accessibility of vaccinations in the population studied, it likely represents a best-case scenario for evaluation of vaccination status, and likely is not representative of all clinical practice settings. Our study did not consider self-reported dates of vaccine administration. This issue is only important for the pneumococcal vaccine regimen where pneumococcal polysaccharide vaccine administration is recommended at least 8 weeks after the pneumococcal conjugate vaccine in immunosuppressed individuals. Additionally, immunosuppressed individuals should receive a second dose of the pneumococcal polysaccharide vaccine after a 5-year interval [45]. Our study focused on whether any pneumococcal vaccination had been administered, rather than the completeness of all recommended pneumococcal vaccinations, and further studies will be necessary to assess accuracy of self-report for completion of vaccine series. Due to the small sample size, we are unable to comment on accurate sensitivity of self-report of MMR and varicella.

In summary, these findings highlight the reliability of self-report of vaccination status in patients with IBD, as well as the benefit of a robust immunization registry and the advantages of ensuring appropriate vaccination.

Conclusion

These findings highlight the importance of longitudinal documentation of vaccination status for IBD patients, who are at increased risk for VPDs, and suggests that self-report is a sensitive and specific means of assessing vaccination status within the IBD patient population. The COVID-19 pandemic has increased the number of video and telephone visits and decreased the opportunity to immunize patients in the clinic. Thus, providers should consider asking for self-report of influenza and pneumococcal immunization to ensure that patients are protected against VPDs. Future studies are necessary to assess self-report reliability with other vaccines including MMR, varicella, tetanus-diphtheria with or without pertussis and herpes zoster.