Introduction

Oncologists are prescribing immune checkpoint inhibitors (ICIs) more frequently for a larger breadth of cancer diagnoses, including nivolumab for melanoma and pembrolizumab for lung cancer [1].This likely corresponds to a higher incidence of immune-related adverse events (irAEs) given consistent frequencies of irAEs across trials [2]. Any organ in the body could potentially be affected by irAEs, which include enteritis, colitis, thyroiditis, hypophysitis, dermatitis, and hepatitis [3]. A single-center descriptive report noted an irAE incidence of 34% in immunotherapy clinical trials, with the most common irAEs being rash (dermatitis), hormonal (hypophysitis), elevated liver function tests (hepatitis), and diarrhea (enterocolitis) [4]. Generally, cytotoxic T-lymphocyte-associated antigen-4 (CTLA-4) inhibitors such as ipilimumab have a higher irAE incidence (60–70%) than programed cell death protein 1 (PD-1) inhibitors such as pembrolizumab or nivolumab (~ 40%) [4].

Many clinicians can be involved in the diagnosis and management of this broad range of presentations for irAEs, including clinicians who work in both outpatient and inpatient settings depending on the severity of the patient’s symptoms. While oncologists, rheumatologists, and other specialists treat irAEs in outpatient clinics, some oncology clinicians and hospitalists have roles in identification and management of irAEs in inpatient settings. Additionally, irAEs that require inpatient treatment are typically of higher grade and have a higher cost burden on the healthcare system [5, 6].

Given the rise in ICI use and the consequent increase in irAEs as well as the number of specialists involved in irAE diagnosis and management, several prior studies have sought to characterize current experience and confidence levels. Each of these studies identified knowledge gaps for rheumatologists regarding irAEs [7,8,9,10]. The one study that surveyed oncologists had only a 2% response rate but did show that they had higher ICI knowledge and rheumatic irAE experience than rheumatologists [7]. There is a gap in our understanding of non-rheumatologists’ comfort in identifying and managing irAEs, and this gap is relevant given these clinicians regularly take care of patients. One intervention study demonstrated the positive effects of addressing knowledge gaps through pharmacist-led education efforts: patients had lower rates of ICI discontinuation due to irAEs [11].

Improving irAE understanding among clinicians who diagnose and treat patients with irAEs through dedicated didactics can positively impact patient care. We quantified the knowledge, confidence, and experience levels of oncology and general medicine clinicians in various roles to inform the development of a future irAE curriculum.

Methods

Participants

In June and July 2022, we administered a web-based survey to all University of Chicago (UChicago)-affiliated oncology clinician-oncology fellows, attendings, nurse practitioners (NPs), and physician assistants (PAs), and all UChicago hospitalists and internal medicine (IM) residents, as well as a comprehensive list of community oncologists in Chicago. Survey invitations were sent via email, and those who had not responded received two reminder emails, spaced at roughly 1-week intervals. Survey implementation was coordinated by the UChicago Survey Lab, and survey responses were collected through Qualtrics. All survey participants had the option of receiving a $10 electronic gift card after completion of the survey.

Survey Instrument

We designed a 25-question survey that assessed knowledge, experience, confidence, and resource utilization related to irAE identification and management (Supplement A). Questions were designed with input from an irAE specialist (PR), oncologists, medical education experts, and the director of the UChicago Survey Lab. The questions were finalized after an iterative process. We created six knowledge-based multiple choice questions based on irAE literature and guidelines and informed by clinical experience. These assessed knowledge level of irAE diagnosis (i.e., myocarditis triad, risk of irAEs if preexisting autoimmune conditions, and diagnostic steps for ICI-associated colitis) and management (i.e., ICI-associated hepatitis, first-line therapy for irAEs, and ICI-related hypothyroidism). We quantified experience level with ICIs and irAEs over the past year. We determined the subspecialty referral patterns and ease of referral for patients with irAEs. We identified the main resources utilized to identify and manage patients with irAEs and the difficulty or ease of accessing resources. We assessed confidence in six aspects of irAE diagnosis and management (i.e., irAE identification, biopsy and lab timing, steroid dosing, steroid-sparing medication selection, and steroid side effect monitoring). We determined respondents’ openness to online resources and continuing medical education irAE sessions.

Statistical Analysis

Respondents were categorized as a community oncologist if not affiliated with UChicago and an oncology attending if affiliated with UChicago, including community satellite locations. Based on the recommendation of the Assistant Director of Advanced Practice, Cancer Service Line (GT), the hematology/oncology NP/PA respondents were retrospectively stratified into two categories: those who primarily treat patients with solid malignancies and those who treat hematologic malignancies. Knowledge question accuracy was calculated in two ways: 1) counting no idea: the total number of correct responses (indicated by an asterisk in Supplement A) divided by the total number of responses and 2) omitting no idea: the total number of correct responses divided by the total number of responses that were not “no idea.” Descriptive statistics were used for “no idea” response frequency as well as responses to experience, resource utilization, and confidence questions. We used Fisher’s exact tests to compare knowledge question responses between oncology physicians (attendings, fellows, or community oncologists) and non-oncology physicians or NPs/PAs (residents, hospitalists, and NPs/PAs). We used stratified simple quantile regressions to analyze the relationships between knowledge, experience, and confidence by clinician type.

Ethical Approval

The UChicago Institutional Review Board determined this study was exempt from further review as it is of minimal risk and comprised of de-identified survey data.

Results

Response Rate

There was an overall response rate of 37% (171/467). This included 51 internal medicine residents, 10 oncology fellows, 30 NPs/PAs (10 solid malignancy and 20 hematologic malignancy), 41 oncology attendings, and 11 community oncologists. Response rates were highest for UChicago oncology clinicians (55–67%), lower for UChicago internal medicine clinicians (43–46%), and lowest for community oncologists (8%) (Table 1).

Table 1 Response rates by respondent type
Full size table

Knowledge

Knowledge question accuracy was highest for oncology attendings (68%), oncology fellows (67%), and NPs/PAs who treat patients with solid malignancies (67%), and it was lowest for hospitalists (38%) and NPs/PAs who treat patients with hematologic malignancies (40%) (Table 2). Oncology attendings, oncology fellows, and community oncologists were less likely to ever respond “no idea” than medicine residents, hospitalists, and hematology/oncology NPs/PAs (23% vs 60%, p < 0.001). Knowledge questions with the lowest accuracy were those related to treatment of ICI-associated hepatitis (23%) and the risk of de novo irAEs in patients with preexisting autoimmune conditions (33%).

Table 2 Knowledge-based questions by respondent type
Full size table

Experience

Experience with ICIs and irAEs was highest for oncology attendings, solid malignancy NPs/PAs, and oncology fellows and lowest for hospitalists, hematologic malignancy NPs/PAs, and internal medicine residents (Table 3). Unadjusted quantile regression models demonstrated that higher ICI and irAE experience was associated with higher knowledge scores for oncology attendings (p = 0.02) and oncology NPs/PAs (p = 0.03). Overall, experience was predictive of knowledge (R2 = 0.15, p < 0.001). Oncology attendings were the least likely to determine that patients with irAEs required subspecialty care beyond oncology, and internal medicine residents and hospitalists were the most likely. Community oncologists found the referral process to subspecialists the most difficult.

Table 3 Prior ICI experience by respondent type
Full size table

Confidence

Overall confidence in irAE diagnosis and management was the highest for community oncologists, oncology attendings, and solid malignancy NPs/PAs and was the lowest for internal medicine residents, hospitalists, and hematologic malignancy NPs/PAs (Supplement B). Generally, clinicians were the least confident with choosing steroid-sparing medications for irAE treatment. Unadjusted quantile regression models demonstrated that higher ICI and irAE experience was associated with higher confidence for medicine residents (p = 0.026), oncology fellows (p = 0.047), and hematology/oncology NPs/PAs (p = 0.042) but not for oncology attendings (p = 0.12), hospitalists (p = 0.20), and community oncologists (p > 0.99). Overall, experience was predictive of confidence (R2 = 0.28, p < 0.001). Unadjusted quantile regression models demonstrated that higher confidence was associated with higher knowledge for medicine residents (p = 0.003), oncology attendings (p = 0.04), and oncology NPs/PAs (p = 0.03) but not for hospitalists (p = 0.2), oncology fellows (p = 0.3), and community oncologists (p = 0.5). Overall, confidence was predictive of knowledge (R2 = 0.19, p < 0.001). Both experience (p = 0.002) and confidence (p = 0.001) were predictive of knowledge in the multivariable quantile regression model (pseudo R2 = 0.13).

Resources

The most common resources utilized by the sample were colleagues (77%) and UpToDate (75%), while the least common were the Society for Immunotherapy of Cancer (SITC, 12%) and PubMed (17%). Respondents found resources on medications and dosing more difficult to access than those on work-up or treatment. While solid malignancy NPs/PAs were extremely open to online resources including online continuing medical education (CME) irAE sessions, oncology fellows and attendings were more likely to use online resources in general than to engage in irAE-specific CME sessions. Overall, most respondents were at least somewhat likely to utilize online resources including online CME irAE sessions (Supplement C).

Discussion

Clinicians caring for patients with irAEs vary in their knowledge, confidence, experience, and openness to curricular interventions based on their roles in the healthcare system. Only rarely did respondents choose “very confident” to irAE diagnosis and management questions, and all respondent types had knowledge scores below 70% on average, reflecting key knowledge gaps that need to be addressed. Curricular interventions have the potential to positively impact the treatment courses of patients on ICIs by avoiding discontinuation due to irAEs [11].

This needs assessment identifies curricular priority areas comprised of the greatest knowledge gaps and where clinicians had the lowest confidence. The two questions with the lowest percentage correct and highest “no idea” responses were related to (1) not using tumor necrosis factor-alpha inhibitors for ICI-hepatitis and (2) the risk of de novo irAEs when patients have preexisting autoimmune disease (pAID). Additionally, respondents expressed the lowest confidence level with choosing steroid-sparing medications. These gaps can be addressed through dedicated, interactive didactics that reference various guideline recommendations for steroid-sparing agents [12,13,14,15]. Currently available management guidelines are available through the American Society of Clinical Oncology (ASCO), SITC, the National Comprehensive Cancer Network (NCCN), and the European Society for Medical Oncology (ESMO) [12,13,14,15]. Patients with pAIDs such as thyroiditis, psoriasis, or rheumatoid arthritis should be counseled on their risks of developing irAEs before starting on ICIs. To accomplish this patient education, clinicians themselves must be informed regarding the risks of cancer ICI toxicities on pAID flares versus de novo irAEs. Future irAE teaching modules for clinicians should provide information on this topic and guidance for use of ICI therapy for patients with pAIDs.

In general, more experience and higher confidence were associated with higher knowledge, but this was not always the case depending on the clinician type. Variations in experience level, confidence, and knowledge did follow roughly expected trends: oncology physicians and solid malignancy NPs/PAs more commonly treat patients with ICIs and at risk for irAEs and generally had higher experience, confidence, and knowledge. On the other hand, generalists and hematologic malignancy NPs/PAs less commonly treat patients with ICIs and thus interface with irAEs with less regularity and generally had lower confidence and knowledge in our study. These three dimensions of experience, confidence, and knowledge can help inform role-specific curricular interventions to boost knowledge and confidence to role-specific responsibility levels that likely correspond to anticipated experience levels. Future curricular interventions should consider a clinician’s role in the diagnosis and management of irAEs and customize interventions to these roles accordingly. Generalists can play a key role in irAE identification with oncologists and autoimmune disease specialists playing vital roles as irAE management experts. Interactive or simulation didactics could supplement the paucity of opportunity for on-the-job experience for clinicians who only take care of occasional patients with irAEs. Such an intervention can help raise confidence and knowledge for those whose clinical work does not allow for organic growth via clinical experience.

All respondents were open to online resources. Given these preferences and likely limited clinician availability due to clinical and administrative responsibilities, future curricular interventions should consider easily-accessible, freely-available online didactics and potentially hybrid courses with in-person discussions and online resources. Virtual didactic options include online non-interactive websites with links to manuscripts and key reference material, prerecorded online presentations, educational podcasts, interactive online modules, and live interactive sessions. SITC is actively developing online resources on toxicities of cancer immunotherapy through their Advances in Cancer Immunotherapy educational series [16]. This current resource does require SITC membership and is not freely accessible to all clinicians, particularly generalists who are less likely to be members. Educational resources available for all relevant clinicians are needed.

In addition to didactic resources, the availability and feasibility of clinical teachers within a given health system are important aspects to consider. These teachers could provide experiential learning opportunities as well as “hands-on,” case-based, role-specific seminars for oncology trainees during ASCO or SITC national meetings and for generalists at American College of Physicians (ACP) or Society of Hospital Medicine (SHM) national meetings. Locally, institutions could develop objective structure clinical examinations (OSCEs) for medicine residents or oncology fellows, or educational interventions could start even further upstream at the medical student stage [10]. Since experience was generally correlated with higher knowledge and confidence, simulated experiences could have direct impact on patient care, particularly if specialized oncology or irAE clinics are not feasible options for clinical rotations. These future educational curricula should be efficacious and sustainable. They could be evaluated by the Kirkpatrick evaluation model and annually updated by expert panels given the rapidly growing nature of this field (Supplement D) [17].

Limitations of this study included the sample selection, response rate, and question design. First, most of the sample was comprised of generalists and oncology specialists from a single academic institution, and the list of community oncologists was obtained from the UChicago’s Assistant Director of Physician Relations. While our sample was limited to the Chicago area, the survey was sent to 467 clinicians of diverse roles, including generalists and NPs/PAs who have not been studied prior. The response rate was much lower for community oncologists (8%) than for all other respondent types (50%). This is likely because non-community oncologists were all affiliated with University of Chicago and may have recognized the email address domain for the survey invitation. Conclusions from this study regarding community oncologists are likely affected by selection bias. Finally, because there are no validated survey instruments specifically on irAEs, we relied on expert-driven question generation.

Future curricula will provide role-specific didactics tailored to expectations for various clinicians who care for patients with irAEs. Key gaps in knowledge of irAE treatment with steroid-sparing agents as well as ICI use in at-risk patient populations (such as that with pAID) will be addressed in future teaching modules within this irAE curricular plan. Finally, irAE education for healthcare professionals with busy clinical schedules should be conducted through a hybrid model that facilitates teaching through online resources, interactive modules, and efficient in-person options. Didactics dedicated to ICI toxicities are vital as ICI use becomes more common and various types of clinicians are increasingly participating in irAE patient care. Our findings justify prospective study of curricular development and implementation that will lead to a multifaceted approach to irAE education for healthcare professionals with aims of improving evaluation and care for patients who suffer from ICI toxicities.