Initial Outcomes of an Online Continuing Education Series Focused on Post-treatment Cancer Survivorship Care

There is a growing number of post-treatment cancer survivors in the USA. Cancer survivors can have a variety of care needs and health care professionals must be prepared to meet these needs. The American Cancer Society (ACS) and the George Washington University (GW) Cancer Center developed The Cancer Survivorship E-Learning Series for Primary Care Providers (E-Learning Series) to address the need for cancer survivorship training and education among health care professionals with a focus on primary care. The GW Cancer Center analyzed evaluation data from 1341 learners who voluntarily completed a module pre- and post-assessment between April 15, 2013, and December 31, 2017, to assess differences in self-rated confidence, on a five-point Likert scale, to meet learning objectives. Descriptive statistics characterize the sample and paired samples t tests were used to assess any statistically significant differences from pre to post (p < 0.05). Most learners were nurses (75.19%) and a majority of learners worked in oncology (74.68%) followed by primary care (11.60%). At pre-assessment, the module with the lowest mean self-confidence rating was 3.16 (SD = 0.81) and the highest was 3.60 (SD = 0.73). At post-assessment, module means in self-confidence rating ranged from 4.08 (SD = 0.46) to 4.26 (SD = 0.56). All differences were statistically significant (p < 0.0001). Results highlight gaps in confidence among health care professionals regarding cancer survivorship care and the need for continuing education. There is also a need for additional uptake of the E-Learning Series among primary care providers. Results suggest that the E-Learning Series is an effective educational tool that increases learners’ confidence in providing cancer survivorship care.


Background
A growing evidence base indicates cancer survivors can face a host of long-term and late effects due to cancer and its treatment, which can affect survivors' physical, emotional, spiritual, and social well-being. For example, cancer survivors are at an increased risk for developing second primary cancers [1], report higher levels of depression and anxiety compared to the general population [2], have difficulty adhering to medical care due to cancer-related financial hardship [3], and do not engage in recommend physical activity levels, which can improve overall quality of life and potentially decrease risk of recurrence [4]. Based on these and other needs, it is clear that the need for specialized health care services for cancer survivors does not end when primary treatment is finished. Due to the potential complexities, survivorship care requires a coordinated, multidisciplinary effort, involving oncology providers, primary care providers, other specialty providers, and public health professionals [5][6][7].
There are approximately 15.5 million cancer survivors living in the USA. It is projected the number of cancer survivors will grow to 20.3 million by 2026, an increase of 30% in 10 years [8]. More than a decade ago, cancer care stakeholders had the foresight that the US health care system would not be fully prepared to care for cancer survivors in the coming years. To help move cancer survivorship to the forefront, The National Academies of Sciences, Engineering, and Medicine (NASEM) Health and Medicine Division (formerly the Institute of Medicine) released its seminal 2005 report, From Cancer Patient to Cancer Survivor: Lost in Transition. The report identified several critical gaps, including the need for survivorship continuing education and training of diverse health care professionals. Per the NASEM report, education should address risks of treatments and cancer recurrence, management of psychosocial and late effects, and benefits of healthy lifestyle changes [9]. In addition to education and training, another key recommendation from the report was the creation and utilization of evidence-based cancer survivorship clinical care guidelines.
To help meet the call for education and development of guidelines, the American Cancer Society (ACS), the GW Cancer Center, and the Centers for Disease Control and Prevention (CDC) developed The National Cancer Survivorship Resource Center in 2010. Emerging from the five-year collaboration were key tools for health care professionals to fill identified gaps: The Cancer Survivorship E-Learning Series for Primary Care Providers (E-Learning Series); the American Cancer Society Survivorship Clinical Care Guidelines for prostate, colorectal, and head and neck cancers; and the American Cancer Society/ American Society of Clinical Oncology Breast Cancer Survivorship Clinical Care Guideline [10][11][12][13]. These clinical guidelines provide evidence-based care recommendations for primary care providers and core content for four modules of the E-Learning Series.
This article provides an overview of the initial outcomes of the E-Learning Series, with a focus on reach as well as effectiveness of the series in increasing learner confidence in defined cancer survivorship learning objectives.

Training Development and Measures
The E-Learning Series is a no-cost, self-paced, 10-module online course and offers free continuing education credits/contact hours to physicians, nurses, physician assistants, and Certified Health Education Specialists (CHES®). Each module corresponds to 1 credit/contact hour. In order to receive credit, participants must successfully complete all activities of at least 1 module. Each module takes approximately 1 hour to complete and includes two 30-minute, evidence-based presentations from experts on the module topic. Each module also highlights the perspective of a cancer survivor and links to additional resources for providers. A learner can select which module(s) they want to take based on their interests and needs. When initially developed, the intended audience for the training was primary care providers; however, the training is open to anyone who wants to enroll.
The E-Learning Series launched on April 15, 2013 with three modules. The remaining modules were released on a rolling basis through March 31, 2016. The first six modules provide an overview of key components of cancer survivorship care, including management of comorbidities, psychosocial health, and tertiary prevention. The last four modules follow the same learning format but focus on the dissemination of the American Cancer Society Survivorship Clinical Care Guidelines.
The GW Cancer Center utilized the Kirkpatrick Evaluation Model to frame the evaluation of the E-Learning Series. This model is frequently used to evaluate trainings, including those for health care professionals [14,15]. The GW Cancer Center focused evaluation on the first two levels of the model (Level 1: Learner satisfaction and Level 2: Learning outcomes) given the on-demand, online format of the learning platform.

Study Population
This study did not meet the definition of human subjects research per guidance from the Institutional Review Board at the George Washington University. The rationale for this determination was that data were collected for the primary purpose of program evaluation and improvement for an online learning offering administered uniquely by GW and were not collected to encourage replication of the educational intervention outside of the local context. A total of 1656 unique learners voluntarily enrolled and participated in the E-Learning Series between April 15, 2013, and December 31, 2017, when data were exported for analysis. Learners who did not complete both a pre-and post-assessment for any given module or who indicated they practiced internationally were excluded from the sample. This evaluation includes learners practicing in the USA, tribes, and territories (n = 1341) who fully completed one or more modules' pre-and post-assessments between launch and December 31, 2017.

Online Training Promotion
The E-Learning Series has been marketed and disseminated nationally through the ACS, GW Cancer Center, and the CDC's various communication channels, including social media and e-mail. In addition, the E-Learning Series has been posted on the National Commission for Health Education Credentialing, Inc. website and promoted at professional and regional conferences across the country.

Data Collection
The GW Cancer Center administered the demographic survey and assessments and managed study data through SurveyMonkey® and our Learning Management System (LMS). Survey Monkey® was used from launch until December 2015, at which point the E-Learning Series pre/ post evaluation data collection moved to the LMS.
At the beginning of the E-Learning Series, learners completed an optional demographic survey, which included questions on age, profession, field of practice, and place of practice. When the E-Learning Series moved to the LMS, additional questions were added such as self-identified gender, race, ethnicity, and practice setting (rural, suburban, urban, unsure or not applicable). Since questions were successively added or could be skipped, the sample size for demographic characteristics reported in Table 1 varies.
For each module, learners also completed a pre-and postassessment. Each assessment had learners' rate their agreement in their confidence to meet the module's specific learning objectives. The number of learning objectives varied for each module, but averaged three to four objectives per module. The post-assessments also included questions on selfreported knowledge, strategies and skills gained, and intention to implement learnings in practice. Questions were measured on a five-point Likert scale (strongly disagree to strongly agree). An open-ended question was added later to the postassessment, so learners could offer feedback or suggestions for improvement.

Statistical Analysis
All statistical analyses were conducted using STATA®/IC 14.2. Response options from Bprofession^and Bfield of practice^were collapsed and made mutually exclusive prior to analysis. For Bfield of practice,^primary care was defined as family medicine, geriatrics, gynecology and internal medicine. For Bprofession,^nursing was defined as nurse, nurse practitioner, and nurse navigator. States were categorized and reported by US census region [16]. Both pre-and postassessment ratings were averaged for each module. Postassessment questions on self-reported knowledge, strategies and skills gained, and intention to implement information learned into practice were averaged across all modules then dichotomized by Bagree/strongly agree^and Bneutral/disagree/strongly disagree.^Descriptive statistics, including frequencies and percentages, were reported to quantify responses. Paired samples t tests were used to assess if the average difference in confidence from pre-to post-assessment was statistically significant for each individual learning objective, and when differences were averaged for each module. A p value of < 0.05 was considered statistically significant.

Results
Demographic characteristics of the sample are presented in Table 1. The majority of learners identified as female (93.00%), White (86.02%), and non-Hispanic (89.34%).
Most participants were between 40 and 59 years old. A majority of learners were nurses (75.19%) followed by physicians (8.17%). Learners worked mainly in oncology (74.68%) and some worked in primary care (11.60%). For practice location, most learners (44.40%) worked in an urban area, followed by suburban (31.04%) and then rural (12.15%). More than two thirds of learners (67.32%) were from the South and Midwest regions of the USA. Sample size for each module, module topic, and launch date are presented in Table 2. Participation was higher for Modules 1 through 4 and tapered for remaining modules with less than 500 learners completing each module. Learning objectives for each module are presented in Table 3.
Reviewing learners' self-rated confidence from pre-to post-assessment (Table 4), there was an increase in mean self-confidence rating for aggregated learning objectives from pre to post for each module, which ranged from 0.66 (SD = 0.67) to 0.92 (SD = 0.77). Change in mean confidence rating was statistically significant (p < 0.0001) for all modules, and the pre-post differences for each individual learning objective were statistically significant (p < 0.0001) (individual objectives' change scores are not presented in the table). For preassessment, the average aggregate confidence rating for learners per module ranged from 3.16 (SD = 0.81) to 3.60 (SD = 0.73). Self-reported confidence in the ability to meet learning objectives was lowest in Module 10 (3.16) and Module 1 (3.20). In the post-assessment, the average aggregate confidence rating for learners per module ranged from 4.08 (SD = 0.46) to 4.26 (SD = 0.56). Also at post-assessment, on average a majority of learners agreed or strongly agreed their knowledge was enhanced (91.59%), they gained new strategies/skills/information they could apply to practice (83.41%), and planned to implement these gains into practice (75.38%).

Discussion
Initial evaluation outcomes support the need for ongoing continuing education in cancer survivorship for all health care professionals as identified in the NASEM report more than a decade ago. When the E-Learning Series was developed, the intended audience was primary care providers. However, based on participation across fields, results suggest that diverse health care professionals are in need of and are seeking information about cancer survivorship.  Results also highlight baseline gaps in confidence among health care professionals regarding cancer survivorship care and suggest the E-Learning Series is effective in reducing these deficits. For example, at the beginning of each module, on average, learners rated their confidence to meet learning objectives as Bneutral.^Upon module completion, learners' confidence on average increased by almost one point, which indicates that most learners agreed they were confident in their ability to meet learning objectives. The learning objective with the greatest increase from pre to post (1.01 points) was BI am confident in my ability to describe national efforts related to survivorship care^(Module 1), suggesting that most learners were unaware of larger national efforts. Integration of individual-level provider behavior with national benchmarks is critical to advance broader health systems change, particularly in the area of care coordination across providers. Observed gains from pre-to post-assessment demonstrate that the E-Learning Series is an effective educational tool to increase learner confidence in addressing physical and psychosocial aspects of cancer survivorship care.
The E-Learning Series reached learners across the USA, yet results indicate the need for additional dissemination and uptake of this educational tool particularly among primary care providers. The 1341 learners who have taken this online training cannot meet the needs of a population of 15.5 million cancer survivors. Modules that focus on follow-up clinical care guidelines for primary care providers (Modules 7-10) have lower participation compared to earlier modules. This could be due to the fact that these modules were released later and have had less time to accrue learners and also due to their disease-specific and clinical nature. Dissemination and uptake of clinical guidelines are critical to move evidence into practice. To support additional dissemination and uptake of the E-Learning Series by primary care providers, the GW Cancer Center recently developed an evidence-based communication toolkit [17] to help health care organizations and primary care professional membership associations promote this learning opportunity. The toolkit includes sample social media messages, e-mails, and blog posts to increase interest in the E-Learning Series. Messages highlight the self-paced nature of the training, the critical role primary care providers play in post-treatment survivorship care, and the need for providers to participate in continuing education to improve cancer survivors' care coordination and health outcomes [18].
The toolkit has been downloaded more than 3500 times since its release in July 2018.
Another possible barrier to uptake is the online delivery of the program. Diverse learners have different preferences for learning formats. Feedback from learners includes technological challenges, such as audio or visual malfunctions, which may have caused some learners to not complete a module. The GW Cancer Center updates modules on a rolling basis as issues are reported in order to improve the user experience and to ensure evidence presented is current. The GW Cancer Center anticipates updating Module 5 in the coming year, since new evidence regarding models of care and the use of survivorship care plans has emerged since the module was released in 2013. Additional methods for dissemination, contingent on resources, will be explored to increase access and uptake.
Limitations of the present analysis include lack of a comparison group, a convenience sample of learners, and the selfreported nature of the data. Future directions for research should include retention of learner confidence longitudinally, as well as behavioral intention, application to practice, such as coordination of care, uptake and impact of recommended guidelines, and patient-reported outcomes.

Conclusions
There is a need for continued education and training of health care professionals to equip them with the knowledge and skills to care for a growing population of post-treatment cancer survivors. The E-Learning Series is an effective educational tool to increase health care providers' confidence in cancer survivorship care. Additional dissemination of this educational tool is warranted to build capacity of a health care professional workforce responsive to the diverse needs of the growing population of cancer survivors. Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.