Database searches resulted in 2555 potentially eligible records. Of these, 24 articles (seven abstracts and 17 full text studies) representing 21 studies and evaluating 21 unique survivorship education programs [38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58] met the inclusion criteria and were included in the review (see PRISMA flow chart: Fig. 1).
Characteristics of included studies
Studies characteristics are detailed in Table 1. All 21 studies utilized single-group designs with no comparators. Thirteen studies [38,39,40, 42, 43, 45, 46, 48, 51, 55,56,57,58] used a pre-test, post-test design. Five studies used a post-test only design [41, 47, 49, 53, 54], two studies used a mixed-methods approach [44, 52], and one study had an unspecified methodology . Of the 21 survivorship education programs evaluated within these studies, 15 were developed in the USA [38, 39, 41,42,43,44,45,46,47,48,49,50, 52, 55, 57], three were developed in Australia [51, 56, 58], one in Germany , and two in Canada [40, 53]. Target learners for these programs were PCPs or residents in primary care training, including internal medicine residents , pediatric physician residents , PCPs only [38,39,40,41,42,43, 45, 47, 48, 51, 56, 58], and mixed health professional groups including PCPs [44, 49, 50, 52,53,54, 57].
Quality of the evidence
All studies were susceptible to bias due to the lack of comparison groups. The quality of most pre-test, post-test and post-test studies was rated poor, with one rating fair (see Supplementary File 5). In addition to the inherently high risk of repeat testing bias, observer bias, the Hawthorne effect, and attrition bias of these types of studies, most of these studies presented limited information about eligibility criteria, sample size calculation, loss to follow-up, fidelity of intervention delivery, and the definition and reliability of outcome measures. Of the two mixed-methods studies, one was of high methodological quality  and the other received a poor rating , presenting issues such as unspecified qualitative methodology, sampling strategy, and statistical methods (see Supplementary 5). Across all studies, outcome measures generally consisted of non-validated self-response measures, increasing risk of self-rater bias. Further, only eight [38,39,40, 42, 44, 48, 52, 55] of 21 studies included a statistical analysis of results and specified the magnitude of change.
Content and modality of cancer survivorship programs
Survivorship care components and content of each program are outlined in Table 2. Intended cancer survivorship content differed across education programs and included clinician education targeted towards the management of fear of cancer recurrence ; utilization of survivorship care plans (SCPs) [42, 53]; management of sexual complications in female cancer survivors ; survivorship management specifically for Hodgkin’s lymphoma, breast cancer and prostate cancer survivors , and cervical, breast, and colon cancer survivors ; childhood cancer survivors [43, 47, 55]; survivors undergoing immunotherapy ; and “general cancer survivorship” across all populations [40, 44,45,46, 49, 51, 52, 54, 56,57,58]. A variety of approaches to survivorship education were described across studies including self-directed online courses [39, 45, 47, 56], in-person presentations [38, 42, 58], workshops and training sessions [40, 44, 46, 48, 54, 55], placement and clinical rotation programs [41, 51, 58], a fellowship program , a referral program , a live webcast , a survivorship conference , an in-person workshop and online webinar , and an in-person seminar and online webinar series .
Cancer survivorship program curricula
A range of sources and guidelines were used to inform survivorship education curricula and content. Eight [43,44,45,46,47,48, 52, 55] of 21 programs specified the utilization of at least one specific recognized guideline or framework to inform survivorship program content or curricula. These guidelines/frameworks included the Children’s Oncology Group Long-Term Follow-up Guidelines [43, 47, 55]; the Quality of Life Model for Cancer Survivors ; the American Cancer Society Clinical Care Guidelines ; the American Society of Clinical Oncology (ASCO) Breast Cancer Survivorship Care Guidelines and ASCO Treatment Summary Survivorship Care Plan ; the Institute of Medicine (IOM) survivorship recommendations ; and the 5A’s communication framework . One study  did not explicitly specify the foundation of curriculum content but described the utilization of videos from the IOM in their program and linked additional resources from the American Academy of Family Physicians (AAFP), National Comprehensive Cancer Network (NCCN), National Cancer Institute (NCI), the Physicians’ Desk Reference (PDR) and ASCO. Lecture content for one fellowship program was informed by the Commission on Cancer (COC), ASCO, and NCCN guidelines . One study described the use of an interdisciplinary committee (incorporating cancer survivor representatives) along with literature, to inform their survivorship curriculum . Two studies engaged clinical stakeholders in informing curricula [38, 50], but did not specify the engagement of cancer survivors. One study engaged a stakeholder group consisting of clinicians, community outreach, marketing, and business development staff in informing program curriculum content . Seven studies did not specify what informed survivorship program content or curricula [40, 42, 51, 54, 56,57,58].
Teaching or learning frameworks and theories underpinning the survivorship programs
Eight [38,39,40, 43,44,45, 52, 55] of 21 survivorship education programs explicitly described the use of a teaching or learning framework or theory to guide program development or evaluation. Buriak and colleagues  used the Analysis, Design, Development, Implementation, and Evaluation (ADDIE) instructional systems process model and a revised version of Bloom’s taxonomy to design their educational program. The same authors also used Gagne’s “Nine Events of Instruction” and Mayer’s principles of multimedia modality to facilitate effective learning. Berrett-Abebe, Chaput, Harvey, and colleagues [38, 40, 45] described the use of Kirkpatrick’s Training Evaluation Framework. Of these three studies, Chaput and colleagues  utilized three levels of the framework (satisfaction, knowledge, and behavior) to develop the outcome measures of their education program, and the remaining two studies [38, 45] used the framework to guide program evaluation. Berrett-Abebe and colleagues  also described use of social cognitive theory to guide the development of their training program. Fulbright and colleagues  used adult learning theory in curriculum development (and in case-based examples). Grant and colleagues  described the utilization of adult learning principles and institutional change theory. Risendal and colleagues  utilized Boot Camp Translation methodology and appreciative inquiry to develop and implement their curriculum, while Schwartz and colleagues  used Kern’s six-step model to integrate their survivorship curriculum into an existing educational structure. The same study also used Bloom’s taxonomy to inform learning objectives. Thirteen studies [41, 42, 46,47,48,49,50,51, 53, 54, 56, 57] did not describe the use of a teaching or learning framework/theory.
Cancer survivorship program evaluation
Education program outcomes are reported in Table 1 as informed by the Kirkpatrick framework. Program outcomes and final interpretations were the same after removal of abstract studies from analysis; thus, outcomes described in full-text papers and conference abstracts were appropriate to be combined during narrative synthesis.
Kirkpatrick level 1—reaction to program
Sixteen studies [38,39,40, 42,43,44, 46, 47, 49, 51,52,53,54,55, 57, 58] evaluated level 1 outcomes. Views on learning experience were mostly measured using a Likert scale (from strongly disagree to strongly agree) with certain studies also incorporating open text [42, 44, 55] and interview feedback [44, 47, 52, 53, 58]. Across all 16 survivorship programs, learner experience and program content were generally reported to be favorable. Outcomes reported included program usefulness and relevance to practice [38, 47, 51, 52, 54, 55, 57]; satisfaction with program design and organization [39, 51]; satisfaction with program content and delivery [39, 40, 43, 44, 49, 51, 53, 58]; recommendations for program improvement [42, 58]; and the recommendation of the program to others [38, 46, 55]. Critical reports on learning experience included insufficient program duration to facilitate the achievement of learning goals [51, 58]; the request for further training in survivorship counseling, additional onco-fertility information, and survivorship training opportunities ; limited exposure to long-term follow-up care and the request for more structured education and quality improvement activities ; and additional support in acquiring well-defined SCPs and follow-up schedules .
Kirkpatrick level 2—learning
Thirteen studies [38, 40, 44,45,46,47,48, 51,52,53, 55, 56, 58] reported positive level 2a outcomes, including increased awareness of cancer survivor needs [47, 52, 58] and increased confidence [38, 40, 45, 51, 56, 58], self-efficacy , and comfort [44, 46, 48, 52, 53, 55] in providing cancer survivorship care. Most outcomes were measured using self-reported questionnaires with three studies utilizing interviews [44, 52, 58]. Fourteen studies [38,39,40, 42, 43, 45, 46, 49,50,51,52, 55, 57, 58] evaluated level 2b outcomes assessing improvements in knowledge, competency , and the recall of topics . Eleven of the fourteen studies evaluated knowledge outcomes using pre-tests and post-tests that were delivered immediately after program cessation (immediate post-test) (e.g. questionnaires, surveys); two studies used a pre-test and immediate post-test followed by a 3 month  or 12 month  delayed post-test; and one study utilized feedback from fellowship program facilitators to determine that learners had sufficient increase in cancer survivorship knowledge .
Kirkpatrick level 3—transfer of learning/behavior change
Nine studies [38, 39, 41, 43, 44, 48, 52, 53, 58] evaluated PCP learner behavior change as a result of survivorship education. Of the eight studies, only two [44, 53] evaluated the transfer of cancer survivorship learning to clinical practice (e.g., increased PCP confidence ordering follow-up tests; self-reported behavior change due to education intervention), both of which also provided follow-up evaluation 12 months post-program commencement. The remaining six studies [38, 39, 41, 43, 48, 52, 58] only measured outcomes immediately post-program; and thus, only evaluated the intention of PCPs to change practice (e.g., implement follow-up guidelines, improve referrals, provision of routine screening, etc.).
Kirkpatrick level 4—program impact/results
Two studies evaluated level 4a outcomes assessing survivorship changes in organizational practice [41, 44]. Daly and colleagues  reported that one PCP practice referred three patients onto lung surveillance and screening because of PCP education sessions within their program. Nearly all (98.1%) participants in Grant and colleagues study  reported that program participation resulted in the uptake of cancer survivorship care at their respective institutions, though specific detail on changes were not provided. Institutional assessments and surveys completed 18-month post-program commencement found significant change and improvement in organization vision and management standards; practice standards; psychosocial, emotional, and social care; communication standards; quality improvement standards; patient and family education; and community network partnerships. However, despite reporting statistical significance, the specific magnitude of change was unclear.
Three studies [50, 52, 53] assessed level 4b outcomes which indicate survivorship program impact at the patient level. Risendal and colleagues’ mixed-methods study  found that training sessions led to varied survivorship care changes across participating rural practices, including more comprehensive history taking, improved referral of patient surveillance scans, and other unspecified immediate changes. Rushton and colleagues  reported that 1 year after initial transfer to their PCP cancer survivorship referral program, patients were satisfied with the overall support, care, and quality of information received and were more knowledgeable about treatment received, potential late effects, and the latent symptoms to report to their primary care providers. Perloff and colleagues  estimated that 590 patients per month were “impacted” by their PCP survivorship webinar; however, it was not specified how authors measured this outcome or whether the impact was positive or negative. Notably, all three studies [50, 52, 53] did not quantify the magnitude of change in outcomes reported.