Introduction

Risk information—information about the probability of future outcomes—is a cornerstone of evidence-based medicine (EBM) and a primary form of evidence in health care. Risk communication, the open two-way exchange of information and opinion about risk [1], is a prerequisite for shared decision making (SDM) and an integral task in numerous important clinical endeavors, ranging from informed consent to advance care planning. Yet effective risk communication is extremely challenging due to several well-documented problems: innumeracy, psychological biases that limit comprehension of risk information, and inadequate risk communication skills among clinicians. Formal training in risk communication has thus been an increasingly recognized need in medical education; an international expert working group recently identified risk communication as a core competency and central focus for efforts to train clinicians in SDM [2]. In recent years, a growing number of institutions have begun to develop and implement risk communication training programs at various levels of medical education [3–6]. However, these programs have been few and varied in content, and their feasibility, effectiveness, and generalizability remain to be established.

A major barrier to integrating risk communication training in undergraduate medical education is a lack of sufficient curricular time and resources. Our group’s experience is an illustrative case. Through a collaborative effort between educators at St. Georges, University of London and Tufts University School of Medicine, we recently developed and piloted a new risk communication teaching program, “Risk Talk,” among a small cohort of second-year medical students at a single medical school [6]. The program consisted of a workshop, 3 h in duration, integrating didactic instruction, experiential role-play with standardized patients (SPs), and small-group faculty feedback. The program was well received by students and was shown, in an experimental study, to be effective in improving students’ subjective and objective risk communication competence as assessed by Observed Structured Clinical Examination (OSCE) [6]. Yet subsequent dissemination of the program to the entire class was constrained by limited curricular time and resources (SPs and faculty). We therefore made several changes to the program to make it more feasible to administer. We reduced its length to a 70-min workshop, offered alongside other ongoing advanced communication skills training, and incorporating brief didactic teaching and dyadic role-play between students. These changes improved the program’s feasibility, but also diminished its content and effectiveness.

We believe more optimal approaches to integrating risk communication training in medical education are needed, and the objective of the current study was to develop and test one approach. We created an e-learning program, “Risk Talk Online,” designed to deliver didactic and observational instruction in risk communication asynchronously, outside of the classroom—thereby restoring critical educational content and freeing up precious in-person classroom time for mentored experiential skills training. The overarching objective of this “flipped classroom” approach was to deliver risk communication teaching in a more efficient, effective, and potentially generalizable manner. In this paper, we describe the development and content of the new program, and report findings from a pilot study of its feasibility, acceptability, and efficacy in two different US medical schools.

Materials and Methods

Study Population and Setting

The study population consisted of medical students at two US medical schools: Tufts University School of Medicine (TUSM) and Washington School of Medicine (WUSM). TUSM students participated at the beginning of their second medical school year, and WUSM students participated at the end of their first year. Students at both schools receive instruction in EBM, clinical epidemiology, and biostatistics during the first and second medical school years. At TUSM, students complete classes in Epidemiology/Biostatistics over a 3-month period at the beginning of the first year, a 6-week course on Evidence-Based Medicine at the end of the first year, and small-group Problem-Based Learning sessions devoted to teaching skills in searching and evaluating evidence. At WUSM, students receive instruction in EBM, Clinical Epidemiology, and Biostatistics through a longitudinal Practice of Medicine Course spanning the first and second years, and additional coursework throughout all 4 years of the MD curriculum. In the WUSM preclinical curriculum, these topics are introduced and reinforced through a mix of lecture-based and small-group (case-based) instruction.

Program Development

The content of the new program was derived from our workshop-based “Risk Talk” curriculum, the development and evaluation of which have been previously reported [6]. Integrating elements of a curriculum originally developed at SGUL and further adapted at TUSM, Risk Talk focused on helping students understand theoretical principles and best practices of risk communication, based on evidence from the risk communication literature. The new program, Risk Talk Online, was designed to preserve these aims and to deliver essential content of the original Risk Talk curriculum through didactic and observational components. Representative screenshots of the program are shown in Appendix 1.

The didactic component consisted of a narrated slide presentation that began by defining and contextualizing risk communication as a key activity within the broader process of shared decision making. The presentation then discussed the challenge of effective risk communication in terms of two major problems: (1) innumeracy and (2) cognitive biases and heuristics. It gave examples of these problems, drawing from relevant literature [7, 8] and employing interactive exercises demonstrating common pitfalls in interpreting risk information. The presentation then discussed practical strategies for addressing the challenges of effective risk communication, summarized the essential steps of risk communication (Table 1), and reviewed essential steps in effective risk communication, operationalized using verbal “scripts”—exemplary language to facilitate learning of communication skills (Table 2). Developed in our original Risk Talk curriculum [6], these scripts focused on (1) improving conceptual understanding of risk [9]; (2) increasing the evaluability, or interpretability, of risk information (e.g., using common denominators and rounding to make risk estimates more interpretable); and (3) reducing cognitive biases in the interpretation of risk information (e.g., using multiple frames—both positive and negative—and emphasizing absolute versus relative risk reduction in describing benefits and harms of treatment).

Table 1 Essential elements of effective risk communication
Full size table
Table 2 “Risk Talk” scripts
Full size table

The observational component of the program consisted of four brief (2–4 min) high-quality videorecordings of an exemplary discussion about statin treatment for cardiovascular disease prevention, conducted by an experienced physician with a standardized patient and employing the Risk Talk “scripts” introduced earlier in the program. The videorecordings modeled different essential steps of risk communication: (1) setting the stage, (2) explaining the magnitude of risk, (3) explaining the reference class and acknowledging uncertainty, and (4) recapping and summarizing the information. The videos were filmed by a professional video production company and edited by the research team.

The Risk Talk Online program utilized the interactive e-learn software program Articulate® and was designed to run automatically while also enabling users to view the program at their own pace, and to stop, restart, and search and advance to different parts of the program as they desired. The program also included electronic links to two downloadable reference materials which students were encouraged to use as reference materials for future learning and workshop participation: (1) a handout summarizing the Risk Talk scripts and essential steps of risk communication, and (2) a brief review paper on clinical risk communication [10]. The entire program took 31.5 min to complete, without any user-initiated pauses.

Program Implementation

Risk Talk Online was implemented in different ways at each study site. At TUSM, the program was administered in September 2014 as a mandatory preparatory exercise prior to a second-year workshop on advanced communication skills. During this workshop, students rotated through separate teaching sessions on risk communication, motivational interviewing, and dealing with “difficult” patients. The 70-min risk communication workshop consisted of a 10-min recap and introduction, followed by three 20-min dyadic case-based role-play sessions, in which each student “doctor” took 15 min conveying risk information to a fellow student “patient,” and then received 5 min of group feedback. One week before the scheduled workshop, students were emailed the URL and instructions on how to access the online program.

At WUSM, the program was administered in March 2015 as part of the Scientific Methods sub-section of the first-year Practice of Medicine course. All first-year WUSM students were required to complete the Risk Talk Online program in preparation for a 90-min workshop session on risk communication. This session included didactic instruction in risk communication and a faculty-led demonstration with an SP. One week before the workshop session, students received an email with a link and directions on how to complete the program and the associated pre- and post-assessments.

A baseline and post-program survey, containing the measures described below, were administered before and after program completion using the HIPAA-compliant web survey platforms Qualtrics® (at TUSM) and DatStat® (at WUSM). At TUSM, survey completion was mandatory, although students could voluntarily opt out of having their responses used for research; at WUSM, all student data was deidentified prior to analysis and thus no voluntary opt-out was required. The study protocol was exempted from Institutional Review Board (IRB) review at Tufts University School of Medicine and Washington University School of Medicine.

Program Evaluation and Measures

Evaluation focused on the program’s feasibility, acceptability, and efficacy. Feasibility was assessed by measuring (1) program completion rate and (2) self-reported program completion time. Students were also asked whether they completed the program in a single sitting. Acceptability was assessed using a three-item measure of students’ perceptions of the program’s value, adapted from our prior study (Appendix 2-1).

Efficacy was assessed by measuring students’ subjective (self-reported) and objective risk communication competence before and after completing Risk Talk Online. Subjective risk competence was assessed using our previously developed three-item measure of confidence in risk communication skills [6] (Appendix 2-2). This measure ascertained students’ perceived level of confidence in their understanding of medical statistics, ability to effectively communicate medical statistics to patients and/or families, and ability to help patients and/or families understand information about probabilities or risks. Objective risk communication competence was assessed by a newly developed four-item measure (Appendix 2-3) consisting of four case vignettes testing students’ knowledge of practical communication strategies for accomplishing key tasks emphasized in our curriculum: (1) framing risk reduction in absolute versus relative terms, (2) explaining the reference class, (3) conveying uncertainty in risk information, and (4) simplifying risk information to facilitate understanding. These items were written to be practical in orientation—i.e., to assess knowledge of not only theoretical concepts, but of preferred approaches (language and communication strategies) to convey risk information in clinical encounters.

Data Analysis

Descriptive statistics for all variables were computed and within-subjects, pre-post changes in subjective and objective risk communication competence were assessed using paired-sample t tests. The proportions of correct responses to individual objective communication competence items were also computed, and pre-post changes were compared using McNemar’s test. The critical level of significance was set at P <0.05.

Results

Feasibility and Acceptability

Overall, 204/210 (97 %) TUSM students and 115/123 (94 %) WUSM students completed the online program; 6 TUSM students opted out of allowing use of their surveys in the study. Average self-reported program completion time was 40.3 min at TUSM and 41.7 min at WUSM. The proportion of students who completed the program in a single sitting was 90.9 % at TUSM and 90.4 % at WUSM. Students gave high value ratings—approaching the six-point maximum value—for the content (M = 4.67, SD = 1.20), videoclips (M = 4.66 SD = 1.29), and impact of the program (M = 4.53, SD = 1.14).

Efficacy

Within-subject pre-post differences in subjective and objective risk communication competence are shown in Table 3. A programming error prevented matching baseline and post-program surveys to unique individuals in the TUSM cohort; we therefore applied probabilistic matching based on survey date, time, and IP addresses, which enabled successful survey matching for 143/204 (70 %) of TUSM students. Students at both sites demonstrated significant improvement on all measures after program completion.

Table 3 Changes in subjective and objective risk communication competence scores before and after viewing the teaching program
Full size table

Performance on individual objective risk communication items are shown in Table 4. The proportion of correct responses was lowest for the item on framing risk reduction in absolute versus relative terms, intermediate for the items on explaining the reference class and simplifying risk information, and highest for the item on communicating uncertainty in risk estimates. Pre-post changes were significant for each item, although largest for the item on the reference class and smallest for the item on explaining uncertainty.

Table 4 Performance on individual objective risk communication measure items
Full size table

Discussion

In this study a new risk communication training program, Risk Talk Online, was developed, implemented at two US medical schools, and found to be feasible, acceptable, and efficacious. The program was completed in approximately 40 min and in a single sitting by over 90 % of students, who rated its value very highly. The program produced significant improvements in subjective and objective risk communication competence. Together, these findings provide strong proof-of-principle evidence supporting the program’s value and generalizability.

Importantly, Risk Talk Online was designed and implemented not as a stand-alone intervention but as part of a more comprehensive “flipped-classroom” training program integrating additional didactic instruction and experiential skills practice in classroom settings. Our findings support the need for such additional instruction. Despite significant improvement in students’ objective risk communication competence after they completed the online program, only a minority (<20 %) were able to correctly answer all four knowledge questions. Students struggled with particular items including the need to frame risk reduction in absolute (versus relative) terms. Although this concept received great emphasis in the teaching program, our data suggest the need for more reinforcement as well as refinement of the program’s content. It should be noted, furthermore, that the knowledge measure used to assess objective risk communication competence was new, and further work to refine the measure and assess its reliability and validity is an important next step for future research.

More research is also needed to determine whether online training in risk communication improves actual communication skills, which were not assessed in the current study. We do have convincing evidence—from our previous OSCE-based experimental study of the Risk Talk curriculum on which the content and approach of the current e-learn program is based—that our overall teaching intervention improves communication skills [6]. It stands to reason, furthermore, that had students in the current study also received experiential training prior to their competence assessments, even greater improvements might have been observed. Nevertheless, whether the Risk Talk Online program—either alone or in combination with workshop-based experiential training—results in similar improvement is a key question for future research.

Broader integration and dissemination of risk communication training in medical education ultimately requires generalizable strategies for overcoming curricular time and resource barriers. We believe that e-learning programs such as Risk Talk Online provide one promising strategy for accomplishing this goal. Similar “flipped classroom” programs have been successfully employed in the teaching of other clinical skills and may facilitate dissemination of risk communicating training at other levels of medical education, including not only undergraduate but graduate and postgraduate training. For all types and levels of learners, e-learning programs for teaching risk communication skills would ideally be deployed as one component of a more comprehensive curriculum, but they could also serve as stand-alone programs if curricular time and resources were limited. We believe risk communication is such a critical advanced communication skill that even some basic training is far preferable to none at all.

Several limitations qualify our study findings and call for further research. Although the Risk Talk Online program was well received, its usability was not evaluated. Formal usability testing and refinement of the program are a necessary focus for future work. The durability of students’ knowledge improvements, and their relationship to patient care outcomes, also remain to be shown. Further research is also needed to assess the feasibility and effectiveness of implementing Risk Talk Online at different points in the medical school curriculum or for different levels of learners. The optimal strategies for integrating risk communication teaching with other related components of the medical school curriculum—e.g., instruction in evidence-based medicine and advanced communication skills—also require further investigation. Finally, more work is needed to explore how risk communication teaching might be incorporated in graduate and continuing medical education.

Despite these limitations, the current study provides promising evidence on the educational value of an online clinical risk communication teaching program. We believe that Risk Talk Online represents an important first step in the development of feasible and effective strategies for teaching risk communication skills, and in disseminating this teaching in medical education. The current study provides proof-of-principle evidence of the program’s value, and paves the way for further work to refine the program and improve physicians’ competence in communicating risk information to patients.

Conclusions

Risk Talk Online, a new online risk communication teaching program, is feasible, acceptable, and efficacious in improving subjective and objective risk communication competence among first- and second-year medical students. More research is needed to refine the program and determine optimal strategies for integrating risk communication training in medical education.