Abstract
Objective
The objective of this study was to develop, pilot, and evaluate an online program for teaching clinical risk communication skills to medical students.
Methods
A new teaching program, “Risk Talk Online,” was developed and piloted at Tufts University School of Medicine (TUSM) and Washington University School of Medicine (WUSM) in 2014–2015. The program’s content was based on “Risk Talk,” a risk communication curriculum previously developed at TUSM, and consisted of didactic and observational instruction using videorecordings of physician-patient discussions illustrating key elements of effective risk communication. The program was implemented among students at the beginning of the second medical school year at TUSM and the end of the first medical school year at WUSM. The program’s feasibility, acceptability, and efficacy in improving subjective and objective risk communication competence were assessed.
Results
A total of 204 TUSM and 115 WUSM students completed the program. Average time of completion was approximately 40 min, and students rated the program’s educational value very highly. Subjective (self-reported) competence in risk communication significantly improved (P < 0.001), as did objective risk communication competence (P < 0.001) assessed by students’ performance on a skills-oriented knowledge assessment administered before and after completion of the program.
Conclusions
A new online risk communication teaching program was feasible, acceptable, and efficacious in improving subjective and objective risk communication competence among first- and second-year students at two medical schools. More research is needed to refine the program and determine optimal strategies for integrating risk communication training in medical education.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Introduction
Risk information—information about the probability of future outcomes—is a cornerstone of evidence-based medicine (EBM) and a primary form of evidence in health care. Risk communication, the open two-way exchange of information and opinion about risk [1], is a prerequisite for shared decision making (SDM) and an integral task in numerous important clinical endeavors, ranging from informed consent to advance care planning. Yet effective risk communication is extremely challenging due to several well-documented problems: innumeracy, psychological biases that limit comprehension of risk information, and inadequate risk communication skills among clinicians. Formal training in risk communication has thus been an increasingly recognized need in medical education; an international expert working group recently identified risk communication as a core competency and central focus for efforts to train clinicians in SDM [2]. In recent years, a growing number of institutions have begun to develop and implement risk communication training programs at various levels of medical education [3–6]. However, these programs have been few and varied in content, and their feasibility, effectiveness, and generalizability remain to be established.
A major barrier to integrating risk communication training in undergraduate medical education is a lack of sufficient curricular time and resources. Our group’s experience is an illustrative case. Through a collaborative effort between educators at St. Georges, University of London and Tufts University School of Medicine, we recently developed and piloted a new risk communication teaching program, “Risk Talk,” among a small cohort of second-year medical students at a single medical school [6]. The program consisted of a workshop, 3 h in duration, integrating didactic instruction, experiential role-play with standardized patients (SPs), and small-group faculty feedback. The program was well received by students and was shown, in an experimental study, to be effective in improving students’ subjective and objective risk communication competence as assessed by Observed Structured Clinical Examination (OSCE) [6]. Yet subsequent dissemination of the program to the entire class was constrained by limited curricular time and resources (SPs and faculty). We therefore made several changes to the program to make it more feasible to administer. We reduced its length to a 70-min workshop, offered alongside other ongoing advanced communication skills training, and incorporating brief didactic teaching and dyadic role-play between students. These changes improved the program’s feasibility, but also diminished its content and effectiveness.
We believe more optimal approaches to integrating risk communication training in medical education are needed, and the objective of the current study was to develop and test one approach. We created an e-learning program, “Risk Talk Online,” designed to deliver didactic and observational instruction in risk communication asynchronously, outside of the classroom—thereby restoring critical educational content and freeing up precious in-person classroom time for mentored experiential skills training. The overarching objective of this “flipped classroom” approach was to deliver risk communication teaching in a more efficient, effective, and potentially generalizable manner. In this paper, we describe the development and content of the new program, and report findings from a pilot study of its feasibility, acceptability, and efficacy in two different US medical schools.
Materials and Methods
Study Population and Setting
The study population consisted of medical students at two US medical schools: Tufts University School of Medicine (TUSM) and Washington School of Medicine (WUSM). TUSM students participated at the beginning of their second medical school year, and WUSM students participated at the end of their first year. Students at both schools receive instruction in EBM, clinical epidemiology, and biostatistics during the first and second medical school years. At TUSM, students complete classes in Epidemiology/Biostatistics over a 3-month period at the beginning of the first year, a 6-week course on Evidence-Based Medicine at the end of the first year, and small-group Problem-Based Learning sessions devoted to teaching skills in searching and evaluating evidence. At WUSM, students receive instruction in EBM, Clinical Epidemiology, and Biostatistics through a longitudinal Practice of Medicine Course spanning the first and second years, and additional coursework throughout all 4 years of the MD curriculum. In the WUSM preclinical curriculum, these topics are introduced and reinforced through a mix of lecture-based and small-group (case-based) instruction.
Program Development
The content of the new program was derived from our workshop-based “Risk Talk” curriculum, the development and evaluation of which have been previously reported [6]. Integrating elements of a curriculum originally developed at SGUL and further adapted at TUSM, Risk Talk focused on helping students understand theoretical principles and best practices of risk communication, based on evidence from the risk communication literature. The new program, Risk Talk Online, was designed to preserve these aims and to deliver essential content of the original Risk Talk curriculum through didactic and observational components. Representative screenshots of the program are shown in Appendix 1.
The didactic component consisted of a narrated slide presentation that began by defining and contextualizing risk communication as a key activity within the broader process of shared decision making. The presentation then discussed the challenge of effective risk communication in terms of two major problems: (1) innumeracy and (2) cognitive biases and heuristics. It gave examples of these problems, drawing from relevant literature [7, 8] and employing interactive exercises demonstrating common pitfalls in interpreting risk information. The presentation then discussed practical strategies for addressing the challenges of effective risk communication, summarized the essential steps of risk communication (Table 1), and reviewed essential steps in effective risk communication, operationalized using verbal “scripts”—exemplary language to facilitate learning of communication skills (Table 2). Developed in our original Risk Talk curriculum [6], these scripts focused on (1) improving conceptual understanding of risk [9]; (2) increasing the evaluability, or interpretability, of risk information (e.g., using common denominators and rounding to make risk estimates more interpretable); and (3) reducing cognitive biases in the interpretation of risk information (e.g., using multiple frames—both positive and negative—and emphasizing absolute versus relative risk reduction in describing benefits and harms of treatment).
The observational component of the program consisted of four brief (2–4 min) high-quality videorecordings of an exemplary discussion about statin treatment for cardiovascular disease prevention, conducted by an experienced physician with a standardized patient and employing the Risk Talk “scripts” introduced earlier in the program. The videorecordings modeled different essential steps of risk communication: (1) setting the stage, (2) explaining the magnitude of risk, (3) explaining the reference class and acknowledging uncertainty, and (4) recapping and summarizing the information. The videos were filmed by a professional video production company and edited by the research team.
The Risk Talk Online program utilized the interactive e-learn software program Articulate® and was designed to run automatically while also enabling users to view the program at their own pace, and to stop, restart, and search and advance to different parts of the program as they desired. The program also included electronic links to two downloadable reference materials which students were encouraged to use as reference materials for future learning and workshop participation: (1) a handout summarizing the Risk Talk scripts and essential steps of risk communication, and (2) a brief review paper on clinical risk communication [10]. The entire program took 31.5 min to complete, without any user-initiated pauses.
Program Implementation
Risk Talk Online was implemented in different ways at each study site. At TUSM, the program was administered in September 2014 as a mandatory preparatory exercise prior to a second-year workshop on advanced communication skills. During this workshop, students rotated through separate teaching sessions on risk communication, motivational interviewing, and dealing with “difficult” patients. The 70-min risk communication workshop consisted of a 10-min recap and introduction, followed by three 20-min dyadic case-based role-play sessions, in which each student “doctor” took 15 min conveying risk information to a fellow student “patient,” and then received 5 min of group feedback. One week before the scheduled workshop, students were emailed the URL and instructions on how to access the online program.
At WUSM, the program was administered in March 2015 as part of the Scientific Methods sub-section of the first-year Practice of Medicine course. All first-year WUSM students were required to complete the Risk Talk Online program in preparation for a 90-min workshop session on risk communication. This session included didactic instruction in risk communication and a faculty-led demonstration with an SP. One week before the workshop session, students received an email with a link and directions on how to complete the program and the associated pre- and post-assessments.
A baseline and post-program survey, containing the measures described below, were administered before and after program completion using the HIPAA-compliant web survey platforms Qualtrics® (at TUSM) and DatStat® (at WUSM). At TUSM, survey completion was mandatory, although students could voluntarily opt out of having their responses used for research; at WUSM, all student data was deidentified prior to analysis and thus no voluntary opt-out was required. The study protocol was exempted from Institutional Review Board (IRB) review at Tufts University School of Medicine and Washington University School of Medicine.
Program Evaluation and Measures
Evaluation focused on the program’s feasibility, acceptability, and efficacy. Feasibility was assessed by measuring (1) program completion rate and (2) self-reported program completion time. Students were also asked whether they completed the program in a single sitting. Acceptability was assessed using a three-item measure of students’ perceptions of the program’s value, adapted from our prior study (Appendix 2-1).
Efficacy was assessed by measuring students’ subjective (self-reported) and objective risk communication competence before and after completing Risk Talk Online. Subjective risk competence was assessed using our previously developed three-item measure of confidence in risk communication skills [6] (Appendix 2-2). This measure ascertained students’ perceived level of confidence in their understanding of medical statistics, ability to effectively communicate medical statistics to patients and/or families, and ability to help patients and/or families understand information about probabilities or risks. Objective risk communication competence was assessed by a newly developed four-item measure (Appendix 2-3) consisting of four case vignettes testing students’ knowledge of practical communication strategies for accomplishing key tasks emphasized in our curriculum: (1) framing risk reduction in absolute versus relative terms, (2) explaining the reference class, (3) conveying uncertainty in risk information, and (4) simplifying risk information to facilitate understanding. These items were written to be practical in orientation—i.e., to assess knowledge of not only theoretical concepts, but of preferred approaches (language and communication strategies) to convey risk information in clinical encounters.
Data Analysis
Descriptive statistics for all variables were computed and within-subjects, pre-post changes in subjective and objective risk communication competence were assessed using paired-sample t tests. The proportions of correct responses to individual objective communication competence items were also computed, and pre-post changes were compared using McNemar’s test. The critical level of significance was set at P <0.05.
Results
Feasibility and Acceptability
Overall, 204/210 (97 %) TUSM students and 115/123 (94 %) WUSM students completed the online program; 6 TUSM students opted out of allowing use of their surveys in the study. Average self-reported program completion time was 40.3 min at TUSM and 41.7 min at WUSM. The proportion of students who completed the program in a single sitting was 90.9 % at TUSM and 90.4 % at WUSM. Students gave high value ratings—approaching the six-point maximum value—for the content (M = 4.67, SD = 1.20), videoclips (M = 4.66 SD = 1.29), and impact of the program (M = 4.53, SD = 1.14).
Efficacy
Within-subject pre-post differences in subjective and objective risk communication competence are shown in Table 3. A programming error prevented matching baseline and post-program surveys to unique individuals in the TUSM cohort; we therefore applied probabilistic matching based on survey date, time, and IP addresses, which enabled successful survey matching for 143/204 (70 %) of TUSM students. Students at both sites demonstrated significant improvement on all measures after program completion.
Performance on individual objective risk communication items are shown in Table 4. The proportion of correct responses was lowest for the item on framing risk reduction in absolute versus relative terms, intermediate for the items on explaining the reference class and simplifying risk information, and highest for the item on communicating uncertainty in risk estimates. Pre-post changes were significant for each item, although largest for the item on the reference class and smallest for the item on explaining uncertainty.
Discussion
In this study a new risk communication training program, Risk Talk Online, was developed, implemented at two US medical schools, and found to be feasible, acceptable, and efficacious. The program was completed in approximately 40 min and in a single sitting by over 90 % of students, who rated its value very highly. The program produced significant improvements in subjective and objective risk communication competence. Together, these findings provide strong proof-of-principle evidence supporting the program’s value and generalizability.
Importantly, Risk Talk Online was designed and implemented not as a stand-alone intervention but as part of a more comprehensive “flipped-classroom” training program integrating additional didactic instruction and experiential skills practice in classroom settings. Our findings support the need for such additional instruction. Despite significant improvement in students’ objective risk communication competence after they completed the online program, only a minority (<20 %) were able to correctly answer all four knowledge questions. Students struggled with particular items including the need to frame risk reduction in absolute (versus relative) terms. Although this concept received great emphasis in the teaching program, our data suggest the need for more reinforcement as well as refinement of the program’s content. It should be noted, furthermore, that the knowledge measure used to assess objective risk communication competence was new, and further work to refine the measure and assess its reliability and validity is an important next step for future research.
More research is also needed to determine whether online training in risk communication improves actual communication skills, which were not assessed in the current study. We do have convincing evidence—from our previous OSCE-based experimental study of the Risk Talk curriculum on which the content and approach of the current e-learn program is based—that our overall teaching intervention improves communication skills [6]. It stands to reason, furthermore, that had students in the current study also received experiential training prior to their competence assessments, even greater improvements might have been observed. Nevertheless, whether the Risk Talk Online program—either alone or in combination with workshop-based experiential training—results in similar improvement is a key question for future research.
Broader integration and dissemination of risk communication training in medical education ultimately requires generalizable strategies for overcoming curricular time and resource barriers. We believe that e-learning programs such as Risk Talk Online provide one promising strategy for accomplishing this goal. Similar “flipped classroom” programs have been successfully employed in the teaching of other clinical skills and may facilitate dissemination of risk communicating training at other levels of medical education, including not only undergraduate but graduate and postgraduate training. For all types and levels of learners, e-learning programs for teaching risk communication skills would ideally be deployed as one component of a more comprehensive curriculum, but they could also serve as stand-alone programs if curricular time and resources were limited. We believe risk communication is such a critical advanced communication skill that even some basic training is far preferable to none at all.
Several limitations qualify our study findings and call for further research. Although the Risk Talk Online program was well received, its usability was not evaluated. Formal usability testing and refinement of the program are a necessary focus for future work. The durability of students’ knowledge improvements, and their relationship to patient care outcomes, also remain to be shown. Further research is also needed to assess the feasibility and effectiveness of implementing Risk Talk Online at different points in the medical school curriculum or for different levels of learners. The optimal strategies for integrating risk communication teaching with other related components of the medical school curriculum—e.g., instruction in evidence-based medicine and advanced communication skills—also require further investigation. Finally, more work is needed to explore how risk communication teaching might be incorporated in graduate and continuing medical education.
Despite these limitations, the current study provides promising evidence on the educational value of an online clinical risk communication teaching program. We believe that Risk Talk Online represents an important first step in the development of feasible and effective strategies for teaching risk communication skills, and in disseminating this teaching in medical education. The current study provides proof-of-principle evidence of the program’s value, and paves the way for further work to refine the program and improve physicians’ competence in communicating risk information to patients.
Conclusions
Risk Talk Online, a new online risk communication teaching program, is feasible, acceptable, and efficacious in improving subjective and objective risk communication competence among first- and second-year medical students. More research is needed to refine the program and determine optimal strategies for integrating risk communication training in medical education.
References
Edwards A, Elwyn G, Mulley A. Explaining risks: turning numerical data into meaningful pictures. BMJ. 2002;324(7341):827–30.
Legare F, Moumjid-Ferdjaoui N, Drolet R, Stacey D, Harter M, Bastian H, et al. Core competencies for shared decision making training programs: insights from an international, interdisciplinary working group. J Contin Educ Heal Prof. 2013;33(4):267–73.
Elwyn G, Edwards A, Hood K, Robling M, Atwell C, Russell I, et al. Achieving involvement: process outcomes from a cluster randomized trial of shared decision making skill development and use of risk communication aids in general practice. Fam Pract. 2004;21(4):337–46.
Sedgwick P, Hall A. Teaching medical students and doctors how to communicate risk. BMJ. 2003;327(7417):694–5.
Joekes K, Sedgwick P, Hall A. Assessment of risk communication by objective structured clinical examination. Med Educ. 2009;43(5):484.
Han PK, Joekes K, Elwyn G, Mazor KM, Thomson R, Sedgwick P, et al. Development and evaluation of a risk communication curriculum for medical students. Patient Educ Couns. 2014;94(1):43–9.
Lipkus IM. Numeric, verbal, and visual formats of conveying health risks: suggested best practices and future recommendations. Med Decis Mak. 2007;27(5):696–713.
Visschers VH, Meertens RM, Passchier WW, de Vries NN. Probability information in risk communication: a review of the research literature. Risk Anal. 2009;29(2):267–87.
Han PK. Conceptual, methodological, and ethical problems in communicating uncertainty in clinical evidence. Med Care Res Rev. 2013;70(1 Suppl):14S–36S.
Gigerenzer G, Edwards A. Simple tools for understanding risks: from innumeracy to insight. BMJ. 2003;327(7417):741–4.
Acknowledgments
The study was supported by an Innovations in Education Grant Award from the Tufts University School of Medicine. The authors wish to thank Paul Begin and Jason Welborn from MaineHealth Educational Services for assistance in video production and editing, and Susie Mueller from the Washington University School of Medicine for her assistance administering the Risk Talk Online program and managing the assessment data.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1 Selected screenshots from Risk Talk Online module
Appendix 2. Study measures
-
1.
A cceptability of Risk Talk Online module
On a scale of 1 to 6 (where 1 = strongly disagree and 6 = strongly agree), please rate your level of agreement with the following
-
1.
The content of the “Risk Talk” module was useful
-
2.
The video clips in the “Risk Talk” module were useful
-
3.
The “Risk Talk” module made me feel more comfortable talking about risk information with patients
-
2.
Subjective risk communication competence
On a scale of 1 to 6 (where 1 = not at all confident and 6 = extremely confident), please rate your level of confidence in the following:
-
1.
Your understanding of medical statistics
-
2.
Your ability to effectively communicate medical statistics to patients and/or families
-
3.
Your ability to help patients and/or families understand information about probabilities or risks
-
3.
Objective risk communication competence*
Please evaluate the information presented in the following scenarios, and select the single best response to each question.
-
Scenario 1: In the MAGIC trial, a large randomized controlled clinical trial, the drug MAGICSTATIN reduced the risk of myocardial infarction from 4/100 to 3/100, a 25 % relative risk reduction. Which of the following is the best way of communicating this information to your patient?
-
A.
Taking MAGICSTATIN will reduce your risk of a heart attack by 25 %.
-
B.
Taking MAGICSTATIN will reduce your risk of a heart attack by 25/100.
-
C.
Taking MAGICSTATIN will reduce your risk of a heart attack by 1 %.
-
D.
Taking MAGICSTATIN will reduce your risk of a heart attack from 4/100 to 3/100.
THE CORRECT ANSWER IS D : This way of communicating provides the most complete picture of the patient’s risk reduction from MAGICSTATIN. It emphasizes the absolute reduction in risk—from an average baseline risk of 4 % to a reduced risk of 3 %.
-
Scenario 2: Mr. Smith is 62 years old, and very interested in taking MAGICSTATIN after seeing a television advertisement showing the MAGIC trial data. His total and LDL cholesterol have remained elevated despite his diligent attempts at diet and exercise over the past 3 years. He also has mild hypertension which has been well controlled on a diuretic. Which of the following is the most important factor that Mr. Smith needs to consider in deciding about whether MAGICSTATIN is the right treatment for him?
-
A.
Whether most doctors tend to recommend MAGICSTATIN or similar drugs for patients with Mr. Smith’s cardiac risk factors.
-
B.
How closely Mr. Smith’s own cardiac risk factors match those of the patients who participated in the MAGIC trial.
-
C.
Whether most patients with Mr. Smith’s cardiac risk factors tend to choose MAGICSTATIN or similar drugs.
-
D.
A and B
THE CORRECT ANSWER IS B: In deciding whether a drug is appropriate for a particular patient, the most important consideration is whether the patient matches the “reference class”—the patient population in which the drug has been evaluated. The more closely that Mr. Smith’s cardiac risk factors match those of the patients for whom MAGICSTATIN has been shown to be effective, the more confidence Mr. Smith can have that he will derive the same benefit from the drug.
Scenario 3: Your patient Mrs. Jones comes to your office to ask your help in interpreting some information she received online. She used an established, well-validated “risk calculator” produced by the National Cancer Institute, which computed a “24.3 % risk” of developing breast cancer in her lifetime, compared to an average risk of 10.8 %. She is worried and asks, “What does this number mean: should I be concerned?” Which of the following is the best response to Mrs. Jones?
-
A.
The number means you are definitely at higher than average risk for breast cancer. There’s reason to be concerned, but it’s impossible to know whether you or any one person will develop cancer or not.
-
B.
The number means you are definitely at higher than average risk for breast cancer. There’s no reason to be concerned, however, because it’s impossible to know whether you or any one person will develop cancer or not.
-
C.
The number means you may be at higher than average risk for breast cancer. There’s reason to be concerned, but it’s impossible to know whether you or any one person will develop cancer or not.
-
D.
The number means you may be at higher than average risk for breast cancer. There’s no reason to be concerned, however, because it’s impossible to know whether you or any one person will develop cancer or not.
THE CORRECT ANSWER IS C: This response appropriately conveys the fundamental uncertainty in all risk estimates, while also acknowledging the value of risk estimates. No matter how strong our risk evidence is, we can never definitely determine a single person’s risk for a future event. It is also impossible to know whether any given person—regardless of their estimated risk—will experience a future outcome such as cancer. Yet in spite of these uncertainties, evidence-based risk evidence, such as the estimates produced by statistical models, provides essential guidance for health care decisions, and should be taken seriously rather than dismissed.
-
Scenario 4: J.S. is a 37-year-old woman who is 16 weeks pregnant and considering prenatal screening. The best available evidence suggests that her baby boy has a 1 in 122 risk of a chromosomal abnormality at delivery, and a 1 in 242 risk of Down syndrome. She is trying to decide whether to undergo amniocentesis for further testing, which carries a 1 in 200 risk of pregnancy loss. Which of the following is the best way of explaining these risks to J.S.?
-
A.
Your child’s risk of a chromosomal abnormality is about 1 %, his risk of Down syndrome is a little less than 0.5 %, and the risk of pregnancy loss as a result of amniocentesis is about 0.5 %.
-
B.
Your child’s risk of a chromosomal abnormality is about twice as high as his risk of Down syndrome or a pregnancy loss as a result of amniocentesis.
-
C.
Your child’s risk of a chromosomal abnormality is 1 in 122, his risk of Down syndrome is 1 in 242, and the risk of pregnancy loss as a result of amniocentesis is 1 in 200.
-
D.
Your child’s risk of a chromosomal abnormality is about 1 in 120, his risk of Down syndrome is about 1 in 240, and the risk of pregnancy loss as a result of amniocentesis is about 1 in 200.
THE CORRECT ANSWER IS A: This response provides the quantitative information that patients need to simultaneously compare three risks, while also simplifying the information and minimizing the amount of arithmetic and computation that patients themselves need to perform. It avoids the “1-in-x” format which is confusing to interpret, converts unusual fractions such as “1/122” into percentages, and then uses rounding so that patients can compare multiple risks more easily.
*Correct answers in italics, provided to respondents after module completion
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Han, P.K.J., Piccirillo, J., Gutheil, C. et al. Development and Evaluation of an Online Risk Communication Teaching Program for Medical Students. Med.Sci.Educ. 26, 557–567 (2016). https://doi.org/10.1007/s40670-016-0290-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40670-016-0290-3