INTRODUCTION

Physicians have contact with at least 70 % of adult smokers in the United States annually1 and have an essential role in helping patients stop smoking. As brief interventions can have a substantial impact on adult smokers,2,3 the U.S. Preventive Services Task Force gives physician-delivered tobacco treatment interventions (e.g. 5A counseling, “Ask, Advise, Assess, Assist, Arrange”) an “A” recommendation, indicating that physicians should offer this service at every office visit. Unfortunately, use of the 5As among physicians is limited1,4–6 as is the amount of tobacco treatment training that physicians receive.7–11 Developing the competencies needed for tobacco dependence treatment and integrating them into routine practice should start early in medical education.12

Medical students are inadequately prepared to help patients modify health behaviors such as tobacco use.13 A curriculum that optimally combines didactics, interactive counseling practice through role-play or the use of standardized patients, and observed clinical experiences with smokers is not the norm.14 Earlier reports indicated that U.S. medical school curricula devote only 4 h during the 4 years to training for tobacco dependence treatment.15 Therefore, there is a gap between the acknowledged importance of developing tobacco dependence treatment skills and the level of tobacco dependence treatment training provided in medical school.13 This is unfortunate, given that competency in tobacco dependence treatment is now recommended as a core graduation requirement for all new physicians and other healthcare professionals.16,17

The goal of the current study (MSQuit) was to develop, implement, and evaluate a comprehensive multi-modal education (MME) intervention for teaching the effective delivery of tobacco dependence treatment to medical students. We hypothesized that MME delivered over multiple school years would improve medical students' tobacco dependence treatment skills beyond traditional education (TE). MSQuit compared the two educational interventions in the context of a group-randomized controlled trial.

METHODS

Trial Design

Ten U.S. medical schools were pair-matched (see "Pair-Matching and Randomization" section) and randomized to MME or TE curriculum. The primary outcome was student performance on a tobacco-specific objective structured clinical examination (OSCE) case given to third-year medical students. Using a cross-sectional design, OSCE scores for students from the class of 2014 who participated in MME or TE were compared with those of students at the same schools from an earlier cohort (class of 2012) who did not participate in the intervention. Using a cohort design, the secondary outcomes of self-reported skills (i.e. self-efficacy) for performing each 5A behavior and pharmacotherapy counseling among MME and TE third-year students (class of 2014) were compared.18

Participating Schools

The ten medical schools that participated in the study are noted in our previous publication.18 Schools met the following criteria: 1) an enrollment of ≥90 matriculating first-year (MS1) students; 2) ≤4 h of tobacco treatment training over 4 years of medical school; 3) a family medicine or internal medicine clerkship for third-year (MS3) students; and 4) a standard evaluation of MS3s using an OSCE examination, with agreement by the medical school to include a new tobacco case. Institutional review boards at each participating medical school approved the study. Three of the ten schools had participated in an earlier study of tobacco dependence treatment training in medical schools;11 the other seven were recruited through networks with colleagues. Schools were both private (N = 3) and public (N = 7), geographically diverse, and located in both inner-city and outlying areas.

Pair Matching and Randomization

Before randomization, schools were pair-matched based on their total overall school ranking in self-reported skills for performing the 5As and pharmacotherapy counseling (see Primary and Secondary Outcomes Measures) via a survey of third-year medical students (MS3) from the class of 2011. Schools were then ranked based on the school’s average score, and were randomized to MME or TE with their matched school.

Educational Interventions

MME: Multi-Modal Education

All research personnel and site principal investigators from each of the ten schools participated in the iterative development of each MME learning component. The MME intervention consisted of components that took place either in the first or third year of medical school. These included 1) a 4-h self-paced Web-based course for MS1s covering all 5As, with particular emphasis on Assist and Arrange and pharmacotherapy counseling; 2) a role-play classroom demonstration and practice implemented in tandem with the Web course; and 3) a clerkship booster session during the third-year family or internal medicine clerkship rotation that reinforced prior tobacco dependence treatment instruction.

Clerkship preceptors from either the family or internal medicine clerkship attended a 30- to 60-min individual or group academic detailing19 session reviewing tobacco dependence treatment guidelines, as well as how to model, observe, instruct, and provide critical feedback to students in the specific 5A and pharmacotherapy counseling skill set for tobacco treatment. MME students, therefore, participated in an enhanced clerkship rotation in which students and preceptors were strongly recommended to work together with patients who smoked. MME schools were encouraged to continue providing their traditional tobacco treatment education concurrent with MME implementation.

TE: Traditional Education

Students and preceptors in the five TE schools continued their traditional tobacco education. This curriculum over the 4 years of medical school was reported to be ≤4 h in duration and varying in content. Traditional teaching was largely didactic, interspersed among basic science and behavioral counseling classes in medical school years 1 and 2. Some schools also included tobacco case studies as part of their behavioral counseling role-plays taught to first- or second-year students.

Intervention Fidelity

Each MME component was provided to MME schools and their faculty instructors with standardized written guidelines, instructions, objectives, and discussion points to ensure that students would receive the intervention in a similar manner and within a specified time frame (e.g. during MS1 year). Academic detailers (ADs) were first trained by UMMS researchers in an 8-h training session, and received follow-up supervision. Prior to the teaching of preceptors, ADs practiced their sessions with “practice participants”, and all sessions were audiotaped. Each AD completed fidelity checklists after each session with preceptors to ensure that all pertinent topics were covered.

Primary and Secondary Outcomes Measures

The primary outcome, observed 5As plus pharmacotherapy counseling for tobacco dependence treatment, was measured by a structured OSCE, a standard method for evaluating medical student skills at U.S. medical schools.20,21 OSCEs were implemented within the context of existing clinical skills assessments. School-level scores from the class of 2012 were compared with school-level scores from the class of 2014. About 50 % of the OSCEs for the class of 2012 were randomly selected for coding in order to provide a pre-intervention school-level comparison. In contrast, 100 % of OSCEs for the class of 2014 were coded. This nested cross-sectional design22 was chosen because OSCEs are not given to MS1s.

OSCEs were videotaped and then scored using a 33-behavior checklist (Yes/No items) covering multiple aspects of each 5A behavior and four additional items for communication skills that were scored from 1 = poor to 5 = excellent; total scores ranged from 4 to 20. The development and quality assurance measure related to the study checklist are described in Mazor and colleagues (in press). Previous research by Mazor23 demonstrated the effectiveness of such an approach, and the 2008 update to the clinical guidelines24 includes a recommendation to utilize interviewing strategies for increasing motivation. Items were approved by a scientific advisory group of external tobacco researchers and clinicians, and are based on clinical guidelines.24 Three blinded and trained raters viewed the video recording of the OSCE and completed the checklist. A random selection of 10 % of raters’ scoring was compared against the rating by a “gold-standard” supervisor, with little variation noted. The gold-standard supervisor received extensive training and practice in completing the checklist from the study staff responsible for developing the checklist. Standardized patients portraying the tobacco case were not responsible for scoring student performance on this OSCE. Secondary outcomes were student self-reported skill level for performing each of the 5As and pharmacotherapy counseling (6 items total). All students from the class of 2014 completed a questionnaire as MS1s and again at post-intervention as MS3s. Items included, “Asking about smoking at every visit”, “Advising smokers to quit”, “Assessing smokers’ willingness to quit”, “Assisting smokers to develop a quit plan”, “Arranging follow-up contact regarding smoking”, and “Discussing and prescribing pharmacotherapy options”. Although the discussion of pharmacotherapy can be considered an “Assist” behavior, this was asked as a separate item to assess self-efficacy for this specific skill. Each item was dichotomized as “moderately or very skilled” vs. “not at all or somewhat skilled”.

Intervention Tracking and Dosage Measures

Study coordinators at each school tracked each student’s participation in the role-play and booster MME component (Yes/No). The study-tracked total MME intervention dose received per student was computed as the total number of components completed (0 through 3). Each school’s academic detailer also recorded the attendance of preceptors in their academic detailing sessions. Third-year students at MME and TE schools were also asked whether they had participated in a Web-based course and role-play classroom demonstration regarding tobacco dependence treatment. In addition, students completed a series of questions to assess the extent of the clerkship preceptors’ provision of tobacco training instruction for students. Students rated on a four-point scale (1 = Strongly disagree to 4 = Strongly agree) how much they agreed with seven statements related to their preceptor providing them with clear instruction, feedback, and training regarding tobacco dependence counseling. The self-reported total intervention dose was computed as the sum of having received each of the MME components (range 0 to 3).

Other Relevant Measures

Third-year students from both cross-sectional (MS3 students from the classes of 2012 and 2014) and cohort (MS1 and MS3 students from the class of 2014) designs also answered questions related to demographics (e.g. age, gender), their intended career choice (primary care, specialty, other, or unsure), and their smoking history (e.g. had they smoked ≥ 100 cigarettes in their lifetime).

Statistical Analysis

The analyses of OSCE scores used a two-stage mixed model analysis of variance (ANCOVA).23,25 In the first stage, 20 (10 schools × 2 time-points) means were estimated from an ANCOVA for student-level OSCEs as a function of school, time-point, and their interaction. In the second stage, the ten post-intervention school means were regressed on intervention assignment and the corresponding pre-intervention school mean. This approach adjusted for the MME and TE comparison for baseline school means while accounting for within-school correlation.

Due to missing data on the OSCE scores for about 27 % of the class of 2014, we completed a sensitivity analysis using a multiple imputation approach appropriate for group-randomized controlled trials.26 The following items were used for imputation: baseline school mean, randomization assignment, history of smoking in the past 30 days, history of providing tobacco dependence treatment counseling or clinical work prior to medical school, frequency of tobacco dependence treatment instruction and observation during medical school, tobacco-related knowledge, and 5A self-efficacy and intentions. We used available data from the students’ baseline and follow-up surveys to impute missing OSCE scores. Ten imputed datasets were combined to estimate the difference in mean OSCE scores for the MME and TE schools, adjusting for baseline mean OSCE score and treating school as a random effect, thereby mimicking the primary analysis.

The 5A and pharmacotherapy counseling self-efficacy skills outcome used a mixed-effects logistic regression,26 adjusting for student’s own pre-intervention 5A and pharmacotherapy counseling dichotomized scores, treating school as a random effect.

To examine the association of MME “dosage” with total OSCE score, we used linear mixed modeling adjusting for the school-specific baseline mean total OSCE score, treating school as a random effect. We conducted two separate analyses for study-tracked dosage scores and student- reported dosage score. All analyses used SAS Version 9.327 or R 2.15.2 statistical software.28 Supplemental analyses of individual OSCE items did not adjust for multiple comparisons, focusing instead on consistency of condition-related differences across items.

RESULTS

Participating Students

There were 766 MS3s (class of 2014) at MME schools who were eligible for the study. Of these, 565 (73.8 %) completed a third-year post-intervention OSCE, and 552 (72.1 %) completed both the baseline and post-intervention surveys. There were 737 MS3s (class of 2014) at TE schools who were eligible for the study. Of these, 531 (72.0 %) completed a third-year post-intervention OSCE, and 495 (67.2 %) completed both the baseline and post-intervention surveys. Among students who completed OSCEs or follow-up self-efficacy surveys 2 years later, no MME vs. TE differences in baseline demographics or smoking history were noted (Table 1). Reasons for missing outcomes are reported in Fig. 1. On the follow-up survey, students reported their intended career as follows: primary care (37 %), medical specialty (37 %), other (20 %), and undecided (6 %); there were no other differences by study condition.

Table 1 Characteristics of Student Cohort by Randomization Assignment
Full size table
Fig. 1
figure 1

Consort diagram/flow diagram. a Primary outcome OSCE data. b 5A self-efficacy survey data

Full size image

MME Intervention Participation and Fidelity

Among 612 eligible MME school students in the OSCE or self-efficacy analyses, study data objectively reported that 89.5 % completed the Web-based course, 82.7 % completed the role-play activity, and 83.5 % completed the third-year booster session; 72.4 % completed all three intervention components. Of the 268 eligible clerkship preceptors from the five MME schools, 83 % participated in an academic detailing session. Self-reported receipt of each type of learning component was higher among MME than TE students at post-intervention: 91.7 % vs. 43.9 % for receipt of any Web-based education (p < 0.001), 90.0 % vs. 63.4 % for receipt of any role-play participation (p = 0.008), and 51.5 % vs. 38.3 % for student-reported optimal preceptor training (p = 0.10).

Additionally, at post-intervention, a greater percentage of MME versus TE students reported >5 h of tobacco education during their pre-clinical years (42.4 % vs. 21.8 %, p = 0.06). In general, among third-year students completing an internal medicine clerkship, MME students were more likely than TE students to agree that their preceptors provided clear instruction, feedback, modeling and clear objectives, and responded to questions about tobacco counseling (Table 2). Likewise, MME students from internal medicine clerkships reported a greater likelihood of advice for patients to quit smoking on medical records and educational materials being prominently displayed. For the family medicine clerkship, there were no differences between MME and TE students.

Table 2 Student Ratings of Preceptor Teaching in Family Medicine and Internal Medicine by Randomized Assignment
Full size table

OSCE Outcomes

There was no significant difference between MME and TE students (Table 3) in total OSCE scores. This was consistent in the multiple imputation analyses (estimated mean MME 0.81 points higher, p = 0.41). MME students outperformed TE students on 27/33 items, and significant differences (ps < 0.05) were seen among 6 of the 33-item OSCE checklist (Table 3). MME students had higher communication scores than TE students, but this difference was not statistically significant (p = 0.21) (Table 3).

Table 3 OSCE Scores and Individual OSCE Items: Comparison of MME and TE Educational Interventions
Full size table

Specific findings included the fact that more than 70 % of students in both MME and TE asked the standardized patient (SP) about their frequency of smoking per day and informed the SP that their cough was related to smoking. MME students, however, were more likely than TE students to report discussing the SP’s confidence to quit, yet fewer than 20 % of students provided this assessment. On each of the 18 Assist behavioral items, MME students scored higher (6 were related to nicotine replacement therapy [NRT] and other pharmacotherapy), and they were statistically significantly more likely than TE students to complete three of these items (Table 2). With regard to the Arrange items, MME students were significantly more likely than TE students to arrange for follow-up care via the quitline and to encourage discussion with a primary care provider.

Association Between MME Dosage and Total OSCE Scores

Total study-tracked intervention dose, relevant only for MME students, was positively associated with total OSCE score, as reflected in an estimated increase of 0.44 points (SE 0.24, p = 0.06) with receipt of each additional intervention component. Among both MME and TE students, the magnitude of the association of total self-reported intervention dose, available for all students with total OSCE score, was weaker than for study-tracked intervention dose, reflected in a 0.12-point (SE = 0.12, p = 0.34) increase in total OSCE score with receipt of each additional intervention component.

5As and Pharmacotherapy Counseling Self-Efficacy Outcomes

At post-intervention, a higher percentage of MME versus TE students agreed or strongly agreed that they were able to advise or assess (p < 0.10) and assist, arrange, and provide pharmacotherapy counseling (p < 0.05) (Fig. 2).

Fig. 2
figure 2

Self-efficacy by randomization assignment

Full size image

DISCUSSION

In this randomized study of ten U.S. medical schools, MME students receiving a combination of a Web-based course, role-play classroom demonstration, and preceptor-facilitated tobacco dependence treatment training and a booster session as MS1s and MS3s, respectively, were more likely to perform several tobacco dependence treatment-related 5A and pharmacotherapy counseling behaviors on an OSCE checklist, but demonstrated no overall change in total OSCE score relative to TE students. However, MME students were more likely to report higher self-efficacy skills for many 5A and pharmacotherapy counseling tobacco dependence behaviors.

At study inception, we anticipated that performance on the OSCE and self-report on the performance of the 5As and pharmacotherapy counseling would evaluate different aspects of student mastery of tobacco dependence treatment competencies. OSCEs attempt to measure actual observed practice, although they provide only a snapshot of student skills in a simulated clinical situation. Student self-report seeks to measure the student’s self-efficacy in tobacco dependence treatment skills that may or may not be associated with actual performance, and possibly subject to over- or under-estimation.

The MSQuit study provided a competency-based multi-modal curriculum with individual components that were evaluated in earlier studies with medical students and for teaching tobacco dependence treatment specialists.29 The four major components of the MSQuit intervention had high penetration (as measured by an objective tracking mechanism and via student self-report). The Academic Detailing18,19,30–35 component, albeit brief, resulted in MME MS3 students being more likely than TE students to report receipt on all measures of interest, including clear instruction, feedback, modeling, and clear objectives, for providing tobacco dependence treatment.

Despite more hours of tobacco dependence treatment training and strong indications of the receipt of tobacco dependence treatment education in years 1 and 3, total OSCE scores among MME students were low, and were not significantly higher than scores of TE students. There are several possible explanations. First, there was little room for improvement for the Ask and Assess measures, as both MME and TE students scored relatively high for these two behaviors at pre-intervention. This finding is consistent with the tobacco dependence treatment literature for practicing physicians, suggesting that this skill is already well-integrated into training programs.6 Second, achieving a higher total score on the OSCE may have been difficult, since third-year medical students rarely have in-depth or long-term contact with patients who smoke. Third, the time-limited nature of the OSCE may have precluded some students from asking additional questions or providing more counseling for components on which they had been well-trained. Notably, on the OSCE, only 61 % of MME students were reported to advise patients to quit. However, more than 80 % of students self-reported as moderately to very skilled in the Advise component of the 5As. Fourth, there was no formal teaching for MME second-year students; thus there was a lapse of at least 18 months between the first-year pre-clinical component and the preceptor-led teaching and booster session for third-year students, which could have led to regression to original skills as first years. A more systems-oriented, longitudinal curriculum embracing the first three years of medical school might have been more beneficial. Fifth, a voluntary 30–60-min academic detailing program, while attended by more than 80 % of MME preceptors, may have been insufficient to change preceptors’ tobacco dependence treatment and precepting skills. In addition, the preceptor may have modeled a modified approach to the 5As, such as Ask and Act, which has been promoted to practicing physicians in recent years.36

Strengths and Limitations

Major strengths evident in this study are the importance of assembling a team of physician and curriculum specialist innovators to develop and implement a medical education study, using a randomized design, with high fidelity across multiple and diverse medical schools, and rigorous evaluation via two time-tested methods for assessment and evaluation, notably an OSCE and student self-report.

The study may have limited generalizability, as only 10 of the 125 existing U.S. medical schools in 2009 were involved. Second, ≤4 h of tobacco treatment information in the curriculum was used as an inclusion criterion. This number indicates that the average number of hours for tobacco treatment training was based on current research at the time. Third, individual schools were not blinded to the type of educational intervention they were receiving. Fourth, while students were expected to complete a full 5As interaction with at least two patients who smoked, we cannot quantify the time students spent in tobacco counseling during the clerkship. While MME students reported greater preceptor-to-student tobacco dependence communication, there were no objective measures of the quantity or quality of the preceptors’ training for individual students, nor was this standardized. Finally, the use of a single OSCE case may limit the generalizability of the results when considering variations in readiness to quit among patients.

CONCLUSIONS

Our findings highlight the continued need for additional tobacco dependence treatment training in U.S. medical schools, consistent with proposed reforms in undergraduate medical education by the Association of American Medical Colleges5 and he Institute of Medicine.37 Important lessons that emerged from the study were that educational models for such training should ideally include core knowledge building and role-play practice in medical school years 1 and 2, more intensive academic detailing and booster follow-up for medical school preceptors, and encounters with at least 2–3 smokers in clinical settings.

The MSQuit study did not demonstrate a significant difference in the total OSCE score for MME students. However, the significant change in self-efficacy will likely result in eventual change in behavior or clinical performance.38 Heightened self-efficacy is an essential prerequisite for effecting a change in behavior.39 Likewise, learning occurs through a series of interactions that reinforce and layer on new, more complex skills over time.38 Building on what we have learned about tobacco treatment training for medical students will be an important next step.