E-Cigarettes: Harmful or Harm-Reducing? Evaluation of a Novel Online CME Program for Health Care Providers



Patients are asking health care providers about e-cigarettes, vaping, and other electronic nicotine delivery systems (ENDS). Provider advice on ENDS has varied greatly, suggesting a need for evidence-based continuing medical education (CME).


A novel free online CME course was developed on ENDS risks and benefits, product types (e.g., vape pens, pods), and screening and counseling best practices for adults, adolescents, and different smoker profiles (e.g., daily, social).


From January 2017 through June 2018, 1061 individuals accessed the course: 46% physicians, 7% physician assistants, 7% nurse practitioners, 15% nurses, 4% pharmacists, and 28% allied health/student/other; 41% were international.

Program Description

The course was built from observed online patient-provider interactions. Through video role-plays, expert interviews, and interactive activities, the course engaged learners in the evidence on ENDS. Completers earned 1.5 CME units.

Program Evaluation

A total of 555 health care providers earned 832.5 CME units. Pre- to post-test scores significantly increased from 57 to 90%; 76% rated the course as above average (41%) or outstanding (35%); 99% indicated the course was free of commercial bias.


Addressing the growing need for balanced provider education on ENDS, this interactive online CME engaged learners and increased knowledge on devices and evidence-based cessation approaches.


Electronic nicotine delivery systems (ENDS) entered the US market in 2007.1 In 2017, 6.9 million (2.8%) adults reported every day or some day use of ENDS, with greater use (7.9%) among current conventional smokers.2 Adults report dual use of ENDS in places where smoking is banned, as a means to reduce the number of cigarettes smoked per day, and in an effort to quit smoking.3,4

While some observational studies suggest a benefit of ENDS use, particularly daily use, for quitting smoking,5 randomized controlled trial evidence of ENDS for quitting smoking is limited.6 Two cessation studies showed no advantage of ENDS relative to placebo (i.e., ENDS without nicotine). The third, and most recent trial (published after release of this CME), tested ENDS with behavioral support against nicotine replacement therapies (NRT).7 The study found significantly higher cigarette quit rates for ENDS (18%) relative to NRT (10%) at 1-year follow-up. Among those who quit smoking, 80% continued to use ENDS and 9% were using NRT at 1-year follow-up, suggesting the potential of developing dependence to ENDS. ENDS and NRT-reported side effects were largely comparable. The UK Royal College of Physicians concluded that ENDS appear effective when used to quit smoking and that their risks are unlikely to exceed 5% of the harm from smoking tobacco.8 In contrast, leading US health agencies concluded there is insufficient evidence to recommend ENDS use for cessation.9,10,11 Instead, their guidelines recommend use of cessation medication treatments with established effectiveness and safety. The National Academies of Sciences concluded that the health benefits of ENDS use among continued smokers (i.e., dual users) are unproven, and the long-term health effects for ENDS users and the effects of maternal ENDS use on fetal development are unknown.6 To date, no ENDS product has been approved as a therapeutic cessation aid by the US Food and Drug Administration (FDA). However, in one study, 38.5% of current smokers believed the FDA had approved ENDS for cessation.12

While ENDS may represent a form of harm reduction for adult smokers, few would suggest a benefit of ENDS use in adolescence. ENDS expose the developing brain to nicotine with serious potential for harm in igniting addiction and driving sustained long-term use.13 Classified as an epidemic by the US Surgeon General, ENDS are now the leading tobacco product used by US youth,14 with 1 in 5 high school students reporting use.15,16 Multiple prospective observational studies of US adolescents have found that among never smokers, ENDS use is significantly associated with initiating combustible cigarette use.17,18

The increasing popularity of ENDS is reflected in patient-physician interactions. In a survey of 561 US physicians, 70% reported that patients ask them about ENDS at least some of the time.19 In two surveys of providers, about a third recommended ENDS to patients for quitting smoking.20,21 In a survey of 918 adult smokers, 18% reported a provider had supported ENDS as a cessation aid.13 Among adolescent health care providers, 41% said they would tell their patients that ENDS are less harmful than cigarettes, and 24% would recommend ENDS to teens for quitting smoking.22

The observed variation in provider attitudes toward ENDS suggests the need for balanced and evidence-based continuing medical education (CME). Notably, 92% of providers surveyed nationally expressed interest in learning more about ENDS and indicated their most frequent sources of information were patients, news sources, and ads, rather than professional or scientific sources.23 Similarly, in one survey of providers, many expressed low confidence for effectively addressing patients’ questions about ENDS, with varied attitudes toward ENDS as cessation aids.24

To address the need for quality provider education on ENDS, we developed a novel online CME course to train health care providers on ENDS risks and benefits, product types (e.g., vape pens, tanks, pods), and screening and counseling best practices for adults and adolescents and for different smoker profiles (e.g., daily, social). The course was offered at no charge and completers could earn 1.5 CME units. We examined course outcomes, including change in ENDS knowledge, and evaluation of course content, instruction, perceived commercial bias, and users’ intention to integrate course content into their daily practice.


The course was designed for physicians, nurse practitioners, physician assistants, nurses, clinical psychologists, tobacco treatment specialists, and other licensed addiction treatment providers. The course was developed in collaboration with the Stanford Center for CME and hosted on their website. It can be accessed at https://tinyurl.com/stanfordocme. The program was promoted via Stanford Online, as well as through eBlasts to departmental mailing lists.

From January 2017 to June 2018, a total of 1061 people enrolled in the course: 59% were located in the USA, with 86 other countries spanning six continents represented. Learners were 54% female and 46% male, with median age of 31 (21% ages 19–25, 52% ages 26–40, 27% ages 41+). Among learners reporting their terminal degree(s), the distribution was 43% MD, 15% RN, 9% MBBS, 7% PA, 7% NP, 7% PhD, 4% PharmD, 3% MD/PhD, 3% DO or DDS, and 2% other. Most learners (85%) reported actively seeing patients. A total of 45 specialties were reported, with the most frequent being family medicine and community health (21%), internal medicine (15%), psychiatry (10%), emergency medicine (5%), pediatrics (4%), surgery (4%), cardiovascular health (3%), and general practice (3%).

Program Description

The program’s video content, shot with a go-pro, was designed to provide an immersive first-person experience of a day in the life of a health care provider encountering ENDS. The day starts with the provider hearing a radio advertisement for ENDS while driving to work and then seeing a person vaping outside the clinic. In clinic, the provider treats patients asking about ENDS. For credit, learners engage in three interactive video cases of different patient-provider visits: a pre-college physical with a nonsmoker; an older heavy smoker with schizophrenia; and a perioperative check-in for reconstructive breast surgery following cancer treatment. At the end of each case, learners watch three expert videos. Optional activities include reading and responding to a patient email inquiry and visiting a virtual vape shop to learn about different ENDS products and components. A resource list orients learners toward further information about ENDS.

The CME is designed within a nonlinear choose-your-own adventure–type framework. Informed by our prior work analyzing ENDS-related questions from real patients and responses from licensed physicians,21 the CME draws from scientific evidence, emphasizes potential benefits and harms of ENDS, and covers FDA-approved cessation medications. It addresses use for smoking reduction or cessation among adults and use among adolescent nonsmokers. The content, storyboard, and scripts were developed by the interdisciplinary research team, which included physicians; scholars from behavioral science, education, and linguistics; and the instructional design, film, and development team with the Stanford Center for CME. Scholars with expertise in pharmacology and inhalation toxicology were engaged for expert interviews. Four physicians external to the development process served as content reviewers and time testers.

The course takes approximately 1.5 h to complete. Course evaluations are optional. However, to receive a certificate of completion and earn 1.5 CME AMA PRA category 1 credits, learners must complete an 8-item ENDS knowledge assessment right after the CME and score at 75% or higher at post-test. Learners were limited to two attempts on the post-test items. They could access the course while completing the post-test.

Program Evaluation

Through June 2018, 1061 people accessed the course; 587 (55%) completed the course and the ENDS knowledge assessment after the CME; and 555 passed by scoring at 75% or higher (94.5% of those taking the post-test), with 832.5 CME units awarded. While learners could leave and return to complete the course, 59% completed the course on the same day they started. From pre- to post-course completion, ENDS knowledge test scores significantly improved from a mean of 57% to 90% (pre/post-knowledge test scores available on n = 499 learners, paired samples t(498) = 29.6, p < .001).

An optional course evaluation survey assessed learners’ past practices and future intentions and perceptions of the course; the number of learners responding to individual items ranged from 444 to 475 (35% of responders were MDs/DOs or medical students). Learners reported their engagement in professional practices related to ENDS (e.g., seeking evidence, assessing and treating ENDS use, recommending FDA-approved smoking cessation aids) prior to taking the CME and then their future intentions having completed the CME. Relative to their prior practices, learners’ intentions significantly increased, by about 1.5 standard deviations, on all assessed items (Table 1). Learners’ mean values at post-test on all items approached a 4, reflecting intention to “often” engage in the behaviors.

Table 1 Learners’ Reported Prior Practices and Future Intentions Regarding ENDS Having Completed the CME Course (N = 475 responding)

Most learners gave the course above average to outstanding ratings (Fig. 1): 76% rated the course’s activity content and module quality as above average (41%) or outstanding (35%); 73% rated the opportunities for active learning (e.g., case studies, skill sessions, written assignments) as above average (40%) or outstanding (33%); 75% considered the course content and delivery easy to navigate; and 99% viewed the course as free of commercial bias. In evaluating the individual course components, most learners gave above average or outstanding ratings for the patient/provider visit role plays (74%), didactic expert videos (70%), case-based content (75%), and animations and visual aids (71%). A majority of learners gave above average or outstanding ratings for level of engagement to the patient/provider visit role plays (58%), didactic expert videos (53%), case-based content (60%), and animations and visual aids (58%).

Figure 1

Learners’ evaluations of CME course content, engagement, and instruction (N = 444–473 responding).

Most learners (60%) intended to make changes in their work based on the course learnings. Top intentions were changing screening practices to include e-cigarettes (83%); modifying treatment plans (66%); enhancing communication with patients, families, or health care team members (59%); and incorporating new skills into patient care (57%). Among the 40% of learners not intending to change their practice, 40% stated the course validated their current practice, 32% were still in school and not seeing patients, and 18% did not see these types of patients. Most learners (92%) identified no barriers to applying course learnings. Among the 8% of learners indicating barriers, most frequently identified were lack of experience (44%), lack of time (33%), and patient obstacles (33%).


The current findings indicate low levels of knowledge of ENDS and confidence in addressing ENDS in clinical practice at pre-test with significant gains following completion of the online CME. By addressing the growing need for unbiased education on ENDS, this interactive online CME engaged learners and increased knowledge on ENDS devices and evidence-based cessation approaches, improving confidence and intentions to address ENDS in clinical practice.

This CME’s interactive video content was intended to mimic patient-provider interactions and offer providers an opportunity to practice appropriate responses to different patient scenarios. Program evaluations suggest that this framework optimizes participant learning and engagement. Pre- to post-test scores increased significantly, most learners rated the course as above average or outstanding, and almost all indicated the course was free of commercial bias. Further course strengths include its efficiency (1.5 h to complete), accessibility and scalability (free and online), and utility for providers working in a diversity of fields and countries. The CME addressed health care providers’ knowledge gaps on ENDS identified in the literature. The course can serve as a model for future CMEs.

A limitation is that our evaluation data do not allow us to assess how course completion affects providers' actual patient interactions. While learners reported intentions to integrate course learnings into their work, future research should assess whether and how these intentions translate into actual clinical practice.

The course was updated and re-released on January 15, 2019, to include timely content on Juul, which is now the leading ENDS device in the USA and highly popular among young people.25 The updated course also incorporates new research on ENDS as a cessation tool and resources such as the National Academy of Sciences statement on the public health consequences of ENDS21. The course will continue to be updated in light of emerging research and policy changes.


  1. 1.

    U.S. Department of Health and Human Services CfDCaP, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health. U.S. Department of Health and Human Services. E-cigarette use among youth and young adults: A report of the Surgeon General—executive summary. Atlanta, GA; 2016.

  2. 2.

    Wang TW, Asman K, Gentzke AS, et al. Tobacco Product Use Among Adults — United States, 2017. MMWR Morb Mortal Wkly Rep. 2018;67:1225–1232.

    Article  Google Scholar 

  3. 3.

    Delnevo CD, Giovenco DP, Steinberg MB, et al. Patterns of Electronic Cigarette Use Among Adults in the United States. Nicotine Tob Res. 2016;18(5):715-719.

    Article  Google Scholar 

  4. 4.

    Caraballo RS, Shafer PR, Patel D, Davis KC, McAfee TA. Quit Methods Used by US Adult Cigarette Smokers, 2014-2016. Prev Chronic Dis. 2017;14:E32.

    Article  Google Scholar 

  5. 5.

    Ghosh S, Drummond MB. Electronic cigarettes as smoking cessation tool: are we there? Curr Opin Pulm Med. 2017;23(2):111-116.

    Article  Google Scholar 

  6. 6.

    National Academies of Sciences E, Medicine, Health, et al. In: Eaton DL, Kwan LY, Stratton K, eds. Public Health Consequences of E-Cigarettes. Washington (DC): National Academies Press (US); 2018.

    Google Scholar 

  7. 7.

    Hajek P, Phillips-Waller A, Przulj D, et al. A Randomized Trial of E-Cigarettes versus Nicotine-Replacement Therapy. N Engl J Med. 2019;380(7):629-637.

    Article  Google Scholar 

  8. 8.

    Physicians RCo. Nicotine without smoke: Tobacco harm reduction. London; 2016.

  9. 9.

    Bhatnagar A, Whitsel LP, Ribisl KM, et al. Electronic cigarettes: a policy statement from the American Heart Association. Circulation. 2014;130(16):1418-1436.

    Article  Google Scholar 

  10. 10.

    Brandon TH, Goniewicz ML, Hanna NH, et al. Electronic nicotine delivery systems: a policy statement from the American Association for Cancer Research and the American Society of Clinical Oncology. J Clin Oncol. 2015;33(8):952-963.

    Article  Google Scholar 

  11. 11.

    NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines): Smoking Cessation. June 18, 2018.

  12. 12.

    Berg CJ, Haardoerfer R, Escoffery C, Zheng P, Kegler M. Cigarette users’ interest in using or switching to electronic nicotine delivery systems for smokeless tobacco for harm reduction, cessation, or novelty: a cross-sectional survey of US adults. Nicotine Tob Res. 2015;17(2):245-255.

    Article  Google Scholar 

  13. 13.

    Office of the Surgeon General. E-cigarette Use among Youth and Young Adults: A Report of the Surgeon General. Washington, DC: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention. 2016.

    Google Scholar 

  14. 14.

    Jamal A, Gentzke A, Hu SS, et al. Tobacco Use Among Middle and High School Students - United States, 2011-2016. MMWR Morb Mortal Wkly Rep. 2017;66(23):597-603.

    Article  Google Scholar 

  15. 15.

    Cullen KA, Ambrose BK, Gentzke AS, Apelberg BJ, Jamal A, King BA. Notes from the Field: Use of Electronic Cigarettes and Any Tobacco Product Among Middle and High School Students - United States, 2011-2018. MMWR Morb Mortal Wkly Rep. 2018;67(45):1276-1277.

    Article  Google Scholar 

  16. 16.

    Miech R, Schulenberg JE, Johnston LD, Bachman JG, O’Malley PM, Patrick ME. National Adolescent Drug Trends in 2018. Ann Arbor; 2018.

  17. 17.

    Berry KM, Fetterman JL, Benjamin EJ, et al. Association of Electronic Cigarette Use With Subsequent Initiation of Tobacco Cigarettes in US Youths. JAMA Netw Open. 2019;2(2):e187794.

    Article  Google Scholar 

  18. 18.

    Barrington-Trimis JL, Kong G, Leventhal AM, et al. E-cigarette Use and Subsequent Smoking Frequency Among Adolescents. Pediatrics. 2018;142(6).

    Article  Google Scholar 

  19. 19.

    Nickels AS, Warner DO, Jenkins SM, Tilburt J, Hays JT. Beliefs, Practices, and Self-efficacy of US Physicians Regarding Smoking Cessation and Electronic Cigarettes: A National Survey. Nicotine Tob Res. 2017;19(2):197-207.

    Article  Google Scholar 

  20. 20.

    Kandra KL, Ranney LM, Lee JG, Goldstein AO. Physicians’ attitudes and use of e-cigarettes as cessation devices, North Carolina, 2013. PLoS One. 2014;9(7):e103462.

    Article  Google Scholar 

  21. 21.

    Steinberg MB, Giovenco DP, Delnevo CD. Patient–physician communication regarding electronic cigarettes. Prev Med Rep. 2015;2:96-98.

    Article  Google Scholar 

  22. 22.

    Pepper JK, Gilkey MB, Brewer NT. Physicians’ Counseling of Adolescents Regarding E-Cigarette Use. J Adolesc Health. 2015;57(6):580-586.

    Article  Google Scholar 

  23. 23.

    Pepper JK, McRee AL, Gilkey MB. Healthcare providers’ beliefs and attitudes about electronic cigarettes and preventive counseling for adolescent patients. J Adolesc Health. 2014;54(6):678-683.

    Article  Google Scholar 

  24. 24.

    Al-Abed A CT, Ismail I, Lin E, Sergakis G, Mays M. Knowledge, perceptions, and awareness of electronic cigarettes among healthcare providers and in-patients. Respir Care. 2014;59(10):OF45.

    Google Scholar 

  25. 25.

    Securities WF. Nielsen: Tobacco “All Channel” Data. Equity Research. San Francisco, CA: Wells Fargo Securities; 2018.

    Google Scholar 

Download references


We acknowledge Mark Rosenberg, Kimberly Walker, and Andrew Baek with the Stanford Center for CME for their collaboration in developing and disseminating the CME; Laura Corbett for assistance accessing the evaluation data; Dr. Maciej Goniewicz and Dr. Suzaynn Schick for participating as experts in the CME; Dr. Mark Rubinstein and Dr. Sean David for their external review and feedback on the course content; and Dr. Sue Kim and Dr. Sang-ick Chang for their course review as time testers.


Intramural research funding was provided by the Stanford Center for Continuing Medical Education, and postdoctoral training support was provided by the National Heart, Lung, and Blood Institute T32 Postdoctoral Training Grant No. 5T32HL007034-43. The Stanford University School of Medicine has received and has used undesignated program funding from Pfizer, Inc. to facilitate the development of innovative CME activities designed to enhance physician competence and performance and to implement advanced technology. A portion of this funding supports this activity.

Author information



Corresponding author

Correspondence to Judith J. Prochaska PhD, MPH.

Ethics declarations

This project, which reports on learners’ outcomes in an online continuing medical education program, falls under exemption 1 of the common rule, 45 CFR 46.104(d)(1).

Conflict of Interest

None of the authors have received any financial or in-kind support from an e-cigarette or tobacco company. Dr. Prochaska has served as an expert witness against the tobacco companies in lawsuits and has provided consultation to pharmaceutical and technology companies that make medications and other treatments for quitting smoking. No other authors have disclosures to report related to this work.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Fielding-Singh, P., Brown-Johnson, C., Oppezzo, M. et al. E-Cigarettes: Harmful or Harm-Reducing? Evaluation of a Novel Online CME Program for Health Care Providers. J GEN INTERN MED 35, 336–340 (2020). https://doi.org/10.1007/s11606-019-05388-7

Download citation


  • e-cigarettes
  • vaping
  • Juul
  • continuing medical education
  • nicotine
  • ENDS