The rapid transition to telemedicine at the onset of the COVID-19 pandemic required many providers to learn telemedicine “on the fly.” As virtual care will likely remain a mainstay of outpatient medicine, it is imperative that telemedicine training be incorporated into graduate medical education.
Design a telemedicine curriculum for internal medicine residents based on principles of experiential learning.
VA-based internal medicine primary care clinic.
Sixteen first-year internal medicine residents participated in the curriculum.
The curriculum included a didactic session followed by four simulated patient encounters focused on troubleshooting technical issues, performing the virtual physical exam, coordinating team-based care, and tackling emergencies.
Participants reported minimal previous experience with telemedicine. After completing the training, resident confidence in conducting video visits increased from an average score of four to seven (on a 10-point scale). Residents were more likely to agree that video visits would allow them to build bonds and effectively address their patients’ needs. This increased confidence persisted at 3 months after training.
Using experiential learning, we identified strategies which increased the confidence of internal medicine trainees in conducting telemedicine visits. Further research is needed to validate our findings across different practice settings.
The COVID-19 pandemic has forced many primary care providers to adopt new approaches to delivering care. While office visits were a mainstay of pre-pandemic primary care,1 social distancing restrictions required a rapid transition to telemedicine.2 During the second quarter of 2020, approximately 35% of all primary care visits were conducted virtually compared to only 1% in 2019.1 Supported by changes in insurance reimbursement3 and ACGME milestones for resident performance,4 telemedicine is likely to maintain a significant role in the practice of post-pandemic primary care.
The abrupt pivot to telemedicine posed a unique problem in graduate medical education. Attending physicians with little formal training5 or experience with telemedicine1 were tasked with teaching trainees to provide virtual care. Further challenges arose in July 2020 with the arrival of a new class of residents who had limited experience in providing primary care both in person and virtually. Previously published telemedicine curricula have utilized a didactic approach with online training modules6, 7 or longitudinal training over the course of residency.8 Based on feedback provided by residents during our initial transition to telemedicine, we identified a need for efficient experiential training in video-based primary care. We developed a simulation-based curriculum with the goal of increasing resident comfort and competence in providing virtual primary care during the pandemic and throughout their careers.
SETTING AND PARTICIPANTS
All first-year internal medicine residents (PGY-1, n= 16) who had their continuity clinic at the Portland VA participated in our telemedicine curriculum, which was delivered over a 3-h session during their clinic orientation in July of 2020. The curriculum focused primarily on conducting primary care visits using video technology with patients who were not in clinic. The instructors included three attending physicians with significant previous telemedicine experience. After orientation, PGY-1s completed 3 months of patient care in the outpatient setting which included a mix of in-person, telephone, and video appointments. Residents’ opinions on the curriculum and their comfort with telemedicine were assessed with short anonymous surveys prior to the curriculum and 1 week and 3 months after completing the course. The survey was adopted from a 2015 study of telemedicine satisfaction9 and was further piloted at the Portland VA amongst primary care and specialty providers.
Prior to the COVID pandemic, residents in our clinic provided patient care primarily through in-person visits and rare telephone encounters. During our urgent transition to virtual medicine at the outset of the pandemic, residents received brief instruction on practical aspects of conducting video and telephone visits to allow them to pivot quickly to telemedicine. A follow-up survey of residents identified a need for simulation training highlighting the flow of the telehealth visit, challenges with common technology issues, and strategies for performing the virtual physical exam. Based on these survey results and discussions with our regional telemedicine community of practice, we developed a formal telemedicine curriculum for incoming PGY-1 residents in our clinic (Figure 1). We identified Kolb’s cycle of experiential learning including a concrete experience, reflective observation, abstract conceptualization, and active experimentation as a foundation on which to structure our curriculum.10, 11
During their orientation to our clinic, PGY-1 residents received 1 hour of small group didactics on telemedicine. This didactic included a brief overview of the field of telemedicine, strategies for appropriately triaging patients between clinical venues, and an overview of virtual care etiquette. We also reviewed VA-specific technology and coding requirements for phone and video visits.
After the didactic training, PGY-1 residents participated in four simulated patient encounters using the VA Video Connect (VVC) technology (see Appendix 1 in the supplementary material for case specifics). These cases provided the PGY-1 residents with concrete experiences in the four areas of need identified by senior residents in our pre-curriculum assessment:
Troubleshooting technical issues
Evaluating a patient with dizziness through the virtual neurological exam
Evaluating a patient with shoulder pain through the virtual physical exam
Triaging a patient with a mental health emergency and communicating with clinic team members during a virtual visit
The cases were facilitated by senior residents who joined the session virtually and alternated between the role of the simulated patient and observer. The use of senior residents as simulated patients reinforced their understanding of virtual care strategies and provided opportunities for near-peer mentoring with PGY-1 residents by leveraging senior residents’ previous experiences in virtual care. Senior residents were provided with an outline of each case and topics for debriefing. PGY-1 residents acted as the physician throughout and were briefed with a chief complaint for each case. Each simulation lasted 10 min with 5 min for debriefing led by the senior residents. After completing the simulations, PGY-1 residents participated in a 30-min attending-led structured debrief. This post-simulation debrief used the Gather Analyze Summarize framework12 to prompt PGY-1 residents to engage in reflective observation of their attitudes towards virtual care and to address areas of discomfort. To encourage abstract conceptualization and maximize opportunities for active experimentation, we scheduled all PGY-1 residents for virtual visits in the 3 months following their orientation. During this period, PGY-1 residents precepted with attendings experienced in virtual care and were provided with feedback on their use of telemedicine modalities.
All 16 PGY-1 residents participated in the telemedicine training curriculum. The residents then conducted 3 weeks of closely supervised virtual primary care visits over a 3-month period, collectively completing 98 phone and 152 video visits.
Ninety-three percent of residents (n=15) responded to the pre-survey. Response rates at 1 week and 3 months were 81% (n=13) and 100% (n=16) respectively. Seven residents identified as male, eight as female, and one as non-binary. Respondents indicated a broad range of career interests including primary care, hospital medicine, and several sub-specialty disciplines. Two residents (12%) reported receiving structured telemedicine training prior to residency, and no residents reported previously conducting a virtual visit.
When surveyed 1 week after the training, 92% of PGY-1 residents reported the curriculum adequately prepared them to conduct virtual visits; this percentage fell to 81% at 3 months. On a Likert scale of 1–10 (with one indicating “not confident at all” and 10 indicating “extremely confident”), confidence in conducting video visits increased from a mean of 4 on the pre-survey to 7 at 1 week and eight at 3 months, respectively (Figure 2). All residents agreed using video technology was helpful for follow-up visits, and 56% indicated video technology was helpful for new patient visits. Residents were also asked to evaluate their agreement on a number of measures related to communicating with patients in the virtual space on a 5-point Likert scale (1=do not agree at all and 5 = very much agree). After they completed the curriculum, residents were more likely to agree that video visits would allow them to build bonds with patients (pre-course 3.4, 1 week post-course 4.3), effectively communicate with patients (pre-course 3.9, 1 week post-course 4.3), and effectively treat patients’ needs (pre-course 3.1, 1 week post-course 4.1).
In qualitative comments, residents highlighted that the greatest barriers to conducting video visits were technological and administrative. Suggestions for improvement included additional training on troubleshooting video technology and conducting the virtual physical exam. Several residents noted that training patients on video technology prior to the visit would help appointments run more smoothly (Appendix 2 in the supplementary material). (Please see our supplemental material for all suggestions for curriculum improvement.)
Our curriculum in telemedicine used case-based simulations rooted in experiential learning to efficiently improve internal medicine PGY-1 confidence in their ability to provide virtual care in the outpatient setting. The trainees who participated in this curriculum had little experience with telemedicine. Despite their lack of formal training prior to this curriculum, the majority of participants reported feeling adequately prepared by our half-day training to perform virtual visits. This increased comfort with telemedicine was reflected in trainees’ survey responses at 1 week and 3 months which noted increased ability to communicate and build bonds with patients.
We attempted to address areas of discomfort identified by senior residents during the transition to telemedicine by developing cases which focused on troubleshooting video technology and the virtual physical exam. Computer anxiety and lack of technical support are common barriers to telemedicine uptake, especially in the older population served by our clinic.13 Our technology troubleshooting case allowed learners to gain experience in addressing technical issues and patient anxieties with telemedicine. While evaluating patients over video has some limitations, many professional societies have developed resources to help providers refine their virtual physical exam.14, 15 The cases focusing on dizziness and shoulder pain provided our trainees with the opportunity to practice these targeted musculoskeletal and neurologic exams in the virtual space.
Finally, the COVID-19 pandemic and associated social isolation have led to higher rates of psychological distress in our patients.16 The mental health emergency case provided trainees with a safe space to explore the challenges of caring for patients experiencing distress in the virtual setting and to engage with clinic-specific protocols and resources for triaging patients with mental illness. By involving clinic-based mental health providers in the case, our learners were able to practice coordinating care within the patient centered medical home.
Our curriculum demonstrates one way in which translational simulation can be used to rapidly address deficiencies in healthcare delivery during the COVID pandemic. Translational simulation has been described as simulation directed towards health service priorities, improving systems and teams, and can be used to rapidly revise care processes in the setting of a pandemic.17 Our simulations were targeted towards an abrupt change to virtual care utilizing recommendations from our residents and our regional telemedicine community of practice and succeeded in rapidly improving resident comfort with new care processes.
Our study is limited by a small sample size and implementation at a single clinic. We also acknowledge the lack of a control group makes it impossible to account for knowledge gained through experience alone. While we were able to assess the perceived benefit of our curriculum, we did not assess objective knowledge of our residents or clinical outcomes. However, the high survey response rate and assessment of multiple measures of resident satisfaction and comfort with telemedicine increase our confidence that the curriculum benefited our residents. Our curriculum was also created prior to the release of new AAMC and ACGME competencies and milestones for digital health.4, 18 While we feel our curriculum targets the ACGME’s milestone and addresses the majority of the AAMC competencies, future iterations of our curriculum will address equitable access to virtual care and include knowledge-based evaluations based on the new milestone.
Telemedicine allows healthcare providers to deliver quality care with greater access, improved patient satisfaction, and potentially lower cost.19, 20 Virtual care has been implemented widely during the COVID-19 pandemic and is likely to remain an integral part of primary and specialty care. The increasing prominence of telemedicine and new expectations of fluency with virtual care demands further research into effective strategies for teaching telemedicine to trainees. Our curriculum adds to a growing body of evidence6,7,8 that telemedicine education can be effectively incorporated into medical training and provides a strategy for teaching key components of the virtual visit through case-based simulation. Additional research is needed to further evaluate telemedicine training on a larger scale and to investigate its impact on patient satisfaction, provider uptake, and clinical outcomes.
Alexander GC, Tajanlangit M, Heyward J, Mansour O, Qato DM, Stafford RS. Use and content of primary care office-based vs telemedicine care visits during the COVID-19 pandemic in the US. JAMA Network Open. 2020;3(10):e2021476-e2021476.
Mann DM, Chen J, Chunara R, Testa PA, Nov O. COVID-19 transforms health care through telemedicine: Evidence from the field. Journal of the American Medical Informatics Association. 2020.
Hoffman DA. Increasing access to care: Telehealth during COVID-19. Journal of Law and the Biosciences. 2020;7(1):lsaa043.
Internal Medicine Milestones. The Accreditation Council for Graduate Medical Education. 2021.
Pourmand A, Ghassemi M, Sumon K, Amini SB, Hood C, Sikka N. Lack of telemedicine training in academic medicine: Are we preparing the next generation? Telemedicine and e-Health. 2020.
Abraham HN, Opara IN, Dwaihy RL, et al. Engaging third-year medical students on their internal medicine clerkship in telehealth during COVID-19. Cureus. 2020;12(6).
Hayes JR, Johnston B, Lundh R. Building a successful, socially-distanced family medicine clerkship in the COVID crisis. PRiMER: Peer-Review Reports in Medical Education Research. 2020;4.
Kirkland EB, DuBose-Morris R, Duckett A. Telehealth for the internal medicine resident: A 3-year longitudinal curriculum. J Telemed Telecare. 2019:1357633X19896683.
Becevic M, Boren S, Mutrux R, Shah Z, Banerjee S. User satisfaction with telehealth: Study of patients, providers, and coordinators. Health Care Manag. 2015;34(4):337-349.
Zigmont JJ, Kappus LJ, Sudikoff SN. The 3D model of debriefing: Defusing, discovering, and deepening. Seminars in Perinatology. 2011;35(2):52-58.
Kolb DA, Boyatzis RE, Mainemelis C. Experiential learning theory: Previous research and new directions. In: Perspectives on thinking, learning, and cognitive styles. Routledge; 2014:227-248.
Phrampus PE, O’Donnell JM. Debriefing using a structured and supported approach. In: The comprehensive textbook of healthcare simulation. Springer; 2013:73-84.
Kruse C, Fohn J, Wilson N, Patlan EN, Zipp S, Mileski M. Utilization barriers and medical outcomes commensurate with the use of telehealth among older adults: Systematic review. JMIR Medical Informatics. 2020;8(8):e20359.
Chari DA, Wu MJ, Crowson MG, Kozin ED, Rauch SD. Telemedicine algorithm for the management of dizzy patients. Otolaryngology–Head and Neck Surgery. 2020;163(5):857-859.
Laskowski ER, Johnson SE, Shelerud RA, et al. The telemedicine musculoskeletal examination. Mayo Clinic Proceedings. 2020;95(8):1715-1731.
Talevi D, Socci V, Carai M, et al. Mental health outcomes of the CoViD-19 pandemic. Riv Psichiatr. 2020;55(3):137-144.
Brazil V, Lowe B, Ryan L, et al. Translational simulation for rapid transformation of health services, using the example of the COVID-19 pandemic preparation. Advances in Simulation. 2020;5(1):1-10.
New and emerging areas in medicine series: Telehealth competencies across the learning continuum. Association of American Medical Colleges. 2020.
De La Torre-Díez I, López-Coronado M, Vaca C, Aguado JS, de Castro C. Cost-utility and cost-effectiveness studies of telemedicine, electronic, and mobile health systems in the literature: A systematic review. Telemedicine and e-Health. 2015;21(2):81-85.
Bashshur RL, Howell JD, Krupinski EA, Harms KM, Bashshur N, Doarn CR. The empirical foundations of telemedicine interventions in primary care. Telemedicine and e-Health. 2016;22(5):342-375.
The authors thank Dr. Kathryn Artis, Dr. Jennifer Barton, and Jacob Dougherty of the Portland VA for their help in survey development. We would also like to thank Dr. Martha Gerrity (Portland VA) for her review of the manuscript and Shelia Markwardt (Oregon Health & Science University) for her statistical advice. Aspects of this curriculum were presented in a workshop at the Northwest Society of General Internal Medicine regional meeting on January 23, 2021, and in a poster at the National Society of General Internal Medicine on April 23, 2021.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Keyserling, K., Janetos, E. & Sprague, C. Teaching Telehealth During a Pandemic and Beyond: an Intern’s Survival Guide for Virtual Medicine. J GEN INTERN MED (2021). https://doi.org/10.1007/s11606-021-07009-8
- video visits
- medical education
- experiential learning