INTRODUCTION

The novel coronavirus SARS-CoV-2 (COVID-19) pandemic disrupted graduate medical education, with social distancing measures compelling training programs to abruptly transition from in-person to virtual educational formats despite limited experience or proficiency.1,2,3,4,5,6 Morning report (MR), a highly valued core educational conference for internal medicine (IM) residents,7,8,9,10,11,12,13,14 was included in this transition at many academic institutions.15 Little is known about learners’ perceptions of virtual educational formats in graduate medical education, and we are only beginning to understand the impact that the COVID-19 pandemic has had on medical education more broadly.16,17,18,19 We surveyed IM residents from academically affiliated residency programs in May and June 2020 to elucidate their experiences with the transition to virtual morning report (virtual MR) during the COVID-19 pandemic.

METHODS

Participants

The Veterans Affairs (VA) National Academic Hospitalist Work Group is a collaboration of more than 140 hospital medicine faculty from academically affiliated VA hospitals. Through the workgroup, we identified a convenience sample of 14 academically affiliated IM residency programs of variable sizes (average 131 trainees, range 35 to 216) that were geographically dispersed across the USA, represented by the study authors. Each site obtained permission from the residency Program Director to survey their trainees. All interns (R1), second-year (R2), and third-year (R3) residents were eligible to participate in the study. Participation was voluntary and no compensation was provided.

Survey Instrument

The anonymous, self-administered, and 25-item survey (Supplement 1) was developed by four of the authors (TJA, JR, HS, PBC), with questions based on prior studies of MR8,12 and input obtained from all authors on the structure and content of the survey. The survey was pilot tested with two internal medicine chief residents for clarity and modified based on feedback. The survey contained questions about level of training; self-identified gender and underrepresented minority status (URM, as defined by the AAMC20); institutional affiliation; self-reported virtual MR attendance rates and reasons for attending or not; opinions on who should attend and who should teach; how specific aspects of the online format affect the learning environment; how virtual MR compares to in-person MR with regard to participation, engagement, and overall education; and whether some aspects of virtual MR should continue after in-person conferences resume. We also collected program-level data to account for differences in size and practice with respect to who facilitated virtual MR during the COVID-19 pandemic (chief residents and/or attending physicians).

The survey included a combination of Likert-style and multiple option questions and open-ended comments about how virtual MR has impacted residents’ learning or clinical practice, what residents perceive to constitute the “best virtual morning report,” and suggestions for improvement. Respondents compared important educational attributes of virtual MR to traditional in-person MR on a 5-point Likert-style scale (1: much better (favors virtual) to 5: much worse (favors in-person)) and also rated the educational value of different aspects of virtual MR (e.g., remote participation, chat boxes, audience response systems) on a 5-point scale (1: strongly enhances to 5: strongly detracts).

Data Collection

At each site, a member of our research group distributed the survey to residents via institutional email. Email reminders were sent 1 and 2 weeks after the initial survey was sent. The University of Washington Institutional Review Board deemed the study exempt from research oversight for all sites.

Statistical Analysis

All items were summarized as the percentage of endorsed responses and examined by level of training (R1 vs. R2/R3 combined), self-identified gender and URM status, and program size using chi-square tests. We used logistic regression to assess differences in overall assessment of virtual MR (“about the same” and “favors virtual MR” vs. reference category of in-person MR), who should attend (IM attendings vs. others), and who should teach (chief residents vs. others), controlling for level of training, self-identified gender and URM status, program size, and region. All analyses were done using STATA version 16.1.

Free-text responses were systematically analyzed through inductive iterative review by three coders (TJA, JR, MK). We created codes after independently reviewing the responses and coming to consensus about the code assignment. We tabulated the frequency of comments by code using Microsoft Excel™ and synthesized comments into generalizable themes (e.g., barriers to trainee participation, how virtual MR can be improved) until theme saturation was reached. A fourth reviewer (JB) independently examined the uncoded data and reviewed all themes and categories as a final check. All coders developed consensus regarding the categorization and generalization of the data.21

RESULTS

Respondent Characteristics

Six hundred fifteen residents completed the survey, representing a 35% response rate. Respondents from each year of training (R1, R2, R3) were equally represented in the sample (Table 1). Overall, 51% identified as female and 10% identified as URMs. Most respondents (93%) were from large programs (> 100 residents) and almost half were from institutions in the Western USA (47%, Supplement 2). Chief residents were the sole virtual MR facilitators at 12 programs representing 80% of the participants, while the other 20% were from 2 programs that were facilitated by a combination of chief residents and attendings.

Table 1 Respondent Characteristics
Full size table

Attendance

Self-reported attendance at virtual MR was variable: 10% (61/615) of respondents always attend, 47% (290/615) attend more than half of the time, 30% (184/615) attend less than half the time, and 13% (80/615) rarely attend. Reasons for participation included clinical education (93%), review of evidence-based medicine (54%), and camaraderie (47%). The main reason respondents cited for not attending was being too busy with clinical duties (77%), with many free-text comments specifying conflicting priorities and distractions as significant barriers to attendance. Other barriers to participation included a lack of quiet space (26%), internet access (10%), or audio/visual hardware (5%).

Respondents across training levels favored attendance of interns (97%), R2/R3s (96%), and medical students (81%); far fewer favored attendance by attendings (47%) or fellows (22%). Respondents preferred chief residents (56%) to be the primary facilitators at virtual MR, followed by residents (25%), and internal medicine (12%) and subspecialist attendings (8%).

Comparison of In-Person and Virtual Morning Report

Figure 1 compares different aspects of in-person and virtual MR formats by level of training (R1 vs. R2/3). Across all participants, 42% of residents preferred in-person, 18% preferred virtual, and 40% felt they were equivalent. R2/3s had a greater preference for the in-person format overall (R1=38%; R2/3=44%, p=0.013), and for group participation (R1=54%; R2/3=59%, p=0.045) and clinical reasoning education (R1=8%; R2/3=14%, p=0.01). Respondents rated in-person and virtual formats similarly regarding interpreting images and the quality of case presentations. For aspects related to social interaction, most preferred in-person for their own participation, sense of camaraderie and peer-engagement.

Figure 1
figure 1

Comparisons of in-person vs. virtual morning report by training year.

Full size image

Comments suggest the benefits of virtual MR include the abilities to participate remotely, multitask (“I can listen on mute in the team room while I finish work”), and use self-directed synchronous learning (“I like being able to look things up in real time while participating in the lecture”).

Contribution of Specific Features of Virtual Platforms to the Learning Environment

Respondents rated whether specific features of the virtual format enhanced or detracted from participation (Fig. 2). Features of virtual MR that enhanced learning included the use of chat boxes (65%), video participation by learners (as compared to audio-only participation, 52%), the use of audience response systems (52%), and the use of smart boards/tablets (51%). Comments described the advantages of using image capture for future reference or further education (e.g., “you can easily take screenshots to save important slides you want to preserve”). Participation from the team room and the ability to synchronously look up and share information had more mixed results, with a higher percentage of respondents (25 and 19%, respectively) reporting these distracted from the learning environment. A portion of respondents did not rate breakout rooms (35%), smart boards (18%), or polls (15%), suggesting that these features may not be frequently used at many institutions. Of all respondents, 47% felt that virtual participation should continue to be an option once in-person conferences can safely resume, and an additional 19% wanted some in-person and some virtual MRs each week; 6% endorsed having virtual only and 28% wanted to return to in-person only.

Figure 2
figure 2

Contribution of specific features of virtual platforms to learning environment.

Full size image

Responses to Free-Text Questions

Residents were asked to describe the qualities of the best virtual MR they had attended within the last year (Table 2). Over a quarter of the respondents (n = 166, 27%) wrote free-text comments, with many addressing more than one theme, for a total of 283 unique comments. The most frequent responses emphasized audience engagement, with many respondents describing increased engagement through the effective use of chat boxes (“easy access to chat for questions without disrupting the presentation”) and audience response systems (“polls and chats help bring easy access to evidence based medicine”).

Table 2 Thematically Grouped Responses to Free-Text Questions
Full size table

Respondents were also asked to provide suggestions about how to improve virtual MR (Table 2). One hundred twenty-six respondents (20%) wrote free-text answers, generating 185 unique comments. Increasing engagement and interaction were again major themes. Suggestions identified opportunities to improve the use of the virtual platform (“ask people to join with audio and video when possible,” “take more advantage of the technology with virtual polls, breakout sessions to increase participation”) and to better protect the educational time and space (“there is no physical escape from the constant distractions and work,” “frequent interruptions make it impossible to pay attention and engage”).

Lastly, respondents were asked to describe which aspects of virtual MR had the greatest impact (either positively or negatively) on their learning or clinical practice (Table 2). One quarter of respondents (n = 152, 25%) wrote free-text answers, generating 193 unique comments. Almost two thirds (64%) of the comments were positive and focused on the accessibility and flexibility of virtual MR (“it’s been wonderful to be able to join morning report on days off or when I’m on backup at home”). Negative comments (36%) described difficulties with engagement (“I miss peer-to-peer contact to bounce ideas off of for clinical cases”), a lack of camaraderie (“my overall sense of connectedness and community is negatively affected”), and difficulty focusing due to interruptions and distractions.

DISCUSSION

This multicenter study provides insight into IM residents’ perceptions of virtual MR after the rapid transition to virtual educational formats due to the COVID-19 pandemic.1,3,4,15 Residents perceived the virtual platform as a valuable teaching modality and highlighted how the effective use of virtual technologies can enhance the learning environment and help mitigate the negative social impact on camaraderie, engagement, and participation. Although born out of necessity, and not without its disadvantages and opportunities for improvement, residents identified virtual MR as an asset to their education during the pandemic, both on its own and in conjunction with in-person conferences, and advocated for the integration of a virtual format into the structure of MR in the future.

While more residents felt in-person MR (40%) was preferable to virtual MR (18%), an equal number felt that virtual MR was as good (40%) and highlighted distinct advantages, most notably increased accessibility. Residents appreciated how the virtual platform improved attendance by removing the physical barriers of distance and travel time for trainees with significantly fragmented workflows.22,23,24 Residents also appreciated coordinating virtual MR across multiple sites, increasing programmatic educational equity and perhaps partially compensating for the loss of camaraderie that is associated with in-person educational conferences.8,25,26,27 Residents described other important advantages: multimodal engagement through chat functions, breakout rooms, annotation and screen capture; improved visibility of clinical images (e.g., radiography, EKGs); and the ability to learn both synchronously and asynchronously using online references and in-team discussion in parallel with the conference, preserving team learning activities that are known to mitigate physician burnout.28

However, residents also identified clear disadvantages of virtual MR, particularly the loss of camaraderie. This may partially explain the drop in attendance at virtual MR, with only 57% of residents attending at least half of the time as compared to 70% in the pre-pandemic era.8 Although residents’ inability to interact and socialize with peers was likely amplified by the broader social restrictions of the COVID-19 pandemic, free-text responses described how the loss of community, and the accompanying individual and group accountability, diminished engagement (“being online feels very detached and distant”; “I don’t have the draw of camaraderie compelling me to go”). Residents also described a lack of consensus on best practices for the virtual platform, including guidance from programs on how to cope with a less protected learning environment vulnerable to more frequent interruptions, both in the hospital and at home.

Multitasking during virtual MR was a recurrent and polarizing theme. Many residents described how frequent interruptions due to the lack of a protected physical space acted as a barrier to their participation and engagement. This coincides with prior studies that describe how interruptions and distractions during educational conferences, many of which turn out to be non-urgent or unnecessary, significantly impact educational outcomes.29,30,31 However, despite these distractions, a greater proportion of residents in our study felt that participating from the team room enhanced the learning environment, rather than detracting from it (37 vs. 25%, respectively). Free-text responses reveal how residents appreciated being able to attend to time-sensitive clinical obligations while listening to virtual MR in their team room, in contrast to leaving the room during in-person MR. Although strategies and guidelines have been created to minimize interruptions,32,33 there is clearly a need to re-implement or restructure these interventions to protect resident education in the virtual arena.

Residents emphasized that a successful virtual MR is contingent upon a variety of factors relating to the creation and protection of a safe learning environment. While it is important to have a quiet space equipped with reliable audio and visual connectivity, residents stressed that it is also important to mitigate some of the competing clinical demands and external disruptions. Residents highlighted that skilled manipulation of technology by the facilitator and the establishment of clear expectations for the use of audio, video, and chat functions lead to effective management of group size and increased engagement and participation.

The consensus that programs should support both in-person and virtual formats following the pandemic invites careful consideration of how modern in-person and virtual interactive instructional techniques can be combined to maximize learning during MR. Our qualitative data reinforce the idea that no one style, structure, or medium will work for all learners. However, a reasonable balance can be reliably derived from co-production of best practices within programs34 and deliberate inclusion of the components that residents consistently value: a sense of community, a safe and supportive learning environment, skilled facilitation that fosters audience engagement and balanced discussion, and expert input with concise clinical pearls, diagnostic reasoning, and the use of evidence-based medicine.8,12,14

This study highlights an opportunity to engage programs nationally in collaborative efforts to define excellence in MR and build a broader community of practice and resources for professional development and education. The virtual MR platform may allow for new approaches that invite collaborative problem solving, increased access to intentionally designed learning experiences, and the co-production of educational approaches which integrate the expertise of both learners and teachers in order to optimize educational outcomes.34

Our qualitative data suggests that the innovative promise of virtual MR may not be realized if programs simply translate traditional in-person MR designs to a virtual format. Even so, there may be important operational benefits. For example, larger programs may be able to host a single virtual MR (vs. multiple MRs at each clinical site) to increase session quality and continuity across rotations while reducing aggregate teaching workload. This could be expanded further to include virtual MR conferences between programs, allowing residents to interact with peers and access educators across the country. Residents’ comments also offered options to foster a sense of community within a virtual context, such as a “social minute” that could be variably structured (“allow time for camaraderie at the beginning/end”). With improvements in how virtual MR is developed and integrated, the gap between residents’ preferences for in-person versus virtual MR may narrow.

Further evaluation of the educational content, structure, and outcomes of MR will benefit from a mixed methods approach to explore how best to adapt MR to maximize engagement, participation, and learning in the in-person and virtual environments. Blended learning (the combination of traditional face-to-face learning and asynchronous or synchronous e-learning) is regarded as more effective for knowledge acquisition in healthcare professions.35 Ultimately, further research is needed to understand the impact different MR educational strategies have on not just learner experiences and perceptions, but on the higher-order evaluation of demonstrated knowledge, skills, and attitudes, clinical behaviors, and patient and institutional outcomes.36

This study has several limitations. The survey was distributed during late spring 2020 as programs responded to teaching during the pandemic, with variable implementation of virtual MR across the 14 participating institutions. The pandemic also directly impacted survey participants at the time of distribution, which likely contributed to the relatively low response rate (35%) and possibly a positive bias towards online learning based on who was able and willing to respond. Geographically, the West was overrepresented in overall response rate which may have affected our results if residents’ perceptions and expectations of MR vary across regions. Our results also may not be applicable to smaller academic or community-based programs given the resources of the larger programs surveyed in our multi-center sample.

In conclusion, survey respondents endorsed virtual MR as a valuable alternative to traditional in-person MR during the COVID-19 pandemic. Residents perceived the virtual platform as a valuable teaching modality and highlighted how the effective use of virtual technologies enhanced the learning environment and helped mitigate the negative social impact on camaraderie, engagement, and participation. Overall, residents identified virtual MR as an asset to their education both on its own and in conjunction with in-person conferences and advocated for the integration of a virtual format into the delivery of MR in the future. Virtual MR may also present an opportunity for innovation that has not yet been realized.