E-learning, or digital learning, offers a profound opportunity to enhance the psychiatric clerkship experience. A recent review of active learning techniques in psychiatry education revealed that students enjoyed the flexibility and classroom engagement of these techniques [1]. E-learning, such as self-directed virtual cases, recorded lectures, and digital question banks, allows students to go through the material at their own pace, fast forward through concepts already mastered, and review more challenging material [2]. As medical schools expand the use of e-learning [3, 4], a trend accelerated by the COVID-19 pandemic, equity and access issues become more relevant. This paper explores considerations when developing a supplemental e-learning curriculum to ensure that information is accessible to all medical students on the psychiatry clerkship and presents student feedback on the usefulness of the curriculum.

Ensuring Access and Equity

Part of the mission at Morehouse School of Medicine, a Historically Black Colleges and Universities (HBCU) medical school where most of the student body is recruited from traditionally marginalized communities, is to increase diversity in medicine. Marginalized communities are those excluded from mainstream social, economic, educational, and/or cultural life. Examples of marginalized populations include, but are not limited to, groups excluded due to race, gender identity, sexual orientation, age, physical ability, language, and/or immigration status [5]. According to the 2019–2020 Association of American Medical Colleges (AAMC) Matriculating Student Questionnaire (MSQ) data for Morehouse School of Medicine, 65% of our graduating students identify as Black, and 12.2% identify as Latinx (compared with 8.3% and 11.1% of all medical students, respectively). The median parental income of our students was almost half the income reported by all students nationally. Our students are more likely to have outstanding educational loans at the onset of medical school and to fund their medical school education with loans than students at other medical schools (Table 1). Consistent with the literature, our students use various digital resources (e.g., OnlineMedEd, UptoDate, Wikipedia, Google, YouTube videos, various mobile apps, and online texts) while on the Clerkship and Clerkship director discussion with our students revealed that many students purchase these supplemental materials [6]. As the clerkship faculty increased the use of digital resources over the past four years, they paid special attention to creating an equitable learning environment. Based on our MSQ financial data, and informal student and faculty feedback, the authors identified several areas of particular concern: technological access (including internet access, digital literacy, and disability access), access to a selection of high-quality digital medical education tools (regardless of ability to self-finance), and diverse learning styles.

Table 1 Racial/ethnic identity and financial data from the Association of American Medical Colleges Matriculating Student Questionnaire: 2019 individual school report for Morehouse School of Medicine compared to all US medical schools
Full size table

All students had adequate devices to access digital learning resources, ensured by providing a laptop upon matriculation and high-speed internet was available on campus. Information technology support was readily available to assist students with any technical difficulties and faculty in creating e-learning material.

The clerkship directors provided multiple modalities for accessing the information to accommodate a variety of learning styles. Lectures were recorded and uploaded to the learning management system for students to review readily. Digital and hard-copy textbooks were available for check out from the library. Students were encouraged to use virtual study groups, and institutional access to a videoconferencing platform (Zoom) was provided for didactics and group sessions. These digital resources were utilized concurrently with in-person clinic experiences and provided opportunities for face-to-face education, real-time utilization of knowledge, and individual learning evaluations.

Digital learning material was chosen to give students exposure to key psychiatric disorders less commonly seen on our rotation. A review of student logs indicated little exposure to patients with eating disorders, obsessive-compulsive disorder, and neurodevelopmental disorders. The authors focused on digital learning modules to fill these knowledge gaps. In addition, modules reviewing depression, anxiety, substance use disorders, and psychosis were selected to reinforce their clinical knowledge. Given the wide variety of digital learning tools available, with limited information about their utility, accuracy, and effectiveness, psychiatry faculty reviewed and selected e-learning material to incorporate into the clerkship. This ensured that students viewed relevant material and allowed all students to have equal access to the tools. Our school’s Committee on Psychiatric Education initially reviewed seven digital learning tools recommended on the Association of Directors of Medical Student Education in Psychiatry (ADMSEP) listserv. The committee selected three resources based on whether the material covered supported the curriculum, ease of use, level of interactivity, disability access (including hearing and learning disabilities), and cost (see Table 2 for a description of selected resources). The digital learning curriculum, textbooks, and other reading materials were provided electronically and at no cost to the student.

Table 2 Digital learning tools and student perception of benefit to learning
Full size table

Psychiatry Clerkship Supplemental E-learning Curriculum

The Psychiatry Clerkship at Morehouse School of Medicine is a six-week course during which students are exposed to key psychiatric disorders as outlined by the ADMSEP [7]. Students spend four days per week at their clinical site, with one day dedicated to didactics. In 2017, the clerkship faculty shifted the didactic from traditional lecture-based to a flipped classroom design where students accessed pre-recorded material and/or readings before the class session. The live session was then dedicated to applying student knowledge to specific cases designed by the teaching faculty. Additionally, students were required to complete a series of electronic modules from the resources in Table 2. The modules were embedded in the learning management system so that faculty could track completion.

Evaluation of the Asynchronous E-learning Curriculum

At the end of the clerkship, students completed a brief survey on the usefulness of each e-learning resource. The survey had four questions, three that rated items from 1 (least) to 5 (most beneficial) beneficial and one open-ended item for additional comments.

Ninety of the 95 students who completed the psychiatry clerkship during the 2020–2021 academic year completed the feedback survey. Table 2 shows the results of the survey. On average, students found the e-learning supplements beneficial to their learning, with Case X rating slightly lower than the ADMSEP e-modules and SymptomMedia. Student comments indicate that most found the material useful:

“I appreciated the mix of resources/media made available to learn about the psychiatric disorders throughout the rotation,”

“Really good case simulation component,”

“I really appreciated the use of high quality, supplementary educational materials, and self-directed learning, as it was very time efficient and aided me in my studies and clinical care.” Two student criticisms were recorded:

“Having to work in multiple sites made this assignment cumbersome,”

“They were helpful, but of course not as beneficial as in-person encounters.”

The National Board of Medical Examiners (NBME) Psychiatry Shelf exam scores were comparable before and after instituting the supplemental virtual curriculum. Student satisfaction with the course, based on the biannual clerkship evaluation, remained high as well. In weekly check-ins with the Clerkship Director, students confirmed equal ability to access and utilize digital learning materials, as well as satisfaction with the offerings.

Future Implications

As schools increase diversity, special attention must be placed on the impact of economic disparities among students, especially as new learning tools are introduced. By providing a device to every student, purchasing digital material, and providing access to study spaces with reliable internet access, medical school clerkship directors can decrease some economic barriers to incorporating digital learning in the clerkship. By vetting the digital learning tools, clerkship directors can reduce the burden on medical students to discern quality educational material, review for disability accessibility, and ensure all students are learning the material needed to complete the clerkship successfully.

Consistent with other experiences with e-learning [8], students utilized and appreciated the inclusion of asynchronous e-modules. The student NBME scores were comparable pre- and post-intervention. However, the authors will need to collect more data over time to examine the overall trend. By offering e-learning as a supplement to clinical education, rather than in lieu of, the authors do not anticipate a decline in clinical skills. Rather, acknowledging that students have different strengths and offering a greater variety of tools can enhance student performance [9, 10].

A significant barrier to more fully incorporating digital learning is resistance by faculty and difficulty with the adoption of innovative technologies. Informal discussion with core faculty indicated discomfort with the digital learning tools and uncertainty about its benefit. One future goal is to create a formal rubric for evaluating digital material for the psychiatry clerkship and include all interested faculty in the review process which may decrease faculty resistance. Based on the student feedback, NBME scores, and evaluations, the authors determined that the incorporation of digital learning materials met our access and equity goals while maintaining the high educational standards of the clerkship. We will continue to examine the e-learning process to better understand student utilization. Additional tools, such as adaptive learning, would provide personalized content and individualized tracking that could one day enhance the learning and feedback environment [11]. Future studies should continue to assess the impact of the digital learning curriculum on learning outcomes including test scores, clinical skills, and feedback from faculty. The authors also hope to assess the suitability of the curriculum to different student cohorts, as well as complete a cost-benefit analysis.

As innovative technologies are introduced to undergraduate psychiatry medical education, it is important to consider whether the material is accessible to all students. Evaluation of accessibility is a crucial component to ensuring the success of a diverse medical student body.