There is increasing scrutiny of all stages of medical education to ensure that training is efficient, meaningful, and competency-based. Those efforts extend to the final year of training so that students will enter residency with all of the abilities necessary for the task on day one of internship. The Accreditation Council for Graduate Medical Education (ACGME) Competencies and Association of American Medical Colleges (AAMC) Core Entrustable Professional Activities (EPAs) [1] can serve as guides to what those needed tasks are [7]. Outcomes of crash courses have been studied for surgery and neurosurgery [8]. However, there is no specific literature on designing “boot camp” or crash courses to prepare students specifically in psychiatry.

In this article, we describe our boot camp curricula development using the six-step approach suggested by Kern et al. [3]. We collaborated on needs assessment but individually developed boot camp curriculums at our respective schools, each course taking into account the resources and needs of our individual programs but sharing materials and information. Here, we describe the process of our curriculum development: Problem Identification and Needs Assessment, Goals and Objectives, Educational Strategies, Implementation, Evaluation, and Feedback.

Curriculum Design—Problem Identification and Needs Assessment

One way to assess the unmet need students have for additional training or assessment prior to matriculation to residency is to survey program directors about strengths and weaknesses of the interns in their programs. Lyss-Lerman et al. [4] is a report of semi-structured interviews with program directors in a variety of disciplines reporting some common difficulties in self-reflection and improvement, organizational skills, professionalism, and weak medical knowledge. A survey of program directors was conducted on the American Association of Directors of Psychiatric Residency Training (AADPRT) list-serve to identify areas of need for final year medical students. The topics the program directors suggested were safety management (risk assessment and emergency management), basic therapy principles including building rapport and motivational interviewing, mental status exam, case formulations, working in the electronic medical record, residents as teachers, relevant topics in internal medicine and neurology, ethics, and compassion fatigue; the first two topics (safety and therapy) were the most common responses. Frayha et al. [2] surveyed program directors who work with interns and graduating students at one institution about their confidence in accomplishing the EPAs. In general, the program directors and students were confident that they could perform the EPAs without supervision, although the confidence varied by the task.

Another source of information about the unmet needs of interns is to assess the students or interns themselves. Peterson et al. [6] used the Readiness for Clerkship Survey (RfC) and Readiness for Residency Survey (RfR) with learners and concluded that while self-assessments are not reliable at the individual level, aggregate assessments of students and interns can indicate the cohort’s relative strengths and weaknesses to assess the program. Lypson et al. [5] used the Postgraduate Orientation Assessment (POA), an eight-station objective structured clinical examination to assess entering interns’ skills and found this to be a feasible strategy to identify knowledge gaps.

Prior to developing the curriculum, the authors collaborated on gathering the literature on fourth-year course work, gathering feedback from interns and graduating students at their programs, solicited faculty input on needed topics, and met regularly to refine the content of the courses.

Curriculum Design—Goals and Objectives

The overarching goal for the University of Texas Medical Branch at Galveston (UTMB) boot camp course was to have the students feel comfortable that they were ready for residency by practicing what they would be doing. This was created so that they would feel more confident when they started internship, and thus, it was placed in the final year of medical school rather than in an internship crash course. It is a classroom-based boot camp so that we can accommodate the volume of students who go into psychiatry (typically 5–8% of the class) without overwhelming the clinical rotations. Because it is based on classroom teaching, we did not cover skills they would learn only on the wards, such as managing time efficiently that have been identified as an important need [4] but would be best covered during an acting internship. However, we did provide a means to simulate skills with role-play exercises and standardized patient encounters. Other disciplines already offer boot camps at our institution, and adding the course allowed students going into psychiatry to participate. Some courses were held with the internal medicine and pediatric boot camp students if the content was also relevant to psychiatry (i.e., handoff communications). The objectives were written around the EPAs.

Some institutions offer boot camp training that spans only over a few days provided the limited time available in the 4th-year curriculum. At University of Hawaii John A. Burns School of Medicine (UH), the boot camp is part of a “Senior Seminar Week.” The goal of the course is to reinforce EPA-related skills and review selected psychiatry level 1 milestones. Other specialties also offer discipline specific skills during this time. Selected psychiatric topics allow students to review and practice foundational clinical skills required to meet level 1 milestones in psychiatry training as outlined by the ACGME. Joint sessions with other specialties are based on EPA’s and also include medicine simulation sessions to practice procedural skills to help review clinical skills that students will need in their primary care rotations during intern year.

Curriculum Design—Educational Strategies

At UTMB, we used small group discussion, peer teaching, role-play exercises, self-study solving management decisions of written cases, and standardized patient encounters to cover the material (see Table 1). For example, the objective for EPA1: “Gather a history on a new psychiatric patient and perform an Mental Status Exam. Give self- and peer-feedback on this skill” included a small group discussion on working through a differential in week 1, a standardized patient encounter in week 3, and group feedback on the encounter in week 4.

Table 1 Content mapped to Entrustable Professional Activities (EPA) and Accreditation Council for Graduate Medical Education (ACGME) Critical Competencies [1]
Full size table

At UH, there are lectures and interactive activities, including simulation sessions (see Table 1). Three days are devoted to general topics across specialties and 3 days are specialty-specific boot camp activities. Seminars are facilitated by faculty and include interactive exercises, self-reflection and group sharing, and roundtable discussions. There is also ample time for sessions facilitated by residents for topics related to workflow, learning strategies, and transition to intern life.

Other modalities the authors considered for the materials include the use of online learning platforms for asynchronous learning of case management skills, but this methodology was limited because neither school was subscribed to a sufficient learning platform. Online learning should be considered by schools with students scattered at different sites.

Curriculum Design—Implementation

The first UTMB boot camp was a 1-month experience in 2016. This was designed to run concurrently with elective boot camps in other disciplines. It is an elective and five students participated. Of note, there are 230 students per year in the school (7% entered psychiatry that year).

The UH boot camp was implemented in 2015 as a joint project of Clerkship Directors across all specialties. It is held soon after the residency match date, and all students are required to participate. Since the first boot camp, the number of UH students entering psychiatry specialty increased from 4% (two students) to 13% (seven students); there are 56 students per class year. The increase in number of participants prompted a revision of curriculum over the past 3 years, from informal, individualized, and small group discussions to more structured seminars. The adoption of psychiatry milestones in the UH Psychiatry Residency Program was also implemented in the recent years and was used as a guide in redesigning the boot camp curriculum.

Curriculum Design—Evaluation and Feedback

Because the curriculum is different at the different schools, we can only present findings within each institution rather than pooling the results. At UTMB, we tracked student learning with the same 50 board-style multiple choice questions (on topics important to intern level) and self-comfort ratings on a variety of skills by survey at the beginning and end of the course (UTMB IRB Submission #17-0276.001 was deemed educational exempt research). There was minimal improvement in the knowledge questions, with a group average improvement of 7% (range − 4 to 16%). The students reported improvement in self-comfort in all areas (see Fig. 1). They noted the most improvement in skills they were not asked to do in the clerkship: gather informed consent, discharge a patient, manage benzodiazepine withdrawal, risk assessment/determine level of care, and provide student feedback. In a verbal feedback session at the end of course, the students also noted that each person who taught the class gave examples of mistakes they had made and what they learned from it. Students felt less pressure to try to be perfect, more confident, and more comfortable asking for help when needed. The students attributed this to the open discussions of mistakes that were held during the course.

Fig. 1
figure 1

Student self-assessment pre- and post-course at UTMB

Full size image

UH conducts a formal, IRB-approved study (UH IRB Submission # 23078), to track student feedback of the boot camp experience in all disciplines. Students receive a pre-survey before the first boot camp session, and a post-survey at the end of the last session as well as 3 months into their internship (3 months after medical school graduation) to assess the sustainability of skills learned and student’s perception of preparation they gained from boot camp. The survey asks the students to rate their confidence on the following items: being prepared to function as an intern, communicating with patients effectively, obtaining informed consent, and assessing decision-making capacity. Students who participate in the psychiatry track boot camp are also asked to rate their comfort level on a 5-point Likert scale (1 = no confidence to 5 = complete confidence) in various areas of patient care such as performing mental status examination (mean rating 3.7 increases to 4.5 pre- to post-course), formulating assessment in a bio-psycho-socio-cultural format (mean rating 3.3 improves to 4.3 pre- to post-course), and evaluating patient safety issues in a psychiatric setting (mean rating 3.2 improves to 4.5 pre- to post-course).

In the psychiatry-specific survey items, the item communicating effectively with families (mean 3.9 improves to 4.2) did not reach statistical significance, but the other six items showed statistically significant improvement from pre- to post-course survey. This suggests more emphasis in this area for future boot camps. Students also improved performance on a four-item written test of knowledge about informed consent. While the response rate of the 3-month post-graduation survey did not capture all students, returned surveys show sustained improvement of confidence in the same items compared to the initial, pre-course survey. The course evaluations from the past 3 years show an overall positive feedback from all students regarding the content of the boot camp curriculum. Suggestions for improvement revolve around logistics and scheduling, which may be related to the challenge of recruiting enough faculty and resident physicians at the most ideal times to facilitate the seminars since facilitators are also often involved in clinical duties during these times.

In conclusion, for educators considering modifications to the final year of medical school, boot camps can be a valuable tool for some students to help with the transition into internship year. At UTMB and UH, we found boot camps are a valuable addition to curriculum, particularly for identity formation of becoming a psychiatrist and confidence of entering the intern year of training.

When designing a boot camp experience, we recommend that educators first establish clear course goals and objectives. The development of these goals and objectives should be informed by assessment methods such as soliciting feedback from senior students, recent graduates, and program directors who have accepted your graduates, if possible, to determine teaching needs. For instance, a curriculum designed to provide documentation that students have mastered certain skills will be entirely different from one which aims to cover non-knowledge or procedural-based material not covered in other courses.

The course goals and objectives should direct if the course is mandatory for all students or an elective for those who have self-assessed the need for additional preparation. Elective courses will likely experience barriers to participation such as competition with other electives, which was experienced at UTMB. The content covered should consider skills specifically needed for the intern level that are generalizable regardless of the program they are matched to, such as how to teach students or how to perform a safety assessment.

Teaching methods, while working within institutional resources (i.e., availability of classroom teaching faculty versus abundance of clinical experiences versus availability of online teaching platform), should be determined by the course goals.

Our boot camps are in the early stages, and we will continue to track the students who took these courses for feedback on the effectiveness during internship training. Limitations of the work include that we do not have a comparison of intern performance with or without the boot camp and that our feedback is based on self-assessment of comfort rather than observed review of performance. We also do not have a comparison of covering this material in an internship crash course, and if the material is retained or mastered better if covered later in training. There is a need for standardization of boot camp methodology if we are to compare data and make this more generalizable (for example, students at UTMB elected to take the course and at UH it was required). Another weakness is that we did not show much improvement in knowledge with the course, thus questioning the effectiveness of the content and process. The generalizability of the feedback at UTMB is limited by the bias incurred by student election of the course.

Additional course details, including schedules and materials, will be provided upon request.