Background

The potential for the first-level medical licensure exams in the U.S. to cause stress for students was apparent in the early 1980s, after more medical schools began requiring students to pass the National Board of Medical Examiners (NBME) comprehensive Part I exam (the predecessor to United States Medical Licensing Exam (USMLE) Step 1) before moving on to their clerkships [1]. NBME Part I and II exams were not high on the list of factors used in residency selection then [2], but by 2008, when the first survey of residency program directors was conducted by the National Resident Matching Program (NRMP), USMLE Step 1 and the equivalent exam for osteopathic schools, Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Level 1, were collectively the top-ranked factor used by programs when selecting which applicants to interview [3]. USMLE Step 1 and COMLEX Level 1 have remained at the top of the list in NRMP program director surveys since then. The pressure that students had felt to compete and perform well on Step 1, and the toll it was taking on them personally, were recently described in articles on “Step 1 climate” [4] and “Step 1 mania” [5]. The sponsors of the USMLE became sufficiently concerned about the potential impact of Step 1 on student well-being to organize the Invitational Conference on USMLE Scoring (INCUS) in March 2019, which resulted in the announcement that USMLE Step 1 will change to pass/fail reporting after January 1, 2022 [6]. The National Board of Osteopathic Medical Examiners announced in December 2020 that it would also change COMLEX Level 1 to pass/fail reporting beginning in May 2022 [7].

Many hope that the change to pass/fail reporting for U.S. first-level licensure exams will relieve stress on medical students. However, evidence suggests that the stress may instead be deferred to second-level licensure exam performance. In a national survey of U.S. program directors conducted after the announcement that Step 1 would become pass/fail, 77% of respondents indicated that they would require USMLE Step 2 Clinical Knowledge (CK) three-digit scores to be reported, and 81% would increase the emphasis on USMLE Step 2 CK scores when selecting applicants [8].

It may also be a testament to the pervasiveness of Step 1-related stress that, despite a significant body of literature related to USMLE exams [9], few investigators have formally studied Step 1’s influence on student well-being. Amid an array of commentaries [4, 5, 10, 11], we found only a statement describing an unpublished survey at one medical school that “concern that curricular content did not match what [students] were expected to know to perform well on the USMLE test … caused anxiety in more than 70% of students.” [10] A recent study at one U.S. medical school indicated that large proportions of its students attributed negative well-being to USMLE Step 1 preparation [12].

Nearly all medical students engage in a period of intense preparation for their first-level licensure exams, which we will refer to as a “dedicated preparation period” (DPP). This is typically a four- to eight-week period before clerkships in which students study for their exams, free from scheduled curricular activities. Schools that allow students to take these exams during clerkship years also typically provide dedicated time to prepare [13]. Studies related to dedicated preparation for USMLE or COMLEX exams have reported on DPP duration, study activities, and study resources [12, 14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]. DPPs are a time in which many students attempt to have a sole focus on their upcoming licensure exam and a time in which exam-related stress is likely to be most intense; however, no study to our knowledge has focused on student well-being during DPPs.

Our primary goal in this study was to characterize student experiences during DPPs for USMLE Step 1 and COMLEX Level 1 at four U.S. medical schools, with a focus on the physical, intellectual, emotional, and social aspects of students’ well-being [32]. A secondary goal was to examine relationships between the financial factors related to exam preparation and student well-being, given the increasing attention being paid to the cost of preparing for and taking USMLE and COMLEX exams in the setting of increasing student debt.

Methods

Subjects and setting

This study was done on data collected through a larger cross-sectional survey (complete instrument included as supplementary material). It was sent electronically to all students in their second year of study at four U.S. medical schools, three allopathic schools (Larner College of Medicine at the University of Vermont, Rush Medical College, University of Illinois College of Medicine) and the largest campus at one osteopathic school (Lake Erie College of Osteopathic Medicine). The survey was sent in 2019 after each school’s respective DPP for USMLE Step 1 or COMLEX Level 1. All four schools required their respective first-level exams to be taken before students began their clerkships. Taking USMLE Step 1 was optional for students at the osteopathic school, although a majority (55%) among those sent the survey chose to take it; students at that school usually would take USMLE Step 1 at the end of their DPP and around the same time as they took COMLEX Level 1. All schools provided students practice tests developed by NBME and additional commercial resources to aid in preparation. The time given to students for DPPs ranged from six to eight weeks.

Appropriate ethical approval was obtained at each institution.

Survey composition

Survey content was based on a review of the literature and our team’s experiences in counseling students on exam preparation at our respective institutions. We included items assessing time spent in DPPs, study resources used, confidence in resources, cost of resources, and perceived financial strain. These items were conceptualized as predictors of 18 items that measured outcomes of DPPs.

Sixteen of the items measuring outcomes asked about changes due to DPPs. Respondents were asked along a five-point Likert-type scale (1 = got much worse, 5 = got much better) to what extent exam preparation influenced: four items about professional development (medical knowledge base, confidence in ability to care for patients, confidence medical school was the right choice, confidence to be competitive for their specialty choice), two items about relationships (with colleagues, with loved ones), two items about quality of life (balance in personal and professional life, overall quality of life), one item about personal anxiety levels, and one item about feeling burned out from medical school. Six items asked along a five-point Likert-type scale (1 = much less, 5 = much more), compared to usual, how much respondents engaged in healthy behaviors (sleeping, eating healthy foods, exercising, spending time with friends, spending time with family, taking time away from academic responsibilities). All 16 of these items included a neutral option (e.g., “no change,” “about the same as usual”).

Two of the 18 outcome items asked how often during the DPP respondents either felt burned out from work and studying or bothered by feeling down, depressed, or hopeless (1 = never, 6 = every day).

Items related to confidence were adapted from a study on medical student doubt [33]. Items related to healthy behaviors were adapted from a study of internal medicine residents [34]. Items on quality of life, burnout, and feeling depressed were based on studies of medical learners using similar items [35,36,37,38]. All items were piloted with medical students who had previously completed their DPPs and revised for clarity and relevance prior to survey administration to our full sample.

Students have major life events while in medical school [39], but the impact of these events is understudied. Surveys included an item asking if respondents experienced “a significant life event that you believe impacted your ability to study.” If students answered yes, they were asked to categorize the event as (1) something that happened to a family or friend, (2) personal illness, (3) financial matter, and/or (4) an academic/extracurricular commitment. Respondents were also asked to describe the event.

Three additional open-ended items asked students (1) “Would you please share how preparing for USMLE Step 1/COMLEX Level 1 influenced you personally and professionally?”; (2) “How did you feel after you completed your exam? What was it like to have the exam behind you?”; and (3) “What did you do after you took your exam?”

Data analysis

We first calculated descriptive statistics. In bivariate analyses, we created quartiles for continuous predictor variables because they were not normally distributed. We dichotomized all outcomes variables based on how we hypothesized they would change as a result of the DPP. We aggregated “got much better” and “somewhat better” (vs the 3 other response options) for the four professional development items. We dichotomized the other 12 items measuring changes to well-being by aggregating “got much worse” and “somewhat worse” (vs the 3 other response options). We dichotomized healthy behavior items by aggregating “much less” and “somewhat less” (vs the 3 other response options) and refer to these as “worse.” Burnout and depressive symptoms were determined to be present if they occurred weekly or more often during the DPP. We used chi-squared tests to examine differences in rates of outcomes across schools. For analyses of predictors for each outcome, we performed logistic regressions adjusting for clustering within schools and report unadjusted p-values. We set statistical significance at alpha < .05. When interpreting results, we applied Bonferroni correction for 18 comparisons, which requires an unadjusted p value of < .0027 for an alpha of < .05. Stata 13 (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX, USA. StataCorp LP) was used for data analysis.

Responses to open-ended prompts on surveys often do not lend themselves to formal qualitative analysis [40]. However, students completing this survey often provided detailed responses to the open-ended item about the personal and professional impact of preparation for exams. Two authors coded responses for themes independently using an editing analysis method. Responses to other open-ended items were less descriptively rich but were likewise analyzed independently by two co-authors to summarize their contents.

Results

A total of 326/750 (43%) students responded to surveys, with response rates ranging from 37 to 53% across schools. Complete data were available for 314/750 (42%). Before DPPs, students spent a median of 10 h/week (interquartile range (IQR) 5–21 h/week) preparing for first-level licensure exams. DPPs lasted a median of 7 weeks (IQR 6–8 weeks), during which students reported spending 70 h/week (IQR 56–80 h/week) studying. Students used a median of 8 study resources (IQR 7–10). Most (78%) reported being very or extremely confident in the resources they selected for exam preparation.

Changes in student activities and wellness

Figure 1 illustrates responses to the 16 items we used to measure changes attributed to students’ DPPs. Most students felt that 2 outcomes improved: medical knowledge base (95%) and confidence in ability to care for patients (56%). Most reported 9 outcomes worsened, including overall quality of life (72%), feeling burned out from medical school (77%), and personal anxiety levels (81%). Most also reported burnout (71%) and feelings of depression (52%) weekly or more often.

Fig. 1
figure 1

Percentages of 316 second-year students across 4 U.S. medical schools who reported changing for better or worse with their dedicated preparation period for first-level licensure exams. a Responses measured along a 5-point Likert-type scale. “No change” responses are not shown. Percentages refer to the absolute value of proportion of students experiencing the change

Full size image

Rates for 5 outcomes showed statistically significant differences across the 4 schools (p < .0027): eating healthy foods, confidence that medical school was the right choice, confidence in being competitive for their specialty, relationships with colleagues, and relationships with loved ones. No statistically significant differences were seen in rates across schools for the other 13 items.

Study time and wellness

Relationships among hours per week studying and the 18 outcomes items are shown in Table 1. More time studying per week was significantly associated with 3 items after adjusting for clustering in schools (p < .0027): better medical knowledge base, worse time with family, and worse time away from academic activities.

Table 1 Relationships among professional and wellness factors and quartiles of hours per week studying during dedicated preparation periods (DPPs)
Full size table

Spending and perceived financial strain and outcomes

Students reported spending a median of $650 (IQR $440–1000) on resources or programs to prepare for their exam. A total of 25% (n = 80) reported that paying for preparation materials strained their finances “a lot” or “extremely.” Greater amount spent for preparation was associated with greater perceived financial strain (p < .001).

Significant associations (p < .0027) were found with increased amount spent on preparation resources and two outcomes items: worse exercising and overall quality of life. Greater strain on finances was associated with less favorable responses to 11 of the 18 outcomes items (p < .0027) (Table 2).

Table 2 Relationships among professional and wellness factors and actual spending and perceived financial strain
Full size table

Life events during DPPs

62/314 (20%) respondents reported experiencing a significant life event that impacted their ability to study during their DPPs, with something that happened to a family member or friend as the most common type of event (39/62 = 63%), followed by personal illness (15/62 = 24%), academic/extracurricular commitments (5/62 = 8%), and financial matters (4/62 = 6%). Examples of life events described by respondents included illnesses or death of loved ones; divorce or break-up; responsibility to work to provide for one’s family; new diagnosis, work-up, or treatment of one’s own illness; and pressure to complete abstracts or other academic projects during the DPP. Compared to those who did not report having a life event, those who reported having a life event had significantly worse changes to exercising (61% vs 79%, p < .001), balance in their personal and professional life (78% vs 87%, p < .001), and personal anxiety levels (69% vs 82%, p < .001) compared to those who did not report having a life event, but had no other statistically significant differences in the other 15 outcomes.

Qualitative items on personal and professional influence of DPPs

Analysis of students’ descriptions of how preparation for their exams influenced them personally and professionally revealed the following themes: (1) opportunity for synthesis of medical knowledge, (2) exercising the endurance and self-discipline required for professional practice, (3) dissonance between exam preparation resource content, formal curriculum, and professional values, (4) isolation, deprivation, and anguish in a competition for the highest possible score, and (5) effects on well-being extending beyond DPPs. Quotes that illustrate each theme are included in Table 3.

Table 3 Themes and illustrative quotes from students’ descriptions of how exam preparation influenced them personally and professionally
Full size table

Students’ descriptions of what it was like to have the exam behind them included feeling “relieved,” “numb,” “tired,” “terrible,” and “hopeful.” Students rested, caught up with friends and family, and traveled in the period immediately after their exams.

Discussion

This is the first multi-institution study to examine medical student well-being during dedicated preparation periods (DPPs) for high-stakes exams. Our findings can be used to inform recommendations for students and schools to make exam preparation less stressful and to add relevant evidence to licensure exam reform discussions.

Students begin medical school with well-being ratings that are similar or higher than their peers’ [41], but subsequently medical students, residents, and physicians have worse well-being than their non-medical colleagues [42, 43]. Exactly when the decline in well-being starts is not clear. Available data from the U.S. have suggested that well-being may remain stable during the first year of medical school [44] and worsen during the second year [45, 46], resulting in approximately 35–50% of medical students having symptoms of burnout and 25% having symptoms of depression [46,47,48]. USMLE Step 1, and to a lesser degree, COMLEX Level 1, have been invoked as worsening medical student well-being although supporting evidence has been limited. At one school, nearly three-fourths of students surveyed at the start of their clerkships reported burnout [49], and at another school, students surveyed shortly after their DPP reported burnout (79%), and feelings of anxiety or depression (61%), which they attributed to preparation for USMLE Step 1 [12]. Our estimates of burnout and depressive symptoms across four schools were substantially higher than the rates for burnout and depression during medical school in general and more consistent with reports from students who recently completed their DPPs. We further found that most other measures of well-being were reported to worsen during DPPs. Our estimates provide a new level of detail that characterizes student well-being and the specific healthy activities that may become compromised during their DPPs. These data may be useful in future studies that make comparisons or attempt to monitor interventions aimed at maintaining wellness during DPPs. As more countries implement standardized national licensure exams [50], they may be more attuned to students’ experiences in preparing for them.

Students’ statements about how DPPs influence them may provide insight into what occurs to students’ professional identity formation during DPPs. High-stakes exams have been implicated in eroding trust in medical education institutions and undermining traditional professional values [51]. Our study suggests that these processes are exacerbated during DPPs, while competition and the pursuit of maximum knowledge at the expense of self-care are incentivized. This may tacitly reinforce a traditional stoic physician stereotype that a growing number of wellness programs [52] seek to counteract. While many students may manage DPPs as a time of temporary imbalance and recover completely from them, our findings indicate that some may have lasting changes. Should a decline in the well-being be precipitated or exacerbated during DPPs and continue through students’ training, this could influence relationships in their personal and professional lives, and perpetuate a professional culture that makes it difficult to address the systemic factors that contribute to the epidemic of provider distress.

Mitigating educational debt has been proposed as a way to improve medical student well-being [53]. Commentaries on Step 1 have also suggested that the cost of exam preparation worsens the learning environment and ultimately contributes to inequities and physician workforce maldistribution [4, 5, 54]. Consistent with previous estimates [4, 5] and empirical work [12, 55], we found that students incur significant out-of-pocket expenses for exam preparation resources, despite being given preparation resources that have been paid for by their institutions. However, the level of student spending on exam preparation was small compared to the approximate median U.S. medical student debt of $200,000, and less than the price of taking each licensing exam. Evidence linking student spending on medical education and their well-being is mixed [56,57,58,59,60]. Our finding that financial strain was more closely associated with worse well-being than was actual spending is consistent with a qualitative study indicating that students experience debt in a variety of ways and that perception of debt may be more important than debt amount [61]. More work is needed to understand the relationship between educational spending and perceived financial strain. In the meantime, monitoring students’ financial strain may be more beneficial than tracking their spending amount. Also, as purchasing exam preparation materials has become an educational expense that all students preparing for their exams bear, schools may consider the benefits of offering needs-based financial assistance for exam preparation, as they do for tuition, to enhance both well-being and equity.

Important limitations must be considered when reviewing our findings. First, our response rate varied across schools, and overall, respondents were less than a majority of those in our sample. While there was consistency across the four schools in many of our findings, our ability to generalize across all students in our sample and to medical schools in general is limited. Second, our measures were self-reported, and we had an inability to adjust for student wellness before they began their DPPs. While we attempted to overcome this by asking for changes attributed to DPPs, responses are subject to bias. Third, while surveys were sent shortly after students completed their DPPs to improve their recall accuracy, not all students took their exams at the same time. Some students knew their exam scores and some did not, which could bias their perceptions of how well-being was influenced during DPPs. Fourth, while we attempted to shed light on student experiences with DPPs by measuring multiple dimensions of wellness and capturing students’ qualitative comments, there were inevitably important aspects of wellness that we did not evaluate and which warrant further investigation. Likewise, there are other variables that may contribute to changes in well-being during DPPs’ that we did not measure (e.g. expectations for their own performance) and could be included in future study. Finally, whether changes in well-being during DPPs are specific only to the particular circumstances of preparing for USMLE Step 1 or COMLEX Level 1, or would generalize to DPPs used for other high-stakes exams is unknown. All indications suggest that scores from Step 2 CK and COMLEX Level 2 will be used by residency programs just as first-level licensure exam scores had been and students may engage in DPPs for those exams that resemble those for students in our study; our methods may inform studies of preparation for those and other high-stakes licensure exams.

Conclusions

Every year, approximately half of the applicants to U.S. medical schools are turned away. The students who matriculate typically have demonstrated academic and personal excellence and an enthusiasm for learning medicine in service to patients and society. Taking time to develop and consolidate one’s medical knowledge is necessary to foster the expertise that all practicing physicians must possess and apply to their patient care. Our study has important implications for how students should develop and consolidate their knowledge foundation. Learning medicine can be gratifying and should be perceived as a privilege of participation in the profession. There is significant risk to student well-being and potentially to the profession if DPPs for high-stakes exams continue in their current forms.

Indeed, we should question whether there should be DPPs at all. Physicians must develop habits of continuous learning, and greater emphasis on continuous assessment that spaces learning over a longer period would be more effective at gaining knowledge than intense DPPs. Greater alignment of curriculum and licensure exam content may also help to make DPPs obsolete. In the meantime, strategies at a system-level to relieve the emphasis placed on a single exam score should be implemented to alleviate the stress students feel during DPPs. Schools should also recognize that many students placed into DPPs for required high-stakes exams experience them as a time of deprivation and suffering, and seek to offer preventative and supportive resources that mitigate DPP-related harms.