Introduction

As annual applications to general surgery residency program increase and, therefore, acceptance into residency becomes more competitive, understanding factors that influence decision-making is critical for both applicants and programs to assess best fit. Beyond identifying best fit, programs will also be challenged by increased demands on managing recruitment and assessing metrics from a growing pool of applicants. Applicants in turn often have minimal objective measures to assess surgical programs to assist in rank determination. While these metrics have always been challenging, it becomes particularly salient with the recent changes in National Board of Medical Examiners (NBME) reporting and adoption of virtual interviews.

The NBME United States Medical Licensing Examination (USMLE) Step 1 exam has been utilized as both a cut-off and a parameter to objectively evaluate applicants by over 90% of program directors (PDs) [1, 2]. However, critical review of the USMLE exams yielded considerable financial and social bias and found it to be a poor predictor of clinical accomplishment [3, 4]. Additionally, medical students reported that the exam negatively affected overall education and mental health [5, 6]. Based on these findings, the NBME changed the exam to a pass/fail report. Programs previously relying on the numerical cut-offs may now need new methods for screening, assessment, and evaluation and/or to rely more heavily on other application aspects.

The COVID-19 pandemic in 2020 disrupted traditional recruitment practices and surgical residency programs had to quickly improvise. New strategies were developed to ensure adequate exposure for critical evaluation of and by prospective applicants. Virtual interviews allowed applicants to apply more broadly as the burden of travel expenses disappeared, leading to a dramatic increase in the number of applications per program and per applicant [7, 8]. Unfortunately, virtual interviews are not synonymous to in-person interviews. The format is incapable of fully immersing an applicant in a program’s culture, location, and experience. The combination of this with increasing applications worsens the already highly competitive environment for both programs and applicants. In fact, studies have shown that one-quarter of program directors and applicants believe virtual interviews adversely impact the ability to match top applicants or match to top programs, respectively [9]. Without interpersonal interactions or direct observations of facilities, the community, and living options, it can be difficult for both applicants and programs to assess interest and genuine investment.

Prior studies evaluating the recruitment and match process have relied on data from the perspective of the residency program [10, 11]. To date, there is no literature describing the preferences and perceptions of general surgery applicants. Understanding applicant opinions and interpretation of the application process may provide new insight into the match process as it continues to undergo evolution. In addition, clarifying what is valued by applicants not only provides benefit for programs and program directors, but also for future applicants. This study aimed to (1) identify what general surgery applicants are looking for in a program; (2) determine what applicants believe programs are looking for; and (3) determine what applicant characteristics are associated with success as defined by number of interview invitations.

Methods

In September 2021, a web-based survey was designed by a stakeholders with diverse roles across the application process. This included a surgical resident, program directors, assistant program directors, a chair of surgery, and prior medical student clerkship directors. Design considerations incorporated limiting response bias, using neutral wording, providing non-leading questions, and intentional question ordering. The self-administered survey was comprised of 29 questions that queried academic and research performance, application characteristics, recruitment strategies, theories on match process, and overall match results as well as several demographic parameters (Appendix A). Excellence at something other than surgery was queried, defined as either collegiate or professional involvement in another discipline such as athletics, music, or art. The survey was anonymous without any email or IP addresses collected and completely voluntary. No participation incentives were offered and a statement indicating consent with proceeding forward was present at the top of the survey. The study was approved by the Institutional Research Board with an exempt status due to anonymous, web-based data collection design.

All past or current surgical residents of any clinical level in the United States were eligible to take the survey. The survey was disseminated through both social media and email with a cover statement and link to the questionnaire. It was posted on social media (Twitter and Facebook) by the authors from their own personal accounts. Additionally, it was disseminated via email to program directors and general surgery resident coordinators with universally accessible email addresses on program websites.

Data were analyzed using standard statistical methods, appropriate to the type of variable and normality of the data. Descriptive statistics including means, standard deviation, median, and interquartile range (IQR) for numeric variables, and counts and percentages for categorical variables were used to describe the study population. Bivariate and multivariable regression analyses were performed to assess factors correlating with success defined by the percentage of applications yielding an interview invitation out of total number of programs applied. A p value of < 0.05 was used for all significance determinations.

Results

The survey was completed by 220 surgical residents nationwide. Of the respondents, the majority were Caucasian with an average age of 31 years (Table 1). Most respondents matched in the last 5 years, 2017–2021 (n = 172), with the rest matching between 2007 and 2016 (n = 39).

Table 1 Demographics and characteristics of applicants and their match process (n (%) if unspecified) and bivariate regression analysis of factors associated with percentage of interview invites
Full size table

Application characteristics

Most respondents earned high honors or honors in both surgery and medicine clerkships with none receiving a failure (Table 1). A third of all applicants took part in an honors surgery rotation if offered by their medical school. Average USMLE Step 1 and Step 2 scores were in 236 (224–245) and 250 (240–258) respectively. A quarter to one-third of applicants were either GHHS inductees and/or alpha omega alpha (AOA) inductees and over twenty percent excelled at something other than surgery. Nearly half attended an away rotation and the majority had research experience with a median of 2 publications. Letter of recommendations (LOR) were often written by clinical faculty, followed by the chair of surgery, a PD, clerkship director, and research faculty.

Residency interviews

Figure 1 presents the number of programs applied and number of interviews offered and attended of each applicant. Applicants applied to an average of 61 programs with 16 interviews offered and 12 attended (Table 1). A third of applicants had a contact reach out to a program to get an interview, and 26% did the same for a match spot. Most applicants told the residency they matched at it was their number one choice and over one-quarter told more than one program it was their top choice.

Fig. 1
figure 1

Number of programs applied, and number of interviews offered/attended by applicants

Full size image

Applicant beliefs

Applicants believed the most important piece of their applications was the LOR, with few believing it to be either the personal statement, medical school grades, or step scores (Table 2). Applicants believed factors most important to programs were personality, followed by LOR writers, then medical school reputation. Top factors that applicants considered in ranking programs were resident culture, location, program reputation, and resident autonomy. Conversely, fewer highly valued research opportunities, diversity, and number of fellows. Additional factors noted to be helpful in ranking included faculty interviews, the program’s website, and the virtual resident interaction.

Table 2 Applicant beliefs regarding the match process
Full size table

Interview invitations

Univariate analysis found that application factors that were associated with an increase in percent of interview invites included age, year of match, higher Step 1 and Step 2 scores, higher grades in surgery and medicine clerkships, participation in an honor surgery rotation, and AOA and GHHS induction (Table 1). Compared to other races (mainly Caucasians), Asians had on average 14.7 decrease in percent of interview invites. Applicants who reported to excel in something other than surgery had a 14.8 decrease in percent of interview invites. LOR, Step 1 and 2 scores, honor surgery rotation, AOA, year of match, Asian race, and excelling at something else remained significant after multivariate analysis (Table 3).

Table 3 Multivariable analysis of factors associated with percentage of interview invites
Full size table

Change in application over time

Regression analysis between match year and percent of interview invites showed a statistically significant decrease in percent of interview invites over time (β = − 5.48, p < 0.001) which was mainly caused by a statistically significant increase (p < 0.01) in number of programs applied to over time (P < 0.01) (Fig. 2). The number of interviews offered and attended remained the same (p = NS, data not shown).

Fig. 2
figure 2

Average number of programs applied, and number of interviews offered/attended over time, along with percent of interview invites

Full size image

Discussion

General surgery residency match is founded on a strict algorithm intended to connect applicants and residency programs based on rank. While there is considerable data available on what programs value in candidates, there is no literature identifying the reciprocal—program components important to applicants for rank and selection. As the match process becomes more challenging with increasing applications, high selection process standards, use of virtual interviews, and changes in test result reporting, insight into factors associated with rank becomes essential. Such insight can not only improve program communication with applicants during the match process but also may lead to a better understanding of candidate preferences which could influence attrition rates and job satisfaction [11]. This study aims to illuminate such factors, hoping to provide transparency into how applicants assess the process of applying to general surgery residencies and what they and programs appear to value.

Prior research has focused primarily on understanding resident selection and interview process through faculty and program director perspectives. Assessments have sought to determine attributes of ‘successful’ or ‘competitive’ applicants based on prior experience and selection processes [10,11,12,13]. However, these studies are limited with the applicant pool often from a single center and the criteria for success being subjective and narrow, such as resident American Board of Surgery scores or a competitive fellowship match, defined by attending surgeons without incorporation of applicant viewpoints. These data found the most important single screening factor is USMLE step 1 scores followed by step 2 scores [14]. USMLE scores were also predictive of a successful match along with other objective components such as honors in surgical clerkships, AOA membership, and research experience and publications [15]. Predictions for resident proficiency and success, based on competency evaluations, found medical school honors as positive predictors alone [12]. The current study’s survey results found applicants beliefs were not necessarily in concert, believing personality and LOR were the most important factors to programs over objective data. Applicants may benefit from recognizing that markers for objective academic success hold considerable weight and may not necessarily be surpassed by personality and/or LOR.

The survey results depicting application characteristics found one-third of candidates had a contact reach out to programs of interest for either an interview or match offer. Over half of candidates let their top program know it was ranked first, however, nearly 30% told more than one program that it was ranked first. The interest of an applicant in a program has been shown to influence program ranking so reaching out to programs may produce benefit although the authors would caution against making multiple statements of intent [2]. Other interesting study findings included applicants favoring resident culture and program location above autonomy, case logs, and research opportunities. This finding may reflect the recent focus on wellness, which has emerged as a pertinent and key topic amongst clinical and administrative leadership [16]. The statistics in general surgery demonstrate over two-thirds of residents meet the standardized criteria for ‘burnout’ [17, 18]. Additionally, one-third are classified as having mild depression and a quarter suffer from clinical psychological distress [19, 20]. As such, the importance of culture over clinical acumen may not be surprising with similar results demonstrated in other literature [21]. Applicants ranked highly the importance of faculty interviews and program websites in ranking decisions. This serves as a compelling reminder to program directors to encourage lively faculty interviews and to keep program websites current and informative [22]. With similar results seen in the emergency medicine literature, website content can be an important factor in an applicant’s decision to apply [23].

Analysis of factors affecting success in the match process, as defined by interview invitations, found percentage of interview invitations to be decreased if Asian (compared to Caucasians) and increased with AOA status, LOR written by a chair of surgery, honor surgery rotation, GHHS status, and step scores. After multivariable analysis, LOR, step scores, honor surgery rotation, and Asian race remained significant. With the changes in test reporting to a pass/fail designation, programs will need to identify new parameters to evaluate candidates as USMLE step scores can no longer be used as cut-offs or objective evaluators. Survey results of program directors have concurring findings with about 50% citing use of both AOA and GHHS as important interview selection factors. This is problematic as honor society selection of candidates is not standardized across institutions. Further, concern for racial inequalities in AOA has emerged with less minority students selected for membership [24,25,26]. Finally, not all medical schools have AOA and/or GHHS chapters. LOR can also be problematic due to lack of standardization, objectivity, and inter-reader reliability [27, 28]. While the majority skew positively with significant academic inflation, they do offer a way to assess ability to work in a team, initiative and drive, professionalism, competence, and communication. The decrease in interviews for Asian applicants was a concerning, albeit not novel, finding. Many studies have demonstrated racial discrimination, particularly in LOR language with use of more agentic language and less personality discussion for Asian applicants [29,30,31,32]. While this study cannot confirm this causation, consideration of a universal, standardized LOR may decrease race-based bias.

Study limitations include not all-inclusive list of variables and survey methodology limited by response and selection bias. Further, this is a descriptive study and cannot identify causation. The survey was not beta tested and the use of social media and program director email limits the ability to determine the actual response rate. While responses were invited nationally, it still is a relatively small sample size and may not be evenly representative, limiting the generalizability of the findings. There was no cut-off in match year for survey respondents which induces recall bias along with experience variability and must be considered while interpreting the data. Further, questions regarding important factors were not specified as to referring to screening versus overall importance to programs. Future study is aimed at partnership with a national agency to disseminate a similar survey to all surgical applicants annually. Future studies might also assess these variables in other specialties in the USA and/or surgical program applicants in other countries. Furthermore, longitudinal studies of surgical applicants in the USA may be useful to monitor these results over time.

Conclusions

This national survey provides interesting insight into applicant beliefs and values which is particularly salient as the process of matching only becomes more challenging with surging application numbers, changes in testing parameters and virtual interviews. This information can further benefit surgical residency programs and PDs by providing targeted information for improvement in program communication, information dissemination, and interview day outline and organization.