Background

Whether applying to medical school, residency programs, fellowships, or for the role of a staff physician, interviews play an important part of the selection process for both applicants and programs. The Coronavirus 2019 (COVID-19) pandemic, and the public health measures that have accompanied it, has necessitated a transition away from face-to-face interviews. This provides an opportunity to scrutinize the benefits and limitations of video-based interviewing as they compare to traditional face-to-face interviews.

Face-to-face interviews come with both financial and opportunity costs that have been reported in previous studies [1,2,3,4]. Financial costs result from application fees, travel and accommodation for interviews, and costs associated with completing elective rotations [1,2,3]. Opportunity costs such as time taken away from clinical service to attend interviews, which applies to applicants and interviewer, should also be considered [3, 4].

The environmental costs of face-to-face interviews have not been well-studied in the literature, but a recent emphasis has been placed on the impact of long-haul flights associated with traveling for interviews on our carbon footprint [5]. In line with this, the healthcare sector is regarded as a major contributor to greenhouse gas emissions, urging for a call to action towards climate change as a public health emergency [6, 7].

Video-based interviewing presents an opportunity to continue with personnel selection by adhering to COVID-19 restrictions while addressing the aforementioned costs associated with face-to-face interviews. While previous studies have reported on their experience with video-based interviews and provided summaries of the current literature on the topic, a formal structured synthesis of the literature on video-based interviewing within healthcare does not exist [8,9,10,11,12,13]. Given the rapid transition to video-based interviewing, an opportunity was identified to consolidate the current literature on the topic.

Methods

Study aim

The aim of this review was to evaluate the extent of previous research on video-based interviewing as it applies to healthcare contexts. We sought to review the benefits and limitations of video-based interviewing and identify gaps in our current knowledge on the implementation of video-based interviewing. The research question that guided this scoping review was as follows:

Within selection processes in healthcare, how do video-based interviews compare with face-to-face interviews in terms of costs, implementation, and candidate selection?

Study design

A scoping review was conducted in anticipation of the heterogeneity of study designs available on this topic. We employed the methodology that has been previously outlined and subsequently refined [14, 15]. Full details to the conduct of this scoping review can be found in our previously published study protocol [16]. In short, a search strategy developed in conjunction with an academic health sciences librarian was used with the following databases: Ovid MEDLINE, Embase, PsycInfo, and Cochrane Central. The search was run on March 31, 2020, to capture studies from inception to March 30, 2020. The gray literature was searched using Google, and the reference list of all studies selected for inclusion was reviewed for any additional studies that fit our inclusion criteria. Our study was reported according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist, which is provided in Additional file 1 [17]. The search strategy can be found in Additional file 2. The search was updated on February 20, 2021.

Inclusion and exclusion criteria that were developed to select studies that were in line with our research question are provided below. Studies that were not published in English or French were excluded due to translation services available. There was no limitation based on publication date.

Inclusion criteria:

  • Involve applicants interviewing via video-based and/or face-to-face interviews AND

  • Involve applicants applying to medical school, residency, fellowship programs, dentistry, pharmacy, nursing, or other healthcare-related fields AND

  • Any study design that involved the implementation of video-based interviews AND

  • Any method of data analysis, including quantitative and qualitative studies AND

  • Assess any outcome of interest including financial costs, environmental impact, or time invested

Exclusion criteria:

  • Review articles OR

  • Editorials or expert opinions that do not describe a particular video-based interview that was implemented OR

  • Studies that are not published in English or French

Data acquisition

The results of our search were initially screened based on title and abstract by two independent reviewers (RS, RH). Cohen’s kappa statistic was determined following this to ensure inter-rater reliability before proceeding to the next stage of screening. Any discrepant studies by either of these reviewers were included in full text review. After initial screen based on title and abstract, the two reviewers (RS, RH) independently screened the full texts of the selected studies in duplicate. Any discrepancies following this were reviewed by a third independent reviewer (HM) who made the final decision on study inclusion. Reasons for exclusion after full text review were documented.

DistillerSR (Evidence Partners, Ottawa, Canada) was used for the data extraction process. A standardized form was created on this platform and independently tested by two reviewers (RS and RH) on the initial 10 studies that were included. Following this, the data extraction form was modified to capture the range of methodologies and outcomes of the studies that were included in our scoping review. No further changes were made to the data extraction form after this modification, and the same standardized form was used on all included studies.

Data analysis

Given the nature of a scoping review, no formal statistical synthesis of the study data was pursued. Study characteristics including geographic location, study methodology, discipline of authors, purpose of interview, and type of analysis are presented as descriptive data. In line with our aim to identify the benefits and limitations of video-based interviewing, outcomes from each study were also grouped thematically as they applied to financial costs, opportunity costs, environmental costs, technical limitations, impact on the rank list, and body language.

Quality assessment

Risk of bias assessment of each included article was performed using the Joanna Briggs Institute (JBI) critical appraisal tools by two independent reviewers (RS, RH) [18]. Although not a typical component of a scoping review, we sought to provide an assessment of the study quality available in the current literature. Abstracts were not critically appraised as they lacked the methodological detail required to assess for quality and risk of bias. The tools relevant for cohort studies, cross-sectional studies, randomized controlled trials, and quasi-experimental studies were used as applicable. Based on the results of the JBI critical appraisal tool, an overall study rating was assigned by the reviewers to describe the risk of bias as either “high,” “moderate,” or “low” risk. All studies that met the inclusion criteria were included in our scoping review, regardless of the results of their quality assessment.

Results

Search results

A total of 3851 studies were retrieved from our search, after duplicates were removed. Following review of titles and abstracts by two independent reviewers, a total of 76 studies were selected for full text review (Cohen’s kappa = 0.84). Following independent full text review by two reviewers, a total of 37 studies were selected. These were reviewed by our third reviewer after which 36 articles were included. Reasons for exclusion after full text review included the lack of any video-based interview being implemented (n = 19), letters to the editor (n = 7), did not involve applicants in healthcare fields (n = 3), did not have any measured outcomes (n = 8), or abstracts which full text was already included (n = 3). The search was updated on February 20, 2021, and 5 additional studies met inclusion criteria. Two additional studies were retrieved from review of the references of the included studies. As such, a total of 43 studies were included in our scoping review [19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61]. A PRISMA flow diagram is provided in Fig. 1 to outline the study selection process [62]. A summary of the demographic data and eligibility criteria of the included studies is provided in Table 1. Our outcomes, which were group thematically, are summarized separately in Table 2.

Fig. 1
figure 1

PRISMA 2020 flow diagram for new systematic reviews which included searches of databases and registers only. From: Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. doi: 10.1136/bmj.n71. For more information, visit: http://www.prisma-statement.org/

Full size image
Table 1 Characteristics of included studies
Full size table
Table 2 Study outcomes grouped thematically
Full size table

Study characteristics

Of the included studies, 17 were conference abstracts and 26 were published manuscripts. Forty-two studies were conducted in the USA, while 1 was conducted in Australia.

Twenty-nine studies included medical students applying to residency, 2 studies of students applying to medical/dental school, 2 studies of pharmacy students applying to their respective pharmacy residency, and 10 studies of residents applying to fellowship. Of studies involving medical residency or fellowship applications, disciplines included urology (n = 1), anesthesiology (n = 1), general surgery (n = 1), pediatric surgery (n = 1), gastroenterology (n = 1), internal medicine (n = 1), orthopedic surgery (n = 2), obstetrics and gynecology (n = 2), family medicine (n = 2), complex general surgical oncology (n = 3), female pelvic medicine and reconstructive surgery (n = 1), maternal fetal medicine (n = 1), advanced gastrointestinal minimally invasive surgery (n = 1), and emergency medicine (n = 22).

There was a range of study methodology. There were 36 cross-sectional studies, 4 cohort studies, 2 quasi-experimental studies, and 1 randomized trial. All studies employed some form of quantitative analysis, while one abstract did not outline what analysis they performed. No studies used qualitative methods.

Virtual panel interview

Eighteen studies reported on their use of panel style virtual interviews. Of these, 6 were adjuncts [19,20,21,22,23,24] to the face-to-face interview while 12 were replacements [25,26,27,28,29,30,31,32,33,34,35,36]. Studies used Skype [19, 21, 24,25,26,27,28, 30], FaceTime [19], Zoom [31, 32, 34,35,36], or a combination of two platforms [22, 23, 29]. Eleven studies provided additional information outside the virtual interview itself including a video tour of the facility, video tour of the surrounding communities, Google hangout session with current residents, electronic brochures, and resident contact information to ask additional questions if interested [19, 26, 28,29,30,31,32,33,34,35,36]. The number of interviews varied from a single 15–30 min video-based interview [20, 22,23,24,25,26,27, 30, 32, 33] to multiple interviews with different faculty members [19, 21, 28, 29, 31, 34,35,36]. The number of interviewers also varied from one-on-one interviews [19, 20, 24, 27, 28, 34, 35] to those with up to five interviewers [22, 23, 26, 30,31,32,33]. Three studies described having a dedicated individual (e.g., program coordinator, information technology specialist) who facilitated movement of applicants and interviewers between breakout rooms on the video-based platform [31, 34, 35]. Three studies had an administrative staff responsible for ensuring that applicants had established appropriate audio/visual connections [19, 27, 28].

Nine studies reported favorable perceptions from applicants on video-based panel interviews [20, 21, 24, 26, 31, 32, 34,35,36]. Applicants in these studies felt that they were able to demonstrate their strengths and personality through a video-based format [26, 32, 35, 36], that it was a fair way to present the program to them [21], and that the interviews flowed well [31]. Most applicants found that the video-based interviews met their expectations [20, 26], and they were satisfied with the process [34].

In one study, applicants felt that the video-based interview was less effective in allowing them to represent themselves [28]. An additional study highlighted that video-based interviewing prevented the “gut feelings” about a program that are typically felt at the time of a face-to-face interview [33]. Most applicants in one study also found that the inability to see the city or meet the faculty face-to-face was a drawback to video-based interviews [34]. Other concerns with video-based interviewing included not being able to understand the program’s culture [35] and interact with current residents [29].

Few studies explored the interviewers’ perspective on video-based interviewing and found that interviewers were mostly satisfied with the video-based interview process overall [26, 31, 34]. In one study however, none of the interviewers recommended using video-based interviews as the sole method of interviewing [21].

In the 12 studies that specifically assessed the role of video-based interviewing as a replacement to the current process, two found that it worked well to replace face-to-face interview [31, 34], while 10 studies felt that it was a useful adjunct or screening tool, but was not ready to replace face-to-face interviews entirely [19,20,21,22, 26, 28, 32, 33, 35, 36].

Overall, the panel-based video interview format appeared to be acceptable to applicants and interviewers, although a complete transition away from face-to-face interviews has been met with hesitancy.

Virtual multiple mini-interviews

One study reported on their use of video-based multiple mini-interviews (MMIs) [37]. With their retrospective cohort study comparing applicants to medical and dental school between years that employed face-to-face MMIs with those that employed video-based MMIs, they found that there was no significant difference between interview scores, although there was greater variability in scores in the video-based MMI group.

A total of 76% of applicants and 78% of interviewers were satisfied with the video-based MMI process. Similar to the face-to-face counterpart, the video-based MMIs involved seven questions with 2-min change over time between stations. The interview process was set up and overseen by five administrative staff and two IT staff. This study was the only study that explored the role of a video-based MMI as a replacement to the face-to-face counterpart. This one study demonstrated that the video-based MMI was an acceptable alternative to an in-person process.

One-way video interviewing

One-way video interviewing describes the process by which applicants submit answers to standardized questions in video format to be evaluated as part of the selection process. There were 24 studies that employed this method of video-based interviewing.

Twenty-two of these studies evaluated the standardized video interview (SVI), an online unidirectional interview that was developed by the Association of American Medical Colleges (AAMC) and piloted with the Accreditation Council for Graduate Medical Education (ACGME)-accredited emergency medicine programs. Twelve of these studies were abstracts [38,39,40,41,42,43,44,45,46,47,48,49] and 10 were articles [50,51,52,53,54,55,56,57,58,59]. Applicants submitted an audio/video response to six questions which was subsequently scored from 6 to 30. The goal of the SVI was to provide standardized information about applicant’s interpersonal and communication skills and professionalism. It was introduced in 2016 as a research project and was administered as an operational pilot in the emergency medicine residency selection during the 2018, 2019, and 2020 match cycles. Although the SVI was not continued for the 2021 match cycle, these studies were included in our scoping review to assess the value of unidirectional interviewing.

Overall, there were small correlations between SVI scores and other aspects of the application such as USMLE scores [41], faculty scores of communication and professionalism [51], electronic standardized letter of evaluation (eLOE) [55], patient evaluation of communication skills [46], and traditional interview scores [47, 48]. One study questioned the utility of the SVI as there was a lack of a relationship between SVI scores and applicant ranking [40], while another study found that the SVI score changed the likelihood of a program to invite the applicant for an interview in 7% of cases [56]. Of studies that explored the applicant perspective, two found that applicants did not feel that the SVI should be part of the application process [39, 51].

Apart from the studies that assessed the AAMC’s SVI specifically, two other studies described their use of one-way video interviewing or asynchronous video interviews [60, 61]. Both these studies utilized one-way interviewing as an adjunct to face-to-face interviews, rather than a replacement. Applicants submitted a video response to three standardized questions that were scored to determine which applicants would be subsequently invited for a face-to-face interview.

Higher scores on this one-way interview were correlated with higher in-person interview scores [60, 61]. While one study reported a positive correlation between one-way interview score and rank list placement [60], the other study reported a nonsignificant positive correlation between the two [61].

Overall, results were ambiguous as to what the SVI was measuring and the value that should be attributed to it amongst other aspects of the emergency medicine selection process. Nonetheless, one-way interviews may hold some promise as an initial screen, but there is no evidence we found to demonstrate that it should replace a bidirectional interview.

Technical limitations

Eleven of the included studies discussed technical limitations of video-based interviewing [19,20,21,22,23, 28,29,30, 34, 36, 37]. Three studies discussed issues with connectivity that were resolved by reconnecting or switching to telephone interviews without video [22, 23, 28]. One study discussed the voice delay associated with using the Skype platform for video-based interviews [21]. One study explained that while faculty was nervous about potential technical issues with video-based interviews, only one interviewer had a temporary technical connectivity problem [36].

No studies reported any major concerns from a technical point of view that limited the use of video-based interviewing.

Financial cost

Fourteen studies discussed the financial costs associated with the selection proces s[19,20,21,22,23,24, 28, 29, 31, 33, 34, 36, 37, 59]. Thirteen of these studies reported a reduction in financial costs for either applicants [19, 20, 22, 33, 34, 36], programs/interviewers [23, 24, 29], or both [21, 28, 31, 37]. One study reported that applicants who matched successfully spent significantly more money compared to those that did not match [19]. In another study where applicants were given the option of interview format, 25% of applicants chose a video-based interview due to financial limitations [29]. Additionally, the importance of modifying the surgical fellowship recruitment process given increases in student debt had been alluded to by one study [36].

Overall, video-based interviewing was seen as a way to improve financial costs for all stakeholders, particularly for applicants in whom financial limitations impact their application process.

Opportunity costs

Fifteen studies discussed the opportunity costs associated with the selection process [19,20,21,22,23, 26, 27, 29, 31, 33, 34, 36, 37, 58, 61]. Eleven of these studies reported that video-based interviewing afforded the applicants the ability to expend less time with the interview process [19,20,21,22,23, 26, 28, 33, 34, 36, 37], which would mean less time being taken away from clinical or educational commitments [20, 21, 28, 34].

Two studies discussed that video-based interviewing was particularly beneficial for applicants that could not get time off work to attend face-to-face interviews [29] and for applicants who would not be able to attend the face-to-face interview due to interview scheduling conflicts [20]. One study also highlighted that residency programs themselves would experience fewer disruptions with video-based interviewing since residents could take less time away from clinical duties [36].

One study explained that the introduction of the SVI may have increased the time required for applicants to prepare for the interview itself [28]. From a program’s perspective, one study reported that the utilization of one-way video interviewing prior to face-to-face interviews delayed the face-to-face interviews by 3 weeks and increased the burden of work from the program [61].

While one-way video interviewing may come at the cost of increased work for both applicants and interviewers, bidirectional video-based interviewing allows applicants to take less time away from personal or professional commitments and results in fewer disruptions for the programs themselves.

Environmental cost

None of the 43 studies discussed the environmental impact of interviewing in healthcare as it applies to personnel selection.

Body language

Three studies discussed the role of body language as it pertained to video-based interviewing [21, 23, 33]. One study commented on the lack of physical contact such as a hand shake but explained that this was more of a concern for interviewers [21]. Another study discussed how the loss of video connection in some interviews prevented the assessment of body language which was part of the evaluation [23]. Finally, one study commented on the lack of subjective details from an interaction that are lost in a video-based format [33].

None of the studies discussed the interpretation of body language and how this plays into applicant selection.

Influence on rank list

There were 12 studies that assessed either the perceived or objective impact of video-based interviews on the rank list [19, 26,27,28,29,30, 35, 45, 56, 53, 60, 61]. Two of these studies reported a positive relationship between video-based interview scores and overall rank list [60, 61]. Three studies found that there were no differences in acceptance rate and/or rank list position based on whether applicants had a face-to-face or video-based interview [27,28,29].

Three studies reported from the applicants’ perspective that most applicants were comfortable ranking programs after a video-based interview [26, 30]. One study reported that only 54% of applicants who had a virtual interview felt that the experience was sufficient to make a ranking decision, compared to 92% of their counterparts who had a face-to-face interview experience [35].

From the interviewer perspective of determining the rank list, three studies reported that there was no significant difference of interview type on the ranking of applicants [28, 29].

In terms of the SVI, one study reported the lack of any significant correlation between the SVI scores and rank list position [45], while another study reported that the SVI score changed the likelihood of inviting an applicant for a face-to-face interview in 7% of cases, with lower SVI scores more likely to decrease the chance of an interview invite than higher scores were to increase the chance of an interview invite [56].

Quality assessment

Quality assessment was completed for the 26 peer-reviewed manuscripts included in our scoping review. Using the JBI critical appraisal tools to evaluate each article, 11 studies were felt to have a high risk of bias, 12 with a moderate risk of bias, and only 3 studies with a low risk of bias. Most studies lacked a valid and reliable tool to evaluate applicant/interviewer perspectives on video-based interviews, did not assess or control for potential confounders, or had poor survey response rates. A summary of our quality assessment is provided in Additional file 3.

Discussion

Our scoping review summarizes the current literature and highlights major themes about video-based interviewing in healthcare. Overall, both financial costs and opportunity costs associated with the selection process were reported to be improved with video-based interviewing, while the impact on environmental costs has not been well explored in the current literature. Bidirectional video-based interviews were well received by both applicants and interviewers, yet a preference remains for face-to-face interviews. One-way video interviewing may be useful in select settings as a screening tool, but was not found to be a good replacement for bidirectional interviewing.

Important features of video-based interviewing that were described by multiple studies included the use of a video tour of the facility/city, an informal video-based session with current trainees, and having a dedicated administrative person to help applicants and interviews navigate through the video-based platform. While technical limitations were anticipated, there were no major technical issues that limited the use of video-based interviews. The interpretation of body language in a video-based setting was not well explored.

In the context of the social distancing measures necessitated by the COVID-19 pandemic, video-based interviewing has garnered recent interest across multiple disciplines within healthcare which were captured by our broad search strategy. Our search was also updated after its initial run to capture the influx of studies that were published since the start of the COVID-19 pandemic. Previous reviews have not been as structured or encompassing of video-based interviewing in healthcare as our scoping review [13, 63], and it has been reported that developments addressing medical school/residency interviews are underreported [63].

The results of our review are in line with those from a previous literature review but also serve to highlight gaps in literature. Joshi and colleagues outlined benefits and drawbacks to video-based interviewing [13]. Similar benefits were found in our study including the decrease in financial costs and time taken away from clinical or academic duties. While Joshi and colleagues allude to the reduction in carbon footprint with video-based interviews, they also did not find any studies that evaluated this aspect.

An important finding in our review was that despite studies reporting that applicants and/or interviewers felt that the video-based interview worked well, completing transitioning away from face-to-face interviews was not seen as a favorable option. From the results of our review, however, reasons for this are unclear. One of the challenges with assessing the reasons for this hesitancy may be the methodology of the available studies. While using surveys to quantify applicant and interviewer perspectives on video-based interviewing is helpful, it falls short in highlighting the thought processes that guide decision-making. It has been suggested that qualitative studies that use an inductive approach to data gathering may be more beneficial in such circumstances to get a better understanding of the reasoning behind our decision-making [64].

Another aspect of video-based interviewing that was not well reported in the included studies is the role of body language and nonverbal communication. While the lack of body language was alluded to in three studies, quantifying the impact of body language on the video-based selection process has inherent challenges and has not been assessed in a healthcare context. In a study by Proost and colleagues from the organizational psychology literature, it was found that applicants are less attracted to organizations that use video-based interviews compared to those that use face-to-face interviews for their selection process [65]. They explain that this perception may be influenced by the decreased ability to convey and interpret nonverbal cues on a virtual platform. While it has been studied on the organizational psychology literature, the results of our scoping review suggest that nonverbal communication as it applies to video-based interviewing in medicine is an area for future research.

Given the nature of the topic, the challenges associated with survey studies, and the inherent biases involved in the selection process, most studies had a moderate or high risk of bias. Additionally, analyzing perceptions around video-based interviewing comes with the challenge of measuring differing viewpoints in a reliable and valid way. We did not come across any validated measurement tool as studies developed and implemented their own surveys. While the goal of our scoping review was not to statistically synthesize data, we appreciate that collating the results of these heterogenous studies has its difficulties. Future research on developing an assessment tool to grade the quality of the interviews may prove helpful in standardizing the evaluation of interview processes.

Conclusion

Our scoping review summarizes the available literature on the use of video-based interviewing in healthcare contexts and has highlighted important areas for further exploration. Video-based interviewing, while necessary during the COVID-19 era, provides benefits from a financial, opportunistic, and environmental point of view that argue for its continued use even after the pandemic. While video-based interviewing has been reported to be a feasible alternative to face-to-face interviewing, hesitancy remains to completely transition from face-to-face interview for reasons that are difficult to extrapolate from the currently available literature. In addition, the role of nonverbal communication and how this factors into decision-making is important to evaluate, as it will help to better understand the complex thought processes that underlie personnel selection in healthcare.