The COVID-19 pandemic precipitated an urgent need for training programs to exponentially make use of telehealth technologies and pivot to the telesupervision modality (Bell et al., 2020). At the onset of the pandemic, a survey of members of the Association of Psychology Training Clinics (Hames et al., 2020), the national organization for directors of psychology training programs, revealed that 86% of clinic directors indicated they primarily moved to telehealth for all clinical services. The rapid and expanded adoption raised questions about telesupervision effectiveness and has prompted a growing body of relevant empirical knowledge (Phillips et al., 2021). Building from the work of Phillips and colleagues (2021), a review of key studies is shown in Table 1. In brief, these studies highlight continued equivalence in supervision satisfaction, supervisory working alliance/relationship, and supervisee self-efficacy across modalities, with increased access, flexibility, and telehealth skills as noted benefits and technology issues, less non-verbal information, and less connection/more distraction as consistent challenges (see Table 1). One limitation of the current literature is that studies primarily focus on trainee perspectives, with few studies including supervisor and Training Director perspectives, and no published research that includes a comparison of trainee and supervisor perspectives.

Table 1 Research on telesupervision
Full size table

Further, the literature highlights the following needs to support telesupervision provision: (1) establish specific training guidelines and structure (Augusterfer et al., 2020; Patel et al., 2021; Phillips et al., 2021; Watters & Northey, 2020); (2) be intentional in building supervisor and trainee relationships (Bernhard & Camins, 2021; Hames et al., 2020; Schmittel et al., 2021; Watters & Northey, 2020); and (3) ensure access to supervisors and experiential learning (Hausman et al., 2021). The competency-based supervision framework, which has a well-established literature base (Falender & Shafranske, 2017; Falender et al., 2014; Grus, 2013), highlights competency-based supervision as an approach for fostering the acquisition of trainee knowledge and skills needed for clinical learning outcomes (Falender & Shafranske, 2017). The framework provides guidance on how to continue trainee competency development and maintain essential elements of effective supervision as the supervision modality evolves. The elements of competency-based supervision include the development of an effective working alliance between a trainee and their supervisor, consistent evaluative feedback, consistent supervision meetings, and direct observation of clinical work (Falender, 2018).

Using a complementary mixed-methods design (Palinkas et al., 2011), quantitative methods were used to compare trainee and supervisor perspectives related to how essential elements of supervision were upheld during the rapid transition to the telehealth modality due to the COVID-19 pandemic. Qualitative methods were used to explore benefits and challenges of telesupervision as well as facilitators and lessons learned.

Procedure

This project was reviewed by the VA Office of Research & Development and determined to be a quality improvement project and therefore not subject to institutional review board approval. All VA psychology training programs are required to be members of the Association of Psychology Postdoctoral and Internship Centers (APPIC) and accredited by the American Psychological Association (APA). According to the APPIC directory, there are 621 VA internship positions across 118 training sites and 372 postdoctoral positions across 76 VA training sites (APPIC, 2023). Training Directors at APA-accredited VA clinical psychology training programs were emailed links to supervisor- and trainee-facing anonymous questionnaires and were asked to send the respective links to supervisors and trainees (psychology interns and postdoctoral residents) who recently finished or were nearing completion of their training programs and who engaged in telesupervision during the 2019 to 2020 training year. Thus, respondents consisted only of VA psychology supervisors (n = 242) and trainees (n = 128) who responded to an email from their Training Directors, and response rates cannot be calculated. The one-time questionnaire responses were collected between 08/06/2020 to 09/04/2020. Given the early-pandemic context of questionnaire dissemination, data are reflective of the variety of modalities used for clinical supervision at that time (e.g., video-based, telephone-based, in-person, hybrid).

Questionnaires

The supervisor and trainee questionnaires differed only slightly and included quantitative and qualitative items as well as demographic items including training setting, location of patients served, and telesupervision and telehealth experience. Quantitative items employed a 5-point Likert scale and assessed supervision consistency, trainee knowledge of telehealth emergency procedure, frequency of evaluative feedback, and frequency and type of direct observation. Qualitative items assessed differences in supervision quality, telesupervision facilitators, and benefits and challenges related to telesupervision. Supervisory alliance was assessed using the supervisor- and trainee-versions of the Supervisory Working Alliance Inventory (SWAI-S and SWAI-T, respectively; Efstation et al., 1990), which measures the quality of the supervisor-trainee relationship. Both versions employ a 7-point Likert scale, with mean scale scores calculated for each subscale. The SWAI-S has 23 items with three subscales: Client Focus (emphasis on clinical needs); Rapport (emphasis on establishing rapport); and Identification (emphasis on the trainee’s agreement/alignment with supervisor). The internal consistency coefficients for the Client Focus, Rapport, and Identification scales were 0.79, 0.66, and 0.81, respectively. The SWAI-T has 19 items with two subscales: Client Focus (emphasis on clinical needs) and Rapport (emphasis on establishing rapport). The internal consistency coefficients for the Client Focus and Rapport subscales were 0.88 and 0.92, respectively.

Data Analysis

Quantitative data were analyzed using SPSS (Version 26) statistical software. A significance level of α = .05 was used for all analyses. The data were checked and screened for univariate and multivariate errors and normality prior to running any analyses. Pearson’s chi-square analyses were employed to illustrate differences between trainee and supervisor experiences on the quantitative variables. Supervisors and trainees used the SWAI to rate supervisory alliance in both the in-person and telesupervision modalities. Separate analyses for trainees and supervisors were conducted using paired samples t-tests to compare mean SWAI ratings between the two modalities.

The qualitative data analysis was conducted using a thematic analysis approach utilizing consensus coding to provide more information on questions of inquiry (Braun & Clarke, 2006; Sundler et al., 2019). Responses of supervisors and trainees were analyzed as two separate datasets. Two project team members completed the qualitative analysis. Both provided telesupervision prior to and at the onset of the pandemic. First, both team members reviewed the qualitative dataset multiple times to gain familiarity with the data. Each team member then independently coded the data to identify emerging patterns, each utilizing an electronic log that included columns to delineate emerging themes and placed relevant data within these themes. The team members then met to review emerging themes and discuss similarities and differences within identified themes. An open discussion strategy was used for reaching consensus on any differences in coding of data. Identified themes were compared to raw data text to determine whether themes were consistent with the meaning of the text as well as detect additional themes and subthemes. The data was independently reviewed again to identify larger patterns of response, identified as superordinate themes. Team members met again to review superordinate themes, themes, and subthemes and returned to the data to ensure themes appropriately captured nuanced meaning of the data. Team members met to compare themes across the two datasets, detecting similarities and differences in identified themes for supervisors and trainees.

Results

Respondents

A total of 242 supervisors and 128 trainees participated (see Table 2). The higher number of supervisors is likely reflective of each trainee being assigned two to three supervisors over the course of the training year. Consistent with the population of psychology supervisors and trainees, respondents primarily identified as female (n = 267, 72.2%), white (n = 302, 81.6%), and at urban training settings (n = 277, 74.9%), with some differences in the racial and ethnic diversity of supervisor and trainee samples (χ2 = 20.3, p < 0.01). Although few previously endorsed prior telesupervision experience (n = 56, 15.1%), it is notable that supervisor and trainee samples differed significantly (χ2 = 14.7, p < 0.001). This is consistent with telesupervision being minimally used across VA training programs as policy was more restrictive prior to pandemic emergency allowances, whereas trainees may have experienced telesupervision in other training settings. Of the trainees, 73 (57.0%) were interns and 55 (43.0%) were postdoctoral fellows. Supervisors averaged 9.2 years of supervision experience (SD = 6.93; range = 1–33 years).

Table 2 Descriptive statistics for study variables
Full size table

Quantitative Findings

Knowledge of Telehealth Emergency Procedures

Chi-square test results are presented in Table 3. Results from these analyses revealed that, compared to supervisors’ ratings of trainees’ knowledge, trainees endorsed having less knowledge of telehealth emergency procedures (χ2 = 5.85, p < 0.05).

Table 3 χ2 tests for supervision elements within the telesupervision modality
Full size table

Essential Elements of Competency-Based Supervision After Pivoting to Telesupervision

Feedback, Supervision, and Oversight

Compared to supervisors, trainees endorsed less frequency of evaluative feedback (χ2 = 17.84, p < 0.001), less consistency in supervision meetings (χ2 = 6.29, p < 0.05), and less frequency of direct observation (χ2 = 11.60, p < 0.01; see Table 3).

Supervisory Alliance

Overall, high levels of alliance were reported across subscales. Though analyzed separately, trainees and supervisors endorsed a similar pattern of experiences as they pivoted to telesupervision from in-person supervision (see Table 4). First, among both supervisors and trainees, mean SWAI ratings on the Client Focus subscale in the in-person modality were statistically significantly higher compared to the telesupervision modality. Although both trainees and supervisors reported statistically different, the means are not practically different (mean differences of 0.11 and 0.10 for supervisors and trainees, respectively). Next, means for the Rapport subscale were not significantly different for either supervisors or trainees. Finally, there were statistically different means between modalities on the Identification subscale for supervisors; however, the mean difference was only 0.10. Given the limited variability (SD ≤ 0.75) on all items, the meaningfulness of small differences between modalities must be interpreted with caution, if at all.

Table 4 Paired sample t-tests for SWAI subscales
Full size table

Qualitative Findings

The qualitative analyses identified three superordinate themes pertaining to the telesupervision questions: benefits, challenges, and implementation needs and feasibility. Of note, a fourth superordinate theme (pandemic response) was identified that included data unique to the experience of pandemic onset (e.g., childcare centers being closed, concerns about illness) and is therefore not included. The synopsis below lists superordinate themes with a description of corresponding themes and subthemes (see also Tables 5, 6, and 7). Most themes and subthemes overlapped across supervisor and trainee responses, with some notable differences.

Table 5 Superordinate theme: benefits of telesupervision
Full size table
Table 6 Superordinate theme: challenges of telesupervision
Full size table
Table 7 Superordinate theme: needs and feasibility
Full size table

Synopsis of Superordinate Themes, Themes, and Subthemes

Superordinate Theme: Benefits of Telesupervision

Supervisors and trainees reported quality was maintained across modalities. In particular, supervisors and trainees reported perceiving telesupervision as equivalent to in-person supervision, while highlighting the benefit of video over phone. Some trainees and supervisors attributed quality being maintained due to an already established in-person relationship while trainees identified additional benefit of increased depth and structure of telesupervision sessions. In addition, improved supervisory oversight was another benefit inclusive of the following: increased availability and access via electronic means, increased access to observation of clinical work, and less interruptions in scheduling of consistent meetings (notably because of reduced need for travel time). Furthermore, supervisors and trainees noted improved training program functioning: flexibility and convenience, increased availability of training experiences and access to supervisors (via engagement with remote supervisors), engagement in practicing telehealth, and sharing of information/resources electronically. Both supervisors and trainees identified a benefit of having increased access to supervisors including the following: being able to access supervisors for oversight or assistance more readily, having a less intrusive manner to engage in live observation, the immediacy of joining video sessions, and diversity of supervisors (i.e., supervisors at other clinics being able to provide telesupervision). Some trainees and supervisors further noted experiencing increased ease in sharing information through electronic means versus physically seeking out a supervisor or resource. There were also benefits in the supervisory relationship: supervisors and trainees reported increase in comfort, while supervisors identified benefits of increased sharing of power and collaboration. Some trainees also noted experiencing more rapport and connection.

Superordinate Theme: Challenges of Telesupervision

Telesupervision challenges included lack of stable technology infrastructure, equipment, and knowledge on how to effectively use technology. The data further supported phone-based supervision inferiority due to inability to observe non-verbal cues and perceived lower quality of supervision. In addition, supervisors and trainees identified negative impacts on supervisory oversight in the following ways: having less accessibility via informal contact and a reduction in observation of trainee clinical skills. In addition, trainees reported initially perceiving less access to supervisors during crisis situations. Furthermore, there were aspects of telesupervision that negatively impacted the supervisory relationship, noting a negative impact on building rapport and connection as it felt more like an efficient and task-driven meeting with less discussion. There was also supervisor-identified increased difficulty in demonstrating skills, observing, or providing needed resources. Problems with engagement in multitasking online that detracted from attending fully to the supervision session were identified. Supervisors reported concerns about being able to fully attend to non-verbal cues. Finally, the lack of system and licensing policy supporting the use of telesupervision was a barrier.

Superordinate Theme: Needs and Feasibility

Supervisors and trainees identified access to technology equipment/platforms, bandwidth of internet, video supervision, and technology knowledge/support as necessary for successful telesupervision. Both identified two components needed for successful telesupervision: (1) the importance of a clear supervision agreement that includes expectations (e.g., trainee and supervisor roles and responsibilities), how to effectively access supervisors, and emergency procedures tailored to modality as opposed to a general supervision contract; and (2) planned logistical infrastructure for electronic sharing of materials, engagement in live/recorded observation, and demonstration or modeling of a skill set. Supervisors further highlighted the importance of intentional rapport building as well as the benefit of engaging in hybrid in-person and telesupervision modalities.

Both trainees and supervisors noted the need for investment in telesupervision engagement, with attitude toward modality having an impact on perceived effectiveness. Trainees highlighted that quality of supervision permeated across modality, with quality and engagement of supervisor being the stable variable, whether in-person or via telesupervision. Finally, data highlight the need for policy (at the program, system, and national level) that enable telesupervision and processes to follow when using telesupervision, while also having policy that supports general provision of supervision including allocated time to effectively supervise (e.g., meet with trainees, live observation, modeling).

Implications and Applications

Consistent with prior literature, findings generally support feasibility and acceptability of telesupervision for psychology training (Bernhard & Camins, 2021; Inman et al., 2019; Jordan & Shearer, 2019; Martin et al., 2017; Schmittel et al., 2021; Tarlow et al., 2020; Thompson et al., 2022), while maintaining essential elements of competency-based supervision regardless of modality (Thompson et al., 2022). Primary and secondary aims were met in comparing perspectives of trainees and supervisors related to telesupervision at the onset of the pandemic, as well as providing insight into lessons learned to further inform future telesupervision practice. The results further support prior literature on the importance of ensuring access to supervisors and experiential learning within telesupervision (Hausman et al., 2021), as well as the need for intentionality in relationship building (Bernhard & Camins, 2021; Hames et al., 2020; Schmittel et al., 2021; Watters & Northey, 2020). Although quantitative analyses revealed few differences between modalities, qualitative data provide more nuanced information on potential benefits (e.g., increased openness or comfort) while highlighting that increased attention to the development of rapport may be particularly important in ensuring telesupervision is not overly transactional. The qualitative data further provide insight into the potential importance of building in informal interactions, both within and beyond the scheduled individual sessions, to enhance the depth of supervision (e.g., impromptu case consultation, observation of trainees in work environment). Furthermore, trainee reports of less knowledge of emergency procedures, is likely indicative of the rapid transition to both telehealth and telesupervision services at the time of the questionnaire and highlights the need for education and practice in the use of technology for both telehealth and telesupervision.

The qualitative and quantitative data collectively support the need for training programs, and the larger institutions in which programs reside, to have policies supporting effective supervision, inclusive of time dedicated to directly observe trainee clinical work and also model expected skill development. Regarding the use of technology, qualitative results support the superiority of video over phone supervision as well as the importance of technology access and knowledge. The results additionally underscore the need for a clear supervision agreement. A supervision agreement is already an important part of competency-based supervision (Falender, 2018), and the data suggest specific components to include in this agreement when engaging in telesupervision inclusive of the following: how to effectively access supervisors, engagement in emergency procedures, engagement in direct observation, sharing of resources and materials, modeling skills, and clarity of expectations.

The results provide insight into identifying other areas of need at the program- and supervisor-level to support robust competency-based supervision regardless of supervision modality (Falender, 2018). For example, the quantitative results suggest that supervisors have a perception of upholding essential elements of supervision (e.g., maintaining regular supervision meetings, providing evaluative feedback, engaging in live observation) at a greater level than what matches endorsed experiences of trainees. Although data were derived from an aggregated, national group of supervisors and trainees, the discrepancy in supervisor and trainee reports regarding the presence of essential elements in supervision provides insight into supervision practice regardless of supervision modality. It further suggests the reported delivery of supervision may not be the experience of the supervision consumer. Thus, there is potential benefit in increasingly examining data from the perspectives of both supervisors and trainees (both within programs and in greater research) to examine cause and subsequent potential solutions (e.g., ongoing clinical supervision trainings, oversight and accountability for essential elements of supervision).

Telesupervision provides an opportunity to increase accessibility and oversight through the ability to work with a greater range of supervisors, ease at which the trainee can access consultation or live observation, and diversification of learning experiences. However, this should not overshadow the quality of supervision interactions. Consistent and attentive engagement in supervision must be maintained to mitigate potential for electronic communication interruptions or temptation to multitask. Furthermore, the manner in which expected competencies are demonstrated or evaluated needs to be altered to fit the modality. Specific recommendations on how to demonstrate required skills and competencies (e.g., administering a psychological test or demonstrating a therapy skill), as well as engage in direct observation (e.g., live observation or recording) must be identified. Further opportunities to leverage telesupervision and technology to enable less intrusive ways of engaging in live observation for both in-person and telehealth care provided by trainees should be explored.

Limitations, Considerations, and Future Directions

In the face of overarching concerns about health and safety, the majority of VA training programs shifted to telesupervision with minimal infrastructure or prior history of engaging in telesupervision. Many programs became completely virtual with variability of on-site presence at the facility, highlighting an important distinction between telesupervision as part of a greater training program as opposed to operating in a completely virtual environment. The forced nature of training programs’ conversion to telesupervision may have influenced responses.

The questionnaire was disseminated when new teleconferencing platforms were being rolled out throughout the VA. With telehealth standardization and increased use since data collection, it is likely that some technology-related concerns and skills have improved over time. Technology is likely to further evolve, and the use of telesupervision and telehealth are slated to remain mainstays, requiring technology-related challenges to continue to be assessed. Although all VA psychology Training Directors were sent links, there may have been bias related to which Training Directors disseminated questionnaires and who decided to complete them. In addition, it is not clear as to what percentage of respondents were engaging primarily in video-based versus telephone-based supervision. Differences in the degree of virtual work, infrastructure, and technology training may have influenced responding. Because of the cross-sectional nature of the data, it is difficult to make direct comparisons between in-person and virtual modalities, particularly across trainee development and telehealth-related skill development. Future explorations of telesupervision may benefit from longitudinal analyses of in-person and telesupervision and the development of trainee competencies over time.

Although the expansion of telesupervision has the potential to create opportunities for lower-resourced training programs (e.g., ability to include remote supervisors, increased rotation opportunities, collaboration across programs), it is also important to recognize the potential to increase inequities. Technology, infrastructure, and electronic materials identified as important require financial resources that are disparate across programs and trainees (Bell et al., 2020). Thoughtful engagement in the utilization of telesupervision includes further identification of how responsible allocation of time and resources can enable increased access versus only expanding opportunities for those programs or trainees that are already heavily resourced.

Findings from the current project suggest future directions for research related to the optimal combination of telesupervision and in-person training that yields greatest benefit in quality of training, individual variables that impact supervision relationships via telesupervision, impact of patient outcomes, and impact of competency building. Future research could also build on lessons learned for telesupervision implementation, while also prompting reflection on areas of potential growth and focus on engagement in supervision, regardless of modality (e.g., consistency of feedback, experiential learning, direct observation, access to supervision). The data further suggest that supervision is complex, whether it is conducted in-person or virtually. Therefore, rather than simply comparing modalities, it is imperative that the presence of essential elements needed for successful training and specific differences in upholding these essential elements across modalities are assessed. Ultimately, telesupervision is likely to remain in the changing landscape of psychology training, with the need to maintain essential elements of effective supervision being critical regardless of modalities and supervision experiences.