Background

The novel coronavirus disease (2019-nCoV/COVID-19) drastically impacted the context in which mental health services are provided. Many community mental health centers (CMHCs) rapidly implemented telehealth services in order to comply with ‘stay at home’ restrictions set across the country [1]. Mental health providers were subsequently tasked with transitioning their services, including the delivery of evidence-based practices (EBPs), from in-person treatment to telehealth. Although evidence for the effectiveness of telehealth services (i.e., services provided via phone and/or video platforms) has existed for over 20 years, numerous barriers have delayed its widespread use until now [2,3,4]. In response to COVID-19, a nationwide telehealth implementation effort occurred in only a matter of weeks. This rapid telehealth implementation [5] directly contrasts with the 17-year research-to-practice pipeline [6] and is even more rapid than the 3-year length of time EBPs are integrated into real-world settings with the aid of an implementation team [7]. In accordance with the STROBE checklist of items included in reports of observational studies (Additional file 1), this paper reports on the impact of the COVID-19 pandemic and rapid implementation of telehealth services from the perspectives of CMHC providers in one Midwest state and discusses implications regarding the quality and sustainment of rapid implementation efforts.

Method

Participants and setting

Data were collected as part of an ongoing service contract between the UC San Diego and the statewide behavioral health services system in one Midwest state to engage the state and their CMHCs in the Leadership and Organizational Change for Implementation strategy [8, 9]. Participants were behavioral health services providers (n = 93, response rate = 77%) from 6 CMHCs that were also contracted to improve upon provider delivery of combined motivational enhancement therapy and cognitive behavioral therapy (MET/CBT) and other EBPs. In response to COVID-19, CMHCs implemented telehealth services, typically including both video conference and telephone modes of delivery. Providers identified mostly as female (n = 77; 84.6%), non-Hispanic (n = 81; 89%), and White (n = 79; 86.8%) and were 41 years old on average (sd = 14.8 years). Most providers had completed master’s level education (n = 65; 71.4%) and identified social work as their primary discipline (n = 44; 48.4%). Providers reported spending the greatest percentage of their work time in psychotherapy and/or counseling (\( \overline{x} \)=44.1%) and reported an average caseload of 52.1 (sd = 39.5) clients per month. See Table 1 for provider demographics.

Table 1 Provider demographics
Full size table

Procedures and measures

CMHC providers completed surveys via the Qualtrics web-based platform that included measures targeting their reactions to the COVID-19 outbreak and transition to providing services via telehealth. The measures that were utilized are described below. Unless otherwise noted, participants responded to each measure using a 5-point response scale ranging from 0 = “Strongly Disagree” to 4 = “Strongly Agree.” When appropriate, the internal consistency of each measure was assessed for this sample using Cronbach’s α; results are included in Tables 2 and 4.

Table 2 Survey scale descriptive statistics
Full size table

Perceptions of personal risk

This 9-item measure was adapted from Wu et al.’s measure assessing perceptions of personal risk around SARS [10]. Items were adapted to assess participants’ perceived risk of being exposed to, and getting infected with, COVID-19.

COVID-19 rumination

This 3-item measure was developed by LeNoble and colleagues to assess participants’ rumination about COVID-19’s interference with their work [11].

Work changes due to COVID-19

Based on feedback from providers and their leaders, three distinct types of work changes were identified. A single item was developed to capture each type of work change through an iterative process of item generation, discussion, and refinement until consensus on item wording was achieved. The resulting three items assessed changes in tasks, settings, and teams that mental health providers experienced following the COVID-19 outbreak. Participants responded using a 5-point response scale ranging from 0 = “Not at all” to 4 = “Very great extent.” Because the three items were developed to capture three distinct types of work changes and were not intended to represent a single underlying construct, internal consistency was not assessed [12].

Burnout

The Copenhagen burnout inventory assesses participants’ emotional exhaustion and work-related frustration [13]. Participants responded to three items targeting work-related burnout within the past 2 weeks using a 5-point scale ranging from 0 = “Never” to 4 = “Always.”

Perceived organizational support

We used three items from Eisenberger et al. to assess perceptions of the helpfulness, care, and concern of the agency for providers [14].

Telehealth self-efficacy

This 4-item measure adapted from a measure developed by Lau and Brookman-Frazee assessed participant’s confidence, knowledge, understanding, and preparation to deliver therapy via telehealth [15, 16].

Collective efficacy

This 3-item measure adapted from Jex and Bliese assessed efficacy beliefs targeting the agency’s transition to telehealth [17].

Telehealth beliefs

This 5-item measure adapted from the University of Michigan’s Behavioral health Workforce Research Center assessed whether providers had a positive view of telehealth [18].

Transition to telehealth

Seven items evaluating the transition to telehealth were developed by the study authors through an iterative process of item generation, discussion, and refinement until consensus on item wording was achieved. These items measured the extent to which different aspects of treatment were better or worse when serving clients via telehealth as opposed to in-person treatment; see Table 4 for the individual items. Participants responded using a 5-point response scale ranging from 0 = “Significantly worse with telehealth relative to in-person” to 4 = “Significantly better with telehealth relative to in-person.”

Open-ended survey questions

Participants responded to two open-ended survey items. The first asked participants about the major impacts of COVID-19 on their work, and the second asked about the major impacts of COVID-19 on the use of a specific EBP (MET/CBT) and other EBPs in general.

Analysis

Participant responses were aggregated across all items to obtain an overall scale mean with the exception of the work changes and the transition to telehealth items, which were analyzed individually. Descriptive statistics of quantitative measures were assessed to explore providers’ responses and/or reactions to the COVID-19 outbreak, and subsequent transition to providing services via telehealth. Potential between-agency differences in provider responses were explored using univariate analysis of variance (UNIANOVA). Provider responses to open-ended survey questions were analyzed using a template organizing style of interpretation. Specifically, responses were first reviewed by authors (MS, KR, and KC) to gain familiarity with the content and to isolate broad themes. Text was then sorted and organized in accordance with broad themes, and new themes were generated when appropriate. All authors held meetings to review “chunks” [19] of text and develop summaries of findings.

Results

Quantitative items

See Tables 2, 3, and 4 for survey item and scale descriptive statistics. On average, provider item and scale scores were lowest on COVID-19 rumination (\( \overline{x} \) = 1.10) and perceptions of personal risk (\( \overline{x} \) = 1.41). Provider item and scale scores were greatest on the telehealth beliefs scale (\( \overline{x} \) = 2.89) indicating generally favorable beliefs and attitudes about telehealth, followed by telehealth self-efficacy (\( \overline{x} \) = 2.68) and perceived organizational support (\( \overline{x} \)=2.66). With regard to the work changes (Table 2), the highest scores were for changes in the work setting (\( \overline{x} \)=3.66), followed by changes in the work tasks (\( \overline{x} \)=2.91). Fewer changes were reported for the work team (\( \overline{x} \) = 2.11). With regard to the questions evaluating the effects of telehealth (Table 4), providers reported the largest benefits for scheduling (\( \overline{x} \) = 3.12), and the biggest challenge with patient/client focus (\( \overline{x} \) = 2.38). Results from UNIANOVA indicated no significant differences between agencies in any of the survey items and/or scale scores (see Table 2).

Table 3 Work changes due to COVID-19 descriptive statistics
Full size table
Table 4 Transition to telehealth descriptive statistics
Full size table

Qualitative items

Major impacts of COVID-19 on work

Analysis of open-ended item responses provided by 89 unique participants identified three major themes (see Table 5). One centered on the impact that the transition to telehealth had on interactions with patients/clients. Providers described technological barriers to high quality interactions identifying challenges such as “blocked cell number,” “caseload lives in rural areas…not all kids have access to internet or stable internet,” and “some clients do not have technological capacity for video conferencing.” Some providers commented on the challenge of developing/maintaining rapport through the transition to telehealth. For example, providers reported “not being able to build rapport with new clients” and that “not being able to provide therapy in person and be able to read client’s body language has been the major impact.” Some providers reported beliefs that telehealth facilitated improvements in communication with clients. For example, providers stated that “it has been more enjoyable regarding relationships with clients…due to the less formal atmosphere as clients are more comfortable in their homes,” and that they experienced “stronger communication” with clients.

Table 5 Provider responses and themes regarding major impacts of COVID-19 on work
Full size table

Another theme centered on the changes in provider expectations regarding productivity. Some providers commented on reduced productivity due to this transition, stating “my productivity has dropped…because I have been having issues coordinating with some families” and “I used to have very high guardian engagement…this has decreased; guardians have limited to no access to technology at times.” Some providers commented on increased demands like “increase in documentation required,” “increased pressure regarding productivity and revenue,” and “more new changes in documentation and more paperwork without receiving productivity for increased time spent.”

A third theme centered on challenges maintaining a work-life balance. For example, providers stated that “work-life balance has been disrupted as I have difficulty separating myself from work” and “lack of home life/work balance…time management is difficult.” Providers also reported challenges with working from home while caring for their children, stating “working from home while taking care of my child makes it difficult to do my best work with clients” and “it can be difficult to work from home with children.”

Major impacts of COVID-19 on EBP delivery

When asked about the major impacts of COVID-19 on delivery of EBPs, three themes emerged based on responses from 86 unique participants (see Table 6). One centered on the challenge delivering certain therapies via telehealth wherein providers stated “barriers to implementing services like play therapy are somewhat dependent on the setting,” “prize draws and drug screens are difficult to do,” “lack of client ability to access the worksheets or use a video format,” and that they “haven’t tried [Eye Movement Desensitization and Reprocessing]-bilateral stimulation” via telehealth.

Table 6 Provider responses and themes regarding major impacts of COVID-19 on implementation of evidence-based practices (EBPs)
Full size table

A second theme centered on the potential limitations to confidentiality and/or lack of privacy when providing treatment with telehealth. Providers reported challenges engaging clients in EBP for the treatment of trauma because some clients are less willing to discuss traumas due to limited confidentiality/privacy. Some providers stated “many clients now have less privacy at home—may have partners/kids around;” other providers reported discomfort processing trauma without being able to see how their clients are responding stating that “I have not been able to work with some of my patients on healing their trauma…I am uncomfortable due to not being able to see if they are upset, being triggered, etc.”

The final theme that emerged centered on the general challenge of engaging children in treatment via telehealth. Providers stated “it is difficult to teach my students over the video sessions at times depending on the subject we are discussing and distractions,” “some parents don’t hold their children accountable for doing video sessions,” “parents prefer I work with their children face-to-face,” and that “younger kids often engage better face to face.”

Discussion

This study investigated major impacts of COVID-19 and the rapid implementation of telehealth services on CMHC providers’ work and EBP delivery. Consistent with a recently proposed definition of rapid implementation, telehealth services were provided to those in need, with speed and efficiency, through a pragmatic reconceptualization of rigor [5]. CMHCs and representative providers utilized various modes of telehealth delivery including both video conference (e.g., via Zoom, Skype, Cisco Webex) and telephone. For clients unable to access videoconferencing sessions, providers typically reported using telephone. Providers viewed CMHC transitions to telehealth positively and reported confidence in their abilities to deliver services via telehealth. Notably, providers held generally favorable views of telehealth. Provider acceptance of telehealth services has been found to play a primary role in the implementation and sustainment of telehealth [20], and lack of provider acceptance is noted as the greatest barrier to widespread implementation of telehealth to date [21]. This suggests it may be fruitful to explore how significant circumstances that necessitate implementation impact attitudes toward implementation and innovation adoption.

In order to support rapid implementation, collaboration between researchers, funders, health systems workers, practitioners, and community partners toward a common cause is necessary [5]. Implementation efforts are most successful when propelled by the alignment of support across system and organizational contexts [22]. Historically, telehealth has been difficult to implement, scale-up, and sustain [3, 23] often due to lack of funding and/or policy support at the system-level [24, 25]. In the context of COVID-19, restrictions around the privacy of patient health information (i.e., HIPAA) and Medicaid billing requirements were adjusted, which ultimately aided the rapid transition to telehealth [26,27,28,29]. The sustained use of telehealth will likely be a function of continued billing capabilities and policies to facilitate sustainment.

To maximize the success of rapid implementation, the needs of a range of stakeholders, focusing on time-pressured, clinically relevant questions, must be considered [5]. Collecting and providing timely information of value to stakeholders (i.e., practitioners, decision-makers, and policy makers) can guide specific actions to support rapid implementation. In this study, results suggested that although providers perceived both therapeutic relationships and clients’ willingness to schedule telehealth sessions as somewhat better than in-person services, some providers reported difficulty maintaining engagement and using EBPs via telehealth. Providers also reported experiencing stress related to billing, documentation, and productivity demands while adapting to work changes and maintaining work-life balance. To sustain rapid implementation of telehealth services, it will be essential for CMHC leaders to consider this information and identify specific actions and supports that are needed such that providers can effectively engage with clients and deliver EBPs via telehealth [30, 31].

This study lends important insight into provider experiences with telehealth implementation and EBP delivery in the context of the COVID-19 outbreak. There are, however, several limitations. Due to the novel nature of this outbreak and the related work changes, some measures were created or adapted for this study and do not yet have published psychometrics, making cross-study comparison challenging. Also, this study did not include client- or treatment-level information, such as client perspectives on telehealth, nor did it include the perspectives of other relevant stakeholders, such as CMHC or system-level leadership. Additionally, though not directly assessed in this study, it is very likely that pandemic-related stressors and the associated impacts on cognitive functioning could have impacted the quality of the rapid implementation to telehealth. Finally, it is unclear how expeditious implementation will impact the sustainability of telehealth; moving too quickly through implementation without adequate planning may result in the omission of key implementation activities, and ultimately non-sustainment [32]. Future research should focus on the extent to which the rapid implementation of telehealth in the context of COVID-19 results in sustainment.

Conclusion

In the context of the COVID-19 outbreak, CMHC providers continued to provide services and deliver EBPs through telehealth. Persistence was needed to connect and engage with clients, and creativity was crucial for continued EBP delivery. Though rapid telehealth implementation was supported by the relaxing of national policies, it remains unclear whether telehealth will be sustained moving forward, and, if so, what support providers need to do so effectively.