Background

The onset of the COVID-19 pandemic necessitated rapid changes within behavioral healthcare organizations, as many shifted to new models of care delivery, such as telehealth, to avoid interruptions to their services. Many organizations were able to do this rapidly, supported by emergency policy changes and the development and implementation of new technologies (Centers for Medicare & Medicaid Services, 2020; Folk et al., 2022; Goldman et al., 2020; Shaver, 2022). As we now enter a “post”-pandemic era, a greater understanding of how to sustain these changes and further improve their implementation is essential. Because organizations had to make such a rapid shift to the use of telehealth at the start of the pandemic, many organizations and their providers did not receive adequate training on the use of telehealth in their practice (Montoya et al., 2022; Perry et al., 2020).

For many organizations, lockdown orders and ongoing loneliness and isolation during the pandemic created significantly higher demand for mental health and substance use care, causing caseloads to increase dramatically (American Psychological Association, 2022). This increased caseload also significantly challenged organizations’ abilities to provide adequate training on the use of telehealth given the lack of training time in providers’ daily schedules as well as the lack of trainers (Esper et al., 2020; Lieneck et al., 2021; Lin et al., 2022). Because of this, there are many providers and organizations with high levels of telehealth utilization who may not feel they have adequate background or training to deliver telehealth services effectively (KFF, 2022; Malliaras et al., 2021; Sugarman et al., 2021; Terry & Buntoro, 2021). At the height of the pandemic, in-depth telehealth training or telehealth training to address special populations or circumstances was often not feasible or a priority; this lack of training now presents a significant challenge to the long-term sustainability of the practice. Additionally, though many clients appreciate its flexibility and accessibility, not all individuals like or are able to use telehealth (Imlach et al., 2020). For many, the opportunity to return to in-person care has been a welcome change, reducing rates of telehealth use in behavioral health organizations, though telehealth use remains quite high, and much of the decline is driven by organizational demands rather than provider concerns (Berry et al., 2022; Lieneck et al., 2021).

Understanding more about the barriers to telehealth use for organizations and the clients they serve and how this knowledge can inform training and workforce development in the future is vital to making telehealth sustainable. Commonly cited reasons why organizations and providers do not use telehealth or have begun to decrease their use of telehealth as the pandemic has eased include, but are not limited to, not knowing how to use the technology, ethical and legal concerns, reimbursement and documentation challenges, lack of technology access among the populations their organization serves, and lack of efficacy data for telehealth versus in-person care (Cortelyou-Ward et al., 2020; Lin et al., 2018; Molfenter et al., 2021; Montoya et al., 2022; Perle et al., 2014; Perry et al., 2020; Spagnolo et al., 2022). Many of these challenges can be addressed through training and technical assistance. Indeed, workforce development and support can help organizations as they navigate these challenges to improve their services.

Given these ongoing challenges, understanding the training needs of behavioral healthcare organizations and providers about using telehealth is vital to encourage the longevity of this practice and promote greater access to telehealth for historically underserved populations. Surveying a large sample of behavioral health professionals (psychologists, counselors, social workers, case managers, administrators, etc.) across settings and geographic locations can help to develop a deeper understanding of whether these factors impact the training needs of behavioral health organizations to continue providing telehealth effectively to their clients. Training needs are expected to be high across all respondents, given the often low levels of training that organizations received at the beginning of the pandemic, but smaller and more rural organizations, among others, may have unique training needs given the client population they serve.

Methods

Respondents

Staff and providers working in behavioral healthcare organizations were invited to complete an online, self-administered survey in REDCap between September and December 2022. This survey was distributed by email to five of the ten Mental Health Technology Transfer Center (MHTTC) service areas, each of which covers a Department of Health and Human Services designated region, as well as the MHTTC Network Coordinating Office’s national listserv. The Substance Abuse and Mental Health Services Administration (SAMHSA)–funded MHTTC Network provides free training and technical assistance to the behavioral health workforce to increase their implementation of evidence-based practices across mental health prevention, treatment, and recovery support. Distribution of the survey was conducted by only five out of ten MHTTCs because of existing connections in the research group, allowing for easier and more efficient distribution. However, the MHTTCs have significant reach outside of their service areas and the Network Coordinating Office’s listserv is national, enabling national distribution of the survey. Email requests to complete the survey reached providers across the US states and territories, including both urban and rural areas. Individuals who are on MHTTC survey distribution lists are typically those who have participated in trainings in the past, are interested in receiving updates about ongoing training efforts, or have otherwise been involved in training efforts, either themselves or via their organization. Anyone who was on a participating MHTTC’s email distribution list at the time of distribution would have received information about the survey and had the opportunity to complete the survey if desired. Because of the survey distribution method used, the exact response rate is unknown. Respondents did not receive any incentive to complete the survey. This study was reviewed and approved by the University of Wisconsin — Madison Institutional Review Board (protocol #: 2020-0551). This analysis includes a total of 365 respondents who completed the quantitative survey, 122 of whom also responded to free response questions.

Measures

General information about respondents was gathered, including location of organization (state and zip code), organization name, type of community the organization is located in, whether the organization serves primarily tribal communities, type of organization they work for (health system, FQHC, specialty behavioral health provider, private practice), and their role in the organization (administrator, clinician, medical provider, etc.). To assess current levels of telehealth service provision, respondents indicated whether their organization provided a variety of different services (screenings, individual therapy, group therapy, etc.) in-person, via telephone, and/or via video. To assess training needs, respondents were asked to rate their level of agreement, on a scale from 1 (strongly disagree) to 5 (strongly agree), with 12 telehealth training topic needs (e.g., how telehealth can be used to reach more diverse populations, use of telehealth in treatment of adults, delivering high-fidelity evidence-based psychotherapy [e.g., cognitive behavioral therapy] via telehealth). At the end of the survey, respondents were asked a free response question regarding what other training is needed to allow their organization to use telehealth services more effectively. Responses to all questions were optional, and respondents had the ability to skip any question. Questions and response options are presented in Table 1.

Table 1 Survey questions and scale
Full size table

Analysis

Frequencies for demographic characteristics were calculated using SPSS. Some variables were recategorized to create fewer categories for analysis. Type of community was recategorized to include only rural and urban using United States Department of Agriculture (USDA) rural-urban commuting area codes, which give each zip code a rating from 1 to 10 (United States Department of Agriculture Economic Research Service, 2023). Based on Health Resources and Services Administration (HRSA) rural health definitions, any zip code given a 4–10 was recategorized as rural while the rest were recategorized as urban (Health Resources and Services Administration, 2022). Provider type was recategorized into administrator (administrator, clinical supervisor), clinician (counselor, social worker, psychologist), medical provider (physician/prescriber, nurse), and other (case worker, peer support worker, other). To assess levels of telehealth service provision, respondents who indicated that they provided at least one service via telephone or video were categorized as utilizing telehealth.

Means and standard deviations were calculated for training needs questions. To assess differences in responses to training needs according to demographic information, linear mixed models were run using the glmmTMB() from the glmmTMB package in R. Qualitative analysis of the free response question regarding training needs was conducted using NVivo 2020 and was completed by one research staff member (KF). Content analysis was used to identify the most common emergent themes in responses (reimbursement, adapting therapeutic techniques, etc.). Some responses covered multiple themes and were coded appropriately into several categories.

Results

Respondent Characteristics

A total of 480 responses were received. Respondents who completed only the demographics questions were removed (n = 115). Of the remaining 365 survey responses received, 262 were complete while 103 were incomplete. Incomplete responses were included in analysis, causing sample size for each question to vary. Respondents were from 43 states and Puerto Rico, with the largest numbers of responses coming from Illinois, Michigan, and Wisconsin. Most respondents were in urban areas (n = 244, 66.8%) and did not serve primarily tribal or indigenous communities (n = 344, 94.5%). Respondents worked in a variety of types of healthcare organizations, including health systems (n = 67, 18.8%), FQHCs (n = 11, 3.1%), specialty behavioral health providers with 6+ sites (n = 73, 20.5%), specialty behavioral health providers with 2–5 sites (n = 83, 23.3%), stand-alone specialty behavioral health providers (n = 54, 15.2%), and private practice (n = 68, 19.1%). The majority of respondents were clinicians (n = 253, 69.3%), followed by administrators (n = 60, 16.4%) and medical providers (n = 50, 13.7%). 85.2% of respondents indicated that their organization provided at least one behavioral health service via telehealth (phone and/or video).

Reported Training Needs

For each question regarding telehealth training and workforce development needs, respondents endorsed similar levels of need, with every question having a mean response between 3.5 and 4.0. Reported agreement for each question can be seen in Table 2. Three items had ratings above 3.9: how to increase digital literacy for patients, how telehealth can be used to reach more diverse populations, and how to integrate recovery apps into the care delivery process.

Table 2 Reported training and workforce development needs to continue providing telehealth
Full size table

Perceived training needs did not vary by organization location, with the exception that respondents located in an urban environment were significantly more likely to report needing training on how to increase digital literacy for patients (M = 4.07) than those in a rural environment (M = 3.80), p = .045. There were some variations in response by type of organization, but there was no systematic way in which this occurred across the six categories of organization types. Additionally, there were no significant differences in reported training needs according to respondent role. Conversely, on nearly every training needs question, there was a significant difference in level of agreement according to the number of telehealth services provided at the organization. In general, providers from organizations that provided more telehealth services reported more need for training.

Qualitative Analysis of Training Needs

When asked “What other training do you think is needed to allow your organization to more effectively use telehealth in the delivery of services?”, the 62 responses fell into four categories: adaptations to existing treatment delivery for specific populations or disorders, technology access, ethics and confidentiality, and concerns around workflow and administrative challenges. Example responses are included in Tables 3, 4, and 5.

Table 3 Examples of responses about treatment adaptations
Full size table
Table 4 Examples of responses about technology access
Full size table
Table 5 Examples of responses about administrative concerns
Full size table

Treatment Adaptations

Of the free text responses, the most frequently cited training need was how to adapt telehealth for use with specific populations or types of therapy. Concerns over the use of telehealth with children, adolescents, and families were frequently discussed, with respondents indicating that it is difficult to keep the attention of younger clients in a virtual environment and that typical forms of treatment used with younger populations are difficult to deliver via telehealth. Additionally, some respondents indicated that individuals mandated to treatment were difficult to engage in virtual treatment, and additional training in how to maintain engagement with telehealth could be beneficial.

Some respondents also indicated concerns over knowing when telehealth use is contraindicated for their treatment population. Providers expressed concern that telehealth could increase isolation for individuals using substances or with certain common mental health disorders, such as anxiety and depression, and that this may make treatment less effective for such individuals. Providers reported particular concerns regarding assessing suicide risk and delivering crisis interventions in a virtual environment.

Another requested area of training was around how telehealth could be better utilized to address issues of diversity, equity, and inclusion within behavioral health organizations and the clients they serve. Respondents indicated concerns regarding inequitable access to the technology required to utilize telehealth services and in the knowledge required to do so effectively. Though providers were generally supportive of the future use of telehealth within their organizations, additional training on how to more appropriately disseminate these technologies within their communities is necessary.

Lastly, several respondents noted a desire to receive additional training on delivering certain types of treatment via telehealth, including acceptance and commitment therapy (ACT), motivational interviewing (MI), creative therapies, and more generally, the use of telehealth outside of traditional individual therapy sessions. Providers expressed concern over the appropriateness of using telehealth for the administration of these types of therapy and a desire for additional support via training around this topic in order to feel more competent in their provision of these services.

Technology Access

Other frequently cited training needs were around methods to increase access to technology for populations that lack such access, such as through the provision of computers, phones, and internet or data plans. This was a particular concern for clients in rural areas where these services can be lacking. Providers are eager to find ways to enhance access to telehealth services for those with limited connection to technology, which may also be related to respondents’ concerns around the use of telehealth to address issues of diversity, equity, and inclusion.

Ethics and Confidentiality

Another major area of needed training cited by respondents was ensuring compliance with ethical standards and confidentiality guidelines while providing treatment in a virtual environment. Respondents expressed a desire to receive additional training on HIPAA, the best platforms for delivering secure virtual behavioral health services, how to communicate with clients outside of appointments in a secure manner, and how to ensure they still meet ethical standards of care while delivering service virtually.

Administrative Concerns

A final frequently cited area for additional training centered around administrative concerns, including how to effectively and efficiently provide both telehealth and in-person care, how to ensure adequate reimbursement for and proper documentation of telehealth care, and methods to integrate telehealth use with existing electronic health record (EHR) systems.

Related to how to effectively offer both in-person and virtual care to clients, several respondents indicated additional training is needed on how to balance both types of care and offer this choice to clients. Providers are seeking additional training on how to schedule, organize, and maintain both types of services as they navigate this transition. Related to reimbursement and documentation for telehealth provision, providers seek training on how to work with insurance companies to lobby for adequate reimbursement for telehealth services. Additionally, because documentation needs and integration with EHR systems are related to reimbursement concerns, many respondents also desired further training on how to better use their existing EHRs to document their use of telehealth.

Discussion

With the rapid onset of the COVID-19 pandemic, many healthcare organizations were pushed into new methods of care delivery with minimal time to prepare or refine these techniques. While organizations rose to the challenge out of necessity, there is now time to reassess the use of telehealth and evaluate what healthcare organizations need to continue to deliver telehealth care effectively and efficiently.

This cross-sectional survey of a large national sample of behavioral health providers identified several key areas for additional training and support, including telehealth use with specific populations, methods to increase technology access in less-connected areas, and administrative challenges, including documentation, billing, and the integration of telehealth with existing care delivery systems. In general, there are significant training needs across provider type, organization type, and organization location. Given the enduring nature of the pandemic and the high demand for mental healthcare, there was little time for healthcare organizations to receive adequate training on the implementation or use of telehealth. This may be causing organizations to now desire additional training across the board. Although organizations may have been meeting the demand for telehealth services with little training, it is clear they feel a need for additional support as they maintain, and in some cases, increase the use of telehealth (Lo et al., 2022). As organizations continue to provide telehealth services, they need to offer or access training on key telehealth-related topics and protect the time for providers to receive this training. Unless organizations prioritize training opportunities for their clinicians, it is unlikely that clinicians will be able to receive the training they are so interested in because of the significant demands on their time.

The current study found a general interest in additional training but also specific areas where the need is greatest. Respondents generally indicated a strong interest in continuing to use telehealth with their clients and expressed that clients appreciated telehealth and found it useful; however, they had concerns about its use with certain populations and its integration into a system that was typically not designed for the use of telehealth. Though most treatment modalities can be delivered in a virtual environment, many providers need support on how to best deliver these treatments in a virtual environment. For example, virtually providing treatment in a group setting can create additional challenges over working one-on-one with individuals because of the lack of direct contact; thus, additional training may be required to ensure appropriate service delivery or fidelity to the treatment. Given these concerns, providers are seeking a greater level of understanding regarding which clients and types of treatments are appropriate for telehealth and which should only be offered in person. This point is supported by the qualitative and quantitative results of this study, with respondents reinforcing that additional support is needed to better understand how to adapt typical treatments to a virtual environment. As telehealth enters a new era, the focus of training needs to shift from general telehealth delivery to more tailored and specific topics, such as the provision of specific evidence-based psychotherapy via telehealth (e.g., cognitive behavioral therapy); best practices for working with children and families in a virtual environment; and how to promote engagement with virtual care. At this point, most providers likely have sufficient foundational knowledge of how to use telehealth but need additional support with more specific areas of care. Focusing training on these areas is vital to allow providers to feel more confident in their ability to deliver effective care via telehealth, particularly as it is now clear that many clients remain interested in its use long-term, at least for some types of care and in particular circumstances (Predmore et al., 2021; Rasmussen et al., 2022).

Additionally, a large concern throughout the pandemic has been the digital divide, which refers to both a lack of technology access for clients from underserved populations, including those in rural areas, older adults, lower-income communities, and individuals with limited English proficiency, and a lack of understanding and ability to utilize technology, even when it is available (Choxi et al., 2022; Clare, 2021; Cortelyou-Ward et al., 2020; Gallegos-Rejas et al., 2023; Rodriguez et al., 2021). Though telehealth has helped to increase access to healthcare for many populations, the digital divide causes challenges for many individuals (Choxi et al., 2022; Clare, 2021; Cortelyou-Ward et al., 2020). As the use of telehealth continues, many providers are seeking a deeper understanding of how to mitigate these factors to promote greater access to services, particularly for underserved populations (Gallegos-Rejas et al., 2023).

In light of this, it is surprising that there were no significant differences in identified training needs according to geographic location, type of organization, provider type, or other demographic characteristics. In general, there has been a strong belief that certain organizational and/or client population characteristics make telehealth use more difficult and less likely to succeed. While this may still be true, our results indicate that high training needs exist regardless of organization type or client population served. While telehealth is largely viewed as a helpful way to reach underserved populations, both qualitative and quantitative results indicated that more training on how to serve diverse populations via telehealth and how to increase digital literacy for clients is needed to allow providers to do so competently. While telehealth may help address some equity issues, it can also introduce new ones, and providers are seeking support as they navigate these ongoing challenges and build a stronger foundation to serve a more diverse client base effectively. Clinicians in all areas indicated that a major barrier to telehealth use was client technology access and understanding while simultaneously recognizing that telehealth use eliminates major barriers to care for some individuals, like transportation and taking time off work. Telehealth represents a significant opportunity to increase access to care for underserved populations, but there are also major barriers for these same populations in accessing this care. Increasing clinician comfort and expertise with these technologies via training about the different types of technology and their use may allow providers to better work with clients who struggle with technology, as they will feel more comfortable explaining and troubleshooting technology challenges.

It is also vital to recognize that some areas of concern identified by respondents are not ones that can be easily addressed through training. While it may be feasible to train providers on adapting certain types of therapy to delivery via telehealth or strategies to engage particular populations more effectively, training alone is not the solution to concerns like documentation, reimbursement, or integrating telehealth into organizational workflows. For example, though providers may prefer to continue providing telehealth services, changes in reimbursement will likely result in fewer telehealth services being offered. Additionally, because workflows and EHR systems were typically not designed with telehealth in mind, in most cases, it is much harder to document the provision of telehealth than traditional in-person care. In sum, training cannot mitigate these concerns identified by respondents, particularly without larger shifts in the healthcare landscape (e.g., permanent rule changes allowing billing for telehealth, upgrading of EHR systems to better document telehealth). For now, it is important to carefully allocate training resources and time to areas that can be addressed by training within already resource-constrained organizations.

Limitations

This study had a fairly limited survey response yield, though the exact response rate is unknown, given the manner of distribution. Because surveys were distributed via newsletter and email lists, and it is possible that they were passed on to others beyond those original lists, it is impossible to know exactly how many people received an invitation to complete the survey. However, this method was also a strength of the study as it allowed for an efficient, diverse geographic distribution. Other limitations include that the sample may be comprised of respondents who are more likely to be using telehealth and, therefore, more likely to respond to a survey of this nature. Thus, their opinions may not be representative of the general consensus within the behavioral health field.

Conclusion

Based on this study’s qualitative and quantitative results, it seems clear that for most behavioral health organizations, telehealth care is here to stay, but changes are needed to allow it to continue effectively. Because of the rapid transition from in-person care to telehealth in the wake of the COVID-19 pandemic, many organizations were never able to establish proper procedures for providing telehealth care and simply did their best, given the circumstances. Organizations and providers showed incredible flexibility and resilience but may now be better equipped to adjust their provision of care to the use of telehealth and ensure that the best quality care is being provided. Therefore, identifying needed areas of training and workforce development, such as how to adapt particular types of therapy to telehealth and how to increase technology access for underserved populations, will allow providers to best adapt their current methods of care to better serve their clients, particularly those who have historically struggled to access traditional forms of treatment.