Background

Telemental health (TMH) refers to delivering mental health care remotely via telecommunications technology (as opposed to face-to-face) [1]. Adoption of TMH has expanded during the COVID-19 pandemic to allow services to continue to offer mental health support despite social distancing restrictions. This shift in care delivery was often conducted rapidly as part of the emergency response to the pandemic, in which services had to adapt their existing face-to-face models of treatment to include remote forms of care [2].

Some benefits to delivering mental health support in this way have been identified, for example, increasing access for service users who live remotely, have difficulty travelling, or find mental health care settings stigmatising or intimidating, and greater convenience for some service users [3, 4]. However, there are also some challenges associated with this approach, for example, some service users may not have access to technology, internet connectivity, or a private space to use during TMH care, whilst others have identified challenges in developing and maintaining a therapeutic relationship [5, 6]. A recent systematic review also identified that TMH may not be suitable for all types of therapy, for example, exposure therapy or when treating trauma [2]. This review also identified challenges in delivering TMH care to certain populations, for example, children and service users with learning difficulties or severe anxiety [2].

The rapid switch to TMH during the pandemic has resulted in great variations in how and to what extent TMH has been adopted and sustained, across different geographical locations and services [7]. Due to the rapid nature of the implementation of TMH, staff have raised concerns around a lack of appropriate training to be able to conduct remote mental health care effectively and safely [6, 8]. Nonetheless, both staff and service users express interest in incorporating TMH in routine care beyond the pandemic, increasing service user and staff choice and convenience. To move beyond the piecemeal pandemic implementation of TMH to strategies for incorporating it in routine care in the future, we need a greater understanding of the best approaches to introducing and sustaining it in contexts where it is potentially helpful.

Implementation research can be defined as “the scientific inquiry into questions concerning implementation—the act of carrying an intention into effect, which in health research can be policies, programmes, or individual practices” [9]. This review will focus on implementation strategies, which represent “the ‘how to’ component of changing healthcare practice” [10] and are key in determining the success of an intervention [10]. Implementation strategies are the methods by which changes to existing healthcare practice are introduced and sustained in practice. There is a need to establish what works for whom in TMH and identify the key mechanisms for acceptable, effective, and efficient integration of this intervention into routine care. Studies using implementation science methods are especially focused on meeting this need, and therefore efforts to synthesise available information on implementation efforts form a key part of bridging the gap between what is known about effective treatment and what can feasibly be utilised within mental health settings [10].

The current review aimed to synthesise evidence on how best to implement and sustain TMH during the recovery from the pandemic and beyond, integrating it across the mental health system in a flexible and sustainable way that both maximises its potential in everyday practice and allows a response to be rapidly mobilised to any future emergency.

This review specifically aimed to:

  1. 1)

    identify and describe strategies that have been used to improve the implementation of TMH approaches;

  2. 2)

    synthesise evidence on how these strategies influence implementation outcomes.

Methods

This systematic review is reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidance [11]. The review was prospectively registered on PROSPERO (CRD42021266245).

Inclusion criteria

We included studies meeting the following criteria:

  • Participants: Staff who worked within any specialist mental health service, including inpatient, outpatient, community and crisis care settings; people of any age who received organised mental health care in specialist mental health services, or their family members or carers.

Studies conducted in primary care or standalone psychotherapy service settings, or that involved service users with substance misuse, neuropsychiatry/neurology or dementia diagnoses were excluded.

  • Interventions: Pre-planned strategies only (strategies which comprise deliberate and purposeful efforts, planned in advance, to support the effective and sustained implementation of TMH [10]). We included various modalities of TMH, including video calls, telephone calls, text messaging platforms and hybrid approaches combining different platforms, or a combination of remote and face-to-face care. TMH care must have included spoken or written communication carried out remotely between mental health professionals or between mental health staff and patients, service users, family members, unpaid carers, or peer supporters.

Studies where the intervention was only delivered to selected participants recruited for the purpose of the study, as opposed to being rolled out across an existing service, were excluded.

  • Outcomes: At least one of the outcomes from Proctor and colleagues’ [12] taxonomy of implementation outcomes, defined for the purposes of this review as the effects of deliberate and purposive actions to implement TMH (see Evidence synthesis for more detail), had to be reported:

    • Acceptability (to service users or staff)

    • Adoption (including any individual differences in those reached or not reached)

    • Appropriateness

    • Feasibility, e.g. actual fit, suitability for use

    • Fidelity

    • Cost and cost effectiveness (of implementation support intervention or strategy)

    • Penetration, e.g. spread, level of institutionalisation

    • Sustainability

Studies with or without a comparator were included. Studies were excluded which reported findings about the extent of implementation of a TMH programme, or described barriers to or facilitators of the implementation of TMH, but did not describe and evaluate an explicit pre-planned strategy designed to achieve more widespread, effective and/or sustained implementation of TMH, or did not report a relevant outcome according to Proctor’s taxonomy.

  • Study designs: There were no restrictions based on study design or language of papers.

Further exclusion criteria

We also excluded conference abstracts, review articles, editorials and opinion pieces. Papers were excluded if they were published before January 2010 as earlier studies may be less relevant due to changes in both the availability of and familiarity with technology that can be used to support telehealth.

Search strategy

The search strategy included the following:

  1. 1)

    Five academic databases (PubMed, EMBASE, PsycINFO, CINAHL and Web of Science) were searched from January 2010 to July 2021. The search strategy used a combination of keyword and subject heading searches relating to mental illness, remote working and implementation.

  2. 2)

    Preprint servers (medRxiv, PsyArXiv, Wellcome Open Research and JMIR Preprints) were searched (October 2021).

  3. 3)

    Forward citation searching using Web of Science and backward citation searching of reference lists of included studies.

The full search strategy is provided in Additional file 1: Appendix 1.

Screening

All references were de-duplicated in Endnote X9 [13] and then imported into Rayyan [14] for title and abstract screening. Title and abstract screening was conducted by five reviewers (RA, PB, SS, ER, MT), with 100% included references and 25% of excluded references checked by another member of the research team (PB, MY, MW) to ensure inclusion criteria had been applied correctly. Full text screening was conducted by four reviewers (PB, RA, ET, NL), with 100% included references and 25% of excluded references checked by another member of the research team to assess if they agreed with the original decision. All disagreements were resolved by discussion with a third reviewer.

Data extraction

All included references were imported into EPPI-Reviewer 4.0 [15] for data extraction. A data extraction form was created and piloted on a small number of included studies by three reviewers (RA, NSJ, PB), before data for the remaining studies was extracted by three reviewers (NL, ER, ET). All data extraction was checked by another member of the research team (PB, NSJ, RA, JP).

Details on the service setting, study design, characteristics of the clinical population, characteristics of the staff and TMH modalities used were extracted from each study. Details of the implementation strategy used, categorised according to the ERIC compilation of implementation strategies [16] (outlined in more detail below) and implementation outcomes (categorised according to Proctor’s taxonomy [12]) were also extracted.

Quality appraisal

Quality appraisal was conducted using the Mixed Methods Appraisal Tool (MMAT) [17] for all primary research studies which aimed to answer a research question. MMAT recommendations are to report the percentage of criteria met for each study (e.g. 20% represents 1/5 criteria met, while 80% represents 4/5 criteria met). Studies were considered of low quality if they met 20% or less, medium quality if they met 40–60% of the criteria, and high quality if they met at least 80% of the criteria. The AACODS (Authority, accuracy, coverage, objectivity, date, significance) tool for descriptive studies [18] was used to appraise the quality of studies which did not meet the MMAT criteria of having a clear research question. We describe these studies as “descriptive studies”. Quality assessment was carried out by one member of the research team (NL, ER, ET, or RA) and checked by another reviewer (RA, PB, or NVSJ). Due to the relatively small number of papers included in this review, the results of quality assessment were not used to determine eligibility for inclusion, although quality assessment along with study design were considered during interpretation of findings.

Evidence synthesis

We conducted a framework synthesis [19] to consolidate findings from the included studies, a method which “allows themes or concepts identified a priori to be specified as coding categories from the outset” (p1). We used two pre-established implementation science frameworks as the basis for our framework synthesis, and mapped data from included papers according to headings from these two frameworks. Firstly, we used the overarching categories from the ERIC compilation of implementation strategies [16], to identify and record strategies used to implement TMH (see Table 1 for further details). Secondly, outcomes of studies were categorised according to the Proctor taxonomy of implementation outcomes [12], which consists of the following: acceptability (stakeholders’ perception that the intervention is agreeable), adoption (uptake of the intervention), appropriateness (the perceived compatibility of the intervention), feasibility (the extent a new intervention can be used in a particular setting), fidelity (whether an intervention was implemented as originally prescribed), implementation cost (the cost effect of implementing the intervention), penetration (the integration of the intervention within a service) and sustainability (the extent to which a new intervention is maintained). Details of outcomes and interventions from the framework were used to create summaries of the strategies employed and resulting outcomes in each study. As per recommendations for the use of these outcomes [12], we recorded within the taxonomy both those outcomes reported as resulting from the pre-planned strategies intended to optimise TMH, and outcomes relating to the TMH interventions themselves (as a result of implementation strategies) where reported by the included studies. For example, we recorded acceptability of training reported by clinicians as well as acceptability of the TMH interventions reported by service users, which may have been impacted by the implementation strategy of 'train clinicians'.

Table 1 Details of all categories covered by the ERIC compilation of implementation strategies – from Powell et al. [16]
Full size table

Results

Study selection

Database searches identified 20,858 papers, of which 14,294 were screened by their title and abstract once duplicates had been removed. A total of 338 papers were screened at full text, resulting in 14 studies identified for inclusion in the review. No additional papers were identified from preprint servers or from forward or backward citation searching. The study selection and screening process is summarised in Fig. 1.

Fig. 1
figure 1

PRIMSA flow chart showing the study selection and screening process

Full size image

Study characteristics

Of the 14 included studies, 11 were conducted in the USA, with one each in the UK, Canada, and Australia. Six studies focused only on service users, seven only on staff, and one involved both staff and service users as participants. Four studies used quantitative methods, four were mixed methods studies, two were qualitative studies and four studies were descriptive in nature (did not aim to address a specific research question and instead provided a description of actions and their outcomes). No trials or studies with a comparison group were identified.

The majority (n = 11) of studies explored implementation in community mental health services, and five studies used an established implementation framework to inform their analysis. Further details of included studies can be found in Table 2.

Table 2 Characteristics of included studies
Full size table

Quality of included studies

MMAT [17] quality appraisal was conducted for the 10 primary studies which aimed to answer a specific research question (as opposed to purely describing the implementation of TMH). Of these, two studies met 20% of criteria (low quality), three studies met 40% of criteria (medium quality), three studies met 60% or criteria (medium quality), and one study each met 80% and 100% of criteria (high quality). Four descriptive studies were appraised using the AACODS checklist [18], and as all studies met all (3/4) or all but one (1/4) criteria, we considered these studies as appropriate descriptive studies but considered primary studies as more important. A full breakdown of the results of the quality assessment is provided in Additional file 2: Appendix 2.

Evidence synthesis

Each type of implementation strategy in ERIC’s taxonomy was reported in at least one study. The most commonly used strategy was ‘Train and educate stakeholders’, which was identified in nine studies, whilst the least used was ‘Utilise financial strategies’, which was only reported in one study [24]. The mean number of strategies used per study was 3.5, while the most common numbers of strategies used per study was 2 or 3. The implementation strategies reported by each study can be found in Table 3.

Table 3 Implementation strategies reported in each study included in this review
Full size table

Most of the implementation strategies were reported as being associated with good outcomes for TMH implementation, such as improved knowledge for clinicians regarding how to conduct care via TMH, increased acceptability or adoption. However, some barriers to TMH implementation remained, including a lack of staff time, higher administrative burden, or a preference for face-to-face appointments (amongst either staff or service users).

While most studies reported implementation outcomes only in relation to the TMH intervention itself, five studies reported implementation outcomes resulting specifically from the strategies used [23, 25, 28, 31, 33]. Felker et al. [23] used three strategies: ‘provide interactive assistance’, ‘adapt and tailor to the context’, and ‘train and educate stakeholders’, the latter strategy resulted in high ratings of the acceptability, adoption, and appropriateness of the training provided. Lindsay et al. [25] also reported using the same three strategies and found that therapists reported a high degree of satisfaction with the external facilitation provided as part of training. Myers et al. [28] used three strategies: ‘provide interactive assistance’, ‘develop stakeholder interrelationships’ and ‘train and educate stakeholders’ during the implementation of a TMH platform, and found that whilst adoption of TMH increased, there were additional implementation costs in terms of time associated with unpaid site champion roles developed to provide interactive assistance. Puspitasari et al. [31] used five implementation strategies in the rapid adoption of TMH due to the COVID-19 pandemic: ‘use evaluative and iterative strategies’, ‘provide interactive assistance’, ‘adapt and tailor to the context’, ‘develop stakeholder interrelationships’, and ‘train and educate stakeholders’. They found education, training, and ongoing supervision were of particular importance at the start of teletherapy implementation to support clinicians’ successful engagement with the technology. Taylor et al. [33] used two strategies—‘use evaluative and iterative strategies’ and ‘develop stakeholder interrelationships’ to investigate the importance of clinical facilitation for the implementation and sustainability of TMH in perinatal and infant mental health services and concluded that ongoing clinical facilitation is necessary for the sustainability of TMH services.

We had originally aimed to make specific links between strategies and general implementation of TMH outcomes, but as all but one study reported several implementation strategies in varying combinations, this was not possible. Instead, strategies and reported outcomes are presented in Table 4 and outlined for each study in Additional file 3: Appendix 3. Outcomes for each study are categorised according to the taxonomy of implementation outcomes [12].

Table 4 Implementation strategies and outcomes for each study included in this review
Full size table

There was no discernible difference in the implementation strategies used by studies which did or did not use implementation frameworks to guide their approach, as all studies used a variety of different strategies. In addition, we did not observe any commonalities across the outcomes of these studies.

We also compared the use of telemental health strategies between the studies which implemented telemental health before and after the start of the COVID-19 pandemic, but as all studies used a variety of different strategies there were no obvious differences in the types of strategy used or reported outcomes. However, Felker et al. had implemented telemental health prior to the pandemic and concluded that “earlier TMH training efforts likely helped foster staff interest and experience using TMH, and in doing so contributed to the relatively strong TMH adoption rate observed in this health care system during the early months of the COVID-19 pandemic.”

Discussion

Summary of findings

In this study, we have reviewed literature on strategies used to improve implementation of TMH and their associated implementation outcomes. We identified as meeting our inclusion criteria fourteen studies, conducted across five countries. Both staff and service user views and outcomes were represented in these studies. Results indicated that using a combination of different planned implementation strategies could be associated with successful implementation of TMH, although the methodologies of most studies were such that firm causal conclusions were difficult to draw. Whilst we had originally planned to explore links between individual types of implementation strategy and implementation outcomes directly, we were unable to isolate the effects of specific mechanisms as all studies reported outcomes of initiatives that combined multiple implementation strategies and none had a comparison group. We are, however, able to propose some tentative conclusions based on the synthesis of findings from studies which reported outcomes of these strategies. Ongoing support and facilitation, for example, through either technical assistance or ongoing consultation, was common and tended to be strongly linked to successful implementation. Providing initial training and the use of ‘digital champions’ to model best practice, also benefited implementation of TMH.

Other recent studies further support and supplement these conclusions. In our recent rapid realist review of TMH [34], we found that providing staff with training on the use of technology to deliver mental health services, a strategy reported by several studies in the current review, was reported to increase confidence in and uptake of TMH. The rapid realist review also found that adapting the use of TMH to take into account service user preferences was beneficial in removing barriers to accessing TMH [34] (see Table 5 for further discussion). For example, a commonly cited barrier to service user engagement with telemental health has been a lack of choice of modality [5, 35], so that offering a choice of platforms to service users may remove some barriers to telementation implementation.

Table 5 Lived experience commentary by Beverly Chipp & Karen Machin, members of the NIHR Mental Health Policy Unit's Lived Experience Working Group
Full size table

Research indicates that ERIC strategies are considered suitable to influence different implementation outcomes [36], but there is currently little consensus or evidence regarding which strategies affect specific outcomes. Furthermore, Powell and colleagues [37] argue that implementation strategies should be tailored to the circumstances and context of each change project, as they may be more likely to result in changes to practice. They suggest four different methods to identify appropriate strategies: concept mapping, group model building, conjoint analysis, and intervention mapping. Essentially, this means that the appropriateness and effectiveness of an implementation strategy or implementation support ‘bundle’ may well depend on the context in which a clinical intervention or service delivery mechanism, such as TMH, are introduced to. It logically follows that the same implementation strategy may be very well suited to one context, but redundant in another if it fails to address specific barriers to implementation. Hence selection of implementation strategies should be tailored to the local context of application. This can be illustrated by studies which have reported various staff barriers or challenges to using TMH. These have included: feeling isolated from colleagues, tasks taking more time when working from home, lack of support, concerns around privacy or confidentiality, and concerns around developing and maintaining a therapeutic relationship [2, 38, 39], indicating the importance of tailoring implementation strategies to overcome local barriers. Whilst some studies included in this review evaluated the barriers and facilitators to TMH before using implementation strategies to address these, not all papers reported taking this approach in a systematic manner.

Implications for future research

As all studies reported using multiple implementation strategies, we were unable to compare the effectiveness of specific strategies. This could be addressed by future research, for example, cluster randomised controlled trials to formally compare the effectiveness of different implementation strategies linked to specific outcomes at a team or organisational level, although it would also be helpful to evaluate a theoretically informed approach to selecting a bundle of implementation strategies to fit a particular context, using more robust study designs. Inclusion of a control group in future evaluations of the implementation of TMH is critical: without this we will remain unable to establish causal links between the presence or absence of a strategy to support implementation, or bundle thereof, and success of implementation offers an illustration of a clustered randomised evaluation of different interventions to support the implementation of four evidenced-based psychosis treatments. Within this study, what was randomised was not the clinical therapies, but rather the level of implementation support, which was limited in the control arm (provision of treatment manuals) and substantially enhanced in the intervention arm to include toolkits, training, implementation facilitation, and data-based feedback. The primary endpoint was fidelity of treatment delivery, and the trial concluded that the implementation ‘bundle’ was successful in enhancing fidelity across all four studied treatments for psychosis. Similarly designed studies in the context of TMH provision will significantly expand our knowledge regarding how best to deliver it sustainably and effectively.

Beyond controlled evaluations, our understanding of the relative effectiveness and suitability of strategies to support implementation of TMH across different settings can be further enhanced. Future observational studies should offer a detailed description of local (or wider, as appropriate) settings in which TMH is offered, a well-articulated rationale for the selection of strategies (such as those we summarised above) to support implementation, and clear details of what each strategy involves – such that subsequent evidence syntheses can offer a better articulation of which strategies may be better suited to which contexts. We further propose that selection of implementation support interventions should be based on a description and mapping of the barriers and drivers an implementation effort is likely to face, for example carried out using one of the methods proposed by Powell et al. [38].

It is also important to note that most studies identified in this review were conducted in the United States. There is therefore a need to replicate these findings and conduct further research in other countries with different healthcare structures and funding models to generalise the findings from this research.

Implications for practice

Our review identified a range of potential implementation strategies to be deployed to improve TMH implementation in routine settings, with evaluations spanning a full range of types of implementation strategy. Although we were unable to identify causal links between implementation strategies and outcomes, findings across the included studies suggest that when implementing TMH, service planners should consider a multi-component implementation strategy. This strategy should be tailored to the local context and designed to address any pre-identified barriers. It is likely that staff training and facilitation support are key factors in the success of implementing TMH.

Strengths and limitations

A strength of our review is that it focused only on studies which implemented TMH as part of their routine service (i.e., not just in a trial), which means findings are more likely to generalise to ‘real world’ settings. This review also integrated data from studies conducted before the COVID-19 pandemic with those conducted during the pandemic, which enables findings to be used to inform future models of service development.

Limitations of this research should be acknowledged. Firstly, as noted earlier, the high heterogeneity of strategies and outcomes reported across studies makes it hard to reach firm judgements about which strategies are linked with effective implementation of TMH. As all studies reported the inclusion of multiple strategies, we were unable to draw conclusions regarding the active ingredients of specific strategies. Secondly, researchers were not blinded to the results of screening and quality assessment during double screening. This was due to the short timeframe in which the review was conducted in order to make its results relevant to the current service context. We limited the scope of our review to the implementation of TMH in specialist mental health services, which typically serve mainly people with severe and enduring mental illness. Our review is therefore unable to provide evidence about TMH implementation in primary care settings, or for other client groups including people with substance misuse (in the absence of a mental illness) or people with dementia or other organic disorders. Finally, included studies were mostly of moderate to low quality, and only five interventions were informed by an established implementation framework.

Conclusion

Using a combination of implementation strategies appeared associated with successful implementation of TMH, but it was not possible to infer conclusively causal relationships between specific types of implementation strategy and outcomes. Potentially valuable strategies to improve the implementation of TMH include providing initial training for clinicians, as well as ongoing support and consultation. Further research utilising more robust study designs to evaluate individual implementation strategies is needed to explore which specific factors can influence implementation of TMH.