Introduction

Telehealth is known to help patients overcome two barriers they face when seeking health care: distance and time [1, 2]. Proponents of telehealth have argued that it has the potential to transform healthcare delivery by reducing costs, increasing quality of care, enhancing patient & provider satisfaction, and improving population health outcomes [1,2,3,4,5]. Throughout the COVID-19 pandemic, the healthcare industry has witnessed a massive acceleration in the use of telehealth services, bolstered largely by the temporary removal of policy-level barriers to telehealth use (e.g., federal or state restrictions to telehealth coverage and reimbursement) [6]. Although uncertainties remain regarding the future sustainability of telehealth, a key point of consensus, is that the permanent removal of policy-level barriers by itself, would not be sufficient to ensure widespread, sustainable use of telehealth services in the post-pandemic era (although it could serve as an important facilitator) [1, 2, 6]. On the other hand, the literature has emphasized the need for healthcare providers and organizations to make systematic and concerted efforts towards effective implementation of telehealth services, for long-term sustainable use [1, 2, 5, 6].

Existing literature on telehealth service implementation frameworks

Over the past three decades, a variety of frameworks have been put forth for understanding the key barriers and facilitators to implementing telehealth services. Notably, a comprehensive review of ‘telehealth service implementation frameworks,’ by van Dyk (2014) [7] identified nine different frameworks. 1) The seven core principles for the successful implementation of telemedicine, which emphasizes the importance of pragmatism, user-friendliness, user-training, and organizational structure in telehealth implementation [8]; 2) Telehealth readiness assessment tools, which emphasize planning, technological, learning, societal, and policy readiness [9, 10]; 3) Barriers to the diffusion of telemedicine, which emphasizes technical, behavioral, economic, and organizational barriers [11]; 4) Lessons in telehealth service innovation, which identify various success factors, including the policy context, perceived benefit, professional roles and willingness to cross boundaries [12, 13]; 5) The Unified Theory of Acceptance and Use of Technology (UTAUT), which describes the interaction among variables influencing technology acceptance [13]; 6) The Khoja-Durrani-Scott (KDS) Framework, which considers various stages in the telehealth lifecycle [14]; 7) The framework on health system challenges in scaling up for telehealth, which includes consideration for policy, organizational, technological, and financial challenges [15]; 8) The layered telemedicine implementation model, which identifies determinants of success associated with each lifecycle phase of telemedicine [16]; and 9) The comprehensive telemedicine evaluation model, which considers several issues related to telehealth implementation, including the cost of education, quality of clinical services, and community access to services, among others [17]. The review (by van Dyk, 2014) concluded that a “holistic” approach is needed to telehealth service implementation, which includes consideration for organizational structures, change management, technology, economic feasibility, societal impacts, perceptions, user-friendliness, evidence and evaluation, and policy and legislation [7].

It would be relevant to note however, that van Dyk’s (2014) review of telehealth implementation frameworks did not include the Consolidated Framework for Implementation Research (CFIR) [18]. The CFIR was introduced in 2009, to serve as a comprehensive meta-theoretical framework that could be used to inform both ‘implementation science’ and ‘implementation strategy,’ in health services delivery. Since its introduction, the CFIR has been leveraged to inform effective implementation of a variety of health service innovations, including evidence-based practices for patient safety, best practices for patient-and-family centered care, and a variety of health information technologies, including Electronic Health Records and clinical decision support systems [19]. Along these lines, in recent years, the CFIR has also been utilized to inform the implementation of telehealth service initiatives [6].

The CFIR comprises five major domains (characteristics of the intervention, the outer setting, the inner setting, characteristics of the individuals involved, and the process by which implementation is accomplished) [18]. Each domain in turn, is mapped to an array of constructs informed by existing implementation theories and conceptual models. For example, the domain of inner setting been mapped to the following constructs: 1) structural characteristics, 2) networks and communication, 3) culture, including norms and values of an organization, and 4) implementation climate or the absorptive capacity for change. Six sub-constructs contribute to a positive implementation climate for an intervention, including readiness for implementation, compatibility, relative priority, organizational incentives & rewards, goals & feedback, and learning climate. Readiness for implementation in turn, includes three sub-constructs, i.e., leadership engagement, available resources, and access to information & knowledge. The five domains (and constructs) in the CFIR, are known to interact in rich and complex ways to influence implementation effectiveness.

As a pragmatic meta-theoretical framework with a comprehensive taxonomy of domains and constructs, the CFIR may be viewed as a “holistic” approach to understanding barriers and facilitators to implementation, compared to any other existing stand-alone framework for telehealth implementation [18, 19]. For example, no other framework mentioned above, is known to give due to consideration to the influence of ‘organizational culture’ or ‘implementation climate’ on implementation effectiveness. By contrast, the CFIR not only emphasizes both these constructs, but it goes a step further in identifying six sub-constructs contributing to a positive implementation climate, including ‘readiness for implementation,’ and three additional sub-constructs contributing to ‘readiness for implementation.’

By virtue of its comprehensive taxonomy (five domains and multiple interrelated constructs for assessing implementation effectiveness), a distinguishing feature of the CFIR is that it can help to understand why a particular implementation initiative succeeded or failed. The CFIR has been leveraged as a framework for guiding formative evaluation of implementation efforts. In addition, the CFIR is known to combine well with other established frameworks that could be used to assess implementation scalability and sustainability, e.g., the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework [20]. Supplementing the CFIR with the RE-AIM framework in turn, has potential to serve the dual purpose of providing insight into 1) implementation effectiveness and 2) implementation scalability & sustainability.

Gaps in the literature

It is noteworthy that the CFIR was introduced to the health services sector only a little over a decade ago. Correspondingly, CFIR applications to the telehealth implementation context have only gained momentum over the past 5–10 years. For example, a keyword search on PubMed of (Telehealth OR Telemedicine) AND (Consolidated Framework for Implementation Research) on March 1, 2022, returned a total of 58 articles. By comparison, the broader keyword search on PubMed of (Telehealth OR Telemedicine) AND (Implementation) on the same day, returned a total of 5587 articles, indicating that the body of literature on telehealth and CFIR, accounts for less than 1% of the broader literature on telehealth implementation on PubMed. It is appropriate to compare the results from the two sets of search terms (mentioned above) on any given academic database (like PubMed) at a given point in time. This is because the broader (latter) search serves to capture the general state of Implementation Research related to telehealth or telemedicine (on PubMed). The CFIR or the Consolidated Framework for Implementation Research is one of many tools that could be used to guide implementation research. As mentioned earlier, since its introduction in 2009, the CFIR has received a lot of attention in implementation science in general and more specifically, in the context of implementing health services innovations like evidence-based practices and Electronic Health Records [19]. The question of interest to this study is the scope of knowledge gained from ‘CFIR applications to telehealth implementation initiatives.’ Therefore, it is appropriate to compare the results from the two sets of search terms mentioned above, to establish a baseline understanding of existing research on this topic and highlight the gap in the literature. Based on the search results, it is not surprising to note that there are no existing reviews of the literature to characterize the scope of knowledge that has been gained exclusively from ‘applications of the CFIR to telehealth service implementation initiatives.’ This paper seeks to address this gap.

Review objective and review questions

This paper undertakes a comprehensive review of the literature to characterize the scope of knowledge that has been gained from applications of the Consolidated Framework for Implementation Research (CFIR) to telehealth service implementation initiatives. The review objective is to “identify and synthesize the literature related to applications of CFIR to telehealth service implementation initiatives.” Correspondingly, this scoping review is expected to be directly relevant to healthcare providers and organizations looking to get started with telehealth and/or to design and implement telehealth services for effective and sustainable use. The specific review questions are outlined below.

  1. 1)

    What have we learned so far from applications of the CFIR to telehealth service implementation initiatives?

    1. a)

      What have we learned about the outcomes (success or failure) of telehealth service implementation initiatives?

    2. b)

      Has the CFIR been combined with other frameworks to enable assessment of both i) effectiveness and ii) scalability or sustainability of telehealth implementation?

    3. c)

      Which CFIR domains (or constructs) have been identified as most influential in explaining success or failure of telehealth implementation initiatives?

  2. 2)

    What are the descriptive characteristics of CFIR applications to telehealth service implementation initiatives?

    1. a)

      What healthcare domains (e.g., primary care, emergency care, post-acute care, mental health, oral health, etc.) have CFIR applications to telehealth initiatives focused on?

    2. b)

      What diagnoses or conditions (e.g., diabetes, stroke, cancer, depression, dementia etc.), have CFIR applications to telehealth initiatives focused on?

    3. c)

      What target populations (e.g., children, adults, seniors, veterans, etc.), have CFIR applications to telehealth initiatives focused on?

    4. d)

      What technology areas (e.g., synchronous technologies such as interactive audio/video, or asynchronous technologies such as store-and-forward, remote monitoring, mHealth apps), have CFIR applications to telehealth initiatives focused on?

    5. e)

      What service areas (e.g., clinical practice/healthcare delivery, medical education, population health management, healthcare administration), have CFIR applications to telehealth initiatives focused on?

Rationale for a scoping review

According to Sucharew and Macaluso (2019) [21], scoping reviews can be useful for answering broad questions, such as “What information has been presented on this topic in the literature?” which in turn is fully consistent with what this review seeks to accomplish. There have been no comprehensive reviews of the literature to-date, to characterize the scope of knowledge that has been gained exclusively from applications of the CFIR to telehealth service implementation initiatives. This paper seeks to address this gap, and the review objective (and questions) in turn, are aligned with this purpose. Moreover, a scoping review is intended to provide an overview of the available research evidence without producing a summary answer to a discrete research question. The questions of this review lend themselves to a scoping review approach (versus other types of review) because they are broad in scope, and the review objective is to describe the available evidence on applications of CFIR to telehealth service implementation initiatives, as opposed to addressing a discrete research question (e.g., “what is the relationship between implementation climate and implementation success of telehealth implementation initiatives?”)

Methodology

The review protocol was developed based on guidelines for scoping reviews provided by the Joanna Briggs Institute (JBI) [22]. The PRISMA-ScR criteria (for scoping reviews) were used to frame the review effort [23]. The protocol was not registered. The review protocol is included in Additional file 1 and the completed PRISMA-ScR checklist is included in Additional file 2.

Information sources

This scoping review sought to identify published original research articles (including quantitative, qualitative, and mixed-method studies) and review articles, to address the review questions. Since the CFIR was officially introduced in 2009, the following five major academic databases were searched for coverage from 01.01.2010 through 12.31.2021: 1) PUBMED, 2) SCIENCE DIRECT (SCIDEIRECT), 3) PROQUEST, 4) CINAHL, and 5) WEB OF SCIENCE (WoS). The article search was conducted in March 2022. The five databases were selected to ensure maximum coverage across medicine and social science domains. Additional searches were conducted on databases relevant to education (ERIC) and engineering domains (IEEE Explore). However, these searches produced negligible results on the topic of interest, and the latter two databases were excluded from information sources for this review.

Search strategy

The following two sets of search terms were used to search all five databases for the period 01.01.2010 through 12.31.2021: 1) (Telehealth OR Telemedicine) AND (Consolidated Framework for Implementation Research); and 2) (Telehealth OR Telemedicine) AND (CFIR). The full electronic search strategy used on PUBMED is included as an example, in Additional file 3. It would be relevant to note that “Telemedicine” is a National Library of Medicine Medical Subject Heading (MeSH) term that includes the synonyms (entry terms) “mobile health,” “mhealth” and “ehealth.” The resulting total number of records from this initial search, for both sets of search terms combined, was 18,388 records (including 73 from PUBMED, 909 from SCIDIRECT, 17,318 from PROQUEST, 32 from CINAHL, and 56 from WoS). These totals included peer-reviewed (scholarly) journal articles, conference papers, working papers, wire feeds, reports, books, trade journals, dissertations, theses, magazines, and other sources. The next section describes the eligibility criteria that were applied to select articles for final inclusion in this scoping review.

Eligibility criteria

This review considered original research articles (including clinical trials, quantitative, qualitative, and mixed-method studies) and review articles, that were published in peer-reviewed journals, in English language, and pertained to the scope of the review (i.e., ‘the application of CFIR to telehealth implementation initiatives’). Since the CFIR was officially introduced only in 2009, papers published between 01.01.10 and 12.31.21, were included for consideration. All forms of telehealth were considered, including telemedicine, digital health, eHealth and mHealth technologies. Research papers considered for inclusion were based on empirical data, including, but not limited to, data collected from clinical trials, surveys, observations, focus groups, and interviews. Among reviews, systematic and scoping reviews were considered for inclusion.

This scoping review excluded: 1) articles that were not published in peer-reviewed journals (e.g., conference papers, working papers, wire feeds, reports, books, trade journals, dissertations, theses, magazines, and other sources); 2) articles that were not pertinent to the review topic (e.g., papers that did not involve use CFIR or telehealth or both). It also excluded 3) articles that were neither original research nor reviews (e.g., study protocols, editorial articles, discussion papers, theoretical reflections, or any other type of article that did not include a methodology section). Additionally, this scoping review excluded 4) articles that did not meet critical appraisal criteria outlined by the Joanna Briggs Institute (JBI) and the Mixed-Method Appraisal Tool (MMAT). While JBI checklists were used for critical appraisal of qualitative studies, reviews, clinical trials, and cross-sectional studies, the MMAT was used for critical appraisal of mixed-method studies. The templates used for article selection (i.e., eligibility and critical appraisal criteria), are included in Additional file 4.

Process for selecting sources of evidence

Following the search, all identified citations were collated and uploaded into a reference management system (Zotero 5.0) for initial screening. After removal of duplicates, article titles and abstracts were screened for potential inclusion, based on the eligibility criteria for the review. Articles identified for inclusion based on screening of titles & abstracts, were retrieved in full text for assessment based on the eligibility criteria. Reasons for article exclusion at each stage of the process were noted and have been reported in detail in the Results section. All articles that were selected based on eligibility criteria, were subjected to critical appraisal. Only articles that met the critical appraisal criteria were selected for final inclusion in the review. The results of the search are reported in full using a PRISMA flow chart.

Process for charting data items

All included articles were reviewed to retrieve two categories of data items, Category #1: data items for characterizing the articles (e.g., Article Name, Authors, Publication Year, Article Type); and Category #2: data items for capturing results based on the review questions (RQ1 and RQ2). Both data categories were retrieved from explicit information presented in the reviewed articles and charted in two separate spreadsheet templates included in Additional file 5. Correspondingly, Additional file 5 constitutes the raw dataset for the study. Together, the two data charting spreadsheets incorporated all the fields needed to capture the data items outlined above. No additional assumptions or simplifications needed to be made in the data charting process.

To elaborate, data items relevant to RQ 1 (“What have we learned so far from applications of the CFIR to telehealth service implementation initiatives?”) included: 1a. “Does the article include an outcome measure of intervention or implementation effectiveness of the telehealth initiative (Yes/No)? 1b. “Is the CFIR combined with other frameworks in assessing the telehealth initiative (Yes/No)?” and 1c. “Which CFIR domains (or constructs) were identified as influential in explain telehealth implementation effectiveness?” Each data item in turn, was directly aligned with the corresponding review questions (RQs 1a, 1b, and 1c) outlined earlier.

Data items relevant to RQ 2 (“What are the descriptive characteristics of CFIR applications to telehealth implementation initiatives?”) were as follows: 2a. Healthcare Domains of Interest; 2b. Targeted Diagnoses or Conditions; 2c. Targeted Patient Populations; 2d. Technology Areas; and 2e. Service Areas of Interest. Each data item in turn, was directly aligned with the corresponding review questions (RQs 2a, 2b, 2c, 2d, and 2e) outlined earlier.

Process for synthesizing results

Data were summarized using counts, aggregates, and proportions for analysis based on the review questions. For example, data on article characteristics (e.g., article type and publication year) and data on review questions (e.g., CFIR domains found to influence telehealth implementation) were summarized for analysis and interpretation. This process in turn, helped to synthesize results and draw inferences related to the state of the science on CFIR applications to telehealth implementation initiatives.

Results

Selection of sources of evidence

The initial database search resulted in a total of 18,388 records. Of these, a total of 15,813 records were excluded for not being peer-reviewed journal articles. Among the remaining 2575 peer-reviewed journal articles, a total of 1024 duplicates were removed, and the remaining 1551 articles were screened for eligibility based on titles & abstracts. Of these, 54 articles were excluded for being wholly or partially in non-English language. An additional 954 articles were excluded for being outside the scope of the review; and an additional 437 articles were excluded for not being an acceptable article type (including 94 study protocols). The remaining 100 articles were retrieved in full text for assessing eligibility for inclusion. Following full review, 32 articles were excluded for being outside the scope of the study, and the remaining 68 articles were subjected to critical appraisal. Following critical appraisal, a total of 64 articles were identified for final inclusion in the scoping review [24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87]. The search results are summarized in full in a PRISMA chart in Fig. 1. The supplementary material includes a breakdown of the aggregate search results for each of the five databases (Additional file 6).

Fig. 1
figure 1

PRISMA Article Selection Flow Chart

Full size image

Characteristics of sources of evidence

Table 1 outlines data items representing characteristics of included articles. The results are synthesized below, and full article citations for all individual sources of evidence are provided under References.

Table 1 Data charting on article characteristics
Full size table

Results of individual sources of evidence

Table 2 outlines data items representing results as they relate to review questions, including RQ1. The results are synthesized below.

Table 2 Data charting on review questions
Full size table

Synthesis of results

The data presented in Tables 1 and 2, provide a foundation for synthesizing the results of this review with respect to article characteristics and both the review questions (RQ1 and RQ2).

Results based on key article characteristics

To begin with, Table 3 Part A provides a summary breakdown of articles reviewed by publication year, while Table 3 Part B provides a summary breakdown by article type. Table 3 Part A shows that a majority 64% of included articles were published in 2020 or later, with 20% published in 2020 and 44% published in 2021. This suggests that CFIR applications to telehealth have gained momentum during the pandemic period. Complementarily, the database search found several protocols for implementation-effectiveness studies on the topic of interest (published during/after 2020), which could not be included in this review. The latter in turn, serves to not only reinforce the gleaning that CFIR applications to telehealth initiatives received a boost during the pandemic, but also, that the science of telehealth implementation informed by the CFIR, has potential for significant advancement in the coming years, as study protocols materialize into completed and published studies.

Table 3 Summary of key article characteristics
Full size table

Table 3 Part B indicates that 85% (54) were research articles, while 14% (9) were review papers. The majority (77%) of all included articles (and 91% of research articles), were qualitative or mixed-method studies seeking to identify barriers or facilitators to telehealth service implementation informed by CFIR domains or constructs. In other words, most studies eligible for inclusion in this review, were focused on a qualitative or mixed-method assessment of barriers and facilitators to effective implementation of telehealth initiatives, using the CFIR. These studies in turn, were based on data collected from key informants involved in the implementation process, through interviews, focus groups, construct-based surveys, observation, content, or archival analysis and/or other mixed-method analytic techniques.

Among original research papers, examples of qualitative or mixed-method studies included the following: One study sought to evaluate perceived determinants of Telemedicine Diabetic Retinopathy Screening (TDRS) in Federally Qualified Health Centers, [24] through semi-structured interviews with key informants (administrators, clinicians, staff) involved in TDRS. Another study sought to identify organizational factors promoting successful implementation of telehealth and adoption of “no test” medication abortion protocols through semi-structured interviews with providers during the COVID-19 pandemic [25].

Among remaining original research papers, 3% (2) were implementation-effectiveness hybrid studies [31, 53]; 5% (3) were comparative effectiveness studies, [57, 63, 65] while only 2% (1) fell in the “other” category [43]. Implementation-effectiveness hybrid studies involved the concurrent evaluation of implementation and intervention effectiveness, and often included a qualitative or mixed-method design component for assessing implementation effectiveness. For example, one implementation-effectiveness study sought to implement and evaluate a complex mHealth intervention in Uganda [31]. Another sought to implement and evaluate a clinical communication tool (known as the Loop) [53] for team-based care in pediatric and adult care settings. Comparative effectiveness studies were clinical trials that sought to assess either the effectiveness of telehealth over usual care, or the comparative effectiveness of two different telehealth initiatives, or one initiative in different contexts. Like implementation-effectiveness hybrid studies, comparative effectiveness studies were often preceded or followed by a qualitative or mixed-method assessment of implementation effectiveness. For example, one comparative effectiveness study sought to compare outcomes for TeleMOVE with standard, facility-based MOVE weight-management services, while also examining factors influencing TeleMOVE implementation across demonstration sites [65]. The one paper that fell into the ‘other’ category, was a secondary data analysis study that sought to identify community and hospital characteristics associated with adoption of tele-stroke among acute care hospitals in the state of North Carolina, United States [43].

Examples of included review papers were as follows: one review paper sought to identify, appraise, and synthesize qualitative research evidence on healthcare stakeholders’ perceptions of factors affecting the implementation of Critical Care Telemedicine [28]. Another review sought to conduct a rapid mixed-methods evidence synthesis to identify barriers, facilitators, and stakeholder experiences of implementing pediatric telemedicine, to inform the pandemic response [29].

Results based on review questions

With respect to results corresponding to the review questions, Table 4 Part A summarizes results pertaining to RQ1a (i.e., if the study included an outcome measure of intervention or implementation effectiveness of the telehealth initiative); Table 4 Part B summarizes results related to RQ1b (i.e., if the study sought to supplement CFIR with another framework); and Table 4 Parts C & D summarize results related to RQ1c (i.e., which CFIR domains or constructs were identified to have influence over implementation effectiveness).

Table 4 Summary of results on review question 1
Full size table

Table 4 Part A shows that a very small proportion of articles 8% (5) included an outcome measure of intervention or implementation effectiveness, while the vast majority 92% did not. This ties in with the findings related to article type, i.e., that most included articles were qualitative studies seeking to examine barriers or facilitators to successful implementation through interactions (e.g., interviews) with key informants among high or low (sometimes, both high and low) implementers. In other words, these studies were not designed to provide outcome measure(s) of success; instead, study sites were selected based on prior observation of implementation success (e.g., usage rates) and were approached post-implementation for the qualitative assessment of barriers/facilitators for the purpose of gaining insight into strategies for pre-implementation readiness assessment in similar contexts. Like the qualitative studies, none of the review articles included outcome measures of intervention or implementation success.

Only the handful of implementation-effectiveness and comparative-effectiveness studies (reported earlier), included outcome measures. For example, one implementation-effectiveness study that sought to compare outcomes for TeleMOVE with standard, facility-based MOVE weight-management services over the evaluation period utilized both the number of patients enrolled per site and the program’s clinical effectiveness (as demonstrated by average weight lost per patient), as key outcome measures [65]. Concurrently, the study sought to understand factors influencing TeleMOVE implementation across demonstration sites. Another comparative effectiveness study that sought to evaluate the success of two health interventions for caregivers of people with dementia (Myinlife and Partner in Balance), [57] used a variety of outcome measures, including eHealth use data, coach evaluation questionnaires, and information on implementation determinants. Although studies incorporating outcome measures of telehealth implementation success were more the exception than the rule in this scoping review, it would be relevant to note that the database search found a growing number of protocols for implementation-effectiveness studies published since 2020 (36 protocols in 2020, and 58 in 2021, for a total of 94 protocols), which suggests that the science of CFIR applications to telehealth initiatives may be poised to generate additional outcome measures of intervention- or implementation-effectiveness in the near future.

Similarly, Table 4 Part B shows that only one article (2%) sought to supplement CFIR with other frameworks (e.g., the RE-AIM framework) to concurrently assess both implementation success and scalability, in the context of a telemedicine-delivered healthy lifestyle program for obesity management in a rural, academic obesity clinic [34]. The scarce use of additional frameworks to supplement CFIR in the context of telehealth initiatives, is unlike other healthcare implementation areas involving CFIR applications, like evidence-based practice implementation or health IT (Electronic Health Record) implementation, where CFIR has been more frequently supplemented with RE-AIM and other frameworks [19, 20].

Table 4 Part C summarizes the CFIR domains found to be of influence in telehealth implementation. As indicated in the table, most articles reviewed 58 (91%) reported the Inner Setting domain of CFIR to be significant in influencing telehealth implementation success, either by itself or alongside other CFIR domains. Only 6 (9%) of the articles reviewed did not report any findings related to Inner Setting. On the other hand, Inner Setting was found to be the sole domain of influence in 7 (11%) of the articles reviewed. Process, Intervention Characteristics, and Individual Characteristics domains, each followed the Inner Setting domain in being identified as influential predictors of telehealth implementation in 33(52%), 32(50%), and 28 (44%) of studies reviewed, respectively. By comparison, Outer Setting received the fewest mentions, with only 9 (14%) of articles reviewed identifying this domain to be of importance in influencing telehealth implementation success.

Given the predominance of Inner Setting in influencing telehealth implementation, it would be relevant to note the domains that appeared most frequently alongside Inner Setting. The Process domain appeared most frequently alongside the Inner Setting domain, with 28 mentions, while the Intervention Characteristics domain was a close second with 27 mentions alongside the Inner Setting domain, while Individual Characteristics received 24 mentions alongside Inner Setting. As shown in Table 4 Part D, the most frequent exclusive domain combinations were “Inner Setting and Intervention Characteristics,” which were found to be influential predictors of telehealth implementation success in 10 (16%) of the 64 studies, and “Inner Setting, Intervention Characteristics, and Process,” which were found to be influential in 9 (14%) of the 64 studies.

To elaborate, one study on determinants of Telemedicine Diabetic Retinopathy Screening in Federally Qualified Health Centers, found that four constructs, two related to Inner Setting (Leadership Engagement, and Goals and Feedback) and two related to Process (Engaging and Champion) were perceived as strongly distinguishing high from low implementation effectiveness [24]. Another study emphasized the relevance of Inner Setting as a facilitator, and Individual Characteristics as a barrier. This study compared the characteristics of EDs with robust and low implementation of tele-stroke and found the CFIR domain of Inner Setting to be of strong relevance to robust implementation [30]. In EDs with robust assimilation, tele-stroke programs had the support of leadership, tele-stroke use and outcomes were measured, and stakeholders received regular feedback about their tele-stroke use. By comparison, in EDs with low implementation, ED physicians felt that tele-stroke had little value beyond a telephone consult and tele-stroke was perceived to increase complexity, indicating that Individual Characteristics of implementers, served as a barrier to tele-stroke implementation. Another study seeking to identify barriers affecting implementation of an online frailty tool in primary care, [42] helped to understand the crucial role of Intervention Characteristics, specifically the potential for the intervention to be integrated into the workflow (i.e.,  intervention adaptability), in influencing implementation success.

One comparative-effectiveness study on telehealth over usual care, sought to understand barriers and facilitators to implementing a national telehealth weight management program (TeleMOVE) for Veterans [65]. Eleven sites reported high program complexity because TeleMOVE required more staff time per participant than MOVE! due to logistical and technical assistance issues related to the devices, indicating that Intervention Characteristics served as a barrier. High-uptake sites overcame implementation challenges by leveraging communication networks with stakeholders, setting programmatic goals, monitoring feedback of results, and taking time to foster incremental delivery improvements, all of which indicated how the Inner Setting could serve as a facilitator to implementation. On the other hand, low-uptake sites reported less leadership support and less communication among stakeholders, highlighting how the Inner Setting could serve as a barrier to implementation.

Next, Table 5 serves to summarize results pertaining to RQ2 (“What are the descriptive characteristics of CFIR applications to telehealth initiatives?”). Based on the sub-questions, Table 5 provides a summary breakdown of included articles by ‘healthcare domains of interest’ (Part A), ‘targeted diagnoses or conditions’ (Part B), ‘targeted populations’ (Part C), ‘technology areas of interest’ (Part D), and ‘service areas of interest’ (Part E).

Table 5 Summary of results on review question 2
Full size table

Table 5 Part A indicates that CFIR applications to telehealth initiatives have largely focused on the Specialty Care domain at 63% (40), followed by Primary Care at 16% (10), Emergency Care at 5% (3), Acute & Intensive Care at 5% (3), and Oral Health, also at 5% (3), followed by other domains. Table 5 Part B indicates that targeted diagnoses/conditions in Specialty Care consist of tele-psychiatry/mental health (including care for dementia, psychosis, audiology, and substance abuse), specialty care referrals (eConsults), tele-cardiology, telemedicine for infectious diseases (HIV, Tuberculosis), lifestyle health (including sleep medicine, smoking cessation, obesity management) geriatrics, maternal health, oncology, ophthalmology, and surgery. Targeted areas in the Primary Care domain, include diabetes (chronic disease) management, diabetic retinopathy screening, frailty screening, home-based primary care, precision health, and provider-to-provider communication (team-based care). Targeted areas in Emergency Care include stroke care and emergency management. Targeted areas in Acute & Intensive Care include patient-provider communication and Critical Care Telemedicine. Pediatrics included general pediatrics and school-based parent training, while the General domain included patient engagement and eConsults across various levels of care.

With respect to targeted populations, Table 5 Part C shows that CFIR applications to telehealth initiatives have largely focused on adults at 69% (including adults living with HIV, adults in rural areas, adults admitted to ICUs, and parents of school age children), followed by veterans at 14%, seniors at 8%, followed by children at 2%. Regarding technology areas of interest, Table 5 Part D shows that most CFIR Applications to telehealth initiatives have sought to leverage the benefits of both Synchronous and Asynchronous technologies at 55%, (e.g., Critical Care Telemedicine, eHealth for rehabilitation care, home-based primary care, pediatric telemedicine, precision health, tele-coaching for diabetes, lifestyle health promotion, including obesity management, tele-psychiatry for youth, and tele-stroke care). This was followed by a focus on Asynchronous-only technology at 31% (e.g., mHealth, remote monitoring, store-and-forward, virtual hub, and other asynchronous digital health initiatives, internet-based patient-provider communication, and SMS texting for medication adherence).

Regarding service areas of interest, Table 5 Part E shows that CFIR Applications to telehealth initiatives have focused mainly on the provision of healthcare delivery (i.e., clinical practice only) at 78%, with significantly lower proportions that have sought to use telehealth for 2) clinical practice and medical education (5%), 3) clinical practice and population health (6%), and 4) all three, i.e., clinical practice, medical education, and population health (6%).

Discussion

Summary of results

This review sought to characterize the scope of knowledge that has been gained thus far, from applications of the CFIR to telehealth service implementation initiatives. Following an extensive search for eligible articles in five major academic databases, a total of 64 peer-reviewed (original research or review) articles were reviewed. The review found that most (64% of) articles on the topic of interest have been published since 2020, and that a majority (77%) are qualitative or mixed-method studies seeking to identify barriers and facilitators to telehealth implementation (using CFIR), through interaction (e.g., interviews, focus groups, surveys) with key stakeholders involved in implementation. With respect to the scope of knowledge gained regarding success or failure of telehealth implementation initiatives, the review found a very small proportion of comparative or implementation-effectiveness studies (5%) that included outcome measure(s) of intervention or implementation effectiveness. Similarly, the review found that very few (2% of) studies sought to supplement the CFIR with other frameworks like the RE-AIM, to gain dual insight into implementation effectiveness and scalability/sustainability.

As discussed earlier, the CFIR’s potential as a comprehensive tool for informing factors influencing implementation-effectiveness has been demonstrated in other domains of health services innovation, including evidence-based practice (EBP) and Electronic Health Records (EHR) implementation [19, 20]. In this context, it would be relevant to acknowledge that research related to EBP and EHR implementation in the United States, has been directly proportional to the substantial attention these areas have received at a federal policy-level [6]. By comparison, telehealth has historically not been a federal health policy priority in the US. In the absence of policy-level support, telehealth adoption has historically been dictated by ad hoc initiatives undertaken at the individual provider or organizational level [1,2,3]. This changed during the COVID-19 pandemic, when telehealth received a much-needed boost from a combined surge in policy and provider-and-organizational-level attention, which in turn may help to explain the growth in research in this area during the pandemic, including the significant increase in study protocols for implementation-effectiveness trials on the topic of interest. These trends suggest that the science on the topic may be gravitating towards generating: 1) more outcome measures of telehealth implementation-effectiveness in the future, alongside 2) barriers and facilitators to implementation (based on CFIR domains) and 3) insights into the scalability and sustainability of telehealth initiatives using additional frameworks like RE-AIM, all of which could help to engender a more nuanced understanding of the determinants of telehealth implementation-effectiveness. These gleanings in turn suggest that a follow-up review on this topic within the next 5 years, may be a fruitful endeavor.

Regarding CFIR domains of influence, the Inner Setting domain was found to be significant in influencing telehealth implementation success in the vast majority (91%) of articles, either by itself, or alongside other domains. Several articles found the Inner Setting to be the sole domain to explain implementation success. On the other hand, none of the remaining four domains were found to be influential by themselves, which in turn serves to underscore the predominance of the Inner Setting domain in helping to explain telehealth implementation effectiveness. Inner Setting constructs that were found to be consistently important included, the availability of resources, goals & feedback, leadership engagement, readiness for implementation, and implementation climate.

The two domains that most frequently appeared alongside the Inner Setting domain were, Intervention Characteristics and Process. With respect to Intervention Characteristics, the construct of Adaptability (i.e., the ability to adapt the intervention to the prevailing context and more specifically, integrate the innovation it into the prevailing workflow), was consistently identified to be important. At a broader level, the intervention needs to align with the prevailing value system (among key stakeholders), for it to be successfully integrated with the workflow. The fact that Intervention Characteristics was often identified as a barrier or facilitator alongside Inner Setting, speaks to the importance of leadership engagement in ensuring intervention adaptability within the organizational context, to facilitate implementation success.

Along these lines it is noteworthy, that whenever the Process domain was found to be relevant, it was almost always accompanied by Inner Setting. Broadly, this helps to understand how implementation “process” goes together with the organizational “structure” (captured by the Inner Setting domain). This is helpful to understand when one considers that the absence of leadership engagement in an intervention, is likely to be accompanied by a lack of stakeholder engagement and championing, reflecting a failure of implementation Process. On the other hand, very few articles reviewed found the Outer Setting to (e.g., reimbursement policies for telehealth) to be of significance in explaining implementation effectiveness. This in turn suggests that permanent removal of policy and regulatory barriers to telehealth reimbursement by itself, may not suffice for ensuring implementation effectiveness. Instead, effective telehealth implementation requires healthcare providers to gain a more comprehensive understanding of implementation science dynamics, including the complex inter-relationships among the CFIR Inner Setting domains (and related constructs) and telehealth implementation effectiveness.

Regarding descriptive characteristics, the breakdown of articles by healthcare domain showed that majority of CFIR applications to telehealth initiatives have focused on the Specialty Care domain (63%), and the dominant targeted condition within Specialty Care was Mental Health (tele-psychiatry). There were fewer telehealth initiatives devoted to improving care coordination during transitions, patient-centered care, and team-based care. With respect to targeted populations, there has been greater focus on adults (69%) and veterans (14%), compared to children and youth (4%), and with respect to technology areas, most initiatives have relied on use of both synchronous and asynchronous technology (55%). Lastly, with respect to service area, the vast majority of CFIR applications to telehealth initiatives have focused on healthcare delivery (clinical practice), (78%) while a much lower proportion have concurrently also used telehealth for medical education and population health promotion.

Implications for practice, policy, and future research

Findings by CFIR domain, suggest that simply focusing on issues impacting the Outer Setting, i.e., the removal of policy barriers including constraints associated with reimbursement and payment for telehealth services may not suffice in ensuring implementation-effectiveness. Instead, findings serve to underscore the predominant role of the Inner Setting domain (including leadership engagement, resources, measurement feedback, implementation readiness and implementation climate) in influencing implementation effectiveness. Findings also reveal that Process (stakeholder engagement and championing) related concerns are often accompanied by concerns associated with Inner Setting, implying that leadership engagement may be crucial in facilitating stakeholder engagement and cultivating champions for implementation.

Findings also point to the importance of interventional adaptability (integration into workflow) and individual characteristics, including perceptions and attitudes of individuals involved in influencing implementation success of telehealth initiatives. However, it would be relevant to note that like Process concerns, strategies for dealing with concerns associated with Intervention and Individual Characteristics often trace back to the Inner Setting, for example, one study found that leadership initiatives to strengthen relationships between stroke experts and ED providers helped to improve intervention adaptability, address resistant providers, and improve processes. From a practice perspective therefore, healthcare organizations need to understand these dynamics to be able to design and implement successful telehealth initiatives. From a policy perspective, both policy makers and advocacy groups, e.g., specialty society organizations such as the American Academy of Family Physicians, would be well advised to channel organized efforts and resources towards educating and training providers in implementation science dynamics to better prepare them for future success in telehealth implementation.

Regarding future research, the database search revealed several study protocols for telehealth implementation-effectiveness studies which could not be included in the review. Since these types of studies are frequently designed to provide outcome measures of implementation success, a follow-up review on the same topic in 5 years (to allow time for protocols to materialize into public studies), may help to better understand the determinants of implementation effectiveness, including the relationship between CFIR domains and success or failure of telehealth implementation. The trends in research growth on this topic may also align synergistically with the concurrent progress being made in conceptualizing outcomes for use with the CFIR, to provide an advanced understanding of the determinants of telehealth implementation-effectiveness [88]. In addition to a follow-up scoping review, a more mature evidence-base of outcome measures from completed implementation-effectiveness studies in the future, would have potential to provide a stronger foundation for systematic review efforts seeking to examine inter-relationships among a variety of CFIR domains & constructs within the telehealth implementation context, e.g., what is the relationship between implementation climate and effectiveness of telehealth implementation? Also, the findings related to descriptive characteristics of CFIR applications to telehealth implementation, by themselves, provide insight into gaps in the literature and potential avenues for future research growth, including the need for more CFIR applications to telehealth initiatives in primary care and acute care, as well as the need for telehealth initiatives that are focused on medical education and population health promotion, as opposed to a singular focus on clinical practice.

Strengths and limitations

A key strength of this scoping review is that it helps to address a gap in the literature related to the scope of knowledge gained thus far from CFIR applications to telehealth implementation initiatives. Another strength was that the review was guided by evidence-based criteria for scoping reviews developed by the Joanna Briggs Institute and the internationally accepted guidelines for scoping reviews outlined in the PRISMA-ScR checklist. A clear rationale for use of scoping review (vs other types of review techniques) is provided at the outset, and the research questions of the scoping review are directly aligned with the review’s broader objective. Also, consistent with the rationale for a scoping review, the review involved a comprehensive database search of 5 major academic databases for individual sources of evidence (i.e., eligible journal articles) on the topic of interest.

One limitation of this review, however, is that other avenues for literature searches were not leveraged, e.g., 1) internet searches to examine the “gray literature” (industry publications, unpublished manuscripts, conference papers), 2) contact with authors to identify additional articles, and 3) review of reference lists of selected articles to identify additional articles. Additionally, the review found few implementation-effectiveness studies on the topic, which in turn limits insights gained into determinants of telehealth implementation success based on the CFIR. Nevertheless, the database search revealed an increasing number of protocols for implementation-effectiveness on the topic published in 2020 and 2021, suggesting that the science on this topic may be gravitating more towards generating additional outcome measures. This also implies that a follow-up review on the same topic within the next 5 years may be helpful in gaining advanced and nuanced insights on the relationship between CFIR domains and telehealth implementation-effectiveness.

Conclusion

This scoping review reviewed 64 eligible articles to examine the scope of knowledge gained thus far from CFIR applications to telehealth implementation initiatives. The review found that most eligible articles were published in 2020 or later indicating that the science of CFIR applications to telehealth initiatives, has gained momentum during the pandemic. The review also found that most eligible articles were qualitative studies designed to examine barriers and facilitators to telehealth implementation through interaction (interviews, focus groups surveys etc.) with key stakeholders involved in implementation. By comparison, a very small proportion of articles were implementation- effectiveness studies that included outcome measures of intervention or implementation success. Similarly, very few articles used additional frameworks like the RE-AIM to supplement the CFIR to gain dual insights into implementation effectiveness and sustainability of telehealth initiatives. Regarding CFIR domains of influence, most included articles reported the Inner Setting domain to be of significance in influencing telehealth implementation effectiveness. By comparison, a very limited number of articles reported the Outer Setting to be of significance in influencing telehealth implementation. The review also found that most telehealth initiatives were undertaken in the specialty care domain, and that mental health/psychiatry was the most targeted condition. Most telehealth initiatives targeted adults (compared to children) and utilized both synchronous & asynchronous telemedicine technologies. The preponderance of articles also focused on utilizing telehealth for clinical practice, as opposed to medical education or population health.

A key takeaway is that to effectively design and implement telehealth initiatives, healthcare providers need to gain a thorough understanding of the inter-relationships between telehealth implementation effectiveness and CFIR domains, especially the CFIR Inner Setting domain and related constructs, including leadership engagement, resource availability, goals & feedback, and implementation climate. This suggests that policy advocacy groups and specialty societies, may need to place as much emphasis (if not more), on educating and training healthcare providers in their respective specialties on Inner Setting dynamics associated with telehealth implementation, in addition to advocating for better telehealth reimbursement policies, since the latter would only be relevant to addressing the Outer Setting domain of influence on telehealth implementation.