Introduction

The COVID-19 pandemic has transformed mental health care, creating increased demand for remotely delivered services. Emerging research has noted a 154% increase in utilization of telehealth (i.e., reception of care without an in-person visit) since the pandemic, and underscored its utility during periods of in-person service closures and physical distancing mandates. [1, 2] Examinations of the impacts of telehealth have found these services to be effective for the general population. [3] Reviews suggest that telehealth delivery of psychological treatments may be particularly useful as a cost-effective alternative to in-person psychotherapy and to supplement access for rural communities. [4, 5] One particular population that could benefit from telehealth services is military and veteran populations.

Evidence of effectiveness of iCBT

One type of telehealth service that has been promising is internet-based cognitive behavioral therapy (iCBT). Loosely described, iCBT involves delivery of cognitive behavioral therapy (CBT) through a computer, phone, or mobile device, often guided by a mental health professional. [6] iCBT has been demonstrated as effective in the treatment of mental health conditions, including posttraumatic stress disorder (PTSD) [8], depressive disorders [6, 9,10,11], anxiety disorders [6, 9, 11], substance use disorders [6], and insonmia [12] in the general population. Indeed, some have reported comparability between guided iCBT and in-person CBT. [6, 11]

While reviews generally agree that iCBT can be useful for treating mental health conditions, authors have highlighted several limitations that may affect interpretations of iCBT effectiveness and generalizability. First, variability in iCBT study design, such as allowance for concurrent treatments, length of intervention, presence of facilitation, delivery format, outcomes measured, and intended treatment targets may largely determine its overall effectiveness. Second, the degree to which iCBT may be tolerated by diverse patient populations remains unclear.

iCBT for military and veteran populations

Veteran and active service military members are considered a vulnerable population on account of elevated rates of mental health concerns and physical and mental health service utilization. [13] Military and veteran communities are distinguished by uniqueness in frequencies and complexities of trauma exposures, which increases risk for the development of mental health conditions. [13, 14] Further, this population exhibits distinctions in symptom presentation and comorbidities [15], and is often less responsive [16], or differentially responsive [17] to mental health treatments compared to general populations. Interestingly, research has found iCBT to be an effective treatment for insomnia [18] and PTSD [7] within military and veteran populations. However, the variability in iCBT study design, as stated above, remain unclear among this population. Specifically, in reviewing available meta-analyses conducted on internet-based interventions, only four articles included, or focused on, military populations. [7,8,9,10] However, none of the meta-analyses focused specifically on iCBT within a military population or examined effects across various demographics, characteristics of iCBT interventions, and mental and physical outcomes.

Aims and scope of review

In consideration of these factors, evaluations of iCBT for military and veteran populations is warranted. Specifically, pandemic-related restrictions and higher rates of mental health concerns coincide with increased telehealth treatment utilization. With the unique experiences and needs of military members and veterans, the effectiveness of iCBT for this population requires further exploration. To extend prior reviews, we sought to include diverse mental health outcomes and methodologically-divergent studies to examine factors that may differentially influence study effects. The goals of the current meta-analysis are to: (1) substantiate the effects of iCBT for military and veteran populations, (2) evaluate its effectiveness compared to control conditions (e.g., waitlist, treatment-as-usual [TAU], and active alternative interventions), and (3) examine potential factors that may influence the effectiveness of iCBT interventions, such as population characteristics, study design, and treatment delivery.

Methods

Search terms

This review was completed following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [19] reporting and Cochrane [20] review guidelines. The literature search was conducted using four databases: PsycInfo, Medline, Embase, and Proquest Dissertation & Theses on June 4, 2021, with no date restriction. See supplementary S1 for search string.

Inclusion and exclusion criteria

Our inclusion criteria included studies that: (1) were restricted to adult military or veteran populations, (2) incorporated iCBT (including computer, phone, mobile) as the primary treatment, and (3) evaluated mental health outcomes (e.g., PTSD, anxiety). Exclusion criteria included: (1) literature reviews, (2) qualitative studies, (3) study protocols, (4) studies that did not include a clinical or analogue population, and (5) studies with no measure of change on outcome variables (e.g., measure collected at pre-iCBT only).

Study selection

A total of 20 studies were included (see Fig. 1). Trained screeners (S.R. & S.S.) examined each study independently. Inter-rater reliability via percentage agreement was high at each phase (title/abstract [97.5%] and full text [95.0%]). Discrepancies were discussed in a group until consensus was reached among all authors; thus, final study selection had 100% inter-rater reliability. For each screening phase, SWIFT-Active Screener, a web-based review software, was used to screen all citations. [21].

Fig. 1
figure 1

PRISMA flow diagram [18]

Full size image

Data extraction

Demographics, study data, and predictor information were extracted from each study (see S1). Demographic information extracted were: (1) age (i.e., mean age of the study sample), and (2) reported gender (i.e., whether participants were primarily men, women, or mixed [multiple genders in sample]). Study data information extracted were: (1) study condition (iCBT or control: active [e.g., psychoeducation]; treatment as usual [TAU]; or waitlist), (2) sample size, (3) pre- and post-intervention mean and standard deviations across study conditions on all outcomes, (4) target outcome were distinguished by primary (e.g., PTSD targeted treatment and PTSD outcome) or secondary (e.g., PTSD targeted treatment and substance use outcome) outcomes. In the event multiple measures were taken for the same categorical outcomes (e.g., two measures of PTSD), the outcome with the largest effect size were kept. Predictor information extracted were: (1) the type of outcome measured (i.e., anxiety (e.g., Beck Anxiety Inventory [BAI] [22]), depression (e.g., Beck Depression Inventory [BDI] [23]), health and functioning (e.g., Brief Inventory of Psychosocial Functioning [B-IPF] [24]), PTSD (e.g., PTSD Checklist [PCL] [25]), quality of life (e.g., Quality of Life Enjoyment and Satisfaction [Q-LES-Q-SF] [26]), substance use (e.g., cigarettes smoked), and behavior (health behavior related to scales or activities; e.g., hours slept); (2) format (i.e., web, computer, phone, mobile app); (3) length of intervention (in weeks); (4) delivery (self-guided or facilitated); (5) concurrent treatment allowed (yes or no); and (6) dropoutrateFootnote 1 (%).

Data Analysis

The meta-analyses were conducted using the Comprehensive Meta-Analysis (CMA) software version 3. [27] All demographics and study data and were inserted into CMA. In addition, a correlation of 0.77 (used in a previous meta-analysis with similar design [28]) was inserted to account for between-subjects variance in within-subjects designs. For all studies, effect direction was selected to represent whether the measures of change on outcomes were in line with our hypotheses (i.e., outcomes improved over time; positive) or not (i.e., outcomes did not improve over time; negative).

Main analyses examined the combined effects of iCBT interventions and the relative effects of iCBT to control conditions across outcome measures. Hedges’ g effect size was used. Effect sizes were interpreted as follows, 0.41 for a minimum effect representing a practically significant effect, 1.15 for a moderate effect, and 2.70 for a strong effect. [29] Further, a continuous meta-regression was used. Following the main analyses, subsequent predictor analyses were examined across outcomes. Sub-group analyses were conducted if there are a minimum of five samples. [30] Lastly, publication bias was assessed via visual inspection of the funnel plot, Egger’s Regression test, Durval and Tweedie’s trim-and-fill, and classic fail-safe.

Results

Study characteristics

Our meta-analysis included 20 studies with 91 samples of data from veterans and active military members. Across samples, the gender distribution was of men onlyFootnote 2 (n = 54) and mixed gendersFootnote 3 (n = 37), with no samples consisting of women only (n = 0). The mean age across samples was 43.66 (ranging from 28 to 67 years; see Table 1 for raw data and Table S1 for predictor information). Across samples, the distribution of samples was as follows: n = 56 for iCBT, and n = 35 for controls (active, n = 14; TAU, n = 17; and waitlist, n = 4). The pooled sample sizes across studies included data from 1614 individuals (iCBT [n = 872] and controls [active, n = 141; TAU, n = 478; and waitlist, n = 123]).

Table 1 Data of Included Studies
Full size table

Overall analyses

First, we examined the effects of iCBT and control interventions on all outcomes. A total of 91 samples were entered into the mixed, random effects model to determine the pooled effect of iCBT interventions relative to control groups on primary and secondary outcomes. Results indicated a significant difference across groups, Q(3) = 50.30, p < .001. The pooled effect size showed a small but meaningful effect for iCBT, g = 0.54, SE = 0.04, 95% CI (0.45, 0.62), Z = 12.32, p < .001. These effects were robust relative to the effects of controls - TAU, g = 0.26, SE = 0.05, 95% CI (0.17, 0.36), Z = 5.25, p < .001, and waitlist, g = -0.04, SE = 0.08, 95% CI (-0.19, 0.11), Z = -0.56, p = .58. However, for studies with an active alternative intervention, the pooled effects were observed to be slightly larger in magnitudeFootnote 4, relative to the iCBT, g = 0.59, SE = 0.13, 95% CI (0.34, 0.84), Z = 4.61, p < .001. Across all samples, effects were found to be considerably heterogeneous (I2 = 87.96), Q(90) = 747.62, p < .001.

Effects of iCBT on primary outcomes across outcomes measured

Next, we examined the effects of iCBT interventions on the primary outcomes of each study. Across all primary outcomes measured, a total of 25 samples were entered into the mixed, random effects model to determine the pooled effect of iCBT intervention. The effect size for primary outcomes showed a small but meaningful effect for iCBT, g = 0.50, SE = 0.05, 95% CI (0.41, 0.59), Z = 10.46, p < .001 (see Fig. 2). Subgroup analyses (between groups with adequate sample representation) on examining the primary outcomes by the types of outcomes measured (PTSD and depression) found no significant differences across groups, Q(1) = 0.49, p = .48. Specifically, the effects were relatively similar between depression [g = 0.55, SE = 0.15, 95% CI (0.27, 0.84), Z = 3.79, p < .001] and PTSD [g = 0.44, SE = 0.06, 95% CI (0.33, 0.55), Z = 7.70, p < .001]. The remaining outcomes (anxiety, behavior, health and functioning, quality of life, and substance use) were not examined due to low sample size.

Fig. 2
figure 2

Forest Plot of Primary Outcome across Outcomes Measured for iCBT Group

Full size image

Effects of iCBT on secondary outcomes across outcomes measured

Next, we examined the effects of iCBT intervention on the secondary outcomes of each study. Across all outcomes measured, a total of 31 samples were entered into the mixed, random effects model to determine the pooled effect of iCBT intervention. The effect size for secondary outcomes showed a small but meaningful effect for iCBT, g = 0.48, SE = 0.04, 95% CI (0.41, 0.55), Z = 12.91, p < .001 (see Fig. 3). Subgroup analyses (between groups with adequate sample representation) on examining the secondary outcomes by outcome measured (depression, health and functioning, and quality of life) found no significant differences across groups, Q(2) = 0.56, p = .76. Specifically, depression showed a small but meaningful effect, g = 0.45, SE = 0.09, 95% CI (0.27, 0.62), Z = 4.96, p < .001. However, effects were non-meaningful for health and functioning (g = 0.38, SE = 0.07, 95% CI (0.25, 0.51), Z = 5.52, p < .001), and quality of life (g = 0.36, SE = 0.09, 95% CI (0.19, 0.53), Z = 4.19, p < .001). The remaining outcomes (anxiety, behavior, PTSD, and substance use) were not examined due to low sample size.

Fig. 3
figure 3

Forest Plot of Secondary Outcome across Outcomes Measured for iCBT Group

Full size image

Meta-regression

A continuous meta-regression was conducted using the moments method as the estimation framework to determine their influence on the pooled effects within iCBT samples [27]. Continuous predictors included age, dropout rate and length of intervention. This model did not account for significant variance within the data. Within Model 1, only length of intervention significantly predicted the study variance, with shorter lengths of interventions having a larger magnitude of effects (see Table 2).

Table 2 Meta-Regression Test Statistics
Full size table

Effects of iCBT on Study Outcomes across categorical predictors

Overall

We examined whether the effects of iCBT (k = 56) varied as a result of categorical predictors (concurrent treatment, delivery, format, gender, outcomes measured) via Q-statistics. [30] Analyses with all outcomes in iCBT group found that concurrent treatment resulted in different effect sizes across sub-groups, while delivery, format, gender, and outcomes measured did not result in different effects across outcomes. Specifically, studies that extended participant eligibility to allow concurrent treatments yielded a small positive effect, while studies that did not allow concurrent treatment did not meet the effect threshold (see Table 3).

Table 3 Meta-Analysis Test Statistics on Sub-Group Analyses Across Predictors for All Outcomes in iCBT Group (k = 56)
Full size table

Primary outcomes

We examined whether the effects of iCBT on primary outcomes (k = 25) varied across categorical predictors (concurrent treatment, delivery, format, and gender) via Q-statistics. Analyses with primary outcomes in iCBT groups found that concurrent treatment and gender resulted in different effect sizes across sub-groups, while delivery and format did not result in different effects across outcomes. Specifically, studies that extended participant eligibility to allow concurrent treatments yielded a marginally stronger effect relative to studies that did not allow concurrent treatment. For gender, samples containing mixed-gender populations yielded more robust effects relative to samples that only contained males (see Table 4).

Table 4 Meta-Analysis Test Statistics on Sub-Group Analyses Across Predictors for Primary Outcomes in iCBT Group (k = 25)
Full size table

Secondary outcomes

We examined whether the effects of iCBT on secondary outcomes (k = 31) varied across categorical predictors (concurrent treatment, delivery, format, and gender) via Q-statistics. Analyses with secondary outcomes in iCBT groups found that concurrent treatment and format resulted in different effect sizes across sub-groups, while delivery and gender did not result in different effects across outcomes. Specifically, studies that extended participant eligibility to allow concurrent treatments met the threshold for practically significant effects. For format, studies that used computers yielded higher effects than those that used mobile applications (see Table 5).

Table 5 Meta-Analysis Test Statistics on Sub-Group Analyses Across Predictors for Secondary Outcomes in iCBT Group (k = 31)
Full size table

Publication Bias

Visual inspection of the funnel plot indicates that a number of studies are clustered towards the center, with a small number scattered to the right and some imputed studies to the left, suggesting publication bias (see Fig. 4). Egger’s regression test detected a significant asymmetry, (B0) = 1.95, t(89) = 2.84, p = .005. For Duval and Tweedie’s (2000) trim-and-fill, under the random-effects model, the point estimate and 95% confidence interval for the combined studies is 0.46 (0.39, 0.52). Using trim-and-fill, the imputed point estimate is 0.34 (0.25, 0.41). Classic fail-safe N revealed that it would take 27,431 ‘null’ studies in order for the combined 2-tailed p-value to exceed 0.05. In other words, there would need to be 301.4 missing studies for every observed study for the effect to be nullified. Taken together, analyses examining publication bias suggest the presence of some biases in that study effects may be over-represented and skewed towards positive, but that these biases, if any, should be marginal in nature.

Fig. 4
figure 4

Funnel Plot across all included studies

Full size image

Discussion

The current meta-analysis sought to determine the relative effectiveness of iCBT for military and veteran populations. Results found iCBT to be more effective across a range of outcome measures when compared against waitlist controls and TAU conditions, but not when compared to active alternative interventions. The pooled effect sizes of iCBT interventions were small, but of practical significance and meaningful importance. [29] Internet-based CBT were effective for a number of mental health conditions, with results robust for both primary (PTSD and depression) and secondary (depression) outcomes.

Effectiveness of iCBT for military populations

The effects of iCBT treatments were robust across a number of primary and secondary study outcomes, and comparable to prior meta-analyses of iCBT treatments conducted with non-military samples on depression (d = 0.41) [11] and PTSD (d = -0.60) [42]. This underscores the robustness of iCBT treatments. In addition, our results (g = 0.54) were similar in magnitude to the previous findings on CBT treatments for depression (g = 0.71) [43] and iCBT for PTSD (g = 0.72) [8] among the general population. Although slightly smaller in size by comparison, the relative comparability of these findings establishes the validity of CBT treatments generally, and its adaptability to diverse modes of delivery, including internet, app, and telephone-based deliveries.

Yet, the effects of iCBT were not superior when compared against groups receiving alternative interventions. This, in part, may be due to variability in treatments considered “active alternative controls”. In our sample of studies, these groups ranged from psychoeducation[31]–[32], supportive counseling [33], in-person therapy [18], and group counseling [34]. The size of the effects within this category are not uniform or homogenous. Instead, the effects are more indicative of diverse treatments received by participants. Indeed, prior reviews note similar considerations, underscoring the superiority of CBT to be highly dependent on the nature of the control conditions tested against. [35].

Considerations for the use of iCBT with military populations

In addition to the pooled effects, iCBT interventions were found to be differentially effective based on several factors. For participants receiving other treatments concurrently with iCBT, such as medications and adjunctive therapies, improvements in symptoms were of greater magnitude relative to those not receiving concurrent treatments. Although this may not be surprising given the cumulative effects of treatments, it also suggests that iCBT may be a complementary treatment when used with others. This has important implications for military and veteran communities in particular, due to the higher prevalence of physical and mental health comorbidities. [36] When engaging in treatment planning, the option to add a course of iCBT, if appropriate, may have advantageous effects. Future research could explore the additive effects of iCBT, whether as the primary or supplementary form of therapy.

While the format in which iCBT was presented (i.e., computer versus mobile and web-based applications) leads to differences in overall effects, the overall effect sizes of were comparable. Meanwhile, in accordance with prior research observing differences between facilitated versus self-guided iCBT programs [37], the current meta-analysis also found significant differences with small effects, though these effects were similar in size. However, these differences may be driven largely by the inclusion of CBT for insomnia, which are exclusively self-guided and have the largest effect sizes.

Results found that iCBT interventions that included both men and women participants were found to have larger effects than male only participants. Historically, diverse genders have been excluded from service, and underrepresented in military research. [38] The increase in effects from mixed gender studies may be reflective of more recent research, and may further be confounded with advancements in technology and improvements in program design, both of which may result in a more efficacious treatment program. Despite these speculations, prior research has underscored gender differences in military and veteran samples following mental health treatments. [39, 40] Finally, shorter iCBT interventions were generally found to have greater magnitude of effects. [44] This may be due to shorter programs being more manageable with less participant attrition. This may also explain the lack of relationship observed between attrition rate and study effects. Thus, future studies should examine these factors in relation to iCBT intervention effectiveness.

Taken together, these considerations provide the deeper contexts for understanding the effectiveness of iCBT for military and veteran populations. While findings are positive, more research is needed to better evaluate the relative weights of various population and intervention characteristics, such as examinations of fidelity [41], treatment adherence, and individual motivations. Importantly, these factors point to considerations and areas of negotiation when engaging in treatment planning and offer insights into the interpretations of treatment efficacy. Finally, results from the current meta-analysis should be considered in lieu of the presence of potential publication biases that may skew the effects of iCBT. As technologies continue to evolve and demands for telehealth and remote delivery of services amplify, we expect that both the quality of evidence and sophistication of treatment will increase over time. As such, continued research is needed to better position the use of iCBT for military personnel and veterans and other high-risk populations with unique service needs.

Conclusions and future directions

With the changing landscape of health service delivery brought on by the pandemic, investigations into telehealth delivery of services may be more relevant than ever before. The current meta-analysis is one of few to investigate the utility of iCBT for military personnel and veterans. Together, findings underscore the potential utility and effectiveness of iCBT interventions for military and veteran populations. Specifically, iCBT may be optimized under conditions of inclusive enrollments (e.g., mixed genders, allowance for concurrent treatments). Clinicians may feel comfortable recommending access to iCBT treatments for military and veteran populations, especially as an adjunctive, add-on, or while awaiting in-person treatment(s). Meanwhile, future research should explore the benefits and obstacles of iCBT through rigorous investigations of the barriers and facilitators of successful deliveries of services. Further, while results from this meta-analysis was not pre-registered (e.g., PROSPERO), future reviews may wish to consider this approach.