Skip to main content
Download PDF

Clinical Predictors of Engagement in Teleintegrated Care and Telereferral Care for Complex Psychiatric Disorders in Primary Care: a Randomized Trial

Journal of General Internal Medicine (2022)Cite this article

Abstract

Background

Telepsychiatry Collaborative Care (TCC) and Telepsychiatry/Telepsychology Enhanced Referral (TER) expand the reach of specialty mental health services to underserved populations.

Objective

Assess clinical predictors of treatment engagement for complex psychiatric conditions in TCC—in which remote specialists consult with primary care teams via an onsite care manager who also provides brief psychotherapy—and TER, in which remote specialists provide direct telehealth treatment.

Design

A randomized pragmatic trial from twenty-four primary care clinics without onsite psychiatrists or psychologists.

Participants

A total of 1,004 adult patients screened positive for posttraumatic stress disorder (PTSD)and/or bipolar disorder were randomized to receive TCC or TER for 1 year.

Main Measures

Psychotherapy engagement was measured by the number of sessions completed, and pharmacotherapy engagement by the medication adherence item from the Schizophrenia Care and Assessment Program Health Questionnaire (SCAP-HQ).

Key Results

Engagement in TCC psychotherapy visits was greater compared to TER. There was no association between the PTSD symptom severity and treatment engagement. The internal state scale (ISS) activation subscale, an indicator of mania, was associated with reduced odds of initiating psychotherapy (odds ratio [OR] = 0.70; 95% CI, 0.59 to 0.84) but not the number of sessions attended once psychotherapy started. The Drug Abuse Screening Test-10(DAST-10) score was associated with receipt of fewer psychotherapy sessions (incidence ratio rate [IRR] = 0.88; 95% CI, 0.81 to 0.95). The number of physical health comorbidities was associated with greater engagement in psychotherapy (IRR = 1.11, 95% CI, 1.03 to 1.19) and pharmacotherapy (OR = 1.54; 95% CI, 1.27 to 1.87). None of the findings varied by intervention group.

Conclusions

Both teleintegrated and telereferral care offer an opportunity to treat patients with complex psychiatric conditions. While there was no difference in clinical characteristics predicting engagement, onsite care managers engaged patients in more psychotherapy sessions than remote therapists.

Trial Registration

ClinicalTrials.gov Identifier: NCT02738944

Due to limited availability of mental health specialists in underserved areas, primary care practices often treat patients with complex psychiatric conditions1,2,3, including posttraumatic stress disorder (PTSD) and bipolar disorder (BD) which respectively affects up to 20%4,5 and 4%3 of their population. Bipolar disorder and PTSD frequently co-occur6 and have a high likelihood of psychiatric comorbidities7,8,9, including substance use disorders10,11, resulting in greater symptom burden and lower quality of life6,8, premature mortality9,12, and higher disengagement from care10,13,14,15. A review of empirically supported treatments for PTSD yielded attrition rates as high as 60% for veterans16 and 57% for the general population17. Premature dropout rates for pharmacotherapy for BD range between 30 and 40%13,18.

Telepsychiatry Collaborative Care (TCC) and Telepsychiatry/Telepsychology Enhanced Referral (TER) expand the reach of specialty mental health services to underserved populations19,20,21,22. TCC uses a team-based approach with a lead primary care provider, support from a care manager and consultation from a telepsychiatrist for treatment recommendations21,22. TER allows for live synchronous audio-visual interactions between patients and mental health clinicians for delivery of care20,21. Telepsychiatrists prescribed medications directly to the patient and telepsychologists deliver psychotherapy. Both models are evidence-based practices for depressive and anxiety disorders20,21 but little is known regarding the engagement of patients with complex psychiatric conditions. Prior studies showed PTSD severity and the symptom cluster of avoidance to be associated with patient dropout in specialty mental health care 23,24,25. Studies of patients with BD demonstrated a switch to hypomania negatively impacts treatment engagement, particularly when associated with substance use8,18. Whether PTSD and BD symptomatology and comorbidities are associated with engagement in TCC or TER, or differences in engagement between models, could inform individual patient care decisions, treatment guidelines, and service implementation for millions of Americans increasingly relying on telehealth services26.

We assessed clinical predictors of psychotherapy and pharmacotherapy engagement using data from the Study to Promote Innovation in Rural Integrated Telepsychiatry (SPIRIT), a large comparative effectiveness trial of TCC (an integrated care model), and TER (a virtually co-located, but not integrated, model), for the management of PTSD and BD in underserved primary care settings27. Improved outcomes were observed in both intervention groups, and there was overall greater engagement in psychotherapy in the TCC arm28. This secondary analysis focuses on potential differences in clinical predictors of engagement in treatment across and between arms to inform patient-level clinical decision-making. We hypothesized that PTSD avoidance symptoms would be associated with decreased engagement across both arms. We also conducted exploratory analyses to assess whether overall PTSD symptoms, mania, and clinical covariates were associated with treatment engagement and whether the effects differed by care model. We hypothesized that negative predictors of engagement would have stronger effects in the TER arm compared to the TCC arm, as the latter offers more proactive efforts to engage and activate patients through the primary care-based care manager.

METHOD

Study Design

SPIRIT is a randomized pragmatic comparative effectiveness trial (Fig. 1). Randomization to TCC or TER was stratified by screening results (PTSD-only group versus BD-only or BD and PTSD group). Virtually, all participants (92%) who screened positive for BD also screened positive for PTSD. If participants randomized to TER had not engaged in the first 6 months of the trial (having ≤ 2 interactive video encounters), they were randomized a second time to receive a phone call from a psychologist encouraging usage of the telehealth services (Fig. 1). The enrollment target was 1,000 patients.

Figure 1
figure 1

SPIRIT Sequential Multiple Assignment Randomized Trials (SMART).

Full size image

Study Settings and Participants

The SPIRIT trial was conducted in 24 clinics associated with 12 Federally Qualified Health Centers (FQHCs) across three states in the USA (Arkansas, Michigan, and Washington). FQHCs are community-based health centers that provide preventive and primary care services to rural, underserved areas and minority communities regardless of patients’ ability to pay29. FQHC clinics were recruited to participate if they had no psychiatrist or psychologist on site. The study protocol was approved and monitored by the Institutional Review Boards (IRB) at the participating state medical schools (University of Arkansas for Medical Sciences, University of Michigan, and University of Washington) and participating FQHCs entered into a reliance agreement to receive oversight from these IRBs. The SPIRIT trial was developed in close collaboration with our Consumer Advisory Board and Policy Advisory Board27.

Eligible patients were 18 years or older, not currently prescribed a psychotropic medication by a mental health specialist, English-speaking, and had positive screen on the abbreviated PTSD Checklist (PCL-6 ≥14) for PTSD30and/or on the Composite International Diagnostic Interview 3.0 (CIDI) (positive stem question responses and score of ≥8) for BD31. Patients with any physical and mental health comorbidities were included as well as patients with safety concerns according to a safety protocol put in place.

Study Recruitment and Screening

The PCL-6 and CIDI were used for recruitment purposes only. To minimize screening burden, only patients who screened positive for depression on the Patient Health Questionnaire32 (PHQ-9 score ≥ 10) were subsequently screened by clinic staff for PTSD and BD to enter the study. A total of 2,464 patients screened positive for PTSD on the PCL-6and/or BD on the CIDI. Among those, 1,931 patients were found eligible and 1,214 consented to participate in the SPIRIT trial. A total of 1,004 participants completed the baseline research assessment and were randomized to an intervention group (TCC vs TER). Patients were recruited from November 16, 2016, to June 30, 2019.

Study Interventions

TCC telepsychiatrists and TER telepsychiatrists and telepsychologists were offsite, university affiliated, and credentialled to practice at the FQHC. Both TCC and TER psychiatrists conducted an initial assessment through telehealth to assign clinical diagnoses and develop a treatment plan. In TER, the telepsychiatrists prescribed all psychotropic medications and provided referrals to telepsychologists. Telepsychologists delivered cognitive processing therapy (CPT) for PTSD, cognitive behavioral therapy (CBT) for BD, and either one or both for patients with both PTSD and BD. In the TCC arm, the onsite primary care physicians prescribed psychotropic medications recommended by the offsite telepsychiatrists who discussed individual cases with the onsite care managers. The care managers were registered nurses and licensed clinical social workers who were trained to deliver evidence-based behavioral activation (BA)33.

Definition of Engagement

We measured medication engagement in the past 2 weeks by the medication adherence item from the Schizophrenia Care and Assessment Program Health Questionnaire (SCAP-HQ)34 administered during the 12-month follow-up survey. Those who responded that they never missed or missed only a couple times their medications on the SCAP-HQ item were considered engaged in pharmacotherapy. Those who refused a prescription or not taking their medications as prescribed were considered not engaged. We measured psychotherapy engagement by the number of visits completed with the offsite telepsychologists or onsite care managers.

Clinical Predictors of Engagement

Psychometrically validated scales were administered to participants at baseline and 6 and 12 months later. This included the PTSD Checklist for DSM-5(PCL-5) for PTSD4; the 20-item Hopkins Symptom Checklist Depression Scale (SCL-20) for depression35; the Drug Abuse Screen Test (DAST-10) for drug use36; the Alcohol Use Disorders Identification Test (AUDIT-C) for alcohol misuse37; and the Depression Outcomes Module Comorbidity Checklist for physical comorbidities38. We used the Internal State Scale (ISS) for BD mood states. It comprises four subscales including the Activation subscale which identifies a manic or mixed state39,40.

Data Analysis

Psychotherapy engagement was modeled with a zero-inflated negative binomial model. The zero-inflated model decomposes engagement into two related outcomes with separate parameter estimates: a logistic regression coefficient for the binary zero/nonzero process, which models whether patients initiated any psychotherapy, and a negative binomial coefficient for the distinct count process, which models the number of sessions they attended. Pharmacotherapy engagement was modeled as a binary outcome using a logistic regression.

We examined hypotheses stepwise using a series of nested models. We tested whether PTSD symptom severity was predictive of treatment engagement by modeling each respective outcome using the PCL-5 total score as the only predictor. To examine whether the effect of avoidance varied from other PTSD symptom domains, we used likelihood ratio tests to compare the model with the overall PCL-5 score to a model with separate cluster-specific scores. We included an interaction term to assess whether PTSD symptoms had differential effects by treatment arm. We then explored the predictive value of the additional clinical characteristics by adding these to the model. These series of nested models were all fitted to the same size of dataset for likelihood ratio tests, which is 981 out of 1004. Twenty-three cases with missing data were excluded. We used the Benjamini-Hochberg41 adjustment with the false discovery rate set to 5% to adjust for multiple comparisons.

RESULTS

Study Participants and Screening results

Participant Characteristics

A total of 1,004 participants were randomized to the TCC arm (n=506) and to the TER arm (n=498). In TER, 228 (46.0%) did not engage and were re-randomized to receive additional phone contacts. This adaptive component of the TER intervention was not effective and therefore was not factored into this analysis. Study participants had a mean age 39.6 years (SD 13.0) and 70% were female and 34% were racial/ethnic minorities. Only 31% were employed full- or part-time, and 62% lived below the 2016 Federal Poverty Level. Medicaid and Medicare beneficiaries accounted for 68% and 24% respectively. The sample had worse than average mental health-related quality of life as measured by the Veterans RAND 12-item Health Survey Mental Health Composite Score (MCS)42(Table 1).

Table 1 Baseline Characteristics of Patient Enrolled in SPIRIT Trial
Full size table

Screening Results

About two-thirds of the study sample (n=637, 63%) screened positive for PTSD only and about one-third (n=367, 37%) screened positive for BD. Nearly all of those who screened positive for BD (92%) also screened positive for PTSD.

Participant Clinical Characteristics

The mean score on the PCL-5 was 47.75 (SD 17.7) suggesting moderate PTSD severity and on the SCL-20 was 2.44 (SD 0.7) indicating that patients were moderately to quite a bit bothered by depressive symptoms. The mean ISS score was 179.44 (SD 121.0), with 53.5% meeting criteria for mania or a mixed state. The AUDIT-C showed no alcohol misuse for 80% of patients. The DAST-10 revealed on average a low to moderate level of problems related to drug abuse. The mean number of physical health comorbidities was 4 (SD 2.7) with high prevalence for repeated problems with neck, back, or spine (62%); migraine (46%); and arthritis or any kind of rheumatism (43%) (Supplementary Table 3).

Psychotherapy Engagement

The number of psychotherapy encounters was 60% greater in the TCC intervention group compared to the TER group. Of the TCC patients, 79.3% had ≥1 BA encounter and averaged 9.6 encounters (SD=7.7). Of the TER patients, 45% had ≥1 telepsychology encounter and averaged 6.4 encounters (SD=4.4). The PCL-5 cluster-specific scores examined separately (intrusion, avoidance, negative alterations in cognitions and mood, and alterations in arousal and reactivity) did not have differential effects on psychotherapy engagement and did not improve the model fit compared to using the total PCL-5 score (χ2(6) = 7.65, p = .26; supplementary Table 4). Therefore, we used the PCL-5 total scores in subsequent models. Overall, the PCL-5 severity score was not significantly associated with psychotherapy initiation, or the overall number of sessions attended, and the association did not vary by intervention group (χ2(2) = 3.03, p = .21). Including additional clinical covariates, such as the SCL-20, ISS activation, DATS-10, AUDIT-C, and physical comorbidities, provided a significantly better fit to the data than excluding them (χ2(10) = 51.83, p < .001), but did not significantly impact the main findings. The ISS activation, indicative of manic or a mixed state, was associated with psychotherapy initiation but not the number of sessions attended once patients initiated treatment. A one standard deviation increase in the ISS activation subscale was associated with reduced odds (odds ratio = 0.70; 95% CI, 0.59 to 0.84) of engaging in psychotherapy. A one standard deviation increase in the DAST-10 score was associated with attending 12% fewer sessions (incidence rate ratio = 0.88; 95% CI, 0.81 to 0.95) and a one standard deviation increase in the number of physical comorbidities was associated with attending 11% more sessions (incidence rate ratio = 1.11; 95% CI, 1.03 to 1.19) (Table 2).

Table 2 Prediction Models for Psychotherapy Engagement and Pharmacotherapy Engagement
Full size table

Pharmacotherapy Engagement

Pharmacotherapy engagement increased over time, with no differences between intervention groups. The PCL-5 cluster-specific scores examined separately showed no differential associations with medication adherence (χ2(3) = 1.79, p = .62; supplementary Table 5). PTSD symptom severity according to total PCL-5 score was not associated with medication adherence and the null association did not vary by intervention condition (χ2(1) = 2.62, p = .11). Including additional clinical covariates provided a better fit to the data than the model excluding them (χ2(5) = 25.98, p < .001; supplementary Table 4), but did not significantly impact findings. A one standard deviation increase in physical comorbidities was associated with significantly increased odds of medication engagement (odds ratio = 1.54; 95% CI, 1.27 to 1.87). No other significant associations were found (Table 2).

Exploratory Post Hoc Tests

To better understand the results, we examined whether associations between the ISS activation and number of physical comorbidities with psychotherapy engagement and medication adherence varied by intervention group. All interaction effects were non-significant.

DISCUSSION

This study examined clinical predictors of treatment engagement for complex psychiatric conditions in primary care clinics. To our knowledge, SPIRIT is the largest mental health trial ever conducted in primary care clinics serving rural populations. Our study had minimal exclusion criteria and enrolled a medically complex and socio-demographically diverse sample. Contrary to our hypothesis, clinical characteristics that were expected to negatively impact engagement in pharmacotherapy or psychotherapy did not have stronger effects in the TER arm compared to the TCC arm despite the overall greater level of psychotherapy engagement in the TCC arm. TCC was able to better engage patients in psychotherapy visits via the onsite care manager and sustained proactive outreach. The differences in outreach practices between arms did not impact engagement. Both models can be offered based on primary care practice resource availability. TCC may be more favorable from a population health perspective insofar as it efficiently reaches and links more patients with mental health services compared to TER.

We hypothesized that patients with a higher level of avoidance would be less likely to engage in psychotherapy as indicated in previous studies23,24,25. Avoidance, a core symptom cluster of PTSD, can impair functioning and lead to isolation to escape unpleasant triggering memories25. We found no benefit of considering avoidance separately from overall PTSD symptom severity. Neither avoidance nor total PTSD symptom severity was associated with treatment engagement, and thus, our findings suggest neither should be used to drive clinicians’ decisions to initiate treatment for patients with PTSD within integrated or referral telehealth care models.

In exploratory analyses, manic symptom severity according to the ISS activation score was the only negative predictor of psychotherapy initiation. This finding may be in part due to clinician prioritization of treatment embedded in a shared decision-making process. Given the biological underpinning of BD, treatment guidelines for mania and mixed states focus generally on pharmacologic interventions43,44 which could result in deferring the initiation of psychotherapy, despite the evidence for effectiveness45,46. Lower psychotherapy initiation among those with manic symptoms could also be due to diminished insight, motivation, or level of organization to engage in psychotherapy44,46. Our finding calls for intensification of engagement efforts through combined pharmacotherapy and psychosocial interventions for patients in a current manic episode. It also warrants further exploration of clinician prioritization of mental health services for manic patients within specific telehealth care models.

Substance use disorders are highly comorbid with PTSD and BD10,11. We found the DAST-10 score was associated with receipt of fewer psychotherapy sessions. Studies conducted with community-based samples47 and in addiction programs48,49 display a similar trend and point to drug cravings and negative emotions as clinical characteristics associated with disengagement from mental health care. By contrast, alcohol use according to the AUDIT-C did not predict engagement which could be due to primary care-based providers having more experience engaging patients on their alcohol use compared to drug use. Also, the psychotherapy provided in our study did not specifically integrate behavioral approaches to substance use, which is a limitation that future implementation of these telehealth care models should consider addressing.

Individuals with PTSD and mood disorders have a greater likelihood of suffering from the most prevalent physical health conditions such as cardiovascular, endocrine, and pain-related conditions7,12,50. This health disparity tends to be greater in rural areas compared to metropolitan sectors 51. In this trial, patients had an average of four physical comorbidities including high rates of pain-related illnesses. We found a favorable correlation between the cumulative burden of physical comorbidities and likelihood of psychotherapy and pharmacotherapy engagement. This is consistent with other studies conducted in primary care clinics with patients with PTSD and mood disorders 50,52,53. The increased contact with mental health services is critical, particularly for patients with pain-related conditions considering their negative impact on mental health, substance use, and global functioning.

Our results should be appreciated with the following limitations in mind. Firstly, PTSD is highly prevalent in our sample with 92% of those who screened positive for BD also screened positive for PTSD. This is representative of the participating FQHCs in three states. Secondly, psychotherapy was accessed differently in each arm, with TER requiring a referral from a psychiatrist; nevertheless, we did not find any difference between arms with respect to clinical predictors of engagement. Thirdly, we used the psychometric scales administered pre-randomization as predictors of engagement rather than the psychiatric diagnoses assigned by clinicians after randomization to ensure we capture both engaged and non-engaged patients.

CONCLUSION

Both teleintegrated and telereferral care offer an opportunity to treat patients with chronic and complex psychiatric conditions with otherwise limited access to mental health specialists. Our results suggest mania and substance use, but not PTSD symptom severity, may require additional intervention to support engagement while medical comorbidities may foster care-seeking behaviors. While there was no difference in clinical characteristics predicting engagement between models, onsite care managers engaged patients in more psychotherapy sessions than remote therapists.

References

  1. Olfson M, Kroenke K, Wang S, et al. Trends in office-based mental health care provided by psychiatrists and primary care physicians. J Clin Psychiatry. 2014 Mar;75(3):247-53.

    PubMed  Google Scholar 

  2. Wang PS, Demler O, Olfson M, et al. Changing profiles of service sectors used for mental health care in the United States. Am J Psychiatry. 2006 Jul;163(7):1187-98.

    PubMed  PubMed Central  Google Scholar 

  3. Cerimele JM, Chwastiak LA, Dodson S, Katon WJ. The prevalence of bipolar disorder in general primary care samples: a systematic review. Gen Hosp Psychiatry. 2014;36(1):19-25.

    PubMed  Google Scholar 

  4. Spoont MR, Williams JW, Kehle-Forbes S, et al. Does This Patient Have Posttraumatic Stress Disorder? Rational Clinical Examination Systematic Review. JAMA. 2015;314(5):501–510.

    CAS  PubMed  Google Scholar 

  5. Liebschutz J, Saitz R, Brower V, et al. PTSD in urban primary care: high prevalence and low physician recognition. J Gen Intern Med. 2007 Jun;22(6):719-26.

    PubMed  PubMed Central  Google Scholar 

  6. Cerimele JM, Bauer AM, Fortney JC, et al. Patients with co-occurring bipolar disorder and posttraumatic stress disorder: a rapid review of the literature. J Clin Psychiatry. 2017;78(5):e506-e514.

    PubMed  Google Scholar 

  7. Hossain S, Mainali P, Bhimanadham NN, et al. Medical and Psychiatric Comorbidities in Bipolar Disorder: Insights from National Inpatient Population-based Study. Cureus. 2019;11(9):e5636.

    PubMed  PubMed Central  Google Scholar 

  8. Bauer MS, Altshuler L, Evans DR, et al. Veterans Affairs Cooperative Study #430 Team. Prevalence and distinct correlates of anxiety, substance, and combined comorbidity in a multi-site public sector sample with bipolar disorder. J Affect Disord. 2005 Apr;85(3):301-15.

    PubMed  Google Scholar 

  9. Roberts AL, Kubzansky LD, Chibnik LB, et al. Association of Posttraumatic Stress and Depressive Symptoms With Mortality in Women. JAMA Netw Open. 2020 Dec 1;3(12):e2027935.

    PubMed  PubMed Central  Google Scholar 

  10. Jacobsen LK, Southwick SM, Kosten TR. Substance use disorders in patients with posttraumatic stress disorder: a review of the literature. Am J Psychiatry. 2001 Aug;158(8):1184-90.

    CAS  PubMed  Google Scholar 

  11. Cerullo MA, Strakowski SM. The prevalence and significance of substance use disorders in bipolar type I and II disorder. Subst Abuse Treat Prev Policy. 2007 Oct 1;2:29.

    PubMed  PubMed Central  Google Scholar 

  12. Crump C, Sundquist K, Winkleby MA, et al. Comorbidities and mortality in bipolar disorder: a Swedish national cohort study. JAMA Psychiatry. 2013 Sep;70(9):931-9.

    PubMed  Google Scholar 

  13. Fornaro M, Novello S, Fusco A, et al. Clinical features associated with early drop-out among outpatients with unipolar and bipolar depression. J Psychiatr Res. 2021 Apr;136:522-528.

    CAS  PubMed  Google Scholar 

  14. Grubbs KM, Fortney JC, Pyne JM, et al. Predictors of Initiation and Engagement of Cognitive Processing Therapy Among Veterans With PTSD Enrolled in Collaborative Care. J Trauma Stress. 2015;28(6):580-584.

    PubMed  Google Scholar 

  15. Graff FS, Griffin ML, Weiss RD. Predictors of dropout from group therapy among patients with bipolar and substance use disorders. Drug Alcohol Depend. 2008;94(1-3):272-275.

    PubMed  Google Scholar 

  16. Maguen S, Li Y, Madden E, et al. Factors associated with completing evidence-based psychotherapy for PTSD among veterans in a national healthcare system. Psychiatry Res. 2019;274:112-128.

    PubMed  Google Scholar 

  17. Schottenbauer MA, Glass CR, Arnkoff DB, et al. Nonresponse and dropout rates in outcome studies on PTSD: review and methodological considerations. Psychiatry. 2008 Summer;71(2):134-68.

    PubMed  Google Scholar 

  18. Li K, Wei Q, Li G. Time to lack of persistence with pharmacological treatment among patients with current depressive episodes: a natural study with 1-year follow-up. Patient Prefer Adherence. 2016;10:2209-2215.

    PubMed  PubMed Central  Google Scholar 

  19. Ramanuj P, Ferenchik E, Docherty M, et al. Evolving Models of Integrated Behavioral Health and Primary Care. Curr Psychiatry Rep. 2019 Jan 19;21(1):4.

    PubMed  Google Scholar 

  20. Adaji A, Fortney J. Telepsychiatry in Integrated Care Settings. Focus (Am Psychiatr Publ). 2017;15(3):257-263.

    PubMed  PubMed Central  Google Scholar 

  21. Fortney JC, Pyne JM, Mouden SB, et al.Practice-Based Versus Telemedicine-Based Collaborative Care for Depression in Rural Federally Qualified Health Centers: A Pragmatic Randomized Comparative Effectiveness Trial. Focus (Am Psychiatr Publ). 2017 Jul;15(3):361-372.

    PubMed  PubMed Central  Google Scholar 

  22. Unützer J, Katon W, Callahan CM, et al. IMPACT Investigators. Improving Mood-Promoting Access to Collaborative Treatment. Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. JAMA. 2002 Dec 11;288(22):2836-45.

    PubMed  Google Scholar 

  23. Zayfert C, DeViva JC, Becker, et al. Exposure utilization and completion of cognitive behavioral therapy for PTSD in a “real world” clinical practice. J Trauma Stress. 2005 Dec;18:637–645.

    PubMed  Google Scholar 

  24. Bryant RA, Moulds ML, Guthrie RM, et al. Imaginal exposure alone and imaginal exposure with cognitive restructuring in treatment of posttraumatic stress disorder. J Consult Clin Psychol. 2003 Aug;71(4):706-12.

    PubMed  Google Scholar 

  25. Sayer NA, Friedemann-Sanchez G, Spoont M, et al. A qualitative study of determinants of PTSD treatment initiation in veterans. Psychiatry. 2009 Fall;72(3):238-55.

    PubMed  Google Scholar 

  26. Patel SY, Mehrotra A, Huskamp HA, et al. Trends in Outpatient Care Delivery and Telemedicine During the COVID-19 Pandemic in the US. JAMA Intern Med. 2021 Mar 1;181(3):388-391.

    PubMed  Google Scholar 

  27. Fortney JC, Heagerty PJ, Bauer AM, et al. Study to promote innovation in rural integrated telepsychiatry (SPIRIT): Rationale and design of a randomized comparative effectiveness trial of managing complex psychiatric disorders in rural primary care clinics. Contemp Clin Trials. 2020 Mar;90:105873.

    PubMed  Google Scholar 

  28. Fortney J, Bauer A, Cerimele J, et al. Comparison of Teleintegrated Care and Telereferral Care for Treating Complex Psychiatric Disorders in Primary Care: A Pragmatic Randomized Comparative Effectiveness Trial. JAMA Psychiatry. 2021 Nov 1;78(11):1189-1199.

  29. Xue Y, Greener E, Kannan V, et al. Federally qualified health centers reduce the primary care provider gap in health professional shortage counties. Nurs Outlook. 2018 May-Jun;66(3):263-272.

    PubMed  Google Scholar 

  30. Lang AJ, Stein MB. An abbreviated PTSD checklist for use as a screening instrument in primary care. Behav Res Ther. 2005;43(5):585-594.

    PubMed  Google Scholar 

  31. Kessler RC, Akiskal HS, Angst J, et al. Validity of the assessment of bipolar spectrum disorders in the WHO CIDI 3.0. J Affect Disord. 2006;96:259-269.

    PubMed  PubMed Central  Google Scholar 

  32. Levis B, Sun Y, He C, et al. Accuracy of the PHQ-2 Alone and in Combination With the PHQ-9 for Screening to Detect Major Depression: Systematic Review and Meta-analysis. JAMA. 2020;323(22):2290–2300.

    PubMed  PubMed Central  Google Scholar 

  33. McFall M. Behavioral activation as a primary care-based treatment for PTSD and depression among returning veterans. J Trauma Stress. 2010 Aug;23(4):491-5.

    PubMed  Google Scholar 

  34. Lehman AF, Fischer EP, Postrado L, et al. The Schizophrenia Care and Assessment Program Health Questionnaire (SCAP-HQ): an instrument to assess outcomes of schizophrenia care. Schizophr Bull. 2003;29(2):247-56.

    PubMed  Google Scholar 

  35. Williams JW Jr, Stellato CP, Cornell J, et al. The 13- and 20-item Hopkins Symptom Checklist Depression Scale: psychometric properties in primary care patients with minor depression or dysthymia. Int J Psychiatry Med. 2004;34(1):37-50.

    PubMed  Google Scholar 

  36. Shirinbayan P, Salavati M, Soleimani F, et al. The Psychometric Properties of the Drug Abuse Screening Test. Addict Health. 2020;12(1):25-33.

    PubMed  PubMed Central  Google Scholar 

  37. Dawson DA, Grant BF, Stinson FS, et al. Effectiveness of the derived Alcohol Use Disorders Identification Test (AUDIT-C) in screening for alcohol use disorders and risk drinking in the US general population. Alcohol Clin Exp Res. 2005 May;29(5):844-54.

    PubMed  Google Scholar 

  38. Rush AJ. Handbook of Psychiatric Measures. 1st ed. Washington, DC: American Psychiatric Association; 2000: 213–15.

    Google Scholar 

  39. Bauer MS, Vojta C, Kinosian B, et al. The Internal State Scale: replication of its discriminating abilities in a multisite, public sector sample. Bipolar Disord. 2000 Dec;2(4):340-6.

    CAS  PubMed  Google Scholar 

  40. Bauer MS, Crits-Christoph P, Ball WA, et al. Independent Assessment of Manic and Depressive Symptoms by Self-rating: Scale Characteristics and Implications for the Study of Mania. Arch Gen Psychiatry. 1991;48(9):807–812.

    CAS  PubMed  Google Scholar 

  41. Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Methodol., 1995;57(1):289-300.

    Google Scholar 

  42. Jones D, Kazis L, Lee A, et al. Health status assessments using the Veterans SF-12 and SF-36: Methods for evaluating outcomes in the Veterans Health Administration. J Ambul Care Manage. 2001;24(3):68-86.

    CAS  PubMed  Google Scholar 

  43. Verdolini N, Hidalgo-Mazzei D, Murru A, et al. Mixed states in bipolar and major depressive disorders: systematic review and quality appraisal of guidelines. Acta Psychiatr Scand. 2018 Sep;138(3):196-222.

    CAS  PubMed  Google Scholar 

  44. American Psychiatric Association. Part A – Treatment Recommendations for Patients with Bipolar Disorder. Practice guideline for the treatment of patients with bipolar disorder (revision). Am J Psychiatry. 2002 Apr;159(4 Suppl):4-7.

    Google Scholar 

  45. Miklowitz DJ, Efthimiou O, Furukawa TA, et al. Adjunctive Psychotherapy for Bipolar Disorder: A Systematic Review and Component Network Meta-analysis. JAMA Psychiatry. 2021;78(2):141–150.

    PubMed  Google Scholar 

  46. O'Brien B, Lee D, Swann AC, et al. Psychotherapy for Mixed Depression and Mixed Mania. Psychiatr Clin North Am. 2020 Mar;43(1):199-211. Epub 2019 Dec 18.

  47. French MT, Roebuck MC, McGeary KA, et al. Using the drug abuse screening test (DAST-10) to analyze health services utilization and cost for substance users in a community-based setting. Subst Use Misuse. 2001 May-Jun;36(6-7):927-46.

    CAS  PubMed  Google Scholar 

  48. Nordheim K, Walderhaug E, Alstadius S, et al. Young adults’ reasons for dropout from residential substance use disorder treatment. Qual Soc Work. 2018;17(1):24-40.

    PubMed  Google Scholar 

  49. King AC, Canada SA.Client-related predictors of early treatment drop-out in a substance abuse clinic exclusively employing individual therapy. J Subst Abuse Treat. 2004 Apr;26(3):189-95.

    PubMed  Google Scholar 

  50. Sripada RK, Pfeiffer PN, Valenstein M, et al. Medical illness burden is associated with greater PTSD service utilization in a nationally representative survey. Gen Hosp Psychiatry. 2014 Nov-Dec;36(6):589-93.

    PubMed  Google Scholar 

  51. Theis KA, Barker L, Shannon Self-Brown S et al. Chronic Disease Disparities by County Economic Status and Metropolitan Classification, Behavioral Risk Factor Surveillance System, 2013. Center for Chronic Disease (CDC). Prev Chronic Dis., Original Research, Volume 13, September 16. https://www.cdc.gov/pcd/issues/2016/16_0088.htm. Accessed 31 May 2021.

  52. Cohen BE, Gima K, Bertenthal D, et al. Mental health diagnoses and utilization of VA non-mental health medical services among returning Iraq and Afghanistan veterans. J Gen Intern Med. 2010 Jan;25(1):18-24.

    PubMed  Google Scholar 

  53. Levine DS, Sripada RK, Ganoczy D, et al. Poorer Physical Health is Associated With Greater Mental Health Service Utilization in a Sample of Depressed U.S. Army National Guard Soldiers. Mil Med. 2016 Aug;181(8):803-10.

    PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank the patients and staff of Family Health Centers, Moses Lake Community Health Center, Sea Mar Community Health Centers, Yakima Neighborhood Health Services, Boston Mountain Rural Health Centers, East Arkansas Family Health Center, Lee County Cooperative Clinic, Cherry Health, Family Medical Center, Great Lakes Bay Health Centers, InterCare Community Health Network, and Upper Great Lakes Family Health Center. We would like to thank the staff of the Community Health Plan of Washington, the Community Health Centers of Arkansas, and the Michigan Primary Care Association. We would also like to thank our Policy Advisory Board: Donald Weaver (National Association of Community Health Centers), Ed Larkins (Family Medical Centers of Michigan), Jay Shore (University of Colorado at Denver), LaShannon Spencer and Victoria Miller (Community Health Centers of Arkansas), Nancy Speck (National Association for Rural Mental Health), Sara Coates and Dawne Velianoff (Michigan Primary Care Association), Susan Ward-Jones (East Arkansas Family Health Center), Jay Shore (University of Colorado at Denver), and Kim Arnold (National Alliance on Mental Illness). We would like to thank our Consumer Advisor Board: Bernadette Mcdonald and Florence Fee (No Health without Mental Health); Ingrid Deetz, Kevin Einbinder, and Betsy O’Brien (Depression and Bipolar Support Alliance); Kim Arnold, Marquitta Magnini, and Shawn McCown (National Alliance on Mental Illness); Jode Freyholtz-London (Wellness in the Woods); Tammy Heral; and Jeremy Mullins. We would like to thank the members of our external Data Safety Monitoring Board: Thomas Belin (University of California, Los Angeles), John Williams (Duke University), and Mark Williams (Mayo Clinic). We would like to thank the following electronic health record vendors for giving us substantial discounts on the user licenses for the telepsychiatrists and telepsychologists: NextGen Healthcare®, eClinicalWorks, Centricity (GE Healthcare), Allscripts®, and SuccessEHS, Inc. (Greenway Health™).

Funding

This work was supported by a grant from the Patient-Centered Outcomes Research Institute (PCORI—PCS-1406-19295) to Drs. Fortney and Unützer. The statements in this article are solely the responsibility of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute, its Board of Governors or Methodology Committee.

Author information

Authors and Affiliations

  1. Department of Psychiatry, University of Michigan Medical School, 4250 Plymouth Rd, Ann Arbor, MI, 48109, USA

    Jennifer Severe MD, Paul N. Pfeiffer M.D., M.S. & Rebecca Sripada Ph.D.

  2. Department of Veterans Affairs, Ann Arbor Veterans Affairs Center for Clinical Management Research, Ann Arbor, MI, USA

    Paul N. Pfeiffer M.D., M.S. & Rebecca Sripada Ph.D.

  3. Department of Psychiatry and Behavioral Sciences, School of Medicine, University of Washington, Seattle, WA, USA

    Katherine Palm-Cruz M.D., Theresa Hoeft Ph.D., Matthew Hawrilenko PhD, Shiyu Chen MPH & John Fortney Ph.D.

  4. Department of Veterans Affairs, Health Services Research and Development, Center of Innovation for Veteran-Centered and Value-Driven Care, Seattle, WA, USA

    John Fortney Ph.D.

Authors
  1. Jennifer Severe MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul N. Pfeiffer M.D., M.S.
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Katherine Palm-Cruz M.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Theresa Hoeft Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rebecca Sripada Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Matthew Hawrilenko PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Shiyu Chen MPH
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. John Fortney Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jennifer Severe MD.

Ethics declarations

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Prior Presentations

This paper has not been presented before at any conference or lecture.

Supplementary Information

ESM 1

(DOCX 37 kb)

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Severe, J., Pfeiffer, P.N., Palm-Cruz, K. et al. Clinical Predictors of Engagement in Teleintegrated Care and Telereferral Care for Complex Psychiatric Disorders in Primary Care: a Randomized Trial. J GEN INTERN MED (2022). https://doi.org/10.1007/s11606-021-07343-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11606-021-07343-x

KEY WORDS

  • telehealth
  • collaborative care
  • treatment engagement
  • psychiatric disorders
  • PTSD
  • bipolar disorder
  • primary care
  • federally qualified
  • health centers