Covid-19 has led to an unprecedented shift to telemental health (TMH) in mental healthcare. This study examines the impact of this transition on visit adherence for mental health services in an integrated behavioral health department. Monthly visit data for 12,245 patients from January, 2019 to January, 2021 was extracted from the electronic medical record. Interrupted time series (ITS) analysis examined the impact of the Covid-19 transition to TMH on immediate level and trend changes in the percentage of cancelled visits and no shows in the 10 months following the transition. ITS also compared changes across the three largest services types: adult, pediatric, and substance use. Following the TMH-transition, completed visits increased by 10% amounting to an additional 3644 visits. In April, 2020, immediately following the TMH-transition, no shows increased by 1.4%, (95% CI 0.1, 2.7, p < 0.05) and cancellations fell by 13.5% (95% CI − 17.9, − 9.0, p < 0.001). Across the 10-month post-TMH period, 18.2% of visits were cancelled, compared to 28.3% across the 14-month pre-TMH period. The proportion of no-shows remained the same. The pattern was similar for pediatric and adult sub-clinics, but no significant changes in cancellations or no shows were observed in the substance use sub-clinic. TMH during the Covid-19 pandemic is associated with improved visit adherence over time and may be a promising model for improving the efficiency of mental health care delivery once it is safe to resume in person care.
The onset of the Covid-19 pandemic precipitated a dramatic shift in the way mental healthcare services are delivered. The need for social distancing to mitigate the spread of Covid-19 has required health care organizations to swiftly develop and implement strategies to provide continuity of care while maintaining the safety of patients and providers. This is critical in mental healthcare where stress related to the virus and mitigation efforts may heighten symptoms among those with mental health concerns (Asmundson et al., 2020; Druss, 2020; Zhou et al., 2020). To address these concerns, telemental health (TMH), which includes the synchronous delivery of mental health services through electronic technologies including video and telephone, has been widely adopted by healthcare organizations (Baum et al., 2021; Connolly et al., 2020; Hoffnung et al., 2021; Mishkind et al., 2020; Shore et al., 2020; Tse et al., 2020). TMH is an effective means for delivering mental health services to diverse populations presenting with a variety of conditions (Bashshur et al., 2016; Fiacco et al., 2021). Furthermore, TMH removes some of the barriers to service access, thus improving accessibility among patients in low-access areas and populations with special needs (Bashshur et al., 2016).
Despite the demonstrated benefits of TMH, insurance coverage barriers have typically made it unfeasible to implement TMH into routine practice in many states (Wilson et al., 2017). Prior to the pandemic, state-level restrictions on TMH allowed fewer than half of states to provide TMH services at parity. Furthermore, organizations experienced barriers to implementing TMH including costs associated with deploying new technology and training providers (Wilson et al., 2017). However, at the onset of Covid-19 in early 2020, many state and federal governmental bodies lifted insurance and licensure barriers to telehealth, paving the way for its widespread adoption. In Massachusetts, the site of the current research, an executive order required local state insurance companies to permit in-network providers to deliver covered services via telehealth (“Order Expanding Access to Telehealth Services and to Protect Health Care Providers,” 2020). Critically, this order eliminated inter-insurance differences in authorizations. Similar orders went into effect in 40 states nationwide, and the Centers for Medicare and Medicaid Services (CMS) expanded coverage to visits conducted via telemedicine (CMS.gov, 2020; Haque, 2021).
In response to the rapid expansion of TMH, a burgeoning body of research has begun to examine the impact of Covid-19 related TMH implementations on patients, providers, and healthcare organizations. The findings of this research have generally been favorable; the majority of patients report feeling as connected to providers in the TMH model and many would prefer TMH or a combination of TMH and in-person in the long term (Nicholas et al., 2021; Tse et al., 2020). Others have described short-term improvements in visit completions and no show rates following the transition to TMH (Mishkind et al., 2020).
The current study builds upon this research to examine the impact of the transition to TMH on the Behavioral Health (BH) Department at Reliant Medical Group (RMG) in central Massachusetts. In the ten months following the transition from exclusively in-person only service delivery to TMH, we noted a 10% improvement in completed visits. This study uses an interrupted time series (ITS) design to examine longitudinal changes in the trends of the two factors that drive visit completion rates, cancellations and no shows, over the 14 months prior to and ten months following the transition to TMH.
[Name redacted] Reliant Medical Group is a multi-specialty medical group located in central Massachusetts serving over 300,000 patients in adult and pediatric primary care. Behavioral health services provided by the department include both treatment for mental health concerns and substance use, as well as behaviorally oriented treatments for physical health problems. This study focuses on visit data for 12,245 patients who received mental health services from the adult, pediatric and substance use services sub-clinics. The BH department operates an integrated model in adult and pediatric primary care, in which BH clinicians are tightly allied with primary care practitioners and available in real-time for consultation, triage, and intervention. The BH Provider is the primary internal treatment force, offering relatively brief, goal-oriented, evidence-based psychotherapy (8–10 sessions) to patients. The BH Consulting Prescriber is an MD or psychiatric advanced practitioner who supports primary care with prescribing advice and brief interventions to evaluate and adjust medications (usually not longer than 6–8 weeks) before passing the case back to primary care for ongoing management. In addition to these roles, there are specialty BH sub-clinics for substance use disorders, bariatrics, and a newly established dialectical behavior therapy (DBT) center. Combining these services, over 40,000 BH visits and consultations were completed in 2020.
On March 17th, 2020 all BH providers and consulting prescribers began to work remotely, conducting visits via telephone until all providers had been trained to use the secure video platform. Prior to this transition, all visits were conducted in-person. At that time nearly all visits transitioned to video. The BH chief trained each provider on the new video software via a one-on-one call. Additionally, all staff participated in a one-hour video training provided by a telemedicine start-up with expertise in providing BH video treatments.
We conducted an interrupted time series (ITS) analysis of routine monthly encounter data spanning from January of 2019 to January of 2021, treating the March, 2020 transition to TMH as the intervention period. Considered to be the strongest quasi-experimental design, ITS is ideal for measuring the population-level impact of healthcare interventions and quality improvement initiatives where random assignment is not feasible (Penfold & Zhang, 2013). By treating the pre-intervention period as the counterfactual had the intervention not occurred, ITS can be used to identify changes in the slope of the data over time (Linden, 2015; Penfold & Zhang, 2013).
Visit data includes monthly scheduled visits, completed visits, cancellations, and no-shows routinely captured in the Electronic Medical Record (EMR). We used visit data for all psychotherapy and psychiatry visits scheduled in the adult, pediatric and substance use service sub-clinics between January, 2019 and January, 2021. This resulted in 14 months of pre-intervention data and ten months of post-intervention data, which included a total of 102,072 scheduled visits during the study period. This covered 10,515 patients seen prior to the TMH-transition, and 12,245 patients seen after. Patient-level characteristics across the three sub-clinics and intervention periods are displayed in Appendix. This study was ruled exempt quality improvement by the Reliant Medical Group IRB.
The primary outcome variables were visit cancellations (including 24-hour cancellations) and no shows (visits where the patient did not cancel or show up to their appointment) as a proportion of total scheduled visits (all visits scheduled with BH providers and prescribers). The analyses focus on adult, pediatric, and substance use service sub-clinics, which cover 89 percent of all services provided in the BH department. We excluded visit data for DBT and bariatrics because both of these programs were initiated after the pandemic started.
Prior to running the analyses, we examined the pre-TMH data for linearity. We assessed autocorrelation at different lags by plotting residuals and partial autocorrelation functions and implementing the Cumby-Huizinga general test of autocorrelated errors for each model to determine the optimal lag (Baum & Schaffer, 2013). The models were run in Stata 15 using the program its a, which estimates ordinary least squares (OLS) regression with Newey West standard errors (Linden, 2015). March 2020, the month TMH was introduced, was excluded from analyses as the intervention took place mid-month (Linden, 2015; Penfold & Zhang, 2013).
First, we estimated the change in the slope and level change of the percentage of cancellations and no-shows across the adult, pediatric, and substance use service sub-clinics combined. We then examined the changes in slopes and level changes for each sub-clinic individually. We also assessed potential seasonal effects and possible confounding due to changes in initial versus follow-up encounters, and psychiatric visits versus therapy visits. We did not identify any differences in the coefficients or standard errors for the primary variables of interest and these variables were omitted from the models.
In the 14 months prior to the transition to TMH, approximately 62.0% (95% CI 60.7, 63.4) of all visits were completed (i.e. patient did not cancel or no-show), a trend that was relatively consistent across all months, with the exception of December, 2019 when completions fell to 55%. With the onset of Covid-19 and the transition to TMH, there was an immediate 10% increase in the proportion of completed visits in April. Overall completions have remained around 72% (95% CI 71.8, 73.25) over the past 10 months. Pre and post-transition visits, cancellations, and no show rates across all sub-clinics, and disaggregated by each unique sub-clinic are displayed in Table 1. Prior to the transition, 27.8% of adult visits were cancelled, compared to only 17.3% following the transition. For pediatric visits, 30.3% were cancelled prior to the transition compared to 18.5% after. In the substance use sub-clinic, cancellations remained constant across the transition. In the postintervention period 98% of all visits were conducted via TMH; the remaining 2% face-to-face visits primarily included patients from the substance use sub-clinic who were not considered appropriate candidates for TMH. In the preintervention period, 68% of all scheduled visits were in the adult sub-clinic, 25% were in the pediatric sub-clinic, and 7% were in the substance use sub-clinic. In the postintervention period, approximately 63% of all scheduled visits were in the adult sub-clinic and 12% were in the substance use sub-clinic. The proportion of pediatric scheduled visits remained constant across the intervention. Patient demographic and psychiatric characteristics across the pre-TMH and post-TMH periods are displayed in the Appendix.
Aggregate Changes in Cancellations and No Show Rates
Table 2 and Fig. 1 display changes in the trend and level of the percentage of cancelled visits over time following the transition to TMH. Overall, cancellations decreased by 13.5% (95% CI − 17.9, − 9.0) in April, immediately following the transition. In the preintervention period, the slope for the percentage of cancellations was not statistically different from zero (B = 0.2, 95% CI − 0.3, 0.8). Although the trend change in the slope of cancellations (B = 0.2, 95% CI − 0.4, 0.9) was not statistically significant, analysis of the postintervention linear trend indicated a significant postintervention slope period (B = 0.4, 95% CI 0.2, 0.7, p < 0.005), indicating a gradual increase in cancellations following the initial drop in the post-transition period.
Table 2 and Fig. 2 display changes in the trend and level of the percentage of no shows following the transition to TMH. The percentage of no shows increased significantly by 1.4% (95% CI 0.1, 2.7, p < 0.05) in April, immediately following the transition. After this initial jump, the percentage of no shows declined at a rate of 0.6% per month (95% CI − 0.8, − 0.4, p < 0.001), a pattern that was not observed in the preintervention period where no shows were rising overtime. The monthly decline from May, 2020 to January, 2021 is depicted in Fig. 2. Due to the combined effects of the initial jump followed by the gradual decline over time, the overall no show rate remained at 9.0% in the pre- and post-TMH periods.
Subgroup Analyses of Cancellation and No Show Rates
Table 3 displays the results of the subgroup ITS analysis including the baseline trends and level and trend changes for adult, pediatric and substance use service sub-clinics, respectively. Both the adult and pediatric service sub-clinics experienced significant level decreases in the percentage of cancellations immediately following the transition. Among adult service users, cancellations dropped by 12.9% in April (95% CI − 17.0, − 8.7, p < 0.001). Among pediatric service users, cancellations fell by 12.0% (95% CI − 16.2, − 7.8, p < 0.001). However, following initial level drops, both adult and pediatric sub-clinics showed significant gradual increases in the rate of cancellations. Among adults, cancellations increased by 0.6% per month (95% CI 0.1, 1.1, p < 0.05) and in the pediatric sub-clinic, cancellations increased by 0.7% per month (95% CI 0.1, 1.3, p < 0.05) in the post-intervention period. We did not observe significant changes in the level or trend of cancellations in the substance use sub-clinic.
No Show Rates
In stratified ITS analyses, both adult and pediatric sub-clinics experienced gradual declines in the percentage of no shows across the TMH-transition period. In the adult sub-clinic, no shows decreased at a rate of 0.7% (95% CI − 0.9%, − 0.5%, p < 0.001) per month after the transition. In the pediatric sub-clinic, no shows decreased at a rate 0.5% (95% CI − 0.8%, − 0.3%, p < 0.001). In the pediatric and adult sub-clinic ITS analyses, the level change that was observed in aggregate analysis of no shows did not reach statistical significance, although there were marginal trends in significance. In the adult service sub-clinic, there was a level change of 1. 2% (95% CI − 0.01, 2.4, p = 0.052) and in the pediatric sub-clinic there was a level change of 1.7 (95% CI − 0.1, 3.5, p = 0.069). In the substance use sub-clinic, neither the level nor trend of no shows changed significantly across the transition to TMH.
By using an ITS design to examine trends in mental health encounter data from 14 months prior to the transition to TMH and ten months following the transition, we were able to examine the impact of TMH on visit cancellation and no show rates in order to better understand the overall improvement in visit completion. The patterns in cancellation and no show rates differed from each other. In the analyses aggregated across service sub-clinics, there was a sharp decline in cancellations in April, 2020 immediately following the transition to TMH, followed by a gradual increase in cancellations in the months after. This pattern was also observed among the adult and pediatric service sub-clinics in stratified analyses. However, the overall level and trend in substance use service cancellations was not impacted by the transition. Level and trend changes in no shows followed a different pattern. Directly after the transition to TMH, the overall level of no shows increased slightly, followed by a gradual decline over time. Similar patterns were observed in adult and pediatric service sub-clinics. Again, no clear changes in no shows emerged in the substance use services sub-clinic.
It is likely that the initial increase in no shows followed by a decrease over time in post-TMH no shows reflects a learning curve for the new virtual technology (Wootton et al., 2020). As providers and patients adapted to telehealth technology, they likely became more efficient engaging in the TMH model of care leading to incremental improvements in no show rates among pediatric and adult service users over time. Furthermore, systematic learning by the department, reflected in improvements in workflows, patient-facing materials, and support mechanisms, also likely contributed to the gradual reduction in no-shows over time.
One possible explanation for the sharp decrease in cancellations is that TMH improves visit adherence by removing some of the typical barriers patients encounter keeping face-to-face visits such as transportation, childcare, and other logistical barriers. It is also critical to consider that the decline in cancellations may have been related to the enhanced need for behavioral health services during the Covid-19 pandemic. Increases in mental health symptomology related to the pandemic and resulting social isolation may motivate patients to keep their behavioral health appointments. However, in one study of failure to attend TMH services using data collected prior to the pandemic, the authors reported that patients missed TMH visits at nearly half the rate that they missed in-person visits (Snoswell & Comans, 2021). This suggests that the reduction in cancellations may indeed be the product of improved convenience due TMH rather than an increased need for services due to the pandemic. Further, the gradual increase in cancellations several months into the pandemic may reflect state-level loosening of restrictions that allowed some workers to return to work sites and gradual reopening of local economies. These changes may have reintroduced logistical barriers to visit adherence as patients had more reasons to leave their homes. It is also possible that patients became more likely to cancel visits as they developed strategies for coping with stress related to the pandemic and relied less on BH services.
Differences Among Sub-clinics
It is promising that TMH appeared to positively impact both the adult and pediatric service sub-clinics. The literature on TMH among pediatric populations has identified mixed support for TMH. One study reported that adults preferred TMH visits and children were twice as likely to return to face-to-face visits following the lock down, compared to adults (Hoffnung et al., 2021). Yet, another study found that young people aged 12–25 reported that TMH was effective and that it made them less likely to cancel visits. Unlike pediatric and adult service users, patients using substance use services did not show changes in the rate of cancellations or no shows across the transition. This suggests that patients seeking substance use services may face specific barriers to engaging in TMH compared to other behavioral health patients. Prior to the onset of Covid-19, TMH was leveraged for patients seeking substance use treatment at twenty-percent the rate of use for other behavioral health patient populations (Huskamp et al., 2018). While research on TMH in behavioral health practice demonstrates broad clinical efficacy of TMH for behavioral health, relatively less is understood about the efficacy of TMH in substance use treatment and this is an important area for future inquiry (Huskamp et al., 2018; Lin et al., 2019; Lin et al., 2020).
Despite promising evidence that the transition to TMH may improve visit adherence, this study has several important limitations. First, the rapid transition to TMH occurred in response to the onset of the Covid-19 pandemic and the effects of TMH cannot be separated from concurrent developments in the pandemic. Nonetheless, these findings are important as they show that in a public health emergency, TMH is a viable means for engaging patients in behavioral health care and may improve adherence. Furthermore, under non-emergency circumstances TMH may still benefit visit completion by removing logistical barriers. Second, although ITS is a powerful approach for examining trends in population-level data over time, individual level effects of TMH are washed out in aggregate analyses. It remains vital to understand how individual-level factors such as diagnosis and severity of illness impact adherence and outcomes related to TMH interventions. This will be especially important in determining the future of TMH and making decisions about the circumstances in which TMH models should be used. Third, we used a single group ITS design which lacked a control group, and it is possible that a concurrent, unobserved changes may have impacted no shows and cancellation rates, potentially confounding these findings (Linden, 2017). Finally, Reliant Medical Group operates under an integrated behavioral health model in which patients undergo relatively brief, targeted treatments for mental health concerns. Therefore, these findings may not be generalizable to more intensive, longer term behavioral health services.
This study has several notable strengths. By focusing on group-level outcomes, ITS is particularly robust to potential confounding by individual level factors (Penfold & Zhang, 2013). ITS allowed us to test both the immediate changes in outcomes as well as the trends in in these outcomes over time. While a mean comparison (e.g. t-test) of no shows would not reveal any change before and after the transition, by examining the time trends, we detected improvements in no-show rates over time following the TMH-transition. While we include data from 10 months following the transition to TMH, it remains vital to examine the longer-term impact of TMH on no shows on overall visit completion as more data becomes available. Finally, the ITS method controls for secular trends that existed in the data prior to the implementation of TMH.
This investigation suggests that TMH may improve visit adherence through significant immediate reductions in cancellations and gradual decreases no show rates over time. However, these improvements may not be universal for all patients or types of services. Future research is needed to understand which services can be delivered effectively via TMH and which benefit from in-person care or hybrid models. As mitigation efforts and vaccine distribution begin to slow the spread of Covid-19, it is important to understand the long-term role that telehealth may play in healthcare. Some states are beginning to pass legislation to require parity for services delivered via telehealth after it becomes safe to resume in person visits, paving the way for its continued use after the pandemic (An Act promoting a resilient health care system that puts patients first., 2020). The expansion of TMH following the Covid-19 presents an opportunity to increase access to behavioral health services, potentially reducing costs to healthcare organizations and payers. However, more work is needed to understand the optimal circumstances for telehealth services, as well as potential barriers that some patients may have to accessing necessary technologies.
Data and study code are available by request.
An Act promoting a resilient health care system that puts patients first. (2020). Boston, MA: The Commonwealth of Massachusetts.
Asmundson, G. J. G., Paluszek, M. M., Landry, C. A., Rachor, G. S., McKay, D., & Taylor, S. (2020). Do pre-existing anxiety-related and mood disorders differentially impact COVID-19 stress responses and coping? Journal of Anxiety Disorders, 74, 102271. https://doi.org/10.1016/j.janxdis.2020.102271
Bashshur, R. L., Shannon, G. W., Bashshur, N., & Yellowlees, P. M. (2016). The empirical evidence for telemedicine interventions in mental disorders. Telemedicine and E-Health, 22(2), 87–113. https://doi.org/10.1089/tmj.2015.0206
Baum, A., Kaboli, P. J., & Schwartz, M. D. (2021). Reduced in-person and increased telehealth outpatient visits during the COVID-19 pandemic. Annals of Internal Medicine, 174(1), 129–131. https://doi.org/10.7326/m20-3026
Baum, C. F., & Schaffer, M. E. (2013). actest: Stata module to perform Cumby-Huizinga general test for autocorrelation in time series. Statistical Software Components S457668. Retrieved from https://ideas.repec.org/c/boc/bocode/s457668.html
CMS.gov. (2020). Medicare telemedicine health care provider fact sheet | CMS. Retrieved February 1, 2021, from https://www.cms.gov/newsroom/fact-sheets/medicare-telemedicine-health-care-provider-fact-sheet
Connolly, S. L., Stolzmann, K. L., Heyworth, L., Weaver, K. R., Bauer, M. S., & Miller, C. J. (2020). Rapid increase in telemental health within the department of veterans affairs during the COVID-19 pandemic. Telemedicine and E-Health. https://doi.org/10.1089/tmj.2020.0233
Druss, B. G. (2020, September 1). Addressing the COVID-19 pandemic in populations with serious mental illness. JAMA Psychiatry. https://doi.org/10.1001/jamapsychiatry.2020.0894
Fiacco, L., Pearson, B. L., & Jordan, R. (2021). Telemedicine works for treating substance use disorder: The STAR clinic experience during COVID-19. Journal of Substance Abuse Treatment. https://doi.org/10.1016/j.jsat.2021.108312
Haque, S. N. (2021, January 1). Telehealth beyond COVID-19. Psychiatric Services. https://doi.org/10.1176/APPI.PS.202000368
Hoffnung, G., Feigenbaum, E., Schechter, A., Guttman, D., Zemon, V., & Schechter, I. (2021). Children and telehealth in mental healthcare: What we have learned from COVID-19 and 40,000+ sessions. Psychiatric Research and Clinical Practice. https://doi.org/10.1176/appi.prcp.20200035
Huskamp, H. A., Busch, A. B., Souza, J., Uscher-Pines, L., Rose, S., Wilcock, A., Landon, B. E., & Mehrotra, A. (2018). How is telemedicine being used in opioid and other substance use disorder treatment? Health Affairs, 37(12), 1940–1947. https://doi.org/10.1377/hlthaff.2018.05134
Lin, L., Casteel, D., Shigekawa, E., Weyrich, M. S., Roby, D. H., & McMenamin, S. B. (2019). Telemedicine-delivered treatment interventions for substance use disorders: A systematic review. Journal of Substance Abuse Treatment, 101, 38–49
Lin, L. A., Fernandez, A. C., & Bonar, E. E. (2020, December 1). Telehealth for substance-using populations in the age of coronavirus disease 2019: Recommendations to enhance adoption. JAMA Psychiatry. https://doi.org/10.1001/jamapsychiatry.2020.1698
Linden, A. (2015). Conducting interrupted time-series analysis for single- and multiple-group comparisons. Stata Journal, 15(2), 480–500. https://doi.org/10.1177/1536867x1501500208
Linden, A. (2017). Challenges to validity in single-group interrupted time series analysis. Journal of Evaluation in Clinical Practice, 23(2), 413–418. https://doi.org/10.1111/jep.12638
Mishkind, M. C., Shore, J. H., Bishop, K., D’Amato, K., Brame, A., Thomas, M., & Schneck, C. D. (2020). Rapid conversion to telemental health services in response to COVID-19: Experiences of two outpatient mental health clinics. Telemedicine and E-Health. https://doi.org/10.1089/tmj.2020.0304
Nicholas, J., Bell, I., Thompson, A., Valentine, L., Simsir, P., Sheppard, H., & Adams, S. (2021). Implementation lessons from the transition to telehealth during COVID-19: A survey of clinicians and young people from youth mental health services. Psychiatry Research. https://doi.org/10.1016/j.psychres.2021.113848
Order Expanding Access to Telehealth Services and to Protect Health Care Providers. (2020). Boston, MA: Office of the Governor. Retrieved from https://www.mass.gov/info-details/covid-19-state-of-emergency
Penfold, R. B., & Zhang, F. (2013). Use of interrupted time series analysis in evaluating health care quality improvements. Academic Pediatrics. https://doi.org/10.1016/j.acap.2013.08.002
Shore, J. H., Schneck, C. D., & Mishkind, M. C. (2020, December 1). Telepsychiatry and the coronavirus disease 2019 pandemic-current and future outcomes of the rapid virtualization of psychiatric care. JAMA Psychiatry. https://doi.org/10.1001/jamapsychiatry.2020.1643
Snoswell, C. L., & Comans, T. A. (2021). Does the choice between a telehealth and an in-person appointment change patient attendance? Telemedicine Journal and E-Health, 27(7), 733–738. https://doi.org/10.1089/TMJ.2020.0176
Tse, J., Lastella, D., Chow, E., Kingman, E., Pearlman, S., Valeri, L., Wang, H., & Dixon, L. B. (2020). Telehealth acceptability and feasibility among people served in a community behavioral health system during the COVID-19 pandemic. Psychiatric Services., 2021(72), 654–660.
Wilson, F. A., Rampa, S., Trout, K. E., & Stimpson, J. P. (2017). Telehealth delivery of mental health services: An analysis of private insurance claims data in the United States. Psychiatric Services, 68(12), 1303–1306. https://doi.org/10.1176/appi.ps.201700017
Wootton, A. R., McCuistian, C., Packard, D. A. L., Gruber, V. A., & Saberi, P. (2020). Overcoming technological challenges: Lessons learned from a telehealth counseling study. Telemedicine and E-Health, 26(10), 1278–1283. https://doi.org/10.1089/tmj.2019.0191
Zhou, X., Snoswell, C. L., Harding, L. E., Bambling, M., Edirippulige, S., Bai, X., & Smith, A. C. (2020, April 1). The Role of Telehealth in Reducing the Mental Health Burden from COVID-19. Telemedicine and E-Health. https://doi.org/10.1089/tmj.2020.0068
This work was supported financially by the Practice Research Network at Reliant Medical Group and OptumLabs.
Conflict of interest
The authors have no conflicts of interest to declare that are relevant to the content of this article.
The research was granted an exemption as a quality improvement study by the Internal Review Board at Reliant Medical Group.
Reporting follows the Reporting of studies Conducted using Observational Routinely-collected health Data (RECORD) guidelines.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix: Behavioral Health Department Patient Characteristics Pre and Post-transition to TMH
Appendix: Behavioral Health Department Patient Characteristics Pre and Post-transition to TMH
Substance Use BH sub-clinic
Unique patients across sub-clinics c
Age, mean (SD)
Mental health diagnosesb
Bipolar disorders and manic episodes
Obsessive compulsive disorder
Stress related disorders
Alcohol use and dependence
Substance use and dependence
Patients may change insurance throughout the year or be enrolled in more than one plan; Patients may receive more than one diagnosis; Patients may see providers in different sub-clinics over the year.
Rights and permissions
About this article
Cite this article
Eyllon, M., Barnes, J.B., Daukas, K. et al. The Impact of the Covid-19-Related Transition to Telehealth on Visit Adherence in Mental Health Care: An Interrupted Time Series Study. Adm Policy Ment Health 49, 453–462 (2022). https://doi.org/10.1007/s10488-021-01175-x
- Telemental health
- Behavioral health
- Mental health