Abstract
The current study (1) characterizes patterns of mental health service utilization over 8 years among youth who received psychotherapy in the context of a community implementation of multiple evidence-based practices (EBPs), and (2) examined youth-, provider- and service-level predictors of service use patterns. Latent profile analyses were performed on 5,663,930 administrative claims data furnished by the county department of mental health. Multinomial logistic regression with Vermunt’s method was used to examine predictors of care patterns. Based on frequency, course, cost, and type of services, three distinct patterns of care were identified: (1) Standard EBP Care (86.3%), (2) Less EBP Care (8.5%), and (3) Repeated/Chronic Care (5.2%). Youth age, ethnicity, primary language, primary diagnosis and secondary diagnosis, provider language and provider type, and caregiver involvement and service setting were significant predictors of utilization patterns. Although the majority of youth received care aligned with common child EBP protocols, a significant portion of youth (13.7%) received no evidence-based care or repeated, costly episodes of care. Findings highlight opportunities to improve and optimize services, particularly for youth who are adolescents or transition-aged, Asian-American/Pacific Islander, Spanish-speaking, or presenting with comorbidities.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Nationally representative surveys estimate that 13% of children and adolescents suffer from a mental health disorder in the U.S., but only a third to a half of these youth receive services (Merikangas et al., 2010, 2011). In a recent meta-analysis, Duong et al. found that outpatient clinics and schools were the most accessed settings for mental health services, although service rates in these settings remain below 45% among youth with elevated symptoms or diagnoses and many youth remain unserved (Duong et al., 2020). Furthermore, among youth who initiate services, evidence suggests that the continuity of that care is poor. For example, Saloner et al. (2014) found that only 33% of youth received minimally adequate care, defined as four or more total visits with a mental health professional. A recent report from Mental Health America found that only 27% of youth with severe depression accessed more than seven visits total in the past year (Mental Health America, 2020). Such sobering figures highlight the continued need to address mental health care gaps among children and adolescents.
Evidence-based practices, or EBPs, are increasingly implemented across publicly funded mental health services for youth as policymakers and payors strive for quality improvement. Many EBPs exist for common presenting concerns for youth such as depression (Weersing et al., 2017), anxiety (Higa-McMillan et al., 2016), trauma (Dorsey et al., 2017), and disruptive behaviors (e.g., Kaminski & Claussen, 2017; McCart & Sheidow, 2016). EBPs are typically designed to be time-limited, focused treatment, often ranging from 12 to 20 sessions. In a study examining the compatibility of EBPs in the context of community mental health, Whiteside et al. (2020) used 13 sessions as a benchmark to assess whether youth received a sufficient dose of evidence-based care in a single year based on average number of sessions in common child treatment manuals and previous efficacy trials. However, there is limited empirical data on the minimum treatment length required for improved outcomes and mixed results on a dose response relationship for youth receiving care in community mental health (Bickman et al., 2002; Kirk et al., 2019).
There is surprisingly limited research on mental health service utilization patterns among youth and this has been identified as a barrier to policy reforms to improve quality of care (Saloner et al., 2014). The research that has been conducted to date has examined youth mental health service use based on number of sessions attended and overall duration of treatment (e.g., Saloner et al., 2014), and does not tell us how youth engage with mental health services. For example, it is unclear if care is episodic, chronic, continuous, or sporadic in course and at what intensity care is delivered. Empirical approaches such as latent class or latent profile analyses can allow us to explore patterns of use related to course and intensity. Past efforts on understanding mental health service utilization patterns have been limited to specific subpopulations (e.g., immigrants, refugees, specific diagnostic groups; Anderson et al., 2013; Derr, 2016; Olfson et al., 2003; Pastor et al., 2017; Salami et al., 2019; Wu et al., 2001). An exception is work by Reid et al. in Ontario, Canada. In two studies (2011, 2019), these authors identified five distinct patterns of care among youth receiving mental health services across 4–5 years. They found significant variability in the course and intensity of services, such that the majority of children (53%) received minimal care, characterized by infrequent visits over the course of a few months, while 23% of youth received prolonged periods of care for more than 2 years. About 20% of youth received a single discrete/acute episode of care with regular visits over 9.6 months. There was also a pattern of episodic care (8%), characterized by infrequent visits across multiple episodes of care spanning 3.5 years. The final two patterns were intensive (13%) and ongoing/intensive-episodic (6%), characterized by high volume of visits over multiple years. This pioneering work challenges the field to consider how common patterns of service receipt align with treatment protocols as tested in effectiveness trials and alternative models of care delivery that address a range of levels of need (e.g., triage or stepped-care models).
A major gap in the youth mental health services literature is the lack of data on the delivery of EBPs and associated patterns of psychotherapy service utilization in routine care settings. Several county- and state-wide EBP initiatives exist including in Los Angeles County (Regan et al., 2017), New York State (Acri et al., 2019), and Washington State (Stoner, 2018). Despite increasing implementation of EBPs, we know little about youth service utilization patterns within community EBP implementation efforts and what factors may affect patterns of service use in these contexts. Triplett et al. (2021) surveyed 376 therapists who participated in a state-funded EBP training initiative in Washington State about their delivery of EBPs following completion of training and consultation. When asked to report on their delivery of an EBP protocol for their most “complete” case, therapists reported delivering 12.4 sessions on average. Furthermore, whereas 20.1% of therapists reported not having completed the EBP with the client, 21.2% of therapists reported that the client’s treatment discontinued after completing the EBP, and 58.7% of therapists reported continuing therapy (with or without continued delivery of EBP elements). These findings suggest that service utilization patterns may influence EBP delivery in routine care. More research is needed to understand the range of patterns of service utilization in a context of multiple EBPs implementation in routine mental health care systems and factors associated with more and less optimal patterns of care.
Drawing on Anderson’s behavioral model of health service utilization (Andersen & Davidson, 2007), several predictors at the youth, parent, and provider level have been identified to be associated with patterns of mental health service use. Research reveals racial/ethnic disparities in mental health service utilization; youth from racial/ethnic minority groups (e.g., Black, Latinx, Asian) are less likely to enter mental health services (Garland et al., 2005; Le Cook et al., 2013; Saloner et al., 2014) and more likely to drop-out of care prematurely (de Haan et al., 2018) relative to non-Hispanic Whites. Youth’s gender is also associated with service use, such that caregivers are more likely to seek care for boys than girls (Merikangas et al., 2010; Reid et al., 2019; Saloner et al., 2014). Scholarship focusing on the effect of youth’s age is limited to older studies and show mixed results (Edlund et al., 2002; Hazen et al., 2004; Stein et al., 2013). Unsurprisingly, perceptions of need for treatment and the presence of comorbidity and impairment are associated with increased service use (Chavira et al., 2009; Reid et al., 2019; Ryan et al., 2015); however, to our knowledge, no research to date has examined differential patterns of service use by youth diagnosis. Caregiver involvement in youth therapy is critical in most EBPs for externalizing behaviors (Kaminski & Claussen, 2017) and has been associated with larger treatment effects in cognitive behavioral therapy for youth internalizing behaviors as well (Sun et al., 2019). In community service contexts serving high proportions of racial/ethnic minority families, lower caregiver engagement has been found when therapists are unable to deliver services in non-English languages (Barnett et al., 2019). Ultimately, identifying predictors of mental health service use patterns can inform strategies for improving treatment access and quality.
The Current Study
There is a dearth of research on youth’s service utilization patterns in community mental health systems implementing EBPs. Past research has focused on describing the types or rates of services received (i.e., number of sessions attended), focused on subpopulations (i.e., a particular diagnosis or demographic group), with brief study duration, and in systems where no EBPs are implemented. The current study extends the children’s mental health services and implementation research literature by (1) characterizing patterns of mental health service utilization among youth receiving psychotherapy using latent profile analyses within a system employing multiple EBPs, (2) focusing on the largest county public mental health service system in the U.S. serving youth with a variety of common presenting concerns, and (3) exploring patterns of care over 8 years of a community EBP implementation initiative. To the extent that the EBP implementation initiative supported the delivery of structured, high-quality care, we may expect to see evidence of service use pattern aligned with these parameters. Furthermore, identifying factors associated with distinct patterns of utilization may inform how to optimize delivery of care within community EBP implementation efforts.
Method
Study Context
Data were collected as part of the Knowledge Exchange on Evidence-Based Practice Sustainment (4KEEPS) study [R01MH100134; MPIs: Lau and Brookman-Frazee (2015)], which examined the sustainment of multiple child EBPs within a system-driven implementation by the Los Angeles County Department of Mental Health (LACDMH, 2016). LACDMH is the largest county public mental health system in the United States and serves more than 250,000 individuals yearly (LACDMH, 2016). As part of the Prevention and Early Intervention (PEI) initiative that began in 2010, LACDMH implemented a fee-for-service reimbursement of EBPs for children and youth across all of their agencies (see LACDMH, 2016 for details about PEI initiative). PEI is intended for youth early in their course of illness who can benefit from brief, time-limited EBPs. Providers were required to deliver approved EBPs (e.g., Trauma Focused Cognitive Behavior Therapy, Parent–Child Interaction Therapy, Triple P, etc.) in order to receive reimbursement to PEI-allocated funds within this model. Other funding mechanisms (e.g., County General Funds, Medicaid) do not impose requirements on EBP delivery for reimbursements. The specific EBP delivered was specified and linked to a procedure code into the billing process. LACDMH specified the expected frequency and treatment length for each EBP (see Table 1 for a list of the most frequently used EBPs in the county) and the recommended maximum treatment episode duration is 18 months. The system-driven implementation of EBPs provides a unique opportunity to characterize mental health service utilization for an entire population served within a single public health system and to examine how patterns of care align with expected EBP course of treatment in an implementation as usual context.
Procedure
This study utilized administrative claims data furnished by LACDMH. All available outpatient claims to PEI funds for Fiscal Year 2009/2010 to Fiscal Year 2017/2018 for children and transition-aged youth (≤ 25 years old) were extracted (n = 6, 914,533 claims). Because the goal of the study was to characterize patterns of mental health service utilization among youth receiving psychotherapy, only youth who had at least one psychotherapy claim were included in analyses. Youth who exclusively received other services (e.g., medical management only) were omitted (n = 20,462; 17%). To ensure that each youth had at least one year opportunity to receive services during the study period, we removed youth whose first claim was initiated in the last fiscal year. The final sample included 5,663,930 claims for 138,359 youth, submitted by 13,067 treatment providers. Characteristics of the youth and therapist samples are found in “Results” below. Study procedures were approved by Institutional Review Boards at [University of California San Diego, University of California Los Angeles, and Los Angeles County Department of Mental Health].
Service Use Indicators
Eight indicators were derived from claims data to characterize the pattern of service utilization for each youth. Each claim was associated with the date of service, procedure code (e.g., individual psychotherapy, group psychotherapy, family therapy, collateral, medication management, crisis management, evaluation/assessment, etc.), EBP code (e.g., MAP, TF-CBT, no/unknown EBP), provider language, provider discipline/type, service setting (e.g., office, home, school, etc.), youth demographics (e.g., age, gender, race/ethnicity, primary diagnosis, language), and cost of service. Service use indicators were developed to characterize the volume, continuity/frequency, duration, EBP receipt, and cost of services.
Volume
The total volume of claims was indexed by total psychotherapy claims and total non-psychotherapy claims. Total psychotherapy claims (TotPsych) were calculated by summing counts of all psychotherapy (e.g., individual, group, family therapy) claims per youth. Total non-psychotherapy claims (TotNonPsych) were the sum of all other claims (e.g., medication management, case management, crisis management, evaluation/assessment) per youth.
Continuity
Continuity was indexed in two ways. First, the average number of days between psychotherapy claims was calculated per youth (GapMean). Second, episode count was indexed by total number of episodes (TotEpisode), with a new episode defined when there was 180 or more days between consecutive psychotherapy claims per youth-provider dyad.
Duration
Duration of service was calculated as the number of active months each youth had a psychotherapy claim (ActiveMonth).
EBP Receipt
EBP Receipt was indexed by the number of unique EBPs received and the percentage of psychotherapy claims not tied to an EBP. The number of EBPs was the count of total unique EBPs received by a youth (EBPCount). The percentage of no EBP was calculated as the percentage of psychotherapy claims tied to a no/unknown EBP code (%NoEBP).
Cost
Total cost was calculated as the sum of costs associated with all claims per youth.
Service Use Predictors
Youth demographics, provider demographics, and service characteristics were included as predictors.
Youth Demographics
The following youth demographic variables associated with youth’s first claim were used: age, gender, race/ethnicity, language, primary diagnosis, secondary diagnosis, and year of first claim. Age was categorized as childhood (0–12), adolescence (13–17), and transition-aged (18–25). Race/ethnicity was categorized as non-Hispanic White, Hispanic, Black/African American, Asian American/Pacific Islander, Another Minoritized Ethnicity, and Not Reported. Youth’s preferred primary language was dichotomized into Spanish or English. Primary diagnosis was categorized into Internalizing Disorders (e.g., anxiety disorders, mood disorders), Externalizing Disorders (e.g., attention-deficit/hyperactivity disorder, disruptive behavior disorders), Trauma or Stress-Related Disorders (e.g., post-traumatic stress disorder, adjustment disorders), and Other Disorders. Because the majority of youth did not have a secondary diagnosis (70.9%), secondary diagnosis was dichotomized into a binary variable (yes/no). Year of first claim was included as a covariate to account for youth’s varying opportunity for ensuing service utilization during the study period.
Provider Demographics
The following provider demographics represent information associated with the youth’s first claim. Provider discipline/type was dichotomized into mental health therapists (i.e., Marriage and Family Therapists, Social Workers, Counselors, Psychologists, Trainees) versus other providers (i.e., Psychiatrist, Nurse, Rehabilitation Practitioner, Occupational Therapist, Other). Language, or language(s) that providers reported they can deliver services in, was dichotomized into English-only and Language(s) other than English.
Service Characteristics
Service setting for the care provided was dichotomized into Office Only or Community (i.e., a youth had at least some claims from a community setting such as home or school). The percentage of caregiver involved claims was calculated per youth and categorized as 0 (less than 50% caregiver involvement) and 1 (50% or more caregiver involvement). The following types of psychotherapy procedural codes indexed caregiver involvement: collateral, family therapy, family therapy with client, and multifamily group therapy.
Data Analytic Plan
There was no to minimal missing data for the majority of variables (no missing data for procedure code, service setting, service date, cost, or youth’s age; < 3% missing for therapist language and discipline and youth’s gender, diagnosis, and primary language, and EBP code). Missing EBP code (n = 6233 claims) was classified as “no EBP”, missing youth’s ethnicity (n = 21,406; 13.9% of youth) was classified as “Ethnicity not reported,” and missing youth’s secondary diagnosis (n = 98,093; 70.9% of youth) was classified as “No Secondary Diagnosis.”
Latent profile analyses (LPA) were conducted to determine underlying subgroups with different patterns of care using eight indicators related to Volume, Continuity, Duration, EBP Receipt, and Cost. LPA was particularly suited because it does not assume that the population sampled is homogenous, does not assume a priori the number of groups/profiles, and enables the identification of patterns where many continuous variables are considered simultaneously (Stanley et al., 2017). LPA was conducted using full maximum likelihood estimation using Mplus 8 (Muthén & Muthén, 2017). Models were built successively with increasing number of profiles beginning with 1, and model fit was assessed. Models were tested on the full sample as well as five subsamples randomly drawing 15% of the sample (n ~ 20,753 youth per subsample) to examine the robustness of LPA solutions. Although no common standard criteria exist for LPA model selection, several guidelines and rules of thumb have been suggested. The following criteria were used to determine the optimal number of profiles to retain: (1) lower Bayesian information criterion (BIC) and sampled-adjusted Bayesian information criterion (SABIC), (2) a statistically significant bootstrapped likelihood ratio test (BLRT; indicating the additional profile resulted in improved model fit), (3) high posterior probabilities for each profile (≥ 0.70), (4) no less than 1% of sample in a given profile, and (5) interpretability of profiles, theoretical meaningfulness, and parsimony (Jung & Wickrama, 2008; Marsh et al., 2009; Nylund et al., 2007; Stanley et al., 2017; Tein et al., 2013). Due to computational demands required to conduct the BLRT, random draws of 1% of the sample (n ~ 1540 individuals per subsample) was selected to run the BLRT and this process was iterated five times for each LPA model. For interpretation of the best-fitting model, indicator scores were z-scored, with scores above + 0.5 labeled as high, scores less than − 0.5 labeled as low, and scores in between labeled as average (Ekblom-Bak et al., 2020).
To examine predictors of care patterns/profile membership, Vermunt’s method was used. Collier and Leite (2017)’s Monte Carlo simulation found that Vermunt’s method, a type of three-step estimation approach, yielded more accurate results for predictors of profile membership than single-step and other three-step approaches. Vermunt’s method first estimates profile membership using only indicator variables, then estimates the most likely profile membership using posterior probabilities, and finally regresses profile membership on predictor variables while accounting for measurement error (Vermunt, 2010). Eleven predictor variables were entered as auxiliary variables with the R3STEP command in Mplus (Asparouhov & Muthén, 2014). Listwise deletion removed 1888 youth for the prediction model resulting in a sample of 136,471 youth (98.6%). Chi-square tests indicated statistically significant differences on the predictor variables between the retained and excluded sample of youth, however, effect sizes were close to 0 (range − 0.01 to 0.09). Given the sensitivity of chi-square test and small effect sizes of differences between retained and excluded youth, data were treated as Missing Completely at Random (MCAR). Listwise deletion of missing data is acceptable under the MCAR mechanism (Enders, 2010).
Results
Youth and Service Characteristics
Sample characteristics are presented in Table 2. The youth sample (n = 138,359) consisted of slightly more males (54.6%) then females (45.4%) with a mean age of 11.82 years (SD = 4.79 years). Youth were predominantly from an ethnic minoritized group (85.7%), with the largest representation of Hispanic/Latinx youth (67.4%). Youth’s primary diagnoses varied across common mental health concerns seen in community mental health, including Mood Disorders (19.6%), Trauma and Stress-Related Disorders (19.6%), Disruptive Behavior Disorders (19.1%), and Anxiety Disorders (12%). The majority of youth did not have a secondary diagnosis (70.9%). Youth were served by providers (n = 13,067) representing multiple disciplines, with social workers and marriage and family therapists comprising 49.2% of providers. A sizable portion of providers were classified as providing services in a language other than English (29.9%).
The majority of mental health service claims were psychotherapy claims (72.8%). Of those, 40.9% of claims involved a caregiver or family members. Services were primarily provided in an office setting (65.8%). The majority of psychotherapy claims were associated with a specified EBP (92.1%) and 7.9% of psychotherapy claims were not.
Aim 1: Patterns of Service Utilization
LPA models specifying 1 to 4 profiles were conducted. Fit indices for LPA models are presented in Table 3. BIC and SABIC values decreased with each additional profile and the BLRT indicated each additional profile significantly improved fit. However, the four-profile solution yielded profiles represented by less than 1% of the sample, limiting its utility. Thus, considering all fit criteria and interpretability, the three-profile model was retained. The three profiles are presented in Fig. 1 and means on indicators across profiles are presented in Table 4.
Mental health service utilization profiles. Figure illustrate z-scores on the following indicators: Total psychotherapy claims (TotPsych), total non-psychotherapy claims (TotNonPsych), average days between psychotherapy claims (GapMean), total number of psychotherapy episodes (TotEpisode), number of psychotherapy active months (ActiveMonth), percentage of psychotherapy claims with no EBP (%NoEBP), number of specified EBPs received (EBPCount), and total cost (TotCost). Scores above + 0.5 are considered high, scores less than − 0.5 are considered low, and scores in between are considered average (Ekblom-Bak et al., 2020)
Profile 1: Standard EBP Care
This profile was the largest of the sample (86.3%) and had average levels across all indicators. Youth on average received 24.98 number of psychotherapy sessions across 7.85 months, with 12.01 days between sessions. On average, only 3.14% of psychotherapy claims were not associated with EBP delivery. Examining the ratio of psychotherapy claims to non-psychotherapy claims suggest that youth in this profile primarily received psychotherapy services. Youth had on average 1.72 discrete episodes of care. Average cost per youth was $6460.87.
Profile 2: Less EBP Care
This profile comprised 8.5% of youth. Youth represented in this profile received on average 19.52 psychotherapy sessions across 6.81 months, with 13 days between sessions. These sessions are primarily not associated with EBP delivery (M = 84.39% of psychotherapy claims were no EBP claims). Youth in this profile received roughly comparable numbers psychotherapy and non-psychotherapy claims and had on average 1.94 discrete episodes of care. Average cost per youth was $5301.20.
Profile 3: Repeated/Chronic Care
This profile represented a small proportion of youth (5.2%). Youth on average received 126.13 sessions of psychotherapy across 35.86 months, with 9.58 days between sessions. They received multiple specified EBPs (M = 2.4), had a high number of psychotherapy claims (M = 126.13), and other types of mental health services (M = 57.42), and had high numbers of discrete episodes of care (M = 6.36). Average cost per youth was $30,925.86.
Aim 2: Predictors of Service Utilization Patterns
Results from the multinomial logistic regression using Vermunt’s method revealed several significant predictors on profile membership. Profile 1, Standard EBP Care, was used as the reference group. Thus, a significant predictor indicated higher or lower risk of being in one of the other profiles relative to the Standard EBP Care group. See Table 5 for results.
Profile 2: Less EBP Care
Youth’s age, language, ethnicity, primary diagnosis, secondary diagnosis, and year of first claim, caregiver involvement in psychotherapy, provider type, and service setting were significant predictors of youth’s membership in the Less EBP Care pattern. Adolescents and transition-aged youth had greater odds of receiving Less EBP Care relative to children, with effects particularly pronounced for transition-aged youth. Youth whose primary language was Spanish had greater odds of being in the Less EBP Care group relative to youth whose primary language was English. Asian American/Pacific Islander youth had greater odds of being in the Less EBP Care group relative to Hispanic youth. Youth with primary diagnoses of externalizing disorders and trauma/stress-related disorders were less likely to be in the Less EBP Care group relative to youth with primary internalizing diagnoses. Youth with a secondary diagnosis were also less likely to be in the Less EBP Care group. Higher caregiver involvement in psychotherapy was associated with greater odds of being in the Less EBP Care group, whereas having a therapist as a provider was associated with lower odds. Exclusive office-based care was associated with higher odds of being in the Less EBP Care group relative to youth receiving services in other settings. All of these effects are significant when accounting for youth’s year of first claim. Youth’s gender and provider language were not significant predictors.
Profile 3: Repeated/Chronic Care
Youth’s age, gender, ethnicity, primary diagnosis, secondary diagnosis, year of first claim, caregiver involvement in psychotherapy, provider type, provider language, and service setting were significant predictors of youth’s membership in the Repeated/Chronic Care pattern. Adolescents and transition-aged youth had again greater odds of being in the Repeated/Chronic Care group relative to children, with effects more pronounced for transition-aged youth. Males had greater odds of being in the Repeated/Chronic Care group relative to females. Youth who are Non-Hispanic White or from a minoritized racial/ethnic group (other than Black or Asian American/Pacific Islander) had greater odds of being in the Repeated/Chronic Care group relative to Hispanic youth. Youth with primary externalizing, trauma/stress-related, and other diagnoses (e.g., substance use) had greater odds of being in the Repeated/Chronic Care group relative to youth with primary internalizing disorders. Youth with a secondary diagnosis had greater odds of being in the Repeated/Chronic Care group and this was one of the strongest effects in the model. Youth with higher caregiver involvement in psychotherapy and who were served by a therapist who spoke a language other than English were associated with lower odds of being in the Repeated/Chronic Care group. Youth receiving exclusively office-based care had lower odds of being in the Repeated/Chronic Care group relative to youth receiving services in other settings, and this effect was large. All of these effects are significant accounting for youth’s year of first claim.
Discussion
The present study examined characteristics of mental health service utilization among youth receiving psychotherapy and determined multiple youth-, provider-, and service-level predictors of utilization patterns within a large administrative dataset reflecting nearly six million claims over eight years in the context of a system-driven multiple EBPs implementation. Based on frequency, course, cost, and type of services, three distinct profiles of care were identified: (1) Standard EBP Care, (2) Less EBP Care, and (3) Repeated/Chronic Care. Although service receipt does not indicate level of need or clinical outcomes, understanding different profiles of care and characteristics that predict who accessed which profile of care have implications for efforts to address disparities with the goal of optimizing services for youth.
The clear majority of youth in the system (86.3%) received services designated as Standard EBP Care. Youth in this profile received psychotherapy sessions relatively consistently, both in attendance and EBP focus, over a service period that lasted under eight months. This finding may speak to the success of the system-wide implementation efforts in LACDMH over the past decade: the majority of youth receiving PEI funding received EBP psychotherapy services on a timeline congruent with the time-limited nature of many child EBPs. This finding is consistent with a recent evaluation of the PEI program in LACDMH. Ashwood et al. (2018) demonstrated that PEI services were associated with clinical improvements: of the youth who entered care with clinically significant symptoms, over half fell below the clinical significance threshold. Other studies focused on the sustainment of EBPs in LACDMH have also documented the specific implementation conditions—including the availability of EBP consultation along with therapist and program leader perceptions about EBPs (Lau et al., in press)—that have likely contributed to the prevalence of this utilization pattern.
In comparison to Reid et al. (2019), our findings as a whole are more optimistic: the majority of youth in the Reid et al. study (53%) received “Minimal” care and only 8% of youth received “Acute” care, which is comparable to the duration and number of sessions in our Standard EBP Care profile. In terms of the volume and duration of services, our finding that Standard EBP Care was delivered, on average, over 7.85 months is largely on par with the delivery time of an effectiveness trial testing EBPs in community settings, as were the number of days that passed between sessions, although the total number of sessions attended was higher in our sample (Weisz et al., 2012). Similarly, the mean number of sessions in the Standard EBP Care profile is higher than what was reported by Triplett et al. (2021), who found youth received an average of 12.4 sessions in a state-funded implementation of cognitive-behavioral therapy based on therapist report. Triplett et al. noted that the majority of youth (58.7%) in their sample continued to receive therapy after completing a course of EBP and that the majority of these continued therapy sessions included EBP elements (72.3%), more similar to the Repeated/Chronic Care pattern.
EBPs are time-limited in nature and designed to be delivered in a standard number of weekly psychotherapy sessions (e.g., TF-CBT is designed to be delivered in 12–15 sessions; Cohen & Mannarino, 2015). Although it is reassuring that youth accessed an average number of sessions ample enough to complete a full course of an EBP as designed, we must also consider the volume of sessions and the duration of the episode of care within a larger treatment literature. Studies describing mental health service dose and response are limited and inconsistent. For example, whereas one study with adults found that a minimum of four therapy sessions were necessary to achieve more than 50% reliable and clinically significant improvement (Delgadillo et al., 2014), Anderson et al. (2001) found that somewhere between 11 and 16 therapy sessions resulted in 50% clinically significant improvement. From the children’s mental health services literature, there is some evidence that the type of treatment moderates the dose–response relation. In a sample of children receiving usual care services, Bickman et al. (2002) found youth response was unrelated to the amount of treatment received. More recently, youth receiving cognitive-behavioral therapy in a school mental health program experienced full benefits of the treatment over the course of 14 sessions (Kirk et al., 2019). An area for future research is to connect utilization data to clinical outcomes. Further, research can dissect the Standard EBP Care pattern at a more granular level and additional correlates not examined in this study. This can help further our understanding of standard EBP practice in community mental health and differential associations with psychotherapy process, quality, engagement, and outcomes.
A portion of youth in the sample (8.5%) received a pattern of utilization labeled as Less EBP Care. This pattern is characterized by relatively consistent psychotherapy sessions across a limited time-frame (7 months), but with sessions that were not associated with the delivery of EBPs according to administrative claims. The lack of EBP sessions relative to the other utilization patterns may signal ‘lower quality’ services or implementation failure. Given the literature documenting EBPs associated with better youth outcomes (Weisz et al., 2013), this utilization pattern indicates room for improvement. There is, however, some evidence that youth in this care pattern may have different needs or may be utilizing mental health services in a different way. In particular, youth in this pattern had the highest number of crisis management claims relative to the other two care patterns, which may have reduced reliance on EBPs.
Several youth characteristics were associated with Less EBP Care. Whereas younger children were more likely to receive Standard EBP Care, adolescents and transition-aged youth were more likely to receive Less EBP Care. Youth’s age emerged as one of the strongest predictors. Research has documented a lack of sensitivity to developmental differences between children and adolescents among existing prevention and intervention programs (Malti, et al., 2016). Our findings suggest there was difficulty retaining youth of older ages in EBP care and may reflect a need to tailor existing EBPs to more effectively engage older youth and address their unique mental health service needs (Malti, et al., 2016).
Spanish-speaking youth were more likely than their English-speaking counterpart to be classified in Less EBP Care, possibly highlighting incongruence between existing EBPs with Spanish-speaking populations and/or the need to translate existing EBPs into additional languages for better service engagement. Furthermore, Asian American/Pacific Islander youth were more likely than Hispanic youth to receive Less EBP Care. This finding may be consistent with literature documenting challenges in engaging Asian American youth or families in evidence-based interventions or needed services (Guo et al., 2014; Kim et al., 2018; Kodish et al, 2020). This may point to the urgent need for culturally sensitive care and efforts to identify strategies to better align intervention with the needs of Asian American youth and families. It could also suggest the need to work closely with therapists who serve Asian American families to ensure they feel efficacious in tailoring EBPs for their clients’ needs (Lau et al., 2010). In contrast, Hispanic youth in this sample were more likely to be represented in the Standard EBP Care group. Although this finding may be evidence that the LACDMH system has adapted to address the needs of its most populous subgroup, Hispanic youth, we must also highlight that Hispanic youth still represent a sizable proportion of the other service patterns; Hispanic youth have variable experiences in EBP service use within the system. Finally, youth with externalizing and trauma/stress-related disorders were less likely to be in the Less EBP Care group. This may speak to the availability of EBPs for these specific presenting concerns within the LACDMH implementation context. For example, LACDMH furnished training and implementation supports for nine EBPs for trauma and twelve EBPs for disruptive behaviors and conduct problems, versus three EBPs for anxiety.
Several provider and service characteristics were also related to Less EBP Care. Youth who exclusively received office-based care were more likely to be in the Less EBP Care group relative to youth who received services in other settings. Coupled with the finding that youth with a secondary diagnosis were also less likely to be in the Less EBP Care group, it may be that youth with more intensive needs are more likely to receive in-home care than youth with less intensive needs. Additionally, the Less EBP Care group appear to receive a higher percentage of non-psychotherapy services (e.g., case management, evaluation and assessment) which may only be offered in clinics. Higher caregiver involvement was also associated with Less EBP Care. This finding is somewhat surprising and is difficult to reconcile with the notion of caregiver attendance in child therapy as a quality indicator of psychotherapy and a precondition for the delivery of many EBP strategies (Barnett et al., 2019; Sun et al., 2019; Wright et al., 2019). This finding may reflect a lower preponderance of caregiver-mediated EBPs for internalizing versus externalizing problems broadly and within in the LACDMH implementation specifically. Indeed, further inspection revealed a higher proportion of youth with primary internalizing disorders in this care pattern relative to the other two patterns. Thus, when caregivers are in attendance, therapists may be more likely to deliver interventions other than those structured by PEI EBPs.
The final utilization pattern, Repeated/Chronic Care (5.2%), represents repeated, costly episodes of care with a greater number of unique EBPs used. Youth in this care pattern may experience higher acuity of mental health problems and require repeated and longer-term care. Although the current study cannot adequately quantify the level of need among youth in this care pattern without clinical data, clinical comorbidity as indicated by the presence of a secondary diagnosis, was uniquely related to higher odds of receiving Repeated/Chronic care. Given the mono-problem focus of many EBPs, it is possible that youth may receive an EBP to first address a primary diagnosis, then another EBP to address the secondary diagnosis. However, it is also possible that other factors may contribute to this care pattern beyond clinical complexity or impairment. For example, youth may receive multiple episodes of care because they do not initially receive an EBP that fully meets their needs, either as a result of incomplete assessment or provider lack of training in an EBP that matches primary presenting concerns. Previous research in the LACDMH context has shown that therapists sometimes extend the use of EBPs they are trained in to alternate problem foci than originally intended (off-label use) (Kim et al., 2018; Park et al., 2018). Future studies can examine the match between EBP received and presenting concern and how (mis)match contributes to utilization patterns and outcomes. Furthermore, future work should incorporate measures of whether youth are completing a full course of an EBP within a treatment episode vs. discontinuing prematurely and thus requiring return to care. Notably, this care pattern is associated with significant costs that are four to six times the average cost of the other two patterns. From a financial standpoint, there is an opportunity to optimize treatment response to reduce costs to families and systems.
In addition to comorbidity, several other youth characteristics predicted Repeated/Chronic Care. Adolescents and transition-aged youth were also more likely to receive Repeated/Chronic Care. This may reflect the importance of intervening earlier in development before mental health problems become more intractable (McGorry & Mei, 2020). Alternately, it may suggest that PEI EBPs—meant to address mental health conditions early in their developmental course—may not address the needs for older youth as well such that they require repeated care. Youth who are Non-Hispanic White and youth from another minoritized racial/ethnic group were more likely to be in the Repeated/Chronic Care pattern than Hispanic youth. This requires further research to identify factors that explain racial differences in repeated care. As discussed previously, this may provide further evidence that the LACDMH system has adapted to the needs of Hispanic youth, or there may be unknown differences in characteristics of groups served in LACDMH that drive these apparent inequities (e.g., other service system involvement, initial severity). Compared to youth with internalizing disorders, youth with externalizing and trauma/stress-related disorders were more likely to receive Repeated/Chronic Care. Paired with the finding that caregiver involvement was associated with a lower likelihood of being in this care pattern, this may speak to the challenge and importance of involving caregivers in care, particularly for conduct and trauma-related problems for which caregiver-mediated interventions are front line (Barnett et al., 2019).
Youth who were served by multilingual providers were less likely to be classified in the Repeated/Chronic Care pattern, suggesting the importance of a multicultural and multilingual workforce. Youth who exclusively received office-based care were less likely to be in the Repeated/Chronic Care pattern. This finding may reflect the discontinuous nature of school and home-based care. For example, therapy provided in schools may be disrupted during breaks in the academic calendar. Alternatively, youth in the Repeated/Chronic Care pattern may be more functionally impaired and require more intensive home-based services. It is also possible that youth in this care pattern may experience more barriers to attend office-based therapy such as transportation and childcare needs. Future research would benefit from examining individual, familial, and neighborhood level social determinants of care that may also impact mental health care utilization patterns.
Although this study’s focus on PEI outpatient services has precluded any group comparisons at higher levels of care, these findings highlight the persistent need for mental health systems to promote equitable care. Ongoing advocacy efforts (APA, 2019; Butler & Rodgers, 2019) emphasize the importance of providing services that (1) address contextual factors unique to youth with minoritized identities (e.g., experiences of discrimination, stigma-related barriers, diversifying the mental health workforce), (2) support families (e.g., accessible and culturally competent parent education programs), and (3) buffer against socioeconomic disadvantage (e.g., adjunct early childhood programs and housing programs).
Notably, we did not identify a pattern resembling a “drop out” group that is often documented in the literature or what Reid et al., (2011, 2019) characterized as “Minimal Care.” This may speak to the success of the LACDMH context adapting to and effectively engaging with its client population. It is also plausible that children who are found eligible for PEI care in the current context differ from children served in other outpatient public sectors where higher dropout rates have been documented. In the current sample, there were nevertheless 5.8% of youth who only received one single psychotherapy session, although combined with other service use indicators, there was not a coherent profile that corresponded to a discrete “drop out” group.
Implications
Taken together, the utilization profiles may point to the need for tailored implementation strategies and innovative models of service delivery to promote accessible, effective, and efficient care in systems implementing multiple EBPs. Findings related to the Less EBP group reveal problems with youth not receiving EBP care even in contexts resourced for implementation. More research is needed to identify mechanism(s) underlying why some youth received EBP care while some did not within a single system of care and whether tailored implementation supports may be required for providers to sustain EBP delivery. The Repeated/Chronic pattern, where youth are receiving high levels of costly care for long periods of time, highlights the importance of optimizing treatment. Youth with high-cost utilization tend to be diagnostically complex presenting with multiple comorbidities (Dickson et al., 2019). It may be expected that some portion of youth treated first within a brief, limited, early intervention approach would be subsequently identified as having more persistent service needs. The dissemination of transdiagnostic approaches that target underlying mechanisms common across child mental health problems (e.g., Unified Protocol, the FIRST treatment program) as opposed to problem-specific EBP protocols may drive more efficient care for youth with comorbidities. Measurement-based care approaches may help providers identify when clients with different diagnostic profiles, developmental needs, and social determinants of health have benefitted sufficiently from particular EBPs, redirecting care resources to more clients in need. Furthermore, stepped-care models have the potential to reduce cost and improve efficiency by first offering low-intensity interventions and only stepping up to a higher intensity intervention for youth who are not responding to treatment (Salloum et al., 2014). For example, Salloum et al. (2014) found that their stepped care TF-CBT intervention not only resulted in comparable treatment benefit compared to standard TF-CBT, it also reduced costs to families by 51.3%. This model may more efficiently serve some youth who fell in the Repeated/Chronic group by delivering low-intensity evidence-based interventions, and stepping up to weekly therapy sessions only for youth who require more intensive care.
Limitations
This study has a number of limitations to consider. Although we described service utilization profiles based on extant literature, we did not include clinical outcome data in our analyses. It is possible that youth in the Less EBP or Repeated/Chronic utilization patterns may nonetheless experience successful treatment response. These data are also limited to the PEI Program in Los Angeles County, representing a minority of mental health services provided; PEI funding represents 19% of Mental Health Services Act allocations in LACDMH. As described earlier, PEI guidelines include a number of restrictions that make it unique; for example, when billing PEI, providers are mandated to deliver from an approved menu of EBPs. Similarly, although EBPs may be delivered in different settings, PEI is intended for outpatient services for youth in the early stage of their course of illness. This reflects a relatively low ceiling to the acuity of the youth included in this sample. Furthermore, we do not have data regarding youth’s receipt of services outside of PEI if PEI did not meet their needs or if they were deemed ineligible to receive PEI care at some point. Many additional factors that likely influence service utilization were not addressed in this study, such as social determinants of health including access issues (e.g., transportation) and poverty-related barriers to care (e.g., caregivers requiring childcare). Future studies may also explore the influence of systemic and policy factors that may influence utilization patterns over time.
Conclusion
Using latent profile analyses, this study identified three distinct patterns of outpatient mental health service utilization in a community multiple EBPs mental health system. Within this context, the majority of youth received services aligned with many child EBP protocols, speaking to the success of the system-wide implementation of EBPs. However, a portion (13.7%) of youth received care patterns designated as Less EBP or Repeated/Chronic Care. Findings highlight opportunities to improve and optimize services, particularly for youth who are adolescents or transition-aged, Asian-American/Pacific Islander, Spanish-speaking, or presenting with comorbidities.
Data Availability
Data analyzed in this study are owned by the Los Angeles Department of Mental Health (LACDMH), and as such, are not publicly available. Data may be available from LACDMH upon reasonable request and with permission.
References
Acri, M., Fuss, A. A., Quintero, P., Hoagwood, K., McKay, M. M., & Cleek, A. (2019). Disseminating clinical and fiscal practices across the New York State behavioral healthcare system. Social Work in Health Care, 58(6), 557–563.
American Psychological Association. (2019). Promoting positive mental health among racial/ethnic minority children: Ensuring and enhancing services, programs, and resources. Retrieved from https://www.apa.org/pi/families/resources/positive-mental-health.pdf
Andersen, R. M., & Davidson, P. L. (2007). Improving access to care in America: Individual and contextual indicators. Changing the US Health Care System: Key issues in health services policy and management (pp. 3–31). Wiley.
Anderson, E. M., & Lambert, M. J. (2001). A survival analysis of clinically significant change in outpatient psychotherapy. Journal of Clinical Psychology, 57(7), 875–888.
Anderson, K. K., Fuhrer, R., Wynant, W., Abrahamowicz, M., Buckeridge, D. L., & Malla, A. (2013). Patterns of health services use prior to a first diagnosis of psychosis: The importance of primary care. Social Psychiatry and Psychiatric Epidemiology, 48(9), 1389–1398.
Ashwood, J. S., Kataoka, S. H., Eberhart, N. K., Bromley, E., Zima, B. T., Baseman, L., … Burnam, M. A. (2018). Evaluation of the mental health services act in Los Angeles County: Implementation and outcomes for key programs. Rand Health Quarterly, 8, 2. https://doi.org/10.7249/RR2327
Asparouhov, T., & Muthén, B. (2014). Auxiliary variables in mixture modeling: Three-step approaches using M plus. Structural Equation Modeling: A Multidisciplinary Journal, 21(3), 329–341.
Barnett, M. L., Lau, A. S., Lind, T., Wright, B., Stadnick, N. A., Innes-Gomberg, D., Pesanti, K., & Brookman-Frazee, L. (2019). Caregiver attendance as a quality indicator in the implementation of multiple evidence-based practices for children. Journal of Clinical Child & Adolescent Psychology, 49(6), 1–15.
Bickman, L., Andrade, A. R., & Lambert, E. W. (2002). Dose response in child and adolescent mental health services. Mental Health Services Research, 4(2), 57–70.
Butler, A. M., & Rodgers, C. R. (2019). Developing a policy brief on child mental health disparities to promote strategies for advancing equity among racial/ethnic minority youth. Ethnicity & Disease, 29(Suppl 2), 421–426.
Chavira, D. A., Garland, A., Yeh, M., McCabe, K., & Hough, R. L. (2009). Child anxiety disorders in public systems of care: Comorbidity and service utilization. The Journal of Behavioral Health Services & Research, 36(4), 492–504.
Cohen, J. A., & Mannarino, A. P. (2015). Trauma-focused cognitive behavior therapy for traumatized children and families. Child and Adolescent Psychiatric Clinics, 24(3), 557–570.
Collier, Z. K., & Leite, W. L. (2017). A comparison of three-step approaches for auxiliary variables in latent class and latent profile analysis. Structural Equation Modeling: A Multidisciplinary Journal, 24(6), 819–830.
de Haan, A. M., Boon, A. E., de Jong, J. T., & Vermeiren, R. R. (2018). A review of mental health treatment dropout by ethnic minority youth. Transcultural Psychiatry, 55(1), 3–30.
Delgadillo, J., McMillan, D., Lucock, M., Leach, C., Ali, S., & Gilbody, S. (2014). Early changes, attrition, and dose–response in low intensity psychological interventions. British Journal of Clinical Psychology, 53(1), 114–130.
Derr, A. S. (2016). Mental health service use among immigrants in the United States: A systematic review. Psychiatric Services, 67(3), 265–274.
Dickson, K. S., Stadnick, N. A., Lind, T., & Trask, E. V. (2019). Defining and predicting high cost utilization in children’s outpatient mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 47, 655–664.
Dorsey, S., McLaughlin, K. A., Kerns, S. E., Harrison, J. P., Lambert, H. K., Briggs, E. C., Revillion Cox, J., & Amaya-Jackson, L. (2017). Evidence base update for psychosocial treatments for children and adolescents exposed to traumatic events. Journal of Clinical Child & Adolescent Psychology, 46(3), 303–330.
Duong, M. T., Bruns, E. J., Lee, K., Cox, S., Coifman, J., Mayworm, A., & Lyon, A. R. (2020). Rates of mental health service utilization by children and adolescents in schools and other common service settings: A systematic review and meta-analysis. Administration and Policy in Mental Health and Mental Health Services Research, 48, 420–439.
Edlund, M. J., Wang, P. S., Berglund, P. A., Katz, S. J., Lin, E., & Kessler, R. C. (2002). Dropping out of mental health treatment: Patterns and predictors among epidemiological survey respondents in the United States and Ontario. American Journal of Psychiatry, 159(5), 845–851.
Ekblom-Bak, E., Stenling, A., Salier Eriksson, J., Hemmingsson, E., Kallings, L. V., Andersson, G., Wallin, P., Ekblom, Ö., Ekblom, B., & Lindwall, M. (2020). Latent profile analysis patterns of exercise, sitting and fitness in adults–Associations with metabolic risk factors, perceived health, and perceived symptoms. PLoS ONE, 15(4), e0232210.
Enders, C. K. (2010). Applied missing data analysis. Guilford Press.
Garland, A. F., Lau, A. S., Yeh, M., McCabe, K. M., Hough, R. L., & Landsverk, J. A. (2005). Racial and ethnic differences in utilization of mental health services among high-risk youths. American Journal of Psychiatry, 162(7), 1336–1343.
Guo, S., Kataoka, S. H., Bear, L., & Lau, A. S. (2014). Differences in school-based referrals for mental health care: Understanding racial/ethnic disparities between Asian American and Latino youth. School Mental Health: A Multidisciplinary Research and Practice Journal, 6(1), 27–39.
Hazen, A. L., Hough, R. L., Landsverk, J. A., & Wood, P. A. (2004). Use of mental health services by youths in public sectors of care. Mental Health Services Research, 6(4), 213–226.
Higa-McMillan, C. K., Francis, S. E., Rith-Najarian, L., & Chorpita, B. F. (2016). Evidence base update: 50 years of research on treatment for child and adolescent anxiety. Journal of Clinical Child & Adolescent Psychology, 45(2), 91–113.
Jung, T., & Wickrama, K. A. (2008). An introduction to latent class growth analysis and growth mixture modeling. Social and Personality Psychology Compass, 2(1), 302–317.
Kaminski, J. W., & Claussen, A. H. (2017). Evidence base update for psychosocial treatments for disruptive behaviors in children. Journal of Clinical Child & Adolescent Psychology, 46(4), 477–499.
Kim, J. J., Kodish, T., Bear, L., El-Hendi, T., Duong, J., & Lau, A. S. (2018). Disparities in follow-up care for Asian American youth assessed for suicide risk in schools. Asian American Journal of Psychology, 9(4), 308–317.
Kirk, A., Michael, K., Bergman, S., Schorr, M., & Jameson, J. P. (2019). Dose response effects of cognitive-behavioral therapy in a school mental health program. Cognitive Behaviour Therapy, 48(6), 497–516.
Kodish, T., Weiss, B., Duong, J., Rodriguez, A., Anderson, G., Nguyen, H., Olaya, C., & Lau, A. S. (2020). Interpersonal psychotherapy - adolescent skills training (IPT-AST) with youth from Asian American and immigrant families: Cultural considerations and intervention process. Cognitive and Behavioral Practice, 2, 147–166.
Lau, A. S., Brookman-Frazee, L. (2015). The 4KEEPS study: identifying predictors of sustainment of multiple practices fiscally mandated in children’s mental health services. Implementation Science, 11, 31. https://doi.org/10.1186/s13012-016-0388-4
Lau, A. S., Fung, J. J., & Yung, V. (2010). Group parent training with immigrant Chinese families: Enhancing engagement and augmenting skills training. Journal of Clinical Psychology. in Session, 66, 880–894.
Le Cook, B., Barry, C. L., & Busch, S. H. (2013). Racial/ethnic disparity trends in children’s mental health care access and expenditures from 2002 to 2007. Health Services Research, 48(1), 129–149.
Los Angeles County Department of Mental Health. (2016). Prevention and early intervention implementation handbook. Los Angeles County Department of Mental Health.
Malti, T., Noam, G. G., Beelmann, A., & Sommer, S. (2016). Toward dynamic adaptation of psychological interventions for child and adolescent development and mental health. Journal of Clinical Child & Adolescent Psychology, 45(6), 827–836.
Marsh, H. W., Lüdtke, O., Trautwein, U., & Morin, A. J. (2009). Classical latent profile analysis of academic self-concept dimensions: Synergy of person-and variable-centered approaches to theoretical models of self-concept. Structural Equation Modeling: A Multidisciplinary Journal, 16(2), 191–225.
McCart, M. R., & Sheidow, A. J. (2016). Evidence-based psychosocial treatments for adolescents with disruptive behavior. Journal of Clinical Child & Adolescent Psychology, 45(5), 529–563.
McGorry, P. D., & Mei, C. D. (2020). Early intervention in youth mental health: Progress and future directions. Evidence-Based Mental Health, 21, 182–184.
Mental Health America. (2020). 2021 The State of Mental Health in America Report. Retrieved from https://www.mhanational.org/research-reports/2021-state-mental-health-america
Merikangas, K. R., He, J. P., Brody, D., Fisher, P. W., Bourdon, K., & Koretz, D. S. (2010). Prevalence and treatment of mental disorders among US children in the 2001–2004 NHANES. Pediatrics, 125(1), 75–81.
Merikangas, K. R., He, J. P., Burstein, M., Swendsen, J., Avenevoli, S., Case, B., Georgiades, K., Heaton, L., Swanson, S., & Olfson, M. (2011). Service utilization for lifetime mental disorders in US adolescents: Results of the National Comorbidity Survey-Adolescent Supplement (NCS-A). Journal of the American Academy of Child & Adolescent Psychiatry, 50(1), 32–45.
Muthén, L. K., & Muthén, B. O. (2017). Mplus: Statistical analysis with latent variables: User’s guide (Version 8). Muthén & Muthén.
Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling: A Multidisciplinary Journal, 14(4), 535–569.
Olfson, M., Gameroff, M. J., Marcus, S. C., & Jensen, P. S. (2003). National trends in the treatment of attention deficit hyperactivity disorder. American Journal of Psychiatry, 160(6), 1071–1077.
Park, A. L., Tsai, K. H., Guan, K., & Chorpita, B. F. (2018). Unintended consequences of evidence-based treatment policy reform: Is implementation the goal or the strategy for higher quality care? Administration and Policy in Mental Health and Mental Health Services Research, 45(4), 649–660.
Pastor, P. N., Simon, A. E., & Reuben, C. A. (2017). ADHD: Insurance and mental health service use. Clinical Pediatrics, 56(8), 729–736.
Regan, J., Lau, A. S., Barnett, M., Stadnick, N., Hamilton, A., Pesanti, K., Bando, L., & Brookman-Frazee, L. (2017). Agency responses to a system-driven implementation of multiple evidence-based practices in children’s mental health services. BMC Health Services Research, 17(1), 1–14.
Reid, G. J., Cunningham, C. E., Tobon, J. I., Evans, B., Stewart, M., Brown, J. B., Lent, B., Neufeld, R. W., Vingilis, E. R., Zaric, G. S., & Shanley, D. C. (2011). Help-seeking for children with mental health problems: Parents’ efforts and experiences. Administration and Policy in Mental Health and Mental Health Services Research, 38(5), 384–397.
Reid, G. J., Stewart, S. L., Barwick, M., Carter, J., Leschied, A., Neufeld, R. W., Pierre, J. S., Tobon, J. I., Vingilis, E., & Zaric, G. S. (2019). Predicting patterns of service utilization within children’s mental health agencies. BMC Health Services Research, 19(1), 1–14.
Ryan, S. M., Jorm, A. F., Toumbourou, J. W., & Lubman, D. I. (2015). Parent and family factors associated with service use by young people with mental health problems: A systematic review. Early Intervention in Psychiatry, 9(6), 433–446.
Salami, B., Salma, J., & Hegadoren, K. (2019). Access and utilization of mental health services for immigrants and refugees: Perspectives of immigrant service providers. International Journal of Mental Health Nursing, 28(1), 152–161.
Salloum, A., Scheeringa, M. S., Cohen, J. A., & Storch, E. A. (2014). Development of stepped care trauma-focused cognitive-behavioral therapy for young children. Cognitive and Behavioral Practice, 21(1), 97–108.
Saloner, B., Carson, N., & Cook, B. L. (2014). Episodes of mental health treatment among a nationally representative sample of children and adolescents. Medical Care Research and Review, 71(3), 261–279.
Stanley, L., Kellermanns, F. W., & Zellweger, T. M. (2017). Latent profile analysis: Understanding family firm profiles. Family Business Review, 30(1), 84–102.
Stein, B. D., Sorbero, M. J., Dalton, E., Ayers, A. M., Farmer, C., Kogan, J. N., & Goswami, U. (2013). Predictors of adequate depression treatment among medicaid-enrolled youth. Social Psychiatry and Psychiatric Epidemiology, 48(5), 757–765.
Stoner, S. A. (2018). Evidence-based Practices in Washington State: Report to DBHR. Seattle: Alcohol & Drug Abuse Institute, University of Washington, September. Retrieved from http://adai.uw.edu/pubs/pdf/2018EBPinWashington.pdf
Sun, M., Rith-Najarian, L. R., Williamson, T. J., & Chorpita, B. F. (2019). Treatment features associated with youth cognitive behavioral therapy follow-up effects for internalizing disorders: A meta-analysis. Journal of Clinical Child & Adolescent Psychology, 48(sup1), S269–S283.
Tein, J. Y., Coxe, S., & Cham, H. (2013). Statistical power to detect the correct number of classes in latent profile analysis. Structural Equation Modeling: A Multidisciplinary Journal, 20(4), 640–657.
Triplett, N. S., Woodard, G. S., AlRasheed, R., Meza, R. D., Robinson, S., Berliner, L., & Dorsey, S. (2021). Application of evidence-based treatment in community mental health settings: Examining EBT delivery duration and client discharge. The Journal of Behavioral Health Services & Research, 48, 1–8.
Vermunt, J. K. (2010). Latent class modeling with covariates: Two improved three-step approaches. Political Analysis, 18, 450–469.
Weersing, V. R., Jeffreys, M., Do, M. C. T., Schwartz, K. T., & Bolano, C. (2017). Evidence base update of psychosocial treatments for child and adolescent depression. Journal of Clinical Child & Adolescent Psychology, 46(1), 11–43.
Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K., Miranda, J., Bearman, S. K., Research Network on Youth Mental Health. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69(3), 274–282.
Weisz, J. R., Kuppens, S., Eckshtain, D., Ugueto, A. M., Hawley, K. M., & Jensen-Doss, A. (2013). Performance of evidence-based youth psychotherapies compared with usual clinical care: A multilevel meta-analysis. JAMA Psychiatry, 70(7), 750–761.
Whiteside, S. P., Sim, L. A., Morrow, A. S., Farah, W. H., Hilliker, D. R., Murad, M. H., & Wang, Z. (2020). A meta-analysis to guide the enhancement of CBT for childhood anxiety: Exposure over anxiety management. Clinical Child and Family Psychology Review, 23(1), 102–121.
Wright, B., Lau, A. S., & Brookman-Frazee, L. (2019). Factors associated with caregiver attendance in the community implementation of multiple evidence-based practices in children’s mental health services. Psychiatric Services, 70, 808–815.
Wu, P., Hoven, C. W., Cohen, P., Liu, X., Moore, R. E., Tiet, Q., Okezie, N., Wicks, J., & Bird, H. R. (2001). Factors associated with use of mental health services for depression by children and adolescents. Psychiatric Services, 52(2), 189–195.
Funding
Funding for this research project, including the study design and data collection, analysis and interpretation of data, was supported by NIMH Grant R01MH100134 awarded to authors Lauren Brookman-Frazee and Anna Lau.
Author information
Authors and Affiliations
Contributions
JHLL assisted with data acquisition, led data analyses and interpretation, and led the writing of the manuscript. LB and AL conceived and supervised the study, provided input on statistical models and data interpretation, and edited the manuscript. ALV assisted with data analyses and interpretation and editing of the manuscript. JRC assisted with editing of the manuscript. DI, KT, and KP furnished data, provided input on data analyses and interpretation, and edited the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
Ethical Approval
Study procedures were approved by multiple institutional review boards (University of California, Los Angeles, University of California, San Diego) and the Human Subjects Research Committee at Los Angeles County Department of Mental Health (LACDMH). The study involved secondary analyses of administrative claims data; thus consents were not applicable. All research activities were performed in accordance with the ethical standards articulated in the 1964 Declaration of Helsinki, its later amendments, and the 1979 Belmont Report. Standards of reporting American Psychological Association’s Journal Article Reporting Standards for Quantitative Research in Psychology.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Lui, J.H.L., Brookman-Frazee, L., Vázquez, A.L. et al. Patterns of Child Mental Health Service Utilization Within a Multiple EBP System of Care. Adm Policy Ment Health 49, 506–520 (2022). https://doi.org/10.1007/s10488-021-01179-7
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-021-01179-7