Introduction

Women with a history of physical or sexual abuse are much more likely to develop substance abuse and mental health problems or co-occurring disorders than those without it (Rosenberg et al. 1996; MacMillan et al. 2001; Najavits et al. 1997; Goodman et al. 1995). Conversely, women with co-occurring disorders are also likely to be exposed to environments and relationships that are prone to violence (Harris 1996). Consequently, physical or sexual abuse victimization is highly prevalent among women with co-occurring disorders. Between 50 and 70% of women in psychiatric care and 55–99% of women with substance abuse disorders reported being abused (Najavits et al. 1997; Briere and Zaidi 1989; Cascardi et al. 1996).

Experience of physical or sexual abuse is likely to significantly complicate treatments of co-occurring disorders (Brown 2000). In the current service delivery systems, however, experience of physical violence or sexual abuse has been significantly under-recognized and thus has not been adequately addressed in the treatment co-occurring disorders (Harris 1996; Rosenberg et al. 1996; Brunette and Drake 1998).

Growing evidence suggests an integrated approach, in which co-occurring disorders are treated simultaneously, is more effective than treating each disorder separately (Drake et al. 1998; Nuttbrock et al. 1998; De Leon et al. 2000). However, little research has examined the effectiveness of trauma-informed services in the treatment of co-occurring disorders (e.g., Triffleman et al. 1999; Hien et al. 2004; Najavits et al. 2005; Cusack et al. 2008), and no previous study has focused specifically on women with co-occurring disorders to examine treatments addressing the impact of physical and sexual abuse on the lives of these women. The women, co-occurring disorders, and violence study (WCDVS) was the first study designed to develop and evaluate the trauma-informed, integrated, comprehensive, and consumer-involved services for women with co-occurring disorders with a history of physical and/or sexual violence.

The WCDVS intervention required both system or organizational level and clinical or service level integration (McHugo et al. 2005). System level integration, which generally involves creating a network of organizations that provides a coordinated continuum of services (Goldman et al. 2000), does not always lead to service level integration, which refers to simultaneous and coordinated provision of care for each care episode (Bachrach 1993). Furthermore, several large scale studies consistently showed lack of association between system level integration and improved clinical outcome (Rosenheck et al. 2002; Bickman 1996; Bickman et al. 1999; Ridgely and Jerrell 1996) while service level integration is generally related to improved clinical outcomes (Drake et al. 1998; Druss et al. 2001; Mueser et al. 1998; Katon et al. 1999).

The WCDVS intervention agencies were required to provide services meeting four criteria: first, integration at both the organizational level and clinical level; second, trauma-informed services that address the impact of trauma on women’s lives and facilitate trauma recovery (Harris and Fallot 2001); third, consumer-involvement by including consumers as trauma group facilitators or consultants who helped design and deliver interventions; and fourth, comprehensive services by including eight core service elements: outreach and engagement, screening and assessment, ongoing treatment activities, parenting skills training, resource coordination and advocacy, trauma-specific services, crisis intervention, and peer-run services.

To meet these four intervention criteria, intervention sites adapted one or two of the following four group-based integrated trauma-specific service models: Seeking Safety model (Najavits 2001), Trauma Recovery and Empowerment Model (Harris 1998; Fallot and Harris 2002), Addiction and Trauma Recovery Integration Model (Miller and Guidry 2001), and a hybrid model called Triad (Clark and Fearday 2003). Specific details of how each model was implemented (e.g., number of sessions, whether outpatient-based or residential) varied across intervention sites. All of these programs focused on maintaining personal safety, teaching empowerment and coping skills, and helping women understand the link between substance abuse, mental health and trauma. Services provided by comparison agencies also varied from site to site and sometimes included some of the same program elements offered by the intervention but did not provide a trauma-specific treatment.

Previous studies indicate that the WCDVS intervention improved clinical outcomes measured by symptoms of drug use, alcohol use, mental health and trauma in both individual level (Morrissey et al. 2005a, b) and program level (Cocozza et al. 2005). However, pathways through the intervention may have improved clinical outcomes, not only of trauma-related but also of mental health and substance abuse problems, have not been fully understood. Studies investigating changes in patterns of service use during the first 6 months of follow-up found no significant change in either levels of services use for any service type (Domino et al. 2006) or total service costs (Domino et al. 2005a) by the intervention. The results were consistent in the 12 month evaluation where no intervention effect on total service costs, either for the overall services or for services external to the study, was found (Domino et al. 2005b).

Taken together, previous studies suggest that the improved clinical outcome in the WCDVS intervention may have been associated with changes in the service content of each episode of care, rather than changes in overall intensity or cost of services. The present study assessed whether the intervention had changed the content of services received during each episode, reported by service recipients during the 12 months following enrollment in WCDVS. We examined whether intervention had changed reported use of services addressing mental health and substance abuse issues as well as trauma-informed and integrated services, for a range of service types described below. Integrated services here were defined as services simultaneously addressing trauma, mental health, and substance abuse during a service episode.

Investigating the content of services in each clinical dimension and for a variety of service types is important for many reasons. First, pathways of the improved outcomes in mental health and substance abuse problem with the intervention could be better understood. While the intervention arm was required to provide trauma-specific elements in an integrated way, by design, the comparison arm continued to provide care as-usual which typically had extensive treatment elements addressing both mental health and substance abuse problems and sometimes addressing trauma. By looking into the content of services, we confirm that the improved clinical outcomes could have been ascribed to the changes in the way services were delivered in each episode level, given that a similar level of treatment intensity was provided by both the intervention and comparison arms. Second, it helps to establish fidelity of the integrated model as perceived by the study participants in both study arms. The type of complex intervention used in the WCDVS is fundamentally different from a simple randomized control drug trial in that woman in the intervention arm may not have received integrated counseling, for e.g., and women in the comparison arms may have received informal integrated services. Therefore, examining the rate of receipt of integrated counseling services in the intervention arm allows us to measure both the dose of the intervention and the degree of contamination. Third, while the intervention arm was required to provide integrated services, the range of those services was not well-defined. Services such as general medical visits or emergency room visits may have been outside the scope of the intervention; therefore exploring the extent to which study participants differentially perceived the intervention across a range of service types reported is an empirical question.

Methods

Data and Study Sample

The study participants were women aged 18 or older with histories of interpersonal violence who have experienced mental health and substance use disorders within the last 5 years, at least one of which occurred within the last 30 days. A total of 2,729 women enrolled in the nine sites nationwide in 2001 and 2002. Data were collected at baseline and 3, 6, 9 and 12 months follow-up interviews. All the questions were read and answers were recorded by interviewers during in-person interviews (McHugo et al. 2005).

Data collection procedures and subject protection plans were approved by institutional review boards at each study site and at two coordinating centers. All participants provided written informed consent to participate in the research. There is no known conflict of interest of investigators throughout the conduct of the WCDVS, data analysis, and dissemination of findings. All the co-authors participated in the analysis and interpretation of the data presented in this paper and they approved the final manuscript.

The present study includes 2,087 women (76% of those who completed the baseline interview) who completed the 12 months follow-up interview within 12 weeks before or after the scheduled date, regardless of the quantity of intervention services received. Among those who completed the 12 months interview, 1,558 women (74.7%) completed all the four (i.e., 3, 6, 9, and 12 months) follow up interviews. Women who dropped out of the study were more likely to be younger, non-white, and uninsured and were more likely to have less than high school education than those who completed the 12 months interview. However, there was no systematic attrition between the intervention and comparison arms in terms of demographic, service use and clinical characteristics (Morrissey et al. 2005a, b).

Study Design

Each of the nine sites had an intervention arm and a comparison arm, both of which were located in the same service area. The WCDVS used a quasi-experimental design with each study site responsible for selecting its intervention and comparable comparison programs. Two study arms (intervention versus comparison) for each site were similar in terms of the nature of services delivered, traditional treatment focus (e.g., mental health or substance abuse) and baseline demographic and clinical conditions (McHugo et al. 2005; Morrissey et al. 2005a, b).

The nine sites varied considerably in their setting such as whether treatment was delivered on an outpatient or residential basis and whether the setting was primarily a substance abuse or mental health agency. Regardless of the variation, all the intervention agencies were required to provide services meeting four criteria: integration, trauma-informed, consumer-involvement, and comprehensiveness, by adopting a trauma-specific service models, as described above. Services provided by comparison agencies also varied from site to site and sometimes included some of the same program elements offered by the intervention but did not provide a formal trauma-specific treatment.

Service Measures

In each interview, participants were asked to answer questions about any occurrence, quantity and treatment content of service use for a variety of service types during the last 3 months or since the previous interview. Thus, women who missed the previous follow-up interview reported service use during the entire period since the previous interview. Service types analyzed in this study are hospital, emergency room, general medical visit, residential treatment, outpatient case management, outpatient group counseling, outpatient individual counseling, and peer support group. We classified these service types into two categories: service types that were largely provided by the participating agencies including residential treatment, outpatient case management, outpatient group counseling, and outpatient individual counseling; and service types that were mostly external to the core interventions including hospital, emergency room and general medical visit.

For each service type, if the respondent reported any service use then she was asked the treatment content during each service episode (visit or admission) and could choose one or more of relevant categories among eight categories: medical services, violence/trauma, psychiatric/emotions services, psychiatric medication check, alcohol/other drug treatment, parenting support, legal assistance, housing assistance, and education/vocation assistance. Our study focused on trauma (violence/abuse/trauma treatment), mental health (services for emotions, nerves, or psychological problem or psychiatric medication check) and substance abuse (alcohol/other drug abuse treatment). Based on the answers to treatment content, we defined integrated service as services addressing trauma, mental health and substance abuse simultaneously in each service episode (i.e., outpatient counseling or case management session, emergency room visit, or admission to hospital or residential facility).

Analytical Methods

We first compared quantity and treatment content of services in the intervention and usual care arms during the intervention period. Then, we examined whether use of services addressing each of the three content areas and use of integrated services were changed by the intervention.

Linear regressions were used to estimate the intervention effect. For each service type and treatment content, the outcome variable was a dichotomous variable indicating whether an individual received any treatment addressing each content area during the 12 months, and the main effect was the treatment condition (intervention versus comparison). Level of service use for the corresponding service type and content area during pre-baseline 3 months was used to adjust for the baseline difference in service use. Participant age, race/ethnicity, marital status and dummy variables indicating study sites were also included as covariates to adjust for heterogeneity in two study arms. All the variables were defined for each individual, service type and content level. Huber-White robust standard errors were used to adjust for site-level clustering and heteroskedasticity. We obtained virtually the same results with logit regressions and thus reported the results from the linear regressions for ease of interpretation.

Results

Baseline Demographic Characteristics

The study sample (n = 2,087) consisted of women aged 19–59 with a mean age of 37 and diverse racial groups, education levels, marital status, and insurance status (Table 1). The majority of women were whites (56.2%), having high school or less education (71.5%), currently not married or partnered (61.1%), and having Medicaid (58%). Participants in the two study arms had similar demographic characteristics, except that those in the intervention arm were more likely to be African–American (P < 0.05) and never married (P < 0.05).

Table 1 Baseline sample characteristics by intervention/comparison group
Full size table

Service Use Pattern During 12 Months Study Period

The probability of using any service for each service type during the 12 months period ranged from 32.3% for hospital to 88.6% for general medical visits. Among the services within the intervention focus of the WCDVS, 79.6% of women used peer support groups and 51.9% of them used residential treatment (Table 2).

Table 2 Service use rates during the 12 months: by intervention/comparison group
Full size table

For the service types which the WCDVS focused on, there were some differences in service use between two study arms during the study period although the direction was not always consistent. Women in the intervention arm were less likely to use residential treatment but, among those who used any, they stayed more days at the facility. For outpatient case management, women in the intervention arm were more likely to receive services and received services more often. For outpatient group counseling, women in the intervention arm were more likely to use the service, but the level of use among users did not differ by intervention/comparison status. No difference was observed in the use of service types that were external to the interventions.

Treatment Content of Services Received

For the all the service types within the scope of the study, participants in the intervention arm were more likely to report receiving services addressing trauma or integrated services during the study period (Table 3). Likewise, women in the intervention arm were more likely to report receiving services addressing mental health through residential treatment, outpatient case management and peer support group service, and services addressing substance abuse through outpatient case management. Contrarily, women in the intervention arm were less likely to report receiving services addressing substance abuse through outpatient group counseling. Generally, the magnitudes of the differences were larger for services addressing trauma or integrated services than services addressing mental health or substance abuse.

Table 3 Any use of services addressing each treatment content during the 12 months: by intervention/comparison group [frequency %]
Full size table

On the other hand, for services external to the intervention, the likelihood of receiving treatment addressing any of these content areas was very low and did not differ between the two study arms. In analyses not presented in the table, we also examined the frequencies during the pre-baseline period and found no difference between the intervention and comparison arms.

Intervention Effect on the Content of Services

After adjusting for the pre-baseline level of service use and other relevant covariates, we found a significant intervention effect on the likelihood of addressing trauma or providing integrated services in service types within the scope of the study (Table 4). For these service types, the magnitude of increase in the likelihood of addressing trauma or providing integrated services ranged from 6 to 22 percentage points. For outpatient case management, the intervention increased the likelihood of addressing mental health or substance abuse problems as well. Similarly, for peer support group sessions, the intervention increased the likelihood of addressing substance abuse issues.

Table 4 Intervention effect on the content of services: linear regression results [coefficient (robust standard error)]
Full size table

For service types external to the study, there was no intervention effect. Finally, in all the regression models, pre-baseline levels of service use addressing corresponding content area and site dummy variables, not presented in the table, were also significant.

Discussion

This study investigated whether the trauma-informed integrated approach in treating women and co-occurring disorders with a history of violence was implemented as intended, by enhancing service integration and changing service content. By examining service recipients’ self-report data on service content, we found strong evidence that the intervention increased the provision of trauma-informed and integrated services as well as services addressing mental health or substance use problems in several service types within the scope of the WCDVS. However, there was no consistent increase in the quantity of services in all these service types where intervention effects were observed.

In contrast to the lack of intervention effect on changes in service use pattern in terms of quantity of service use in the WCDVS (Domino et al. 2005a, b, 2006), the present study showed significant changes in content of services by the intervention by examining the unique aspect of the intervention, the provision of trauma-specific integrated services. Studies evaluating clinical outcomes of the intervention have consistently demonstrated improved symptoms of mental health and substance abuse as well as trauma (Morrissey et al. 2005a, b; Cocozza et al. 2005). Taken together, incorporation of trauma-specific element may have been successfully implemented at the services level, which may have led to the improved clinical outcomes in both mental health and substance abuse problems.

We identified service content based solely on service recipients’ self-reporting. An alternative useful source of service content might be provider records, which may be more accurate, but are more expensive to collect and were not available for this study. Although imperfect, clients’ self-reported data have many advantages over provider records in terms of practicality and comprehensiveness. More importantly, content of services is subjective and may be different from those by providers (Bogwald 2001), and self-reports are the only source of information capturing service content measured by service recipients. Evidence suggests that self-reported service use by clients of behavioral services may be valid (Rozario et al. 2004; Booth et al. 2006). The service content measures used in the present study have also been shown to be reliable with moderate to substantial levels of reliability (Chung et al. 2008). Indirect evidence supporting the validity of self reported service content in the present study is that the intervention effect was observed only for service settings within the scope of the WCDVS intervention not for service types that were outside the scope of the intervention.

Similarly, there are many ways to define and evaluate service integration. Our study evaluated clinical integration at the service-level using information on treatment content during each service episode reported by service recipients. An alternative measure might be subjective perception of the levels of integration in services overall. Different measures may capture different dimensions of service integration and could result in different conclusions. To capture the wide range of integration and concordance or discordance in results based on different sources, further validation studies using other data sources are recommended.

There are several limitations to be noted. First, potential recall errors and misclassification is inherent in self-reporting. However, the potential errors could not have led to biased estimates unless the errors were systematic across intervention conditions, which is unlikely. Second, we did not evaluate individual trauma-specific program or service elements. Therefore, the overall findings are attributed to the package of intervention services, not to individual trauma-specific services. Only two (Seeking Safety and TREM) of the four models had enough power for separate analyses, so, we reran these on the relevant sites. The results (see Appendix, Table 5) did not show a consistent difference across service types between two models. Third, self-reported service use includes all the services received in all settings, from all providers, not limited to the services received from the participating sites. Therefore, this study finding can be interpreted as changes in content of services associated with the intervention rather than services provided by the intervention itself. Given the extensive scope and intensity of services provided by both intervention and comparison programs, the amount of services recipients received through other providers for the service types within the scope of the intervention might be negligible. Fourth, women who did not complete the 12 month assessment were likely to have had more severe drug use and trauma symptoms at baseline than women who were included in this study. However, our findings of an intervention effect based on regression estimates may not be biased because there was no difference between intervention and comparison groups in the characteristics of people who did and did not complete the 12 months assessment. Finally, women in our study were enrolled in the study through the participating agencies and may represent most complicated and difficult to treat populations. Therefore, any differences observed in this study could be interpreted as a conservative estimate.

To confirm the findings of our study and address these limitations, we suggest further focused research. First, future studies should validate self-reporting with agency or provider record abstracts. Second, the link between improved awareness of service content and clinical outcomes also warrants further investigation. How perceptions of service content may translate to clinical outcomes is largely unknown (Lloyd-Evans et al. 2007). Third, the performance of individual trauma-specific programs and service elements should be evaluated to understand the most effective and efficient approaches for particular client groups. Finally, evidence of concordance of service integration defined in different ways and measuring different dimensions, would inform future evaluations of service integration.

Implication for Community Mental Health

The successful implementation of integrated service as demonstrated by the WCDVS serves as an example of how community agencies providing a wide range of services to clients with complex conditions could work together for the improvement of consumer outcomes. However, these services require a considerable amount of start-up cost and expertise (Dalton et al. 2003) that may not be available to many community agencies. Funding of these types of integrated services remains a formidable challenge to the wider dissemination of these practices.