Introduction

The concept of recovery from mental illness is becoming an organizing principle in the design of mental health services (Davidson and White 2007). Recovery from mental illness, as defined by the National Consensus Statement on Mental Health Recovery is, “a journey of healing and transformation enabling a person with a mental health problem to live a meaningful life in a community of his or her choice while striving to achieve his or her full potential” (SAMHSA 2004).

Such recovery from mental illness is the focus of The Village Integrated Service Agency in Long Beach California. The Village began its work in 1990, having been awarded a grant as part of California Assembly Bill 3777 to promote integrated service delivery systems. The arguments used to gain passage of California’s 2004 Mental Health Services Act, which levies a 1% income tax on adjusted gross incomes over $1 million to be used for transformational activities in the state mental health system, were, to a significant extent, based on the success of The Village’s pioneering efforts (Bambauer 2005; Christie-Smith and Gartner 2006; Mulligan 2005; Scheffler and Adams 2005).

In a qualitative study of The Village’s practice, Erickson and Straceski (2004) attempt to identify what has been described as The Village’s “magic bullet” of success. The principal answer that Erickson and Straceski posit is The Village’s culture—its “articulation of a philosophy of care” that “places fundamental importance on” the practice of The Village and with that practice “a respect for and appreciation of whole persons.”

The Village, however, does not treat and care for its members with “culture” alone. To become a member of The Village a person must be 18 or older, live in the Long Beach area, have a DSM-IV(Diagnostic and Statistical Manual of Mental Disorders, 4th edition) Axis 1 diagnosis of schizophrenia, bi-polar disorder, or major depression, and be homeless, incarcerated, or at risk of being so (Village 2007). As described in detail below, the program provides affordable housing, job coaches, employment opportunities, case-management, collateral contact, and rehabilitation and treatment services in the context of an assertive community treatment services model (Salyers and Tsemberis 2007). These services have been found to be efficacious in producing specific outcomes (Lewin Group 2000), but to our knowledge no evidence exists about the effect of these services on the trajectory of recovery.

The Village describes itself as adhering to the following principles (1) choice; (2) quality of life; (3) community focus; and (4) “whatever it takes.” Choice is operationalized by members choosing services based on their personal goals. Traditional “professional to patient” relationships are de-emphasized. Staff and members are considered equal partners in the recovery process. Quality of life is operationalized by a focus on goals that address each part of members’ lives (work, education, financial, and social). Community focus is operationalized by a focus on integrating members into the community rather than being segregated. To accomplish this, Village staff members spend most of their time out of the office in order to support members as they pursue their personal goals. “Whatever it takes” refers to services being available for as long as needed and staff being available 24 h/day in the event of a crisis (Village 2009). Village funds are not focused on providing hospitalization, partial hospitalization, structured day treatment, or psychotherapy, but rather are focused on maintaining smaller case management loads, increasing the availability of psychiatrists, and providing social and employment opportunities for members (Ragins 2006).

The Village applies these principles using collaborative case management teams and psychosocial rehabilitation. Each collaborative case management team is comprised of a director, specialists (psychiatrist, money manager, employment counselor), and a group of personal service coordinators (PSC). Each PSC is the lead responsible for 15–20 members (Ragins 2006).

Teams are integrated into a comprehensive recovery-oriented program that emphasizes collaborative interactions, encouraging members to take risks by attempting new things in working towards their recovery goals, and not being afraid to fail. In-house programs aid members in obtaining housing, social skills, financial management skills, job skills, and employment (Ragins 2006).

The housing department matches members with housing options [including options available via the U.S. Department of Housing and Urban Development (HUD)]. It also offers an on-call response line to landlords to minimize crises and evictions (Ragins 2006).

The social skills of members are cultivated through the efforts of community integration/outreach specialists where the emphasis is on learning by doing. Social activities include everything from using the gym at the YMCA to dancing at a night club. Members with more advanced social skills are employed as “life coaches” to support members in crisis and to encourage isolated members to participate in social activities (Ragins 2006).

The financial skills of members are built by The Village (in coordination with The Village bank) through a system that gradually increases members’ financial responsibility: helping members to manage their social security benefits so that members may provide for themselves (food, clothing, housing), teaching members to create budgets and manage their own funds, and aiding members in opening accounts in community banks (apart from The Village bank) (Ragins 2006).

Opportunities to learn job skills include a café/deli, maintenance unit, and clerical unit maintained by The Village and staffed by members. Each job requires regular work hours, pays minimum wage or higher, and lasts 9 months. There are no prerequisites to obtaining any job other than applying (Ragins 2006).

Apart from employment at The Village, opportunities for employment in competitive jobs in the community are facilitated by job developers who work with The Village work sites (café/deli, maintenance unit, clerical unit), The Village supported education Career Center, and support services at Long Beach City College. This service is open to all members (Ragins 2006).

A key question relates to the recovery trajectory of Village members. What does the pathway of recovery look like? The trajectory of recovery can be operationalized from the viewpoint of the mental health consumer or the mental health service provider. Recovery from mental illness from the viewpoint of the mental health consumer will vary significantly from person to person to the extent that what is meaningful, what constitutes full potential, and what type of community a person prefers to live in varies from person to person. A number of instruments are in development that attempt to operationalize recovery from the consumer’s point of view including the Recovery Assessment Scale and the Stages of Recovery Instrument (McNaught et al. 2007; Andresen et al. 2006; Corrigan et al. 2004; Ralph et al. 2000). On the other hand, the trajectory of recovery from mental illness as operationalized from the viewpoint of the mental service provider will be different, but complementary to the viewpoint of the mental health consumer. This approach focuses on a given consumer’s level of function (Fisher et al. 2009). Such measures are fairly new. A recently validated instrument in this category is the Milestones of Recovery Scale (Fisher et al. 2009). This study will incorporate this measure to describe the trajectory of recovery by members of The Village.

Methods

Data

As a pilot project to determine if the provision of additional data (sociodemographic characteristics, psychiatric diagnoses, and services received) and accompanying research would be warranted, The Village Integrated Service Agency provided us only with data on the Milestones of Recovery Scale for each member served by The Village during the period of November 2005 to January 2006 (658 observations). Note that due to the use of the simulation model described below, it is not important that our sample of members from the Village be individuals who only initiated membership at the beginning of our data collection period. Our models use an established method to simulate the long-run recovery trajectory of Village members based on an analysis of the recovery trajectory of a cohort of Village members sampled over a shorter period of time.

Measure

The MORS is based on a framework developed by the California Association of Social Rehabilitation Agencies in 1997 that suggested that mental health consumers could be grouped according to three criteria: (1) risk to themselves and others, (2) skills and supports, and (3) engagement with the mental health system (Fisher et al. 2009). Pilon et al. (2006) used these three criteria to create the MORS. Risk to self and others is measured by the likelihood that a consumer poses harm to themselves and/or others, whether a consumer is involved in risky or unsafe behaviors, and the consumer’s level of chemical dependency. Coping skills and supports include the ability of a consumer to care for themselves, the extent of support a consumer receives from family and friends, and the extent to which a consumer needs mental health staff involvement in order to get their basic needs met. Finally, engagement with the mental system is simply a judgment regarding whether a mental health consumer is appropriately interacting with the mental health system.

The MORS ranges from 1 to 8 with 8 being the highest level of recovery (MORS 1 through MORS 8). Each member’s recovery score is determined monthly by a Village mental health professional (Pilon and Ragins 2007). In order make the results of our analysis more understandable to readers, we consulted with Pilon and Ragins and constructed an aggregate MORS (AMORS) scale consisting of six levels that combined MORS levels 2 and 4 and MORS levels 3 and 5 based on the commonality of being “engaged” or “unengaged” in mental health treatment. See Table 1 for a description of the MORS levels and the AMORS levels.

Table 1 Milestones of recovery scale—original and aggregated
Full size table

The MORS has been shown to be valid and reliable (Fisher et al. 2009). The inter-rater reliability of the MORS is 0.85 [confidence interval (CI) 0.81, 0.89], test-retest reliability is 0.85 (CI, 0.81, 0.87); and validity coefficients for the Level of Care Utilization System (LOCUS) are at or above r = .49 for all subscales except one (Level of Support) (Fisher et al. 2009; Pilon and Ragins 2007).

Analysis

To analyze the AMORS data, we employ the mathematical theory of Markov Chains (Ching 2006). Mathematical models (such as Markov chain models) of the outcomes of organizational activity are useful as they provide simplified pictures of the outcomes of complex human actions. These models are valuable in predicting outcomes at the system level (Pandiani et al. 1994). It is necessary that the measured states of the world be exhaustive and mutually exclusive for Markov Chain models to be successfully applied. With regard to mental health applications, Markov Chain models have been used as far back as the 1970s with an early application successfully predicting inpatient utilization (Sweillam and Tardiff 1978) and more recent applications successfully describing the pattern of movement across mental health states in cohorts of individuals diagnosed with depression (Patten 2005; Patten and Lee 2004, 2005), the pattern of movement across mental health states in cohorts of individuals with schizophrenia (James et al. 2006), the physical movement of individuals with schizophrenia between various types of mental health services (Moreno et al. 2007), and the physical movement of persons with severe mental illness between the public mental health system and jail (Norton et al. 2006).

To apply Markov Chain models in the current case, we must first determine the probabilities of moving from any AMORS level to any other AMORS level. These are called transition probabilities and describe both probabilities of progress toward recovery and probabilities of regression from recovery. We assume that the transition probabilities in the starting AMORS level sum to 1, that the transition probabilities apply to all members at The Village, that the transition probabilities are constant over time, and that MORS levels are independent over time.

In other words, this is an analysis of a sequence of monthly member assessments using the AMORS where the probability of a member’s AMORS level in any particular month depends only on the level of the member’s AMORS level in the previous month. In a system with such temporal dependency, the probabilities relating to the future depend only on the present stage. They do not depend on the way the present stage emerged from the past. These are standard Markov Chain assumptions (Ching 2006). While not completely realistic (all models necessarily simplify reality by definition), they are reasonable assumptions. Since all MORS and AMORS levels are mutually exclusive, transition probabilities necessarily sum to one. Since we are modeling a system, the transition probabilities would apply to the “average” cohort of Village members. Transition probabilities between AMORS levels would be expected to be close to constant over time provided the Village program and member case mix do not change. Finally, since we are modeling the “average” member of The Village, the assumption that the probabilities relating to the future depend only on the present stage is also reasonable.

An example of how Markov models work is presented in Fig. 1. This example assumes that there are only three recovery levels, does not use actual data, and is for illustrative purposes only. We use the term “STAGE” rather than “AMORS” to avoid confusion. The figure starts at period 0 where a mental health consumer is in STAGE 1 and shows all possible pathways to move from STAGE 1 to any of the other STAGE states. The probability of moving from STAGE 1 to STAGE 1 is 0.5, the probability of moving from STAGE 1 to STATE 2 is 0.3, and the probability of moving from STAGE 1 to STAGE 3 is 0.2. These probabilities are called transition probabilities and are constant. Each transition probability is determined by calculating a quotient where the denominator is the total number of individuals who start in STAGE 1 and the numerator is the total number of people who have moved from STAGE 1 to any particular one of the three STAGE states. The three quotients will necessarily add to one. The probabilities at the end of the chain (in this case, at period 2) are the product of the probabilities across any given pathway. A similar figure could be developed where the starting point was STAGE 2 or STAGE 3.

Fig. 1
figure 1

Example of Markov process. Note: this is not actual data and is for illustrative purposes only

Full size image

In our example the probability of being in STAGE 1 in period 2 is 0.38 (0.25 + 0.09 + 0.04), the probability of being in STAGE 2 in period 2 is 0.35 (0.15 + 0.12 + 0.08), and the probability of being in STAGE 3 in period 2 is 0.27 (0.10 + 0.09 + 0.08). Note that these three probabilities sum to one (1 = 0.39 + 0.35 + 0.27). We apply this approach using Mathematica 6.0 and consecutive months of origin-destination AMORS data. Confidence intervals were calculated using the method of bootstrapping (Efron and Tibshirani 1993). Two thousand random samples of the integers between one and 658 were randomly drawn with replacement from uniform distributions and 95% confidence intervals constructed.

This study has been reviewed by both the Office for the Protection of Human Subjects at the University of California at Berkeley. It has also been reviewed by the Los Angeles County of Mental Health Human Subjects Research Committee.

Results

Transitional Probabilities

Table 2 provides the basis for our simulation analysis. It shows the actual one-month transitional probabilities and frequencies of various levels of the AMORS. The diagonal in Table 2 shows that the probabilities that members remain in an AMORS level in any given month, in other words, these are the probabilities of there being no movement, of the origin and destination states being the same. These probabilities range from 0.459 to 1.000. Also note that members who are currently in any given AMORS level may move forward or backward (and not all movements need be sequential). While a member who is in AMORS 1 has no place to go but forward, once such a member does move forward, there is no guarantee that only forward movements will subsequently occur. For example, for members who reach AMORS 3, the transitional probability of moving backward to AMORS 2 is larger (0.126) than moving forward to AMORS 4 (0.117). Similarly for members who reach AMORS 4, the transitional probability of moving forward to AMORS 5 (0.115) is less than the transitional probability of moving to AMORS 3 (0.187).

Table 2 One-month AMORS transition probabilities and frequency counts for members of The Village
Full size table

In sum, movement along the recovery continuum is not always forward, and as we will see below, is often not linear. However, in the aggregate, we will see that the average movements that do occur tell a clear story. We use the transitional probabilities in Table 2 to simulate movements between AMORS levels from 2 to 100 months. We present the results of 2–24 months graphically for each of the 6 AMORS levels.

Figure 2 presents the relationship between months of care at The Village and the probability of being in the AMORS levels of “extreme risk” (red), “unengaged” (purple), “poorly self-coordinating” (orange), “coping/rehabilitating” (yellow), “early recovery” (blue), and “self-reliant” (green) given that a member started in the category of “extreme risk” (red). Each level is shown separately with estimates shown as solid lines and 95% confidence intervals shown as dotted lines. For example, in Fig. 2, the red line represents the relationship between months of care and the probability of being evaluated as “extreme risk” after the number of months of care listed on the horizontal axis. The sum of estimated probabilities from any given month across the six AMORS levels will be equal to one. Recall that in Fig. 2 every Village member represented by the graphs started at the “extreme risk” level. This explains the high initial probabilities of being at the “extreme risk” level. Note that the probability of staying at this level diminishes quickly. By approximately 10 months the probability of being at “extreme risk” has fallen to ~0.10. In other words, in less than one year, ~90% of Village members who were at “extreme risk” are no longer so. After the 0.06 probability range, at ~20 months of care, the reductions in probability continue, but slowly.

Fig. 2
figure 2

Probability that members who begin as “extreme risk” will continue as “extreme risk”, or become “unengaged”, “poorly self-coordinating”, “coping/rehabilitating”, “early recovery”, or “self reliant” over a two year period. 95% Confidence Intervals displayed as dotted lines

Full size image

Figure 2 also shows evidence of recovery where movement occurs from the “extreme risk” level to the subsequent levels of “unengaged” (purple), “poorly self-coordinating” (orange), “coping/rehabilitating” (yellow), “early recovery” (blue) and “self-reliant” (green). After a year of care there is an approximate probability of 0.80 that initial “extreme risk” members have progressed to one of the following stages: “poorly self-coordinating” (orange), “coping/rehabilitating” (yellow), or “early recovery” (blue), or “self-reliant” stages (green). After two years of care there is a probability of almost half that the initial “extreme risk” members move to the recovery-oriented “coping/rehabilitating” (yellow), “early recovery” (blue), or “self-reliant” (green) levels.

Figure 3 presents the relationship between months of care at The Village and the probability of being in the AMORS levels of “extreme risk” (red), “unengaged” (purple), “poorly self-coordinating” (orange), “coping/rehabilitating” (yellow), “early recovery” (blue), and “self-reliant” (green) given an initial assessment of “unengaged” (purple). Note that the probability of staying “unengaged” (purple) diminishes quickly. There is a probability of ~0.80 that members will leave this stage in the first 6 months of care. There is a small amount of retrogression with their being an ~0.07 probability of entering the “extreme risk” (red) stage in the first 6 months of care and a probability of ~0.05 of being at “extreme risk” (red) after 2 years. However, the vast majority of Village members who begin in this stage improve. In the first 6 months of care, there is a probability of approximately one-half that members will progress to “poorly self-coordinating” (orange) and then to other levels (note the spike occurring during the first 6 month period in “poorly self-coordinating” (orange) which then declines thereafter. By the end of 2 years, there is a probability of ~0.40 that initially “unengaged” members are in the recovery-oriented coping/rehabilitating” (yellow), “early recovery” (blue), or “self-reliant” (green) levels.

Fig. 3
figure 3

Probability that members who begin as “unengaged” will continue as “unengaged”, or become “extreme risk”, “poorly self-coordinating”, “coping/rehabilitating”, “early recovery”, or “self reliant” over a two year period. 95% Confidence Intervals displayed as dotted lines

Full size image

Figure 4 presents the relationship between months of care at The Village and the probability of being in the AMORS levels of “extreme risk” (red), “unengaged” (purple), “poorly self-coordinating” (orange), “coping/rehabilitating” (yellow), “early recovery” (blue), and “self-reliant” (green) given that the initial assessment was “poorly self-coordinating” (orange). Note that the probability of staying “poorly self-coordinating” (orange) diminishes quickly. There is a probability of ~0.50 that members will leave this stage in the first 6 months of care. There is a moderate amount of retrogression with there being an ~0.05 probability of entering the “extreme risk” (red) stage in the first 6 months of care and an ~0.04 probability of being in this stage after 2 years. There is also retrogression to the “unengaged” (purple) stage. This occurs with a probability of ~0.16 at 6 months and a probability of ~0.13 at 2 years. In short, there is a probability of about one-fifth that members will have retrogressed after 2 years given an initial assessment of “poorly self-coordinating” (orange).

Fig. 4
figure 4

Probability that members who begin as “poorly self-coordinating” will continue as “poorly self-coordinating”, or become “extreme risk”, “unengaged”, “coping/rehabilitating”, “early recovery”, or “self reliant” over a two year period. 95% Confidence Intervals displayed as dotted lines

Full size image

However, there is a probability of about one-third that members beginning as “poorly self-coordinating” (orange) will move to the recovery-oriented “coping/rehabilitating” (yellow), “early recovery” (blue) and “self-reliant” stages within six months. At the end of 2 years, there is a probability of ~0.40 that members will be in one of these three stages.

Figure 5 presents the relationship between months of care at The Village and the probability of being in the AMORS levels of “extreme risk” (red), “unengaged” (purple), “poorly self-coordinating” (orange), “coping/rehabilitating” (yellow), “early recovery” (blue), and “self-reliant” (green) given that the initial assessment was “coping/rehabilitating” (yellow). In the initial 6 months, there is a probability of approximately one-half that members in this stage will retrogress with the probability of moving to “extreme risk” (red) being ~0.03, the probability of moving to “unengaged” (purple) being ~0.11 and the probability of moving to “poorly self-coordinating” (orange) being ~0.36. However, in the initial 6 months there is a probability of just under one-fifth that members will move ahead to “early recovery” (blue) or to becoming “self-reliant” (green). At the end of 2 years, there is a probability of ~0.55 that members have retrogressed, a probability of ~0.25 that members continue to be “coping/rehabilitating” (yellow), and a probability of ~0.20 that members move ahead to either “early recovery” (blue) or “self-reliant” (green).

Fig. 5
figure 5

Probability that members who begin as “coping/rehabilitating” will continue as “coping/rehabilitating”, or become “extreme risk”, “unengaged”, “poorly self-coordinating”, “early recovery”, or “self reliant” over a two year period. 95% Confidence Intervals displayed as dotted lines

Full size image

Figure 6 presents the relationship between months of care at The Village and the probability of being in the AMORS levels of “extreme risk” (red), “unengaged” (purple), “poorly self-coordinating” (orange), “coping/rehabilitating” (yellow), “early recovery” (blue), and “self-reliant” (green) given that the initial assessment was “early recovery” (blue). In the initial 6 months, there is a probability of ~0.70 of members retrogressing, with there being a probability of ~0.02 that members retrogress to “extreme risk” (red), a probability of ~0.08 that members will retrogress to “unengaged” (purple), a probability of ~0.30 that members will retrogress to “poorly self-coordinating” (orange), and a probability of ~0.30 that members will retrogress to “coping/rehabilitating” (yellow). The probability of remaining in “early recovery” (blue) at 6 months is ~0.25. However, there is only a probability of ~0.05 that members will be “self-reliant” (green) at 6 months. At the end of 2 years, the probability of being “self-reliant” is ~0.10, the probability of being in “early recovery” is ~0.15, and the probability of retrogressing is ~0.75.

Fig. 6
figure 6

Probability that members who begin as “early recovery” will continue as “early recovery”, or become “extreme risk”, “unengaged”, “poorly self-coordinating”, “coping/rehabilitating”, or “self reliant” over a two year period. 95% Confidence Intervals displayed as dotted lines

Full size image

Finally, Fig. 7 shows the movement of members who have entered the “self-reliant” stage. In this stage, there is no evidence of retrogression.

Fig. 7
figure 7

Probability that members who begin as “self reliant” will continue as “self reliant” over a two year period. 95% Confidence Intervals displayed as dotted lines

Full size image

Discussion

We found that using a Markov Chain model, an established approach in analyzing mental health outcomes at the system level, we are able to describe the trajectory of recovery, including retrogressions, of individuals participating in a model integrated service delivery system, The Village, a program of Mental Health America of Los Angeles. Using a validated measure of recovery that assesses recovery from the perspective of the mental health service provider, the Milestones of Recovery Scale, we have for the first time illustrated the trajectory of recovery given months of care and current recovery level.

In short, as members recover, the majority are able to maintain their level of recovery up to the level of “poorly self-coordinating”. However, the next level, “coping/rehabilitating” appears to be a threshold that is difficult for many to maintain as there is only a probability of 0.45 that individuals who reach this level will be able to maintain it or progress to higher levels of recovery over a two-year period. Even fewer who enter “early recovery” will maintain or move past this level of recovery over a two-year period with the probability being ~0.25. However, it appears that virtually everyone who reaches the stage of being “self reliant” remains there.

Limitations

There are several limitations to this study. First, since the data used in the analysis are only outcomes data, we are only able to determine the average recovery trajectory. Thus, while our analysis is a reasonable description of the outcomes of The Village when seen as a system, it cannot be used to determine how various subgroups within the system may behave. An extension to do this would require additional data on members’ sociodemographic characteristics, psychiatric diagnoses, and the particular sets of services received by each subgroup.

In addition, our analysis implicitly assumes that the culture, philosophy, and case mix of The Village do not change over time. To the extent that such changes do occur, our analysis will be less accurate.

Conclusion

The success of The Village in helping members to move forward in a recovery process suggests the potential outcomes that are possible from California’s Mental Health Service Act which is attempting to extend the success of The Village across the state in the form of Full Service Partnerships (FSP) (Mental Health Services Act 2004). These FSPs, which have similar features, but are not identical to the program administered by The Village, are designed to provide a full array of mental health services to the state’s most seriously ill individuals. The experience of The Village suggests that evaluations of FSPs that take into account less than 2 years of outcomes data may understate the success of FSPs. This is critical in any evaluation of FSPs.

Extensions of the current work should include information that is presently collected by the Village, such as sociodemographic characteristics, psychiatric diagnoses, services provided to members of The Village, type of employment and living arrangement, but will probably require changes to the Village’s information collection to include the purpose of service application. Such an analysis would allow policy makers to have a better idea about what types of services, in what situations, are most productive in promoting the recovery of various types of members. Such knowledge would ultimately facilitate the development of the public mental health system, and the delivery of the most effective care to the severely mentally ill citizens of California.