Longitudinal Outcomes for Youth Receiving Runaway/Homeless Shelter Services
- First Online:
- Cite this article as:
- Pollio, D.E., Thompson, S.J., Tobias, L. et al. J Youth Adolescence (2006) 35: 852. doi:10.1007/s10964-006-9098-6
- 442 Views
This research examined outcomes and use of specific types of services 6 weeks, 3 and 6 months post-discharge for a large sample of runaway/homeless youth using crisis shelter services. Data were collected for 371 runaway/homeless youth using emergency shelter and crisis services at eleven agencies across a four-state midwestern region. Outcomes were assessed for runaway behavior, family relationships, substance use, school behavior, employment, sexual behavior, and self-esteem. Additionally, the impact of services on outcomes was assessed. Findings indicated that although youth achieved a wide variety of positive outcomes 6 weeks post-discharge, there were signs of attenuation of certain outcomes by the 6 month follow-up. Service use after discharge did not demonstrate a strong impact on maintaining outcomes. The substantive findings reported here present encouraging evidence for providers of services for runaway/homeless youth. Crisis shelter services appear to facilitate broad-based short-term gains, but do not appear sufficient to maintain these gains over an extended period.
KeywordsHomeless/runaway youthYouth sheltersServicesLongitudinal methodsService use
Research on runaway and homeless youth has focused primarily on the causes, antecedents, and individual characteristics associated with running away. A substantial amount of research has documented the disproportionate risk for problems faced by these youth. High-risk problem behaviors include: substance use and abuse (Greene et al., 1997; Kipke et al., 1997), HIV/AIDS risk behaviors (Clatts et al., 1998; Ennett et al., 1999), depressive symptoms (Whitbeck et al., 2000), and suicide (Yoder, 1999). In addition, these adolescents often report problems in the family such as parental physical/sexual/emotional abuse, neglect, conflict, and substance use (Ringwalt et al., 1998; Thompson et al., 2001; Whitbeck et al., 1997a). While these studies provide a view of the challenges experienced by runaway/homeless youth, limited research has addressed service provision and outcomes of service use among this high-risk youth population. The purpose of the present study is to examine the long-term outcomes of runaway/homeless adolescents who utilize emergency shelter services.
Community-based youth emergency shelters have been designed as the primary service resource for runaway and homeless adolescents, offering an array of services including temporary housing, crisis intervention, outreach, individual and family counseling and case management (Greene et al., 1997). Youth crisis shelters, such as those examined in this study, serve adolescents 12–18 years of age, are limited in size to less than 20 beds, restrict stays to fewer than 15 days, and are partially funded by the U.S. Department of Health and Human Services (Thompson et al., 2001). The first priority is ensuring that youth are provided with the basic necessities, such as adequate shelter, food and clothing. Federally-funded shelters are mandated to attempt reunification of youths with their families or provide assistance, such as vocational training and employment readiness activities, designed to develop skills to live independently (Aviles and Helfrich, 2004; Karabanow and Clement, 2004).
Evaluating outcomes of youth using shelter services provides an opportunity to document intervention effectiveness, as well as guide future service development (Teare et al., 1994). Outcomes targeted by interventions to runaway/homeless youth have been conceptualized in terms of school status, employment, self-esteem, behavioral problems, mental health/substance use, and family relationships (Cauce et al., 1994; Song et al., 2001). To date, few studies have assessed the outcomes of runaway/homeless youth seeking assistance from youth crisis shelters. Systematic evaluations of outcomes for these service-using youth would provide much needed documentation of intervention effectiveness and point to new directions for service development. A methodologically sound evaluation of these services represents a critical next step for service provision and policy decisions for this under-served population.
The current study extends previous research conducted by the authors in which an exploratory study of runaway/homeless youths receiving services at a single shelter in a Midwestern city was conducted (Thompson et al., 2000). The most significant finding showed that shelter-using youths returning home to live with their parents experienced greater positive outcomes than youths discharged to other locations. Additional research by these authors compared short-term (6 week) outcomes for a sample of runaway youth following crisis shelter service use in eleven crisis shelter across four Midwestern states with a comparable group of high-risk youth using longer-term services (Thompson et al., 2002). Findings demonstrated that youth utilizing crisis shelter services had strong positive outcomes six weeks post-discharge, equivalent to youth utilizing ongoing, long-term services.
While these studies demonstrate that runaway youth successfully achieve positive short-term improvement following emergency shelter use, to date no longitudinal studies of service use outcomes for this population have been published. However, research of homeless adults suggests that short-term outcomes attenuate or become unstable over time for a portion of the population (Cauce et al., 1994; Toro, 1999; Zlotnick and Robertson, 1999). These findings argue strongly for the importance of examining outcomes longitudinally. It seems reasonable that the short-term positive outcomes found in our previous studies may diminish over time, as crisis services likely have a positive influence for a relatively short period of time.
To address this gap in the literature and build upon previous research, the present study examined the long-term outcomes and service use of a large sample of runaway/homeless youth following discharge from youth crisis shelters. Based on the literature, we hypothesize that participants who utilize runaway/homeless emergency shelters demonstrate improved outcomes across a variety of domains six weeks following discharge; however, the improvement diminishes during the six months post-discharge. Further, it is hypothesized that youth who receive additional services post-shelter have improved outcomes relative to those who do not receive additional services during the six-month time period.
Sample and setting
Data for participants (runaway/homeless youth using emergency shelter and crisis services) were gathered from eleven agencies that provide services to these youth in Federal Region VII (Missouri, Iowa, Nebraska, Kansas: MINK). The participant agencies were part of the MINK Network of Runaway and Youth Services, a consortium of programs providing emergency shelter and crisis services for youth. All MINK agencies are federally funded through the U.S. Department of Health and Human Services, Administration for Children and Families. The eleven MINK agencies participating in the study provided services across a variety of settings including urban, suburban, and rural locations; some were freestanding shelters and others were part of large service networks. Agency populations showed considerable variation in composition and size. Some agencies served only males or only females, others served both. All agencies provided similar emergency shelter and crisis services, including short-term basic shelter, crisis support, and counseling. Agencies were funded from a variety of sources, but received a majority of their funding from governmental supports and the United Way.
The study sample consisted of all youth discharged from each runaway/homeless youth emergency shelter for a 6 month period. Due to agency availability, there was some variation in start and end dates for study recruitment. All subjects were initially recruited between November 1997 and August 1998. Because the majority of study participants were minors, it was necessary to gain parental consent in person, over the telephone, or by mail. Washington University’s Human Subjects Committee approved all study procedures.
Interviews were conducted at admission and 6 weeks, 3 months, and 6 months post-discharge. Agency staff conducted all interviews, generally the same person who had conducted the intake interview or were the youth’s primary caseworker at the agency. In cases where youth could not be located after discharge, but a family member with recent information was available, proxy interviews were conducted. Previous studies using these methods of tracking subjects have been shown to yield a follow-up sample representative of initial intakes on key demographic and behavioral variables (Pollio et al., 2000; Thompson et al., 2002).
Demographic information was collected for participants at intake using the Runaway and Homeless Youth Management Information System (RHY MIS) instrument, a well-recognized data collection tool developed by the Administration for Children and Families (ACF). An additional instrument, developed specifically for this study, assessed outcomes across seven domains: (1) runaway behavior, (2) family relationships, (3) substance use, (4) school behavior, (5) employment, (6) sexual behavior, and (7) self-esteem. The RHY MIS and outcome instruments (including test-retest information) have been described elsewhere (Thompson et al., 2000, 2001, 2002).
Variables in the analysis
Gender, ethnicity, age, family contact prior to intake, living situation at time of admission, referral source, lifetime substance use and type, and previous suicidal ideation were collected at intake. Outcome data collected at intake and all follow-ups included twelve variables across the seven domains: (1) runaway behavior (days on the run), (2) family relationship (family contact yes/no, perceived family support rating, range 1–4), (3) substance use (used drugs/alcohol during time period, yes/no), (4) school behavior (suspended, expelled, received detention from school, had at least one of these events, all coded yes/no), (5) employment (currently employed yes/no, fired in last 6 weeks yes/no), (6) sexual behavior (currently active yes/no), and (7) self-esteem (5-item scale, range 5–20). Data for the self-esteem scale and family contact items were not available for analysis for the six-month follow-ups. To correct for skew on the ‘days on the run’ variable, it was transformed via taking the square root prior to analysis.
Service use was collected for seven service types at each wave of data collection. Because pilot efforts found that estimates of amounts or units of services used were relatively unreliable, each service was operationalized dichotomously as used/not used during each time period. The seven service categories were drawn from the RHY MIS instrument and included: (a) crisis phone service, (b) health care services, (c) alcohol/drug treatment, (d) mental health services, (e) housing services, (f) employment/skills training, (g) legal services.
Imputation of missing data was required due to the substantial attrition (approximately 25–30% per time period) and missing data within individual interviews. Missing data were replaced by multiple imputation (Schafer, 1997) to create an analytic sample of 371 (all eligible subjects providing substantially complete intake data). Complete datasets were created from the original data using Norm 2.1 software (Schafer, 1999). One thousand Monte Carlo Markov Chain imputations were run, with every 200 used to create a total of 5 implicative data sets. Data were then analyzed separately for each dataset and estimates “rolled up” to create final estimates. Using a scalar method of reconciling scores (Rubin, 1987), estimates, standard errors, t-ratios and degrees of freedom were calculated for each comparison.
Analyses following Schafer’s (1999) recommendations were conducted to examine outcomes over time. For each set of outcome variables, t-tests were performed by comparing the following: baseline with all outcomes, 6 week with 3 month and 6 month outcomes, 3 month with 6 month outcomes. Because the multiple imputation methods created continuous variables for outcomes in which only two conditions were possible (e.g., achieved/did not achieve), imputed continuous variables were recalculated to 1/0 prior to analysis. To determine significance, probabilities for dichotomous variables were recomputed into McNemar scores.
To examine the impact of various types of service use on outcomes, t-tests were calculated for continuous-level change scores between individuals receiving/not receiving services during that period. For dichotomous outcome variables, z-tests were calculated to identify the net percent change (positive change-negative change/total n) for individuals’ receiving/not receiving specific services during that time period. For this analysis, only the “any negative school event” variable was analyzed from the school domain, resulting in examination of nine outcome variables. For these outcome variables, significance tests were calculated for each service type and each distinct time period (baseline vs. 6 weeks, 6 weeks vs. 3 months, and 3 months vs. 6 months).
Youth using runaway/homeless emergency shelters were generally female (61%), white (73%), and averaged nearly 15 years of age (mean = 14.7, S.D. ± 1.5 years). Most had run away at least once (62%) and averaged more than two episodes (mean = 2.4, S.D. ± 5.0) for greater than 7 days total (mean = 7.2, S.D. ± 18.2). The vast majority of these youth had been living at home (81%) prior to the runaway episode leading to admission. Lifetime substance use was highly prevalent across a wide range of substances, marijuana being the most frequently reported (94%), followed by alcohol (76%), cocaine (17%), and inhalants (13%). Over one third of the sample (36%) reported they had previously contemplated suicide. Most youth had been referred to crisis services by their family (36%) or by formal systems (54%).
Baseline data across the outcome domains indicated that the vast majority of youth reported recent family contact (88%) with a moderate level of perceived family support (mean = 2.4, S.D. ± 1.2). Almost half the sample reported currently being in school or having recently graduated (49%). A substantial proportion of those in school (45%) reported a recent negative school event; 31% had been in detention, 27% suspended, and 8% expelled in the three months prior to admission at the youth agency. A relatively small proportion (13%) was employed at and 10% had been fired in the time immediately prior to intake. Over three-quarters of the sample (77%) reported current substance use. Almost half (43%) were currently sexually active and self-esteem was generally quite low (mean = 9.3, S.D. ± 2.9).
Percent using services for each time period post-discharge
Differences in variables for each time period
6 week vs.
3 month vs.
6 week vs.
vs. 6 weeks
vs. 3 month
vs. 6 month
Days on the runa
Family contact (%)b
Perceived family supportc
Currently using substances (%)
Any negative event (%)
Currently employed (%)
Fired since last survey (%)
Currently sexually active (%)
Self esteem scaleb,c
Analysis of outcomes over time
Table 2 presents changes in outcomes over time. For ease in determining direction and magnitude of change, baseline rates reported are included in the first column, with subsequent columns reporting direction and amount of change; exact scores can be calculated by adding or subtracting from the baseline column to obtain the rate for each time period. For days on the run, significant decreases in days on the run occurred at all three time periods relative to baseline (6 weeks t=5.16, p < .0001; 3 months t=6.35, p < .0001; 6 months t=3.22, p=.001), with significant decreases also occurring from 6 week to 3 months (t=2.69, p=.009). However, increases in days on the run were observed by 6 months relative to both other follow-up time periods (3 vs. 6 t=−5.22, p < .0001, 6 vs. 6 t=−2.82, p=.006). Figure 1 presents these results graphically.
The proportion of youth reporting family contact increased significantly at each measurement period relative to baseline (6 week p=.05; 3 month p=.001), as did 6 week versus 3 month (p=.04) (because significance testing is based on McNemar scores, no chi-square statistic is reported for these results). For perceived family support, significant improvement occurred at all three time periods relative to baseline (6 weeks t=5.62, p < .0001; 3 months t=3.56, p=.001; 6 months t=5.44, p < .0001) with no significant differences detected between other follow-up time periods.
Substantial and significant decreases were reported at all three follow-up periods versus baseline for current substance use (all follow-ups p < .0001). There was an additional significant decrease in the proportion of youth reporting substance use from 6 weeks to 3 months (p=.0002). This change, however, reversed itself between 3 and 6 months (p=.002), resulting in no significant differences in proportion using substances between 6 week and 6 month follow-ups.
Analyses of outcomes relative to school events yielded mixed results. Although there was a significant decrease in proportion of youth experiencing any negative school event (suspension, expulsion, or detention) between 6 weeks and 3 months follow-ups (p=.008), this change did not occur during other time periods. Findings for individual types of negative events varied, with no consistent pattern emerging.
Analysis of impact of service use on outcomes
Generally, the analysis revealed that service use had little consistent impact on individual outcomes. Across all nine outcomes analyzed for each time period, only seven individual tests (out of a total of seventy-five) proved significant. Four of the seven significant findings occurred in the analysis of the impact of service type on days on the run. Receipt of alcohol/drug treatment between 3 and 6 month follow-up (t=−2.50, p=.02) and employment services between 6 week and 3 month follow-up (t=3.10, p=.005) were each associated with significantly greater decreases in number of days on the run for those receiving that service than for those not. Receipt of mental health services between 6 week and 3 month follow-ups (t=−3.40, p=.001) and health services between 3 and 6 month follow-ups (t=−2.32, p=.02) were associated with significantly greater increases in days on the run for those receiving the service than for those not.
For the remaining three significant findings, two occurred in analyzing the self-esteem outcome, and the last in the parental support outcome. For self-esteem, between baseline and 6 week follow-up, receipt of employment services was associated with significantly greater increases in self-esteem scores for those receiving services than for those not (t=4.12, p < .0001). In the opposite direction, receipt of legal services was associated with significantly decreased change scores for those receiving services than for those not (t=−2.39, p=.03). For the parental support outcome, receipt of health services was associated with significantly greater increases in perceived parental support for those receiving services than for those not (t=3.5, p=.005).
The results of this study are informative for understanding treatment effects over time for adolescents who use emergency shelter services. The findings appear to confirm our hypotheses that argue for the broad, short-term effectiveness of these services in crisis resolution for runaway/homeless youth, but question their long-term influence.
In assessing outcomes over various time periods, results appeared to follow three primary patterns. First, for some domains, outcomes did not consistently improve over time. For example, the measure of sexual activity showed an increase in the numbers of youth reporting current sexual activity at 6 months post shelter use. School related outcomes were inconsistent, with improvements in some areas counterbalanced by declines in others; overall rates of negative events were unchanged over time. The second pattern showed more positive outcomes for three major domains: runaway behavior, family interaction, and substance use. All of these areas maintained significant improvement over baseline for all time periods.
The third pattern of outcomes suggests that for some domains, the gains achieved in the initial periods diminished or were reversed at later assessment. For example, positive changes in employment and self-esteem had disappeared by final assessment. Additionally, although days on the run and substance use remained significantly lower at 6 months compared to baseline, significant increases in days on the run and those using substances were identified between 3 months and 6 months. This pattern supports our first hypothesis that positive effects of crisis shelters are broad-based, but relatively short-lived.
In examining the impact of post-intervention service use on outcomes, receipt of various types of services was not consistently associated with achieving or maintaining positive change. For example, post-discharge receipt of employment and alcohol/drug services (where significant) did predict greater improvement in outcomes relative to those not receiving services. As the vast majority of youth reported pre-service substance use, this finding may suggest the potential benefit for referrals to these types of services. For mental health and legal service use post-discharge, however, receipt of that service type was associated with decreases in gains relative to those not receiving the service. One explanation for this service use may be that use of mental health or legal services may represent a proxy for the presence of an additional existing condition (e.g., major mental illness, conduct disorder, juvenile justice problems) among a subgroup of this sample of runaway/homeless youth. These sub-group’s difficulties may be more persistent than that of the other youth in the sample, making it less likely that they can achieve improvement through crisis-oriented services and require additional assistance on a long-term basis. Given the limitations of these findings, however, these explanations are offered with caution.
Although informative as an initial longitudinal study of crisis service effectiveness among runaway/homeless youth, several significant limitations must be considered in interpreting the results. First, lack of a control group limits conclusions concerning causal assertions about the effectiveness of services. Certainly, future research on service use for this population needs to include a comparison condition of other troubled youth, perhaps runaway/homeless youth not seeking crisis services. Second, aspects of the sampling strategy limit generalizability of the findings. The sample includes only service-using youth; thus, it is not generalizable to the entire runaway/homeless population. We believe that the youth in this sample are representative of the population of service-using runaway/homeless youth from Missouri, Illinois, Nebraska, and Kansas. Other research has suggested, however, that this population is not representative of other geographic areas across the U.S. in terms of demographics and other factors (Thompson et al., 2003) and provides support for the need to examine service-seeking populations in other regions of the U.S.
Additionally, outcomes were only measured for 6 months post-intervention. Although this extends the length of longitudinal studies currently in the literature, whether further attenuation occurs or outcomes maintain can not be ascertained by the current study. Finally, agency personnel interviewing their own program participants may have introduced bias; however, the trade-off of re-locating individuals and tracking a representative sample without a copious budget made this bias unavoidable. A further challenge of using agency personnel as interviewers was that the outcome interviews had to be relatively brief. As a result, the instrument that measured outcomes and service use was limited in complexity. Finally, the limited number of significant differences detected relative to the large numbers of analyses makes it impossible to rule out the possibility that the few significant findings achieved are merely artifacts of the number of analyses.
Implications for services
The findings of this study have direct implications for service provision with runaway/homeless youth. Most importantly, the evidence of short-term outcome improvement supports the need to continue provision of short-term crisis services for this population.
The results also point to the need for additional services following discharge from emergency shelter services. While it appears that these services can be instrumental to the successful outcomes of these high-risk youth, high-risk youth exhibit low rates of social service utilization (Brooks et al., 2004; De Rosa et al., 1999; Kipke et al., 1993; Slesnick et al., 2001). Barriers to appropriate utilization have been reported as lack of insurance, transportation, and knowledge of the systems of care (Yates et al., 1991) and youth are often distrustful of formal services and authority figures, as many have been exploited and victimized by adults, including members of their own families (Farrow et al., 1992). Many fear they will be reported to the police or child protective services due to status offenses or other outstanding warrants. Outreach services and drop-in centers are the most frequently utilized types of services offered to runaway/homeless youth and often act as a gateway to other services (De Rosa et al., 1999; Mallett et al., 2004).
Results, both in terms of high prevalence and significant associations, most clearly point to the need for follow-up services addressing substance-related issues. Although current services offered by emergency shelters must continue to focus on crisis intervention and offering safe, short-term residential care, substance use issues must be acknowledged. Providing substance use treatment is an immense challenge and likely impossible for these agencies; however, they are in a prime position to facilitate screening, assessment, and referral to substance abuse treatment (Thornberry et al., 1991).
Finally, the finding that outcomes in the family domain maintained for at least six months suggests the potential benefit for family involvement in post-discharge services, especially when viewed in light of previous research arguing that reunification is associated with improved outcomes (Thompson et al., 2000; Whitbeck et al., 1997a). Research has consistently suggested potential benefits for including families in treatment of high-risk youth. Prevention efforts with delinquent and drug-abusing youth indicate that the single most effective form of prevention involves working with the total family system (Kumpfer et al., 1998). Identification of situations where families may be engaged in services is a potentially beneficial method for addressing problems experienced by youth. Parents could benefit from learning new management skills, such as effective monitoring, supervision, and discipline. Providing youth and parents with strategies to better communicate would lead to an improved family environment (MacLean et al., 1999; Whitbeck et al., 1997b). Inclusion of appropriate families in after-care represents a promising direction for service development.
Implications for evaluation and future research
An additional contribution this study can offer is a demonstration for the potential importance and utility of academic and provider partnerships in completing community-based evaluations. The care in development of data collection methods (instrument reliability/validity, tracking, and agency training) utilized in this research has resulted in a replicable methodology, as well as information useful to the provider agencies. This study further demonstrated the feasibility of agency-based tracking and evaluation for this difficult-to-follow population. The use of sophisticated statistical and data manipulation procedures, such as imputation of missing data, can to some degree compensate for limitations imposed by budgetary constraints. While studies such as this one can in no way replace more sophisticated research inquiries, they can add practical knowledge useful for policy and program development.
Beyond addressing the obvious methodological limitations of this study (e.g., including a comparison group, sampling outside the service-using population, more complete instrumentation, etc.), a critical next step for research on runaway/homeless youth is to examine the impact of organizational and environmental characteristics on outcomes. Because service settings vary (e.g., rural versus urban, freestanding shelters versus multi-service agencies), presuming that service populations are identical within these various agencies is overly simplistic. Examination of service use and outcomes across various settings would allow better specification of service needs and appropriate interventions for this heterogeneous population. Further, assessments would enhance our ability to predict client outcomes by increasing ability to account for sources of variation. Understanding which agencies serve youth that are less likely to achieve outcomes and those with greater likelihood of success would avoid the potential for invidious comparisons of effectiveness between agencies.
Short-term shelters are a central part of the national service response to the needs of runaway and homeless youth. The current study adds to our knowledge of the effectiveness of this model through testing hypotheses around their short- and longer-term effectiveness, as well as exploring the role of post-discharge service use. The results affirm the hypotheses that runaway shelters are effective across a broad range of outcomes, but that over time many of these outcomes attenuate. Post-discharge service use does not appear to be consistently associated with improved outcomes; however, this may be an artifact of the measurement or reflect differences in severity of youth’s problems. The findings suggest the need for post-discharge coordination of care, particularly around substance use and family issues.
The authors wish to thank the agencies in the MINK (Missouri, Iowa, Nebraska, Kansas) Network of Runaway and Homeless Youth Shelters. This research was supported by a grant from Administration for Children and Families, Department of Health and Human Services to Dr. Pollio.