Improving Programming in Juvenile Detention: The Impact of Project Safe Neighborhoods Youth Outreach Forums

Abstract

Objectives

A growing body of evidence suggests focused deterrence strategies successfully reduce criminal behavior. Very little of this evidence comes from randomized experiments. This paper takes a step toward filling this gap in the literature. We present the results of a randomized experiment evaluating a series of youth outreach forums that leverage several focused deterrence strategies.

Methods

This paper presents the results of a randomized controlled trial of a youth outreach forums program run in the Cook County Juvenile Detention Center (JTDC) by the Northern Illinois Project Safe Neighborhoods Task Force.

Results

We find the program caused a 20 percent reduction in the number of new spells at the JTDC in the eight months after random assignment and reduced total arrests by 18 percent in the first year after random assignment. While both of these impacts are somewhat imprecisely estimated, the reduction in total arrests is driven by statistically significant 43 and 40 percent reductions in arrests for violent and drug crime, respectively, and a large but less precisely estimated 30 percent reduction in arrests for property crime. These correspond to very valuable and proportionally large reductions in the social costs of crime. Our estimates also suggest the forums increase attachment to school.

Conclusion

The results of our study suggest juvenile detention centers may better reduce the future criminal behavior of residents by implementing similar programs to the youth outreach forums program.

Introduction

Nearly 50, 000 youth, who hail disproportionately from the most economically disadvantaged and socially isolated neighborhoods in America, are incarcerated in a residential facility on a given day (Sickmund et al. 2019). This juvenile incarceration has the potential to serve several policy purposes. Incarceration prevents youth from committing additional crimes while they are detained; the threat of being imprisoned may deter some youth from committing crimes in the first place; and incarceration could serve as an intensive intervention designed to redirect youth to more pro-social activities than crime, like school or formal employment. These potential benefits come at a cost. The state of Illinois spends over $300 per day for every incarcerated youth (Petteruti et al. 2014). But this direct cost may be dwarfed by the long-term costs of incarceration: Youth are unable to attend their regular school while incarcerated. After leaving detention, many individuals have reduced economic opportunities and lower earnings which also affects the individual’s family and community (Western and Pettit 2010). The costs are not limited to economic outcomes: nationally, nearly one in ten incarcerated youth is sexually assaulted while in custody (Beck et al. 2013).

The best evidence to date suggests juvenile incarceration in its current form primarily serves to incapacitate youth. Lee and McCrary (2017) show there is little difference in criminal behavior of youth before and after their 18th birthday, despite a substantial increase in potential punishments in the adult justice system. This suggests the threat of incarceration is not a serious deterrent of juvenile crime. Aizer and Doyle (2015) use differences in the sentencing patterns of randomly assigned judges to identify the impact of juvenile incarceration on adult outcomes. They find juvenile incarceration reduces the probability of graduation and increases the probability of incarceration later in life. This suggests juvenile incarceration may be pushing youth further into crime rather than redirecting them to more pro-social activities. Since incapacitation is the shortest lived of the potential benefits of juvenile detention, these results suggest juvenile incarceration in its current form may not be an efficient policy. This raises the question: are there aspects of juvenile detention that can be improved?

This study addresses the narrower question of whether a Youth Outreach Forums program that has been run in the Cook County Juvenile Temporary Detention Center (JTDC) by the Northern Illinois Project Safe Neighborhoods (PSN) Task Force since 2015 can improve the outcomes of detained youth. The core of the program is a series of four forums that were usually held over four consecutive days during school breaks. Each forum provides participants the opportunity to have a structured discussion in as neutral a setting as possible with authority figures in their lives, including family members, teachers, and law enforcement agents, as well as individuals with insights about the consequences of crime, including community members and ex-offenders. The program was inspired by quasi-experimental evidence that a similar program serving adult probationers led to a significant reduction in homicide rates (Papachristos et al. 2007).

The theoretical basis for the program is the idea that most people voluntarily comply with laws and rules they consider legitimate (Tyler 2006a, b; Tyler and Trinkner 2018). According to procedural justice theory, this legitimacy can be established by treating individuals with dignity and respect and giving them a voice in the interaction and by demonstrating that the authority figure is neutral and trustworthy. Bottoms and Tankebe (2012) go further and explain that legitimacy is dialectic in nature, requiring both the audience and the power-holders to agree on the legitimacy. The hope is giving youth the opportunity to have an open discussion with the authority figures in their lives about the logic behind the rules that are in place and the consequences of breaking those rules will better establish this “audience legitimacy” by enhancing perceptions of the procedural justice of their interactions with the legal authorities with whom they are dealing (Tyler and Huo 2002). Although specifically motivated by procedural justice theory, the program can more generally be considered to be a focused deterrence strategy. Braga and Kennedy (2021, p. 3) explain that focused deterrence strategies “seek to change offender behavior by understanding underlying violence-producing dynamics and conditions that sustain recurring violence problems, and implementing a blended set of law enforcement, informal social control, and social service actions.” In this spirit, the program leverages many elements that could all reduce crime. For example, the program could reduce crime by making the long-term consequences of youth’s actions more salient, developing participants’ skills, introducing the youth to positive adult mentors, or connecting the youth with community support.

To credibly measure the impact of the program on youths’ outcomes, the program was implemented in the JTDC as a randomized controlled trial in 2015 and 2016. The research team randomly selected groups of youth to be invited to voluntarily participate in the program using a cluster-randomized design. Because the treatment is randomly assigned, any differences in outcomes between the treatment and control groups can be credibly attributed to the program.

Our preferred estimates indicate that the program caused a 20 percent reduction in the number of new spells at the JTDC in the eight months after random assignment and reduced total arrests by 18 percent in the first year after random assignment. While both of these impacts are somewhat imprecisely estimated, the reduction in total arrests is driven by statistically significant 43 and 40 percent reductions in arrests for violent and drug crime, respectively, and a large but less precisely estimated 30 percent reduction in arrests for property crime. These correspond to very valuable and proportionally large reductions in the social costs of crime. There is a small and insignificant reduction in other crime. Our estimates indicate proportionally large, but imprecisely estimated, increases in school attendance and grade point averages (GPAs). However, these increases are relative to very low baseline levels of school attachment. These results provide suggestive evidence that offering similar programming to incarcerated youth could reduce future criminal behavior and increase attachment to school.

These results contribute to a growing literature that demonstrates the potential of focused deterrence strategies to reduce crime. Braga et al. (2018) conduct a systematic review of 24 quasi-experimental evaluations of focused deterrence strategies. They note that 22 of the 24 studies in their review found beneficial impacts on crime with the other 2 studies finding basically no difference between the treatment and comparison conditions. Notably, none of the studies in their review used a randomized experimental design. Our study helps fill this gap in the literature.

Our results are also an encouraging contribution to the small experimental literature evaluating whether particular programs can improve outcomes for incarcerated individuals. Farrington and Welsh (2005) review 14 experiments evaluating correctional programs. Two of these studies found “scared straight” programs increase recidivism and four found juvenile bootcamps have no effect or have a detrimental effect on re-offending. The evidence from eight experiments evaluating therapeutic programs offered to inmates are more encouraging, but the effects are usually statistically insignificant. Grommon et al. (2018) experimentally evaluate a dog-training program offered to youth in a midwestern juvenile detention center and find that it has no effect on self-reported psychosocial measures, including self-esteem, empathy, optimism, pessimism, compassion, and social competence. Heller et al. (2017) leverage a natural experiment in the JTDC, the same setting as the present study, and find residents who were randomly selected to receive a set of reforms that increased the educational requirements for staff, instituted a new system of rewards for youth, and introduced youth to cognitive behavioral therapy techniques for managing conflict and bad behavior were 21 percent less likely to return to the JTDC. Programs that seek to keep youth out of the juvenile justice system through restorative justice conferencing also have shown mixed results. Recently, Kimbrell et al. (2023) find promising results through a meta-analysis of both experimental and quasi-experimental research designs that restorative justice programs are more impactful than other conventional methods. Shem-Tov et al. (2021) find that participation in the Make-it-Right (MIR) restorative justice program in San Francisco, which worked with youth ages 13 to 17 who committed a felony, reduced recidivism. In contrast, Vooren et al. (2022) find that participation in Halt, a restorative justice program for youth in the Netherlands, increased the probability of recidivism one year after the program and decreased educational attainment.

Finally, our study provides lessons on running experiments in detention centers. In the first year of the study, random assignment occurred about two weeks before the program was scheduled to begin in order to give JTDC staff time to prepare for the program. This had the unintended consequence of making treatment uncorrelated with program participation because of youth’s typically short spells in the JTDC. Many youth who were assigned to the treatment group were no longer at the JTDC when the program was run. After observing the first year results, the research team and program administrators made adjustments to the randomization protocol in order to increase the relevance of random assignment in the second year of the experiment, including moving it much closer to the start of the program. Given that these adjustments were effective, our preferred estimates are from the second year of the study. Because the first year of the study is not informative about treatment effects, our estimates are less precise than expected in the experimental design. Given the imprecision, we emphasize effect sizes and confidence intervals in addition to p-values. Recent debates in the statistics literature on the value of hypothesis testing have emphasized that a focus on effect sizes and confidence intervals may better address whether a hypothesis is correct than the traditional focus on p-values (Wasserstein and Lazar 2016; Benjamin et al. 2018).

The remainder of this paper is organized as follows. Section 2 provides details about the program and its development. Section 3 explains the experimental design and the details of the analysis. Section 4 provides descriptive statistics on the youth in our sample. Section 5 presents the program’s impact on youth’s post-randomization juvenile detention, arrests, and schooling. Section 6 concludes.

Context and Intervention

Antecedents of the Intervention

The Project Safe Neighborhoods (PSN) initiative was launched in 2001 by the United States Department of Justice to reduce gun violence (McGarrell et al. 2009). The initiative provides each of the 94 U.S. Attorney districts with financial resources, training, and a framework to support local solutions to gun violence. Each region formed a PSN taskforce that included representatives of the U.S. Attorney’s Office, other federal, state, and local law enforcement agencies, members of the broader community, including representatives from local governments, schools, and social service agencies, and researchers. While the PSN taskforces could develop solutions developed to their own local context, each taskforce was provided with training on intensively prosecuting violent gun crime, interrupting the supply of illegal guns, holding offender deterrence meetings, and offering support services to help shift individuals out of crime. In the first seven years of the program, Congress allocated nearly 3 billion dollars to support the PSN initiative (McGarrell et al. 2009). In 2022, the Office of Justice programs awarded about 17.5 million in PSN grants.¹

The PSN initiative was inspired by the perceived success of Richmond’s Project Exile and Boston’s Operation Ceasefire. Richmond’s Project Exile was a joint effort between law enforcement in Richmond, Virginia and the regional U.S. Attorney’s office to prosecute all drug and domestic violence cases involving guns and all felon-in-possession-of-a-firearm cases in federal courts (Raphael and Ludwig 2003). Federal sentencing guidelines for gun crime were more severe than those in Virginia when the policy was in operation and would normally result in sentences in out-of-state prison. An advertising campaign was used to inform the community and potential offenders about the increased certainty of enforcement. The policy took effect in February 1997. A 35 percent decline in the homicide rate and a 37 percent decline in the gun homicide rate between 1997 and 1998 contributed to the perception of Project Exile’s success. However, Raphael and Ludwig (2003) show that most of this decline was driven by an unusually high homicide rate in 1997. They show that more credible research designs suggest the program had little to no effect.

Boston’s Operation Ceasefire was a partnership between the Boston Police Department, the Bureau of Alcohol, Tobacco, and Firearms, the U.S. Attorney, the Suffolk County District Attorney, the Massachusetts Department of Probation, the City of Boston, the Massachusetts Department of Parole, social service workers, and researchers (Kennedy 1997). The intervention was motivated by the insights that the justice system was not effectively deterring crime because punishments were uncertain and insubstantial and that a small number of individuals are responsible for a large share of crime. The program aimed to more effectively deter violent crime using a “focused deterrence strategy.” Legal authorities would respond to violent crime by “pulling every lever” to impose additional costs on committing crime. A violent crime will bring heightened law enforcement attention to all crimes, including to more minor violations, like probation violations, loitering, or drinking in public. This new enforcement strategy was communicated at formal meetings with gangs and gang members and presentations at schools and juvenile detention centers, and by word of mouth through social workers and law enforcement agents.

Braga et al. (2001) measure the impact of Operation Ceasefire by comparing the number of youth homicides in Boston in the years before and after the intervention took place and also by comparing trends in Boston’s crime outcomes to analogous trends in other cities. The pre-post comparison shows that Operation Ceasefire led to a large proportional reduction in youth homicides. The comparison with other cities showed that Boston’s crime reduction was relatively large compared to typical changes during that time period, but several other cities saw similar declines.

This type of focused deterrence strategy has also been used to address drug markets. The High Point Intervention in High Point, North Carolina addressed drug markets by targeting a particular drug market, arresting any violent drug dealers in that market, and having non-violent drug dealers attend a “call-in” meeting where community members, social service providers, former offenders, and law enforcement tell the drug dealers that they are valued in the community and that the drug dealing needs to stop (Kennedy 2009). The non-violent dealers are informed that law enforcement has developed a case against them, but the case will not be brought unless they re-offend. Using quasi-experimental methods, Corsaro et al. (2012) find that the High Point Intervention led to modest violent crime reductions, especially in the first two areas that were targeted.

In addition to addressing group crime dynamics, like gang violence or drug markets, focused deterrence strategies have also been used to address recidivism of individual offenders. In Chicago, a key element of the PSN strategy was a series of “Offender Notification Forums” (Papachristos et al. 2007). Offenders with a history of gun violence were asked to attend an hour-long forum about the consequences of gun violence and how to transition away from crime. The meetings began with a 15 min discussion with law enforcement about local and federal laws regulating gun violence and the consequences of violating these laws. Next, an ex-offender would lead a discussion of how he had successfully avoided reoffending. The forum would conclude with a discussion with social service providers in the community about what resources are available to support their success. These forums were designed with the goal of establishing legitimacy. Meetings were held at neutral locations, like parks or libraries. All participants would sit around a table on a level playing field. The PSN taskforce complemented these forums with federal prosecutions of gun crime, gun recovery efforts, and community outreach. Papachristos et al. (2007) find that these efforts caused a large proportional reduction in homicides in targeted police districts using quasi-experimental methods.

The Northern District of Illinois’s PSN task force also used group violence reduction strategies. Papachristos and Kirk (2015) discuss Chicago’s ‘call-in” program. These meetings brought together groups of gang members who had been identified by the police as being involved in potentially violent conflicts, police leadership, and community members. The police informed the gang members that the next shooting they were involved in would get the full attention of law enforcement. Community members emphasized that the gang members were part of their community but that the violence had to stop.

Focused deterrence strategies are now an important component of crime prevention policies. As demonstrated by the above examples, focused deterrence strategies generally follow the same framework. An interagency task force is brought together to address a particular crime problem. The crime is then addressed with special enforcement efforts that are complemented with an outreach campaign informing potential offenders about the enforcement, the impact of crime on the community, and the services available to support their success (Kennedy 2006). Braga and Kennedy (2021) provide a more detailed overview of the theory, development, and applications of focused deterrence strategies. They highlight that the effectiveness of focused deterrence is supported by its “strong logic model” that creates several mechanisms that could all help reduce crime. First, focused deterrence more effectively generates a deterrent effect by combining enhanced punishments with clear communication to potential offenders about the change in enforcement. Second, focused deterrence promotes legitimacy and procedural justice by encouraging respectful, direct communication between legal authorities and potential offenders (Papachristos et al. 2007; Braga and Kennedy 2021; Bottoms and Tankebe 2012). Third, the participation of community members and social service providers reinforces informal social control. They actively highlight the negative impact of crime on the community, increasing awareness and accountability. Furthermore, by connecting potential offenders with services that not only support their personal success, but also underscore the community’s investment in their well-being.

Braga et al. (2018) conduct a meta-analysis of the evaluations of focused deterrence strategies. They identified 24 quasi-experimental evaluations. The evidence presented in these studies suggests that focused deterrence strategies causes a moderately large reduction in crime outcomes. However, they find that the effects are smaller, but still beneficial, in studies using research designs that they identify to be more credible. Their review did not identify any evidence from randomized controlled trials.

The Program

The PSN Youth Outreach Forums program was developed by the Northern District of Illinois’ PSN Task Force and was inspired by adult outreach forums studied by Papachristos et al. (2007). Like the adult forums, the youth forums can be thought of as an individual focused deterrence strategy because the forum participants are not necessarily members of a shared group. With that said, the youth intervention studied here and the adult intervention studied by Papachristos et al. (2007) have some key differences. The youth program is implemented while participants are incarcerated in a concentrated four-day setting rather than a long-term setting when they are released. The primary component of the program is a series of four forums which provide participants the opportunity to have a structured discussion in as neutral a setting as possible with authority figures in their lives, including family members, teachers, and law enforcement agents, as well as individuals with insights about the consequences of crime, including community members and ex-offenders who have moved away from crime. Youth sit in a circle with the speakers in order to put the youth and speakers on as level a footing as possible consistent with the tenets of the theory, which emphasize, among other factors, the importance of recognizing individual dignity.

The program was managed by the Northern District of Illinois’ U.S. Attorney’s Office and implemented by the PSN Task Force in partnership with the Cook County Juvenile Detention Center. The Northern District of Illinois’ PSN Task Force implements many programs reaching both youth and adults. All of the forums were facilitated by a member of the PSN Task Force from the Chicago Police Department.²

The curriculum consists of four forums which were typically held over four consecutive days during school breaks. The forums were offered over breaks because residents attend school based on a typical school year calendar at the JTDC’s Nancy B. Jefferson school. Figure 1 shows the daily population of youth in JTDC and the date when youth were randomized into the program. During school breaks, there may not have been as much regular programming, so this program also provided additional programming to youth in JTDC.

The first forum is “Walking the Walk.”³ In this forum, ex-offenders talk to youth about the consequences of their criminal life. This includes a presentation by ex-gang members affiliated with the organization “In My Shoes” who have permanent disabilities due to violence resulting from their gang involvement⁴ and an adult who spent time in the JTDC as a teenager who spent most of his adult life in prison because of his continued involvement with the criminal justice system as a young adult. He emphasizes that he felt like he was “getting away with it” as a teenager until he was the subject of a federal indictment that resulted in more than two decades in prison. These presentations aim to reinforce how participants’ lives may be adversely affected by ongoing involvement in crime and continued contact with the criminal justice system. The “In My Shoes” presentation emphasizes the potential non-legal consequences of being involved in crime (Braga and Kennedy 2021). The second presentation enhances the salience of potentially escalating criminal consequences. Both presentations contribute to deterrence by emphasizing the costs associated with criminal behavior.

The second forum is titled “Mutual Respect with the Law.” This forum is designed to increase youth’s trust in law enforcement and establish the legitimacy of law enforcement while decreasing youth’s cynicism about the law. In the forum, youth have a neutral conversation with uniformed police officers with the goal of showing the participants that police officers are regular people, explaining why police behave the way they do, and reinforcing the consequences of continued involvement with the justice system. All of the officers were members of the Chicago Police Department’s command staff. Most of the youth recognized that the gold stars on the officers’ uniforms signaled that they were high-ranking officers. This is intentional because it gives the youth the opportunity to talk about their negative interactions with police officers with decision makers who can also speak with authority about the department’s policies and procedures. As Bottoms and Tankebe (2012) discuss, misconduct by authority figures undermines legitimacy and creates resentment. The discussion of how police are themselves held accountable directly addresses this source of illegitimacy. The presence of command staff at the forum could also have a deterrent effect by emphasizing the importance of the forums and showing the youth that they are under scrutiny (Braga and Kennedy 2021). Anecdotally, this forum had a big impact on some participants. Many youth would begin the forum defensively and would not speak to the officers, but by the end of the session, most of the youth were willing to talk to the officers and even shake hands.

The third forum is titled “Community and a Shared Moral Voice.” The goal of this session is to help youth understand community norms, values, and morals. This is accomplished in two ways. First, community members are invited to have a conversation with youth about how crime affects the community. This promotes legitimacy by establishing that rules are supported by a set of shared beliefs (Bottoms and Tankebe 2012). Second, youth are introduced to resources available in the community that are designed to help them continue to make better decisions once they are released. These social supports could reduce crime for a number of reasons. For example, they could directly solve a problem (i.e. helping pay bills) that the youth may have tried to solve by committing a crime (i.e. selling drugs). As another example, they could also enhance youth’s sense of belonging in their community.

The final forum’s focus changed over the course of the experiment. It was originally called “Hope Outside the Law.” The session aimed to help youth develop more positive relationships with non-legal authorities, their parents and teachers in particular, understand the importance of behavioral standards in non-legal environments, and increase youth’s future orientation. This session was designed to be split evenly between parent–child relationships and teacher-student relationships. Youth would be introduced to a school re-engagement specialist employed by the program at the end of this session. Reengagement specialists work with the youth to find the school within Chicago Public Schools that will best meet their needs and maximize their chances of graduating. In practice, it was difficult to implement “Hope Outside the Law” as intended because it required parents to make a sometimes long trip to the JTDC to participate and teachers to volunteer their time on their off days. In one case, no parents showed up and that portion of the session needed to be improvised. Even when some parents did attend, the program facilitators worried that this emphasized the absence of caring adults in some of the other children’s own lives.

Given the challenges in implementing the “Hope Outside the Law” forum, it was replaced by a new forum focused on conflict resolution strategies in the second year of the study. This session is built around cognitive behavioral therapy principles and gives youth the opportunity to practice conflict resolution strategies in the context of interactive games. For example, in one activity, a single youth is given instructions to do a non-verbal action. The other kids are then asked to discuss what message the youth is sending. The facilitator would highlight that the wide variety of responses highlights that people interpret things in different ways. Recognizing that they may mistakenly interpret something as aggressive or antagonistic could help youth avoid entering into a conflict unnecessarily. Similar cognitive behavioral therapy programming has been convincingly shown to reduce youth’s future criminal behavior (Heller et al. 2017).

Although the vast majority of youth in the study are boys, several adjustments are made when girls participate. In particular, the “Walking the Walk” forum is adjusted to include female ex-offenders and the program “In Her Shoes” which focuses on domestic violence issues. The “Mutual Respect with the Law” forum includes female command staff. Inviting female command staff not only gives the girls the opportunity to have the same discussion about their rights and responsibilities while dealing with law enforcement, but also gives them exposure to women in positions of power.

The forums were funded by a $500,000 Bureau of Justice Assistance grant to support PSN initiatives in the Northern Illinois District. Much of this funding supported the cost of personnel required to develop and coordinate the youth forums program. The marginal cost of running each forum is much less, only a couple hundred dollars, given that many of the presenters volunteered their time.

The Facility

The Juvenile Temporary Detention Center (JTDC) in Cook County, Illinois is the largest juvenile detention center in the United States. More than 80 percent of youth admitted to the JTDC are between 15 and 17 years old, but youth as young as 12 and as old as 20 were admitted during our sample window (March 1st, 2015 through April 30th, 2017). The average youth in our sample is almost exactly 16 years old on the day of their admission.

There were 2,622 admissions to the JTDC by 1,718 unique youth in the year starting July 1st, 2015. On average, there were 298 youth housed at the JTDC on any given day during this period. Figure 1 shows the daily population based on the juvenile court records shared with the research team covering all admissions between March 1st, 2015 and April 30th, 2017. There is a clear seasonality to the population, with the population highest in the summer and lowest in the winter. Most stays at the JTDC are relatively short. The median completed spell in our data is 15 days and the interquartile range is 3 to 31 days. The subset of youth who are being housed in the JTDC until they are old enough to transfer to the adult court system are classified as “automatic transfers” (AT). These youth may have substantially longer stays as they are awaiting transfer to the adult court system.

Residents of the JTDC are organized into a series of residential “pods” which typically house 8 to 12 youth. With few exceptions, youth participate in nearly all offered activities together with the other youth in their pod and rarely interact with youth from other pods. Although youth may be offered activities within their pod, participation is voluntary. Pods are segregated by gender and whether or not a youth is an AT.

Experimental Design and Analysis

The experiment includes seven rounds of forums that were run between March 2015 and August 2016. For each round of forums, JTDC administrators provided the research team with a list of pods eligible to participate in the forums. The research team randomly selected a pre-specified number of pods using a block randomized design. Typically, three or four pods were selected for treatment in each round of the lottery so that each session of the forum could be held on the same day. Blocks were defined by lottery, AT status, and gender. Randomization was at the pod level because, with few exceptions, youth complete all activities together with their pods.

Table 1 shows the number of clusters included and selected for treatment across all lotteries and separately for the first and second year of the experiment. A cluster is defined as a pod-lottery combination. Most pods were included in multiple lotteries, but because most youth spend less than a month in the JTDC, only a small number of youth were included in multiple lotteries. The experiment includes a total of 86 clusters of which 26 were selected to receive the program. The first and second year of the study include 56 and 30 total clusters of which 17 and 9 were selected for the treatment group, respectively.

Individual youth are considered treated if, on the day of the lottery, they resided in a pod which was selected for treatment. Youth are part of the control group if, on the day of the lottery, they reside in a pod that was not selected for treatment. To be conservative, we do not include individuals who move into a treatment or control pod after the day of randomization in the study because knowledge of a pod’s treatment status may have affected how youth were assigned to pods.

Table 2 shows the number of youth participating in at least one forum by treatment status and year of the study. In order to protect youth from self-incrimination, youth were asked not to share their names or any details about their cases during the forums. However, youth were asked to submit a notecard with their name to the JTDC’s staff in order to receive a certificate for completing the program. We measure participation by linking the names on these notecards with our analysis sample.⁵ Youth were not required to submit a card or to write their real name.⁶ A total of 16 participants who submitted a notecard could not be matched. We do not have data on how many youth did not submit a notecard. Therefore, our participation measure is a lower bound on actual participation. Participation information is available for six of the seven lotteries. Participation data is not available for the first forum series.

In the first year of the study, the lottery was conducted by the research team about two weeks before the scheduled start of the forums in order to give the JTDC’s staff time to prepare for the program. Unfortunately, few youth living in the pod on the day of the lottery were still in that pod when the program began, either because they were released or moved pods. As a result, treatment group youth were only 6 percentage points (pp) more likely to participate in the program than control group youth in the first year. After controlling for individual covariates and randomization block fixed effects (see Sect. 3.2 for details), the first-stage impact of treatment status on participation shown in Panel B is $-$0.02 with a standard error (SE) of 0.08 and p-value on the null hypothesis that the impact is zero of 0.81. Therefore, treatment assignment is not a relevant predictor of program participation in the first year of the study.

In the second year, after observing the low participation rates and substantial control cross-over in the first year, we ran the lottery just a few days before the program was scheduled to begin.⁷ This change solved the problem: In the second year of the study, treatment group youth were 35 pp more likely to have participated in the program than control group youth (38% vs 3%). The first-stage coefficient in the second year is 0.35 (SE$=0.09$, $p<0.01$).

Because random assignment is not a relevant predictor of program participation in the first year, our preferred analysis focuses on the second year of the program.⁸

Data

Our sample is derived from Juvenile Court Data on the housing history of residents of the JTDC. This data covers all youth who were admitted to the JTDC between March 1st, 2015 and April 30th, 2017. Each record includes the youth’s name, date of birth, and admission and release dates to the JTDC and to each pod. A youth is included in our analysis sample if he or she was assigned to a pod that was included in a lottery on the date the lottery was run. We also use these juvenile court data to measure details about youth’s pre- and post-randomization spells in the JTDC.

We measure arrests using Illinois State Police (ISP) arrest records provided by the Illinois Criminal Justice Information Authority (ICJIA) which combine police records from departments across the state. Youth are matched to these records by ICJIA using their names and dates of birth. The data cover both juvenile and adult arrests from 2001 through September 2017, a little over a year after the final lottery was run. Each record includes information about the arrested individual as well as the date of the incident and a description of the charge. We use the charge description to classify whether each arrest is for a violent, property, drug, or other crime. 98 percent of youth are matched to a pre-randomization arrest. In principle all youth should have a prior arrest, but some arrests may have been missed because of substantial typos in names or birth dates.

We measure schooling histories and outcomes using student-level records from Chicago Public Schools. These data include information on youth’s enrollment status, attendance, and grades. We match residents to these records using fuzzy matching techniques on names and dates of birth.⁹ 98 percent of residents in our sample are matched to a Chicago Public Schools record.

To improve our statistical power, we also report treatment effects on a “standardized index” of detention, arrest, and schooling outcomes. To generate this index, we first standardize each outcome using the control group mean and standard deviation. Then we average the standardized outcomes.

Estimation and Inference

Our sample includes one observation for each unique youth-lottery combination for a total of 626 observations on 609 youth. Of these 609 youth, 594 were included in one lottery, 13 were included in two lotteries, and 2 were included in 3 lotteries.

Our analysis is based on the following individual-level panel regression:

$$\begin{aligned} Y_{ipst} = \gamma T_{pt} + X_{i,t-1} \beta + \alpha _{st}+\varepsilon _{ipst}, \end{aligned}$$

(1)

where i indexes youth, p indexes pods, s indicates whether the pod housed juvenile male, AT, or female residents, and t tracks the date of the lottery. $Y_{ipst}$ is an outcome for resident i living in pod p of type s that was included in the lottery at time t. $T_{pt}$ is an indicator for whether or not pod p was assigned to the treatment group on date t. This indicator equals zero if a youth lived in a pod which participated in the lottery but was not selected for the treatment group.

Randomization blocks are defined by whether the pod housed juvenile males, AT males, or female residents and the date of the lottery. $\alpha _{st}$ is a randomization block fixed effect. $X_{i,t-1}$ is a vector of pre-randomization covariates, including controls for age (in years) on the day of the lottery, spells in the JTDC since March 1st, 2015, length of the current spell in the JTDC on the day of the lottery, number of arrests for violent, property, drug, and other crimes in the year before the lottery, indicators for having zero violent, property, drug, or other arrests, days attended in the school year before the lottery at non-prison schools, GPA in the semester before the lottery, the number of prior lotteries the youth was included in, and the number of prior forum series the youth attended. Separate indicator variables for missing any of these control variables are also included.¹⁰ These baseline controls are not necessary for identification because $T_{pt}$ is random conditional on randomization block. However, their inclusion improves our statistical power.

The coefficient $\gamma$ is interpretable as the intent-to-treat (ITT) effect of the program as it is the impact of being randomly assigned to the treatment group, regardless of whether or not the resident actually participates in the program. This is a policy-relevant parameter as it is informative about the average impact of the program given the actual participation rates associated with the program.¹¹

Due to the implementation challenges in the first year of the study, our preferred estimates are from the second year of the program. These estimates come from the following adapted version of Eq. (1):

$$\begin{aligned} Y_{ipst} = \gamma _{1} T_{pt} \times (t\le 4) + \gamma _2 T_{pt} \times (t \ge 5) + X_{i,t-1} \beta + \alpha _{st}+\varepsilon _{ipst}. \end{aligned}$$

(2)

$\gamma _1$ and $\gamma _2$ are the intent-to-treat effects in the first and second years of the study, respectively. $\gamma _2$ is our preferred estimate. We use a pooled regression that includes the first year of the study, instead of estimating Eq. (1) separately by study year, to more precisely estimate the coefficients on the control variables.

Standard errors are clustered by both individual and pod-lottery cluster. We implement this two-way clustering using the method of Cameron and Miller (2015).

Descriptive Statistics

Table 3 shows descriptive statistics of the control group and estimates of treatment-control differences separately for the first and second year of the study. We show results separately by year, as we will focus on the second year of the study in the remainder of the analysis because of the compliance issues in the first year.

An average resident in the control group in the first year of the study was 15.8 years old with 1.4 spells in the JTDC at baseline, including their current spell at the time of the lottery, who had been incarcerated in the JTDC for about 25 days. In the second year of the study, an average resident in the control group was 16.2 years old, had 2.8 spells in the JTDC, and had been in the JTDC for 28 days. Not surprisingly, nearly all youth have a pre-randomization arrest in the year before the study (99 percent in year one, 97 percent in year two).¹² On average, control group youth in the first year of the study had 1.74 arrests for violent crime, 1.15 arrests for property crime, 1.36 arrests for drug crime, and 4.93 arrests for other crime in the year before the study. In the second year, control youth had 2.29, 1.42, 1.29, and 5.19 arrests for violent, property, drug, and other crimes.¹³

Nearly all control group youth (99 percent in year one, 97 percent in year two) are matched to a Chicago Public Schools record, but these youth are quite disconnected from school on average. About 75 percent of youth in the first year of the study and 69 percent of youth in the second year of the study attended at least one day of school at a non-prison school in the school year before the program, but they only attended 61 and 44 days on average in the first and second year of the study. Among the group that attended at least one day, youth attended about 81 and 64 days on average in the first and second year of the study, respectively. Given that the school year is 178 days long, this indicates that even youth who attend some school still miss more than half of the school year. Among the subsample of youth with nonmissing grades, the average GPA in the semester before the lottery was a 1.4 in the first year of the study and a 1.5 in the second year of the study, which are between a C and a D average.

Among the 27 treatment-control comparisons in Table 3 one is statistically significant at the 5 percent level and two are significant at the 10 percent level. This is almost exactly what we would expect from chance variation. We test whether the covariates are jointly insignificant by regressing treatment on our full set of baseline covariates, controlling for block fixed effects, and running an F-test of the null hypothesis that the covariates are jointly insignificant. The p-value of the joint test is 0.14 in the first year and 0.43 in the second year. Therefore, as expected because treatment was randomly assigned, there is little evidence of observable differences between the treatment and control group in either year of the study.

Results

This section describes the program’s impact on juvenile incarceration in the JTDC, arrests, and schooling. Table 4 shows impacts for the second year of the experiment. The results in Table 4 are our preferred estimates because, as discussed in Sect. 3, random assignment was not a relevant predictor of program participation in the first year of the experiment. As a result, we do not expect the program to have any impact, positive or negative, in the first year of the experiment. We include the pooled results and first-year results in Appendix Tables 5 and 6, respectively, for completeness and transparency.

Impact on Detention Outcomes

We begin by looking at the impact of being randomly invited to participate in the program on youth’s post-randomization juvenile detention outcomes.

Although we are most interested in youth’s behavior after being released from the JTDC, we first explore whether the program affected the time until youth are released from the JTDC. Because youth cannot be arrested and cannot enroll at non-prison schools while incarcerated, any impact on time until release could affect our interpretation of the program’s impact on other outcomes. Figure 2 shows the Kaplan-Meier survival function for the time until release by treatment status, or the proportion of youth in each group who had not been released after D days. The treatment and control group curves look very similar, which suggests that the program did not impact time until release.

The first two rows of Table 4, Panel A provide additional evidence that there is no impact on the time to release. The first row shows that the youth in the treatment group were 3 pp less likely to have been released by the end of our court records data. Although this is a 30 percent reduction relative to the control group’s mean of 10 pp, it is not statistically significant at conventional levels. The 95 percent confidence interval includes a reduction as large as 9 pp up to an increase of 3 pp. The second row shows the impact on days until release among the sub-sample which was released during our sample period. Treatment group youth were released 0.02 days later than the youth in the control group. This effect is also statistically insignificant with a 95 percent confidence interval ranging from a 16-day decrease up to a 16-day increase. The control group’s average days to release is 34.69.

The third and forth rows of Panel A show the impacts on having any new spells in the JTDC in the first eight post-randomization months¹⁴ and on the number of new spells during this period. Both point estimates are negative. The impact on having any new admissions to the JTDC in the first eight post-randomization months is not statistically significant, but suggests treatment caused a 6 pp reduction in the probability of having any new spells. This is a 10 percent reduction relative to the control group mean of 0.58. The confidence interval ranges from a 33 percent reduction to a 12 percent increase. The estimated impact on the number of post-randomization spells, $-$0.21, indicates the program caused a 19 percent reduction in the number of new spells. This effect is more precisely estimated than the other estimated impacts on juvenile incarceration ($p=0.11$). The 95 percent confidence interval includes up to a 0.46 decline to a 0.04 increase in the number of new spells. Therefore, we can rule out that the program caused more than a 4 percent increase in the number of new spells, but it potentially caused as much as a 43 percent reduction in the number of new spells. Our point estimate is somewhat larger, but of a similar magnitude, to comparable intent-to-treat estimates on having any readmissions to the JTDC and on the number of readmissions to the JTDC after 8 months in Heller et al. (2017).¹⁵

The final row of Panel A reports the ITT effect on a standardized index of detention outcomes. This index averages standardized versions of the previous four outcomes. Being randomly assigned to treatment causes a 0.11 standard deviation (SD) reduction in the detention index. Because greater values for all of the outcomes indicate more involvement with the justice system, a reduction is a beneficial outcome. The impact on this index is also statistically insignificant with the 95 percent confidence interval around the estimate ranging from a decline as large as 0.28 SDs up to an increase of 0.07 SDs.

Impact on Arrests

Readmission to the JTDC is an imperfect measure of youth’s ongoing criminal behavior and interaction with the justice system. Youth could be arrested and released or if they have turned 18, they could be admitted to the adult jail. To better capture the extent of youth’s ongoing criminal activity and involvement with the justice system Table 4, Panel B shows the impact of the program on total, violent, property, drug, and other arrests in the first year after the lottery. We look separately at these types of crime because they have different social costs and causes. It is not uncommon for programs to have different impacts on different types of crime (Davis and Heller 2020). All outcomes are multiplied by 100 and are therefore interpretable as being the impact per 100 treatment youth. We also report impacts on the social cost of crime in Table 4, Panel C.

We estimate that the program caused a reduction of 36 arrests per 100 youth in the treatment group in the year after the program. This is an 18 percent reduction relative to the control group mean of 206 arrests per 100 youth. While large, this estimate is not significantly different from zero at conventional levels ($p=0.20$). The 95 percent confidence interval around this point estimate ranges from a decline of 92 arrests per 100 youth to an increase of 19 arrests per 100 youth. So, consistent with the impact on future spells in the JTDC, we can rule out that the program caused more than a 9 percent increase in total arrests, but it potentially caused as much as a 44 percent reduction in total arrests.¹⁶

Figure 3 shows the dynamics of how the cumulative impact on total arrests in Table 4 develops in the first year after random assignment. The cumulative impact on total arrests is negative in each month and becomes more negative until 10 months after random assignment when it peaks at a reduction of nearly 50 arrests per 100 youth. Although the cumulative impact on total arrests is never statistically significant at the 5 percent level, the confidence intervals include large reductions and reject anything but small increases in total arrests.

Looking separately across types of crime, all of the point estimates in Panel B are negative. This uniformity in the sign of the effects is itself suggestive that the program reduces crime. Under the null hypothesis that the treatment effects are all zero with the noise distribution symmetric around zero, the probability of finding negative estimates for each type of crime is 0.06 ($=(1/2)^{4}$). With the exception of other crime, the effects are also quite large.

We estimate that the program caused a reduction of 15 arrests for violent crime per 100 youth in the year after the program. This is a 43 percent reduction relative to the control group mean of 34 violent crime arrests per 100 youth. This estimate is statistically significant at the 10 percent level ($p=0.07$). The confidence interval around the point estimate ranges from a decline of 30.4 to an increase of 1.2 violent crime arrests per 100 youth. As with total arrests, the confidence interval includes proportionally large reductions in violent crime (of up to 87 percent) and rejects all but a small increase (up to 3 percent).

We find similarly large proportional reductions in arrests for property and drug crime. Our point estimates imply reductions of 30 and 40 percent, respectively. Specifically, the estimates suggest that the program caused 6.2 fewer property arrests and 7.4 fewer drug arrests per 100 youth. The drug crime estimate is significant at the 10 percent level ($p=0.06$). The 95 percent confidence interval ranges from a decline of 15 drugs arrests to an increase of 0.2 drug arrests per 100 youth. Therefore, we can reject anything but a tiny adverse impact on drug arrests. The property crime estimate, however, is quite noisy with a 95 percent confidence interval ranging from a reduction of 19 arrests to an increase of 7 arrests per 100 youth. While a small increase in arrests for property and drug crime is not ruled out, the confidence intervals indicate that there may also be a large reduction.

The estimated impact on other crime is proportionally small, negative, and statistically insignificant. We estimate the program caused a reduction of 8 arrests per 100 youth in the treatment group in the year after the program. This is a 6 percent reduction relative to the control group mean of 133 arrests per 100 youth. The estimate is also noisy, with a 95 percent confidence interval ranging from a reduction of 49 arrests to an increase of 33 arrests per 100 youth.

Figure 4 shows the dynamics of how the cumulative impacts on arrests for violent, property, drug, and other crime shown in Panel B of Table 4 develop over the first 12 months after random assignment. The estimated impact on violent arrests begins negative and becomes more negative over the year. The estimate is statistically significant at the 5 percent level in each of the first 5 months and 9 months after random assignment. In all months, we can reject anything but a small increase in violent arrests. The impact on property arrests is positive until about 6 months after random assignment when it becomes negative. The impact on drug arrests starts negative and becomes more negative in the first 4 months after random assignment when it begins to level out or grow more negative at a slower rate. The drug impact is statistically significant at the 5 percent level in all but the 7th and 12th month after random assignment. Finally, the impact on other arrests starts at zero and becomes more negative until 5 months after randomization when it begins to gradually shrink towards zero.

The final row of Panel B shows the impact on our standardized index of arrest outcomes. This index averages standardized arrests for violent, property, drug, and other crimes. We find a marginally statistically significant 0.13 SD reduction in arrest outcomes ($p=0.07$). The 95 percent confidence interval includes reductions as large as 0.27 SDs up to basically no effect, 0.01 SDs.

This standardized index puts equal weight on violent, property, drug, and other crimes. However, the social costs of these crimes are quite different. An alternative to this approach is to look at the program’s impact on the social costs of crime directly. The social cost of crime outcome can be thought of as a weighted index of total arrests with greater weights given to more socially costly crimes. Panel C of Table 4 shows these results.

Assigning social costs to crime is an inherently noisy endeavor that relies on strong assumptions. We show the results for the same social cost of crime outcomes as are shown in the appendix of Davis and Heller (2020).¹⁷ The first and second rows of Panel C use estimates of the direct costs of crime from Miller et al. (1996) as reported in Cohen and Piquero (2009). These are based on direct victim costs and direct costs to the justice system. The third and fourth rows of Panel C show results using contingent valuation based estimates of people’s willingness to pay to avoid crime from Cohen et al. (2004) which are also reported in Cohen and Piquero (2009). To account for the fact that we observe arrests, not all crimes committed, the first and third rows inflate the number of arrests by the estimated ratio of crimes to arrests reported in Cohen and Piquero (2009). The second and fourth rows show estimates using only observed arrests.

Our estimates in Panel C are all large and negative, indicating reductions in social costs ranging from $9911 to $360,676 per treated youth. The estimates indicate about a 40 percent reduction in the social cost of crime using the direct estimates and just over a 60 percent decline using the willingness-to-pay estimates. The willingness-to-pay estimates are both statistically significant at the 5 percent level. The least beneficial impacts in the 95 percent confidence intervals around these estimates suggest a roughly 20 percent reduction in the social costs of crime. The direct cost of crime estimate is statistically significant at the 10 percent level when using observed arrests, but not when using all crimes.

Scaling the point estimates by the 78 youth who were in the treatment group in the second year of the study suggests that the program has a potentially high rate of return. Even the smallest point estimate, $9911, suggests a total societal savings of over $750,000 from just the second year of the forums. This would be a 50 percent return on the initial $500,000 grant to support PSN work in the Northern District of Illinois. Of course, the confidence interval around the total societal savings using the direct estimates includes no benefit and even some smaller detrimental effects. The magnitude of these benefits is plausible because crime is incredibly socially costly. A single armed robbery is estimated to cost the criminal justice system about $16,000 and the victim about $32,000. Cohen and Piquero (2009) report estimates suggesting that youth are arrested for about 1 out of every 2.8 armed robberies they commit. Therefore, a reduction of a single arrest for armed robbery is estimated to create societal savings of over $130,000 using the more conservative direct estimates.

Impact on Schooling

Table 4, Panel D shows the program’s impact on post-randomization schooling outcomes among the 96 percent of youth who are linked to a Chicago Public School record in the second year of the experiment. The first and second rows of Panel D show the impact on attending any days at a non-prison public school in Chicago and on the number of days attended during the school year following random assignment, respectively.¹⁸ We focus on non-prison schools because youth are required to attend Nancy B. Jefferson when they are incarcerated at the JTDC.

Youth in the treatment group are 7 pp more likely to have attended any days of school. This a 15 percent increase relative to the control group mean of 51 percent but is not significantly different from zero. The 95 percent confidence interval ranges from a decline of 3 pp to an increase of 18 pp.

We also find that the program did not significantly increase the number of days attended. The point estimate is relatively small in absolute terms, an increase of 5.72 days of school attended or just over one extra week of school. But this is a 26 percent increase relative to the control group’s very low average level of attendance of 21.60 days attended. The 95 percent confidence interval around the point estimate ranges from a decline of 2.14 days attended to an increase of 13.58 days attended. Mirroring the crime results, the confidence interval includes proportionally large increases in attendance of up to 63 percent and rejects all but relatively small decreases of up to 10 percent.

The third row of Panel D shows the program caused a 13.6 pp increase in the probability that youth have a GPA in the semester following random assignment. Having a GPA is a measure of school attachment because youth with very low attendance are disenrolled from school and not graded. This is a statistically significant 35 percent increase relative to the control group mean of 39 percent. The 95 percent confidence interval around the point estimate ranges from an increase of 6 pp up to 22 pp.

The fact that we observe GPAs more often in the treatment group than the control group complicates our estimates of the impact of treatment on GPAs. In particular, we may expect that the program pushed more marginal students into school who otherwise would not have attended enough to be graded at all. This would bias our estimate of the impact on GPAs downward. Even so, we find a proportionally large, but statistically insignificant, 0.3 grade point increase in GPAs. This is a 27 percent increase relative to the control group’s mean of 1.1. In grade terms, treatment shifts students from a D average to a D+ average. The confidence interval ranges from a decline of $-$0.07 grade points up to an increase of 0.66 grade points. The lower bound of this grade point is not large enough to correspond to a change in students’ average grades. The upper bound, in contrast, corresponds to an increase from a D to a C- average.

The final row of Panel D shows the impact of treatment on a standardized schooling outcome. This outcome includes the average standardized value of each observed schooling outcome. In particular, GPA is included when it is observed but is otherwise excluded. This assumes GPAs are missing at random so that students who are missing a GPA would have performed about as well as students with observed GPAs, on average. This should be conservative given that we find evidence that treatment reduces missingness and improves performance. The estimate suggests treatment improves schooling outcomes by 0.21 SDs. This effect is statistically significant at the 5 percent level. The 95 percent confidence interval includes improvements of between 0.08 and 0.33 SDs. This demonstrates that, when taken together, these schooling results provide relatively strong evidence that the youth outreach forums improve attachment to schooling.

Discussion

This paper presents the results of a randomized controlled trial designed to credibly measure the impact of a series of “youth outreach forums.” These forums give youth the opportunity to talk to members of their community, authority figures in their life, and individuals who have left a life crime in a neutral, constructive setting.

Our estimates suggest that being randomly assigned to the treatment group offered the youth outreach forums caused an 18 percent reduction in total arrests driven by 43 percent, 30 percent, and 40 percent reductions in violent, property, and drug crime, respectively. Although the violent crime and drug crime impacts are both statistically significant at the 10 percent level, the confidence intervals around all of these estimates reject all but small increases in arrests and include even larger proportional reductions. We find a marginally significant reduction in a standardized index of arrest outcomes that weights different crimes equally. Looking at the social costs of crime, which can be thought of as another arrest index that gives greater weight to more socially costly crimes, we find proportionally large and very socially valuable reductions in the social costs of crime. Three of our four social cost estimates are significant at the 10 percent level and two are significant at the 5 percent level. Our estimates also suggest that the forums may increase attachment to school.

The forums were motivated by the theory that giving youth the opportunity to discuss the rationale for the law with authority figures in their lives will increase their perception of its legitimacy (Tyler 2006a; Bottoms and Tankebe 2012). In practice, however, the forums can be viewed as part of a more general focused deterrence strategy. The program “pulls many levers” that could drive our results (Braga and Kennedy 2021). One lever is improving youth’s perceptions of legitimacy. Unfortunately, we were not allowed to survey the youth in our study about their attitudes and beliefs so we cannot directly test this. Nevertheless, several components of the program could operate through this channel. Meeting with police command staff provides youth with the opportunity to ask questions about the rationale behind the law and policing and also the rules that police are expected to follow themselves. Discussions with community members, parents, and teachers also provide insight to the logic behind laws and rules and promote the sense that these laws and rules are based on a shared set of values.

Another lever is deterrence (Becker 1968; Nagin 2013). Deterrence theory suggests individuals will commit a crime when the perceived benefits outweigh the perceived costs. Meeting with former offenders who spent decades of their lives in prison or who were paralyzed because of gun violence could both increase the perceived costs of committing crime. Meeting with command staff from the Chicago Police Department also highlights that the youth are under scrutiny which may suggest that they are more likely to be caught if they do commit a crime.

Yet another lever is connecting youth to social supports and building their sense of community belonging. Youth were connected with a school reengagement specialist working at the JTDC whose job was to help them transition back to the school that best meets their needs. But they also were introduced to other social programs operating in Chicago in the “Community and a Shared Moral Voice” forum. Being offered help shows youth that their community would like them to succeed and potentially addresses root causes of criminal behavior, like needing to make extra money to help pay bills. Moreover, almost all of the forums created opportunities for youth to meet positive adult mentors.

Finally, the program could also build skills that help the youth stay out of trouble and succeed in school. The “Hope Outside the Law” forum was replaced by a forum that explicitly taught the youth conflict resolution tools based on cognitive behavioral therapy principles. Indeed, credible experimental evidence has shown that teaching these types of skills can reduce crime (Landenberger and Lipsey 2005; Heller et al. 2017).

The challenges that arose in this project are relevant for future researchers who hope to conduct similar experiments in juvenile detention centers. The first year of the study was uninformative about treatment effects because of short spells in the JTDC. By working with the JTDC’s administration to run the randomization closer to the start of the program, we were able to correct this issue in the second year.

Our estimates are somewhat imprecise because of the issues that arose in the first year. Given the imprecision, we take care to emphasize effect sizes and confidence intervals in addition to p-values. Recent debates in the statistics literature on the value of hypothesis testing have emphasized that a focus on effect sizes and confidence intervals may better address whether a hypothesis is correct than the traditional focus on p-values (Wasserstein and Lazar 2016; Benjamin et al. 2018). Therefore, this may be a useful template for other criminology research.

Table 1 Treatment assignment details

Table 2 Treatment status and participation by year

Table 3 Baseline balance

Table 4 Intent-to-treat effect estimates

Fig. 1

Daily Population in the JTDC. This figure shows the daily population of youth admitted between January 1, 2015 and April 30, 2017. The vertical bars indicate the dates of each lottery

Fig. 2

Empirical survival curve for time until release. This figure shows the proportion of youth in the treatment and control group who had not been released after X days

Fig. 3

Dynamic impacts of intervention on total arrests. This figure shows the intent-to-treat estimates on number of total crime arrests over the first 12 months after random assignment for the second year of the study. 95% confidence interval based on standard errors clustered by pod and youth

Fig. 4

Dynamic impacts of intervention on arrests by type of crime. This figure shows the intent-to-treat estimates on crime outcomes over the first 12 months after random assignment for the second year of the study. 95% confidence interval based on standard errors clustered by pod and youth

Appendix

Table 5 Intent-to-treat effect estimates, pooling both years

Table 6 Intent-to-treat effect estimates, first year only