Incremental Propensity Score Effects for Criminology: An Application Assessing the Relationship Between Homelessness, Behavioral Health Problems, and Recidivism

Abstract

Objectives

This study examines the relationship between homelessness and recidivism among people on probation with and without behavioral health problems. The study also illustrates a new way to summarize the effect of an exposure on an outcome, the Incremental Propensity Score (IPS) effect, which avoids pitfalls of other approaches commonly used in criminology.

Methods

We assessed the impact of homelessness at probation start on rearrest within one year among a cohort of people on probation (n = 2453). We estimated IPS effects, considering general and crime-specific recidivism if subjects were more or less likely to be unhoused, and assessed effect variation by behavioral health problem status. We used a doubly robust machine learning estimator to flexibly but efficiently estimate effects.

Results

A substantial intervention—reducing homelessness by roughly 65%—corresponded to a 9% reduction in the estimated average rate of recidivism (p < 0.05). Milder interventions showed smaller, non-significant effect sizes. Stratifying by behavioral health problem and rearrest type led to similar results without statistical significance.

Conclusions

Minding limitations related to observational data and generalizability, this study suggests large reductions in homelessness lead to significant reductions in rearrest rates. Efforts to reduce recidivism should include interventions that make homelessness less likely, but notable differences in recidivism will require these interventions be sizable. Meanwhile, efforts to establish recidivism risk factors should consider alternative effects, like IPS effects, to maximize validity and reduce bias.

Appendix 1: Balance Table

Variable	Type	Unadjusted standardized mean difference	Adjusted mean housed	Adjusted mean homeless	Adjusted standardized mean difference
Age	Cont	− 0.2915	36.1	36.4	− 0.0195
Sex	Bin	0.0389	0.1519	0.1609	− 0.0090
Race/ethnicity
Black	Bin	0.0234	0.4945	0.4815	0.0130
Latino	Bin	0.0296	0.1439	0.1518	− 0.0079
Another race/ethnicity	Bin	0.0388	0.1165	0.1271	− 0.0106
White	Bin	− 0.0919	0.2452	0.2397	0.0055
Recidivism Risk	Cont	− 0.5371	− 0.0057	0.0662	− 0.0775
Financial Insecurity	Cont	− 0.6336	− 0.0018	0.0515	− 0.0567
Diagnosis
Co-occurring	Bin	− 0.0599	0.0393	0.0439	− 0.0046
No Diagnosis	Bin	0.1508	0.8114	0.8019	0.0095
SMI Only	Bin	− 0.0290	0.0639	0.0643	− 0.0004
SUD Only	Bin	− 0.0609	0.0854	0.0898	− 0.0045
Effective sample size
Homeless		648			383.3
Housed		1805			1661.7

Standardized mean differences are housed minus homeless, divided by pooled standard deviation across both groups; SMI = Serious mental illness; SUD = Substance use disorder; Cont. = Continuous; Bin. = Binary.

The table shows traditional balance diagnostics. The first column shows the variable in the data while the second column shows the type of the variable (“cont.” equals continuous and “bin.” equals binary). The third column shows the unadjusted mean differences between the housed and homeless groups. For binary variables, this is the raw difference in proportions, while for continuous variables it is the standardized mean difference, where the standardized mean difference equals the mean among housed individuals minus the mean for homeless individuals, divided by the pooled standard deviation across both groups. The fourth and fifth columns show the adjusted means among housed and homeless individuals, respectively. Here, the data is re-weighted according to the propensity score estimates. Specifically, the weights for individual $i$, which we denote by $W_{i}$, are:

$$W_{i} = \frac{{A_{i} }}{{\hat{\pi }\left( {X_{i} } \right)}} + \frac{{1 - A_{i} }}{{1 - \hat{\pi }\left( {X_{i} } \right)}}$$

(6)

where $A_{i} = 1$ if individual $i$ was housed at the started of probation and $0$ otherwise, and $\hat{\pi }\left( {X_{i} } \right)$ are the estimated propensity score values for individual $i$. Finally, the sixth column shows the adjusted standardized mean differences. For binary variables, this is again the raw difference in proportions; i.e., the adjusted mean for housed individuals minus the adjusted mean for homeless individuals. For continuous variables, this is the standardized mean difference. The final two rows of the table show the true and weighted sizes for the housed and homeless groups.

The table demonstrates that balance is improved by weighting according to the estimated propensity scores, indicating that the SuperLearner estimator for the propensity score is appropriately capturing confounding in the observed data.