Estimating Treatment Effects and Predicting Recidivism for Community Supervision Using Survival Analysis with Instrumental Variables

Abstract

Criminal justice researchers often seek to predict criminal recidivism and to estimate treatment effects for community corrections programs. Although random assignment provides a desirable avenue to estimating treatment effects, often estimation must be based on observational data from operating corrections programs. Using observational data raises the risk of selection bias. In the community corrections contexts, researchers can sometimes use judges as instrumental variables. However, the use of instrumental variable estimation is complicated for nonlinear models, and when studying criminal recidivism, researchers often choose to use survival models, which are nonlinear given right-hand-censoring or competing events. This paper discusses a procedure for estimating survival models with judges as instruments. It discusses strengths and weaknesses of this approach and demonstrates some of the estimation properties with a computer simulation. Although this paper’s focus is narrow, its implications are broad. A conclusion argues that instrumental variable estimation is valuable for a broad range of topics both within and outside of criminal justice.

Appendix

The appendix shows why the two-step estimator does not work in a survival model. The DGP for commonly used proportional hazard survival models (Lancaster 1990, p. 35) can be written in a linear logarithmic form.

$$ \ln \left( Y \right) = X\alpha_{X} + T\delta + e $$

(A1)

Y is the failure time, X are risk factors, and T is a dichotomous variable representing treatment. The distribution of e determines the parametric model. For example, if exp(e) is a type one extreme value distribution, then the failure time has an exponential distribution conditional on X and T. As before assume that the probability of treatment is a linear function of X and Z, so:

$$ T = 1\quad {\text{if}}\;X\beta_{X} + Z\beta_{Z} + u \ge 0\;{\text{and}}\;T = 0\;{\text{otherwise}} $$

(A2)

Both Eqs. A1 and A2 are parts of the DGP. The first equation is the outcome equation. The second is the selection equation.

Suppose that Eqs. A1 and A2 are the DGP, but that the model includes some of the X variables (the X _O variables) and omits other X variables (the X _U variables). The Z are always observed. Placing the omitted variables in the error term, rewrite the same DGP for the outcome equation as:

$$ \ln \left( Y \right) = X_{O} \alpha_{{X_{O} }} + T\delta + e^{*}\quad {\text{where}}\;e^{*} = X_{U} \alpha_{{X_{U} }} + e $$

(A3)

Likewise, the same DGP for the selection equation is written as:

$$ T = 1\quad {\text{if}}\;X_{O} \beta_{{X_{O} }} + Z\beta_{Z} + u^{*} \ge 0\;{\text{and}}\;T = 0\;{\text{otherwise}}\;{\text{where}}\;u^{*} = X_{U} \beta_{{X_{U} }} + u $$

(A4)

This new error term changes the survival distribution (Heckman and Singer 1984), but we ignore that fact in this discussion. Generally estimates of the δ will be biased and inconsistent because T is not orthogonal to e*. The survival model will suffer from selection bias, but can a two-step estimator overcome this problem? In the two-step estimator, the evaluator estimates T based on a model of the DGP for the selection equation. The model will not represent the β consistently, but nevertheless the model will provide a consistent estimate of T conditional on X _O and Z. Replacing the treatment variable T with its estimate leads to another representation of the DGP for the outcome equation as:

$$ \ln \left( Y \right) = X_{O} \alpha_{{X_{O} }} + \hat{T}\delta + e^{**} $$

(A5)

The error term is now $e^{**} = \left( {T - \hat{T}} \right)\delta + X_{U} \alpha_{{X_{U} }} + e$. Conditional on X _O , $\hat{T}$ will be orthogonal to e** by construction, but X _O will not be orthogonal to e** if the observed variables are correlated with the unobserved variables.

Holding X _O constant and letting P be the probability of treatment conditional on X _O and Z, and using a least squares regression to estimate the parameters, the expected value of the outcome Y can be written:

$$ \begin{aligned} E\left[ {\ln \left( Y \right)} \right] & =\,& P\left\{ {X_{O} a_{{X_{O} }} + \delta + E\left[ {v|T = 1} \right]} \right\} + \left( {1 - P} \right)\left\{ {X_{O} a_{{X_{O} }} + E\left[ {v|T = 0} \right]} \right\} \\ & =\,& \left\{ {X_{O} a_{{X_{O} }} + P\delta } \right\} \\ \end{aligned} $$

(A6)

$E\left[ {v|T = 1} \right]$ is the expected value of the residual (not the error term) conditional on treatment. The expected value is not zero because of the presence of the omitted variable X _U . Likewise $E\left[ {v|T = 0} \right]$ is the expected value of the residual conditional on no treatment. If there is a constant in the model, so that $E\left[ v \right] = 0$, $PE\left[ {v|T = 1} \right] + \left( {1 - P} \right)E\left[ {v|T = 0} \right] = 0$. The second line shows that the expectation conditional on X _O and the probability of treatment admission equals the two-step estimator. The estimator works provided there is no censoring.

Now introduce censoring at time C = ln(censoring time). Let $\varphi \left( v \right)$ represent the density for v; let $\varphi \left( {v|T = 1} \right)$ represent the density of v conditional on treatment and let $\varphi \left( {v|T = 0} \right)$ represent the density of v conditional on no treatment. The probability of surviving to time C implies that:

$$ v \ge C - X_{O} a_{{X_{O} }} - \hat{P}\delta $$

(A7)

The probability of survival as of time C can be written as:

$$ \begin{aligned} {\text{PROB}}_{\text{survival}} \left( {C|X_{O} ,\hat{P}} \right) & = & P\left\{ {\int\limits_{{C - X_{O} a_{{X_{O} }} - \delta }}^{\infty } {\varphi (v|T = 1)} } \right\} + (1 - P)\left\{ {\int\limits_{{C - X_{O} a_{{X_{O} }} }}^{\infty } {\varphi (v|T = 0)} } \right\} \\ & \ne & \int\limits_{{C - X_{O} a_{{X_{O} }} - P\delta }}^{\infty } {\varphi (v)} \\ \end{aligned} $$

(A8)

The inequality holds because of Jenkin’s inequality. This shows why the two-step estimator will not work: It cannot account for nonlinearities due to censoring. One might solve this problem by making assumptions about the conditional distribution of v. This leads to what are known as Heckman-type adjustment models, but obviously making assumptions about the distribution of v is uncertain and getting the assumptions wrong will lead to biased and inconsistent parameter estimates.

Possibly the density function is approximately linear over the range of interest to the estimation, namely: $\left( {C - X_{O} a_{{X_{O} }} } \right) - \left( {C - X_{O} a_{{X_{O} }} - \delta } \right)$. If so, then the two-step estimator would likely improve on the approach that relies on selection on the observables. Furthermore, the approximation is especially plausible over ranges where the density in increasing or decreasing monotonically. This will always be true of an exponential survival model and it will be true over a range for other survival models.