Skip to main content

Opting out of treatment: Self-selection bias in a randomized controlled study of a focused deterrence notification meeting



This study investigated the role of self-selection in an evaluation of the impact of a focused deterrence notification meeting on subsequent arrests.


We conducted a randomized controlled study that randomly assigned probationers and parolees to a treatment group asked to attend a focused deterrence notification meeting and a control group that was not asked to attend the meeting. A sizable proportion of the treatment group did not attend the meeting. We estimated intent-to-treat, average treatment, and local average treatment models to evaluate the effect of attending the notification meeting on future arrests and the effect of self-selection on the results.


Subjects who attended the notification meeting were less likely than those who did not receive treatment to be arrested over the following 17 months. The results were not significantly affected by selection effects.


Future evaluations of focused deterrence and related criminal justice interventions should be based on randomized controlled research designs that address selection effects on the outcome.

This is a preview of subscription content, access via your institution.


  1. 1.

    Additional missing data further reduced the sample size in some analyses, as noted in the presentation of results.

  2. 2.

    All model and test results were estimated in STATA 13.1.

  3. 3.

    All results not shown are available from the authors on request.

  4. 4.

    We acknowledge concerns regarding the application of OLS and related linear models to dichotomous outcomes. Angrist (2006) defends the use of such models, however, when dichotomous indicators appear on both sides of the equation.

  5. 5.

    We re-estimated the models without covariates on the 103 cases used for the models including the covariates. The results are nearly identical to those presented in Table 4.

  6. 6.

    We conducted a similar comparison of subjects who attended the notification meeting (the compliers in the treatment group) with those who did not attend the meeting (the noncompliers in the treatment group + the control group). The results were nearly identical to those reported.


  1. Angrist, J. D. (2006). Instrumental variables methods in experimental criminological research: What, why and how. Journal of Experimental Criminology, 2(1), 23–44.

    Article  Google Scholar 

  2. Angrist, J. D., & Pischke, J.-S. (2008). Mostly harmless econometrics: An Empiricist’s companion. Princeton: Princeton University Press.

    Google Scholar 

  3. Becker, G. S. (1968). Crime and punishment: An economic approach. In The economic dimensions of crime (pp. 13–68). New York: Palgrave Macmillan.

  4. Braga, A. A., & Weisburd, D. L. (2011). The effects of focused deterrence strategies on crime: A systematic review and meta-analysis of the empirical evidence. Journal of Research in Crime and Delinquency, 49(3), 323–358.

    Article  Google Scholar 

  5. Braga, A. A., & Weisburd, D. L. (2012a). The effects of “pulling levers” focused deterrence strategies on crime. Campbell Systematic Reviews, 8(6).

  6. Braga, A. A., & Weisburd, D. L. (2012b). The effects of focused deterrence strategies on crime: A systematic review and meta-analysis of the empirical evidence. Journal of Research in Crime and Delinquency, 49, 323–358.

    Article  Google Scholar 

  7. Braga, A. A., & Weisburd, D. L. (2014). Must we settle for less rigorous evaluations in large area-based crime prevention programs? Lessons from a Campbell review of focused deterrence. Journal of Experimental Criminology, 10(4), 573–597.

    Article  Google Scholar 

  8. Braga, A. A., Apel, R., & Welsh, B. C. (2013). The spillover effects of focused deterrence on gang violence. Evaluation Review, 37(3–4), 314–342.

    Article  Google Scholar 

  9. Braga, A. A., Hureau, D. M., & Papachristos, A. V. (2014). Deterring gang-involved gun violence: Measuring the impact of Boston’s Operation Ceasefire on street gang behavior. Journal of Quantitative Criminology, 30(1), 113–139.

    Article  Google Scholar 

  10. Corsaro, N., & Brunson, R. K. (2013). Are suppression and deterrence mechanisms enough? Examining the “pulling levers” drug market intervention strategy in Peoria, Illinois, USA. International Journal of Drug Policy, 24(2), 115–121.

    Article  Google Scholar 

  11. Corsaro, N., & Engel, R. S. (2015). Most challenging of contexts: Assessing the impact of focused deterrence on serious violence in New Orleans. Criminology & Public Policy, 14(3), 471–505.

    Article  Google Scholar 

  12. Engel, R. S., Tillyer, M. S., & Corsaro, N. (2013). Reducing gang violence using focused deterrence: Evaluating the Cincinnati Initiative to Reduce Violence (CIRV). Justice Quarterly, 30(3), 403–439.

    Article  Google Scholar 

  13. Falk, Ö., Wallinius, M., Lundström, S., Frisell, T., Anckarsäter, H., & Kerekes, N. (2014). The 1% of the population accountable for 63% of all violent crime convictions. Social Psychiatry and Psychiatric Epidemiology, 49, 559–571.

    Article  Google Scholar 

  14. Farabee, D., Zhang, S. X., & Wright, B. (2014). An experimental evaluation of a nationally recognized employment-focused offender reentry program. Journal of Experimental Criminology, 10(3), 309–322.

    Article  Google Scholar 

  15. Hawken, A., & Kleiman, M. (2009). Managing drug involved probationers with swift and certain sanctions: Evaluating Hawaii’s HOPE: Executive summary. Washington: National Criminal Justice Reference Services.

    Google Scholar 

  16. Kennedy, D. M. (2009). Deterrence and crime prevention: Reconsidering the prospect of sanction. New York: Routledge.

    Google Scholar 

  17. Kennedy, D. M., Braga, A. A., & Piehl, A. M. (2001). Reducing gun violence: The Boston Gun Project’s Operation Ceasefire. Washington: National Institute of Justice.

    Google Scholar 

  18. Nagin, D. S. (2013). Deterrence in the twenty-first century. Crime and Justice, 42(1), 199–263.

    Article  Google Scholar 

  19. Nagin, D. S. (2016). Project HOPE: Does it work? Criminology and Public Policy, 15, 1005–1007.

    Article  Google Scholar 

  20. Papachristos, A. V., & Kirk, D. S. (2015). Changing the street dynamic: Evaluating Chicago’s Group Violence Reduction Strategy (VRS). Criminology & Public Policy, 14(3), 525–558.

    Article  Google Scholar 

  21. Papachristos, A. V., Meares, T. L., & Fagan, J. (2007). Attention felons: Evaluating Project Safe Neighborhoods in Chicago. Journal of Empirical Legal Studies, 4(2), 223–272.

    Article  Google Scholar 

  22. Paternoster, R. (2010). How much do we really know about criminal deterrence? The Journal of Criminal Law and Criminology, 100(3), 765–824.

    Google Scholar 

  23. Paternoster, R., Saltzman, L. E., Waldo, G. P., & Chiricos, T. G. (1983). Perceived risk and social control: Do sanctions really deter? Law and Society Review, 17(3), 457–479.

    Article  Google Scholar 

  24. Piquero, A. R. (2000). Assessing the relationships between gender, chronicity, seriousness, and offense skewness in criminal offending. Journal of Criminal Justice, 28, 103–115.

    Article  Google Scholar 

  25. Sampson, R. J. (2010). Gold standard myths: Observations on the experimental turn in quantitative criminology. Journal of Quantitative Criminology, 26(4), 489–500.

    Article  Google Scholar 

  26. Wallace, D., Papachristos, A. V., Meares, T., & Fagan, J. (2016). Desistance and legitimacy: The impact of offender notification meetings on recidivism among high risk offenders. Justice Quarterly, 33(7), 1237–1264.

    Article  Google Scholar 

  27. Williams, K. R., & Hawkins, R. (1986). Perceptual research on general deterrence: A critical review. Law and Society Review, 20(4), 545–572.

    Article  Google Scholar 

  28. Williams, D. J., Currie, D., Linden, W., & Donnelly, P. D. (2014). Addressing gang-related violence in Glasgow: A preliminary pragmatic quasi-experimental evaluation of the Community Initiative to Reduce Violence (CIRV). Aggression and Violent Behavior, 19(6), 686–691.

    Article  Google Scholar 

  29. Wolfgang, M. E., Thornberry, T. P., & Figlio, R. M. (1987). From boy to man, from delinquency to crime. Chicago: University of Chicago Press.

    Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Richard Rosenfeld.



Table 6 Tests for equivalence between treatment group subjects who attended and did not attend a notification meeting

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hamilton, B., Rosenfeld, R. & Levin, A. Opting out of treatment: Self-selection bias in a randomized controlled study of a focused deterrence notification meeting. J Exp Criminol 14, 1–17 (2018).

Download citation


  • Focused deterrence
  • Notification meeting
  • Randomized controlled experiment
  • Selection effects