Journal of Experimental Criminology

, Volume 12, Issue 3, pp 347–371 | Cite as

Predictions put into practice: a quasi-experimental evaluation of Chicago’s predictive policing pilot

  • Jessica SaundersEmail author
  • Priscillia Hunt
  • John S. Hollywood



In 2013, the Chicago Police Department conducted a pilot of a predictive policing program designed to reduce gun violence. The program included development of a Strategic Subjects List (SSL) of people estimated to be at highest risk of gun violence who were then referred to local police commanders for a preventive intervention. The purpose of this study is to identify the impact of the pilot on individual- and city-level gun violence, and to test possible drivers of results.


The SSL consisted of 426 people estimated to be at highest risk of gun violence. We used ARIMA models to estimate impacts on city-level homicide trends, and propensity score matching to estimate the effects of being placed on the list on five measures related to gun violence. A mediation analysis and interviews with police leadership and COMPSTAT meeting observations help understand what is driving results.


Individuals on the SSL are not more or less likely to become a victim of a homicide or shooting than the comparison group, and this is further supported by city-level analysis. The treated group is more likely to be arrested for a shooting.


It is not clear how the predictions should be used in the field. One potential reason why being placed on the list resulted in an increased chance of being arrested for a shooting is that some officers may have used the list as leads to closing shooting cases. The results provide for a discussion about the future of individual-based predictive policing programs.


Predictive policing Program evaluation Propensity score matching Quasi-experimental design Risk assessment Time series analysis 



We would like to thank the Chicago Police Department and Dr. Miles Wernick from the Illinois Institute of Technology for their participation and support of this evaluation. We would also like to acknowledge research assistance provided by Sam Cooper and Alessandra Sienra-Canas. This publication was made possible by Award Number 2009-IJ-CX-K114 - Predictive Policing Analytic & Evaluation Research Support awarded by the National Institute of Justice, Office of Justice Programs. The opinions, findings, conclusions and recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the Department of Justice.


  1. Abrahamse, A. F., Ebener, P. A., Greenwood, P. W., Fitzgerald, N., & Kosin, T. E. (1991). An experimental evaluation of the Phoenix repeat offender program. Justice Quarterly, 8(2), 141–168.CrossRefGoogle Scholar
  2. Aitchison, J., & Dunsmore, I. R. (1980). Statistical prediction analysis. CUP Archive.Google Scholar
  3. Auerhahn, K. (1999). Selective incapacitation and the problem of prediction. Criminology, 37(4), 703–734.CrossRefGoogle Scholar
  4. Bar-Hillel, M. (1980). The base-rate fallacy in probability judgments. Acta Psychologica, 44(3), 211–233.CrossRefGoogle Scholar
  5. Beck, C., & McCue, C. (2009). Predictive policing: what can we learn from Wal-Mart and Amazon about fighting crime in a recession? Police Chief, 76(11), 18.Google Scholar
  6. Becker, G. S. (1993). Nobel lecture: the economic way of looking at behavior. Journal of Political Economy, 101, 385–409.CrossRefGoogle Scholar
  7. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B: Methodological, 57, 289–300.MathSciNetzbMATHGoogle Scholar
  8. Berk, R. (2008). Forecasting methods in crime and justice. Annual Review of Law and Social Science, 4, 219–238.CrossRefGoogle Scholar
  9. Berk, R. (2011). Asymmetric loss functions for forecasting in criminal justice settings. Journal of Quantitative Criminology, 27(1), 107–123.CrossRefGoogle Scholar
  10. Berk, R., & Bleich, J. (2013). Statistical procedures for forecasting criminal behavior. Criminology and Public Policy, 12(3), 513–544.CrossRefGoogle Scholar
  11. Berk, R., Sherman, L., Barnes, G., Kurtz, E., & Ahlman, L. (2009). Forecasting murder within a population of probationers and parolees: a high stakes application of statistical learning. Journal of the Royal Statistical Society: Series A (Statistics in Society), 172(1), 191–211.MathSciNetCrossRefGoogle Scholar
  12. Berry, M. J., & Linoff, G. S. (2004). Data mining techniques: for marketing, sales, and customer relationship management. Indianapolis, IN: John Wiley & Sons.Google Scholar
  13. Blumstein, A. (1986). Criminal Careers and “Career Criminals” (Vol. 2). Washington, DC: National Academies.Google Scholar
  14. Bordua, D. J., & Reiss, A. J., Jr. (1966). Command, control, and charisma: reflections on police bureaucracy. American Journal of Sociology, 72, 68–76.CrossRefGoogle Scholar
  15. Braga, A. (2005). Hot spots policing and crime prevention: a systematic review of randomized controlled trials. Journal of Experimental Criminology, 1(3), 317–342.CrossRefGoogle Scholar
  16. Braga, A., & Weisburd, D. L. (2010). Policing problem places: Crime hot spots and effective prevention. New York, NY: Oxford University Press on Demand.Google Scholar
  17. Braga, A., & Weisburd, D. L. (2012). The effects of focused deterrence strategies on crime: a systematic review and meta-analysis of the empirical evidence. Journal of Research in Crime and Delinquency, 49(3), 323–358.CrossRefGoogle Scholar
  18. Braga, A., Papachristos, A. V., & Hureau, D. M. (2012). The effects of hot spots policing on crime: An updated systematic review and meta-analysis. Justice Quarterly, 1–31.Google Scholar
  19. Bruinsma, G., & Weisburd, D. (2014). Encyclopedia of Criminology and Criminal Justice. New York: Springer.CrossRefGoogle Scholar
  20. Caldwell, M. F., Vitacco, M., & Van Rybroek, G. J. (2006). Are violent delinquents worth treating? A cost–benefit analysis. Journal of Research in Crime and Delinquency, 43(2), 148–168.CrossRefGoogle Scholar
  21. Chicago Police Department. (2014).Gang Violence Reuction Strategy: General Order G10-01. Chicago, IL.Google Scholar
  22. Chinman, M., Imm, P., & Wandersman, A. (2004). Getting To Outcomes™ 2004. Santa Monica: Rand Corporation.Google Scholar
  23. Cohen, J., Gorr, W. L., & Olligschlaeger, A. M. (2007). Leading indicators and spatial interactions: a crime‐forecasting model for proactive police deployment. Geographical Analysis, 39(1), 105–127.CrossRefGoogle Scholar
  24. Cope, N. (2004). ‘Intelligence led policing or policing led intelligence?’Integrating volume crime analysis into policing. British Journal of Criminology, 44(2), 188–203.CrossRefGoogle Scholar
  25. Cornish, D. B., & Clarke, R. V. (2014). The reasoning criminal: Rational choice perspectives on offending. New Brunswick, NJ: Transaction Publishers.Google Scholar
  26. Dvoskin, J. A., & Heilbrun, K. (2001). Risk assessment and release decision-making: Toward resolving the great debate. American Academy of Psychiatry and the Law, 29, 6–10.Google Scholar
  27. Eck, J., Chainey, S., Cameron, J., & Wilson, R. (2005). Mapping crime: Understanding hotspots (Vol. NCJ 209393). Washington, DC: National Institute of Justice.Google Scholar
  28. Erbentraut, J. (2014). Chicago’s controversial new police program prompts fear of racial profiling. The Huffington Post.Google Scholar
  29. Foster, E. M., & Jones, D. (2006). Can a costly intervention be cost-effective?: an analysis of violence prevention. Archives of General Psychiatry, 63(11), 1284–1291.PubMedPubMedCentralCrossRefGoogle Scholar
  30. Funk, M. J., Westreich, D., Wiesen, C., Stürmer, T., Brookhart, M. A., & Davidian, M. (2011). Doubly robust estimation of causal effects. American Journal of Epidemiology, 173(7), 761–767.PubMedPubMedCentralCrossRefGoogle Scholar
  31. Gendreau, P., Little, T., & Goggin, C. (1996). A meta-anallysis of the predictors of adult offender recidivism: what works! Criminology, 34(4), 575–608.CrossRefGoogle Scholar
  32. Gorr, W., & Harries, R. (2003). Introduction to crime forecasting. International Journal of Forecasting, 19(4), 551–555.CrossRefGoogle Scholar
  33. Gottfredson, M., & Hirschi, T. (1986). The true value of Lambda would appear to be zero: an essay on career criminals, criminal careers, selective incapacitation, cohort studies, and related topics*. Criminology, 24(2), 213–234.CrossRefGoogle Scholar
  34. Greenwood, P. W., & Abrahamse, A. F. (1982). Selective incapacitation. Santa Monica: Rand Corporation.Google Scholar
  35. Groff, E. R., & La Vigne, N. G. (2002). Forecasting the future of predictive crime mapping. Crime Prevention Studies, 13, 29–58.Google Scholar
  36. Grove, W. M., & Meehl, P. E. (1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: the clinical–statistical controversy. Psychology, Public Policy, and Law, 2(2), 293.CrossRefGoogle Scholar
  37. Huber, P. J. (1973). Robust regression: asymptotics, conjectures and Monte Carlo. The Annals of Statistics, 1, 799–821.MathSciNetzbMATHCrossRefGoogle Scholar
  38. Hunt, P., Saunders, J., & Hollywood, J. S. (2014). Evaluation of the Shreveport Predictive Policing Experiment. Santa Monica: RAND Corporation.Google Scholar
  39. Kang, J. D., & Schafer, J. L. (2007). Demystifying double robustness: a comparison of alternative strategies for estimating a population mean from incomplete data. Statistical Science, 25, 523–539.Google Scholar
  40. Kennedy, D. M. (1996). Pulling levers: chronic offenders, high-crime settings, and a theory of prevention. Valparaiso University Law Review, 31, 449.Google Scholar
  41. Kovandzic, T. V., Sloan, J. J., III, & Vieraitis, L. M. (2004). “Striking out” as crime reduction policy: the impact of “three strikes” laws on crime rates in US cities. Justice Quarterly, 21(2), 207–239.CrossRefGoogle Scholar
  42. Lewin, J., & Wernick, M. (2015). Chicago Police Department Data Analytics and Predictive Policing. Paper presented at the International Association of Chief’s of Police, Chicago, IL.Google Scholar
  43. Lipsey, M. W. (1999). Can intervention rehabilitate serious delinquents? The Annals of the American Academy of Political and Social Science, 564(1), 142–166.CrossRefGoogle Scholar
  44. Litwack, T. R., & 2. (2001). Actuarial versus clinical assessments of dangerousness. Psychology, Public Policy, and Law, 7, 409.CrossRefGoogle Scholar
  45. Llenas, B. (2014). Brave New World of “Predictive Policing” Raises Specter of High-Tech Racial Profiling, Fox News Latino.Google Scholar
  46. Loeber, R., & Farrington, D. P. (1998). Serious and violent juvenile offenders: Risk factors and successful interventions. Thousand Oaks, CA: Sage Publications.Google Scholar
  47. Lum, C., Koper, C. S., & Telep, C. W. (2011). The evidence-based policing matrix. Journal of Experimental Criminology, 7(1), 3–26.CrossRefGoogle Scholar
  48. Martin, S. E., & Sherman, L. W. (1986). Selective apprehension: a police strategy for repeat offenders. Criminology, 24, 155.CrossRefGoogle Scholar
  49. Mazerolle, L. G., Kadleck, C., & Roehl, J. (1998). Controlling drug and disorder problems: the role of place managers. Criminology, 36(2), 371–404.CrossRefGoogle Scholar
  50. Mazerolle, L. G., Ready, J., Terrill, W., & Waring, E. (2000). Problem-oriented policing in public housing: the Jersey City evaluation. Justice Quarterly, 17(1), 129–158.CrossRefGoogle Scholar
  51. McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods, 9(4), 403.PubMedCrossRefGoogle Scholar
  52. McCleary, R., Hay, R. A., Meidinger, E. E., & McDowall, D. (1980). Applied time series analysis for the social sciences. Beverly Hills: Sage Publications.Google Scholar
  53. McCord, J. (2003). Cures that harm: unanticipated outcomes of crime prevention programs. The Annals of the American Academy of Political and Social Science, 587(1), 16–30.CrossRefGoogle Scholar
  54. McGarrell, E. F., Chermak, S., Wilson, J. M., & Corsaro, N. (2006). Reducing homicide through a “lever‐pulling” strategy. Justice Quarterly, 23(02), 214–231.CrossRefGoogle Scholar
  55. Mohler, G. O., Short, M. B., Malinowski, S., Johnson, M., Tita, G. E., Bertozzi, A. L., & Brantingham, P. J. (2015). Randomized controlled field trials of predictive policing. Journal of the American Statistical Association, 110(512), 1399–1411.Google Scholar
  56. Ohri, A. (2013). Forecasting and Time Series Models R for Business Analytics (pp. 241–258). Springer.Google Scholar
  57. Papachristos, A. (2009). Murder by structure: dominance relations and the social structure of gang homicide. American Journal of Sociology, 115(1), 74–128.MathSciNetCrossRefGoogle Scholar
  58. Papachristos, A. V., & Kirk, D. S. (2015). Changing the street dynamic. Criminology and Public Policy, 14(3), 525–558.CrossRefGoogle Scholar
  59. Papachristos, A., Braga, A., & Hureau, D. (2011). Six-degrees of violent victimization: Social networks and the risk of gunshot injury.Google Scholar
  60. Papachristos, A., Braga, A., & Hureau, D. (2012). Social networks and the risk of gunshot injury. Journal of Urban Health, 89(6), 992–1003.PubMedPubMedCentralCrossRefGoogle Scholar
  61. Pate, T., Bowers, R. A., & Parks, R. (1976). Three approaches to criminal apprehension in Kansas City: An evaluation report. Washington, DC: Police Foundation.Google Scholar
  62. Perry, W. L., McInnis, B., Price, C. C., Smith, S. C., & Hollywood, J. S. (2013). Predictive policing: The role of crime forecasting in law enforcement operations. Santa Monica, CA: Rand Corporation.Google Scholar
  63. Quinsey, V. L., Harris, G. T., & Rice, M. E. (2000). Violent Offenders: Appraising and Managing Risk. Psychiatric Services, 51(3), 395Google Scholar
  64. Ratcliffe, J. (2002). Intelligence-led policing and the problems of turning rhetoric into practice. Policing and Society, 12(1), 53–66.MathSciNetCrossRefGoogle Scholar
  65. Ratcliffe, J. (2005). The effectiveness of police intelligence management: a New Zealand case study. Police Practice and Research, 6(5), 435–451.CrossRefGoogle Scholar
  66. Ratcliffe, J. H. (2012). Intelligence-led policing. New York, NY: Routledge.Google Scholar
  67. Ratcliffe, J. H., & Guidetti, R. (2008). State police investigative structure and the adoption of intelligence-led policing. Policing: An International Journal of Police Strategies and Management, 31(1), 109–128.CrossRefGoogle Scholar
  68. Ridgeway, G. (2013). Linking prediction and prevention. Criminology and Public Policy, 12(3), 545–550.CrossRefGoogle Scholar
  69. Ridgeway, G., & MacDonald, J. M. (2009). Doubly robust internal benchmarking and false discovery rates for detecting racial bias in police stops. Journal of the American Statistical Association, 104(486), 661–668.MathSciNetzbMATHCrossRefGoogle Scholar
  70. Ridgeway, G., Braga, A. A., Tita, G., & Pierce, G. L. (2011). Intervening in gun markets: an experiment to assess the impact of targeted gun-law messaging. Journal of Experimental Criminology, 7(1), 103–109.CrossRefGoogle Scholar
  71. Ridgeway, G., McCaffrey, D., Morral, A., Burgette, L., & Griffin, B. A. (2014). Toolkit for Weighting and Analysis of Nonequivalent Groups: A tutorial for the twang package. R vignette. RAND.Google Scholar
  72. Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55.MathSciNetzbMATHCrossRefGoogle Scholar
  73. Sailor, W., Dunlap, G., Sugai, G., & Horner, R. (2008). Handbook of positive behavior support. New York, NY: Springer.Google Scholar
  74. Sherman, L. W. (1986). Policing communities: what works? Crime and justice, 343–386.Google Scholar
  75. Sherman, L. W. (1992). The influence of criminology on criminal law: evaluating arrests for misdemeanor domestic violence. Journal of Criminal Law and Criminology, 83, 1–45.CrossRefGoogle Scholar
  76. Sherman, L. W., & Berk, R. A. (1984). The specific deterrent effects of arrest for domestic assault. American Sociological Review, 49, 261–272.PubMedCrossRefGoogle Scholar
  77. Sherman, L. W., & Weisburd, D. (1995). General deterrent effects of police patrol in crime “hot spots”: a randomized, controlled trial. Justice Quarterly, 12(4), 625–648.CrossRefGoogle Scholar
  78. Sherman, L. W., Gottfredson, D., MacKenzie, D., Eck, J., Reuter, P., & Bushway, S. (1997). Preventing crime: What works, what doesn’t, what’s promising: A report to the United States Congress. Washington, DC: US Department of Justice, Office of Justice Programs.Google Scholar
  79. Silver, E., & Miller, L. L. (2002). A cautionary note on the use of actuarial risk assessment tools for social control. Crime and Delinquency, 48(1), 138–161.CrossRefGoogle Scholar
  80. Starr, S. B. (2014). Evidence-based sentencing and the scientific rationalization of discrimination. Stanford Law Review, 66, 803–953.Google Scholar
  81. Stroud, M. (2014). The Minority Report: Chicago’s new police computer predicts crimes, but is it racist? The Verge.Google Scholar
  82. Tonry, M. (1987). Prediction and classification: legal and ethical issues. Crime and Justice, 9, 367–413.CrossRefGoogle Scholar
  83. Weisburd, D., & Mazerolle, L. G. (2000). Crime and disorder in drug hot spots: implications for theory and practice in policing. Police Quarterly, 3(3), 331–349.CrossRefGoogle Scholar
  84. Welsh, B. C., & Rocque, M. (2014). When crime prevention harms: a review of systematic reviews. Journal of Experimental Criminology, 10(3), 245–266.CrossRefGoogle Scholar
  85. Wright, K. N., Clear, T. R., & Dickson, P. (1984). Universal applicability of probation risk‐assessment instruments. Criminology, 22(1), 113–134.CrossRefGoogle Scholar
  86. Yang, M., Wong, S. C., & Coid, J. (2010). The efficacy of violence prediction: a meta-analytic comparison of nine risk assessment tools. Psychological Bulletin, 136(5), 740.PubMedCrossRefGoogle Scholar
  87. Zikopoulos, P., & Eaton, C. (2011). Understanding big data: Analytics for enterprise class hadoop and streaming data. McGraw-Hill Osborne Media.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  1. 1.RAND CorporationSanta MonicaUSA

Personalised recommendations