In 2013, the Chicago Police Department conducted a pilot of a predictive policing program designed to reduce gun violence. The program included development of a Strategic Subjects List (SSL) of people estimated to be at highest risk of gun violence who were then referred to local police commanders for a preventive intervention. The purpose of this study is to identify the impact of the pilot on individual- and city-level gun violence, and to test possible drivers of results.
The SSL consisted of 426 people estimated to be at highest risk of gun violence. We used ARIMA models to estimate impacts on city-level homicide trends, and propensity score matching to estimate the effects of being placed on the list on five measures related to gun violence. A mediation analysis and interviews with police leadership and COMPSTAT meeting observations help understand what is driving results.
Individuals on the SSL are not more or less likely to become a victim of a homicide or shooting than the comparison group, and this is further supported by city-level analysis. The treated group is more likely to be arrested for a shooting.
It is not clear how the predictions should be used in the field. One potential reason why being placed on the list resulted in an increased chance of being arrested for a shooting is that some officers may have used the list as leads to closing shooting cases. The results provide for a discussion about the future of individual-based predictive policing programs.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
This is only one component of the larger long-term collaboration which broadly explores whether and how crime can be predicted.
Here, “numerical instability” refers to the likelihood that the estimates that an SSL member’s calculated risk of a specified thousands of times more likely to be killed is due more to statistical artifacts from fitting the quadratic curve than an accurate estimate.
Initially, CPD said they would put all the highest-risk individuals on the list; however, they decided to vet the list through their Deployment Operations Center (DOC), who made some changes to who would appear on the list, and therefore, the 426 individuals did not represent the highest scoring individuals based on the model.
Details of GVRS are available at http://directives.chicagopolice.org/directives/data/a7a57bf0-136d1d31-16513-6d1d-382b311ddf65fd3a.html
It is important to note that Chicago has gone through transformative changes over this time period, including a new Superintendent in 2011 and the integration of COMPSTAT to provide oversight. In addition to changes in leadership and management style, CPD has implemented a large number of homicide reduction strategies during this time period, including the multiple changes to the Gang Violence Reduction Strategy, and gang call-ins across different districts starting in 2010.
While there is always the possibility that the groups are different on unobservable variables, we have captured many of the important research-validated criminogenic factors. That is why we specify the approach reduces, rather than eliminates, bias.
With the exception of the winter of 2012 which did not experience the same degree of a “cooling off” period.
Most arrestees were not incapacitated for any significant period of time, but rather were booked into the Cook County jail and released within a few hours to a few days.
Abrahamse, A. F., Ebener, P. A., Greenwood, P. W., Fitzgerald, N., & Kosin, T. E. (1991). An experimental evaluation of the Phoenix repeat offender program. Justice Quarterly, 8(2), 141–168.
Aitchison, J., & Dunsmore, I. R. (1980). Statistical prediction analysis. CUP Archive.
Auerhahn, K. (1999). Selective incapacitation and the problem of prediction. Criminology, 37(4), 703–734.
Bar-Hillel, M. (1980). The base-rate fallacy in probability judgments. Acta Psychologica, 44(3), 211–233.
Beck, C., & McCue, C. (2009). Predictive policing: what can we learn from Wal-Mart and Amazon about fighting crime in a recession? Police Chief, 76(11), 18.
Becker, G. S. (1993). Nobel lecture: the economic way of looking at behavior. Journal of Political Economy, 101, 385–409.
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B: Methodological, 57, 289–300.
Berk, R. (2008). Forecasting methods in crime and justice. Annual Review of Law and Social Science, 4, 219–238.
Berk, R. (2011). Asymmetric loss functions for forecasting in criminal justice settings. Journal of Quantitative Criminology, 27(1), 107–123.
Berk, R., & Bleich, J. (2013). Statistical procedures for forecasting criminal behavior. Criminology and Public Policy, 12(3), 513–544.
Berk, R., Sherman, L., Barnes, G., Kurtz, E., & Ahlman, L. (2009). Forecasting murder within a population of probationers and parolees: a high stakes application of statistical learning. Journal of the Royal Statistical Society: Series A (Statistics in Society), 172(1), 191–211.
Berry, M. J., & Linoff, G. S. (2004). Data mining techniques: for marketing, sales, and customer relationship management. Indianapolis, IN: John Wiley & Sons.
Blumstein, A. (1986). Criminal Careers and “Career Criminals” (Vol. 2). Washington, DC: National Academies.
Bordua, D. J., & Reiss, A. J., Jr. (1966). Command, control, and charisma: reflections on police bureaucracy. American Journal of Sociology, 72, 68–76.
Braga, A. (2005). Hot spots policing and crime prevention: a systematic review of randomized controlled trials. Journal of Experimental Criminology, 1(3), 317–342.
Braga, A., & Weisburd, D. L. (2010). Policing problem places: Crime hot spots and effective prevention. New York, NY: Oxford University Press on Demand.
Braga, A., & Weisburd, D. L. (2012). The effects of focused deterrence strategies on crime: a systematic review and meta-analysis of the empirical evidence. Journal of Research in Crime and Delinquency, 49(3), 323–358.
Braga, A., Papachristos, A. V., & Hureau, D. M. (2012). The effects of hot spots policing on crime: An updated systematic review and meta-analysis. Justice Quarterly, 1–31.
Bruinsma, G., & Weisburd, D. (2014). Encyclopedia of Criminology and Criminal Justice. New York: Springer.
Caldwell, M. F., Vitacco, M., & Van Rybroek, G. J. (2006). Are violent delinquents worth treating? A cost–benefit analysis. Journal of Research in Crime and Delinquency, 43(2), 148–168.
Chicago Police Department. (2014).Gang Violence Reuction Strategy: General Order G10-01. Chicago, IL.
Chinman, M., Imm, P., & Wandersman, A. (2004). Getting To Outcomes™ 2004. Santa Monica: Rand Corporation.
Cohen, J., Gorr, W. L., & Olligschlaeger, A. M. (2007). Leading indicators and spatial interactions: a crime‐forecasting model for proactive police deployment. Geographical Analysis, 39(1), 105–127.
Cope, N. (2004). ‘Intelligence led policing or policing led intelligence?’Integrating volume crime analysis into policing. British Journal of Criminology, 44(2), 188–203.
Cornish, D. B., & Clarke, R. V. (2014). The reasoning criminal: Rational choice perspectives on offending. New Brunswick, NJ: Transaction Publishers.
Dvoskin, J. A., & Heilbrun, K. (2001). Risk assessment and release decision-making: Toward resolving the great debate. American Academy of Psychiatry and the Law, 29, 6–10.
Eck, J., Chainey, S., Cameron, J., & Wilson, R. (2005). Mapping crime: Understanding hotspots (Vol. NCJ 209393). Washington, DC: National Institute of Justice.
Erbentraut, J. (2014). Chicago’s controversial new police program prompts fear of racial profiling. The Huffington Post.
Foster, E. M., & Jones, D. (2006). Can a costly intervention be cost-effective?: an analysis of violence prevention. Archives of General Psychiatry, 63(11), 1284–1291.
Funk, M. J., Westreich, D., Wiesen, C., Stürmer, T., Brookhart, M. A., & Davidian, M. (2011). Doubly robust estimation of causal effects. American Journal of Epidemiology, 173(7), 761–767.
Gendreau, P., Little, T., & Goggin, C. (1996). A meta-anallysis of the predictors of adult offender recidivism: what works! Criminology, 34(4), 575–608.
Gorr, W., & Harries, R. (2003). Introduction to crime forecasting. International Journal of Forecasting, 19(4), 551–555.
Gottfredson, M., & Hirschi, T. (1986). The true value of Lambda would appear to be zero: an essay on career criminals, criminal careers, selective incapacitation, cohort studies, and related topics*. Criminology, 24(2), 213–234.
Greenwood, P. W., & Abrahamse, A. F. (1982). Selective incapacitation. Santa Monica: Rand Corporation.
Groff, E. R., & La Vigne, N. G. (2002). Forecasting the future of predictive crime mapping. Crime Prevention Studies, 13, 29–58.
Grove, W. M., & Meehl, P. E. (1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: the clinical–statistical controversy. Psychology, Public Policy, and Law, 2(2), 293.
Huber, P. J. (1973). Robust regression: asymptotics, conjectures and Monte Carlo. The Annals of Statistics, 1, 799–821.
Hunt, P., Saunders, J., & Hollywood, J. S. (2014). Evaluation of the Shreveport Predictive Policing Experiment. Santa Monica: RAND Corporation.
Kang, J. D., & Schafer, J. L. (2007). Demystifying double robustness: a comparison of alternative strategies for estimating a population mean from incomplete data. Statistical Science, 25, 523–539.
Kennedy, D. M. (1996). Pulling levers: chronic offenders, high-crime settings, and a theory of prevention. Valparaiso University Law Review, 31, 449.
Kovandzic, T. V., Sloan, J. J., III, & Vieraitis, L. M. (2004). “Striking out” as crime reduction policy: the impact of “three strikes” laws on crime rates in US cities. Justice Quarterly, 21(2), 207–239.
Lewin, J., & Wernick, M. (2015). Chicago Police Department Data Analytics and Predictive Policing. Paper presented at the International Association of Chief’s of Police, Chicago, IL.
Lipsey, M. W. (1999). Can intervention rehabilitate serious delinquents? The Annals of the American Academy of Political and Social Science, 564(1), 142–166.
Litwack, T. R., & 2. (2001). Actuarial versus clinical assessments of dangerousness. Psychology, Public Policy, and Law, 7, 409.
Llenas, B. (2014). Brave New World of “Predictive Policing” Raises Specter of High-Tech Racial Profiling, Fox News Latino.
Loeber, R., & Farrington, D. P. (1998). Serious and violent juvenile offenders: Risk factors and successful interventions. Thousand Oaks, CA: Sage Publications.
Lum, C., Koper, C. S., & Telep, C. W. (2011). The evidence-based policing matrix. Journal of Experimental Criminology, 7(1), 3–26.
Martin, S. E., & Sherman, L. W. (1986). Selective apprehension: a police strategy for repeat offenders. Criminology, 24, 155.
Mazerolle, L. G., Kadleck, C., & Roehl, J. (1998). Controlling drug and disorder problems: the role of place managers. Criminology, 36(2), 371–404.
Mazerolle, L. G., Ready, J., Terrill, W., & Waring, E. (2000). Problem-oriented policing in public housing: the Jersey City evaluation. Justice Quarterly, 17(1), 129–158.
McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods, 9(4), 403.
McCleary, R., Hay, R. A., Meidinger, E. E., & McDowall, D. (1980). Applied time series analysis for the social sciences. Beverly Hills: Sage Publications.
McCord, J. (2003). Cures that harm: unanticipated outcomes of crime prevention programs. The Annals of the American Academy of Political and Social Science, 587(1), 16–30.
McGarrell, E. F., Chermak, S., Wilson, J. M., & Corsaro, N. (2006). Reducing homicide through a “lever‐pulling” strategy. Justice Quarterly, 23(02), 214–231.
Mohler, G. O., Short, M. B., Malinowski, S., Johnson, M., Tita, G. E., Bertozzi, A. L., & Brantingham, P. J. (2015). Randomized controlled field trials of predictive policing. Journal of the American Statistical Association, 110(512), 1399–1411.
Ohri, A. (2013). Forecasting and Time Series Models R for Business Analytics (pp. 241–258). Springer.
Papachristos, A. (2009). Murder by structure: dominance relations and the social structure of gang homicide. American Journal of Sociology, 115(1), 74–128.
Papachristos, A. V., & Kirk, D. S. (2015). Changing the street dynamic. Criminology and Public Policy, 14(3), 525–558.
Papachristos, A., Braga, A., & Hureau, D. (2011). Six-degrees of violent victimization: Social networks and the risk of gunshot injury.
Papachristos, A., Braga, A., & Hureau, D. (2012). Social networks and the risk of gunshot injury. Journal of Urban Health, 89(6), 992–1003.
Pate, T., Bowers, R. A., & Parks, R. (1976). Three approaches to criminal apprehension in Kansas City: An evaluation report. Washington, DC: Police Foundation.
Perry, W. L., McInnis, B., Price, C. C., Smith, S. C., & Hollywood, J. S. (2013). Predictive policing: The role of crime forecasting in law enforcement operations. Santa Monica, CA: Rand Corporation.
Quinsey, V. L., Harris, G. T., & Rice, M. E. (2000). Violent Offenders: Appraising and Managing Risk. Psychiatric Services, 51(3), 395
Ratcliffe, J. (2002). Intelligence-led policing and the problems of turning rhetoric into practice. Policing and Society, 12(1), 53–66.
Ratcliffe, J. (2005). The effectiveness of police intelligence management: a New Zealand case study. Police Practice and Research, 6(5), 435–451.
Ratcliffe, J. H. (2012). Intelligence-led policing. New York, NY: Routledge.
Ratcliffe, J. H., & Guidetti, R. (2008). State police investigative structure and the adoption of intelligence-led policing. Policing: An International Journal of Police Strategies and Management, 31(1), 109–128.
Ridgeway, G. (2013). Linking prediction and prevention. Criminology and Public Policy, 12(3), 545–550.
Ridgeway, G., & MacDonald, J. M. (2009). Doubly robust internal benchmarking and false discovery rates for detecting racial bias in police stops. Journal of the American Statistical Association, 104(486), 661–668.
Ridgeway, G., Braga, A. A., Tita, G., & Pierce, G. L. (2011). Intervening in gun markets: an experiment to assess the impact of targeted gun-law messaging. Journal of Experimental Criminology, 7(1), 103–109.
Ridgeway, G., McCaffrey, D., Morral, A., Burgette, L., & Griffin, B. A. (2014). Toolkit for Weighting and Analysis of Nonequivalent Groups: A tutorial for the twang package. R vignette. RAND.
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55.
Sailor, W., Dunlap, G., Sugai, G., & Horner, R. (2008). Handbook of positive behavior support. New York, NY: Springer.
Sherman, L. W. (1986). Policing communities: what works? Crime and justice, 343–386.
Sherman, L. W. (1992). The influence of criminology on criminal law: evaluating arrests for misdemeanor domestic violence. Journal of Criminal Law and Criminology, 83, 1–45.
Sherman, L. W., & Berk, R. A. (1984). The specific deterrent effects of arrest for domestic assault. American Sociological Review, 49, 261–272.
Sherman, L. W., & Weisburd, D. (1995). General deterrent effects of police patrol in crime “hot spots”: a randomized, controlled trial. Justice Quarterly, 12(4), 625–648.
Sherman, L. W., Gottfredson, D., MacKenzie, D., Eck, J., Reuter, P., & Bushway, S. (1997). Preventing crime: What works, what doesn’t, what’s promising: A report to the United States Congress. Washington, DC: US Department of Justice, Office of Justice Programs.
Silver, E., & Miller, L. L. (2002). A cautionary note on the use of actuarial risk assessment tools for social control. Crime and Delinquency, 48(1), 138–161.
Starr, S. B. (2014). Evidence-based sentencing and the scientific rationalization of discrimination. Stanford Law Review, 66, 803–953.
Stroud, M. (2014). The Minority Report: Chicago’s new police computer predicts crimes, but is it racist? The Verge.
Tonry, M. (1987). Prediction and classification: legal and ethical issues. Crime and Justice, 9, 367–413.
Weisburd, D., & Mazerolle, L. G. (2000). Crime and disorder in drug hot spots: implications for theory and practice in policing. Police Quarterly, 3(3), 331–349.
Welsh, B. C., & Rocque, M. (2014). When crime prevention harms: a review of systematic reviews. Journal of Experimental Criminology, 10(3), 245–266.
Wright, K. N., Clear, T. R., & Dickson, P. (1984). Universal applicability of probation risk‐assessment instruments. Criminology, 22(1), 113–134.
Yang, M., Wong, S. C., & Coid, J. (2010). The efficacy of violence prediction: a meta-analytic comparison of nine risk assessment tools. Psychological Bulletin, 136(5), 740.
Zikopoulos, P., & Eaton, C. (2011). Understanding big data: Analytics for enterprise class hadoop and streaming data. McGraw-Hill Osborne Media.
We would like to thank the Chicago Police Department and Dr. Miles Wernick from the Illinois Institute of Technology for their participation and support of this evaluation. We would also like to acknowledge research assistance provided by Sam Cooper and Alessandra Sienra-Canas. This publication was made possible by Award Number 2009-IJ-CX-K114 - Predictive Policing Analytic & Evaluation Research Support awarded by the National Institute of Justice, Office of Justice Programs. The opinions, findings, conclusions and recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the Department of Justice.
Rights and permissions
About this article
Cite this article
Saunders, J., Hunt, P. & Hollywood, J.S. Predictions put into practice: a quasi-experimental evaluation of Chicago’s predictive policing pilot. J Exp Criminol 12, 347–371 (2016). https://doi.org/10.1007/s11292-016-9272-0
- Predictive policing
- Program evaluation
- Propensity score matching
- Quasi-experimental design
- Risk assessment
- Time series analysis