A quasi-experimental synthetic control evaluation of a place-based police-directed patrol intervention on violent crime

Abstract

Objectives

This research evaluates the impact of an implementation of a place-based police-directed patrol intervention—originally based on the Data Driven Approaches to Crime and Traffic Safety (DDACTS) model—on violent crime in Flint, Michigan, USA.

Methods

We utilize recent advances in synthetic control methods to implement a retrospective quasi-experimental design across seven separate intervention areas, producing a counterfactual estimate of what would have happened to violent crime had the intervention never been implemented. We use survey weight calibration to produce counterfactual intervention areas using comparison block groups in Flint, and account for treatment diffusion by using comparison block groups from Detroit.

Results

The synthetic control method calibrated a set of weights to exactly match the intervention hot spots to counterfactuals from Flint and Detroit. Although basic trend analyses suggested declines in violent crime in the treatment areas, the synthetic controls raised questions about treatment effects. Specifically, the Flint comparison revealed an unexpected increase in aggravated assaults associated with the intervention, whereas the Detroit comparison suggested a similar effect but also possible reduction in robberies.

Conclusions

This evaluation presents mixed findings regarding the effect of the intervention on violent crime. Inconsistent program effects may be attributable to incongruences between the program as implemented and the prescribed DDACTS model on which it was based. The findings also suggest the need for future research to investigate potential differential effects of directed patrol on specific types of violent crime. The synthetic control method provides a powerful means for counterfactual estimation in retrospective evaluations.

This is a preview of subscription content, access via your institution.

Fig. 1

Data Availability Statement

Replication data pertaining to the outcome evaluation of the program described in this manuscript, as well as (relatively messy) script for implementing the analyses are available from the primary author upon reasonable request.

Notes

  1. 1.

    It is important to note that the implications of the distributional misspecification in applying WLS to count outcomes are mitigated through the use of permutation tests (described later). For peace of mind, we replicated the analyses for Hotspot 1 using negative binomial regression, and the substantive conclusions were unchanged from the WLS analyses. However, the variance estimates produced by WLS were somewhat more conservative than the negative binomial regression models.

  2. 2.

    It is worth noting that Robbins et al. (2017) also propose a Wald-type omnibus test which can approximate an omnibus p value without the use of placebo tests. However, as the authors note, the computation of the quantities necessary for this statistic is non-trivial. We use the original permutation-based omnibus statistic (Robbins et al. 2015) in its place.

  3. 3.

    Indeed, Robbins et al. (2015) note that the significance of the omnibus statistic can only be assessed via comparison to a distribution of permutation values. This is because of the potential for the omnibus statistic to simply continue to grow as more outcomes are included. Assuming this value was sampled from a normal distribution would have an increased type I error rate.

  4. 4.

    It should be noted that the traffic stops were much more densely concentrated within the hot spots (9.6 monthly traffic stops per square mile outside the hot spots, compared to 73.8 within the hot spots).

  5. 5.

    Our contention is not that virtually any intervention for a given city could successfully draw a comparison set of blocks from some other city. Rather, in the present case, Detroit is uniquely situated to act as a counterfactual donor pool to Flint, due to the cities' geographic proximity, shared socioeconomic history, and violent crime levels (Jacobs 2004; Matthews 1997). Further, it is important to note that, although Detroit was part of the Secure Cities Initiative, it did not receive the same DDACTS/directed patrol intervention as in Flint during the study period. Instead, the Detroit program involved embedding Michigan Department of Corrections community corrections agents within the Detroit Police Department, and provided additional Michigan State Police personnel to supplement homicide detectives. The State Police did conduct vehicle patrols in Detroit but these did not involve the intensity or the geographic hot spot focus of the Flint DDACTS/directed patrol initiative.

  6. 6.

    Specifically, two sets of weights are estimated for each comparison (Flint and Detroit). The first set balances on the individual outcomes, and the second set balances on the aggregated total violence outcome. The two sets were estimated so that both the individual and aggregated outcomes would have corresponding synthetic controls which exactly matched the hot spots (i.e., the weights for the individual outcomes do not guarantee an exact match for the aggregated outcomes). Descriptive statistics for the weights presented in the following sections describe the individual outcome weights, given that these will have a larger variance and corresponding design effect.

  7. 7.

    Table 2 displays weighted totals for outcomes and covariates across blocks in hot spot 1 and comparisons (estimated using svytotal). It should be noted that the weighted outcome and covariate means between the hot spot and comparison areas were also equivalent (produced using svymean).

  8. 8.

    We focus on aggravated assaults and robberies, and do not plot homicides due to their relative infrequency, the fact that the synthetic control weights only balance on the total number of homicides over the pre-intervention period, rather than at individual quarters, and the largely null impacts for the program on homicides.

  9. 9.

    A random-effects meta-analysis, consistent with the approach used by Saunders et al. (2015), produced identical results.

References

  1. Abadie, A., Diamond, A., & Hainmueller, J. (2015). Comparative politics and the synthetic control method. American Journal of Political Science, 59(2), 495–510.

    Article  Google Scholar 

  2. Braga, A. A. (2005). Hot spots policing and crime prevention: A systematic review of randomized controlled trials. Journal of Experimental Criminology, 1(3), 317–342.

    Article  Google Scholar 

  3. Braga, A. A. (2010). Setting a higher standard for the evaluation of problem-oriented policing initiatives. Criminology and Public Policy, 9(1), 173–182.

    Article  Google Scholar 

  4. Braga, A. A., & Bond, B. J. (2008). Policing crime and disorder hot spots: A randomized controlled trial. Criminology, 46(3), 577–607.

    Article  Google Scholar 

  5. Braga, A., & Weisburd, D. (2010). Policing problem places: Crime hot spots and effective prevention. New York: Oxford University Press.

  6. Braga, A. A., Papachristos, A. V., & Hureau, D. M. (2010). The concentration and stability of gun violence at micro places in Boston, 1980–2008. Journal of Quantitative Criminology, 26(1), 33–53.

    Article  Google Scholar 

  7. Braga, A. A., Hureau, D. M., & Papachristos, A. V. (2011). An ex post facto evaluation framework for place-based police interventions. Evaluation Review, 35(6), 592–626.

    Article  Google Scholar 

  8. Braga, A. A., Papachristos, A. V., & Hureau, D. M. (2014). The effects of hot spots policing on crime: An updated systematic review and meta-analysis. Justice Quarterly, 31(4), 633–663.

    Article  Google Scholar 

  9. Brantingham, P. L., & Brantingham, P. (1993). Environment, routine, and situation: Toward a pattern theory of crime. In R. V. Clarke & M. Felson (Eds.), Routine activity and rational choice (pp. 27–54). New Brunswick: Transaction.

    Google Scholar 

  10. Bryant, K. M., Collins, G., & Villa, J. (2014). Data Driven Approaches to Crime and Traffic Safety: Shawnee, Kansas 2010–2013. Report prepared for the Bureau of Justice Assistance.

  11. Carsey, T. M., & Harden, J. J. (2014). Monte carlo simulation and resampling methods for social science. Thousand Oaks: Sage.

    Google Scholar 

  12. Carter, J. G., & Piza, E. L. (2017). Spatiotemporal convergence of crime and vehicle crash hotspots: Additional consideration for policing places. Crime and Delinquency. https://doi.org/10.1177/0011128717714793

  13. Cohen, J., & Ludwig, J. (2003). Policing gun crimes. In J. Ludwig & P. J. Cook (Eds.), Evaluating gun policy: Effects on crime and violence (pp. 217–250). Washington, DC: Brookings Institution.

    Google Scholar 

  14. Cordner, G. W. (1981). The effects of directed patrol: A natural quasi-experiment in Pontiac. In J. J. Fyfe (Ed.), Contemporary issues in law enforcement (pp. 37–58). Beverly Hills: Sage.

    Google Scholar 

  15. Cornish, D. B., & Clark, R. V. (1987). Understanding crime displacement: An application of rational choice theory. Criminology, 25(4), 933–948.

    Article  Google Scholar 

  16. Corsaro, N. (2013). The high point drug market intervention: Examining impact across target areas and offense types. Victims and Offenders, 8, 416–445.

    Article  Google Scholar 

  17. CrimeSolutions.gov (2016). Program Profile: Data Driven Approaches to Crime and Traffic Safety (DDACTS) in Kansas. Retrieved from https://www.crimesolutions.gov/ProgramDetails.aspx?ID=479.

  18. Deville, J. C., Särndal, C. E., & Sautory, O. (1993). Generalized raking procedures in survey sampling. Journal of the American Statistical Association, 88(423), 1013–1020.

    Article  Google Scholar 

  19. Grady, S. C., & Enander, H. (2009). Geographic analysis of low birthweight and infant mortality in Michigan using automated zoning methodology. International Journal of Health Geographics, 8(1), 10–28.

    Article  Google Scholar 

  20. Groff, E. R., Weisburd, D., & Yang, S. M. (2010). Is it important to examine crime trends at a local “micro” level? A longitudinal analysis of street to street variability in crime trajectories. Journal of Quantitative Criminology, 26(1), 7–32.

    Article  Google Scholar 

  21. Haberman, C. P., Groff, E. R., Ratcliffe, J. H., & Sorg, E. T. (2016). Satisfaction with police in violent crime hotspots: Using community surveys as a guide for selecting hot spots policing tactics. Crime and Delinquency, 62(4), 525–557.

    Article  Google Scholar 

  22. Hardy, E. (2010). Data-driven policing: How geographic analysis can reduce social harm. Geography Public Safety: A Quarterly Bulletin of Applied Geography for the Study of Crime and Public Policy, 2(3), 1–2.

  23. IACP & NHTSA. (2001). Traffic safety in the new millennium: Strategies for law enforcement: A planning guide for law enforcement executives, administrators and managers. Washington, DC: National Highway Traffic Safety Administration.

    Google Scholar 

  24. Jacobs, A. J. (2004). Inter-local relations and divergent growth: The Detroit and Tokai auto regions, 1969 to 1996. Journal of Urban Affairs, 26(4), 479–504.

    Article  Google Scholar 

  25. Kreml, F. (1954). The specialized traffic division source. Annals of the American Academy of Political and Social Science, 291, 63–72.

    Article  Google Scholar 

  26. Kubrin, C. E., Messner, S. F., Deane, G., McGeever, K., & Stucky, T. D. (2010). Proactive policing and robbery rates across US cities. Criminology, 48(1), 57–97.

    Article  Google Scholar 

  27. Lumley, T. (2014). survey: analysis of complex survey samples. R package version 3.31.

  28. Matthews, R. (1997). What’s good for GM…: Deindustrialization and crime in four Michigan cities, 1975–1993. Dissertation: Western Michigan University, Department of Sociology.

  29. McClure, D., Levy, J., La Vigne, N., & Hayeslip, D. (2014). DDACTS Evaluability assessment: Final report on individual and cross-site findings. Washington, DC: Justice Policy Center at the Urban Institute.

    Google Scholar 

  30. McGarrell, E., Chermak, S., Weiss, A., & Wilson, J. (2001). Reducing firearms violence through directed police patrol. Criminology and Public Policy, 1, 119–148.

    Article  Google Scholar 

  31. NHTSA. (2014). Data-driven approaches to crime and traffic safety, perational guidelines. Washington, DC: Author.

    Google Scholar 

  32. R Core Team. (2016). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing http://www.R-project.org/

    Google Scholar 

  33. Ratcliffe, J. H., Taniguchi, T., Groff, E. R., & Wood, J. D. (2011). The Philadelphia foot patrol experiment: A randomized controlled trial of police patrol effectiveness in violent crime hotspots. Criminology, 49(3), 795–831.

    Article  Google Scholar 

  34. Robbins, M. W., Saunders, J., & Kilmer, B. (2015). A framework for synthetic control methods with high-dimensional, micro-level data: Evaluating a neighborhood-specific crime intervention. Santa Monica: RAND Corp Available at: http://www.rand.org/pubs/working_papers/WR1080.html

    Google Scholar 

  35. Robbins, M. W., Saunders, J., & Kilmer, B. (2017). A framework for synthetic control methods with high-dimensional, micro-level data: Evaluating a neighborhood-specific crime intervention. Journal of the American Statistical Association, 112(517), 109–126.

    Article  Google Scholar 

  36. Rosenbaum, P. R. (2005). Heterogeneity and causality: Unit heterogeneity and design sensitivity in observational studies. The American Statistician, 59(2), 147–152.

    Article  Google Scholar 

  37. Rosenfeld, R., Deckard, M. J., & Blackburn, E. (2014). The effects of directed patrol and self-initiated enforcement on firearm violence: A randomized controlled study of hot spot policing. Criminology, 52(3), 428–449.

    Article  Google Scholar 

  38. Rudolph, K. E., Stuart, E. A., Vernick, J. S., & Webster, D. W. (2015). Association between Connecticut’s permit-to-purchase handgun law and homicides. American Journal of Public Health, 105(8), e49–e54.

    Article  Google Scholar 

  39. Sampson, R. J., & Cohen, J. (1988). Deterrent effects of the police on crime: A replication and theoretical extension. Law and Society Review, 22(1), 163–189.

    Article  Google Scholar 

  40. Saunders, J., Lundberg, R., Braga, A. A., Ridgeway, G., & Miles, J. (2015). A synthetic control approach to evaluating place-based crime interventions. Journal of Quantitative Criminology, 31(3), 413–434.

    Article  Google Scholar 

  41. Schwarzer, G. (2007). Meta: An R package for meta-analysis. R News, 7(3), 40–45.

    Google Scholar 

  42. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Belmont: Wadsworth, Cengage Learning.

    Google Scholar 

  43. Sherman, L. W. (1993). Defiance, deterrence, and irrelevance: A theory of the criminal sanction. Journal of Research in Crime and Delinquency, 30(4), 445–473.

    Article  Google Scholar 

  44. Sherman, L., & Rogan, D. (1995). Effects of gun seizure on gun violence: ‘Hot spots’ patrol in Kansas City. Justice Quarterly, 12(4), 673–693.

    Article  Google Scholar 

  45. Sherman, L. W., & Weisburd, D. (1995). General deterrent effects of police patrol in crime “hot spots”: A randomized, controlled trial. Justice Quarterly, 12(4), 625–648.

    Article  Google Scholar 

  46. Skogan, W., & Frydl, K. (2004). Fairness and effectiveness in policing: The evidence. Washington DC: National Academies Press.

  47. Sorg, E. T., Wood, J. D., Groff, E. R., & Ratcliffe, J. H. (2014). Boundary adherence during place-based policing evaluations: A research note. Journal of Research in Crime and Delinquency, 51(3), 377–393.

    Article  Google Scholar 

  48. Stanojević, P., Jovanović, D., & Lajunen, T. (2013). Influence of traffic enforcement on the attitudes and behavior of drivers. Accident Analysis and Prevention, 52, 29–38.

    Article  Google Scholar 

  49. Stuster, J., Sheehan, M., & Monford, G. (1997). Contributions of traffic enforcement to the war on crime: Case study of the grand prairie, Texas police department. Washington, D.C.: National Highway Traffic Safety Administration.

    Google Scholar 

  50. Telep, C. W., Mitchell, R. J., & Weisburd, D. (2014). How much time should the police spend at crime hot spots? Answers from a police agency directed randomized field trial in Sacramento, California. Justice Quarterly, 31(5), 905–933.

    Article  Google Scholar 

  51. Thomas, F. D., Blomberg, R. D., Peck, R. C., Cosgrove, L. A., & Salzberg, P. M. (2008). Evaluation of a high visibility enforcement project focused on passenger vehicles interacting with commercial vehicles. Journal of Safety Research, 39(5), 459–468.

    Article  Google Scholar 

  52. Trojanowicz, R. C. (1986). Evaluating a neighborhood foot patrol program: The Flint, Michigan, project. In D. P. Rosenbaum (Ed.), Community crime prevention: Does it work? Newbury Park: Sage.

    Google Scholar 

  53. Viechtbauer, W. (2010). Conducting meta-analyses in R with the metaphor package. Journal of Statistical Software, 36(3), 1–48.

    Article  Google Scholar 

  54. Wasserstein, R. L., & Lazar, N. A. (2016). The ASA's statement on p-values: Context, process, and purpose. The American Statistician, Online Advance Publication, 70(2), 129–133.

    Article  Google Scholar 

  55. Weisburd, D. (2005). Hot spots policing experiments and criminal justice research: Lessons from the field. The Annals of the American Academy of Political and Social Science, 599(1), 220–245.

    Article  Google Scholar 

  56. Weisburd, D. (2008). Place-based policing. Washington: Police Foundation.

    Google Scholar 

  57. Weisburd, D. (2015). The law of crime concentration and the criminology of place. Criminology, 53(2), 133–157.

    Article  Google Scholar 

  58. Weisburd, D., Telep, C., Hinkle, J., & Eck, J. (2010). Is problem-oriented policing effective in reducing crime and disorder? Criminology and Public Policy, 9, 139–172.

    Article  Google Scholar 

  59. Weiss, A. (2013). Data-driven approaches to crime and traffic safety (DDACTS): An historical overview. Washington DC: NHSTA.

    Google Scholar 

  60. Weiss, A., & Freels, S. (1996). The effects of aggressive policing: The Dayton traffic enforcement experiment. American Journal of Police, 15, 45–64.

    Article  Google Scholar 

  61. Weiss, A., & McGarrell, E.F. (1999). Traffic enforcement and crime: Another look. Police Chief, 66(7), 25–28.

  62. Worden, R. E., & McLean, S. J. (2009). DDACTS in theory and practice. Albany: John F. Finn Institute for Public Safety.

    Google Scholar 

  63. Yannis, G., Papadimitriou, E., & Antoniou, C. (2007). Multilevel modelling for the regional effect of enforcement on road accidents. Accident Analysis and Prevention, 39(4), 818–825.

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported grant number 2012-BJ-CX-K036 awarded by the Bureau of Justice Statistics, Office of Justice Programs, U.S. Department of Justice. This research was made possible by substantial assistance from the Michigan State Police. Points of view in this document are those of the author and do not necessarily represent the official position or policies of the US Department of Justice or the Michigan State Police. We would like to thank the editorial team and three anonymous reviewers for their helpful comments, and especially Michael Robbins and Jessica Saunders for their insights on implementing the methods described here.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jason Rydberg.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Rydberg, J., McGarrell, E.F., Norris, A. et al. A quasi-experimental synthetic control evaluation of a place-based police-directed patrol intervention on violent crime. J Exp Criminol 14, 83–109 (2018). https://doi.org/10.1007/s11292-018-9324-8

Download citation

Keywords

  • Synthetic control
  • Quasi-experimental design
  • Place-based policing
  • Directed patrol
  • Program evaluation