This research evaluates the impact of an implementation of a place-based police-directed patrol intervention—originally based on the Data Driven Approaches to Crime and Traffic Safety (DDACTS) model—on violent crime in Flint, Michigan, USA.
We utilize recent advances in synthetic control methods to implement a retrospective quasi-experimental design across seven separate intervention areas, producing a counterfactual estimate of what would have happened to violent crime had the intervention never been implemented. We use survey weight calibration to produce counterfactual intervention areas using comparison block groups in Flint, and account for treatment diffusion by using comparison block groups from Detroit.
The synthetic control method calibrated a set of weights to exactly match the intervention hot spots to counterfactuals from Flint and Detroit. Although basic trend analyses suggested declines in violent crime in the treatment areas, the synthetic controls raised questions about treatment effects. Specifically, the Flint comparison revealed an unexpected increase in aggravated assaults associated with the intervention, whereas the Detroit comparison suggested a similar effect but also possible reduction in robberies.
This evaluation presents mixed findings regarding the effect of the intervention on violent crime. Inconsistent program effects may be attributable to incongruences between the program as implemented and the prescribed DDACTS model on which it was based. The findings also suggest the need for future research to investigate potential differential effects of directed patrol on specific types of violent crime. The synthetic control method provides a powerful means for counterfactual estimation in retrospective evaluations.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Data Availability Statement
Replication data pertaining to the outcome evaluation of the program described in this manuscript, as well as (relatively messy) script for implementing the analyses are available from the primary author upon reasonable request.
It is important to note that the implications of the distributional misspecification in applying WLS to count outcomes are mitigated through the use of permutation tests (described later). For peace of mind, we replicated the analyses for Hotspot 1 using negative binomial regression, and the substantive conclusions were unchanged from the WLS analyses. However, the variance estimates produced by WLS were somewhat more conservative than the negative binomial regression models.
It is worth noting that Robbins et al. (2017) also propose a Wald-type omnibus test which can approximate an omnibus p value without the use of placebo tests. However, as the authors note, the computation of the quantities necessary for this statistic is non-trivial. We use the original permutation-based omnibus statistic (Robbins et al. 2015) in its place.
Indeed, Robbins et al. (2015) note that the significance of the omnibus statistic can only be assessed via comparison to a distribution of permutation values. This is because of the potential for the omnibus statistic to simply continue to grow as more outcomes are included. Assuming this value was sampled from a normal distribution would have an increased type I error rate.
It should be noted that the traffic stops were much more densely concentrated within the hot spots (9.6 monthly traffic stops per square mile outside the hot spots, compared to 73.8 within the hot spots).
Our contention is not that virtually any intervention for a given city could successfully draw a comparison set of blocks from some other city. Rather, in the present case, Detroit is uniquely situated to act as a counterfactual donor pool to Flint, due to the cities' geographic proximity, shared socioeconomic history, and violent crime levels (Jacobs 2004; Matthews 1997). Further, it is important to note that, although Detroit was part of the Secure Cities Initiative, it did not receive the same DDACTS/directed patrol intervention as in Flint during the study period. Instead, the Detroit program involved embedding Michigan Department of Corrections community corrections agents within the Detroit Police Department, and provided additional Michigan State Police personnel to supplement homicide detectives. The State Police did conduct vehicle patrols in Detroit but these did not involve the intensity or the geographic hot spot focus of the Flint DDACTS/directed patrol initiative.
Specifically, two sets of weights are estimated for each comparison (Flint and Detroit). The first set balances on the individual outcomes, and the second set balances on the aggregated total violence outcome. The two sets were estimated so that both the individual and aggregated outcomes would have corresponding synthetic controls which exactly matched the hot spots (i.e., the weights for the individual outcomes do not guarantee an exact match for the aggregated outcomes). Descriptive statistics for the weights presented in the following sections describe the individual outcome weights, given that these will have a larger variance and corresponding design effect.
Table 2 displays weighted totals for outcomes and covariates across blocks in hot spot 1 and comparisons (estimated using svytotal). It should be noted that the weighted outcome and covariate means between the hot spot and comparison areas were also equivalent (produced using svymean).
We focus on aggravated assaults and robberies, and do not plot homicides due to their relative infrequency, the fact that the synthetic control weights only balance on the total number of homicides over the pre-intervention period, rather than at individual quarters, and the largely null impacts for the program on homicides.
A random-effects meta-analysis, consistent with the approach used by Saunders et al. (2015), produced identical results.
Abadie, A., Diamond, A., & Hainmueller, J. (2015). Comparative politics and the synthetic control method. American Journal of Political Science, 59(2), 495–510.
Braga, A. A. (2005). Hot spots policing and crime prevention: A systematic review of randomized controlled trials. Journal of Experimental Criminology, 1(3), 317–342.
Braga, A. A. (2010). Setting a higher standard for the evaluation of problem-oriented policing initiatives. Criminology and Public Policy, 9(1), 173–182.
Braga, A. A., & Bond, B. J. (2008). Policing crime and disorder hot spots: A randomized controlled trial. Criminology, 46(3), 577–607.
Braga, A., & Weisburd, D. (2010). Policing problem places: Crime hot spots and effective prevention. New York: Oxford University Press.
Braga, A. A., Papachristos, A. V., & Hureau, D. M. (2010). The concentration and stability of gun violence at micro places in Boston, 1980–2008. Journal of Quantitative Criminology, 26(1), 33–53.
Braga, A. A., Hureau, D. M., & Papachristos, A. V. (2011). An ex post facto evaluation framework for place-based police interventions. Evaluation Review, 35(6), 592–626.
Braga, A. A., Papachristos, A. V., & Hureau, D. M. (2014). The effects of hot spots policing on crime: An updated systematic review and meta-analysis. Justice Quarterly, 31(4), 633–663.
Brantingham, P. L., & Brantingham, P. (1993). Environment, routine, and situation: Toward a pattern theory of crime. In R. V. Clarke & M. Felson (Eds.), Routine activity and rational choice (pp. 27–54). New Brunswick: Transaction.
Bryant, K. M., Collins, G., & Villa, J. (2014). Data Driven Approaches to Crime and Traffic Safety: Shawnee, Kansas 2010–2013. Report prepared for the Bureau of Justice Assistance.
Carsey, T. M., & Harden, J. J. (2014). Monte carlo simulation and resampling methods for social science. Thousand Oaks: Sage.
Carter, J. G., & Piza, E. L. (2017). Spatiotemporal convergence of crime and vehicle crash hotspots: Additional consideration for policing places. Crime and Delinquency. https://doi.org/10.1177/0011128717714793
Cohen, J., & Ludwig, J. (2003). Policing gun crimes. In J. Ludwig & P. J. Cook (Eds.), Evaluating gun policy: Effects on crime and violence (pp. 217–250). Washington, DC: Brookings Institution.
Cordner, G. W. (1981). The effects of directed patrol: A natural quasi-experiment in Pontiac. In J. J. Fyfe (Ed.), Contemporary issues in law enforcement (pp. 37–58). Beverly Hills: Sage.
Cornish, D. B., & Clark, R. V. (1987). Understanding crime displacement: An application of rational choice theory. Criminology, 25(4), 933–948.
Corsaro, N. (2013). The high point drug market intervention: Examining impact across target areas and offense types. Victims and Offenders, 8, 416–445.
CrimeSolutions.gov (2016). Program Profile: Data Driven Approaches to Crime and Traffic Safety (DDACTS) in Kansas. Retrieved from https://www.crimesolutions.gov/ProgramDetails.aspx?ID=479.
Deville, J. C., Särndal, C. E., & Sautory, O. (1993). Generalized raking procedures in survey sampling. Journal of the American Statistical Association, 88(423), 1013–1020.
Grady, S. C., & Enander, H. (2009). Geographic analysis of low birthweight and infant mortality in Michigan using automated zoning methodology. International Journal of Health Geographics, 8(1), 10–28.
Groff, E. R., Weisburd, D., & Yang, S. M. (2010). Is it important to examine crime trends at a local “micro” level? A longitudinal analysis of street to street variability in crime trajectories. Journal of Quantitative Criminology, 26(1), 7–32.
Haberman, C. P., Groff, E. R., Ratcliffe, J. H., & Sorg, E. T. (2016). Satisfaction with police in violent crime hotspots: Using community surveys as a guide for selecting hot spots policing tactics. Crime and Delinquency, 62(4), 525–557.
Hardy, E. (2010). Data-driven policing: How geographic analysis can reduce social harm. Geography Public Safety: A Quarterly Bulletin of Applied Geography for the Study of Crime and Public Policy, 2(3), 1–2.
IACP & NHTSA. (2001). Traffic safety in the new millennium: Strategies for law enforcement: A planning guide for law enforcement executives, administrators and managers. Washington, DC: National Highway Traffic Safety Administration.
Jacobs, A. J. (2004). Inter-local relations and divergent growth: The Detroit and Tokai auto regions, 1969 to 1996. Journal of Urban Affairs, 26(4), 479–504.
Kreml, F. (1954). The specialized traffic division source. Annals of the American Academy of Political and Social Science, 291, 63–72.
Kubrin, C. E., Messner, S. F., Deane, G., McGeever, K., & Stucky, T. D. (2010). Proactive policing and robbery rates across US cities. Criminology, 48(1), 57–97.
Lumley, T. (2014). survey: analysis of complex survey samples. R package version 3.31.
Matthews, R. (1997). What’s good for GM…: Deindustrialization and crime in four Michigan cities, 1975–1993. Dissertation: Western Michigan University, Department of Sociology.
McClure, D., Levy, J., La Vigne, N., & Hayeslip, D. (2014). DDACTS Evaluability assessment: Final report on individual and cross-site findings. Washington, DC: Justice Policy Center at the Urban Institute.
McGarrell, E., Chermak, S., Weiss, A., & Wilson, J. (2001). Reducing firearms violence through directed police patrol. Criminology and Public Policy, 1, 119–148.
NHTSA. (2014). Data-driven approaches to crime and traffic safety, perational guidelines. Washington, DC: Author.
R Core Team. (2016). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing http://www.R-project.org/
Ratcliffe, J. H., Taniguchi, T., Groff, E. R., & Wood, J. D. (2011). The Philadelphia foot patrol experiment: A randomized controlled trial of police patrol effectiveness in violent crime hotspots. Criminology, 49(3), 795–831.
Robbins, M. W., Saunders, J., & Kilmer, B. (2015). A framework for synthetic control methods with high-dimensional, micro-level data: Evaluating a neighborhood-specific crime intervention. Santa Monica: RAND Corp Available at: http://www.rand.org/pubs/working_papers/WR1080.html
Robbins, M. W., Saunders, J., & Kilmer, B. (2017). A framework for synthetic control methods with high-dimensional, micro-level data: Evaluating a neighborhood-specific crime intervention. Journal of the American Statistical Association, 112(517), 109–126.
Rosenbaum, P. R. (2005). Heterogeneity and causality: Unit heterogeneity and design sensitivity in observational studies. The American Statistician, 59(2), 147–152.
Rosenfeld, R., Deckard, M. J., & Blackburn, E. (2014). The effects of directed patrol and self-initiated enforcement on firearm violence: A randomized controlled study of hot spot policing. Criminology, 52(3), 428–449.
Rudolph, K. E., Stuart, E. A., Vernick, J. S., & Webster, D. W. (2015). Association between Connecticut’s permit-to-purchase handgun law and homicides. American Journal of Public Health, 105(8), e49–e54.
Sampson, R. J., & Cohen, J. (1988). Deterrent effects of the police on crime: A replication and theoretical extension. Law and Society Review, 22(1), 163–189.
Saunders, J., Lundberg, R., Braga, A. A., Ridgeway, G., & Miles, J. (2015). A synthetic control approach to evaluating place-based crime interventions. Journal of Quantitative Criminology, 31(3), 413–434.
Schwarzer, G. (2007). Meta: An R package for meta-analysis. R News, 7(3), 40–45.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Belmont: Wadsworth, Cengage Learning.
Sherman, L. W. (1993). Defiance, deterrence, and irrelevance: A theory of the criminal sanction. Journal of Research in Crime and Delinquency, 30(4), 445–473.
Sherman, L., & Rogan, D. (1995). Effects of gun seizure on gun violence: ‘Hot spots’ patrol in Kansas City. Justice Quarterly, 12(4), 673–693.
Sherman, L. W., & Weisburd, D. (1995). General deterrent effects of police patrol in crime “hot spots”: A randomized, controlled trial. Justice Quarterly, 12(4), 625–648.
Skogan, W., & Frydl, K. (2004). Fairness and effectiveness in policing: The evidence. Washington DC: National Academies Press.
Sorg, E. T., Wood, J. D., Groff, E. R., & Ratcliffe, J. H. (2014). Boundary adherence during place-based policing evaluations: A research note. Journal of Research in Crime and Delinquency, 51(3), 377–393.
Stanojević, P., Jovanović, D., & Lajunen, T. (2013). Influence of traffic enforcement on the attitudes and behavior of drivers. Accident Analysis and Prevention, 52, 29–38.
Stuster, J., Sheehan, M., & Monford, G. (1997). Contributions of traffic enforcement to the war on crime: Case study of the grand prairie, Texas police department. Washington, D.C.: National Highway Traffic Safety Administration.
Telep, C. W., Mitchell, R. J., & Weisburd, D. (2014). How much time should the police spend at crime hot spots? Answers from a police agency directed randomized field trial in Sacramento, California. Justice Quarterly, 31(5), 905–933.
Thomas, F. D., Blomberg, R. D., Peck, R. C., Cosgrove, L. A., & Salzberg, P. M. (2008). Evaluation of a high visibility enforcement project focused on passenger vehicles interacting with commercial vehicles. Journal of Safety Research, 39(5), 459–468.
Trojanowicz, R. C. (1986). Evaluating a neighborhood foot patrol program: The Flint, Michigan, project. In D. P. Rosenbaum (Ed.), Community crime prevention: Does it work? Newbury Park: Sage.
Viechtbauer, W. (2010). Conducting meta-analyses in R with the metaphor package. Journal of Statistical Software, 36(3), 1–48.
Wasserstein, R. L., & Lazar, N. A. (2016). The ASA's statement on p-values: Context, process, and purpose. The American Statistician, Online Advance Publication, 70(2), 129–133.
Weisburd, D. (2005). Hot spots policing experiments and criminal justice research: Lessons from the field. The Annals of the American Academy of Political and Social Science, 599(1), 220–245.
Weisburd, D. (2008). Place-based policing. Washington: Police Foundation.
Weisburd, D. (2015). The law of crime concentration and the criminology of place. Criminology, 53(2), 133–157.
Weisburd, D., Telep, C., Hinkle, J., & Eck, J. (2010). Is problem-oriented policing effective in reducing crime and disorder? Criminology and Public Policy, 9, 139–172.
Weiss, A. (2013). Data-driven approaches to crime and traffic safety (DDACTS): An historical overview. Washington DC: NHSTA.
Weiss, A., & Freels, S. (1996). The effects of aggressive policing: The Dayton traffic enforcement experiment. American Journal of Police, 15, 45–64.
Weiss, A., & McGarrell, E.F. (1999). Traffic enforcement and crime: Another look. Police Chief, 66(7), 25–28.
Worden, R. E., & McLean, S. J. (2009). DDACTS in theory and practice. Albany: John F. Finn Institute for Public Safety.
Yannis, G., Papadimitriou, E., & Antoniou, C. (2007). Multilevel modelling for the regional effect of enforcement on road accidents. Accident Analysis and Prevention, 39(4), 818–825.
This research was supported grant number 2012-BJ-CX-K036 awarded by the Bureau of Justice Statistics, Office of Justice Programs, U.S. Department of Justice. This research was made possible by substantial assistance from the Michigan State Police. Points of view in this document are those of the author and do not necessarily represent the official position or policies of the US Department of Justice or the Michigan State Police. We would like to thank the editorial team and three anonymous reviewers for their helpful comments, and especially Michael Robbins and Jessica Saunders for their insights on implementing the methods described here.
About this article
Cite this article
Rydberg, J., McGarrell, E.F., Norris, A. et al. A quasi-experimental synthetic control evaluation of a place-based police-directed patrol intervention on violent crime. J Exp Criminol 14, 83–109 (2018). https://doi.org/10.1007/s11292-018-9324-8
- Synthetic control
- Quasi-experimental design
- Place-based policing
- Directed patrol
- Program evaluation