Does Hot Spots Policing Have Meaningful Impacts on Crime? Findings from An Alternative Approach to Estimating Effect Sizes from Place-Based Program Evaluations

Abstract

Objectives

Prior meta analyses of hot spots policing show that the approach reduces crime, but report relatively small mean effect sizes based on Cohen’s d. The natural logarithm of the relative incidence rate ratio (log RIRR) has been suggested as a more suitable effect size metric for place-based studies that report crime outcomes as count data. We calculate the log RIRR for hot spots policing studies to assess whether it changes interpretation of hot spots policing’s impact on crime.

Methods

Cohen’s d and log RIRR effect size metrics were calculated for 53 studies representing 60 tests of hot spots policing programs. Meta-analytic techniques were used to compare the estimated impacts of hot spots policing on crime and investigate the influence of moderating variables using the two differing effect size metrics.

Results

The Cohen’s d meta-analysis revealed a “small” statistically significant mean effect size favoring hot spots policing in reducing crime outcomes at treatment places relative to control places (d = .12) of approximately 8.1%. In contrast, the log RIRR meta-analysis suggests that hot spots policing generated a more substantive 16% (d = .24) statistically significant crime reduction. The two metrics also produced differing rank orders in magnitudes of effect for the same studies.

Conclusion

Cohen’s d provides misleading results when used to calculate mean effect size in place based studies both in terms of the relative ranking of the magnitude of study outcomes, and in the interpretation of average impacts of interventions. Our analyses suggest a much more meaningful impact of hot spots policing on crime than previous reviews.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Notes

  1. 1.

    While some meta-analytic scholars had raised concerns about using Cohen’s d to represent program effects in studies that reported area-based crime count data (Farrington et al. 2007; Farrington and Welsh 2013), the general approach used in the ongoing hot spots policing systematic review was considered an acceptable methodology. Indeed, the effect size approach used in the hot spots policing review passed ongoing scientific scrutiny by Campbell Collaboration methods reviewers and high-quality social science journal peer reviewers.

  2. 2.

    In their meta-analysis of an English national quasi-experimental multi-site evaluation of the effects of closed-circuit television (CCTV), Farrington et al. (2007) derived this adjustment for overdispersion through a linear regression analysis of 70 sets of monthly crime counts in treatment, control, buffer, and police division areas. As they described (Farrington et al. 2007, pp. 36–37), “For each area in each year, the total number of crimes N was compared with V/N, where V is the estimated variance of the number of crimes (based on monthly numbers). In a Poisson process, V/N = 1. It was clear that V/N increased with the total number of crimes. The correlation between V/N and N was 0.77 (p < 0.0001). A linear regression analysis showed that V/N = 0.0008 * N + 1.2.”.

  3. 3.

    Wilson (2020) suggests the following formula to estimate the quasi-Poisson overdispersion parameter: \(\emptyset = \frac{1}{{\sum n_{k} - 2}} \sum \frac{{s_{k}^{2} \left( {n_{k} - 1} \right)}}{{\overline{{x_{k} }} }},\) where \(\overline{{x_{k} }}\) is the mean count (or rate) for the treatment and control areas both pretest and post-test, resulting in four means, sk is the standard deviation for each of the four mean counts, and nk is the number of counts contributing to each mean and standard deviation. For studies that reported the log IRR and its standard error, we used those metrics in the meta-analysis as these count regression models already adjusted for overdispersion.

  4. 4.

    When included studies reported an aggregate crime category (such as total incidents or total calls for service), we calculated our effect size measures based on the provided metrics. In the absence of an aggregate crime category, we combined all reported outcomes into an overall average effect size statistic (designated “combined” in the meta-analysis forest plots).

  5. 5.

    Q = 67.516, degrees of freedom = 27, p < 0.001, I2 = 60.009.

  6. 6.

    Q = 78/944, degrees of freedom = 59, p < 0.05, I2 = 25.264.

  7. 7.

    For the Cohen’s d meta-analysis, the between group Q = 20.573, degrees of freedom = 1, p < 0.001. For the log RIRR meta-analysis, the between group Q = 61.658, degrees of freedom = 1, p < 0.001.

  8. 8.

    https://www.rand.org/well-being/justice-policy/centers/quality-policing/cost-of-crime.html (accessed May 22, 2020).

  9. 9.

    While the identification process yielded 56 distinct violent crime hot spots, the Jersey City Police Department’s Violent Crimes Unit only had the resources to implement the problem-oriented policing intervention at 12 treatment locations that were matched in pairs to 12 control locations (one member of each pair was allocated to treatment and control conditions). As such, the resulting randomized controlled trial only considered N = 24 total violent crime hot spots. For discussion of the hot spots identification process and the implementation of the randomized controlled trial, please see Braga (1997) and Braga et al. (1999). We used FBI UCR citywide counts to estimate the number of aggravated assaults because the randomized experiment included simple assaults and aggravated assaults in one total assault incident outcome.

  10. 10.

    https://www.ucrdatatool.gov/Search/Crime/Local/RunCrimeJurisbyJurisLarge.cfm (accessed May 22, 2020).

  11. 11.

    https://www2.fbi.gov/ucr/cius_04/documents/CIUS_2004_Section2.pdf (accessed May 22, 2020).

  12. 12.

    https://www.rand.org/well-being/justice-policy/centers/quality-policing/cost-of-crime.html (accessed May 17, 2020).

  13. 13.

    When we limited our main effects meta-analysis to the hot spots policing studies included in the Blueprints review (Buckley et al. 2020) that included the appropriate information to estimate log RIRR effect size metrics (N = 18), our meta-analysis estimated a 10.3% reduction in crime at treatment places relative to control places. Using the violent crime counts and cost estimates presented in the main text, the much more restrictive Blueprints review suggests that the hot spots policing intervention would have generated roughly $7.8 million in cost savings if the program was applied to all violent crime hot spots in Jersey City for one year and $2.3 million if applied to violence in all crime and disorder hot spots in Lowell for one year.

References

  1. Ariel B, Partridge H (2017) Predictable policing: Measuring the crime control benefits of hotspots policing at bus stops. J Quant Criminol 33:809–833

    Article  Google Scholar 

  2. Ariel B, Sherman LW, Newton M (2020) Testing hot-spots police patrols against no-treatment controls: temporal and spatial deterrence effects in the London Underground experiment. Criminology 58:101–128

    Article  Google Scholar 

  3. Ariel B, Weinborn C, Sherman LW (2016) “Soft” policing at hot spots—do police community support officers work? A randomized controlled trial. J Exp Criminol 12:277–317

    Article  Google Scholar 

  4. Attermann CV (2017) Investigating hot spots policing. Ministry of Justice Research Office, Copenhagen

    Google Scholar 

  5. Beck JW (2010) The Effects of Directed Patrols in Unincorporated Areas: A Case Study of the Data-Driven Approach to Crime and Traffic Safety. University of Nevada, Reno, NV

    Google Scholar 

  6. Berk R, MacDonald JM (2008) Overdispersion and Poisson regression. J Quant Criminol 24:269–284

    Article  Google Scholar 

  7. Bond BJ, Hajjar L (2013) Measuring congruence between property crime problems and response strategies: enhancing the problem-solving process. Police Q 16:323–338

    Article  Google Scholar 

  8. Braga AA (1997) Solving violent crime problems: an evaluation of the Jersey City police department’s pilot program to control violent places, Ph.D. diss., Rutgers University, University Microfilms International, Ann Arbor, MI

  9. Braga AA (2001) The effects of hot spots policing on crime. Annals 578:104–125

    Google Scholar 

  10. Braga AA (2005) Hot spots policing and crime prevention: a systematic review of randomized controlled trials. J Exp Criminol 1:317–342

    Article  Google Scholar 

  11. Braga A, Bond B (2008) Policing crime and disorder hot spots: a randomized controlled trial. Criminology 46:577–608

    Article  Google Scholar 

  12. Braga A, Hureau D, Papachristos A (2011) An ex-post-facto evaluation framework for place-based police interventions. Eval Rev 35:592–626

    Article  Google Scholar 

  13. Braga AA, Papachristos AV, Hureau DM (2014) The effects of hot spots policing on crime: an updated systematic review and meta-analysis. Just Q 31:633–663

    Article  Google Scholar 

  14. Braga AA, Turchan B, Papachristos AV, Hureau DM (2019) Hot spots policing of small geographic areas effects on crime. Campbell Syst Rev 15:e1046

    Google Scholar 

  15. Braga AA, Weisburd D (2010) Policing problem places: crime hot spots and effective prevention. Oxford University Press, New York

    Book  Google Scholar 

  16. Braga A, Weisburd D, Waring E, Mazerolle LG, Spelman W, Gajewski F (1999) Problem-oriented policing in violent crime places: a randomized controlled experiment. Criminology 37:541–580

    Article  Google Scholar 

  17. Bryant KM, Collins G, Villa J (2014) Data driven approaches to crime and traffic safety: Shawnee, Kansas 2010–2013. US Bureau of Justice Assistance, Washington, DC

    Google Scholar 

  18. Buckley P, Cook T, Elliott D, Gardner F, Gottfredson D, Hawkins JD, Hedges LV, Murry V, Tolan P, Tipton E, Vollmer L, Wadhwa M (2020) Does hot spots policing reduce crime? An alternative interpretation based on a meta-analysis of randomized experiments, University of Colorado, Boulder, CO

  19. Clarke RV, Weisburd D (1994) Diffusion of crime control benefits: observations on the reverse of displacement. Crime Prev Stud 2:165–184

    Google Scholar 

  20. Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Lawrence Erlbaum, Hillsdale

    Google Scholar 

  21. Cook T, Cooper H, Cordray D, Hartmann H, Hedges LV, Light R, Louis T, Mosteller F (1992) Meta-analysis for explanation: a casebook. Russell Sage Foundation, New York

    Google Scholar 

  22. Cooper H, Hedges LV (eds) (1994) The handbook of research synthesis. Russell Sage Foundation, New York

    Google Scholar 

  23. Dario LM (2016) Crime at convenience stores: assessing an in-depth problem-oriented policing initiative, Ph.D. diss., Arizona State University, Phoenix, AZ

  24. Farrington D, Gill M, Waples S, Argomaniz J (2007) The effects of closed-circuit television on crime: meta-analysis of an English national quasi-experimental multi-site evaluation. J Exp Criminol 3:21–38

    Article  Google Scholar 

  25. Farrington D, Petrosino A (2001) The Campbell collaboration crime and justice group. Annals 578:35–49

    Google Scholar 

  26. Farrington D, Welsh B (2013) Measuring effect size in meta-analysis, with special reference to area-based crime prevention programmes and the effects of closed-circuit television on crime. In: Kuhn A, Margot P, Aebi M, Schwarzenegger C, Donatsch A, Jositsch D (eds) Criminology, criminal policy and criminal law in international perspective. Stämpfli Verlag, Bern, Switzerland, pp 75–89

    Google Scholar 

  27. Fielding M, Jones V (2012) ‘Disrupting the optimal forager’: Predictive risk mapping and domestic burglary reduction in Trafford, Greater Manchester. Int J Police Sci Mgmt 14:30–41

    Article  Google Scholar 

  28. Gerell M (2016) Hot spot policing with actively monitored CCTV cameras: does it reduce assaults in public places? Int Crim Just Rev 26:187–201

    Article  Google Scholar 

  29. Groff ER, Ratcliffe JH, Haberman CP, Sorg ET, Joyce NM, Taylor RB (2015) Does what police do at hot spots matter? The Philadelphia policing tactics experiment. Criminology 53:23–53

    Article  Google Scholar 

  30. Hasselblad V, Hedges LV (1995) Meta-analysis of screening and diagnostic tests. Psych Bull 117:167–178

    Article  Google Scholar 

  31. Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M, Welch V (2019) Cochrane handbook for systematic reviews of interventions, https://www.training.cochrane.org/handbook. Accessed 24 May 2020

  32. Hinkle J, Weisburd D, Telep C, Peterson K (2020) Problem-oriented policing for reducing crime and disorder: an updated systematic review and meta-analysis, unpublished manuscript submitted to the Campbell collaboration

  33. Johnson SD, Tilley N, Bowers KJ (2015) Introducing EMMIE: an evidence rating scale to encourage mixed-method crime prevention synthesis reviews. J Exp Criminol 11:459–473

    Article  Google Scholar 

  34. Kennedy L, Caplan J, Piza E (2015) A multi-jurisdictional test of risk terrain modeling and a place-based evaluation of environmental risk-based patrol deployment strategies. Rutgers University, Newark, NJ

    Google Scholar 

  35. Kochel TR, Burruss G, Weisburd D (2015) St. Louis county hot spots in residential areas (SCHIRA) final report: assessing the effects of hot spots policing strategies on police legitimacy, crime, and collective efficacy, US Department of Justice, Washington, DC

  36. Koper CS, Lum C, Hibdon J (2015) The uses and impacts of mobile computing technology in hot spots policing. Eval Rev 39:587–624

    Article  Google Scholar 

  37. Koper CS, Taylor BG, Woods DJ (2013) A randomized test of initial and residual deterrence from directed patrols and use of license plate readers at crime hot spots. J Exp Criminol 9:213–244

    Article  Google Scholar 

  38. Lipsey MW (1990) Design sensitivity: statistical power for experimental research. Sage Publications, Thousand Oaks, CA

    Google Scholar 

  39. Lipsey MW (2000) Statistical conclusion validity for intervention research: a significant (p < .05) problem. In: Bickman L (ed) Validity and social experimentation: Donald Campbell’s legacy. Sage Publications, Thousand Oaks, pp 101–120

    Google Scholar 

  40. Lipsey M, Wilson DB (2001) Practical meta-analysis. Sage Publications, Thousand Oaks, CA

    Google Scholar 

  41. Lum C, Hibdon J, Cave B, Koper CS, Merola L (2011) License plate reader (LPR) police patrols in crime hot spots: an experimental evaluation in two adjacent jurisdictions. J Exp Criminol 7:321–345

    Article  Google Scholar 

  42. Manning M, Johnson SD, Tilley N, Wong GT, Vorsina M (2016) Economic analysis and efficiency in policing, criminal justice and crime reduction: What works? Springer, New York

    Book  Google Scholar 

  43. Marklund F, Holmberg S (2015) Camera monitoring in Stureplan and civic square: full report. Brå, Stockholm, Sweden

    Google Scholar 

  44. Marklund F, Merenius S (2014) Policing crime “hot spots”: results and experiences from two projects against personal abuse and abuse. Brå, Stockholm, Sweden

    Google Scholar 

  45. Martinez NN (2013) Team policing revisited: a quasi-experimental evaluation in Las Vegas. University of Nevada, Las Vegas, NV, Nevada

    Google Scholar 

  46. Mazeika DM (2014) General and specific displacement effects of police crackdowns: criminal events and “local” criminals, Ph.D. diss., University of Maryland, College Park, MD

  47. Mazerolle L, Price J, Roehl J (2000) Civil remedies and drug control: a randomized field trial in Oakland, California. Eval Rev 24:212–241

    Article  Google Scholar 

  48. Nagin DS, Sampson RJ (2019) The real gold standard: measuring counterfactual worlds that matter most to social science and policy. Ann Rev Criminol 2:123–145

    Article  Google Scholar 

  49. Phillips SW, Wheeler A, Kim D (2016) The effect of police paramilitary unit raids on crime at micro-places in Buffalo, New York. Int J Police Sci Mgmt 18:206–219

    Article  Google Scholar 

  50. Piza E, Caplan JM, Kennedy LW, Gilchrist AM (2015) The effects of merging proactive CCTV monitoring with directed police patrol: a randomized controlled trial. J Exp Criminol 11:43–69

    Article  Google Scholar 

  51. Piza E, O’Hara BA (2014) Saturation foot-patrol in a high-violence area: a quasi-experimental evaluation. Just Q 31:693–718

    Article  Google Scholar 

  52. Ratcliffe JH, Breen C (2011) Crime diffusion and displacement: measuring the side effects of police operations. Prof Geograph 63:230–243

    Article  Google Scholar 

  53. Ratcliffe J, Taniguchi T, Groff E, Wood J (2011) The Philadelphia foot patrol experiment: a randomized controlled trial of police patrol effectiveness in violent crime hot spots. Criminology 49:795–831

    Article  Google Scholar 

  54. Ratcliffe J, Taylor RB, Askey AP, Thomas K, Grasso J, Bethel K (2020) The Philadelphia predictive policing experiment. J Exp Criminol. https://doi.org/10.1007/s11292-019-09400-2

    Article  Google Scholar 

  55. Reppetto T (1976) Crime prevention and the displacement phenomenon. Crime Delinq 22:166–177

    Article  Google Scholar 

  56. Rosenfeld R, Deckard MJ, Blackburn E (2014) The effects of directed patrol and self-initiated enforcement on firearm violence: a randomized controlled study of hot spot policing. Criminology 52:428–449

    Article  Google Scholar 

  57. Rosenthal R (1994) Parametric measures of effect size. In: Cooper H, Hedges LV (eds) The handbook of research synthesis. Russell Sage Foundation, New York, pp 231–244

    Google Scholar 

  58. Rossi PH (1987) The iron law of evaluation and other metallic rules. Res Soc Prob Publ Pol 4:3–20

    Google Scholar 

  59. Rydberg J, McGarrell EF, Norris A, Circo G (2017) Applying synthetic control to evaluate the impact of data driven approaches to crime and traffic safety on violent crime, unpublished manuscript

  60. Santos RB, Santos RG (2016) Offender-focused police intervention in residential burglary and theft from vehicle hot spots: a partially blocked randomized control trial. J Exp Criminol 12:373–402

    Google Scholar 

  61. Santos RG, Santos RB (2015) An ex post facto evaluation of tactical police response in residential theft from vehicle micro-time hot spots. J Quant Criminol 31:679–698

    Google Scholar 

  62. Sedelmaier CM, Hipple NK (2016) New Haven, Connecticut Smart Policing Initiative: employing evidence-based policing strategies to engage the community and reduce crime. US Bureau of Justice Assistance, Washington, DC

    Google Scholar 

  63. Shadish WR, Cook TD, Campbell DT (2002) Experimental and quasi-experimental designs for generalized causal inference. Wadsworth Cengage Learning, Belmont, CA

    Google Scholar 

  64. Shadish WR, Ragsdale K (1996) Random versus nonrandom assignment in controlled experiments: Do you get the same answer? J Consult Clin Psychol 64:1290–1305

    Google Scholar 

  65. Sherman L, Buerger M, Gartin P (1989) Beyond dial-a-cop: a randomized test of repeat call policing (RECAP). Crime Control Institute, Washington, DC

    Google Scholar 

  66. Sherman L, Gartin P, Buerger M (1989) Hot spots of predatory crime: routine activities and the criminology of place. Criminology 27:27–56

    Article  Google Scholar 

  67. Sherman L, Rogan D (1995a) Deterrent effects of police raids on crack houses: a randomized controlled experiment. Just Q 12:755–782

    Article  Google Scholar 

  68. Sherman L, Rogan D (1995b) Effects of gun seizures on gun violence: ‘Hot spots’ patrol in Kansas City. Just Q 12:673–694

    Article  Google Scholar 

  69. Sherman L, Weisburd D (1995) General deterrent effects of police patrol in crime hot spots: a randomized controlled trial. Just Q 12:625–648

    Article  Google Scholar 

  70. Sherman L, Williams S, Ariel B, Strang LR, Wain N, Slothower M, Norton A (2014) TTPS hotspot experiment: Murder, wounding, shooting data graphs. Unpublished report

  71. Sorg ET (2015) An ex post facto evaluation of the Philadelphia GunStat model, Ph.D. diss., Temple University, Philadelphia, PA

  72. Sviridoff M, Sadd S, Curtis R, Grinc R (1992) The neighborhood effects of street-level drug enforcement: tactical narcotics teams in New York. Vera Institute of Justice, New York

    Google Scholar 

  73. Taylor B, Koper C, Woods D (2011) A randomized controlled trial of different policing strategies at hot spots of violent crime. J Exp Criminol 7:149–181

    Article  Google Scholar 

  74. Telep CW, Mitchell RJ, Weisburd D (2014) How much time should the police spend at crime hot spots? Answers from a police agency directed randomized field trial in Sacramento, California. Just Q 31:905–933

    Article  Google Scholar 

  75. Weisburd D, Braga AA (2019) Hot spots policing as a model for police innovation. In: Weisburd D, Braga AA (eds) Police innovation: Contrasting perspectives, 2nd edn. Cambridge University Press, New York, pp 291–313

    Google Scholar 

  76. Weisburd D, Bushway S, Lum C, Yang S (2004) Trajectories of crime at places: a longitudinal study of street segments in the city of Seattle. Criminology 42:283–321

    Article  Google Scholar 

  77. Weisburd D, Green L (1995) Policing drug hot Spots: the Jersey City DMA experiment. Just Q 12:711–736

    Article  Google Scholar 

  78. Weisburd D, Eck J, Braga A, Telep C, Cave B et al (2016) Place matters: criminology for the 21st century. Cambridge University Press, New York

    Book  Google Scholar 

  79. Weisburd D, Hinkle JC, Famega C, Ready J (2012) Legitimacy, fear and collective efficacy in crime hot spots: assessing the impacts of broken windows policing strategies on citizen attitudes. US National Institute of Justice, Washington, DC

    Google Scholar 

  80. Weisburd D, Lum C (2005) The diffusion of computerized crime mapping in policing: linking research and practice. Police Prac Res 6:419–434

    Article  Google Scholar 

  81. Weisburd D, Lum C, Petrosino A (2001) Does research design affect study outcomes in criminal justice? Annals 578:50–70

    Google Scholar 

  82. Weisburd D, Majmundar MK (eds) (2018) Proactive policing: effects on crime and communities, Committee on proactive policing, National Academies Press, Washington, DC

  83. Weisburd D, Mastrofski S, McNally AM, Greenspan R, Willis J (2003) Reforming to preserve: compstat and strategic problem solving in American policing. Criminol Publ Pol 2:421–456

    Article  Google Scholar 

  84. Weisburd D, Wyckoff L, Ready J, Eck J, Hinkle J, Gajewski F (2006) Does crime just move around the corner? A controlled study of spatial displacement and diffusion of crime control benefits. Criminology 44:549–592

    Article  Google Scholar 

  85. Welsh B, Farrington D (2002) Crime prevention effects of closed circuit television: a systematic review. Home Office, London

    Google Scholar 

  86. Welsh B, Peel M, Farrington D, Elffers H, Braga A (2011) Research design influence on study outcomes in crime and justice: a partial replication with public area surveillance. J Exp Criminol 7:183–198

    Article  Google Scholar 

  87. Wheeler AP, Phillips SW (2018) A quasi-experimental evaluation using roadblocks and automatic license plate readers to reduce crime in Buffalo. NY Sec J 31:190–207

    Article  Google Scholar 

  88. Williams SA (2015) Do visits or time spent in hot spots patrol matter most? A randomized control trial in the West Midlands Police. Fitzwilliam College, Cambridge

    Google Scholar 

  89. Wilson DB (2001) Meta-analytic methods for criminology. Annals 578:71–89

    Google Scholar 

  90. Wilson, D. B. (2020). The relative incident rate ratio effect size for count-based impact evaluations: When an odds ratio is not an odds ratio, unpublished manuscript.

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Anthony A. Braga.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Braga, A.A., Weisburd, D.L. Does Hot Spots Policing Have Meaningful Impacts on Crime? Findings from An Alternative Approach to Estimating Effect Sizes from Place-Based Program Evaluations. J Quant Criminol (2020). https://doi.org/10.1007/s10940-020-09481-7

Download citation

Keywords

  • Hot spots policing
  • Meta-analysis
  • Effect size
  • Cohen’s d
  • Log RIRR