Understanding the time-course of an intervention’s mechanisms: a framework for improving experiments and evaluations
The crime prevention evaluation literature has identified several potential side effects of interventions. These often-unintended consequences occur at different stages of prevention processes, including before official start dates. They can improve or reduce intervention impacts. Evaluations using before-and-after designs with or without controls can fail to identify these effects. We describe a longitudinal framework to guide the design and evaluation of interventions that can account for these side effects when causal mechanisms are better understood.
Our time-course framework provides a comprehensive assessment of the prevention process. Using place-based examples as illustrations, it builds on previously identified temporal benefits and backfires—such as anticipatory benefits, residual deterrence, and initial backfire—that have never been systematically organized into a single framework. We show how our framework can be incorporated into the EMMIE framework for assessing prevention utility.
The proposed time-course framework links together all temporal effects, their underlying mechanisms, and shows how they can vary by context.
The framework suggests that considering all decisions within these timelines will be more cost-effective and produce greater crime reductions in the long run. By considering the mechanisms that can be triggered at various points in an intervention’s time-course, we can better design experiments to test them and generate stronger evaluations of programs.
KeywordsCrime prevention policy EMMIE framework Initial backfire Intervention time-course Program evaluation
We would like to thank the three anonymous reviewers for their comments on this manuscript. Their insights lead to substantial improvements in the final draft. This research was supported by the Social Sciences and Humanities Council of Canada (SSHRC).
- Bowers, K. J., & Johnson, S. D. (2004). Who commits near repeats? A test of the boost explanation. Western Criminology Review, 5(3), 12–24.Google Scholar
- Cartwright, N., & Hardie, J. (2012). Evidence-based policy: A practical guide to doing it better. New York, NY: Oxford University Press.Google Scholar
- Clarke, R. V., & Weisburd, D. (1994). Diffusion of crime control benefits: observations on the reverse of displacement. Crime Prevention Studies, 2, 165–184.Google Scholar
- Eck, J. E., & Wartell, J. (1998). Improving the management of rental properties with drug problems: a randomized experiment. In L. G. Mazerolle & J. Roehl (Eds.), Civil remedies and crime prevention (crime prevention studies) (Vol. 9, pp. 161–185). Monsey, NY: Criminal Justice Press.Google Scholar
- Farrell, G., Bowers, K. J., & Johnson, S. D. (2005). Cost-benefit analysis for crime science: making cost-benefit analysis useful through a portfolio of outcomes. In M. Smith & N. Tilley (Eds.), Crime science: new approaches to preventing and detecting crime (pp. 56–84). Portland, OR: Willan Publishing.Google Scholar
- Gambetta, D. (1998). Concatenations of mechanisms. In P. Hedstrom & R. Swedberg (Eds.), Social mechanisms: an analytical approach to social theory. Cambridge University Press.Google Scholar
- Johnson, S. D. & Bowers, K. (2005) Using publicity for preventive purposes, in Nick Tilley (Ed) Handbook of crime prevention: theory, policy and practice (first edition). London: Willan.Google Scholar
- Merton Metropolitan Police (2015). A guide to alleygates. Retrieved from: January 12, 2018. https://www.bexley.gov.uk/sites/bexley-cms/files/A-guide-to-Alleygates.pdf
- Millie, A. & Hough, M. (2004). Assessing the impact of the reducing burglary initiative in southern England and Wales (Home Office Online Report 42/04). London: Home Office.Google Scholar
- Pawson, R., & Tilley, N. (1997). Realistic evaluation. London: Sage.Google Scholar
- Ross, H. L. (1981). Deterring the drinking driver: legal policy and social control. Lexington, Mass: Health.Google Scholar
- Sidebottom, A., Tompson, L., Thornton, A., Bullock, K., Tilley, N., Bowers, K., & Johnson, S. D. (2017). Gating alleys to reduce crime: a meta-analysis and realist synthesis. Justice Quarterly, 1–32.Google Scholar
- Smith, M. J., Clarke, R. V., & Pease, K. (2002). Anticipatory benefits in crime prevention. Crime Prevention Studies, 13, 71–88.Google Scholar
- Tompson, L. Belur, J, Thornton, A., Bowers, K., Johnson, S., Sidebottom, A. Tilley, N. and Laycock, G. (2019). Taking Stock of Systematic Reviews in Crime Reduction: An Evidence Appraisal Using the EMMIE Framework. Under Review.Google Scholar