Skip to main content

Advertisement

SpringerLink
Log in

Further improving reporting in crime and justice: an addendum to Perry, Weisburd and Hewitt (2010)

  • Published:
Journal of Experimental Criminology Aims and scope Submit manuscript

Abstract

Objectives

To elaborate on guidelines designed to improve the descriptive validity of reports of criminological research, whatever research design, focusing in particular on the inclusion of items relating to the causal mechanisms through which interventions are believed to operate and to the setting, circumstances, and procedures in which measures are applied.

Methods

The literature on reporting guidelines in criminology and health is reviewed. Particular attention is paid to the SQUIRE guidelines, designed to improve reporting quality in the field of healthcare improvement.

Results

Criminological treatments of descriptive validity have focused on reporting quality in randomized controlled trials only, drawing on the CONSORT Statement, which is widely used in reports of medical trials. These guidelines, reflecting the field from which they originate, pay little attention to issues that are important in appraising crime and justice studies and for replication and scale-up efforts. Reporting guidelines items related to causal mechanisms and implementation are presented, drawing heavily on the SQUIRE guidelines.

Conclusions

Sloppy, incomplete reporting threatens to frustrate the advancement of criminological research and constrain its utility for informing policy and practice. The development of criminology-specific reporting guidelines to stimulate improvements in the descriptive validity of criminological research is welcomed. A plan for the future development of reporting guidelines in criminology is suggested.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. http://www.equator-network.org/home/

  2. www.consort-statement.org/consort-statement/overview0/

  3. The difference between the present CONSORT Statement and the 2001 version used by Perry et al. relates to the addition of three items: Registration, Protocol and Funding. These items are arguably less relevant to most criminological trials and are not pertinent to the aims of Perry et al. (2010).

  4. http://www.implementationscience.com/

  5. These items have been adapted slightly to match the purposes of this paper.

  6. http://www.campbellcollaboration.org/

References

  • Archer, M. (1995). Realist Social Theory. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Astbury, B., & Leeuw, F. L. (2010). Unpacking Black Boxes: Mechanisms and Theory Building in Evaluation. American Journal of Evaluation, 31(3), 363–381.

    Article  Google Scholar 

  • Bennett, T., Holloway, K., & Farrington, D. P. (2009). A Review of the Effectiveness of Neighborhood Watch. Security Journal, 22, 143–155.

    Article  Google Scholar 

  • Berk, R. A. (2005). Randomized experiments as the bronze standard. Journal of Experimental Criminology, 1, 417–433.

    Article  Google Scholar 

  • Berk, R. A., & Rossi, P. H. (1999). Thinking About Program Evaluation (2nd ed.). Thousand Oaks: Sage.

    Google Scholar 

  • Bhaskar, R. (1997). A Realist Theory of Science. London: Verso.

    Google Scholar 

  • Boudon, R. (1974). Education, Opportunity and Social Inequality. New York: Wiley.

    Google Scholar 

  • Bunge, M. (1999). The Sociology-Philosophy Connection. New Brunswick: Transaction Publishers.

    Google Scholar 

  • Cartwright, N. (1999). The Dappled World: A Study of the Boundaries of Science. Cambridge: Cambridge University Press.

    Google Scholar 

  • Cartwright, N. (2007). Hunting Causes and Using Them. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Davidoff, F., Batalden, P., Stevens, D., Ogrinc, G., & Mooney, S. (2008). Publication Guidelines for Improvement Studies in Health Care: Evolution of the SQUIRE Project. Annals of Internal Medicine, 149(9), 670–676.

    Google Scholar 

  • Davis, R., Weisburd, D., & Hamilton, E. (2007). Preventing Repeat Incidents of Family Violence: A Randomized Field Test of a Second Responder Program in Redlands, California. Washington DC: Police Foundation.

    Google Scholar 

  • Ekblom, P. (2002). From the Source to the Mainstream is Uphill: The Challenge of Transferring Knowledge of Crime Prevention Through Replication, Innovation and Anticipation. In N. Tilley (Ed.), Analysis for Crime Prevention (Crime Prevention Studies 13). Monsey: Criminal Justice Press.

    Google Scholar 

  • Farrington, D. P. (2003). Methodological quality standards for evaluation research. The Annals of the American Academy of Political and Social Science, 587(1), 49–68.

    Article  Google Scholar 

  • Farrington, D. P., & Welsh, B. C. (2005). Randomized experiments in criminology: what have we learned in the last two decades? Journal of Experimental Criminology, 1(1), 9–38.

    Article  Google Scholar 

  • Gasper, D. (2000). Evaluating the ‘logical framework approach’: Towards learning-oriented development evaluation. Public Administration and Development, 20, 17–28.

    Article  Google Scholar 

  • Grove, L. and Farrell, G. (in press). Once Bitten, Twice Shy? Repeat Victimization and its Prevention, in B. Welsh and D. Farrington (eds.) The Oxford Handbook of Crime Prevention, Oxford: Oxford University Press.

  • Hanmer, J. (2003). Mainstreaming Solutions to Major Problems: Reducing Repeat Domestic Violence. In K. Bullock & N. Tilley (Eds.), Crime Reduction and Problem-oriented Policing (pp. 252–284). Cullompton: Willan.

    Google Scholar 

  • Harré, R., & Madden, E. (1975). Causal Powers: A Theory of Natural Necessity. Oxford: Basil Blackwell.

    Google Scholar 

  • Hedstrom, P. (2005). Dissecting the Social. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Hedstrom, P., & Bearman, P. (2009). The Oxford Handbook of Analytic Sociology. Oxford: Oxford University Press.

    Google Scholar 

  • Henry, G., Mark, M., & Julnes, G. (1998). Realist Evaluation: An Emerging Theory in Support of Practice. New Directions for Evaluation (Vol. 78). San Francisco: Jossey-Bass.

    Google Scholar 

  • Hirschhorn, L. R., Ojikutu, B., & Rodriguez, W. (2007). Research for Change: Using Implementation Research to Strengthen HIV Care and Treatment Scale-Up in Resource-Limited Settings. The Journal of Infectious Diseases, 196, 516–22.

    Article  Google Scholar 

  • Hope, T., & Murphy, J. (1983). Problems of implementing crime prevention: The experience of a demonstration project. Howard Journal, 22, 38–50.

    Article  Google Scholar 

  • Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010). Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research. PLoS Biology, 8(6), e1000412.

    Article  Google Scholar 

  • Knutsson, J., & Clarke, R. V. (2006). Putting theory to work: Implementing situational prevention and problem-oriented policing (Crime Prevention Studies, Vol. 20). Monsey, NY: Criminal Justice Press.

    Google Scholar 

  • Lawson, T. (1997). Economics and Reality. London: Routledge.

    Book  Google Scholar 

  • Lum, C., & Yang, S.-M. (2005). Why do evaluation researchers in crime and justice choose non-experimental methods? Journal of Experimental Criminology, 1, 191–213.

    Article  Google Scholar 

  • MacKenzie, D. L. (2006). What works in corrections: reducing the criminal activities of offenders and delinquents. New York: Cambridge University Press.

    Book  Google Scholar 

  • Moher, D., Schulz, K. F., Altman, D., & for the CONSORT Group. (2001). The CONSORT Statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA, 285, 1987–1991.

    Article  Google Scholar 

  • Pawson, R. (2009). Evidence-based policy: a realist perspective. London: Sage Publications.

    Google Scholar 

  • Pawson, R., & Tilley, N. (1997). Realistic Evaluation. London: Sage Publications.

    Google Scholar 

  • Perry, A. E. (2010). Descriptive Validity and Transparent Reporting in Randomized Controlled Trials. In A. R. Piquero & D. Weisburd (Eds.), Handbook of Quantitative Criminology (Vol. Part 3, pp. 333–352). New York City: Springer Science.

    Chapter  Google Scholar 

  • Perry, A. E., & Johnson, M. (2008). Applying the Consolidated Standards of Reporting Trials (CONSORT)to studies of mental health provision for juvenile offenders: a research note. Journal of Experimental Criminology, 4(2), 165–185.

    Article  Google Scholar 

  • Perry, A. E., Weisburd, D., & Hewitt, C. (2010). Are criminologists describing randomized controlled trials in ways that allow us to assess them? Findings from a sample of crime and justice trials. Journal of Experimental Criminology, 6(3), 245–262.

    Article  Google Scholar 

  • Petrosino, A. J., Kiff, P., & Lavenberg, J. (2006). Research note: Randomized field experiments published in the British Journal of Criminology, 1960–2004. Journal of Experimental Criminology, 2, 99–111.

    Article  Google Scholar 

  • Rothstein, H. R. (2008). Publication bias as a threat to the validity of meta-analytic results. Journal of Experimental Criminology, 4, 61–81.

    Article  Google Scholar 

  • Sampson, R. J. (2006). How does community context matter? Social mechanisms and the explanation of crime rates. In P. O. Wikström & R. Sampson (Eds.), The explanation of crime (pp. 31–60). Cambridge: Cambridge University Press.

    Google Scholar 

  • Sampson, R. J. (2010). Gold Standard Myths: Observations on the Experimental Turn in Quantitative Criminology. Journal of Quantitative Criminology, 26, 489–500.

    Article  Google Scholar 

  • Sayer, A. (1992). Method in Social Science: A Realist Approach. London: Routledge.

    Google Scholar 

  • Sherman, L. W., Farrington, D. P., Welsh, B. C., & MacKenzie, D. L. (Eds.). (2002). Evidence based crime prevention. London: Routledge.

    Google Scholar 

  • Sidebottom, A., & Tilley, N. (2011). Improving Problem-Oriented Policing: The need for a new model? Crime Prevention and Community Safety, 13(2), 79–101.

    Article  Google Scholar 

  • Simera, I., Moher, D., Hoey, J., Schulz, K. F., & Altman, D. G. (2010). A catalogue of reporting guidelines for health research. European Journal of Clinical Investigation, 40(1), 35–53.

    Article  Google Scholar 

  • Sims, L. (2001). Neighborhood Watch: Findings from the 2000 British Crime Survey (Research Findings, Vol. 150). London: Home Office.

    Google Scholar 

  • Smith, M., Clarke, R., & Pease, K. (2002). Anticipatory benefits in crime prevention. In N. Tilley (Ed.), Analysis for Crime Prevention (Crime Prevention Studies, Vol. 13). Monsey, NY: Criminal Justice Press.

    Google Scholar 

  • Sparrow, M. (2011). Governing Science. New Perspectives in Policing. National Institute of Justice, Washington D.C. & The Program in Criminal Justice Policy & Management, Harvard Kennedy School, Cambridge, Massachusetts.

  • Tilley, N. (1993). After Kirkholt: Theory, methods and results of replication evaluations (Crime Prevention Unit Paper, Vol. 47). London: Home Office.

    Google Scholar 

  • Tilley, N. (2009). Crime Prevention. Willan: Cullompton.

    Google Scholar 

  • Tilley, N., & Hopkins, M. (1998). Business as Usual: An Evaluation of the Small Business and Crime Initiative (Police Research Series Paper, Vol. 95). London: Home Office.

    Google Scholar 

  • Van der Knaap, L. M., Leeuw, F. L., Bogaerts, S., & Nijssen, L. T. J. (2008). Combining Campbell standards and the realist evaluation approach – the best of two worlds? American Journal of Evaluation, 29(1), 48–57.

    Article  Google Scholar 

  • Vandenbroucke, J. P. (2009). Reporting Guidelines: STREGA, STROBE, STARD, SQUIRE, MOOSE, PRISMA, GNOSIS, TREND, ORION, COREQ, QUOROM, REMARK, and CONSORT: for whom does the guideline toll? Journal of Clinical Epidemiology, 62, 594–596.

    Article  Google Scholar 

  • Weisburd, D. (2003). Ethical practice and evaluation of interventions in crime and justice: the moral imperative for randomized trials. Evaluation Review, 27(3), 336–354.

    Article  Google Scholar 

  • Wilson, D. B. (2009). Missing a critical piece of the pie: simple document search strategies inadequate for systematic reviews. Journal of Experimental Criminology, 5(4), 429–440.

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank Paul Ekblom, Louise Grove, and the anonymous reviewers and editor for their useful comments on earlier drafts of this article.

Author information

Authors and Affiliations

  1. UCL Jill Dando Institute of Security and Crime Science, University College London, 35 Tavistock Square, London, England, WC1H 9EZ

    Aiden Sidebottom & Nick Tilley

Authors
  1. Aiden Sidebottom
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nick Tilley
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aiden Sidebottom.

Appendix

Appendix

Table 2 Key reporting guidelines for different study designs
Full size table
Table 3 SQUIRE Guidelines (Standards for QUality Improvement Reporting Excellence)
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sidebottom, A., Tilley, N. Further improving reporting in crime and justice: an addendum to Perry, Weisburd and Hewitt (2010). J Exp Criminol 8, 49–69 (2012). https://doi.org/10.1007/s11292-011-9128-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11292-011-9128-6

Keywords

Search

Navigation