Skip to main content
SpringerLink
Log in

Descriptive Validity and Transparent Reporting in Randomised Controlled Trials

  • Chapter
  • First Online:
Handbook of Quantitative Criminology

Abstract

While the concept of validity has been around for a number of decades, descriptive validity has been considered to a lesser extent in criminal justice research. In this chapter, we illustrate and evaluate several examples of descriptive validity in randomized controlled trials conducted in criminal justice. In particular, we introduce the idea of the CONSORT Statement, which was originally developed in healthcare to assess the descriptive validity of trials. This Statement was developed originally over concerns about the appropriateness of synthesising trials together in meta-analyses with differing levels of descriptive validity. We then go on to demonstrate the use of descriptive validity, using two recent studies. We examine examples of descriptive validity which report poorly in criminal justice trials and provide examples of good practice.Throughout this chapter, we emphasize the importance of descriptive validity to criminal justice researchers in upholding the quality of research within the field. A comparison and discussion about the applicability of the CONSORT Statement with criminal justice and healthcare trials suggests that some amendments may need to be made to make relevance to trials conducted in criminal justice. Generally adoption of a CONSORT-like Statement in criminal justice is thought to improve the reporting of descriptive validity in future research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    PsychINFO, Applied Social Sciences Index and Abstracts, International Bibliography of the Social Sciences, Social Policy and Practice, Social Services Abstracts, Sociological Abstracts, ERIC, Criminal Justice Abstracts, PAIS International, C2 SPECTR, Swets-Wise, DARE and Cochrane Database of Systematic Reviews.

  2. 2.

    Department of Health, Department of Education and Skills, Ministry of Justice, Joseph Rowntree Foundation, Royal College of Psychiatrists, Youth Justice Board, Policy Studies Institute, Mental Health Foundation, Young Minds and NACRO.

References

  • Altman DG (2001) The revised CONSORT Statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 134:663–694

    Google Scholar 

  • Altman DG (2005) Endorsement of the CONSORT statement by high impact medical journal: a survey of instructions for authors. Br Med J 330:1056–1057

    Article  Google Scholar 

  • Borduin CM, Mann BJ, Cone LT, Henggeler SW, Ficci BR, Blaske DM et al. (1995) Multisystemic treatment of serious juvenile offenders: long term prevention of criminality and violence. J Consult Clin Psychol 63:569–578

    Article  Google Scholar 

  • Boutron I, Moher D, Tugwell P, Giraudeau B, Poiraudeau S, Nizard R, Ravaud P (2005) A checklist to evaluate a report of a nonpharmocological trial (CLEAR NPT) was developed using consensus. J Clin Epidemiol 58(10):1233–12240

    Article  Google Scholar 

  • Boutron I, Estellat C, Guittet L, Dechartres A, Sackett DL, Hróbjartsson A et al. (2006) Methods of blinding in reports of randomised controlled trials assessing pharmacologic treatments: a systematic review. PLoS Med 3(10):1931–1939

    Article  Google Scholar 

  • Boruch RF (1997) Randomized field experiments for planning and evaluation: a practical guide. Sage Publications, Newbury Park, CA

    Google Scholar 

  • Campbell DT (1957) Factors relevant to the validity of experiments in social settings. Psychol Bull 54:297–312

    Article  Google Scholar 

  • Campbell DT & Stanley JC (1963) Experimental and quasi-experimental designs for research on teaching. In: Gage NL (ed) Handbook of research on teaching. Chicago, Rand McNally, 171–246

    Google Scholar 

  • Campbell FA, Ramey CT, Pungello E, Sparling J, Miller-Johnson S (2002) Early childhood education: young adult outcomes from the Abercedarian project. Appl Dev Sci 6:42–57

    Article  Google Scholar 

  • Devereaux PJ, Manns BJ, Ghali WA, Quan H, Guyatt GH (2002) The reporting of methodological factors in randomized controlled trials and the association with a journal policy to promote adherence to the Consolidated Standards of Reporting Trials (CONSORT) checklist. Control Clin Trials 23:380–388

    Article  Google Scholar 

  • Egger M, Smith GD (1998) Bias in location and selection of studies. Br Med J 316:61–66

    Google Scholar 

  • Emerson JD, Burdick E, Hoaglin DC, Mosteller F, Chalmers TC (1990) An empirical study of the possible relation of treatment differences to quality scores in controlled randomized clinical trials. Control Clin Trials 11:339–352

    Article  Google Scholar 

  • Farrington DP (2003) Methodological quality standards for evaluation research. Ann Am Acad Pol Soc Sci 587:49–68

    Article  Google Scholar 

  • Farrington DP, Welsh BC (2005) Randomized experiments in criminology: what have we learned in the last two decades? J Exp Criminol 1(1):9–38

    Article  Google Scholar 

  • Fiorucci S, Mencarelli A, Lechi A, Renga B et al. (2004) Co administration of nitric oxide aspirin prevents platelet and monocyte activation and protects against gastric damage induced by aspirin in humans. J Am Coll Cardiol 44:635–641

    Article  Google Scholar 

  • Francis CW, Berkowitz SD, Comp PC, Lieberman JR Ginsberg JS et al. (2003) Comparison of ximelagatran and warfarin for the prevention of venous thromboembolism after total knee replacement. N Engl J Med 349: 1703–1712

    Google Scholar 

  • Freemantle N (2001) Interpreting the results of secondary end points and subgroup analyses in clinical trials: should we lock the crazy aunt in the attic? Br Med J 322(7292):989–991

    Article  Google Scholar 

  • Graf J, Doig GS, Cook DJ, Vincent JL, Sibbald WJ (2002) Randomized controlled trials in sepsis: has methodological quality improved over time? Crit Care Med 30(2):461–472

    Article  Google Scholar 

  • Grossman JB, Tierney JP (1998) Does mentoring work? An impact study of the Big Brothers Big Sisters Program. Eval Rev 22:403–426

    Article  Google Scholar 

  • Hewitt C, Hahn S, Torgerson DJ, Watson J, Bland MJ (2005) Adequacy and reporting of allocation concealment: review of recent trials published in four general medical journals. Br Med J 330:1057–1058

    Article  Google Scholar 

  • Hopewell S, Altman DG, Moher D, Schulz KF (2008) Endorsement of the CONSORT statement by high impact factor medical journals: a survey of journal editors and journals ‘Instructions to Authors’. Trials 9:20

    Article  Google Scholar 

  • Ioannidis JP, Cappelleri JC, Lau J (1998) Issues in comparisons between meta-analyses and large trials. JAMA 279:1089–1093

    Article  Google Scholar 

  • Jüni P, Altman DG, Egger M (2001) Assessing the quality of controlled clinical trials. In: Egger M, Davey-Smith G, Altman DG (eds) Systematic reviews in health care: meta-analysis in context, 2nd ed. BMJ Books, London

    Google Scholar 

  • Jüni P, Holenstein F, Sterne J, Bartlett C, Egger M (2002) Direction and impact of languate bias in meta analyses of controlled trials: empirical study. International Journal of Epidemiology 31:115–123

    Article  Google Scholar 

  • Jüni P, Tallon D, Egger M (2000) ‘Garbage in – garbage out’? Assessment of the quality of controlled trials in meta-analyses published in leading journals. In: Proceedings of the 3rd symposium on systematic reviews: beyond the basics. St. Catherines College, Centre for Statistics in Medicine, Oxford

    Google Scholar 

  • Kjaergard LL, Villumsen J, Gluud C (1999) Quality of randomised clinical trials affects estimates of intervention efficacy. In: Proceedings of the 7th Cochrane Colloquium. Universitia S. Tommaso D’Aquino Rome. Milan, Centrro Cochrane Italiano, poster B10

    Google Scholar 

  • Kjaergard LL, Nikolova J, Gluud C (1999) Reported methodological quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med 135:982–989

    Google Scholar 

  • Kjaergard LL, Villumsen J, Gluud C (2001) Reported methodological quality and discrepancies between large and small randomized trials in meta-analyses. Annals of Intern Medicine 135:982–989

    Google Scholar 

  • Lao L, Bergman S, Hamilton GR, Langenberg P, Berman B (1999) Evaluation of acupuncture for pain control after oral surgery. Arch Otolaryngol Head Neck Surg 125:567–572

    Google Scholar 

  • Latessa E, Moon (1992) The effectiveness of acupuncture in an outpatient drug treatment program, Journal of Contemporary Criminal Justice 8(4):317–331

    Google Scholar 

  • Lösel F, Köferl P (1989) Evaluation research on correctional treatment in West Germany: a meta-analysis. In: Wegener H, Lösel F, Haisch J (Eds) Criminal behavior and the justice system: psychological perspectives. Springer-Verlag, New York, pp 334–355

    Google Scholar 

  • Moher D, Jadad AR, Nichol G Penman M, Tugwell M, Wash S (1995) Assessing the quality of RCT: an annotated bibliography of scales and checklists. Control Clin Trials 16:62–73

    Google Scholar 

  • Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP (1998) Does quality of reports of randomized trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 352:609–613

    Article  Google Scholar 

  • Moher D, Cook DJ, Jadad AR, Tugwell P, Moher M, Jones A, et al. (1999) Assessing the quality of reports of randomised trials: implications for the conduct of meta-analyses. Health Technol Assess 3(i–iv):1–98

    Google Scholar 

  • Moher D, Schulz KF, Altman D, for the CONSORT Group (2001a) The CONSORT Statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 285:1987–1991

    Article  Google Scholar 

  • Moher D, Jones A, Lepage L (2001b) Use of the CONSORT statement and quality of reports of randomized trials: a comparative before and after evaluation. JAMA 285:1992–1995

    Article  Google Scholar 

  • Moher D, Altman D, Schulz K, for the CONSORT Group. The CONSORT statement: revised recommendation for improving the quality of reports of parallel-group randomised trials [Chinese]. Chin J Evid-Based Med 2005; 5(9):702–707

    Google Scholar 

  • Needles K, James-Burdumy S, Burghardt J (2005) Community case management for former jail inmates: its impacts on re-arrest, drug use and HIV risk. J Urban Health 82(3):420–431

    Google Scholar 

  • Perry AE, Johnson M (2008) Applying the Consolidated Standards of Reporting Trials (CONSORT) to studies of mental health provision for juvenile offenders: a research note. J Exp Criminol 4:165–185

    Article  Google Scholar 

  • Perry AE, Gilbody S, Akers J, Light K (2008) Access and provision of services for young people with mental health needs. Youth Justice Board, England and Wales

    Google Scholar 

  • Perry AE, Weisburd D, Hewitt C (in press) Are criminologists reporting experiments in ways that allow us to assess? J Exp Criminol

    Google Scholar 

  • Peters M, Thomas D, Zanberlon C (1997) Boot camps for juvenile offenders. Office of Juvenile Justice and Delinquency Prevention (Program Summary, Washington, DC

    Google Scholar 

  • Petrosino A, Turpin-Petrosino C, Buehler J (2002) Scared straight and other juvenile awareness programs for preventing juvenile delinquency: a systematic review of randomized experimental evidence. Campbell Collaboration, USA

    Google Scholar 

  • Plint AC, Moher D, Morrison A, Schulz K, Altman D, Hill C, Gaboury I (2006) Does the CONSORT checklist improve the quality of reporting randomized controlled trials? A systematic review. Med J Aust 185(5):263–267

    Google Scholar 

  • Prady SL, Richmond SJ, Morton VM, MacPherson H (2008) A systematic evaluation of the impact of STRICTA and CONSORT recommendations on quality of reporting for acupuncture trials. PLos One 3(2):1–10

    Article  Google Scholar 

  • Rhodes W, Gross M (1997) Case management reduces drug use and criminality among drug-involved arrestees: an experimental study of an HIV prevention intervention (PDF Version). Washington, D.C.: National Institute of Justice, U.S. Department of Justice

    Google Scholar 

  • Robinson D (1995) The impact of cognitive skills training on post release recidvisim among Canadian federal offenders. Correctional Service of Canada, Ottowa, Research Report R-41

    Google Scholar 

  • Schulz KF, Chalmers I, Hayes RJ, Altman DG (1995) Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273:408–412

    Article  Google Scholar 

  • Shea B, Boers M, Grimshaw JM, Hamel C, Bouter LM (2006) Does updating improve the methodological and reporting quality of systematic reviews? BMC Med Res Methodol 6:27

    Article  Google Scholar 

  • Sherman LW, Farrington DP, Welsh BC, Mackenzie DL (eds) (2002) Evidence-Based Crime Prevention. Routledge, London

    Google Scholar 

  • Torgerson CJ & Elbourne D (2002) A systematic review and meta analysis of the effectiveness of information and communication technology (ICT) on the teaching of spelling. J Res Read 35:129–143

    Article  Google Scholar 

  • Torgerson CJ, Torgerson DJ, Birks YF, Porthouse J (2005) A comparison of RCTs in health and education. Br Educ Res J 31(6):761–785

    Article  Google Scholar 

  • Torgerson CJ, Torgerson DJ (2008) Designing and running randomised trials in health and the social sciences, Palgrave, MacMillan

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for Criminal Justice Economics and Psycholog, University of York, Heslington, York, UK

    Amanda E. Perry

Authors
  1. Amanda E. Perry
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. College of Criminology, Florida State University, West Call Street 643, Tallahassee, 32306, U.S.A.

    Alex R. Piquero

  2. Inst. Criminology, Hebrew University of Jerusalem, Jerusalem, 91905, Israel

    David Weisburd

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Perry, A.E. (2010). Descriptive Validity and Transparent Reporting in Randomised Controlled Trials. In: Piquero, A., Weisburd, D. (eds) Handbook of Quantitative Criminology. Springer, New York, NY. https://doi.org/10.1007/978-0-387-77650-7_17

Download citation

Publish with us

Policies and ethics

Search

Navigation