Skip to main content
SpringerLink
Log in

Developing a reporting guideline for social and psychological intervention trials

Journal of Experimental Criminology volume 9, pages 355–367 (2013)Cite this article

A Commentary to this article was published on 07 August 2013

A Commentary to this article was published on 07 August 2013

A Commentary to this article was published on 07 August 2013

Abstract

Objectives

Social and psychological interventions are often complex. Understanding randomised controlled trials (RCTs) of these complex interventions requires a detailed description of the interventions tested and the methods used to evaluate them; however, RCT reports often omit, or inadequately report, this information. Incomplete and inaccurate reporting hinders the optimal use of research, wastes resources, and fails to meet ethical obligations to research participants and consumers.

Methods

In this paper, we explain how reporting guidelines have improved the quality of reports in medicine, and describe the on-going development of a new reporting guideline for RCTs: CONSORT-SPI (an extension for social and psychological interventions).

Results and conclusions

We invite readers to participate in the project by visiting our website, in order to help us reach the best-informed consensus on these guidelines (http://tinyurl.com/CONSORT-study).

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  • Abraham, C.. (2009). Wider recommendations to improve reporting of the content of behaviour change interventions. (For the Workgroup for Intervention Development and Evaluation Research). Retrieved November 14, 2012 from http://interventiondesign.co.uk/wp-content/uploads/2009/02/wider-recommendations.pdf

  • American Educational Research Association. (2006). Standards for reporting on empirical social science research in AERA publications. Educational Researcher, 35(6), 33–40.

    Article  Google Scholar 

  • American Psychological Association Journal Article Reporting Standards Group. (2008). Reporting standards for research in psychology: why do we need them? What might they be? American Psychologist, 63, 839–851.

    Article  Google Scholar 

  • Armstrong, R., Waters, E., Moore, L., Riggs, E., Cuervo, L. G., Lumbiganon, P., et al. (2008). Improving the reporting of public health intervention research: advancing TREND and CONSORT. Journal of Public Health, 30(1), 103–109.

    Article  Google Scholar 

  • Bonell, C. (2002). The utility of randomized controlled trials of social interventions: an examination of two trials of HIV prevention. Critical Public Health, 12(4), 321–334.

    Article  Google Scholar 

  • Bonell, C., Oakley, A., Hargreaves, J., Strange, V., & Rees, R. (2006). Assessment of generalisability in trials of health interventions: suggested framework and systematic review. BMJ, 333, 346–349.

    Article  Google Scholar 

  • Bonell, C. P., Hargreaves, J., Cousens, S., Ross, D., Hayes, R., Petticrew, M., et al. (2011). Alternatives to randomisation in the evaluation of public health interventions: design challenges and solutions. Journal of Epidemiological Community Health, 65, 582–587.

    Article  Google Scholar 

  • Bonell, C., Fletcher, A., Morton, M., Lorenc, T., & Moore, L. (2012). Realist randomised controlled trials: a new approach to evaluating complex public health interventions. Social Science & Medicine, 75(12), 2299–2306.

    Article  Google Scholar 

  • Boutron, I., Moher, D., Altman, D. G., Schulz, K., Ravaud, P., & for the CONSORT group. (2008a). Extending the CONSORT Statement to randomized trials of nonpharmacologic treatment: explanation and Elaboration. Annals of Internal Medicine, 148, 295–309.

    Article  Google Scholar 

  • Boutron, I., Moher, D., Altman, D. G., Schulz, K., Ravaud, P., & for the CONSORT group. (2008b). Methods and processes of the CONSORT group: Example of an extension for trials assessing nonpharmacologic treatments. Annals of Internal Medicine, 148(4), W60–W67.

    Google Scholar 

  • Calvert, M., Blazeby, J., Revicki, D., Moher, D., & Brundage, M. (2011). Reporting quality of life in clinical trials: a CONSORT extension. The Lancet, 378, 1684–1685.

    Article  Google Scholar 

  • Campbell Collaboration. (2013). Retrieved 26 April 2013 from http://campbellcollaboration.org/.

  • Campbell, M. K., Elbourne, D. R., & Altman, D. G. (2004). CONSORT statement: extension to cluster randomised trials. BMJ, 328, 702–708.

    Article  Google Scholar 

  • Cartwright, N., & Munro, E. (2010). The limitations of randomized controlled trials in predicting effectiveness. Journal of Evaluation in Clinical Practice, 16, 260–266.

    Article  Google Scholar 

  • Chalmers, I. (2003). Trying to do more good than harm in policy and practice: the role of rigorous, transparent, up-to-date evaluations. The ANNALS of the American Academy of Political and Social Science, 589(22), 22–40.

    Article  Google Scholar 

  • Cochrane Collaboration. (2013). Retrieved 26 April 2013 from http://www.cochrane.org/.

  • Craig, P., Dieppe, P., Macintyre, S., Mitchie, S., Nazareth, I., & Petticrew, M. (2008). Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ, 337, 979–983.

    Article  Google Scholar 

  • Davidson, K. W., Goldstein, M., Kaplan, R. M., Kaufmann, P. G., Knatterud, G. L., Orleans, C. T., et al. (2003). Evidence-based behavioural medicine: what is it and how do we achieve it? Annals of Behavioral Medicine, 26, 161–171.

    Article  Google Scholar 

  • Des Jarlais, D. C., Lyles, C., Crepaz, N., & the TREND Group. (2004). Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. American Journal of Public Health, 94, 361–366.

    Article  Google Scholar 

  • Evans, T., & Brown, H. (2003). Road traffic crashes: operationalizing equity in the context of health sector reform. Injury Control and Safety Promotion, 10(1–2), 11–12.

    Article  Google Scholar 

  • Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: a synthesis of the literature. Tampa: University of South Florida.

    Google Scholar 

  • Fraser, M. W., Galinsky, M. J., Richman, J. M., & Day, S. H. (2009). Intervention Research: Developing Social Programs. Oxford: Oxford University Press.

    Google Scholar 

  • Gill, C. E. (2011). Missing links: how descriptive validity impacts the policy relevance of randomized controlled trials in criminology. Journal of Experimental Criminology, 7(3), 201–224.

    Article  Google Scholar 

  • Glasziou, P., Meats, E., Heneghan, C., & Shepperd, S. (2008). What is missing from descriptions of treatment in trials and reviews? BMJ, 336, 1472–1474.

    Article  Google Scholar 

  • Goldbeck, L., & Vitiello, B. (2011). Reporting clinical trials of psychosocial interventions in child and adolescent psychiatry and mental health. Child and Adolescent Psychiatry and Mental Health, 5(4).

  • Grant, S., Montgomery, P., & Mayo-Wilson, E. (2012). Development of a CONSORT extension for interventions in public health and related disciplines. The Lancet, 380(Supp. 3), p. S14. Retrieved 26 April 2013 from http://www.thelancet.com/abstracts/public-health-science.

  • Hardeman, W., Michie, S., Fanshawe, T., Prevost, A. T., McLoughlin, K., & Kinmonth, A. L. (2008). Fidelity of delivery of a physical activity intervention: predictors and consequences. Psychology & Health, 23, 11–24.

    Article  Google Scholar 

  • Hawe, P., Shiell, A., & Riley, T. (2004). Complex interventions: how "out of control" can a randomised controlled trial be? BMJ, 328(7455), 1561–1563.

    Article  Google Scholar 

  • Henggeler, S. W., Schoenwald, S. K., Rowland, M. D., & Cunningham, P. B. (2002). Serious emotional disturbances in children and adolescents: Multisystemic therapy. New York: Guilford Press.

    Google Scholar 

  • Hopewell, S., Clarke, M., Moher, D., Wager, E., Middleton, P., Altman, D. G., et al. (2008). CONSORT for reporting randomised trials in journal and conference abstracts. Lancet, 371, 281–283.

    Article  Google Scholar 

  • Ivers, N., Jamtvedt, G., Flottorp, S., Young, J. M., Odgaard-Jensen, J., French, S. D., et al. (2012). Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, 6, CD000259.

  • Ladd, B. O., McCrady, B. S., Manuel, J. K., & Campbell, W. (2010). Improving the quality of reporting alcohol outcome studies: effects of the CONSORT statement. Addictive Behaviors, 35, 660–666.

    Article  Google Scholar 

  • Littell, J. H., Campbell, M., Green, S., & Toews, B. (2009). Multisystemic Therapy for social, emotional, and behavioral problems in youth aged 10-17. Cochrane Library of Systematic Reviews, 4, CD004797.

  • Marshall, M., Lockwood, A., Bradley, C., Adams, C., Joy, C., & Fenton, M. (2000). Unpublished rating scales: a major source of bias in randomised controlled trials of treatments for schizophrenia. The British Journal of Psychiatry, 176, 249–252.

    Article  Google Scholar 

  • Mayo-Wilson, E. (2007). Reporting implementation in randomized trials: proposed additions to the Consolidated Standards of Reporting Trials statement. American Journal of Public Health, 97, 630–633.

    Article  Google Scholar 

  • McGrath, P. J., Stinson, J., & Davidson, K. (2003). Commentary: the Journal of Pediatric Psychology should adopt the CONSORT statement as a way of improving the evidence base in pediatric psychology. Journal of Pediatric Psychology, 28, 169–171.

    Article  Google Scholar 

  • Medical Research Council. (2008). A framework for development and evaluation of RCTs for complex interventions to improve health. London: Medical Research Council.

    Google Scholar 

  • Medical Research Council. (2011). Using natural experiments to evaluate population health interventions: guidance for producers and users of evidence. London: Medical Research Council.

    Google Scholar 

  • Michie, S., Fixsen, D., Grimshaw, J. M., & Eccles, M. P. (2009). Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implementation Science, 4, 40.

    Article  Google Scholar 

  • Michie, S., Abraham, C., Eccles, M. P., Francis, J. J., Hardeman, W., & Johnston, M. (2011). Strengthening evaluation and implementation by specifying components of behaviour change interventions: a study protocol. Implementation Science, 6, 10.

    Article  Google Scholar 

  • Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., et al. (1999). Does quality of reports of randomized trials affect estimates of inter- vention efficacy reported in meta-analyses? Lancet, 352, 609–613.

    Article  Google Scholar 

  • Moher, D., Altman, D. G., Schulz, K. F., & Elbourne, D. R. (2004). Opportunities and challenges for improving the quality of reporting clinical research: CONSORT and beyond. Canadian Medical Association Journal, 171(4), 349–350.

    Article  Google Scholar 

  • Moher, D., Hopewell, S., Schultz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., et al. (2010a). CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. BMJ, 340, c869.

    Article  Google Scholar 

  • Moher, D., Schulz, K. F., Simera, I., & Altman, D. G. (2010b). Guidance for developers of health research reporting guidelines. PLoS Medicine, 7(2), e1000217.

    Article  Google Scholar 

  • Moore, L. (2002). Research design for the rigorous evaluation of complex educational interventions: lessons from health services research. Building Research Capacity, 1, 4–5.

    Google Scholar 

  • Moore, L., & Moore, G. F. (2011). Public health evaluation: which designs work, for whom and under what circumstances? Journal of Epidemiology and Community Health, 65, 596–597.

    Article  Google Scholar 

  • Murphy, M. K., Black, N. A., Lamping, D. L., McKee, C. M., Sanders, C. F. B., Askham, J., et al. (1998). Consensus development methods, and their use in clinical guideline development. Health Technology Assessment, 2(3).

  • Naleppa, M. J., & Cagle, J. G. (2010). Treatment fidelity in social work intervention research: a review of published studies. Research on Social Work Practice, 20, 674–681.

    Article  Google Scholar 

  • Newman, M., & Elbourne, D. (2004). Improving the usability of educational research: guidelines for the REPOrting of primary empirical research Studies in Education (The REPOSE Guidelines). Evaluation & Research in Education, 18(4), 201–212.

    Article  Google Scholar 

  • Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2004). Realist Synthesis: An Introduction. (University of Manchester: ESRC Research Methods Programme)

  • Perry, A. E., & Johnson, M. (2008). Applying the Consolidated Standards of Reporting Trials (CONSORT) to studies of mental health provision for juvenile offenders: a research note. Journal of Experimental Criminology, 4, 165–185.

    Article  Google Scholar 

  • Perry, A. E., Weisburd, D., & Hewitt, C. (2010). Are criminologists describing randomized controlled trials in ways that allow us to assess them? findings from a sample of crime and justice trials. Journal of Experimental Criminology, 6, 245–262.

    Article  Google Scholar 

  • Pildal, J., Hróbjartsson, A., Jørgensen, K. J., Hilden, J., Altman, D. G., & Gøtzsche, P. C. (2007). Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. International Journal of Epidemiology, 36(4), 847–857.

    Article  Google Scholar 

  • Prescott, R. J., Counsell, C. E., Gillespie, W. J., Grant, A. M., Russell, I. T., Kiauka, S., et al. (1999). Factors that limit the quality, number and progress of randomised controlled trials. Health Technology Assessment, 3(20).

  • Schulz, K. F., Chalmers, I., Hayes, R. J., & Altman, D. G. (1995). Allocation concealment in randomised trials: defending against deciphering. Lancet, 359, 614–617.

    Article  Google Scholar 

  • Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group. (2010). CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ, 340, 698–702.

    Article  Google Scholar 

  • Semaan, S., Kay, L., Strouse, D., Sogolow, E., Mullen, P. D., Neumann, M. S., et al. (2002). A profile of U.S.-based trials of behavioral and social interventions for HIV risk reduction. Journal of Acquired Immune Deficiency Syndromes, 30, S30–S50.

    Google Scholar 

  • Shepperd, S., Lewin, S., Straus, S., Clarke, M., Eccles, M. P., Fitzpatrick, R., et al. (2009). Can we systematically review studies that evaluate complex interventions? PLoS Medicine, 6(8), 31000086.

    Article  Google Scholar 

  • Stinson, J. N., McGrath, P. J., & Yamada, J. T. (2003). Clinical trials in the Journal of Pediatric Psychology: applying the CONSORT statement. Journal of Pediatric Psychology, 28, 159–167.

    Article  Google Scholar 

  • Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19(6), 349–357.

    Article  Google Scholar 

  • Torgerson, C. J., Torgerson, D. J., Birks, Y. F., & Porthouse, J. (2005). A comparison of RCTs in health and education. British Educational Research Journal, 31(6), 761–785.

    Article  Google Scholar 

  • Tugwell, P., Petticrew, M., Kristjansson, E., Welch, V., Ueffing, E., et al. (2010). Assessing equity in systematic reviews: realising the recommendations of the Commission on Social Determinants of Health. BMJ, 341, c4739.

    Article  Google Scholar 

  • Turner, L., Shamseer, L., Altman, D. G., Weeks, L., Peters, J., Kober, T., et al. (2012). Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database of Systematic Reviews, 11, MR000030.

  • von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gotzsche, P. C., & Vandenbroucke, J. P. (2007). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies. Annals of Internal Medicine, 147(8), 573–577.

    Article  Google Scholar 

  • Wang, S., Moss, J. R., & Hiller, J. E. (2006). Applicability and transferability of interventions in evidence-based public health. Health Promotion International, 21(1), 76–83.

    Article  Google Scholar 

  • Welch, V., Petticrew, M., Tugwell, P., Moher, D., O'Neill, J., Waters, E., et al. (2012). PRISMA-Equity 2012 extension: reporting guidelines for systematic reviews with a focus on health equity. PLoS Medicine, 9(10), e1001333.

    Article  Google Scholar 

  • Zwarenstein, M., Treweek, S., Gagnier, J. J., Altman, D. G., Tunis, S., Haynes, B., et al. (2008). Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ, 337, a2390.

    Article  Google Scholar 

Download references

Acknowledgments

The CONSORT-SPI (Social and Psychological Interventions) International Advisory Group includes: J. Lawrence Aber, Distinguished Professor of Applied Psychology and Public Policy, Steinhardt School of Culture, Education, and Human Development, New York University; Chris Bonell, Professor of Sociology and Social Intervention, Centre for Evidence Based Intervention, University of Oxford; David M. Clark, Chair of Psychology, Department of Experimental Psychology, University of Oxford; Frances Gardner, Professor of Child and Family Psychology, Centre for Evidence Based Intervention, University of Oxford; Steven Hollon, American Psychological Association Guidelines Committee (Chair), Gertrude Conaway Professor of Psychology, Department of Psychology, Vanderbilt University; Jim McCambridge, Senior Lecturer in Behaviour Change, Department of Social and Environmental Health Research, London School of Hygiene and Tropical Medicine; Susan Michie, Professor of Health Psychology, Department of Clinical, Educational & Health Psychology, University College London; Laurence Moore, Professor of Public Health Improvement, Cardiff School of Social Sciences, Cardiff University; Mark Petticrew, Professor of Public Health Evaluation, Department Social and Environmental Health Research, London School of Hygiene and Tropical Medicine; Lawrence Sherman, Wolfson Professor of Criminology, Cambridge Institute of Criminology, Cambridge University; Steve Pilling, Director, Centre for Outcomes Research and Effectiveness, University College London; James Thomas, Associate Director EPPI-Centre, Reader in Social Policy, Institute of Education, University of London; Elizabeth Waters, Jack Brockhoff Chair of Child Public Health, McCaughey VicHealth Centre for Community Wellbeing, Melbourne School of Population & Global Health, University of Melbourne, Australia; David Weisburd, Director and Walter E. Meyer Professor of Law and Criminal Justice, Institute of Criminology, Hebrew University Faculty of Law, Jerusalem; Joanne Yaffe, Associate Professor, College of Social Work, University of Utah. This project is funded by the UK Economic and Social Research Council (ES/K00087X/1). We thank the Centre for Evidence-Based Intervention (Oxford University), the Centre for Outcomes Research and Effectiveness (University College London), and the National Collaborating Centre for Mental Health (NCCMH) for their support. S.G. is supported by a linked Clarendon Fund-Green Templeton College Annual Fund Scholarship to support his doctoral studies and research. D.M. is supported by a University Research Chair.

Conflict of interest

None.

Author statement

PM, EMW, and SG conceived of the idea for the project. All authors helped to draft the manuscript, and all have read and approved the final manuscript.

Author information

Authors and Affiliations

  1. Centre for Evidence-Based Intervention, University of Oxford, Barnett House, 32 Wellington Square, Oxford, OX1 2ER, UK

    Sean Grant & Paul Montgomery

  2. Centre for Outcomes Research and Effectiveness, Research Department of Clinical, Educational & Health Psychology, University College London, 1-19 Torrington Place, London, WC1E 7HB, UK

    Evan Mayo-Wilson

  3. Centre for Statistics in Medicine, University of Oxford, Botnar Research Centre, Windmill Road, Oxford, OX3 7LD, UK

    Sally Hopewell

  4. Institute of Child Care Research, Queen’s University Belfast, 6 College Park, Belfast, BT7 1LP, UK

    Geraldine Macdonald

  5. Clinical Epidemiology Program, Ottawa Hospital Research Institute, Centre for Practice-Changing Research (CPCR), The Ottawa Hospital, General Campus, 501 Smyth Rd, Room L1288, Ottawa, ON, K1H 8L6, Canada

    David Moher

Authors
  1. Sean Grant
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Evan Mayo-Wilson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sally Hopewell
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Geraldine Macdonald
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David Moher
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Paul Montgomery
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sean Grant.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Grant, S., Mayo-Wilson, E., Hopewell, S. et al. Developing a reporting guideline for social and psychological intervention trials. J Exp Criminol 9, 355–367 (2013). https://doi.org/10.1007/s11292-013-9180-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11292-013-9180-5

Keywords

search

Navigation