Journal of Experimental Criminology

, Volume 9, Issue 3, pp 355–367 | Cite as

Developing a reporting guideline for social and psychological intervention trials

  • Sean Grant
  • Evan Mayo-Wilson
  • Sally Hopewell
  • Geraldine Macdonald
  • David Moher
  • Paul Montgomery
Article

Abstract

Objectives

Social and psychological interventions are often complex. Understanding randomised controlled trials (RCTs) of these complex interventions requires a detailed description of the interventions tested and the methods used to evaluate them; however, RCT reports often omit, or inadequately report, this information. Incomplete and inaccurate reporting hinders the optimal use of research, wastes resources, and fails to meet ethical obligations to research participants and consumers.

Methods

In this paper, we explain how reporting guidelines have improved the quality of reports in medicine, and describe the on-going development of a new reporting guideline for RCTs: CONSORT-SPI (an extension for social and psychological interventions).

Results and conclusions

We invite readers to participate in the project by visiting our website, in order to help us reach the best-informed consensus on these guidelines (http://tinyurl.com/CONSORT-study).

Keywords

Randomised controlled trial RCT CONSORT-SPI Reporting guideline Reporting standards 

References

  1. Abraham, C.. (2009). Wider recommendations to improve reporting of the content of behaviour change interventions. (For the Workgroup for Intervention Development and Evaluation Research). Retrieved November 14, 2012 from http://interventiondesign.co.uk/wp-content/uploads/2009/02/wider-recommendations.pdf
  2. American Educational Research Association. (2006). Standards for reporting on empirical social science research in AERA publications. Educational Researcher, 35(6), 33–40.CrossRefGoogle Scholar
  3. American Psychological Association Journal Article Reporting Standards Group. (2008). Reporting standards for research in psychology: why do we need them? What might they be? American Psychologist, 63, 839–851.CrossRefGoogle Scholar
  4. Armstrong, R., Waters, E., Moore, L., Riggs, E., Cuervo, L. G., Lumbiganon, P., et al. (2008). Improving the reporting of public health intervention research: advancing TREND and CONSORT. Journal of Public Health, 30(1), 103–109.CrossRefGoogle Scholar
  5. Bonell, C. (2002). The utility of randomized controlled trials of social interventions: an examination of two trials of HIV prevention. Critical Public Health, 12(4), 321–334.CrossRefGoogle Scholar
  6. Bonell, C., Oakley, A., Hargreaves, J., Strange, V., & Rees, R. (2006). Assessment of generalisability in trials of health interventions: suggested framework and systematic review. BMJ, 333, 346–349.CrossRefGoogle Scholar
  7. Bonell, C. P., Hargreaves, J., Cousens, S., Ross, D., Hayes, R., Petticrew, M., et al. (2011). Alternatives to randomisation in the evaluation of public health interventions: design challenges and solutions. Journal of Epidemiological Community Health, 65, 582–587.CrossRefGoogle Scholar
  8. Bonell, C., Fletcher, A., Morton, M., Lorenc, T., & Moore, L. (2012). Realist randomised controlled trials: a new approach to evaluating complex public health interventions. Social Science & Medicine, 75(12), 2299–2306.CrossRefGoogle Scholar
  9. Boutron, I., Moher, D., Altman, D. G., Schulz, K., Ravaud, P., & for the CONSORT group. (2008a). Extending the CONSORT Statement to randomized trials of nonpharmacologic treatment: explanation and Elaboration. Annals of Internal Medicine, 148, 295–309.CrossRefGoogle Scholar
  10. Boutron, I., Moher, D., Altman, D. G., Schulz, K., Ravaud, P., & for the CONSORT group. (2008b). Methods and processes of the CONSORT group: Example of an extension for trials assessing nonpharmacologic treatments. Annals of Internal Medicine, 148(4), W60–W67.Google Scholar
  11. Calvert, M., Blazeby, J., Revicki, D., Moher, D., & Brundage, M. (2011). Reporting quality of life in clinical trials: a CONSORT extension. The Lancet, 378, 1684–1685.CrossRefGoogle Scholar
  12. Campbell Collaboration. (2013). Retrieved 26 April 2013 from http://campbellcollaboration.org/.
  13. Campbell, M. K., Elbourne, D. R., & Altman, D. G. (2004). CONSORT statement: extension to cluster randomised trials. BMJ, 328, 702–708.CrossRefGoogle Scholar
  14. Cartwright, N., & Munro, E. (2010). The limitations of randomized controlled trials in predicting effectiveness. Journal of Evaluation in Clinical Practice, 16, 260–266.CrossRefGoogle Scholar
  15. Chalmers, I. (2003). Trying to do more good than harm in policy and practice: the role of rigorous, transparent, up-to-date evaluations. The ANNALS of the American Academy of Political and Social Science, 589(22), 22–40.CrossRefGoogle Scholar
  16. Cochrane Collaboration. (2013). Retrieved 26 April 2013 from http://www.cochrane.org/.
  17. Craig, P., Dieppe, P., Macintyre, S., Mitchie, S., Nazareth, I., & Petticrew, M. (2008). Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ, 337, 979–983.CrossRefGoogle Scholar
  18. Davidson, K. W., Goldstein, M., Kaplan, R. M., Kaufmann, P. G., Knatterud, G. L., Orleans, C. T., et al. (2003). Evidence-based behavioural medicine: what is it and how do we achieve it? Annals of Behavioral Medicine, 26, 161–171.CrossRefGoogle Scholar
  19. Des Jarlais, D. C., Lyles, C., Crepaz, N., & the TREND Group. (2004). Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. American Journal of Public Health, 94, 361–366.CrossRefGoogle Scholar
  20. Evans, T., & Brown, H. (2003). Road traffic crashes: operationalizing equity in the context of health sector reform. Injury Control and Safety Promotion, 10(1–2), 11–12.CrossRefGoogle Scholar
  21. Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: a synthesis of the literature. Tampa: University of South Florida.Google Scholar
  22. Fraser, M. W., Galinsky, M. J., Richman, J. M., & Day, S. H. (2009). Intervention Research: Developing Social Programs. Oxford: Oxford University Press.Google Scholar
  23. Gill, C. E. (2011). Missing links: how descriptive validity impacts the policy relevance of randomized controlled trials in criminology. Journal of Experimental Criminology, 7(3), 201–224.CrossRefGoogle Scholar
  24. Glasziou, P., Meats, E., Heneghan, C., & Shepperd, S. (2008). What is missing from descriptions of treatment in trials and reviews? BMJ, 336, 1472–1474.CrossRefGoogle Scholar
  25. Goldbeck, L., & Vitiello, B. (2011). Reporting clinical trials of psychosocial interventions in child and adolescent psychiatry and mental health. Child and Adolescent Psychiatry and Mental Health, 5(4).Google Scholar
  26. Grant, S., Montgomery, P., & Mayo-Wilson, E. (2012). Development of a CONSORT extension for interventions in public health and related disciplines. The Lancet, 380(Supp. 3), p. S14. Retrieved 26 April 2013 from http://www.thelancet.com/abstracts/public-health-science.
  27. Hardeman, W., Michie, S., Fanshawe, T., Prevost, A. T., McLoughlin, K., & Kinmonth, A. L. (2008). Fidelity of delivery of a physical activity intervention: predictors and consequences. Psychology & Health, 23, 11–24.CrossRefGoogle Scholar
  28. Hawe, P., Shiell, A., & Riley, T. (2004). Complex interventions: how "out of control" can a randomised controlled trial be? BMJ, 328(7455), 1561–1563.CrossRefGoogle Scholar
  29. Henggeler, S. W., Schoenwald, S. K., Rowland, M. D., & Cunningham, P. B. (2002). Serious emotional disturbances in children and adolescents: Multisystemic therapy. New York: Guilford Press.Google Scholar
  30. Hopewell, S., Clarke, M., Moher, D., Wager, E., Middleton, P., Altman, D. G., et al. (2008). CONSORT for reporting randomised trials in journal and conference abstracts. Lancet, 371, 281–283.CrossRefGoogle Scholar
  31. Ivers, N., Jamtvedt, G., Flottorp, S., Young, J. M., Odgaard-Jensen, J., French, S. D., et al. (2012). Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, 6, CD000259.Google Scholar
  32. Ladd, B. O., McCrady, B. S., Manuel, J. K., & Campbell, W. (2010). Improving the quality of reporting alcohol outcome studies: effects of the CONSORT statement. Addictive Behaviors, 35, 660–666.CrossRefGoogle Scholar
  33. Littell, J. H., Campbell, M., Green, S., & Toews, B. (2009). Multisystemic Therapy for social, emotional, and behavioral problems in youth aged 10-17. Cochrane Library of Systematic Reviews, 4, CD004797.Google Scholar
  34. Marshall, M., Lockwood, A., Bradley, C., Adams, C., Joy, C., & Fenton, M. (2000). Unpublished rating scales: a major source of bias in randomised controlled trials of treatments for schizophrenia. The British Journal of Psychiatry, 176, 249–252.CrossRefGoogle Scholar
  35. Mayo-Wilson, E. (2007). Reporting implementation in randomized trials: proposed additions to the Consolidated Standards of Reporting Trials statement. American Journal of Public Health, 97, 630–633.CrossRefGoogle Scholar
  36. McGrath, P. J., Stinson, J., & Davidson, K. (2003). Commentary: the Journal of Pediatric Psychology should adopt the CONSORT statement as a way of improving the evidence base in pediatric psychology. Journal of Pediatric Psychology, 28, 169–171.CrossRefGoogle Scholar
  37. Medical Research Council. (2008). A framework for development and evaluation of RCTs for complex interventions to improve health. London: Medical Research Council.Google Scholar
  38. Medical Research Council. (2011). Using natural experiments to evaluate population health interventions: guidance for producers and users of evidence. London: Medical Research Council.Google Scholar
  39. Michie, S., Fixsen, D., Grimshaw, J. M., & Eccles, M. P. (2009). Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implementation Science, 4, 40.CrossRefGoogle Scholar
  40. Michie, S., Abraham, C., Eccles, M. P., Francis, J. J., Hardeman, W., & Johnston, M. (2011). Strengthening evaluation and implementation by specifying components of behaviour change interventions: a study protocol. Implementation Science, 6, 10.CrossRefGoogle Scholar
  41. Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., et al. (1999). Does quality of reports of randomized trials affect estimates of inter- vention efficacy reported in meta-analyses? Lancet, 352, 609–613.CrossRefGoogle Scholar
  42. Moher, D., Altman, D. G., Schulz, K. F., & Elbourne, D. R. (2004). Opportunities and challenges for improving the quality of reporting clinical research: CONSORT and beyond. Canadian Medical Association Journal, 171(4), 349–350.CrossRefGoogle Scholar
  43. Moher, D., Hopewell, S., Schultz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., et al. (2010a). CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. BMJ, 340, c869.CrossRefGoogle Scholar
  44. Moher, D., Schulz, K. F., Simera, I., & Altman, D. G. (2010b). Guidance for developers of health research reporting guidelines. PLoS Medicine, 7(2), e1000217.CrossRefGoogle Scholar
  45. Moore, L. (2002). Research design for the rigorous evaluation of complex educational interventions: lessons from health services research. Building Research Capacity, 1, 4–5.Google Scholar
  46. Moore, L., & Moore, G. F. (2011). Public health evaluation: which designs work, for whom and under what circumstances? Journal of Epidemiology and Community Health, 65, 596–597.CrossRefGoogle Scholar
  47. Murphy, M. K., Black, N. A., Lamping, D. L., McKee, C. M., Sanders, C. F. B., Askham, J., et al. (1998). Consensus development methods, and their use in clinical guideline development. Health Technology Assessment, 2(3).Google Scholar
  48. Naleppa, M. J., & Cagle, J. G. (2010). Treatment fidelity in social work intervention research: a review of published studies. Research on Social Work Practice, 20, 674–681.CrossRefGoogle Scholar
  49. Newman, M., & Elbourne, D. (2004). Improving the usability of educational research: guidelines for the REPOrting of primary empirical research Studies in Education (The REPOSE Guidelines). Evaluation & Research in Education, 18(4), 201–212.CrossRefGoogle Scholar
  50. Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2004). Realist Synthesis: An Introduction. (University of Manchester: ESRC Research Methods Programme)Google Scholar
  51. Perry, A. E., & Johnson, M. (2008). Applying the Consolidated Standards of Reporting Trials (CONSORT) to studies of mental health provision for juvenile offenders: a research note. Journal of Experimental Criminology, 4, 165–185.CrossRefGoogle Scholar
  52. Perry, A. E., Weisburd, D., & Hewitt, C. (2010). Are criminologists describing randomized controlled trials in ways that allow us to assess them? findings from a sample of crime and justice trials. Journal of Experimental Criminology, 6, 245–262.CrossRefGoogle Scholar
  53. Pildal, J., Hróbjartsson, A., Jørgensen, K. J., Hilden, J., Altman, D. G., & Gøtzsche, P. C. (2007). Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. International Journal of Epidemiology, 36(4), 847–857.CrossRefGoogle Scholar
  54. Prescott, R. J., Counsell, C. E., Gillespie, W. J., Grant, A. M., Russell, I. T., Kiauka, S., et al. (1999). Factors that limit the quality, number and progress of randomised controlled trials. Health Technology Assessment, 3(20).Google Scholar
  55. Schulz, K. F., Chalmers, I., Hayes, R. J., & Altman, D. G. (1995). Allocation concealment in randomised trials: defending against deciphering. Lancet, 359, 614–617.CrossRefGoogle Scholar
  56. Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group. (2010). CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ, 340, 698–702.CrossRefGoogle Scholar
  57. Semaan, S., Kay, L., Strouse, D., Sogolow, E., Mullen, P. D., Neumann, M. S., et al. (2002). A profile of U.S.-based trials of behavioral and social interventions for HIV risk reduction. Journal of Acquired Immune Deficiency Syndromes, 30, S30–S50.Google Scholar
  58. Shepperd, S., Lewin, S., Straus, S., Clarke, M., Eccles, M. P., Fitzpatrick, R., et al. (2009). Can we systematically review studies that evaluate complex interventions? PLoS Medicine, 6(8), 31000086.CrossRefGoogle Scholar
  59. Stinson, J. N., McGrath, P. J., & Yamada, J. T. (2003). Clinical trials in the Journal of Pediatric Psychology: applying the CONSORT statement. Journal of Pediatric Psychology, 28, 159–167.CrossRefGoogle Scholar
  60. Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19(6), 349–357.CrossRefGoogle Scholar
  61. Torgerson, C. J., Torgerson, D. J., Birks, Y. F., & Porthouse, J. (2005). A comparison of RCTs in health and education. British Educational Research Journal, 31(6), 761–785.CrossRefGoogle Scholar
  62. Tugwell, P., Petticrew, M., Kristjansson, E., Welch, V., Ueffing, E., et al. (2010). Assessing equity in systematic reviews: realising the recommendations of the Commission on Social Determinants of Health. BMJ, 341, c4739.CrossRefGoogle Scholar
  63. Turner, L., Shamseer, L., Altman, D. G., Weeks, L., Peters, J., Kober, T., et al. (2012). Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database of Systematic Reviews, 11, MR000030.Google Scholar
  64. von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gotzsche, P. C., & Vandenbroucke, J. P. (2007). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies. Annals of Internal Medicine, 147(8), 573–577.CrossRefGoogle Scholar
  65. Wang, S., Moss, J. R., & Hiller, J. E. (2006). Applicability and transferability of interventions in evidence-based public health. Health Promotion International, 21(1), 76–83.CrossRefGoogle Scholar
  66. Welch, V., Petticrew, M., Tugwell, P., Moher, D., O'Neill, J., Waters, E., et al. (2012). PRISMA-Equity 2012 extension: reporting guidelines for systematic reviews with a focus on health equity. PLoS Medicine, 9(10), e1001333.CrossRefGoogle Scholar
  67. Zwarenstein, M., Treweek, S., Gagnier, J. J., Altman, D. G., Tunis, S., Haynes, B., et al. (2008). Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ, 337, a2390.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Sean Grant
    • 1
  • Evan Mayo-Wilson
    • 2
  • Sally Hopewell
    • 3
  • Geraldine Macdonald
    • 4
  • David Moher
    • 5
  • Paul Montgomery
    • 1
  1. 1.Centre for Evidence-Based InterventionUniversity of OxfordOxfordUK
  2. 2.Centre for Outcomes Research and Effectiveness, Research Department of Clinical, Educational & Health PsychologyUniversity College LondonLondonUK
  3. 3.Centre for Statistics in MedicineUniversity of OxfordOxfordUK
  4. 4.Institute of Child Care ResearchQueen’s University BelfastBelfastUK
  5. 5.Clinical Epidemiology Program, Ottawa Hospital Research Institute, Centre for Practice-Changing Research (CPCR)The Ottawa HospitalOttawaCanada

Personalised recommendations