Background

When feasible and appropriate, randomised controlled trials (RCTs) are used to evaluate social and psychological interventions, and to inform policy and practice decisions [1,2,3,4,5]. To use reports of RCTs, readers need information about their design, context, conduct, analysis, results, and interpretation. Like other types of research, RCTs can provide biased estimates of intervention effects if they are not conducted well, and syntheses of these RCTs may be biased if the trials are not reported completely [6, 7]. Consequently, accurate, complete, and transparent reports of RCTs are essential for maximising their value [8], allowing replication studies to build the evidence base [9], and facilitating the comparison and implementation of effective interventions in real-world contexts [10].

Recent reviews have shown that reports of RCTs of social and psychological interventions are often insufficiently accurate, comprehensive, and transparent to replicate trials, assess their quality, and understand for whom and under what circumstances an intervention should be delivered [11,12,13]. For instance, authors often do not report data on intervention implementation [14], such as the specific techniques employed by intervention providers; adaptation or tailoring of the intervention to specific groups or individuals; materials used to support intervention implementation; and participant behaviours [15]. Inadequate reporting can make it difficult for researchers to replicate trials, for intervention developers to design effective interventions, and for providers to use the interventions in practice [16]. A lack of sharing trial protocols, outcome data, and materials required to implement social and psychological interventions has been identified as a major reason for limitations in the ability of behavioural and social scientists to reproduce trial procedures, replicate trial results, and effectively synthesise evidence on these interventions [16,17,18,19,20,21]. The review of trials that we conducted in the first phase of this project (n  =  239) revealed that many CONSORT items were poorly reported in the behavioural and social science literature. Such items included identification as a randomised trial in titles; information about masking, methods for sequence generation, and allocation concealment; and details about the actual delivery of the interventions. Only 11 of 40 journals we examined referenced reporting guidelines in ‘Instructions to Authors’ [11]. This inefficient use of resources for research likely contributes to the suboptimal dissemination of potentially effective interventions [8, 22], overestimations of intervention efficacy [23], and research waste of investment to the order of hundreds of billions of dollars [22]. As in other areas of research, transparent and detailed reporting of social and psychological intervention RCTs is needed to minimise reporting biases and maximise the credibility and utility of this research evidence [24, 25].

The CONSORT Statement

To address the problems in scientific manuscripts outlined above, reporting guidelines have been developed that include minimum standards for describing specific types of research [26]. Reporting guidelines do not provide recommendations for study design or conduct. Instead, they focus on reporting what was done (methods) and what was found (results). In 1996, a group of scientists and journal editors published the CONSORT (Consolidated Standards of Reporting Trials) Statement to help authors report RCTs in biomedicine completely and transparently [27]. In light of feedback and emerging evidence, the CONSORT Group updated this reporting guideline in 2001 [28] and again in 2010 [29]. CONSORT 2010 includes a 25-item checklist and flow diagram. An extensive Explanation and Elaboration (E&E) document serves as a user manual that explains the rationale behind each checklist item, provides the methodological rationale for each checklist item, and gives examples of trial details adequately reported in accordance with each checklist item [26].

The CONSORT Statement has had an important impact in medicine. An early evaluation showed that reporting in the BMJ, Lancet, and JAMA improved after the publication of the first CONSORT Statement [30]. Systematic reviews comparing articles in medical journals endorsing CONSORT compared with journals not endorsing it found that the former are significantly more likely to describe the method of sequence generation, allocation concealment, and participant flow [31]. These effects remain even after controlling for the impact factor of the journals and study outcomes [32]. Over 600 journals and prominent editorial groups (including the International Committee of Medical Journal Editors, the Council of Science Editors, and the World Association of Medical Editors) officially endorse the CONSORT Statement.

Scope of CONSORT-SPI 2018

The CONSORT 2010 Statement focuses on individually randomised two-group parallel trials [29]. To address the varying amount of additional information needed for different types of trial, the CONSORT Group has created extensions (http://www.consort-statement.org/extensions). These extensions target different types of trial designs, such as cluster randomised [33], noninferiority [34], pragmatic [35], N-of-1 [36], and feasibility [37]; different types of trial data, such as patient-reported outcomes [38], abstracts [39], and harms [40]; and different types of intervention (see next section) [41,42,43]. Intervention extensions of CONSORT are organised by techniques, such as non-pharmacologic [41], herbal medicinal products [42], and acupuncture [43].

Social and psychological interventions go beyond simply adding techniques or using different techniques compared to biomedical interventions; they often use concepts, theories, and taxonomies that are distinct from those used by the biomedical scientists targeted by the CONSORT extension for non-pharmacologic treatments [21, 44,45,46,47,48]. To delineate the scope of CONSORT for social and psychological interventions (CONSORT-SPI), we define interventions by their mechanisms of action: i.e., how these interventions function to affect desired outcomes [49, 50]. That is, social and psychological interventions are actions intended to modify processes and systems that are social and psychological in nature (such as cognitions, emotions, behaviours, norms, relationships, and salient aspects of the environment) and are hypothesised to be influences on outcomes of interest [51, 52].

Social and psychological interventions can be complex in several ways [12, 50]. For example, these interventions cover an assortment of coordinated actions—such as practices, programmes, and policies—that often involve multiple interacting components. The units targeted by these interventions may include individuals, groups, or even places, and outcomes may be measured at any of these levels. The behaviours of both providers and recipients must be understood if the intervention and its effects are to be understood [53,54,55]. Social and psychological interventions may not follow strictly standardised implementation procedures [56], and effects may depend on aspects of the hard-to-control dynamic systems in which they occur [57,58,59]. For these reasons, readers of social and psychological intervention research are interested in more than just effect estimates—they require information about how and why these interventions work, for whom, and under what conditions [60].

Methods

We developed an official CONSORT Extension that addresses the minimum criteria that need to be met when reporting RCTs evaluating the effects of social and psychological interventions (CONSORT-SPI 2018). We followed recommended practices for developing and disseminating reporting guidelines [26] as described in the study protocol [61]. The methods and results of the systematic review, Delphi process, and consensus meeting followed a pre-specified protocol reported in full elsewhere [11, 61]. We briefly summarise the process below (Fig. 1).

Fig. 1
figure 1

Flow of potential checklist items through CONSORT-SPI 2018 project

Full size image

Systematic reviews

We first conducted a systematic review to assess the adherence of RCTs evaluating social and psychological interventions to existing reporting standards, and to identify potential items for the CONSORT-SPI 2018 checklist and flow diagram [11].

Online Delphi process

We then conducted an international online Delphi process between September 2013 and February 2014 to prioritise the list of potential items for the CONSORT-SPI 2018 checklist and flow diagram that were identified in the systematic review. To encourage widespread participation, we published commentaries in several journals publishing trial reports in the fields of addiction, criminology, education, adult and child psychology and psychiatry, public health, and social work [11, 62,63,64,65,66,67,68], directing readers to a recruitment website where they could register. We also invited members of professional bodies, funders, policymakers, journal editors, practitioners, user representatives, and other stakeholders to participate. We encouraged all identified stakeholders to invite any further colleagues to participate. We sent these participants a two-round survey to rate the importance of including proposed items in the CONSORT-SPI 2018 checklist and to provide qualitative feedback (survey items can be accessed at the project’s ReShare site: https://doi.org/10.5255/UKDA-SN-851981). We synthesised the results of the first survey and sent these to participants, who then completed the second survey, which was designed to explore areas of disagreement and to resolve questions arising during the first round.

Consensus meeting

Following the Delphi process, we held a three-day in-person consensus meeting to determine the content of the CONSORT-SPI 2018 checklist and flow diagram, as well as the accompanying E&E document (March 2014). We used established methods [69] from previous CONSORT meetings [29, 35, 41, 70]. Participants included 31 experts from the Delphi process (see Table 6 in the Appendix), whom we selected purposively to include key stakeholders from targeted disciplines (e.g. public health, social work, education, criminology, and clinical psychology) and professional roles (e.g. trialists, funders, and journal editors) [71].

Prior to the meeting, we sent participants background literature [9, 11, 26, 39, 61, 64, 72], results from the Delphi process, and the meeting agenda. On the first day, participants discussed the background literature and its applicability to the various disciplines and professional roles represented at the meeting. During the second day, participants discussed and voted on potential checklist and flow diagram items nominated during the Delphi process using anonymous electronic ballots. On the third day, participants voted on the remaining items and discussed strategies for dissemination. Participants were asked to consider the value of each item based on the evidence presented and to vote on whether each item was essential when reporting all social and psychological intervention RCTs. When voting, participants could select ‘exclude’, ‘include’, or ‘unsure’.

In the first round of voting, only items endorsed as ‘include’ by ≥70% of participants were included in the checklist [73, 74]. We excluded all other items unless at least two participants proposed they be reconsidered. In this second round of voting, items endorsed as ‘include’ by ≥80% of participants were also incorporated in the CONSORT-SPI 2018 checklist. Participants suggested that several ‘excluded’ items should be discussed in the E&E document.

Post-meeting activities

After the consensus meeting, we finalised the CONSORT-SPI 2018 checklist and flow diagram. We then drafted the Extension Statement (this manuscript), as well as an E&E document that serves as a user manual for the checklist. We distributed these documents to consensus meeting participants for feedback and revision, and we incorporated their comments in the final version of this manuscript and the accompanying E&E. We also discussed how best to optimise our strategy for disseminating and implementing these documents.

Results

Systematic review

The systematic review of reporting guidance identified 14 relevant reporting guidelines and 5 reporting assessment tools. These tools included a total of 147 potential items to consider for the CONSORT-SPI 2018 checklist, 89 of which were not included in the CONSORT checklist [11].

Online Delphi process

With input from the project’s International Advisory Group, we included 77 potential checklist items from the systematic review in the first round of the modified Delphi process. We recruited 384 Delphi participants from 32 countries working in over a dozen areas of social and psychological intervention, including academics, researchers, practitioners, journal editors, research funders, policymakers, and recipients of social and psychological interventions. The Delphi process yielded 58 potential items as important to consider for inclusion in the CONSORT-SPI 2018 checklist.

Consensus meeting

During the consensus meeting, participants voted to extend 9 of the 25 items in the CONSORT 2010 checklist: background and objectives, trial design, participants, interventions, statistical methods, participant flow, baseline data, outcomes and estimation, and funding. These extended checklist items addressed the need for reports of RCTs of social and psychological interventions to describe: the hypotheses for how the intervention might work, the eligibility criteria for settings and providers, the actual provider delivery and participant uptake of the interventions, the intervention materials, how missing data were handled, participant recruitment, socioeconomic baseline variables, availability of trial data, author declarations of interest, involvement of the intervention developer in the trial, and details of any incentives offered (Table 1). Participants also voted to add a new item about stakeholder involvement, and they recommended modifications to existing CONSORT 2010 checklist items (Table 2). The flow diagram (Fig. 2) to address the unique needs of social and psychological intervention trials was also modified—specifically, the number of participants approached during enrolment and the number of providers, organisations, and areas (as appropriate) allocated to each trial arm. To further facilitate use of CONSORT-SPI 2018, we have provided a tailored CONSORT Extension for Abstracts (Table 3) [39] and a CONSORT Extension for Cluster Randomised Trials (Tables 4 and 5) [33] for social and psychological intervention trials.

Table 1 The CONSORT-SPI 2018 checklist
Full size table
Table 2 Noteworthy changes to CONSORT 2010 items in the CONSORT-SPI 2018 checklist
Full size table
Fig. 2
figure 2

The CONSORT-SPI 2018 flow diagram

Full size image
Table 3 Items to report in journal or conference abstracts for social and psychological intervention trials [39]
Full size table
Table 4 Items to report in the abstract for cluster randomised social and psychological intervention trials [33]
Full size table
Table 5 Items to report in the main text for cluster randomised social and psychological intervention trials [33]
Full size table

Discussion

The CONSORT-SPI 2018 Extension is designed to assist authors in writing reports of social and psychological intervention RCTs and to assist peer reviewers and editors in assessing these manuscripts. While we recommend that authors report items in the checklist in the relevant manuscript section (i.e., introduction, methods, results, or discussion), the format of an article will depend on journal style, editorial decisions, expectations within a particular research area, and author discretion. At a minimum, authors should address each checklist item somewhere in the article with the appropriate level of detail and clarity. We recommend subheadings within major sections—particularly the methods and results sections—to help ease of reading. The accompanying CONSORT-SPI 2018 E&E document is a user manual for the CONSORT-SPI 2018 checklist, providing a concise rationale for and description of how best to adhere to each checklist item. We recommend that authors preparing reports of social and psychological intervention RCTs consult the CONSORT-SPI 2018 E&E document when using the CONSORT-SPI 2018 checklist.

This guideline may prove useful to several different stakeholders [75]. Researchers can use CONSORT-SPI 2018, along with the SPIRIT Statement, during trial design to ensure they consider the essential study aspects they will have to describe in future manuscripts. Use of CONSORT-SPI 2018 throughout a trial (from design to reporting) can help improve the accuracy, completeness, and transparency of the final manuscript. Journal editors can enforce policies and procedures to ensure that CONSORT-SPI 2018 is actually used by authors, editors and peer reviewers to improve the social and psychological RCT manuscripts they publish [76]. Research funders who adopt CONSORT-SPI 2018 and other reporting guidelines may receive higher quality grant applications, as well as facilitate the commissioning of the most important and rigorous studies while helping to reduce research waste. Policymakers, practitioners, and systematic reviewers who encourage researchers to use CONSORT-SPI 2018 may find this leads to higher quality publications, which these stakeholders can then use to identify and implement effective interventions for populations and settings of interest. In addition, faculty could use reporting guidelines to train the next generation of researchers, peer reviewers, and journal editors [77].

In highlighting prospective trial registration [78], the publication of protocols [79], and increased sharing of trial data [16, 80], all of which are uncommon in social and psychological intervention research, CONSORT-SPI 2018 also complements other efforts to improve research transparency. Examples of such efforts include the Template for Intervention Description and Replication (TIDieR) checklist (which will replace CONSORT 2010 Item 5) [9], the Behaviour Change Technique taxonomy [21, 44], the Berkeley Initiative for Transparency in the Social Sciences [81], the Data Access and Research Transparency Statement [82], the Center for Open Science [19], the Transparency and Openness Promotion guidelines [16], and the Human Behaviour-Change Project [83].

Strengths and limitations

We followed recommended best practices in the development of these reporting guidelines and advocate their use to future reporting guideline developers [26]. A challenge that we experienced, and which other reporting guideline developers have faced [84], was the large number of potential checklist items that participants considered to be important for a CONSORT-SPI 2018 guideline. As with the CONSORT 2010 Statement, CONSORT-SPI 2018 represents a set of minimum reporting criteria and does not preclude individual authors from addressing other issues that they deem important to ensure complete and transparent reporting. For example, for social and psychological interventions utilising mobile phones, additional details may need to be reported in trial manuscripts [85].

In addition, as in the development of previous CONSORT guidelines, other items fundamental to an RCT have not been included (such as approval by an institutional ethical review board) because journals and institutions address these issues in other ways [29]. We encourage users of this guideline to provide feedback on the appropriateness of the content in the CONSORT-SPI 2018 checklist and its accompanying E&E document.

Endorsement

As a recognised extension of the CONSORT 2010 Statement, journals and organisations already endorsing the CONSORT guidelines can easily extend their support to CONSORT-SPI 2018. We encourage other journals and organisations that publish social and psychological intervention RCTs to endorse CONSORT-SPI 2018 and to register their official support on the CONSORT website (http://www.consort-statement.org/about-consort/endorsement). Journal endorsement policies that include monitoring of adherence to the checklist are essential for complete and transparent reporting [31]. To maximise the potential impact of CONSORT-SPI 2018, editors should consider requiring authors to submit a completed CONSORT-SPI 2018 checklist as a separate document when reporting social and psychological intervention RCTs, and we recommend that editors should check that all items have been included before sending manuscripts for peer review. Endorsing journals should consider adding the following statement to their ‘Instructions to Authors’ [36]:

JOURNAL NAME requires a completed CONSORT-SPI 2018 checklist as a condition for submitting manuscripts about randomised trials of social and psychological interventions. We recommend that your submission addresses each item in the CONSORT-SPI 2018 checklist. Taking the time to ensure your manuscript meets these basic reporting requirements will greatly improve your manuscript, and may potentially enhance its chances for eventual publication.

We also recommend that researchers, editors, peer reviewers, funders, and educators consult the CONSORT website (http://www.consort-statement.org) for other relevant CONSORT Extensions (e.g. the extension for cluster randomised trials) [33], as well as the Enhancing the Quality and Transparency of Health Research (EQUATOR) Network for up-to-date information on other reporting guidelines (http://www.equator-network.org) that may be of relevance to their study.

Conclusion

CONSORT-SPI 2018, like other CONSORT guidelines, is an evolving tool that requires regular reappraisal and modifications as new evidence emerges and as scientific consensus changes. We invite interested stakeholders to contact us with feedback or to contribute to the guideline’s ongoing development, including individuals or groups who wish to translate the CONSORT-SPI 2018 checklist into other languages or those who wish to evaluate the impact of the CONSORT-SPI 2018 checklist on future trial reporting [31, 86]. To provide feedback and access the most recent version of the CONSORT-SPI 2018 checklist and E&E document, visit the project (https://www.birmingham.ac.uk/schools/social-policy/departments/social-policy-sociology-criminology/research/projects/2017/Consort-SPI.aspx) and CONSORT websites (http://www.consort-statement.org).