Background

Randomized controlled trials (RCTs) which are designed to provide the best quality of evidence required for health care decisions [1, 2] should ideally be reported according to pre-defined standards. Most readers of articles reporting RCT usually start by making an initial assessment of the interest of the article after reading the content of the abstract, which subsequently guides the decision on whether to read the entire article or not [3]. With an overwhelming day-to-day workload, the continuous availability of large volumes of new scientific publications, limited access to many full-text articles (particularly in resource limited settings) [4], many health professionals tend to make recourse to information on abstracts to take health care decisions [3]. As such abstracts of trials reported in journals should contain sufficiently accurate and clear information that permits the reader to get a good synopsis of the findings of the trial [5]. Formerly, there was no standardized method of reporting RCT and this created many discordances in the reporting of RCTs. These discrepancies, stimulated the establishment of the Consolidated Standards of Reporting Trials (CONSORT) statement, the first of which was published in 1996 and revised in 2001 [6, 7], which aimed at “standardizing” and improving the way RCTs are reported [8]. In order to avoid inconsistencies between the content of the abstract and the full text of the article, as was seen in many RCT reports, an extension to the CONSORT statement was published in 2008, which provided a list of essential items to be included for reporting of abstracts of RCTs [9]. This statement comprises seventeen items distributed in eight sections which include: the title, authors contact details, trial design, methods (participants, interventions, objective, outcomes, randomization, blinding), results (numbers randomized, recruitment, numbers analyzed, outcome, harms), conclusions, trial registration and funding [9].

Following the publication of this CONSORT extension for abstract reporting, studies to assess the quality of abstract reporting in general medicine journals and other specific fields of medicine have been done. These studies have shown that there is substantial room for improvement in the adherence to these reporting guidelines [1019].

However, as concerns RCTs in HIV/AIDS medicine, there are limited studies investigating the impact of CONSORT extension for abstract reporting checklist. In this field of HIV/AIDS where guidelines to tackle the pandemic are continually changing, adherence to this statement is particularly important, most especially as HIV/AIDS disproportionally affects resource limited settings (mainly sub-Saharan African countries) where most of the RCTs are done but most of health professionals do not have access to full-text articles required for health care decision making. We therefore, in the present study, using abstracts of the periods before and after the publication of CONSORT extension for abstracts, aim to first of all compare the overall reporting quality of RCTs in HIV/AIDS medicine published in general medicine and infectious diseases journals, and secondly determine the factors associated with better reporting quality.

Methods

Design

We conducted a systematic review of abstracts published in 2006, 2007, 2014 and 2015 in 20 leading general medicine and 20 leading infectious diseases journals based on 2014 impact factor.

Data sources

We conducted a MEDLINE/PubMed search of all RCTs published in the years 2006–2007 and 2014–2015. The search strategy used MeSH terms like “randomized controlled trial” as publication type, journal names and HIV and AIDS as terms in title/abstract. We limited search for the following periods: (2006/01/01 to 2007/12/31 and 2014/01/01 to 2015/12/31). The choice of the time limit of 2014 and 2015 was arbitrary and based on the fact that 6 years after the publication of the first CONSORT statement for abstracts in 2008, is sufficient time for researchers to have become versed with the recommendations. Other types of articles (editorial, case reports, comment, observational study, review, meta-analysis and letters) were excluded. Searches were done regularly during the study period and the last one was conducted on February 26, 2016 as could be seen presented in Additional file 1.

Studies selection

RCT abstracts where interventions were provided to HIV-infected patients were selected. These abstracts were included when participant allocation to interventions was described by the terms “random”, “randomly allocated”, “randomized”, “randomization” or another word in the abstract suggesting that participants were randomly distributed between the trial arms. We considered journals as CONSORT statement endorser if they referenced CONSORT statement in their ‘Instruction to authors/Submission guidelines’ or if they reference as endorser in the CONSORT webpage (http://www.consort-statement.org/about-consort/consort-endorsement/consort-endorsers---journals/). Two reviewers independently selected abstracts.

Data extraction and covariates

Data extraction was independently done by two reviewers using a pre-made and pretested data extraction form in compliance with the items of the recommendations of the evaluation of RCTs using the CONSORT for abstracts [9]. Each question had a ‘yes’ or ‘no’ response for each item indicating whether the authors had reported it or not. Agreement between the two reviewers was measured using the Kappa statistic [20] and discrepancies were resolved by discussion among the authors or by arbitration of a third author. Other information collected from journals other than the CONSORT abstracts items, were journal name, type of abstract format (IMRAD [introduction, method, results and discussion] or eight-heading [objective, design, setting, participants, intervention, main outcome measure, results and conclusions] or one-block), type of interventions in the trial (pharmacological or non-pharmacological), number of authors, publication on behalf of a research collaboration group, and journal field (infectious diseases or general medicine).

Main outcomes definition

The primary outcome was the number of items reported in each selected abstract. The secondary one was the proportion of abstracts reporting each of 17 recommended items.

Statistical analysis

IBM Statistical Package for Social Science (IBM SPSS) version 23.0 for Windows (IBM Corp. Released 2014. IBM SPSS Statistics for Windows, Version 23.0. Armonk, NY: IBM Corp.) was used to code, enter and analyze data. Categorical variables were expressed as numbers with percentages (%). Continuous variables were expressed as means with standard deviation.

We expressed the number of items for each year as mean (standard deviation [SD]). We estimated mean differences using the independent two-sample T-test for unadjusted means and the generalized estimation equations (GEEs) for adjusted means [21]. All these mean differences were reported with 95 % confidence intervals (95 % CI). The χ2 was used to compare compliance with the 17 items of CONSORT extension for abstracts between pre-CONSORT and post-CONSORT for abstracts publication. This was also done using GEE for adjusted analysis. We reported measures of association as odd ratios (OR) for univariate analyses and as adjusted odds ratios (aOR) for multivariate analyses both with their 95 % CI. A p-value < 0.05 was considered statistically significant.

Incidence rate ratios (IRRs) were computed to identify factors associated with better reporting in 2014–15 abstracts using GEE. For this analysis, Poisson distribution and unstructured correlation matrix were assumed. Univariate and multivariate analyses were conducted.

For GEE, adjustments were made for CONSORT endorser journal (yes/no), abstract format (IMRAD/8-heading/one-block), publication on behalf of a group (yes/no), number of authors (≤6/7–12/≥ 13), and journal field (general medicine/infectious diseases). Journal name has been used as grouping factor due to fact that they can similarity in articles published in the same journal. The adjustment was done for abstract format because there are previous studies reporting relationship between abstract format and quality reporting [15, 22]. The adjustment was also done for number of authors and publication on behalf a group because it was reported an association with higher quality of work with number of collaborators [23, 24].

Results

General characteristics of selected abstracts

Our search yielded 444 articles of which 95 were published in 2006, 103 in 2007, 140 in 2014 and 106 in 2015. One hundred and thirty-two abstracts did not meet our inclusion criteria, hence were excluded. We therefore included 76 abstracts from 2006, 83 from 2007 (2006–07, n = 159), 81 from 2014 and 72 from 2015 (n = 153) as shown in the flow diagram (Fig. 1).

Fig. 1
figure 1

Flow chart of included studies

The inter-observer agreement value was 0.86 for abstract selection, 0.71 for overall rating of abstract reporting quality, and varied between 0.74 and 0.98 for each items of CONSORT for abstracts.

All articles came from 19 of 40 journals included in search strategy. Interventions were pharmacological/vaccine in 76 % (238/312) of the abstracts. The IMRAD format was used in 85 % (266/312) of abstracts. Among all the journals, 84 % of abstracts (262/312) were from journals which endorsed CONSORT, 80 % (248/312) were from infectious diseases journals, 63.5 % (198/312) were published on behalf of a group, and 75 % (233/312) were published in journals with impact factor less than 10. The mean number of authors among articles was 12.2 (SD 6.8) (Table 1).

Table 1 Distribution of HIV abstracts by year and characteristics

Overall quality of reporting

Only 1 % (2/153) of abstracts reported all items in the 2014–15 period. The mean number of items reported in 2006–07 (6.2; SD = 2.5; Median 6; Range 1–14) differed significantly from that reported in 2014–15 (7.7; SD = 4.0; Median 7; Range 1–17), mean difference (MD): 1.6; 95 % CI 0.8–2.3; p <0.001.

In the GEE analysis, after adjusting for covariates among the type of intervention, journal endorser of CONSORT, format of abstract, journal field, numbers of authors, journal impact factor and publication on a behalf of a group, the adjusted mean number of items reported in 2006–07 (7.2; 95 % CI 6.6–7.7) differed significantly from that reported in 2014–15 (7.8; 95 % CI 7.3–8.4), adjusted MD: 0.7; 95 % CI 0.1–1.2; p = 0.015.

Quality of reporting of each item

In univariate analysis, there was improvement from 2006–07 to 2014–15 for ‘title’, ‘trial design’, ‘participants’, ‘objective’, ‘randomization’, ‘blinding’, ‘recruitment’, ‘number analyzed’, ‘outcome’, ‘conclusion’, ‘trial registration’ and ‘funding’. However, the quality of abstracts decreased for ‘author contact’. After adjustment, the quality of abstract remained improved for the following items: ‘participants’ (aOR 1.92; 95 % CI 1.05–3.51), ‘randomization’ (p < 0.001), ‘outcome’ of results section (aOR 2.26; 95 % CI 1.17–4.35), ‘trial registration’ (aOR 8.32; 95 % CI 3.66–10.88) and ‘funding’ (p < 0.001). From the 2006–07 period to the 2014–15 period, the quality of the item ‘author contact’ decreased (aOR 0.06; 95 % CI 0.03–0.12) (Table 2).

Table 2 Crude and adjusted odds ratios for adherence to the 12 items of the CONSORT statement for HIV/AIDS abstracts in 2014–2015 compared to 2006–2007

Factors associated with better reporting quality in 2014–2015 abstracts

In univariate analysis, factors associated with better reporting included publishing in general medicine journal, publishing in CONSORT endorser journals, structured abstract (IMRAD or 8-heading), high impact journal, more than 13 authors and publishing on behalf of a research collaboration group. In multivariate analysis, factors independently associated with better reporting of abstracts in 2014–15 included publishing on non-pharmacological/vaccine intervention (aIRR 1.19; 95 % CI 1.03–1.37), more than 13 authors (aIRR 1.39; 95 % CI 1.07–1.79) and high journal impact factor (aIRR 2.16; 95 % CI 1.83–2.54) (Table 3).

Table 3 Unadjusted and adjusted incidence rate ratios for the total number of CONSORT extension for HIV abstract items reported

Discussion

This study aimed to assess, according to the CONSORT for abstract checklist, the differences in the reporting quality of abstracts of RCTs in HIV/AIDS medicine published in general medicine and infectious diseases journals before (in the years 2006–07) and after (in the years 2014–15) the publication of the CONSORT extension for abstracts [9]. Our findings demonstrate that for some items (participants, randomization, outcome of results section, trial registration and funding), the reporting quality of HIV/AIDS RCT abstracts has improved significantly in the post-CONSORT era as compared to the pre-CONSORT era, while for others the quality has remained unchanged or regressed (authors contact). These results are consistent with previous studies that have reported inconsistencies and patterns of non-compliance to the CONSORT for abstracts guidelines in most of journals [1019].

The mean number of items reported increased significantly from 2006–07 to 2014–15. This is in line with the average of 3 more items reported in RCT abstracts in top-tiers of general medicine journals since 2008 as found in a recent study [19]. High impact factor was independently associated with more adherence to CONSORT abstract items as in a study in which high impact factor is associated with high quality reporting for CONSORT items [15, 2530]. Many people and institutions consider journal impact factor as the reflect of articles’ quality [31]. High number of authors was also independently associated with more adherence to CONSORT abstract items as in previous studies [23, 24]. One can argue that the involvement of a large number of authors in an article could improve quality. Many authors would increase the chances that one or more of the authors ensure to the respect of the reporting guidelines. We also found non pharmacological/vaccine intervention in trial a factor for better reporting. This finding requires more investigation to understand this relationship.

As a whole, the reporting of some items improved significantly over time: more abstracts in 2014–15 as compared to 2006–07 identified the study as randomized in the title, provided details on the trial design, randomization, blinding, trial registration and funding. For most abstract elements there was progress in reporting even though this was not significant. One of the main reasons that can explain the improvement of the overall reporting quality of abstracts found in our study is the endorsement of the CONSORT statement by most of the journals surveyed. Indeed, 84 % of articles included in our analysis were published in journals that endorsed CONSORT. A systematic review that examined the effectiveness of the CONSORT statement in journals that have formally endorsed it concluded that journals’ adoption of CONSORT is associated with improved reporting of RCTs [8]. Moreover, another study has also shown that the endorsement of recommendation and effective implementation of CONSORT guidelines lead to the improvement of reporting quality of abstracts [12]. In line with these studies, our findings stress the need to expand journal endorsement and implementation of the CONSORT extension for abstracts before manuscript submission (in journal submission guidelines), during review process and editorial assessment. Authors should be required to submit a completed CONSORT checklist with each manuscript with an emphasis for CONSORT extension for abstracts [32]; reviewers should assess the completeness of abstract reporting; and final manuscript copyediting would surely improve abstract reporting quality [19].

Despite this overall improvement, the quality of reporting of some items remained sub-optimal, consistent with findings from other studies [1019]. Poorly reported items specifically include methods of randomization and allocation concealment, and blinding. These items are essential methodological elements which are critical, therefore adequate standardized reporting is indispensable so as to demonstrate that selection bias was avoided, thereby allowing unequivocal interpretations of the study findings [5]. Nevertheless, less abstracts in 2014–15 reported corresponding authors’ details (postal and email addresses). This is a major drawback when readers need to contact authors for further information on the study. This depends on journal indexing policy, rather than a deficiency in reporting. In addition, author contact details are more relevant for conference abstracts, none of which were included in this study [5].

Though we found that there has been some improvement in HIV RCT abstract reporting, our study had some limitations. We not investigated others factors that can influence reporting quality. These non-investigated factors include abstract word count, types of utilization of CONSORT guidelines in the ‘instructions for authors’ section of journals, and awareness of the CONSORT statement. Furthermore, this study reports on the adequacy of reporting using the CONSORT checklist items, but the accuracy of reporting cannot be assured because a comparison of the abstract and the full text was beyond our scope.

Conclusions

The reporting quality of RCT abstracts in HIV/AIDS medicine in general medicine and infectious diseases journals has sub-optimally improved after the publication of the CONSORT extension for abstracts. This suboptimal improvement is associated with journal high impact factor, high number of authors and non-pharmacological/vaccine intervention in the trial. However, there is still much to do for improvement to meet the standards of the CONSORT for abstracts guidelines. Journal endorsement and more strict adherence to the CONSORT for abstracts standards by both authors and journal editors will contribute to better RCT reports in HIV/AIDS medicine. Further researches are necessary to investigate why authors, reviewers, journal editors, funders, institutions and readers not fully adhere to CONSORT guidelines.