Introduction

End-stage kidney disease (ESKD) is a chronic, devastating illness in children with an incident rate of approximately 8 cases per million of the population in Australia [1]. Kidney transplantation is the first-line treatment for ESKD in children [2, 3]. Treatment guidelines for children receiving kidney transplants rely heavily on empirical evidence derived from studies in adults as a result of the under-representation of children in randomized controlled trials (RCT) throughout the medical literature [4, 5]. The paucity of pediatric-specific research is multifactorial, but translates into sub-par evidence for clinical decision-making in the treatment of children with ESKD. It is therefore of the utmost importance that pediatric research that has been done or will be undertaken has both the best design and transparent reporting as these elements are fundamental to clinical research.

Although there is a correlation between poorly designed RCTs and poor trial reporting, it has been widely recognized that poor reporting quality alone can lead to exaggeration of treatment efficacy, and that deficiencies in reporting trials are unfortunately very common even in well-designed clinical trials in prominent medical journals [6, 7]. In response to the substantial gap in the optimal reporting of RCTs, the CONSORT (CONsolidated Standards Of Reporting Trials) statement was developed by two international, multi-disciplinary work groups to re-establish robust reporting standards as the foundation for reliable, evidenced-based research [8]. Verification of improvement in the quality of reporting with implementation of the CONSORT statement was followed by its widespread endorsement by leading medical journals and the completion of internal audits in several medical disciplines [9, 10].

Measurement of the reporting quality of pediatric trials to date has been limited to a few studies completed in psychology, dentistry, complementary medicine, and cerebral palsy physiotherapy [11–14]. Although no studies have examined the quality of reporting in pediatric renal transplantation, trials in adults investigating immunosuppressive interventions for kidney transplant recipients have demonstrated failings in trial reporting, with clinicians expressing that poor reporting quality contributed to difficulty in establishing best practice despite a substantial body of literature in this field [15, 16].

In this review, we aimed to evaluate the quality of reporting of transplantation trials in children published in contemporary biomedical literature. By measuring and describing the standard of reporting, we aimed to highlight what has been reported well, and what has been under-reported, to provide a bench-mark against which future improvement in reporting standards can be measured. Furthermore, we aimed to explore any potential differences in reporting standards among trials involving only children and trials involving a mixed population of adults and children.

Materials and methods

Review design

All RCTs and quasi-randomized trials (where allocation was not truly random, but based on day of week or patient record number or similar) in kidney transplant recipients that had enrolled at least one child (defined as age 17 or less), and that were reported from 2000 onwards were eligible for inclusion in the review. These trials could involve either children alone (referred to in this paper as pediatric trials), or mixed populations of adults and children (referred to in this paper as mixed population trials). Trials enrolling only adult participants (18 years and over) were excluded, as were trials in which recipients had received another solid organ in addition to kidney transplantation. We included only reports of trials published in peer-reviewed biomedical literature, and excluded trials reported only as abstracts from conference proceedings or in non-peer-reviewed journals. Non-English language publications were excluded because of a lack of feasibility as there were no resources available for translation. There were no other exclusions.

Identification of cohort of trials for inclusion

Relevant trials were obtained from the Cochrane Renal Group’s specialized register of RCTs (2000 to 30 April 2008). The search was designed to retrieve contemporary reports of RCTs published since CONSORT reporting standards became widely accepted, and so did not include reports prior to this century. The search strategy was developed in collaboration with the Cochrane Renal Group’s trial search coordinators, and included trials assigned the keyword “Child”, OR trials assigned any or all of the medical subject heading (MeSH) terms “adolescence”, “adolescent”, “child”, or “infant”. Transplantation trials were further separated from trials investigating general renal medical conditions, chronic kidney disease, peritoneal dialysis or hemodialysis by a subset search strategy using text words (transplant*), (donor*), (graft*), or (recipient*). Where necessary, we reviewed the full citations to clarify all trial reports meeting the inclusion criteria. A second reviewer (AW) independently reviewed the search results and discrepancies were resolved by discussion. In the event of identification of multiple reports from the same trial, only the earliest publication reporting fullest clinical results was selected, and subsequent reports were discarded. In doing this, we reasoned that the first report of any trial to enter the public domain might be expected to report methods clearly and unambiguously.

Assessment tool

The revised CONSORT statement was used as a basis for assessing reporting quality. CONSORT is a validated, evidence-based guideline demonstrated to improve the quality of reporting of RCTs, which has been endorsed by the International Committee of Medical Journal Editors, and by over 343 medical journals including the New England Journal of Medicine and The Lancet, as well as the leading nephrology and transplantation journals (such as the Journal of the American Society of Nephrology, the American Journal of Transplantation and Transplantation) (Fig. 1) [8, 9, 17].

Fig. 1
figure 1

The 2001 revised CONSORT statement checklist of essential items that should be included in reports of randomized controlled trials (RCTs). This checklist was used to score reports of transplantation trials including children in this review

Full size image

Data abstraction and analysis

Data abstraction was completed by a single reviewer (RB) not blinded to trial authors or journal, with any uncertainties discussed with a second reviewer (AW) using a standardized data form to document trial characteristics and the presence or absence of CONSORT checklist items. The reporting quality of included trials was assessed using the 22-item CONSORT checklist (Fig. 1), with the successful reporting of a CONSORT criteria item only awarded if all of the required components detailed in the CONSORT statement were completely satisfied [8]. Characteristics of each included trial were summarized descriptively, and quantitative frequencies for each individual item and overall trial CONSORT scores were tabulated. Comparisons of the reporting quality of CONSORT items were analyzed as binary variables (did not meet CONSORT requirement/did meet CONSORT requirement) using a two-tailed Fisher’s exact test. Overall CONSORT score in pediatric trials versus mixed population trials was analyzed using the two-tailed non-parametric Mann–Whitney U test. A value of P < 0.05 was considered significant.

Results

A total of 1,775 abstracts were identified by the initial search strategy. A full paper review was undertaken for 161 trials that could not be excluded on title and abstract alone, and 27 were identified as meeting the inclusion criteria (Fig. 2). The 27 trials included comprised a total of 6,082 randomized participants; 928 participants in pediatric trials and 5,154 participants in mixed population trials containing both adults and children. For the mixed population trials it was not always clear exactly how many children were included, of the total randomized participants. The 27 trials included were reported in 12 different journals. The characteristics of the trials included are summarized in Table 1.

Fig. 2
figure 2

Identification of reports of randomized controlled trials for inclusion in the systematic review of the quality of reporting of transplantation trials in children. Criteria for inclusion: all randomized controlled trials of kidney transplant recipients, reported in peer reviewed journals, from 2000 onwards, in the English language, conducted either exclusively in children, or including children within a mixed population of child and adult participants

Full size image
Table 1 Characteristics of trial reports included in review, stratified into those conducted exclusively in children, and those in mixed populations of both adults and childrena
Full size table

CONSORT items: title, abstract, and introduction

Only 4 out of 9 (44%) pediatric trials and 10 out of 18 (56%) mixed population trials described the trial as randomized in the title of the publication, but all the trials included appropriately described the fact that participants were randomly allocated to interventions in the trial abstracts (Table 2).

Table 2 Identification of CONSORT criteria items (1 through 12) presented in the title, abstract, introduction, and methods of the kidney transplantation trials included in the study
Full size table

CONSORT items: methods

With the exception of the excellent reporting of specific objectives and hypotheses (27 out of 27) and the statistical methods for planned analyses (26 out of 27), the essential components of a well-reported methods section were absent from many of the trials (Table 2). Clearly defined trial eligibility criteria and sufficient information on data collection settings were not reported in one-third of the papers. Precise details of the interventions intended for each group and how and when they were received was reported in 78% of the trials included (7 out of 9 pediatric trials, 14 out of 18 mixed population trials). Sample size calculation was poorly reported with only 4 pediatric trials (44%) and 9 mixed population trials (50%) mentioning this information. Randomization reporting was unsatisfactory as many trials failed to include detailed descriptions of the methods used to generate the random allocation sequence (10 out of 27), to implement allocation concealment (9 out of 27), and to separate creation of the allocation sequence from assignment to the study groups (8 out of 27). Pediatric trials reported these randomization details marginally better than mixed population trials (44%, 56%, and 44% respectively compared with 33%, 22%, and 22% respectively), but the difference was not statistically significant. Blinding of participants, intervention administrators, and outcome assessors was under-reported. In particular, understanding who was blinded to treatment intervention was frequently unclear. Thirteen trials (48%) were clearly identified as open-label studies. For the remaining trials, only 2 mixed population trials (7%) provided complete details on blinding. Five trials (19%) reported partial information whereby at least one of the participants, administrators, or assessors was specifically reported as blinded. Three trials (11%) gave no other information beyond stating that the trial was either “double-blinded” or “blinded.” There was no statistical difference (P > 0.05) in the quality of reporting for any methodological items between pediatric and mixed population trials.

CONSORT items: results

Although the flow of participants through each stage of the study was reported in 8 of the pediatric trials (89%) and 12 of the mixed population trials (67%), only 6 trials (22%) in total made adequate use of the recommended CONSORT flow diagrams, with fewer mixed population trials (11%) than pediatric trials (44%) using these visual flow diagrams (difference not significant P > 0.05, Table 3). When describing the participant analysis set, we found appropriately reported intention-to-treat (ITT) analysis in only 13 trials (48%). Furthermore, 5 trials (28%) mislabeled their analyses as ITT when they were in fact not examined using a full analysis set. Outcome reporting was extremely poor, with only 9 trials (33%) reporting complete details for all trial arms of the intervention summary results, estimated effect size, and precision. Ancillary analyses including exploratory and sub-group analyses were common with one third of trials in total completing analyses not pre-specified in the methods section. There was no statistical difference (P > 0.05) in the quality of reporting for any CONSORT results items between pediatric and mixed population trials.

Table 3 Identification of CONSORT criteria items (13 through 22) presented in the results and discussion sections of the kidney transplantation trials included in the study
Full size table

CONSORT items: discussion

Reporting of result interpretation was poor with only 10 trials (37%) in total clearly discussing both the summary of key findings and the trial limitations (Table 3). Failure to report these items was largely a result of deficiencies in reporting any trial limitations, with only 2 pediatric trials (22%) and 8 mixed population trials (44%) describing this CONSORT item as recommended. All included trials interpreted their results in the context of other current evidence, but 0 pediatric trials and only 3 mixed population trials (17%) incorporated a systematic review of current evidence into the discussion section. There was no statistical difference (P > 0.05) in the quality of reporting for any of the required CONSORT discussion item components between pediatric and mixed population trials.

Overall CONSORT score

As demonstrated in Fig. 3, neither pediatric nor mixed population trials scored well overall when compared against the 22 item CONSORT checklist. In fact, one trial scored as low as 8 out of 22 of the CONSORT criteria (36%) [39]. The difference between the mean CONSORT scores for pediatric trials (14.78) and mixed population trials (14.50) was not statistically significant (P = 0.56, Mann–Whitney U test).

Fig. 3
figure 3

Quality of reporting of included kidney transplantation trials: the percentage of CONSORT criteria that were completely reported for each pediatric (black) trial and mixed population (gray) trial. Pediatric trials comprised solely children 17 years or younger, and mixed population trials consisted of a mixed population of adults and children in which there must be at least one child aged 17 years or younger. Mean CONSORT criteria score for the pediatric trials was 14.78 (±2.68; 67%) and 14.50 (±3.94; 66%) for the mixed population trials (P = 0.56, Mann–Whitney U test)

Full size image

Discussion

Summary of main findings

This systematic review of trial reporting quality in pediatric transplantation trials indicates that it is generally not optimal, and has room for further improvement when evaluated against the established reporting criteria endorsed by the majority of journals that published these reports. None of the trials included completely satisfied the requirements of the CONSORT statement despite all but one of these trials being published in journals that endorse CONSORT (as of May 2009) [39]. Moreover, the trials on average reported absolutely no information for one third of the recommended 22-item CONSORT checklist. In general, while the CONSORT statement requirements for titles, abstracts, and introductions were well reported, the reporting of essential components of the methods, results and discussion domains was unsatisfactory.

This review identified no evidence of discrepancy between the quality of reporting of trials that included only children compared with trials in which children were a subset of patients in an otherwise adult study. Trials including children alone were no better or no worse than those that included both children and adults, with no statistical difference identified between the reporting quality in any of the 22 required CONSORT items, nor any significant difference (P = 0.56) in the overall average CONSORT criteria scores. We acknowledge that the sample size of our trial cohort was small, but nevertheless, this suggests that sub-optimal reporting of trials might be a widespread problem, and that report standards of RCTs in both adult and pediatric transplantation require major improvement [16].

Findings in context

To our knowledge this is the first systematic review of the reporting quality of RCTs in the pediatric transplantation population, but is among a small but growing number of studies assessing reporting standards in the broader pediatric literature [11–14]. Other studies in reporting quality in pediatric research to date have been limited to other fields and specialist areas and findings from these studies have unfortunately been equally unimpressive, showing similar deficiencies in reporting standards. Despite our findings in the transplantation trials, the results of our study compare favorably with those of the other pediatric settings. We demonstrated that the reporting of pediatric transplantation trials met an average of 67% of the CONSORT criteria for trials including only children and 66% for trials including a mixed population of adults and children. In contrast, only 40% of the CONSORT criteria items in complementary medicine reports in children and 47% of the CONSORT criteria items in reports in cerebral palsy physiotherapy were well reported [13, 14]. Another study in pediatric psychology found that half of CONSORT items were reported less than 25% of the time [11]. Although this is not the first time that many of these reporting deficiencies have been identified in the broader literature, our findings provide a somber summary of the quality of the reports of trials available to inform treatment choices in pediatrics.

Strengths and weaknesses

Our systematic review identified and summarized all available trials from the Cochrane Renal Group’s specialized register of RCTs that met our inclusion criteria in an effort to capture the totality of RCT evidence available on kidney transplantation in children. In an effort to maintain a robust methodology, two different reviewers worked independently to identify trials, and in data abstraction agreement was sought with a second reviewer for any data that were in any way ambiguous. However, although there is the intention to be meticulous, this type of research is time-consuming and open to human error, which is not helped by an absence of structured rational reporting in many of the trial reports, and differences in the formatting of manuscripts among different journals [14].

The exclusion of non-English publications has been demonstrated to introduce bias into meta-analyses of interventions. A limitation of our work was to exclude non-English language literature, due to difficulty in obtaining translations of these reports, and it is possible that in this way we have introduced some bias to our findings [45].

In limiting this review to only transplantation trials, we included a relatively small number of trials, and as a result we may have had inadequate power to detect differences in reporting quality between pediatric and mixed populations trials that do exist. In fact, only one third of trials assessed in this review involved transplantation in an exclusively pediatric population. This may also explain why the findings were more comparable to the results of a systematic review of renal transplantation immunosuppression in an adult population (adequate reporting of 69.1% of the CONSORT criteria) rather than those of other pediatric systematic reviews that had lower overall reporting adequacy [16]. Thus, a larger work examining the totality of RCTs related to pediatric nephrology might be more informative.

Implications for clinical practice and conclusion

Consistent with the literature in other medical disciplines, there is much room for improvement in the reporting of RCTs in children requiring kidney transplantation. Although a multitude of factors contribute to the under-representation of children in randomized trials compared with adult-centered research, our findings are the first to suggest that the quality of reporting in pediatric transplantation trials is no worse than that of the adult literature. However, it does suggest that even when children with kidney transplants do take part in an RCT, the results of these trials are not reported as adequately and as transparently as they could be, which is a general failing warranting action.

The evidence supports initiatives such as the CONSORT statement and efforts by the International Committee of Medical Journal Editors to correct reporting deficiencies found throughout the biomedical literature, but more must be done to ensure guidelines are stringently applied to transplantation trials involving children. Endorsement by journals appears to be insufficient alone, and there is evidence that unfamiliarity with the CONSORT criteria by authors and reviewers can contribute to this problem [46].

Responsibility for ensuring that trial reporting is good quality rests with all involved in clinical research, from funders to publishers. Researchers bear responsibility for conducting their research ethically and responsibly, and ensuring that they report details of their work according to best practice, using the appropriate reporting guidelines for their study design, which for an RCT when writing for journals is the CONSORT statement. Peer reviewers are the expert gate-keepers, who as part of peer review bear responsibility for checking research validity, which includes assessing study design and reporting, and alerting journal editors where they find this lacking. Ultimate responsibility for reports of RCTs appearing in journals rests with the journal editorial board.

Better training of junior researchers in research methodology might improve the standards of research writing, and peer review. Increasing literacy in trial reporting among trainees, junior researchers, and scientists might be addressed by offering training opportunities as part of postgraduate continuing medical education (CME). Professional societies might endorse this by including trial design and reporting workshops as part of CME education at conferences and scientific meetings.

It is encouraging that there is a movement toward better reporting within the transplant community with CONSORT-derived, transplantation-specific reporting criteria currently being recommended and developed for individual journals [15, 46, 47]. Unfortunately, history has demonstrated that guidelines only benefit the quality of reporting when they are followed.