Background

The results from randomized clinical trials (RCTs) provides the best clinical evidence and forms that basis for clinical practice in medicine [1]. The knowledge gleaned from RCTs can change the clinical management of patients and practice patterns of clinicians. Given the importance of RCTs, sufficient information on intervention details must be provided in the trial reports to allow the reader to come to his or her conclusions about the recommended treatment. There must be a minimum standard in the quality of reporting of randomized trials so that information is presented in a complete and unambiguous manner [2]. For radiation oncology, the quality of radiotherapy (RT) reporting is important for several reasons. Firstly, radiation treatment parameters such as the radiation prescription, planning process and treatment delivery should be clearly reported to allow accurate reproducibility of the treatment in real world practice. Secondly, clear specifications of treatment parameters will minimize treatment variations that can affect patient outcomes. Thirdly, radiation oncology is a discipline, which is heavily dependent on quality assurance (QA) [3]. Newer and modern developments in radiation oncology have led to new planning systems and treatment delivery machines being used for patient treatment. Routine reporting of the use of QA is now expected in RCTs to ensure that participating centers are able to deliver radiation in a clinically consistent and reproducible manner.

The CONSORT statement provides evidence based minimum set of recommendations for reporting of randomized trials [4]. It aims to help authors prepare their reports in a transparent and unbiased manner. The details of the ‘interventions’ in the checklist of the CONSORT checklist entails reporting of the interventions for each group with sufficient details to allow replication of the intervention [5]. However, the minimum standard of reporting for intervention will vary from one discipline to the other. In addition, there is limited evidence available on the reporting of radiotherapy treatment in RCTs.

Prostate cancer has the highest incidence amongst men in developed countries [6]. External beam radiotherapy (EBRT) is one of the options in the curative treatment of prostate cancer. Many RCTs have been conducted to improve the outcome of patients with prostate cancer, both in terms of improving biochemical disease free survival, overall survival and reducing treatment toxicity. Radiation treatment has advanced from 2 dimensional radiotherapy (2DRT), to 3 dimensional conformal radiotherapy (3DCRT) and to advanced technologies such as intensity modulated radiotherapy (IMRT) and proton beam therapy in the modern era today [7].

A previous report by Bekelman and colleagues examined the quality of radiotherapy reporting in RCTs of Hodgkin’s and Non-Hodgkin’s lymphoma and showed that the reporting of radiotherapy was deficient [8]. Only 38% of the included trials described the target volume; 21% of trials specified the RT prescription point and 20% of included trials described using a RT QA process. It is unclear if the same observation can be made for RCTs of prostate cancer.

The aim of our study is to assess the quality of prostate RT treatment reporting in RCTs of prostate cancer, the factors associated with adequate quality reporting and its impact on the bias in reporting of the trial’s primary efficacy and toxicity outcomes.

Methods

Trial criteria

This study incorporated randomized trials including patients with histologically proven prostate cancer. At least one of the intervention arms needed to include curative intent radiotherapy treatment to the prostate. If the trial had more than one report, all the reports were assessed but would be counted as one trial. We used the trial protocol in the assessment of quality of radiotherapy reporting if they were referenced in the trial report or provided as supplementary materials accompanying the trial report.

Search strategy

From January 1996 to December 2016, we searched MEDLINE (PubMed) for “Prostate neoplasms” and “Radiotherapy”. We limited our search to publications in the English language and to randomized trials. We then hand searched the results for eligible trials. The final selection was made from reading the full text article.

Data extraction

We had 3 independent reviewers using a standardized form to collect the trial data.. We extracted trial characteristics such as sample size, interventions, primary outcome measures and publication characteristics such as year of publication, impact factor of journal in the year of publication for each report.

Quality assessment

We assessed the quality of RT treatment reporting in the trial report primarily and used information in the trial protocol for assessment if it was mentioned in the trial report or was part of supplementary materials. Based on the Radiation Therapy Oncology Group (RTOG) Procedure Manual (available from https://www.rtog.org/Researchers/PoliciesManuals/ProceduresManual.aspx), we used 10 RT reporting measures to assess the quality of the prostate RT treatment reporting. The 10 measures are: RT dose prescription method; RT dose-planning procedures; organs at risk (OAR) dose constraints; target volume definition, simulation procedures; treatment verification procedures; total RT dose; fractionation schedule; conduct of QA as well as presence or absence of deviations in RT treatment planning and delivery. The adequacy definitions for these measures are summarized in Table 1. Any differences were resolved consensually between the 3 reviewers. For each trial, key descriptors were scored as “No” if they were absent in the trial report/ protocol, and “yes” if these were reported We defined a trial as having adequate quality in reporting of RT treatment if seven or more key descriptors were reported adequately.

Table 1 The Key Descriptors (quality measures) and the corresponding definition for adequacy. Ten Key Descriptors are described: Target volume definition, definition of dose planning procedures, radiation dose specification, fractionation specification, radiation prescription point specification, OAR constraints, simulation procedures, verification procedures, QA process for RT, QA process adherence reporting for RT
Full size table

Assessment of bias in the reporting of the trial’s primary efficacy and toxicity outcomes

We defined biased reporting using published criteria [9]. Bias in the reporting of primary efficacy endpoint is defined as no statistical significant difference in primary efficacy endpoint yet one of the treatment arms was highlighted as beneficial. Bias in the reporting of toxicity endpoint was explored using a hierarchy scale from 1 (excellent) to 7 (very poor) to indicate whether reporting of toxicities occurred in the abstract, discussion, concluding statement or results table (Fig. 1). Reports with scores 5 to 7 were judged as having bias in the reporting of toxicity endpoint.

Fig. 1
figure 1

Bias in reporting of toxicity endpoint assessment was explored using a hierarchy scale from 1 (excellent) to 7 (very poor) to indicate whether reporting of toxicities occurred in the abstract, discussion, concluding statement or results table (Fig. 1). Reports with scores 5 to 7 were judged as having bias in the reporting of toxicity endpoint

Full size image

Statistical analysis

The descriptive statistics were presented as frequencies and percentages. Multivariable logistic regression was performed (using forward selection) to determine the potential factors associated with adequate quality reporting. Continuous variables such as impact factor, year of publication and sample size were recategorized as dichotomized nominal variables into various categories determined a priori. Univariable logistic regression was used to determine the impact of adequate reporting on the bias of reporting of primary efficacy and toxicity endpoints. Factors with P < 0.05 in logistic regression were considered statistically significant. All statistical analyses were performed using Stata version 13 (StataCorp LP, TX, USA).

Results

Selection of trials

The result of the search strategy was summarized in Fig. 2. We identified 59 eligible trials.

Fig. 2
figure 2

Results of the search strategy. Four hundred thirty-six records were identified. One hundred one full text articles were assessed for eligibility. Fifty-nine studies were included in the analysis

Full size image

Characteristics of trials

The characteristics of the trials are shown in Table 2. Majority of the trials treated patients with 3D conventional radiotherapy techniques. 73% of the trials had a sample size of more than 200 patients and 80% of trials were not industry sponsored.

Table 2 Characteristics of all trials. Majority of trials treated patients with 3D conventional techniques, had a sample size of 200 or more, were not industry sponsored, were listed in clinicaltrials.gov, were RT focused, had non-overall survival as the primary efficacy endpoint and were published in Europe. There were more trials published in radiotherapy journals, published form 1996-2005, published in journals with impact factor of more then 15. Majority of trials were non-cooperative group trials, did not publish their trial protocol, did not publish their QA processes, and did not have biases in reporting of primary efficacy endpoints and toxicity
Full size table

Quality of RT reporting

Table 3 shows the number of trials and the quality measures they reported. Almost all trials reported target volume definition, total radiation dose and fractionation adequately. Radiation prescription point was reported in 37 (63%) trials. Description of dose planning procedures was present in 47 trials (80%). Verification procedures were reported in 27 trials (46%). OAR constraints, simulation procedures QA process for RT, QA process adherence reporting for RT was reported in approximately one third of all trials. Six trials (10%) reported all ten quality measures adequately. Figure 3 summarizes the number of quality measures that were adequately reported by the included trials. Trials included and their scores for each criteria is shown in Additional file 1.

Table 3 Quality of radiotherapy reporting. The quality measures are shown on the left column and the number and percentage of trials which reported that quality measure adequately shown on the right columns. This shows that the reporting of the quality measures were variable. Almost all trials reported target volume definition, radiation dose specification and fractionation specification adequately. Only one third of all trials reported OAR constraints, simulation procedures, and QA adherence reporting for RT adequately
Full size table
Fig. 3
figure 3

Bar graph showing number of quality measures which were adequately reported in all trials. X-axis shows number of quality measures reported. Y-axis shows number of trials reporting the corresponding number of quality measures adequately

Full size image

Factors associated with adequate quality reporting

Twenty-four trials (40%) reported seven criteria or more adequately. Multivariate analysis showed that trials that published their QA results were 4 times more likely than trials that did not published their QA results {odds ratio (OR) 4.3, 95% confidence interval (CI) 0.99 to 19 and p value (p) = 0.05), and cooperative group trials were more 5 times more likely to have adequate quality in reporting (OR4.8, 95% CI 1.0 to 23.0, p = 0.047) compared to non-cooperative group trials. Other factors including region, primary outcome, industry sponsorship, trial design, sample size, journal’s impact factor, publication in radiotherapy focused journals, year of publication, listed in trial registry and trial protocol availability did not predict for adequate quality of reporting (Additional file 2).

Impact on bias in reporting of primary efficacy and toxicity endpoints

Univariable logistic regression showed that trials with adequate quality of RT reporting were less likely to have bias in reporting of treatment toxicity. (OR 0.20. CI 0.04 to 0.99, p = 0.049).

Discussion

To our knowledge, this is the first study to examine the quality of prostate radiotherapy reporting in RCTs of prostate cancer. Our study shows the quality of prostate radiotherapy reporting in RCTs of prostate cancer is variable, with only 40% of included trials reporting seven or more quality measures adequately. The reporting of target volumes, radiation dose and fractionation specifications were adequate in almost all trials. However, OAR constraints, simulation procedures QA process for RT, QA process adherence reporting for RT was poor and was only reported in approximately one third of all trials. Multivariate analysis of the trial and publication characteristics showed that cooperative group trials and trials which published their QA process and were more likely to be associated with adequate quality of radiotherapy reporting.

These findings emphasize the need to standardize the way we report radiotherapy treatment. Documenting and reporting the results of a QA process is an important component of radiotherapy in RCTs [10] Radiotherapy protocol deviations have been shown to lead to adverse outcomes for patients. Ohri et al. performed a meta-analysis to determine the impact of radiotherapy protocol deviations on clinical outcomes in cooperative group trials. He found that the frequency of RT QA deviations ranged from 8 to 71% in the included studies and this was associated with a significant decrease in OS (HR of death = 1.74, 95% confidence interval [CI] = 1.28 to 2.35; P < .001) [11]. This highlights the need for trials to adhere to protocol radiotherapy planning guidelines if patients are to benefit from the proposed treatment.

In Hodgkin’s lymphoma, Weiner et al. reported a randomized controlled trial of intensive chemotherapy with or without low dose radiation in the treatment of stage IIB to IV Hodgkin’s disease in paediatric patients [12]. An early report revealed no advantage of RT in this group of patients. However, in a subsequent evaluation of the data, it was found that there was a 10% survival advantage in patients receiving compliant RT. As much as 30% of patients had treatment deviation which included RT to involved sites. This highlights the need for studies to report protocol compliance. This may encourage and lead to improvements of processes to reduce protocol deviations.

The global quality assurance of Radiation Therapy Clinical Trials Harmonization group aims to broaden the acceptance of clinical trial results by harmonizing and improving the quality assurance of RT implemented world-wide. This will provide a framework for quality assurance in clinical trials [13].

The CONSORT statement proposes broad recommendations for reporting of interventions [14]. The updated CONSORT extension for non-pharmacologic treatment checklist entails reporting of precise details of both the experimental treatment and its comparator. However, specific recommendations are lacking for reporting of radiotherapy treatment. Bekelman and colleagues use six parameters, including target volume description, radiation dose specification, fractionation specification, radiation prescription point specification, QA process use and QA process adherence reporting to assess the quality of RT treatment reporting in Hodgkin’s and Non-Hodgkin’s Lymphoma trials. With improvement in technology, the way we treat patients with radiotherapy is changing. Computed tomography (CT) planning is currently the standard for treatment for many tumour subsites. CT planning allows us to visualize tumor volumes as well as organs at risk [15]. In addition to assessing tumour coverage, we are able to assess OAR constraints using dose volume histograms (DVH). There is increased emphasis on simulation and verification procedures with the implementation of image guided radiotherapy (IGRT) in the treatment of many tumour subsites [16]. In view of this, for our study we adopted 4 additional criteria from the recommendations by RTOG in addition to the 6 proposed by Bentzen et al. [17]and included description of simulation procedures, OAR constraints, simulation procedures and verification procedures in our assessment of quality of radiotherapy reporting.

Up to two thirds of trials reported prescription method adequately. The prescription method is integral to radiotherapy planning and delivery. With increasing use of IMRT for the treatment of prostate cancer, it is important for clinical trials to follow ICRU 83 in the prescribing and reporting of IMRT treatments [18].

Only one third of trials reported organs at risk (OAR) constraints adequately. OARs are extremely important in radiotherapy treatment planning as they typically represent avoidance structures that the planning system would minimize radiation dose delivery to. Increased doses to organs at risk will increase treatment toxicities. Not reporting organs at risk constraints will not allow the readers interpret the toxicity outcomes of the trials properly. Furthermore, overly conservative OAR constraints might lead to compromised target volume coverage, leading to increased risk of tumour recurrence.

One would expect the quality of radiotherapy reporting to improve over the years with increasing use of sophisticated technology for radiotherapy planning and delivery. We analyzed the year of publication as a variable to determine if trials published in the recent years were more likely to have adequate reporting. Interestingly, the quality of radiotherapy reporting did not improve over time. This may be because there are no published consensus guidelines on the reporting of RT treatment technique for trials including radiotherapy treatment.

We conducted a univariable logistic regression analysis, to determine if the quality of radiotherapy treatment reporting influenced the presence of bias in the reporting of primary efficacy and toxicity endpoints. We found that trials with adequate quality of RT reporting were less likely to have bias in reporting of treatment toxicity, but not primary efficacy endpoints. This is intuitive and suggests that trials with adequate reporting of RT treatment techniques may be more familiar with reporting standards of clinical trials.

The strengths of our study is that we used published tools to evaluate the quality of prostate RT reporting. In addition, we included only randomized trials for our study as the results from randomized trials are more likely to be impact clinical practice. Our study was limited by the relatively small sample size. In addition, not all the trial protocols were available for assessment. However, we felt that the primary trial report should describe at least seven key RT descriptors adequately as not all clinicians have time to examine the trial protocol in detail.

Conclusion

The quality of prostate radiotherapy reporting in randomized controlled trials in prostate cancer is variable. Future efforts to develop consensus guidelines to standardize the reporting of radiotherapy treatment are warranted.