Background

Pancreatic cancer is the fourth largest cancer-related cause of death worldwide, with an overall 5‑year survival rate of 2–7% [1,2,3,4]. Although approximately 15% of patients are diagnosed with a seemingly resectable tumour, 10–35% of these tumours prove to be irresectable during explorative laparotomy [5]. Also, although some studies suggest that the percentage of resectable tumours may be increased by preoperative radiochemotherapy, those studies were mainly single-arm phase II trials and many did not report using the intention-to-treat approach [5].

The Dutch Pancreatic Cancer Group (DPCG) started the multicentre, randomised, controlled PREOPANC trial (Netherlands Trial Registry: NTR 3709), which became open for accrual in April 2013 [6]. In this phase III trial, patients with (borderline) resectable pancreatic tumours are randomized between (1) direct explorative laparotomy and (2) preoperative radiochemotherapy followed by explorative laparotomy.

For pancreatic cancer, it is well known that target volume delineation can be difficult [7]. In our department, the planning CT scan is combined with a contrast medium and a four-dimensional CT scan (4DCT) to determine the respiratory-induced motion of the tumour and neighbouring organs [8]. Because of the large day-to-day variation in tumour position, in our clinic we currently use intratumoural fiducial markers for delineation and daily online position verification [8,9,10,11,12]. The radiation oncologist delineates the gross tumour volume (GTV) on the CT scan and expands this volume to encompass the GTV in all respiratory phases. This expanded volume is defined as the internal gross tumour volume (iGTV). The iGTV combined with an additional margin forms the internal clinical target volume (iCTV), which is then expanded to create the planning target volume (PTV).

A dummy run procedure at the beginning of a clinical trial is a good tool to improve protocol compliance with the radiotherapy prescriptions and to improve uniformity of the treatment. In multicentre trails, a dummy run can diminish heterogeneity in treatment quality and can also be used to detect correctable variations in treatment [13,14,15,16,17,18].

The aim of this dummy run procedure was to evaluate compliance with the radiotherapy protocol, and the uniformity of delineation and treatment planning, among institutions participating in the DPCG PREOPANC trial.

Methods

General

All 11 radiation oncology departments participating in the PREOPANC trial were asked to participate in the dummy run and were encouraged to complete the dummy run before the start of patient accrual. The opening for accrual varied between the different institutions: the first opened in April 2013 and the last in November 2015. In the PREOPANC trial, the preoperative radiochemotherapy consists of a total dose of 36 Gy in 15 fractions of 2.4 Gy (5 fractions/week), with concurrent gemcitabine (1000 mg/m2 on days 1, 8 and 15) preceded and followed by a cycle of gemcitabine (1000 mg/m2 on days 1 and 8) with one week rest in between the three cycles [6].

Dummy run procedure

The ‘dummy’ patient had a borderline resectable pancreatic tumour in the pancreatic tail and was selected by the Quality Assurance (QA) team. This team consisted of one radiation oncologist (GT), one radiation oncology resident (EV), two medical physicists (AB, JV) and two researchers (EL, AH). Institutions were provided with a set of images and clinical data of this ‘dummy’ patient to be uploaded to the local delineation and treatment planning system. The data contained the diagnostic CT scan (4 phases with vascular contrast) with the radiological report, the planning CT scans, including a contrast-enhanced 3DCT (fast scan) and a 4DCT, the dummy run instructions and a questionnaire.

Prior to the planning CT scan, the patient had three intratumoural fiducial markers (Visicoil, Core Oncology, Santa Barbara, CA, USA) implanted by the gastroenterologist during endoscopic ultrasound. In our institution, intratumoural fiducial markers are standard practice for daily position verification of the tumour with cone beam CT [11, 12]. The planning CT scan images were acquired using a GE LightSpeed RT16 scanner (General Electric Co., Waukesha, WI, USA): slice thickness 2.5 mm. First, the CT scan with vascular contrast was obtained (fast scan), followed by a 4DCT scan which provided the respiratory-induced motion of the tumour in 10 respiratory phase scans. In addition, a maximum intensity projection and an average intensity projection (Ave-IP) were reconstructed from the data of the 10 respiratory phase scans. On the provided planning CT scan, the organs at risk (OARs; kidneys, liver and spinal cord) were already delineated to exclude dosimetric differences due to differences in OAR delineation.

The responsible radiation oncologists were instructed to delineate and expand the target volumes (GTV, iGTV, iCTV and PTV) and supply a radiation plan according to the PREOPANC trial protocol (Electronic Supplementary Material; [6]). The radiation oncologists received this protocol at the moment the institution had received approval for inclusion of patients in the PREOPANC trial. Furthermore, the responsible radiation oncologists were asked to fill out a questionnaire concerning treatment facilities and procedures. The QA team performed the quality evaluation of all submissions.

Delineation prescriptions

Observers were asked to delineate the GTV on the Ave-IP projection according to the protocol (Electronic Supplementary Material). This GTV had to be expanded to encompass the tumour on all respiratory phases of the 4DCT scan, thus creating the iGTV. The iCTV was defined as the iGTV with a 5-mm uniform margin to account for possible microscopic tumour extensions. An alternative (but more time-consuming) possibility was to delineate the GTV in all respiratory phases, expanding each of these GTVs with 5 mm and summing these volumes, forming the internal target volume (ITV). The PTV included the iCTV (or ITV) plus a 10-mm uniform margin. Consultation of a diagnostic radiologist during the delineation process was allowed.

Treatment planning technique

In the PREOPANC trial, at least a 3D conformal treatment delivery technique was required (Electronic Supplementary Material; [6]). More sophisticated techniques, such as intensity-modulated radiation therapy (IMRT) and volumetric modulated arc therapy (VMAT) were also allowed. The dose of 3600 cGy in 15 fractions should be specified according to the International Commission on Radiation Units and Measurements (ICRU) guidelines; at least 95% of the prescribed dose should cover ≥98% of the PTV [19,20,21]. For the OARs, the prescriptions in the PREOPANC trial were recommended in equivalent doses (fraction dose of 2 Gy). Due to a fraction dose of 2.4 Gy, we also supplied the physical doses of the constraints for the OARs to calculate treatment planning. If the mean physical dose to one kidney exceeded 1680 cGy, irradiation of the contralateral kidney should be avoided as much as possible. The mean physical dose of the liver was not to exceed 2640 cGy. Since the total dose in the slightly hypofractionated schedule of 36 Gy is relatively low, the dose to the spinal cord will (at maximum) be well below the tolerance. Similarly, the dose constraints for stomach and small bowel will not be exceeded.

Dummy run assessment

For assessment of the first three institutions, we arranged a meeting with all members of the QA team to ensure that the procedure of evaluating the target delineation and treatment planning/dosimetry was uniform, and conform the dummy run protocol. The first five delineations were evaluated by the senior radiation oncologist (GT) and the radiation oncology resident (EV) together (both experienced in delineation of pancreatic tumours) This evaluation determined whether the delineation was adequate with respect to the iGTV location and delineation, compared with the tumour seen on the diagnostic CT and described in the diagnostic report. After a standard procedure of evaluating the delineations was drafted between the radiation oncologist and the resident, the radiation oncology resident alone assessed the remaining six delineations. The mean volumes of the iGTV and PTV, as well as the ratio of the smallest and largest iGTV and PTV, were calculated.

The treatment plans were evaluated based on dose coverage of the PTV, dose to the OARs, and the radiation delivery technique used. The conformity index (CI) described by the Radiation Oncology Therapy Group (RTOG), defined as the ratio between the volume enclosed by the reference isodose (95%) and the PTV [22], was used as a measure for conformity with the high-dose region to the PTV. The first three submitted treatment plans were evaluated by the entire QA team to assess the standard procedure that was drafted for this dummy run in the protocol. After evaluating the procedure, the remaining treatment plans and dosimetric parameters were evaluated by one of the researchers (EL) and the radiation oncologist resident (EV).

An overall conclusion concerning protocol compliance was made for each institution. Feedback was provided to each participating institution on an individual basis within 6 weeks after submission. In case of deviations that occur in the majority of the participating institutions, the protocol instructions were to be modified.

Results

All 11 institutions submitted the delineation and treatment plan for the ‘dummy’ patient and the questionnaire.

Questionnaire

Only three institutions indicated they treated >10 patients per year (Table 1). The delineation of the target volumes for the ‘dummy’ patient was experienced as moderately difficult. Most institutions (8 of 11) used 4DCT scans for delineation in patients with pancreatic cancer. In three institutions, fiducial markers were placed in the tumour as part of the routine treatment; two of those use 3–4 Visicoil intratumoural markers per patient; one of these institutions used fiducial markers only for stereotactic treatment of the pancreas. In this institution 3 fiducials are consecutively placed in the tumour or close to the tumour (within 3 cm of the tumour).

Table 1 Questionnaire outcomes
Full size table

Target volume delineation

The range of the iGTV was 19.3–77.2 cm3 (Fig. 1), with a mean iGTV of 41.5 cm3 (standard deviation [SD] 14.8 cm3). The ratio between the largest to the smallest iGTV was 4.0. The range of the PTV volume was 162.4–341.6 cm3 (Fig. 2), with a mean PTV volume of 251.4 cm3 (SD 48.7 cm3; Fig. 3). In most cases, the iGTV was found to be smaller than expected when compared with the tumour seen on the diagnostic CT scan and described in the diagnostic report (Fig. 1). In 8 of the 11 institutions, in some CT slices, the ventral side of the tumour was delineated with tight borders. All these tumour borders were included in the iCTV, even though the iCTV is not the correct target volume for macroscopic tumour volume. One institution delineated the tumour larger than described on the diagnostic CT (especially in the caudal direction) and had the largest iGTV of all the institutions. For one institution (institution #10), the iGTV was created after a margin of 5 mm was added to the GTV for the CTV. In this case, the respiration motion of the tumour was accounted for in the iCTV; therefore, no iGTV was available and we included the GTV as delineated in Figs. 1 and 3. All institutions used the correct margins to obtain iCTV and PTV.

Fig. 1
figure 1

Gross tumour volumes (GTVs) of one institution (red) and internal gross tumour volumes (iGTVs) of 10 institutions projected on the average scan of the four-dimensional computed tomography (4DCT). One intratumoural fiducial marker is visible within the delineated target. Colours: institution 1: orange; 2: yellow; 3: light blue; 4: green; 5: pink; 6: dark green; 7: brown; 8: dark blue; 9: black; 10: red, 11: purple

Full size image
Fig. 2
figure 2

Planning target volumes (PTVs) of the 11 institutions projected on the average scan of the four-dimensional computed tomography (4DCT), with one intratumoural fiducial marker visible within the PTVs. Colours: institution 1: orange; 2: yellow; 3: light blue; 4: green; 5: pink, 6: dark green; 7: brown; 8: dark blue; 9: black; 10: red; 11: purple

Full size image
Fig. 3
figure 3

Box plots of a the conformity index (CI) of the treatment plan, b the volume of internal gross tumour volumes (iGTVs) and planning target volumes (PTV) and c the doses to the organs at risk (OARs) of the 11 institutions. Boxes: median value and upper and lower quartiles; whiskers: lowest and highest data point within 1.5 × interquartile range; dots: outliers

Full size image

Treatment planning/dosimetry

The majority of institutions (82%) used VMAT for treatment delivery. Institution #7 used IMRT and institution #2 used a 3-field conformal technique.

The dose prescription to the PTV (V95% ≥ 98%) was achieved in all treatment plans. The mean CI was 1.16 (range 1.04–1.52), with the largest CI in the 3‑field conformal plan (Fig. 3a). The doses to the OARs are shown in Fig. 3c. The mean left kidney dose was 579 cGy (range 326–849 cGy) and the mean right kidney dose was 283 cGy (range 68–564 cGy). The mean liver dose was 519 cGy (range 394–690 cGy), and the mean maximum dose to the spinal cord was 1724 cGy (range 1214–3016 cGy). The highest dose to the spinal cord was observed in the 3‑field conformal treatment plan.

Verification of the protocol

Some minor recommendations concerning these delineations were given as feedback to the institutions. In most of the institutions (8 of 11), suggestions for the delineations concerned the tight borders of the iGTV in ventral direction; it was advised to include these borders in the iGTV (Table 2). We advised institution #7 to use a more sophisticated technique in order to reduce the dose to the spinal cord. Any recommendations for the delineation, treatment planning, or further questions for assessment, were included in the feedback. No adaptations to the PREOPANC protocol prescriptions were required.

Table 2 Reported feedback and recommendations
Full size table

Discussion

In multicentre clinical trials, variation in protocol interpretation and noncompliance are well-known pitfalls that may lead to protocol deviations and discrepancies between participating institutions. A modern protocol prescription of radiotherapy leaves some freedom with regard to radiation technique, but clearly describes target volumes, dose requirements and constraints for OARs [23]. Quality assurance in the form of a dummy run can be beneficial because this can minimise clinical variations in treatment within the trial. Previous dummy run studies (in various tumour groups) showed that a dummy run ensures optimal radiotherapy delivery based on recommendations and protocol adaptations [13, 15, 16]. Our results showed acceptable deviations, i. e. essentially, inevitable interobserver variations in target delineation, and no prescription adaptations were required for the PREOPANC protocol. Because no adaptations to the protocol were required we did not repeat the dummy run and did not perform a central review of individual patients prospectively.

In the present dummy run, we considered our assessment (EV, GT) as the standard for evaluation of the pancreatic tumour delineations of the ‘dummy’ patient. Consultation between the QA team and the diagnostic radiologist concerning the borders of the tumour took place before evaluation of the delineations. We are aware that no gold standard is available for the delineation of pancreatic tumours and that evaluation of the delineations by two clinicians is subjective and prone to error. Creating a consensus delineation with more observers as a gold standard may be helpful, as was shown in cervical cancer [24]. In pancreatic cancer, the study of Carvatta et al. [25] proposed guidelines for high-risk nodal areas and CTV delineation, showing acceptable interobserver variation. No guideline is available for the GTV delineation in pancreatic cancer and large interobserver variation in pancreatic cancer delineation is reported [26, 27]. For that reason, we did not provide a reference contour.

The volumes of GTV and iGTV can be used as an objective measure to indicate variation [28]. Our results show that, for this ‘dummy’ patient, the mean iGTV volume of the pancreatic tumour was 41.5 cm3, with a ratio between the largest and the smallest iGTV of 4.0. Two earlier dummy runs, in three different cases of pancreatic cancer, reported ratios of the largest to the smallest GTV volume of 9, 3 and 6.8 [29, 30]. In the present study, the questionnaire revealed that there is little experience in the Netherlands with preoperative radiotherapy of pancreatic tumours. The variation in delineation between observers may decrease in time when institutions gain more experience.

Tumour extension outside the pancreas is an important problem in the delineation of the GTV. Tumour extension may result in only slight differences in density compared with the surrounding fat tissue [29]. Also, no delineation guidelines are available for pancreatic cancer, only for postoperative radiotherapy of pancreatic cancer [31].

In our study, institution #10, applied the CTV margin before expanding the GTV to account for respiratory motion. This suggested that the CTV was manually expanded in the 10 respiratory phases to include the visible tumour; this means that the microscopic tumour extensions are not properly accounted for in the CTV. As a result, the PTV margin used may be insufficient to account for all remaining uncertainties. In this institution, the PTV volume (241 cm3) was not the smallest of all the institutions. On an annual basis, this institution treats (on average) only 5 patients with pancreatic cancer, implying that the experience in delineation can be considered only moderate as compared with the other institutions. Unfortunately we only noticed this deviation in a later stage of the study, so this feedback was reported in a later stage of the dummy run to the concerned institution. The consequences of the minor deviations in iGTV delineation for the PREOPANC trial outcome are unclear, as the impact of interobserver variation on a trial outcome is not yet fully explored [28]. However, by producing a clear protocol and using a dummy run to test protocol compliance we aimed to reduce this variation between institutions. RTOG studies reporting on deviations in delineation in pancreatic trials showed unacceptable deviations in around 5% of the cases in a total of four studies, without knowing the impact on outcome [32]. In a multivariate analysis, one study showed that the radiation therapy quality assurance (per-protocol failure of adherence to guidelines) was significant for survival of patients with pancreatic tumours [33].

One of our institutions used conformal 3‑field treatment planning, with a CI of 1.52 and higher doses to the spinal cord and left kidney compared with the other institutions. This illustrates that conformity and OAR sparing can be improved with techniques such as VMAT or IMRT, as described by Nabavizadeh et al. [34]. These latter authors showed that the conformity for both VMAT and IMRT techniques was superior compared with a 3D-CRT technique. VMAT provided a comparable CI to IMRT, with reduced mean doses to the PTV and an overall reduction in treatment time [34].

It is known that respiratory-induced abdominal tumour motion can be substantial. Almost all our institutions used 4DCT in the treatment of patients with pancreatic cancer, taking into account the differences in position of the GTV during the respiratory cycle. This can result in better coverage of the pancreatic tumour within one fraction. Intratumoural fiducial markers and daily online position verification are used to compensate for day-to-day position variation [11, 35]. The above-mentioned procedures in radiotherapy of pancreatic cancer are recommended in the PREOPANC trial. Also, the interobserver variation in pancreatic tumour delineation is currently being quantified in the Netherlands, by means of a study comparing delineation on 3DCT and 4DCT.

Unfortunately, no quantitative evaluation of the variation in delineations, such as the kappa index, could be performed with the software used for the present study. This is a limitation of this study. The kappa index can be used for measuring observer agreement when there is no gold standard [36]. However the added value of the kappa index in this study would be limited since we only had a single dummy patient. No other radiotherapy quality assurance measures were undertaken, as the compliance to protocol prescriptions was acceptable and the observed deviations were inevitable interobserver variations in target delineation. Although guidelines are available for the postoperative setting [31], no gold standard exists for preoperative target delineation for pancreatic cancer. Also, since extensive QA procedures are known to hamper accrual [37], we considered the value of adding prospective individual case review (or other measures) to be too small to implement them.

Conclusion

In this dummy run, all participating institutions were allowed to continue participation in the PREOPANC trial since their compliance to the radiotherapy protocol had acceptable deviations, all of which were due to inevitable interobserver variation in target delineation. A few recommendations were made. Further studies are required to improve visualisation of pancreatic cancer and the delineation tools used, in order to decrease target delineation variation between individual clinicians.