Introduction

Knowledge-based planning is a technique where knowledge from historical planning data is combined with patient anatomical information to inform achievable plan quality for the individual patient. The introduction of knowledge-based planning (KBP) for an automated, individualised approach for optimising modulated radiotherapy has been shown to increase plan quality and decrease inter-planner variation [1, 2]. Its implementation into the Eclipse treatment planning system, under the name RapidPlan (Varian Oncology Systems, Palo Alto, Ca, USA) has been validated in numerous studies [3,4,5,6,7]. RapidPlan uses dose-volume histogram (DVH) predictions based on previous plans to create individualised optimisation criteria.

The New South Wales RapidPlan Consortium was formed in late 2018 to investigate possible benefits which could be gained through collaboration between centres using RapidPlan. Whilst multiple publications have reported methods for optimising RapidPlan models for local use, few publications have reported potential benefits of model sharing across multiple institutes.

Schubert et al. [8], representing the German RapidPlan Consortium, distributed a model from a single centre to seven separate institutions and found that the model produced clinically acceptable plans at all centres. However, it was postulated that further fine tuning of the model at the local site may be beneficial given the differences between institutions in terms of contouring, planning techniques and planning goals. Ueda et al. [9] investigated the differences between models produced independently at 5 institutions and concluded that sharing models would require that the plan designs used for the DVH estimate match those used in the institution receiving the model. In contrast, rather than sharing models, the Victorian Public Sector RapidPlan Group created a prostate model jointly between eight separate radiotherapy centres [10]. When comparing the retrospective, manually produced plans with the Group-model generated plans, they found a general improvement in organ-at-risk doses and dose homogeneity. However, they also found that the introduction of a common automated model did not reduce the range of OAR doses (ie the minimum to maximum values) within each centre.

With the growing evidence of the benefits of using RapidPlan, many centres are looking to implement KBP. However, limited guidance on the best and most efficient ways to do this is provided in the literature. To investigate this, we designed a study to ask the following question: Is it better to develop an in-house model, use an externally developed model or modify an externally developed model? To answer this question, three centres evaluated two locally generated models and one publically available model. Each centre used the models to generate plans for a set of previously treated patients. Each centre evaluated the generated plans according to their standard (in-house) qualitative and quantitative planning goals.

Methods and materials

RapidPlan, implemented in the Eclipse Treatment Planning System (Varian Oncology Systems, Palo Alto, USA), was used for KBP model generation. For this study, three radiation therapy centres were enrolled, referred to as Centre A, B and C. Centre C had no prior KBP experience or local models. They aimed to determine from this study if an existing model would satisfy their requirements or whether they should instead create their own model. Centres A and B had existing prostate models which were included in the testing, referred to as Model A and Model B, respectively. Centres A and B aimed to determine whether an external model would better satisfy their planning requirements compared with their in-house models.

In addition to the in-house models from Centres A and B, the University of California, San Diego (UCSD) ‘UCSD Prostate’ [11] model was included in the testing as an example of a freely-available, shared model. Information regarding the 3 models is given in Table 1. Model A and B both include multiple target structures to allow the higher doses to be constrained to the central portion of the planning target volume (PTV).

Table 1 Details of the initial KBP models
Full size table

At each of the three centres, 19–20 consecutive patients were selected after excluding those who would not normally be planned with a standard beam configuration, for example patients with bi-lateral hip prostheses. Each centre planned their selected cohort using each of the three KBP models. Planners were asked to use the prescription dose, beam configuration and energy they would normally use in their centre. All centres chose to plan using volumetric modulated arc therapy (VMAT). Patients were planned with a single-click optimisation with no adjustments made by the planner. Plans were completed in Eclipse V13 using the AAA algorithm (Centres A and B) or Eclipse V15 using the AXB dose-to-medium algorithm (Centre C). In line with current practise at each centre, Centre A plans used 10MV while plans at Centres B and C used 6MV.

In addition to the three initial models, Centres A and B created and applied an edited version of the UCSD model, labelled as UCSD Edit. This new model was created by adjusting the objectives, including both dose level and priority, in the UCSD model to better satisfy the planning goals of those centres.

Dose volume histogram (DVH) data were then extracted using the Eclipse Scripting API and compared to the centres’ own prostate planning goals for each patient’s dose prescription. The planning goals of the three centres for a 78 Gy prostate plan are presented in Table 2. In addition, the planning goals from the eviQ ‘Prostate adenocarcinoma definitive EBRT conventional high risk’ protocol [12] are listed for comparison. eviQ does not provide recommendations on target coverage. In addition to the values listed in Table 2, eviQ quotes planning goals from trial protocols. These have not been included as they often differ from the eviQ recommendations. The eviQ protocol also includes constraints for the penile bulb, small intestines and large intestines. These are not listed as none of the centres in this study contour these organ-at-risk (OAR) structures.

Table 2 Prostate planning goals for 78 Gy prostate plans
Full size table

To allow comparison between the models as well as between the centres, the following dosimetry parameters were evaluated: PTV V95% and D1cc, bladder V40Gy, V50Gy and V65Gy, rectum V40Gy, V65Gy and V75Gy and femoral heads V50Gy, where VxxGy refers to the volume receiving xx in Gy. For these parameters, the mean values for each centre were calculated and these data presented in graphical format for comparison.

Statistical comparisons of the results were performed using Matlab™ with an unpaired 2-sample t-test used where P < 0.05 indicates significance in the difference of the mean values. The boxplots produced (Figs. 1, 2 and 3) to compare results display the interquartile range as a blue box with the median indicated by a red line and outliers shown as red ‘ + ’ symbols.

Fig. 1
figure 1

a The volume of PTV receiving 95% of the prescription dose (V95%) for the three centres (A, B, C) and for the four models (Model A, Model B, UCSD and UCSD edit), shown as a percentage. The cyan line indicates the eviQ acceptable level. b Percentage dose received by 1 cc of PTV (D1cc) for the three centres (A, B and C) and for the four models (Centre A, Centre B, UCSD and UCSD edit)

Full size image
Fig. 2
figure 2

The percentage of bladder receiving a dose of 51.3% or more (equivalent to V40Gy for a 78 Gy prescription) for the three centres (A, B, C) and for the four models (Centre A, Centre B, UCSD and UCSD edit). The cyan line indicates the 78 Gy prostate eviQ acceptable tolerance

Full size image
Fig. 3
figure 3

The percentage of rectum receiving a dose of 51.3% or more (equivalent to V40Gy for a 78 Gy prescription) for the three centres (A, B, C) and for the four models (Centre A, Centre B, UCSD and UCSD edit). The cyan line indicates the 78 Gy prostate eviQ acceptable tolerance

Full size image

Each centre was asked to provide feedback regarding the performance of the RapidPlan models and which model(s) that centre intended to use in the future.

Results

Overall model performance

Considering the plans against all the local planning goals at each centre, as shown in Table 3, all models gave similar results. Centres A and C found Model A more likely to meet the planning goals while the edited UCSD model resulted in the highest proportion of planning goals met for Centre B.

Table 3 The average percentage of achieved local planning goals
Full size table

PTV model performance

Model B meet a higher proportion of PTV planning goals at Centres A and C, while the UCSD edit Model met the higher proportion of PTV metrics at Centre B. Figure 1 compares the PTV V95% and D1cc data, showing the 3 centres and the 3 or 4 different models used at each centre. Note that the UCSD Edit models in the presented graphs are unique to each centre based on the changes made by that centre. At Centres B and C, all models generally achieved V95% greater than 99%, whereas at Centre A, only the Models A and B consistently achieved this objective. Comparing PTV V95% between centres, the results from Centres B and C do not show a significant difference between any of the models. The Centre A data are significantly different from Centre C for the UCSD model (P = 0.007) and also from Centre B for Model A (P = 0.048). Figure 1b shows the PTV D1cc for all centres. The 107% objective was meet by all plans. However, all centres found the mean D1cc to be higher with Model B (P < 0.001).

OAR model performance

Model B achieved the lowest proportion of OAR planning goals at all centres while Model A, the UCSD model and the two UCSD edit models gave similar results. Figures 2 and 3 show the bladder and rectum V51.3% (equivalent to V40Gy for a 78 Gy prescription) DVH information for all centres and models. All models performed similarly with regard to the bladder doses. However, the dose to the rectum was lower with the UCSD model compared to both Models A and B (P < 0.001 for both Models A and B at Centres B and C and for Model B at Centre A).The UCSD Edit models also produced lower rectum doses than Model A and B. The increased rectal sparing of the UCSD model can be seen in Fig. 4 where the 50% isodose line covers less of the rectum posteriorly.

Fig. 4
figure 4

The dose distribution of a patient from Centre B representative of the majority of patients. PTV shown in pink, CTV shown in orange, 50% isodose shown as orange, 100% isodose shown as cyan and the colour wash indicating 95–107% dose. Top Left—Model A. Top Right—Model B. Bottom Left—UCSD. Bottom Right—UCSD edit

Full size image

Centre A

Model A achieved the highest proportion of planning goals at Centre A. While PTV planning objectives were more likely to be meet by Model B, the lower performance against the OAR planning objectives meant that this model was unacceptable to Centre A. Conversely, the UCSD and UCSD Edit models provided better rectal sparing and met more of the planning goals but performed poorly against the PTV planning goals.

It can be seen from Figs. 1, 2, 3 and 4 that UCSD edit model managed to maintain the low rectal doses of the original model. However, Centre A felt that the PTV V95% dose coverage achieved was inferior, visible in Fig. 1, and they were not able to remedy this.

They noted that the UCSD model achieved lower rectal doses. Information about the plans used to create each model is included in Table 4 in Appendix. These data show that the mean rectum doses for the plans in the UCSD model are lower than the plans Model A and Model B leading to lower predicted DVH curves and a lower line dose optimisation objective. In addition, the UCSD model, which included a wider range of rectal volumes than those in Model A or B, contained a number of upper objectives on the rectum which were not present in Model A or B.

The staff at Centre A found that the sharing of models was beneficial and discussed all models in a multi-disciplinary meeting. They chose to continue improving their in-house model (Model A) informed by the results of this study.

Centre B

For Centre B, the UCSD edit model achieved the PTV and OAR planning objectives more consistently than any of the other models performed. Despite this, Centre B did not select to use this model clinically.

Centre B is the only centre with a mean PTV dose goal. While the UCSD edit model satisfied more of the PTV planning goals than the other models, this particular goal was not satisfied for any patients using the UCSD and the UCSD Edit Models. In comparison, Model B achieved this objective for 95% of patients and Model A achieved this objective for 50% of patients. It is possible that renormalising the plans could have resulted in more plans satisfying this planning goal while still achieving better OAR doses. However, the planner at Centre B also noted that the global maximum dose with the UCSD and UCSD edit models was often outside the CTV (seen in 95% of plans using the UCSD model and 90% of plans using the UCSD edit model). While not a documented planning goal, the planner preferred the maximum dose to be in the CTV and this was achieved more often with the model developed in their own centre, Model B, with only 15% of plans using Model B having a global maximum dose outside the CTV. For Model A, which also uses multiple target structures, 75% of plans had the maximum dose outside the CTV.

Centre C

Model A achieved the most local planning goals at Centre C. The UCSD model fared poorly on the PTV planning goals and Model B was not able to satisfy as many OAR goals as Model A.

Based on their experience with the three models, Centre C expected that Model A would be the best model to adopt as their planning goals aligned better with Centre A, compared with Centre B. A lack of physics resources meant that the implementation of Model A was delayed and during this time an in-house model was developed by planning staff which was eventually implemented. The building of the in-house RapidPlan model at Centre C was informed by the other 3 models and hence participating in this project was considered beneficial.

Centre C reported that the sharing of models allowed them to become familiar with the software and to build a set of prostate plans to use in their own model. One of the main challenges at Centre C was finding agreement between their oncologists on the planning goals they should aim for. Centre C have implemented RapidPlan models for other treatment sites following this work.

Discussion

Creating a RapidPlan model requires multiple good quality treatment plans. Using an externally generated model has the potential to produce superior plans while reducing the repetition of work already performed by others. This study investigates this further by comparing in-house RapidPlan models with externally developed models. Our results, from both centres A and B, show that their in-house models produce plans which are more likely to satisfy the local planning goals compared to externally produced models. This is not surprising given these models were specifically created with the planning goals of their centres in mind.

However, the assessment of models based on the performance of plans against local planning goals was incomplete. This style of assessment failed to highlight the reductions in rectal dose achieved by the UCSD model and other considerations such as location of global maximum dose.

Knowledge-based planning, as implemented in RapidPlan, aims to predict the optimal OAR DVHs and, based on these, set realistic optimisation criteria. As the UCSD model creates plans with lower rectal doses, it could be argued that this is the model which should be adopted for planning at the three centres. However, it may be that the lower rectal dose predictions given by this model are not compatible with the PTV requirements of the centres involved and, even with editing, the UCSD model could not produce plans clinically acceptable at the centres.

The failure of either Centre A and B to create a satisfactory model from the UCSD model may indicate adjustment of a model created with different aims is limited. Consequently, the centres may be biased towards their own in-house models and were not invested in further developing the external model.

Also, the adjustments to the UCSD model were limited by the single target structure used in this model. While additional target structures can be added to a model for each patient, the addition of an extra target by default for all patients invalidates the model within Eclipse and renders it unusable. All centres in this study found the PTV doses from the UCSD model were unsatisfactory and, without additional control provided by an extra target structure, were not able to adjust the model to produce satisfactory plans.

The lower rectal doses achieved with the UCSD model indicate a potential for improvement of the in-house models. Utilising additional fixed objectives on the rectum, when producing plans in the future, may produce lower rectal DVH curves. New iterations of models using plans with lower rectal DVH curves would lead to lower model predicted DVH curves and the production of plans with lower rectal doses.

While the in-house models were selected for use by all centres, these results may not be true for other comparisons where the shared model is better suited to the centre involved and to customisation. In addition, situations exist where generating a local model may not be feasible e.g. if patient numbers are low or planning practises have recently changed (ie. using a new planning system or changing from an IMRT to VMAT technique).

This study has highlighted the differing planning goals which exist between centres but the data also shows that the same model can give different results at different centres. Our results show that, for all models, the bladder doses are lower at Centre C, that the rectum doses are higher at Centre B and that PTV coverage is lower at Centre A. The differences observed could be caused by differences in contours or differences in planning techniques. As all centres chose to use a 2-arc planning technique, this is unlikely to be the cause. Centres B and C used 6MV whereas Centre A used 10MV. This may contribute to the higher rectum doses at Centre B but higher doses were not seen at Centre C using the same energy, and this does not explain the lower bladder doses at Centre C or lower PTV coverage at Centre A. A review of the rectal contouring guidelines at the three centres shows that these are not well documented at Centre B. This may be contributing to the higher rectal doses. Additionally, the patients’ prostate size as well as bladder and bowel preparation may also differ which could result in differing OAR and PTV contours.

While the shared models were not implemented clinically by the centres involve in this investigation, benefits were seen from the assessment of different models. All centres were able to benchmark in-house models against external models, determine areas of potential improvement for future model creation and learn by viewing the objectives set in other models.

Based on this work, an option to improve the success for future sharing of RapidPlan models and similar collaborative efforts is to work towards better agreement between centres, whereby planning goals and contouring guidelines are unified, prior to attempting to create a model. This option has several advantages in that it would incorporate external review of planning goals between centres, promote collaboration and discussion between centres and simplifies the task of creating a satisfactory model. This approach has been used successfully to create regional models in Germany and Victoria [8, 10]. In the Victorian model, plans from all centres were included in the creation of the model. This should ensure that the geometric range of patients at all centres is included. However, these models may need to exclude patient cohorts where systematic differences exist between target and OAR segmentations (eg potentially stratifying patients by use SpaceOAR gel to increase the space between the posterior prostate and rectum). When planning aims are agreed a-priori, for example in the context of a clinical trial, a share RapidPlan model has the potential to harmonise plan quality between centres and improve trial protocol compliance.

It has been seen in this study that considering only predefined planning metrics (i.e. DVH planning constraints) is not the only method used to assess plan quality. Even with prior agreement on planning goals, additional organ sparing following the as-low-as-reasonably-achievable (ALARA) [13] principal or improved PTV coverage may be achievable. The relative prioritisation and importance of these potential improvements may differ between radiation oncologists.

The success of sharing of RapidPlan models could be further improved by ensuring planning goals contained in the model are defined, and flexibility is built within the model. These features would allow wider sharing of models and would not require agreement or discussion prior to the creation of models. Centres could select a model which is most likely to align with their planning priorities and adapt this model in response to future changes in practice. In the specific context of RapidPlan, ensuring flexibly in shared models would mean including multiple PTV structures and a wide range of OAR structures to allow for the variation in planning goals between centres. This option would still require DVH predictions to be achievable in addition to the other added constraints, and hence all models need to be validated at the local level prior to clinical implementation.

A limitation of this study is that it only considered prostate radiotherapy. However, a comparison between an in-house created model and available shared models is likely to be beneficial for all treatment sites and ensure to best available RapidPlan model is used. Where the creation of an in-house model is not practical a flexible, shared model from a centre with similar planning goals is likely to be beneficial. However, internal testing and adjustment of the model are likely to still be required.

Conclusions

When comparing radiotherapy plans generated using an in-house model with an externally generated model, it was found that the in-house model would most likely achieve the local planning goals and satisfy the requirements of the radiation oncologist. When an in-house model is not available, the model most likely to be considered acceptable is a model that has been created by a centre with similar planning goals. While the UCSD prostate model was found to achieve the lowest rectal doses, a lack of flexibility in adjusting the model meant that it could not provide plans which were satisfactory to any of the centres.

Participating in a comparison of RapidPlan models was found to be beneficial for the centres involved, even when the centre chose not to adopt the new models for patient planning. Where possible, centres should consider a range of RapidPlan models to find the model that best suits their needs.

When developing RapidPlan models for sharing or for internal use, models should be designed to be flexible, in particular, multiple target structures should be included as well as all available OAR structures. This allows more adjustment of the model where additional boost regions or additional OAR limitations are required. Model documentation should include details of the planning goals and beam arrangements of the plans used to create the model. This allows centres to select the most compatible models and then adapted these models to different planning constraints or techniques. The success of shared RapidPlan models may be increased through standardisation of planning goals and contouring guidelines as this will ensure more consistently DVH predictions.