A comparison of in-house and shared RapidPlan models for prostate radiation therapy planning

Knowledge-based planning (KBP) can increase plan quality, consistency and efficiency. In this study, we assess the success of a using a publicly available KBP model compared with developing an in-house model for prostate cancer radiotherapy using a single, commercially available treatment planning system based on the ability of the model to achieve the centre’s planning goals. Two radiation oncology centres each created a prostate cancer KBP model using the Eclipse RapidPlan software. These two models and a third publicly-available, shared model were tested at three centres in a retrospective planning study. The publicly-available model achieved lower rectum doses than the other two models. However, the planning-target-volume (PTV) doses did not meet the local planning goals and the model could not be adjusted to correct this. As a result, the plans most likely to satisfy local planning goals and requirements were created using an in-house model. For centres without an existing in-house model, a model created by another centre with similar planning goals was found to be preferred. Variations in local planning practices including contouring, treatment technique and planning goals can influence the relative performance of KBP. The value of publicly available KBP models could be enhanced through standardisation of planning goals and contouring guidelines, providing information related to the planning goals used to create the model and increased flexibility to allow local adaptation of the KBP model.


Introduction
Knowledge-based planning is a technique where knowledge from historical planning data is combined with patient anatomical information to inform achievable plan quality for the individual patient. The introduction of knowledge-based planning (KBP) for an automated, individualised approach for optimising modulated radiotherapy has been shown to increase plan quality and decrease inter-planner variation [1,2]. Its implementation into the Eclipse treatment planning system, under the name Rapid-Plan (Varian Oncology Systems, Palo Alto, Ca, USA) has been validated in numerous studies [3][4][5][6][7]. RapidPlan uses dose-volume histogram (DVH) predictions based on previous plans to create individualised optimisation criteria.
The New South Wales RapidPlan Consortium was formed in late 2018 to investigate possible benefits which could be gained through collaboration between centres using RapidPlan. Whilst multiple publications have reported methods for optimising RapidPlan models for local use, few publications have reported potential benefits of model sharing across multiple institutes.
Schubert et al. [8], representing the German RapidPlan Consortium, distributed a model from a single centre to seven separate institutions and found that the model produced clinically acceptable plans at all centres. However, it was postulated that further fine tuning of the model at the local site may be beneficial given the differences between institutions in terms of contouring, planning techniques and planning goals. Ueda et al. [9] investigated the differences between models produced independently at 5 institutions and concluded that sharing models would require that the plan designs used for the DVH estimate match those used in the institution receiving the model. In contrast, rather than sharing models, the Victorian Public Sector Rapid-Plan Group created a prostate model jointly between eight separate radiotherapy centres [10]. When comparing the retrospective, manually produced plans with the Groupmodel generated plans, they found a general improvement in organ-at-risk doses and dose homogeneity. However, they also found that the introduction of a common automated model did not reduce the range of OAR doses (ie the minimum to maximum values) within each centre.
With the growing evidence of the benefits of using RapidPlan, many centres are looking to implement KBP. However, limited guidance on the best and most efficient ways to do this is provided in the literature. To investigate this, we designed a study to ask the following question: Is it better to develop an in-house model, use an externally developed model or modify an externally developed model? To answer this question, three centres evaluated two locally generated models and one publically available model. Each centre used the models to generate plans for a set of previously treated patients. Each centre evaluated the generated plans according to their standard (in-house) qualitative and quantitative planning goals.

Methods and materials
RapidPlan, implemented in the Eclipse Treatment Planning System (Varian Oncology Systems, Palo Alto, USA), was used for KBP model generation. For this study, three radiation therapy centres were enrolled, referred to as Centre A, B and C. Centre C had no prior KBP experience or local models. They aimed to determine from this study if an existing model would satisfy their requirements or whether they should instead create their own model. Centres A and B had existing prostate models which were included in the testing, referred to as Model A and Model B, respectively. Centres A and B aimed to determine whether an external model would better satisfy their planning requirements compared with their in-house models.
In addition to the in-house models from Centres A and B, the University of California, San Diego (UCSD) 'UCSD Prostate' [11] model was included in the testing as an example of a freely-available, shared model. Information regarding the 3 models is given in Table 1. Model A and B both include multiple target structures to allow the higher doses to be constrained to the central portion of the planning target volume (PTV).
At each of the three centres, 19-20 consecutive patients were selected after excluding those who would not normally be planned with a standard beam configuration, for example patients with bi-lateral hip prostheses. Each centre planned their selected cohort using each of the three KBP models. Planners were asked to use the prescription dose, beam configuration and energy they would normally use in their centre. All centres chose to plan using volumetric modulated arc therapy (VMAT). Patients were planned with a single-click optimisation with no adjustments made by the planner. Plans were completed in Eclipse V13 using the AAA algorithm (Centres A and B) or Eclipse V15 using the AXB dose-tomedium algorithm (Centre C). In line with current practise at each centre, Centre A plans used 10MV while plans at Centres B and C used 6MV.
In addition to the three initial models, Centres A and B created and applied an edited version of the UCSD model, labelled as UCSD Edit. This new model was created by adjusting the objectives, including both dose level and priority, in the UCSD model to better satisfy the planning goals of those centres.
Dose volume histogram (DVH) data were then extracted using the Eclipse Scripting API and compared to the centres' own prostate planning goals for each patient's dose prescription. The planning goals of the three centres for a 78 Gy prostate plan are presented in Table 2. In addition, the planning goals from the eviQ 'Prostate adenocarcinoma definitive EBRT conventional high risk' protocol [12] are listed for comparison. eviQ does not provide recommendations on target coverage. In addition to the values listed in Table 2, eviQ quotes planning goals from trial protocols.

3
These have not been included as they often differ from the eviQ recommendations. The eviQ protocol also includes constraints for the penile bulb, small intestines and large intestines. These are not listed as none of the centres in this study contour these organ-at-risk (OAR) structures.
To allow comparison between the models as well as between the centres, the following dosimetry parameters were evaluated: PTV V95% and D1cc, bladder V40Gy, V50Gy and V65Gy, rectum V40Gy, V65Gy and V75Gy and femoral heads V50Gy, where VxxGy refers to the volume receiving xx in Gy. For these parameters, the mean values for each centre were calculated and these data presented in graphical format for comparison.
Statistical comparisons of the results were performed using Matlab™ with an unpaired 2-sample t-test used where P < 0.05 indicates significance in the difference of the mean values. The boxplots produced (Figs. 1, 2 and 3) to compare results display the interquartile range as a blue box with the median indicated by a red line and outliers shown as red ' + ' symbols.
Each centre was asked to provide feedback regarding the performance of the RapidPlan models and which model(s) that centre intended to use in the future.

Overall model performance
Considering the plans against all the local planning goals at each centre, as shown in Table 3, all models gave similar results. Centres A and C found Model A more likely to meet the planning goals while the edited UCSD model resulted in the highest proportion of planning goals met for Centre B.

PTV model performance
Model B meet a higher proportion of PTV planning goals at Centres A and C, while the UCSD edit Model met the higher proportion of PTV metrics at Centre B. Figure 1 compares the PTV V95% and D1cc data, showing the 3 centres and the 3 or 4 different models used at each centre. Note that the UCSD Edit models in the presented graphs are unique to each centre based on the changes made by that centre. At Centres B and C, all models generally achieved V95% greater than 99%, whereas at Centre A, only the Models A and B consistently achieved this objective. Comparing PTV V95% between centres, the results from Centres B and C do not show a significant difference between any of the models. The Centre A data are significantly different from Centre C for the UCSD model (P = 0.007) and also from Centre B for Model A (P = 0.048). Figure 1b shows the PTV D1cc for all centres. The 107% objective was meet by all plans. However, all centres found the mean D1cc to be higher with Model B (P < 0.001).

OAR model performance
Model B achieved the lowest proportion of OAR planning goals at all centres while Model A, the UCSD model and the two UCSD edit models gave similar results. Figures 2  and 3 show the bladder and rectum V51.3% (equivalent to V40Gy for a 78 Gy prescription) DVH information for all centres and models. All models performed similarly with regard to the bladder doses. However, the dose to the rectum was lower with the UCSD model compared to both Models A and B (P < 0.001 for both Models A and B at Centres B and C and for Model B at Centre A).The UCSD Edit models also produced lower rectum doses than Model A and B. The increased rectal sparing of the UCSD model can be seen in Fig. 4 where the 50% isodose line covers less of the rectum posteriorly.

Centre A
Model A achieved the highest proportion of planning goals at Centre A. While PTV planning objectives were more likely to be meet by Model B, the lower performance against the OAR planning objectives meant that this model was unacceptable to Centre A. Conversely, the UCSD and UCSD Edit models provided better rectal sparing and met more of the planning goals but performed poorly against the PTV planning goals.
It can be seen from Figs. 1, 2, 3 and 4 that UCSD edit model managed to maintain the low rectal doses of the original model. However, Centre A felt that the PTV V95% dose coverage achieved was inferior, visible in Fig. 1, and they were not able to remedy this.
They noted that the UCSD model achieved lower rectal doses. Information about the plans used to create each model is included in Table 4 in Appendix. These data show that the mean rectum doses for the plans in the UCSD model are lower than the plans Model A and Model B leading to lower predicted DVH curves and a lower line dose optimisation objective. In addition, the UCSD model, which included a wider range of rectal volumes than those in Model A or B, contained a number of upper objectives on the rectum which were not present in Model A or B.

3
The staff at Centre A found that the sharing of models was beneficial and discussed all models in a multi-disciplinary meeting. They chose to continue improving their inhouse model (Model A) informed by the results of this study.

Centre B
For Centre B, the UCSD edit model achieved the PTV and OAR planning objectives more consistently than any of the other models performed. Despite this, Centre B did not select to use this model clinically.
Centre B is the only centre with a mean PTV dose goal. While the UCSD edit model satisfied more of the PTV planning goals than the other models, this particular goal was not satisfied for any patients using the UCSD and the UCSD Edit Models. In comparison, Model B achieved this objective for 95% of patients and Model A achieved this objective for 50% of patients. It is possible that renormalising the plans could have resulted in more plans satisfying this planning goal while still achieving better OAR doses. However, the planner at Centre B also noted that the global maximum dose with the UCSD and UCSD edit models was often outside the CTV (seen in 95% of plans using the UCSD model and 90% of plans using the UCSD edit model). While not a documented planning goal, the planner preferred the maximum dose to be in the CTV and this was achieved more often with the model developed in their own centre, Model B, with only 15% of plans using Model B having a global maximum dose outside the CTV. For Model A, which also uses multiple target structures, 75% of plans had the maximum dose outside the CTV.

Centre C
Model A achieved the most local planning goals at Centre C. The UCSD model fared poorly on the PTV planning goals and Model B was not able to satisfy as many OAR goals as Model A.
Based on their experience with the three models, Centre C expected that Model A would be the best model to adopt as their planning goals aligned better with Centre A, compared with Centre B. A lack of physics resources meant that the implementation of Model A was delayed and during this time an in-house model was developed by planning staff which was eventually implemented. The building of the in-house RapidPlan model at Centre C was informed by the other 3 models and hence participating in this project was considered beneficial.
Centre C reported that the sharing of models allowed them to become familiar with the software and to build a set of prostate plans to use in their own model. One of the main challenges at Centre C was finding agreement between their oncologists on the planning goals they should aim for. Centre C have implemented RapidPlan models for other treatment sites following this work.

Discussion
Creating a RapidPlan model requires multiple good quality treatment plans. Using an externally generated model has the potential to produce superior plans while reducing the repetition of work already performed by others. This study investigates this further by comparing in-house RapidPlan models with externally developed models. Our results, from both centres A and B, show that their in-house models produce plans which are more likely to satisfy the local planning goals compared to externally produced models. This is not surprising given these models were specifically created with the planning goals of their centres in mind.
However, the assessment of models based on the performance of plans against local planning goals was incomplete. This style of assessment failed to highlight the reductions in rectal dose achieved by the UCSD model and other considerations such as location of global maximum dose.
Knowledge-based planning, as implemented in Rapid-Plan, aims to predict the optimal OAR DVHs and, based on these, set realistic optimisation criteria. As the UCSD model creates plans with lower rectal doses, it could be argued that this is the model which should be adopted for planning at the three centres. However, it may be that the lower rectal dose predictions given by this model are not compatible with the PTV requirements of the centres involved and, even with editing, the UCSD model could not produce plans clinically acceptable at the centres.
The failure of either Centre A and B to create a satisfactory model from the UCSD model may indicate adjustment of a model created with different aims is limited. Consequently, the centres may be biased towards their own inhouse models and were not invested in further developing the external model. Also, the adjustments to the UCSD model were limited by the single target structure used in this model. While additional target structures can be added to a model for each patient, the addition of an extra target by default for all patients invalidates the model within Eclipse and renders it unusable. All centres in this study found the PTV doses from the UCSD model were unsatisfactory and, without additional control provided by an extra target structure, were not able to adjust the model to produce satisfactory plans.
The lower rectal doses achieved with the UCSD model indicate a potential for improvement of the in-house models. Utilising additional fixed objectives on the rectum, when producing plans in the future, may produce lower rectal DVH curves. New iterations of models using plans with lower rectal DVH curves would lead to lower model predicted DVH curves and the production of plans with lower rectal doses.
While the in-house models were selected for use by all centres, these results may not be true for other comparisons where the shared model is better suited to the centre involved and to customisation. In addition, situations exist where generating a local model may not be feasible e.g. if patient numbers are low or planning practises have recently changed (ie. using a new planning system or changing from an IMRT to VMAT technique).
This study has highlighted the differing planning goals which exist between centres but the data also shows that the same model can give different results at different centres. Our results show that, for all models, the bladder doses are lower at Centre C, that the rectum doses are higher at Centre B and that PTV coverage is lower at Centre A. The differences observed could be caused by differences in contours or differences in planning techniques. As all centres chose to use a 2-arc planning technique, this is unlikely to be the cause. Centres B and C used 6MV whereas Centre A used 10MV. This may contribute to the higher rectum doses at Centre B but higher doses were not seen at Centre C using the same energy, and this does not explain the lower bladder doses at Centre C or lower PTV coverage at Centre A. A review of the rectal contouring guidelines at the three centres shows that these are not well documented at Centre B. This may be contributing to the higher rectal doses. Additionally, the patients' prostate size as well as bladder and bowel preparation may also differ which could result in differing OAR and PTV contours.
While the shared models were not implemented clinically by the centres involve in this investigation, benefits were seen from the assessment of different models. All centres were able to benchmark in-house models against external models, determine areas of potential improvement for future model creation and learn by viewing the objectives set in other models.
Based on this work, an option to improve the success for future sharing of RapidPlan models and similar collaborative efforts is to work towards better agreement between centres, whereby planning goals and contouring guidelines are unified, prior to attempting to create a model. This option has several advantages in that it would incorporate external review of planning goals between centres, promote collaboration and discussion between centres and simplifies the task of creating a satisfactory model. This approach has been used successfully to create regional models in Germany and Victoria [8,10]. In the Victorian model, plans from all centres were included in the creation of the model. This should ensure that the geometric range of patients at all centres is included. However, these models may need to exclude patient cohorts where systematic differences exist between target and OAR segmentations (eg potentially stratifying patients by use SpaceOAR gel to increase the space between the posterior prostate and rectum). When planning aims are agreed a-priori, for example in the context of a clinical trial, a share RapidPlan model has the potential to harmonise plan quality between centres and improve trial protocol compliance.
It has been seen in this study that considering only predefined planning metrics (i.e. DVH planning constraints) is not the only method used to assess plan quality. Even with prior agreement on planning goals, additional organ sparing following the as-low-as-reasonably-achievable (ALARA) [13] principal or improved PTV coverage may be achievable. The relative prioritisation and importance of these potential improvements may differ between radiation oncologists.
The success of sharing of RapidPlan models could be further improved by ensuring planning goals contained in the model are defined, and flexibility is built within the model. These features would allow wider sharing of models and would not require agreement or discussion prior to the creation of models. Centres could select a model which is most likely to align with their planning priorities and adapt this model in response to future changes in practice. In the specific context of RapidPlan, ensuring flexibly in shared models would mean including multiple PTV structures and a wide range of OAR structures to allow for the variation in planning goals between centres. This option would still require DVH predictions to be achievable in addition to the other added constraints, and hence all models need to be validated at the local level prior to clinical implementation.
A limitation of this study is that it only considered prostate radiotherapy. However, a comparison between an inhouse created model and available shared models is likely to be beneficial for all treatment sites and ensure to best available RapidPlan model is used. Where the creation of an in-house model is not practical a flexible, shared model from a centre with similar planning goals is likely to be beneficial. However, internal testing and adjustment of the model are likely to still be required.

Conclusions
When comparing radiotherapy plans generated using an in-house model with an externally generated model, it was found that the in-house model would most likely achieve the local planning goals and satisfy the requirements of the radiation oncologist. When an in-house model is not available, the model most likely to be considered acceptable is a model that has been created by a centre with similar planning goals. While the UCSD prostate model was found to achieve the lowest rectal doses, a lack of flexibility in adjusting the model meant that it could not provide plans which were satisfactory to any of the centres.
Participating in a comparison of RapidPlan models was found to be beneficial for the centres involved, even when the centre chose not to adopt the new models for patient planning. Where possible, centres should consider a range of RapidPlan models to find the model that best suits their needs.
When developing RapidPlan models for sharing or for internal use, models should be designed to be flexible, in particular, multiple target structures should be included as well as all available OAR structures. This allows more adjustment of the model where additional boost regions or additional OAR limitations are required. Model documentation should include details of the planning goals and beam arrangements of the plans used to create the model. This allows centres to select the most compatible models and then adapted these models to different planning constraints or techniques. The success of shared RapidPlan models may be increased through standardisation of planning goals and contouring guidelines as this will ensure more consistently DVH predictions.