Introduction

Rooted in the ethical imperative for respect of patient autonomy, shared decision-making has become a core aspect of patient-centered care that seeks to empower patients to consider with their physicians the available options, taking into account their personal circumstances and articulating their values, perspectives, and concerns [1,2,3]. Being promoted as an avenue for improving individual and population health [4], shared decision-making may reduce decisional conflict and improve patient adherence [5]. It is particularly pertinent to contexts of personal uncertainty, when evidence points to multiple, clinically valid therapeutic alternatives [5].

Neuroendocrine tumors (NETs) have been reported to have a rising incidence, likely due to improved diagnosis and classification [6]. Gastroenteropancreatic (GEP) NETs are often slow-growing, with nonspecific symptoms that mimic a variety of disorders, obstructing and delaying diagnosis. Diagnosis typically occurs at an advanced stage, often inadvertently and after metastases have developed [7, 8]. Complete tumor resection is recommended whenever possible [9]. However, for unresectable disease, two randomized, double-blind, placebo-controlled trials, PROMID and CLARINET, demonstrated that the somatostatin analogues (SSAs) octreotide and lanreotide significantly prolong time to progression and progression-free survival (PFS) in patients with well- or moderately differentiated midgut or GEP-NETs, respectively [10, 11]. Both SSAs are administered every 4 weeks, octreotide via intramuscular injection, lanreotide via deep subcutaneous injection with the option of self-administration [12, 13]. On the basis of this evidence, National Comprehensive Cancer Network (NCCN) guidelines recommend, for patients with unresectable, asymptomatic (nonfunctioning) GEP-NET and low tumor burden, either observation with markers and scans (watchful waiting) or initiation of treatment with octreotide or lanreotide [9]. There is no consensus on timing of treatment initiation [9]. The nature of the disease and the different options imply that patients’ preferences and circumstances are central to decision-making in this context, thus suggesting a systematic approach to help identify the relevant factors bearing on the decision and support shared decision-making.

The International Patient Decision Aids Standards (IPDAS) Collaboration recommends that a decision-making tool deliver information about the disease and the available options, including their risks and benefits; clarify and make explicit patient values and preferences; and help share perspectives to support reflection and discussion between patients and physicians [5, 14, 15]. Multi-criteria decision analysis (MCDA) represents a structured and transparent approach to decision-making [16,17,18,19,20,21,22,23]. Sharing the intent of IPDAS, the open-source EVIDEM MCDA framework includes tools to clarify and make explicit individual values, synthesize complex evidence for the decision criteria to facilitate their consideration in real-life scenarios, and help share perspectives through visual outputs and insights [24, 25].

Building on these features, this study’s objectives were to (1) explore patients’ and clinicians’ reflection when deciding on management options for unresectable GEP-NET by elucidating which criteria are considered and how they are considered; and (2) develop a comprehensive decision framework that can facilitate elicitation and communication of benefits, risks, preferences, and concerns to help patients and physicians engage in meaningful conversations on the decisions faced. Before application to real-life settings with patient–physician dyads, a small group workshop design, which allowed collection of patients’ and physicians’ inputs and feedback, was selected to study how such a framework can support reflection and communication (proof-of-concept).

Methods

Study Overview

The study involved identification and validation of the criteria that are considered by US patients and physicians in their decision-making (including input from a criteria workshop), design of the framework for two decision scenarios, and its application with an extended group of patients and clinicians (decision support workshop) (Fig. 1). Framework design encompassed two approaches: the first focused on the benefits and risks of the management options in the sense of intervention outcomes (core benefit-risk criteria). The second explored the impact of other decision-making factors that may modulate the benefit-risk balance (modulating criteria).

Fig. 1
figure 1

Study overview. Colloquial evidence: “anything that establishes a fact or gives reason for believing something” [55]

Full size image

Workshop Design

Patients and clinicians were recruited as experts in decision-making to collect their expert views and insights, stemming from their living with or treating the disease. No clinical data was collected. Both were selected via predefined recruitment criteria (Appendix 1). Clinicians were oncologists specializing in treating patients with locally advanced or metastatic GEP-NET from both academic and community practices. Patients were recruited through patient associations and support groups. All study participants were offered an honorarium as well as reimbursement for their expenses. All participants provided informed consent (patients) or signed agreements (clinicians).

To facilitate open discussion, sessions were held under the Chatham House Rule [26] and were limited to six (three patients, three clinicians) and 11 (five patients, six clinicians) participants for the criteria and the decision support workshops, respectively. Study investigators (i.e., researchers performing the study) guided participants through the workshops.

Criteria Workshop and Design of GEP-NET MCDA Benefit-Risk Trees

At the criteria workshop, patients and clinicians were presented with the 20 generic EVIDEM framework criteria (along with their definitions), organized into seven domains of evaluation [27], plus 10 intervention outcomes subcriteria specific to GEP-NET, which had been identified through targeted review of the scientific literature, including regulatory requirements for oncology trials [28, 29]. Participants were asked to validate each (sub)criterion and suggest others for inclusion. For framework design, to allow the expression of a broad range of perspectives, a (sub)criterion was included if at least one participant deemed it to be relevant. (See Appendix 2 for (sub)criteria.) The core and modulating (sub)criteria were organized by domain into a core benefit-risk and a modulated benefit-risk tree, respectively.

Decision Scenarios and Evidence on GEP-NET and Management Options

Two decision scenarios were defined on the basis of current practice guidelines [9]: the first explored the primary decision whether to initiate treatment with an SSA (using lanreotide as reference case) or to pursue watchful waiting; and the second explored the decision between two SSAs treatments: treatment 1 (lanreotide) versus treatment 2 (octreotide). To provide necessary and sufficient evidence to assign a score for each criterion, MCDA evidence matrices for each scenario were created following EVIDEM methodology [27], which entails a standardized approach for the identification, analysis, synthesis, and reporting of evidence.

Evidence was obtained from public sources, including major biomedical literature databases (PubMed/Medline), Cochrane systematic reviews, clinical trial registries, conference websites (ASCO, NANETS), and bibliographies of full-text articles. Additional sources were patient and professional association websites. Recent peer-reviewed, comprehensive studies were preferentially selected; for clinical data, only key phase 3 or 4 randomized controlled trials (RCTs) and prospective studies were eligible for inclusion. (See Supplementary Appendix 4 for more details on the targeted systematic literature review methodology.) The evidence matrix included 33 references.

Decision Support Workshop

At the decision support workshop, participants were first introduced to the approach and the definitions of the decision criteria and then asked to express their individual value systems and preferences by weighing the relative importance of the criteria (independently of the scenarios). Next, participants explored scenario 1: on the basis of the evidence matrix (see excerpt in Appendix 5), the available evidence for a criterion was presented by study investigators, then the group exchanged their comments and insights, which was followed by each participant providing his/her individual score, comments, and insights in writing. Scores were provided using a constructed, cardinal scale, designed to capture individual interpretations, and expressed how consideration of each criterion would impact the decision, ranging from +5 (much in favor of option 1) to − 5 (much in favor of option 2). For qualitatively considered criteria, participants indicated how their consideration impacted their decision (positive, no, or negative impact). Participants were also asked to provide for each criterion any additional information they wanted to see. This was repeated for each criterion and then for scenario 2. Feedback on the process was collected through structured discussion and in written form.

Data Analysis

As reported previously [25], the relative benefit-risk balance (RBRB) is defined as the sum of the benefit-risk contributions (BRC x ) of all (n) subcriteria of the benefit-risk tree, where BRC x is the product of the normalized weight (W x , ∑ W x  = 1) and standardized score (S x  = score/5) for the subcriterion x (linear additive model):

$${\text{RBRB}} = \sum\limits_{x = 1}^{n} {{\text{BRC}}_{x} } = \sum\limits_{x = 1}^{n} {\left( {W_{x} \times S_{x} } \right)}$$

For example, if the criterion impact on autonomy received a normalized weight of 0.1 and a score of + 2, its contribution to the RBRB was 0.1× 2/5 = + 0.04. RBRBs can range from − 1 to 1. Positive RBRBs indicate that the sum of the criteria contributions favors option 1 and negative RBRBs favor option 2. A corresponding linear additive model including all core and modulating (sub)criteria was used to derive the modulated RBRB (mRBRB).

As the approach was designed to be applied at the individual level for patient–physician interactions, weights, scores and RBRBs were computed at the level of individual participants. For the purpose of illustration and reporting, mean values were calculated and are presented here at the group level. Variability across participants was quantified using standard deviations (SD). Written and verbal comments were organized and summarized by criteria. Visual representations of the outputs were designed.

Face Validity and Exploration of Uncertainty

To assess face validity, participants received a visual and tabular representation of their own assessment (including weights, RBRBs, mRBRBs, and qualitative impacts) and were asked whether this reflected their reasoning during the exercise. To explore impact on weights distribution, two weighting methods were used: hierarchical point allocation (HPA), which involved distributing weighting points across the domains, criteria, and subcriteria of the benefit-risk trees, was used for the primary analysis [24]. For an alternative analysis, the direct rating scale (DRS) method was used [30], which involves rating the relative importance of each (sub)criterion on a scale from 1 to 5. Using the sum of these rates as denominator, relative weights for each (sub)criterion were calculated and used to compute the RBRBs.

Results

Criteria Workshop: What Matters?—Criteria Set Adaptation and Validation

Of the 30 (sub)criteria presented to the criteria workshop participants, 26 were considered relevant by more than 90% of the participants.

Participants unanimously agreed on the relevance of the core benefit-risk subcriteria, progression-free survival, worsening of symptoms, health-related quality of life (HRQoL), impact on autonomy, impact on dignity, serious adverse events, and fatal adverse events to their evaluation of GEP-NET management options. All other outcomes presented were deemed relevant by at least one-third of the participants. Tumor regression rate was identified as an additional relevant outcome and was incorporated in the core benefit-risk tree (Fig. 2a).

Fig. 2
figure 2

MCDA core benefit-risk tree (a) and modulated benefit-risk tree (b). *Qualitatively assessed criteria

Full size image

Modulating factors deemed relevant by all were type of therapeutic benefit, disease severity, unmet needs, costs/constraints to patients, and quality of evidence. Population priorities and access were unanimously considered not relevant and therefore omitted from the framework. On the basis of participant input, subcriteria were created to segregate costs and constraints incurred by patients from those incurred by the healthcare system, and system capacity and appropriate use was incorporated into the modulated tree for quantitative consideration (Fig. 2b).

Decision Support Workshop Step 1: What are the Trade-Offs?—Clarifying Values and Preferences

Weighting the criteria of the core benefit-risk tree, decision support workshop participants assigned the greatest relative weight to overall survival (mean normalized weight 0.16, scale 0 to 1), followed by progression-free survival (0.15), fatal adverse events (0.14), and disease symptoms (0.11) (Fig. 3a). Level of chromogranin A (0.01), and convenience and ease of use (0.04) received the least weight. Progression-free survival (SD ± 0.15) and overall survival (± 0.12) showed the largest variations in weights.

Fig. 3
figure 3

Mean (SD) normalized weights assigned by participants using hierarchical point allocation to each domain/criterion/subcriterion of the MCDA a core and b modulated benefit-risk trees

Full size image

In the modulated tree (Fig. 3b), comparative effectiveness received the greatest mean weight (0.18), followed by disease severity (0.12), comparative safety/tolerability (0.10), and type of therapeutic benefit (0.10). The least weight was assigned to size of affected population (0.02) and non-medical costs and constraints (0.03). Participants’ weights varied most for comparative effectiveness (± 0.12), disease severity, and quality of evidence (± 0.10). Among the economic consequences and constraints subcriteria, cost of intervention to patient received the greatest normalized weight (0.04), followed by non-medical costs and constraints (0.03) and other medical costs and constraints to patients (0.03) (data not shown).

Decision Support Workshop Step 2: How Does Consideration of Each Criterion Favor Either Option?—Criteria Scores and Insights

Scoring, on the basis of the available evidence and personal insights, how consideration of each criterion favors either option prompted a rich exchange of knowledge and rationales among participants. Details—including a condensed summary of the evidence available to the participants as well as the scores and comments they provided—are presented for scenario 1 (watchful waiting vs treatment [lanreotide]) in Table 1 (core benefit-risk criteria) and 2 (modulating criteria). A high-level summary is reported for scenario 2 further on.

Table 1 Core benefit-risk criteria: condensed evidence and participant scores and comments exploring scenario 1: treatment (lanreotide as a reference case) versus watchful waiting
Full size table

For comparative effectiveness, data on PFS was seen as clearly favoring treatment over watchful waiting, with a mean score of 4.0 (± 1.2, scale − 5 to 5) (Table 2). While it was noted that the PFS appeared to encourage a low threshold for starting SSA therapy, participants raised questions around the relevance of the RCT data to clinical practice regarding patient population and the criteria to define progression. Overall survival (OS), the criterion that had received the highest weight, was scored slightly in favor of treatment (mean 0.3 ± 2.2). Clinicians noted that the trial was not designed to measure OS and that this outcome would be difficult to quantify. No data was available on effect on tumor-related disease symptoms in the target population; nevertheless, most participants scored this criterion as favoring SSA therapy (mean 2.9 ± 2.1), in part because of its well-known symptom relief effects. The effect on level of chromogranin A was also considered to be in favor of treatment (1.2 ± 1.9), but clinicians commented that its usefulness in decision-making was limited (Table 1).

Table 2 Modulating benefit-risk criteria: condensed evidence and participant scores and comments exploring scenario 1: treatment (lanreotide as a reference case) versus watchful waiting
Full size table

For comparative patient-perceived health/PROs, consideration of HRQoL marginally favored treatment (mean 0.5 ± 1.8). Clinical trial data showed no significant difference in HRQoL outcomes. Some participants commented that treatment-induced PFS extension would be expected to have a positive emotional impact, whereas receiving no treatment could take an emotional toll. Comments on impact on autonomy and convenience/ease of administration revolved around the option of lanreotide self-injection, its relation to insurance coverage, and its impact on office visits. Although scores varied widely, generally autonomy in drug administration was viewed as favoring treatment (1.0 ± 2.6), whereas overall convenience aspects favored watchful waiting (− 1.3 ± 2.0), which, as commented by one participant, was particularly relevant for frail patients (Table 1).

Among comparative safety/tolerability subcriteria, non-serious AEs and non-fatal serious AEs moderately favored watchful waiting (mean scores − 1.3 ± 1.5 and − 0.5 ± 0.9), whereas fatal AEs did not favor either option (0.0 ± 0.0), congruent with the absence of fatal AEs in the pivotal trial. Participants stressed that potential AEs must be considered, especially as treatment is initiated in asymptomatic patients. They also noted that more information on the management of side effects, their impacts, and whether or not they are reversible would have been helpful.

Among modulating criteria (Table 2), the type of therapeutic benefit lanreotide provides was considered to be in favor of treatment (3.8 ± 1.7). One participant commented that “Keeping the disease stable is very significant.” Consideration of disease severity and unmet needs favored treatment overall (2.5 ± 2.7 and 2.1 ± 1.6), but views differed widely, particularly with respect to disease severity where scores ranged from − 2 to 5. On the watchful waiting side, it was commented that “In slow-growing asymptomatic tumors the benefit of any treatment is unclear”, whereas on the treatment side, a patient stated that “Doing something is better than doing nothing.” It was also noted that the favored option would depend on the presence of co-morbidities.

With respect to economic consequences and constraints, considerations around the cost of treatment to patients, in terms of co-payments, deductibles, and possible increases in monthly contributions, favored watchful waiting (mean score − 2.8 ± 1.8). Other medical and non-medical costs and constraints for patients also favored watchful waiting (− 1.2 ± 1.3 and − 1.0 ± 1.6, respectively). It was noted that there may be additional costs and medical appointments if patients experience side effects. Patients expressed major concerns about time away from work and family, travel costs, and fear of losing employment as a result of insurance charges.

Both quality of evidence and expert consensus/clinical practice guidelines (CPGs) were viewed in favor of treatment (2.9 ± 1.8 and 2.2 ± 2.8). Participants commented that CLARINET was well conducted but it was only one trial. (Note the PROMID trial was presented for scenario 2 only.) With respect to CPGs, clinicians noted that the guidelines provide general guidance but treatment decisions are driven by individual patient characteristics.

Across all (sub)criteria, the greatest variations in scores (SD ≥ 2.0) were for overall survival, disease symptoms, tumor regression rate, impact on autonomy, convenience/ease/mode of administration, disease severity, expert consensus/CPGs, and system capacity/appropriate use (Tables 1, 2).

Qualitative criteria were considered by the majority of participants but had no impact on their decision-making. Several participants indicated that mandate and scope of the healthcare system was in favor of treatment, while opportunity costs and affordability were in favor of watchful waiting (data not shown).

Decision Support Workshop Step 3: What is the Overall Balance Between the Criteria?

Combining weights and scores, at the group level, an RBRB of 0.18 (± 0.20) favored treatment (with lanreotide as reference case) over watchful waiting (Fig. 4). Progression-free survival (0.12 ± 0.13) and disease symptoms (0.06 ± 0.06) contributed most to the RBRB in favor of treatment. Non-fatal non-serious adverse events (− 0.02 ± 0.03), non-fatal serious adverse events (− 0.01 ± 0.02), and convenience (− 0.01 ± 0.01) contributed most towards watchful waiting. The largest variations among participants were noted for progression-free survival and impact on autonomy (± 0.13 and ± 0.07).

Fig. 4
figure 4

Mean RBRB contributions* of each quantitative criterion and overall RBRB† for treatment (using lanreotide as reference case) versus watchful waiting: a core benefit-risk model, b modulated benefit-risk model. *Values shown represent the contribution of criteria to the relative benefit-risk balance calculated as normalized weight (summing to 1) multiplied by score for each criterion (theoretical range from −1 to +1). †Relative benefit-risk balance is the sum of contributions from all criteria (theoretical range from −1 to +1). Error bars show standard deviations across 11 participants

Full size image

Taking into account modulating criteria, at the group level, an mRBRB of 0.29 favored treatment over watchful waiting, with large individual variations (± 0.28). Beside comparative effectiveness (0.07 ± 0.08), type of therapeutic benefit, disease severity, and quality of evidence contributed most towards treatment (0.08 ± 0.06, 0.07 ± 0.12, and 0.06 ± 0.10), whereas economic considerations, including cost of intervention and other medical costs and constraints, favored watchful waiting (− 0.03 ± 0.02 and − 0.01 ± 0.02).

For scenario 2 (treatment 1 [lanreotide] vs treatment 2 [octreotide]), the majority of criteria did not favor one option over another, with mean RBRB (0.00 ± 0.04) and mRBRB (0.01 ± 0.04) close to 0 at the group level (data not shown). Participants commented that although the pivotal trials (PROMID and CLARINET) were difficult to compare because of different patient populations, effectiveness, impact on HRQoL, and safety and tolerability appeared to be similar. With respect to the RBRB, convenience/ease of use (0.02 ± 0.01) and impact on autonomy (mean 0.01 ± 0.01) contributed to favoring lanreotide, as participants commented, owing to the possibility to self-inject and potentially less painful injection, while tumor regression rate (− 0.01 ± 0.03) contributed towards octreotide (no data on lanreotide). With respect to the mRBRB, system capacity (0.02 ± 0.02) contributed in favor of lanreotide, while cost of intervention favored octreotide (− 0.02 ± 0.02) (participants used cost comparison data and their own knowledge/experience), although with large variations across participants.

Face Validity and Exploration of Uncertainty

All of the seven participants who participated in the face validity exercise noted that the visual representations of their quantitative outputs (i.e., weights, RBRBs, and mRBRBs) reflected their thinking process. One participant commented specifically that the visual representation clarified how the combination of relative weights and the consideration of available evidence had impacted their thought process.

Replacing HPA-derived with DRS-derived weights shifted the mean RBRB for scenario 1 from 0.18 (± 0.20) to 0.14 (± 0.19) and the mRBRB from 0.29 (± 0.28) to 0.19 (± 0.20). For scenario 2, DRS-derived weights shifted the RBRB and the mRBRB slightly more in favor of lanreotide, from 0.00 (± 0.04) to 0.04 (± 0.04) and from 0.01 (± 0.04) to 0.02 (± 0.04), respectively.

Particpants’ Feedback on Experience and Process

At the end of the exercise, participants noted that the criteria discussed reflected questions with which they struggle and highlighted the importance of conducting a shared reflection on these criteria. They also emphasized the importance of considering the available evidence, while noting its limitations in real-world applications. One participant would have wanted more time to read and process the information. In general, participants found the process innovative and helpful since it allowed them to structure their thinking and make it more explicit, which allowed it to be shared with others. The interactive component of the exercise was found to foster an environment where patients and clinicians had the opportunity to discuss sensitive elements. This was appreciated by all participants.

Discussion

Investigating a complex clinical decision problem on the basis of a range of criteria and the available evidence allowed an in-depth exploration of what matters and how it matters to patients and clinicians. This exploration revealed that a variety of criteria are relevant to the reality of decision-making, including potential clinical and patient-reported outcomes, but also other factors, such as disease severity, quality of evidence, type of benefit, and personal costs and constraints. This suggests that true understanding of the decision requires embracing its complexity, which also has to be reflected in a framework that is intended to support the decision-making process. This is echoed in other work reporting that, beyond clinical factors, family situation, impact on family and work as well as disease stage are important for cancer patients’ decision-making [31].

Assigning criteria weights prompts stakeholders to reflect on their values and preferences (i.e., what matters most to them). In the GEP-NET management context, on the group level, OS and PFS were the most important outcomes, closely followed by fatal AEs. PFS is recommended as primary endpoint for NET trials, while OS is deemed not a practical endpoint because of important trial design constraints [32]. Thus, although observational data suggests a correlation between PFS and OS [33], a reliable estimate of the OS benefit associated with SSA therapy is likely to remain unavailable. Weights varied widely, highlighting differences in values and preferences and the importance of communication. Although the study was not designed to compare patient and clinician perspectives, exploratory analysis suggests that, while clinicians tended to assign greater importance to efficacy outcomes, patients put greater emphasis on fatal and serious AEs (mean normalized weights vs clinicians: fAEs: 0.18 vs 0.11, sAEs: 0.11 vs 0.06). Recent studies confirm that risk aversion differs from one group to another [34,35,36] and also varies depending on patients’ health status, as lower disease severity correlates with less willingness to accept risk [31, 35]. Compared to clinicians, patients also tended to assign greater weight to the criteria impact on autonomy and impact on dignity at the expense of impact on HRQoL, which may suggest that these terms more directly expressed what matters to patients than the HRQoL concept, as was discussed previously [37, 38].

Communicating scientific evidence in a way that ensures patients’ understanding is a challenge [39, 40], particularly in the presence of preconceived biases regarding the risks or benefits of interventions [40]. Furthermore, both patients and physicians may be unaware of their assumptions as related to the decision [41]. The scoring exercise, including the comments it elicited, helped uncover assumptions and differences in interpretation. Particularly when data was absent or inconclusive, large variations in scores were observed, reflecting uncertainty and diversity of views. For example, for overall survival, one-quarter of participants provided positive scores, thus expressing their expectation that treatment may extend survival even though the clinical study design was not able to show an OS benefit. Scores for disease severity showed particularly large variation (SD 2.7), indicating that the same information on the natural course of the disease can be interpreted in favor of either option, as was also confirmed by the comments provided. It is also possible that participants’ own experience with the disease and its symptoms played into how they considered this criterion. Sharing scores with others can bring interpretations to light, and thereby help patients and clinicians understand each other’s reasoning in order to create a “shared mind”, which, as Epstein and Street noted, is central to achieving truly shared decision-making [41].

Combining weights and scores across criteria (RBRB) indicates that, at a group level, patients and clinicians preferred treatment rather than watchful waiting (scenario 1). This appears to be in line with the natural progression of the disease and comments from physicians that eventually all patients would receive SSA therapy. Group discussion also highlighted the emotional impact of “doing something” following a GEP-NET diagnosis. Beyond core benefit-risk criteria, disease severity, type of benefit, and quality of evidence played impactful modulating roles in patients’ and clinicians’ considerations. With respect to scenario 2, on a group level, the RBRB and mRBRB did not appear to favor one treatment option over the other, although some of the decision criteria were deemed in favor of one SSA or the other.

Exploratory analysis suggested that safety/tolerability contributed in favor of watchful waiting among clinicians but not patients. This likely reflects the relatively safe profile of SSAs. In their comments, patients highlighted their willingness to accept non-serious AEs if treatment slows disease progression. Other studies also observed that drug effects may be valued differently by patients and physicians [42].

While participants deemed the visual representation of their assessment helpful and affirmed its face validity, the potential value of a decision framework lies not primarily in quantitative outputs but in its ability to stimulate fruitful conversations about values, preferences, the evidence and its meaning [3]. To that end, weighting provided a means for sharing values and preferences, while scoring scales were designed to help decision-makers express their thinking in a semi-quantitative way, which, combined with insights, can be easily shared with others. The comprehensiveness of the framework helped conversations go beyond risks and benefits to include autonomy, convenience, feasibility, personal costs and constraints, and affordability. Feedback from participants highlighted the usefulness of the exercise to clarify the complexity of decision-making and support communication.

This study had several limitations. First, as a result of lack of data, for some of the criteria (e.g., non-medical costs and constraints) participants had to rely solely on their own insights and experiences to inform their reflection. In practice, accurate cost information is often not available at the time of decision-making, although, as was shown, study participants saw cost considerations as very relevant to decision-making. Further, in real-life applications, the benefit of a decision framework lies at the individual level and for patient–physician interactions. In this study, results are presented on a group level to illustrate the approach and identify aspects relevant to the management of GEP-NET. However, in view of the relatively small number of participants and the large variability in how individuals consider evidence and make trade-offs, these group values should be interpreted with caution and are not broadly generalizable. Still, the small workshop format was designed to create an atmosphere where patients and physicians could freely exchange perspectives and thus allowed an in-depth exploration of thought processes, to the benefit of the larger GEP-NET community.

Conclusion

The multi-criteria framework offered patients and clinicians a common interpretive frame to structure the complexity of decision-making, support individual reflection, and help share what matters to them and how it matters when exploring treatment decisions for unresectable, asymptomatic, non-functioning, well- or moderately differentiated GEP-NETs. This work can be leveraged to provide a decision-making tool for the clinical setting (e.g., a Web-based application)—which features relevant criteria, scientific evidence, and tools to elicit individual values, preferences, judgments, and insights—to support patients and clinicians in jointly arriving at evidence-informed decisions.