FormalPara Key Points for Decision Makers

Value assessment framework recommendations call for improving value measures to better align with what is important to patients.

This paper presents patient-informed value elements that were developed with continuous patient engagement throughout the process.

The work will advance the field of value assessment because it provides a set of novel and measurable patient-informed value elements that can be incorporated into existing value frameworks and economic evaluations to improve the health technology assessment, data-generation, and decision-making processes.

1 Introduction

Value assessment frameworks have emerged as a way to inform a range of audiences about the value of health technologies under review. Existing value frameworks are anchored on value measures related to clinical outcomes and their associated costs [1,2,3]. Recent criticisms are that these elements do not reflect other constructs of value important to patients [4], they are not patient centered, the inputs are not informed by patients [5], and there has been insufficient patient engagement in the process [5, 6]. Consequently, current value frameworks are unable to weight the results according to treatment preferences, even though it is known that patients value specific aspects of treatment differently [7].

Several recent reports draw attention to the advancement in novel value measures, where gaps exist, and where more work is needed to make value assessment more patient centered. A focus has been on identifying novel value elements [8, 9], such as severity of disease, equity, or value of hope [7, 9, 10]. The International Society for Pharmaceutical Outcomes Research (ISPOR) Value Assessment Frameworks Special Task Force Report [11] recently emphasized the need for patient perspectives in value assessments [7] and offered a set of recommendations to address gaps in the field, one of which is to test novel elements [6, 8, 9, 12]. This would begin to address the call for meaningful patient input in value assessment [6, 13,14,15].

Some barriers to making value assessments more patient centered do exist. For one, there appears to be no set of empirically derived patient-informed value elements, and this hinders their application in existing value assessment frameworks. To adequately reflect the views of a patient community, we must elicit directly from patients their values, experiences, and needs for outcomes and treatment options. Without meaningfully engaging patients, value assessments may not authentically reflect their perspectives on what is most important in healthcare decisions. Second, guidance on how to quantify patient-informed value elements consistently in value assessment is limited. The Patient-Driven Values in Healthcare Evaluation (PAVE) Center was established at the University of Maryland, USA, with the specific goals of using a stakeholder-engaged and empirically driven process to identify a set of patient-informed value elements, establish quantifiable measures that can be incorporated into existing value frameworks, and test methods for using these measures in economic evaluations. The goals of this paper are to describe our process of identifying a core set of patient-informed value elements with continuous patient–stakeholder engagement and to provide guidance on methodological approaches to quantify and apply the patient-informed value elements.

2 Methods

2.1 Design

Qualitative and quantitative methods were used iteratively to elicit patient-informed value elements, defined in this paper as elements that patients consider important when selecting among treatment and healthcare options. Qualitative methods involved one-on-one discussions with patient stakeholders. Quantitative methods included guided activities that were administered via an electronic instrument. A methodological iterative process to integrate qualitative and quantitative data collection and synthesis was facilitated by the four stages described in the following sections. This work was deemed exempt as non-human subjects research from the University of Maryland Baltimore Institutional Review Board.

2.2 Patient Stakeholder Involvement

Three patient representative members of the PAVE Stakeholder Advisory Committee (SAC) were involved in all stages of this work. The SAC patient representatives have first-hand experience advocating for the healthcare needs of patients. One patient stakeholder is involved in national policy and advocacy for the rare disease patient community, one delivers patient support to one of the largest behavioral health providers in a mid-Atlantic state, and one represents Hispanic communities.

Individuals from a broader patient stakeholder community were involved in the final phase of the value-element identification. In total, 14 individuals representing eight diverse national patient advocacy groups, which included mental health, osteoporosis, blindness, lupus, eczema, oncology, chronic obstructive pulmonary disease, and hypercholesterolemia, were members of the National Health Council (NHC) Value Workgroup and familiar with value assessment. This workgroup was an established entity preceding this work.

2.3 Patient-Informed Value Element Development

A staged qualitative and quantitative methodologic approach engaged patient–stakeholder representatives directly to elicit the elements of value important to the patient community. The three PAVE SAC patient representatives met via telephone with the first author (SDR) three times to identify the issues that are important to patients in making healthcare decisions. It was through these iterative discussions that they derived a list of value elements, which were compared with value elements in the published literature. Representatives from the broader patient community then assessed the importance of the value elements. Four stages were used to arrive at patient-informed value elements subsequently organized into a comprehensive core set.

2.3.1 Stage 1: Identify Existing Elements of Value Important to the Patient Community

An important initial step was to identify any congruence and differences in elements of value defined by patients and those assessed in research. A thorough review of the published literature using PubMed and Embase identified the breadth of elements incorporated into existing value assessments. The search terms to retrieve relevant literature are provided in the electronic supplementary material (ESM; Appendix Table A1). Two research assistants scanned the articles to assess the relevance of the peer-reviewed publications with respect to the elements in existing value frameworks, patient-centered research on healthcare decision making, and value elements in economic evaluations. We considered how researchers and value framework developers defined and measured the elements of value and how this was incorporated into value frameworks or healthcare decision making. The result of this review was a list of existing elements used to assess the value of healthcare interventions.

In the first round of review and discussion, the three SAC patient representatives provided input on elements of value to patients in making healthcare decisions. The goal for this round was to first familiarize everyone with the current landscape for value assessment and elicit the elements that are important to the patient community. Next, they considered the list of elements that emerged from the literature review and commented on whether any elements were missing. During the group discussion, elements were added, some elements were rephrased, and some were marked for deletion. We incorporated this feedback and revised the element list.

In the second round of review and discussion, the SAC patient representatives examined the revised element list in greater depth. The goals of this round were to confirm the changes from the prior discussion, obtain advice on whether elements were missing after the first round, and, if yes, what elements needed to be added, and to remove elements that were not important to patients in making healthcare decisions. Each SAC patient representative reviewed the list independently and made suggestions for additions, deletions, or re-phrasing. During the group discussion, all input was reviewed, and any differences of opinion were resolved through consensus. At the conclusion of the discussion, a master list of patient-informed elements incorporated the collective feedback from the three SAC patient representatives.

The third and final round of review and discussion focused on defining each element and confirming whether further edits were needed. The goal was to capture the meaning of the patient-informed element in the voice of the patient. Again, each of the three SAC patient representatives independently reviewed the list, revised from the previous round, and provided a brief definition of each element given their respective lens. The group discussion was used to refine the meaning and achieve consensus on the definition.

2.3.2 Stage 2: Prioritize Patient-Informed Value Elements

The second stage of this work assessed the validity and completeness of the patient-informed value elements using a wider patient community that represented various medical conditions. The patient-community representatives were recruited from the membership of the NHC Value Workgroup and were invited to participate in a guided activity with two of the authors (SDR, YDH) where they could provide feedback on the patient-informed elements. We developed an online tool (Appendix Table A2 in the ESM) in Qualtrics® that elicited input on whether the element label and definition required rephrasing to improve clarity, and if so, what would they recommend. Individuals were also asked to rate whether the element was of high, medium, or low importance when making decisions about a healthcare intervention. The one-on-one guided activity was conducted via WebEx® teleconference where the tool could be screen shared to facilitate the guided activity. For each element, individuals provided their input by answering three questions: Would you rephrase the element label or the definition? (yes/no); If yes, how would you rephrase the element label or the definition? (open-ended); How important to treatment decision making is this value element to the patient community? (high/medium/low). The research team reviewed the feedback, modified the definition phrasing where recommended, and summarized the importance ratings.

2.3.3 Stage 3: Refine Patient-Informed Value Elements

The third stage ensured we did not overlook any elements that are important to patients, confirmed that elements flagged for removal in stage 1 should be removed, and implemented any additional refinements. The three SAC patient representatives independently reviewed the 47 elements flagged for removal after stage 1 and provided input on whether to delete, retain, or combine the element with an existing element, and noting which one. A research assistant consolidated the feedback from the three independent reviews, and elements where two or more patient stakeholders recommended deletion were removed.

2.3.4 Stage 4: Synthesis of Patient-Informed Value Elements

The final stage synthesized the value elements into the core set of patient-informed value elements. First, the elements that reflected the same construct were grouped into categories. Next, the categories were clustered into broader domains. The three SAC patient representatives reviewed the model of thematic domains, categories, and individual elements and provided suggestions for changes. All three SAC patient stakeholders approved the final principal set of value elements.

3 Results

3.1 Identifying Elements Important to Patient Values in Healthcare

Of the 94 elements of value identified in stage 1, there were 60 from the literature and 34 from patient stakeholder input (Fig. 1). The SAC patient representatives flagged 47 elements for deletion. Of the 47 elements retained for the candidate set in the stage 2 prioritization, 26 were derived directly from patient experiences and seven were retained from the literature as being important to patients (Appendix Table A3 in the ESM). An additional five elements from the literature were rephrased and nine elements were modified to be more patient centered (Appendix Table A3 in the ESM). The definitions of the patient-informed value elements are in Table 1.

Fig. 1
figure 1

Identification and refinement of patient-informed elements of value with continuous patient stakeholder involvement. SAC stakeholder advisory committee

Full size image
Table 1 Stakeholder-derived definitions for the 47 patient-informed value elements at stage 1
Full size table

After the final round of stage 1 discussions, three of the elements, i.e., benefit–risk tradeoff for patients, benefit–risk tradeoff for the community, and maximum willingness to pay for benefit gained were eliminated, leaving 44 value elements for the stage 2 prioritization. The research team, in consultation with the three SAC patient representatives, determined that these three elements were not independent elements but rather a combination of more than one element. It was not clear what the benefit or risk tradeoff represented, and it was determined that these would be redundant with the existing element explanation of treatment benefits and risks.

3.2 Prioritizing Elements Important to Patient Values in Healthcare

Table 2 illustrates the value elements rated as high priority after the stage 2 evaluation. A total of 32 elements were rated high priority by ≥ 50% of the patient stakeholders and only three elements were rated high priority by < 25% of the patient stakeholders. The 12 elements deemed high importance by > 75% of the patient stakeholders were tolerability, side effects, ability to maintain relationships with family, ability to work, impact on depression, affordability, long-term costs, reimbursed care, available treatment, appropriateness of care, provider willing to deliver care, and explanation of the treatment benefits and risks. The three elements rated as of high importance by < 25% of the patient stakeholders were conflict with religious beliefs, fear of rejection by family, and fear of rejection by society. Elements such as social activities, support network and relationships with family and peers, and the physical impact on fatigue, ability to work, impact on anxiety, life expectancy, and the financial impact of treatment were of high importance for 50–75% of patient stakeholders. Notably, none of the patient stakeholders recommended deleting any elements or adding missing elements.

Table 2 Value elements rated as high priority by patient stakeholders in the stage 2 evaluation
Full size table

3.3 Refining Patient-Informed Value Elements

The research team and the three SAC patient representatives made minor revisions to the value element list after stage 2. The addition of four new elements included physical abilities, embarrassment/self-conscious, autonomy/dependence, and medication frequency. Physical abilities included getting ready in the morning and ability to exercise, which was unique from ability to work. Stigma became an overarching category that included embarrassment/self-conscious along with rejection by society and family. Autonomy/dependence was a related but distinct element from relocation costs to be closer to family and from long-term effects on the family. Medication frequency was an aspect of tolerability, so that became an element and tolerability the overarching category. Two elements, impact on depression and impact on anxiety, were combined into emotional status. Finally, alternative treatments, pharmacy access to medication, and treatment access in socially neglected populations were absorbed in available treatment and not retained as unique elements. The patient-informed value element development in stages 1 and 2 is in the ESM (Appendix Table A3). This resulted in 42 unique patient-informed value elements.

The element refinement in stage 3 is shown in the ESM (Appendix Table A4). The three SAC patient representatives confirmed removal of 26 of 47 elements flagged for removal after stage 1. Of the 21 that remained, all were combined or encompassed within an element that emerged from stage 1 or 2. No new elements were added to the list of 42 patient-informed elements that emerged from stage 2.

3.4 Synthesizing a Core Set of Patient-Informed Value Elements

The 42 patient-informed value elements were organized into 11 categories that were subsumed within five thematic domains (Fig. 2). The 11 categories included tolerability, disease burden, forecasting, accessibility of care/treatment, healthcare service delivery, cost incurred on the patient, cost incurred on the family, personal well-being, stigma, social well-being, and personal values. The categories of tolerability, disease burden, and forecasting were within the domain of short- and long-term effects of treatment. The treatment access domain included accessibility of care/treatment and healthcare service delivery. Cost incurred by the patient and cost incurred by the family were in the treatment costs domain. The life impact domain included personal well-being and stigma, and the social impact domain included social well-being and social values. The individual patient-informed value elements within each category are listed in Fig. 2.

Fig. 2
figure 2

Domains related to the patient-informed value elements

Full size image

3.5 Recommendations for the Application of Patient-Informed Value Elements

It was not the intent to develop a single metric or a patient-reported outcome measure, because not all elements are important to all groups. These elements ideally would be tailored to a specific patient stakeholder group. To implement this in patient-centered value assessment, the research team developed recommendations for the methodological quantification and application of the elements.

3.5.1 Recommendation 1: Tailoring Patient-Informed Value Elements to Stakeholder Groups

To tailor the core set of 42 patient-informed value elements to those that are most important to a patient community with a specific medical condition, we recommend that stakeholders be engaged in individual interviews or in focus group discussions. We recommend that individuals are asked to identify the most important elements within each of the five domains. From here, further reduction can be accomplished by identifying the top priorities across domains. Future research may consider literature-based approaches and the US FDA Patient-Focused Drug Development reports for specific medical conditions to identify a starting point for each subgroup. The end goal for use of the value elements—be it clinical practice guidance, benefit plan design, utilization management decisions, or new drug development—should guide this process.

3.5.2 Recommendation 2: Quantifying Patient-Informed Value Elements

Once the core set of patient-informed value elements is tailored to a patient community, these can be quantified depending upon the desired end goal. It is possible to evaluate the rank-order importance in a descriptive assessment of the value elements. The research team used stated-preference methods to quantify preferences using regression models [16,17,18,19]. The resulting beta estimates, or preference weights, inform their relative importance in patient-informed value-based decisions. It is also possible to evaluate the trade-off among different value elements using the preference weights.

3.5.3 Recommendation 3: Applying Patient-Informed Value Elements in Economic Evaluations

Value elements that have been quantified for a particular patient community can ultimately inform components of value assessments. Some patient-informed value elements such as patient costs already exist in economic evaluations that use the societal perspective, whereas others are novel and rarely considered in evaluations, such as sibling costs, stigma from family and society, personal values, and social well-being through support and relations with family and peers. Domains and attributes for health state utility estimation should be compared with the elements ranked highly by the patient community to ascertain whether quality-of-life instruments measure the prioritized value elements. The patient-informed value elements also may be a platform from which to assess the patient centeredness of widely used quality-of-life instruments. If not, alternatives such as stated-preference methods may be explored to quantify patient preferences for health states and quality of life. Our patient-informed value elements can address the patient-centeredness gaps identified in existing value frameworks [5, 6]. Examples include weighting elements based on patient preferences, ensuring impact is given to qualitatively assessed elements, and measuring tradeoffs for productivity, illness burden, and innovation [5, 6].

4 Discussion

This set of patient-informed value elements was directed by patient stakeholders throughout the entire process and was grounded in patient experiences that enriched the contextual meaning and relevance of each element. Our SAC patient representatives and the patient communities involved in this work reflected patients across the age range, from caregivers of young children to older individuals living with chronic conditions. In total, 75% of the value elements in the core set were patient derived and distinct from the elements used in existing value frameworks. Notably, many of the value elements highlighted in a recent ISPOR task force report [9], such as fear of contagion, real option value, scientific spillovers, and equity were not endorsed by patient stakeholders. Of the three SAC patient stakeholders involved in guiding this formative work, one was well-versed in value assessment and familiar with the topics to guide the discussion in their relevance to patients. It is possible that these are more relevant to societal and population perspectives in a situational context than they are to the patient perspective. The ISPOR task force report noted that improving adherence is a common value element but is rarely or inconsistently measured even though patient values for treatment attributes can indirectly affect health through adherence [9]. Our patient-informed elements can build upon this call for clarification and research on adherence-improving factors. The recently updated Institute for Clinical and Economic Review (ICER) 2020–2023 value assessment framework explicitly recognized that patients value benefits beyond clinical outcomes [20]. Several value elements that their framework mentions as important to individual patients are similar to our findings, such as complexity of the treatment regimen, impact of care options on the ability to return to work, and the negative impact of the condition on family and caregivers. ICER has proposed methodologies for including these in assessments. We hope our set of elements provides a basis for further efforts in developing and standardizing methods such as these.

It is important to consider how the patient-informed value elements fit within the current discussion of patient-centered value assessment across jurisdictions. Cost-effectiveness analyses traditionally reflect population-level costs for the average patient experience rather than considering individualized outcomes, sparking recent criticism over this “one-size-fits-all” approach [21]. Garrison et al. [7] posited that a challenge with assessing value from the individual perspective relates to bounded rationality: the limitation and biases that affect a patient’s decision making. Others describe patients’ healthcare decisions as guided by value traits and life priorities [22]: Armstrong and Mullins [22] described a taxonomy of patient values that encompassed decisional, situational, and external values. The patient-informed value elements such as tolerability, life impact, and treatment costs are decisional values. The patient-informed value elements within the domains of treatment access and life impact reflect situational values. Finally, the social impact domain, along with the patient-informed value elements within the stigma category, reflect external values. This provides some evidence of the external validity of our patient-informed value elements. The recommendations for tailoring, quantifying, and applying the patient-informed value elements offer a range of opportunities that do not impose a “one-size-fits-all” approach. Rather, the patient-informed value elements are a comprehensive guide that can be leveraged to enhance the patient voice in value assessment and make value assessments more patient centric [5]. The ISPOR Task Force Report, with international stakeholder representation, suggests this is a global concern [11]. Recently, changes to value frameworks, such as in the updated ICER framework [20], illustrate this interest in the USA. This is further evidenced by efforts among jurisdictions in the UK and Europe as well [15, 23,24,25].

Our approach is not without limitations. First, while we engaged patient stakeholders from diverse medical communities, it is difficult to account for all possible patient perspectives. For example, the aspect of altruism in protecting the community from disease spread emerged in a recent study of patients with hepatitis C [26]. It may be that the fear of contagion was not prioritized in our patient-informed elements because of a lack of a patient representative from the infectious disease community. We acknowledge this model is not static and will require ongoing refinement with additional patient groups, and additional recommendations will follow as we gain more experience with their application in value assessment. We continue to build our connections, such as in women’s health, to bring this important work to other patient communities not represented in our initial effort. This work is a first step in deriving patient-informed value elements. Although one of the SAC patient representatives was from the Hispanic community, we did not engage Spanish-speaking patients in this phase of the work. It is possible that non-English-speaking communities may have different values in healthcare that stem from different experiences. One immediate next step for our team is to conduct similar patient engagement within the Hispanic community using Spanish language instruments. Our experience with the Hispanic community has revealed a limited number of organized communities that advocate on behalf of patients. This has taught us that we must first understand the needs of the community before we can introduce the concept of value assessment. Our team has invested significant effort into understanding where to start, which is enabling replication of this work in the Hispanic community. A similar investment will be needed for other cultural and racial ethnic groups. The patient voice represented in this work reflected individuals already familiar with value assessment. Nonetheless, this work provides a foundation from which to advance the patient voice in value assessment.

The main goal of this work was to identify patient-informed value elements that could be tested and used in existing value frameworks or value assessments, such as cost-effectiveness analyses. It was not the intent to develop a value framework de novo nor to develop a patient-reported outcome instrument. While the patient-informed value elements are an important advancement, we acknowledge that more work needs to be done. Future applications will be to examine the potential for the candidate, patient-driven value elements described here to be used in economic models for target medical conditions. Phelps et al. [27] described that potential value elements may fall into a category of “conceptually feasible but generally impractical” for use in economic evaluations. Additional work by this team will determine elements that may be impractical for economic evaluations but could be meaningfully applied in other health technology assessments, such as patient preference evaluations. In addition, we plan to quantify the importance of our 42 proposed patient-informed value elements across the range of medical conditions. This will address issues of comparability in priorities for the value of health interventions across medical conditions. For example, value elements important to patients with cancer diagnoses may not be equally important to those living with chronic conditions, such as diabetes mellitus, or to those living with rare diseases. Careful consideration of patient-driven elements and their generalizability across medical conditions will be important for future patient-centered value assessments. We hope this work will stimulate a dialogue on approaches to quantifying patient-informed value in economic evaluations.

5 Conclusion

The formative work described herein responds to the call for patient involvement in value assessment. We present a methodologic process used to derive patient-informed value elements with continuous patient engagement throughout the process. The hope is that these elements can be implemented in forthcoming research to advance the field of value assessment. The ultimate goal is to improve health technology assessment, data-generation, and decision-making processes.