FormalPara Key Points for Decision Makers

A modeling effort has been launched to test the hypothesis that continuous stakeholder engagement may result in health economic (HE) models more widely considered by decision makers in the US.

Early-stage findings show that such an engagement approach has yielded valuable insights for the model design, and have identified several implementation challenges.

The stakeholder engagement approach to model development may result in HE models more widely considered by decision makers, and has the potential to greatly improve health technology assessment in the US.

1 Introduction

Under pressure to curb rising health care costs in the US [1], private and public payers, policymakers, clinical societies, and other stakeholders have increasingly turned to formalized health technology assessments (HTA) to assess the relative value of health interventions [2,3,4]. Health economic (HE) modeling [5] is used to predict and compare the costs and health outcomes of interventions and is commonly utilized to inform HTA outside the US [6, 7]. Although evidence generated from HE models is being used more commonly in coverage and reimbursement deliberations in the US, it is not consistently adopted by different stakeholder groups or across therapeutic areas [3, 8,9,10].

Experts from multiple disciplines have called for a more open and inclusive process in the development of HE models and HTA frameworks, and they emphasize the importance of broader inclusion of patient and employer perspectives [2, 11, 12]. We hypothesize that actively engaging with multiple stakeholder groups throughout the HE model development process—and particularly during initial planning and model design phases—may result in more relevant and useful models.

To implement such an approach, the Innovation and Value Initiative (IVI) has established a process that empanels an advisory group in the disease state of interest. These advisors provide guidance on key modeling considerations, identify data sources, and generate use cases throughout different phases of the modeling process. As participants in the HE modeling process (Table 1), the advisory group guides modeling decisions from the beginning (Fig. 1).

Table 1 Primary actors in model development process
Full size table
Fig. 1
figure 1

Development process for health economic models with multi-stakeholder input

Full size image

2 Inclusion of Multiple Stakeholders in Health Economic Model Development

In the US, HE models are often developed by for-profit economic consultants, academic researchers, or health economics and outcomes research experts within life sciences companies [13, 14]. Key model assumptions, specifications, and data inputs are often determined by technical experts, and access to key assumptions or data inputs in the HE models can be limited due to intellectual property concerns [13, 15]. Additionally, HE models seldom consider or incorporate direct patient perspectives and inputs, and they might fail to capture real-world patient experiences valued by decision makers [11, 16]. Perspectives from external stakeholders are often sought ex post and used as contextual inputs or considerations rather than being consistently incorporated into models. The lack of transparency about key modeling decisions and limited stakeholder participation in the model development process can lead to misalignment between model design and decision needs of end users [13]. These limitations cause relevant decision makers to question the credibility and relevance of cost-effectiveness estimates and, ultimately, the ability of models to inform HTA in meaningful ways [17, 18].

In July 2020, IVI launched its third Open-Source Value Project initiative to build an HE model to support HTA for major depressive disorder (MDD). We chose MDD due to its prevalence, significant societal burden, and broad interest among stakeholders who are looking for better treatments and more cost-effective resource allocation [19, 20]. As a first step, IVI convened a 20-member Advisory GroupFootnote 1 (AG) consisting of patients (n = 5), employers (n = 5), clinicians (n = 5), innovators (n = 3), payers (n = 2), and researchers (n = 2) to weigh in throughout the modeling process.

To date, the AG has provided feedback on the model’s conceptual framework [21] as well as the model design, including identifying (1) decision needs (model objectives); (2) gaps in existing economic models; (3) data sources for model inputs; and (4) the most appropriate analytic framework when multiple approaches exist. Feedback was provided through group meetings, surveys, emails, and individual discussions. In prioritizing feedback from the AG, we solicited additional insights from patients and employers, two traditionally under-represented stakeholder groups in the HE modeling process [11, 12, 22].

2.1 Early Insights

While still in the early stages, this ongoing stakeholder engagement has already yielded valuable insights, recommendations, and resources (Table 2). This input is meaningfully impacting model design in three key areas discussed here.

Table 2 Advisory Group input guiding model development
Full size table

AG input has highlighted important decision contexts and factors considered by end users that help inform the model specification For example, the Hamilton Depression Rating Scale and Montgomery-Asberg Depression Rating Scale are the clinical instruments most widely used to define health states in existing models, but stakeholder input indicated that the Patient Health Questionnaire-9 is more commonly used in clinical practice and increasingly in clinical trials and observational studies. In discussions about potential model structure, all stakeholder groups emphasized that care setting (primary vs specialty care) is an important variable that will impact prescribed treatments and their effectiveness. Therefore, we are reviewing additional literature to evaluate how care settings will impact patient trajectory in model simulations. Patient groups also emphasized the importance of prioritizing data inputs from samples more representative of the MDD population in the US.

AG members have shared preliminary, and sometimes proprietary, findings and data from ongoing research projects that can be used as model inputs For example, a researcher shared data on caregiver burden that can be a direct model input [23]. Employer, patient, and researcher representatives highlighted the need to measure different types of productivity impacts (e.g., absenteeism, presenteeism) and provided suggestions on data sources and literature.

AG members who are potential end users are interested in engaging in the modeling process and applying the model in their decision making Despite their differing decisions needs, many stakeholders emphasized similar considerations in model design. These included reflecting patient heterogeneity in treatment experience, facilitating subgroup analyses, and fully measuring different types of productivity loss (Table 2). Moreover, AG members also proactively proposed use case development. For example, one stakeholder suggested that the model design would allow her organization to assess the burden of MDD without active treatments.

The discussions also identified key areas of consensus across stakeholder groups regarding the purpose, design, and use of existing HE models (Table 2 and “Appendix 2”, see ESM). Suggestions included (1) models should be flexibly designed to accommodate emerging and evolving data; (2) models should incorporate real-world data beyond just clinical trials; (3) entire treatment pathways should be modeled rather than just individual treatment comparisons; and (4) models should incorporate patient perspectives and inputs.

2.2 Implementation Challenges

Eight months in, we have identified several challenges to implementing this new approach. First, eliciting and incorporating feedback from diverse stakeholders to inform HE model development requires experience with multiple methodologies (both qualitative and quantitative), as well as creativity and flexibility in their use. For example, roundtable discussions required focused questions and well-identified goals to generate specific feedback on research questions. Small-group discussions were sometimes more appropriate for engaging those with specific expertise. Qualitative AG input sometimes needed to be supplemented with additional literature reviews to identify appropriate methods—to understand the measures and estimates of presenteeism, for example. Finally, to ensure inclusive input, discussion materials must account for varying levels of clinical or technical knowledge. While time consuming, these steps are crucial for soliciting useful input.

This approach requires foresight to address differing priorities. Through email communications, meeting summaries, and discussions, we have worked to ensure that all viewpoints are documented and acknowledged, and that methods for prioritizing input are transparent. We are committed to communicating with the AG about our decision-making process, and plan to prioritize considerations with the broadest stakeholder buy-in, inputs from traditionally under-represented stakeholders, and inputs that are most feasible to implement [24].

Finally, engaging a large, diverse stakeholder group increases the scope and complexity of model development. The research team leading the process must invest considerable time and resources to planning and managing engagement activities, synthesizing conflicting viewpoints, and determining how to incorporate feedback into the model design. This may be particularly challenging for those lacking engagement experience. Refining the methodology through further implementation may increase efficiency, however.

Advisory Group members must also commit their time and energy to an ongoing process, and the research team may need to account for missing stakeholder perspectives. Some key stakeholder groups were not able to participate in our Advisory Group, namely pharmacy benefit managers, benefit consultants, and government agencies (e.g., Centers for Medicare & Medicaid Services). As these are key decision makers and potential users, we intend to use public comment periods and individual outreach to solicit feedback.

3 Learning Through Action

As the next phases of the MDD development transpire, we will continue to evaluate the process with the AG. Specifically, we will assess how this engagement has impacted model design, whether it will increase the consideration of HE models by different stakeholders, and how well it facilitates open dialogue and trust across stakeholder groups.

As US policymakers and payers continue to prioritize a transition to value-based care, IVI is advocating a concurrent evolution of methods to better reflect the inputs and needs of broader stakeholders. Our multi-stakeholder engagement approach to developing an open-source HE model has the potential to greatly improve HTA in the US.