FormalPara Key Points

When asked their views on the possibility of involving patients in health economics modeling, 22 health economists described a variety of potential benefits, harms, and variables of interest with respect to the process.

Potential benefits and harms pertained to the model, modeling process, modeler, and patient, respectively.

Findings suggest that modeler and patient characteristics, details about patients' roles, the goals of modeling, and high-level procedures surrounding patient involvement in modeling should be considered in future research.

1 Introduction

Patient involvement in health economics (HE) has been advocated on numerous grounds, including patient rights, democratic principles, methodological rigor, and instrumental goals [1, 2]. In addition to involving patients in health technology assessment [1, 2], health economists have advocated involving patients in the modeling process [3]. Specifically, van Voorn et al. [3] argue that patient involvement in HE modeling could result in better incorporation of patient expertise, enhance patient understanding and acceptance of HE models, and improve model validation. Additionally, patients might assist in managing value judgments in HE modeling [4]. Value judgments are decisions that invoke not only “scientific criteria” but social and ethical values [4], sometimes called “unforced methodological choices” in modeling [5]. How to manage value judgments in science is a topic of rigorous debate, central to the conception of scientific objectivity [6, 7].

Value judgments in HE modeling have recently been described in some detail [4], and are consistent with descriptions of value judgments in other modeling areas, such as climate modeling [5, 8,9,10,11,12] and mainstream economic modeling [13]. Value judgments in modeling are described in different ways, but generally include determining the purpose of a model, informing “trade-offs” in the modeling process, managing uncertainties and setting standards of evidence, choosing evidential categories, and using normative concepts [4, 5, 8,9,10,11,12,13]. How best to describe value judgments in HE modeling and where to encourage patient involvement in these decisions are open questions. For example, van Voorn et al. [3] (p.131) suggest that patients could be involved in six modeling activities—formulating the problem, designing the conceptual model, setting model requirements and applicability limitations, selecting decision-making indicators, and discussing uncertainties—all of which involve value judgments, as described in the values in modeling literature [4, 5, 8,9,10,11,12,13].

The observation that HE modeling involves value judgments is consistent with analyses of values in science, generally, across a range of fields [6]. To manage value judgments in science, Elliott [6] (p.14) recommends following three principles: transparency (making value judgments clear), engagement (inviting people to scrutinize value judgments), and representativeness (informing value judgments with widely held social and ethical values). Although Elliott's recommendation invites further discussion [14, 15], it provides theoretical support for involving patients in HE modeling; Elliott's [16] framework suggests patient involvement in HE modeling can enhance transparency and allow researchers and patients to inform value judgments together.

Although there is a strong theoretical rationale for involving patients in HE modeling, there are reasons to consider some potential risks. For example, van Voorn et al. [3] mention the considerable investment needed to educate patients about HE models and to train patients and modelers to work together, the perceived risk of bias from patient involvement, and the concern that patients might “over-emphasize” treatment side effects or advocate for specific interventions. In general, Sampson et al. [17] identified both risks and benefits associated with mechanisms to increase transparency in decision modeling, though patient involvement was not itself analyzed as a transparency mechanism. Despite the potential risks, van Voorn et al. [3] support involving patients in HE modeling, arguing that the benefits will likely outweigh the risks using strategies such as developing a “proper selection procedure”, involving only patients capable of taking a “neutral view”, and including a minimum of five patients with significant differences [3] (p.131). In the future, studies could explore these variables (e.g., selection procedures, number of patients involved, patient characteristics) and measure their impact on risks and benefits of patient involvement. Other variables to explore may be indicated by the broader literature on patient and public involvement (PPI) in health research, including the various methods and levels of patient involvement [18].

The PPI in health research literature should help inform future studies on the risks and benefits of patient involvement in HE modeling. However, this literature has tended to focus on PPI in qualitative health research, and PPI in quantitative research may present unique variables, such as statisticians' ability to communicate about analytic decisions [19]. Furthermore, the health literature has not emphasized the role of PPI in managing value judgments in research, as literature in philosophy of science has done [4, 16]. To guide future research, there is a need to identify potential benefits, harms, and variables relevant to patient involvement in HE modeling specifically, including those relevant to managing value judgments. The purpose of this study was to address this need, generating data and hypotheses to inform future research on patient involvement in HE modeling.

2 Methods

This analysis used data from a broader qualitative study on social, ethical, and other value judgments in HE modeling, whose methods have been published elsewhere [4]. In brief, the lead author conducted face-to-face qualitative interviews with English-speaking HE modeling professionals in British Columbia (BC), Canada, between February and May, 2019 (BC Provincial Research Ethics Platform H18-03694). Recruitment was done by email through an academic community group, and participants received advance material before the interview (for details, see Online Resource 1 and 2). Given the complexity of the research topic, data saturation was not a target; rather, researchers aimed to interview as many participants as possible within a pragmatically determined data collection period [20, 21]. The semi-structured interview guide developed for the study (see Online Resource 3) included one question about the possibility of involving patients in modeling, which was asked at the end of the interview (“Before we close, I would like to ask your views about having patients involved in the modelling process. Do you have any thoughts about this as a potential?”). The focus of the current report is this question.

Data analysis was performed by both authors, who read and reviewed written transcripts derived from tape-recorded interviews. Following Braun and Clarke’s [22] theoretical thematic analysis, the lead author organized participants' responses into three theory-driven categories: potential benefits, potential harms, and variables of interest. These categories were driven by the theory that patient involvement in HE modeling has benefits and risks, with numerous factors influencing the net benefit [3]. The lead author then developed a list of codes to identify themes in participants' responses within those categories (i.e., data-driven themes within theory-driven categories). N-vivo 11 software was used to facilitate data management and development of coding schemes. Both authors subsequently reviewed codes and coded material together. Decisions as to the level at which to distinguish themes (i.e., whether and how to collapse codes) were informed by the purpose of the study. Specifically, the authors aimed to preserve important conceptual distinctions and unique insights from individual participants (i.e., to document every conceptually unique response, even if made by a single participant), while considering what level of detail would be useful to inform future research. The results below reflect the authors' shared judgment and interpretation of the data. The authors' theoretical position and the influence of their judgment on the results are addressed in Online Resource 1.

3 Results

22 HE modeling professionals were interviewed. Participants were generally highly experienced and held a variety of positions in university, hospital, and institutional research settings; further detail on participant characteristics and interview locations is provided in Online Resource 1. This section describes their responses to a question on the possibility of involving patients in the modeling process. Material within theory-driven categories (potential benefits, potential harms, and variables of interest) was coded into one or more of several data-driven themes and subthemes (see figures). Below are qualitative descriptions of each one. To protect anonymity, the pronoun “they” is used for all participants (P1–P22) and details about specific models are omitted.

3.1 Potential Benefits: Model, Modeler, Process

Participants described potential benefits of patient involvement in modeling in a variety of terms, which the authors interpreted within three themes: benefits to the model, modeler, or process (Fig. 1). Benefits to the model were descriptions of improvements to models themselves, including observed (i.e., based on experience working with patients) and hypothetical (i.e., possible) improvements. Specifically, participants said patient involvement had led or might lead to models' including things that would have otherwise been excluded, or vice versa. For example, P17 described modeling the impact of a personal health technology, saying patients had influenced the choice to include “two different outcomes to make a decision”, i.e., the incremental cost-effectiveness ratio (ICER) and a patient-level outcome (“convenience and stigma and socially, how they feel about living with that outcome”). P17 framed this as an improvement, saying “I think it adds value for the entire model built and we would not have done that if we have not talked to patients”. Similarly, P16 said patient involvement might lead to models' excluding irrelevant things, commenting “there may be pathways that we think seem perfectly sensible that are completely implausible”. Among participants who mentioned potential benefits to the model, some spoke of the need to ensure that models examine relevant phenomena, framing the benefit of patient involvement as ensuring models represent what is important to patients (e.g., P11: “patients are in the best position to tell us what is the most important outcome”). Others spoke to the need for models to be consistent with patients' factual knowledge, interpreting the potential benefit of patient involvement in these terms. For example, P20 said, “As a modeler, I try to model the pathway of some disease, or some interventions, and I try to make it correct as I can according to the literature, talking to the expert...but it might be useful to have a patient’s view”.

Fig. 1
figure 1

Potential benefits of patient involvement in modeling. The potential benefits of patient involvement are represented in distinct bubbles: benefits to the model, modeler, modeling process, and patient. “Benefits to the patient” appears with a question mark because it was not directly identified by study participants. Potential benefits to the model are detailed in three bubbles (excludes irrelevant phenomena; includes relevant phenomena; consistent with patients’ factual knowledge). Potential benefits to the modeler are detailed in four bubbles (knowledge; level of interest in the model; decision-making; communication). Potential benefits to the modeling process are detailed in six bubbles (more adequate for model conceptualization; more adequate for decision-making; transparent; democratic; objective; legitimate)

Full size image

Benefits to the modeler were descriptions of improvements in modelers' knowledge or other attributes, both observed and hypothetical. For example, P18 said patients are able to “challenge a lot of the assumptions that you’re making”, describing their recent experience working with patients as “blowing my perception of what I’ve been doing out of the water”. Although they did not report having direct experience working with patients, P1 thought patients could hypothetically provide “valuable context” and a “better way of conceptualizing and understanding the disease process”, with modelers “gaining meaningful information about how to model”. Beyond providing knowledge, P13 suggested that patients might improve modelers' motivation, saying they personally tend to be “more interested” in modeling things someone they know has experienced. For P7, another possibility was that giving modelers a “sense of the potential impact” of decisions “might on some unconscious level make them make better decisions”. While P7 pointed to a potential indirect effect on modelers' decision-making, P22 described an indirect effect on modelers' communication practices. They remarked: “the patient is doing nothing except being there, and forcing you to articulate. That on its own seems to have some power”.

Benefits to the process were descriptions of improvements to the modeling process (without direct reference to models), generally given in theoretical or abstract terms. For some participants, process-level benefits equated to making the modeling process more democratic, or otherwise more trustworthy. For example, P17 gave a democratic rationale for patient involvement in modeling (“you need to involve everyone that’s affected in that process because their values might be different from ours”), while P12 suggested it might improve “the legitimacy or transparency or objectivity of the exercise”. For others, patient involvement implied making the modeling process more adequate for model conceptualization and decision-making. Specifically, P21 spoke of involving patients in reviewing model results “to explore unintended consequences of recommendations that could stem from your results”, which the authors interpreted as a potential improvement to decision-making.

3.2 Potential Harms: Model, Modeler, Process, Patient

Participants described potential harms of patient involvement in modeling in a variety of terms, which the authors interpreted within four overlapping themes: harms to the model, modeler, patient, or process (Fig. 2). Harms to the model were descriptions of negative effects on models themselves, which were given in hypothetical terms by two participants (P1, P18). Both feared that patient involvement in modeling could increase model complexity in an undesirable way. For P18, modeling involved balancing “between complexity and a simple, actionable piece of work”, and they worried that patients might object to model simplification. Similarly, P1 said patients might want to include irrelevant decision nodes, ones that would not impact the model results but make models “difficult to operationalize” and “time-consuming”. Furthermore, P1 noted the results of such complex models would be difficult to communicate to people.

Fig. 2
figure 2

Potential harms of patient involvement in modeling. The potential harms of patient involvement are represented in distinct bubbles: benefits to the model, modeler, modeling process, and patient. Potential harms to the model are detailed in three bubbles (includes irrelevant phenomena; overly complex; difficult to communicate results). Potential harms to the modeler are detailed in one bubble (comfort on the job). Potential benefits to the modeling process are detailed in three bubbles (slowing the process; reducing rigor; complicating project management). Potential harms to the patient are detailed in four bubbles (unnecessary burden; being offended; tokenism; paternalism)

Full size image

Harms to modelers and patients, respectively, were descriptions of negative effects on individuals, which were given in hypothetical terms. Speaking to potential harms to modelers, P1 drew an analogy to negative experiences working with clinicians. They explained “It can be a nightmare when they have expert level knowledge of their field but they don’t connect the dots...I actually have to go away and program this- I don’t want to do that unless it’s necessary”. The authors interpreted this comment as evoking judgments around model complexity versus actionability (as referenced by P18), but centering the negative effect on modelers of their judgment being overruled. Elaborating on their concerns, P1 said, “I would feel more comfortable getting information from a survey from a variety of patients, rather than having a patient partner sit down with me”. The authors interpreted P1's comments as alluding to potential harms to modelers (e.g., comfort on the job). Speaking to potential harms to patients, P1 suggested that participating in modeling could be an unnecessary burden: “If a patient is thinking about cost-effectiveness, to me alarm bells are going off. They should be advocating for their own health and that’s it”. P1 also expressed concern about involving patients without guaranteeing their contribution would be meaningful, linking this potential to the harms of tokenism and paternalism. For P18, a potential concern was offending patients, as model simplifications could imply “you're ignoring this nuance of the disease which we know is important”.

Harms to the process were descriptions of negative effects on the modeling process (without direct reference to models), which were given in hypothetical terms. For example, P17 referenced the time it would take to train patients, especially if patients were expected to interpret or analyze data, which for them raised “some concern in terms of project management”. The potential for patient involvement to slow the modeling process was reinforced by P21, who referenced “the opportunity cost of producing timely scientific evidence to help inform policy, as opposed to taking the time to ask everyone their opinion”. Furthermore, P1 suggested that tokenistic patient involvement could make the modeling process less rigorous. They commented, “there’s no value-add [if] they’re just rubberstamping me. If anything, it’s going to lull me into a false sense of security about how good a job I’m doing”.

3.3 Variables of Interest

Participants referenced many variables that could influence the net benefit of patient involvement in modeling, which the authors interpreted within six themes (Fig. 3). Some participants raised questions about the patient's role within the modeling team, including which decisions patients would be involved in and how much influence they would have over those decisions. In this regard, participants suggested there would be benefit to patient involvement in certain decisions (e.g., defining the research question, conceptualizing the model, determining model structure), but not in others (e.g., choice of modeling approach, choice of distribution to fit to a parameter). One consideration raised was the effect of involving patients in decisions they could not realistically influence. For example, P17 mentioned that decision-makers often want outcomes measured a certain way (“if you bring in patient partners [who] don’t like that outcome measure, there’s not much you can do”), while others said modeling decisions are often influenced by feasibility, including data availability, and informed by specific types of expertise.

Fig. 3
figure 3

Variables of interest to patient involvement in modeling. The variables of interest relevant to patient involvement in modeling are represented in distinct bubbles: goals, procedures, modeler characteristics, patient characteristics, participators, and patient role. Goals are detailed in three bubbles (diversity; representation; target model user). Procedures are detailed in five bubbles (target projects for involvement; guidelines; timelines; methods of engagement; recruitment procedures). Modeler characteristics are detailed in five bubbles (belief in patient involvement; experience with patients; assumptions; openness to change; communication strategies). Patient characteristics are detailed in three bubbles (capabilities; training; perspectives). Participators are detailed in five bubbles (number of patients; identity of patients; number of other stakeholders; identity of other stakeholders; power dynamics). Patient role is detailed in two bubbles (which decisions patients involved in; how much influence patients have)

Full size image

Participants also pointed to variables in high-level procedures around involvement, i.e., outside patients' role within the modeling team. For example, P6 questioned whether all modeling projects would be targets for patient involvement or just some, saying they prioritize involvement “at different levels” across models (“If it is a model that is going to be used by patients and providers they must be involved, no question”). Regarding identifying patient partners, P2 said it currently “takes quite a bit of effort”, raising the question of how recruitment procedures might vary. Other participants focused on methods of engagement, saying focus groups or surveys could be alternatives to having patients join the research team. Another variable identified was timelines: according to P22, in some modeling projects, there is “time pressure”, in others “no one’s hammering at the door”. Speaking directly to the many variables surrounding patient involvement, P6 said, “we need better guidelines”, expressing doubt that procedures should be “left to individual researchers to decide”. The importance of considering the high-level procedures around patient involvement in modeling was underscored by P15:

“Practically speaking, I’m not sure how to manage that procedurally and make sure that people are involved enough, but not too much. I think the mechanics of how that works and processes take more thought. At a basic level, making sure that we talk with patients about what we’re measuring, about who we’re including, about what might be a confounder versus something that’s on the pathway of effects- all makes really good sense”.

Participants also reflected on modeler and patient characteristics. With respect to modeler characteristics, participants highlighted variables like previous experience working with patients, openness to change (e.g., to changing methods), assumptions, belief in the value of patient involvement, and strategies for communicating about modeling and model results. Regarding patient characteristics, participants mentioned capabilities (e.g., understanding modeling concepts and results), training, and personal perspectives. For example, P17 questioned whether involvement in modeling would cause patients to lose their unique perspective, while P3 thought patients would not be oriented to the “best interest of society” (“if you’re a patient, you want your treatment to be paid and that’s it”). Expressing a similar view, P19 commented “if I have diabetes, every question related to diabetes is important for me”.

Another variable of interest was participators in modeling, i.e., patients and other stakeholders participating in the process. Regarding which patients, participants raised distinct questions regarding both patient identities (i.e., who are the patients?) and the number of patients involved in a given project. Regarding other stakeholders, participants suggested many stakeholders other than patients could be involved in modeling, including physicians, members of the public, and other people “on the receiving end” of the model (P3). On this topic, P12 said, “patient” need not mean having a “specific condition” but could mean “patient-as-a-society-member”. A distinct issue surrounding participators in modeling pertained to power dynamics. In particular, P22 mentioned that “you’re often doing it [building the model] for a policy-maker who, in some ways, has the most power, because they’re the ones who make the decision”. At the same time, P22 remarked, clinicians who participate in modeling have considerable power (“the doctor says something, you sort of have to do it”), while modelers themselves have a discreet control over the process (“the team who’s actually doing the work is making many of the judgments that you don’t think about”). For P22, existing power dynamics in modeling had implications for patient involvement (“let’s just be honest, the patient is going to be low-down in the power”). Unlike P22, P3 expected patients to have considerable power, based on personal experience (“I’ve done work with patients and I want to just give them the treatment”). For this reason, P3 said “the power needs to be balanced out” within modeling teams.

The last variable distinguished concerned the goals of modeling. In the authors' interpretation, goals were sometimes implied by participants. For example, some participants worried that the patients involved would not be representative of a larger target population of patients, without questioning that representativeness was a relevant goal. P21, for example, said it was important that patient input be “sufficiently representative of patients that it can be incorporated without concerns of introducing bias or reducing generalizability”. However, the assumption that representativeness would always be a relevant goal was challenged by P15, who remarked that in their own field “questions about the diversity of perspectives represented in the process are especially important”. Although the goals of modeling were sometimes implied, one comment from P8 was relatively explicit: “Maybe we need to think about who this model is really for”.

4 Discussion

4.1 Key Findings in Context

Initiatives to involve patients in health research have expanded, supported by programs like the United Kingdom's INVOLVE and Canada's Strategy for Patient-Oriented Research [23, 24]. However, initiatives focused on HE have been limited [25, 26]. One possibility is to involve patients in HE modeling, which could help ensure models are acceptable to patients and incorporate their expertise [3, 25]. Involving patients in HE modeling could also be a direct way to enhance transparency, while giving health economists and patients a chance to mutually inform social and ethical value judgments that arise in the HE modeling process [4, 16]. From a theoretical perspective, the evidence that modeling requires making social and ethical value judgments [4, 5, 8,9,10,11,12,13] lends support to initiatives to involve patients in the process: at least, where scientists must make social and ethical decisions, the need for democratic accountability in science suggests consulting the people those decisions will affect [27].

Participants in this study pointed to numerous possible benefits of patient involvement in HE modeling, which we categorized as benefits to HE models, HE modelers as individuals, and the HE modeling process generally. Conceptually and in practice, many of these benefits may overlap with each other. For example, participants suggested that patient involvement might help ensure that phenomena included in HE models are relevant and important to patients and are represented in a way that is consistent with patients' factual knowledge. Participants also suggested that patient involvement might have a positive influence on modelers, increasing their knowledge and interest, improving their communication skills, and informing their decision-making. To the extent that the quality of HE models is directly influenced by modelers as individuals, these two categories of benefits may be seen as overlapping. Benefits to HE models may also be seen as more or less overlapping with benefits to the modeling process, from making the process more transparent, democratic, objective, legitimate, and generally more adequate for model conceptualization and decision-making. The findings of our study add to the literature that helps conceptualize the benefits sought through patient involvement in HE activities [1, 2] and should encourage further conversation about how best to describe and distinguish between these benefits in the context of HE modeling. In particular, our participants' descriptions of potential benefits to HE models and modelers, specifically, appear to be a novel contribution to the literature.

In addition to benefits, participants in this study identified certain harms that could potentially result from patient involvement in HE modeling. Speaking of HE models themselves, participants voiced the concern that patient involvement might lead to overly complex models, to models that include irrelevant phenomena or whose results would be difficult to communicate. Conceptually, these potential harms invite comparison to potential benefits, as described above. For example, the inclusion of relevant phenomena was framed by participants as a potential benefit of patient involvement, the inclusion of irrelevant phenomena as a potential harm, raising the question of how to distinguish between relevant and irrelevant. This finding underscores lessons from the values in modeling literature, specifically that modeling requires “relevance judgments” which are “irreducibly normative” (i.e., value laden) [28] (p.3). On our interpretation of theoretical accounts within the philosophy of modeling, what is included in a model, a model's complexity, and the ease with which a model's results can be communicated are not only interrelated model attributes, but values, which stakeholders may weigh differently [5, 8,9,10,11,12,13]. Distinguishing between potential harms and benefits to HE models, then, will require establishing from whose perspective a model should be evaluated. Furthermore, the same principle applies to harms and benefits to the modeling process. In this study, participants voiced concern about slowing or complicating the modeling process, or reducing its “rigor”, which reflects their values and perspectives (indeed, some stakeholders might consider a slower process beneficial). The current study should thus encourage further conversation around whom to involve in establishing procedures around HE modeling, including patient involvement, as stakeholders will likely have different values around processes and outcomes. In this context, still another important finding is that participants had concerns about potential harms to patients and modelers from working together. Although the potential harms identified were different for modelers (comfort on the job) and patients (unnecessary burden, being offended, tokenism, paternalism), both are worthy of attention.

In addition to potential benefits and harms, this study generated numerous variables relevant to patient involvement in modeling, i.e., factors that could influence the net benefit of patient involvement. Our findings reinforce van Voorn et al.’s [3] observations (on the importance of patient characteristics and selection procedures), while also generating further detail and many other points of interest. One particularly important variable identified was the goals of HE modeling; indeed, whether the goal is to model “representative” perspectives or “diverse” perspectives and whether the goal is to meet the needs of one type of model user or another are core questions which must be addressed in order to produce adequate models. Other important categories of variables identified included high-level procedures surrounding patient involvement (i.e., target projects for patient involvement, guidelines, timelines, methods of engagement, recruitment procedures), details around the patient role (i.e., which decisions patients are involved in and how much influence they have), and participators in modeling generally (i.e., patients and other stakeholders, both in terms of numbers and identities, and the power dynamics among them). In our view, these three categories of variables seem particularly likely to be influenced by institutional factors (e.g., funding, institutional-level values), which were not explored in-depth by participants in this study and warrant further attention. Still other variables of interest identified included modeler characteristics (i.e., belief in patient involvement, experience with patients, assumptions, openness to change, communication strategies) and patient characteristics (i.e., capabilities, training, perspectives). In our view, the individual-level variable that may be most likely to be influenced by institutional factors is training, at least to the extent that this might be provided by institutions supporting patient involvement in modeling. It should be noted that participants in this study did not discuss the possibility of training modelers to work with patients, although this is a possibility that warrants exploration.

4.2 Strengths and Limitations

To our knowledge, this is the first qualitative study to describe HE modelers' perspectives on working with patients specifically, and its findings will facilitate comparison to HE modelers' views on working with clinicians [29] and other stakeholders, including the public. A core strength of this research is its capacity to identify key questions for future inquiry, as described below. Although this study generates valuable knowledge, it also has limitations. The data were derived from qualitative interviews mostly focused on value judgments in modeling, and it is not possible to know in what way participants' responses were influenced by their engagement with the other questions asked earlier in the interview. Furthermore, we do not believe that saturation was achieved in the current analysis. This does not mean the current findings are invalid, but rather that the topic has not been fully explored [21]. Finally, the study findings represent the perspectives of the 22 HE modelers who volunteered to participate in interviews, and it is unknown to what extent other HE modelers will share their views. This limitation is a reason to explore the topic among HE modelers in other settings.

4.3 Future Questions

The findings of this study raise numerous questions for future research. First, participants did not speak to certain benefits and harms that are nonetheless possible in theory; it would be helpful for future inquiries to include direct questions about these possibilities. For example, no participants spoke to potential direct benefits to patients from participating in HE modeling, though they spoke to potential direct benefits to HE modelers. One potential for future research is to ask patients for their perspectives on participating in HE modeling, including what benefits and harms they would anticipate. It would also be useful for future studies to ask HE modelers and patients whether they want to work together (considering that both potential benefits and harms have been identified), and what variables they would expect to influence the overall benefit of the process. In this context, it would be fruitful to explore the perceived importance of structural supports—such as training and guidelines for patients and modelers—as this would help to guide initiatives supporting patient involvement in HE modeling.

Ultimately, it will be important for empirical studies to determine to what extent patient involvement in HE modeling actually affords the benefits that are being sought. However, such studies will require first addressing the overarching question: what benefits are sought by patient involvement in HE modeling? If one of the benefits is giving patients the opportunity to inform social and ethical value judgments in modeling, this raises several more key questions, many of which are philosophical [16]. Just some key questions include how best to describe value judgments in HE modeling, which value judgments patients should be invited to inform, and what other stakeholders—such as the general public—should given the same opportunity.

5 Conclusion

This qualitative analysis has contributed to the nascent literature on patient involvement in HE modeling, providing a broad view of potential benefits, harms, and variables of interest from the perspective of HE modelers. The results should encourage future research to explore how patient involvement in HE modeling may result in benefits and harms at different levels—including HE models, modelers, processes, and patients—and to determine what will ultimately influence the net benefit. A priority is to clarify what benefits are sought through patient involvement in HE modeling, as just one possibility is the benefit of better informing social and ethical value judgments in HE models.