FormalPara Key Points for Decision Makers

Collaborating with patient partners resulted in a more comprehensive microsimulation Markov model that better represented the current standard of care for adults with major depression in British Columbia, Canada. Benefits included more precise terminology, validation of the treatment pathways, and inclusion of elements that the researchers on the team had not considered.

An unexpected benefit of this collaboration was informing directions for future research and modeling that are based on patient partner priorities.

Moving through the “uncertainty” and “mapping” stages as a team set the stage for more robust structured collaboration later on.

1 Introduction

Patient engagement is becoming more common in healthcare research and engaging patients in co-production of research has been described as more ethical and democratic [1]. As defined by the Canadian Institutes of Health Research’s Strategy for Patient-Oriented Research—Patient Engagement Framework, patient engagement is “meaningful and active collaboration in governance, priority setting, conducting research and knowledge translation” [2].

Many opportunities for patient engagement in health research in general exist, though patient engagement in simulation modeling is relatively scarce. Simulation modeling is used to assess the costs and benefits of healthcare interventions and as a basis for policy decisions that affect the public at large, and patients, more specifically. While Staniszewska et al. recently presented a framework for public involvement in economic modeling, the authors note the distinction between public engagement and patient engagement [3]. Another recent paper proposed principles for patient and public engagement in health economic research more generally, but does not give specific suggestions for simulation modeling [4]. These papers reveal an increasing awareness of and need for greater exploration of the inclusion of patient partners in the design and development of economic simulation models to embed their perspectives in the decision-making process.

Guidelines for meaningful engagement of patients in simulation modeling are absent. However, there are more general frameworks for conceptual economic modeling [5] and for public involvement in the same [3], which can be built upon to include particulars for patient engagement. Although coding intricate and often divergent healthcare pathways into a simulation model is a complex task, the opportunity to involve patients in the modeling process offers rewards for modelers and patient partners alike. These include increasing modelers’ contextual knowledge and giving a sense of potential impact and motivation and increasing patient partners’ feelings of ownership of findings as well as fostering a greater understanding and possible public acceptance of the model [5,6,7]. There are also arguments that the model itself is likely to benefit from increased credibility, salience, and legitimacy as a result of patient involvement [7].

Our microsimulation Markov model, part of a larger research study, was designed to approximate the standard of care for major depressive disorder in adult patients in British Columbia (BC), Canada, with and without the use of pharmacogenomic testing as part of routine care [8]. Our modeling team, which included two patient partners, combined quantitative and qualitative evidence from the literature, administrative data, and patient, clinician, and other stakeholder perspectives to create an analytical infrastructure with three main uses. These include (1) predicting impacts of changes to current care, (2) developing more complicated modeling scenarios in the future, and (3) use as a template from which other jurisdictions could build models that accurately reflect their own contexts.

Squires et al. [5] suggest that future publications around models should include qualitative descriptions of the quantitative models, which is the focus of another manuscript in development. This paper goes one step beyond a model description to exemplify patient-specific participation in economic modeling and its benefits. We contend that part of the qualitative description of the modeling process should include the authentic patient partner voice; these perspectives and experiences formed an integral part of the current model development, but rarely make it to publication in full form. In the following, we describe our experience of working with patient partners to co-design and develop a microsimulation-Markov model of depression care and offer insight into how the process fits into three sequential stages of model co-development: uncertainty, mapping, and structured collaboration. We provide key examples of where and how patient partner contributions shaped model development and a full list of all suggestions they provided. Finally, we present the patient partners’ thoughts on the modeling process in their own words. We hope that this account of our experience will inspire research teams to engage patient partners in modeling by showing benefits to the model, collective research team, and to the patient partners themselves. We also aim to provide the reader with high-priority ideas for future research in major depression care and modeling endeavors by presenting the patient partners’ suggestions for the same.

2 Collaboration Process

Our team moved through three distinct stages, starting in a place of uncertainty, mapping the way together, and finally reaching a place of structured collaboration, enabling the development of the care pathway for the current model and producing questions and hypotheses for future research.

2.1 Uncertainty

At first, the research team was uncertain about how each team member might best contribute to the modeling. There were considerations of potential benefits and costs to the model and modeling process, and perhaps most importantly, to the patient partners and the other members of the research team [6, 9]. These considerations did not exist independently of one another. For example, the modeling process was constrained from the outset by the project length and budget which, in turn, impacted the amount of training that could be provided. However, after discussing potential benefits, we believed they would outweigh the potential costs. For example, we expected that patient partners would provide a richer picture of the current standard of care, emphasize parameters and outcomes of importance to them, and illustrate where and how the current healthcare system is not meeting patient needs. With these considerations in mind, we met over Zoom to discuss the overall scope of the model, our hopes and concerns, and what we hoped to accomplish with the patient partners. This discussion helped both the modelers and the patient partners clarify issues and expectations, resolving some of the uncertainty and building rapport among team members.

2.2 Mapping

The patient partners helped map the way forward and develop a plan for working together. At their request, documents were pre-circulated ahead of each meeting to allow for a greater scope and depth of reflection and to facilitate preparation. These documents included a living list of assumptions underlying the model and any new questions about the clinical pathway as well as specific questions about areas where patient partners’ perspectives were absent or inadequate. We also created a living document that served as a record of decisions, ideas, and questions from each meeting.

We decided on a structure for our meetings: first, the researchers would present the patient partners with a set of assumptions underlying the care pathway. The patient partners could then reflect on and question these assumptions and, thus, provide a new lens through which to see them. By setting a structure, all team members knew what to expect at meetings with each involving a presentation of new ideas and built-in time for feedback.

2.3 Structured Collaborating and Questioning

With the framework for working together mapped out, the modeling collaboration could begin. The lead modeler (SG) presented assumptions and questions about the model structure and evidence parameters and the patient partners confirmed or questioned them. In several instances, the patient partners asked questions about our evidence parameters that we did not have answers to and that for which data did not exist. On these occasions, assumptions had to be made or limitations acknowledged. Together, we discussed possible solutions until a compromise could be reached: items were marked as “for future research,” a limitation of the current model was acknowledged, or the model was coded flexibly to allow for updates when new information becomes available. In addition to contributing to the current model (Table 1), the patient partners shed light on key priorities for future research (Table 2) that we would have otherwise missed, an unexpected benefit of collaboration.

Table 1 Patient partner suggestions included in the current model
Full size table
Table 2 Patient partner suggestions for future research
Full size table

3 Patient Partner Contributions

In accordance with the GRIPP2 Guidelines for reporting patient partner involvement in research [10], the following sections describe in detail where and how patient partners contributed to the development and structure of the model.

3.1 Impact on Process

A major impact that patient partners had on the modeling process was influencing the language that was used in the model. Together, we created a team-wide glossary, a protocol about using abbreviations in meetings, and a preferred terminology document. Guided by the patient partners, the team saw this as an opportunity to stop using language in research that puts the onus of depression and its perceived treatability on the patient, which in turn reinforces stigma around depression. For example, we chose the term “refractory depression” to replace “treatment-resistant depression” in the model and in the other areas of the project.

The patient partners also influenced and improved the larger research team’s understanding of the model by modifying the flow diagrams that depict the model structure and pathway. Squires et al. [5] underline the importance of the iterative production of diagrams with stakeholders to make assumptions explicit and to communicate the pathways within the model efficiently. Our diagrams were originally intended to be used to keep the larger project team members (including our research oversight committee) up to date on the modeling progress and structure, regardless of their familiarity with modeling. The patient partners had us add color to the figures to make them more accessible by visually grouping relevant information together. We also included an acknowledgment of important limitations and assumptions within the model diagrams to increase their visibility, as opposed to listing these in an appendix. These changes were met with positive feedback from the entire research team and external collaborators, who all found the diagrams easier to follow and subsequently noted they had a better understanding of our modeling process and our goals, enabling them to give more detailed feedback.

3.2 Impact on the Model

Patient partners had multiple impacts on the model itself by making suggestions that were implemented, acknowledged as limitations of the current model, or listed as questions for future research. In some instances, the model was coded to be flexible to allow for changes in the future (when more data become available). The limitations and assumptions in the model represent gaps in the current state of knowledge that are of interest or seen as “high priority” by our patient partners. For this reason, we have documented both the suggestions and modifications that were adopted into the model and those that were not in Tables 1 and 2, as both are valuable.

3.2.1 Checking Assumptions

Assumptions are a necessary part of modeling, and the patient partners were vital in checking our assumptions about the treatment pathway in the current standard of depression care in BC. Through a process of presenting our assumptions, listening to patient partner feedback, then changing or refining our assumptions, we built a model that better reflects actual patient experience in BC.

For example, in the original treatment pathway, we had planned a decision point at week 4 where patients in the model would either continue their medications or discontinue their medications because of an adverse effect or a lack of efficacy. We brought this assumption to the patient partners, who pointed out that the current model did not capture discontinuation for financial reasons. According to them, medication affordability owing to out-of-pocket costs or a lack of coverage by insurance providers is an important and prevalent reason for discontinuing otherwise effective medications. They suggested that we include “discontinuation for financial reasons” as an option at this decision point, but we were unable to find data on the number of patients who discontinue medications for financial reasons in BC.

Because we could not model discontinuation for financial reasons, we noted this limitation in a prominent place in our model diagrams and set this as a direction for future research. However, in order to ensure that the financial burden of medications on patients was reflected in the model, we changed our costing scenario to include two sets of costs of medication: (1) full price (out of pocket and paid by public health) and (2) only the portion paid for by public health. Discussions with the patient partners made it clear that our model needed to include loss of income, out-of-pocket costs, and informal caregiving costs in order to accurately reflect patients’ realities. These modifications were looked on favorably by the entire team.

3.2.2 New Questions and Model Limitations

An unexpected benefit of collaborating was crafting new research questions based on the patient partners’ lived experience, especially when it could not be modeled. For example, though psychotherapy, especially cognitive behavioral therapy, has similar efficacy to pharmacotherapy in treating depression [11] and is recommended by the Canadian Network for Mood and Anxiety Treatments as a treatment for depression in Canada for patients with all severities of depressive symptoms and of all ages, education levels, and cultural and ethnic backgrounds [12], psychotherapy is largely inaccessible to many patients who might benefit from it because it is not covered in BC. The patient partners emphasized access, affordability, and efficacy as variables of importance to model. We searched for data in order to include these as parameters in the model, but were unable to find this information.

We acknowledged several limitations such as this one. Some were due to practical issues (lack of time to code, too complicated to model at this scale), whereas many other limitations were due to a lack of available research and data. Out of these limitations came research questions from the patient partners, examples of which include: “what percentage of those who want to access psychotherapy are able to?” and “how long is the average wait time for referral to a psychiatrist or funded treatment option in BC?” These questions are especially noteworthy because they are clear priorities for patients with lived experience, but there is a lack of research in these areas.

4 Patient Partners’ Perspective: In Their Own Words

“Both of us patient partners came to the modeling project with the same neutral, open expectations as the modelers did. This could partly be because neither of us had ever been involved with modeling, just as the modelers had not previously collaborated with patient partners on this kind of work. Nevertheless, we asked specifically to be included in the modelling and received the full support of the research team.”

“We looked forward to each session due to its collaborative nature. For example, when a term/label did not reflect patient experience clearly enough, we worked together to select and use more pertinent language. Everyone wanted the model to be as complete and clear as possible, and we worked hard to have it reflect real life experience. We were concerned that the model might not accommodate everything we wanted it to, but the modelers clearly explained the rationale for exclusions. We worked with the modelers to clarify what the model could and could not cover.”

“Knowing that readers will be able to see all of the feedback we gave, even what didn’t make it into the model, gives us, as patients, a lot of hope. There is intense validation and confidence in having one’s experiences heard and believed. What’s more, hearing that lessons learned from those experiences will be passed on and shared with other researchers is incredible. Assisting in building the model gave us opportunities to explore thoughts and have discussions that would not have occurred if we had not been a part of the model development.”

“We appreciated the opportunity to learn about and participate in the modeling process. We were encouraged to challenge assumptions made by the team and to engage in meaningful dialogue as we learned about microsimulation-Markov modeling and its strengths and limitations. The research team was welcoming and treated us as equals. They were respectful of our lived experience and of our knowledge of the current standard of care and gaps in the system. This resulted in effective and satisfying collaboration.”

5 Modeler’s Perspective: In Her Own Words

“This was my first experience working with patient partners to create a model. Therefore, I had some concerns at the beginning regarding limitations of the model and how they might affect incorporating suggestions from the patient partners. This concern was resolved after the first couple of meetings as I found our patient partners very understanding about the constraints in the simulation modeling process. As a team, we were able to find a balance around including necessary factors that would impact depression outcomes and creating an operating framework that could be delivered within our specific timeframe. Getting the chance to learn about the treatment pathway directly from those with lived experience of depression was an amazing journey. Just as certainly as collaborating with patient partners improved the model structure, it also increased our confidence in its potential outcomes. These outcomes, in turn, will guide decision making around implementing pharmacogenomic testing as part of standard care for depression in BC.”

6 Discussion

This paper is a rare description of the process of collaborating with patient partners on a complex simulation model, and a testament to the value this adds to modeling clinical scenarios. Patient partner perceptions and experiences formed an integral and beneficial part of model development. While it is uncommon to see their original voice in academic publications, the depth and authenticity provided here would be lost had this been paraphrased. An example of the value of the authentic patient partner voice in the literature is Burch’s commentary on patient-oriented research, where she encourages teams to “expect to learn together” [13]. Indeed, this was one of the most valuable experiences our team had. Experiencing the uncertainty of patient partner economic modeling together (what others have referred to as ‘entering a liminal space’ [3]) was an important part of team building that facilitated our collaborative process through the rest of the project, resulting in a more robust model that was mutually acceptable to the modelers and patient partners alike. This is in line with the understanding that patient partner participation in modeling increases the credibility, saliency, and legitimacy of the health economic model [7].

Our patient partners’ input will directly influence organization-level impacts through their suggestions for research on healthcare system access, as well as long-term (system-level) impacts through a valuable contribution to examining the cost effectiveness of pharmacogenomic testing for depression care. These are especially valuable contributions because there is a gap in the literature on patient-partnered research effects on intermediate and long-term research impacts [1]. Finally, our findings are in line with a recent survey of health economists regarding patient collaboration [6], in that we discovered benefits, not only to the model and the research process, but also to the modeler and patient partners themselves, despite initial feelings of uncertainty.

Regarding current proposed frameworks for economic modeling, in general, or those that include the public [3, 5], we would add that specific considerations for working with patient partners merit attention. For example, Squires et al. note that co-production of diagrams is important so that the “assumptions and beliefs [of stakeholders can] be made explicit so that they can be agreed upon or questioned” [5]. In our project, patient partners checked assumptions developed by the modeling team. The resulting model diagrams performed a double duty by displaying the team’s assumptions about the treatment pathway, and by including footnotes from the patient partners about areas for future research that would make subsequent iterations of the model stronger. We would encourage others to take this ‘patients as experts on lived experience’ approach and, indeed, this seems to be reflected in the recent literature [3, 4]. The strengths of our project included that our simulation model was informed by two patient partners, rather than only one, and their recommendations and viewpoints are presented here in full, providing a transparent account. Given the lack of guidelines for patient partner involvement in modeling, our account and the inclusion of the patient partners’ authentic voice are important additions to the literature.

Our paper is unique in that it includes lists of every patient partner suggestion regarding the model, regardless of whether it is reflected in the final version, and it notes directions for future research based on what our model could not currently accommodate. We included an extended direct quotation from the patient partners themselves, bringing their authentic voice into a space where it is largely absent. These are important features that future guidelines around patient partner involvement in modeling should consider in order to encourage transparency around their processes: reporting all suggestions, reporting directions for future research, and including the patient partners’ thoughts in the description of how the model came to be.

We present three recommendations based on our experience. First, and most importantly, is that patient partners should be meaningfully included in modeling projects. We found this to be a positive experience for both parties that enhanced the final simulation model and left the patient partners with a feeling of “intense validation”, as described above. Second, we recommend embracing the feeling of uncertainty and working through it as a team, not only to build rapport, but to clarify expectations and ideas for the modeling process. Finally, we suggest that in the absence of formal established frameworks, research teams should commit to presenting their experiences fully and authentically, including the patient voice where possible, to enhance others’ learning.

7 Conclusions

Patient partner engagement in the modeling process is an important way to increase the “democracy” of the scientific decision-making process, and as we have seen, leads to more robust models and relevant directions for future research.