FormalPara Key Points for Decision Makers

Most of the health economic decision models used in current practice are single-use models, which are dedicated to the evaluation of one specific intervention. Also, they may lack transparency, validity testing or flexibility.

Multi-use disease models (MUDMs) aim to model a disease rather than a decision problem, thus a single MUDM can potentially evaluate many interventions for the disease in question.

The use of MUDMs requires adequate investment and expertise for their development, but offers substantial long-term organizational and methodological advantages to HTA agencies and policy makers.

1 Introduction

Health care policy makers in many jurisdictions use health economic decision models as a basis for information regarding the cost effectiveness of medicines and other health care interventions. Such information serves to inform policy decisions regarding coverage and price negotiation. The current use of health economic decision models is mostly confined to what will be referred to as single-use models. That is, for each evaluation, a new, dedicated model is developed. In many settings, these models are part of dossiers to apply for coverage. Such dossiers are then assessed by an HTA agency, for instance NICE in England, the Healthcare Institute (ZIN) in the Netherlands, CADTH in Canada and many more throughout the world [46].

A range of challenges arises concerning single-use models in the process of assessing and using model-based health economic evaluations. An important challenge concerns prioritization of different health care interventions when their cost effectiveness is assessed by different models. The initiative for development of health economic decision models in most settings rests with applicants for reimbursement and hence varies for each application. With different initiators, models for different treatments for the same disorder may show little consistency concerning model choices. For instance, models may differ in terms of model states, model assumptions or input data used. Furthermore, single-use models developed by applicants are often proprietary and may lack transparency [47, 48] and offer little insight into model validity. Single-use models often disregard or incorrectly model adverse events and comorbidities. Finally, by their very nature, single-use models tend to ignore interconnections between treatments. For instance, if a medicine is added to first-line treatment, this may affect the results of second- and third-line treatments [1,2,3].

Further organizational challenges in HTA processes exist; many settings involve stakeholders relatively late in the process. Model developers are commissioned and paid by applicants, who have a stake in reimbursement. HTA agencies or other policy makers get the reactive role to review and judge the developed models.

Single-use models may lack the flexibility to allow for structural adaptations. Time and resource limitations play a role for all parties involved. As a result, single-use models often come with only a limited uncertainty analysis and model validation, which seems reasonable given their single use, but is a waste of resources when more than one intervention is being considered for the same disorder, even if not immediately, but in the near future [48]. Another consistency issue arises when budget impact studies or appropriate care evaluations are conducted separately from the cost-effectiveness study, and are not necessarily using the same model and assumptions.

Multi-use disease models (MUDMs) could help address these challenges and support more structured use of disease models for various purposes, including health economic evaluations. The basic idea is to model a disease rather than a decision problem and use the same model for evaluating potentially all interventions treating the disease in question. The aims of this paper are therefore to clarify the terminology and define the concept of MUDMs and to identify and prioritize issues that should be solved before multi-use models can be implemented as part of actual coverage decisions. This works towards a blueprint supporting HTA agencies and other stakeholders in deliberate choices regarding the application of MUDMs.

2 Methods

2.1 Literature Review

A pragmatic desk literature review was performed during May 2019 to August 2019 to identify existing knowledge and previous attempts in defining the concept of MUDMs and developing such models. The review led to a first inventory of potential applications and methodological issues concerning MUDM.

2.2 Expert Panel Survey

A modified Delphi process was designed involving a panel of modeling experts, a group of external advisors (they were not part of the expert panel), our research team and policy experts from an HTA agency. Our research team consisted of modeling experts from five Dutch academic and institutional HTA groups. The team collaborated closely with the policy advisors from the National Health Care Institute (ZIN) who commissioned this study. Our external advisors are listed in Supplement 1 (see electronic supplementary material [ESM]) and are renowned international health economic decision modeling experts. The modified Delphi panel approach is summarized in Fig. 1.

Fig. 1
figure 1

Process to develop the Blueprint for Multi-use Disease Models. (White boxes display work by the research team; light blue box display input from ZIN and project advisors; dark blue boxes display input from our expert panel.)

Full size image

Purposive sampling was used to recruit a panel of experts from academia, the pharmaceutical industry, consultancy, and policy makers and civil servants. The expert panel included all active participants of the AdViSHE [4] panel and added participants with modelling knowledge from the research team’s network. Team members distributed invitations to increase response, adding relevant contacts from their personal network to the contact list, and contacting them personally.

Based on discussion with the team, the topics of the first-round survey were determined as (i) definition and terminology; (ii) applications; and (iii) organizational and methodological concerns. These served as the basis for formulating the questions in the first round of the expert panel survey. The exact formulation was the result of four rounds of revision by the team members. The prefinal round one questionnaire was then sent to our external advisors for comments, which led to the final questionnaire as sent to our expert panel. The first-round survey was sent out on 25th September 2019 and closed on 19th October 2019.

Responses from Round 1 (Supplement 2, see ESM) were summarized, processed by the research team, and then fed back to the panel in Round 2, with the second round serving to comment on results from the first round, to generate consensus on terminology and definition and to prioritize potential applications and issues to be addressed (Supplement 2, see ESM).

To construct the Round 2 questionnaire, the research team met with the commissioning experts from ZIN in a consensus meeting on 13th December 2019, in Utrecht, to discuss the results from Round 1. To reduce the length of the results, team members prepared this meeting by grouping similar terms and comments for each element to be discussed. In a session with all team members, consensus was reached on these groupings and the formulation of the second survey questions. A prefinal version was then sent out for comment to the external advisors, leading to the final round-2 questionnaire as distributed. The second-round survey was sent out on 27th January 2020 and closed on 1st April 2020.

2.2.1 Terminology and Definition

The term ‘disease-specific model’ and its definition were initially proposed by the team and shown to the participants in Round 1. The Round 1 survey asked the participants to comment on the term, suggest alternatives, and discuss the proper definition.

Furthermore, the Round 1 questionnaire presented to the panel participants a list of 10 elements that were considered essential to characterize an MUDM based on a discussion of the results of the desk review by the project team. Panel members were asked to comment on this list, indicate which elements they considered most important and add any elements they missed. The percentage of respondents considering an element as important was calculated for each element. This resulted in the essential characteristics identified by the panel, which supported formulating the novel definition of MUMD for Round 2.

In the Round 2 questionnaire, a new term (‘multi-use disease model’) was suggested for approval by the panel, based on Round 1 results. Furthermore, a revised definition of MUDM was proposed to the panel members, based on elements that were present in the definitions suggested by panel members in Round 1.

2.2.2 Potential Applications of Multi-Use Disease Models (MUDMs)

In Round 1, the panel members were asked to append and comment on a list of possible applications (for instance, resource allocation, budget impact estimation, guideline development, epidemiological projections, policy evaluation) where MUDM could be relevant. This initial list had been drafted by the research team based on the desk top review and did not aim for completeness.

Based on Round 1 results, a gross list was made listing all applications identified by the panel members. In a research team meeting that also hosted HTA agency representatives (Fig. 1, step 4), the research team grouped the applications from the gross list. Then they removed applications that were considered infeasible, that is, that would be impossible to realize even with a large amount of resources and time. During this same meeting, applications that were deemed irrelevant for an HTA agency in its consideration of applying MUDMs for health technology assessment were discarded to further reduce the list of potential applications. This led to a shortlist of ten potential applications. Participants in Round 2 were asked to select a maximum of five most important applications from the remaining list (see Supplement 2 in the ESM) and rank them in order of importance (from 1 to 5, with 5 indicating highest priority).

2.2.3 Potential Issues and Challenges of MUDMs

In Round 1, the panel members were asked to comment on the potential issues and add any relevant issues (Supplement 2). The research team then condensed and structured panel responses in step 4.

The resulting new list of issues was provided to the panel members in Round 2. In a table, we asked the experts to score the issues for relevance (highly relevant, moderately important, not important) and feasibility (not possible, ambitious, certainly doable). To reduce the workload for each panel member, a multifactorial design was employed so that each panel member only had to score a subset of all these issues. To each expert, 7 issues were assigned, ensuring all issues were scored by at least 8 panel members. Experts could still score all 32 issues if they wished to do so. The resulting scores were re-scaled to have a total score of 10N for each expert, where N is the number of issues scored by this expert, so that the average score of all issues was 10 for every expert.

Similarly, in Round 2 a list of recommendations as suggested by panel members in the first round as part of their comments on the challenges presented was provided. Participants were asked to give their opinion on the acceptability (highly desirable, desirable, acceptable, and unacceptable) of these. Again, to reduce the workload, each panel member only had to rate a subset of the full list of recommendations.

2.3 Final Steps Towards a Blueprint

After Round 2, in step 8 (Fig. 1), the research team discussed the summarized findings and used these to draft the pre-final results. This included (i) the choice of a term and definition for MUDM; (ii) a list of priority applications and (iii) a list of priority challenges and recommendations. The priorities chosen were then discussed with ZIN (step 9), comparing priorities from the expert panel with ZIN priorities. Issues were grouped and then re-ordered based on the following three criteria: (i) the need for further research; (ii) policy relevance/need for policy decision; (iii) acceptability of policy decision/expected differences among stakeholders. This resulted in a final overview of applications, challenges and recommendations, to become part of a blueprint for MUDM application.

3 Results

3.1 Findings from the Literature Review

Several authors have addressed ideas similar to that of MUDMs with different names, for instance ‘whole disease models’ by Tappenden et al. [2], ‘reference models’ by Afzali et al. and Frederix et al. [5,6,7], ‘policy model’ by Weinstein et al. [8] and the ‘treatment pathways models’ in Lord’s MAPGuide project [9]. Finally, ‘generic models’ were coined by Snyder et al. [10].

Applied examples of MUDMs [11,12,13,14] did not always explicitly address them as such. In more complex disorders, the efforts required for a proper disease model supported its repeated use, for instance in diabetes mellitus, chronic obstructive pulmonary disease and oncology.

In conclusion, the idea of MUDM has been around, but many different terms have been used, and direct applications in health technology assessment were relatively scarce, with the exception of the above-mentioned examples.

Notably, in several fields outside reimbursement decisions, for instance in the field of health impact models [15,16,17,18,19], multi-use has been the standard. Examples are the RIVM Chronic Disease model [20], the Prevent model [21], the PopMod model [22], the DYNAMO-HIA model [23], the ECONda tool and the UKHF microsimulation model [24]. These models are not MUDMs, since a typical health impact model covers multiple diseases and several do not include economic outcomes.

Being suitable for multiple use implies a number of requirements. These varied in previous studies. Tappenden et al. [2] listed the following requirements for ‘whole disease models’: (i) “The model boundary and breadth should capture all relevant aspects of the disease and its treatment—from preclinical disease through to death.”; (ii) “the model should be developed such that the decision node is conceptually transferable across the model.”; (iii) “The costs and consequences of service elements should be structurally related.” All of these ensure that the model is truly system level and allows for the full comparison and evaluation of interventions across the entire disease pathway, from primary prevention to palliative care [2]. Afzali and Karnon [5] introduced the concept of reference models and state that they need to be “subject to a comprehensive development process by an independent team of investigators taking input from a wide range of stakeholders.” Also, “The resulting models could incorporate a comprehensive, unbiased representation of the disease, and be able to evaluate a wide set of interventions within a particular disease area.” Finally, “Models could be populated using best available evidence, and validated.” Regarding maintenance, they stated “Over time, the model might require some updating, with respect to both structure and data inputs, which could be undertaken by sponsors, but following the approach specified in the original model development process. Deviations from this process would require full justification.”

3.2 Participants in the Expert Panel Surveys

In the first round of the expert panel survey (September/October 2019), a total of 102 questionnaires were sent out by email, and after sending two reminder emails, 51 responses were received, with Supplement 2 showing response rates per subtopic (see ESM). The response rate for Round 1 was 50% (51/102). In the second round, 61 questionnaires were sent out, since we did not approach persons who in Round 1 were clearly not willing or able to respond (those who replied to us indicating they are not willing to participate in the survey or who did not reply after two reminder emails). After sending two reminder emails, 42 responses were received (69%). Relatively few respondents worked in industry, while people from academia, consultancy, and policy were more prominently represented (Table 1).

Table 1 Background information of expert panel members in Round 1 and Round 2 surveys
Full size table

3.3 Results from Surveys

3.3.1 Terminology and Definition

The panel mentioned seven alternatives for our first proposal for terminology, which was ‘disease-specific model’, and the research team in step 4 proposed two more alternatives. The term ‘multi-use disease model’ was then used in the second-round panel survey and panelists were asked to approve this. This term was approved in Round 2 by 83% (35/42) of the respondents. For those who disagreed (7/42, 17%), the concerns were mainly focused on the word ‘multi-use’. It was brought forward that ‘multi-use’ itself might be confusing, since it can refer to several ‘multi’ things, for example times, purposes, diseases, treatments, countries.

The panel also suggested a large number of edits for our definition of disease-specific model/MUDM in Round 1. The definition of an MUDM was based on an elaborate analysis of Round 1 panel answers. A matrix was set up, structuring all elements that were included in definitions offered by panel members. Next, similar concepts were grouped together and concepts that received support from multiple panel members were considered for inclusion in the new definition. Additionally, the list of applications as indicated by the panel was used to check whether the definition would cover these application. A final definition was agreed upon in the consensus meeting of December 2019 and submitted to the panel in the Round 2 survey.

The revised definition of a ‘multi-use disease model’ was as follows: “A health economic decision model that properly represents the length and dynamics of a disease trajectory to accommodate the evaluation of a range of current and future health care interventions. It enables projections of policy scenarios, based on setting specific epidemiological parameters. When several disease stages are included, consistent comparisons over these stages are possible. This enables its repeated use, possibly after adaptations, for health economic evaluations and to support evidence-based health care policy regarding a certain condition.” The definition was subsequently approved by most (35/42, 83%) panel members in Round 2.

3.3.2 Characteristics of an MUDM

Panel scores for the characteristics of an MUDM are shown in Supplement 3, Supplementary Fig. 1 (see ESM). Two elements were considered important by > 60% of the respondents, namely covering a wide range of interventions and being suitable for repeated use. Two further elements were considered important by 50% of respondents: being able to produce policy projections, and estimates that are consistent over different disease stages. Based on these results, the use of the term ‘multi-use’, and the elements of projections and the inclusion of a wide range of interventions in our definition and terminology is supported by the panel. Note, however, that a wide range of interventions does not necessarily imply that a large part of the disease course needs to be covered, although often this will be the case.

3.3.3 Potential Applications

The extended list of potential applications covered several intended uses, including support of reimbursement decisions, support of clinical guideline drafting, and evaluations of current health care policy. Supplement 3, Table 2 lists the applications along with the team judgement concerning applicability for HTA agencies (see ESM). Figure 2 presents priorities based on the second round. The top two potential applications were comparing alternative policies and supporting resource allocation decisions, which received average scores of around 2.5. Three further potential applications had average scores around 2: budget impact estimation, guideline development, and identification of key uncertainties and their potential impact. On the bottom end, equity analyses and umbrella trials were rated as less relevant applications of MUDMs.

Fig. 2
figure 2

Potential applications for multi-use disease models along with their scores and ranks

Full size image

3.3.4 Issues and Recommendations

Results from Round 1 as they were summarized in the Round 2 survey can be found in Supplement 2 (see ESM). Issues mentioned varied in level of detail and recommendations in degree of concreteness. Figure 3 shows Round 2 scores for issues, with issues receiving an average score above 10, which is the expected average score of all issues, being considered as important. The most important issues were How to ensure sufficient transparency of model structure, assumptions and input data, Role of stakeholders, Transferability and Model complexity (Fig. 3).

Fig. 3
figure 3

Average score of issues for multi-use disease models (Round 2 expert panel)

Full size image

Figure 4 summarizes recommendations reflecting panel member opinions on features of MUDMs and operationalization of use. Two important findings were that the experts advised against models only being used by their developers, and against excluding the healthy population. Large support (≥ 50% of respondents scoring highly desirable) was expressed for regular updates (>80%), revalidation after updates (> 80%), proper storage of results, strong validation requirements, including time trends, FAIRFootnote 1 [25]/transparent modelling, including subgroups and heterogeneity, and independent model owners. From this it can be concluded that the expert panel in their definition of MUDMs tended towards more extensive models (including healthy population, strong validation requirements, regular updates, FAIR), and public ownership.

Fig. 4
figure 4

Recommendations on applying multi-use models (Round 2 expert panel)

Full size image

3.4 Prioritization of Issues to Address by ZIN

The initial 32 issues and recommendations sent to experts in Round 2 (the left-most and right-most columns in Fig. 5) were discussed with ZIN. The separate prioritization by ZIN mostly confirmed the priorities as selected by the panel regarding issues and recommendations for the development and use of MUDMs (see Supplement 4 in the ESM), but was somewhat more pronounced. That is, panel members had more priorities than the ZIN experts. Next, the research team in an online meeting merged similar items and/or moved items to a more appropriate category. As a result, 20 topics and one overarching topic (inner columns in Fig. 5) were derived from the original 32 items. This result was also discussed with and agreed upon by the ZIN experts. The categorization of ‘organizational’ and ‘methodological’ issues was further divided into seven categories: organization, development of model, input data, validation and transparency, model use, model results and model maintenance. The colours in Fig. 5 represent the category of each item and topic.

Fig. 5
figure 5

Overview of issues and re-arrangements made by the research team and ZIN, priorities indicated in dark colours

Full size image

4 Discussion

This study offers a starting point towards further development and application of MUDMs in the form of a clear definition, a list of potential applications, and an overview of related issues and challenges of MUDMs using input from a large group (N = 54) of international HTA modelling experts. Results were validated using an independent advisory board of academic experts and HTA agency representatives.

MUDMs can be defined as a health economic decision model that can be repeatedly used for a certain disease condition, to accommodate the evaluation of a range of health care interventions over several disease stages. While a number of challenges and issues remain to be solved, such models have many promising potential applications. The most important of these challenges were transparency and stakeholder involvement, while high priority recommendations on handling these issues included again transparency, but also regular updates and the model’s ability to account for time trends. Model ownership organization and proper choice of the level of complexity were other relevant issues highlighted by the panel as well as the project’s advisory board.

Several previous studies have addressed the definition and terminology related to MUDMs [2, 3, 5,6,7, 10]. Tappenden et al.’s [2] definition of a ‘whole disease model’ differed from the current one by its explicitly very wide scope and by the requirement of consistency throughout. That is, whole disease models should be suitable for the health economic evaluation of interventions for prevention, diagnosis and treatment across the whole disease pathway. As such, a ‘whole disease model’ is the most complete implementation of the idea of a MUDM. The disadvantage may be lack of feasibility, due to its very stringent requirements. Afzali et al. [5, 6] defined ‘reference model’, or ‘disease-specific reference model’ as a model that should represent “the knowledge and uncertainty about states/events relating to the disease progression on the basis of the best available evidence.” It is to be applied to a wide set of interventions for a specific disease. Compared with our current definition, reference models seem to require a certain ‘gold standard’ status, which is left open for MUDM. That is, a reference model is seen as the best possible model; in contrast, MUDMs do not necessarily claim this. As an example, in diabetes, more than 10 different MUDMs exist [26].

While the panel indicated explicitly that they also wanted to include models that did not cover the complete disease pathway, many of their recommendations and choices and especially suggested applications would require a model that covers a large part, if not all, of the disease pathway. The highest scores for applications of MUDMs were given to “comparing alternative policies in prevention and treatment” and “resource allocation over the entire disease pathway of interest”. This points at the most important challenge for developers of an MUDM, namely to choose a scope for the model that balances feasibility with applicability.

Insight and scientific evidence on diseases develop over time, while epidemiology and other model inputs may change. Therefore, regular maintenance is crucial for MUDMs, as also indicated by panel results. Maintenance and options for updates have to be integrated into the model development right from the start, for instance by using a modular model structure. This has been previously underlined, among others, by the ISPOR-SMDM Modeling Good Research Practices Task Force [27].

In contrast to the publications on reference models [6, 7], the panel did not clearly advocate using a single model as the gold standard, and some panel members warned of a potential lack of insight into structural uncertainty that may come with such a reference model. However, in itself, a reference model does not prohibit use of alternative model structures alongside the reference model analysis [7].

Finally, most previous studies on MUDMs [2, 3, 6] paid little attention to more practical and organizational issues like model ownership, maintenance and access for external users, while the current study shows that about half of the issues identified fall into these categories. To enable useful policy advice regarding the application of MUDMs, possible solutions for the methodological and organizational issues that were identified should be investigated.

It may appear that MUDMs bring a lot of challenges from this final overview. However, a very important advantage is the reduction of the inefficiency involved in repeated development and validation of new single-use models. When properly implemented, MUDMs could benefit several stakeholders. HTA agencies and assessors would benefit from improved consistency and transparency of model-based economic evaluations, while model developers could also benefit, when they do not need to develop a model from scratch, but could start with an existing MUDM and tailor it to their needs.

In addition to showing these advantages, the implementation of MUDMs will reduce the amount of model review time needed by HTA agencies and external reviewers during assessments of specific treatments. Additionally, MUDMs are typically available to HTA agencies, in contrast to single purpose models which are often built under commission of applicants. Therefore, usage of MUDMs could broaden the scope of treatments that may be evaluated by HTAs by allowing more opportunities to the HTA agency to perform independent evaluations. Furthermore, MUDMs will enhance consistency in the model-based evaluations to support (a broader range of) decisions within disease areas and potentially improve validity of model results, with models being more elaborately tested and used.

The current study focused on the definition and potential applications of MUDMs as well as the identification of challenges and recommendations. How to practically develop MUDMs was not addressed here. Several of the challenges identified by the panel indeed concerned topics that concern these practical matters, for instance the funding of model development, their ownership and their maintenance (see Fig. 5). In our project, the research team used the findings from the panel surveys to develop five business cases based on different choices regarding model ownership as potential blueprints for the implementation of MUDMs [28]. This, however, was not part of the current study, which only reported on the panel consultation.

Our study has several limitations. First, the panel was asked to reflect on largely theoretical issues. Not having to actually implement or develop an MUDM, or being confronted with an actual application, implies that some panel recommendations may lack practical relevance. We tried to reduce this risk by using a second round of feedback to the panel, and by having ZIN experts and an advisory board of three independent experts comment on the results. Nevertheless, the results serve as a starting point rather than a final blueprint for the application of multi-use models.

Second, we were able to recruit only a limited number of participants from industry to participate in the panel. More participants were approached, but these often declined based on a stated lack of experience in health economic decision modelling. Industry often hires consultancy firms to develop health economic decision models and the panel did contain a number of participants affiliated with a consultancy.

Third, our findings should be interpreted with care, in that we cannot claim that our panel was representative of all stakeholders in health economic decision modelling. The group of active participants may over-represent people with a prior interest in the topic of MUDMs. This may have an impact on the priority setting in the Round 2 survey, for example the priority scores for applications, issues and recommendations.

Fourth, the research team and ZIN were both from the Netherlands. In particular, the final prioritization of issues was influenced by input from ZIN experts. Also, the transferability of MUDMs to other countries was not extensively assessed and discussed. That is, the typical MUDM foreseen in the current project was a locally adapted model, using local input data where appropriate. Yet, the findings of the current study concerning MUDMs would also be relevant outside of the Netherlands. The expert panel was international, and the results presented in the current study therefore reflect the opinions of these international experts. Our external advisors were international as well. Furthermore, similar work on MUDMs is ongoing in UK and Canada, which indicates the relevance of MUDMs outside of the Dutch setting.

Strengths of the current research are that we did use a broad panel of experts, so that the findings reflect the insights of people from academia, decision makers and industry representatives, including consultancy. Another strength is the systematic approach, using expert panel input for a clear definition of concepts and terminology, as well as a two-step Delphi-like procedure to derive an inventory of challenges and recommendations. The expert panel members were asked to also reflect on practical and organizational aspects, which were further commented on by employees from the Dutch HTA agency.

This study was commissioned by ZIN, the Dutch HTA agency, as part of a larger project investigating the possibilities, advantages and disadvantages of MUDM for ZIN. The research team subscribed to a competitive tender with a project proposal. During the resulting project, the research team had regular contact with ZIN experts to keep them informed on project progress. Discussion of Round 1 panel results took place in a consensus meeting of the research team and ZIN experts together. However, Round 2 results were first processed by the research team, after which the ZIN experts provided their input, which led to a combined final set of results. The latter allows for better insights into the role of the HTA agency experts (Supplement 4, see ESM). Results of this project were intended to support future steps by the HTA agency. Therefore, it was important to closely collaborate with the ZIN experts throughout the entire project and one of them is a co-author of this publication. However, the current paper reflects the opinion of the co-authors and not necessarily the official policy of ZIN.

In our view, MUDMs may have varying levels of comprehensiveness. For some applications, it may be desirable and feasible to develop an extensive model with multiple decision points, a broad range of outcomes and costs (health and other), and all parameter estimates available (perhaps even linked to patient registries for regular updates). For other applications, a less extensive model may be suitable, or even a set of mandatory model components (for example concerning costing parameters, the core risk engine for a patient level model, or the most important model states and their care as usual transitions for a state-transition model). Such partial solutions may help to gradually introduce multi-use disease modelling in an efficient and feasible way and overcome issues with inconsistencies across assessments and with technical validation. Developing model code in a structured and modular way will further support this [29].

Ultimately, MUDMs will improve consistency among coverage decisions for various treatments in the same disease area. In addition, they will enable engagement in an overall evaluation of several treatments for one disease. Examples of the latter can already be found in the multiple technology appraisals at NICE, for instance when comparing different modalities for glucose monitoring in type 1 diabetes [30]. Other agencies may apply MUDMs in support of policy making concerning management of infectious diseases, public health policy aiming at prevention through a healthy lifestyle, and population screening programs. Examples are the vaccination policy advice documents by the Dutch health council which apply MUDMs of infectious diseases [31, 32]; several COVID-19 epidemiology models which can also be considered MUDMs [33, 34]; evaluations of tobacco control [35, 36], alcohol misuse [37, 38] and overweight policy [39] using health impact models, and evaluations of cost effectiveness of colon cancer screening using extensive colon cancer models [40,41,42,43].

As a logical extension, the support of clinical guideline development is a very attractive application of MUDMs. Several authors have argued for better integration of cost-effectiveness information in clinical guidelines [44, 45]. In some jurisdictions such as in England, cost effectiveness is integrated into clinical guideline development. In that case, MUDMs have a clear role to support clinical guideline development. Also, in settings that do not consider cost effectiveness as an explicit criterion in clinical guidelines, MUDMs have a potential role in guideline development, since they offer a consistent framework to evaluate implications for capacity requirements or resource use of choices advised by clinical guidelines. While this was not an explicit element in our panel survey, the potential application ranked 4th in our panel and future research could pay more attention to the specific requirements that application for guideline development may bring.

Finally, MUDMs can be applied in budget impact analyses, but this brings an extra requirement for the model that its model population reflects the total patient population under consideration, that is, sufficient information on disease incidence and prevalence, and on population characteristics is included. This is another area for future research, with close links to existing public health models.

5 Conclusion

The introduction of MUDMs offers several potential advantages over the use of de-novo models for each application, both organizational (consistency of decision support, initiative for HTA agency) and methodological (increased transparency and validity of the health economic decision models applied). However, challenges exist concerning development, maintenance and access. The current paper provides a clear definition as well as a list of possible criteria that may serve as a starting point for groups interested in either developing or using an MUDM. Furthermore, the study helps to support the implementation of MUDMs by identifying and discussing a range of methodological issues associated with their development, maintenance and use. Once solved satisfactorily, MUDMs may put the HTA agency in the lead. When the agency initiates the development or commissioning of MUDMs, they can require the use of the MUDM for reimbursement applications. When they also apply the same models in analyses that support other policies, for instance as part of guideline development, this may enhance the consistency of methodologies used across different policy decisions.