Reimbursement decisions and price negotiation of healthcare interventions often rely on health economic model results. Such decisions affect resource allocation, patient outcomes and future healthcare choices. To ensure optimal decisions, assessing the validity of health economic models may be crucial. Validation involves much more than identifying (and hopefully correcting) errors in the model implementation. It also includes assessing the conceptual validity of the model and validation of the model input data, and checking whether the model’s predictions align sufficiently well with real-world data [1, 2]. In the context of health economics, validation can be defined as “the act of evaluating whether a model is a proper and sufficient representation of the system it is intended to represent in view of an application” [3], meaning that the model complies with what is known about the system and its outcomes provide a robust basis for decision making.

In recent years, recognition of the importance of validation as a fundamental step in the modelling process seems to arise among researchers and decision makers [1, 2, 4,5,6]. Despite this, validation efforts on health economic models remain unreported. A quick PubMed search for “cost effectiveness” and “model” returned 1126 hits, but when “validation” was added, it dropped to 27 hits (2.4%). This contrasts with searches for “sensitivity analysis” (48%) and “uncertainty” (18%). Recent reviews in cost effectiveness, with a focus on the models used, point out that validation is a missing element in many model-based studies [7, 8]. Even though model developers will probably have validated their models, this lack of reporting might raise questions about the rigor of model validation in daily practice. It is difficult to assess whether health economic model validation is unreported or unperformed [9], given that validation efforts are not reported in detail and that models that were subject to extensive validation (in theory) are still found to have errors [10]. Guidance and reporting guidelines exist, but their implementation is lagging behind.

Terminology in health economic model validation can often be confusing because of different interpretations and a lack of clear definitions. The term “internal validation” can be used to describe the act of comparing model outcomes to empirical data that were used to build the model [11,12,13,14]. The same definition is referred to as “dependent validation” elsewhere [15, 16]. However, other studies use “internal validation” to refer to model “verification” [1, 17, 18], or even to double programming [19]. In line with the first use of “internal validation” previously mentioned, the concept of “external validation” requires comparing model outcomes to empirical data that were not used to build the model [11,12,13, 18]. However, the same definition is referred to as “independent validation” as well [15, 16], whereas “independent validation” has also been employed to indicate validation undertaken by a third party [20]. “External validation” has been used for the comparison between model outcomes against outcomes produced by other models [14], but this is often referred to as “cross-validation” [1, 15, 21]. Finally, some publications include a sensitivity analysis as part of model validation [22, 23], although a sensitivity analysis aims to explore uncertainty, not validate models: a model full of errors may still produce robust results. These are just a few examples to highlight that efforts to establish standardised definitions and guidelines are essential for clarity and consistency in the field.

In addition to this lack of standardisation in the terminology used, the disparity regarding validation requirements among different guidelines around the world does not help model developers when deciding if, and to what extent, validation efforts should be reported. Many guidelines still address the subject of model validation in a brief and unspecific way, or not at all. Exceptions to this are for example the Dutch or the Australian pharmacoeconomic guidelines [6, 24]. We believe that this should be a subject of concern and debate, especially because reporting and good practice guidance is available. The report of the International Society Modeling Good Research Practices Task Force-7 in 2012 established a framework for conducting validation of health economic models [1]. The Assessment of the Validation Status of Health-Economic decision models (AdViSHE) tool was developed in 2016 for the purpose of documenting validation efforts and offered a selection of items to balance feasibility and rigor [15]. We believe these are the minimum validation standards that all model-based studies should meet. Since then, several validation-specific tools have been developed to provide structured approaches to different aspects of model validation [25,26,27,28]. Additionally, some studies have addressed the challenge of metrics: how to judge results of validation tests and when it is good enough [29, 30]. As a result, model developers (and users such as heath technology assessment agencies) are, in principle, well equipped to conduct validations of health economic models, report and interpret their results. Regarding health economic decision models in general, organisations such as the International Society for Pharmacoeconomics and Outcomes Research and the Society for Medical Decision Making have developed and refined through the years guidelines and reporting standards [1, 31,32,33,34,35,36,37]. Many scientific journals in the field of health economics require authors to adhere to specific reporting guidelines, such as the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist [38]. These standards help ensure that health economic evaluations are reported transparently in research articles. Given the available tools, it is strongly recommended to encourage similar standards for model validations. This would entail researchers, public institutions or private health stakeholders to provide sufficiently detailed descriptions of their validation tests, and their results, in publications as well.

Furthermore, notable changes have been occurring in the field of health economic modelling, and data science in general, in recent years [39]. The development of more complex methods (e.g., in the analysis of survival data or evidence synthesis) [40,41,42], a change in modelling habits (e.g., the use of R in economic modelling) [43,44,45,46], the increasing use of open-source models [47], real-world data [48] and artificial intelligence/machine learning methods are a few examples of these changes [49]. Health economic models are growing in complexity, owing to for example the importance of personalised medicine [50, 51], advanced therapeutic medicinal products [52], modelling of vaccines and immunisation frameworks [53], and the increasing interest for multiple-use models such as whole disease or pathway models [54,55,56,57,58,59,60,61]. Complex models require more extensive validation efforts than straightforward cohort-level state-transition models or decision trees to ensure their accuracy and reliability. The use of open-source modelling software has increased, promoting transparency and collaboration among different stakeholders [44, 47]. However, this does not replace the need for model validation. Integrating real-world data in health economic models has to be combined with adequate validation of model outcomes against such data, for instance by replicating observed data [62]. We believe all these changes, together with a general lack of reporting, call for updated guidance on model validation.

Model validation can play a crucial role in healthcare decision making, at least in theory. Health economic models can be complex, often relying on intricate assumptions and multiple data sources. Ensuring that these models meet high-quality standards is vital to their utility and credibility [2]. This can be achieved by systematically checking all elements of model validation: validating the conceptual model, verifying data sources, testing the plausibility of modelling assumptions, conducting extensive model verification, comparing model outcomes to independent real-world data when available, and addressing potential conflicts of interest when experts are involved in any aspect of model validation. One way to enhance this is through standardisation. As part of this, consistent use of terminology is helpful. The development of guidelines and best practices for health economic model validation can help ensure that critical steps are not overlooked and that models consistently adhere to standards. Furthermore, transparency and credibility should be cornerstones of the validation process, making detailed documentation of validation efforts available. More consensus in the field, or clarity by model users, is needed on whether independent third parties should conduct validations, or it is sufficient when model developers clearly report their validation steps and results. The use of external independent validators without conflicts of interest, such as the National Institute for Health and Care Excellence External Assessment Groups, can provide an impartial assessment of the model’s strengths and weaknesses, helping to identify potential pitfalls and areas for improvement, enhancing not only the credibility of the model but also improving the transparency of the decision-making process. Nevertheless, such procedures are costly and do not solve all issues raised previously.

Validation of health economic models should be seen as a critical component of evidence-based decision making in healthcare. However, as of today, it still faces several important challenges, including the lack of consensus guidance and standardised procedures, the need for greater rigour or the question of who should oversee the validation process. To address these challenges, we encourage model developers, agencies requiring models for their decision making and editors of journals that publish models to recommend the use of state-of-the-art tools for reporting (and conducting) validations of health economic models, such as those mentioned in this editorial.