FormalPara Key Points for Decision Makers

Differences in definitions of disease severity (e.g. levels of respiratory support) and long-term sequelae reflect the methodological variation across decision–analytic models used in the economic evaluation of COVID-19 drug treatments.

Modelling methods in the field of COVID-19 are still in the early stages of development, leaving substantial room for improvement.

Insight into key methodological choices made in decision–analytic models enhance the discussion on best modelling practices in the field of infectious diseases.

1 Introduction

In a world where resources are limited and healthcare costs keep rising, health technology assessment (HTA) becomes crucial to making informed decisions about the allocation of resources for health-related interventions. Even (and perhaps especially) in the case of a pandemic with a large impact on healthcare systems and economies as a whole, HTA is of vital importance to inform the decision making process on potential interventions. Despite the overwhelming and unprecedented threats of a pandemic, a comprehensive approach to assess the impact of an intervention is needed as of its initial stage. HTA can support decision makers on a comprehensive response to a public health emergency by not only addressing the immediate health impacts, such as the number of infections and critical care capacity, but also considering the broader societal consequences that are pivotal to determine the value of an intervention [1].

In the urgent situation of the coronavirus disease 2019 (COVID-19) pandemic, national authorities around the world had to respond quickly to mitigate the threatening consequences caused by the severe acute respiratory syndrome coronavirus (SARS-CoV-2). A wide diversity of public health measures was implemented and numerous therapeutic and prophylactic interventions were developed or repurposed to control symptoms and prevent disease progression. Furthermore, central emergency funds were expanded and regulatory processes were adapted to accelerate market access of promising interventions. The European Union (EU) strategy on COVID-19 therapeutics is a sound example of this. The strategy included an intensified use of rolling reviews and conditional marketing authorisation, and one of its first commitments was the joint procurement of the antiviral drug remdesivir for 36 European countries before full marketing authorisation [2,3,4]. In contrast to recent times beyond the pandemic, reimbursement decisions of COVID-19 treatments were largely made without a thorough HTA of clinical and cost-effectiveness aspects [5]. As the pandemic progressed and the level of vaccine-induced and natural immunity increased, healthcare systems returned to normal and central emergency funds were phased out. The relief of healthcare systems and the return to fixed healthcare budgets have once again shifted priorities towards an increased relevance of HTA [6].

Several trial- and model-based economic evaluations have been performed since clinical evidence of potential COVID-19 treatments was studied. Although many relied on data from large-scale clinical trials, evidence on quality of life (QoL) of patients with COVID-19 was almost absent, clinical pathways and treatment populations were heterogeneous and shifted over time as a consequence of the rapidly evolving nature of the pandemic and subsequent policy choices, and the proportion of patients suffering from long-term sequelae as well as its duration and impact were uncertain [7, 8]. The limited evidence on several inputs challenged health economists to provide informative output about the expected costs and consequences of interventions and to account for varying stages of disease and long-term outcomes in the context of COVID-19.

This study aimed to compare and summarise key methodological characteristics of model-based economic evaluations of COVID-19 drug treatments. Similar to the objective of the current study, another review identified the perspective, comparators, type of economic model, types of costs included, data sources and methods for estimating productivity costs in economic evaluations of interventions against viral pandemics, including COVID-19 [9]. However, only a small part of the included studies focussed on drug treatments. The current study specifically focussed on modelling choices which pertain to disease severity levels during hospitalisation, model structure, sources of treatment effectiveness and QoL, and long-term sequelae. Further, input parameters that exerted a high impact on the output values as well as frequently mentioned limitations were analysed. Ultimately, the outcome of the analysis will give direction to future decision–analytic modelling of COVID-19 and infectious diseases in general (whether in the context of a public health emergency), and address the need for valuation and measurement of certain input parameters that are key in health economic evaluations.

2 Methods

2.1 Literature Databases

We conducted a systematic literature review to identify original articles on full economic evaluations of COVID-19 drug treatments. The following key databases were searched: MEDLINE, EMBASE, Web of Science, Scopus, Econlit and the International Health Technology Assessment Database (International Network of Agencies for Health Technology Assessment). The search was last rerun on 22 July 2023.

2.2 Search Strategy

Manual searches for relevant studies were the starting point to develop a search string in PubMed. The search string captured all 17 studies that were found through manual searches. After this validation check, the search string was translated for application with other databases to search for fully published studies in peer-reviewed journals as primary research (not review or meta-analysis) in English language, performed in all countries. The scope of research was narrowed down to drug treatments against symptomatic COVID-19, both in and outside a hospital setting. This ruled out studies concerning vaccines, diagnostic techniques and non-pharmaceutical interventions (NPIs). The review did not impose any limits on dates, as studies pertained to the recently emerged context of COVID-19. Search strategies are provided in the supplementary material. The search protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO; CRD42023407646). This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

2.3 Selection Criteria

After removing duplicate articles, two reviewers (CV, ADIvA) independently screened titles and abstracts for initial inclusion. Full texts of included articles were screened for final inclusion. Disagreements between the two reviewers were discussed to achieve consensus. The number and reasons for inclusion or exclusion at each stage of the review were recorded and are presented in Fig. 1. Studies were included if both costs and health outcomes were compared (i.e., ‘full’ economic evaluation), regardless of the type of comparator intervention. Further, trial-based economic evaluations were excluded from the review, as the objective of the study was focussed on modelling choices made in economic evaluations of COVID-19 drug treatments. A trial-based economic evaluation is performed alongside a single randomized controlled trial (RCT) and does not primarily synthesise outputs of different sources by means of a decision–analytic model. Furthermore, cost-effectiveness analyses (CEAs) of hospital-level treatment strategies [e.g., intensive care unit (ICU) treatment for the management of patients with COVID-19] were excluded. This was because these types of interventions include more than just a drug treatment, which was the main focus of the study.

Fig. 1
figure 1

PRISMA flow diagram showing the study selection process in the systematic review of model-based economic evaluations of COVID-19 drug treatments

Full size image

2.4 Data Extraction and Synthesis

One reviewer (CV) extracted data from the included studies onto a data chart using Microsoft Excel, which was checked by the other reviewer (ADIvA). The content of the data chart was developed during the formal screening of search results against eligibility criteria. The extracted information included publication details, study country, study design, patient population, intervention(s) and comparator(s), model structure, healthcare setting, time horizon, perspective, sources of treatment effectiveness, sources of QoL data, cost types, patient subgroups, model validation, uncertainty measures, sensitivity analyses outcomes and reported limitations.

Studies were among others grouped by type of economic evaluation and model type. As a preliminary synthesis, extracted data were summarised in tabular form, using the above-mentioned aspects as table headings. Consequently, aspects that were relevant to the research questions were categorised into more broader components or rubrics, which enabled the exploration of differences and similarities across the studies. Disease severity levels during hospitalisation that were used in the model were extracted from the author’s text and model parameters. Any differentiation outside the scope of hospitalisation (e.g., outpatient care) was not included to gain a proper comparison across studies. Model structure was mainly retrieved from model visualisations. Long-term sequelae and sources of both treatment effectiveness and QoL were derived from the author’s text. Thematic analysis was used to identify recurrent themes regarding input parameters with a high impact on the results and limitations mentioned in the studies. Other aspects not directly related to the research questions though relevant to draw context were reported quantitatively (e.g. study country and study perspective).

3 Results

3.1 General Study Characteristics

Of the 1047 records identified through the search in electronic databases, 665 articles were screened on title and abstract, and 37 studies were selected for full-text screening, of which 12 studies were excluded for several reasons: 4 studies analysed the cost-effectiveness of a hospital-level treatment; rather than a patient-level treatment [10,11,12,13]; 1 study analysed the cost-effectiveness of a non-pharmaceutical treatment [14]; 5 studies did not use any decision–analytic modelling technique, of which 2 used a cost-calculator rather than an economic model [15, 16] and 3 based their analyses on trial results only [17,18,19]; 1 study analysed clinical and economic benefits though did not report a cost-effectiveness ratio and was not considered a full economic evaluation [20]; and finally, 1 study was not published in English language [21]. In addition to the full-text screened studies, two articles with identical first three authors and a similar model structure were found through cross-referencing. These studies were considered unique here because of substantial differences in modelling characteristics that are of interest to our study. One grey-literature report was included, identified through the article of Beinfeld et al., concerning a CEA on three treatments for outpatients with COVID-19, performed by the Institute for Clinical and Economic Review (ICER) organisation [22, 23]. Consequently, 27 studies met the inclusion criteria and were analysed according to the predefined methodology. The PRISMA flow diagram is presented in Fig. 1.

An overview of several study characteristics is presented in Table 1. Approximately half of included studies originated from the USA. Remdesivir was the most commonly studied intervention in model-based economic evaluations of COVID-19 drug treatments. The category with treatments other than those specified in the table involved treatments that appeared less than twice among selected studies. Time horizon of analyses differed substantially across studies and ranged from 1 month to a lifetime horizon. One study assumed a COVID-19 episode time horizon, though no further details about the length of the episode were reported [24]. The majority of analyses applied a healthcare system or payer perspective, meaning that only direct medical costs were included. Five out of seven studies that applied a societal perspective used a healthcare system or institutional perspective in the base case analysis. All seven studies with a societal perspective included productivity loss as the only indirect cost type in the analysis. One study did not report any sensitivity analysis, although the use of probability distributions was reported to account for potential variation in measured effectiveness [25]. All other studies applied and reported one or more uncertainty measures to assess the level of confidence regarding the results of the evaluation. Eight studies subdivided their patient population into different subgroups, whether or not as part of a scenario analysis. Depending on the study context, the level of subgroups varied from the requirement of oxygen support only to including age, sex, vaccination status, disease severity, risk of hospitalisation and comorbidities. One study subdivided patient populations at an even more detailed level, including multiple biomarkers [24]. A few studies discussed the validation of the model results against real-world data, for example, local epidemiological data and observational datasets [22, 26, 27]. In general, it was mentioned that validation of (long-term) outcomes could not be carried out due to paucity of real-world data at the time that the studies were conducted.

Table 1 Frequencies of general study characteristics of model-based economic evaluations of COVID-19 drug treatments
Full size table

3.2 Narrative Synthesis of Specific Study Characteristics

3.2.1 Disease Severity Levels During Hospitalisation

The majority of modelling studies (n = 23, 85.2%) applied several levels of disease severity among patients with COVID-19 during hospitalisation. Three types of differentiation were identified, namely: level of respiratory support, level of care management (i.e., ICU versus non-ICU) and symptom levels (mild, moderate, severe). Nine (39.1%) studies grouped on respiratory support [22, 28,29,30,31,32,33,34,35], six (26.1%) on level of care management [25, 27, 36,37,38,39], seven (30.4%) on the both respiratory support and level of care management [24, 26, 40,41,42,43,44] and one (4.3%) on symptoms [45]. Four studies did not apply any differentiation during hospitalisation (14.8%). The study of Sinha et al. analysed patients with severe COVID-19 only [46]. The study of Jo et al. differentiated the modelled population by underlying disease and age prior to hospitalisation, though no distinction between different hospitalisation compartments was made [47]. The study of Savinkina et al. analysed a treatment (nirmatrelvir/ritonavir) for individuals newly positive with COVID-19 in the outpatient setting. The decision tree included risk for severe COVID-19 and for hospitalisation, both determined by age and presence of comorbidities and vaccination status. However, no differentiation during hospitalisation was used (i.e. ‘hospitalised’ versus ‘not hospitalised’) [48]. Zhang et al. used two disease severity levels (severe, non-severe), though the stage of hospitalisation was not specified [49].

Respiratory support was subdivided into five categories: (I) no respiratory support; (II) low-flow oxygenation (LFO); (III) noninvasive ventilation (NIV) or high-flow oxygenation (HFO); (IV) mechanical ventilation (MV) or extracorporeal membrane oxygenation (ECMO); and (V) supplemental oxygen (as the unspecified combination of the second and third category). Differentiation of the first four types of respiratory support was most frequently applied. Two studies did not explicitly distinguish between LFO and HFO or NIV as the method of oxygen delivery (Fig. 2).

Fig. 2
figure 2

Levels of respiratory support incorporated in decision-analytic models for the economic evaluation of COVID-19 drug treatments. ECMO extracorporeal membrane oxygenation, HFO high-flow oxygen, LFO low-flow oxygen, MV mechanical ventilation, NIV non-invasive ventilation

Full size image

3.2.2 Model Types and Structures

A total of 16 studies used a Markov state-transition model, including 12 models that were preceded by either a decision tree, an algebraic model or an epidemiological model. Six studies used a decision tree to model patients during the acute phase, succeeded by a Markov model to simulate patients in the post-acute phase [22, 26, 29, 32, 41, 42]. Two studies used an algebraic model to calculate quality-adjusted life years (QALYs) and costs of patients during hospitalisation and post-discharge care, followed by a Markov model to estimate lifetime QALYs and costs after recovery [30, 35]. Three studies constructed an epidemiological model to simulate the pandemic course over a 20-week time horizon. These studies applied a contagion index based on historical data to estimate the number of COVID-19 infections, number of hospitalisations, ICU admissions and deaths. The output of the epidemiological model served as input for a Markov model, in which costs and consequences of the treatment were estimated [36, 37, 43]. Of the studies with a Markov model, two studies partly followed the conventional Susceptible, Exposed, Infected, and Recovered (SEIR) structure and included the S, I and R compartments as health stages in their Markov model [27, 45, 50]. Rattanavivapong et al. used a Markov model consisting of five health states (susceptible, infection, severe infection, recovered and death) [27]. Jiang et al. designed a Markov model following a susceptible—exposed—asymptomatic—presymptomatic—awaiting diagnosis—infected—recovered (SEAPWIR) structure [45]. Two studies by Jo et al. used a compartmental transmission model and extended the SEIR structure by multiple disease-related compartments (to a total of 12 and 13 compartments), to account for different patient profiles and treatment pathways [38, 47]. The compartmental transmission models simulated the course of the pandemic over the whole population (South Korea; South Africa) and estimated the number of people infected with COVID-19 and the treatment population. The model in the South Korean study included a compartment for the vaccinated susceptible population. The model in the South African study included five different compartments to represent different stages of hospitalisation. Both models distinguished between symptomatic and asymptomatic infections and allowed for recovery without hospitalisation for the symptomatic population. One study used a partitioned survival model and consisted of three mutually exclusive health states: hospitalised, discharged and death [31].

Six studies using a state-transition model included more than one hospitalisation state and allowed for transition between those health states. In the models designed by Chow et al. and Jiang et al., patients could shift from non-ICU to ICU and vice versa [25, 45]. The stochastic compartmental transmission model used by Jo et al. included a general ward compartment before and after ICU treatment [38]. The three models of Ruggeri et al. incorporated a transition from general ward to ICU, though a return from ICU to general ward was not possible. The model of Rattanavipapong et al. allowed for susceptibility to infection after being recovered, and the ICER report incorporated costs and disutility as a consequence of rehospitalisation for 9% of the population in the first Markov cycle [22, 27].

The post-acute phase was differently composed across models, ranging from a two-state model (‘alive–dead’) to the inclusion of a ‘rehospitalisation’ [41, 42] or ‘recovered with long-term sequelae’ health state [26].

3.2.3 Sources of Treatment Effectiveness

Six studies based their effectiveness estimates of the intervention on multiple clinical studies, of which five studies included one or more RCTs [28, 32, 33, 38, 41, 48]. Three studies based their effectiveness estimates on the results of a meta-analysis, including two meta-analyses that used results from RCTs [25, 39, 45]. Two studies based their data on a single cohort study [40, 44]. One study included results from another CEA and from two studies based on real-world data as a basis for the effectiveness parameters in the model [49]. All other studies based the effectiveness of the intervention on a single clinical trial.

3.2.4 Sources of Quality of Life

Of the 17 cost-utility analyses (CUAs), two used utility measures that were derived directly from patients suffering from COVID-19 [25, 27]. Both studies (USA and Thailand) based utility values on the outcome of an Iranian study. The study used time trade-off (TTO) questions to estimate the health utility value of patients with COVID-19, and utility was grouped by several factors, including but not limited to having an underlying disease and hospitalisation status (i.e. hospitalised at general wards, hospitalised at ICU without intubation and hospitalised at ICU with intubation) [51]. The CUA of Popping et al. used data from a Dutch cohort study and used QALYs from a study that was also performed under the Dutch population. The study estimated QALYs lost because of COVID-19 on the basis of excess mortality in COVID-19 waves [52].

Other CUAs used a variety of other studies reporting on utilities of patients without COVID-19. One study in the USA performed a de novo vignette-based utility study to derive health states pertinent to the natural history of COVID-19. Consequently, on the basis of the seven health states, an EQ-5D-5L questionnaire was completed by a sample of the United Kingdom general population as a proxy for patients with COVID-19, and utility values were transformed to the US value set [26]. The ICER report calculated a disutility for outpatient management from a COVID-19-specific utility study [53], whereas other (dis)utilities were based on measurements among patients who did not have COVID-19 [22].

3.2.5 Lifetime Models and Long-Term Sequelae

Of ten studies with a lifetime horizon, seven made adjustments to the general population estimates of mortality rates, health utilities and healthcare costs to account for long-term sequelae of patients recovered from COVID-19. Jovanoski et al. adjusted background mortality ratio, utility and healthcare costs for patients with moderate (temporary, lasting 1 year) or severe health issues (patient’s lifetime) after discharge [29]. Kelton et al. and Ohsfeldt et al. accounted for long-term sequelae for patients with severe comorbidities over the lifetime horizon by using multipliers for post-discharge mortality, utility and post-discharge costs. The proportion of patients with severe comorbidities was 34.2% in the study of Kelton et al. and 32.1% in the study of Ohsfeldt et al. Additionally, productivity loss was included as indirect non-medical cost in both analyses [30, 35]. In the analysis of Rafia et al., hospitalised patients were at an elevated risk of death (7.7 times higher compared with the general population [54]), had a reduced QoL (utility decrement of − 0.097) and increased costs (a one-off cost of monitoring patients with multi-organ dysfunctions) during the first year after entering the model. After the first year, adjustments in survival and QoL returned to an unadjusted background mortality and pre-COVID-19 QoL values [31]. The ICER report on oral drug treatments applied adjustments as was done in the analysis of Sheinson et al. [22, 32]. For patients discharged after mechanical ventilation, an increased mortality ratio relative to the general population at a hazard ratio of 1.33, a disutility of − 0.13 in the first year and − 0.04 in years 2–5, and an average annual personal healthcare costs of $7859 US dollars in the first year were applied. Additionally, both market and non-market productivity were incorporated.

The model structure of Goswami et al. included a health state for long-term sequelae, which was defined as ‘cardiovascular, pulmonary, and neurologic conditions or other general symptoms related to COVID-19 that remained up to 6 months after the initial onset of symptoms’. Rates of long-term sequelae were derived from a national database containing records of patients treated for COVID-19 at academic medical centres. The study applied a standardized mortality ratio of 1.5 to the background mortality rate for individuals with serious severe cardiovascular conditions or diabetes. Utility values originated from the EQ-5D-5L questionnaire, which was conducted alongside the economic evaluation, as described earlier. Costs of long-term sequelae management included outpatient visit costs and pharmacy costs for a period of 5 months following the 1 month of hospitalisation, as analysed in the decision tree [26].

Other studies with a lifetime horizon did not adjust all three types of inputs (mortality, QoL, costs) to account for long-term sequelae. Sinha et al. applied a QALY weight to adjust for QoL lost due to chronic lung disease, while mortality and annual health expenses of COVID-19 survivors were based on general population estimates [46]. The analysis of Rattanavipapong et al. estimated incremental QALYs from COVID-19 deaths averted over a lifetime horizon, though no long-term sequelae were taken into account [27]. The study of Whittington et al. used general population estimates for long-term sequelae, rather than quantifying ongoing cost or disutility associated with COVID-19 after hospital discharge. Several reasons were mentioned for this choice, including the lack of consensus on a standardised definition and duration of long-COVID and mixed estimates of the rate of long-COVID. Furthermore, long-term sequelae were thought to have no influence on the findings of the analysis [33].

3.2.6 Input Parameters with a High Impact on Results

A total of 22 studies performed deterministic sensitivity analyses (DSAs) to assess the sensitivity of the results to variations in one parameter or a set of parameters. Parameters related to the treatment effect of the intervention were most often reported as the input parameter having a high impact on the results (n = 18). Other parameters mentioned to a lesser extent were related to costs of the intervention or costs of hospitalisation (n = 10), risk of disease progression (n = 6) and the probability of a SARS-CoV-2 infection (n = 5). Other parameters were related to treatment effects of the comparator, healthcare costs after discharge or recovery, patient’s age, baseline clinical characteristics, inclusion of indirect costs and time horizon. One study reported no significant impact of parameters to the output values following a DSA [34].

3.2.7 Frequently Reported Limitations

Most commonly reported limitations were related to imprecise cost estimates for both resource use and drug costs (mentioned in 11 of 27 studies), a lack of (significant) evidence of treatment effectiveness (n = 10), the impact of the pandemic evolution and policy choices on different outcomes of the analysis (n = 10) and the impact of missing data of various input parameters or assumptions made on the outcome of the analysis (n = 10). Further, limited data on QoL data and the lack of COVID-19-specific utilities (n = 10) and the (partial) omission of adverse events or contraindications of the intervention (n = 8) were frequently reported. One study did not report any limitations [34].

4 Discussion

This systematic review identified 27 full economic evaluations using decision–analytic models to estimate the cost-effectiveness of drug treatments against COVID-19. The results illustrate the variety across studies with regard to the differentiation of inpatient clinical management, the structure of models and the width and extend of incorporating long-term sequelae.

4.1 Evidence Discussion

The initial pandemic response had varying practices for ICU admission and decisions regarding ventilatory support of patients with COVID-19 [55]. The treatment approaches in economic evaluations of COVID-19 treatments were mainly informed by clinical trials (either ongoing or completed) and were also influenced by assumptions. As described here, the majority of studies differentiated the population in the model at the stage of hospitalisation. Differentiating COVID-19 patient populations by level of respiratory support is pivotal to reflect actual clinical practice, as included studies showed substantial differences in length of stay, health outcomes and costs per method of oxygen delivery and treatment setting. However, methods of oxygen support were incorporated in various ways and were not always clearly defined. Some studies did not explicitly define whether supplemental oxygen therapy consisted of LFO, HFO/NIV or both, which may be a small but meaningful difference regarding clinical outcomes and associated costs [56, 57]. Other studies combined MV and ECMO as one level of disease severity on the basis of the differentiation used in the clinical trial of interest (e.g. the ACTT-1 trial on remdesivir) [58]. However, clinical outcomes and healthcare costs of both types of respiratory support may differ substantially. For instance, the need for dialysis was substantially higher in patients with COVID-19 receiving ECMO compared with patients with COVID-19 admitted to the ICU in general [59, 60]. To differentiate a certain disease or disease group by severity, standardized measures of patient illness are useful, of which the World Health Organisation (WHO) clinical progression scale is a good example [61].

The results illustrate the variety of model structures used in economic evaluations of treatments in the field of COVID-19. The choice of model type and structure is shaped by several factors, including the healthcare system of the study country, healthcare setting (e.g. emergency care, outpatient or inpatient care), the costs that are deemed to be relevant to answer the research question(s) of the analysis, the role of time to capture the costs and consequences and the treatment effect on disease transmission or the number of prevented hospitalisations. These and more dependencies are a likely explanation for the variety of model structures observed across studies. As was reported by the review of Rasmussen et al., a SEIR transmission model was used in approximately half of the economic evaluations of interventions against viral pandemics [9]. However, the review mainly included NPIs. In the current review concerning drug treatments, five studies were found to include disease transmission in the model. Remarkably, none of these considered any effect of the treatment on the number of infections. Two studies measured the number of prevented severe cases [36, 47], whereas the rest measured clinical improvement during hospitalisation. Of interest, two studies adjusted the infection rates in different scenarios (e.g. vaccination) to investigate the potential of the treatment of interest under different pandemic circumstances [37, 43].

The option of rehospitalisation was included in several models, whether or not in an explicit way. There is a growing body of evidence concerning the return to hospital after a COVID-19 infection. A systematic review and meta-analysis found that one in ten patients who recovered from COVID-19 require hospital readmission after discharge [62]. Another study reported an even higher proportion (28%) for the Omicron variant in a population with significant comorbidities [63]. These findings address that hospital readmission is an essential contextual consideration and advocate for the inclusion in health economic models of COVID-19 treatments. An important sidenote to the incorporation of hospital readmission is that patients may follow a different treatment trajectory with different timelines, rather than following the exact same route as treatment-naïve patients. In addition, for the proportion returning from ‘recovered’ to ‘susceptible’ in transmission models, the risk of infection and subsequent length of stay on treatment may differ between the once-infected and the non-infected population [64].

Long-term sequelae were incorporated over different follow-up times, were applied to either a certain subgroup or to all patients discharged from the hospital and involved various types of long-term costs. Evidence on long-term sequelae after a COVID-19 infection has been immature and inconclusive during the first 2 years of the pandemic, and started to grow as the number of survivors of COVID-19 continued to rise globally. Meanwhile, many clinical definitions have been proposed for long-term sequelae of COVID-19. The follow-up time and scope of symptoms defined as long-term sequelae by Goswami et al. rather aligns with the later published WHO definition of post-COVID-19 condition (PCC), which leaves room for more than 200 different new symptoms that continue or develop ‘3 months after the initial SARS-CoV-2 infection, with these symptoms lasting for at least 2 months with no other explanation’ [26, 65]. An accurate estimate of PCC prevalence has been challenging, due to the use of various definitions and the inclusion of heterogeneous populations. Moreover, PCC rates may have changed over time, as the risk of PCC development seems to be lower with Omicron infections (versus Delta infections) and was inversely associated with the number of COVID-19 vaccine doses [66]. Nonetheless, occurrence of PCC in approximately 13% of SARS-CoV-2 infections in the general population addresses the relevance of long-term sequelae as a contextual factor in the economic evaluation of COVID-19 treatments [67]. However, health economic modellers are challenged to find a balance between less sophisticated modelling requiring less data and omitting long-term sequelae on the one hand, and using a lifetime horizon and incorporating long-term sequelae often on the basis of low-quality evidence or assumptions on the other hand. The reasoning behind these decisions is influenced by factors such as the emergency nature of the pandemic, the far-reaching long-term consequences of the disease, and the perspective chosen to sufficiently demonstrate the value of effective COVID-19 treatments.

Treatment effectiveness was the most often reported parameter influencing the outcome of the analysis. One of the most frequently reported limitations across studies was related to limited or insignificant evidence on treatment effectiveness as well. The results of sensitivity analyses and the limitations reported in the studies are highly dependent on the methodological characteristics and study context, which differ substantially across the included studies. Although the methodological heterogeneity across studies makes it arduous to quantitatively analyse the outcome of sensitivity analyses and reported limitations, our results illustrate the need for evidence generation concerning both clinical and economic input parameters for future economic evaluations of treatments of COVID-19 and related diseases.

Of the 27 studies included, only 7 included productivity loss as indirect costs. The inclusion of indirect costs in the economic evaluation of COVID-19 drug treatments could contribute substantially to the total estimated costs. Importantly, productivity gains or losses of the treatment population are not the only indirect costs that would apply for an effective treatment. Besides the direct impact on productivity, an effective treatment can also save costs related to other disease prevention measures (e.g. lockdowns) that would be necessary in the absence of effective treatments. Even in the case of a treatment that is effective in reducing mortality but no other dimensions of the disease, a broader inclusion of indirect costs can still make the treatment a cost-effective option. However, incorporating cost savings of other disease preventive measures in the economic evaluation would require a much broader scope of the analysis.

The outcome of the economic evaluations may be subject to the impact of several external factors, such as new variants of concern and vaccination against COVID-19. The economic profiles of clinical treatment are likely to be influenced by vaccination, for example, through affecting the probability of suffering severe diseases upon infection. A few studies incorporated vaccination into their model by means of an alternative intervention strategy, patient subgroup analysis or scenario analysis. However, the majority of studies did not incorporate any impact of vaccination to the model outcome.

Studies identified were highly heterogeneous in objective, scope, and context. Therefore, several key methodological characteristics were difficult to summarise. For instance, assumptions were often related to the model structure (e.g. ‘patients who were admitted to the ICU were assumed to be transferred back to non-ICU inpatient care prior to discharge or otherwise have died’ [25]) and the type of drug treatment (e.g. ‘remdesivir affects the hospital length of stay for all patients and dexamethasone affects the mortality for patients requiring external oxygen’ [28]). Further, the types of cost included are largely dependent on the perspective, time horizon and costing method. For example, the study of Jovanoski et al. applied a fixed amount for hospitalisation cost, irrespective of the length of stay, whereas the study of Ruggeri et al. included daily costs of general ward and ICU, and the study of Popping et al. used a micro-costing (bottom-up) approach and included cost of paramedical care and care facilitated by the general practitioners’ offices [29, 37, 44]. Though relevant in the analysis of methodological approaches used in the economics evaluations of COVID-19 drug treatments, assumptions and cost parameters and sources appeared quite specific for each individual study and were left out of the narrative synthesis.

Internet searches show that this study is one of the few summarising methodological characteristics and modelling choices, and the first describing long-term sequelae incorporated in model-based economic evaluations that have been published thus far. Previously published systematic reviews on economic evaluations in the context of COVID-19 had various scopes and objectives and went beyond drug treatments as the type of intervention [7, 68,69,70]. The systematic review conducted by Elvidge et al. was closely related to our study, and focussed on methodological approaches used in economic evaluations of diagnostics and treatments for COVID-19 [8]. Besides key study characteristics, the study reported on the most influential parameters on cost-effectiveness as well. However, the review was last updated on 12 July 2021 and included a much lower number of studies (n = 7) than our study did. Moreover, results were described in a less extensive way compared with the results presented here, for which we believe our study provides new and more in-depth insights about some similar subjects.

4.2 Limitations of Evidence in Review

Literature searches were restricted to (1) full economic evaluations of (2) drug treatments against COVID-19 (3) using a decision–analytic model. Studies meeting these requirements were included regardless of type of intervention, comparator treatment, healthcare setting, objective, perspective or any other methodological topic. Only studies published in English were included, which likely excluded other relevant studies (at least one study was excluded during full-text screening because of language). The majority of studies were performed in high-income countries, with others in upper-middle income countries, indicating that the results presented here may be less generalisable to low- and middle-income countries. This may be particularly the case when considering respiratory support strategies, which are dependent on ICU capacity, availability of advanced respiratory support, and ICU admission criteria (e.g. age, comorbidities) [71].

Although several study characteristics were reported quantitatively, the qualitative part of the review was most important in this study. As such, risk of bias of included studies to the outcome of the review was considered potentially less relevant in the present case, since the goal of this study was to compare and summarise predefined modelling characteristics and methods rather than synthesising study results of such studies. Of note, study designs of the three included reports of Ruggeri et al. were largely similar, although the analyses were performed for different countries (Italy, Portugal, Saudi Arabia) and different interventions (casirivimab/imdevimab, remdesivir), and included a variety of disease severity categories (level of care, with or without MV/ECMO). As such, frequencies of modelling characteristics can still be interpreted as based on unique studies.

4.3 Limitations of Review Processes

Some methodological characteristics were not explicitly mentioned in all studies and had to be derived indirectly. Despite the risk of misinterpreting an author’s presentation of the methodological choices made, all study features that were not explicitly stated in the main text could be derived from tables and figures.

Parameter uncertainty and study limitations were categorised by topic to measure frequencies and identify potential commonalities across studies. Grouping by topic did not follow any existing conceptual framework and is therefore acknowledged to be arbitrary. Moreover, limitations only included the ones that were reported, leaving aside other possible shortcomings of the methods used.

4.4 Implications for Future Practice

On the basis of the findings of the current review, several implications for future practice of modelling COVID-19 drug treatments can be drawn. Basically, to respond quickly and effectively to future public health emergencies, the publication of an early HTA model in which a standardized model structure is presented for a specific disease or a certain patient group would allow health economists and decision makers to assess the value for money of different treatments in a consistent way. Some countries had a certain dynamic transmission model in place, such as the South African National COVID-19 Epidemiology Model (NCEM) and the South Korean COVID-19 epidemiology model. In addition, the concept model of a hypothetical treatment published by Sheinson et al. is an example for such a standardised model, and several input parameters have been adopted by at least two other studies included in the review [22, 32, 42]. Notwithstanding the usefulness of an early HTA model that can be applied to different treatments and customised to the healthcare setting of a specific country, a ‘one-size-fits-all’ model is likely unfeasible, as healthcare systems, healthcare capacities and treatment pathways of acute care vary per jurisdiction. For instance, the definition of what constitutes an ICU differs per country, if not per hospital [72]. The road towards a concept model therefore starts with clear and concise definitions of the different compartments of healthcare systems that are frequently applied in decision–analytic models. In addition, transparency in clinical pathways, for instance via clinical practice guidelines, would serve the development of a standardised model structure for a specific disease [73]. Moreover, since patient differentiation in a health economic model is usually based on the differentiation applied in clinical trials, the levels of disease severity should be carefully specified during the design phase of the trial, thereby considering potentially significant differences that impact clinical and economic evidence.

As was observed in the current study, most common model approaches used in economic evaluation are Markov models, whether or not combined with decision trees [74]. In the case of infectious diseases, a dynamic transmission model is a common approach and can be used to describe and predict a change in infection rate over time due to novel viral strains and increased versus waning immunity [75]. A working group report by Pitman et al. described that dynamic transmission models are relevant ‘when evaluating an intervention against an infectious disease that (1) has an impact on disease transmission in the target population or (2) alters the frequency distribution of strains (e.g., genotypes or serotypes)’ [76]. The changed incidence and the expected likelihood of different results as a consequence of a change in infection rates was also reported in several studies included in our study. However, it would be less straightforward to justify the use of a dynamic transmission model for economic evaluations where the impact of the intervention on disease transmission is deemed implausible. Therefore, health economic modellers should carefully assess the relevance of modelling the impact of averted infections on the cost-effectiveness of an intervention.

Finally, regarding the acute phase, the simulation of patients with COVID-19 by only their highest level of respiratory support received may be a pseudorealistic representation of actual clinical practice. A prospective cohort study of more than 60,000 patients with COVID-19 found that a substantial part of patients transit between ventilation treatments during their hospital stay [71]. As such, it can be argued that the possibility to shift between different respiratory health states at the stage of hospitalisation would improve the dynamic properties of a time-dependent health economic model.

5 Conclusion

The study found substantial differences across the methodological choices made in health economic models of COVID-19 drug treatments. Differentiation of the modelled population during the acute phase and long-term sequelae have been incorporated in various ways. A more standardized approach is pivotal to increase consistency and transparency across model-based economic evaluations. Further, the results revealed a lack of economic evidence, such as resource use, cost and QoL data. This implies the need for a broadened scope of evidence generation in future public health emergencies.