Introduction

COVID-19 and economic studies: a new era calling for a societal approach

The COVID-19 pandemic has impacted on the scope of health economics literature, which will increasingly examine value beyond health care interventions such as government policies (Mullins and Drummond 2020). The aim is to analyze economic evaluations and methodologies assessing COVID-19 government policies suppressing or mitigating transmission, reducing disease, and impacting national income loss; health system innovations and models of care. This can facilitate future economic evaluations and assist government and public health policy decisions during pandemics.

The global situation during a pandemic requires new economic evaluation methods in healthcare beyond traditional approaches to CBA and CEA. There are large economic impacts of social distancing, declining stock markets, increasing government spending, and unemployment. Some governments use CBA, monetizing reduced death risks, extending lifetimes, and comparing costs to benefits to determine whether prevention costs outweigh health benefits.

Greenstone and Nigam (2020) and Thunstrom et al. (2020) found mortality benefits from social distancing are between $8tn and $12tn, or $60,000 to $95,000 per household in the USA. Costs are high but benefits are greater. Thunstrom et al. (2020) calculated $5tn net benefits when including costs to GDP from social distancing. These studies assume society should pay $10m to save a person’s life, viz the value of statistical life (VSL). The USA Department of Transport and Environmental Protection Agency analyses are consistent with these findings (Freeman and Groom 2020). Greenstone and Nigam’s (2020) age adjusted data calculated $16.1m to save a 20–29-year-old and $1.5m for those aged 80 and over.

In CEA there is a preference to estimate value based on QALYs rather than a VSL. NICE pays £20,000–£30,000 for drugs to facilitate good health for one extra year. USA’s Institute for Clinical and Economic Review QALY is higher at $50,000–$150,000. In the UK a 70-year-old has 15 years life expectancy. Using cost/QALY of £25,000, this amounts to £375,000 for their future life. This is an over-estimate for those likely to die from COVID-19. However, it is lower than $3.7m Greenstone and Nigam (2020) assign to a life of an individual aged 70–79. Given £25,000/QALY and mortality estimates, UK households WTP is £6750 each to prevent 600,000 COVID deaths; £185bn in total or £330,000 per life saved. Social costs of distancing such as domestic violence and mental health should be included (Freeman and Groom 2020).

Society’s position on whether health benefits of COVID-19 interventions outweigh costs can depend on whether cost/QALY or VSL is used. The implications of economic evaluations depend on how society values change in risk of death or length and quality of lives extended. Trade-offs can be explicit in CBA, quantifying them using values implied by societal decisions. However, Freeman and Groom (2020) argue neither method, VSL or QALY, is applicable. VSL uses stated WTP to avoid a small increase in risk of death. Lower estimates of cost/QALY reflects health effects of relatively small changes in NHS expenditure, rather than what society “ought” to pay to improve health status (Freeman and Groom 2020). There is a paucity of economic evaluations addressing the mitigation of pandemics at the national and societal level, along with equity in decision making. This paper investigates these issues.

The pandemic raises difficult ethical choices for governments such as lockdowns and vaccine prioritization. To what extent and time period should costly lockdowns be used to enable good population health? Who should be prioritized in the allocation of vaccines, ventilators, or intensive care unit beds? Should vaccine doses be equitably shared globally, or should countries prioritize the health and economic well-being of their citizens? Equity in decision making can be addressed by social welfare function (SWF) analyses. SWF is attractive given it accounts for societal inequalities and population-wide policy impact. Policy choices involve health and income trade-offs, with consequences frequently heterogeneously distributed across populations. Older adults are at highest risk of severe COVID-19. The working-age encounter the burden of economic lockdowns, with excessive impacts on low socioeconomic-status individuals given they may not work at home (Ferranna et al. 2021a).

The following sections commence with a systematic review of full economic evaluations of pandemic interventions. They address COVID-19 government policies suppressing or mitigating transmission, reducing disease, and impacting national income loss, health system innovations and models of care. The government policies concern restrictions such as social distancing, stay at home orders, and institutional closures as lockdowns. Health system innovations involve COVID-19 workforce prevention, sheltered homelessness interventions, staffing ICU bed reserve capacity in Europe, and epidemic control strategies such as testing, contact-tracing, isolation centers, screening, and quarantine centers. New models of care concern home maintenance allergen immunology versus clinics, hospitalized COVID-19 patients versus supportive care, standard COVID-19 care versus telemonitoring for heart failure patients, COVID-19 patients in public versus private-health systems, and COVID-19 testing for diagnosing and discharging patients.

Given the scope of health economics literature is expanding to evaluate value beyond healthcare interventions, such as other government policies, the research also analyses theoretical issues concerning welfare economics, SWF, social return on investment (SROI), WHO’s pandemic economic risk framework for social and movement measures, and MCDA.

The current research makes a significant contribution to the economic literature as there were no prior systematic or scoping reviews that addressed the breadth of research question being addressed in this study. A study by Dawoud and Soliman (2020) undertook a systematic review of the cost effectiveness of antiviral treatments for pandemics and outbreaks of respiratory illnesses, including COVID-19. A systematic review by Rees et al. (2020) analyzed COVID-19 length of hospital stay to project future system demands by various levels of care. A systematic review concerning the economic evaluation of programs against COVID-19 was undertaken by Rezapour et al. (2021). Their aim was different as it focused on COVID-19 treatments and programs rather than government policies and excluded more recent studies undertaken since July 2020. They reviewed studies from December 2019 to July 2020. The current study included studies from 2020 to 2021 and explored broader issues concerning health system changes, government policies aimed at suppressing or mitigating transmission, reducing disease, and national income loss. Importantly, it analyses important methodological issues.

Rasmussen et al. (2022) published a scoping review of economic evaluations against viral pandemics. It was different to the current study as it included a broader number of diseases such as Ebola, Zika, SARS, MERS, H1NI, and H5N1. It did not include the study results and their implications and did not assess the quality of the economic evaluations. Rather, the authors only included summary statistics on study perspective, costs, comparators, and economic models.

Further, the study by Rasmussen et al. (2022) did not explore economic methodological issues pertinent to public health decision making during a pandemic. Importantly, in addition to analyzing the full economic evaluations of pandemic interventions, the current study analyses methodological issues in economics impacting on pandemics including welfare economics, social return on investment, social welfare function, multi-criteria decision analysis, and the WHO pandemic social and movements decision-making and economics framework. The study thereby provides an important and unique contribution to the literature.

Methodology

Literature review framework and protocol registration

The research aimed to analyze economic evaluations and methodologies assessing COVID-19 interventions and government policies to suppress or mitigate transmission, reduce disease, and impacting on national income loss; health system innovations and models of care. Hence, the study analyses economic studies and novel methodologies that can evaluate broader health system and societal impacts of pandemic interventions. A systematic scoping review of the literature is appropriate in circumstances where the study aim is to identify the types of evidence available in the field and any knowledge gaps (Peters et al. 2020a; Munn et al. 2018; Peters et al. 2020b). A systematic scoping review was undertaken using PRISMA-ScR including the review design, population, concept and context (PCC), protocol and PRISMA flow charts (Peters et al. 2020b). The review was based on an a priori defined PRISMA-Scr protocol, which was registered by Open Science Framework (OSF). [https://osf.io/4wzac/?view_only=a412c5177f624fcd94f06e76c583a31f] The OSF registered PRISMA-Scr protocol is also in Appendix 1. The completed PRISMA-Scr checklist is in Appendix 2. Full economic evaluations were assessed for quality and methodological rigor using the CHEERS Checklist (Husereau et al. 2022) with the assessment scoring tool published in the “European Journal of Health Economics” by Antioch et al. (2002), and NICE’s (2012) cost benefit analysis checklist.

Search strategy and selection of studies

Searches were undertaken using PUBMED, Medline, PUBMED Central (PMC), and the National Centre for Biotechnology Information (NCBI) Bookshelf for April 2020 to June 2021. The search engine was PUBMED Advanced Search Builder, National Library of Medicine, USA. Search terms: (COVID-19[MeSH Terms]) AND (cost effectiveness[MeSH Terms]) April 2020 to April 2021, resulting in 109 references. The second search included terms (cost benefit analysis) AND (COVID-19) June 2020 to June 2021, resulted in 196 references. See Appendix 3 for details of PUBMED searches. The additional 25 studies were identified from other sources, including references from selected journal articles and Google Scholar. Once duplicates were removed, title and abstract reviews were conducted to assess eligibility. Full text reviews were also conducted, and full economic evaluations assessed for methodological quality. A PRISMA flow diagram summarizing the study selection process is shown in Fig. 1, based on the PRISMA 2020 statement: an updated guideline for reporting systematic review (Page et al. 2021)

Fig. 1
figure 1

PRISMA flow chart for reporting the systematic scoping review of economic evaluations of pandemic interventions. Source: based on framework in Page et al. (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ;372:(71). doi: 10.1136/bmj.n71

Full size image

Eligibility criteria

All records identified were assessed for inclusion against the following participants, concept and context (PCC) criteria. The results of quality assessments of the economic evaluation methodology were also used for full economic evaluations.

Participants

Populations impacted by the COVID-19 pandemic, including those in health services, the health care system, and society at large. Populations and patients with COVID-19 infection and those causing outbreaks.

Concept

  • (a) Full economic evaluations: CEA, where results were expressed in monetary units per case averted or life year saved; CUA, where results were expressed in monetary units per QALY or DALYs; CBA or CMA, where results were expressed as an intervention’s total saving or loss in monetary units. Primary research was preferred.

  • (b) Peer-reviewed review articles and books concerning broad methodological issues addressing pandemic impacts such as welfare economics and MCDA. All published 2020 and 2021.

Context

All contexts including all countries of origin. The context should relate to COVID-19 evaluations and methodologies concerning government policies suppressing or mitigating transmission, reducing disease, impacting national income loss, health system innovations, and models of care.

Exclusion criteria

Cost of illness studies, studies not adopting a comparator for full economic evaluations, conference abstracts, newspaper articles, and dissertations. Studies not reported in English were also excluded.

Assessing quality of studies

The methodology to grade full economic evaluations, discussed in Antioch et al. (2017), was used to assess the quality of the full economic evaluations. This included the Consolidated Health Economics Evaluation Reporting Standards (CHEERS) checklist, updated in 2022 (Husereau et al. 2022) in conjunction with Drummond’s 10-point checklist (Drummond et al. 2005; Drummond et al. 1997) which is included and scored in Antioch et al.’s (2002) assessment tool published in the “European Journal of Health Economics.” Cost benefit analysis studies were also assessed using the foregoing checklists and NICE (2012) Cost Benefit Analysis Checklist. A total score achievable for each study was 6 for costs and 10 for consequences, for a total composite score of 16 (Antioch et al. 2002). Studies graded as strong on cost effectiveness methodology and achieving a composite score of at least 10 out of 16, were selected for the study. The scoring system by Antioch et al. (2002), which uses Drummond et al.’s (2005, 1997) checklist has also been utilized extensively in other studies grading the methodological rigor of health economic evaluations.

Results

Overall 330 records were identified, of which 40 studies met the inclusion criteria. Fifteen (37%) were full economic evaluations and were mainly model based. They achieved scores higher than 10 and were selected for the systematic review. Of these, six (40%) were CUA, six (40%) were CEA, one (7%) was a CMA, and two (13%) were CBA. Six CEAs evaluated health-system innovations using decision tree, Monte Carlo, and other stochastic-models. They concerned COVID-19 workforce-prevention, sheltered homelessness interventions, staffing ICU bed-reserve capacity, epidemic control strategies: testing, contact tracing, isolation centers, screening, quarantine-centers. Five CUAs evaluated new models of care using decision tree and Markov. They evaluated home-maintenance allergen immunology versus clinics, hospitalized COVID-19 patients versus supportive-care, standard COVID-19 care versus telemonitoring for heart-failure patients, COVID-19 patients in public versus private-health systems, COVID-19 testing for diagnosing and discharging patients. Two CBA and a CUA analyzed government policies concerning restrictions, mitigation, and suppression such as social distancing, stay at home orders, institutional closures such as lockdowns. Strategies were generally found to be either cost-saving or cost effective at the study specific willingness to pay thresholds. The remaining 25 (63%) studies addressed welfare economics, social return on investment, social welfare function, multi-criteria decision analysis, and the WHO pandemic economic risk framework. Table 1 presents the characteristics of included full economic evaluation studies on COVID-19, providing overview statistics on the origin (continent) of each study, economic evaluation study type, study design model for estimation, and the study perspective. It includes a frequency count and percentage distribution. Appendix 4 includes the completed data chart of included full economic evaluation studies, providing valuable details of all results. This includes author, year, continent of origin, types of economic evaluation, type of model for estimation, types of interventions and comparators, primary outcome, study perspective, economic evaluation results reported in the relevant currency and economic evaluation ratios, and methodological quality assessment scores. All included studies had high methodological quality scoring 15 (or 94%) or higher. The implications of the most relevant studies relating to government policies on pandemic restrictions, such as mitigation, suppression, health system innovation, and models of care are discussed in the next section. A consolidated overview of all other results from Appendix 4 are also discussed. Methodological studies concerning welfare economics, social return on investment, social welfare function, MCDA, and WHO pandemic decision and economics framework are also analyzed.

Table 1 Characteristics of included studies on COVID-19 – full economic evaluation studies
Full size table

Discussion

Full economic evaluation studies

The results from all the full economic evaluations are included in the table in Appendix 4. Studies concerning assessments of the value of COVID-19 interventions involving health system innovations and government policies aimed at suppressing or mitigating transmission, reducing disease, and minimizing national income loss are discussed below. The other full economic evaluation studies in Appendix 4 concerning health system innovations and new models of care are also discussed. Methodological issues concerning welfare economics, social return of investment, social welfare function, multi-criteria decision analysis (MCDA) and the WHO pandemic decision and economics framework are also analyzed below.

We turn to three important full economic evaluations concerning government policy to suppress or mitigate transmission, reduce COVID-19 disease, with impacts on national income loss. A promising CBA methodology was undertaken by Broughel and Kotrous (2021) who estimated the benefits and costs of state suppression policies to “bend the curve” during the initial outbreak of COVID-19 in 50 USA States. U.S. states enforced non-pharmaceutical interventions (NPIs) to suppress COVID-19 transmission by closing nonessential businesses and enforcing stay-at-home orders for all residents. Almost 90% of the population was required to stay at home unless engaged in “essential” activities. They valued benefits and costs in terms of additions or subtractions to total production, using the value of production (VOP) approach. Costs included losses to output associated with the enforcement of nonessential business closure and stay-at-home orders. Indirect costs occurred from increased mortality risks via suicide related to lost income. The benefits valued prevented COVID-19 deaths in terms of total production gained by lengthening lives. Cost-savings of preventing COVID-19 illnesses and health-care utilization were also estimated.

Relative to a baseline where only infected and at-risk populations mitigate the COVID-19 spread, total benefits of suppression policies to economic output ranged from $632.5b to $765.0b from March to August 2020. Relative to private mitigation, output lost due to suppression policies was between $214.2b to $331.5b. Cost estimates were based on length of non-essential business closures and stay-at-home orders, enforced for between 42 and 65 days. Net benefits of suppression were between $301b and $550.8b. The most significant factor was reduced mortality.

Given there is controversy concerning methodologies to valuing benefits of life-saving public health interventions during the pandemic, they also presented mortality benefit estimates using VSL and QALYs saved. Net mortality benefits were between $10.6t and $11.9t using VSL of $11.3m. Gross mortality benefits using QALYs, based on the $50,000 threshold, were between $285b and $530b. Their estimate using the VoP approach falls within this interval suggesting similarities between the QALY and VoP approach (Broughel and Kotrous 2021). The study achieved a methodological rigor score of 15 (or 94%) representing high methodological rigor.

Their finding of positive net benefits is consistent with other CBAs of social distancing during the pandemic. However, their net benefit estimates are smaller (see Broughel and Kotrous 2021 for a review). The difference in their findings is attributable to two issues. First, they estimated the costs and benefits associated with the policy response to COVID-19. They did not estimate the costs and benefits associated with social distancing more generally, which includes public and private actions that reduce economic and health impacts of COVID-19. Second, their analysis focuses on the costs and benefits of COVID-19 suppression with regard to its effects on economic output and production (Broughel and Kotrous 2021). Their VOP approach to valuing mortality benefits enables more direct comparisons of mortality benefits to other relevant benefits and costs. Healthcare utilization is most easily measured by observed service cost. Similarly, costs associated with policy interventions, such as losses to GDP, are comparable to these production benefits. Their focus on production is similar to other CBAs in the literature, such as evaluations of COVID-19 screening tests (see Broughel and Kotrous 2021 for a review).

Their unique contribution is accounting for potential increases in mortality risks, owing to economic costs associated with income losses stemming from public policies. They found that suppression policies are, on net, cost-saving.

The policies prevented additional deaths through this income-saving channel, in addition to preventing COVID-19 deaths more directly. While the indirect mortality benefits are small relative to benefits overall, it is important to calculate such ancillary mortality risks given concerns about suppression policy impacts on depression that could outweigh the health consequences of COVID-19. Conversely, they found suppression policies had minimal short-run effects on overall mortality through such indirect mechanisms. While economic dislocation is not the only factor impacting mental health during suppression policies, their findings are in accord with a study that found the number of suicides in several US states and high-income countries during the summer and fall months of 2020 did not diverge significantly from pre-COVID-19 trends (Broughel and Kotrous 2021). Their analysis has some limitations. There is uncertainty regarding the number of COVID-19 deaths that would have occurred in the counterfactual scenario in which suppression policies were not enforced. Their choice of counterfactual is the forecast of COVID-19’s progression in USA by Ferguson and colleagues, published in March 2020 and based on early evidence about disease transmission and mortality (Broughel and Kotrous 2021).

Miles et al. (2021) compared the UK population “lockdown” response and outcomes with European countries of comparable income and healthcare resources using a CBA involving macro costing using QALY. They measured estimates of the economic costs as different percentage losses in GDP against possible benefits of avoiding life years lost, for different scenarios where current COVID-19 mortality and comorbidity rates were used to calculate the loss in life expectancy. Adjustments were made for their levels of poor health and quality of life. They then applied a QALY value of £30,000, which is the maximum under NICE national guidelines. The costs of continuing severe restrictions were so high relative to likely benefits in lives saved that an expeditious easing in restrictions was justified. The smallest estimate for lockdown costs incurred was 40% higher than highest benefits from avoiding the worst mortality case scenario at full life expectancy tariff. In more realistic estimations, the lockdown costs were over five times higher. Future scenarios showed in the best case, a QALY value of £220k, which is seven times (x7) the NICE guideline. In the worst-case, £3.7m, which is 125 times the NICE guideline, was required to justify lockdown continuation (Miles et al. 2021). The study achieved a methodological rigor score of 16 representing very high methodological rigor.

The economic evaluation methodology used by Zala et al. (2020) was a CUA using a patient simulation model with attached costs. They calculated the relative cost-effectiveness of hypothetical suppression policies in the Imperial College COVID-19 Response Team model in the UK. Population level disease projections in deaths, ICU bed days, and non–ICU bed days were analyzed. National income loss estimates were derived from a study on the impact of a hypothetical pandemic on the UK economy, with sensitivity analyses based on more recent projections. Individual QALY loss and costed resource use inputs were analyzed. They compared two hypothetical suppression strategies to a mitigation policy and an unmitigated pandemic. An unmitigated pandemic assumed no government control measures. The mitigation aim was to decrease the pandemic impact by flattening the curve to reduce peak ICU demand and overall deaths, assuming (1) individual case isolation, (2) home quarantine (household with a suspected case), and (3) social distancing advice for individuals aged over 70.

There were two suppression strategies. Suppression involves more extensive controls, viz general social distancing, school, and university closures: (1) Suppression 1, triggered “on” at 100 ICU cases in a week and “off” when weekly cases halve to 50 cases; (2) Suppression 2, triggered “on” at 400 ICU cases in a week and “off” when weekly cases halve to 200 cases. Results for base case settings (R = 2.4) included the following findings. Suppression 1 versus unmitigated resulted in £19,653 per additional QALY. Suppression 1 versus mitigation resulted in £33,346 per additional QALY. Suppression 2 versus unmitigated £20,977 per additional QALY. Suppression 2 versus mitigated £38,314 per additional QALY.

Assuming a maximum reduction in national income of 7.75%, incremental cost effectiveness ratios (ICERs) for Imperial model versus mitigation are below £60 000 per QALY. However, results are uncertain and conditional on the accuracy of the Imperial model projections. They are also sensitive to estimates of national income loss. Nevertheless, it would be arduous to claim that these suppression policies are cost-ineffective relative to the alternatives available. The article provides some early insight into the trade-offs that are involved (Zala et al. 2020). The study achieved a methodological rigor score of 16 representing high methodological rigor.

The remaining 12 full economic evaluation studies that used CEA, CMA, or CUA addressed health system innovations and new models of care. All 12 studies achieved high methodological rigor scores of at least 15 (or 94%). The CEAs and a CMA evaluated COVID-19 health system innovations. CEAs using decision tree and simulation models evaluated new forms of sheltered homelessness interventions to reduce disease spread (Baggett et al. 2020), universal COVID-19 screening versus personal protective equipment (PPE) for obstetric health workers (Savitsky and Albright 2020), and staffing ICU bed reserve capacity in Europe (Gandjour 2021). CEAs using Monte Carlo simulation models evaluated health system epidemic control strategies involving testing, contact tracing, isolation centers, screening, and quarantine centers (Reddy et al. 2021) and evaluations of health system PPE for health workers in low- and middle-income countries (Risko et al. 2020). A CEA using stochastic analyses evaluated COVID-19 testing strategies combined with isolation periods (Du et al. 2021). A CMA evaluated the UK National Health Services testing pathways for suspected COVID-19 patients using community versus standard hospital practices (Currie et al. 2020).

CUA using decision tree and simulation models such as Markov evaluated new models of care such as home maintenance allergen immunology versus in office clinics (Shaker et al. 2020), hospitalized COVID-19 patients versus best supportive care (Sheinson et al. 2021), standard COVID-19 care versus telemonitoring for older heart failure patients (Jiang et al. 2021) and management of COVID-19 patients in public versus private health systems (Cleary et al. 2021). CUA using patient simulation models (SEIR) with attached costs evaluated COVID-19 tests for diagnosing and discharging patients (Jiang et al. 2020). Appendix 4 includes valuable details of the results of all full economic evaluations.

An analysis of the foregoing studies raises important issues, especially those by Broughel and Kotrous (2021), Miles et al. (2021), and Zala et al. (2020) that evaluate government policies suppressing or mitigating transmission, reducing disease, and impacting national income loss. Key issues concern the economy-health tradeoff and the equity-efficiency tradeoff and the scope of economic evaluations. To what extent should economically costly lockdowns be imposed to ensure good population health? What are the health and income tradeoffs? How can policies account for societal inequalities and the distribution of policy impacts across the population? To what extent should economic evaluations include a broad array of socio-economic and environmental outcomes such as pollution reduction caused by lock downs when assessing social return on investment? Is there scope to capture health and non-health impacts underpinned by the “triple bottom line” viz social, economic, and environmental issues? What frameworks can assess the intended and unintended economic impact of social and movement measures during the pandemic?

While the intention is to limit virus spread and reduce deaths, unintended consequences involve disrupting access to health care and delays to diagnosis. It can exacerbate economic slowdown, socio-economic inequality, and harm workers unable to telework. How can we facilitate the systematic identification of new, emerging, or obsolete technologies impacting health and society during the pandemic? Are there frameworks to address these issues and facilitate stakeholder engagement in the process? Such methodological issues are explored below.

Methodological issues

Welfare economics

Governments face challenging trade-offs. Flattening the curve and saving lives requires major fiscal economy-wide support. The UK government has committed 20–40% of GDP. Total health, fiscal, and social costs are borne later. The societal perspective is important, with social wellbeing treated as a much broader public health issue. Welfare economics involves an ex-ante, proactive consideration of alternative policies. The economy-health trade-off and the equity–efficiency trade-off are central. Allocative efficiency which maximizes welfare, internalizing positive or negative externalities, is considered. Decision makers should move from a “medical problem” perspective to a much broader array of issues and stakeholders (Chilton et al. 2020).

Resource allocation is important and economic evaluation during a pandemic can involve calculating intervention benefits and costs, and the aggregate outcome of many marginal stakeholders’ decisions and trade-offs. Society can choose the best response, identifying social and health related opportunity costs, such as social isolation, inequalities, surgeries cancelled, and treatments displaced. Current scenarios can be costed into future projects, along with the benefits of factors such as additional ventilators, nurses, ICU beds, schoolteachers, and local services. Economic evaluation enables analyses of opportunity cost and health outcomes of alternatives such as contact-tracing, testing, regional versus national approach, random sampling, antigen, and antibody tests (Chilton et al. 2020)

During COVID-19 “excess” deaths in the UK, including the effects across age cohorts, can be calculated. The impact on life expectancy and quality of life data can be analyzed by decision makers. Willingness to pay (WTP) methods such as value of a prevented fatality (VPF), value of a statistical life year (VLY), and QALY can be used, including the negative externalities of pain and suffering. WTP assumes true preferences and values are used. However, the resulting increases in taxes and associated ethical issues should be considered even where the benefit/cost ratio is positive for a high cost intervention (Chilton et al. 2020)]. The Regulatory Office of the Australian Department of Prime Minister and Cabinet provides guidance on preparing cost-benefit analysis in Regulation Impact Statements. This includes how to treat the benefits of regulations designed to reduce the risk of physical harm. WTP estimates the value of reductions in the risk of physical harm known as the value of statistical life (VSL). The VSL is $5.0m and the value of statistical life year (VLY) is $217,000 in 2020 Australian dollars (Department of Prime Minister and Cabinet 2020). Some USA regulatory government agencies and American economic evaluations of the pandemic use $10m and $11.3m as VSL. This is more than double the value of the Australian VSL of $5m used by the Commonwealth Government for regulatory decisions.

Chilton et al. (2020) indicate that allocative efficiency analyses are important and smaller-scale decisions such as rationing ventilators or Do Not Attempt Resuscitation guidance to care homes could be established before an epidemic. Costs can be incurred now and benefits identified later, which are likely inter-generational. High discount rates for future consequences and the rationale for optimal responses should be considered.

An ex-ante, viz “before the event” approach can identify low up-front cost interventions with significant future benefits, such as life expectancy and lower inequalities. Alternatively, costly interventions with uncertain or low benefits can be articulated. Previous UK policy involved comparatively fewer ICU beds than Germany and austerity, implying a low present value (high discount rate) on the future of a potential pandemic. Would such a policy for future pandemics accurately reflect the country’s preferences? (Chilton et al. 2020).

The benefits and costs of future programs should be re-evaluated and include crucial areas such as public service, safety nets, population health, inequality, health services strength, and essential workers. Smaller, extra expenditure or redistribution in future, reflecting these priorities and benefits, may prevent higher costs of future crises (Chilton et al. 2020). Analysis of these economic issues during a global pandemic can be further assisted through Social Welfare Function, Social Return on Investment, the WHO’s decision and economics framework, and MCDA.

Social welfare function

The SWF approach can explicitly account for societal inequalities and the distribution of policy impacts across the population. It may therefore be more appealing than more traditional approaches. In the design of vaccine prioritization strategies, trade-offs can emerge between protecting the health of high-fatality-risk individuals, such as older adults and those with comorbidities, and the health of high-exposure-risk individuals undertaking essential societal activities or in economically critical sectors. Methodologies to adequately evaluate health-related interventions that can have differential socioeconomic and health consequences in the community are challenging (Ferranna et al. 2021a).

Traditional evaluation methods, such as CEA and CBA, are restricted to some extent in dealing with the complexity of these issues. CEA has a health-centric approach focusing on benefits involving the changes in mortality and morbidity and healthcare cost savings. It does not permit differential socioeconomic benefits across different units of health. Where health is measured in QALYs, CEA assumes that each additional QALY has equal value regardless of the individual characteristics experiencing the QALY such as income level or age.

On the other hand, CBA analysis evaluates an intervention by converting its health and non-health benefits into monetary equivalents and summating them (Hammitt 2020; Greenstone and Nigam 2020). Unlike CEA, CBA can include differential socioeconomic benefits across different units of health through defining individual-specific WTP measures. However, WTP is dependent on ability to pay, and CBA analysis can therefore inflate benefits accruing to rich individuals relative to similar benefits attributable to poorer counterparts. Since CBA attributes the equivalent value to currency paid by the wealthy and those paid by the poor, an intervention’s value is independent of whether its cost burden falls on wealthy or poorer individuals. This is a consequence of CBA using an unweighted sum of monetary equivalents. An alternative approach to policy evaluation is social welfare function (SWF) analysis (Adler 2019). SWF measures an intervention’s health and non-health effects on individual well-being. It then aggregates individual well-being impacts to calculate an overall measure of the intervention’s value. The aggregation is calculated through a SWF, including concerns for the distribution of well-being across the population.

Like CBA, SWF analysis allows for differential socioeconomic benefits across different units of health. Unlike CBA, it does not necessarily inflate the value of health benefits accruing to wealthy individuals relative to the poor, given the criterion of evaluation is well-being and not currency. Further, it is sensitive to the population distribution of burdens and benefits (Adler et al. 2014).

SWF is a generalization of CBA. Operationally, it is equal to an equity-weighted CBA, where the configuration of the weights is based on the specific SWF (Ferranna et al. 2021b)

The choice of the COVID-19 intervention value framework is a significant issue (Ferranna et al. 2021a). The issues of whether to impose a lockdown, its level of restrictiveness and duration is pertinent. A permissive lockdown policy can result in an uncontrolled pandemic with harms tending to be excessively endured by the worse off and poor. Contributing factors include working and living conditions, which put them at above average risk of infection. Some may also experience reduced access to quality health care when infected.

Lockdowns can impose costs, such as job and income losses, that can be disproportionately borne by the worse off. They have less savings and more limited re-employment potential, especially where social safety nets are lacking. This is important in developing countries with less ability to provide public income support.

The optimal lockdown stringency and duration is based on interacting empirical and normative issues. A first empirical likelihood is that the burdens to the worse off from an uncontrolled pandemic are higher than the burdens of a lockdown. In this scenario, interests of the worse-off will be served by a stringent lockdown policy. The second possibility is that the burdens to the worse-off from a lockdown are greater than the burdens from an uncontrolled pandemic. Here, the interests of the worse-off are better served by less stringent lockdowns.

The optimal policy for CBA occurs where the sum of individual WTP is highest. This WTP will reflect willingness to both avoid pandemic harms and avoid policy harms. However, CBA weighs every dollar of WTP equally, even across the rich and less wealthy with differential ability to pay. It therefore underweights the interests of the worse off, rendering it insensitive to the abovementioned considerations compared with an SWF approach. For the first empirical scenario, CBA will recommend a lockdown policy that is too permissive considering equity concerns. When the second empirical possibility holds, it will recommend a lockdown policy that is too stringent. Across the two empirical scenarios, the worse off do better under the SWF function approach compared to the CBA method (Ferranna et al. 2021a).

Ferranna et al. (2021a, b) discuss their studies of lockdowns whose burdens are disproportionately borne by the worse off. In America, CBA may support policies that eliminate infection spread even if they cause a 30% GDP loss. SWF analysis will assess these policies to be unacceptable if low-income groups pay a disproportionate amount of net costs. In this scenario, applying plausible levels of inequality aversion, SWF would recommend only policies that cost a maximum of 10–15% of GDP. The larger the decision makers’ aversion to inequality, the lower the support for strict lockdown policies when their costs are regressive.

Vaccine prioritization highlights important differences between traditional evaluation methods and SWF analysis. Modeling the best allocation of vaccines (e.g., Bubar et al. 2021) is often based on epidemiological outcomes such as deaths, years of life lost, or numbers of infections and possibly on the costs of delivering such outcomes.

However, they can neglect structural inequities, such as concentrating vaccinations in socially vulnerable areas, the economic benefits of alternative vaccination rules that consider linkages between vaccination and relaxing nonpharmaceutical interventions; and the impact of characteristics such as age. Contrastingly, SWF can include imperatives for social equity and socioeconomic impacts of alternative vaccine allocation strategies (Ferranna et al. 2021b).

Assume the risk of severe COVID-19 outcomes increases with age and decreases with socioeconomic status. This may occur since rich individuals can more effectively protect themselves from infection risk and obtain more effective treatments. Older adults in low socioeconomic groups are the most exposed to risk of severe outcomes. Assume the vaccine enables very good protection but is less effective at reducing transmission. Where a policy goal is to decrease the number of severe cases, the vaccine would be administered initially to older adults in low socioeconomic groups. It would subsequently be administered to older adults in high socioeconomic groups. Where the policy goal is to maximize social welfare, older adults in low socioeconomic groups would still be prioritized given their high risk. However, if social equity is the imperative, younger adults in low socioeconomic groups would come next, rather than older wealthy adults given the former are in a less privileged position from a socioeconomic perspective and are younger and have not lived a full life yet (Adler et al. 2021), (Ferranna et al. 2021b).

The SWF approach is more data intensive compared with more traditional methods, given it requires information on the distribution and correlation of the populations’ different attributes. It also requires data on the distributional effects of the policy on the attributes. Social welfare analysis takes a consequentialist perspective. It therefore does not fully capture other ethical issues, such as human rights, individual responsibility versus luck, or the restitution principle. Nevertheless, because SWF analyses the distribution of policy impacts across the population and the correlation with background inequalities, it is more attractive than CBA and CEA. It can also potentially include equity and justice issues in policy evaluation (Ferranna et al. 2021a). Social return on investment is also important during a pandemic and is discussed below.

Social Return On Investment (SROI)

The objective of SROI is to assess if an intervention is worth the investment. Costs are analyzed in monetary value and benefits can be linked to health and broader social improvements. Compared to traditional health economic evaluation tools, SROI is an extension of CBA, including a broader array of socio-economic and environmental outcomes. It captures health and non-health impacts, underpinned by the “triple bottom line” viz social, economic, and environmental. There is broad stakeholder engagement in valuing outcomes, unlike more traditional approaches. SROI evaluates and places proxy values on personal, social, and community outcomes where necessary, capturing social impact at the societal level. It also values the potential negative effects (Banke-Thomas et al. 2015).

During a pandemic there is a need to capture social, economic, health, environmental costs and benefits given the nature of the impacts globally. The total health, fiscal, and social costs may be borne later. In this societal perspective, social wellbeing should be seen as a broader public health issue. It requires expanding the traditional scope of health economics beyond health care interventions to include government policies to suppress or mitigate transmission, suppress disease, and direct economic support addressing the consequences of policies involving prolonged lockdowns.

It can also capture environmental impacts such as reductions in CO2 emissions and pollution which are environmental and health benefits of lockdowns that can be included in the analysis. It is reported as monetary value or welfare benefit. Financial proxies are used to estimate the monetary value of benefits not easily monetized. SROI is the ratio of discounted value of benefits (social value) divided by total investment. SROI > 1 is worthwhile. The SROI ratio 3:1 means $3 of social value created for $1 invested after discounts. The main output of analysis is an SROI ratio, net present value, and payback period. It can be used for priority setting, resource allocation, and stakeholder building. Its strengths involve use of a singular ratio which captures positive and negative outcomes with stakeholder engagement. The challenges involve difficulty attaching financial values to “soft outcomes” and the “counterfactual.” There is poor comparability of SROI ratios across interventions (Banke-Thomas et al. 2015).

During the pandemic, the environmental impact of lockdowns can be included. These led to a reduction in human activities, energy use, CO2 emissions, and pollution impacting on health outcomes. There was an abrupt 8.8% decrease in global CO2 emissions in 2020 compared to the same period in 2019. The timing of emission decreases corresponds to lockdown measures in each country. Substantial differences in emissions persist between countries, with continuing emission declines in USA (Liu et al. 2020),

Some relevant studies have analyzed SROI in health services, involving methodologies that analyze both health and social impact of policies, interventions, and services (Ashton et al. 2020), SROI from public health policies to support implementing the Sustainable Development Goals by building on Health 2020 (Dyakova et al. 2017), and the methodological challenges facing economists, health services planners, and policy experts (Edwards and Lawrence 2021; Hutchinson et al. (2019); Leck et al. 2016; Yates and Marra (2017); Gibson et al. (2011). SROI is relevant to pandemic scenarios.

WHO decision framework for social and movement measures and economics

The aim of public health and social measures during COVID-19 was to limit the virus spread and reduce deaths. Public health and social measures can be implemented together. It is difficult to measure their individual impact. The WHO’s decision framework for calibrating social and movement measures during the COVID-19 pandemic (WHO 2020) can assist in addressing societal impact and can be used for economic evaluation studies. It can be helpful when undertaking an analysis using CEA, CBA, SROI, and SWF.

There is evidence that social, physical distancing, and international travel-related measures which WHO calls social and movement measures can decrease face-to-face interactions/movement, reduce pressure on health services, and protect the most vulnerable. There can be unintended consequences if this disrupts access to care and delays diagnosis, treatment, and impacts on mental health and behavioral risk factors. The measures can also exacerbate economic slowdown, socio-economic inequality, harm workers unable to telework and those with precarious employment conditions and limited social protection (WHO 2020).

Complex decisions are required to sustain lives, livelihoods, and protect the vulnerable. A delay in calibrating social and movement measures during widespread transmission and risk of overwhelmed health services could increase morbidity and mortality, and the need to sustain stringent measures for longer.

Further, easing social and movement measures too quickly can jeopardize health and economic recovery. Strong, sustained policies mitigating harmful economic consequences of COVID-19 are necessary to support workers and businesses. Health, economic, and social welfare may be valued differently in different settings. It is challenging to collect context-specific evidence on multiple dimensions in a rapidly evolving situation. A five-step framework to support decision-making can assist in this process (WHO 2020). This framework includes the following steps which can facilitate comprehensive economic evaluations.

  1. 1.

    Assess the situational level and optimize health system response.

  2. 2.

    Identify possible social and movement measures for each context and possible calibration options and assess their health impacts.

  3. 3.

    Develop and populate an “Extended Assessment Matrix” of important health and non-health dimensions including implementation costs, economic cost, equity impact, and political barriers.

  4. 4.

    Establish a dialogue and a decision-making process.

  5. 5.

    Monitor, adapt, and communicate regularly throughout steps 1–4 (WHO 2020).

Step 3 above involves developing and populating an “extended assessment matrix” which involves WHO’s social and movement measures and their impact on health and non-health dimensions (WHO 2020). This framework can facilitate broad economic evaluation studies and could assist in conceptualizing the broad framework around analyses in CEA, CBA, SROI, and SWF. Such analyses can involve extensive collaboration with stakeholders, facilitated through steps 1 and 2, the extended assessment matrix in step 3, dialogue and decision making for steps 4, and communication for step 5. WHO (2020) indicates that the extended matrix in step 3 includes situation levels 3 and 4. For each situation level there are subclasses. Level 3 involves community transmission with limited capacity to respond and risk of overwhelming the health system. The subclasses are partial closure of businesses, school, e-learning, gathering size limitation, and no additional measures. A matrix analysis of health and non-health dimension for each sub-class can define the impact as low, medium, and high. The health dimensions impacts are COVID-19, non-COVID-19, and health system. The non-health dimension impacts are implementation cost, economic cost, equity impact, and political barriers. The same matrix framework is applied to level 4, involving uncontrolled pandemic requiring extensive measures to avoid overwhelming the health services. Subclasses include complete closure of businesses and institutions, prohibiting gatherings, and no additional measures. The abovementioned health and non-health dimensions are also applied to level 4 and with impact defined as low, medium, or high (WHO 2020).

Multi Criteria Decision Analysis (MCDA)

MCDA for horizon scanning of health innovations: HTA during pandemics

Horizon scanning involves the systematic identification of new, emerging, or obsolete technologies impacting health and society. Brief assessments can be undertaken using MCDA. This supports a full Health Technology Assessment (HTA) for innovative, potentially affordable technologies. Robust assessments and transparency principles without conflicts of interest are central. They can be used to rapidly identify technologies and interventions such as diagnostic, therapeutic, vaccines, and technology innovation. They assist in containing risks and ensuring high effectiveness, safety, ethics, with economical outcomes (Ruggeri et al. 2020).

MCDA can assist governments in decision making in supporting new models of care dependent on new technologies such as vaccines, medicines, diagnostic tools, and contact tracing systems. A tool for the early medical technology assessment using MCDA for horizon scanning was developed and used in Italy by the National Centres for HTA and Innovative Technologies. In the model, each HTA domain is attributed a score reflecting pros and cons along with opportunities and threats. Scores, which are weighted according to different perspectives, are plotted on a Cartesian graph, and positioned according to the potential value and perceived risk. Results are included in a table with a matrix of potential recommended outcomes such as that shown in Table 2 (Ruggeri et al 2020).

Table 2 Technology positioning zones and recommendations
Full size table

The approach can be demonstrated using a case study on the early assessment of a contact tracking system App. A brief assessment was undertaken using MCDA. A Panel involving medical, health, economics, statistician medical engineer and IT experts reported their views on the App, which were included on a matrix by scoring the balance between strengths and limits, and threats and opportunities. This included perceptions of effectiveness, safety, economic, legal, social, and ethics for each HTA domain. Scores on the Likert scale ranged from 1 (min added value or min risk) to 7 (max added value or max risk). Total score assigned to the value and risk was the sum of scores assigned to each domain of the HTA. Total scores were weighted using MCDA. There were three perspectives used, including health, decision-makers, and citizens/patients. The weights system was derived from the literature. The weights were varied using Monte Carlo simulation assuming 1000 scenarios for each perspective. Overall values and risk weights were included on a scatter plot graph. The ratio of risk/value was placed in one of four areas in the graph and Table 2. The recommendation was for full HTA and was compliant with Core Model of European Network for Health Technology Assessment (EuNetHTA) (Ruggeri et al. 2020). The recommendation was a consequence of the positioning in the comfort zone for low risk and high value. This scenario is considered a “no negative scenario.”

During a pandemic, MCDA can facilitate vaccine prioritization, equitable health access (Roy and Kar 2022), and technology evaluation.

The present systematic review has some potential limitations. The results were limited to articles published in English, which represents a potential limitation. The model structures, sources of information, and time horizons in the full economic evaluations varied across studies, and consequently, it was difficult to generalize the results of a study to other settings. Most full economic evaluation studies were conducted in North America and the United Kingdom, contributing to 60% of the studies included. Only 13% of studies concerned Asia and another 13% related to Africa. Another 7% concerned 139 low-and middle-income countries. An additional 7% of studies were from Europe. Cost of illness studies, conference abstracts, newspaper articles, and dissertations were excluded, and some relevant information might be omitted as a result.

Conclusion

The studies using cost utility analysis and cost benefit analysis, and analyzing mortality, morbidity, QALY gained, national income loss, and value of production approaches have effectively evaluated government policies to mitigate or suppress COVID-19 transmission, disease, and national income loss. Stakeholder engagement and the quality and relevance of economic evaluations could be improved by using the WHO’s pandemic decision-making and economics framework, social return on investment methods, and multi-criteria decision analysis. The WHO’s model facilitates economic evaluations of social and movement restrictions. SROI links benefits to health and broader social improvements. MCDA facilitates vaccine-prioritization, equitable health-access, and technology evaluation.

Unlike CEA, CBA can incorporate differential socioeconomic benefits across different units of health through individual WTP measures. However, WTP depends on ability-to-pay and can inflate benefits accruing to rich relative to less wealthy counterparts. Social welfare function can account for social inequalities and population-wide policy impact. It is a generalization of CBA and operationally, it is equal to an equity-weighted CBA. Welfarist frameworks, evaluating social value of mortality-risk, such as value of a statistical life are often used by governments for regulatory decisions.

Economic evaluations of health system innovations and new models of care during the pandemic have involved high quality studies using CEA and CUA. Cost effectiveness analyses utilizing decision tree and Monte Carlo models have been used to evaluate health system innovations. Cost utility analyses have effectively utilized decision trees and Markov models to evaluate new models of care.

The entire range of foregoing methodologies are instructive for government decision-making internationally, in addition to their current use of value of a statistical life and CBA in regulatory decisions. Social welfare function is noteworthy as it can represent prospective patterns of collective choice as to alternative social states. It can provide governments with a guideline for achieving the optimal distribution of income, which is vital during pandemics.