Malaria and Economic Evaluation Methods: Challenges and Opportunities
- 895 Downloads
There is a growing evidence base on the cost effectiveness of malaria interventions. However, certain characteristics of malaria decision problems present a challenge to the application of healthcare economic evaluation methods. This paper identifies five such challenges. The complexities of (i) declining incidence and cost effectiveness in the context of an elimination campaign; (ii) international aid and its effect on resource constraints; and (iii) supranational priority setting, all affect how health economists might use a cost-effectiveness threshold. Consensus and guidance on how to determine and interpret cost-effectiveness thresholds in the context of internationally financed elimination campaigns is greatly needed. (iv) Malaria interventions are often complimentary and evaluations may need to construct intervention bundles to represent relevant policy positions as sets of mutually exclusive alternatives. (v) Geographic targeting is a key aspect of malaria policy making that is only beginning to be addressed in economic evaluations. An approach to budget-based geographic resource allocation is described in an accompanying paper in this issue and addresses some of these methodological challenges.
Key Points for Decision Makers
The use of cost-effectiveness thresholds is complicated by the disease elimination objective and international aid.
To be relevant to key policy questions, economic evaluations may need to evaluate compatible (rather than mutually exclusive) interventions and provide information to support geographic targeting.
Total annual spending on malaria has grown to over US$ 2.6 billion globally . The current availability of resources represents an arguably unprecedented opportunity to reduce the global public health burden of malaria and perhaps eradicate the disease altogether. Promising gains have been made, with global malaria-associated mortality decreasing by 60% between 2000 and 2015 (down to an estimated 438,000 deaths) . It is important to continue to make the most of the currently available resources by spending in a way that maximises impact.
Healthcare economic evaluation can offer useful information to support this goal by appraising the balance of costs and consequences of technologies or services in different contexts. A recent review by Gray and Wilkinson describes the development of economic evaluation methods  including two key initiatives for priority setting in low and middle income countries (LMICs) (i) the World Health Organisation’s (WHO) Generalised Cost Effectiveness Analysis (GCEA)  and (ii) the more recent reference case, an initiative to support methodological standardisation backed by the Bill and Melinda Gates Foundation .
Scale up vector control (typically through long-lasting insecticide treated bed nets).
Scale up access to malaria diagnosis and treatment.
Forms of mass drug administration (including mass screening and treatment).
Active case follow-up.
Deployment of an emergent malaria vaccine.
The malaria economic evaluation evidence base to guide policy makers is substantial and growing. In 2011 a systematic review of the cost and cost effectiveness of malaria control interventions identified 43 economic evaluations published between 2000 and 2010 . A broader review of economic evaluation in LMICs published in 2015 found that malaria comprised 20% (n = 41) of the published literature between 2000 and 2013 . The most recent and broadest review identifies a further 29 malaria economic evaluations published between 2012 and May 2014, representing a 13% share of the literature across all disease and income settings . Since different time periods are covered by the 2011 and 2016 reviews, there are at least 72 malaria economic evaluations published to date, 40% of which have been published since 2012. However, there are some important idiosyncrasies; this paper identifies some common characteristics of malaria decision problems that affect both the application of economic evaluation methods and interpretation of results.
2 Elimination: Intervention Cost-Effectiveness Decreases with Incidence (Or Appears To)
In the context of disease elimination it is possible to draw a distinction between two types of economic evaluation; (i) evaluation of the policy of elimination (or eradication ); and (ii) evaluation of the component interventions required to achieve elimination.
Economic evaluation of malaria or other elimination campaigns are relatively uncommon; a systematic review in 2015 identified 43 economic analyses, though many are not economic evaluations1 . Such evaluations are methodologically challenging due to the complexity of elimination campaigns and expected impacts. In malaria the expected benefits of elimination are not only to population health but various other sectors including (though evidence is mixed) education [11, 12], tourism , and economic productivity [14, 15]. Cost benefit or cost consequence analysis  can be used to appraise the multi-sectoral impacts of elimination, though concrete results may remain elusive. To capture the full costs and benefits of an elimination campaign, the evaluation time horizon must extend some years beyond the expected date of elimination. Such analysis inherently entails a great degree of uncertainty, in particular due to influence of secular trends that can play a decisive role in driving transmission. It is very difficult to anticipate the impact of economic development, deforestation or climate change on malaria, and such factors could play important roles over the course of 10 or 20 years. There is also a risk of failure, either through never reaching elimination  or from a post-elimination resurgence of transmission .
If the campaign is successful and incidence declines, so too (almost paradoxically) does the apparent cost effectiveness of the interventions. At some point on the decline in transmission the relevant incremental cost-effectiveness ratios (ICERs) will no longer fall under the appropriate cost-effectiveness threshold (CET) (see Box 1 and Box 2). Prima facie, it is not possible to achieve malaria elimination without investing in cost ineffective interventions because the benefits of investing in the late phases of malaria elimination lie in the difficult-to-define post-elimination period.
Cost-effectiveness priority setting: a brief summary
Comprehensive descriptions of economic evaluation methods can be found elsewhere [9, 21, 22]. The most common frameworks, cost-effectiveness or cost utility analysis, typically include the following steps:
• Define a decision problem in terms of a set of alternative healthcare technologies or services (interventions)
• Measure or model intervention costs and effects
• Express the differences between competing interventions on the cost effectiveness frontiera as incremental cost effectiveness ratios (ICERs)b
• Use a cost-effectiveness threshold (CET) to define the intervention with the highest ICER below the CET as the optimal choice
The cost-effectiveness threshold (CET) is considered to represent the willingness-and-ability-to-pay of the Ministry of Health or society in general, and would ideally be set at such a level that it reasonably reflects real budget constraints. In theory, the CET is equal to the opportunity cost of alternative public healthcare spending. Defining an accurate CET is a challenge and it is often indexed to the national gross domestic product (GDP) per capita. In low and middle income countries (LMICs), thresholds of 1× and 3× GDP per capita are commonly used , though a recent study suggests a lower threshold may be more appropriate . An earlier review by Shillcutt et al. summarises the methodological debate in defining CETs in LMICs 
3 International Aid: Available Resources Exceed Local Funding Constraints
The purpose of international aid is to overcome the severe resource constraints faced by LMICs. While there is a longstanding debate surrounding so-called vertical aid [27, 28], earmarked disease-specific funding continues to facilitate the implementation of malaria programmes that otherwise would not have been affordable. As outlined in Box 2, a CET is used to represent the resource constraints of the society or healthcare provider. Regardless of the CET chosen, on introducing a non-negligible sum of aid the CET would, in theory, need to be raised to reflect the increase in healthcare services now affordable to this society. It may be that across the health sector total aid inflows are not significant; however, in many countries there is substantial additional financing directed specifically towards malaria control and elimination. In these contexts a CET that accurately reflects governmental or societal constraints may be inaccurate in reflecting resource constraints in malaria planning.
Probabilistic analysis and cost-effectiveness acceptability curves enable analysts to consider a range of possible CETs. Nevertheless, again, consensus is needed on whether and how to adjust an expected health-sector CET where aid affects affordability of malaria control and elimination programmes.
4 International Aid: Supranational Priority Setting
Economic evaluation typically aims to address allocative efficiency within a nation state. However, before international aid arrives in-country to be allocated to malaria interventions, a series of decisions have already been made regarding which disease areas and which countries to prioritise. No formal economic analysis of supranational priority setting of malaria funds has been published to our knowledge.
There are several important factors that affect the interpretation of cost-effectiveness evidence for supranational priority setting. As outlined in Box 2, standard decision rules (including GDP indexed CETs) are intended to reflect the resource constraints of a particular country. Therefore, conclusions about whether an intervention is cost effective (or not) to a large degree reflect the affordability of the intervention in that context as much as the efficiency with which investment may be converted to health gains. For example, a multi-country study of pandemic preparedness found that, in general, stockpiling antivirals is not cost-effective for LMICs but may be cost effective in high-income countries . As the authors note, the use of GDP based CETs means this conclusion reflects local affordability and does not reflect where investment would yield the most health gains. The reverse would likely be true . Decision makers and analysts should therefore not use GDP-based CETs for between-country priority setting. Notably, recent work on decision rules for health system strengthening in the context of supranational priority setting focuses on budget allocation rather than threshold analysis . To an extent, GDP-based CETs may remain relevant to supranational priority setting in terms of allocative efficiency within the health sector that aid is delivered into. However regional malaria elimination or global eradication is a weakest link global public good, requiring effort from all affected countries for success . GDP-based CETs are a decision framework for optimal allocation of healthcare resources at the national level yet regional malaria elimination policy must consider more than simply the efficient allocation of resources from a country perspective.
Supranational priority setting perhaps implies a role for a common or global CET , matching the perspective of the funder. That is, health gains are valued equally in all countries and funds directed to where impact is greatest. A global CET might reflect supply-side2 constraints of international aid budgets or demand-side norms regarding the extent to which poorer countries should be supported to provide healthcare services that would otherwise be unaffordable. GiveWell, an advisory organisation for charity donors, is effectively applying a global CET of US$5000 per life saved in their assessments of philanthropic causes and organisations .
Ultimately, if malaria elimination is cost effective (and political commitments to achieve elimination suggests it is considered to be in the countries that have made them) then other factors such as the rate at which local health systems can efficiently absorb additional financing, the period for which this can be sustained, equity considerations and rises in domestic financing may drive international aid allocations.
5 Choosing Between Compatible Interventions
Use a base-case comparator scenario for all interventions.
Construct intervention bundles defined as mutually exclusive alternatives.
In the first approach, the costs, effects and cost-effectiveness ratios of intervention options are all expressed in comparison to a common baseline, most likely a null or “no additional intervention” scenario, which is similar in this respect to the generalised cost effectiveness analysis (GCEA) framework . These generalised cost-effectiveness ratios allow the reader to make a judgement about which interventions, in isolation, yield the greatest health gains per unit of investment. The cost-effectiveness rank could be applied to determining the order in which interventions are added to a package of services but may be less applicable when choosing one intervention instead of another (when an incremental cost-effectiveness ratio based on the difference in costs and effects between the two interventions is required). An additional limitation is that intervention interactions are not accounted for. This approach assumes that the costs and, in particular, effects of intervention A are unchanged by the presence of intervention B, which is rarely a tenable assumption for interventions addressing the same disease.
To incorporate both intervention compatibility and interactions, several malaria economic evaluations have constructed priority setting landscapes based on discrete packages of interventions [37, 38, 39]. Intervention bundles can then be treated as mutually exclusive alternatives and incremental analysis applied to assess the value of switching from one package to another. A limitation of this approach is that the intervention combinations are defined a priori and may restrict the decision space since it may not be possible to include all possible combinations. Input from decision makers into the design of intervention packages will improve the relevance of the evaluation to policy decisions. Evidence users should take care not to interpret the package with the lowest cost-effectiveness ratio as the optimal choice. This applies to all economic evaluation results but can be particularly tempting with evaluations of intervention bundles.
6 Geographic Targeting
The epidemiology of malaria varies considerably between different geographical areas. A core aspect of priority setting in malaria control and elimination is to determine which interventions, or combinations of interventions, are provided where.
There are several options to address geographic heterogeneity in economic evaluation. Stratification of results, sensitivity analysis or scenario analysis can be used to present the cost effectiveness of healthcare interventions in contexts with certain characteristics. Readers are then free to match their context of interest to the closest example given. In this approach the reader still has a degree of work to do in translating from illustrative examples to applied policy, particularly if targeting resources across a large number of geographical units such as districts or townships. Moreover, any distinction between heterogeneity and uncertainty in results may be lost. Calculation of costs, effects and cost effectiveness in all geographic units of interest would provide a decision maker with a fuller picture of the decision landscape. Decision rules or constraints can then be applied to yield a recommended geographic allocation of interventions according to a CET, or, given the limitations discussed here, direct allocation of a relevant budget. Two recent studies address the geographic allocation of resources in malaria planning [26, 37] but such examples are rare and methods are varied.
We identify five areas where the realities of malaria decision problems affect the application of economic evaluation methods. The complexities of (i) declining incidence and cost effectiveness in the context of elimination; (ii) international aid and resource constraints; and (iii) supranational priority setting, all affect the way in which health economists might use a CET. Guidance and consensus regarding best practice on when and how to use a CET for malaria economic evaluation, and the selection or calculation of this threshold, is greatly needed. Alternatively, a policy of malaria elimination may allow economic evaluation to focus on direct budget allocation within the elimination campaign. (iv) In order to properly inform priority setting in malaria policy, economic evaluations must be able to assess combinations of compatible interventions, rather than sets of mutually exclusive alternatives. Assessment of exhaustive sets of intervention combinations is not possible and communication with policy makers will be essential to construct relevant evaluation questions. (v) Geographic priority setting is a core element of planning in both the control and elimination of malaria. Economic evaluation can do more to support this priority setting by incorporating geographic heterogeneity into analyses and making a clear distinction between heterogeneity and uncertainty.
In an accompanying paper in this issue  we outline an approach to geographic resource allocation in malaria planning that addresses several of the challenges outlined in this paper.
Defined by Drummond et al. as “the comparative analysis of alternative courses of action in terms of their costs and consequences” .
Ochalek et al. introduce the concept of supply- and demand-side perspectives on the cost-effectiveness threshold broadly reflecting resource contraints and societal norms, respectively .
Both authors contributed to manuscript content. TD lead the manuscript drafting with YL making revisions.
Compliance with Ethical Standards
No specific ethical approval was sought for this opinion article.
Conflict of interest
Tom Drake and Yoel Lubell declare no conflicts of interest.
TD and YL are supported by a grant from the Bill and Melinda Gates Foundation. The Mahidol Oxford Tropical Medicine Research Unit is supported by the Wellcome Trust major overseas programme in South East Asia (Grant Number 106698/Z/14/Z).
- 1.WHO. World Malaria Report 2015. In: WHO. http://www.who.int/malaria/publications/world-malaria-report-2015/report/en/. Accessed 12 July 2016.
- 3.Tan-Torres Edejer T, Baltussen R, Adam T, Hutubessy R, Acharya A, Evans DB, Murray CJ, Organization WH. Making choices in health: who guide to cost effectiveness analysis. World Health Organization; 2003.Google Scholar
- 4.Wilkinson T, Chalkidou K, Claxton K, Revill P, Sculpher M, Briggs A, Teerawattananon Y, Rattanavipapong W. The Reference Case for Economic Evaluation. Bill and Melinda Gates Foundation & NICE International. 2014. http://www.idsihealth.org/knowledge_base/the-reference-case-for-economic-evaluation. Accessed 6 Jan 2017.
- 6.Santatiwongchai B, Chantarastapornchit V, Wilkinson T, Thiboonboon K, Rattanavipapong W, Walker DG, Chalkidou K, Teerawattananon Y. Methodological variation in economic evaluations conducted in low- and middle-income countries: information for reference case development. PLoS One. 2015. doi: 10.1371/journal.pone.0123853.PubMedPubMedCentralGoogle Scholar
- 9.Drummond MF, Sculpher MJ, Claxton K, Stoddart G, Torrance GW. Methods for the economic evaluation of health care programs. 4th ed. Oxford: Oxford University Press; 2015.Google Scholar
- 11.Clarke SE, Jukes MCH, Njagi JK, Khasakhala L, Cundill B, Otido J, Crudder C, Estambale BBA, Brooker S. Effect of intermittent preventive treatment of malaria on health and education in schoolchildren: a cluster-randomised, double-blind, placebo-controlled trial. Lancet. 2008;372:127–38.CrossRefPubMedPubMedCentralGoogle Scholar
- 21.Briggs A, Sculpher M, Claxton K. Decision modelling for health economic evaluation. 1st ed. Oxford: Oxford University Press; 2006.Google Scholar
- 22.Gray AM, Clarke PM, Wolstenholme JL, Wordsworth S. Applied methods of cost-effectiveness analysis in healthcare. Oxford: Oxford University Press; 2011.Google Scholar
- 23.WHO | WHO-CHOICE. http://www.who.int/choice/en/. Accessed 10 Apr 2014.
- 24.Ochalek JM, Lomas J, Claxton KP. Cost per DALY averted thresholds for low- and middle-income countries: evidence from cross country data. York; 2015. p. 1–50. (CHE Research Paper; 122).Google Scholar
- 33.GiveWell’s Cost-Effectiveness Analyses. In: GiveWell. http://www.givewell.org/how-we-work/our-criteria/cost-effectiveness/cost-effectiveness-models. Accessed 31 Oct 2016.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.