FormalPara Key Points for Decision Makers
Table 1
Full size table

1 Introduction

The pledge to attain sustainable development goals (SDGs) has compelled low- and middle-income countries (LMICs) to strengthen their health systems rigorously [1]. An integral part of strengthening the health system is ensuring patients’ access to high-value innovative health technologies, including novel pharmaceuticals. Adopting an innovation may cause an increase in healthcare expenditure [2, 3], which can be a significant issue for LMICs.

There are various tools that decision makers could use to manage the risk of adopting a new health technology, including health technology assessment (HTA) [2]. The World Health Organization (WHO) defined HTA as “the systematic evaluation of properties, effects and/or impacts of health technologies and interventions,” which includes intended and unintended consequences [4]. HTAs may involve consideration of the cost and outcome of implementing a certain health technology, usually using an economic evaluation that compares the cost and outcome of two or more alternatives of health technologies.

Unfortunately, the full potential use of HTA, particularly economic evaluations in LMICs, is hampered by various issues including the low availability and quality of local data leading to uncertainties surrounding the results of the evaluation [5, 6]. This situation creates a concern that the results may falsely inform decisions in LMICs and prevent patients’ access to high-value health technologies or facilitate access to costly but ineffective health technologies. A case of inaccurate information was found in the initial economic evaluation of a novel tuberculosis (TB) diagnostic tool, Xpert/MTB.RIF (Xpert), which informed the decision to implement the tool in South Africa rapidly. The initial economic evaluation, performed with limited evidence, concluded that the new technology would be cost effective [7]. However, a pragmatic trial showed that Xpert did not improve the cost effectiveness of TB diagnosis due to unforeseen constraints, such as the high level of diagnosis loss-to-follow-up, and the low rate of follow-up tests for negative results [8].

The nature of economic evaluations for new health technologies in LMICs, with limited evidence and abundant uncertainties, has similarities to the situation in high-income countries where early HTAs are performed for new medical products in development [9]. The early HTAs not only assess the potential cost effectiveness of new technologies but also aim to identify and manage uncertainties due to information gaps and illustrate the impact of this uncertainty on the cost and outcome measures [9]. By exploring beyond the primary cost-effectiveness conclusion, early HTAs can inform how to optimize resource allocation for the technology, including how to manage risks and implementation barriers. Thus, the approaches in early HTAs might be beneficial when implemented for new health technologies in LMICs.

New short-course regimens for the management of drug-sensitive and drug-resistant TB are developed to avoid the negative outcome of the long period of treatment with the current anti-TB regimen [10, 11]. The intervention is a new technology with limited evidence, and therefore a good case study for assessing economic evaluations for new technologies in LMICs. A WHO-endorsed new short-course regimen for the treatment of drug-resistant TB has been evaluated in a clinical trial (STREAM 1), with an inconclusive result regarding the clinical non-inferiority of the new regimen compared with the current regimen [12]. Several short-course regimens containing a new anti-TB, bedaquiline, for drug-sensitive and drug-resistant TB treatment, are currently still in phase III of a clinical trial [13].

The urgency of curtailing TB prevalence and achieving TB control goals have pushed decision makers in LMICs to make decisions on implementing a new anti-TB regimen based on results of economic evaluations performed with limited evidence. There is little knowledge about whether the economic evaluations of the new regimens addressed uncertainties caused by the limited evidence as rigorously as in the early HTAs and could produce useful information for decision making. Thus, we performed a systematic review on economic evaluations of implementing the new short-course regimens for drug-sensitive and drug-resistant TB, to explore how uncertainties in the studies were dealt with and to identify useful information for decision making from the studies.

2 Method

2.1 Literature Search

One author (TIAP) performed a literature review to identify economic evaluations of novel short-course regimens (SCRs) published up until September 2018 in four databases, PubMed, EMBASE, NHSEED, and CEA registry, using the search strings detailed in the Electronic Supplementary Materials, and screened the studies using the criteria given in Table 1. We did not restrict the choice of economic evaluation study design, which included trial, real-world data, simulation, or decision-model-based design. We excluded studies that only reported their economic evaluation methods and results partially, such as study protocols, conference abstracts, editorials, comments, and qualitative studies. We did not extend our search to grey literature. To validate the screening process, another author (SMS) performed an independent search and screening in one of the databases in which most of the studies were found—PubMed. The agreement rate between the two authors on the study inclusion reached 75%, and differences could be resolved through deliberation. We did not validate the screening process in other databases since we assumed that the agreement rate would be similar.

Table 1 Inclusion and exclusion criteria
Full size table

2.2 Data Extraction

One author (TIAP) extracted the general information from the studies, mainly to obtain an overview on the methodological approach, the intervention and comparator, and the study setting. Furthermore, gaps in the information on cost and clinical evidence that could lead to uncertainties were extracted. These gaps could mainly be identified through the assumptions taken in the studies since assumptions are often included in economic evaluations to overcome the information gaps.

The author extracted information on how uncertainties were dealt with, comparing them with the approaches used in early HTAs. In early HTAs, sensitivity analyses are applied rigorously to address the uncertainties upfront, including a deterministic sensitivity analysis that can inform the main drivers of the cost effectiveness, and a probabilistic sensitivity analysis (PSA) that shows the likelihood of a health technology to be cost effective given various thresholds [9, 14]. The PSA informs decision makers on the scale of uncertainties surrounding the cost effectiveness estimates [15]. The sensitivity analyses in early HTAs also take into account the health system dynamics and patient pathways to include pragmatic aspects that may be encountered during the real-world implementation [9].

Information that could be beneficial for decision making was also extracted and analyzed. This includes not only the cost-effectiveness summary [e.g. incremental cost-effectiveness ratio (ICER)] and conclusion, but also additional information that can help optimize the implementation of the new SCRs, such as barriers to implementing the SCRs. This information could notify decision makers regarding additional efforts required to ensure the cost effectiveness of implementing the intervention. Other useful information relates to the main drivers of the cost effectiveness, which raises the awareness regarding factors that can alter the cost effectiveness of implementing the intervention.

2.3 Quality Appraisal

The quality of the included studies was assessed prior to the data extraction with the help of the Consensus Health Economic Criteria (CHEC) checklist [16]. We used the checklist since its assessment questions are explicit and straightforward. Furthermore, it is suitable to appraise full economic evaluation studies, where both cost and health outcome are considered. The checklist was developed as a minimum set of items describing the quality of economic evaluation studies [16]; therefore, it is not methodologically rigid and applicable to studies with different methodological approaches.

The quality appraisal was performed independently by two authors (SMS and TIAP). Any disagreement between the two authors was resolved through a discussion. A third author (AH) was involved when the disagreement could not be resolved.

3 Result

After removal of duplications, the search identified 322 articles. Based on the abstract, 313 studies were excluded for reasons detailed in Fig. 1. Further exclusion based on the full text was performed on five other studies. A total of four studies fulfilled the inclusion criteria and were included [17,18,19,20].

Fig. 1
figure 1

Summary of the literature search in the electronic databases

Full size image

3.1 Quality Appraisal

The studies fulfilled most of the quality criteria in the appraisal checklist. The complete appraisal result of these studies can be found in the Electronic Supplementary Material. Some studies only partially addressed uncertainty such as through scenario analyses [19, 20]. However, one study using a complex model mentioned that sensitivity analyses other than a deterministic analysis and scenario analyses would be computationally expensive [19].

3.2 General Characteristics of the Studies

Three of the included studies were model-based economic evaluations [17,18,19], while one was an empirical study based on publicly available and local data [20]. Two of the model-based studies used a decision analytic model, which did not take into account TB transmission process (i.e., a static model) [17, 18]. One study did not specify the type of static model used [17], while another used a decision tree model [18]. The other study used a model that considers the transmission process; i.e., an individual-based dynamic transmission model (dynamic model) [19]. The model-based studies analyzed the cost effectiveness of adopting a hypothetical new first-line SCR that was given in 4 months compared with the standard, 6-month, first-line regimen. These studies expressed the health outcome of implementing the new first-line SCR in disability-adjusted life-years (DALY) averted [17,18,19].

The empirical study analyzed the cost effectiveness of long-course regimens (LCRs, 20 months or more) and SCRs (9–12 months) containing bedaquiline compared with the regimens containing injectable drugs for drug-resistant TB treatment [20]. This study expressed the health outcome of implementing the regimens in the number of successful treatments [20].

3.3 Information Gaps

There was no clinical evidence for the hypothetical first-line SCR analyzed in the model-based studies. Hence, the studies took various assumptions, which were consistent in all studies. The studies assumed that the novel SCR was non-inferior and non-superior to the current regimen. The same risk of mortality during treatment and probability of cure upon treatment completion were applied for both regimens. They also assumed that the default or loss-to-follow-up rate per month during treatment was the same for both regimens. Hence, the studies only analyzed the impact of shortening the duration of treatment [19]. There was also no information about the potential price of the new regimen; hence, the studies performed a price-threshold analysis with 1 USD per day as a baseline price. The study by Knight et al. did not assume a baseline price for the new regimen; however, it looked at the price that resulted in a cost per DALY averted equal to the gross domestic product (GDP) per capita of the study setting, that is, South Africa [19]. The relatively high GDP per capita of South Africa (compared with most of the settings of the other studies) resulted in a regimen price higher than 1 USD per day [19].

A long-term follow-up of a clinical trial showed different levels of cure following patients’ default from treatment [18]. Based on this information, the model-based studies assumed a possibility of cure after treatment default; depending on how long the individuals had been on treatment before defaulting. They assumed that early defaulters who received the novel first-line SCR would have a higher probability for cure compared with those who received the standard regimen.

In the empirical study, bedaquiline-based regimens were assumed to be more effective than the injectable-based regimens since they shortened the time to sputum smear conversion (an indication of treatment success and reduction of transmission risk). The efficacy of injectable-based SCR was taken from the STREAM 1 clinical trial, while the efficacy of injectable-based LCR referred to the WHO-reported global treatment success rate for multi-drug resistant (MDR) TB [20]. For bedaquiline-based LCR, the efficacy was taken from a meta-analysis, and for bedaquiline-based SCR, it was taken from a modeling study since there was no information from clinical studies [20].

3.4 Dealing with Uncertainties

All model-based studies performed a deterministic sensitivity analysis. The deterministic analysis was mainly performed on parameters that were determined with assumptions, such as default rate, as well as cost parameters, such as treatment delivery costs (i.e., treatment costs excluding the drug regimen costs).

Two of the model-based studies considered the health system dynamic by performing the analysis with different scenarios, i.e., a ‘perfect’ and ‘current’ scenario [17, 19]. The analysis with the ‘perfect’ scenario assumed that the adherence of healthcare providers and patients to the TB treatment guideline was high. The high adherence caused a high rate of TB care utilization, high treatment delivery costs, and low rate of treatment default. The analysis with the ‘current’ scenario applied the actual level of adherence to the TB treatment guideline in the study setting. With this scenario, the rate of TB care utilization and the treatment delivery costs were lower, while the treatment default rate was higher than the rate with the ‘perfect’ scenario. In one of the model-based studies, the analysis was also performed with another scenario, which assumed that TB could only be cured when the treatment was completed.

The empirical study also performed several scenario analyses. However, the implications of the different scenarios were limited to capacity utilization and regimens’ price. One scenario assumed a high prevalence of extensive drug-resistant TB, which increased the utilization of inpatient care and hospitalization costs. Another scenario assumed local procurement for the regimens, which decreased the price of injectable-based regimens.

PSA was only performed by one study, under the assumption that the regimen price was 1 USD per day, and the willingness-to-pay threshold (cost per DALY averted) was equivalent to the GDP per capita of the settings (see Table 2 for details on the study settings) [17]. Standard deviation and 95% uncertainty ranges of the ICER were estimated from the PSA (see Table 2). The uncertainty range covers the true ICER of implementing the new regimen. Thus, the wider the range, the higher the uncertainties surrounding the ICER. The threshold also helps in interpreting the range. Implementing the new regimen is not cost saving but cost effective when the ICER lies below the threshold.

Table 2 Cost effectiveness result from the model-based studies
Full size table

3.5 Information for Decision Making

3.5.1 Cost-Effectiveness Outcomes

The results from the three model-based studies were summarized in Table 2. The studies with static models showed that the new SCR for drug-sensitive TB would be cost saving in most settings [17, 18]. Despite causing an increase in drug acquisition costs, the new first-line SCR decreased the treatment delivery costs (i.e., treatment costs excluding the drug regimen costs) due to the short period of treatment. The first-line SCR was not cost saving but still cost effective in settings with low treatment delivery costs. However, the novel first-line SCR was not cost effective at a price of 1 USD per day in Bangladesh due to the significantly low treatment delivery costs in the country [17]. The study that used a dynamic model showed the small impact of the new first-line SCR on TB prevalence due to its limited effect on the TB transmission process [19].

The empirical study showed that the cost per treatment success for MDR TB with injectable-based SCR was lower compared with the cost with injectable-based LCR (around 30% lower) [20]. The use of injectable-based SCR instead of LCR mainly reduced the drug acquisition and hospitalization costs. Switching to a bedaquiline-based regimen further reduced the hospitalization costs for SCR and LCR (Fig. 2). It also eliminated the costs of injectable-related adverse event management for both regimens. Overall, switching to bedaquiline-based SCR and LCR was cost saving. However, the cost saving was higher for LCR than for SCR in all settings.

Fig. 2
figure 2

Reduction in health system cost per treatment success (without the costs for ototoxicity management) due to the use of bedaquiline-based regimens compared with injectable-based regimens. AE adverse event

Full size image

3.5.2 Results of the Deterministic Sensitivity and Scenario Analysis

The deterministic sensitivity analysis of the three model-based studies identified the main drivers for the cost-effectiveness outcomes of the new first-line SCR, that is, treatment default rate and the treatment delivery costs. The health benefit of implementing new first-line SCR was large when the default rate was high. Implementing the new first-line SCR could result in a significant cost saving when the treatment delivery cost was high. Hence, the price threshold for the hypothetical first-line SCR could be high if the treatment delivery costs were high.

One study showed that, with a ‘perfect’ scenario, implementing the new first-line SCR instead of the current regimen would not result in any additional health benefit [19]. However, another study showed that implementing the new first-line SCR resulted in high cost saving due to the high treatment delivery costs with the ‘perfect’ scenario [17]. The cost saving with the ‘current’ scenario was lower compared with the saving with the ‘perfect’ scenario [17]. One study also showed that assuming no cure for patients who defaulted from TB treatment slightly increased the cost effectiveness of the new first-line SCR [19].

In all scenarios, bedaquiline-based regimens for MDR TB treatment significantly reduced hospitalization costs and eliminated the costs to manage injectable-related adverse events. Thus, these regimens were always cost saving despite a lower cost for the injectable drugs, and higher utilization rate of TB care.

3.5.3 Results of the Probabilistic Sensitivity Analysis (PSA)

The PSA performed in one model-based study showed that the likelihood of the new first-line SCR being cost saving or cost effective was higher in settings with a relatively strong health system (i.e., a system with high adherence to TB treatment guidelines, high TB care utilization rate, and high treatment delivery cost), compared with those with a weak health system (i.e., a system with low adherence to TB treatment guidelines, low TB care utilization rate, and low treatment delivery cost). The PSA result in Table 2 showed that the 95% uncertainty range was wider when the analysis was performed with the ‘perfect’ scenario than with the ‘current’ scenario.

4 Discussion

Most economic evaluations of new shortened regimens for drug-sensitive and drug-resistant TB took a similar approach to the early HTAs by thoroughly analyzing the uncertainties upfront, including their impact on the cost-effectiveness outcomes. Thus, the studies could provide substantial information for decision making, including the potential cost effectiveness of implementing the SCRs, the influence of health system characteristics on the cost effectiveness, and the main drivers of the cost and cost effectiveness.

The analysis with different scenarios of health system characteristics (i.e., with the ‘perfect’ and ‘current’ scenarios) showed a potential barrier that could make the implementation of the new first-line SCR not cost effective, that is, a weak health system with a low utilization rate of TB care. Thus, this barrier might need to be addressed prior to implementing the new regimen. This finding also showed the benefit of including pragmatic scenarios to identify potential barriers to implementing an intervention, which can be caused by the organization of the health system, the providers and patients’ behaviors, and other context-specific factors [9, 17, 21].

The information on the main drivers of cost effectiveness could help identify important information gaps to fill. The clinical efficacy of the new first-line SCR should be established; however, the treatment default rate and delivery costs in the study setting should also be carefully determined. An information gap that was barely considered in the studies was the timing of the treatment default. If treatment default happens at a higher rate during the last months of the standard treatment (the 5th or 6th month, when symptoms are no longer apparent) [18], shortening the regimen to 4 months would prevent most of the default cases. Unfortunately, the evidence regarding the timing of default is inconclusive since different timings of treatment default were reported [22,23,24]. Furthermore, the STREAM 1 clinical trial results showed that shortening the treatment period did not affect the retention of patients on treatment [12]. However, this finding could be caused by the rigorous monitoring performed in the trial. Other information that needed confirmation was about cure from TB following treatment default. These information gaps can also be quantified through value-of-information analysis to further prioritize additional research required to fill them [25].

The inadequate sensitivity analyses performed in the economic evaluation of the new regimens for MDR TB resulted in limited information for decision making. The analyses did not address the assumptions on the efficacy of the regimens for MDR TB treatment although there were uncertainties surrounding the efficacy estimates. The STREAM 1 trial could not confirm the assumed non-inferiority of the new injectable SCR compared with the conventional injectable LCR for MDR TB treatment [12]. The study also assumed that the bedaquiline-based regimens did not contain any injectable drugs. However, the efficacy estimate of the bedaquiline-based regimen was taken from clinical trials that tested the addition of bedaquiline to a background regimen containing injectable drugs [26].

The scenario analyses performed in the empirical study could only inform decision makers on the cost drivers of implementing the bedaquiline-based regimens. As observed in other studies, the use of bedaquiline mainly reduced the hospitalization costs due to the rapid culture conversion that allowed early discharge from inpatient care [27,28,29]. The study also showed significant cost saving due the elimination of costs for the management of injectable-related adverse events. Unfortunately, the study was not clear on the inclusion of adverse event management related to bedaquiline. Although bedaquiline is well tolerated, a serious adverse event of cardiotoxicity has been reported, which caused a requirement for cardiac function monitoring during treatment [30, 31].

There could be other barriers to implementing the new regimens for the treatment of MDR TB. The benefit of new regimens for MDR TB treatment was highly dependent on the long-term efficacy of the regimen, the regimen’s ability to save resources to expand access, and the level of resistance to the second-line drugs in the population [32]. The new regimens could also have less benefit in settings with a low rate of rapid and accurate MDR TB diagnosis due to the limited access to molecular diagnosis and drug sensitivity testing.

The PSA performed in one of the included studies showed that decision makers from countries with high utilization of TB care and treatment delivery costs, such as Brazil and South Africa, could be more confident in adopting a new first-line SCR compared with those from countries with low utilization rates of TB care and treatment delivery costs, such as Tanzania and Bangladesh. The wide range of uncertainties with a ‘perfect’ scenario, which assumed a high rate of adherence to TB treatment guidelines, showed the paucity in long-term data, since the scenario was highly hypothetical. One benefit of PSA that was not thoroughly explored in the study was informing the value of additional information [9, 15].

Considering the importance of addressing uncertainties in economic evaluations for new technologies with significant information gaps, the sensitivity analysis should be carefully planned and include potential constraints of implementing the technology. Recently, a conceptual framework was developed to empirically identify potential constrains of implementing a new technology in LMICs, which could be a guide to explore uncertainties beyond those caused by the choice of parameters [21]. This process should involve stakeholders with various fields of expertise to ensure optimal identification of the potential constraints and scenarios where and when implementing the technology could be harmful [21].

This review may have several limitations such as the potential exclusion of relevant studies due to the design of the search strings. A systematic review of studies addressing the cost and cost effectiveness of MDR TB treatment found a similar number of studies as in this review, that is, four included studies out of 420 studies [33]. The low number of studies found in the previous and current review reflected the low rate of new anti-TB regimen development since the 1960s [34]. The omission of grey literature could have also excluded relevant studies; However, despite including grey literature, the previous review found only a few studies [33]. Many LMICs concerned with TB have not yet fully implemented HTA and economic evaluations [5, 35, 36]. The low rate of studies conducted could limit the number of reports published in the grey literature. Despite the small number of the included studies, this review was able to confirm the importance of addressing uncertainties surrounding the economic evaluation’s estimates to inform decision making in LMICs, which was also indicated in other studies [21, 37]. Most of the included studies also showed consistent cost-effectiveness findings.

We did not perform a meta-analysis because of the high heterogeneity between the studies and the limited use of a single cost-effectiveness summary estimate. In addition, we understand that the CHEC checklist used in this review is developed for systematic reviews that include trial-based economic evaluations. Although the included studies used different methodological approaches, we chose to use only one checklist to ensure comparability of the appraisal outcome between the studies.

Other economic evaluations of different new health technologies in LMICs might not use the approach identified in this review. However, we believe that the management of uncertainties similar to the early HTA approach is useful for all new technologies in LMICs and should be used in future studies.

In conclusion, we found that by using approaches similar to early HTAs, economic evaluations for new anti-TB regimens in LMICs could provide useful information for decision makers despite the limited evidence. Using this approach, the economic evaluations not only showed the impact of the intervention on health outcome and cost, but also generated important information to manage risks and implementation barriers including information about the influence of health-system characteristics on the cost effectiveness, and the main drivers of the cost and cost-effectiveness outcomes.