FormalPara Key Points for Decision Makers

The number of economic evaluations conducted in Ethiopia was limited.

Though some studies had poor quality, overall, economic evaluations conducted in Ethiopia were of good quality.

Scholars must conduct quality economic evaluations and urge policymakers to use the results of these studies.

1 Introduction

Healthcare resources are limited, making it challenging to provide adequate healthcare services to all patients with different health conditions, while health costs have also been increasing dramatically over recent years [1,2,3,4]. Due to this scarcity of resources, it is impossible to close the gap between available resources and the demand for healthcare simply by increasing healthcare budgets [3, 5, 6]. The cost of medical technology and pharmaceuticals is extremely high, which is why many countries have implemented health economic evaluation processes to guide the selection and use of such interventions [7,8,9]. There is a need for a method for evaluating options and allocating and utilizing available resources more effectively. During this critical period, economic evaluations play a key role in evaluating the cost effectiveness of alternative healthcare programs, services, or interventions [5, 9,10,11]. All countries, including developing countries, can benefit from health economic evaluation [10, 11].

Economic evaluations aim to provide an efficient allocation of scarce resources by identifying, measuring, valuing, and comparing the costs and consequences of alternative strategies. With the help of economic evaluations, we can make better decisions about health technology pricing, health insurance reimbursement, and formulary acceptance of newly innovated technologies [9, 11, 12]. Economic evaluations can reduce resource wastage and the irrational utilization of healthcare resources, thus helping to maximize net health benefits through effective resource allocation [1, 9].

Although evidence from economic evaluations is increasingly being used, the quality of published data remains suspect. Consequently, it is difficult to assess the validity of the conclusions drawn from such pieces of evidence [13]. Their usefulness for policy and decision making is determined by their methodological rigor, accuracy, and generalizability. Hence, critical appraisal is essential for assessing the reporting quality of studies, as poorly constructed economic evaluations may lead to incorrect decisions [12,13,14]. Quality and well-designed economic evaluations are essential for enhancing decision making. Therefore, reviewing and appraising the already published economic evaluations is one method for improving study quality [5]. Various tools are available for evaluating the quality of health economic evaluations, with the Quality of Health Economics Studies (QHES) scale and Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist being the most widely accepted and used [12, 15, 16]. Therefore, this systematic review was conducted using the QHES and CHEERS instruments to evaluate the quality of published economic evaluations in Ethiopia.

2 Methodology

2.1 Study Search Strategies

To identify studies conducted in Ethiopia, a PRISMA protocol-compliant search of studies published until April 2022 was conducted. We searched electronic databases such as PubMed, Cochrane, EMBASE, Medline, Web of Science, and Africa Journals OnLine to find relevant studies. Additionally, gray literature was also carefully searched from various websites, including university repositories, key government websites, NGOs, ResearchGate, and Google Scholar. Moreover, the reference lists of previously identified studies were also manually searched for relevant studies.

Several keywords were used during our search for studies, both individually and in combination, including pharmacoeconomics, pharmacoeconomics studies, cost-effectiveness analysis OR cost-effectiveness, cost-utility analysis OR cost-utility, cost-minimization analysis OR cost-minimization, cost-benefit analysis OR cost-benefit, health economics, economic evaluation, economic analysis, economic impact, economics, cost evaluation, outcome evaluation, healthcare cost, efficiency analysis, efficacy, effectiveness, and Ethiopia (see Appendix A in the electronic supplementary material [ESM]).

2.2 Inclusion and Exclusion Criteria

This study included only original studies conducted in Ethiopia that evaluated the cost and consequences of at least two health interventions or programs. Studies were not restricted by publication period. We excluded abstracts (with no full text available), treatment guidelines, reviews, and expert opinions. Additionally, we excluded studies that involved other countries besides Ethiopia.

2.3 Study Screening Strategy

Studies were evaluated twice by two separate reviewers, with any disagreements resolved by a third reviewer. In the first step of the screening process, the abstracts and titles of the studies were reviewed. Following the primary screening, all pertinent studies identified were selected for full-text review based on eligibility criteria. The summary of the study search strategy and eligibility screening is presented in Fig. 1.

Fig. 1
figure 1

PRISMA flow chart of study selection

Full size image

2.4 Data Extraction

The data extraction tool was developed to extract pertinent information from selected studies. The relevant data collected included authors' names, publication year, study base year, medical condition, perspective, evaluated alternatives, economic evaluation framework, economic evaluation technique, outcome metric, discount rate, incremental cost-effectiveness ratio (ICER), study conclusion, funding sources, first author residence, and. impact factors of the journal. The studies were then grouped based on the type of economic evaluation, as follows:

  1. 1.

    Cost-minimization analysis (CMA) when costs were compared with evidence of equal effectiveness;

  2. 2.

    Cost-effectiveness analysis (CEA) when health outcomes were expressed in physical therapeutic units;

  3. 3.

    cost-utility analysis (CUA) when health outcomes were expressed in terms of quality-adjusted life-years (QALYs) or disability-adjusted life-years (DALYs); and

  4. 4.

    Cost-benefit analysis (CBA) when health outcomes were expressed in monetary terms.

2.5 Evaluation of the Quality of the Studies

In this review, the QHES scale was used to evaluate the reporting quality of current economic evaluations. There are 100 points on the QHES scale, and lower scores indicate poor quality [17] (Supplementary Table 1, see ESM). For a clearer understanding of which categories most of the studies fall into, the QHES scores of the studies were categorized as ≥ 75, ≥ 50 to < 75, and < 50 [12]. The percentage distribution of the studies for each QHES score category is presented in Fig. 2. The percentage of studies that met each of the QHES items is also shown in Fig. 3. Additionally, the reporting quality of the economic evaluations conducted in Ethiopia was evaluated using the CHEERS 2022 checklist (Supplementary Table 4, see ESM).

Fig. 2
figure 2

Percentage of studies in each Quality of Health Economics Studies (QHES) quality category

Full size image
Fig. 3
figure 3

Percentage of studies that met each Quality of Health Economics Studies (QHES) item

Full size image

3 Results

3.1 Study Screening Process

Through the use of keywords mentioned in the methodology section, 168 articles were initially identified from various electronic databases. Following the removal of 55 duplicate articles, the titles and abstracts of the remaining articles were reviewed, and 42 articles were identified for eligibility assessment. At the time of eligibility assessment, 14 articles were not full pharmacoeconomic evaluations, three had incomplete content, and four included other countries (besides Ethiopia). Finally, 21 articles were included in this study to evaluate the quality and characteristics of economics evaluations conducted in Ethiopia [18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38] (Fig. 1).

3.2 General Characteristics of Included Studies

All except four studies [20, 22, 29, 37] mentioned the perspective of the analysis. Twelve studies (57.14%) were conducted from a healthcare provider’s perspective [21, 23, 25, 26, 30,31,32,33,34,35,36, 38] and three studies (14.29%) were conducted from a societal perspective [18, 27, 28]. HIV was the medical condition evaluated in four (19.06%) of the studies [19, 20, 23, 36], and general public health [30, 37], tuberculosis [25, 38], pneumonia [21, 27], and maternal and child health [31, 32] were all investigated in two (9.52%) studies each. Seventeen of the studies (80.95%) compared healthcare services or programs [18, 20,21,22,23,24,25, 27, 29,30,31,32,33,34, 36,37,38] and the remaining studies compared pharmaceutical products [19, 26, 28, 35]. Cost-utility analysis was the economic evaluation method in 14 (66.67%) of the studies [18, 19, 24,25,26,27,28,29, 31, 32, 34, 35, 37, 38], while the rest used a cost-effectiveness analysis [20,21,22,23, 30, 33, 36]. The most commonly used economic evaluation framework was modeling, which was employed in 12 (57.14%) studies [19, 21, 23,24,25,26, 28,29,30, 35,36,37]. Modeling was combined with two randomized controlled trials [27, 34] and with two cohort studies as well [18, 38]. However, three (14.29%) of the studies did not express their economic evaluation framework [22, 32, 33].

All studies disclosed their funding sources except four (19.05%) [24, 35,36,37]. Addis Ababa University funded two studies independently [23, 38] and another two studies in collaboration with the Canada Research Chair in Economics of Infectious Diseases [19], and McGill University Health Centre [18]. The Bill and Melinda Gates Foundation funded three (21.43%) studies independently [22, 30, 33] and one study in collaboration with the Norwegian Research Council [32]. The first authors of 13 (61.90%) studies were living in Ethiopia [18, 19, 23,24,25, 28, 31,32,33, 35,36,37,38], and in five studies the first authors were from Norway [21, 26, 29, 30, 34] (23.81%).

Among all studies, nine (42.86%) used DALYs as the outcome metric [25,26,27, 31, 32, 34, 35, 37, 38], and seven used the physical therapeutic unit (33.33%) [20,21,22,23, 30, 33, 36]. Eighteen studies (85.71%) have been published since 2015 [18, 19, 21,22,23,24,25,26,27,28,29,30,31,32,33,34,35], with eight (38.1%) exclusively published in 2021. The first study was conducted in 2002, and the last one was conducted in 2021 (Supplementary Fig. 1, see ESM). Seventeen studies (80.95%) specified the discounting rate; 16 of them used 3% as the discounting rate [18,19,20, 24, 25, 27, 28, 30, 31, 33,34,35, 38, 39] and 1 study used 5% [23]. Thirteen studies (61.90%) were published in journals with an impact factor ≥3 [18,19,20,21,22, 25,26,27, 29, 31,32,33, 38], while two studies were published by journals without impact factors [36, 37] (Table 1).

Table 1 The characteristics of included studies (N = 21)
Full size table

3.3 Quality Evaluation of the Studies with QHES Instrument

Overall, the mean QHES score for the studies was 82%. There were 14 (66.67%) studies with a QHES score ≥ 75% [18, 19, 21, 23,24,25,26,27,28, 31, 34,35,36, 38], and 11 of them were cost-utility analyses [18, 19, 24,25,26,27,28, 31, 34, 35, 38]. The QHES score for five (23.81%) of the studies fell in the range of ≥ 50% and < 75% [20, 22, 29, 32, 33]. On the other hand, the QHES scores for the other two studies (9.52%) were < 50% [30, 37]. The first author of one of these two studies was from Norway [30]. There were five studies with 100% QHES scores [23, 26, 28, 34, 35], and four of them were cost-utility analyses [26, 28, 34, 35]. A generalized cost-effectiveness analysis with a QHES score of 35% was the study with the lowest quality [30], followed by a study that examined general public health with a score of 40% [37] (Fig. 2).

Only criterion 7 (data abstraction method) of the QHES scale was adequately articulated in all studies (100%). Criterions 4 (pre-specified subgroup analysis) and 10 (specifying the primary outcome measures) were explicitly stated in 95.24% of the studies. Criteria 1 (objective), 5 (addressing the uncertainty), 9 (measurement and valuation of cost), and 13 (expressing the assumptions and study limitations) were clearly expressed in 90.48% of the studies. The most frequently overlooked criterion was criterion 14 (discussing potential biases), addressed in only 57.14% of the studies. This was followed by criterion 12 (describing the economic model structure and components), expressed in 66.67% of the studies (Fig. 3).

3.4 Quality Evaluation of the Studies with CHEERS Checklist

On average, the quality of the studies evaluated by the CHEERS criteria was 78.23%, ranging from 42.9% [32] to 92.9% [25]. The highest quality study, which investigated tuberculosis-related interventions using a cost-utility analysis, met 92.9% of the CHEERS criteria [25]. Four (19.05%) of the studies met 89.3% of the CHEERS criteria, and they were all cost-utility analyses [18, 19, 27, 38]. Furthermore, two studies fulfilled 85.7% of the criteria [28, 34]. The study that fulfilled the fewest CHEERS criteria was a cost-utility analysis of mother and child health-related interventions, which addressed only 42.9% of the criteria [32]. It was followed by a study that examined general public health-related initiatives, which met 67.9% of the criteria [30] (Fig. 4).

Fig. 4
figure 4

Percentage of CHEERS (Consolidated Health Economic Evaluation Reporting Standards) items met by each study [18,19,20,21,22,23,24, 26,27,28,29,30,31,32,33,34,35,36, 38]

Full size image

Abstract, background and objectives, selection of outcomes, measurement of outcomes, valuation of outcomes, and cost estimation were the CHEERS criteria that were clearly stated in all studies (100%). Besides, health economics analysis plan, comparators, analytics and assumptions, study parameters, and summary of main results were the criteria that were defined in 95.24% of the studies. Setting and location, currency/base year/conversion rate, study findings/limitations/generalizability, and characterizing the uncertainty were the criteria properly expressed in 85.71% of the studies. The effect of the study engagement with patients or other study participants, and characterizing the heterogeneity were the least addressed criteria, which were addressed in only 9.52% and 14.29% of the studies, respectively. Following these criteria, characterizing distributional effects, and approach to engagement with patients or other study participants were the criteria properly addressed in only 28.57% and 42.58% of the studies, respectively. (Supplementary Fig. 2, see ESM).

4 Discussion

The quality and quantity of economic evaluations conducted in Ethiopia were assessed in this study. The selection criteria led to the inclusion of 21 studies. It is evident from this that economic evaluations of healthcare-related interventions in Ethiopia are limited. To properly analyze and comprehend the results, everyone should be familiar with the various elements of economic evaluations. In this context, in-depth training is required for those planning to conduct economic evaluations [10, 40, 41]. The limited number of studies in Ethiopia may be attributed to a lack of health economists or pharmaco-economists, as well as a need for advanced training in economic evaluation among non-health economists [42]. Therefore, it is reasonable to conclude that this application is not widely used in Ethiopia. However, conducting studies has been increasing in recent years, indicating that, even though not significantly, awareness of the role of economic evaluations in decision making is growing over time.

Infectious diseases were the most commonly examined medical condition in the studies analyzed, with HIV being the most frequently investigated [19, 20, 23, 36]. This suggests that the economic evaluation of chronic or non-communicable diseases was not thoroughly explored. Therefore, further studies on non-communicable diseases are needed in Ethiopia, which also has a large chronic disease burden. This result is consistent with a Vietnamese review, which found HIV to be the most frequently evaluated medical condition [7]. Cost-utility analysis was the economic evaluation method used in 66.67% of the studies, with the rest being cost-effectiveness analyses. This is comparable to the results of a Chinese review [6], but an Egyptian review reported that cost-effectiveness analysis was the most widely applied method [5]. Healthcare services or programs were the alternative interventions compared in most of the studies (80.95%), while the remaining studies compared pharmaceutical interventions. Despite 64.29% of the studies having an Ethiopian first author, all studies had at least one foreign author. This may be due to a lack of resources as well as a lack of skills and knowledge regarding the economic evaluation of healthcare decisions.

Overall, eight foreign and one local institution were involved in funding economic evaluations in Ethiopia, both individually and in collaboration. Among studies that disclosed their financial sources, only two were funded by an Ethiopian institution independently. Addis Ababa University was the only Ethiopian institution that funded health economic evaluations, funding two studies independently [23, 38] and another two in collaboration with foreign funders [18, 19]. This indicates that the Ethiopian government and other local organizations have been relatively reluctant to allocate funding for economic evaluations. This may be due to the lack of awareness regarding the importance of these studies and their applicability in assisting with various healthcare decisions. The Bill & Melinda Gates Foundation alone sponsored three studies [22, 30, 33] and one study together with the Norwegian Research Council [32]. Four studies failed to disclose the sources of their financing [35,36,37]. One study was conducted without any funding sources [28].

DALYs were the most frequently used outcome metric, employed in 42.86% of the studies. This was followed by physical therapeutic units, used in 33.33% of the studies. Similarly, a Vietnamese study reported that DALYs were the most commonly used outcome metric [7]. Conversely, a Bangladeshi study reported that physical therapeutic units were the most commonly used outcome metric [44]. A total of 85.71% of the studies were published since 2015, and 38.1% of the studies were published in 2021 alone [18, 19, 21, 28,29,30, 33, 35]. This suggests that scholars are becoming increasingly aware of the importance of including economic evaluations in healthcare decisions in recent times.

The mean QHES score across all included economic evaluations was 82%, ranging from 35% to 100%. This overall mean score was almost equivalent to that reported by the Chinese study [43]. Of the studies, 14 had a mean QHES score ≥ 75% [18, 19, 21, 23,24,25,26,27,28, 31, 34,35,36, 38]. Of these studies, five had a score of 100% [23, 26, 28, 34, 35] and 23.81% of the evaluated studies fell in the range between ≥ 50% and < 75% [20, 22, 29, 32, 33]. Additionally, 14.29% of the studies had QHES scores < 50% [30, 37]. Studies from the Asia-Pacific region and Zimbabwe have both produced findings consistent with these results [12, 45]. Only criterion 7 (data abstraction methods) of the QHES scale was explicitly explained in every study. This was followed by criterion 4 (pre-specified subgroup analysis) and criteria 10 (specifying the primary outcome measures), which were clearly defined in 95.24% of the studies. Criterion 14 (discussing the potential bias) was the most frequently overlooked, explicitly addressed only in 57.14% of the studies.

The average quality of studies evaluated using the CHEERS criteria was 78.23%, which is close to the average QHES score of 82%. Of the 28 CHEERS criteria, the study with the highest quality fulfilled 92.6% of them [25], while the study with the lowest quality met only 42.9% [32]. Among all criteria, only six (abstract, background and objectives, selection of outcomes, measurement of outcomes, valuation of outcomes, and cost estimation) were addressed appropriately in all studies (100%). Criteria such as the effect of the study engagement with patients or other participants, and characterizing the heterogeneity were the most frequently overlooked, addressed in only 9.52% and 14.29% of the studies, respectively. Healthcare decision makers should be aware that poorly designed economic evaluations can lead to misleading findings. Therefore, economic evaluations of high quality are necessary for better decision making [5, 12, 46]. Healthcare decision makers must have a thorough understanding of economic evaluations in order to take into account the methodological and policy uncertainties present in economic evaluations when interpreting their results [13, 40, 41].

4.1 Study Limitations

Because some of the studies' methodologies were not stated clearly, it was difficult for reviewers to categorize and code them using the instruments. Thus, each reviewer may have interpreted it differently, although a third reviewer was used to address any discrepancies. Most of the questions in the QHES scale are a combination of multiple questions that should be addressed separately. Therefore, research that explicitly states one dimension of the question without addressing the other side will be considered to have not answered it at all. Furthermore, despite our systematic search for articles, we may have missed a few other articles for a variety of reasons. Future researchers should, therefore, consider these limitations when conducting research in this area.

5 Conclusion

The review found that Ethiopia has a scarcity of health economic studies, demonstrating that economic evaluation is still in its infancy there. However, according to the findings of this study, the quality of health economic evaluations in Ethiopia is good. The role of Ethiopian institutions in funding such studies was found to be very poor, highlighting the need for active engagement from local institutions. Furthermore, advanced economic evaluation training for Ethiopians is also needed, as this study revealed that the majority of the studies were conducted with the technical assistance of foreign scholars.