Background

Economic evaluations are widely utilized to guide healthcare decision-making, and in many countries, cost-utility analysis (CUA) is the preferred form of economic evaluations [1]. CUA estimates health gains in terms of quality-adjusted life-years (QALYs), a measure combining life expectancy and quality of life, quantified using health-state utilities (HSUs) [2]. HSU is a measure of the value of health outcomes, or health-related quality of life (HRQoL), usually based on the stated preferences of the concerned general public. In current practice, HSU is measured on a cardinal scale, where 1 represents full health, 0 represents being dead, and negative values indicate states worse than dead [3]. There are different ways of obtaining HSU data including use of preference-based measures (PBM), direct elicitation techniques, and mapping algorithms [4].

HSU is a crucial data input in cost-utility models, and it significantly impacts the results of cost-utility analyses (CUA) [3,4,5]. However, there is no universally accepted method for determining HSU values, and different approaches may yield varying HSU values for the same health states [6, 7]. These variations can lead to different CUA conclusions which may undermine the goal of consistent decision-making [7, 8]. As a result, some health technology assessment (HTA) agencies have recommended methods for deriving HSU data in their technical guidance. For instance, England's National Institute for Health and Care Excellence (NICE) recommends using EQ-5D and its value set, based on the health preferences of the general UK population [9]. Nevertheless, there is no international consensus on the best practices for generating and utilizing HSUs in CUA [1]. Furthermore, the reporting of HSU information in CUA models is often inadequate [5, 7, 10, 11]. Frequently, studies do not reference the original sources, and did not report essential details such as preference sources, sample sizes, elicitation techniques, and justifications for the selected utility values [5, 7, 10, 11].

In the past few decades, HTA has been rapidly developing in Asia [12]. To address the increasing healthcare demands and costs, some Asian countries have formally adopted HTA [12, 13]. For instance, South Korea introduced the positive list system in 2006 and established a formal HTA process to inform reimbursement decisions for new drugs [14]. Other national HTA agencies in Asia include the Health Intervention & Technology Assessment Program (HITAP) in Thailand, the HTA Section within the Ministry of Health in Malaysia, and the Agency for Care Effectiveness (ACE) in Singapore [15]. Despite the rapid development, many challenges still persist, including a lack of well-trained personnel, tools, and local data, which might have impeded the quality of the evidence generated [16, 17].

In spite of the HTA growth in Asia, there is a dearth of research examining the methodology and process for cost-effectiveness evidence generation. Only one study investigated the methodological quality of CUA studies targeting Asian populations. In a systematic review of 175 CUA studies published between 2000 and 2012, Thorat et al. found that good methodology was generally adhered to, although some reporting issues were present [15]. However, this study did not assess the quality of HSUs, which plays a vital role in CUA and decision-making. In this study, we aimed to systematically evaluate the HSU data quality in published CUA studies targeting Asian populations by examining 1) the proportion of studies which failed to report the details of HSU data used; and 2) the reported characteristics of the HSU data including estimation method, preference source, HRQoL data source, and sample size of reported HSU data. We also examined how the reporting and characteristics of HSU evolved over the past few decades. We hope that findings in this review can facilitate the improvement of the practices related to HSU reporting and usage in economic evaluations in Asia.

Methods

Data sources and search strategies

A systematic literature search was performed to identify published CUA studies targeting Asian populations, which is defined as populations of any Asian ethnicity living in Asia. Four databases, including PubMed, Web of Science, Medline, and Embase, were systematically searched from inception to December 2019. The searches used the following search terms: “cost-utility analysis”, “cost–benefit analysis”, “cost-effectiveness analysis”, “economic evaluation”, “QALY” and “Asia”. The search algorithms can be found in the Additional file 1.

Inclusion/Exclusion criteria

Studies were included if they: 1) were economic evaluations; 2) used QALY as the outcome measure and reported the use of HSU; 3) targeted an Asian population; and 4) were available in English. Studies were excluded if they were reviews, editorials, conference abstracts, or methodological articles, or if their full text was not available.

Data extraction

Two authors (XZ and QC/AZ) independently selected studies and extracted data using a standardized Microsoft Excel-based data extraction form. Any disagreement was resolved by consulting the corresponding author (NL).

Information was extracted for both the general characteristics of selected studies and the characteristics of reported HSU data. Extracted study characteristics included year of publication, studied population/country, type of intervention, study perspective, time frame of the analysis, model type, type of sensitivity analysis, and discount rate. We also reviewed the included studies to understand whether any literature or systematic reviews was undertaken to obtain HSU values.

Extraction of the HSU data characteristics was restricted to the HSU data used in the base-case analysis of each study. For each CUA, we first identified all the health states used in the estimation of QALYs. Next, we identified the HSUs for the respective health states used in the base-case analysis. As a result, the number of HSUs we extracted was equal to the number of health states used in the CUA studies. For each HSU value identified, we extracted data for four key characteristics, including 1) estimation method, e.g., PBMs, direct elicitation methods, mapping etc.; 2) source of health-related quality of life (HRQoL) data, being defined as the country, district or region from which HRQoL data was collected and used to estimate the HSU values; 3) source of preference data, being defined as the country, district or region from which health preferences were elicited and used to estimate the HSU values; and 4) sample size, being defined the population sample size used for collecting i) HRQoL data (in the case of using an indirect estimation method, such as EQ-5D) or ii) health preference data (in the case of using a direct estimation methods such as time trade-off).

Data analysis

All the four key HSU characteristics were coded into categorical variables and analyzed at the HSU level. First, for each HSU characteristic, we calculated the percentage of nonreporting, that is, the proportion of HSUs for which the characteristic was not reported. We also examined how the proportion changed from years 1999–2010 to years 2011–2019. To examine the changes in reporting quality, we additionally conducted sub-group analysis for the top three countries for which most CUA studies were conducted. To investigate changes in HSU estimation methods, we excluded HSUs estimated using expert opinions or unjustifiable methods, and categorized the remaining HSUs into one of three groups including EQ-5D, SF-6D/HUI/QWB/mapping, and time trade-off (TTO)/standard gamble (SG)/visual analogue scale (VAS). For sources of HRQoL and preference data, we identified for each HSU whether the sources matched the country targeted by the CUA, and calculated the proportion of HSUs estimated using HRQoL and preference data from the country which the CUA was conducted for.

Results

Study selection

The study selection process is summarized in Fig. 1. The initial search identified a total of 3,379 studies from four databases. After removing duplicates, 1,958 studies were included in the review process. Based on assessment of titles and abstracts, 932 studies were excluded. Of the remaining 1026 studies, full-text review excluded 237 studies. Among those, 142 did not target an Asian population; 53 did not use QALY as the outcome measure (n = 10) or report any HSU data (n = 43); 29 were not original CUAs; 11 were protocols or unpublished manuscripts (n = 8), methodological articles (n = 2), or systematic review (n = 1); and 2 were not available in full-text. Therefore, a total of 789 studies were included. Among those, 577 studies obtained HSUs from published literature and 123 studies used unpublished empirical data to calculate the HSUs.

Fig. 1
figure 1

Flow diagram showing the process of study selection. QALYs, quality-adjusted life-years; CUA, cost-utility analysis; DALYs, disability-adjusted life years

Full size image

Study characteristics

The studies included in this review were published between 1999 and 2019, with the majority published in the period 2017 to 2019 (43.9%) or 2014 to 2016 (26.6%). There was a significant increase in the number of studies after 2013 (Fig. 2). Most of the CUA studies targeted the population of mainland China (27.0%), Japan (20.8%), or Thailand (9.3%). The majority of the CUAs evaluated a therapeutic intervention (73.0%), took a payer’s perspective (54.0%), and used a time horizon of more than 10 years (52.0%). The CUA studies were primarily model-based, using either Markov models (53.0%) or decision tree models (12.5%), and a probabilistic sensitivity analysis (62.2%). Detailed characteristics of the included studies are summarized in Table 1. Notably, only 5% of the CUA studies reported the use of a literature review to identify HSU values.

Fig. 2
figure 2

The number of published QALY-based CUA studies in Asia by year

Full size image
Table 1 Characteristics of included papers (n = 789)
Full size table

Characteristics of HSUs

A total of 4,052 HSUs were identified from the base-case analyses of the included studies. The number of HSUs per study ranged from 1 to 25, with the median being 5. Of the 4,052 HSUs, 3,351 (82.7%) were from published literature and 656 (16.2%) were unpublished empirical data. The full characteristics of the HSUs are shown in Table 2.

Table 2 Characteristics of HSUs included in the review (n = 4,052)
Full size table

Nonreporting

Key characteristics were not reported for the majority the 4,052 HSUs. The percentage of nonreporting was 65.4% for estimation methods, 76.9% for source of sample, 84.3% for sample size, and 91.0% for source of health preferences. Figure 3 shows that the percentages of nonreporting decreased after year 2010 as a whole and for three countries with the most publications, except that Thailand had a higher percentage of estimation method nonreporting after year 2010.

Fig. 3
figure 3

The proportion of HSUs whose characteristics were not reported by publication year

Full size image

Estimation method

Of 1,349 HSUs for which estimation methods were reported, EQ-5D (n = 781, 55.7%) was the most frequently used method, followed by TTO (n = 170, 12.1%) and SF-6D (n = 147, 10.5%). A total of 20 HSUs (0.5%) were obtained using an unjustifiable method, for example, using non-utility HRQoL instrument. Figure 4 shows that there was a significant increase in the use of EQ-5D after 2010.

Fig. 4
figure 4

Characteristics of HSUs by publication year

Full size image

Source of HRQoL data

In total, the HRQoL data source was reported for 938 HSUs. Most of the HSU (n = 862, 91.9%) were estimated using Asian HRQoL data (see Table 2), with Japan (25.8%), mainland China (14.2%), and Thailand (11.1%) being the main source. The majority of the HSUs (n = 631, 67.3%) were estimated using HRQoL data collected from the target country and the proportion of such HSUs increased considerably after 2010 (Fig. 4).

Source of preference data

The source of preferences was reported for only 365 of the 4,025 HSUs. Among those, 320 HSUs (87.7%) were derived from an Asian value set (see Table 2). The majority of the health preferences used were from Japan (19.7%), Thailand (13.2%), mainland China (12.6%), or a non-Asian country (12.3%). Most HSU values (n = 303, 83.0%) were estimated using health preference data collected from the target country, with a significant increase of such HSUs after 2010 (Fig. 4).

Sample size

Among the 637 HSUs for which the estimation sample size was reported, almost half of these values was estimated with a sample of 100 or more individuals (45.7%) and the proportion increased after 2010 (Fig. 4).

Discussion

In this study, we systematically summarized the HSU characteristics in published CUA studies targeting Asian populations. There is an evident increase in CUA publications in the past two decades in Asia, especially after the year 2013. This upward trend in the number of CUA studies is not surprising because countries in this region have been actively using economic evaluations to inform reimbursement decision-making within their respective health systems in the past two decades [13, 18,19,20].

Despite the proliferation in publications, most studies failed to report the basic characteristics of the HSU values used. Although the issue of nonreporting showed an improvement, the estimation method, source of HRQoL data, source of preference data, and sample size were not reported for over 60% to 90% of the HSUs used in the CUA studies published between years 2011–2019. This finding is in line with previous studies of CUA studies not targeting a specific region. For example, Ara et al. found that out of 24 published CUA studies in the cardiovascular disease area published after 2014, none of the studies reported HSU-related details (e.g., sample size, estimation method) used [7]. These findings suggest that poor reporting quality for HSUs used in CUA studies may be a global issue and not just specific to Asia. This suboptimal practice is disappointing since ISPOR’s Consolidated Health Economic Evaluation Reporting Standards (CHEERS) [21] was published in 2013 simultaneously in ten international health economics and medical journals. The targets of this reporting guidance are researchers as well as editors and reviewers of journals publishing economic evaluation studies, including CUA. CHEERS recommends economic evaluation studies to describe HSU estimation methods, together with the population sample size, and sources of the HRQoL and preference data [21]. It is worth noting that those recommendations have been maintained in CHEERS-2022 [22], which was simultaneously published in twenty-one journals. Hopefully, this updated guidance will help improve the HSU data reporting quality in future economic evaluation studies.

It is concerning that a very large proportion of CUA studies used HSUs from the literature but did not report how those HSUs were identified. It is possible that the HSUs were identified through comprehensive literature reviews but the CUA papers failed to report them. If so, this is a reporting issue that should be improved in future studies [22]. If the HSUs in those CUA studies were not identified using formal literature reviews, the appropriateness and validity of the HSUs as well as the conclusions of those CUAs would be questionable. Future CUA studies in Asia should adopt a more rigorous approach in searching, evaluating, and selecting HSUs by following emerging good practices such as those recommended by an ISPOR taskforce [10].

Among the HSUs for which basic characteristics were reported, the majority of those used in CUA studies published after 2010 were derived using recommended methodology such as using an established preference-based measure EQ-5D and its value sets reflecting the health preferences of targeted populations. The trend of using EQ-5D to acquire HSU data is consistent with findings from CUA-related systematic reviews worldwide [23,24,25,26]. For example, in a systematic review of HSU values for stroke, 87 of 111 (78%) studies used EQ-5D to generate HSU data. The predominant use of EQ-5D data in CUAs around the world is probably attributable to both recommendations of HTA agencies [13] and its extensive adoption by outcome researchers and clinicians. A recent review study found that there is a proliferation of clinical studies using EQ-5D as an outcome measure [27], forming a database that provides CUA studies with HSU data. In Asia, most HTA agencies recommend the use of either EQ-5D or validated preference-based measures, and local public’s health preferences to estimate HSU values [13]. Also importantly, many Asian countries have established their national EQ-5D value sets [28,29,30] and validated the desirable measurement properties of EQ-5D in their populations [31].

A main limitation of this study is that we only included CUA studies published in English. We explored the possibility of including CUA studies published in both English and non-English such as Chinese literature into this review. We found it is technically difficult and resource-demanding, and eventually decided to only include English databases in this study. Nevertheless, in a similar systematic review study evaluating QALY-based CUA studies in the Chinese literature, we found similar trends [32]. Another important limitation of this review is the poor reporting quality of the reviewed studies. Only a small portion of the reviewed CUA studies reported the characteristics of the HSUs used. As a result, our findings may not reflect the true status quo of Asian CUA studies. This is also because there are many unpublished CUA studies. In HTA practice, CUA studies may be performed with limited resources and are therefore of low quality. For example, Hong and Bae found that, in addition to poor reporting, CUA reports submitted to the Health Insurance Review and Assessment Service (HIRA) in South Korea between 2014 and 2018 inadequately adhered to the recommendation of using HSUs reflecting Korean’s health preferences [33]. Lastly, we did not investigate the extent to which HSUs in the literature were appropriately used in the CUAs. A previous review of CUA studies in cardiovascular disease identified potentially inappropriate use of published HSU data such as modifying original HSU values without justification [19].

Conclusion

Over the past two decades, the number of CUA studies targeting Asian populations significantly increased and the methods used to derive HSUs in those studies improved. However, HSU characteristics were not reported in most of published Asian CUA studies, making it difficult to evaluate the true quality and appropriateness of the HSUs used in those cost-effectiveness studies. We advocate better reporting of HSUs in future CUAs.