FormalPara Key Points for Decision Makers

This is the first study focusing on whether and how disutilities of adverse events (AEs) are used in cost-utility analyses of cancer drug therapy, and if not, why not.

It has been found that detailed information about the use of disutilities of AEs is not reported and essential methods to analyze the impact of AEs on quality-adjusted life-years are not thoroughly conducted.

An application process for the disutilities of AEs has been developed to remind and guide researchers to correctly use the disutilities of AEs as parameters in the decision-analytic model.

1 Introduction

Disutility is a complement to health state utility values (HSUVs) [1] and describes the decrement in utility (valued quality of life) due to a particular symptom or complication [2]. Disutility values are often expressed as negative values, which may be derived by subtracting utility values for the health state with the symptom or complication of interest from those of the health state without that symptom or complication [2]. As research in this area has grown, more disutility values have been estimated, associated with particular characteristics [3], including mode of administration [4], dose frequency [5], medical device attributes [6], treatment convenience [7], and caregivers [8]. Including disutility in economic models is helpful to evaluate the impact of all factors on health outcomes and achieve a more accurate assessment of quality-adjusted life-years (QALYs).

Disutilities of adverse events (AEs) are particularly recommended to be included in economic evaluations. Technical support document 12, developed by the UK’s National Institute for Health and Care Excellence (NICE), suggested that a clear justification should be provided for the non-inclusion of adverse effects in economic models [9]. The same suggestion was made by Craig et al. in 2010 [10] and Ara and Wailoo in 2012 [11]. The Professional Society for Health Economics and Outcomes Research (ISPOR) released a Good Practices for Outcomes Research Task Force Report in 2016 [12], which advised researchers to consider whether their economic model needed additional utility deficits of acute events and short-term (but often severe) treatment-related AEs. The 2019 ISPOR report also gave detailed recommendations on the disutilities of AEs [13]. Guidelines for economic evaluations in Australia, Belgium, Canada, the UK, and South Africa also recommend including disutilities of AEs [14].

Although the above recommendations on disutility applications have been made, there is still a situation in which some current economic evaluations input the disutilities of AEs, while some do not. For example, when carrying out a cost-utility analysis (CUA) related to ovarian cancer, Guy et al. included the disutilities of AEs [15] (e.g., anemia, thrombocytopenia, neutropenia, fatigue, hypertension, nausea, and vomiting), but Wolford et al. [16] and Leung et al. [17] did not. Even when CUAs include disutilities of AEs, the approach may not be fully standardized with recommendations proposed by institutions and scholars. For example, Chongqing et al. [18] included the disutilities of grades 1 and 2 AEs. However, the recommendation suggests that applying decrements for grades 1 and 2 AEs can introduce an element of double counting as the cohort used for the main HSUVs may have included a proportion of patients who had experienced these grades 1 and 2 AEs [11, 12]. The neglect and non-standard application of disutility may result in an incomplete or repeated estimation of QALY decrements, which may impair the accuracy of evaluation results. However, no studies of how disutility values were used in CUAs are available, and the extent and issues of disutility applications are unclear.

Worldwide, cancer is the main cause of death among diseases. Approximately 19.3 million new cases with 10.0 million deaths were recorded in 2020 [19]. Statistics released by IQVIA in 2021 estimated that 169 antitumor drugs were launched in the past decade [4]. Although the introduction of new drugs has improved the quality of life of patients, it has also led to an increase in the cost of cancer treatments [5]. Coupled with the existence of finite healthcare budgets, CUA for cancer treatment has become more important. In the past 2 decades, oncology appraisals accounted for 45.14% of 370 positive NICE technology appraisals [20]. Therefore, this study focused on CUAs of cancer drug therapy and aimed to identify whether and how the disutilities of AEs were used, and if not, why not. We also aimed to clarify discrepancies with recommendations proposed by health technology assessment (HTA) institutions and scholars for including disutilities in economic evaluation models.

2 Methods

2.1 Identification of Cancer and Adverse Events (AEs)

This study focused on the CUAs of drug therapy for cancer. The International Classification of Disease, 10th Revision (ICD-10) was used to identify cancers (ICD code: C00-C97) [21], and AEs were identified using the Common Terminology Criteria for Adverse Events (CTCAE) version 5.0 [22]. Grade related to the severity of the AEs, graded from Grades 1–5.

2.2 Data Sources

This systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [23]. Databases including PubMed, Web of Science, and the Cochrane Library, as well as the official websites of the National Institute for Health and Care Research (NIHR), the Institute for Clinical and Economic Review (ICER), the Institute for Quality and Efficiency in Health Care (IQWiG), the Canadian Agency for Drugs and Technologies in Health (CADTH), and the Centre for Reviews and Dissemination (CRD) were searched for CUAs published from January 2019 to April 2022, to analyze the use of disutilities since the 2019 ISPOR report was published providing more detailed recommendations. The keywords used for the search included those related to disease (‘cancer’ or ‘neoplasm’ or ‘malignancy’ or ‘neoplasia’ or ‘tumor’ or ‘carcinoma’ or ‘sarcoma’ or ‘leukemia’ or ‘leukaemia’ or ‘lymphoma’ or ‘mesothelioma’ or ‘glioma’ or ‘germinoma’ or ‘choriocarcinoma’ or ‘myeloma’ or ‘melanoma’ or ‘malignant neoplasm’), and evaluation techniques (‘economic evaluation’ or ‘pharmacoeconomic analysis’ or ‘cost utility’ or ‘cost-utility’ or ‘cost effectiveness’ or ‘cost-effectiveness’). The reference lists from the selected studies were also reviewed to provide a comprehensive list of studies. The complete electronic search strategy for PubMed is provided in Box 1.

2.4 Study Selection

Articles were independently screened by two authors (YQL and ZJD). Disagreements were resolved by consensus, however if consensus could not be reached, a third author (FC) provided arbitration and consensus. The inclusion criteria were defined by the population, intervention, comparison, outcomes and study (PICOS) strategy (see Table 1). Studies with the following criteria were excluded: (1) duplicate and non-full-text publications; (2) reviews, editorials, and comment letters; and (3) studies that were not published in English.

Table 1 Study inclusion criteria
Full size table

2.5 Data Extraction

Two groups of authors (JFH and PHS, and LW and ZJD) independently extracted data from each selected study using a standard abstraction Microsoft Excel spreadsheet (Microsoft Corporation, Redmond, WA, USA). Any discrepancies were examined by another author (YQL) to ensure that accurate data were obtained. The extracted data included study characteristics (title, first author, journal, year of publication, country, target population, study perspective, and model structure), and application of disutilities of AEs (whether and how disutilities were used, or why they were not used).

2.6 Comparison Between the Use of Disutilities of AEs and Published Recommendations

By reviewing published journals and guidelines, this study summarized the recommendations regarding the use of disutility values of AEs (review process and original text shown in electronic supplementary material [ESM] 1). After analyzing and merging homogeneous content, five criteria were formed to compare the discrepancies between practice and recommendations: (1) providing a clear justification for the inclusion or exclusion of disutilities of AEs in the decision-analytic model; (2) providing a detailed description of parameter values and data source; (3) focusing on grade 3 or greater AEs, excluding grades 1 and 2 AEs to avoid double counting; (4) providing a detailed description of calculation, e.g., AEs were justified by the incidence rates, and/or duration; and (5) conducting uncertainty analyses, including univariate sensitivity analyses, probabilistic analyses and scenario analyses.

The included CUAs were divided into two groups based on whether disutilities were used as parameters in the model: (1) CUAs using disutilities of AEs; and (2) CUAs not using disutilities of AEs and compared with the criteria, respectively. Descriptive analyses were used to summarize the results.

2.7 Quality Assessment

The quality of the included studies was evaluated through the Quality of Health Economic Studies (QHES) instrument [24], an internationally recognized quality checklist. The QHES contains 16 items, each with specific weight values ranging from 1 to 9. The quality score can be calculated by adding up all the points for questions answered ‘yes’. Studies were categorized as highest quality (100 points), high quality (75–99 points), general quality (50–74 points), low quality (25–49 points), and lowest quality (≤ 24 points). Each article was independently evaluated and scored by two groups of reviewers (JFH and PHS; LW and ZJD). Any disagreement was solved by referring to another reviewer (YQL).

3 Results

Figure 1 shows that 12,092 articles were retrieved from the initial search. After screening titles and abstracts, 595 studies were potentially eligible and these were retrieved for full-text review. After reading the full text, 128 articles were excluded because they did not meet the inclusion criteria. Ultimately, 467 studies were included that fully met the pre-established inclusion criteria.

Fig. 1
figure 1

Study selection process. AEs adverse events, CUAs cost-utility analyses

Full size image

3.1 Characteristics and Quality Assessment of Identified Studies

A summary of the study characteristics and quality assessment is shown in Table 2. Of the 467 CUAs reviewed, 54% (254/467) used disutilities of AEs and 46% (213/467) did not. The proportion of CUAs using disutilities of AEs in 2019, 2020, and 2021 was 47% (51/107), 50% (65/129), and 61 % (116/190), respectively, showing an increasing trend. The search ended in April 2022 and thus the 54% (22/41) in that year does not represent the overall situation for the year. In the US, Canada, Japan, and the UK, slightly more CUAs used disutilities of AEs. Furthermore the proportion of CUAs using disutilities of AEs was higher in the fields of non-small cell lung cancer, hematological malignancies, melanoma, and prostate and hepatocellular carcinoma.

Table 2 Characteristics and quality assessment of included cost-utility analyses for cancer treatment
Full size table

Ninety-seven percent (455/467) of the included CUAs were assessed as ‘high quality’, with the results showing that the quality of CUAs using disutility was better than that of CUAs not using disutility, as the proportion of ‘highest quality’ and ‘high quality’ was higher in the group using disutility. The quality assessment of each included CUA is provided in ESM 2.

3.2 Use of Disutility Values in Cost-Utility Analyses (CUAs)

3.2.1 Reasons for Including Disutilities

Of 254 CUAs using disutilities of AEs, 15% (38/254) provided a clear justification for this. The main reason (95%, 36/38) was that AEs were expected to have a significant impact on health-related quality of life (HRQoL). All reasons for including disutilities are summarized in ESM 3 Table S3-1.

3.2.2 Scope of Disutilities Input

Of the 254 CUAs using disutilities of AEs, 74% (187/254) stated disutilities, of which AEs were selected as model parameters. The scope of disutility input was specified by severity (98%, 183/187), incidence (36%, 68/187), incidence difference (2%, 4/187) of AEs, and/or expert opinions (2%, 3/187). For example, in the study by Dong et al. [25], grade 3 or 4 AEs with an incidence rate > 5% or significant differences between the two strategies were included in the model. Addo et al. [26] restricted AEs to vaginal bleeding, musculoskeletal disorders such as arthralgia, deep vein thrombosis, and pulmonary embolism according to expert opinions.

3.2.3 Parameters of Disutilities

Overall, 60% (153/254) of the CUAs using disutilities of AEs input disutility values in terms of specific AEs, such as disutilities caused by anemia (− 0.073), thrombocytopenia (− 0.19), neutropenia (− 0.20), decreased platelet count (− 0.19), decreased neutrophil count (− 0.20), and febrile neutropenia (− 0.42) [27]. In the remaining studies, disutility values of AEs were integrated using certain criteria: 23% (59/254) of the CUAs integrated disutilities of AEs as one parameter, such as AE disutility (− 0.28) [28]; 9% (23/254) integrated disutility values based on therapy, such as disutilities of AEs in pertuzumab plus trastuzumab plus docetaxel (− 0.056) and trastuzumab plus docetaxel (− 0.04) [29]; 6% (16/254) were based on severity, such as disutility due to grade 1 and 2 AEs (− 0.014), and grade 3–5 AEs (− 0.157) [30]; and 1% (2/254) were based on the care setting, such as disutilities of AEs in outpatient (− 0.13) and inpatient (− 0.17) settings [31].

3.2.4 Source of Disutilities Data

The disutilities of AEs used in the CUAs came from three main sources: literature citations, research hypotheses, and utility measurement. Overall, 90% (229/254) of the CUAs cited data from published literature, including original health utility studies and CUA studies. The lack of data means that the disutility values cited by CUAs might not be measured for the study population, setting and location of the CUAs. For example, Chisaki et al. [32] carried out a cost-effectiveness analysis in the Japanese healthcare system but used disutility values of AEs from other countries because no data were available from Japan. In total, 14% (35/254) of the CUAs made assumptions about the disutility values of AEs based on expert opinion or clinical judgment. For example, the disutility might be assumed to be 0 [33] or be the mean value of disutilities of AEs that were available [34] or the disutility value of other single AEs that were available [35]. A further 6% (14/254) of the CUAs performed an original health measurement study to get utility data (e.g., a CUA alongside a clinical trial that administered EQ-5D to conduct measurement [36], which is a preference-based instrument). Nine studies included disutility values of AEs (4%, 9/254) but did not specify data sources.

3.2.5 Form of Disutilities Data

The disutility values in the CUAs were presented as negative values, non-negative values, or non-negative values combined with the basic health state. A negative value represented the utility decrement from AEs (e.g., the utility decrement of neutropenia was − 0.09 [37]). This approach was used in 87% (220/254) of CUAs. Non-negative values represent the utility of AEs (e.g., the utility of febrile and hospitalized neutropenia or leukopenia was 0.33 [38]). This approach was used in 2% (6/254) of CUAs. A non-negative value combined with the basic health state represents the utility of AEs that occurred in a particular disease state (e.g., the utility of the progression-free survival state plus neutropenia was 0.604 [39]). This approach was used in 7% (19/254) of the CUAs. In the other 5% (12/254) of the CUAs, disutility values of AEs were included in the model but it was difficult to judge their form because no data were shown.

3.2.6 Calculation of Quality-Adjusted Life-Year Decrements

Of 254 CUAs using disutilities of AEs, 52% (132/254) highlighted the methods used to calculate QALY decrements resulting from AEs. They adjusted the disutility values of AEs based on the incidence rates (49%, 65/132), duration (48%, 64/132), cycle of occurrence (36%, 48/132), and/or frequency (20%, 26/132) of AEs to calculate the effect of AEs on QALYs in the intervention and control arms. For instance, Bensimon et al. [40] subtracted one-time AE-related utility decrements at the beginning of the first cycle based on treatment-specific AE risks, mean durations of AEs, and the additive disutility associated with AEs.

3.2.7 Methods for Uncertainty Analyses for Disutilities

Eighty-four percent (214/254) of CUAs using disutilities of AEs conducted uncertainty analyses and 4% (8/214) used a scenario analysis. Overall, 9% (23/254) of the CUAs did not perform an uncertainty analysis for disutilities, and in 5% (13/254) of the CUAs, information was insufficient to determine whether an uncertainty analysis had been carried out.

3.3 Reasons for Not Using Disutility Values in CUAs

Of the 213 CUAs that did not use disutilities of AEs, only 16% (34/213) explained the reasons for this. There were five main reasons, and all the reasons for excluding disutilities are summarized in ESM 3 Table S3-2. First, the utility values of the disease states already reflected the impact of AEs (50%, 17/34). Pruis et al. [41] reported utility values of the progression-free state for the sunitinib and interferon-α arms as 0.721 and 0.715, respectively. This difference was likely the result of improved efficacy, AE profile, and use of injectable medication. Additional disutilities for specific AEs were therefore not included. Second, the incidence of AEs between the intervention and control arms were not significant enough to cause differences in quality of life (12%, 4/34), as in the study by Phua et al. [42]. Third, the AEs seen had little impact on quality of life because of their low incidence or mild severity (32%, 11/34), as in the study by Mulder et al. [43] Fourth, no disutilities data were available (6%, 2/34). The study of Sussell et al. [44] did not consider AE-related disutility for both the third and fourth reasons. Fifth, no disutility values were attributed to AEs to match other economic evaluations (6%, 2/34), as in the study by Takushima et al. [45]. Moreover, the study by Bastos-Oreiro et al. [46] stated that no disutility values were attributed to AEs, similar to other economic evaluations developed for axi-cel. The remaining 84% (179/213) of CUAs did not use disutilities of AEs or explain the reasons for not doing so.

3.4 Discrepancies with Recommendations

Only 15% (72/467) of CUAs provided a clear justification for the inclusion or exclusion of disutilities for AEs. The proportion was 15% (38/254) for the group that included disutilities of AEs and 16% (34/213) for the group that excluded them.

Most of the 254 CUAs using disutilities of AEs provided a detailed description of parameter values and data sources. Only 8% (21/254) did not provide values or data sources, including 5% (13/254) that did not show the specific disutility values, 4% (9/254) had no statement of data sources, and 2% (4/254) did not display either the disutility values or the data source.

Overall, 69% (175/254) of CUAs focused on grade 3 or higher AEs, and 11% (28/254) of CUAs applied utility decrements for grades 1 and 2 AEs, which had the potential to introduce an element of double counting. The grades of AEs used in 20% (51/254) of CUAs were unclear.

Disutilities of AEs were generally calculated using the incidence rates, which were clearly stated in 49% (65/132) of CUAs that used disutilities of AEs. Moreover, 48% (64/132) of the CUAs justified the calculation of disutilities by duration, 36% (48/132) by the cycle of occurrence, and 20% (26/132) by the frequency of AEs.

Most of the 254 CUAs that included disutilities of AEs performed uncertainty analyses to assess the effect of these disutilities on the economic evaluation results. Only 9% (23/254) of CUAs did not conduct uncertainty analyses, and it was impossible to determine whether uncertainty analyses had been carried out in 5% (13/254) of CUAs.

Overall, in the group of CUAs using disutilities of AEs, only 6% (15/254) of the CUAs that used disutilities of AEs considered all five recommendations. In the group of CUAs not using disutilities of AEs, only 16% (34/213) clarified the reasons for not using disutility values.

4 Discussion

In this study, as a result of screening eight databases and reviewing 467 included CUAs of cancer drug therapy to identify whether and how disutilities of AEs were used as model parameters, it turned out that 54% (254/467) of CUAs included disutility values of AEs. This proportion was similar to the result reported in the study by Craig et al. [10]; however, these researchers only investigated the published HTA reports commissioned by the NIHR and found that the most common method (53%, 42/80) was to derive utilities from patients on treatment. In addition, due to the lack of detailed reporting on the derivation of utilities, they considered that if one can infer that utilities derived from patients on treatment are likely to encompass AEs, then one could surmise that almost 53% of models incorporated AEs through utilities.

Moreover, this study found that the proportion of CUAs using disutilities of AEs increased from the years 2019 to 2021, from 47% (51/107) to 61% (116/190). The proportion of using disutilities of AEs was higher in countries such as the US, Canada, Japan and the UK and the field of non-small cell lung cancer, hematological malignancies, melanoma, and prostate and hepatocellular carcinoma. The main reason (95%, 36/38) for inclusion of disutilities for AEs was that AEs were expected to have a significant impact on HRQoL. The main scope (98%, 183/187) of disutilities input was related to the severity of the AEs. The main parameter (60%, 153/254) was the disutility value for specific AEs, with published literature being the major source (90%, 229/254) of disutility values. Disutilities were often expressed as negative values (87%, 220/254), which is consistent with the definition of the York Health Economics Consortium [2].

When comparing the use of the disutilities of AEs with the five criteria summarized from the published literature, only 6% (15/254) of CUAs that included the disutilities of AEs were consistent with them, indicating that there were some issues. First, detailed information about the use of disutilities of AEs was not reported. To be specific, 85% (395/467) of included CUAs provided no clear justification for the inclusion or exclusion of disutilities for AEs, 8% (21/254) did not provide the parameter values or data sources, and 5% (13/254) were unclear about whether to conduct uncertainty analyses due to insufficient information. Second, essential methods to analyze the impact of AEs on QALYs were not thoroughly conducted. The calculation method of QALY decrements resulting from AEs was not highlighted in 48% (122/254) of CUAs, and 9% (23/254) did not perform an uncertainty analysis. This may be attributed to researchers’ unfamiliarity with existing recommendations about the disutilities of AE applications since there is no universally agreed set of recommendations. However, researchers who included the disutilities of AEs but did not conduct uncertainty analyses may be unfamiliar with the common application process of HSUVs.

Based on our findings, it is suggested that researchers should follow the relevant guidelines [9,10,11,12,13] on the application of HSUVs, especially the recommendations on the application of the disutilities of AEs. The five criteria summarized in this study can be used as a reference. In general, disutilities of AE applications could accord to the following steps.

First, researchers should pay attention to the impact of AEs on the HRQoL and consider carefully whether to include the disutilities of AEs since they may be expected to affect the cost-effectiveness estimate [12], and published recommendations emphasized the importance of considering the disutilities of AEs in economic evaluation models [9, 12, 13].

Second, researchers need to collect information about whether AEs influence patients’ HRQoL and whether the impact of AEs had already been captured in the HSUVs used for the model’s health states to help estimate which AEs needed to be included. Where the AEs are known to affect HRQoL, they should be included in the economic evaluation model [9]. If the utility effects of important AEs are captured by the HSUV data [13], there is no need to add additional disutility values of AEs [47,48,49,50,51]. It is worth noting that grade 3 and higher AEs may have a greater impact on HRQoL [11, 12], and that checking whether unreasonable exclusion has been performed is necessary. Conversely, care is required to ensure the decrements associated with grade 1 and 2 AEs are not double-counted as the cohort used for the main HSUVs may have included a proportion of patients who had experienced these AEs [11].

Third, researchers need to define a wide search range and appropriate approach to collecting the disutility values of AEs. In general, search strategies and selection criteria should be formulated according to data requirements to systematically search some well-known professional databases such as PubMed. Literature on systematic reviews or measurement studies of the disutility values of AEs or the economic evaluation with the same subject needs to be screened to comprehensively collect the needed data. Meanwhile, researchers need to determine whether the obtained values need to be adjusted, especially for non-negative values. If there are no available values, measurement studies of disutilities are required to be conducted following the utility measurement guidelines [52,53,54,55] to provide data support. The guideline of Matza et al. [55] is recommended as a methodological reference because it provided the greatest number of recommendations for estimating disutilities.

Fourth, researchers need to clarify the methods of calculation and uncertainty analyses of the disutility values of AEs. The QALY loss resulting from each AE can be estimated by multiplying the disutility values by the incidence rates, duration, and/or frequency; the cycle in which each AE occurs should also be noted [13]. One-way sensitivity analyses should be used to determine which parameter value the model results are most sensitive to [13]. The choices of statistical distributions and correlations in probabilistic sensitivity analyses should be fully documented and justified [9]. Scenario analyses are also recommended where possible since several CUAs conducted these practices [34, 56,57,58,59,60,61,62].

Fifth, researchers need to put an increasing emphasis on transparency in application reporting of the disutilities of AEs. Justification for inclusion or exclusion of disutilities of AEs, parameter values and data source, methods used to source evidence, methods for data adjustments, calculation, and uncertainty analyses should be described clearly to allow readers, reviewers, and healthcare decision makers to evaluate the credibility of the application.

This study did have some limitations. First, it focused on evaluating the information about disutilities of AEs in the CUAs but did not trace the source literature for the disutility values or verify whether or not the data citations were correct. Second, the study focused on disutilities of AEs related to drug therapy for cancer, therefore the issues should be extrapolated with caution to disutilities related to other domains and diseases, such as complications, mode of administration, and dose frequency.

5 Conclusions

The current use of disutilities of AEs in CUAs shows some discrepancies with recommendations proposed in the literature. One is that detailed information about the use of disutilities of AEs was not reported and the other is that essential methods to analyze the impact of AEs on QALYs were not thoroughly conducted. Therefore, it is suggested that researchers should attach importance to the impact of AEs on HRQoL. Furthermore, in this study, an application process for the disutilities of AEs was developed to remind and guide researchers to correctly use the disutilities of AEs as parameters in the decision-analytic model.