FormalPara Key Points for Decision Makers

Menorrhagia affects health and non-health aspects of life

Broader benefits of the treatment should also be considered

Willingness to pay is feasible and acceptable for use in menorrhagia

The cost-benefit analysis suggests oral treatment as a first-line treatment for menorrhagia

1 Introduction

Economic evaluation offers a formal toolkit to assess both the costs and consequences of competing services. In the UK, decision makers such as the National Institute for Health and Care Excellence (NICE) have adopted cost-utility analysis as the economic evaluation method of choice, which measures outcomes using quality-adjusted life-years (QALYs) with a focus on health-related outcomes [1]. The conventional criterion for decision making is based on a health-maximisation principle with the aim of maximising QALYs relative to the resources available. This approach to economic evaluation, with its focus on health outcomes, is described in theoretical terms as an ‘extra-welfarist’ approach [2]. To construct QALYs, it is recommended that either the EQ-5D or the SF-6D instrument is used to measure health-related quality of life. The use of cost-utility analysis offers a framework for evidence-based decision making in which the objective is to maximise health, but it offers limited support for the evaluation of interventions for which there are gains that go beyond health alone. Cost-benefit analysis is an alternative approach within the economic evaluation toolkit and in contrast is based on the welfarist approach. Cost-benefit analysis places a monetary value on outcomes using stated preferences methods such as contingent valuation or ‘willingness to pay’ (WTP). A cost-benefit analysis takes a wider perspective compared with a cost-utility analysis and thus offers the potential to incorporate costs and consequences that go beyond the healthcare sector.

Measures used to capture outcomes underpinned by the extra-welfarism framework are commonly used across all types of clinical conditions, including those that are chronic but have symptoms that occur in episodes [3]. One such condition is heavy menstrual bleeding, which is clinically termed ‘menorrhagia’. Menorrhagia can be defined as “excessive menstrual blood loss which interferes with the woman’s social, emotional, physical and material quality of life” [4]. The principal driver for treatment is based on women’s experience of its interference in their lives [5]. An objective measure of volume of blood loss is therefore no longer considered to be suitable, and it is a woman’s subjective assessment of her ability to cope and the perceived impact on her quality of life that is increasingly used to assess treatment success [5]. As impact on quality of life is the key indicator of treatment success, it is important to ensure that the quality-of-life measure is used accurately to reflect women’s concerns and experiences.

Historically, women had surgery to treat menorrhagia; however, non-hormonal and hormonal pharmaceutical treatments are now available as first-line treatment for women with menorrhagia. The first robust, UK-based economic evaluation of these pharmaceutical treatments for menorrhagia was conducted alongside a trial using both EQ-5D and SF-6D to compare levonorgestrel-releasing intrauterine system (LNG-IUS) with usual medical treatment as first-line treatment for menorrhagia [6]. LNG-IUS is an intrauterine device that can be inserted by the general practitioner (GP) and also provides contraception. Usual medical treatment can include one of the following: tranexamic acid, mefenamic acid, norethisterone, depo-provera, or combined estrogen/progestogen or progestogen-only oral contraceptive pill (any formulation), which is prescribed by the GP (a description of each treatment is presented in the online resource).

Concerns around the use of these measures, which are underpinned by the extra-welfarist perspective in menorrhagia, were highlighted as the treatment recommendation to decision makers differed depending on the measure used to generate the QALY [6]. Despite being advocated by decision makers, there is evidence to suggest that these measures, which focus on health, may not be suitable for a condition such as menorrhagia because women believe both health and non-health aspects of life are affected by the condition [7]. Furthermore, the standard recall periods of typically used measures and the episodic nature of the condition also mean results could be affected by the timing of assessment. This combined reasoning raises questions about the suitability of QALYs as an outcome measure.

The WTP measure, underpinned by welfarist theory, enables the respondent to take into consideration both health and non-health outcomes and may overcome the issue of timing of assessment. To demonstrate its feasibility, we explore the use of the WTP approach in a preliminary cost-benefit analysis to assess the cost effectiveness of LNG-IUS compared with usual medical treatment (also referred to as oral treatment) as the first-line treatment for menorrhagia.

2 Methods

A cost-benefit analysis was carried out based on an outcome of WTP. The analysis is related to the UK primary care setting and provides an assessment of the difference in costs and WTP between interventions over a 24-month time horizon. The reporting of the cost-benefit analysis follows the CHEERS guidelines [8].

2.1 Participants and Study Design

For this exploratory study, a convenience sample of 110 women were recruited from general gynaecology outpatient clinics based in the Birmingham Women’s Hospital between December 2012 and January 2013. Women who were menstruating but did not necessarily have experience of menorrhagia or its treatments were sought, so all women attending an appointment were approached to complete a booklet questionnaire, either in the clinic or at home, and provided written informed consent to participate. Respondents who took the questionnaire home to complete were given a stamped addressed envelope. Women were asked to value the two pharmaceutical treatments of LNG-IUS and oral treatment.

2.2 Outcome Measures

WTP is elicited from the ex-ante perspective. Individuals are asked to express in monetary terms how much they value a good or a service that leads to a change in outcome [9]. In this context, maximum WTP values were derived prior to the change in outcome occurring, from respondents who are ‘at risk’ of the condition, or ‘at risk’ of requiring treatment. Given the UK is a tax-funded system that offers healthcare ‘free at the point of use’, we designed the WTP study to elicit the views of the at-risk population. The rationale being that because society is funding the healthcare system, it is the views of those at risk that should be sought.

The questionnaire booklet was reviewed by clinical experts in menorrhagia, psychologists and health economists for face and content validity. Maximum WTP values were elicited for both LNG-IUS and oral treatment using a self-complete booklet questionnaire. The booklet captured data on WTP and sociodemographic details.

A description of menorrhagia (without treatment) was first presented and was based on the domains of the disease-specific quality-of-life Menorrhagia Multi-attribute Assessment Scale (MMAS). This measure incorporates both the health and non-health outcomes associated with menorrhagia and consists of six attributes, ‘practical difficulties, ‘social life’, ‘psychological health’, ‘physical health and well-being’, ‘work/daily routine’ and ‘family life/ relationships’ [10]. We used baseline MMAS data from a recent trial (ECLIPSE, ISRCTN86566246) to generate the description. We then presented a scenario describing the expected average ‘outcome’ associated with the two treatments, LNG-IUS (termed Mirena in the scenarios) and oral treatment, using average follow-up MMAS data from the ECLIPSE trial [11].

Using the same method the scenarios for the outcomes associated with LNG-IUS and oral treatments, using the 6-month ECLIPSE MMAS data, were generated. Information describing the process of care was also described in the treatment scenarios (see online resource for method used for scenario development).

Respondents were asked for their preferred treatment, and their maximum monthly out-of-pocket WTP value up until menopause first for oral treatment, and then LNG-IUS. A payment scale, which presents respondents with a range of monetary values, was used to elicit WTP values as it has a higher completion rate than other methods that can be used in a postal questionnaire [12]. The payment scale was derived from a previous applied WTP study [12], and used a range from £0 to £500, which was considered to be most suitable, as the questionnaire asked respondents to provide a monthly WTP value. An open-ended option for values greater than £500 was offered. Following the WTP question, we asked respondents to outline the reasons for their WTP values in an open-ended question to assess the validity of the WTP responses. The respondents were then asked to indicate whether they found the WTP question difficult to answer, and to provide reasons for their response. The time frame of payment ‘up until menopause’ was explicitly stated to ensure that WTP values were not overestimated [13]. The questionnaire included a reminder to consider the amount that they can afford to pay to ensure that the responses obtained were realistic and within the respondent’s means [14]. The time period was intuitive given the nature of the condition. The monthly payment time frame was used because women generally pay monthly (or every 3 months) for prescriptions for menorrhagia, for sanitary protection and will experience the benefits of treatment on a monthly basis.

The booklet questionnaire is presented in the online resource.

2.3 Cost and Resource Use

Given that an ex-ante perspective was adopted, the women were not typically being treated with LNG-IUS or oral treatment, and therefore primary cost data were not available. The costs were consequently derived using the ECLIPSE trial data as the most appropriate available source and also to enable comparability between the cost-utility analysis alongside the ECLIPSE trial and our cost-benefit analysis [6]. Briefly, the general healthcare costs for both treatments included healthcare staff costs and the cost of the treatments. The costs of LNG-IUS and oral treatment were estimated using the British National Formulary [15]. Staff costs were calculated using the nationally recognised reference costs [16]. All costs are reported in 2011 prices in UK (£) sterling using the UK hospital and community health services index [16]. The overall costs for both LNG-IUS and oral treatment at the 2-year time point in the ECLIPSE trial included crossover between treatment arms, as the analysis within the trial was ‘intention to treat’. The average costs of LNG-IUS and oral treatment per person were taken from the average results of a trial-based economic evaluation, where a decision model was used as the basis for the evaluation, and were reported to be £430 and £330, respectively [6]. All costs are from a UK National Health Service perspective. A societal perspective for costs was considered but was not used to enable a comparison between previous analyses using EQ-5D and SF-6D [6].

2.4 Analysis

Average maximum WTP values are compared to the cost of providing the service to generate the net present value (NPV) for each treatment option. If the present value of benefits (expressed through WTP) outweighs the present value of costs (present value of benefits − present value of costs), then the net benefits are said to be positive (NPV > 0) and it is in society’s interest to recommend the treatment choice. The treatment choice that yields the maximum NPV is the most efficient.

The incremental net benefit that shows the difference between the net benefits across the treatments (NPV oral treatment − NPV LNG-IUS) is also presented. To adjust to the present value, the recommended discount rate of 3.5 % was applied to both the costs and outcomes [3]. The WTP values derived for both LNG-IUS and oral treatment were based on a monthly amount, to obtain the present value the WTP value was discounted for every month up to and including 24 months. WTP data were found to be non-normal and were therefore log transformed [17]. A paired t-test was then applied to the log transformed data to explore the difference between the WTP values for each treatment. Protest answers and non-response were removed from the analysis.

The base-case analysis is presented using the cost data described above, which relates to the outcome of the economic evaluation alongside the ECLIPSE trial [6] and is based on an ‘intention-to-treat’ analysis. The base-case analysis was carried out in addition to two sensitivity analyses to assess uncertainty in the results.

Sensitivity analysis:

  1. 1.

    An assessment of uncertainty in the mean NPV is carried out by bootstrapping. Bootstrapping involves randomly sampling values, with replacement from the observed values. Multiple samples are drawn, as 1000 bootstrap datasets are generated using STATA (Version 11.0), and each dataset is considered to be a reiteration of the trial [18]. Bootstrapped 95 % confidence intervals around the mean values are presented and the distributions of the bootstrapped values are then presented graphically.

  2. 2.

    A second sensitivity analysis using alternative cost data, which were not derived from the ECLIPSE model, was applied to identify the impact of the source of cost data. In this sensitivity analysis, the resource use and cost data related to the exclusive use of either LNG-IUS or oral treatment are applied, treatment cross-over is not considered. Table 1 outlines the cost data used in the sensitivity analysis. As oral treatment comprises a range of pharmaceutical treatments, the average cost of oral treatment was weighted according to the frequency with which each treatment is prescribed [6].

    Table 1 Cost data used in sensitivity analysis
    Full size table

3 Results

3.1 Base-Case Results

The maximum average WTP for LNG-IUS was £365 and for oral treatment was £372. This difference was not statistically significant (p = 0.1247; p < 0.05). The maximum average WTP for oral treatment was 13 % higher than the cost of the intervention, and the maximum average WTP for LNG-IUS was 15 % lower than the cost of treatment (Table 2).

Table 2 Base-case results: mean WTP and cost of treatment
Full size table

The base-case results indicate that oral treatment provides a positive NPV of £45, resulting in a welfare gain, and LNG-IUS produces a negative NPV of £−68, leading to a welfare loss (Table 2). When comparing the two treatments, the incremental net benefit exceeds zero suggesting that oral treatment is cost beneficial compared with LNG-IUS. Based on the mean values, oral treatment could be considered the most cost-beneficial intervention.

3.2 Sensitivity Analysis

In sensitivity analysis 1, the confidence intervals associated with the NPVs for both treatments overlap. The NPV is £−68 [95 % CI £−186 to £50] for LNG-IUS and £45 [95 % CI £−55 to £146] for oral treatment. This suggests there is some uncertainty between which treatment is most cost beneficial. However, when presented using graphed plots the bootstrapped NPV for LNG-IUS and oral treatments show a clearer picture with respect to the welfare gains and losses (Figs. 1, 2). In most cases, oral treatment produces a positive NPV, as a greater proportion of the bootstrapped NPV values lie above £0 (Fig. 1). In contrast, the plots for LNG-IUS are the inverse of those for oral treatment, as in most cases LNG-IUS produces a negative NPV. These bootstrapped plots suggest that oral treatment is more likely to be cost beneficial relative to LNG-IUS and reinforce the base-case result.

Fig. 1
figure 1

Base-case results: bootstrapped net present value—oral treatment

Full size image
Fig. 2
figure 2

Base-case results: bootstrapped net present value—levonorgestrel-releasing intrauterine system

Full size image

In sensitivity analysis 2, the mean WTP for LNG-IUS is 41 % greater than the cost of LNG-IUS. The mean WTP for oral treatment is 280 % greater than the cost of oral treatment (Table 3). Both treatments generate a positive NPV.

Table 3 Sensitivity analysis: mean WTP and cost of treatment
Full size table

The results still indicate that oral treatment remains the most cost-beneficial treatment as it generates a greater NPV than LNG-IUS and the incremental net benefit exceeds zero.

3.3 Response to Outcome Measure

One hundred and ten women completed and returned the questionnaire. Both LNG-IUS and oral treatment received the same number of non-responses (four in each). Seven protest answers, which relate to the individual refusing to provide a WTP value, were identified from the qualitative explanations offered.

Ninety-nine women with an average age of 37 years provided a WTP value for LNG-IUS and oral treatment. LNG-IUS was the preferred treatment (47), followed by oral treatment (39) and no preference (11). Two respondents did not answer the question (see online resource for further information). Eighty percent of women said they had experience of heavy periods at one time in their lives, but this may not necessarily mean experience of heavy periods over consecutive cycles as defined by menorrhagia. The two most commonly cited reasons for a WTP value were related to the ‘effect of the treatment’ and ‘affordability’. There were three respondents that misunderstood the WTP question.

Over 60 % of women who completed at least one WTP question said that the question was not difficult to answer. Of those who did find the question difficult to answer, the most common reason was related to ‘not being used to valuing healthcare’. For those who did not find the valuation difficult, the most commonly cited reason was ‘a reasonable amount to pay for the expected benefits’.

4 Discussion

Over a 24-month time horizon, the total cost of oral treatment is cheaper than LNG-IUS (£326 compared with £433 respectively). The NPV of oral treatment is greater than LNG-IUS (£45 compared with £−68, respectively). Thus, oral treatment produced a positive incremental net benefit (equal to £113). On the basis of these results, oral treatment could be recommended as the first-line treatment for menorrhagia.

The findings from both sensitivity analyses support the base-case analysis. The bootstrapped plots in sensitivity analysis 1 demonstrate that oral treatment is the most likely treatment to be cost beneficial. In sensitivity analysis 2, where cost data are not taken from the ECLIPSE trial decision model, which used intention-to-treat analysis but instead relate to the exclusive use of either LNG-IUS or oral treatment, both oral treatment and LNG-IUS generated a positive NPV of £274 and £106 respectively. However, oral treatment yielded the maximum NPV and therefore was still indicated to be the most efficient choice.

Therefore, the base-case analysis and sensitivity analyses suggest that oral treatment is the most cost-beneficial treatment and therefore should be recommended as the first-choice treatment for menorrhagia in clinical practice.

In a privately financed healthcare system, the resource allocation decision from a cost-benefit analysis is relatively straightforward as a positive NPV indicates that the intervention(s) be recommended for use in practice. In contrast, when making resource allocation decisions in a publicly funded healthcare system where a budget constraint exists, it is unlikely to be feasible that all interventions with a positive NPV are recommended for clinical practice [18]. Under budget constraints, the aim is to maximise benefits and therefore the interventions could be ranked against one another and the intervention with the greatest NPV implemented [9]. Whilst this issue is not resolved, in this case, we adopted the decision rule that the treatment choice that yields the maximum NPV is the most efficient and should be implemented.

4.1 Strengths and Limitations

This is the first study, to our knowledge, that applies a cost-benefit analysis to compare LNG-IUS against oral treatment in menorrhagia. Furthermore, a cost-benefit analysis is rarely conducted and reported in the literature and a strength of the current analysis, is that it is based on WTP values that have been elicited from the ex-ante perspective, which is theoretically preferred [9]. An additional strength is that once the questionnaires were developed, they were checked by clinical experts in menorrhagia, by psychologists and external health economists to assess and improve their face and content validity. Thus, rather than basing the ex-ante questionnaire scenarios, for menorrhagia and treatment effectiveness, on expert opinion alone or expected outcomes, novel methods were used to base the scenarios on observed evidence from the ECLIPSE trial, which increases the reliability of the findings.

A limitation of the exploratory study is that we did not determine how many women from our convenience sample had experience of the treatments for menorrhagia. This information would help to determine the extent to which our sample reflects a true ex-ante perspective. The sample used does to some extent reflect the ‘at-risk’ population group, which would be made up of both women who have, and do not have, the condition. However, where women have experience of both menorrhagia and its treatments this does not strictly meet the ex-ante criterion.

The costs for the base-case analysis were taken from the average results of a trial-based economic evaluation to enable comparability between that cost-utility analysis and our cost-benefit analysis [6]. A potential limitation of this approach is that the possibility of changing and stopping treatment was not presented in the WTP scenario and therefore would not have been considered when providing a WTP value. However, when using cost data that are only related to the WTP scenario the same treatment was found to be superior. In this case, although the overall cost-benefit decision did not differ, the extent of the welfare gain produced by oral treatment compared with LNG-IUS did vary, and was dependent on the cost data used in the cost-benefit analysis.

It was not possible to conduct a comprehensive cost-benefit analysis because societal costs were not available. Only healthcare costs were considered in this evaluation. The differing perspectives across costs and benefits could bias the results in favour of the benefits of the treatments as a broader perspective is used. However, when assessing incremental net present values across treatments, the impact is limited as the same approach is applied to both interventions assessed. Incorporating societal costs, such as lost productivity and out-of-pocket prescription fees, would not be straightforward because of double counting. Arguably, the WTP outcome already incorporates lost productivity as the WTP scenarios included impact on work/daily routine. Therefore, if changes in productivity are also counted on the cost side of the equation, it is possible that the benefits of treatment are double counted [19]. The other aspect of societal cost is the cost of prescriptions, which would only be relevant to oral treatment and the exclusion of this could be considered as bias in favour of oral treatment. However, we estimate this cost to be small and unlikely to change the treatment recommendation.

Finally, we recognise the limitation associated with reporting costs in the 2011 price year and WTP values in 2013. Different years of valuation are not unusual in health economics as the utility values derived from other standard measures such as EQ-5D, which is based on time trade-off preference values from the 1990s, are similarly not derived during the same year as costs but are presented in the same economic evaluation. Between these two particular price years, inflation has been particularly low and therefore the lack of adjustment would introduce little if any bias. Furthermore, by not changing the cost year we have presented results that are directly comparable to the cost utility analysis results.

Given the limited availability of cost-benefit analyses currently reported in the literature, a strength of the current article is that we have reported all relevant methods and results as explicitly as possible including additional information to that which is required by recommended guidelines, such as CHEERS, to which published economic evaluations are typically recommended to adhere. As far as we are aware, the current study is the first cost-benefit analysis to attempt to follow CHEERS guidelines [8] and the shortcomings of those guidelines in terms of their relevance to the full and clear reporting of economic evaluations that take the form of a cost-benefit analysis have been apparent.

4.2 Comparison with Other Studies

Very few cost-benefit analyses have been published. To the best of our knowledge, this is the only cost-benefit analysis focussing on menorrhagia. A recent cost-benefit analysis has been carried out but in the area of spinal surgery in Switzerland, where WTP was elicited from the ex-post perspective using patient values [20]. The authors suggested that further methodological work be carried out on the use of ex-ante WTP values, as this perspective is recommended for publicly funded healthcare systems. Despite the ex-ante perspective WTP and cost-benefit analysis being theoretically preferred [9], WTP is typically elicited from the ex-post perspective [21].

In terms of comparisons with other UK studies reporting treatment recommendations for menorrhagia, in contrast to our findings, the NICE guidelines recommend LNG-IUS as the first-line treatment for menorrhagia [4]. Similarly, the economic evaluation alongside the ECLIPSE trial using EQ-5D also found LNG-IUS the most cost-effective intervention [6]. However, the recommendation for oral treatment to be first-line treatment in our cost-benefit analysis does correspond with the recommendation from the economic evaluation alongside the ECLIPSE trial using SF-6D [6]. Decision makers currently recommend EQ-5D for the valuation of outcomes [1], therefore LNG-IUS would be considered the most cost-effective treatment, despite other measures demonstrating that LNG-IUS is not the most cost-effective intervention.

4.3 Implications and Further Research

The results of the current analysis are not attempting to overturn the NICE guideline recommendation but instead present an exploration of the use of an alternative measure. The results of this analysis present potentially important issues about the use of the conventional measures from the extra-welfarist perspective, EQ-5D and SF-6D, within the context of decision making for certain diseases such as menorrhagia. The cost-benefit analysis approach showed oral treatment to be the most efficient use of society’s resources. We have previously shown [6] that the type of measure used to value outcomes has important implications for recommendations to decision makers. To improve the generalisability and robustness of the results, more research needs to be conducted using the WTP approach on a larger sample size that more closely resembles the general population.

Further research to explore the role of cost-benefit analysis and the use of the welfarist approach for certain conditions that affect non-health aspects of quality of life is required both generally by methodologists and specifically in applied research for clinical conditions, such as menorrhagia.