Skip to main content

Advertisement

SpringerLink
Log in

The precision of health state valuation by members of the general public using the standard gamble

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Background

Precision is a recognised requirement of patient-reported outcome measures but no previous studies of the precision of methods for obtaining health state values from the general public, based on specific health state descriptions or vignettes, have been carried out. The methodological requirements of policy makers internationally is driving growth in the use of methods to obtain utilities from the general public to inform cost per quality-adjusted life-year (QALY) analyses of health technologies being considered for adoption by health systems.

Methods

The precision of five comparisons of the outcomes of treatments, based on health state descriptions, was assessed against the results of clinical trials which showed a statistically and clinically significant improvement using an internet panel of members of the UK general public. Health states were developed to depict the baseline and post-treatment states from these exemplar clinical trials. Preferences for health states were obtained using bottom-up titrated standard gamble over the internet, and differences between summary health state values corresponding to the treatment and comparator groups within each exemplar study were compared. Results are considered in the context of various estimates for the minimally important difference in utility values.

Results

Participation among members of the internet panel in the five exemplars ranged from 27 to 59. In four of the five exemplars, the utility-based estimates of treatment benefit showed significant differences between groups and were greater than an assumed minimally important difference of 0.1. Mean utility differences between groups were: 0.23 (computerised cognitive behavioural therapy for depression, P < 0.001), 0.11 (hip resurfacing for hip osteoarthritis, P < 0.001), 0.0005 (cognitive behavioural therapy for insomnia, P = 0.98), 0.15 (pulmonary rehabilitation for COPD, P < 0.001) and 0.11 (infliximab for Crohn’s disease, P < 0.001). The confidence intervals around the estimates of utility-based treatment effect in three of the five examples did not exclude the possibility of a difference smaller than a minimally important difference of 0.1. Recent empirical evidence suggests a lower minimally important difference (0.03) may be more appropriate, in which case our results provide further reassurance of preservation of precision in health state description and valuation.

Conclusions

The precision of estimates of treatment effects based on preference data obtained from disease-specific measurements in clinically significant studies of health technologies was acceptable using an internet-based panel of members of the general public and the standard gamble. Definition of the minimally important difference in utility estimates is required to adequately assess precision and should be the subject of further research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Fitzpatrick, R., Davey, C., Buxton, M., & Jones, D. (1998). Evaluating patient based outcome measures for use in clinical trials. Health Technology Assessment, 2(14), i–iv.

  2. Kessler, R. C., & Mroczek, D. K. (1995). Measuring the effects of medical interventions. Medical Care, 33, AS109–AS119.

    PubMed  CAS  Google Scholar 

  3. Stewart, A. L. (1992). Conceptual and methodologic issues in defining quality of life: State of the art. Progress in Cardiovascular Nursing, 7, 3–11.

    Google Scholar 

  4. Testa, M. A., & Simonson, D. C. (1996). Assessment of quality-of-life outcomes. The New England Journal of Medicine, 334, 835–840. doi:10.1056/NEJM199603283341306.

    Article  PubMed  CAS  Google Scholar 

  5. Dolan, P. (1997). Modeling valuations for EuroQol health states. Medical Care, 35, 1095–1108. doi:10.1097/00005650-199711000-00002.

    Article  PubMed  CAS  Google Scholar 

  6. McCabe, C., Stevens, K., & Brazier, J. (2005). Utility scores for the HUI2: An empirical comparison of alternative mapping functions. Medical Care, 43, 627–635. doi:10.1097/01.mlr.0000163666.00471.8e.

    Article  PubMed  Google Scholar 

  7. Stein, K., Dyer, M., Crabb, T., Milne, R., Round, A., Ratcliffe, J., & Brazier, J. (2006). An internet “value of health” panel: Recruitment, participation and compliance. Health and Quality of Life Outcomes, 4, 90.

  8. Lenert, L. A., Cher, D. J., Goldstein, M. K., Bergen, M. R., & Garber, A. (1998). The effect of search procedures on utility elicitations. Medical Decision Making, 18, 76–83. doi:10.1177/0272989X9801800115.

    Article  PubMed  CAS  Google Scholar 

  9. Dolan, P., Gudex, C., Kind, P., & Williams, A. (1996). Valuing health states: A comparison of methods. Journal of Health Economics, 15, 209–231. doi:10.1016/0167-6296(95)00038-0.

    Article  PubMed  CAS  Google Scholar 

  10. Dolan, P., & Sutton, M. (1997). Mapping visual analogue scale health state valuations onto standard gamble and time trade-off values. Social Science and Medicine, 44, 1519–1530. doi:10.1016/S0277-9536(96)00271-7.

    Article  PubMed  CAS  Google Scholar 

  11. Hammerschmidt, T., Zeitler, H.-P., Gulich, M., & Leidl, R. (2004). A comparison of different strategies to collect standard gamble utilities. Medical Decision Making, 24, 493–503. doi:10.1177/0272989X04269239.

    Article  PubMed  Google Scholar 

  12. Brazier, J., & Dolan, P. (2005). Evidence of preference construction in a comparison of variants of the standard gamble method. Health Economics and Decision Science Section Discussion Papers, University of Sheffield.

  13. Lenert, L. A., & Sturley, A. E. (2002). Use of the internet to study the utility values of the public. In AMIA annual symposium proceedings (pp. 440–444).

  14. Selmi, P. M., Klein, M. H., Greist, J. H., Sorrell, S. P., & Erdman, H. P. (1990). Computer-administered cognitive-behavioral therapy for depression. The American Journal of Psychiatry, 147, 51–56.

    PubMed  CAS  Google Scholar 

  15. Beck, A. T., Rial, W. Y., & Rickels, K. (1974). Short form of depression inventory: Cross-validation. Psychological Reports, 34, 1184–1186.

    PubMed  CAS  Google Scholar 

  16. McMinn, D., Treacy, R., Lin, K., & Pynsent, P. (1996). Metal on metal surface replacement of the hip. Clinical Orthopaedics and Related Research, 329, 89S–98S. doi:10.1097/00003086-199608001-00009.

    Article  Google Scholar 

  17. Vale, L., Wyness, L., McCormack, K., McKenzie, L., Brazelli, M., & Stearns, S. (2001). Systematic review of the effectiveness and cost effectiveness of metal on metal hip resurfacing for treatment of hip disease. Health Services Research Unit, University of Aberdeen.

  18. Morgan, K., Dixon, S., Mathers, N., Thompson, J., & Tomeny, M. (2004). Psychological treatment for insomnia in the regulation of long-term hypnotic drug use. Health Technology Assessment, 8, 1–94.

    Google Scholar 

  19. Buysse, B. J., Reynolds, C. F., Monk, T. H., Berman, S. R., & Kupfer, D. J. (1989). The Pittsburgh Sleep Quality Index for psychiatric practice and research. Psychiatry Research, 28, 193–213. doi:10.1016/0165-1781(89)90047-4.

    Article  PubMed  CAS  Google Scholar 

  20. Man, W. D., Polkey, M. I., Donaldson, N., Gray, B. J., & Moxham, J. (2004). Community pulmonary rehabilitation after hospitalisation for acute exacerbations of chronic obstructive pulmonary disease: Randomised controlled study. BMJ (Clinical Research Ed.), 329, 1209. doi:10.1136/bmj.38258.662720.3A.

    Article  Google Scholar 

  21. Lacasse, Y., Wong, E., & Guyatt, G. (1974). A systematic overview of the measurement properties of the chronic respiratory questionnaire. Canadian Respiratory Journal, 4, 131–139.

    Google Scholar 

  22. Guyatt, G., Mitchell, A., Irvine, E. J., Singer, J., Williams, N., Goodacre, R., et al. (1989). A new measure of health status for clinical trials in inflammatory bowel disease. Gastroenterology, 96, 804–810.

    PubMed  CAS  Google Scholar 

  23. Hanauer, S. B., Feagan, B. G., Lichtenstein, G. R., Mayer, L. F., Schreiber, S., Colombel, J. F., et al. (2002). Maintenance infliximab for Crohn’s disease: The ACCENT I randomised trial. Lancet, 359, 1541–1549. doi:10.1016/S0140-6736(02)08512-4.

    Article  PubMed  CAS  Google Scholar 

  24. Stein, K., & Milne, R. (1998). Health technology assessment. In M. Baker & S. Kirk (Eds.), Research and development in the NHS. Oxford: Radcliffe Medical.

    Google Scholar 

  25. Williams, A. (1995). The measurement and valuation of health: A chronicle. University of York, York.

  26. Garside, R., Stein, K., Castelnuovo, E., Pitt, M., Aschcroft, D., Dimmock, P., et al. (2005). The effectiveness and cost-effectiveness of pimecrolimus and tacrolimus for atopic eczema: A systematic review and economic evaluation. Health Technology Assessment, 9, 1–264.

    Google Scholar 

  27. Juniper, E. F., Guyatt, G., Willan, A., & Griffith, L. (1994). Determining the minimal clinically important change in a disease-specific quality of life questionnaire. Journal of Clinical Epidemiology, 47, 81–87. doi:10.1016/0895-4356(94)90036-1.

    Article  PubMed  CAS  Google Scholar 

  28. Barrett, B., Brown, D., Mundt, M., & Brown, R. (2005). Sufficiently important difference: Expanding the framework of clinical significance. Medical Decision Making, 25, 250–261. doi:10.1177/0272989X05276863.

    Article  PubMed  Google Scholar 

  29. Brozek, J., Guyatt, G. H., & Schunemann, H. (2006). How a well-grounded minimal important difference can enhance transparency of labelling claims and improve interpretation of a patient reported outcome measure. Health and Quality of Life Outcomes, 4, 69.

  30. Brazier, J., Roberts, J., & Deverill, M. (2002). The estimation of a preference-based measure of health from the SF-36. Journal of Health Economics, 21, 292. doi:10.1016/S0167-6296(01)00130-8.

    Article  Google Scholar 

  31. Kulkarni, A. V. (2006). Distribution-based and anchor-based approaches provided different interpretability estimates for the Hydrocephalus Outcome Questionnaire. Journal of Clinical Epidemiology, 59, 176–184. doi:10.1016/j.jclinepi.2005.07.011.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Funding

NHS R&D Programme; National Institute for Health and Clinical Excellence (NICE); NHS Quality Improvement Scotland (NHSQIS). We are extremely grateful to the following for their help: the members of the internet panel, the patients and clinicians who provided help in the development of health state descriptions, Joanne Perry for her project support, Dan Fall (University of Sheffield) and Stephen Elliott (Llama Digital) for website development.

Competing interests

None.

Authors’ contributions

K.S., R.M., J.B. and A.R. conceived the study and, with J.R., designed the evaluation. M.D. developed some of the health state descriptions and contributed to data collection. All authors contributed to the drafting of this report.

Author information

Authors and Affiliations

  1. Peninsula Technology Assessment Group, Peninsula Medical School, University of Exeter, Exeter, UK

    Ken Stein, Matthew Dyer & Alison Round

  2. University of Southampton, Southampton, UK

    Ruairidh Milne

  3. University of Sheffield, Sheffield, UK

    Julie Ratcliffe & John Brazier

Authors
  1. Ken Stein
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthew Dyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ruairidh Milne
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alison Round
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Julie Ratcliffe
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. John Brazier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ken Stein.

Appendix 1: Vignettes

Appendix 1: Vignettes

Depression

The 21 items in the BDI were reviewed and where it was considered they addressed similar constructs, were considered as a single potential statement for the vignette. The wording used in the BDI items was retained as much as possible, with the severity of statements being scaled differently according to the target BDI category. For the mild vignette, all items were scaled as being at level one and for moderate most were considered to be level two. In the minimal vignette, self-harm was not included. The remaining statements were based on those from the mild vignette with compound statements having the first part at level zero (no problems) and the second part at level one (some problems) e.g. “your appetite is poor but you are no more worried about your health than usual”.

Mild depression

  • You feel sad and guilty a good part of the time and feel disappointed in yourself.

  • You cry more than you used to and are discouraged about the future.

  • You feel you have failed more than the average person and feel you may be punished.

  • You have lost interest in other people and get annoyed or irritated easily.

  • You have thoughts of killing yourself but would not carry them out.

  • You don’t enjoy things much and it takes an extra effort to get started at doing something or to take decisions.

  • You get tired easily but don’t sleep well.

  • Your appetite is poor and you worry about physical problems such as aches and pains, or upset stomach or constipation.

Minimal depression

  • You feel sad, but don’t feel particularly guilty or disappointed in yourself.

  • You cry more than you used to but feel ok about the future.

  • You feel you have failed more than the average person but don’t feel you may be punished.

  • You have lost some interest in other people but are no more irritated than you ever are.

  • You don’t enjoy things much but can work normally.

  • You get tired more easily than you used to but you are sleeping ok.

  • Your appetite is poor but you are no more worried about your health than usual.

Moderate depression

  • You feel sad all the time and feel disappointed in yourself.

  • You cry more than you used to and feel you have nothing to look forward to.

  • All you can see in your life is a lot of failures and feel you may be punished.

  • You have lost most interest in other people and feel irritated all the time.

  • You have thoughts of killing yourself but would not carry them out.

  • You don’t get any satisfaction out of doing things and have to push yourself very hard to get anything done.

  • You get tired from doing almost anything and wake up 1–2 h earlier than usual and find it hard to get back to sleep.

  • Your appetite is very poor and you are worried about physical problems such as aches and pains, upset stomach or constipation.

Severe depression

  • You feel so sad that you can’t stand it and hate yourself.

  • You used to be able to cry, but now can’t (even though you want to) and feel the future is completely hopeless.

  • You feel you are a complete failure as a person and that you are being punished.

  • You have lost all interest in other people and don’t get irritated at all by the things that used to irritate you.

  • You would kill yourself if you had the chance.

  • You are dissatisfied or bored with everything and can’t work at all.

  • You are too tired to do anything, wake up several hours before usual and can’t get back to sleep.

  • You have no appetite at all and you are so worried about your physical problems that you can’t think about anything else.

Osteoarthritis

Statements for these vignettes were constructed to depict a combination of impacts in terms of pain and mobility that would result in scores consistent with the results of McMinn et al., with the baseline vignette indicating impact “most of the time”. For the postoperative vignette, since pain and walking scores were both between 5 and 6, statements therefore indicated symptoms as happening “about half the time”.

Osteoarthritis baseline

  • You have a condition that affects your hips.

  • One hip is worse than the other.

  • You experience pain in your hip which varies.

  • It is a constant pain which may also affect your knee. On the whole it is tolerable and permits limited activity.

  • On some days the pain makes you stop carrying out activities.

  • The pain is worse after activity.

  • Your walking is limited but you are able to stand for long periods without difficulty.

  • On days when the pain is worse you find it difficult to walk but on good days you are able to walk for quite long distances with the help of a walking stick.

  • The mobility of your hip joint is moderately limited and you often feel stiff in the morning.

Osteoarthritis post operative

  • You have pain in your hip which is slight or intermittent.

  • You experience pain on about three days a week.

  • The pain improves with activity.

  • When your hip is painful you walk with a limp but the rest of the time you are able to walk normally for your age.

  • You do not need a walking stick or other aid, even when your hip is painful.

Insomnia

The habitual sleep efficiency domain is derived from the answers to other questions and was therefore excluded from vignette construction. The sleep disturbances component is made up of responses to nine questions in the PSQI about various reasons for not sleeping. Morgan et al. did not identify the causes of sleep disturbance and items in this domain were therefore condensed into one statement about disturbed sleep. The item on global sleep quality was also excluded. After writing the baseline scenario, the 12-month vignette was adjusted to represent an approximate 50% drop in sleep latency (time spent awake before sleeping), a 10–15% increase in sleep time and twice as many hypnotic free nights per week.

Baseline insomnia

  • It usually takes you between 30 and 60 min to fall asleep at night but sometimes you lie awake for less than 30 min.

  • Usually you sleep for less than 5 h a night.

  • Less than once a week your sleep is disturbed and you wake up at night.

  • You have to take medication once or twice a week to help you sleep.

  • During the day you sometimes lack energy and enthusiasm to get things done and two or three times a month you have difficulty staying awake when driving, eating or socialising.

Insomnia after cognitive behavioural therapy

  • It usually takes you between 30 and 60 min to fall asleep at night but often you lie awake for less than 30 min.

  • Usually you sleep for 5–6 h a night.

  • Less than once a week your sleep is disturbed and you wake up at night.

  • You have to take medication less than once a week to help you sleep.

  • During the day you sometimes lack energy and enthusiasm to get things done and less than once a week you have difficulty staying awake when driving, eating or socialising.

COPD

The CRDQ includes 20 items with seven category Likert responses. Domain-specific mean scores for groups were available from the clinical trial report. The average scores on each domain were described using nine statements (two each from dyspnoea, fatigue and mastery, and three from emotion) reflecting the distribution of items in the original measure. Items in the CRDQ describe the presence and amount of symptoms. All symptoms were included in the health state descriptions. No item-specific scores were available and it was therefore assumed that the item scores were in proportion to the overall domain score.

The following statements have been derived from the results of a clinical trial, using a specific questionnaire. Within that questionnaire, responses were scaled according to how often the people in the study experienced particular problems and how severely they experienced them:

Frequency

Severity

1. All of the time

2. Most of the time

3. A good bit of the time

4. Some of the time

5. A little of the time

6. Hardly any of the time

7. None of the time

1. Extremely

2. Very

3. Quite a bit

4. Moderately

5. Some

6. A little

7. Not at all

Usual care discharge scenario

  • You get short of breath quite a bit when you carry out activities of daily living like having a bath or shower, getting dressed or preparing meals.

  • You get very short of breath when walking around your own home, going out for a walk or shopping.

  • You feel very tired most of the time.

  • You feel moderately energetic some of the time.

  • A little of the time you feel frustrated or impatient and are embarrassed about your coughing and breathing.

  • Some of the time you are upset, anxious and you feel down in the dumps.

  • You are somewhat dissatisfied and unhappy with your personal life.

  • A good bit of the time you feel fearful and panic when you have difficulty getting your breath. Some of the time you feel confident you can deal with your illness.

  • You have complete control of your breathing a good bit of the time.

Usual care three month scenario

  • You get short of breath quite a bit when you carry out activities of daily living like having a bath or shower, getting dressed or preparing meals.

  • You get very short of breath when walking around your own home, going out for a walk or shopping.

  • You feel quite a bit tired a good part of the time.

  • You feel moderately energetic some of the time.

  • A little of the time you feel frustrated or impatient and are embarrassed about your coughing and breathing.

  • Some of the time you are upset, anxious and you feel down in the dumps.

  • You are somewhat dissatisfied and unhappy with your personal life.

  • A good bit of the time you feel fearful and panic when you have difficulty getting your breath. Some of the time you feel confident you can deal with your illness.

  • You have complete control of your breathing most of the time.

Early rehab discharge scenario

  • You get short of breath quite a bit when you carry out activities of daily living like having a bath or shower, getting dressed or preparing meals.

  • You get very short of breath when walking around your own home, going out for a walk or shopping.

  • You feel quite a bit tired a good part of the time.

  • You have little energy most of the time.

  • Some of the time you feel frustrated or impatient and are embarrassed about your coughing and breathing.

  • A good bit of the time you are upset, anxious and you feel down in the dumps.

  • You are generally dissatisfied and unhappy with your personal life.

  • A good bit of the time you feel fearful and panic when you have difficulty getting your breath. Some of the time you feel confident you can deal with your illness.

  • You have complete control of your breathing a little of the time.

Early rehab three month scenario

  • You get moderately short of breath when you carry out activities of daily living like having a bath or shower, getting dressed or preparing meals.

  • You get moderately short of breath when walking around your own home, going out for a walk or shopping.

  • You feel moderately tired some of the time.

  • You have quite a bit of energy a good bit of the time.

  • A good bit of the time you feel frustrated or impatient and are embarrassed about your coughing and breathing.

  • A little of the time you are upset, anxious and you feel down in the dumps.

  • You are very happy with your personal life most of the time.

  • Some of the time you feel fearful and panic when you have difficulty getting your breath. A good bit of the time you feel confident you can deal with your illness.

  • You have complete control of your breathing almost all of the time.

Crohn’s disease: IBDQ—trial baseline

The following statements have been derived from the results of a clinical trial, using a specific questionnaire. Within that questionnaire, responses were scaled according to how often the people in the study experienced particular problems:

  1. 1.

    all of the time

  2. 2.

    most of the time

  3. 3.

    a good bit of the time

  4. 4.

    some of the time

  5. 5.

    a little of the time

  6. 6.

    hardly any of the time

  7. 7.

    none of the time

In this scenario, you experience the following “a good bit of the time”

  • very frequent loose bowel movements, bloating, wind, pain and some nausea

  • soiling of underwear.

  • feeling generally unwell, tired and worn out with low energy.

  • problems maintaining weight.

  • worries about the illness, depression, tearfulness.

  • some anger and irritability.

  • a fair bit of difficulty working and/or carrying out social engagements.

  • somewhat dissatisfied and unhappy with personal life.

IBDQ—remission

The following statements have been derived from the results of a clinical trial, using a specific questionnaire. Within that questionnaire, responses were scaled according to how often the people in the study experienced particular problems:

  1. 1.

    all of the time

  2. 2.

    most of the time

  3. 3.

    a good bit of the time

  4. 4.

    some of the time

  5. 5.

    a little of the time

  6. 6.

    hardly any of the time

  7. 7.

    none of the time

In this scenario, you experience the following, for the most part, “hardly any of the time”

  • very frequent loose bowel movements, bloating, wind, pain and some nausea.

  • soiling of underwear.

  • feeling generally unwell, tired and worn out with low energy.

  • problems maintaining weight.

  • worries about the illness, depression, tearfulness.

  • some anger and irritability.

  • a fair bit of difficulty working and/or carrying out social engagements.

  • somewhat dissatisfied and unhappy with personal life.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Stein, K., Dyer, M., Milne, R. et al. The precision of health state valuation by members of the general public using the standard gamble. Qual Life Res 18, 509–518 (2009). https://doi.org/10.1007/s11136-009-9446-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-009-9446-6

Keywords

Search

Navigation