Skip to main content
SpringerLink
Log in

Validity and Reliability of Willingness-to-Pay Estimates

Evidence from Two Overlapping Discrete-Choice Experiments

  • Conference Paper
  • Published:
The Patient: Patient-Centered Outcomes Research Aims and scope Submit manuscript

Abstract

Background: Discrete-choice experiments (DCEs), while becoming increasingly popular, have rarely been tested for validity and reliability.

Objective: To address the issues of validity and reliability of willingness-to-accept (WTA) values obtained from DCEs. In particular, to examine whether differences in the attribute set describing a hypothetical product have an influence on preferences and willingness-to-pay (WTP) values of respondents.

Methods: Two DCEs were designed, featuring hypothetical insurance contracts for Swiss healthcare. The contract attributes were pre-selzected in expert sessions with representatives of the Swiss healthcare system, and their relevance was checked in a pre-test. Experiment A contained rather radical health system reform options, while experiment B concentrated on more familiar elements such as copayment and the benefit catalogue. Three attributes were present in both experiments: delayed access to innovation (‘innovation’), restricted drug benefit (‘generics’), and the change in the monthly premium (‘premium’). The issue to be addressed was whether WTA values for the overlapping attributes were similar, even though they were embedded in widely differing choice sets.

Two representative telephone surveys with 1000 people aged >25 years were conducted independently in the German and French parts of Switzerland during September 2003. Socioeconomic variables collected included age, sex, education, total household income, place of residence, occupation, and household size. Three models were estimated (a simple linear model, a model allowing interaction of the price attribute with socioeconomic characteristics, and a model with a full set of interaction terms).

Results: The socioeconomic characteristics of the two samples were very similar. Theoretical validity tends to receive empirical support in both experiments in all cases where economic theory makes predictions concerning differences between socioeconomic groups. However, a systematic inappropriate influence on measured WTA seems to be present in at least one experiment. This is likely to be experiment A, in which respondents were far less familiar with proposed alternatives than in experiment B.

Conclusions: Measuring preferences for major, little-known innovations in a reliable way seems to present particular challenges for experimental research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Table I
Table II
Table III
Fig. 1
Table IV
Fig. 2
Table V

Notes

  1. Hanley et al.[2] give a literature overview for convergent validity in the environmental context.

  2. Reduced premiums apply for young adults (aged <26 years) and children.

  3. See also San Miguel et al.[31] on the importance of a priori information for consistency of choices in DCE. The authors propose a summary sheet describing attributes and their levels. Such a sheet was provided in both experiments A and B.

  4. Because of imprecise and partially missing income data of the respondents it was not possible to adequately derive WTA with respect to income.

References

  1. Ryan M, Gerard K. Using discrete choice experiments to value health care programmes: current practice and future reflections. Appl Health Econ Health Policy 2003; 2(1): 55–64

    PubMed  Google Scholar 

  2. Hanley N, Ryan M, Wright R. Estimating the monetary value of health care: lessons from environmental economics. Health Econ 2003; 12: 3–16

    Article  PubMed  Google Scholar 

  3. Scanlon DP, Chernew ME, Lave JR. Consumer health plan choice. Annu Rev Public Health 1997; 18: 507–28

    Article  PubMed  CAS  Google Scholar 

  4. Ryan M. A comparison of stated preference methods for estimating monetary values. Health Econ 2004; 13: 291–6

    Article  PubMed  Google Scholar 

  5. Gyrd-Hansen D, Sogaard J. Analysing public preferences for cancer screening programmes. Health Econ 2001; 10: 617–34

    Article  PubMed  CAS  Google Scholar 

  6. Merino-Castellò A. Demand for pharmaceutical drugs: a choice modelling experiment [working paper]. Barcelona: University of Barcelona, 2003

    Google Scholar 

  7. Ryan M, Wordsworth S. Sensitivity of willingness to pay estimates to the level of attributes in discrete choice experiments. Scott J Polit Econ 2000; 47: 504–24

    Article  Google Scholar 

  8. San Miguel F, Ryan M, Mclntosh E. Applying conjoint analysis in economic evaluations: an application to menorrhagia. Appl Econ 2000; 32: 823–33

    Article  Google Scholar 

  9. Ryan M, Mclntosh E, Shackley P. Methodological issues in the application of conjoint analysis in health care. Health Econ 1998; 7: 373–8

    Article  PubMed  CAS  Google Scholar 

  10. Ryan M, Hughes J. Using conjoint analysis to assess women’s preferences for miscarriage management. Health Econ 1997; 6: 261–73

    Article  PubMed  CAS  Google Scholar 

  11. Scott A, Vick S. Patients, doctors and contracts: an application of principal-agent theory to the doctor-patient relationship. Scott J Polit Econ 1999; 46: 111–34

    Article  Google Scholar 

  12. Slothuus-Skoldborg U, Gyrd-Hansen D. Conjoint analysis: the cost variable. An Achilles’ heel? Health Econ 2003; 12(6): 479–97

    Article  Google Scholar 

  13. Louviere JL, Hensher DA, Swait J. Stated choice methods: analysis and applications. Cambridge (MA): Cambridge University Press, 2000

    Book  Google Scholar 

  14. DeShazo JR, Fermo G. Designing choice sets for stated preference methods: the effects of complexity on choice consistency. J Environ Econ Manage 2002; 44: 123–43

    Article  Google Scholar 

  15. Luce DR. Individual choice behavior. New York: Wiley and Sons, 1959

    Google Scholar 

  16. Manski C, Lerman SR. The estimation of choice probabilities from choice based samples. Econometrica 1977; 45(8): 1977–88

    Article  Google Scholar 

  17. McFadden D. Economic choices. Am Econ Rev 2001; 91(3): 351–78

    Article  Google Scholar 

  18. Lancaster K. Consumer demand: a new approach. New York: Columbia University Press, 1971

    Google Scholar 

  19. Ben-Akiva M, Lerman SR. Discrete choice analysis. Cambridge (MA): The MIT Press, 1985

    Google Scholar 

  20. Jöreskog K, Goldberger AS. Estimation of a model with multiple indicators and multiple causes of a single latent variable. J Am Stat Assoc 1975; 70: 631–9

    Google Scholar 

  21. Schoenberg R, Arminger G. Linear covariance structures version 2.0, user guide. Kensington (MD): RJS Software, 1989

    Google Scholar 

  22. Litwin MS. How to measure survey reliability and validity. London, New Delhi: SAGE Publications, 1995

    Google Scholar 

  23. Bryan S, Gold L, Sheldon R, et al. Preference measurement using conjoint methods: an empirical investigation of reliability, Health Econ 2000; 9: 385–95

    Article  PubMed  CAS  Google Scholar 

  24. Cairns J, van der Pol M. Repeated follow-up as a method for reducing non-trading behaviour in discrete choice experiments. Soc Sci Med 2004; 58: 2211–8

    Article  PubMed  Google Scholar 

  25. Farrar S, Ryan M. Response-ordering effects: a methodological issue in conjoint analysis. Health Econ 1999; 8: 75–9

    Article  PubMed  CAS  Google Scholar 

  26. Verlegh PWJ, Schifferstein HNJ, Wittink DR. Range and number-of-levels effects in derived and stated measures of attribute importance. Marketing Letters 2002; 13(1): 41–52

    Article  Google Scholar 

  27. Ryan M, Bate A. Testing the assumptions of rationality, continuity and symmetry when applying discrete choice experiments in health care. Appl Econ Letters 2001; 8: 59–63

    Article  Google Scholar 

  28. Scott A. Eliciting GP’s preferences for pecuniary and nonpecuniary job characteristics. J Health Econ 2001; 20: 329–47

    Article  PubMed  CAS  Google Scholar 

  29. Scott A. Identifying and analysing dominant preferences in discrete choice experiments: an application in health care. J Econ Psychol 2002; 23: 383–98

    Article  Google Scholar 

  30. Lloyd A. Threats to the estimation of benefit: are preference elicitation methods accurate? Health Econ 2003; 12: 393–402

    Article  PubMed  Google Scholar 

  31. San Miguel F, Ryan M, Amaya-Amaya M. ‘Irrational’ stated preferences: a quantitative and qualitative investigation. Health Econ 2005; 14(3): 307–22

    Article  Google Scholar 

  32. Telser H, Zweifel P. Measuring willingness-to-pay for risk reduction: an application of conjoint analysis. Health Econ 2002; 11(3): 129–39

    Article  PubMed  Google Scholar 

  33. Hausman JA, editor. Contingent valuation: a critical assessment. Amsterdam, London, New York, Tokyo: North-Holland, 1993

    Google Scholar 

  34. Nocera S, Bonato D, Telser H. The contingency of contingent valuation: what are people willing to pay against Alzheimer’s Disease? Int J Health Care Finance Econ 2002; 2: 219–40

    Article  PubMed  Google Scholar 

  35. Telser H, Zweifel P. Validity of discrete-choice experiments: evidence for health risk reduction. Appl Econ 2007; 39(1): 69–78

    Article  Google Scholar 

  36. Kuhfeld WF, Tobias RD, Garratt M. Efficient experimental design with marketing research applications. J Mark Res 1994; 31: 545–57

    Article  Google Scholar 

  37. Hardin RH, Sloane NJA. A new approach to the construction of optimal designs. J Stat Plan Inference 1993; 37: 229–369

    Article  Google Scholar 

  38. Hardin RH, Sloane NJA. Operating manual for Gosset: a general purpose program for constructing experimental designs. 2nd ed. Murray Hill (NJ): AT&T Bell Laboratories, 1994

    Google Scholar 

  39. Hedayat AS, Sloane NJA, Stufken J. Orthogonal arrays: theory and applications. New York, Berlin, Heidelberg: Springer, 1999

    Book  Google Scholar 

  40. Ryan M. Methodological issues in the application of conjoint analysis in health care. Health Econ 1998; 7: 373–8

    Article  PubMed  CAS  Google Scholar 

  41. Ryan M, San Miguel F. Revisiting the axiom of completeness in health care. Health Econ 2003; 12: 295–307

    Article  PubMed  Google Scholar 

  42. Maddala T, Phillips KA, Johnson RF. An experiment on simplifying conjoint analysis designs for measuring preferences. Health Econ 2003; 12: 1035–47

    Article  PubMed  Google Scholar 

  43. Johnson RF, Ozdemir S, Hauber AB. Motivating out-of-pocket treatment costs with cheap talk [working paper]. Research Triangle Park (NC): Research Triangle Institute, 2007

    Google Scholar 

  44. Cook J, Whittington D, Canh DG, et al. Reliability of stated preferences for cholera and typhoid vaccines with time to think in Hue, Vietnam. Econ Inq 2007; 45(1): 100–14

    Article  Google Scholar 

  45. Johnson RF, Desvouges WH. Estimating stated preferences with rated pair data: environmental, health, and employment effects of energy programs. J Environ Econ Manage 1997; 34: 79–99

    Article  Google Scholar 

  46. Samuelson W, Zeckhauser RJ. Status quo bias in decision making. J Risk Uncertain 1988; 1: 7–59

    Article  Google Scholar 

  47. Horowitz JK, McConnell KE. A review of WTA/WTP studies. J Environ Econ Manage 2002; 44: 426–47

    Article  Google Scholar 

  48. Zweifel P, Telser H, Vaterlaus S. Consumer resistance against regulation; the case of health care. J Regulatory Econ 2006; 29(3): 319–32

    Article  Google Scholar 

  49. Bundesamt für Sozialversicherung. Bern: Prämien, 2003

  50. Andreoni J. Giving with impure altruism: applications to charity and ricardian equivalence. J Polit Econ 1989; 97(6): 1447–58

    Article  Google Scholar 

  51. Andreoni J. Warm-glow versus cold-prickle: the effects of positive and negative framing on cooperation in experiments. Q J Econ 1995; 110(1): 1–21

    Article  Google Scholar 

  52. Shepard DS, Zeckhauser RJ. Survival and consumption. Manage Sci 1994; 30(4): 423–39

    Article  Google Scholar 

  53. Becker K, Zweifel P. Age and choice in health insurance: evidence from Switzerland. Patient 2008; 1(1): 27–40

    Article  PubMed  Google Scholar 

  54. Dellaert BGC, Brazell JD, Louviere JL. The effect of attribute variation on consumer choice. Marketing Letters 1999; 10(2): 139–47

    Article  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge financial support from Interpharma, Santesuisse, the Association of Swiss Pharma Companies (VIPS), MSD Switzerland, the Federal Social Insurance Office (FSIO), the State Secretariat for Economic Affairs SECO, the Swiss Medical Students’ Association, and the Merian-Iselin Hospital. The authors have no conflicts of interest that are directly relevant to the content of this review.

Author information

Authors and Affiliations

  1. Polynomics, Baslerstrasse 44, Olten, CH-4600, Switzerland

    Harry Telser & Karolin Becker

  2. Socioeconomic Institute, University of Zurich, Zurich, Switzerland

    Harry Telser & Peter Zweifel

Authors
  1. Harry Telser
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karolin Becker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter Zweifel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Harry Telser.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Cite this article

Telser, H., Becker, K. & Zweifel, P. Validity and Reliability of Willingness-to-Pay Estimates. Patient-Patient-Centered-Outcome-Res 1, 283–298 (2008). https://doi.org/10.2165/1312067-200801040-00010

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.2165/1312067-200801040-00010

Keywords

Search

Navigation