Skip to main content
SpringerLink
Log in

Accounting for Scale Heterogeneity in Healthcare-Related Discrete Choice Experiments when Comparing Stated Preferences: A Systematic Review

  • Systematic Review
  • Published:
The Patient - Patient-Centered Outcomes Research Aims and scope Submit manuscript

Abstract

Background

Scale heterogeneity, or differences in the error variance of choices, may account for a significant amount of the observed variation in the results of discrete choice experiments (DCEs) when comparing preferences between different groups of respondents.

Objective

The aim of this study was to identify if, and how, scale heterogeneity has been addressed in healthcare DCEs that compare the preferences of different groups.

Methods

A systematic review identified all healthcare DCEs published between 1990 and February 2016. The full-text of each DCE was then screened to identify studies that compared preferences using data generated from multiple groups. Data were extracted and tabulated on year of publication, samples compared, tests for scale heterogeneity, and analytical methods to account for scale heterogeneity. Narrative analysis was used to describe if, and how, scale heterogeneity was accounted for when preferences were compared.

Results

A total of 626 healthcare DCEs were identified. Of these 199 (32%) aimed to compare the preferences of different groups specified at the design stage, while 79 (13%) compared the preferences of groups identified at the analysis stage. Of the 278 included papers, 49 (18%) discussed potential scale issues, 18 (7%) used a formal method of analysis to account for scale between groups, and 2 (1%) accounted for scale differences between preference groups at the analysis stage. Scale heterogeneity was present in 65% (n = 13) of studies that tested for it. Analytical methods to test for scale heterogeneity included coefficient plots (n = 5, 2%), heteroscedastic conditional logit models (n = 6, 2%), Swait and Louviere tests (n = 4, 1%), generalised multinomial logit models (n = 5, 2%), and scale-adjusted latent class analysis (n = 2, 1%).

Conclusions

Scale heterogeneity is a prevalent issue in healthcare DCEs. Despite this, few published DCEs have discussed such issues, and fewer still have used formal methods to identify and account for the impact of scale heterogeneity. The use of formal methods to test for scale heterogeneity should be used, otherwise the results of DCEs potentially risk producing biased and potentially misleading conclusions regarding preferences for aspects of healthcare.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. McFadden D. The choice theory approach to market research. Mark Sci. 1986;5:275–97.

    Article  Google Scholar 

  2. Wright SJ, Ulph F, Dharni N, Payne K. Eliciting preferences for information provision in newborn bloodspot screening programs. Value Health. 2017;20:651–61.

    Article  Google Scholar 

  3. Mirelman A, Mentzakis E, Kinter E, Paolucci F, Fordham R, Ozawa S, et al. Decision-making criteria among national policymakers in five countries: a discrete choice experiment eliciting relative preferences for equity and efficiency. Value Health. 2012;15:534–9.

    Article  Google Scholar 

  4. Bansback N, Hole AR, Mulhern B, Tsuchiya A. Testing a discrete choice experiment including duration to value health states for large descriptive systems: addressing design and sampling issues. Soc Sci Med. 2014;114:38–48.

    Article  Google Scholar 

  5. de Bekker-Grob EW, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2012;21:145–72.

    Article  Google Scholar 

  6. Clark M, Determann D, Petrou S, Moro D, de Bekker-Grob EW. Discrete choice experiments in health economics: a review of the literature. Pharmacoeconomics. 2014;32:883–902.

    Article  Google Scholar 

  7. StataCorp. Stata Statistical Software: Release 12. College Station, TX: StataCorp LP; 2011.

  8. Sawtooth. Sawtooth Software SSI Web 8.3.8. 2012.

  9. Econometric Software Inc. NLOGIT. 2015. Available from: http://www.limdep.com/. Accessed 26 Feb 2018.

  10. Hole AR. Modelling heterogeneity in patients’ preferences for the attributes of a general practitioner appointment. J Health Econ. 2008;27:1078–94.

    Article  Google Scholar 

  11. Hensher D, Greene W. The mixed logit model: the state of practice. Transport. 2003;30:133–76.

    Article  Google Scholar 

  12. Greene WH, Hensher D. A latent class model for discrete choice analysis: contrasts with mixed logit. Transp Res Part B Methodol. 2003;37:681–98.

    Article  Google Scholar 

  13. Huber J, Train K. On the similarity of classical and bayesian estimates of individual mean partworths. Mark. Lett. 2001;123:259–69.

    Article  Google Scholar 

  14. Vass CM, Wright S, Burton M, Payne K. Scale heterogeneity in healthcare discrete choice experiments: a primer. Patient. 2017. https://doi.org/10.1007/s40271-017-0282-4 (Epub 14 Oct 2017).

    Article  Google Scholar 

  15. Innovations Statistical. Latent gold. Belmont: Statistical Innovations; 2013.

    Google Scholar 

  16. Hess S, Train K. Correlation and scale in mixed logit models. J Choice Model. 2017;23:1–8.

    Article  Google Scholar 

  17. Hensher D, Louviere J, Swait J. Combining sources of preference data. J Econ. 1998;89:197–221.

    Article  Google Scholar 

  18. Swait J, Louviere J. The role of the scale parameter in the estimation and comparison of multinomial logit models. J Mark Res. 1993;30:305–14.

    Article  Google Scholar 

  19. Moher D, Liberati A, Tetzlaff J, Altman DG, Altman D. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097.

    Article  Google Scholar 

  20. Vass C, Gray E, Payne K. Discrete choice experiments of pharmacy services: a systematic review. Int J Clin Pharm. 2016;38(3):620–30.

    Google Scholar 

  21. Vass C, Rigby D, Payne K. The role of qualitative research methods in discrete choice experiments. Med Decis Mak. 2017;37:298–313.

    Article  Google Scholar 

  22. Ryan M, Gerard K. Using discrete choice experiments to value health care programmes: current practice and future research reflections. Appl Health Econ Health Policy. 2003;2:55–64.

    Google Scholar 

  23. Barnes AJ, Hanoch Y, Rice T. Determinants of coverage decisions in health insurance marketplaces: consumers’ decision-making abilities and the amount of information in their choice environment. Health Serv Res. 2015;50:58–80.

    Article  Google Scholar 

  24. Koopmanschap MA, Stolk EA, Koolman XHE. Dear policy maker: have you made up your mind? A discrete choice experiment among policy makers and other health professionals. Int J Technol Assess Health Care. 2010;26:198–204.

    Article  Google Scholar 

  25. Ostermann J, Njau B, Mtuy T, Brown DS, Muhlbacher A, Thielman N. One size does not fit all: HIV testing preferences differ among high-risk groups in Northern Tanzania. AIDS Care. 2015;27:595–603.

    Article  Google Scholar 

  26. Araña JE, León CJ, Quevedo JL. The effect of medical experience on the economic evaluation of health policies. A discrete choice experiment. Soc Sci Med. 2006;63:512–24.

    Article  Google Scholar 

  27. Bech M, Kjaer T, Lauridsen J. Does the number of choice sets matter? Results from a web survey applying a discrete choice experiment. Health Econ. 2011;20:273–86.

    Article  Google Scholar 

  28. Hall J, Fiebig DG, King MT, Hossain I, Louviere JJ. What influences participation in genetic carrier testing? Results from a discrete choice experiment. J Health Econ. 2006;25:520–37.

    Article  Google Scholar 

  29. Van Helvoort-Postulart D, Dellaert BGC, Van Der Weijden T, Von Meyenfeldt MF, Dirksen CD. Discrete choice experiments for complex health-care decisions: does hierarchical information integration offer a solution ? Health Econ. 2009;920:903–20.

    Article  Google Scholar 

  30. Holte JH, Kjaer T, Abelsen B, Olsen JA. The impact of pecuniary and non-pecuniary incentives for attracting young doctors to rural general practice. Soc Sci Med. 2015;128:1–9.

    Article  Google Scholar 

  31. Kinter ET, Prior TJ, Carswell CI, Bridges JFP. A comparison of two experimental design approaches in applying conjoint analysis in patient-centered outcomes research: a randomized trial. Patient. 2012;5:279–94.

    Article  Google Scholar 

  32. Knox SA, Viney RC, Gu Y, Hole AR, Fiebig DG, Street DJ, et al. The effect of adverse information and positive promotion on women’s preferences for prescribed contraceptive products. Soc Sci Med. 2013;83:70–80.

    Article  Google Scholar 

  33. Meghani SH, Chittams J, Hanlon AL, Curry J. Measuring preferences for analgesic treatment for cancer pain: how do African-Americans and Whites perform on choice-based conjoint (CBC) analysis experiments? BMC Med Inform Decis Mak. 2013;13:118.

    Article  Google Scholar 

  34. Milte R, Ratcliffe J, Chen G, Lancsar E, Miller M, Crotty M. Cognitive overload? An exploration of the potential impact of cognitive functioning in discrete choice experiments with older people in health care. Value Health. 2014;17:655–9.

    Article  Google Scholar 

  35. Owen K, Pettman T, Haas M, Viney R, Misan G. Individual preferences for diet and exercise programmes: changes over a lifestyle intervention and their link with outcomes. Public Health Nutr. 2010;13:245–52.

    Article  Google Scholar 

  36. Payne K, Fargher EA, Roberts SA, Tricker K, Elliott RA, Ratcliffe J, et al. Valuing pharmacogenetic testing services: a comparison of patients’ and health care professionals’ preferences. Value Health. 2011;14:121–34.

    Article  Google Scholar 

  37. Pedersen LB, Kjær T, Kragstrup J, Gyrd-Hansen D. Do general practitioners know patients’ preferences? an empirical study on the agency relationship at an aggregate level using a discrete choice experiment. Value Health. 2012;15:514–23.

    Article  Google Scholar 

  38. Salampessy BH, Veldwijk J, Jantine Schuit A, van den Brekel-Dijkstra K, Neslo REJ, Ardine de Wit G, et al. The predictive value of discrete choice experiments in public health: an exploratory application. Patient. 2015;8:521–9.

    Article  Google Scholar 

  39. Veldwijk J, Lambooij MS, van Til JA, Groothuis-Oudshoorn CGM, Smit HA, de Wit GA. Words or graphics to present a discrete choice experiment: does it matter? Patient Educ Couns. 2015;98:1376–84.

    Article  Google Scholar 

  40. Finkelstein EA, Bilger M, Flynn TN, Malhotra C. Preferences for end-of-life care among community-dwelling older adults and patients with advanced cancer: a discrete choice experiment. Health Policy. 2015;119:1482–9.

    Article  Google Scholar 

  41. Miller F, Hayeems RZ, Bombard Y, Cressman C, Barg CJ, Carroll JC, et al. Public perceptions of the benefits and risks of newborn screening. Pediatrics. 2015;136:e413–23.

    Article  Google Scholar 

  42. Carroll FE, Al-Janabi H, Flynn T, Montgomery AA. Women and their partners’ preferences for Down’s syndrome screening tests: a discrete choice experiment. Prenat Diagn. 2013;33:449–56.

    Article  Google Scholar 

  43. Erdem S, Thompson C. Prioritising health service innovation investments using public preferences: a discrete choice experiment. BMC Health Serv Res. 2014;14:360.

    Article  Google Scholar 

  44. Fiebig DG, Haas M, Hossain I, Street DJ, Viney R. Decisions about pap tests: what influences women and providers? Soc Sci Med. 2009;68:1766–74.

    Article  Google Scholar 

  45. Arbiol J, Yabe M, Nomura H, Borja M, Gloriani N, Yoshida SI. Using discrete choice modeling to evaluate the preferences and willingness to pay for leptospirosis vaccine. Hum Vaccines Immunother. 2015;11:1046–56.

    Article  Google Scholar 

  46. de Bekker-Grob EW, Bliemer MCJ, Donkers B, Essink-Bot M-L, Korfage IJ, Roobol MJ, et al. Patients’ and urologists’ preferences for prostate cancer treatment: a discrete choice experiment. Br J Cancer. 2013;109:633–40.

    Article  Google Scholar 

  47. Essers BAB, Van Helvoort-Postulart D, Prins MH, Neumann M, Dirksen CD. Does the inclusion of a cost attribute result in different preferences for the surgical treatment of primary basal cell carcinoma? A comparison of two discrete-choice experiments. Pharmacoeconomics. 2010;28:507–20.

    Article  Google Scholar 

  48. Gerard K, Salisbury C, Street D, Pope C, Baxter H. Is fast access to general practice all that should matter? A discrete choice experiment of patients’ preferences. J Health Serv Res Policy. 2008;13(Suppl 2):3–10.

    Article  Google Scholar 

  49. Gidman W, Elliott R, Payne K, Meakin GH, Moore J. A comparison of parents and pediatric anesthesiologists’ preferences for attributes of child daycase surgery: a discrete choice experiment. Paediatr Anaesth. 2007;17:1043–52.

    Article  Google Scholar 

  50. Reed Johnson F, Ozdemir S, Phillips KA. Effects of simplifying choice tasks on estimates of taste heterogeneity in stated-choice surveys. Soc Sci Med. 2010;70:183–90.

    Article  Google Scholar 

  51. Özdemir S, Johnson FR, Hauber AB. Hypothetical bias, cheap talk, and stated willingness to pay for health care. J Health Econ. 2009;28:894–901.

    Article  Google Scholar 

  52. Rockers PC, Jaskiewicz W, Wurts L, Kruk ME, Mgomella GS, Ntalazi F, et al. Preferences for working in rural clinics among trainee health professionals in Uganda: a discrete choice experiment. BMC Health Serv Res. 2012;12:212.

    Article  Google Scholar 

  53. Scott A, Witt J, Duffield C, Kalb G. What do nurses and midwives value about their jobs? Results from a discrete choice experiment. J Health Serv Res Policy. 2014;20:31–8.

    Article  Google Scholar 

  54. Song K, Scott A, Sivey P, Meng Q. Improving Chinese primary care providers’ recruitment and retention: a discrete choice experiment. Health Policy Plan. 2015;30:68–77.

    Article  Google Scholar 

  55. Tinelli M, Nikoloski Z, Kumpunen S, Knai C, Brinovec RP, Warren E, et al. Decision-making criteria among European patients: exploring patient preferences for primary care services. Eur J Public Health. 2015;25:3–9.

    Article  Google Scholar 

  56. Hole AR. Small-sample properties of tests for heteroscedasticity in the conditional logit model. Health, Econometrics and Data Group, University of York. HEDG Working Paper 06/04. May 2006. Available at: https://www.york.ac.uk/media/economics/documents/herc/wp/06_04.pdf,

  57. Fiebig D, Keane M, Louviere J, Wasi N. The generalized multinomial logit model: accounting for scale and coefficient heterogeneity. Mark Sci. 2010;29:393–421.

    Article  Google Scholar 

  58. Hess S, Rose JM. Can scale and coefficient heterogeneity be separated in random coefficients models? Transportation. 2012;39:1225–39.

    Article  Google Scholar 

  59. Magidson J, Vermunt J. Removing the scale factor confound in multinomial logit choice models to obtain better estimates of preference. Sawtooth Software; 2007. p. 1–18.

  60. Swait J, Louviere J. The role of the scale parameter in the estimation and comparison of multinomial logit choice models. J Mark Res. 1993;30:305–14.

    Article  Google Scholar 

  61. Chow GC. Tests of equality between sets of coefficients in two linear regressions. Econometrica. 1960;28:591–605.

    Article  Google Scholar 

  62. Louviere J, Hensher D, Swait J. Stated choice methods: analysis and application. Cambridge: Cambridge University Press; 2000.

  63. DeShazo JR, Fermo G. Designing choice sets for stated preference methods: the effects of complexity on choice consistency. J Environ Econ Manag. 2002;44:123–43.

    Article  Google Scholar 

  64. Hole AR. Stata modules. 2017. Available at: https://www.sheffield.ac.uk/economics/people/hole/stata.

  65. Van Helvoort-Postulart D, Dellaert BGC, Van Der Weijden T, Von Meyenfeldt MF, Dirksen CD. Discrete choice experiments for complex health-care decisions: does hierarchical information integration offer a solution? Health Econ. 2008;18:903–20.

    Article  Google Scholar 

  66. Blaauw D, Erasmus E, Pagaiya N, Tangcharoensathein V, Mullei K, Mudhune S, et al. Policy interventions that attract nurses to rural areas: a multicountry discrete choice experiment. Bull World Health Organ. 2010;88:350–6.

    Article  CAS  Google Scholar 

  67. Zhou M, Thayer WM, Bridges JFP. Using latent class analysis to model preference heterogeneity in health: a systematic review. Pharmacoeconomics. 2018;36(2):175–87.

    Article  Google Scholar 

  68. Rigby D, Burton M, Pluske J. Preference stability and choice consistency in discrete choice experiments. Environ Resour Econ. 2016;65:441–61.

    Article  Google Scholar 

  69. Burton M, Davis KJ, Kragt ME. Interpretation issues in heteroscedastic conditional logit models. School of Agricultural and Resource Economics, The University of Western Australia. Working Paper 1603. 15 May 2016. Crawley.

  70. Louviere JJ, Meyer RJ, Bunch DS, Carson R, Hanemann WM, Hensher D, et al. Combining sources of preference data for modeling complex decision processes. Mark Lett. 1999;103:205–17.

    Article  Google Scholar 

  71. Louviere J, Eagle T. Confound it! That pesky little scale constant messes up our convenient assumptions. Sequem, WA: Sawtooth Software; 2006. Available at: https://www.statisticalinnovations.com/wp-content/uploads/Louviere.Eagle_.2006.pdf.

Download references

Acknowledgements

The researchers would like to thank Dr. Ewan Gray (The University of Edinburgh) and Logan Trenaman (The University of British Columbia) who screened abstracts as part of the initial creation of a database of discrete choice experiments in healthcare.

Funding

Caroline M. Vass and Katherine Payne were supported in the preparation and submission of this article by Mind the Risk, from The Swedish Foundation for Humanities and Social Sciences. The views and opinions expressed are those of the authors, and not necessarily those of other Mind the Risk members or The Swedish Foundation for Humanities and Social Sciences.

Author information

Authors and Affiliations

  1. Manchester Centre for Health Economics, Division of Population Health, Health Services Research and Primary Care, The University of Manchester, Jean McFarlane Building, Oxford Road, Manchester, M13 9PL, UK

    Stuart J. Wright, Caroline M. Vass, Gene Sim & Katherine Payne

  2. UWA School of Agriculture and Environment, University of Western Australia, Perth, WA, Australia

    Michael Burton

  3. School of Economics, University of New South Wales Business School Building, University of New South Wales, Sydney, NSW, Australia

    Denzil G. Fiebig

Authors
  1. Stuart J. Wright
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Caroline M. Vass
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gene Sim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Burton
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Denzil G. Fiebig
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Katherine Payne
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SJW conceptualised the project, screened abstracts in the systematic review, conducted data extraction, contributed to writing the paper and approved the final manuscript. CMV conceptualised the project, updated the existing systematic review, screened abstracts in the systematic review, contributed to writing the paper and approved the final manuscript. GS conceptualised the project, screened abstracts in the systematic review, contributed to writing the paper and approved the final manuscript. MB conceptualised the project, contributed to writing the paper and approved the final manuscript. DGF conceptualised the project, contributed to writing the paper and approved the final manuscript. KP conceptualised the project, updated the existing systematic review, contributed to writing the paper and approved the final manuscript.

Corresponding author

Correspondence to Katherine Payne.

Ethics declarations

Conflict of interest

Stuart Wright, Caroline Vass, Gene Sim, Michael Burton, Denzil Fiebig and Katherine Payne declare that they have no conflicts of interest.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 668 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wright, S.J., Vass, C.M., Sim, G. et al. Accounting for Scale Heterogeneity in Healthcare-Related Discrete Choice Experiments when Comparing Stated Preferences: A Systematic Review. Patient 11, 475–488 (2018). https://doi.org/10.1007/s40271-018-0304-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40271-018-0304-x

Search

Navigation