We provide a user guide on the analysis of data (including best–worst and best–best data) generated from discrete-choice experiments (DCEs), comprising a theoretical review of the main choice models followed by practical advice on estimation and post-estimation. We also provide a review of standard software. In providing this guide, we endeavour to not only provide guidance on choice modelling but to do so in a way that provides a ‘way in’ for researchers to the practicalities of data analysis. We argue that choice of modelling approach depends on the research questions, study design and constraints in terms of quality/quantity of data and that decisions made in relation to analysis of choice data are often interdependent rather than sequential. Given the core theory and estimation of choice models is common across settings, we expect the theoretical and practical content of this paper to be useful to researchers not only within but also beyond health economics.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
This can include presenting a single profile and asking respondents to accept or reject it.
In our case study, the status quo is no treatment; however, more generally, status quo and no treatment need not coincide.
Our overview is not exhaustive, as other software packages capable of estimating some of the discrete-choice models in our review are available. However, the three packages we have reviewed are among the most commonly used for estimating these models.
This implies we will not cover software such as Gauss, Matlab and R, despite there being excellent routines written in these packages for estimating, for example, mixed logit models. A prominent example is Kenneth Train’s codes for mixed logit estimation (http://eml.berkeley.edu/~train/software.html), which served as inspiration for many of the routines later introduced in other statistical packages.
Two versions of Biogeme are available: BisonBiogeme and PythonBiogeme. We focus on BisonBiogeme, which is designed to estimate a range of commonly used discrete-choice models.
Nlogit also optionally allows the data to be organized in wide form, although the manual suggests that long form is typically more convenient.
Interested readers are referred to chapters 8–10 in Train  for more information about the issues covered in this section.
Both Nlogit and Stata will use a default set of starting values unless explicitly specified by the user, whereas Biogeme requires the user to specify the starting values.
The default number of draws is 100 in Nlogit, 50 in Stata and 150 in Biogeme.
In models with several random coefficients, alternative approaches such as shuffled or scrambled Halton draws  or Sobol draws [51, 52] are sometimes used to minimize the correlation between the draws, which can be substantial for standard Halton draws in higher dimensions. See chapter 9 in Train  for a discussion.
Nlogit and Stata’s default starting values are the MNL parameters for the means of the random coefficients and 0 (Nlogit)/0.1 (Stata) for the standard deviations.
Differences can still arise, for example because the optimization algorithms differ in the three packages, subtle differences in terms of how the Halton draws are generated and different starting values (in this case Stata/Biogeme vs Nlogit).
Applying this procedure modifies the data from the standard set-up in Supplementary Appendix 1 to the exploded set-up in Supplementary Appendix 3.
Log-normal parameter distributions are supported by all of the packages. The negative of the log-normal can easily be implemented by multiplying the price attribute by −1 before entering the model. This is equivalent to specifying the negative of the price coefficient to be log-normally distributed. The sign of the coefficient can easily be reversed post-estimation.
One exception is when both the attribute coefficient and the negative of the price coefficient are log-normally distributed, in which case the distribution of mWTP is also log-normal.
Louviere JJ, Hensher DA, Swait JD. Stated choice methods: analysis and applications. Cambridge: Cambridge University Press; 2000.
de Bekker-Grob EW, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2012;21(2):145–72.
Clark MD, Determann D, Petrou S, Moro D, de Bekker-Grob EW. Discrete choice experiments in health economics: a review of the literature. Pharmacoeconomics. 2014;32(9):883–902.
Viney R, Lancsar E, Louviere J. Discrete choice experiments to measure consumer preferences for health and healthcare. Expert Rev Pharmacoeconomics Outcomes Res. 2002;2(4):319–26.
Lancsar E, Louviere J. Conducting discrete choice experiments to inform healthcare decision making. Pharmacoeconomics. 2008;26(8):661–77.
Bridges J, Hauber A, Marshall D, Lloyd A, Prosser L, Regier D, et al. A checklist for conjoint analysis applications in health: report of the ISPOR Conjoint Analysis Good Research Practices Taskforce. Value Health. 2011;14(4):403–13.
Coast J, Horrocks S. Developing attributes and levels for discrete choice experiments using qualitative methods. J Health Serv Res Policy. 2007;12(1):25–30.
Johnson FR, Lancsar E, Marshall D, Kilambi V, Mühlbacher A, Regier DA, et al. Constructing experimental designs for discrete-choice experiments: report of the ISPOR conjoint analysis experimental design good research practices task force. Value Health. 2013;16(1):3–13.
Lancsar E, Swait J. Reconceptualising the external validity of discrete choice experiments. Pharmacoeconomics. 2014;32(10):951–65.
Hauber B, Gonzalez J, Groothuis-Oudshoorn C, Prior T, Marshall D, Cunningham C, et al. Statistical methods for the analysis of discrete choie experiments: a report of the ISPOR conjinta analysis good research practice task force. Value Health. 2016;19:300–15.
Ghijben P, Lancsar E, Zavarsek S. Preferences for oral anticoagulants in atrial fibrillation: a best–best discrete choice experiment. Pharmacoeconomics. 2014;32(11):1115–27.
Dillman DA. Mail and internet surveys: the tailored design method. New York: Wiley; 2000.
Tourangeau R, Rips LJ, Rasinski K. The psychology of survey response. Cambridge: Cambridge University Press; 2000.
Lancsar E, Louviere J, Donaldson C, Currie G, Burgess L. Best worst discrete choice experiments in health: methods and an application. Soc Sci Med. 2013;76:74–82.
Louviere JJ, Flynn TN, Marley A. Best–worst scaling: theory, methods and applications. Cambridge: Cambridge University Press; 2015.
Bartels R, Fiebig DG, van Soest A. Consumers and experts: an econometric analysis of the demand for water heaters. Empir Econ. 2006;31(2):369–91.
King MT, Hall J, Lancsar E, Fiebig D, Hossain I, Louviere J, et al. Patient preferences for managing asthma: results from a discrete choice experiment. Health Econ. 2007;16(7):703–17.
Jung K, Feldman R, Scanlon D. Where would you go for your next hospitalization? J Health Econ. 2011;30(4):832–41.
Harris KM, Keane MP. A model of health plan choice: inferring preferences and perceptions from a combination of revealed preference and attitudinal data. J Econ. 1998;89(1):131–57.
Swait J, Erdem T. Brand effects on choice and choice set formation under uncertainty. Market Sci. 2007;26(5):679–97.
Swait J, et al. Context dependence and aggregation in disaggregate choice analysis. Market Lett. 2002;13:195–205.
McFadden D. Conditional logit analysis of qualitative choice behavior. Berkeley, CA: University of California; 1974.
McFadden D. Disaggregate behavioral travel demand’s RUM side. A 30 year retrospective. Travel Behav Res. 2000:17–63.
Maddala G. Limited dependent and qualitative variables in econometrics. Cambridge: Cambridge University Press; 1983.
Fiebig DG, Keane MP, Louviere J, Wasi N. The generalized multinomial logit model: accounting for scale and coefficient heterogeneity. Market Sci. 2010;29(3):393–421.
Hess S, Rose JM. Can scale and coefficient heterogeneity be separated in random coefficients models? Transportation. 2012;36(6):1225–39.
Revelt D, Train K. Mixed logit with repeated choices: households’ choices of appliance efficiency level. Rev Econ Stat. 1998;80(4):647–57.
Brownstone D, Train K. Forecasting new product penetration with flexible substitution patterns. J Econ. 1998;89(1):109–29.
Hall J, Fiebig DG, King MT, Hossain I, Louviere JJ. What influences participation in genetic carrier testing? Results from a discrete choice experiment. J Health Econ. 2006;25(3):520–37.
Hole AR. Modelling heterogeneity in patients’ preferences for the attributes of a general practitioner appointment. J Health Econ. 2008;27(4):1078–94.
Louviere JJ, Street D, Burgess L, Wasi N, Islam T, Marley AA. Modeling the choices of individual decision-makers by combining efficient choice experiment designs with extra preference information. J Choice Model. 2008;1(1):128–64.
Keane M, Wasi N. Comparing alternative models of heterogeneity in consumer choice behavior. J Appl Econ. 2013;28(6):1018–45.
Lancsar E, Louviere J, Flynn T. Several methods to investigate relative attribute impact in stated preference experiments. Soc Sci Med. 2007;64(8):1738–53.
Fiebig DG, Knox S, Viney R, Haas M, Street DJ. Preferences for new and existing contraceptive products. Health Econ. 2011;20(S1):35–52.
Lancsar E, Louviere J. Estimating individual level discrete choice models and welfare measures using best–worst choice experiments and sequential best–worst MNL. Sydney: University of Technology, Centre for the Study of Choice (Censoc); 2008. p. 08-004.
Scarpa R, Notaro S, Louviere J, Raffaelli R. Exploring scale effects of best/worst rank ordered choice data to estimate benefits of tourism in alpine grazing commons. Am J Agric Econ. 2011;93(3):813–28.
Punj GN, Staelin R. The choice process for graduate business schools. J Market Res. 1978;15(4):588–98.
Chapman RG, Staelin R. Exploiting rank ordered choice set data within the stochastic utility model. J Market Res. 1982;19(3):288–301.
Beggs S, Cardell S, Hausman J. Assessing the potential demand for electric cars. J Econ. 1981;17(1):1–19.
Gu Y, Hole AR, Knox S. Fitting the generalized multinomial logit model in Stata. Stata J. 2013;13(2):382–97.
Hole AR. Fitting mixed logit models by using maxium simulated likelihood. Stata J. 2007;7:388–401.
Pacifico D, Yoo HI. lclogit: a stata module for estimating latent class conditional logit models via the Expectation-Maximization algorithm. Stata J. 2013;13(3):625–39.
Baker MJ. Adaptive Markov chain Monte Carlo sampling and estimation in Mata. Stata J. 2014;14(3):623–61.
Suits DB. Dummy variables: mechanics v. interpretation. Rev Econ Stat. 1984;66:177–80.
Bech M, Gyrd-Hansen D. Effects coding in discrete choice experiments. Health Econ. 2005;14(10):1079–83.
Hole AR, Yoo I. The use of heuristic optimization algorithms to facilitate maximum simulated likelihood estimation of random parameter logit models. J R Stat Soc C. 2017;. doi:10.1111/rssc.12209.
Czajkowski M, Budziński W. An insight into the numerical simulation bias—a comparison of efficiency and performance of different types of quasi Monte Carlo simulation methods under a wide range of experimental conditions. In: Environmental Choice Modelling Conference; Copenhagen; 2015.
Bhat CR. Quasi-random maximum simulated likelihood estimation of the mixed multinomial logit model. Transp Res Part B: Methodol. 2001;35(7):677–93.
Train KE. Discrete choice methods with simulation. Cambridge: Cambridge University Press; 2009.
Hess S, Train KE, Polak JW. On the use of a Modified Latin Hypercube Sampling (MLHS) method in the estimation of a Mixed Logit Model for vehicle choice. Transp Res Part B: Methodol. 2006;40(2):147–63.
Garrido RA. Estimation performance of low discrepancy sequences in stated preferences. In: 10th International Conference on Travel Behaviour Research; Lucerne; 2003.
Munger D, L’Ecuyer P, Bastin F, Cirillo C, Tuffin B. Estimation of the mixed logit likelihood function by randomized quasi-Monte Carlo. Transp Res Part B: Methodol. 2012;46(2):305–20.
Hole AR. A comparison of approaches to estimating confidence intervals for willingness to pay measures. Health Econ. 2007;16(8):827–40.
Train K, Weeks M. Discrete choice models in preference space and willingness-to-pay space: Applications of simulation methods in environmental and resource economics. Berlin: Springer; 2005. p. 1–16.
Ben-Akiva M, McFadden D, Train K. Foundations of stated preference elicitation consumer behavior and choice-based conjoint analysis. In: 2015, Society for economic measurement annual conference, Paris, 24 July 2015.
Greene WH, Hensher DA. A latent class model for discrete choice analysis: contrasts with mixed logit. Transp Res Part B: Methodol. 2003;37(8):681–98.
Johar M, Fiebig DG, Haas M, Viney R. Using repeated choice experiments to evaluate the impact of policy changes on cervical screening. Appl Econ. 2013;45(14):1845–55.
Lancsar E, Wildman J, Donaldson C, Ryan M, Baker R. Deriving distributional weights for QALYs through discrete choice experiments. J Health Econ. 2011;30:466–78.
Lancsar E, Savage E. Deriving welfare measures from discrete choice experiments: inconsistency between current methods and random utility and welfare theory. Health Econ. 2004;13(9):901–7.
Elshiewy O, Zenetti G, Boztug Y. Differences between classical and Bayesian estimates for mixed logit models: a replication study. J Appl Econ. 2017;32(2):470–76.
Ryan M, Bate A. Testing the assumptions of rationality, continuity and symmetry when apply- ing discrete choice experiments in health care. Appl Econ Lett. 2001;8:59–63.
Ryan M, San MF. Revisiting the axiom of completeness in health care. Health Econ. 2003;12:295–307.
Lancsar E, Louviere J. Deleting, “irrational” responses from discrete choice experiments: a case of investigating or imposing preferences? Health Econ. 2006;15(8):797–811.
Fiebig DG, Viney R, Knox S, Haas M, Street DJ, Hole AR, et al. Consideration sets and their role in modelling doctor recommendations about contraceptives. Health Econ. 2017;26(1):54–73.
Hensher DA, Greene WH. Non-attendance and dual processing of common-metric attributes in choice analysis: a latent class specification. Empir Econ. 2010;39(2):413–26.
Lagarde M. Investigating attribute non-attendance and its consequences in choice experiments with latent class models. Health Econ. 2013;22(5):554–67.
Hole AR, Kolstad JR, Gyrd-Hansen D. Inferred vs. Stated attribute non-attendance in choice experiments: a study of doctors’ prescription behaviour. J Econ Behav Organ. 2013;96:21–31.
Flynn TN, Bilger M, Finkelstein EA. Are efficient designs used in discrete choice experiments too difficult for some respondents? A case study eliciting preferences for end-of-life care. Phamacoeconomics. 2016;34(3):273–84.
Mark TL, Swait J. Using stated preference and revealed preference modeling to evaluate prescribing decisions. Health Econ. 2004;13(6):563–73.
Ben-Akiva M, Bradley M, Morikawa TJ, Benjamin T, Novak H, Oppewal H, et al. Combining revealed and stated preferences data. Market Lett. 1994;5(4):335–49.
Lancsar E, Burge P. Choice modelling research in health economics. In: Hess S, Daly A, editors. Handbook of choice modelling. Cheltenham, UK: Edward Elgar Press; 2014. p. 675–87.
Hess S, Daly A. Handbook of choice modelling. Cheltenham: Edward Elgar Publishing; 2014.
The authors thank Peter Ghijben, Emily Lancsar and Silva Zavarsek for making available the data used in Ghijben et al. . All authors jointly conceived the intent of the paper, drafted the manuscript and approved the final version.
No funding was received for the preparation of this paper.
Conflicts of interest
Emily Lancsar is funded by an ARC Fellowship (DE1411260). Emily Lancsar, Denzil Fiebig and Arne Risa Hole have no conflicts of interest.
About this article
Cite this article
Lancsar, E., Fiebig, D.G. & Hole, A.R. Discrete Choice Experiments: A Guide to Model Specification, Estimation and Software. PharmacoEconomics 35, 697–716 (2017). https://doi.org/10.1007/s40273-017-0506-4