Comparing Designs for Choice Experiments: A Case Study

  • Leonie BurgessEmail author
  • Deborah J. Street
  • Nada Wasi


This paper describes an empirical comparison of the performance of four designs for a discrete choice experiment. These designs were chosen to represent the range of construction techniques that are currently popular for choice experiments when no prior knowledge of the parameters is available. Each design had 320 respondents who each completed 16 choice sets. The results suggest that for the multinomial logit model (MNL) the design that is used at this stage is fairly unimportant. As the sample size gets smaller, however, differences between the designs become apparent. We also analysed the results using four different models which accommodate preference heterogeneity. We find that any of these models are able to predict choices more accurately for both in-sample and out-of-sample than the MNL model for the designs used here, and that the differences across designs are larger for models with more parameters, although preliminary results suggest the gain appears to depend on the underlying preference structure.


MNL model GMNL model Mixed logit Stated preference experiments 

AMS Subject Classification

62K05 62P20 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Araña, J.E., León, C.J., Hanemann, M.W., 2008. Emotions and decision rules in discrete choice experiments for valuing health care programmes for the elderly. Journal of Health Economics, 27, 753–769.CrossRefGoogle Scholar
  2. Arora, N., Huber, J., 2001. Improving parameter estimates and model prediction by aggregate customization in choice experiments. Journal of Consumer Research, 28, 273–283.CrossRefGoogle Scholar
  3. Bech, M., 2003. County council politicians’ choice of hospital payment scheme: A discrete choice study. Applied Health Economics and Health Policy, 2, 225–232.Google Scholar
  4. Blamey, R., Bennett, J., Louviere, J., Morrison, M., Rolfe, J., 2000. A test of policy labels in environmental choice modelling studies. Ecological Economics, 32, 269–286.CrossRefGoogle Scholar
  5. Bunch, D.S., Louviere, J.J., Anderson, D., 1996. A comparison of experimental design strategies for multinomial logit models: The case of generic attributes. Technical report, University of California, Davis. Available at Scholar
  6. Carlsson, F., Martinsson, P., 2003. Design techniques for stated preference methods in health economics. Health Economics, 12, 281–294.CrossRefGoogle Scholar
  7. Ferrini, S., Scarpa, R., 2007. Designs with a priori information for nonmarket valuation with choice experiments: A Monte Carlo study. Journal of Environmental Economics and Management, 53, 342–363.CrossRefGoogle Scholar
  8. Fiebig, D.G., Keane, M.P., Louviere, J., Wasi, N., 2010. The generalized multinomial logit model: Accounting for scale and coefficient heterogeneity. Marketing Science, 29, 393–421.CrossRefGoogle Scholar
  9. Gerard, K., Salisbury, C., Street, D.J., Pope, C., Baxter, H., 2008. Is fast access to general practice all that should matter? a discrete choice experiment of patients’ preferences. Journal of Health Services Research & Policy, 13, 3–10.CrossRefGoogle Scholar
  10. Greene, W.H., Hensher, D.A., 2010. Does scale heterogeneity across individuals matter? an empirical assessment of alternative logit models. Transportation, 37(3), 413–428.CrossRefGoogle Scholar
  11. Hensher, D.A., 2006. Revealing differences in willingness to pay due to the dimensionality of stated choice designs: An initial assessment. Environmental and Resource Economics, 34, 7–44.CrossRefGoogle Scholar
  12. Kessels, R., Goos, P., Vandebroek, M., 2006. A comparison of criteria to design efficient choice experiments. Journal of Marketing Research, 43, 409–419.CrossRefGoogle Scholar
  13. Kessels, R., Jones, B., Goos, P., Vandebroek, M., 2008. Recommendations on the use of bayesian optimal designs for choice experiments. Quality and Reliability Engineering International, 24, 737–744.CrossRefGoogle Scholar
  14. Kuhfeld, W.F., 2009. Marketing research methods in SAS. Technical report, SAS Institute. Available at Google Scholar
  15. Louviere, J.J., Islam, T., Wasi, N., Street, D., Burgess, L., 2008. Designing discrete choice experiments: Do optimal designs come at a price? Journal of Consumer Research, 35, 360–375.CrossRefGoogle Scholar
  16. McFadden, D., Train, K., 2000. Mixed MNL models of discrete response. Journal of Applied Econometrics, 15, 447–470.CrossRefGoogle Scholar
  17. McKenzie, L., Cairns, J., Osman, L., 2001. Symptom-based outcome measures for asthma: The use of discrete choice methods to assess patient preferences. Health Policy, 57, 193–204.CrossRefGoogle Scholar
  18. Sandor, Z., Wedel, M., 2001. Designing conjoint choice experiments using managers’ prior beliefs. Journal of Marketing Research, 38, 430–444.CrossRefGoogle Scholar
  19. Sandor, Z., Wedel, M., 2002. Profile construction in experimental choice designs for mixed logit models. Marketing Science, 21, 455–475.CrossRefGoogle Scholar
  20. Street, D.J., Burgess, L., 2007. The Construction of Optimal Stated Choice Experiments: Theory and Methods. Wiley, Hoboken, NJ.CrossRefGoogle Scholar
  21. Train, K.E., 2003. Discrete Choice Methods with Simulation. Cambridge University Press, Cambridge.CrossRefGoogle Scholar
  22. Viney, R., Savage, E., Louviere, J., 2005. Empirical investigation of experimental design properties of discrete choice experiments in health care. Health Economics, 14, 349–362.CrossRefGoogle Scholar
  23. Yu, J., Goos, P., Vandebroek, M., 2009. Efficient conjoint choice designs in the presence of respondent heterogeneity. Marketing Science, 28, 122–135.CrossRefGoogle Scholar

Copyright information

© Grace Scientific Publishing 2011

Authors and Affiliations

  1. 1.Department of Mathematical SciencesUniversity of TechnologySydneyAustralia
  2. 2.School of Finance and EconomicsUniversity of TechnologySydneyAustralia

Personalised recommendations