Journal of Statistical Theory and Practice

, Volume 11, Issue 2, pp 296–321 | Cite as

Possible design-induced artifacts associated with designs for discrete choice experiments

  • Tiago Ribeiro
  • Richard Carson
  • Jordan J. LouviereEmail author
  • John M. Rose


Discrete choice experiments (DCEs) are widely used in many areas of applied social science research. The results of DCEs depend on the particular experimental design for the identification of the key parameters of interest and the statistical efficiency with which those parameters are estimated. Work on experimental designs for DCEs has almost always assumed that the particular design one uses does not influence the nature of the responses to the choice tasks other than via the precision with which parameters are estimated. We examine this assumption by testing whether particular experimental designs influence the probability that a separating hyperplane exists that perfectly predicts the observed choices at the individual level in four DCE data sets. Our empirical results suggest that the particular statistical design used can influence the nature of the choice responses obtained.


Behavioral response choice model experimental design 

AMS Subject Classification

Primary 62K05 secondary 62K99 62P25 91B42 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Adamowicz, V., D. Dupont, and A. Krupnick. 2006. Willingness to pay to reduce community health risks from municipal drinking water, a stated preference study. Paper presented at the Third World Congress of Environmental and Resource Economics, Kyoto, Japan, August.Google Scholar
  2. Anderson, D. A., and J. B. Wiley. 1992. Efficient choice set designs for estimating availability cross-effects models. Marketing Letters 3 (4):357–70.CrossRefGoogle Scholar
  3. Arentze, T., A. Borgers, H. Timmermans, and R. Del Mistro. 2003. Transport stated choice responses: Effects of task complexity, presentation format and literacy. Transportation Research Part E 39 (3):229–44.CrossRefGoogle Scholar
  4. Atkinson, A. C., and L. M. Haines. 1996. Designs for nonlinear and generalized linear models. In Handbook of statistics, ed. S. Ghosh and C. R. Rao, Vol. 13, 437–75. Amsterdam, The Netherlands: Elsevier.Google Scholar
  5. Bateman, I., R. T. Carson, B. Day, M. Hanemann, N. Hanley, T. Hett, M. Jones-Lee, G. Loomes, S. Mourato, E. Ozdemiroglu, D. W. Pearce, R. Sugden, and J. Swanson. 2003. Economic valuation with stated preference techniques: A manual. Northampton, MA: Edward Elgar.Google Scholar
  6. Beck, M., T. Kjaer, and J. Lauridsen. 2011. Does the number of choice sets matter?: Results from a Web survey applying a discrete choice experiment. Health Economics 20 (3):273–86.CrossRefGoogle Scholar
  7. Bliemer, M. C. J., and J. M. Rose. 2010. Serial choice conjoint analysis for estimating discrete choice models. In Choice modelling: The state of the art and the state of practice, eds. S. Hess and A. Daly, 139–61. Northampton, MA: Edward Elgar.Google Scholar
  8. Bliemer, M. C. J., and J. M. Rose. 2011. Experimental design influences on stated choice outputs: An empirical study in air travel choice. Transportation Research Part A 45 (1):63–79.Google Scholar
  9. Bliemer, M. C. J., J. M. Rose, and S. Hess. 2008. Approximation of Bayesian efficiency in experimental choice designs. Journal of Choice Modelling 1 (1):98–127.CrossRefGoogle Scholar
  10. Box, G. E., J. S. Hunter, and W. G. Hunter. 2005. Statistics for experimenters: Design, innovation, and discovery, 2nd ed. New York, NY: Wiley-Interscience.zbMATHGoogle Scholar
  11. Boxall, P., W. L. Adamowicz, and A. Moon. 2009. Complexity in choice experiments: Choice of the status quo alternative and implications for welfare measurement. Australian Journal of Agricultural and Resource Economics 53 (4):503–19.CrossRefGoogle Scholar
  12. Boyle, K. J., and S. Özdemir. 2009. Convergent validity of attribute-based, choice questions in stated-preference studies. Environmental Resource Economics 42 (2):247–64.CrossRefGoogle Scholar
  13. Brazell, J. D., C. G. Diener, E. Karniouchina, W. L. Moore, V. Severin, and P. F. Uldry. 2006. The no-choice option and dual response choice designs. Marketing Letters 17 (4):255–68.CrossRefGoogle Scholar
  14. Brazell, J. D., and J. J. Louviere, 1998. Length effects in conjoint choice experiments and surveys: An explanation based on cumulative cognitive burden. Working paper, Department of Marketing, University of Sydney, Sydney, Australia, July.Google Scholar
  15. Carlsson, F., and P. Martinsson. 2003. Design techniques for stated preference methods in health economics. Health Economics 12 (4):281–94.CrossRefGoogle Scholar
  16. Carson, R. T., and T. Groves. 2007. Incentive and informational properties of preference questions. Environmental and Resource Economics 37 (1):181–210.CrossRefGoogle Scholar
  17. Carson, R. T., and T. Groves. 2011. Incentive and information properties of preference questions: Commentary and extensions. In The international handbook on non-market environmental valuation, ed. J. Bennett, 300–21. Northampton, MA: Edward Elgar.Google Scholar
  18. Caussade, S., J. De D’Ortúzar, L. Rizzi, and D. A. Hensher. 2005. Assessing the influence of design dimensions on stated choice experiment estimates. Transportation Research Part B 39 (7):621–40.CrossRefGoogle Scholar
  19. Chaloner, K., and I. Verdinelli. 1995. Bayesian experimental design: A review. Statistical Science 10 (3):273–304.CrossRefMathSciNetzbMATHGoogle Scholar
  20. Day, B., I. J. Bateman, R. T. Carson, D. Dupont, J. J. Louviere, S. Morimoto, R. Scarpa, and P. Wang. 2012. Ordering effects and choice set awareness in repeat-response stated preference studies. Journal of Environmental Economics and Management 63 (1):73–91.CrossRefGoogle Scholar
  21. Dellaert, B. G. C., J. D. Brazell, and J. J. Louviere. 1999. The effect of attribute variation on consumer choice consistency. Marketing Letters 10 (2):139–47.CrossRefGoogle Scholar
  22. DeShazo, J. R, and G Fermo. 2002. Designing choice sets for stated preference methods: The effects of complexity on choice consistency. Journal of Environmental Economics and Management 44 (1):123–43.CrossRefzbMATHGoogle Scholar
  23. Dhar, R. 1997. Consumer preference for a no-choice option. Journal of Consumer Research 24 (2):215–31.CrossRefMathSciNetGoogle Scholar
  24. Dhar, R., and I. Simonson. 2003. The effect of forced choice on choice. Journal of Marketing Research 40 (2):146–60.CrossRefGoogle Scholar
  25. Farrar, S., and M. Ryan. 1999. Response-ordering effects: A methodological issue in conjoint analysis. Health Economic Letters 8 (1):75–79.CrossRefGoogle Scholar
  26. Finney, D. J. 1978. Statistical method in biological assay. London, UK: Charles Griffin.zbMATHGoogle Scholar
  27. Goegebeur, Y., P. Goos, and M. L Vandebroek, 2007. A hierarchical Bayesian approach to robust parameter design. Working paper, Department of Decision Sciences and Information Management, Katholieke Universiteit Leuven, Leuven, Belgium.Google Scholar
  28. Green, P. E., and V. Srinivasan. 1990. Conjoint analysis in marketing: New developments with implications for research and practice. Journal of Marketing 54 (1):3–19.CrossRefGoogle Scholar
  29. Hensher, D. A. 2004. Accounting for stated choice design dimensionality in willingness to pay for travel time savings. Journal of Transport Economics and Policy 38 (3):425–46.Google Scholar
  30. Hensher, D. A. 2006. How do respondents process stated choice experiments? Attribute consideration under varying information load. Journal of Applied Econometrics 21 (6):861–78.CrossRefMathSciNetGoogle Scholar
  31. Hensher, D. A., J. M. Rose, and W. H. Greene. 2005. Applied choice analysis: A primer. New York, NY: Cambridge University Press.CrossRefzbMATHGoogle Scholar
  32. Hensher, D. A., P. R. Stopher, and J. J. Louviere. 2001. An exploratory analysis of the effects of numbers of choice sets in designed choice experiments: An airline choice application. Journal of Air Transport Management 7 (6):373–79.CrossRefGoogle Scholar
  33. Huber, J., and K. Zwerina. 1996. The importance of utility balance in efficient choice designs. Journal of Marketing Research 33 (3):307–17.CrossRefGoogle Scholar
  34. Kanninen, B. J. 2002. Optimal design for multinomial choice experiments. Journal of Marketing Research 39 (2):214–17.CrossRefGoogle Scholar
  35. Kessels, R., B. Jones, P. Goos, and M. Vandebroek. 2009. An efficient algorithm for constructing Bayesian optimal choice designs. Journal of Business and Economic Statistics 27 (2):279–91.CrossRefMathSciNetGoogle Scholar
  36. Kjaer, T., M. Bech, D. Gyrd-Hansen, and K. Hart-Hansen. 2006. Ordering effect and price sensitivity in discrete choice experiments: Need we worry? Health Economics 15 (11):1217–28.CrossRefGoogle Scholar
  37. Krosnick, J. A. 1991. Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology 5 (3):213–36.CrossRefGoogle Scholar
  38. Kuhfeld, W. F., R. D. Tobias, and M. Garratt. 1994. Efficient experimental design with marketing research applications. Journal of Marketing Research 31 (4):545–57.CrossRefGoogle Scholar
  39. Lazari, A. G., and D. A. Anderson. 1994. Designs of discrete choice set experiments for estimating both attribute and availability cross effects. Journal of Marketing Research 31 (3):375–83.CrossRefGoogle Scholar
  40. Louviere, J. J., 2011. Modelling single individuals: The journey from psych lab to app store. Keynote Address, International Choice Modelling Conference, Leeds, UK.Google Scholar
  41. Louviere, J. J. 2013. Modelling single individuals: The journey from psych lab to app store. In Choice modelling: The state of the art and the state of practice, ed. S. Hess and A. Daly, 1–47. Northhampton, MA: Edward Elgar.Google Scholar
  42. Louviere, J. J., D. A. Hensher, and J. D. Swait. 2000. Stated choice methods: Analysis and applications. New York, NY: Cambridge University Press.CrossRefzbMATHGoogle Scholar
  43. Louviere, J. J., T. Islam, N. Wasi, D. Street, and L. Burgess. 2008. Designing discrete choice experiments: Do optimal designs come at a price? Journal of Consumer Research 35 (2):360–75.CrossRefGoogle Scholar
  44. Louviere, J. J., D. Pihlens, and R. T. Carson. 2011. Design of discrete choice experiments: A discussion of issues that matter in future applied research. Journal of Choice Modelling 4 (1):1–8.CrossRefGoogle Scholar
  45. Mangasarian, O. L. 1965. Linear and nonlinear separation of patterns by linear programming. Operations Research 13 (3):444–52.CrossRefMathSciNetzbMATHGoogle Scholar
  46. Manski, C. F. 1977. The structure of random utility models. Theory and Decision 8 (3):229–54.CrossRefMathSciNetzbMATHGoogle Scholar
  47. Mazzotta, M., and J. Opaluch. 1995. Decision making when choices are complex: A test of Heiner’s hypothesis. Land Economics 71 (4):500–15.CrossRefGoogle Scholar
  48. McFadden, D. 1974. Conditional logit analysis of qualitative choice behavior. In Frontiers in economics, ed. P. Zarembka, 105–42. New York, NY: Academic Press.Google Scholar
  49. Mellers, B. A., and K. Biagini. 1994. Similarity and choice. Psychological Review 101 (3):505–18.CrossRefGoogle Scholar
  50. Meyer, R. J., and T. C. Eagle. 1982. Context-induced parameter instability in a disaggregate-stochastic model of store choice. Journal of Marketing Research 19 (1):62–71.CrossRefGoogle Scholar
  51. Ohler, T., A. Li, J. J. Louviere, and J. Swait. 2000. Attribute range effects in binary response tasks. Marketing Letters 11 (3):249–60.CrossRefGoogle Scholar
  52. Orne, M. T. 1962. On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. American Psychologist 17 (11):776–83.CrossRefGoogle Scholar
  53. Orne, M. T. 1969. Demand characteristics and the concept of quasi-controls. In Artifact in behavioral research, ed. R Rosenthal and R Rosnow, 143–79. New York, NY: Academic Press. Rolfe, J., and J. Bennett. 2009. The impact of offering two versus three alternatives in choice modelling experiments. Ecological Economics 68 (4):1140–48.Google Scholar
  54. Rose, J. M., and M. C. Bliemer. 2009. Constructing efficient stated choice experimental designs. Transport Reviews 29 (5):587–617.CrossRefGoogle Scholar
  55. Rose, J. M., D. A. Hensher, S. Caussade, J. De D’Ortuzar, and J. Rong-Chang. 2009. Identifying differences in preferences due to dimensionality in stated choice experiments: A cross cultural analysis. Journal of Transport Geography 17 (1):21–29.CrossRefGoogle Scholar
  56. Rose, J. M., and S. Hess. 2009. Dual response choices in reference alternative related stated choice experiments. Transportation Research Records 2135 (1):25–33.CrossRefGoogle Scholar
  57. Sandor, Z., and M. Wedel. 2001. Designing conjoint choice experiments using managers’ prior beliefs. Journal of Marketing Research 38 (4):430–44.CrossRefGoogle Scholar
  58. Street, D. J., D. S. Bunch, and B. J. Moore. 2001. Optimal designs for 2 k paired comparison experiments. Communications in Statistics-Theory and Methods 30 (10):2149–71.CrossRefMathSciNetzbMATHGoogle Scholar
  59. Street, D. J., and L. Burgess. 2007. The construction of optimal stated choice experiments: Theory and methods. New York, NY: John Wiley.CrossRefzbMATHGoogle Scholar
  60. Swait, J., and W. Adamowicz. 2001. Choice environment, market complexity, and consumer behavior: A theoretical and empirical approach for incorporating decision complexity into models of consumer choice. Organizational Behavior and Human Decision Processes 86 (2):141–67.CrossRefGoogle Scholar
  61. Train, K. E. 2009. Discrete choice methods with simulation, 2nd ed. New York, NY: Cambridge University Press.CrossRefzbMATHGoogle Scholar
  62. Tudela, A., and G. Rebolledo. 2006. Optimal design of stated preference experiments when using mixed logit models. Proceedings of the European Transport Conference, Leiden, The Netherlands.Google Scholar
  63. Vincent, J. R., R. T. Carson, J. R. DeShazo, K. A. Schwabe, I. Ahmad, S. K. Chong, Y. T. Chang, and M. D. Potts. 2014. Tropical countries may be willing to pay more to protect their forests. Proceedings of the National Academy of Sciences 111 (28):10113–18.CrossRefGoogle Scholar
  64. Viney, R., E. Savage, and J. J. Louviere. 2005. Empirical investigation of experimental design properties of discrete choice experiments in health care. Health Economics 14 (4):349–62.CrossRefGoogle Scholar
  65. Wittink, D. R., J. Huber, P. Zandan, and R. M. Johnson. 1992. The number of levels effect in conjoint: Where does it come from and can it be eliminated? Sawtooth Software Conference Proceedings, 355–64. Ketchum, ID: Sawtooth Software.Google Scholar
  66. Wittink, D. R., L. Krishnamurthi, and D. Reibstein. 1989. The effect of differences in the number of attribute levels in conjoint results. Marketing Letters 1 (2):113–23.CrossRefGoogle Scholar
  67. Yu, J., P. Goos, and M. Vandebroek. 2008. Model-robust design of conjoint choice experiments. Communications in Statistics—Simulation and Computation 37 (8):1603–21.CrossRefMathSciNetzbMATHGoogle Scholar
  68. Yu, J., P. Goos, and M. Vandebroek. 2010. Comparing different sampling schemes for approximating the integrals involved in the efficient design of stated choice experiments. Transportation Research Part B: Methodological 44 (10):1268–89.CrossRefGoogle Scholar
  69. Yu, J., P. Goos, and M. Vandebroek. 2011. Individually adapted sequential Bayesian conjoint-choice designs in the presence of consumer heterogeneity. International Journal of Research in Marketing 28 (4):378–88.CrossRefGoogle Scholar

Copyright information

© Grace Scientific Publishing, 20 Middlefield Ct, Greensboro, NC 27455 2017

Authors and Affiliations

  • Tiago Ribeiro
    • 1
  • Richard Carson
    • 2
  • Jordan J. Louviere
    • 3
    Email author
  • John M. Rose
    • 1
  1. 1.Institute for ChoiceUniversity of South AustraliaNorth SydneyAustralia
  2. 2.Department of EconomicsUniversity of California San DiegoLa JollaUSA
  3. 3.School of MarketingUniversity of South AustraliaAdelaideAustralia

Personalised recommendations