Are Efficient Designs Used in Discrete Choice Experiments Too Difficult for Some Respondents? A Case Study Eliciting Preferences for End-of-Life Care
- 327 Downloads
Although efficient designs have sample size advantages for discrete choice experiments (DCEs), it has been hypothesised that they may result in biased estimates owing to some respondents using simplistic heuristics.
The main objective was to provide a case study documenting that many respondents choose on the basis of a single attribute when exposed to highly efficient DCE designs but switch to a conventional multi-attribute decision rule when the design efficiency was lowered (resulting in less need to trade across all attributes). Additional objectives included comparisons of the sizes of the estimated coefficients and characterisation of heterogeneity, thus providing evidence of the magnitude of bias likely present in highly efficient designs.
Five hundred and twenty-five respondents participating in a wider end-of-life survey each answered two DCEs that varied in their design efficiency. The first was a Street and Burgess 100 % efficient Orthogonal Main Effects Plan design (27 in 8), using the top and bottom levels of all attributes. The second DCE comprised one eighth of the full Orthogonal Main Effects Plan in 32 pairs, (a 2 × 46). Linear probability models estimated every respondent’s complete utility function in DCE1. The number of respondents answering on the basis of one attribute level was noted, as was the proportion of these who then violated this rule in DCE2, the less efficient DCE. Latent class analyses were used to identify heterogeneity.
Sixty per cent of respondents answered all eight tasks comprising DCE1 using a single attribute; most used the rule “choose cheapest end-of-life care plan”. However, when answering the four less efficient tasks in DCE2, one third of these (20 % overall) then traded across attributes at least once. Among those whose decision rule could not be described qualitatively, latent class models identified two classes; compared to class one, class two was more concerned with quality rather than cost of care and wished to die in an institution rather than at home. Higher efficiency was also associated with smaller regression coefficients, suggesting either weaker preferences or lower choice consistency (larger errors).
This is the first within-subject study to investigate the association between DCE design efficiency and utility estimates. It found that a majority of people did not trade across attributes in the more efficient design but that one third of these then did trade in the less efficient design. More within-subject studies are required to establish how common this is. It may be that future DCEs should attempt to maximise some joint function of statistical and cognitive efficiency to maximise overall efficiency and minimise bias.
KeywordsDecision Rule Latent Class Analysis Discrete Choice Experiment Efficient Design Latent Class Model
We acknowledge participants at the International Choice Modelling Conference in Sydney 2013 for helpful comments and Prof. Jordan Louviere for providing the conceptual basis for the two designs used and for comments on the results. This research was supported by the Singapore Ministry of Health’s National Medical Research Council (NMRC/NIG/1036/2010) and Lien Centre for Palliative Care (Duke-NUS-LCPC(I)/2011/0001). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
Compliance with Ethical Standards
TNF designed the discrete choice experiments, conducted principal analyses, drafted the paper and agreed edits to the manuscript. He has no conflict of interest.
MB co-designed the overall survey, suggested secondary analyses and edited the manuscript. He has no conflict of interest.
CM co-designed the overall survey, led administration of the survey and edited the manuscript. She has no conflict of interest.
EAF co-designed the overall survey, suggested secondary analyses, led the overall project and edited the manuscript. He has no conflict of interest.
- 7.Ben-Akiva M, Lerman SR. Discrete choice analysis: theory and application to travel demand. Cambridge: MIT Press; 1985.Google Scholar
- 19.Giergiczny M, Czajkowski M. Investigating preference heterogeneity through individual level modelling. International Choice Modelling Conference. Leeds. 2011.Google Scholar