Choice-based conjoint (CBC) analysis has long been a popular technique in market research. Because CBC is dependent upon respondents’ stated preferences, respondent variability should be taken into account in part-worth estimation. In the spirit of Bayesian residuals within the probit framework, this paper proposes a novel respondent variability measure for CBC, called the “utility deviation” (UD), to detect outliers who have unusually high respondent variability. UD is constructed based on the standardized deviation between a respondent’s true and representative utilities on the made choices. We compare UD with the largest absolute realized deviation (LARD) statistic and the typically used metric, root likelihood (RLH), in the performance of outlier detection using simulated and empirical data. The results show that UD performs slightly better than LARD and significantly outperforms RLH. Finally, we show that performing outlier detection to exclude misleading data can significantly improve the quality of estimation and resultant applications.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
One of the anonymous reviewers suggested using the median instead of mean in the posterior summary to measure the central tendency of the ordinal ranking scores. Based on a real data study, we found that the difference between mean and median is minor.
We first converted the log price into the original scale by taking the exponential. The camera price was then discretized into four categories: (0,200), [200,300), [300,400), and ≥400.
Aizaki, H. (2012). Basic functions for supporting an implementation of choice experiments in R. Journal of Statistical Software, 50, 1–24.
Albert, J. H., & Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association, 88(422), 669–679.
Albert, J., & Chib, S. (1995). Bayesian residual analysis for binary response regression models. Biometrika, 82–4, 747–759.
Allenby, G.M., & Ginter, J.L. (1995). Using extremes to design products and segment markets. Journal of Marketing Research, 32(4), 392–403.
Bradlow, E. T., Weiss, R. E., & Cho, M. (1998). Bayesian identification of outliers in computerized adaptive tests. Journal of the American Statistical Association, 93(443), 910–919.
DeSarbo, W. S., Ramaswamy, V., & Cohen, S. H. (1995). Market segmentation with choice-based conjoint analysis. Marketing Letters, 6(2), 137–147.
Gilbride, T. J., & Allenby, G. M. (2004). A choice model with conjunctive, disjunctive, and compensatory screening rules. Marketing Science, 23(3), 391–406.
Goldstein, H., & Healy, M.J. (1995). The graphical presentation of a collection of means. Journal of the Royal Statistical Society. Series A (Statistics in Society), 158(1), 175–177.
Greene, W. H. (2008). Econometric analysis, 6th edn. Upper Saddle River: Pearson Education.
Haaijer, R., Wedel, M., Vriens, M., & Wansbeek, T. (1998). Utility covariances and context effects in conjoint mnp models. Marketing Science, 17(3), 236–252.
Hauser, J. (1978). Testing the accuracy, usefulness, and significance of probabilistic choice models: an information-theoretic approach. Operations Research, 26–3, 406–421.
Hofstede, S.N., van Bodegom-Vos, L., Wentink, M.M., Vleggeert-Lankamp, C.L., Vlieland, T.P.V., Marang-van de Mheen, P.J., Study group, D., et al. (2014). Most important factors for the implementation of shared decision making in sciatica care: ranking among professionals and patients. PloS one, 9(4),e94176.
Hoogerbrugge, M. and van der Wagt, K. (2006). How many choice tasks should we ask? In Sawtooth Software Conference Proceedings. Sequim, WA, 97–110.
Lenk, P. J., DeSarbo, W. S., Green, P. E., & Young, M. R. (1996). Hierarchical bayes conjoint analysis: recovery of partworth heterogeneity from reduced experimental designs. Marketing Science, 15(2), 173–191.
Louviere, J. J. (2001). What if consumer experiments impact variances as well as means? Response variability as a behavioral phenomenon. Journal of Consumer Research, 28(3), 506–511.
Marschak, J. (1960). Binary-choice constraints and random utility indicators. In Proceedings of a Symposium on Mathematical Methods in the Social Sciences.
Meißner, M., Essig, K., Pfeiffer, T., Decker, R., & Ritter, H. (2008). Eye-tracking decision behaviour in choice-based conjoint analysis. In Perception, vol 37. PION LTD, 97–97.
Orme, B., & Johnson, R. (2006). External effect adjustments in conjoint analysis. In SAWTOOTH SOFTWARE CONFERENCE.
Rossi, P., McCulloch, R., & Allenby, G. M. (1996). The value of purchase history data in target marketing. Marketing Science, 15–4, 321–340.
Rossi, P., Allenby, G. M., & McCulloch, R. (2005). Bayesian statistics and marketing. England: Wiley.
Sawtooth Software (2013). The CBC System for Choice-Based Conjoint Analysis Version 8. Sawtooth Software Technical Paper Series. Sawtooth Software, Inc. Download: https://www.sawtoothsoftware.com/download/techpap/cbctech.pdf.
Selove, M., & Hauser, J. (2011). The strategic importance of predictive uncertainty in conjoint design. Working Paper.
Sermas, R., & Colias, M.J.V. (2012). Package “ChoiceModelR” V1.2. Download: https://cran.rproject.org/web/packages/ChoiceModelR/ChoiceModelR.pdf.
Shu, S.B., Zeithammer, R., & Payne, J. (2013). Consumer preferences for annuities: beyond NPV. Working Paper.
Steckel, J. H., & O’Shaughnessy, J. (1989). Towards a new way to measure power: applying conjoint analysis to group decisions. Marketing Letters, 1(1), 37–46.
Suresh, N., & Conklin, M. (2010). Quantifying the impact of survey design parameters on respondent engagement and data quality. In CASRO Panel Conference.
Tanner, M. A., & Wong, W. H. (1987). The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association, 82(398), 528–540.
Zaslavsky, A.M., & Bradlow, E.T. (2010). Posterior predictive outlier detection using sample reweighting. Journal of Computational and Graphical Statistics, 19(4), 790–807.
Zeithammer, R., & Lenk, P. (2006). Bayesian estimation of multivariate-normal models when dimensions are absent. Quantitative Marketing and Economics, 4–3, 241–265.
We are grateful to the editor and two anonymous reviewers for their careful review and constructive suggestions. We also thank Dr. Pankaj Kumar at Comcast, Dr. Rajan Sambandam at TRC Market Research, and Dr. Yu-Ru Su at Fred Hutchinson for their valuable comments on this research.
Rights and permissions
About this article
Cite this article
Ku, YC., Chiang, TF. & Chang, SM. Is what you choose what you want?—outlier detection in choice-based conjoint analysis. Mark Lett 28, 29–42 (2017). https://doi.org/10.1007/s11002-015-9389-3
- Bayesian residuals
- Choice-based conjoint
- Hierarchical Bayes
- Outlier detection
- Random utility theory
- Respondent variability