Skip to main content

Is what you choose what you want?—outlier detection in choice-based conjoint analysis

Marketing Letters volume 28, pages 29–42 (2017)Cite this article

Abstract

Choice-based conjoint (CBC) analysis has long been a popular technique in market research. Because CBC is dependent upon respondents’ stated preferences, respondent variability should be taken into account in part-worth estimation. In the spirit of Bayesian residuals within the probit framework, this paper proposes a novel respondent variability measure for CBC, called the “utility deviation” (UD), to detect outliers who have unusually high respondent variability. UD is constructed based on the standardized deviation between a respondent’s true and representative utilities on the made choices. We compare UD with the largest absolute realized deviation (LARD) statistic and the typically used metric, root likelihood (RLH), in the performance of outlier detection using simulated and empirical data. The results show that UD performs slightly better than LARD and significantly outperforms RLH. Finally, we show that performing outlier detection to exclude misleading data can significantly improve the quality of estimation and resultant applications.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3

Notes

  1. One of the anonymous reviewers suggested using the median instead of mean in the posterior summary to measure the central tendency of the ordinal ranking scores. Based on a real data study, we found that the difference between mean and median is minor.

  2. We first converted the log price into the original scale by taking the exponential. The camera price was then discretized into four categories: (0,200), [200,300), [300,400), and ≥400.

References

  • Aizaki, H. (2012). Basic functions for supporting an implementation of choice experiments in R. Journal of Statistical Software, 50, 1–24.

    Article  Google Scholar 

  • Albert, J. H., & Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association, 88(422), 669–679.

    Article  Google Scholar 

  • Albert, J., & Chib, S. (1995). Bayesian residual analysis for binary response regression models. Biometrika, 82–4, 747–759.

    Article  Google Scholar 

  • Allenby, G.M., & Ginter, J.L. (1995). Using extremes to design products and segment markets. Journal of Marketing Research, 32(4), 392–403.

  • Bradlow, E. T., Weiss, R. E., & Cho, M. (1998). Bayesian identification of outliers in computerized adaptive tests. Journal of the American Statistical Association, 93(443), 910–919.

    Article  Google Scholar 

  • DeSarbo, W. S., Ramaswamy, V., & Cohen, S. H. (1995). Market segmentation with choice-based conjoint analysis. Marketing Letters, 6(2), 137–147.

    Article  Google Scholar 

  • Gilbride, T. J., & Allenby, G. M. (2004). A choice model with conjunctive, disjunctive, and compensatory screening rules. Marketing Science, 23(3), 391–406.

    Article  Google Scholar 

  • Goldstein, H., & Healy, M.J. (1995). The graphical presentation of a collection of means. Journal of the Royal Statistical Society. Series A (Statistics in Society), 158(1), 175–177.

  • Greene, W. H. (2008). Econometric analysis, 6th edn. Upper Saddle River: Pearson Education.

  • Haaijer, R., Wedel, M., Vriens, M., & Wansbeek, T. (1998). Utility covariances and context effects in conjoint mnp models. Marketing Science, 17(3), 236–252.

    Article  Google Scholar 

  • Hauser, J. (1978). Testing the accuracy, usefulness, and significance of probabilistic choice models: an information-theoretic approach. Operations Research, 26–3, 406–421.

    Article  Google Scholar 

  • Hofstede, S.N., van Bodegom-Vos, L., Wentink, M.M., Vleggeert-Lankamp, C.L., Vlieland, T.P.V., Marang-van de Mheen, P.J., Study group, D., et al. (2014). Most important factors for the implementation of shared decision making in sciatica care: ranking among professionals and patients. PloS one, 9(4),e94176.

  • Hoogerbrugge, M. and van der Wagt, K. (2006). How many choice tasks should we ask? In Sawtooth Software Conference Proceedings. Sequim, WA, 97–110.

  • Lenk, P. J., DeSarbo, W. S., Green, P. E., & Young, M. R. (1996). Hierarchical bayes conjoint analysis: recovery of partworth heterogeneity from reduced experimental designs. Marketing Science, 15(2), 173–191.

    Article  Google Scholar 

  • Louviere, J. J. (2001). What if consumer experiments impact variances as well as means? Response variability as a behavioral phenomenon. Journal of Consumer Research, 28(3), 506–511.

    Article  Google Scholar 

  • Marschak, J. (1960). Binary-choice constraints and random utility indicators. In Proceedings of a Symposium on Mathematical Methods in the Social Sciences.

  • Meißner, M., Essig, K., Pfeiffer, T., Decker, R., & Ritter, H. (2008). Eye-tracking decision behaviour in choice-based conjoint analysis. In Perception, vol 37. PION LTD, 97–97.

  • Orme, B., & Johnson, R. (2006). External effect adjustments in conjoint analysis. In SAWTOOTH SOFTWARE CONFERENCE.

  • Rossi, P., McCulloch, R., & Allenby, G. M. (1996). The value of purchase history data in target marketing. Marketing Science, 15–4, 321–340.

    Article  Google Scholar 

  • Rossi, P., Allenby, G. M., & McCulloch, R. (2005). Bayesian statistics and marketing. England: Wiley.

    Book  Google Scholar 

  • Sawtooth Software (2013). The CBC System for Choice-Based Conjoint Analysis Version 8. Sawtooth Software Technical Paper Series. Sawtooth Software, Inc. Download: https://www.sawtoothsoftware.com/download/techpap/cbctech.pdf.

  • Selove, M., & Hauser, J. (2011). The strategic importance of predictive uncertainty in conjoint design. Working Paper.

  • Sermas, R., & Colias, M.J.V. (2012). Package “ChoiceModelR” V1.2. Download: https://cran.rproject.org/web/packages/ChoiceModelR/ChoiceModelR.pdf.

  • Shu, S.B., Zeithammer, R., & Payne, J. (2013). Consumer preferences for annuities: beyond NPV. Working Paper.

  • Steckel, J. H., & O’Shaughnessy, J. (1989). Towards a new way to measure power: applying conjoint analysis to group decisions. Marketing Letters, 1(1), 37–46.

    Article  Google Scholar 

  • Suresh, N., & Conklin, M. (2010). Quantifying the impact of survey design parameters on respondent engagement and data quality. In CASRO Panel Conference.

  • Tanner, M. A., & Wong, W. H. (1987). The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association, 82(398), 528–540.

    Article  Google Scholar 

  • Zaslavsky, A.M., & Bradlow, E.T. (2010). Posterior predictive outlier detection using sample reweighting. Journal of Computational and Graphical Statistics, 19(4), 790–807.

  • Zeithammer, R., & Lenk, P. (2006). Bayesian estimation of multivariate-normal models when dimensions are absent. Quantitative Marketing and Economics, 4–3, 241–265.

    Article  Google Scholar 

Download references

Acknowledgments

We are grateful to the editor and two anonymous reviewers for their careful review and constructive suggestions. We also thank Dr. Pankaj Kumar at Comcast, Dr. Rajan Sambandam at TRC Market Research, and Dr. Yu-Ru Su at Fred Hutchinson for their valuable comments on this research.

Author information

Authors and Affiliations

  1. Department of Statistics, North Carolina State University, Raleigh, NC, 27695, USA

    Yu-Cheng Ku

  2. School of Economics, Henan University, Kaifeng, China

    Tsun-Feng Chiang

  3. Department of Statistics, National Cheng Kung University, Tainan, Taiwan

    Sheng-Mao Chang

Authors
  1. Yu-Cheng Ku
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tsun-Feng Chiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sheng-Mao Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu-Cheng Ku.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ku, YC., Chiang, TF. & Chang, SM. Is what you choose what you want?—outlier detection in choice-based conjoint analysis. Mark Lett 28, 29–42 (2017). https://doi.org/10.1007/s11002-015-9389-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11002-015-9389-3

Keywords

  • Bayesian residuals
  • Choice-based conjoint
  • Hierarchical Bayes
  • Outlier detection
  • Random utility theory
  • Respondent variability