Predictive Validity in Choice-Based Conjoint Analysis: A Comparison of Hypothetical and Incentive-Aligned ACBC with Incentive-Aligned CBC: An Abstract

Abstract

Up to the present day, conjoint analysis is one of the most widely applied methods in marketing research for understanding customer requirements and anticipating consumers’ purchase decisions (Kim, Bailey, Hardt & Allenby, 2016; Voleti, Srinivasan & Gosh, 2016). Implications extracted from conjoint analysis often have great influence on managerial decision-making regarding product innovation processes, pricing questions, and market penetration decisions, leading to strong demands that conjoint results should be trustworthy (Aaker, Kumar, Leone, & Day, 2013).

Against this background, we explore the differences in predictive validity by comparing two well-established conjoint techniques: choice-based conjoint (CBC) and the newer, adaptive choice-based conjoint (ACBC) analysis. In a study on Sony’s PlayStation 4, ACBC analysis incorporated all four stages (BYO, screener, choice tournament, and calibration) to allow for an analysis of the impact that the optionally applicable calibration section has on the results’ predictive validity. Moreover, we took appropriate incentive-aligning mechanisms into account, because, in the domain of CBC, incentive alignment has already been shown to enhance predictive validity (e.g., Ding, Grewal & Liechty, 2005; Ding, 2007). Implementing a lottery procedure that rewards one participant in each condition, we applied the direct mechanism procedure (Toubia, de Jong, Stieger & Füller, 2012) to induce incentive alignment in CBC, while introducing a new mechanism for incentive aligning ACBC. More precisely, we integrated the final winner concept of ACBC’s choice tournament section as a reward option besides the participant’s holdout task choice. Potential buyers of PlayStation 4 participated in the study, either undergoing an incentive-aligned CBC, an incentive-aligned ACBC, or a hypothetical ACBC.

We find incentive-aligned ACBC to perform slightly better than its hypothetical counterpart. More importantly, in both cases, the prediction quality increases significantly if the “None” parameter is derived from ACBC’s calibration section, allowing ACBC to even outperform incentive-aligned CBC which, to date, represents the gold standard (Steiner & Hendus, 2012; Wlömert & Eggers, 2016). Other parameters, such as the mean absolute error and the mean hit probability, support the results. Consequently, our results highlight a new field of inquiry, as incentive-aligned ACBC may help companies gain the maximum benefit from marketing research studies, preventing, e.g., product failures and additional expenditure.