Marketing Letters

, 19:337

Beyond conjoint analysis: Advances in preference measurement

Authors

    • Columbia Business SchoolColumbia University
  • Olivier Toubia
    • Columbia Business SchoolColumbia University
  • Eric T. Bradlow
    • University of Pennsylvania
  • Ely Dahan
    • UCLA Anderson SchoolUniversity of California, Los Angeles
  • Theodoros Evgeniou
    • INSEADBoulevard de Constance
  • Fred M. Feinberg
    • Stephen M. Ross School of BusinessUniversity of Michigan
  • Eleanor M. Feit
    • Stephen M. Ross School of BusinessUniversity of Michigan
  • Sam K. Hui
    • Stern School of BusinessNew York University
  • Joseph Johnson
    • School of Business AdministrationUniversity of Miami
  • John C. Liechty
    • Smeal College of BusinessThe Pennsylvania State University
  • James B. Orlin
    • Sloan School of ManagementMassachusetts Institute of Technology
  • Vithala R. Rao
    • Johnson Graduate School of ManagementCornell University
Article

DOI: 10.1007/s11002-008-9046-1

Cite this article as:
Netzer, O., Toubia, O., Bradlow, E.T. et al. Mark Lett (2008) 19: 337. doi:10.1007/s11002-008-9046-1

Abstract

We identify gaps and propose several directions for future research in preference measurement. We structure our argument around a framework that views preference measurement as comprising three interrelated components: (1) the problem that the study is ultimately intended to address; (2) the design of the preference measurement task and the data collection approach; (3) the specification and estimation of a preference model, and the conversion into action. Conjoint analysis is only one special case within this framework. We summarize cutting edge research and identify fruitful directions for future investigations pertaining to the framework’s three components and to their integration.

Keywords

Preference measurementConjoint analysisMarketing research

1 Introduction: beyond conjoint analysis

Researchers and practitioners often equate preference measurement with conjoint analysis. Indeed, since its introduction (Green and Rao 1971), conjoint analysis (and its variants) has become the method of choice for quantitative preference measurement, and is considered among the major contributions of marketing science to marketing practice. However, conjoint analysis is only a special case of the broader field of preference measurement (Gustafsson et al. 2007). While academic research in conjoint analysis may be viewed by some as mature, the field of preference measurement remains very active, important, and growing.

In this paper, we review recent developments in preference measurement that go beyond the “traditional” set of tools that are familiar to many practitioners and academics, and offer directions for future research. We propose viewing preference measurement as comprising three main components (see Fig. 1): (1) the problem that the study is ultimately intended to address; (2) the design of the preference measurement task and the data collection approach; (3) the specification and estimation of a preference model, and the conversion into action. In the context of conjoint analysis, these three components typically take the following form: (1) the problem is to help (profit-maximizing) firms design products and/or predict market shares; (2) data collection involves consumers rating, ranking, or choosing among hypothetical profiles designed according to traditional statistical efficiency measures; and (3) the output consists of individual-level partworths estimated assuming additive and normative utility model specifications.
https://static-content.springer.com/image/art%3A10.1007%2Fs11002-008-9046-1/MediaObjects/11002_2008_9046_Fig1_HTML.gif
Fig. 1

The changing landscape of preference measurement

In the past few decades, many of the advances in the area of preference measurement have revolved around proposing better methods for designing conjoint analysis questionnaires and estimating individual-level partworths using relatively sparse data. However, in recent years, preference measurement researchers have contributed to all three components of the proposed preference measurement framework. Specifically, users of preference measurement studies now include, in addition to firms, consumers (e.g., using recommendation agents), policy makers, and researchers from various fields. Accordingly, the problems being addressed extend well beyond opportunistic profit maximization to altruistic consumer and social welfare objectives. Researchers have developed novel data collection methods based on interactions between consumers and firms as well as among consumers, making the preference measurement task more engaging and appealing. In addition, incentive-compatible mechanisms have substantially improved the quality of preference measurement data. Finally, researchers have started incorporating behavioral context effects, non-compensatory processes, and dynamic effects into preference models. Thus, as a field, we are moving toward better, faster, easier-to-collect, and truer data.

We hope to see more research in the future that will continue to investigate alternatives to traditional conjoint analysis along the three components of our proposed framework. Moreover, we argue that these three components are interrelated; thus the the optimal decisions in each component are influenced by the other components. For example, the problem being addressed by the preference measurement study should be taken into account in all stages of the study, from the design of the task through model estimation, to the conversion of the estimated preferences into action.

2 Problem

The types of problems being addressed by preference measurement studies are evolving. Companies have started using preference measurement in new ways that go beyond partworth estimation, and users increasingly include consumers, policy makers, and health care professionals, as well as academic researchers from fields where preference measurement is less ubiquitous.

2.1 Helping companies

Conjoint analysis has helped a large number of companies make decisions in areas such as new product development, pricing, segmentation, positioning, and advertising (Cattin and Wittink 1982; Wittink and Cattin 1989). Such decisions have relied primarily on the estimation of partworths. Given the growing diversity and complexity of the shopping environment, companies are increasingly interested in modeling and understanding the actual process through which consumers choose products, in addition to consumers’ partworths. For example, Erdem et al. (2005) estimated a choice model that captures the role of active information search and learning in consumer decision making in the context of high-involvement consumer durables. Iyengar et al. (2008) built a structural model of consumer preferences for non-linear contracts (e.g., two- or three-part tariff cell phone plans). Gilbride and Allenby (2004) and Jedidi and Kohli (2005) went beyond partworth estimation and utilized preference measurement techniques to study the formation of consideration sets. Preference measurement could also be used more extensively by companies to guide project selection and investment decisions.

2.2 Helping consumers

The last few years have seen a great increase in the number of preference measurement methods designed to help consumers make better choices. The most prevalent example is that of recommendation agents. Recommenders have been and continue to be a popular research topic in various fields, such as information systems, computer science and machine learning (Adomavicius and Tuzhilin 2005; Srebro et al. 2005), psychology (Häubl and Murray 2003) and marketing (Ansari et al. 2000; Arora et al. 2008; Häubl and Trifts 2000; Liechty et al. 2001; Ying et al. 2006). A good example of the revived interest in this topic is the “Netflix Prize” (www.netflixprize.com). The use of preference measurement methods in recommendation systems requires researchers to modify current methodologies in ways that substantially shorten the preference measurement task, and, in some cases, allow practitioners to estimate and utilize partworths in real time (De Bruyn et al. 2008).

2.3 Helping policy makers and health care professionals

Policy makers and health care professionals (e.g., doctors, pharmaceutical companies, hospitals) have become increasingly interested in preference measurement techniques. Their objective may be opportunistic (e.g., maximize profit, maximize chances of winning an election) or altruistic. For example, in medical decision analysis, Bleichrodt and Pinto (2000) developed a non-parametric method to elicit the probability weighting function in the context of choices between medical treatments. Saigal et al. (2007) used conjoint analysis to optimize treatment for prostate cancer based on each patient’s unique tradeoffs between various outcomes and side effects. Parker and Srinivasan (1976) used preference measurement techniques to incorporate patients’ preferences in planning a rural primary health care delivery system.

2.4 Helping academic researchers

Preference measurement is inherently an interdisciplinary field. For example, some of its origins may be traced back to mathematical psychology and transportation. While most of the active work on the topic is currently linked to marketing, we expect the preference measurement community to expand to new fields in the coming years. For example, behavioral economists are increasingly interested in individual-level estimates of the parameters of the value function and the probability weighting function (Kahneman and Tversky 1979; Prelec 1998; Gonzalez and Wu 1999) in the context of cumulative prospect theory (Tversky and Kahneman 1992). Such estimates allow studying the relationship among parameters that represent loss aversion or risk aversion and individual characteristics such as age, income, or education (Tanaka et al. 2007), or between such parameters and behavior (Fehr and Goette 2007; Jarnebrant et al. 2008). We believe that advances in preference measurement, such as adaptive questionnaire design and Bayesian estimation, may benefit this community of researchers. In like manner, researchers in preference measurement may greatly benefit from collaborating with colleagues in fields such as computer science (Evgeniou et al. 2005), education (Bradlow 2005), engineering (Michalek et al. 2005), and psychology (Otter et al. 2008).

3 Design and data collection

3.1 Optimal experimental design: beyond A-efficiency and D-efficiency

The design of conjoint experiments has traditionally focused on maximizing design efficiency measures such as D-efficiency or A-efficiency (Addelman 1962; Kuhfeld et al. 1994). These measures of efficiency are based on matrix norms defined on the covariance matrix of the estimates of the partworths. In other words, in the context of an individual-level regression, D-efficient or A-efficient designs (such as the well-known orthogonal designs) produce partworth estimates that have minimal variance and intercorrelation.

However, those traditional efficiency measures overlook the managerial objective of the preference measurement study. In particular, while traditional measures of efficiency focus on the covariance matrix of the partworths, managers typically take actions that are based on some functions of these partworths (e.g., willingness to pay for a specific feature), and put more weight on some decisions than others. Toubia and Hauser (2007) proposed M-efficiency measures that account for such managerial considerations. Future research may incorporate other aspects of the environment, such as engineering constraints (Michalek et al. 2005) or prior knowledge of consumers’ preferences, into the design stage of preference measurement studies. For example, Gensler et al. (2007) consider acceptable ranges of willingness to pay in the design of an adaptive choice-based conjoint analysis. Along similar lines, the existence of unacceptable product features or combinations of features may have an impact on the criteria used to evaluate possible designs. Note, however, that one should be cautious in asking consumers directly which attribute levels are unacceptable (Green et al. 1988).

More generally, we believe that experimental design may be greatly enhanced by being systematically approached using Bayesian Decision Theory (Chaloner and Verdinelli 1995). Bayesian experimental designs minimize an expected loss function over the posterior distribution of the parameter estimates. For example, Sandor and Wedel (2001) proposed a method for eliciting managers’ prior beliefs about attribute preferences and used this prior information to design Bayesian D-efficient choice experiments. Sandor and Wedel (2005) showed how taking prior information about heterogeneity across consumer preferences into account affects design optimality. In particular, they show how the use of a small set of different conjoint designs improves efficiency over a single design administered to all participants. However, A-efficiency and D-efficiency are just special cases corresponding to two particular loss functions. The specific context of the study may give rise to alternative loss functions and/or prior distributions on the parameters that more accurately reflect the objectives and beliefs of the user. In summary, when designing a preference measurement task, we encourage researchers to incorporate aspects such as managerial objectives, prior beliefs, constraints and characteristics of the task into the criteria used to evaluate the design.

3.2 New forms of interactions

Preference measurement data have been traditionally collected using pencil and paper questionnaires or one-on-one or mail–telephone–mail interviews involving sorting or rating tasks. Since the early 1990s, many respondent interactions have been relegated to computer and web interfaces. The use of web-based questionnaires triggered the development of adaptive methods that allow collecting more information per question. Adaptive methods include the commercially available adaptive conjoint analysis (ACA, Johnson 1987), the Fast Polyhedral approach (Toubia et al. 2003, 2007b, 2004; Vadali et al. 2007), the Adaptive Self-Explicated approach (Netzer and Srinivasan 2008).

However, the technological advances and easier accessibility to respondents afforded by the web come at the cost of decreased respondent patience and attentiveness. Thus, it is becoming more important than ever to keep respondents engaged with the task. Dahan and Hauser (2002) surveyed several virtual interactive web-based interfaces that have been proposed in the past few years to address that issue. For example, the user design approach collects preference data by allowing respondents to design their ideal virtual product (von Hippel and Katz 2002). The Information Pump (Prelec 2001) and the Securities Trading of Concepts (STOC; Dahan et al. 2007a, b) collect preference data by allowing respondents to interact with one another in game-like mechanisms, making the task more engaging and fun. Note that, when designing data collection methods that are based on interactions among consumers, one needs to be aware of biases that such interactions may induce (Johnson et al. 2005). Keeping respondents engaged may also be achieved by showing them physical prototypes to increase the realism of the task (Luo et al. 2008; Srinivasan et al. 1997). Dahan and Srinivasan (2000) took this approach even further and reduced its cost by developing a web interface to measure preferences using static or dynamic virtual prototypes.

Another method to increase consumer involvement is to replace the commonly used hypothetical data collection exercises with incentive-aligned tasks, in which respondents have to “live with” their decisions (Ding 2007; Ding et al. 2005; Park et al. 2008; Prelec 2001). A recent study by Ding (2007) suggested that incentive-aligned mechanisms may be used even when not all the product profiles exist in the market. Incentive-aligned mechanisms were empirically found to increase not only respondents’ engagement but also out-of-sample predictive validity. For example, the incentive-aligned mechanism proposed by Ding et al. (2005) increased hit rates (correct prediction of the first choice out of 21 options) by almost a factor of two (from 26% to 48%). Incentive-aligned mechanisms have been shown to be very effective also in economic experiments for market design such as matching and public goods problems (Amaldoss et al. 2008).

In summary, when building a data collection mechanism, it is important to keep in mind the experience of the consumer completing the task. To be specific, since the ultimate goal is usually to predict actual behavior, engaging and incentive-compatible mechanisms should be favored over hypothetical tasks.

3.3 Dealing with a large number of attributes and products

As products become more complex, consumer preferences need to be measured over a larger number of product attributes and levels. Applications of conjoint analysis have been conducted on products involving as many as 50 product attributes (Wind et al. 1989). Several methods have been proposed to handle the demand for complex problems. The traditional self-explicated approach (Srinivasan 1988) can deal with a large number of attributes and levels. However, this approach carries several limitations (Green and Srinivasan 1990), which have been partially overcome by hybrid estimation methods that combine self-explicated data with preference data from full or partial profile tasks (Green et al. 1981; Johnson 1987; Marshall and Bradlow 2002; Ter Hofstede et al. 2002). Utilizing the concept of complexity control from machine learning, Cui and Curry (2005) and Evgeniou et al. (2005) used a support vector machine approach to handle complex preference measurement problems.

Researchers recently proposed to address the problem of large product dimensionality by developing innovative data collection mechanisms. For example, the Conjoint Adaptive Ranking Database System (CARDS) method proposed by Dahan (2007) simplifies the conjoint analysis task by asking respondents to choose only among the very limited number of sets that are perfectly mapped to specific utility functions proposed in advance by the researcher. Park et al. (2008) proposed an auction-based approach in which respondents can auction a large number of product feature upgrades. Taking a different approach, Netzer and Srinivasan (2008) developed an adaptive self-explicated approach to solve the self-explicated constant sum question problem when the number of product attributes becomes large, demonstrating significant improvement in predictive validity. We expect that many of the advances in our ability to study complex problems will come from the development of such innovative data collection techniques and from the use of auxiliary information.

3.4 Combining multiple sources of data

Traditionally, preference measurement studies have relied on data provided explicitly and consciously by consumers during the preference measurement task. Marketers recently started identifying new sources of data and supplementing stated preference data with auxiliary revealed preference data to: (1) improve predictive ability; (2) ask fewer questions; (3) correct biases related to the preference measurement task.1 Such auxiliary data may be either internal or external to the preference measurement task.

3.4.1 Internal sources of data

Examples of data that are internal to the task include response latencies, eye movement, and mouse movement. Haaijer et al. (2000) demonstrated that response time is related to preference by means of choice uncertainty, whereby shorter response times represent more certain choices. Otter et al. (2008) proposed a Poisson race model to capture response time in conjoint analysis. Netzer et al. (2008) modeled and exploited the relation between response time and choice conflict. Liechty et al. (2003) utilized eye movement data to identify the attention state of respondents when evaluating stimuli. In the future, we expect that more decision process data such as mouse movement, click-stream data and brain images will be utilized in preference measurement.

3.4.2 External sources of data

Examples of auxiliary data that are external to the task include, but are not limited to, sales and market share data. Feit et al. (2007) developed a method for melding experimental choice data and data on market purchases to leverage the best properties of both. Along the same lines, Horsky et al. (2006) demonstrated the benefits of combining scanner-based data with survey-based preference data. Gilbride et al. (2006) proposed a loss function approach to incorporate market share information as constraints in the estimation of choice-based conjoint analysis partworths. De Bruyn et al. (2008) combined preference measurement data with intended product use and customer characteristics data, in the context of recommendation agents. Some less traditional sources of auxiliary data have also been investigated recently. For example, Hui et al. (2008a) and Hui et al. (2008b) measured consumer preferences by combining shopping path data (collected using RFID technology) with transaction data. Another promising external source of data includes readily available data posted on the internet, such as product reviews (Lee and Bradlow 2008).

With the advantages offered by combining multiple sources of information comes the difficulty and complexity of combining data sets that are often not fully aligned with one another. Several approaches have been suggested including data fusion (Gilula et al. 2006), common individual characteristics (Feit et al. 2007) and common latent constructs underlying the multiple data sets (Hui et al. 2008a).

We encourage researchers to identify unique sources of data that could improve our ability to measure consumers’ preferences and to develop methods to overcome the difficulties involved in combining multiple sources of data.

4 Model specification, estimation, and action

4.1 Taking social interactions into account

Preference measurement models have almost exclusively assumed that consumers make choices independently of one another. Some noteworthy exceptions include Rao and Steckel (1991) who studied the polarizing effects of group decision making, Arora and Allenby (1999) who modeled decisions made jointly by husbands and wives, and Ding and Eliashberg (2007) who proposed formal models of multi-party decision-making and applied them to choices of pharmaceutical prescriptions by doctors and patients. Recent research in marketing has continued to highlight and illustrate the importance of social interactions in consumption and choices (e.g., Godes and Mayzlin 2004; Goldenberg et al. 2002). We believe that capturing such interactions more systematically in preference measurement is an important area for future research.

4.2 Meta-attributes

Preferences are often modeled and estimated in the space defined by product attributes and levels. Working in this space makes the translation of consumer preferences into engineering terms easier. However, consumers often think in terms of “meta-attributes” such as needs, motivations, and goals, which may correspond to bundles of physical product attributes. There are several advantages to working in meta-attribute spaces. First, if consumers indeed evaluate products according to meta-attributes, the preference measurement task may become more natural. Second, using dimensions like goals and needs, which are the true drivers of decision making, is likely to lead to better preference measurement. Finally, needs, motivation, and goals are likely to be more stable over time than preferences for specific product attributes (e.g., consumers may have stable preference for faster computers, but their preference for a specific processor may change over time as technology evolves). While working with meta-attributes may be beneficial, identifying and constructing meta-attributes can prove to be difficult. Methods such as factor analysis may provide some insights but lack the fundamental ability to create maps between physical attributes and meta-attributes. The challenge of finding these maps is confounded with issues of language that could be used to describe meta-attributes. Text mining of consumer-written product reviews (Lee and Bradlow 2008) is a potentially valuable tool for automating the process of identifying the language consumers use to describe products. Furthermore, the translation between meta-attributes defined in consumer language and engineering specifications used in product design may not be straightforward.

A few successful attempts to integrate meta-attributes in preference measurement include Luo et al. (2008) who incorporated meta-attributes such as “comfort” and “power” along with more objective characteristics. In the context of recommendation agents, De Bruyn et al. (2008) used tree-based methods combined with higher level “ask-once” questions to group consumers, suggesting that meta-attributes may be related to and identified with “ask-once” questions in online or offline recommendations. Ghose and Rao (2007) tackled directly the topic of how one could construct and utilize meta-attributes in the context of conjoint analysis. We hope to see more work along these lines in the future.

4.3 More flexible utility functions

Preference measurement has typically assumed linear and additive utility functions. An increasing number of papers have explored utility functions that deviate from these assumptions. For example, Kim et al. (2007) modeled preferences using Bayesian splines with endogenous knot configurations, finding hold-out choice prediction improvement in the 10–20% range. Ben-Akiva et al. (2002) proposed a hybrid choice model that integrates many types of discrete choice modeling methods, draws on different types of data, and allows the explicit modeling of latent psychological explanatory variables. Other researchers have explored non-compensatory utility functions. Yee et al. (2008) and Kohli and Jedidi (2007) proposed dynamic programming methods to estimate lexicographic preference structures. Non-compensatory processes seem particularly relevant in the context of consideration sets. Gilbride and Allenby (2004) modeled a two stage process in which the first stage consists of a (potentially) non-compensatory screening of alternatives and the second stage of a compensatory choice among the remaining alternatives. They estimated their model using hierarchical Bayes methods, augmenting the latent consideration sets within their MCMC approach. Jedidi and Kohli (2005) introduced subset-conjunctive screening rules, which generalize disjunctive and conjunctive rules. Non-compensatory decision process may be viewed as the result of simplifying heuristics used by boundedly rational consumers during the preference measurement task. For example, Kim (2004) used a Bayesian hidden Markov model to describe changes in individual consumers’ latent choice heuristics over time.

We hope that future work in this area will enhance the ecological rationality of preference measurement models, i.e., will improve the fit between the structural properties of the model and the structure of the environment to which it is applied.

4.4 Incorporating behavioral effects

The process of data collection in preference measurement often involves a sequence of choices, ranking, ratings, or tradeoffs between attributes and/or products. Much of the research in Behavioral Decision Theory has been focused on studying context and other behavioral effects that may be prevalent when consumers are making such decisions. Therefore, it is surprising that only a handful of studies have attempted to test and apply the battery of robust and significant behavioral effects documented in the consumer behavior literature to preference measurement.

Some of the early work on incorporating behavioral effects into preference measurement explored the effect of the number of attribute levels on the perceived attribute importances (Wittink et al. 1989). The authors suggested that researchers should try to keep the number of attribute levels similar across attributes to avoid biases. Bradlow et al. (2004) investigated and modeled the behavioral effects caused by omitting product attributes in partial profile designs. A few studies have also attempted to model context effects in preference measurement. Kivetz et al. (2004a) proposed several choice models that could capture the well-known compromise effects given a set of partworths collected using alternative preference measurement tasks. In a follow-up paper, the authors suggested that their models could capture additional context effects such as asymmetric dominance, attraction, and detraction (Kivetz et al. 2004b). Haaijer et al. (1998) proposed a flexible covariance matrix that could potentially capture context effects in choice-based conjoint analysis. The paper by Adamowicz et al. (2008), appearing in the current issue of the journal, provides a detailed overview of behavioral effects in choice modeling.

One of the difficulties involved with studying behavioral effects in preference measurement is that one cannot claim that a model describes behavior better than another model based on superior fit or predictive ability only. In particular, more complex models naturally tend to fit better and can often predict worse (due to potential overfitting). Therefore, many factors may influence fit and predictive ability, beyond the accuracy of the behavioral assumptions made by the model. Claiming that a model is isomorphic to the true underlying decision process (i.e., it actually captures the underlying behavior) seems to require exogenous manipulations and/or a set of process measures. Otherwise, a model may only be shown to be paramorphic to the true underlying decision process (i.e., it gives rise to similar outcomes).

Nevertheless, we believe that with the increase in the number of contact points between firms and consumers, and therefore in the number of ways in which practitioners may influence the choice process, consumer psychology is more relevant than ever to preference measurement from a managerial perspective. From an academic perspective, we hope to see a two-way exchange between the preference measurement and consumer psychology community. Psychologists can suggest behavioral effects that may improve the accuracy of preference measurement while preference measurement researchers in turn can develop new methods for measuring and testing alternative behavioral effects.

4.5 Modeling learning, dynamics and preference formation

Most preference measurement models assume that consumers have well-defined and stable preferences. The above discussion suggests that preferences may not be well formed and may be influenced by the task itself and by its context. Furthermore, if preferences are not well formed we are likely to observe dynamics throughout the preference measurement task as a result of preference formation, learning or fatigue. DeSarbo et al. (2005) and Liechty et al. (2005) proposed models that allow the partworth estimates to vary throughout the preference measurement task using a dynamic random effects model. Su and Rao (2007) studied the evolution of willingness to pay for different types of attributes and how such changes affect new product adoption. Many of the flexible models developed to capture dynamics in repeated choice (e.g., Kim et al. 2005; Lachaab et al. 2006) could be applied to preference measurement. Bradlow et al. (2004) take a first step in understanding the antecedents of dynamics by studying consumer learning about preferences for missing attribute levels in a partial profile design. We join Bradlow (2005) in the call for more work attempting to disentangle the different sources of dynamic effects in preference measurement.

4.6 Recent tools for estimation

The standard estimation method for conjoint analysis has become hierarchical Bayes (Lenk et al. 1996; Rossi and Allenby 2003). Although this estimation method has been researched extensively, it continues to be an exciting research area. For example, Sonnier et al. (2007) showed that specifying a normal heterogeneity distribution on the parameters of the multinomial logit model implies a distribution on willingness-to-pay that has substantial mass in the tail, leading to extreme behavior for some individuals. This suggests that prior or heterogeneity distributions should be specified on meaningful quantities (e.g., willingness-to-pay) instead of on latent constructs, like partworths.

An alternative approach to conjoint estimation is based on optimization. This approach has a long history, starting with the Linmap method of Srinivasan and Shocker (1973a, b). More recently, Toubia et al. (2003) and Toubia et al. (2004) proposed polyhedral methods for conjoint estimation and questionnaire designs. These methods are based on interpreting the answer to each conjoint question as a constraint on the respondent’s partworths. Toubia et al. (2007b) and Vadali et al. (2007) generalized the polyhedral methods to capture response error and informative priors on the parameters. Evgeniou et al. (2005), Cui and Curry (2005) and Evgeniou et al. (2007) proposed conjoint estimation methods based on machine learning and statistical learning theory. The method of Evgeniou et al. (2007) has been shown to outperform, in some cases, hierarchical Bayes in estimation accuracy and predictive ability. The two methods are comparable conceptually, with the fundamental difference that all parameters are endogenous in the machine learning method of Evgeniou et al. (2007) while some parameters are typically set exogenously in hierarchical Bayes (e.g., the hyperparameters). Finally, Toubia et al. (2007a) showed that many optimization methods for conjoint estimation may be integrated within the framework of statistical learning theory.

One of the current limitations of optimization-based methods is that they produce point estimates, whereas likelihood-based methods such as hierarchical Bayes provide full distributions on the parameter estimates. While Evgeniou et al. (2007) illustrated a bootstrapping approach to obtaining confidence intervals for their method, we believe that future research may explore alternative approaches to allow statistical inference and hypothesis testing for optimization-based methods. More generally, a fundamental challenge that we hope will be addressed in future research is linking optimization-based methods with likelihood-based methods. For example, Toubia et al. (2007b) and Vadali et al. (2007) gave a likelihood interpretation of polyhedral methods. Bridging the likelihood-based and optimization-based approaches may benefit both approaches. For example, Evgeniou et al. (2007) showed an example of how principles from statistical learning theory may be used to significantly improve the estimation accuracy and predictive ability of hierarchical Bayes estimation.

4.7 From model to action

Parameter estimation is often thought of as the final stage of a preference measurement study. However, at the conclusion of a study, it is imperative to come back to the original problem that motivated the study and ensure that a solution is provided to that problem. Some of the key decisions in marketing are those of optimal product design and product line optimization (Dobson and Kalish 1993; Green and Krieger 1985; Kohli and Sukumar 1990; McBride and Zufryden 1988). Recently, Luo et al. (2005) proposed an approach that takes into account variations in the conditions under which the product will be used, and introduced the concept of “robust product design,” which offers excellent performance under worst-case variations and low sensitivity to variations. Recent models in the area of product line optimization have also emerged from engineering, using detailed physical models to determine which products can be produced (Michalek et al. 2005; Wassenaar et al. 2005). These models combine innovative ways to define feasibility constraints with tailored optimization algorithms. For example, Michalek et al. (2007) used analytical target cascading (ATC) to formally coordinate models from marketing and engineering, and design “optimal” marketing-based products and product lines that are technically feasible.

Beyond product line optimization, we believe that the managerial relevance and impact of preference measurement studies may be enhanced by systematically modeling the Bayesian-decision theoretic loss function of the stakeholder (company, consumers, policy makers, etc.), and providing decision support tools for identifying the action that will minimize this loss function over the entire posterior distribution of the parameters being estimated. Currently, most preference measurement studies are used to produce point estimates of some parameters such as partworths. However, basing decisions on point estimates is suboptimal, as decisions should be based on the expected loss across the entire posterior distribution of the estimates (Chaloner and Verdinelli 1995). For example, Blattberg and George (1992) showed that incorporating the manufacturer’s profit-maximizing goal into the Bayesian loss function leads to smaller price-sensitivity estimates and higher optimal prices. Note that in some of the new domains of application identified earlier in this paper, the loss function may take very different forms from that of a profit-maximizing firm. For example, the appropriate loss function for a recommendation agent may include both the utility derived by the consumer from the recommended product and the effort spent by the consumer throughout his or her interactions with the agent. Given the fact that Bayesian Decision Theory involves integrating over posterior distributions, we believe that there is an opportunity to construct decision support tools that will simplify the choice of actions, based on the output of the preference measurement study and all other relevant information.

5 In conclusion...“every generation needs a new revolution”2

Preference measurement is a very exciting and active field that goes well beyond conjoint analysis. We proposed a framework consisting of three interrelated components for approaching this field. We have summarized some cutting edge research and identified fruitful directions for future research pertaining to the framework’s three components, and to their mutual integration. The past two decades have seen great advances in conjoint analysis through the use of computerized adaptive questionnaires and the development of new estimation methods that account for consumer heterogeneity. Moving forward, we encourage researchers to go beyond conjoint analysis and explore new problems and applications of preference measurement, develop new forms of data collection that engage and entice respondents, take advantage of the availability of new sources of data, model new phenomena such as behavioral effects and dynamics, and combine statistical and optimization methods to improve estimation. Moreover, we encourage researchers to take into account the objectives and context of the preference measurement study throughout each step of the process.

Footnotes
1

We refer the reader to the previous Choice Symposium papers by Ben-Akiva et al. (1994) and Louviere et al. (1999) for a summary of the benefits and difficulties of combining stated and revealed preference data.

 
2

Thomas Jefferson.

 

Acknowledgments

The authors would like to thank the SEI Center for Advanced Studies in Management at Wharton for partially supporting this research and for supporting the 7th Triennial Choice Conference held at the Wharton School. The first two authors (session co-chairs) and the third to the 12th authors are listed alphabetically.

Copyright information

© Springer Science+Business Media, LLC 2008