Thurstonian-Based Analyses: Past, Present, and Future Utilities
- 3.4k Downloads
Current psychometric models of choice behavior are strongly influenced by Thurstone’s (1927, 1931) experimental and statistical work on measuring and scaling preferences. Aided by advances in computational techniques, choice models can now accommodate a wide range of different data types and sources of preference variability among respondents induced by such diverse factors as person-specific choice sets or different functional forms for the underlying utility representations. At the same time, these models are increasingly challenged by behavioral work demonstrating the prevalence of choice behavior that is not consistent with the underlying assumptions of these models. I discuss new modeling avenues that can account for such seemingly inconsistent choice behavior and conclude by emphasizing the interdisciplinary frontiers in the study of choice behavior and the resulting challenges for psychometricians.
Key wordsconsistency conditions identifiability random utility models social dependencies
Most psychometric models for the analysis of multivariate discrete choice data are based on the assumption that choices among options are determined by their underlying utilities for the decision-makers. Because of significant computational advances in estimating the parameters of the latent utility distribution, the analysis of multivariate choice data is now well on its way of becoming a routine matter (Böckenholt & Tsai, 2006; Caffo & Griswold, 2006). Even the joint modeling of discrete and continuous choice outcomes is starting to be commonplace (Hanemann, 1984; Gueorguieva & Agresti, 2001). It is therefore appropriate to revisit this methodology and to assess its past, current, and future value in modeling choice data.
Because this paper reflects my personal journey and views on the analysis of choice data, many important subjects as well as contributors are left out, for which I apologize. In particular, this paper will not get into the extensive literature on estimation and inference—careful discussions of these topics can be found in the books by Hensher, Rose, and Greene (2005), Train (2003), and Louviere, Hensher, and Swait (2000). The papers presented at the 2004 Invitational Choice Symposium (Chakravarti, Sinha, & Kim, 2005) provide valuable overviews of many interesting topics in modeling choice behavior. In addition, the addresses of two nobel prize laureates in economics (Kahneman, 2003; McFadden, 2001) contain excellent reviews of this field.
My journey is linked to two papers by Thurstone (1927, 1931) that started the era of psychometric choice modeling. The next section will briefly review the contributions of these papers, which is followed by a discussion of current versions of Thurstonian-based choice models. Subsequently, I will point to new research avenues in extending these models to account for seemingly inconsistent choice behavior. The paper concludes by pointing to the interdisciplinary frontiers of studying choice behavior and the resulting future challenges for psychometricians.
Utility Theories and Choice Data
Most psychometric choice models are based on the notion of decision utility. This approach infers utility from observed choices and in turn uses utility to explain choices. For example, if a person chooses a and rejects b, this behavior is interpreted as indicating that a has a higher utility than b to this person. A number of consistency conditions have been proposed in the literature to characterize choices based on decision utilities. Transitivity is awell-known consistency condition stating that a preference for a over b and b over c implies a preference for a over c.
In psychometrics, Thurstone’s (1927, 1931)work proved to be highly influential in promoting an extended version of the decision-utility concept that allowed for stochastic variability in choices. Invoking Fechner’s (1860) psychophysical concept of a sensory continuum, Thurstone (1927) argued in his “Law of Comparative Judgment” that choice options can be represented along a utility continuum by random variables that describe the options’ effects on a person’s cognitive apparatus. Thurstone used randomness as a device to represent factors that determine the formation of preferences but are unknown to the observer. This specification was driven by pragmatic considerations since Thurstone was well aware that he could not distinguish between the positions of whether a choice process is inherently random or determined by a multitude of different factors that may not be measurable. By not specifying the “discriminal” process by which a person “identifies, distinguishes, discriminates, or reacts to stimuli” (1927, p. 369) and defining it to be exogenous to all the factors that can affect a choice process, he broadened tremendously the applicability of this approach to the modeling of choice data. Importantly, he also utilized these ideas for modeling individual differences in choice behavior by arguing that heterogeneity in individual assessments of the same option can be described as realizations of the normal distribution. Introducing Thurstone’s work to economics, Marschak (1960) coined the term Random Utility Maximization (RUM) when referring to choice probabilities that result from the maximization of utilities with random elements.
Not unlike the notion of personality traits, a controversial corollary of decision utility is that the measured utilities are predetermined, decomposable, and stable over both time and across situations. Consistency conditions defined for stochastic preferences include stochastic transitivity and contraction and expansion consistency (Block & Marschak, 1960; Falmagne, 1985). Under contraction consistency, if a choice set T is narrowed to U and the options chosen from T are still in U, then no unchosen options should be chosen and no previously chosen options should be unchosen. Under expansion consistency, if a choice set T is expanded to U, then the probability of choosing an option from U should not exceed the probability of choosing the same option from T.
In addition to providing a probabilistic version of decision utility, Thurstone (1931) also promoted the benefits of experimental methods in collecting choice data. By asking respondents to state directly their preferences for a single or multiple sets of options in controlled experimental settings, he showed that it is possible to obtain information about a person’s preferences that may not be available by observing choices in the person’s daily context. These two data types became known later as “stated preference” and “revealed preference” data.
Thurstone (1931) showed that stated preference data can be used to test for indifference curves which at that time played a pivotal role in consumer demand theory. Initially, economists conceptualized the unit of utility as the “just perceivable increment of pleasure” (Edgeworth, 1881, p. 99) but moved later towards the concept of decision utility and, in this context, adopted Pareto’s (1900) work on indifference curves that depict different bundle combinations a person is indifferent to choose among. Interestingly, Thurstone’s (1931) experimental investigation of indifference curves was met initially with severe criticism by economists (Georgescu-Roegen, 1936; Wallis & Friedman, 1942) and had little impact on subsequent developments of demand theory. Reasons for the negative reactions included the hypothetical nature of the choice situation, the apparent empirical difficulties in detecting indifference, and the experimental lack of control for the effect of income and prices. Looking back, however, Thurstone’s (1931) study was rather significant because it is the first reported experiment in behavioral economics, and also prompted much future work on conjoint measurement and related methods (Luce & Tukey, 1964).
Thurstone’s contributions to experimental paradigms for collecting choice data and to the psychometric foundation of choice models with latent variables, in combination with parallel developments in biometrics, sociometrics, and econometrics, led to a rich set of choice models (Ashford & Sowdon, 1970; McFadden, 1984; Hausman & Wise, 1978). The next section presents a short discussion of these Thurstonian-based models for stated preference data and also points to related developments.
Thurstonian Random Utility Models
Typically, stated choice data are collected in the form of incomplete and/or partial rankings. Incomplete ranking data are obtained when a decision-maker considers only a subset of the choice options. For example, in the method of paired comparison, two choice options are presented at a time, and the decision-maker is asked to select the more preferred one (David, 1988). In contrast, in a partial ranking task, a decision-maker is confronted with all choice options and asked to provide a ranking for a subset of the options. For instance, in the best-worst method, a decisionmaker is instructed to select the best and worst options out of the offered set of choice options. Both partial and incomplete approaches can be combined by offering multiple distinct subsets of the choice options and obtaining partial or complete rankings for each of them. Pick any constant-sum, and ordinal versions of paired comparison and rankings are alternative methods for collecting stated preference data (Böckenholt, 1992, 2001a, 2001b; Böckenholt & Dillon, 1997a; Yao & Böckenholt, 1999).
The Response Model
There are important links between multivariate, non-linear mixed IRT models for nominal and binomial responses and random-utility models (Rijmen, Tuerlinckx, De Boeck, & Kuppens, 2003). Many of these models are based on random-effects versions of Luce’s (1959) choice model which is consistent with a RUM model if and only if the random component in (2) follows a Gumbel distribution (Holman & Marley reported in Luce & Suppes, 1965). Examples include Bock’s (1969, 1972) multinomial logit and nominal models, McFadden’s (1974) conditional logit model, McFadden and Train’s (2000) mixed multinomial logit model, Böckenholt’s (2001b) ranking model, and Skrondal and Rabe-Hesketh’s (2003) multilevel logit model.
The interpretation of the RUM model parameters is limited by the comparative and discrete nature of choice data. Most importantly, it is not possible to identify the origin and scale of the individual utility scales. One option may be preferred to another but this result does not allow any conclusions about the attractiveness level of the options or about interpersonal differences in the utilities (Guttman, 1946). Tsai (2000, 2003) discusses the identifiability of parameter estimates for Thurstonian ranking and paired comparison models (see also Tsai & Böckenholt, 2002). An important implication of this work is that the Case distinctions that were originally proposed by Thurstone (1927) are misleading and have to be interpreted as equivalence classes of covariance structures. However, as noted already by Thurstone and Jones (1957), it is possible to identify the origin of the utility scale by extending the choice task, for example, by including comparisons between pairs of options. Böckenholt (2004) provides a recent review and discussion of different methods that can be used for this purpose. In general, methods for identifying an utility scale origin are not only instrumental for avoiding difficulties in the interpretation of the estimated parameters of a choice model, they also provide useful insights about the underlying judgmental process.
Current Uses of Thurstonian Random Utility Models
This section discusses three major current psychometric research streams in the analysis of choice data. First, much work is underway to enhance our current tool set for modeling individual differences both at a particular point in time and over time. Second, there is strong interest in going beyond modeling the relationship between choice options and decision-makers and to consider a wider range of influences including social context and social influences. A third research stream focusses on ways to supplement and complement choice data because choice outcomes alone provide only limited information about the choice process and its determinants.
Aside from temporal dependencies, there is also strong interest in modeling proximity-based dependencies that are introduced by known or unknown (latent) relationships among individuals (Anselin, 2002; Bradlow et al., 2005). This work goes beyond the standard assumption that individuals make choices in isolation by allowing explicitly for the possibility that choices of individuals are correlated or influenced by each other.
the identification of the reference group for which social interaction effects are sought to be established;
self-selection processes of peer or group members;
controls for correlated effects that affect all group members in a similar way; and
controls for contextual effects such as exogenous social background characteristics of group members.
Because these issues are difficult to tackle in any observational study, progress is most likely to be made by combining observational with quasi-experimental or laboratory studies.
In general, the modeling of interaction structures has been based mainly on mathematical models originating in the area of statistical mechanics (Yeomans, 1992). Although the physical interpretation of these models may be of little interest to social scientists, their mathematical properties are intriguing and deserve to be explored in experiments on social decision-making.
Beyond Choice Data
Because choice data contain little information about the underlying choice process and its determinants, there have been many attempts both to supplement them by considering other data sources such as reaction times or process-tracing data (Böckenholt & Hynan, 1994; Johnson & Busemeyer, 2005) and to complement them by combining revealed and stated preference data (Ben-Akiva et al., 1997), or by collecting comparative judgment data on perceived attributes of the choice options. Below I discuss examples of both approaches by considering risky and multiattribute choices.
Uncertain Choice Outcomes
Many, if not most, real-life decisions are based on a mix of information and subjective expectations about the choice options under consideration. For example, the selection of a job may be based on a job description as well as on expectations about the career path. Purchases of over-the-counter drugs may be influenced by the drugs’ ingredients but also by expectations about the ingredients’ effectiveness and quality perceptions of the manufacturers. If these subjective expectations are rational or well-calibrated, it is possible to infer both expectations and utilities from choices alone. However, there is much evidence to suggest that this assumption is difficult to justify in general (see Kahneman, 2003, for a recent review). As a possible solution to this dilemma, Manski (2004) proposed to measure separately expectations in the form of subjective probabilities and combining them with the choice outcomes. Although much care needs to be taken in the elicitation of subjective probabilities, this approach is likely to mitigate problems arising from assuming that expectations are rational.
Future Uses of Thurstonian-Based Analyses
Shafir and LeBoeuf (2002) review a long list of factors that have been shown to affect decision processes as well as possible explanations that can account for these effects. This list includes contextual effects (e.g., relational features such as dominance among choice options, temporal features of the choice situation), choice processes (e.g., the evaluability of options, decision strategies, information search strategies), presentation formats, frames, cultural and social norms, as well as characteristics of the decision-maker (e.g., emotional state, general intelligence, numeracy). In view of this list, it is perhaps surprising how well (2) can work in providing satisfactory accounts of choice data.
Tversky’s (1969) Gambling Study
Observed and expected probabilities for RUM models with stable and correlated utilities
Pr(a ≻ b)
Pr(a ≻ c)
Pr(a ≻ d)
Pr(b ≻ c)
Pr(b ≻ d)
Pr(c ≻ d)
Pr(a ≻ b, b ≻ c, c ≻ a)
Pr(b ≻ a, c ≻ b, a ≻ c)
Pr(b ≻ c, c ≻ d, d ≻ b)
Pr(c ≻ b, d ≻ c, b ≻ d)
Tsai and Böckenholt (2007) illustrate further the usefulness of distinguishing explicitly between within- and between-judge variability based on (15) in combination with (16) in a reanalysis of the intertemporal choice data reported by Roelofsma and Read (2000). As in the replication of the Tversky (1969) study, (16) proved to be well-suited to account for systematic transitivity violations which were caused in this study by inconsistent trade-offs between “time” and “money” attributes characterizing the choice options. Extensions of the model framework to analyze other choice anomalies are currently underway. These studies show that both the compromise and the attraction effect can be modeled parsimoniously using the decomposition (16) of the within-judge covariance matrix (Böckenholt & Tsai, 2007).
Relaxing the assumption of fixed and predetermined utilities appears to be a promising approach to describe seemingly inconsistent choice behavior. By allowing the individual-level utilities to vary in repeated evaluations of the same options in different choice sets, this approach can accomodate a considerably wider range of inconsistent choice behavior than has been possible so far. Equally important, with this framework it becomes possible to relate the utilities’ reliability estimates to both context- and person-specific covariates with the result that one can test rigorously determinants of variables and possibly inconsistent utility assessments.
Do people choose the options they enjoy most? There is much evidence to suggest that the answer is negative: People do not always know what they like and their ability to forecast future utilities of potential choices appears to be systematically biased (Kahneman & Thaler, 2006). The subsequent question about necessary and/or sufficient (environmental) conditions that facilitate utility maximization has received less attention so far. Notable exceptions include the notion of “libertarian paternalism” (Sunstein & Thaler, 2003) which sets up default options in such a way as to help people in their utility maximization. Along the same lines, a recent large-scale study by McFadden (2006) on choices among Medicare-approved plans demonstrates the importance of aiding consumers in helping themselves instead of relying on their self-interest in making optimal choices (see also Lynch & Wood, 2006).
Although behavioral research on choice behavior points convincingly to limitations of random utility models, few would dispute the usefulness of these models in rendering parsimonious descriptions of how individuals perceive and evaluate choice options. Because they are based on a limited explanatory framework for how people make choices, random utility models can provide both a parsimonious quantitative description of choice outcomes and a flexible framework for modeling individual and contextual differences in choice behavior at a particular point in time and over time. However, generalizations of RUM results to different choice situations and options require care and need to be based on additional validation studies. The consideration of random response errors and unstable preferences alone is not sufficient to account for deviations from utility maximization.
There are many research opportunities on the horizon for psychometricians. Better measures and indicators of utility are needed that capture the hedonic experience associated with a choice outcome. To infer utility from choices alone without taking into account anticipated or experienced hedonic reactions no longer seems sufficient (Kahneman, 2003). Measures of brain activity and brain electrochemistry in combination with experimental treatments are starting to become available to provide much needed insights on the links between choice and sensations of pleasure and pain but statistical methods are lacking for effective analyses of these links (Montague, King-Casas, & Cohen, 2006). The development of new measures and indicators is facilitated further by expanding their connections to psychological concepts and processes. For example, affective and motivational mechanisms are becoming integrated in theories of individual choice behavior and have led to new concepts such as “irrational wanting” and “subrational liking” (Winkielman & Berridge, 2003), pointing to obvious limitations in current choice modeling frameworks. Recently, Mourali, Böckenholt, and Laroche (in press) showed that the compromise and asymmetric dominance effects can be weakened or strengthened depending on whether a person has a promotion or prevention focus (Higgins, 1997), demonstrating that motivational factors need to be taken into account when modeling choice data. In general, the search for choice models that are behaviorally more realistic but still tractable is complicated greatly by identifiability and endogeneity issues. Current choice models for unstable preferences, uncertain outcomes, or social interactions suffer from difficulties both in separating different sources of heterogeneity and in identifying multiple equilibria, all of which needs to be studied carefully with the help of well-designed empirical studies. Clearly, challenges for modeling and predicting choice behavior are abundant but they also assure the future well-being of psychometrics.
- Ansari, A., & Iyengar, R. (in press). Semiparametric Thurstonian models for recurrent choices: A Bayesian analyis. Psychometrika, 71.Google Scholar
- Block, H., & Marschak, J. (1960). Random orderings and stochastic theories of response. In I. Olkin et al. (Eds.), Contributions to probability and statistics (pp. 97–32). Stanford, CA: Stanford University Press.Google Scholar
- Bock, R.D. (1969). Estimating multinomial response relations. In R.C. Bose et al. (Eds.), Essays in probability and statistics (pp. 111–32). Chapel Hill: University of North Carolina Press.Google Scholar
- Bock, R.D., & Jones, L.V. (1968). The measurement and prediction of judgment and choice. San Francisco: Holden-Day.Google Scholar
- Böckenholt, U. (1992). Thurstonian models for partial ranking data. British Journal of Mathematical and Statistical Psychology, 45, 31–9.Google Scholar
- Böckenholt, U. (1996). Analyzing multi-attribute ranking data: Joint and conditional approaches. British Journal of Mathematical and Statistical Psychology, 49, 57–8.Google Scholar
- Böckenholt, U., & Tsai, R. (2006). Random-effects models for preference data. In C.R. Rao & S. Sinharay (Eds.), Handbook of statistics (Vol. 26, pp. 447–68). Amsterdam: Elsevier Science.Google Scholar
- Böckenholt, U., & Tsai, R. (2007). An unstable preference view of the compromise and asymmetric dominance effects. Manuscript in preparation.Google Scholar
- Brock, W.A., & Durlauf, S.N. (2001). Interactions-based models. In Handbook of econometrics (Vol. 5, pp. 3297–380). Amsterdam: North-Holland.Google Scholar
- Chakravarti, D., Sinha, A., & Kim, J. (2005). Choice research: A wealth of perspectives. Marketing Letters, 16, 173–82.Google Scholar
- Coombs, C.H. (1964). A theory of data. New York: Wiley.Google Scholar
- David, H.A. (1988). The method of paired comparisons. London: Griffin.Google Scholar
- Edgeworth, F.Y. (1881). Mathematical physics. London: Kegan Paul.Google Scholar
- Falmagne, J.C. (1985). Elements of psychophysical theory. Oxford: Clarendon Press.Google Scholar
- Fechner, G.T. (1860). Elemente der Psychophysik. Leipzig: Breitkopf und Härtel.Google Scholar
- Hensher, D.A., Rose, J.M., & Greene, W.H. (2005). Applied choice analysis: A primer. Cambridge: Cambridge University Press.Google Scholar
- Louviere, J.J., Hensher, D.A. & Swait, J.D. (2000). Stated choice methods. New York: Cambridge University Press.Google Scholar
- Luce, R.D. (1959). Individual choice behavior. New York: Wiley.Google Scholar
- Luce, R.D. (2000). Utility of gains and losses: Measurement-theoretical and experimental approaches. Hillsdale, NJ: Erlbaum.Google Scholar
- Luce, R.D., & Suppes, P. (1965). Preference, utility, and subjective probability. In R.D. Luce, R.R. Bush, & E. Galanter (Eds.), Handbook of mathematical psychology (Vol. III, pp. 235–06). New York: Wiley.Google Scholar
- Lynch, J.G., & Wood, W. (2006). Helping consumers help themselves. Journal of Public Policy and Marketing, 26, 1–.Google Scholar
- Marschak, J. (1960). Binary choice constraints on random utility indictor. In K.I. Arrow, S. Karlin, & P. Suppes (Eds.), Stanford symposium on mathematical methods in the social sciences (pp. 312–29). Stanford, CA: Stanford University Press.Google Scholar
- McFadden, D. (1974). Conditional logit analysis of qualitative choice behavior. In P. Zarembka (Ed.), Frontiers in econometrics (pp. 105–42). New York: Academic Press.Google Scholar
- McFadden, D. (1984). Qualitative choice models. In Z. Griliches & M.D. Intriligator (Eds.), Handbook of econometrics (pp. 1395–457). Cambridge, MA: MIT Press.Google Scholar
- Mourali, M., Böckenholt, U., & Laroche, M. (in press). Compromise and attraction effects under prevention and promotion motivations. Journal of Consumer Research.Google Scholar
- Pareto, V. (1900). Sunto di alcuni capitoli di un nuovo trattato di economia pura del Prof. Pareto. Giornale degli Economisti, 20, 216–35, 511–49.Google Scholar
- Rudas, T., Clogg, C.C., & Lindsay, B.G. (1994). A new index of fit based on mixture methods for the analysis of contingency tables. Journal of the Royal Statistical Society, Series B, 56, 623–39.Google Scholar
- Soetevent, A.R., & Kooreman, P. (in press). A discrete choice model with social interactions: With an application to high school teen behavior. Journal of Applied Econometrics.Google Scholar
- Takane, Y. (1987). Analysis of covariance structures and probabilistic binary choice data. Cognition and Communication, 20, 45–2.Google Scholar
- Train, K. (2003). Discrete choice methods with simulations. Cambridge, MA: MIT Press.Google Scholar
- Tsai, R., & Böckenholt, U. (2007). On the importance of distinguishing between within- and between-subject effects in intransitive intertemporal choice. Manuscript submitted for publication.Google Scholar
- Veblen, T. (1899). The theory of the leisure class. New York: Macmillan.Google Scholar
- Wallis, W.A., & Friedman, M. (1942). The empirical derivation of indifference functions. In O. Lange (Ed.), Studies in mathematical economics and econometrics (pp. 175–89). Chicago: University of Chicago Press.Google Scholar
- Wedel, M., & Kamakura, W.A. (1999). Market segmentation: Conceptual and methodological foundations. Dodrecht: Kluwer Academic.Google Scholar
- Yeomans, J. (1992). Statistical mechanics of phase transitions. Oxford: Oxford University Press.Google Scholar
- Yu, P.L.H., Lam, K.F., & Lo, S.M. (1998). Factor analysis for ranking data. Unpublished manuscript. Department of Statistics, The University of Hong Kong.Google Scholar