, Volume 62, Issue 3, pp 411–434 | Cite as

Modeling within-subject dependencies in ordinal paired comparison data

  • Ulf Böckenholt
  • William R. Dillon


This paper presents two probabilistic models based on the logistic and the normal distribution for the analysis of dependencies in individual paired comparison judgments. It is argued that a core assumption of latent class choice models, independence of individual decisions, may not be well-suited for the analysis of paired comparison data. Instead, the analysis and interpretation of paired comparison data may be much simplified by allowing for within-person dependencies that result from repeated evaluations of the same options in different pairs. Moreover, by relating dependencies among the individual-level responses to (in)consistencies in the judgmental process, we show that the proposed graded paired comparison models reduce to ranking models under certain conditions. Three applications are presented to illustrate the approach.

Key words

repeated measurements rankings intransitivities Thurstonian models 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Agresti, A. (1992). Analysis of ordinal paired comparison data.Applied Statistics, 41, 287–297.Google Scholar
  2. Bishop, Y. M. M., Fienberg, S. E., & Holland, P. W. (1975).Discrete multivariate analysis. Cambridge: MIT Press.Google Scholar
  3. Babington-Smith, B. (1950). Discussion of Professor Ross' paper.Journal of the Royal Statistical Society, Series B, 12, 153–162.Google Scholar
  4. Bock, R. D., & Jones, L. V. (1968).The measurement and prediction of judgment and choice. San Francisco: Holden-Day.Google Scholar
  5. Böckenholt, U. (1992). Thurstonian representation for partial ranking data.British Journal of Mathematical and Statistical Psychology, 45, 31–49.Google Scholar
  6. Böckenholt, I., & Gaul, W. (1986). Analysis of choice behavior via probabilistic ideal point and vector models.Applied Stochastic Models and Data Analysis, 2, 209–226.Google Scholar
  7. Bradley, R. A., & Terry, M. E. (1952). Rank analysis of incomplete block designs: The method of paired comparisons.Biometrika, 39, 324–345.Google Scholar
  8. Carroll, J. D., & DeSoete, G. (1991). Toward a new paradigm for the study of multiattribute choice behavior.American Psychologist, 46, 342–351.Google Scholar
  9. Cook, W. D., & Kress, M. (1985). Ordinal ranking with intensity of preference.Management Science, 31, 26–32.Google Scholar
  10. David, H. A., (1988).The method of paired comparisons. London: Griffin.Google Scholar
  11. Davidson, R. R. (1970). On extending the Bradley-Terry model to accommodate ties in paired comparison experiments.Journal of the American Statistical Association, 65, 317–328.Google Scholar
  12. DeSoete, G., & Winsberg, S. (1993). A Thurstonian pairwise choice model with univariate and multivariate spline transformations.Psychometrika, 58, 233–256.Google Scholar
  13. Dillon, W. R., Kumar, A., & de Borrero, M. (1993). Capturing individual differences in paired comparisons: An extended BTL model incorporating descriptor variables.Journal of Marketing Research, 30, 42–51.Google Scholar
  14. Edwards, A. L. & Thurstone, L. L. (1952). An interval consistency check for the method of successive intervals and the method of graded dichotomies.Psychometrika, 17, 169–180.Google Scholar
  15. Fienberg, S. E. & Larntz, K. (1976). Log-linear representation for paired and multiple comparisons models.Biometrika, 63, 245–254.Google Scholar
  16. Formann, A. K. (1989). Constrained latent class models: Some further applications.The British Journal of Mathematical and Statistical Psychology, 42, 37–54.Google Scholar
  17. Formann, A. K. (1992). Linear logistic latent class analysis for polytomous data.Journal of the American Statistical Association, 87, 476–486.Google Scholar
  18. Glenn, W. A., & David, H. A. (1960). Ties in paired comparison experiments using a modified Thurstone-Mosteller model.Biometrics, 16, 86–109.Google Scholar
  19. Goodman, L. (1979). Simple models for the analysis of association in cross-classifications having ordered categories.Journal of the American Statistical Association, 74, 537–552.Google Scholar
  20. Haberman, S. (1988). A stabilized Newton-Raphson algorithm for log-linear models for frequency tables derived by indirect observations. In C. C. Clogg (Ed.),Sociological methodology 1988 (pp. 193–212). Oxford: Blackwell.Google Scholar
  21. Halff, H. M. (1976). Choice theories for differentially comparable alternatives.Journal of Mathematical Psychology, 14, 244–246.Google Scholar
  22. Joe, H. (1988). Majorization, entropy and paired comparisons.Annals of Statistics, 16, 915–925.Google Scholar
  23. Kendall, M. G. (1962). Ranks and measures.Biometrika, 49, 133–137.Google Scholar
  24. Kendall, M. G. (1975).Rank correlation methods. London: Griffin.Google Scholar
  25. Kroeger, K. (1992). Unpublished data set. University of Illinois, Urbana-Champaign.Google Scholar
  26. Luce, R. D. (1959).Individual choice behavior. New York: Wiley.Google Scholar
  27. Mellers, B. A., & Biagini, K. (1994). Similarity and choice.Psychological Review, 101, 505–518.Google Scholar
  28. Morrison, H. W. (1963). Testable conditions for triads of paired comparison choices.Psychometrika, 28, 369–390.Google Scholar
  29. Park, C. W. (1978). A seven-point scale and a decision-maker's simplifying choice strategy: An operationalized satisficing-plus model.Organizational Behavior and Human Performance, 21, 252–271.Google Scholar
  30. Pendergrass, P. N., & Bradley, R. A. (1960). Ranking in triple comparison. In I. Olkin (Ed.),Contributions to probability and statistics (pp. 331–351), Palo Alto: Stanford University Press.Google Scholar
  31. Rumelhart, D. L., & Greeno, J. G. (1971). Similarity between stimuli: An experimental test of the Luce and Restle choice models.Journal of Mathematical Psychology, 8, 370–381.Google Scholar
  32. Sjöberg, L. (1967). Successive categories scaling of paired comparisons.Psychometrika, 32, 297–308.Google Scholar
  33. Stevens, S. S. (1986)Psychophysics: Introduction to its perceptual, neural, and social prospects. New Brunswick: Transaction Books.Google Scholar
  34. Takane, Y. (1980). Maximum likelihood estimation in the generalized cases of Thurstone's law of comparative judgment.Japanese Psychological Research, 22, 188–196.Google Scholar
  35. Takane, Y. (1987). Analysis of covariance structures and probabilistic binary choice data.Cognition and Communication, 20, 45–62.Google Scholar
  36. Takane, Y. & deLeeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables.Psychometrika, 52, 393–408.Google Scholar
  37. Thurstone, L. L. (1927). A law of comparative judgment.Psychological Review, 34, 273–286.Google Scholar
  38. Torgerson, W. S. (1958)Theory and method of scaling, New York: John Wiley & Sons.Google Scholar
  39. Tutz, G. (1986). Bradley-Terry-Luce models with an ordered response.Journal of Mathematical Psychology, 30, 306–316.Google Scholar
  40. Tversky, A. (1969). Intransitivity of preference.Psychological Review, 76, 31–48.Google Scholar
  41. Vermunt, J. K. (1993).Lem: Log-linear and event history analysis with missing data using the EM algorithm. Unpublished manuscript, Tilburg University.Google Scholar
  42. van Acker, P. (1990). Transitivity revisited.Annals of Operations Research, 23, 1–35.Google Scholar
  43. Yakowitz, S. J., & Spragins (1968). On the identifiability of finite mixtures.Annals of Mathematical Statistics, 39, 209–214.Google Scholar

Copyright information

© The Psychometric Society 1997

Authors and Affiliations

  • Ulf Böckenholt
    • 1
  • William R. Dillon
    • 2
  1. 1.Department of PsychologyUniversity of IllinoisChampaign
  2. 2.School of BusinessSouthern Methodist UniversityDallas

Personalised recommendations