Psychometrika

, Volume 66, Issue 2, pp 209–227 | Cite as

Limited information estimation and testing of Thurstonian models for paired comparison data under multiple judgment sampling

Articles

Abstract

We relate Thurstonian models for paired comparisons data to Thurstonian models for ranking data, which assign zero probabilities to all intransitive patterns. We also propose an intermediate model for paired comparisons data that assigns nonzero probabilities to all transitive patterns and to some but not all intransitive patterns.

There is a close correspondence between the multidimensional normal ogive model employed in educational testing and Thurstone's model for paired comparisons data under multiple judgment sampling with minimal identification restrictions. Alike the normal ogive model, Thurstonian models have two formulations, a factor analytic and an IRT formulation. We use the factor analytic formulation to estimate this model from the first and second order marginals of the contingency table using estimators proposed by Muthén. We also propose a statistic to assess the fit of these models to the first and second order marginals of the contingency table. This is important, as a model may reproduce well the estimated thresholds and tetrachoric correlations, yet fail to reproduce the marginals of the contingency table if the assumption of multivariate normality is incorrect.

A simulation study is performed to investigate the performance of three alternative limited information estimators which differ in the procedure used in their final stage: unweighted least squares (ULS), diagonally weighted least squares (DWLS), and full weighted least squares (WLS). Both the ULS and DWLS show a good performance with medium size problems and small samples, with a slight better performance of the ULS estimator.

Key words

UMD EWMD WMD GLS estimation LISREL categorical data analysis preference data MPLUS binary data goodness of fit 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Albert, J.H. (1992). Bayesian estimation of normal ogive item response curves using Gibbs sampling.Journal of Educational Statistics, 17, 251–269.Google Scholar
  2. Arbuckle, L., & Nugent, J.H. (1973). A general procedure for parameter estimation for the law of comparative judgment.British Journal of Mathematical and Statistical Psychology, 26, 240–260.Google Scholar
  3. Bekker, P.A., Merckens, A., & Wansbeek, T.J. (1994).Identification, equivalent models and computer algebra. San Diego: Academic Press.Google Scholar
  4. Bock, R.D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm.Psychometrika, 46, 443–459.Google Scholar
  5. Bock, R.D., & Jones, L.V. (1968).The measurement and prediction of judgment and choice. San Francisco, CA: Holden-Day.Google Scholar
  6. Böckenholt, U., & Dillon, W.R. (1997). Modeling within-subject dependencies in ordinal paired comparisons data.Psychometrika, 62, 411–434.Google Scholar
  7. Box, G.E.P. (1954). Some theorems on quadratic forms applied in the study of analysis of variance problems: I. Effect of inequality of variance in the one-way classification.Annals of Mathematical Statistics, 16, 769–771.Google Scholar
  8. Brady, H.E. (1989). Factor and ideal point analysis for interpersonally incomparable data.Psychometrika, 54, 181–202.Google Scholar
  9. Browne, M.W. (1984). Asymptotically distribution free methods for the analysis of covariance structures.British Journal of Mathematical and Statistical Psychology, 37, 62–83.Google Scholar
  10. Carroll, J.D. (1980). Models and methods for multidimensional analysis of preference choice (or other dominance) data. In E.D. Lantermann & H. Feger (Eds.),Similarity and choice. Bern, Switzerland: Hans Huber.Google Scholar
  11. Christoffersson, A. (1975). Factor analysis of dichotomized variables.Psychometrika, 40, 5–32.Google Scholar
  12. De Soete, G., & Carroll, J.D. (1983). A maximum likelihood method for fitting the wandering vector model.Psychometrika, 48, 553–566.Google Scholar
  13. De Soete, G., Carroll, J.D., & DeSarbo, W.S. (1986). The wandering ideal point model: A probabilistic multidimensional unfolding model for paired comparison data.Journal of Mathematical Psychology, 30, 28–41.Google Scholar
  14. Dijkstra, T. K. (1992). On statistical inference with parameter estimates on the boundary of parameter space.British Journal of Mathematical and Statistical Psychology, 45, 289–309.Google Scholar
  15. Hajivassiliou, V.A. (1993). Simulation estimation methods for limited dependent variable models. In G.S. Maddala, C.R. Rao, & H.D. Vinod (Eds.),Handbook of Statistics (Vol. 11). New York, NY: Elsevier Science.Google Scholar
  16. Heiser, W., & de Leeuw, J. (1981). Multidimensional mapping of preference data.Mathématiques et Sciences Humaines, 19, 39–96.Google Scholar
  17. Hubert, L., & Arabie, P. (1995). Iterative projection strategies for the least squares fitting of tree structures to proximity data.British Journal of Mathematical and Statistical Psychology, 48, 281–317.Google Scholar
  18. Küsters, U.L. (1987).Hierarchische Mittelwert- und Kovarianztrukturmodelle mit nichtmetrischen endogenen Variablen [Hierarchical mean and covariance structure models on nonmetric endogenous variables]. Heidelberg, Germany: Physica-Verlag.Google Scholar
  19. Maydeu-Olivares, A. (1999). Thurstonian modeling of ranking data via mean and covariance structure analysis.Psychometrika, 64, 325–340.Google Scholar
  20. McDonald, R.P. (1980). A simple comprehensive model for the analysis of covariance structures: Some remarks on applications.British Journal of Mathematical and Statistical Psychology, 33, 161–183.Google Scholar
  21. McDonald, R.P. (1985).Factor analysis and related methods. New York, NY: Lawrence Erlbaum.Google Scholar
  22. Muthén, B. (1978). Contributions to factor analysis of dichotomous variables.Psychometrika, 43, 551–560.Google Scholar
  23. Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators.Psychometrika, 49, 115–132.Google Scholar
  24. Muthén, B. (1993). Goodness of fit with categorical and other non normal variables. In K.A. Bollen & J.S. Long (Eds.),Testing structural equation models (pp. 205–234). Newbury Park, CA: Sage.Google Scholar
  25. Muthén, B., du Toit, S.H.C., & Spisic, D. (in press). Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes.Psychometrika.Google Scholar
  26. Muthén, B., & Hofacker, C. (1988). Testing the assumptions underlying tetrachoric correlations.Psychometrika, 53, 563–578.Google Scholar
  27. Muthén, L., & Muthén, B. (1998).Mplus. Los Angeles, CA: Muthén & Muthén.Google Scholar
  28. Muthén, B., & Satorra, A. (1995). Technical aspects of Muthén's LISCOMP approach to estimation of latent variable relations with a comprehensive measurement model.Psychometrika, 60, 489–503.Google Scholar
  29. Reboussin, B.A., & Liang, K. Y. (1998). An estimating equations approach for the LISCOMP model.Psychometrika, 63, 165–182.Google Scholar
  30. Satorra, A. (1989). Alternative test criteria in covariance structure analysis: A unified approach.Psychometrika, 54, 131–151.Google Scholar
  31. Satorra, A., & Bentler, P.M. (1994). Corrections to test statistics and standard errors in covariance structure analysis. In A. von Eye and C.C. Clogg (Eds.),Latent variable analysis: Applications to developmental research (pp. 399–419). Thousand Oaks, CA: Sage.Google Scholar
  32. Schilling, S. (1993).Advances in full-information item factor analysis using the Gibbs sampler. Unpublished doctoral dissertation, University of Chicago.Google Scholar
  33. Shapiro, A. (1985). Asymptotic distribution of test statistics in the analysis of moment structures under inequality constraints.Biometrika, 72, 133–144.Google Scholar
  34. Shapiro, A. (1988). Towards a unified theory of inequality constrained testing in multivariate analysis.International Statistical Review, 56, 49–62.Google Scholar
  35. Takane, Y. (1980). Maximum likelihood estimation in the generalized case of Thurstone's model of comparative judgment.Japanese Psychological Research, 22, 188–196.Google Scholar
  36. Takane, Y. (1987). Analysis of covariance structures and probabilistic binary choice data.Communication and Cognition, 20, 45–62.Google Scholar
  37. Takane, Y., & de Leeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables.Psychometrika, 52, 393–408.Google Scholar
  38. Teugels, J.L. (1990). Some representations of the multivariate Bernoulli and binomial distributions.Journal of Multivariate Analysis, 32, 256–268.Google Scholar
  39. Thurstone, L.L. (1927). A law of comparative judgment.Psychological Review, 79, 281–299.Google Scholar

Copyright information

© The Psychometric Society 2001

Authors and Affiliations

  1. 1.University of Illinois at Urbana-ChampaignUSA

Personalised recommendations