, Volume 54, Issue 1, pp 75–91 | Cite as

Ordinal consistency and ordinal true scores

  • Norman Cliff


This paper argues that test data are ordinal, that latent trait scores are only determined ordinally, and that test data are used largely for ordinal purposes. Therefore it is desirable to develop a test theory based only on ordinal assumptions. A set of ordinal assumptions is presented, including an ordinal version of local independence. From these assumptions it is first shown that the gamma-correlation between two tests is the product of their gamma-correlations with the true latent order. The theory is generalized to allow for heterogeneous tests by defining a weighted average local independence. The tau-correlations between total score and the latent order can be found in both homogeneous and heterogeneous cases, and a system of differential item weighting to maximize the tau-correlation between weighted items and the latent order is provided. Thus a purely ordinal test theory seems possible.

Key words

test theory ordinal regression local independence reliability 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Andersen, E. B. (1973). Conditional inference for multiple choice questionnaires.British Journal of Mathematical and Statistical Psychology, 26, 42–54.Google Scholar
  2. Bartholomew, D. J. (1984). The foundations of factor analysis.Biometrika, 71, 221–232.Google Scholar
  3. Bartholomew, D. J. (1985). Foundations of factor analysis: Some practical implications.British Journal of Mathematical and Statistical Psychology, 38, 127–137.Google Scholar
  4. Birnbaum, A. (1968). Some latent trait models and their uses in inferring an examinee's ability. In F. M. Lord & M. R. Novick,Statistical theories of mental test scores (pp. 397–472). Reading, MA: Addison, Wesley.Google Scholar
  5. Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm.Psychometrika, 46, 443–459.Google Scholar
  6. Cliff, N. (1977). A theory of consistency of ordering generalizable to tailored testing.Psychometrika, 42, 373–393.Google Scholar
  7. Cliff, N. (1979). Test theory without true scores?Psychometrika, 44, 373–393.Google Scholar
  8. Douglas, G. A., & Wright, B. D. (1968).The two category model for objective measurment. Chicago: University of Chicago, Department of Education Multilith report.Google Scholar
  9. Gans, L. P., & Robertson, C. A. (1981). Distributions of Goodman and Kruskal's gamma and Spearman's rho in 2 × 2 tables for small and moderate sample sizes.Journal of the American Statistical Association, 76, 942–946.Google Scholar
  10. Goodman, L., & Kruskal, W. B. (1955). Measures of association for crossclassifications.Journal of the American Statistical Association, 49, 732–764.Google Scholar
  11. Green, B. F. (1976). Comments on computers in testing. In C. L. Clark (Ed.),Proceedings of the First Conference on Computerized Adaptive Testing (U.S. Civil Service Commission Personal & Development Center PS 75-6). Washington, DC: U.S. Government Printing Office. (Superintendent of Documents Stock No. 006-00940-9)Google Scholar
  12. Guttman, L. (1950). The basis for scalogram analysis. In S. A. Stouffer (Ed.),Measurement and prediction. Princeton, NJ: Princeton University.Google Scholar
  13. Guttman, L. (1971). Measurement as structural theory.Psychometrika, 36, 329–347.Google Scholar
  14. Hubert, L. J. (1985). Combinatorial data analysis: Association and partial association.Psychometrika, 59, 449–467.Google Scholar
  15. Humphreys, L. G. (1956). The normal curve and the attenuation paradox in test theory.Psychological Bulletin, 53, 472–476.Google Scholar
  16. Keats, J. A. (1951).A statistical theory of mental test scores. Melbourne: Australian Council for Educational Research.Google Scholar
  17. Kendall, M. G. (1970).Rank correlation methods (3rd ed.). New York: Hafner.Google Scholar
  18. Krantz, D. H., Luce, R. D., Suppes, P., & Tversky, A. (1971).Foundations of measurement. New York: Academic Press.Google Scholar
  19. Lazarsfeld, P. F. (1950a). The logical and mathematical foundation of latent structural analysis. In S. A. Stouffer (Ed.),Measurement and prediction (pp. 362–412). Princeton, NJ: Princeton University.Google Scholar
  20. Lazarsfeld, P. F. (1950b). The interpretation and computation of some latent structures. In S. A. Stouffer (Ed.),Measurement and prediction (pp. 413–473). Princeton, NJ: Princeton University.Google Scholar
  21. Lazarsfeld, P. F., & Henry, N. W. (1968).Latent structure analysis. New York: Houghton Mifflin.Google Scholar
  22. Levine, M. V. (1970). Transformations that render curves parallel.Journal of Mathematical Psychology, 7, 410–443.Google Scholar
  23. Loevinger, J. A. (1947). A systematic approach to the construction and evaluation of tests of ability.Psychological Monographs, 61, (4, Whole No. 285).Google Scholar
  24. Loevinger, J. A. (1948). A technique of homogeneous test evaluation compared with some aspects of “scale analysis”.Psychological Bulletin, 45, 507–529.Google Scholar
  25. Lord, F. M., & Novick, M. R. (1968).Statistical theories of mental test scores. Reading, MA: Addison, Wesley.Google Scholar
  26. Luce, R. D., & Tukey, J. W. (1964). Simultaneous conjoint measurement: A new type of fundamental measurement.Journal of Mathematical Psychology, 1, 1–27.Google Scholar
  27. Mokken, R. J. (1971).A theory and procedure of scale analysis. Hawthorne, NY: Mouton & Co.Google Scholar
  28. Mokken, R. J., & Lewis, C. (1982). A nonparametric approach to the analysis of dichotomous item responses.Applied Psychological Measurement, 6, 417–430.Google Scholar
  29. Mokken, R. J., Lewis, C., & Sÿtsma, K. (1986). Rejoinder to: “The Mokken Scale: A critical discussion.”Applied Psychological Measurement, 10, 229–285.Google Scholar
  30. Novick, M. R., & Lewis, C. (1967). Coefficient alpha and the reliability of composite measurements.Psychometrika, 32, 1–4.Google Scholar
  31. Puri, M. L., & Sen, P. K. (1985),Nonparametric methods in general linear models. New York: Wiley.Google Scholar
  32. Rasch, G. (1980).Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press. (Reprint, with a Foreword and Afterword by Benjamin D. Wright)Google Scholar
  33. Reynolds, T. J., & Suttrick, K. H. (1986). Assessing the correspondence of a vector to a symmetric matrix using ordinal regression.Psychometrika 51, 101–112.Google Scholar
  34. Roskam, E. E., & Jansen, P. G. W. (1984). A new derivation of the Rasch model. In E. Degreef & J. van Buggenhaut (Eds.),Trends in mathematical psychology (pp. 293–307). Amsterdam: Elsevier Science Publishers.Google Scholar
  35. Roskam, E. E., van den Wollenberg, A. L., & Jansen, P. G. W. (1986). The Mokken Scale: A critical discussion.Applied Psychological Measurement, 10, 267–277.Google Scholar
  36. Schulman, R. S. (1976). Correlation and prediction in ordinal test theory.Psychometrika, 41, 329–340.Google Scholar
  37. Schulman, R. S., & Haden, R. L. (1975). A test theory for ordinal measurements.Psychometrika, 40, 455–472.Google Scholar
  38. Shepard, R. N. (1962). The analysis of proximities: Multidimensional scaling with an unknown distance function.Psychometrika, 27, 219–246.Google Scholar
  39. Smith, R. B. (1974). Continuities in ordinal path analysis.Social forces, 53, 229–246.Google Scholar
  40. Somers, R. H. (1962). A new asymmetric measure of association for ordinal variables.American Sociological Review, 27, 799–811.Google Scholar

Copyright information

© The Psychometric Society 1989

Authors and Affiliations

  • Norman Cliff
    • 1
  1. 1.Department of Psychology, MC 1061University of Southern CaliforniaLos Angeles

Personalised recommendations