Abstract
Measures of consistency and completeness of order relations derived from test-type data are proposed. The measures are generalized to apply to incomplete data such as from tailored testing. The measures are based on consideration of the items-plus-persons by items-plus-persons matrix as an adjacency matrix in which a 1 means that the row element, whether item or person, dominated the column element. From this the number of item-item and person-person dominance relations can be derived. Indices of consistency are developed which compare the number of dominance relations in a matrix to the number that would occur if it were perfectly consistent and to the number in a random matrix; relations to other indices are noted.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Reference note
Shevell, S. K.A scalability coefficient for dominance and proximity data. (Report No. MMPP 75-6) Ann Arbor: University of Michigan, Department of Psychology, 1975.
References
Airasian, P. W., & Bart, W. M. Ordering theory: A new and useful measurement model.Educational Technology, 1973,5, 56–60.
Bart, W. M., & Krus, D. J. An ordering-theoretic method to determine hierarchies among items.Educational and Psychological Measurement, 1973,33, 291–300.
Birnbaum, A. Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.),Statistical theories of mental test scores. New York: Addison-Wesley, 1968.
Cliff, N. Complete orders from incomplete data: Interactive ordering and tailored testing.Psychological Bulletin, 1975,82, 289–302.
Coombs, C. H.A theory of data. New York: Wiley, 1964.
Cronbach, L. J. Coefficient alpha and the internal structure of tests.Psychometrika, 1951,16, 297–334.
Ducamp, A., & Falmagne, J. C. Composite measurement.Journal of Mathematical Psychology, 1969,6, 359–390.
Ferguson, G. A. On the theory of test discrimination.Psychometrika, 1949,14, 61–68.
Gulliksen, H.Theory of mental tests. New York: Wiley, 1950.
Guttman, L. The quantification of a class of attributes: A theory and method of scale construction. In P. Horst (Ed.),The prediction of personal adjustment. New York: Social Science Research Council, 1941.
Guttman, L. The basis for scalogram analysis. In S. A. Stouffer (Ed.)Measurement and prediction. New York: Wiley, 1950.
Kaiser, H. F., and Michael, W. B. Domain validity and generalizability.Educational and Psychological Measurement, 1975,35, 31–35.
Krus, D. J. A computer program for deterministic and probablistic models of order analysis.Educational and Psychological Measurement, 1973,33, 677–683.
Krus, D. J., & Bart, W. M. An ordering theoretic method of multidimensional scaling of items.Educational and Psychological Measurement, 1974,34, 525–535.
Lazarsfeld, P. F., & Henry, N. H.Latent structure analysis. Boston: Houghton Mifflin, 1968.
Loevinger, J. A systematic approach to the construction and evaluation of tests of ability.Psychological Monographs, 1947,61 (4, Whole No. 285).
Loevinger, J. The technique of homogeneous test compared with some aspects of “scale analysis” and factor analysis.Psychological Bulletin, 1948,45, 507–529.
Lord, F. M., & Novick, M. R.Statistical theories of mental test scores. New York: Addison-Wesley, 1968.
Author information
Authors and Affiliations
Additional information
The research reported here was supported by the Office of Naval Research, Contract N00014-75-C-0684, NR150-373. The author wishes to thank Dr. Thomas J. Reynolds for a number of helpful discussions of this material.
Rights and permissions
About this article
Cite this article
Cliff, N. A theory of consistency of ordering generalizable to tailored testing. Psychometrika 42, 375–399 (1977). https://doi.org/10.1007/BF02293657
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF02293657