Ordinal test fidelity estimated by an item sampling model
 Norman Cliff,
 John R. Donoghue
 … show all 2 hide
Rent the article at a discount
Rent now* Final gross prices may vary according to local VAT.
Get AccessAbstract
A test theory using only ordinal assumptions is presented. It is based on the idea that the test items are a sample from a universe of items. The sum across items of the ordinal relations for a pair of persons on the universe items is analogous to a true score. Using concepts from ordinal multiple regression, it is possible to estimate the tau correlations of test items with the universe order from the taus among the test items. These in turn permit the estimation of the tau of total score with the universe. It is also possible to estimate the odds that the direction of a given observed score difference is the same as that of the true score difference. The estimates of the correlations between items and universe and between total score and universe are found to agree well with the actual values in both real and artificial data.
 Birnbaum, A. Some latent trait models and their uses in inferring an examinee's ability. In: Lord, F. M., Novick, M. R. eds. (1968) Statistical theories of mental test scores. AddisonWesley, Reading, MA, pp. 397472
 Charlin, V. L. (1987).Implementation and inferential issues of a multiple regression system with ordinal variables. Unpublished doctoral dissertation, University of Southern California.
 Cliff, N. (1977) A theory of consistency of ordering generalizable to tailored testing. Psychometrika 42: pp. 375401
 Cliff, N. (1979) Test theory without true scores?. Psychometrika 44: pp. 373393
 Cliff, N. (1986) Ordinal analogues of multiple regression. 1986 Proceedings of the Social Statistics Section of the American Statistical Association. American Statistical Association, Washington, DC, pp. 320324
 Cliff, N. (1989a). “Multiple regression” with ordinal variables. Unpublished manuscript.
 Cliff, N. (1989) Ordinal consistency and ordinal true scores. Psychometrika 54: pp. 7591
 Cliff, N., & Charlin, V. L. (In press). Variances and covariances of Kendall's tau and their estimation.Multivariate Behavior Research.
 Donoghue, J. R. (1990).Properties of ability estimates derived from logistic item response theory and ordinal item sampling test theory. Unpublished doctoral dissertation, University of Southern California.
 Donoghue, J. R., & Cliff, N. (In press). An investigation of ordinal true score test theory.Applied Psychological Measurement.
 Douglas, G. A., Wright, B. D. (1986) The twocategory model for objective measurement. University of Chicago, Department of Education, MESA Psychometric Laboratory, Chicago
 Gulliksen, H. (1950) Theory of mental tests. Wiley, New York
 Guttman, L. (1944) A basis for scaling qualitative data. American Sociological Review 9: pp. 139150
 Guttman, L. The basis for scalogram analysis. In: Stouffer, S.A. eds. (1950) Measurement and Prediction. Princeton University, Princeton, NJ, pp. 6090
 Hawkes, R. J. (1971) The multivariate analysis of ordinal data. American Journal of Sociology 76: pp. 908926
 Hettmansperger, T. P. (1984) Statistical inference based on ranks. Wiley, New York
 Holland, P. W. (1990) The Dutch identity: A new tool for understanding item response theory. Psychometrika 55: pp. 518
 Kendall, M. G. (1970) Rank correlation methods. Hafner, New York
 Loevinger, J. A. (1947). A systematic approach to the construction and evaluation of tests of ability.Psychological Monographs, 61, (Whole No. 285).
 Loevinger, J. A. (1948) A technique of homogeneous test evaluation compared with some aspects of “scale analysis”. Psychological Bulletin 45: pp. 507529
 Lord, F. M. (1968) An analysis of the verbal Scholastic Aptitude Test using Birnbaum's threeparameter logistic model. Educational and Psychological Measurement 28: pp. 9891020
 Lord, F. M. (1975) The “ability” scale in item response theory. Psychometrika 40: pp. 205217
 Lord, F. M., Novick, M. R. (1968) Statistical theories of mental test scores. AddisonWesley, Reading, MA
 Mislevy, R. J. (1987) Recent developments in item response theory with implications for teacher certification. Review of Educational Research 14: pp. 239275
 Mokken, R. J. (1971) A theory and procedure of scale analysis. Mouton & Co, Hawthorne, NY
 Mokken, R. J., Lewis, C. (1982) A nonparametric approach to the analysis of dichotomous item responses. Applied Psychological Measurement 6: pp. 417430
 Reckase, M. D. (1985) The difficulty of test items that measure more than one ability. Applied Psychological Measurement 9: pp. 401412
 Reynolds, T. J., Sutrick, K. H. (1986) Assessing the correspondence of one or more vectors to a symmetric matrix. Psychometrika 51: pp. 101112
 Roskam, E. E., Jansen, P. G. W. A new derivation of the Rasch Model. In: deGreef, E., Buggenhaut, J. eds. (1984) Trends in mathematical psychology. Elsevier, Amsterdam, pp. 293307
 Samejima, F. (1979) A new family of models for the multiplechoice item. University of Tennessee, Knoxville, TN
 Schulman, R. S., Haden, R. L. (1975) A test theory for ordinal measurements. Psychometrika 40: pp. 329340
 Smith, R. B. (1974) Continuities in ordinal path analysis. Social Forces 53: pp. 200229
 Somers, R. H. (1974) Analysis of partial rank correlation measures based on the productmoment model: Part One. Social Forces 53: pp. 229246
 Stout, W. (1987) A nonparametric approach for assessing latent dimensionality. Psychometrika 52: pp. 589617
 Stout, W. (1988) A nonparametric multidimensional IRT approach with applications to ability estimation and test bias. University of Illinois, Champaign, IL
 Stout, W. (1990) A new itemresponse theory modeling approach with applications to unidimensionality assessment and ability estimation. Psychometrika 55: pp. 293325
 Thissen, D., Steinberg, L. (1984) A response model for multiple choice items. Psychometrika 47: pp. 501519
 Thissen, D., Wainer, H. (1982) Some standard errors in item response theory. Psychometrika 45: pp. 397412
 Yen, W. (1986) The choice of scale for educational measurement: An IRT perspective. Journal of Educational Measurement 23: pp. 299325
 Title
 Ordinal test fidelity estimated by an item sampling model
 Journal

Psychometrika
Volume 57, Issue 2 , pp 217236
 Cover Date
 19920601
 DOI
 10.1007/BF02294506
 Print ISSN
 00333123
 Online ISSN
 18600980
 Publisher
 SpringerVerlag
 Additional Links
 Topics
 Keywords

 test theory
 item sampling
 ordinal methods
 item universe
 Industry Sectors
 Authors

 Norman Cliff ^{(1)}
 John R. Donoghue ^{(2)}
 Author Affiliations

 1. Department of Psychology, MC1061, University of Southern California, 90089, Los Angeles, CA
 2. Educational Testing Service, USA