Abstract
The validity of a univocal multiple-choice test is determined for varying distributions of item difficulty and varying degrees of item precision. Validity is a function ofσ 2d +σ 2v , whereσ d measures item unreliability andσ v measures the spread of item difficulties. When this variance is very small, validity is high for one optimum cutting score, but the test gives relatively little valid information for other cutting scores. As this variance increases, eta increases up to a certain point, and then begins to decrease. Screening validity at the optimum cutting score declines as this variance increases, but the test becomes much more flexible, maintaining the same validity for a wide range of cutting scores. For items of the type ordinarily used in psychological tests, the test with uniform item difficulty gives greater over-all validity, and superior validity for most cutting scores, compared to a test with a range of item difficulties. When a multiple-choice test is intended to reject the poorestF per cent of the men tested, items should on the average be located at or above the threshold for men whose true ability is at theFth percentile.
Similar content being viewed by others
References
Brogden, H. E. Variation in test validity with variation in the distribution of item difficulties, number of items, and degree of their intercorrelation.Psychometrika, 1946,11, 197–214.
Carroll, J. B. The effect of difficulty and chance success on correlations between items or between tests.Psychometrika, 1945,10, 1–19.
Gulliksen, H. The relation of item difficulty and interitem correlation to test variance and reliability.Psychometrika, 1945,10, 79–91.
Lord, F. M. A theory of test scores and their relation to the trait measured.Res. Bull. 51–13, Educational Testing Service, 1951. See also A theory of test scores. Psychometric Monograph No. 7, 1952.
Richardson, M. W. The relation between the difficulty and the differential validity of a test.Psychometrika, 1936,1, 33–49.
Tucker, L. R. Maximum validity of a test with equivalent items.Psychometrika, 1946,11, 1–13.
Author information
Authors and Affiliations
Additional information
This research was performed under contract Nop 536 with the Bureau of Naval Personnel, and received additional support from the Bureau of Research and Service, College of Education, University of Illinois.
Rights and permissions
About this article
Cite this article
Cronbach, L.J., Warrington, W.G. Efficiency of multiple-choice tests as a function of spread of item difficulties. Psychometrika 17, 127–147 (1952). https://doi.org/10.1007/BF02288778
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02288778