Efficiency of multiple-choice tests as a function of spread of item difficulties
The validity of a univocal multiple-choice test is determined for varying distributions of item difficulty and varying degrees of item precision. Validity is a function ofσd2 +σv2, whereσd measures item unreliability andσv measures the spread of item difficulties. When this variance is very small, validity is high for one optimum cutting score, but the test gives relatively little valid information for other cutting scores. As this variance increases, eta increases up to a certain point, and then begins to decrease. Screening validity at the optimum cutting score declines as this variance increases, but the test becomes much more flexible, maintaining the same validity for a wide range of cutting scores. For items of the type ordinarily used in psychological tests, the test with uniform item difficulty gives greater over-all validity, and superior validity for most cutting scores, compared to a test with a range of item difficulties. When a multiple-choice test is intended to reject the poorestF per cent of the men tested, items should on the average be located at or above the threshold for men whose true ability is at theFth percentile.
Unable to display preview. Download preview PDF.
- 1.Brogden, H. E. Variation in test validity with variation in the distribution of item difficulties, number of items, and degree of their intercorrelation.Psychometrika, 1946,11, 197–214.Google Scholar
- 2.Carroll, J. B. The effect of difficulty and chance success on correlations between items or between tests.Psychometrika, 1945,10, 1–19.Google Scholar
- 3.Gulliksen, H. The relation of item difficulty and interitem correlation to test variance and reliability.Psychometrika, 1945,10, 79–91.Google Scholar
- 4.Lord, F. M. A theory of test scores and their relation to the trait measured.Res. Bull. 51–13, Educational Testing Service, 1951. See also A theory of test scores. Psychometric Monograph No. 7, 1952.Google Scholar
- 5.Richardson, M. W. The relation between the difficulty and the differential validity of a test.Psychometrika, 1936,1, 33–49.Google Scholar
- 6.Tucker, L. R. Maximum validity of a test with equivalent items.Psychometrika, 1946,11, 1–13.Google Scholar