Efficiency of multiple-choice tests as a function of spread of item difficulties

Abstract

The validity of a univocal multiple-choice test is determined for varying distributions of item difficulty and varying degrees of item precision. Validity is a function ofσ 2d +σ 2v , whereσ d measures item unreliability andσ v measures the spread of item difficulties. When this variance is very small, validity is high for one optimum cutting score, but the test gives relatively little valid information for other cutting scores. As this variance increases, eta increases up to a certain point, and then begins to decrease. Screening validity at the optimum cutting score declines as this variance increases, but the test becomes much more flexible, maintaining the same validity for a wide range of cutting scores. For items of the type ordinarily used in psychological tests, the test with uniform item difficulty gives greater over-all validity, and superior validity for most cutting scores, compared to a test with a range of item difficulties. When a multiple-choice test is intended to reject the poorestF per cent of the men tested, items should on the average be located at or above the threshold for men whose true ability is at theFth percentile.

This is a preview of subscription content, access via your institution.

References

  1. 1.

    Brogden, H. E. Variation in test validity with variation in the distribution of item difficulties, number of items, and degree of their intercorrelation.Psychometrika, 1946,11, 197–214.

    Google Scholar 

  2. 2.

    Carroll, J. B. The effect of difficulty and chance success on correlations between items or between tests.Psychometrika, 1945,10, 1–19.

    Google Scholar 

  3. 3.

    Gulliksen, H. The relation of item difficulty and interitem correlation to test variance and reliability.Psychometrika, 1945,10, 79–91.

    Google Scholar 

  4. 4.

    Lord, F. M. A theory of test scores and their relation to the trait measured.Res. Bull. 51–13, Educational Testing Service, 1951. See also A theory of test scores. Psychometric Monograph No. 7, 1952.

  5. 5.

    Richardson, M. W. The relation between the difficulty and the differential validity of a test.Psychometrika, 1936,1, 33–49.

    Google Scholar 

  6. 6.

    Tucker, L. R. Maximum validity of a test with equivalent items.Psychometrika, 1946,11, 1–13.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Additional information

This research was performed under contract Nop 536 with the Bureau of Naval Personnel, and received additional support from the Bureau of Research and Service, College of Education, University of Illinois.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Cronbach, L.J., Warrington, W.G. Efficiency of multiple-choice tests as a function of spread of item difficulties. Psychometrika 17, 127–147 (1952). https://doi.org/10.1007/BF02288778

Download citation

Keywords

  • Public Policy
  • Statistical Theory
  • Psychological Test
  • Item Difficulty
  • Valid Information