Under certain assumptions an expression, in terms of item difficulties and intercorrelations, is derived for the curvilinear correlation of test score on the “ability underlying the test,” this ability being defined as the common factor of the item tetrachoric intercorrelations corrected for guessing. It is shown that this curvilinear correlation is equal to the square root of the test reliability. Numerical values for these curvilinear correlations are presented for a number of hypothetical tests, defined in terms of their item parameters. These numerical results indicate that the reliability and the curvilinear correlation will be maximized by (1) minimizing the variability of item difficulty and (2) making the level of item difficulty somewhat easier than the halfway point between a chance percentage of correct answers and 100 per cent correct answers.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Brogden, H. E. Variation in test validity with variation in the distribution of item difficulties, number of items, and degree of their intercorrelation.Psychometrika, 1946,11, 197–214.
Carroll, J. B. The effect of difficulty and chance success on correlations between items or between tests.Psychometrika, 1945,10, 1–20.
Cronbach, L. J., and Warrington, W. G. Design study for sonar pitch memory test. Bureau of Research and Service, College of Education, Univ. of Illinois, Urbana, Ill., 1951. See also Efficiency of multiple-choice tests as a function of spread of item difficulties,Psychometrika, 1952,17, 127–147.
Gulliksen, H. The relation of item difficulty and inter-item correlation to test variance and reliability.Psychometrika, 1945,10, 79–91.
Kuder, G. F., and Richardson, M. W. The theory of the estimation of test reliability.Psychometrika, 1937,2, 151–160.
Lord, F. M. A theory of test scores. Psychometric Monograph No. 7, 1952.
Pearson, K. Tables for statisticians and biometricians. London: Cambridge Univ. Press, 1924.
Plumlee, L. B. The effect of difficulty and chance success on item-test correlations and test reliability.Psychometrika, 1952,17, 69–86.
Tucker, L. R. Maximum validity of a test with equivalent items.Psychometrika, 1946,11, 1–13.
Wherry, R. J., and Gaylord, R. H. Factor pattern of test items and tests as a function of the correlation coefficient: content, difficulty, and constant error factors.Psychometrika, 1944,9, 237–244.
Yule, G. U., and Kendall, M. G. An introduction to the theory of statistics. London: Charles Griffin and Company, 1940.
About this article
Cite this article
Lord, F.M. The relation of the reliability of multiple-choice tests to the distribution of item difficulties. Psychometrika 17, 181–194 (1952). https://doi.org/10.1007/BF02288781
- Public Policy
- Test Score
- Statistical Theory
- Correct Answer
- Common Factor