The relation of the reliability of multiple-choice tests to the distribution of item difficulties
Under certain assumptions an expression, in terms of item difficulties and intercorrelations, is derived for the curvilinear correlation of test score on the “ability underlying the test,” this ability being defined as the common factor of the item tetrachoric intercorrelations corrected for guessing. It is shown that this curvilinear correlation is equal to the square root of the test reliability. Numerical values for these curvilinear correlations are presented for a number of hypothetical tests, defined in terms of their item parameters. These numerical results indicate that the reliability and the curvilinear correlation will be maximized by (1) minimizing the variability of item difficulty and (2) making the level of item difficulty somewhat easier than the halfway point between a chance percentage of correct answers and 100 per cent correct answers.
Unable to display preview. Download preview PDF.
- 1.Brogden, H. E. Variation in test validity with variation in the distribution of item difficulties, number of items, and degree of their intercorrelation.Psychometrika, 1946,11, 197–214.Google Scholar
- 2.Carroll, J. B. The effect of difficulty and chance success on correlations between items or between tests.Psychometrika, 1945,10, 1–20.Google Scholar
- 3.Cronbach, L. J., and Warrington, W. G. Design study for sonar pitch memory test. Bureau of Research and Service, College of Education, Univ. of Illinois, Urbana, Ill., 1951. See also Efficiency of multiple-choice tests as a function of spread of item difficulties,Psychometrika, 1952,17, 127–147.Google Scholar
- 4.Gulliksen, H. The relation of item difficulty and inter-item correlation to test variance and reliability.Psychometrika, 1945,10, 79–91.Google Scholar
- 5.Kuder, G. F., and Richardson, M. W. The theory of the estimation of test reliability.Psychometrika, 1937,2, 151–160.Google Scholar
- 6.Lord, F. M. A theory of test scores. Psychometric Monograph No. 7, 1952.Google Scholar
- 7.Pearson, K. Tables for statisticians and biometricians. London: Cambridge Univ. Press, 1924.Google Scholar
- 8.Plumlee, L. B. The effect of difficulty and chance success on item-test correlations and test reliability.Psychometrika, 1952,17, 69–86.Google Scholar
- 9.Tucker, L. R. Maximum validity of a test with equivalent items.Psychometrika, 1946,11, 1–13.Google Scholar
- 10.Wherry, R. J., and Gaylord, R. H. Factor pattern of test items and tests as a function of the correlation coefficient: content, difficulty, and constant error factors.Psychometrika, 1944,9, 237–244.Google Scholar
- 11.Yule, G. U., and Kendall, M. G. An introduction to the theory of statistics. London: Charles Griffin and Company, 1940.Google Scholar