Skip to main content
SpringerLink
Log in

The relation of the reliability of multiple-choice tests to the distribution of item difficulties

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

Under certain assumptions an expression, in terms of item difficulties and intercorrelations, is derived for the curvilinear correlation of test score on the “ability underlying the test,” this ability being defined as the common factor of the item tetrachoric intercorrelations corrected for guessing. It is shown that this curvilinear correlation is equal to the square root of the test reliability. Numerical values for these curvilinear correlations are presented for a number of hypothetical tests, defined in terms of their item parameters. These numerical results indicate that the reliability and the curvilinear correlation will be maximized by (1) minimizing the variability of item difficulty and (2) making the level of item difficulty somewhat easier than the halfway point between a chance percentage of correct answers and 100 per cent correct answers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Brogden, H. E. Variation in test validity with variation in the distribution of item difficulties, number of items, and degree of their intercorrelation.Psychometrika, 1946,11, 197–214.

    Google Scholar 

  2. Carroll, J. B. The effect of difficulty and chance success on correlations between items or between tests.Psychometrika, 1945,10, 1–20.

    Google Scholar 

  3. Cronbach, L. J., and Warrington, W. G. Design study for sonar pitch memory test. Bureau of Research and Service, College of Education, Univ. of Illinois, Urbana, Ill., 1951. See also Efficiency of multiple-choice tests as a function of spread of item difficulties,Psychometrika, 1952,17, 127–147.

    Google Scholar 

  4. Gulliksen, H. The relation of item difficulty and inter-item correlation to test variance and reliability.Psychometrika, 1945,10, 79–91.

    Google Scholar 

  5. Kuder, G. F., and Richardson, M. W. The theory of the estimation of test reliability.Psychometrika, 1937,2, 151–160.

    Google Scholar 

  6. Lord, F. M. A theory of test scores. Psychometric Monograph No. 7, 1952.

  7. Pearson, K. Tables for statisticians and biometricians. London: Cambridge Univ. Press, 1924.

    Google Scholar 

  8. Plumlee, L. B. The effect of difficulty and chance success on item-test correlations and test reliability.Psychometrika, 1952,17, 69–86.

    Google Scholar 

  9. Tucker, L. R. Maximum validity of a test with equivalent items.Psychometrika, 1946,11, 1–13.

    Google Scholar 

  10. Wherry, R. J., and Gaylord, R. H. Factor pattern of test items and tests as a function of the correlation coefficient: content, difficulty, and constant error factors.Psychometrika, 1944,9, 237–244.

    Google Scholar 

  11. Yule, G. U., and Kendall, M. G. An introduction to the theory of statistics. London: Charles Griffin and Company, 1940.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Educational Testing Service, USA

    Frederic M. Lord

Authors
  1. Frederic M. Lord
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lord, F.M. The relation of the reliability of multiple-choice tests to the distribution of item difficulties. Psychometrika 17, 181–194 (1952). https://doi.org/10.1007/BF02288781

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02288781

Keywords

Search

Navigation