Skip to main content
Log in

The concept of test and item reliability in relation to factor pattern

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

It is shown that approaches other than the internal consistency method of estimating test reliability are either less satisfactory or lead to the same general results. The commonly attendant assumption of a single factor throughout the test items is challenged, however. The consideration of a test made up ofK sub-tests each composed of a different orthogonal factor disclosed that the assumption of a single factor produced an erroneous estimate of reliability with a ratio of (nK)/(n−1) to the correct estimate. Special difficulties arising from this error in application of current techniques to short tests or to test batteries are discussed. Application of this same multi-factor concept to item-analysis discloses similar difficulties in that field. The item-test coefficient approaches √1/K as an upper limit rather than 1.00 and approaches √1/n as a lower limit rather than .00. This latter finding accounts for an over-estimation error in the Kuder-Richardson formula (8). A new method of isolating sub-tests based upon the item-test coefficient is proposed and tentatively outlined. Either this new method or a complete factor analysis is regarded as the only proper approach to the problem of test reliability, and the item-sub-est coefficient is similarly recommended as the proper approach for item analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Brown, W. Some experimental results in the correlation of mental abilities.Brit. J. Psychol., 1909–1910,3, 296–322.

    Google Scholar 

  2. Edgerton, H. A., and Kolbe, L. E. The method of minimum variation for the combination of criteria.Psychometrika, 1936,1, 183–187.

    Google Scholar 

  3. Edgerton, H. A., and Thomson, K. F. Test scores examined with the lexis ratio.Psychometrika, 1942,7, 281–288.

    Google Scholar 

  4. Horst, P. Obtaining a composite measure from a number of different measures of the same attribute.Psychometrika, 1936,1, 53–60.

    Google Scholar 

  5. Hotelling, H. The most predictable criterion.J. educ. Psychol., 1935,26, 139–142.

    Google Scholar 

  6. Hoyt, C. Test reliability estimated by analysis of variance.Psychometrika, 1941,6, 153–160.

    Google Scholar 

  7. Jackson, R. W. B. Reliability of mental tests.Brit. J. Psychol., 1938–39,29, 267–287.

    Google Scholar 

  8. Kelley, T. L. The reliability coefficient.Psychometrika, 1942,7, 75–83.

    Google Scholar 

  9. Kuder, G. F., and Richardson, M. W. The theory of the estimation of reliability.Psychometrika, 1937,2, 151–160.

    Google Scholar 

  10. Lentz, T. F., and Whitmer, E. F. Item synonymization: a method for determining the total meaning of pencil-paper reactions.Psychometrika, 1941,6, 131–139.

    Google Scholar 

  11. Mosier, C. A note on item analysis and the criterion of internal consistency,Psychometrika, 1936,1, 275–282.

    Google Scholar 

  12. Paulsen, G. B. A coefficient of trait variability.Psychol. Bull., 1931,28, 218.

    Google Scholar 

  13. Richardson, M. W. Note on the rationale of item analysis,Psychometrika, 1936,1, 69–76.

    Google Scholar 

  14. Royer, E. B. Some recent developments in test construction.Proc. Okla. Acad. Sci., 1936,16, 107–109.

    Google Scholar 

  15. Spearman, C. Correlation calculated from faulty data.Brit. J. Psychol., 1909–1910,3, 271–295.

    Google Scholar 

  16. Thompson, G. A. Weighting for battery reliability and prediction.Brit. J. Psychol., 1939–40,30, 357–366.

    Google Scholar 

  17. Thouless, R. H. Test unreliability and functional fluctuation.Brit. J. Psychol., 1935–36,26, 325–343.

    Google Scholar 

  18. Woodrow, H. Quotidian variability.Psychol. Rev., 1932,32, 245–256.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wherry, R.J., Gaylord, R.H. The concept of test and item reliability in relation to factor pattern. Psychometrika 8, 247–264 (1943). https://doi.org/10.1007/BF02288707

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02288707

Keywords

Navigation