Abstract
For methods using statistical optimization to estimate lower bounds to test-score reliability, we investigated the degree to which they overestimate true reliability. Optimization methods do not only exploit real relationships between items but also tend to capitalize on sampling error and do this more strongly as sample size is smaller and tests are longer. The optimization methods were Guttman’s λ 4, λ 5, and λ 6 and the greatest lower bound to the reliability (GLB). Method λ 2 was used as benchmark. We used a simulation study to investigate the relation of the methods’ discrepancy, bias, and sampling error with the proportion of simulated data sets in which each method overestimated true test-score reliability. Method λ 4 and the GLB often overestimated test-score reliability. When sample size exceeded 250 observations, methods λ 2, λ 5, and λ 6 provided reasonable to good statistical results, in particular when data were two-dimensional. Benchmark method λ 2 produced the best results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
P.M. Bentler, J.A. Woodward, Inequalities among lower bounds to reliability: with applications to test construction and factor analysis. Psychometrika 45, 249–267 (1980). doi:10.1007/BF02294079
T. Benton, An empirical assessment of Guttman’s lambda 4 reliability coefficient, in Quantitative Psychology Research: The 78th Annual Meeting of the Psychometric Society, ed. by R.E. Millsap, D.M. Bolt, L.A. van der Ark, W.-C. Wang (Springer, New York, 2015), pp. 301–310
K.A. Bollen, Structural Equations with Latent Variables (Wiley, New York, 1989)
L.J. Cronbach, Coefficient alpha and the internal structure of tests. Psychometrika 16, 297–334 (1951). doi:10.1007/BF02310555
R.J. De Ayala, The Theory and Practice of Item Response Theory (Guilford Press, New York, 2009)
L. Guttman, A basis for analyzing test-retest reliability. Psychometrika 10, 255–282 (1945). doi:10.1007/BF02288892
P.H. Jackson, C.C. Agunwamba, Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: I. Algebraic lower bounds. Psychometrika 42, 567–578 (1977). doi:10.1007/BF02295979
G.F. Kuder, M.W. Richardson, The theory of estimation of test reliability. Psychometrika 2, 151–160 (1937). doi:10.1007/BF02288391
F.M. Lord, M.R. Novick, Statistical Theories of Mental Test Scores (Addison-Wesley, Reading, 1968)
R.P. McDonald, Test Theory: A Unified Treatment (Erlbaum, Mahwah, 1999)
P.R. Oosterwijk, L.A. Van der Ark, K. Sijtsma, Numerical differences between Guttman’s reliability coefficients and the GLB, in Quantitative Psychology Research: The 80th Annual Meeting of the Psychometric Society, Beijing, 2015, ed. by L.A. van der Ark, D.M. Bolt, W.-C. Wang, J.A. Douglas, M. Wiberg (Springer, New York, 2016), pp. 155–172
W. Revelle, psych: Procedures for personality and psychological research Version 1.5.8 [computer software] (2015). Evanston, IL. Retrieved from https://cran.r-project.org/web/packages/psych/index.html
A. Shapiro, J.M.F. Ten Berge, The asymptotic bias of minimum trace factor analysis, with applications to the greatest lower bound to reliability. Psychometrika 65, 413–425. doi:10.1007/BF02296154 2000
K. Sijtsma, On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika 74, 107–120 (2009). doi:10.1007/s11336-008-9101-0
K. Sijtsma, L.A. Van der Ark, Conceptions of reliability revisited and practical recommendations. Nurs. Res. 64, 128–136 (2015). doi:10.1097/NNR.0000000000000077
J.M.F. Ten Berge, G. Sočan, The greatest lower bound to the reliability of a test and the hypothesis of unidimensionality. Psychometrika 69, 613–625 (2004). doi:10.1007/BF02289858
J.M.F. Ten Berge, F.E. Zegers, A series of lower bounds to the reliability of a test. Psychometrika 43, 575–579 (1978). doi:10.1007/BF02293815
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Oosterwijk, P.R., van der Ark, L.A., Sijtsma, K. (2017). Overestimation of Reliability by Guttman’s λ 4, λ 5, and λ 6 and the Greatest Lower Bound. In: van der Ark, L.A., Wiberg, M., Culpepper, S.A., Douglas, J.A., Wang, WC. (eds) Quantitative Psychology. IMPS 2016. Springer Proceedings in Mathematics & Statistics, vol 196. Springer, Cham. https://doi.org/10.1007/978-3-319-56294-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-56294-0_15
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-56293-3
Online ISBN: 978-3-319-56294-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)