Abstract
Sijtsma and Pfadt (2021) published a thought-provoking article on coefficient alpha. I make the following arguments against their work. 1) Kuder and Richardson (1937) deserve more credit for coefficient alpha than Cronbach (1951). 2) We should distinguish between the definition of reliability and its meaning. 3) We should be wary of overfitting in the use of FA reliability. 4) Our primary concern is to obtain accurate reliability estimates rather than conservative estimates. 5) Several reliability estimators, such as \(\lambda _{2}\), \(\mu _{2}\), congeneric reliability and the Gilmer-Feldt coefficient are more accurate than coefficient alpha. 6) The name omega should not be used to refer to a specific reliability estimator.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Allen, M. P. (1973). Construction of composite measures by the canonical-factor-regression method. Sociological Methodology, 5, 51–78. https://doi.org/10.2307/270832.
Bentler, P. M. (1968). Alpha-maximized factor analysis (alphamax): Its relation to alpha and canonical factor analysis. Psychometrika, 33(3), 335–345. https://doi.org/10.1007/BF02289328.
Cho, E. (2016). Making reliability reliable: A systematic approach to reliability coefficients. Organizational Research Methods, 19(4), 651–682. https://doi.org/10.1177/1094428116656239.
Cho, E., & Chun, S. (2018). Fixing a broken clock: A historical review of the originators of reliability coefficients including Cronbach’s alpha. Survey Research, 19(2), 23–54. https://doi.org/10.20997/sr.19.2.4
Cho, E., & Kim, S. (2015). Cronbach’s coefficient alpha: Well known but poorly understood. Organizational Research Methods, 18(2), 207–230. https://doi.org/10.1177/1094428114555994
Cronbach, L. J. (1943). On estimates of test reliability. Journal of Educational Psychology, 34(8), 485–494. https://doi.org/10.1037/h0058608.
Cronbach, L. J. (1947). Test “reliability”: Its meaning and determination. Psychometrika, 12(1), 1–16. https://doi.org/10.1007/BF02289289
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555.
Cronbach, L. J. (1978). Citation classics. Current Contents, 13, 263.
Cronbach, L. J., & Shavelson, R. J. (2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64(3), 391–418. https://doi.org/10.1177/0013164404266386.
Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. https://doi.org/10.1111/bjop.12046.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272.
Feldt, L. S., & Brennan, R. L. (1989). Reliability. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 105–146). American Council on Education and Macmillan.
Feldt, L. S., & Charter, R. A. (2003). Estimation of internal consistency reliability when test parts vary in effective length. Measurement and Evaluation in Counseling and Development, 36(1), 23–27. https://doi.org/10.1080/07481756.2003.12069077.
Ferguson, G. A. (1951). A note on the Kuder-Richardson formula. Educational and Psychological Measurement, 11(4–1), 612–615. https://doi.org/10.1177/001316445101100409.
Gilmer, J. S., & Feldt, L. S. (1983). Reliability estimation for a test with parts of unknown lengths. Psychometrika, 48(1), 99–111. https://doi.org/10.1007/BF02314679.
Graham, J. M. (2006). Congeneric and (essentially) tau-equivalent estimates of score reliability what they are and how to use them. Educational and Psychological Measurement, 66(6), 930–944. https://doi.org/10.1177/0013164406288165.
Guttman, L. (1945). A basis for analyzing test-retest reliability. Psychometrika, 10(4), 255–282. https://doi.org/10.1007/BF02288892.
Hägglund, G. (2001). Milestones in the history of factor analysis. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.), Structural equation modeling: Present and future–A festschrift in honor of Karl Jöreskog (pp. 11–38). Scientific Software International.
Hancock, G., & Mueller, R. O. (2001). Rethinking construct reliability within latent variable systems. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.), Structural equation modeling: Present and future–A festschrift in honor of Karl Jöreskog (pp. 195–216). Scientific Software International.
Harzing, A.-W. (2007). Publish or Perish. https://harzing.com/resources/publish-or-perish
Hayes, A. F., & Coutts, J. J. (2020). Use omega rather than Cronbach’s alpha for estimating reliability. But.... Communication Methods and Measures, 14(1), 1–24. https://doi.org/10.1080/19312458.2020.1718629
Heise, D. R., & Bohrnstedt, G. W. (1970). Validity, invalidity, and reliability. Sociological Methodology, 2, 104–129. https://doi.org/10.2307/270785.
Hoyt, C. (1941a). Note on a simplified method of computing test reliability. Educational and Psychological Measurement, 1(1), 93–95. https://doi.org/10.1177/001316444100100109.
Hoyt, C. (1941b). Test reliability estimated by analysis of variance. Psychometrika, 6(3), 153–160. https://doi.org/10.1007/BF02289270.
Hunt, T. D. (2013). Lambda4: Collection of internal consistency reliability coefficients. https://cran.r-project.org/web/packages/Lambda4/index.html
Hunt, T. D., & Bentler, P. M. (2015). Quantile lower bounds to reliability based on locally optimal splits. Psychometrika, 80(1), 182–195. https://doi.org/10.1007/s11336-013-9393-6.
Jackson, P. H., & Agunwamba, C. C. (1977). Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: I: Algebraic lower bounds. Psychometrika, 42(4), 567–578. https://doi.org/10.1007/BF02295979.
Jackson, R. W. B., & Ferguson, G. A. (1941). Studies on the reliability of tests. University of Toronto Department of. Educational Research Bulletin, 12, 132.
Jöreskog, K. G. (1966). Testing a simple structure hypothesis in factor analysis. Psychometrika, 31, 165–178. https://doi.org/10.1007/BF02289505.
Jöreskog, K. G. (1967a). A computer program for restricted maximum likelihood factor analysis. Research Memorandum 66-20. Educational Testing Service.
Jöreskog, K. G. (1967b). Some contributions to maximum likelihood factor analysis. Psychometrika, 32(4), 443–482. https://doi.org/10.1007/BF02289658.
Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183–202. https://doi.org/10.1007/BF02289343.
Jöreskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36(2), 109–133. https://doi.org/10.1007/BF02291393.
Kaiser, H. F., & Caffrey, J. (1965). Alpha factor analysis. Psychometrika, 30(1), 1–14. https://doi.org/10.1007/BF02289743.
Kamata, A., Turhan, A., & Darandari, E. (2003). Estimating reliability for multidimensional composite scale scores (pp. 1–27). Chicago: Annual Meeting of American Educational Research Association.
Kristof, W. (1971). On the theory of a set of tests which differ only in length. Psychometrika, 36(3), 207–225. https://doi.org/10.1007/BF02297843.
Kuder, G. F., & Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2(3), 151–160. https://doi.org/10.1007/BF02288391.
Lawley, D. N. (1940). The estimation of factor loadings by the method of maximum likelihood. Proceedings of the Royal Society of Edinburgh, 60(1), 64–82. https://doi.org/10.1017/S037016460002006X.
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Addison-Wesley.
McDonald, R. P. (1970). The theoretical foundations of principal factor analysis, canonical factor analysis, and alpha factor analysis. British Journal of Mathematical and Statistical Psychology, 23, 1–21. https://doi.org/10.1111/j.2044-8317.1970.tb00432.x.
McDonald, R. P. (1985). Factor analysis and related methods. Lawrence Erlbaum.
McDonald, R. P. (1999). Test theory: A unified treatment. Lawrence Erlbaum
Miller, M. B. (1995). Coefficient alpha: A basic introduction from the perspectives of classical test theory and structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 2(3), 255–273. https://doi.org/10.1080/10705519509540013.
Novick, M. R. (1966). The axioms and principal results of classical test theory. Journal of Mathematical Psychology, 3(1), 1–18. https://doi.org/10.1016/0022-2496(66)90002-2.
Oosterwijk, P. R. (2016). Statistical properties and practical use of classical test-score reliability methods. the Netherlands: Tilburg University. PhD dissertation.
Oosterwijk, P. R., van der Ark, L. A., & Sijtsma, K. (2017). Overestimation of reliability by Guttman’s \(\lambda \) 4, \(\lambda \) 5, and \(\lambda \) 6 and the Greatest Lower Bound. In L. A. van der Ark, M. Wiberg, S. A. Culpepper, J. A. Douglas, & W.-C. Wang (Eds.), Quantitative psychology research: The 81th Annual Meeting of the Psychometric Society 2016, Asheville NC, USA (Vol. 196, pp. 159–172). Springer. https://doi.org/10.1007/978-3-319-56294-0_15
Raykov, T. (2007). Reliability if deleted, not ‘alpha if deleted’: Evaluation of scale reliability following component deletion. British Journal of Mathematical and Statistical Psychology, 60(2), 201–216. https://doi.org/10.1348/000711006X115954
Raykov, T., & Marcoulides, G. A. (2019). Thanks coefficient alpha, we still need you!. Educational and Psychological Measurement, 79(1), 200–210. https://doi.org/10.1177/0013164417725127.
Reuterberg, S.-E., & Gustafsson, J.-E. (1992). Confirmatory factor analysis and reliability: Testing measurement model assumptions. Educational and Psychological Measurement, 52(4), 795–811. https://doi.org/10.1177/0013164492052004001.
Revelle, W. (2020). psych: Procedures for psychological, psychometric, and personality research. https://cran.r-project.org/web/packages/psych/index.html.
Revelle, W., & Condon, D. M. (2019). Reliability from \(\alpha \) to \(\omega \): A tutorial. Psychological Assessment, 31(12), 1395–1411. https://doi.org/10.1037/pas0000754.
Rodgers, J., & Nicewander, A. (1988). Thirteen ways to look at the correlation coefficient. American Statistician, 42, 59–66. https://doi.org/10.1080/00031305.1988.10475524.
Rönkkö, M., & Cho, E. (2020). An updated guideline for assessing discriminant validity. Organizational Research Methods. https://doi.org/10.1177/1094428120968614
Rulon, P. J. (1939). A simplified procedure for determining the reliability of a test by split-halves. Harvard Educational Review, 9(1), 99–103.
Savalei, V., & Reise, S. P. (2019). Don’t forget the model in your model-based reliability coefficients: A reply to McNeish (2018). Collabra: Psychology, 5(1), 36. https://doi.org/10.1525/collabra.247
Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107–120. https://doi.org/10.1007/s11336-008-9101-0
Sijtsma, K., & Pfadt, J. (2021). Invited review part II: On the use, the misuse, and the very limited usefulness of Cronbach’s alpha: Discussing lower bounds and correlated errors. Psychometrika.
Sörbom, D. (2001). Karl Jöreskog and LISREL: A personal story. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.), Structural equation modeling: Present and future–A festschrift in honor of Karl Jöreskog (pp. 3–10). Scientific Software International.
Spearman, C. (1904a). “General intelligence,” objectively determined and measured. American Journal of Psychology, 15(2), 201–292. https://doi.org/10.2307/1412107
Spearman, C. (1904b). The proof and measurement of association between two things. American Journal of Psychology, 15(1), 72–101. https://doi.org/10.2307/1412159.
Tang, W., & Cui, Y. (2012). A simulation study for comparing three lower bounds to reliability. Paper Presented on April 17, 2012 at the AERA Division D: Measurement and Research Methodology, Section 1: Educational Measurement, Psychometrics, and Assessment., 1–25.
Ten Berge, J. M. F., & Zegers, F. E. (1978). A series of lower bounds to the reliability of a test. Psychometrika, 43(4), 575–579. https://doi.org/10.1007/BF02293815.
Votaw, D. F. (1948). Testing compound symmetry in a normal multivariate distribution. Annals of Mathematical Statistics, 19(4), 447–473. https://doi.org/10.1214/aoms/1177730145.
Watkins, M. W. (2017). The reliability of multidimensional neuropsychological measures: From alpha to omega. The Clinical Neuropsychologist, 31(6–7), 1113–1126. https://doi.org/10.1080/13854046.2017.1317364.
Yang, Y., & Green, S. B. (2010). A note on structural equation modeling estimates of reliability. Structural Equation Modeling, 17(1), 66–81. https://doi.org/10.1080/10705510903438963.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The present research was supported by the Research Grant of Kwangwoon University in 2020.
Rights and permissions
About this article
Cite this article
Cho, E. Neither Cronbach’s Alpha nor McDonald’s Omega: A Commentary on Sijtsma and Pfadt. Psychometrika 86, 877–886 (2021). https://doi.org/10.1007/s11336-021-09801-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-021-09801-1