Abstract
This paper studies the asymptotic distributions of three reliability coefficient estimates: Sample coefficient alpha, the reliability estimate of a composite score following a factor analysis, and the estimate of the maximal reliability of a linear combination of item scores following a factor analysis. Results indicate that the asymptotic distribution for each of the coefficient estimates, obtained based on a normal sampling distribution, is still valid within a large class of nonnormal distributions. Therefore, a formula for calculating the standard error of the sample coefficient alpha, recently obtained by van Zyl, Neudecker and Nel, applies to other reliability coefficients and can still be used even with skewed and kurtotic data such as are typical in the social and behavioral sciences.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Allen, M.J., & Yen, W.M. (1979).Introduction to measurement theory. Monterey, CA: Brooks-Cole.
Azzalini, A., & Valle, A.D. (1996). The multivariate skew-normal distribution.Biometrika, 83, 715–726.
Bentler, P.M. (1968). Alpha-maximized factor analysis (Alphamax): Its relation to alpha and canonical factor analysis.Psychometrika, 33, 335–345.
Bentler, P.M. (1972). A lower-bound method for the dimension-free measurement of internal consistency.Social Science Research, 1, 343–357.
Bentler, P.M., & Woodward, J.A. (1983). The greatest lower bound to reliability. In H. Wainer & S. Messick (Eds.),Principals of modern psychological measurement: A Festschrift for Frederic M. Lord (pp. 237–253). Hillsdale, NJ: Erlbaum.
Crocker, L., & Algina, J. (1986).Introduction to classical and modern test theory. New York, NY: Hott, Rinehart, and Winston.
Cronbach, L.J. (1951). Coefficient alpha and the internal structure of tests.Psychometrika, 16, 297–334.
Drewes, D.W. (2000). Beyond the Spearman-Brown: A structural approach to maximal reliability.Psychological Methods, 5, 214–227.
Feldt, L.S. (1965). The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty.Psychometrika, 30, 357–370.
Ferguson, T. (1996).A course in large sample theory. London, England: Chapman & Hall.
Fleishman, A. (1978). A method for simulating non-normal distributions.Psychometrika, 43, 521–532.
Guttman, L. (1945). A basis for analyzing test-retest reliability.Psychometrika, 10, 255–282.
Hancock, G.R., & Mueller, R.O. (2001). Rethinking construct reliability. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.),Structural equation modeling: Present and future (pp. 195–216). Lincolnwood, IL: Scientific Software International.
Headrick, T.C., & Sawilowsky, S.S. (1999). Simulating correlated multivariate nonnormal distributions: Extending the Fleishman power method.Psychometrika, 64, 25–35.
Kristof, W. (1963). The statistical theory of stepped-up reliability when a test has been divided into several equivalent parts.Psychometrika, 28, 221–238.
Li, H. (1997). A unifying expression for the maximal reliability of a linear composite.Psychometrika, 62, 245–249.
Li, H., Rosenthal, R., & Rubin, D.B. (1996). Reliability of measurement in psychology: From Spearman-Brown to maximal reliability.Psychological Methods, 1, 98–107.
Li, H., & Wainer, H. (1997). Toward a coherent view of reliability in test theory.Journal of Educational and Behavioral Statistics, 22, 478–484.
Lord, F.M., & Novick, M.R. (1968).Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
Magnus, J.R., & Neudecker, H. (1999).Matrix differential calculus with applications in statistics and econometrics (rev. ed.). New York, NY: Wiley.
McDonald, R.P. (1999).Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum Associates.
Mendoza, J.L., Stafford, K.L., & Stauffer, J.M. (2000). Large-sample confidence intervals for validity and reliability coefficients.Psychological Methods, 5, 356–369.
Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures.Psychological Bulletin, 105, 156–166.
Nunnally, J.C., & Bernstein (1994).Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.
Osburn, H.G. (2000). Coefficient alpha and related internal consistency reliability coefficients.Psychological Methods, 5, 343–355.
Raykov, T. (1997). Estimation of composite reliability for congeneric measures.Applied Psychological Measurement, 21, 173–184.
Satorra, A., & Bentler, P.M. (1986). Some robustness properties of goodness of fit statistics in covariance structure analysis.American Statistical Association 1986 Proceedings of Business and Economics Sections (pp. 549–554). Alexander, VA: American Statistical Association.
Shapiro, A., & ten Berge, J.M.F. (2000). The asymptotic bias of minimum trace factor analysis with applications to the greatest lower bound to reliability.Psychometrika, 65, 413–425.
Tadikamalla, P.R. (1980). On simulating non-normal distributions.Psychometrika, 45, 273–279.
ten Berge, J.M.F. (2000). Linking reliability and factor analysis: Recent developments in some classical psychometric problems. In S.E. Hampson (Ed),Advances in personality psychology, Vol. 1 (pp. 138–156). Philadelphia, PA: Psychology Press/Taylor & Francis.
ten Berge, J.M.F., & Hofstee, W.K.B. (1999). Coefficients alpha and reliabilities of unrotated and rotated components.Psychometrika, 64, 83–90.
ten Berge, J.M.F., & Kiers, H.A.L. (1991). A numerical approach to the exact and the approximate minimum rank of a covariance matrix.Psychometrika, 56, 309–315.
Thomson, G.H. (1940). Weighting for battery reliability and prediction.British Journal of Psychology, 30, 357–366.
Vale, D., & Maurelli, V.A. (1983). Simulating multivariate nonnormal distributions.Psychometrika, 48, 465–471.
van Zyl, J.M., Neudecker, H., & Nel, D.G. (2000). On the distribution of the maximum likelihood estimator of Cronbach's alpha.Psychometrika, 65, 271–280.
Woodhouse, B., & Jackson, P.H. (1977). Lower bounds for the reliability of a total score on a test composed of nonhomogeneous items: II. A search procedure to locate the greatest lower bound.Psychometrika, 42, 579–591.
Yuan, K.-H., & Bentler, P.M. (1997). Generating multivariate distributions with specified marginal skewness and kurtosis. In W. Bandilla & F. Faulbaum (Eds.),SoftStat'97—Advances in statistical software 6 (pp. 385–391). Stuttgart, Germany: Lucius & Lucius.
Yuan, K.-H., & Bentler, P.M. (1999). On normal theory and associated test statistics in covariance structure analysis under two classes of nonnormal distributions.Statistica Sinica, 9, 831–853.
Yuan, K.-H., & Bentler, P.M. (2000). Inferences on correlation coefficients in some classes of nonnormal distributions.Journal of Multivariate Analysis, 72, 230–248.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was supported by grants DA01070 and DA00017 from the National Institute on Drug Abuse and a University of North Texas faculty research grant. We would like to thank the Associate Editor and two reviewers for suggestions that helped to improve the paper.
Rights and permissions
About this article
Cite this article
Yuan, KH., Bentler, P.M. On robusiness of the normal-theory based asymptotic distributions of three reliability coefficient estimates. Psychometrika 67, 251–259 (2002). https://doi.org/10.1007/BF02294845
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02294845