Abstract
The use of one-way analysis of variance tables for obtaining unbiased estimates of true score variance and error score variance in the classical test theory model is discussed. Attention is paid to both balanced (equal numbers of observations on each person) and unbalanced designs, and estimates provided for both homoscedastic (common error variance for all persons) and heteroscedastic cases.
It is noted that optimality properties (minimum variance) can be claimed for estimates derived from analysis of variance tables only in the balanced, homoscedastic case, and that there they are essentially a reflection of the symmetry inherent in the situation. Estimates which might be preferable in other cases are discussed. An example is given where a natural analysis of variance table leads to estimates which cannot be derived from the set of statistics which is sufficient under normality assumptions. Reference is made to Bayesian studies which shed light on the difficulties encountered.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Graybill, F. A., & Hultquist, F. A. Theorems concerning Eisenhart's Model II.Annals of Mathematical Statistics, 1961,32, 261–269.
Harville, D. A. Quadratic unbiased estimation of variance components for the one-way classification.Biometrika, 1969(a),56, 313–326.
Harville, D. A. Variance component estimation for the unbalanced one-way random classification-a critique.Aerospace Research Laboratories Report ARL 69-0180, 1969(b).
Hewitt, E., & Savage, L. J. Symmetric measures on cartesian products.Transactions of the American Mathematical Society, 1955,80, 470–501.
Hill, B. M. Inference about variance components in the one-way model.Journal of the American Statistical Association, 1965,60, 806–825.
Jackson, P. H. Simple approximations in the estimation of many parameters.Technical Bulletin No.2. Iowa City, Iowa: The American College Testing Program, 1972.
Lindley, D. V. The estimation of many parameters.foundations of Statistical Inference. V. P. Godambe and D. A. Sprott (Eds.) Toronto: Holt, Rinehart, and Winston, 1971, 435–455.
Novick, M. R. Bayesian considerations in educational information systems. In G. V. Glass, (Ed.)Proceedings of the 1970 Invitational Conference on Testing Problems. Princeton, New Jersey: Educational Testing Service, 1971.
Novick, M. R., Jackson, P. H., & Thayer, D. T. Bayesian inference and the classical test theory model: Reliability and true scores.Psychometrika, 1971,36, 261–288.
Tiao, G. C., & Tan, W. Y. Bayesian analysis of random-effect models in the analysis of variance: I Posterior distribution of variance components.Biometrika, 1965,52, 37–53.
Tukey, J. W. Variances of variance components: II The unbalanced single classification.Annals of Mathematical Statistics, 1957,28, 43–56.
Author information
Authors and Affiliations
Additional information
Work on this paper was carried out at the headquarters of the American College Testing Program, Iowa City, Iowa, while the author was on leave from the University College of Wales.
Rights and permissions
About this article
Cite this article
Jackson, P.H. The estimation of true score variance and error variance in the classical test theory model. Psychometrika 38, 183–201 (1973). https://doi.org/10.1007/BF02291113
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02291113