Skip to main content
SpringerLink
Log in

The relationship between variance components and mean difference effect size

  • Articles
  • Published:
Current Psychology Aims and scope Submit manuscript

Abstract

Variance components in the context of Generalizability Theory are useful indices for attributing the amount of variance to a particular facet or object of measurement. The mean difference effect size (MDES) has proven to be a useful tool both in synthesizing the results of multiple studies and in interpreting individual study results. A mathematical relationship is drawn, therefore, between the variance components of a nested two-facet design configuration on the one hand, and the MDES that can be calculated from the descriptive statistics of an hypothetical measurement study on the other. Using two different variance components estimation procedures, we show that the relationship is both statistically and conceptually meaningful in that all of these estimates closely approximates the true Glassian MDES.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Boodoo, G. M. (1982). On describing an incidence sample. Journal of Educational Statistics, 7, 311–331.

    Article  Google Scholar 

  • Brennan, R. L. (1992a). Elements of generalizability theory. Iowa City, IA: American College Testing.

    Google Scholar 

  • Brennan, R. L. (1992b). Generalizability theory. Educational Measurement: Issues and Practice, 11, 27–34.

    Article  Google Scholar 

  • Cardinet, J., & Allai, L. (1983). Estimation of generalizability parameters. In L. J. Fyans, Jr. (Ed.), Generalizability Theory: Inferences and Practical Applications. New Directions for Testing and Measurement, 18, 17–48. San Francisco: Jossey-Bass.

    Google Scholar 

  • Carver, R. P. (1978). The case against statistical significance testing. Harvard Educational Review, 48, 378–399.

    Google Scholar 

  • Carver, R. P. (1993). The case against statistical significance testing, revisited. Journal of Experimental Education, 61, 287–292.

    Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Fort Worth, TX: Harcourt Brace Jovanovich.

    Google Scholar 

  • Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York:Wiley.

    Google Scholar 

  • Figueredo, A. J., Cox, R. L., & Rhine, R. J. (1995). A generalizability analysis of subjective personality assessments in the stump tail macaque and the zebra finch. Multivariate Behavioral Research, 30, 167–197.

    Article  Google Scholar 

  • Fyans, L. J., Jr. (1983). Multilevel analysis and cross-level inference of validity using generalizability theory. In L. J. Fyans, Jr. (Ed.), Generalizability Theory: Inferences and Practical Applications. New Directions for Testing and Measurement, 18, 49–66. San Francisco: Jossey-Bass.

    Google Scholar 

  • Gaylor, D. W., Lucas, H. L., & Anderson, R. L. (1970). Calculation of expected mean squares by the abbreviated Doolittle and Square Root Methods. Biometrics, 26, 641–655.

    Article  Google Scholar 

  • Glass, G. V. (1977). Integrating findings: The meta-analysis of research, In Review of research in education, Vol. 5 (pp. 351–379). Washington, D.C.: American Educational Research Association.

    Google Scholar 

  • Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.

    Google Scholar 

  • Hartley, H. O., Rao, J. N. K., & LaMotte, L. (1978). A simple synthesis-based method of variance component estimation. Biometrics, 34, 233–244.

    Article  Google Scholar 

  • Hohnes, C. T. (1984). Effect size estimation in meta-analysis. Journal of Experimental Education, 52, 106–109.

    Google Scholar 

  • Jarjoura, D., & Brennan, R. L. (1982). A variance components model for measurement procedures associated with a table of specifications. Applied Psychological Measurement, 6, 161–171.

    Article  Google Scholar 

  • Knapp, T. R. (1978). Canonical correlational analysis: A general parametric significance testing system. Psychological Bulletin, 85, 410–416.

    Article  Google Scholar 

  • Marcoulides, G. A. (1996). Estimating variance components in Generalizability Theory: The covariance structure analysis approach. Structural Equation Modeling, 3, 290–299.

    Article  Google Scholar 

  • Rosenthal, R., & Rubin, D. B. (1982). A simple, general purpose display of magnitude of experimental effect. Journal of Educational Psychology, 74, 166–169.

    Article  Google Scholar 

  • SAS Institute, Inc. (1990). SAS/STAT users guide (Version 6) Vol. 2 (4th Ed). Cary, NC: SAS Institute, Inc.

    Google Scholar 

  • Shavelson, R., & Webb, N. M. (1981). Generalizability theory: 1973-1980. British Journal of Mathematical and Statistical Psychology, 34, 133–166.

    Google Scholar 

  • Shavelson, R., & Webb, N. M. (1991). Generalizability theory. A primer. Newbury Park: Sage.

    Google Scholar 

  • Shavelson, R. J., Webb, N. M., & Rowley, G. L. (1989). Generalizability theory. American Psychologist, 44, 922–932.

    Article  Google Scholar 

  • Shaver, J. P. (1993). What statistical significance testing is, and what it is not. Journal of Experimental Education, 61, 293–316.

    Google Scholar 

  • Sirotnik, K., & Wellington, R. (1977). Incidence sampling: An integrated theory for “matrix sampling.” Journal of Educational Measurement, 14, 343–399.

    Article  Google Scholar 

  • Smith, P.L. (1978). Sampling errors of variance components in small sample multifacet generalizability studies. Journal of Educational Statistics, 3, 319–346.

    Article  Google Scholar 

  • Smith, P. L. (1981). Gaining accuracy in generalizability theory: Using multiple designs. Journal of Educational Measurement, 18, 147–154.

    Article  Google Scholar 

  • Smith, P. L. (1982). A confidence interval approach for variance component estimates in the context of generalizability theory. Educational and Psychological Measurement, 42, 459–466.

    Article  Google Scholar 

  • Snyder, P., & Lawson, S. (1993). Evaluating results using corrected and uncorrected effect size estimates. Journal of Experimental Education, 61, 334–339.

    Google Scholar 

  • Taylor, M. J., and White, K. R. (1992). An evaluation of alternative methods for computing standardized mean difference effect size. Journal of Experimental Education, 61, 63–72.

    Article  Google Scholar 

  • Thompson, B. (1986). ANOVA versus regression analysis of ATI designs: An empirical investigation. Educational and Psychological Measurement, 46, 917–928.

    Article  Google Scholar 

  • Thompson, B. (1991). A primer on the logic and use of canonical correlation analysis. Measurement and Evaluation in Counseling and Development, 24, 80–95.

    Google Scholar 

  • Thompson, B. (1993). The use of statistical significance tests in research: Bootstrap and other alternatives. Journal of Experimental Education, 61, 361–377.

    Google Scholar 

  • Thompson, B. (1996). AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educational Researcher, 25, 26–30.

    Article  Google Scholar 

  • Wellington, R. (1976). Extending generalized symmetric means to arbitrary matrix sampling designs. Psychometrika, 41, 375–384.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Mississippi, USA

    Michael J. Chen

  2. Utah State University, USA

    Xitao Fan

Authors
  1. Michael J. Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xitao Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chen, M.J., Fan, X. The relationship between variance components and mean difference effect size. Curr Psychol 17, 301–311 (1998). https://doi.org/10.1007/s12144-998-1013-8

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12144-998-1013-8

Keywords

Search

Navigation