Abstract
We show that power and sample size tables developed by Cohen (1988, pp. 289–354, 381–389) produce incorrect estimates for factorial designs: power is underestimated, and sample size is overestimated. The source of this bias is shrinkage in the implied value of the noncentrality parameter, λ, caused by using Cohen’s adjustment ton for factorial designs (pp. 365 and 396). The adjustment was intended to compensate for differences in the actual versus presumed (by the tables) error degrees of freedom; however, more accurate estimates are obtained if the tables are used without adjustment. The problems with Cohen’s procedure were discovered while testing subroutines in DATASIM 1.2 for computing power and sample size in completely randomized, randomized-blocks, and split-plot factorial designs. The subroutines give the user the ability to generate power and sample size tables that are as easy to use as Cohen’s, but that eliminate the conservative bias of his tables. We also implemented several improvements relative to “manual” use of Cohen’s tables: (1) Since the user can control the specific values of 1- β,n, andf used on the rows and columns of the table, interpolation is never required; (2) exact as opposed to approximate solutions for the noncentralF distribution are employed; (3) solutions for factorial designs, including those with repeated measures factors, take into account the actual error degrees of freedom for the effect being tested; and (4) provision is made for the computation of power for applications involving the doubly noncentralF distribution.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Abramowitz, M., &Stegun, I. A. (Eds.) (1970).Handbook of mathematical functions (NBS Special Publication No. 55). Washington, DC: National Bureau of Standards.
Bradley, D. R. (1988).DATASIM. Lewiston, ME: Desktop Press.
Bradley, D. R. (1989). Computer simulation with DATASIM.Behavior Research Methods, Instruments, & Computers,21, 99–112.
Bradley, D. R., &Fleisher, C. L. (1994). Generating multivariate data from non-normal distributions: Mihal and Barrett revisited.Behavior Research Methods, Instruments, & Computers,26, 156–166.
Bradley, D. R.,Russell, R. L., &Reeve, C. P. (1996).The accuracy of four approximations to non-central F. Manuscript submitted for publication.
Cohen, J. (1962). The statistical power of abnormal-social psychological research: A review.Journal of Abnormal & Social Psychology,65, 145–153.
Cohen, J. (1969).Statistical power analysis for the behavioral sciences. New York: Academic Press.
Cohen, J. (1977).Statistical power analysis for the behavioral sciences (rev. ed.). New York: Academic Press.
Cohen, J. (1988).Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
Erdfelder, E., Faul, F., &Buchner, A. (1996). GPOWER: A general power analysis program.Behavior Research Methods, Instruments, & Computers,28, 1–11.
Koele, P. (1982). Calculating power in the analysis of variance.Psychological Bulletin,92, 513–516.
Koele, P., &Hoogstraten, J. (1980).Power and sample size calculations in the analysis of variance (Révész Berichten No. 12). Amsterdam: Psychologisch Laboratorium, Universiteit van Amsterdam.
Laubscher, N. F. (1960). Normalizing the noncentralt andF distributions.Annals of Mathematical Statistics,31, 1105–1112.
Patnaik, P. B. (1949). The non-central χ2 andF distributions and their applications.Biometrics,36, 202–232.
Reeve, C. P. (1986).An algorithm for computing the doubly noncentral F C.D.F. to a specified accuracy (Statistical Engineering Division Note 86-4). Gaithersburg, MD: National Bureau of Standards.
Rossi, J. S. (1982).Meta-analysis, power analysis and artifactual controversy: The case of spontaneous recovery of verbal associations. Paper presented at the meeting of the Eastern Psychological Association, Baltimore.
Rossi, J. S. (1990). Statistical power of psychological research: What have we gained in 20 years?Journal of Consulting & Clinical Psychology,58, 646–656.
Scheffé, H. (1959).Analysis of variance. New York: Wiley.
Sedlmeier, P., &Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies?Psychological Bulletin,105, 309–316.
Tiku, M. L. (1965). Laguerre series forms of non-central χ2 andF distributions.Biometrika,52, 415–427.
Tiku, M. L. (1967). Tables of the power of theF-test.Journal of the American Statistical Association,62, 525–539.
Wright, J. (1993). Datasim 1.0.Psychology Software News,4 (1), 15–16.
Author information
Authors and Affiliations
Corresponding author
Additional information
The authors thank David Howell, Richard Lehman, Glenn Meyer, Patrick Onghena, Martin Richter, and Robert Schutz for helpful comments and suggestions on an initial draft of this paper. Any errors that remain are the responsibility of the authors.
Rights and permissions
About this article
Cite this article
Bradley, D.R., Russell, R.L. & Reeve, C.P. Statistical power in complex experimental designs. Behavior Research Methods, Instruments, & Computers 28, 319–326 (1996). https://doi.org/10.3758/BF03204790
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03204790