Psychonomic Bulletin & Review

, Volume 1, Issue 4, pp 476–490 | Cite as

Using confidence intervals in within-subject designs

Article

Abstract

We argue that to best comprehend many data sets, plotting judiciously selected sample statistics with associated confidence intervals can usefully supplement, or even replace, standard hypothesis-testing procedures. We note that most social science statistics textbooks limit discussion of confidence intervals to their use in between-subject designs. Our central purpose in this article is to describe how to compute an analogous confidence interval that can be used in within-subject designs. This confidence interval rests on the reasoning that because between-subject variance typically plays no role in statistical analyses of within-subject designs, it can legitimately be ignored; hence, an appropriate confidence interval can be based on the standard within-subject error term—that is, on the variability due to the subject × condition interaction. Computation of such a confidence interval is simple and is embodied in Equation 2 on p. 482 of this article. This confidence interval has two useful properties. First, it is based on the same error term as is the corresponding analysis of variance, and hence leads to comparable conclusions. Second, it is related by a known factor (√2) to a confidence interval of the difference between sample means; accordingly, it can be used to infer the faith one can put in some pattern of sample means as a reflection of the underlying pattern of population means. These two properties correspond to analogous properties of the more widely used between-subject confidence interval.

References

  1. Anderson, V., &McLean, R. A. (1974).Design of experiments: A realistic approach. New York: Marcel Dekkar.Google Scholar
  2. Bakan, D. (1966). The test of significance in psychological research.Psychological Bulletin,66, 423–437.CrossRefPubMedGoogle Scholar
  3. Bayes, T. (1763). An essay towards solving a problem in the doctrine of chances.Philosophical Transactions of the Royal Society,53, 370–418.CrossRefGoogle Scholar
  4. Berger, J. O., &Berry, D. A. (1988). Statistical analysis and the illusion of objectivity.American Scientist,76, 159–165.Google Scholar
  5. Boik, R. J. (1981). A priori tests in repeated measures designs: Effects of nonsphericity.Psychometrika,46, 241–255.CrossRefGoogle Scholar
  6. Box, G. E. P. (1954). Some theorems on quadratic forms applied in the study of analysis of variance problems: II. Effect of inequality of variance and of correlation between errors in the two-way classification.Annals of Mathematical Statistics,25, 484–498.CrossRefGoogle Scholar
  7. Box, G. E. P. (1986). An apology for ecumenism in statistics. In G. E. P. Box, T. Leonard, & C.-F. Wu (Eds.),Scientific inference, data analysis, and robustness (pp. 51–84). New York: Academic Press.Google Scholar
  8. Box, G. E. P., &Tiao, G. C. (1973).Bayesian inference in statistical analysis. Reading, MA: Addison-Wesley.Google Scholar
  9. Camilli, G. (1990). The test of homogeneity for 2 × 2 contingency tables: A review of and some personal opinions on the controversy.Psychological Bulletin,108, 135–145.CrossRefGoogle Scholar
  10. Cohen, J. (1990). Things I have learned (so far).American Psychologist,45, 1304–1312.CrossRefGoogle Scholar
  11. Fisher, R. A. (1925).Statistical methods for research workers. Edinburgh: Oliver & Boyd.Google Scholar
  12. Fisher, R. A. (1935). The logic of inductive inference.Journal of the Royal Statistical Society,98, 39–54.CrossRefGoogle Scholar
  13. Fisher, R. A. (1947).The design of experiments. New York: Hafner Press.Google Scholar
  14. Fisher, R. A. (1955). Statistical methods and scientific induction.Journal of the Royal Statistical Society, Series B,17, 69–78.Google Scholar
  15. Gigerenzer, G., Swijtink, Z., Porter, T., Daston, L., Beatty, J., &Krüger, L. (1989).The empire of chance. Cambridge: Cambridge University Press.Google Scholar
  16. Greenhouse, S. W., &Geisser, S. (1959). On methods in the analysis of profile data.Psychometrika,24, 95–112.CrossRefGoogle Scholar
  17. Hays, W. (1973).Statistics for the social sciences (2nd ed.). New York: Holt.Google Scholar
  18. Hertzog, C., &Rovine, M. (1985). Repeated-measures analysis of variance in developmental research: Selected issues.Child Development,56, 787–809.CrossRefPubMedGoogle Scholar
  19. Huynh, H., &Feldt, L. S. (1970). Conditions under which mean square ratios in repeated measures designs have exactF distributions.Journal of the American Statistical Association,65, 1582–1589.CrossRefGoogle Scholar
  20. Huynh, H., &Feldt, L. S. (1976). Estimation of the Box correction for degrees of freedom from sample data in the randomized block and split plot designs.Journal of Educational Statistics,1, 69–82.CrossRefGoogle Scholar
  21. Lehmann, E. L. (1993). The Fisher, Neyman-Pearson theories of testing hypotheses: One theory or two?Journal of the American Statistical Association,88, 1242–1249.CrossRefGoogle Scholar
  22. Lewis, C. (1993). Bayesian methods for the analysis of variance. In G. Kerens & C. Lewis (Eds.),A handbook for data analysis in the behavioral sciences: Statistical issues (pp. 233–258). Hillsdale, NJ: Erlbaum.Google Scholar
  23. Loftus, G. R. (1991). On the tyranny of hypothesis testing in the social sciences.Contemporary Psychology,36, 102–105.Google Scholar
  24. Loftus, G. R. (1993a). Editorial Comment.Memory & Cognition,21, 1–3.Google Scholar
  25. Loftus, G. R. (1993b, November).On the overreliance of significance testing in the social sciences. Paper presented at the annual meeting of the Psychonomic Society, Washington, DC.Google Scholar
  26. Loftus, G. R. (1993c). Visual data representation and hypothesis testing in the microcomputer age.Behavior Research Methods, Instrumentation, & Computers,25, 250–256.Google Scholar
  27. Loftus, G. R., &Loftus, E. F. (1988).Essence of statistics (2nd ed.). New York: Random House.Google Scholar
  28. Neyman, J. (1957). “Inductive behavior” as a basic concept of philosophy of science.Review of the International Statistical Institute,25, 7–22.CrossRefGoogle Scholar
  29. Neyman, J., &Pearson, E. S. (1928). On the use and interpretation of certain test criteria for purposes of statistical inference.Biometrika,20A, 175–240, 263–294.Google Scholar
  30. Neyman, J., &Pearson, E. S. (1933). On the problem of the most efficient tests of statistical hypotheses.Philosophical Transactions of the Royal Society of London, Series A,231, 289–337.CrossRefGoogle Scholar
  31. O’Brien, R. G., &Kaiser, M. K. (1985). MANOVA method for analyzing repeated measures designs: An extensive primer.Psychological Bulletin,97, 316–333.CrossRefPubMedGoogle Scholar
  32. Sternberg, S. (1966). High-speed scanning in human memory.Science,153, 652–654.CrossRefPubMedGoogle Scholar
  33. Tufte, E. R. (1983).The visual display of quantitative information. Cheshire, CT: Graphics Press.Google Scholar
  34. Tufte, E. R. (1990).Envisioning information. Cheshire, CT: Graphics Press.Google Scholar
  35. Tukey, J. W. (1974). The future of data analysis.Annals of Mathematical Statistics,33, 1–67.CrossRefGoogle Scholar
  36. Tukey, J. W. (1977).Exploratory data analysis. Reading, MA: Addison-Wesley.Google Scholar
  37. Wainer, H., &Thissen, D. (1993). Graphical data analysis. In G. Kerens & C. Lewis (Eds.),A handbook for data analysis in the behavioral sciences: Statistical issues (pp. 391–458). Hillsdale, NJ: Erlbaum.Google Scholar
  38. Winer, B. J. (1971).Statistical principles in experimental design (2nd ed.). New York: McGraw-Hill.Google Scholar
  39. Winkler, R. L. (1993). Bayesian statistics: An overview. In G. Kerens & C. Lewis (Eds.),A handbook for data analysis in the behavioral sciences: Statistical issues (pp. 201–232). Hillsdale, NJ: Erlbaum.Google Scholar

Copyright information

© Psychonomic Society, Inc. 1994

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of WashingtonSeattle
  2. 2.University of VictoriaVictoriaCanada

Personalised recommendations