Advertisement

Reform of statistical inference in psychology: The case ofMemory & Cognition

  • Sue FinchEmail author
  • Geoff Cumming
  • Jennifer Williams
  • Lee Palmer
  • Elvira Griffith
  • Chris Alders
  • James Anderson
  • Olivia Goodman
Articles

Abstract

Geoffrey Loftus, Editor ofMemory & Cognition from 1994 to 1997, strongly encouraged presentation of figures with error bars and avoidance of null hypothesis significance testing (NHST). The authors examined 696Memory & Cognition articles published before, during, and after the Loftus editorship. Use of figures with bars increased to 47% under Loftus’s editorship and then declined. Bars were rarely used for interpretation, and NHST remained almost universal. Analysis of 309 articles in other psychology journals confirmed that Loftus’s influence was most evident in the articles he accepted for publication, but was otherwise limited. An e-mail survey of authors of papers accepted by Loftus revealed some support for his policy, but allegiance to traditional practices as well. Reform of psychologists’ statistical practices would require more than editorial encouragement.

Keywords

Statistical Inference American Psychological Association Lead Author Reporting Practice Psychology Journal 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. American Psychological Association (2001).Publication manual of the American Psychological Association (5th ed.). Washington, DC: Author.Google Scholar
  2. Bakan, D. (1966). The test of significance in psychological research.Psychological Bulletin,66,423–437.PubMedCrossRefGoogle Scholar
  3. Carver, R. (1978). The case against significance testing.Harvard Educational Review,48, 378–399.Google Scholar
  4. Carver, R. (1993). The case against statistical significance testing, revisited.Journal of Experimental Education,61, 287–292.Google Scholar
  5. Cohen, J. (1990). Things I have learned (so far).American Psychologist,45,1304–1312.CrossRefGoogle Scholar
  6. Cohen, J. (1994). The earth is round (p < 0.05).American Psychologist,49,997–1003.CrossRefGoogle Scholar
  7. Cumming, G., &Finch, S. (2001). A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions.Educational & Psychological Measurement,61, 530–572.Google Scholar
  8. Cumming, G., & Finch, S. (2003).Inference by eye: Confidence intervals, and how to read pictures of data. Manuscript submitted for publication.Google Scholar
  9. Estes, W. K. (1997). On the communication of information by displays of standard errors and confidence intervals.Psychonomic Bulletin & Review,4,330–341.CrossRefGoogle Scholar
  10. Finch, S., Cumming, G., &Thomason, N. (2001). Reporting of statistical inference in theJournal of Applied Psychology: Little evidence of reform.Educational & Psychological Measurement,61,181–210.Google Scholar
  11. Finch, S., Thomason, N., &Cumming, G. (2002). Past and future American Psychological Association guidelines for statistical practice.Theory & Psychology,12, 825–853.CrossRefGoogle Scholar
  12. Friedrich, J. (2000). The road to reform: Of editors and educators.American Psychologist,55,961–962.CrossRefGoogle Scholar
  13. Gernsbacher, M. A. (1998). Editorial.Memory & Cognition,26, 1.CrossRefGoogle Scholar
  14. Hammond, G. [R.] (1996). The objections to null hypothesis testing as a means of analysing psychological data.Australian Journal of Psychology,48,104–106.CrossRefGoogle Scholar
  15. Harlow, L. L., Mulaik, S. A., &Steiger, J. H. (Eds.) (1997).What if there were no significance tests? Mahwah, NJ: Erlbaum.Google Scholar
  16. Institute for Scientific Information (2000).Social sciences citation index. Philadelphia: Author.Google Scholar
  17. Intons-Peterson, M. J. (1990). Editorial.Memory & Cognition,18, 1–2.CrossRefGoogle Scholar
  18. Kirk, R. E. (1996). Practical significance: A concept whose time has come.Educational & Psychological Measurement,56,746–759.CrossRefGoogle Scholar
  19. Loftus, G. R. (1993a). Editorial comment.Memory & Cognition,21, 1–3.CrossRefGoogle Scholar
  20. Loftus, G. R. (1993b). A picture is worth a thousandp values: On the irrelevance of hypothesis testing in the microcomputer age.Behavior Research Methods, Instruments, & Computers,25,250–256.CrossRefGoogle Scholar
  21. Loftus, G. R. (1996). Why psychology will never be a real science until we change the way we analyze data.Current Directions in Psychological Science,5,161–171.CrossRefGoogle Scholar
  22. Loftus, G. R. (2002). Analysis, interpretation, and visual presentation of experimental data. In J. Wixted & H. Pashler (Eds.),Stevens’ handbook of experimental psychology: Vol. 4. Methodology in experimental psychology (3rd ed., pp. 339–390). New York: Wiley.Google Scholar
  23. Loftus, G. R., &Masson, M. E. J. (1994). Using confidence intervals in within-subject designs.Psychonomic Bulletin & Review,1,476–490.CrossRefGoogle Scholar
  24. Lykken, D. T. (1968). Statistical significance in psychological research.Psychological Bulletin,70(3, Pt. 1), 151–159.PubMedCrossRefGoogle Scholar
  25. Masson, M. (2003). Introduction to the special issue on alternative methods of data interpretation.Canadian Journal of Experimental Psychology,57, 139.Google Scholar
  26. Masson, M., &Loftus, G. R. (2003). Using confidence intervals for graphically based interpretation.Canadian Journal of Experimental Psychology,57,203–220.PubMedGoogle Scholar
  27. Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology.Journal of Consulting & Clinical Psychology,46,806–843.CrossRefGoogle Scholar
  28. Nickerson, R. (2000). Null hypothesis significance testing: A review of an old and continuing controversy.Psychological Methods,5, 241–301.PubMedCrossRefGoogle Scholar
  29. Oakes, M. (1986).Statistical inference: A commentary for the social and behavioural sciences. Chichester, U.K.: Wiley.Google Scholar
  30. Schmidt, F. L. (1992). What do data really mean? Research findings, meta-analysis and cumulative knowledge in psychology.American Psychologist,47,1173–1181.CrossRefGoogle Scholar
  31. Sedlmeier, P., &Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies?Psychological Bulletin,105,309–316.CrossRefGoogle Scholar
  32. Tukey, J. W. (1969). Analyzing data: Sanctification or detective work?American Psychologist,24,83–91.CrossRefGoogle Scholar
  33. Wilkinson, L., &the Task Force on Statistical Inference, APA Board of Scientific Affairs (1999). Statistical methods in psychology journals: Guidelines and explanations.American Psychologist,54,594–604.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2004

Authors and Affiliations

  • Sue Finch
    • 1
    Email author
  • Geoff Cumming
    • 2
  • Jennifer Williams
    • 2
  • Lee Palmer
    • 2
  • Elvira Griffith
    • 2
  • Chris Alders
    • 2
  • James Anderson
    • 2
  • Olivia Goodman
    • 2
  1. 1.Statistical Consulting CentreUniversity of MelbourneMelbourneAustralia
  2. 2.School of Psychological ScienceLa Trobe UniversityMelbourneAustralia

Personalised recommendations