Advertisement

Effect Sizes and Power Analysis in HCI

  • Koji Yatani
Chapter
Part of the Human–Computer Interaction Series book series (HCIS)

Abstract

Null hypothesis significance testing (NHST) is a common statistical analysis method in HCI. But its usage and interpretation are often misunderstood. In particular, NHST does not offer the magnitude of differences observed, which is more desirable to determine the effect of comparative studies than the p value. Effect sizes and power analysis can mitigate over-reliance on the p value, and offer researchers better informed preparation and interpretation on experiments. Many research fields now require authors to include effect sizes in NHST results, and this trend is expected to be more and more common. In this chapter, I first discuss common misunderstandings of NHST and p value, and how effect sizes can complement them. I then present methods for calculating effect sizes with examples. I also describe another closely related topic, power analysis. Power analysis can be useful for appropriately designing experiments though it is not frequently used in HCI. I present power analysis methods and discuss how they should and should not be used.

References

  1. Cohen J (1998) Statistical power analysis for the behavioral sciences, 2nd edn. Academic Press, New YorkGoogle Scholar
  2. Field A (2005) Statistical hell: effect sizes. http://www.statisticshell.com/docs/effectsizes.pdf
  3. Field A (2009) Discovering statistics using SPSS, 3rd edn. Sage PublicationsGoogle Scholar
  4. Fisher RA (1925) Statistical methods for research workers. Oliver and BoydGoogle Scholar
  5. Hoenig JN, Heisey DM (2001) The abuse of power: the pervasive fallacy of power calculations for data analysis. Am Stat 55:19–24MathSciNetCrossRefGoogle Scholar
  6. Kirk RE (2003) The importance of effect magnitude. In: Davis SF (ed) Handbook of research methods in experimental psychology. Oxford, pp 83–105Google Scholar
  7. Mizumoto A, Takeuchi O (2008) Basics and considerations for reporting effect sizes in research papers. English Language Education Society 31:57–66. (written in Japanese)Google Scholar
  8. Nakagawa S, Cuthill IC (2007) Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol Rev 82(4):591–605CrossRefGoogle Scholar
  9. Newcombe RG (2012) Confidence intervals for proportions and related measures of effect size. CRC Press, Boca RatonGoogle Scholar
  10. Thompson B (2007) Effect sizes, confidence intervals, and confidence intervals for effect sizes. Psychol Schools 44(5):423–432CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.University of TokyoTokyoJapan

Personalised recommendations