The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective
In the practice of data analysis, there is a conceptual distinction between hypothesis testing, on the one hand, and estimation with quantified uncertainty on the other. Among frequentists in psychology, a shift of emphasis from hypothesis testing to estimation has been dubbed “the New Statistics” (Cumming 2014). A second conceptual distinction is between frequentist methods and Bayesian methods. Our main goal in this article is to explain how Bayesian methods achieve the goals of the New Statistics better than frequentist methods. The article reviews frequentist and Bayesian approaches to hypothesis testing and to estimation with confidence or credible intervals. The article also describes Bayesian approaches to meta-analysis, randomized controlled trials, and power analysis.
KeywordsNull hypothesis significance testing Bayesian inference Bayes factor Confidence interval Credible interval Highest density interval Region of practical equivalence Meta-analysis Power analysis Effect size Randomized controlled trial Equivalence testing
For comments on previous versions of this article, the authors gratefully acknowledge Geoff Cumming, Zoltan Dienes, Gregory Hickock, Michael D. Lee, Joachim Vandekerckhove, and an anonymous reviewer. Correspondence can be addressed to John K. Kruschke, Department of Psychological and Brain Sciences, Indiana University, 1101 E. 10th St., Bloomington IN 47405-7007, or via electronic mail to email@example.com. More information can be found at http://www.indiana.edu/~kruschke/.
- Allenby, G.M., Bakken, D.G., & Rossi, P.E. (2004). The hierarchical Bayesian revolution: How Bayesian methods have changed the face of marketing research. Marketing Research, 16, 20–25.Google Scholar
- Beaumont, M.A., & Rannala, B. (2004). The Bayesian revolution in genetics. Nature Reviews Genetics.Google Scholar
- Berry, S.M., Carlin, B.P., Lee, J.J., & Müller, P. (2011). Bayesian adaptive methods for clinical trials. Boca Raton, FL: CRC Press.Google Scholar
- Carlin, B.P., & Louis, T.A. (2009). Bayesian methods for data analysis, 3rd edn. Boca Raton, FL: CRC Press.Google Scholar
- Cohen, B.H. (2008). Explaining psychological statistics, 3rd edn. Hoboken, New Jersey: Wiley.Google Scholar
- Cohen, J. (1988). Statistical power analysis for the behavioral sciences, 2nd edn. Hillsdale, NJ: Erlbaum.Google Scholar
- Cohen, J. (1994). The world is round (p<:05), American Psychologist, 997–1003.Google Scholar
- Doyle, A.C. (1890). The sign of four. London: Spencer Blackett.Google Scholar
- Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A., & Rubin, D.B. (2013). Bayesian data analysis third edition,3, Boca Raton, Florida, CRC Press.Google Scholar
- Greenland, S., Senn, S.J., Rothman, K.J., Carlin, J.B., Poole, C., Goodman, S.N., & Altman, D.G. (2016). Statistical tests, p values, confidence intervals, and power: A guide to misinterpretations. The American Statistician. Retrieved from 10.1080/00031305.2016.1154108#tabModule.
- Gregory, P.C. (2001). A Bayesian revolution in spectral analysis, AIP (American Institute of Physics) Conference Proceedings, 568, 557. Retrieved from http://dx.doi.org/10.1063/1.1381917.
- Hartung, J., Knapp, G., & Sinha, B.K. (2008). Bayesian meta-analysis, Statistical Meta-analysis with Applications, 155–170, Hoboken, NJ, Wiley.Google Scholar
- Howell, D.C. (2013). Statistical methods for psychology 8th edition, 8th edn. Wadsworth / Cengage Learning: Belmont, CA.Google Scholar
- Howson, C., & Urbach, P. (2006). Scientific reasoning: The Bayesian approach, 3rd edn. Open Court: Chicago.Google Scholar
- Jeffreys, H. (1961). Theory of probability Oxford. UK: Oxford University Press.Google Scholar
- Kelley, K. (2013). Effect size and sample size planning. In Little, T. D. (Ed.) Oxford Handbook of Quantitative Methods (Vols. Volume 1, Foundations, pp. 206–222). New York: Oxford University Press.Google Scholar
- Kruschke, J.K. (2011b). Doing Bayesian data analysis: A tutorial with R and BUGS. Burlington, MA: Academic Press / Elsevier.Google Scholar
- Kruschke, J.K. (2015). Doing Bayesian data analysis, Second Edition: A tutorial with R, JAGS, and Stan. Burlington, MA: Academic Press / Elsevier.Google Scholar
- Kruschke, J.K., & Liddell, T.M. (2015). Bayesian data analysis for newcomers. (in preparation).Google Scholar
- Kruschke, J.K., & Vanpaemel, W. (2015). Bayesian estimation in hierarchical models. In Busemeyer, J. R., Townsend, J. T., Wang, Z. J., & Eidels, A. (Eds.) Oxford Handbook of Computational and Mathematical Psychology: Oxford University Press.Google Scholar
- Lee, M.D., & Wagenmakers, E. -J. (2014). Bayesian cognitive modeling: A practical course, Cambridge, England, Cambridge University Press.Google Scholar
- Liddell, T.M., & Kruschke, J.K. (2014). Ostracism and fines in a public goods game with accidental contributions: The importance of punishment type. Judgment and Decision Making, 9(6), 523– 547.Google Scholar
- Lunn, D., Jackson, C., Best, N., Thomas, A., & Spiegelhalter, D. (2013). The BUGS book: A practical introduction to Bayesian analysis. Boca Raton, Florida: CRC Press.Google Scholar
- Maxwell, S.E., & Delaney, H.D. (2004). Designing experiments and analyzing data: A model comparison perspective, 2nd edn. Mahwah, NJ: Erlbaum.Google Scholar
- Mayo, D.G. (2016). Don’t throw out the error control baby with the bad statistics bathwater: A commentary. The American Statistician. Retrieved from 10.1080/00031305.2016.1154108#tabModule.
- Mayo, D.G., & Spanos, A. (2011). Error statistics. In Bandyopadhyay, P.S., & Forster, M.R. (Eds.) Handbook of the Philosophy of Science. Volume 7: Philosophy of Statistics (pp. 153–198): Elsevier.Google Scholar
- McGrayne, S.B. (2011). The theory that would not die, Yale University Press.Google Scholar
- Meehl, P.E. (1997). The problem is epistemology, not statistics: Replace significance tests by confidence intervals and quantify accuracy of risky numerical predictions, What if there Were no Significance Tests, 395–425, Mahwah, NJ, Erlbaum Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (Eds.)Google Scholar
- Morey, R.D., Rouder, J.N., & Jamil, T. (2015). BayesFactor package for R. http://cran.r-project.org/web/packages/BayesFactor/index.html.
- Pitchforth, J.O., & Mengersen, K.L. (2013). Bayesian meta-analysis, Case Studies in Bayesian Statistical Modelling and Analysis Alston, C.L., Mengersen, K.L., & Pettitt, A.N. (Eds.), Wiley.Google Scholar
- Plummer, M. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling, Proceedings of the 3rd International Workshop on Distributed Statistical Computing (dsc 2003), Vienna, Austria, ISSN 1609-395X.Google Scholar
- Plummer, M. (2012). JAGS version 3.3.0 user manual [Computer software manual].Google Scholar
- Rothman, K.J. (2016). Disengaging from statistical significance. The American Statistician. Retrieved from 10.1080/00031305.2016.1154108#tabModule .
- Schönbrodt, F. D., Wagenmakers, E.-J., Zehetleitner, M., & Perugini, M. (2016). Sequential hypothesis testing with Bayes factors: Efficiently testing mean differences, Psychological Methods. http://dx.doi.org/10.1037/met0000061.
- Serlin, R.C., & Lapsley, D.K. (1993) In Keren, G., & Lewis, C. (Eds.), Rational appraisal of psychological research and the good enough principle, (pp. 199–228). Hillsdale, NJ: Erlbaum.Google Scholar
- Singh, K., Xie, M., & Strawderman, W.E. (2007). Confidence distribution (CD) distribution estimator of a parameter, Complex Datasets and Inverse Problems, 54, 132–150, Beachwood, OH, Institute of Mathematical Statistics Liu, R., & et al. (Eds.)Google Scholar
- Stan Development Team (2012). Stan: A C++ library for probability and sampling, version 1.1. Retrieved from http://mc-stan.org/citations.html.
- Westlake, W.J. (1981). Response to bioequivalence testing −− a need to rethink. Biometrics, 37, 591–593.Google Scholar
- Woodworth, G. (2004). Biostatistics: A Bayesian introduction, Wiley.Google Scholar