Introduction to Bayesian Inference for Psychology

Article

Abstract

We introduce the fundamental tenets of Bayesian inference, which derive from two basic laws of probability theory. We cover the interpretation of probabilities, discrete and continuous versions of Bayes’ rule, parameter estimation, and model comparison. Using seven worked examples, we illustrate these principles and set up some of the technical background for the rest of this special issue of Psychonomic Bulletin & Review. Supplemental material is available via https://osf.io/wskex/.

Keywords

Bayesian inference and parameter estimation Bayesian statistics Tutorial 

References

  1. Bem, D J (2011). Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology, 100, 407–425.PubMedCrossRefGoogle Scholar
  2. Berger, J O, & Wolpert, R L. (1988). The likelihood principle, 2nd edn. Hayward, CA: Institute of Mathematical Statistics.Google Scholar
  3. Consonni, G, & Veronese, P (2008). Compatibility of prior specifications across linear models. Statistical Science, 23, 332–353.CrossRefGoogle Scholar
  4. Cumming, G (2014). The new statistics: Why and how. Psychological Science, 25, 7–29.PubMedCrossRefGoogle Scholar
  5. De Finetti, B. (1974). Theory of probability Vol. 1. New York: Wiley.Google Scholar
  6. Dickey, J M (1971). The weighted likelihood ratio, linear hypotheses on normal location parameters. The Annals of Mathematical Statistics, 42, 204–223.CrossRefGoogle Scholar
  7. Dienes, Z, & McLatchie, N. (this issue). Four reasons to prefer Bayesian over orthodox statistical analyses. Psychonomic Bulletin and Review.Google Scholar
  8. Edwards, W, Lindman, H, & Savage, L J (1963). Bayesian statistical inference for psychological research. Psychological Review, 70, 193–242.CrossRefGoogle Scholar
  9. Efron, B, & Morris, C (1977). Stein’s paradox in statistics. Scientific American, 236, 119–127.CrossRefGoogle Scholar
  10. Etz, A, & Vandekerckhove, J (2016). A Bayesian perspective on the reproducibility project. PLOS ONE, 11, e0149794. doi:10.1371/journal.pone.0149794.PubMedPubMedCentralCrossRefGoogle Scholar
  11. Etz, A, & Wagenmakers, E-J (in press). J. B. S. Haldane’s contribution to the Bayes factor hypothesis test. Statistical Science. Retrieved from arXiv:1511.08180.
  12. Evans, M (2014). Discussion of “On the Birnbaum Argument for the Strong Likelihood Principle”. Statistical Science, 29(2), 242– 246.CrossRefGoogle Scholar
  13. Garthwaite, P H, Kadane, J B, & O’Hagan, A (2005). Statistical methods for eliciting probability distributions. Journal of the American Statistical Association, 100(470), 680–701.CrossRefGoogle Scholar
  14. Gelman, A (2010). Bayesian statistics then and now. Statistical Science, 25(2), 162–165.CrossRefGoogle Scholar
  15. Gelman, A, Carlin, J B, Stern, H S, & Rubin, D B. (2004). Bayesian data analysis, 2nd edn. Boca Raton, FL: Chapman and Hall/CRC.Google Scholar
  16. Gelman, A, & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge: Cambridge University Press.Google Scholar
  17. Haldane, J B S (1932). A note on inverse probability. Mathematical Proceedings of the Cambridge Philosophical Society, 28, 55– 61.CrossRefGoogle Scholar
  18. Hill, B M (1974). On coherence, inadmissibility and inference about many parameters in the theory of least squares. In Fienberg, S. E., & Zellner, A. (Eds.), Studies in Bayesian Econometrics and Statistics (pp. 555–584). North-Holland Amsterdam: Amsterdam.Google Scholar
  19. Hoeting, J A, Madigan, D, Raftery, A E, & Volinsky, C T (1999). Bayesian model averaging: A tutorial. Statistical Science, 14, 382–417.CrossRefGoogle Scholar
  20. Jaynes, E T (1984). The intuitive inadequacy of classical statistics. Epistemologia, 7(43), 43–74.Google Scholar
  21. Jaynes, E T Justice, J. H. (Ed.) (1986). Bayesian methods: General background. Cambridge: Cambridge University Press.Google Scholar
  22. Jeffreys, H (1935). Some tests of significance, treated by the theory of probability. Mathematical Proceedings of the Cambridge Philosophy Society, 31, 203–222.CrossRefGoogle Scholar
  23. Jeffreys, H. (1939). Theory of probability, 1st edn. Oxford, UK: Oxford University Press.Google Scholar
  24. Jeffreys, H. (1961). Theory of probability, 3rd edn. Oxford, UK: Oxford University Press.Google Scholar
  25. Jeffreys, H. (1973). Scientific inference, 3rd edn. Cambridge, UK: Cambridge University Press.Google Scholar
  26. Jern, A, Chang, K-M K, & Kemp, C (2014). Belief polarization is not always irrational. Psychological Review, 121(2), 206.PubMedCrossRefGoogle Scholar
  27. Kass, R E, & Raftery, A E (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795.CrossRefGoogle Scholar
  28. Kruschke, J, & Liddell, T (this issue). The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and planning from a Bayesian perspective. Psychonomic Bulletin and Review.Google Scholar
  29. Leahy, F (1960). Bayes marches on. National Security Agency Technical Journal, 5(1), 49–61.Google Scholar
  30. Lee, M D, & Vanpaemel, W (this issue). Determining informative priors for cognitive models. Psychonomic Bulletin and Review.Google Scholar
  31. Lee, M D, & Wagenmakers, E-J (2013). Bayesian cognitive modeling: A practical course. Cambridge University Press.Google Scholar
  32. Lindley, D V. (1985). Making decisions, 2nd edn. London: Wiley.Google Scholar
  33. Lindley, D V (1993). The analysis of experimental data: The appreciation of tea and wine. Teaching Statistics, 15, 22–25.CrossRefGoogle Scholar
  34. Lindley, D V (2000). The philosophy of statistics. The Statistician, 49, 293–337.Google Scholar
  35. Link, W A, & Barker, R J (2009). Bayes factors and multimodel inference. In Modeling Demographic Processes in Marked Populations (pp. 595–615). Springer.Google Scholar
  36. Ly, A, Verhagen, A J, & Wagenmakers, E-J (2016). Harold Jeffreys’s default Bayes factor hypothesis tests: Explanation, extension, and application in psychology. Journal of Mathematical Psychology, 72, 19–32.CrossRefGoogle Scholar
  37. Marin, J-M, & Robert, C P (2010). On resolving the Savage–Dickey paradox. Electronic Journal of Statistics, 4, 643–654. doi:10.1214/10-EJS564.CrossRefGoogle Scholar
  38. Marsman, M, & Wagenmakers, E-J (2016). Three insights from a Bayesian interpretation of the one-sided P value. Educational and Psychological Measurement.Google Scholar
  39. Matzke, D, Boehm, U, & Vandekerckhove, J. (this issue). Bayesian inference for psychology, part III: Parameter estimation in nonstandard models. Psychonomic Bulletin and Review.Google Scholar
  40. Morey, R D, Hoekstra, R, Rouder, J N, Lee, M D, & Wagenmakers, E-J (2016). The fallacy of placing confidence in confidence intervals. Psychonomic Bulletin and Review, 23(1), 103–123.PubMedCrossRefGoogle Scholar
  41. Morey, R D, Romeijn, J W, & Rouder, J N (2016). The philosophy of Bayes factors and the quantification of statistical evidence. Journal of Mathematical Psychology, 72, 6–18.CrossRefGoogle Scholar
  42. Neyman, J (1977). Frequentist probability and frequentist statistics. Synthese, 36, 97–131.CrossRefGoogle Scholar
  43. Open Science Collaboration, T (2015). Estimating the reproducibility of psychological science. Science, 349, aac4716.CrossRefGoogle Scholar
  44. Development Core Team, R (2004). R: A language and environment for statistical computing [Computer software manual]. Vienna, Austria. Retrieved from http://www.R-project.org.
  45. Raiffa, H, & Schlaifer, R. (1961). Applied statistical decision theory. Cambridge, MA: The MIT Press.Google Scholar
  46. Robert, C P (2007). The Bayesian choice: From decision-theoretic foundations to computational implementation. Springer Science and Business Media.Google Scholar
  47. Robert, C P (2016). The expected demise of the Bayes factor. Journal of Mathematical Psychology, 72, 33–37.Google Scholar
  48. Roberts, H V (1965). Probabilistic prediction. Journal of the American Statistical Association, 60(309), 50–62.CrossRefGoogle Scholar
  49. Rouder, J N (2014). Optional stopping: No problem for Bayesians. Psychonomic Bulletin and Review, 21, 301–308.PubMedCrossRefGoogle Scholar
  50. Rouder, J N, & Morey, R D (2011). A Bayes-factor meta analysis of Bem’s ESP claim. Psychonomic Bulletin and Review, 18, 682–689.PubMedCrossRefGoogle Scholar
  51. Rouder, J N, Morey, R D, & Pratte, M S (in press). Bayesian hierarchical models. In New Handbook of Mathematical Psychology, volume. 1: Measurement and Methodology. Cambridge University Press.Google Scholar
  52. Rouder, J N, Morey, R D, Verhagen, J, Province, J M, & Wagenmakers, E-J (2016). Is there a free lunch in inference?. Topics in Cognitive Science, 8, 520–547.PubMedCrossRefGoogle Scholar
  53. Rouder, J N, & Vandekerckhove, J. (this issue). Bayesian inference for psychology, part IV: Parameter estimation and Bayes factors. Psychonomic Bulletin and Review.Google Scholar
  54. Royall, R M. (1997). Statistical evidence: A likelihood paradigm. London: Chapman and Hall.Google Scholar
  55. Scamander, N A F. (2001). Fantastic beasts and where to find them. London, UK: Obscurus Books.Google Scholar
  56. Spiegelhalter, D J, Best, N G, Carlin, B P, & Van der Linde, A (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society B, 64, 583–639.Google Scholar
  57. The Mathworks, Inc. (2015). MATLAB version R2015a[Computer software manual]. Natick, MA.Google Scholar
  58. Van Ravenzwaaij, D, Cassey, P, & Brown, S. (this issue). A simple introduction to Markov chain Monte-Carlo sampling. Psychonomic Bulletin and Review.Google Scholar
  59. Vandekerckhove, J, Matzke, D, & Wagenmakers, E. -J. (2015). Model comparison and the principle of parsimony. In J. Busemeyer, J. Townsend, Z. J. Wang, & A. Eidels (Eds.), Oxford handbook of computational and mathematical psychology (pp. 300–319). Oxford: University Press.Google Scholar
  60. Vehtari, A, & Ojanen, J (2012). A survey of Bayesian predictive methods for model assessment, selection and comparison. Statistics Surveys, 6, 142–228.CrossRefGoogle Scholar
  61. Verhagen, A J, & Wagenmakers, E.-J. (2014). Bayesian tests to quantify the result of a replication attempt. Journal of Experimental Psychology: General, 143, 1457–1475.CrossRefGoogle Scholar
  62. Wagenmakers, E.-J., Lodewyckx, T, Kuriyal, H, & Grasman, R (2010). Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method. Cognitive Psychology, 60, 158–189.PubMedCrossRefGoogle Scholar
  63. Wagenmakers, E.-J., Love, J, Marsman, M, Jamil, T, Ly, A, Verhagen, J, & Morey, R D. (this issue). Bayesian inference for psychology, part II: Example applications with JASP. Psychonomic Bulletin and Review.Google Scholar
  64. Wagenmakers, E.-J., Marsman, M, Jamil, T, Ly, A, Verhagen, J, Love, J, & Morey, R. (this issue). Bayesian inference for psychology, part I: Theoretical advantages and practical ramifications. Psychonomic Bulletin and Review.Google Scholar
  65. Wagenmakers, E.-J., Morey, R D, & Lee, M D. (in press). Bayesian benefits for the pragmatic researcher. Perspectives on Psychological Science.Google Scholar
  66. Wagenmakers, E.-J., Verhagen, A J, & Ly, A (2016). How to quantify the evidence for the absence of a correlation. Behavior Research Methods, 48, 413–426.PubMedCrossRefGoogle Scholar
  67. Wasserman, L (2000). Bayesian model selection and model averaging. Journal of Mathematical Psychology, 44, 92–107.PubMedCrossRefGoogle Scholar
  68. Winkler, R. (1972). An introduction to Bayesian inference and decision. Winston, New York: Holt, Rinehart.Google Scholar
  69. Wrinch, D, & Jeffreys, H (1919). On some aspects of the theory of probability. Philosophical Magazine, 38, 715–731.CrossRefGoogle Scholar
  70. Wrinch, D, & Jeffreys, H (1921). On certain fundamental principles of scientific inquiry. Philosophical Magazine, 42, 369–390.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2017

Authors and Affiliations

  1. 1.University of CaliforniaIrvineUSA

Personalised recommendations