Bayesian Versus Frequentist Inference

  • Eric-Jan Wagenmakers
  • Michael Lee
  • Tom Lodewyckx
  • Geoffrey J. Iverson
Part of the Statistics for Social and Behavioral Sciences book series (SSBS)

Abstract

Throughout this book, the topic of order restricted inference is dealt with almost exclusively from a Bayesian perspective. Some readers may wonder why the other main school for statistical inference – frequentist inference – has received so little attention here. Isn’t it true that in the field of psychology, almost all inference is frequentist inference?

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Abelson, R.P.: On the surprising longevity of flogged horses: Why there is a case for the significance test. Psychological Science, 8, 12–15 (1997)CrossRefGoogle Scholar
  2. [2]
    Anscombe, F.J.: Sequential medical trials. Journal of the American Statistical Association, 58, 365–383 (1963)CrossRefMathSciNetGoogle Scholar
  3. [3]
    Bakan, D.: The test of significance in psychological research. Psychological Bulletin, 66, 423–437 (1966)CrossRefGoogle Scholar
  4. [4]
    Barnard, G.A.: The meaning of a significance level. Biometrika, 34, 179–182 (1947)MATHMathSciNetGoogle Scholar
  5. [5]
    Basu, D.: On the elimination of nuisance parameters. Journal of the American Statistical Association, 72, 355–366 (1977)MATHCrossRefMathSciNetGoogle Scholar
  6. [6]
    Batchelder, W.H.: Cognitive psychometrics: Combining two psychological traditions. CSCA Lecture, Amsterdam, The Netherlands, October 2007.Google Scholar
  7. [7]
    Berger, J.O.: Statistical Decision Theory and Bayesian Analysis (2nd ed.). New York, Springer (1985)MATHGoogle Scholar
  8. [8]
    Berger, J.O.: Robust Bayesian analysis: Sensitivity to the prior. Journal of Statistical Planning and Inference, 25, 303–328 (1990)MATHCrossRefMathSciNetGoogle Scholar
  9. [9]
    Berger, J.O.: Could Fisher, Jeffreys and Neyman have agreed on testing? Statistical Science, 18, 1–32 (2003)MATHCrossRefMathSciNetGoogle Scholar
  10. [10]
    Berger, J.O., Berry, D.A.: The relevance of stopping rules in statistical inference. In: Gupta, S.S., Berger, J.O. (eds) Statistical Decision Theory and Related Topics: Vol. 1. New York, Springer (1988)Google Scholar
  11. [11]
    Berger, J.O., Berry, D.A.: Statistical analysis and the illusion of objectivity. American Scientist, 76, 159–165 (1988)Google Scholar
  12. [12]
    Berger, J.O., Liseo, B., Wolpert, R.L.: Integrated likelihood methods for eliminating nuisance parameters. Statistical Science, 14, 1–28 (1999)MATHCrossRefMathSciNetGoogle Scholar
  13. [13]
    Berger, J.O., Pericchi, L.R.: The intrinsic Bayes factor for model selection and prediction. Journal of the American Statistical Association, 91, 109–122 (1996)MATHCrossRefMathSciNetGoogle Scholar
  14. [14]
    Berger, J.O., Wolpert, R.L.: The Likelihood Principle. Institute of Mathematical Statistics (2nd ed.), Hayward, CA (1988)Google Scholar
  15. [15]
    Bernardo, J.M., Smith, A.F.M.: Bayesian Theory. New York, Wiley (1994)MATHCrossRefGoogle Scholar
  16. [16]
    Bowers, J.S., Vigliocco, G., Haan, R.: Orthographic, phonological, and articulatory contributions to masked letter and word priming. Journal of Experimental Psychology: Human Perception and Performance, 24, 1705–1719 (1998)CrossRefGoogle Scholar
  17. [17]
    Burdette, W.J., Gehan, E.A.: Planning and Analysis of Clinical Studies. Charles C. Springfield, IL, Thomas (1970)Google Scholar
  18. [18]
    Christensen, R.: Testing Fisher, Neyman, Pearson, and Bayes. The American Statistician, 59, 121–126 (2005)CrossRefMathSciNetGoogle Scholar
  19. [19]
    Cox, D.R.: Some problems connected with statistical inference. The Annals of Mathematical Statistics, 29, 357–372 (1958)MATHCrossRefGoogle Scholar
  20. [20]
    Cox, R.T.: Probability, frequency and reasonable expectation. American Journal of Physics, 14, 1–13 (1946)MATHCrossRefMathSciNetGoogle Scholar
  21. [21]
    Dawid, A.P.: Statistical theory: The prequential approach. Journal of the Royal Statistical Society, Series A, 147, 278–292 (1984)MATHCrossRefMathSciNetGoogle Scholar
  22. [22]
    De Finetti, B.: Theory of Probability, Vols. 1 and 2. New York, Wiley (1974)Google Scholar
  23. [23]
    DeGroot, M.-H.: Optimal Statistical Decisions. New York, McGraw-Hill (1970)MATHGoogle Scholar
  24. [24]
    Dennis, S., Humphreys, M.S.: A context noise model of episodic word recognition. Psychological Review, 108, 452–477 (2001)CrossRefGoogle Scholar
  25. [25]
    Dickey, J.M.: Scientific reporting and personal probabilities: Student’s hypothesis. Journal of the Royal Statistical Society, Series B, 35, 285–305 (1973)MathSciNetGoogle Scholar
  26. [26]
    Edwards, W., Lindman, H., Savage, L.J.: Bayesian statistical inference for psychological research. Psychological Review, 70, 193–242 (1963)CrossRefGoogle Scholar
  27. [27]
    Efron, B.: Why isn’t everyone a Bayesian? The American Statistician, 40, 1–5 (1986)MATHCrossRefMathSciNetGoogle Scholar
  28. [28]
    Estes, W.K.: The problem of inference from curves based on group data. Psychological Bulletin, 53, 134–140 (1956)CrossRefGoogle Scholar
  29. [29]
    Estes, W.K.: Traps in the route to models of memory and decision. Psychonomic Bulletin & Review, 9, 3–25 (2002)Google Scholar
  30. [30]
    Fishburn, P.C.: The axioms of subjective probability. Statistical Science, 1, 335–345 (1986)CrossRefMathSciNetGoogle Scholar
  31. [31]
    Fisher, R.A.: Statistical Methods for Research Workers (5th ed.). London, Oliver and Boyd (1934)Google Scholar
  32. [32]
    Fisher, R.A.: Statistical Methods for Research Workers (13th ed.). New York, Hafner (1958)Google Scholar
  33. [33]
    Forster, K.I., Mohan, K., Hector, J.: The mechanics of masked priming. In: Kinoshita, S., Lupker, S.J. (eds) Masked Priming: The State of the Art. New York, Psychology Press (2003)Google Scholar
  34. [34]
    Galavotti, M.C.: A Philosophical Introduction to Probability. Stanford, CA, CSLI Publications (2005)Google Scholar
  35. [35]
    Gelfand, A.E., Smith, A.F.M., Lee, T. M.: Bayesian analysis of constrained parameter and truncated data problems using Gibbs sampling. Journal of the American Statistical Association, 87, 523–532 (1992)CrossRefMathSciNetGoogle Scholar
  36. [36]
    Gelman, A., Hill, J.: Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge, Cambridge University Press (2007)Google Scholar
  37. [37]
    Gigerenzer, G.: The superego, the ego, and the id in statistical reasoning. In: Keren, G., Lewis, C. (eds) A Handbook for Data Analysis in the Behavioral Sciences: Methodological Issues. Hillsdale, NJ, Erlbaum (1993)Google Scholar
  38. [38]
    Gigerenzer, G.: We need statistical thinking, not statistical rituals. Behavioral and Brain Sciences, 21, 199–200 (1998)CrossRefGoogle Scholar
  39. [39]
    Gigerenzer, G.: Mindless statistics. The Journal of Socio–Economics, 33, 587–606 (2004)Google Scholar
  40. [40]
    Gigerenzer, G., Krauss, S., Vitouch, O.: The null ritual: What you always wanted to know about significance testing but were afraid to ask. In: Kaplan, D. (ed) The Sage Handbook of Quantitative Methodology for the Social Sciences. Thousand Oaks, CA, Sage (2004)Google Scholar
  41. [41]
    Gill, J.: Bayesian Methods: A Social and Behavioral Sciences Approach. Boca Raton, FL, CRC Press (2002).MATHGoogle Scholar
  42. [42]
    Good, I.J.: Weight of evidence: A brief survey. In: Bernardo, J.M., DeGroot, M.-H., Lindley, D.V., Smith, A.F.M. (eds) Bayesian Statistics 2. New York, Elsevier (1985)Google Scholar
  43. [43]
    Goodman, S.N.: P values, hypothesis tests, and likelihood: Implications for epidemiology of a neglected historical debate. American Journal of Epidemiology, 137, 485–496 (1993)Google Scholar
  44. [44]
    Haller, H., Krauss, S.: Misinterpretations of significance: A problem students share with their teachers? Methods of Psychological Research, 7, 1–20 (2002)Google Scholar
  45. [45]
    Heathcote, A., Brown, S., Mewhort, D.J.K.: The power law repealed: The case for an exponential law of practice. Psychonomic Bulletin & Review, 7, 185–207 (2000)Google Scholar
  46. [46]
    Hoffman, L., Rovine, M.J.: Multilevel models for the experimental psychologist: Foundations and illustrative examples. Behavior Research Methods, 39, 101–117 (2007)Google Scholar
  47. [47]
    Hoijtink, H.: Confirmatory latent class analysis: Model selection using Bayes factors and (pseudo) likelihood ratio statistics. Multivariate Behavioral Research, 36, 563–588 (2001)CrossRefGoogle Scholar
  48. [48]
    Howson, C., Urbach, P.: Scientific Reasoning: The Bayesian Approach (3rd ed.). Chicago, Open Court (2006)Google Scholar
  49. [49]
    Hubbard, R., Bayarri, M.J.: Confusion over measures of evidence (p’s) versus errors (α’s) in classical statistical testing. The American Statistician, 57, 171–182 (2003)CrossRefMathSciNetGoogle Scholar
  50. [50]
    Huntjens, R.J.C., Peters, M.L., Woertman, L., Bovenschen, L.M., Martin, R.C., Postma, A.: Inter-identity amnesia in dissociative identity disorder: A simulated memory impairment? Psychological Medicine, 36, 857–863 (2006)CrossRefGoogle Scholar
  51. [51]
    Jaynes, E.T.: Prior probabilities. IEEE Transactions on Systems Science and Cybernetics, 4, 227–241 (1968)CrossRefGoogle Scholar
  52. [52]
    Jaynes, E.T.: Confidence intervals vs Bayesian intervals. In: Harper, W.L., Hooker, C.A. (eds) Foundations of Probability Theory, Statistical Inference, and Statistical Theories of Science, Vol. 2. Dordrecht, Reidel (1976)Google Scholar
  53. [53]
    Jaynes, E.T.: Probability Theory: The Logic of Science. Cambridge, Cambridge University Press (2003)MATHGoogle Scholar
  54. [54]
    Jeffreys, H.: On the relation between direct and inverse methods in statistics. Proceedings of the Royal Society of London. Series A: Mathematical and Physical Sciences, 160, 325–348 (1937)Google Scholar
  55. [55]
    Jeffreys, H.: Theory of Probability. Oxford, Oxford University Press (1961)MATHGoogle Scholar
  56. [56]
    Kass, R.E., Raftery, A.E.: Bayes factors. Journal of the American Statistical Association, 90, 377–395 (1995)Google Scholar
  57. [57]
    Kass, R.E., Wasserman, L.: A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. Journal of the American Statistical Association, 90, 928–934 (1995)MATHCrossRefMathSciNetGoogle Scholar
  58. [58]
    Kass, R.E., Wasserman, L.: The selection of prior distributions by formal rules. Journal of the American Statistical Association, 91, 1343–1370 (1996)MATHCrossRefGoogle Scholar
  59. [59]
    Klugkist, I., Kato, B., Hoijtink, H.: Bayesian model selection using encompassing priors. Statistica Neerlandica, 59, 57–69 (2005)MATHCrossRefMathSciNetGoogle Scholar
  60. [60]
    Klugkist, I., Laudy, O., Hoijtink, H.: Inequality constrained analysis of variance: A Bayesian approach. Psychological Methods, 10, 477–493 (2005)CrossRefGoogle Scholar
  61. [61]
    Klugkist, I., Laudy, O., Hoijtink, H.: Bayesian eggs and Bayesian omelettes: Reply to Stern (2005). Psychological Methods, 10, 500–503 (2005)CrossRefGoogle Scholar
  62. [62]
    Laudy, O., Zoccolillo, M., Baillargeon, R.H., Boom, J., Tremblay, R.E., Hoijtink, H.: Applications of confirmatory latent class analysis in developmental psychology. European Journal of Developmental Psychology, 2, 1–15 (2005)CrossRefGoogle Scholar
  63. [63]
    Lee, M.D., Webb, M.R.: Modeling individual differences in cognition. Psychonomic Bulletin & Review, 12, 605–621 (2005)Google Scholar
  64. [64]
    Lindley, D. V.: A statistical paradox. Biometrika, 44, 187–192 (1957)MATHMathSciNetGoogle Scholar
  65. [65]
    Lindley, D.V.: Bayesian Statistics, a Review. Philadelphia, PA, SIAM (1972)Google Scholar
  66. [66]
    Lindley, D.V.: Scoring rules and the inevitability of probability. International Statistical Review, 50, 1–26 (1982)MATHCrossRefMathSciNetGoogle Scholar
  67. [67]
    Lindley, D.V.: The analysis of experimental data: The appreciation of tea and wine. Teaching Statistics, 15, 22–25 (1993)CrossRefGoogle Scholar
  68. [68]
    Lindley, D.V.: The philosophy of statistics. The Statistician, 49, 293–337 (2000)Google Scholar
  69. [69]
    Lindley, D.V., Scott, W.F.: New Cambridge Elementary Statistical Tables. London, Cambridge University Press (1984)MATHGoogle Scholar
  70. [70]
    MacKay, D.J.C.: Information Theory, Inference, and Learning Algorithms. Cambridge, Cambridge University Press (2003)MATHGoogle Scholar
  71. [71]
    Morey, R.D., Pratte, M.S., Rouder, J.N.: Problematic effects of aggregation in zROC analysis and a hierarchical modeling solution. Journal of Mathematical Psychology (in press)Google Scholar
  72. [72]
    Morey, R.D., Rouder, J.N., Speckman, P.L.: A statistical model for discriminating between subliminal and near-liminal performance. Journal of Mathematical Psychology, 52, 21–36 (2008)MATHCrossRefMathSciNetGoogle Scholar
  73. [73]
    Myung, I.J., Forster, M.R., Browne, M.W.: Model selection [Special issue]. Journal of Mathematical Psychology, 44(1–2) (2000)CrossRefGoogle Scholar
  74. [74]
    Navarro, D.J., Griffiths, T.L., Steyvers, M., Lee, M.D.: Modeling individual differences using Dirichlet processes. Journal of Mathematical Psychology, 50, 101–122 (2006)MATHCrossRefMathSciNetGoogle Scholar
  75. [75]
    Nelson, N., Rosenthal, R., Rosnow, R.L.: Interpretation of significance levels and effect sizes by psychological researchers. American Psychologist, 41, 1299–1301 (1986)CrossRefGoogle Scholar
  76. [76]
    Neyman, J., Pearson, E.S.: On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society A, 231, 289–337 (1933)MATHCrossRefGoogle Scholar
  77. [77]
    O’Hagan, A.: Fractional Bayes factors for model comparison. Journal of the Royal Statistical Society, Series B, 57, 99–138 (1997)MathSciNetGoogle Scholar
  78. [78]
    O’Hagan, A.: Dicing with the unknown. Significance, 1, 132–133 (2004)CrossRefMathSciNetGoogle Scholar
  79. [79]
    O’Hagan, A., Forster, J.: Kendall’s Advanced Theory of Statistics Vol. 2B: Bayesian Inference (2nd ed.). London, Arnold (2004)Google Scholar
  80. [80]
    Peto, R., Pike, M.C., Armitage, P., Breslow, N.E., Cox, D.R., Howard, S.V., Mantel, N., McPherson, K., Peto, J., Smith, P.G.: Design and analysis of randomized clinical trials requiring prolonged observation of each patient, I: Introduction and design. British Journal of Cancer, 34, 585–612 (1976)Google Scholar
  81. [81]
    Pocock, S.J.: Group sequential methods in the design and analysis of clinical trials. Biometrika, 64, 191–199 (1977)CrossRefGoogle Scholar
  82. [82]
    Raftery, A.E.: Bayesian model selection in social research. In: Marsden, P.V. (ed) Sociological Methodology. Cambridge, Blackwells (1995)Google Scholar
  83. [83]
    Ramsey, F.P.: Truth and probability. In: Braithwaite, R.B. (ed) The Foundations of Mathematics and Other Logical Essays. London, Kegan Paul (1926)Google Scholar
  84. [84]
    Rosenthal, R., Gaito, J.: The interpretation of levels of significance by psychological researchers. The Journal of Psychology, 55, 33–38 (1963)Google Scholar
  85. [85]
    Rosnow, R.L., Rosenthal, R.: Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276–1284 (1989)CrossRefGoogle Scholar
  86. [86]
    Rouder, J.N., Lu, J.: An introduction to Bayesian hierarchical models with an application in the theory of signal detection. Psychonomic Bulletin & Review, 12, 573–604 (2005)Google Scholar
  87. [87]
    Rouder, J.N., Lu, J., Morey, R.D., Sun, D., Speckman, P.L.: A hierarchical process dissociation model. Journal of Experimental Psychology: General (in press)Google Scholar
  88. [88]
    Rouder, J.N., Lu, J., Speckman, P.L., Sun, D., Jiang, Y.: A hierarchical model for estimating response time distributions. Psychonomic Bulletin & Review, 12, 195–223 (2005)Google Scholar
  89. [89]
    Rouder, J.N., Lu, J., Sun, D., Speckman, P., Morey, R., Naveh-Benjamin, M.: Signal detection models with random participant and item effects. Psychometrika (in press)Google Scholar
  90. [90]
    Royall, R.: The effect of sample size on the meaning of significance tests. The American Statistician, 40, 313–315 (1986)MATHCrossRefGoogle Scholar
  91. [91]
    Royall, R.M.: Statistical Evidence: A Likelihood Paradigm. London, Chapman & Hall (1997)MATHGoogle Scholar
  92. [92]
    Savage, L.J.: The Foundations of Statistics. New York, Wiley (1954)MATHGoogle Scholar
  93. [93]
    Savage, L.J.: The foundations of statistics reconsidered. In: Neyman, J. (ed) Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1. Berkely, CA, University of California Press (1961)Google Scholar
  94. [94]
    Smith, A.F.M., Roberts, G.O.: Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods. Journal of the Royal Statistical Society, Series B, 55, 3–23 (1993)MATHMathSciNetGoogle Scholar
  95. [95]
    Spiegelhalter, D.J., Thomas, A., Best, N., Lunn, D.: WinBUGS Version 1.4 User Manual. Medical Research Council Biostatistics Unit, Cambridge (2003)Google Scholar
  96. [96]
    Stuart, A., Ord, J.K., Arnold, S.: Kendall’s Advanced Theory of Statistics Vol. 2A: Classical Inference & the Linear Model (6th ed.). London, Arnold (1999)Google Scholar
  97. [97]
    Wagenmakers, E.-J.: A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14, 779–804 (2007)Google Scholar
  98. [98]
    Wagenmakers, E.-J., Grünwald, P.: A Bayesian perspective on hypothesis testing. Psychological Science, 17, 641–642 (2006)CrossRefGoogle Scholar
  99. [99]
    Wagenmakers, E.-J., Grünwald, P., Steyvers, M.: Accumulative prediction error and the selection of time series models. Journal of Mathematical Psychology, 50, 149–166 (2006)MATHCrossRefMathSciNetGoogle Scholar
  100. [100]
    Wagenmakers, E.-J., Waldorp, L.: Model selection: Theoretical developments and applications [Special issue]. Journal of Mathematical Psychology, 50, 99–214 (2006)CrossRefMathSciNetGoogle Scholar
  101. [101]
    Wasserman, L.: All of Statistics: A Concise Course in Statistical Inference. New York, Springer (2004)MATHGoogle Scholar
  102. [102]
    Wilkinson, L., the Task Force on Statistical Inference: Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594–604 (1999)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Eric-Jan Wagenmakers
    • 1
  • Michael Lee
    • 2
  • Tom Lodewyckx
    • 3
  • Geoffrey J. Iverson
    • 2
  1. 1.Department of PsychologyUniversity of AmsterdamRoetersstraat 15Amsterdamthe Netherlands
  2. 2.Department of Cognitive SciencesUniversity of California at IrvineIrvineUSA
  3. 3.Department of Quantitative and Personality PsychologyUniversity of LeuvenBelgium

Personalised recommendations