Skip to main content

Bayesian and frequentist evidence in one-sided hypothesis testing

Abstract

In one-sided testing, Bayesians and frequentists differ on whether or not there is discrepancy between the inference based on the posterior model probability and that based on the p value. We add some arguments to this debate analyzing the discrepancy for moderate and large sample sizes. For small and moderate samples sizes, the discrepancy is measured by the probability of disagreement. Examples of the discrepancy on some basic sampling models indicate the somewhat unexpected result that the probability of disagreement is larger when sampling from models in the alternative hypothesis that are not located at the boundary of the hypotheses. For large sample sizes, we prove that the Bayesian one-sided testing is, under mild conditions, consistent, a property that is not shared by the frequentist procedure. Further, the rate of convergence is \(O(e^{nA})\), where A is a constant that depends on the model from which we are sampling. Consistency is also proved for an extension to multiple hypotheses.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  • Berger JO (2003) Could Fisher, Jeffreys and Neyman have agreed on testing? Stat Sci 18:1–32

    MathSciNet  Article  Google Scholar 

  • Berger JO, Mortera J (1999) Default Bayes factors for one-sided hypothesis testing. J Am Stat Assoc 94:542–554

    Article  Google Scholar 

  • Berger JO, Pericchi LR (1996) The intrinsic Bayes factor for linear models (with discussion). In: Bernardo JM et al (eds) Bayesian statistics, vol 5. Oxford University Press, New York, pp 25–44

    Google Scholar 

  • Berger JO, Sellke T (1987) Testing a point null hypothesis: the irreconcilability of p values and evidence. J Am Stat Assoc 82:112–122

    MathSciNet  MATH  Google Scholar 

  • Casella G, Berger R (1987) Reconciling Bayesian and frequentist evidence in the one-sided testing problem. J Am Stat Assoc 82:106–111

    MathSciNet  Article  Google Scholar 

  • Casella G, Girón FJ, Martínez ML, Moreno E (2009) Consistency of Bayesian procedures for variable selection. Ann Stat 37:1207–1228

    MathSciNet  Article  Google Scholar 

  • Casella G, Moreno E, Girón FJ (2014) Cluster analysis, model selection, and prior distributions on models, Bayesian. Analysis 9:613–658

    MathSciNet  MATH  Google Scholar 

  • Dickey JM (1977) Is the tail area useful as an approximate Bayes factor? J Am Stat Assoc 72:138–142

    MathSciNet  Article  Google Scholar 

  • Dudley RM, Haughton D (1997) Information criteria for multiple data sets and restricted parameters. Stat Sin 7:265–284

    MathSciNet  MATH  Google Scholar 

  • Edwards W, Lindman H, Savage L (1963) Bayesian statistical inference for psychological research. Psychol Rev 70:193–242

    Article  Google Scholar 

  • Efron B, Gous A (2001) Scales of evidence for model selection: Fisher versus Jeffreys. In: Lahiri P (ed) Lecture notes-monograph series, vol 38. Institute of Mathematical Statistics, Hayward, pp 208–246

    Google Scholar 

  • Girón FJ, Martínez ML, Moreno E, Torres F (2006) Objective testing procedures in linear models: calibration of the p-values. Scand J Stat 33:765–784

    MathSciNet  Article  Google Scholar 

  • Kass RE, Vaidyanathan SK (1992) Approximate Bayes factors and orthogonal parameters. With application to testing equality of two binomial proportions. J R Stat Soc Ser B 54:129–144

    MathSciNet  MATH  Google Scholar 

  • Micheas AC, Dey DK (2003) Prior and posterior predictive p-values in the one-side location parameter testing problem. Sankhya Ser A 65:158–178

    MATH  Google Scholar 

  • Moreno E (1997) Bayes factors for intrinsic and fractional priors in nested models, Bayesian robustness. In: Dodge Y (ed) L\(_{1}\) Statistical procedures and related topics, vol 31 of lecture notes-monograph series. Institute of Mathematical Statistics, Hayward, CA, pp 257–270

  • Moreno E (2005) Objective Bayesian analysis for one-sided testing. Test 14:181–198

    MathSciNet  Article  Google Scholar 

  • Moreno E, Girón FJ (2005) Consistency of Bayes factors for linear models. C R Acad Sci Paris Ser I 340:911–914

    Article  Google Scholar 

  • Moreno E, Bertolino F, Racugno W (1998) An intrinsic limiting procedure for model selection and hypothesis testing. J Am Stat Assoc 93:1451–1460

    Article  Google Scholar 

  • Moreno E, Girón FJ, Casella G (2010) Consistency of objective Bayes factors as the model dimension grows. Ann Stat 38:1937–1952

    MathSciNet  Article  Google Scholar 

  • Moreno E, Girón FJ, Casella G (2015) Posterior model consistency in variable selection as the model dimension grows. Stat Sci 30:228–241

    MathSciNet  Article  Google Scholar 

  • Morris C (1987) Comment to Casella and Berger (1987). J Am Stat Assoc 82:112–139

    Article  Google Scholar 

  • Mulder J (2014) Prior adjusted default Bayes factors for testing (in) equality constrained hypotheses. Comput Stat Data Anal 71:448–463

    MathSciNet  Article  Google Scholar 

  • Mulder J, Raftery AE (2019) BIC extensions for order-constrained model selection. Sociol Methods Res. https://doi.org/10.1177/0049124119882459

    Article  Google Scholar 

  • Wang M, Maruyama Y (2016) Consistency of Bayes factor for nonnested model selection when the model dimension grows. Bernoulli 22:2080–2100

    MathSciNet  MATH  Google Scholar 

  • Wilks SS (1963) Mathematical statistics. Wiley, New York

    MATH  Google Scholar 

Download references

Acknowledgements

The first author of this research was supported by the Junta de Andalucía Grant A-FQM-456-UGR18. We are grateful to two anonymous Referees for their comments that lead to improve the presentation of the article.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Elías Moreno.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Moreno, E., Martínez, C. Bayesian and frequentist evidence in one-sided hypothesis testing. TEST 31, 278–297 (2022). https://doi.org/10.1007/s11749-021-00778-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11749-021-00778-8

Keywords

  • Bayesian one-sided test
  • Consistency
  • Decision rule
  • Intrinsic priors
  • p Values
  • Reference priors

Mathematics Subject Classification

  • 62F15
  • 62F03