Advertisement

Confidence Intervals vs Bayesian Intervals

  • E. T. Jaynes
  • Oscar Kempthorne
Chapter
Part of the The University of Western Ontario Series in Philosophy of Science book series (WONS, volume 6b)

Abstract

For many years, statistics textbooks have followed this ‘canonical’ procedure: (1) the reader is warned not to use the discredited methods of Bayes and Laplace, (2) an orthodox method is extolled as superior and applied to a few simple problems, (3) the corresponding Bayesian solutions are not worked out or described in any way. The net result is that no evidence whatsoever is offered to substantiate the claim of superiority of the orthodox method.

To correct this situation we exhibit the Bayesian and orthodox solutions to six common statistical problems involving confidence intervals (including significance tests based on the same reasoning). In every case, we find that the situation is exactly the opposite; i.e., the Bayesian method is easier to apply and yields the same or better results. Indeed, the orthodox results are satisfactory only when they agree closely (or exactly) with the Bayesian results. No contrary example has yet been produced.

By a refinement of the orthodox statistician’s own criterion of performance, the best confidence interval for any location or scale parameter is proved to be the Bayesian posterior probability interval. In the cases of point estimation and hypothesis testing, similar proofs have long been known. We conclude that orthodox claims of superiority are totally unjustified; today, the original statistical methods of Bayes and Laplace stand in a position of proven superiority in actual performance, that places them beyond the reach of mere ideological or philosophical attacks. It is the continued teaching and use of orthodox methods that is in need of justification and defense.

Keywords

Common Sense Bayesian Method Bayesian Solution Bayesian Test Ancillary Statistic 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bayes, Rev. Thomas, ‘An Essay Toward Solving a Problem in the Doctrine of Chances’, Phil. Trans. Roy. Soc. 330–418 (1763). Reprint, with biographical note by G. A. Barnard, in Biometrika 45, 293–315 (1958) and in Studies in the History of Statistics and Probability, E. S. Pearson and M. G. Kendall, (eds), C. Griffin and Co. Ltd., London, (1970). Also reprinted in Two Papers by Bayes with Commentaries, ( W. E. Deming, ed.), Hafner Publishing Co., New York, (1963).Google Scholar
  2. Birnbaum, Allen, ‘On the Foundations of Statistical Inference’, J. Am. Stat. Ass’n 57 269 (1962).Google Scholar
  3. Crow, E. L., Davis, F. A., and Maxfield, M. W., Statistics Manual, Dover Publications, Inc., New York (1960).Google Scholar
  4. Fisher, R. A., Statistical Methods and Scientific Inference, Hafner Publishing Co., New York (1956).Google Scholar
  5. Friedman, K. and Shimony, A., ‘Jaynes’ Maximum Entropy Prescription and Probability Theory’,. Stat. Phys 3, 381–384 (1971).CrossRefGoogle Scholar
  6. Good, I. J., Probability and The Weighing of Evidence, C. Griffin and Co. Ltd., London (1950).Google Scholar
  7. Good, I. J., The Estimation of Probabilities, Research Monograph #30, The MIT Press, Cambridge, Mass. (1965); paperback edition, 1968.Google Scholar
  8. Jaynes, E. T Probability Theory in Science and Engineering, No. 4 of Colloquium Lectures on Pure and Applied Science, Socony-Mobil Oil Co., Dallas, Texas Google Scholar
  9. Jaynes, E. T., ‘New Engineering Applications of Information Theory’, in Engineering Uses of Random Function Theory and Probability, J. L. Bogdanoff and F. Kozin, (eds.), J. Wiley & Sons, Inc., N.Y. (1963); pp. 163–203.Google Scholar
  10. Jaynes, E. T., ‘Information Theory and Statistical Mechanics’, in Statistical Physics, K. W. Ford, (ed.), W. A. Benjamin, Inc., (1963); pp. 181–218.Google Scholar
  11. Jaynes, E. T., ‘Gibbs vs. Boltzmann Entropies’, Am. J. Phys 33, 391 (1965).CrossRefGoogle Scholar
  12. Jaynes, E. T. T., ‘Foundations of Probability Theory and Statistical Mechanics’, Chap. 6 in Delaware Seminar in Foundations of Physics, M. Bunge, (ed.), Springer-Verlag, Berlin (1967); Spanish translation in Modern Physics, David Webber, (ed.), Alianza Editorial s/a, Madrid 33 (1973).Google Scholar
  13. Jaynes, E. T., ‘The Well-Posed Problem’, in Foundations of Statistical Inference, V. P. Godambe and D. A. Sprott, (eds.), Holt, Rinehart and Winston of Canada, Toronto (1971).Google Scholar
  14. Jaynes, E. T., ‘Survey of the Present Status of Neoclassical Radiation Theory’, in Coherence and Quantum Optics, L. Mandel and E. Wolf, (eds.), Plenum Publishing Corp., New York (1973), pp. 35–81.Google Scholar
  15. Jeffreys, H., Theory of Probability, Oxford University Press (1939).Google Scholar
  16. Jeffreys, H., Scientific Inference, Cambridge University Press (1957).Google Scholar
  17. Kempthorne, O., ‘Probability, Statistics, and the Knowledge Business’, in Foundations of Statistical Inference, V. P. Godambe and D. A. Sprott, (eds.), Holt, Rinehart and Winston of Canada, Toronto (1971).Google Scholar
  18. Kendall, M. G. and Stuart, A., The Advanced Theory of Statistics Volume 2, C. Griffin and Co., Ltd., London C1961).Google Scholar
  19. Lehmann, E. L., Testing Statistical Hypotheses, J. Wiley & Sons, Inc., New York, (1959) p. 62.Google Scholar
  20. Pearson, E. S., Discussion in Savage (1962); p. 57.Google Scholar
  21. Roberts, Norman A., Mathematical Methods in Reliability Engineering, McGraw-Hill Book Co., Inc., New York (1964) pp. 86–88.Google Scholar
  22. Rowlinson, J. S., ‘Probability, Information and Entropy’, Nature 225, 1196–1198 (1970).CrossRefGoogle Scholar
  23. Savage, L. J., The Foundations of Statistics, John Wiley, & Sons, Inc., New York (1954).Google Scholar
  24. Savage, L. J., The Foundations of Statistical Inference, John Wiley & Sons, Inc., New York (1962).Google Scholar
  25. Schlaifer, R Probability and Statistics for Business Decisions, McGraw-Hill Book Co. Inc., New York (1959).Google Scholar
  26. Sobel, M. and Tischendorf, J. A., Proc. Fifth Nat’l Symposium on Reliability andGoogle Scholar
  27. Quality Control, I.R.E., pp. 108–118 (1959).Google Scholar
  28. Smith, C. A. B., Discussion in Savage (1962); p. 60.Google Scholar
  29. Cornfield, J., 1969, ‘The Bayesian Outlook and its Applications’, Biometrics 25, 617–642, for a good listing of background material.Google Scholar
  30. Dawid, A. P., M. Stone and J. V. Sidek, 1973, ‘Marginalization Paradoxes in Bayesian and Structural Inference’, J. Roy. Stat. Soc. B in press.Google Scholar
  31. Dawid, A. P. and M. Stone, 1972, ‘Expectation Consistency of Inverse Probability Distributions’, Biometrika 59, 486–489.CrossRefGoogle Scholar
  32. Easterling, R. G., 1972, ‘A Personal View of the Bayesian Controversy in Reliability and Statistics’, IEEE Trans. R-21, 186–194.Google Scholar
  33. Fisher, R. A., 1959, Statistical Methods and Scientific Inference, 2nd ed., Hafner, New York.Google Scholar
  34. Kalbfleisch, J. G., 1971, Probability and Statistical Methods, Dept. of Statistics, Waterloo.Google Scholar
  35. Kalbfleisch, J. G., and D. A. Sprott, 1973, ‘On Tests of Significance’, this volume, p. 259.Google Scholar
  36. Kempthorne, O., 1969, Biometrics 25, 647–654. Discussion of paper by J. Cornfield.Google Scholar
  37. Kempthorne, O., 1971, ‘Probability, Statistics, and the Knowledge Business’, in Foundations of Statistical Inference, V. P. Godambe, and D. A. Sprott, (j, Holt, Rinehart and Winston, New York; 1971.Google Scholar
  38. Kempthorne, O., 1972, ‘Theories of Inference and Data Analysis’, in Statistical Papers in Honor of George W. Snedecor, Iowa State Univ. Press, Ames.Google Scholar
  39. Kempthorne, O. and J. L. Folks, 1971, Probability, Statistics, and Data Analysis, Iowa State Univ. Press, Ames.Google Scholar
  40. Luce, R. D. and H. Raiffa, 1957, Games and Decisions, Wiley, New York.Google Scholar
  41. Ramsey, F. P., 1926, ‘Truth and Probability’, in The Foundations of Mathematics and Other Logical Essays, Kegan, London.Google Scholar
  42. Savage, L. J., 1954, The Foundations of Statistics, Wiley, New York.Google Scholar
  43. Snedecor, G. W. and W. G. Cochran, 1967, Statistical Methods, 6th ed., Iowa State Univ. Press, Ames.Google Scholar
  44. Stone, M. and A. P. Dawid, 1972, ‘Un-Bayesian Implications of Improper Bayes Inference in Routine Statistical Problems’, Biometrika 59, 369–375.Google Scholar
  45. Barnard, G. A., ‘Comments on Stein’s “ Remark on the Likelihood Principle”Google Scholar
  46. J. Roy. Stat. Soc. (A) 125 569 (1962) Google Scholar
  47. Cox, R. T., Am. J. Phys 17, 1 (1946).CrossRefGoogle Scholar
  48. Cox, R. T., The Algebra of Probable Inference, Johns Hopkins University Press, 1961; Reviewed by E. T. Jaynes, Am. J. Phys 31, 66 (1963).CrossRefGoogle Scholar
  49. Deming, W. E., Statistical Adjustment of Data, J. Wiley, New York (1943).Google Scholar
  50. Fisher, R. A., Contributions to Mathematical Statistics, W. A. Shewhart, (ed.), J. Wiley and Sons, Inc. New York (1950); Referred to above as ‘Collected Works’.Google Scholar
  51. Fisher, R. A., Statistical Methods and Scientific Inference, Hafner Publishing Co., New York (1956).Google Scholar
  52. Fisher, R. A., Statistical Methods for Research Workers, Hafner Publishing Co., New York: Thirteenth Edition (1958).Google Scholar
  53. Hoel, P. G., Introduction to Mathematical Statistics, Fourth Edition, J. Wiley and Sons, Inc., New York (1971).Google Scholar
  54. Jaynes, E. T., ‘Review of Noise and Fluctuations’, by D. K. C. MacDonald, Am. J. Phys 31, 946 (1963).Google Scholar
  55. Kendall, M. G., ‘Ronald Aylmer Fisher, 1890–1962’, Biometrika 50, 1–15 (1963); reprinted in Studies in the History of Statistics and Probability, E. S. Pearson and M. G. Kendall, (eds)., Hafner Publishing Co., Darien, Conn. (1970).Google Scholar
  56. Mandel, J., The Statistical Analysis of Experimental Data, Interscience Publishers, New York (1964); p. 290.Google Scholar
  57. McColl, H., ‘The Calculus of Equivalent Statements’, Proc. Lond. Math. Soc 28, p. 556 (1897).Google Scholar
  58. Pearson, Karl, ‘Method of Moments and Method of Maximum Likelihood’, Biometrika 28, 34 (1936).Google Scholar
  59. Pratt, John W., ‘Review of Testing Statistical Hypothesis’ (Lehmann, 1959); /. Am. Stat. Assoc. Vol. 56, pp. 163–166 (1961).CrossRefGoogle Scholar
  60. Roberts, Harry V., ‘Statistical Dogma: One Response to a Challenge’, Multilithed, University of Chicago (1965).Google Scholar
  61. Thornber, Hodson, ‘An Autoregressive Model: Bayesian Versus Sampling Theory Analysis’, Multilithed, Dept. of Economics, University of Chicago, Chicago, Illinois (1965).Google Scholar
  62. Wilbraham, H., Phil. Mag. Series, 4, Vol. vii, (1854).Google Scholar
  63. Zellner, Arnold, ‘Bayesian Inference and Simultaneous Equation Models’, Multilithed, University of Chicago, Chicago, Illinois (1965).Google Scholar

Copyright information

© D. Reidel Publishing Company, Dordrecht-Holland 1976

Authors and Affiliations

  • E. T. Jaynes
    • 1
  • Oscar Kempthorne
    • 2
  1. 1.Dept. of PhysicsWashington UniversitySt. LouisUSA
  2. 2.Statistical LaboratoryIowa State UniversityUSA

Personalised recommendations