Point estimation using the Kullback-Leibler loss function and MML

  • David L. Dowe
  • Rohan A. Baxter
  • Jonathan J. Oliver
  • Chris S. Wallace
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1394)


Machine Learning Noise handling Algorithmic complexity Bayesian and Statistical Learning Methods Minimum Message Length Induction in KDD 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [BC91]
    A.R. Barron and T.M. Cover. Minimum complexity density estimation. IEEE Transactions on Information Theory, 37:1034–1054, 1991.MathSciNetCrossRefMATHGoogle Scholar
  2. [Cov92]
    T.M. Cover. IEEE Information Theory Newsletter, pages 1–6, 1992.Google Scholar
  3. [CT91]
    T.M. Cover and Joy A. Thomas. Elements of Information Theory. John Wiley and Sons,Inc., New York, 1991.CrossRefMATHGoogle Scholar
  4. [DFHL96]
    D.L. Dowe, G.E. Farr, A.J. Hurst, and K.L. Lentin. Information-theoretic football tipping. In N. de Mestre, editor, Proc. 3rd Conference on Mathematics and Computers in Sport, pages 233–241, Bond University, Qld., Australia, 1996.Google Scholar
  5. [DOW96]
    D.L. Dowe, J.J. Oliver, and C.S. Wallace. MML estimation of the parameters of the spherical Fisher distribution. In A. et al. Sharma, editor. Proc. 7th Conf. Algorithmic Learning Theory (ALT'96), LNAI 1160, pages 213–227. Sydney. Australia, October 1996.Google Scholar
  6. [DW97]
    D.L. Dowe and C.S. Wallace. Resolving the Neyman-Scott problem by Minimum Message Length. In Proc. Computing Science and Statistics — 28th Symposium on the interface, volume 28, pages 614–618. 1997.Google Scholar
  7. [Fis53]
    R.A. Fisher. Dispersion on a sphere. Journal of the Royal Statistical Society (Series A), 217:295–305, 1953.MathSciNetMATHGoogle Scholar
  8. [Goo83]
    I.J. Good. Explicativity, corroboration, and the relative odds of hypotheses. In Good thinking-The Foundations of Probability and its applications. University of Minnesota Press, Minneapolis,MN, 1983.Google Scholar
  9. [NS48]
    J. Neyman and E.L. Scott. Consistent estimates based on partially consistent observations. Econometrika, 16:1–32, 1948.MathSciNetCrossRefMATHGoogle Scholar
  10. [Sol64]
    R.J. Solomonoff. A formal theory of inductive inference. Information and Control. 7:1–22,224–254, 1964.MathSciNetCrossRefMATHGoogle Scholar
  11. [Wal95]
    C.S. Wallace. Multiple Factor Analysis by MML Estimation. Technical Report 95/218, Dept. of Computer Science, Monash University, Clayton, Victoria 3168. Australia, 1995. submitted to J. Multiv. Analysis.Google Scholar
  12. [Wal96]
    C.S. Wallace. False Oracles and SMML Estimators. In D.L. Dowe, K.B. Korb, and J.J. Oliver, editors, Proceedings of the Information. Statistics and Induction in Science (ISIS) Conference, pages 304–316, Melbourne, Australia. August 1996. World Scientific. Was Tech Rept 89/128, Dept. Comp. Sci., Monash Univ., Australia, June 1989.Google Scholar
  13. [WB68]
    C.S. Wallace and D.M. Boulton. An information measure for classification. Computer Journal, 11:185–194, 1968.CrossRefMATHGoogle Scholar
  14. [WB75]
    C.S. Wallace and D.M. Boulton. An invariant Bayes method for point estimation. Classification Society Bulletin, 3(3):11–34, 1975.Google Scholar
  15. [WD93]
    C.S. Wallace and D.L. Dowe. MML estimation of the von Mises concentration parameter. Tech Rept TR 93/193, Dept. of Comp. Sci.. Monash Univ., Clayton 3168, Australia, 1993. prov. accepted, Aust. J. Stat.Google Scholar
  16. [WD94]
    C.S. Wallace and D.L. Dowe. Intrinsic classification by MML — the Snob program. In C. Zhang and et al., editors, Proc. 7th Australian Joint Conf on Artif. Intelligence, pages 37–44. World Scientific, Singapore, 1994.Google Scholar
  17. [WF87]
    C.S. Wallace and P.R. Freeman. Estimation and inference by compact cooling. J. Royal Statistical Society (Series B), 49:240–252, 1987.MathSciNetMATHGoogle Scholar
  18. [WF92]
    C.S. Wallace and P.R. Freeman. Single factor analysis by MML estimation. Journal of the Royal Statistical Society (Series B). 54:195–209. 1992.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • David L. Dowe
    • 1
  • Rohan A. Baxter
    • 1
  • Jonathan J. Oliver
    • 1
  • Chris S. Wallace
    • 1
  1. 1.School of Computer Science and Software EngineeringMonash UniversityClaytonAustralia

Personalised recommendations