Skip to main content
Book cover

European Working Session on Learning

EWSL 1991: Machine Learning — EWSL-91 pp 138–150Cite as

On estimating probabilities in tree pruning

Part 3: Numeric And Statistical Approaches

Part of the Lecture Notes in Computer Science book series (LNAI,volume 482)

Abstract

In this paper we introduce a new method for decision tree pruning, based on the minimisation of the expected classification error method by Niblett and Bratko. The original Niblett-Bratko pruning algorithm uses Laplace probability estimates. Here we introduce a new, more general Bayesian approach to estimating probabilities which we call m-probability-estimation. By varying a parameter m in this method, tree pruning can be adjusted to particular properties of the learning domain, such as level of noise. The resulting pruning method improves on the original Niblett-Bratko pruning in the following respects: apriori probabilities can be incorporated into error estimation, several trees pruned to various degrees can be generated, and the degree of pruning is not affected by the number of classes. These improvements are supported by experimental findings. m-probability-estimation also enables the combination of learning data obtained from various sources.

Keywords

  • Classification Accuracy
  • Class Distribution
  • Tree Pruning
  • Pruning Method
  • Complete Binary Tree

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Download conference paper PDF

References

  • Berger, J.O. (1985), Statistical Decision Theory and Bayesian Analysis, Springer-Verlag, New York.

    Google Scholar 

  • Bratko, I., Kononenko, I. (1986), Learning diagnostic rules from incomplete and noisy data, AI Methods in Statistics, UNICOM Seminar, London, December 1986. Also in Interactions in AI and Statistics (ed. B. Phelps) London; Gower Technical Press, 1987.

    Google Scholar 

  • Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J. (1984), Classification and Regression Trees, Belmont, California: Wadsworth Int. Group.

    Google Scholar 

  • Cestnik, B., Kononenko, I., Bratko, I. (1987), ASSISTANT 86: A Knowledge-Elicitation Tool for Sophisticated Users, Progress in Machine Learning, Eds. I. Bratko & N. Lavrac, Sigma Press, Wilmslow.

    Google Scholar 

  • Cestnik, B. (1990), Estimating Probabilities: A Crucial Task in Machine Learning. In Proceedings of ECAI 90, Stockholm, August 1990.

    Google Scholar 

  • Clark, P., Niblett, T. (1987), Induction in Noisy Domains, Progress in Machine Learning, Eds. I. Bratko & N. Lavrac, Sigma Press, Wilmslow.

    Google Scholar 

  • Good, I.J. (1965), The Estimation of Probabilities, M.I.T. Press, Cambridge, Massachusetts.

    Google Scholar 

  • Mingers, J. (1989), An Empirical Comparison of Pruning Methods for Decision Tree Induction, Machine Learning vol. 4, no. 2, Kluwer Academic Publishers.

    Google Scholar 

  • Niblett, T., Bratko, I. (1986), Learning decision rules in noisy domains, Expert Systems 86, Cambridge University Press (Proceddings of Expert Systems 86 Conf., Brighton 1986).

    Google Scholar 

  • Pearl, J. (1988), Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann: San Mateo, CA.

    Google Scholar 

  • Quinlan, J.R. (1986), Learning from noisy data, Machine Learning vol. 2, Eds. R. Michalski, J. Carbonell and T. Mitchel, Palo Alto, CA: Tioga.

    Google Scholar 

  • Quinlan, J.R. (1987), Simplifying decision trees, International Journal of Man-Machine Studies, 27, pp. 221–234.

    Google Scholar 

  • Smyth, P., Goodman, R.M., Higgins, C. (1990), A Hybrid Rule-based/Bayesian Classifier, In Proceedings of ECAI 90, Stockholm, August 1990.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Jožef Stefan Institute, Jamova 39, 61000, Ljubljana, Yugoslavia

    Bojan Cestnik & Ivan Bratko

  2. Faculty of Electrical Eng. and Computer Science, Tržaška 25, 61000, Ljubljana, Yugoslavia

    Ivan Bratko

Authors
  1. Bojan Cestnik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ivan Bratko
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

    Rights and permissions

    Reprints and Permissions

    Copyright information

    © 1991 Springer-Verlag Berlin Heidelberg

    About this paper

    Cite this paper

    Cestnik, B., Bratko, I. (1991). On estimating probabilities in tree pruning. In: Kodratoff, Y. (eds) Machine Learning — EWSL-91. EWSL 1991. Lecture Notes in Computer Science, vol 482. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0017010

    Download citation

    • DOI: https://doi.org/10.1007/BFb0017010

    • Published:

    • Publisher Name: Springer, Berlin, Heidelberg

    • Print ISBN: 978-3-540-53816-5

    • Online ISBN: 978-3-540-46308-5

    • eBook Packages: Springer Book Archive