Skip to main content

On estimating probabilities in tree pruning

Part 3: Numeric And Statistical Approaches

Part of the Lecture Notes in Computer Science book series (LNAI,volume 482)


In this paper we introduce a new method for decision tree pruning, based on the minimisation of the expected classification error method by Niblett and Bratko. The original Niblett-Bratko pruning algorithm uses Laplace probability estimates. Here we introduce a new, more general Bayesian approach to estimating probabilities which we call m-probability-estimation. By varying a parameter m in this method, tree pruning can be adjusted to particular properties of the learning domain, such as level of noise. The resulting pruning method improves on the original Niblett-Bratko pruning in the following respects: apriori probabilities can be incorporated into error estimation, several trees pruned to various degrees can be generated, and the degree of pruning is not affected by the number of classes. These improvements are supported by experimental findings. m-probability-estimation also enables the combination of learning data obtained from various sources.


  • Classification Accuracy
  • Class Distribution
  • Tree Pruning
  • Pruning Method
  • Complete Binary Tree

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  • Berger, J.O. (1985), Statistical Decision Theory and Bayesian Analysis, Springer-Verlag, New York.

    Google Scholar 

  • Bratko, I., Kononenko, I. (1986), Learning diagnostic rules from incomplete and noisy data, AI Methods in Statistics, UNICOM Seminar, London, December 1986. Also in Interactions in AI and Statistics (ed. B. Phelps) London; Gower Technical Press, 1987.

    Google Scholar 

  • Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J. (1984), Classification and Regression Trees, Belmont, California: Wadsworth Int. Group.

    Google Scholar 

  • Cestnik, B., Kononenko, I., Bratko, I. (1987), ASSISTANT 86: A Knowledge-Elicitation Tool for Sophisticated Users, Progress in Machine Learning, Eds. I. Bratko & N. Lavrac, Sigma Press, Wilmslow.

    Google Scholar 

  • Cestnik, B. (1990), Estimating Probabilities: A Crucial Task in Machine Learning. In Proceedings of ECAI 90, Stockholm, August 1990.

    Google Scholar 

  • Clark, P., Niblett, T. (1987), Induction in Noisy Domains, Progress in Machine Learning, Eds. I. Bratko & N. Lavrac, Sigma Press, Wilmslow.

    Google Scholar 

  • Good, I.J. (1965), The Estimation of Probabilities, M.I.T. Press, Cambridge, Massachusetts.

    Google Scholar 

  • Mingers, J. (1989), An Empirical Comparison of Pruning Methods for Decision Tree Induction, Machine Learning vol. 4, no. 2, Kluwer Academic Publishers.

    Google Scholar 

  • Niblett, T., Bratko, I. (1986), Learning decision rules in noisy domains, Expert Systems 86, Cambridge University Press (Proceddings of Expert Systems 86 Conf., Brighton 1986).

    Google Scholar 

  • Pearl, J. (1988), Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann: San Mateo, CA.

    Google Scholar 

  • Quinlan, J.R. (1986), Learning from noisy data, Machine Learning vol. 2, Eds. R. Michalski, J. Carbonell and T. Mitchel, Palo Alto, CA: Tioga.

    Google Scholar 

  • Quinlan, J.R. (1987), Simplifying decision trees, International Journal of Man-Machine Studies, 27, pp. 221–234.

    Google Scholar 

  • Smyth, P., Goodman, R.M., Higgins, C. (1990), A Hybrid Rule-based/Bayesian Classifier, In Proceedings of ECAI 90, Stockholm, August 1990.

    Google Scholar 

Download references

Author information

Authors and Affiliations


Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 1991 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Cestnik, B., Bratko, I. (1991). On estimating probabilities in tree pruning. In: Kodratoff, Y. (eds) Machine Learning — EWSL-91. EWSL 1991. Lecture Notes in Computer Science, vol 482. Springer, Berlin, Heidelberg.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-53816-5

  • Online ISBN: 978-3-540-46308-5

  • eBook Packages: Springer Book Archive