An Empirical Comparison of Probability Estimation Techniques for Probabilistic Rules

  • Jan-Nikolas Sulzmann
  • Johannes Fürnkranz
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5808)


Rule learning is known for its descriptive and therefore comprehensible classification models which also yield good class predictions. However, in some application areas, we also need good class probability estimates. For different classification models, such as decision trees, a variety of techniques for obtaining good probability estimates have been proposed and evaluated. However, so far, there has been no systematic empirical study of how these techniques can be adapted to probabilistic rules and how these methods affect the probability-based rankings. In this paper we apply several basic methods for the estimation of class membership probabilities to classification rules. We also study the effect of a shrinkage technique for merging the probability estimates of rules with those of their generalizations.


Probability Estimation Rule Learning Class Probability Default Rule Rule Pruning 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Asuncion, A., Newman, D.J.: UCI machine learning repository (2007)Google Scholar
  2. Cestnik, B.: Estimating probabilities: A crucial task in Machine Learning. In: Aiello, L. (ed.) Proceedings of the 9th European Conference on Artificial Intelligence (ECAI 1990), Stockholm, Sweden, pp. 147–150. Pitman (1990)Google Scholar
  3. Chen, S.F., Goodman, J.T.: An empirical study of smoothing techniques for language modeling. Technical Report TR-10-98, Computer Science Group, Harvard University, Cambridge, MA (1998)Google Scholar
  4. Cohen, W.W.: Fast effective rule induction. In: Prieditis, A., Russell, S. (eds.) Proceedings of the 12th International Conference on Machine Learning (ML 1995), Lake Tahoe, CA, pp. 115–123. Morgan Kaufmann, San Francisco (1995)CrossRefGoogle Scholar
  5. Demsar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1–30 (2006)MathSciNetzbMATHGoogle Scholar
  6. Ferri, C., Flach, P.A., Hernández-Orallo, J.: Improving the AUC of probabilistic estimation trees. In: Proceedings of the 14th European Conference on Machine Learning, Cavtat-Dubrovnik, Croatia, pp. 121–132 (2003)Google Scholar
  7. Fürnkranz, J.: Pruning algorithms for rule learning. Machine Learning 27(2), 139–171 (1997)CrossRefGoogle Scholar
  8. Fürnkranz, J., Flach, P.A.: Roc ’n’ rule learning-towards a better understanding of covering algorithms. Machine Learning 58(1), 39–77 (2005)CrossRefzbMATHGoogle Scholar
  9. Fürnkranz, J., Widmer, G.: Incremental Reduced Error Pruning. In: Cohen, W., Hirsh, H. (eds.) Proceedings of the 11th International Conference on Machine Learning (ML 1994), New Brunswick, NJ, pp. 70–77. Morgan Kaufmann, San Francisco (1994)CrossRefGoogle Scholar
  10. Hand, D.J., Till, R.J.: A simple generalisation of the area under the roc curve for multiple class classification problems. Machine Learning 45(2), 171–186 (2001)CrossRefzbMATHGoogle Scholar
  11. Hüllermeier, E., Vanderlooy, S.: Why fuzzy decision trees are good rankers. IEEE Transactions on Fuzzy Systems (to appear, 2009)Google Scholar
  12. Manning, C.D., Schütze, H.: Foundations of Statistical Natural Language Processing. The MIT Press, Cambridge (1999)zbMATHGoogle Scholar
  13. Provost, F.J., Domingos, P.: Tree induction for probability-based ranking. Machine Learning 52(3), 199–215 (2003)CrossRefzbMATHGoogle Scholar
  14. Wang, B., Zhang, H.: Improving the ranking performance of decision trees. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 461–472. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  15. Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Jan-Nikolas Sulzmann
    • 1
  • Johannes Fürnkranz
    • 1
  1. 1.Department of Computer ScienceTU DarmstadtDarmstadtGermany

Personalised recommendations