Encyclopedia of Machine Learning

2010 Edition
| Editors: Claude Sammut, Geoffrey I. Webb

Rule Learning

  • Johannes Fürnkranz
Reference work entry
DOI: https://doi.org/10.1007/978-0-387-30164-8_738



Inductive rule learning solves a  classification problem via the induction of a rule set or a  decision list. The principal approach is the so-called separate-and-conquer or covering algorithm, which learns one rule at a time, successively removing the covered examples. Individual algorithms within this framework differ primarily in the way they learn single rules. A more extensive survey of this family of algorithms can be found in Fürnkranz (1999).

The Covering Algorithm

Most covering algorithms operate in a  concept learning framework, that is, they assume a set of positive and negative training examples. Adaptations to the multi-class case are typically performed via class binarization, transforming the original problem into a set of binary problems. Some algorithms, most notably CN2 (Clark & Niblett, 1989; Clark & Boswell, 1991), learn multi-class rules directly by optimizing...
This is a preview of subscription content, log in to check access.

Recommended Reading

  1. Cestnik, B. (1990). Estimating probabilities: A crucial task in machine learning. In L. Aiello (Ed.), Proceedings of the ninth European conference on artificial intelligence (ECAI-90), Stockholm, Sweden (pp. 147–150). Pitman, London.Google Scholar
  2. Clark, P., & Boswell, R. (1991). Rule induction with CN2: Some recent improvements. In Proceedings of the fifth European working session on learning (EWSL-91), Porto, Portugal (pp. 151–163). London: Springer.Google Scholar
  3. Clark, P., & Niblett, T. (1989). The CN2 induction algorithm. Machine Learning, 3(4), 261–283.Google Scholar
  4. Cohen, W. W. (1995). Fast effective rule induction. In A. Prieditis & S. Russell (Eds.), Proceedings of the 12th international conference on machine learning (ML-95), Lake Tahoe, California (pp. 115–123). Morgan Kaufmann, San Mateo, CA.Google Scholar
  5. Cohen, W. W., & Singer, Y. (1999). A simple, fast, and effective rule learner. In Proceedings of the 16th national conference on artificial intelligence (AAAI-99), Orlando (pp. 335–342). Menlo Park: AAAI/MIT Press.Google Scholar
  6. Domingos, P. (1996). Unifying instance-based and rule-based induction. Machine Learning, 24, 141–168.Google Scholar
  7. Fürnkranz, J. (1997). Pruning algorithms for rule learning. Machine Learning, 27(2), 139–171.CrossRefGoogle Scholar
  8. Fürnkranz, J. (February 1999). Separate-and-conquer rule learning. Artificial Intelligence Review, 13(1), 3–54.zbMATHCrossRefGoogle Scholar
  9. Fürnkranz, J., & Flach, P. (2005). ROC ‘n’ rule learning – Towards a better understanding of covering algorithms. Machine Learning, 58(1), 39–77.zbMATHCrossRefGoogle Scholar
  10. Fürnkranz, J., & Widmer, G. (1994). Incremental reduced error pruning. In W. Cohen & H. Hirsh (Eds.), Proceedings of the 11th international conference on machine learning (ML-94), New Brunswick, NJ (pp. 70–77). Morgan Kaufmann, San Mateo, CA.Google Scholar
  11. Liu, B., Hsu, W., & Ma, Y. (1998). Integrating classification and association rule mining. In R. Agrawal, P. Stolorz, & G. Piatetsky-Shapiro (Eds.), Proceedings of the fourth international conference on knowledge discovery and data mining (KDD-98), New York City, NY (pp. 80–86).Google Scholar
  12. Michalski, R. S. (1969). On the quasi-minimal solution of the covering problem. In Proceedings of the fifth international symposium on information processing (FCIP-69), Bled, Yugoslavia. Switching circuits (Vol. A3, pp. 125–128).Google Scholar
  13. Quinlan, J. R. (1990). Learning logical definitions from relations. Machine Learning, 5, 239–266.Google Scholar
  14. Quinlan, J. R. (1993). C4.5: Programs for machine learning. San Mateo: Morgan Kaufmann.Google Scholar
  15. Webb, G. I. (1995). OPUS: An efficient admissible algorithm for unordered search. Journal of Artificial Intelligence Research, 5, 431–465.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Johannes Fürnkranz

There are no affiliations available