Explicit Feature Construction and Manipulation for Covering Rule Learning Algorithms

  • Nada Lavrač
  • Johannes Fürnkranz
  • Dragan Gamberger
Part of the Studies in Computational Intelligence book series (SCI, volume 262)


Features are the main rule building blocks for rule learning algorithms. They can be simple tests for attribute values or complex logical terms representing available domain knowledge. In contrast to common practice in classification rule learning, we argue that the separation of feature construction and rule construction processes has a theoretical and practical justification. Explicit usage of features enables a unifying framework of both propositional and relational rule learning and we present and analyze procedures for feature construction in both types of domains. It is demonstrated that the presented procedure for constructing a set of simple features has the property that the resulting feature set enables the construction of complete and consistent rules whenever possible, and that the set does not include obviously irrelevant features. It is also shown that feature relevancy may improve the effectiveness of rule learning. It this work, we illustrate the relevancy concept in the coverage space, and show that the transformation from the attribute to the feature space enables a novel, theoretically justified way of handling unknown attribute values. The same approach enables that the estimated imprecision of continuous attributes can be taken into account, resulting in the construction of features that are robust to attribute imprecision.


Continuous Attribute Coverage Space Inductive Logic Programming Irrelevant Feature Feature Construction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bergadano, F., Matwin, S., Michalski, R.S., Zhang, J.: Learning two-tiered descriptions of flexible concepts: The POSEIDON system. Machine Learning 8, 5–43 (1992)Google Scholar
  2. Bruha, I., Franek, F.: Comparison of various routines for unknown attribute value processing: The covering paradigm. International Journal of Pattern Recognition and Artificial Intelligence 10(8), 939–955 (1996)CrossRefGoogle Scholar
  3. Clark, P., Niblett, T.: The CN2 induction algorithm. Machine Learning 3(4), 261–283 (1989)Google Scholar
  4. Cohen, W.W.: Fast effective rule induction. In: Prieditis, A., Russell, S. (eds.) Proceedings of the 12th International Conference on Machine Learning (ML 1995), Lake Tahoe, CA, pp. 115–123. Morgan Kaufmann, San Francisco (1995)Google Scholar
  5. Fayyad, U.M., Irani, K.B.: On the handling of continuous-valued attributes in decision tree generation. Machine Learning 8(2), 87–102 (1992)zbMATHGoogle Scholar
  6. Flach, P., Lachiche, N.: 1BC: A first-order bayesian classifier. In: Džeroski, S., Flach, P.A. (eds.) ILP 1999. LNCS (LNAI), vol. 1634, pp. 92–103. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  7. Fürnkranz, J.: Separate-and-conquer rule learning. Artificial Intelligence Review 13(1), 3–54 (1999)zbMATHCrossRefGoogle Scholar
  8. Fürnkranz, J., Flach, P.A.: ROC ’n’ rule learning – Towards a better understanding of covering algorithms. Machine Learning 58(1), 39–77 (2005)zbMATHCrossRefGoogle Scholar
  9. Gamberger, D., Lavrač, N.: Expert-guided subgroup discovery: Methodology and application. Journal of Artificial Intelligence Research 17, 501–527 (2002)zbMATHGoogle Scholar
  10. Golub, T., Slonim, D., Tamayo, P., Huard, C., Gaaseenbeek, M., Mesirov, J., Coller, H., Loh, M., Downing, J., Caligiuri, M., Bloomfield, C., Lander, E.: Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring. Science 286, 531–537 (1999)CrossRefGoogle Scholar
  11. Idestam-Almquist, P.: Generalization of clauses under implication. Journal of Artificial Intelligence Research 3, 467–489 (1995)zbMATHGoogle Scholar
  12. Idestam-Almquist, P.: Generalization of clauses. PhD thesis, Stockholm University, Department of Computer and Systems Sciences (1993)Google Scholar
  13. Kaufman, K.A., Michalski, R.S.: An adjustable rule learner for pattern discovery using the AQ methodology. Journal of Intelligent Information Systems 14, 199–216 (2000)CrossRefGoogle Scholar
  14. Larson, J.B., Michalski, R.S.: Inductive inference of VL decision rules. SIGART Newsletter 63, 38–44 (1977)Google Scholar
  15. Lavrač, N., Džeroski, S.: Inductive Logic Programming: Techniques and Applications. Ellis Horwood (1994)Google Scholar
  16. Lavrač, N., Flach, P.: An extended transformation approach to inductive logic programming. ACM Transactions on Computational Logic 2(4), 458–494 (2001)CrossRefGoogle Scholar
  17. Lavrač, N., Džeroski, S., Grobelnik, M.: Learning nonrecursive definitions of relations with LINUS. In: Kodratoff, Y. (ed.) EWSL 1991. LNCS, vol. 482, pp. 265–281. Springer, Heidelberg (1991)CrossRefGoogle Scholar
  18. Lavrač, N., Gamberger, D., Jovanoski, V.: A sudy of relevance for learning in deductive databases. Journal of Logic Programming 40(2/3), 215–249 (1999)zbMATHCrossRefMathSciNetGoogle Scholar
  19. Matheus, C.J.: A constructive induction framework. In: Proceedings of the 6th International Workshop on Machine Learning, pp. 474–475 (1989)Google Scholar
  20. Mehra, P., Rendell, L., Wah, B.: Principled constructive induction. In: Proceedings of the 11th International Joint Conference on Artificial Intelligence (IJCAI 1989), pp. 651–656. Morgan Kaufmann, San Francisco (1989)Google Scholar
  21. Michalski, R.S.: On the quasi-minimal solution of the covering problem. In: Proceedings of the 5th International Symposium on Information Processing (FCIP 1969), Bled, Yugoslavia. Switching Circuits, vol. A3, pp. 125–128 (1969)Google Scholar
  22. Michalski, R.S.: Pattern recognition and rule-guided inference. IEEE Transactions on Pattern Analysis and Machine Intelligence 2, 349–361 (1980)zbMATHCrossRefGoogle Scholar
  23. Michalski, R.S.: A theory and methodology of inductive learning. Artificial Intelligence 20(2), 111–162 (1983)CrossRefMathSciNetGoogle Scholar
  24. Michalski, R.S.: AQVAL/1 — computer implementation of a variable-valued logic system VL1 and examples of its application to pattern recognition. In: Proceedings of the 1st International Joint Conference on Pattern Recognition, pp. 3–17 (1973a)Google Scholar
  25. Michalski, R.S.: Discovering classification rules using variable-valued logic system vl1. In: Proceedings of the 3rd International Joint Conference on Artificial Intelligence (IJCAI 1973), Stanford, CA, pp. 162–172 (1973b)Google Scholar
  26. Michalski, R.S., Larson, J.B.: Selection of most representative training examples and incremental generation of vl1 hypotheses: the underlying methodology and the description of programs ESEL and AQ11. Technical Report 78-867, Department of Computer Science, University of Illinois at Urbana-Champaign (1978)Google Scholar
  27. Michalski, R.S., Mozetič, I., Hong, J., Lavrač, N.: The multi-purpose incremental learning system AQ15 and its testing application to three medical domains. In: Proceedings of the 5th National Conference on Artificial Intelligence (AAAI 1986), Philadelphia, PA, pp. 1041–1045 (1986)Google Scholar
  28. Provost, F., Fawcett, T.: Robust classification for imprecise environments. Machine Learning 42, 203–231 (2001)zbMATHCrossRefGoogle Scholar
  29. Quinlan, J.R.: Induction of decision trees. Machine Learning 1, 81–106 (1986)Google Scholar
  30. Quinlan, J.R.: Learning logical definitions from relations. Machine Learning 5, 239–266 (1990)Google Scholar
  31. Wojtusiak, J., Michalski, R.S., Kaufman, K.A., Pietrzykowski, J.: The AQ21 natural induction program for pattern discovery: Initial version and its novel features. In: Proceedings of The 18th IEEE International Conference on Tools with Artificial Intelligence, Cincinnati, OH (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Nada Lavrač
    • 1
    • 2
  • Johannes Fürnkranz
    • 3
  • Dragan Gamberger
    • 4
  1. 1.Jožef Stefan InstituteLjubljanaSlovenia
  2. 2.University of Nova GoricaNova GoricaSlovenia
  3. 3.TU DarmstadtDarmstadtGermany
  4. 4.Rudjer Bošković InstituteZagrebCroatia

Personalised recommendations