A Brief Overview of Rule Learning

  • Johannes Fürnkranz
  • Tomáš KliegrEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9202)


In this paper, we provide a brief summary of elementary research in rule learning. The two main research directions are descriptive rule learning, with the goal of discovering regularities that hold in parts of the given dataset, and predictive rule learning, which aims at generalizing the given dataset so that predictions on new data can be made. We briefly review key learning tasks such as association rule learning, subgroup discovery, and the covering learning algorithm, along with their most important prototypes. The paper also highlights recent work in rule learning on the Semantic Web and Linked Data as an important application area.


Association Rule Frequent Itemsets Association Rule Mining Inductive Logic Programming Rule Discovery 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alcala-Fdez, J., Alcala, R., Herrera, F.: A fuzzy association rule-based classification model for high-dimensional problems with genetic rule selection and lateral tuning. IEEE Transactions on Fuzzy Systems 19(5), 857–872 (2011)CrossRefGoogle Scholar
  2. 2.
    Agrawal, R., Imielinski, T., Swami, A.N.: Mining association rules between sets of items in large databases. In: Buneman, P., Jajodia, S. (eds.) Proceedings of the ACM International Conference on Management of Data (SIGMOD 1993), Washington, D.C., pp. 207–216 (1993)Google Scholar
  3. 3.
    Azevedo, P.J., Jorge, A.J.: Ensembles of jittered association rule classifiers. Data Mining and Knowledge Discovery 21(1), 91–129 (2010). Special Issue on Global Modeling using Local PatternsMathSciNetCrossRefGoogle Scholar
  4. 4.
    Bay, S.D., Pazzani, M.J.: Detecting group differences: Mining contrast sets. Data Mining and Knowledge Discovery 5(3), 213–246 (2001)CrossRefzbMATHGoogle Scholar
  5. 5.
    Bayardo Jr., R.J.: Brute-force mining of high-confidence classification rules. In: Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining (KDD 1997), pp. 123–126 (1997)Google Scholar
  6. 6.
    Bringmann, B., Nijssen, S., Zimmermann, A.: Pattern-based classification: a unifying perspective. In: Knobbe, A., Fürnkranz, J. (eds.) Proceedings of the ECML/PKDD 1909 Workshop From Local Patterns to Global Models (LeGo 1909), Bled, Slovenia, pp. 36–50 (2009)Google Scholar
  7. 7.
    Cestnik, B.: Estimating probabilities: a crucial task in Machine Learning. In: Aiello, L. (ed.) Proceedings of the 9th European Conference on Artificial Intelligence (ECAI 1990), Pitman, Stockholm, Sweden, pp. 147–150 (1990)Google Scholar
  8. 8.
    Clark, P., Boswell, R.: Rule induction with CN2: Some recent improvements. In: Kodratoff, Y. (ed.) Machine Learning – EWSL-91. LNCS, vol. 482, pp. 151–163. Springer, Heidelberg (1991)CrossRefGoogle Scholar
  9. 9.
    Clark, P., Niblett, T.: The CN2 induction algorithm. Machine Learning 3(4), 261–283 (1989)Google Scholar
  10. 10.
    De Raedt, L.: Logical and Relational Learning. Springer-Verlag (2008)Google Scholar
  11. 11.
    Dong, G., Li, J.: Efficient mining of emerging patterns: discovering trends and differences. In: Proceedings of the 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 1999), San Diego, CA, pp. 43–52 (1999)Google Scholar
  12. 12.
    Džeroski, S., Lavrač, N. (eds.): Relational Data Mining: Inductive Logic Programming for Knowledge Discovery in Databases. Springer-Verlag (2001)Google Scholar
  13. 13.
    Eineborg, M., Boström, H.: Classifying uncovered examples by rule stretching. In: Rouveirol, C., Sebag, M. (eds.) ILP 2001. LNCS (LNAI), vol. 2157, pp. 41–50. Springer, Heidelberg (2001) CrossRefGoogle Scholar
  14. 14.
    Fayyad, U.M., Irani, K.B.: Multi-interval discretization of continuous-valued attributes for classification learning. In: Proceedings of the 13th International Joint Conference on Artificial Intelligence (IJCAI 1993), pp. 1022–1029 (1993)Google Scholar
  15. 15.
    Frank, E., Witten, I.H.: Generating accurate rule sets without global optimization. In: Shavlik, J. (ed.) Proceedings of the 15th International Conference on Machine Learning (ICML 1998), pp. 144–151. Morgan Kaufmann, Madison (1998)Google Scholar
  16. 16.
    Fürnkranz, J.: Pruning algorithms for rule learning. Machine Learning 27(2), 139–171 (1997)CrossRefGoogle Scholar
  17. 17.
    Fürnkranz, J.: Separate-and-conquer rule learning. Artificial Intelligence Review 13(1), 3–54 (1999)CrossRefGoogle Scholar
  18. 18.
    Fürnkranz, J., Flach, P.A.: ROC ’n’ rule learning - Towards a better understanding of covering algorithms. Machine Learning 58(1), 39–77 (2005)CrossRefzbMATHGoogle Scholar
  19. 19.
    Fürnkranz, J., Gamberger, D., Lavrač, N.: Foundations of Rule Learning. Springer-Verlag (2012)Google Scholar
  20. 20.
    Galárraga, L.A., Teflioudi, C., Hose, K., Suchanek, F.: AMIE: association rule mining under incomplete evidence in ontological knowledge bases. In: Proceedings of the 22nd International Conference on World Wide Web (WWW 2013), Switzerland, pp. 413–422 (2013)Google Scholar
  21. 21.
    Galárraga, L.A., Preda, N., Suchanek, F.M.: Mining rules to align knowledge bases. In: Proceedings of the 2013 Workshop on Automated Knowledge Base Construction (AKBC 2013), pp. 43–48. ACM, New York (2013)Google Scholar
  22. 22.
    Goethals, B.: Frequent set mining. In: Maimon, O., Rokach, L. (eds.) The Data Mining and Knowledge Discovery Handbook, 2nd edn., pp. 321–338. Springer-Verlag (2010)Google Scholar
  23. 23.
    Hájek, P., Holena, M., Rauch, J.: The GUHA method and its meaning for data mining. Journal of Computer and System Sciences 76(1), 34–48 (2010). Special Issue on Intelligent Data AnalysisMathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer Series in Statistics. Springer, New York (2001)CrossRefzbMATHGoogle Scholar
  25. 25.
    Han, J., Pei, J., Yin, Y., Mao, R.: Mining frequent patterns without candidate generation: A frequent-pattern tree approach. Data Mining and Knowledge Discovery 8(1), 53–87 (2004)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Hhn, J., Hllermeier, E.: Furia: an algorithm for unordered fuzzy rule induction. Data Mining and Knowledge Discovery 19(3), 293–319 (2009)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Jovanoski, V., Lavrač, N.: Classification rule learning with APRIORI-C. In: Brazdil, P.B., Jorge, A.M. (eds.) EPIA 2001. LNCS (LNAI), vol. 2258, pp. 44–51. Springer, Heidelberg (2001) Google Scholar
  28. 28.
    Kliegr, T., Kuchař, J., Sottara, D., Vojíř, S.: Learning business rules with association rule classifiers. In: Bikakis, A., Fodor, P., Roman, D. (eds.) RuleML 2014. LNCS, vol. 8620, pp. 236–250. Springer, Heidelberg (2014) Google Scholar
  29. 29.
    Klösgen, W.: Explora: a multipattern and multistrategy discovery assistant. In: Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R. (eds.) Advances in Knowledge Discovery and Data Mining, chap. 10, pp. 249–271. AAAI Press (1996)Google Scholar
  30. 30.
    Kralj Novak, P., Lavrač, N., Webb, G.I.: Supervised descriptive rule discovery: A unifying survey of contrast set, emerging pattern and subgroup mining. Journal of Machine Learning Research 10, 377–403 (2009)zbMATHGoogle Scholar
  31. 31.
    Li, W., Han, J., Pei, J.: CMAR: accurate and efficient classification based on multiple class-association rules. In: Proceedings of the IEEE Conference on Data Mining (ICDM 2001), pp. 369–376 (2001)Google Scholar
  32. 32.
    Lindgren, T., Boström, H.: Resolving rule conflicts with double induction. Intelligent Data Analysis 8(5), 457–468 (2004)Google Scholar
  33. 33.
    Lisi, F.: Building Rules on Top of Ontologies for the Semantic Web with Inductive Logic Programming. Theory and Practice of Logic Programming 8(3), 271–300 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Lisi, F., Esposito, F.: An ilp perspective on the semantic web. In: Bouquet, P., Tummarello, G. (eds.) Semantic Web Applications and Perspectives - Proceedings of the 2nd Italian Semantic Web Workshop (SWAP-05), pp. 14–16. University of Trento, Trento (2005)Google Scholar
  35. 35.
    Liu, B., Hsu, W., Ma, Y.: Integrating classification and association rule mining. In: Agrawal, R., Stolorz, P., Piatetsky-Shapiro, G. (eds.) Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining (KDD 1998), pp. 80–86 (1998)Google Scholar
  36. 36.
    Liu, B., Ma, Y., Wong, C.K.: Improving an association rule based classifier. In: Zighed, D.A., Komorowski, J., Żytkow, J.M. (eds.) PKDD 2000. LNCS (LNAI), vol. 1910, pp. 504–509. Springer, Heidelberg (2000) CrossRefGoogle Scholar
  37. 37.
    Lucchese, C.: DCI closed: a fast and memory efficient algorithm to mine frequent closed itemsets. In: Proceedings of the IEEE ICDM 2004 Workshop on Frequent Itemset Mining Implementations (FIMI 2004) (2004)Google Scholar
  38. 38.
    Lucchese, C., Orlando, S., Perego, R.: Parallel mining of frequent closed patterns: harnessing modern computer architectures. In: Proceedings of the 7th IEEE International Conference on Data Mining (ICDM 2007), pp. 242–251 (2007)Google Scholar
  39. 39.
    Michalski, R.S.: On the quasi-minimal solution of the covering problem. In: Proceedings of the 5th International Symposium on Information Processing (FCIP-69) (Switching Circuits), vol. A3, Bled, Yugoslavia, pp. 125–128 (1969)Google Scholar
  40. 40.
    Morishita, S., Sese, J.: Traversing itemset lattice with statistical metric pruning. In: Proceedings of the 19th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS 2000), pp. 226–236. ACM (2000)Google Scholar
  41. 41.
    Muggleton, S.H.: Inverse entailment and Progol. New Generation Computing 13(3,4), 245–286 (1995). Special Issue on Inductive Logic ProgrammingCrossRefGoogle Scholar
  42. 42.
    Muggleton, S.H., De Raedt, L.: Inductive Logic Programming: Theory and methods. Journal of Logic Programming 19–20, 629–679 (1994)CrossRefGoogle Scholar
  43. 43.
    Mutter, S., Hall, M., Frank, E.: Using classification to evaluate the output of confidence-based association rule mining. In: Webb, G.I., Yu, X. (eds.) AI 2004. LNCS (LNAI), vol. 3339, pp. 538–549. Springer, Heidelberg (2004) CrossRefGoogle Scholar
  44. 44.
    Negrevergne, B., Termier, A., Rousset, M.C., Mhaut, J.F.: Para miner: a generic pattern mining algorithm for multi-core architectures. Data Mining and Knowledge Discovery 28(3), 593–633 (2014)MathSciNetCrossRefGoogle Scholar
  45. 45.
    Negrevergne, B., Termier, A., Rousset, M.C., Mhaut, J.F., Uno, T.: Discovering closed frequent itemsets on multicore: parallelizing computations and optimizing memory accesses. In: Proceedings of the International Conference on High Performance Computing and Simulation (HPCS 2010), pp. 521–528 (2010)Google Scholar
  46. 46.
    Pasquier, N., Bastide, Y., Taouil, R., Lakhal, L.: Discovering frequent closed itemsets for association rules. In: Beeri, C., Bruneman, P. (eds.) ICDT 1999. LNCS, vol. 1540, pp. 398–416. Springer, Heidelberg (1998) CrossRefGoogle Scholar
  47. 47.
    Paulheim, H.: Generating possible interpretations for statistics from linked open data. In: Simperl, E., Cimiano, P., Polleres, A., Corcho, O., Presutti, V. (eds.) ESWC 2012. LNCS, vol. 7295, pp. 560–574. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  48. 48.
    Paulheim, H., Browsing linked open data with auto complete. In: Proceedings of the Semantic Web Challenge co-located with ISWC 2012. Univ., Mannheim, Boston (2012)Google Scholar
  49. 49.
    Paulheim, H., Bizer, C.: Type inference on noisy rdf data. In: Alani, H., Kagal, L., Fokoue, A., Groth, P., Biemann, C., Parreira, J.X., Aroyo, L., Noy, N., Welty, C., Janowicz, K. (eds.) ISWC 2013, Part I. LNCS, vol. 8218, pp. 510–525. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  50. 50.
    Paulheim, H., Fürnkranz, J.: Unsupervised feature construction from linked open data. In: Proceedings of the ACM International Conference Web Intelligence, Mining, and Semantics (WIMS 2012) (2012)Google Scholar
  51. 51.
    Quinlan, J.R.: Learning logical definitions from relations. Machine Learning 5, 239–266 (1990)Google Scholar
  52. 52.
    Quinlan, J.R.: Generating production rules from decision trees. In: Proceedings of the 10th International Joint Conference on Artificial Intelligence (IJCAI 1987), pp. 304–307. Morgan Kaufmann (1987)Google Scholar
  53. 53.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)zbMATHGoogle Scholar
  54. 54.
    Rauch, J.: Observational Calculi and Association Rules, Studies in Computational Intelligence, vol. 469. Springer (2013)Google Scholar
  55. 55.
    Sulzmann, J.N., Fürnkranz, J.: A comparison of techniques for selecting and combining class association rules. In: Knobbe, A.J. (ed.) Proceedings of the ECML/PKDD 2008 Workshop From Local Patterns to Global Models (LeGo 2008), Antwerp, Belgium, pp. 154–168 (2008)Google Scholar
  56. 56.
    Uno, T., Kiyomi, M., Arimura, H.: LCM ver. 2: efficient mining algorithms for frequent/closed/maximal itemsets. In: Proceedings of the IEEE ICDM 2004 Workshop on Frequent Itemset Mining Implementations (FIMI 2004) (2004)Google Scholar
  57. 57.
    Webb, G.I.: OPUS: An efficient admissible algorithm for unordered search. Journal of Artificial Intelligence Research 5, 431–465 (1995)Google Scholar
  58. 58.
    Witten, I.H., Frank, E.: Data Mining - Practical Machine Learning Tools and Techniques with Java Implementations, 2nd edn. Morgan Kaufmann Publishers (2005)Google Scholar
  59. 59.
    Wrobel, S.: An algorithm for multi-relational discovery of subgroups. In: Komorowski, J., Żytkow, J.M. (eds.) PKDD 1997. LNCS, vol. 1263, pp. 78–87. Springer, Heidelberg (1997) CrossRefGoogle Scholar
  60. 60.
    Yin, X., Han, J.: CPAR: classification based on predictive association rules. In: Proceedings SIAM Conference on Data Mining (SDM 2003) (2003)Google Scholar
  61. 61.
    Völker, J., Niepert, M.: Statistical schema induction. In: Antoniou, G., Grobelnik, M., Simperl, E., Parsia, B., Plexousakis, D., De Leenheer, P., Pan, J. (eds.) ESWC 2011, Part I. LNCS, vol. 6643, pp. 124–138. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  62. 62.
    Zaki, M.J., Parthasarathy, S., Ogihara, M., Li, W.: New algorithms for fast discovery of association rules. In: Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining (KDD 1997), Newport, CA, pp. 283–286 (1997)Google Scholar
  63. 63.
    Zhang, C., Zhang, S.: Association Rule Mining –Models and Algorithms. Springer (2002)Google Scholar
  64. 64.
    Zimmermann, A., De Raedt, L.: Cluster grouping: From subgroup discovery to clustering. Machine Learning 77(1), 125–159 (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Department of Computer ScienceTU DarmstadtDarmstadtGermany
  2. 2.Department of Information and Knowledge EngineeringUniversity of Economics, PraguePragueCzech Republic

Personalised recommendations