Skip to main content
SpringerLink
Log in

Learning Single Rules

  • Chapter
  • First Online:
Foundations of Rule Learning

Part of the book series: Cognitive Technologies ((COGTECH))

  • 2127 Accesses

Abstract

This chapter describes algorithms and strategies for constructing single rules in the concept learning framework. The starting point in Sect. 6.2 is a generic algorithm for finding the best rule which searches the hypothesis space for a rule that optimizes some quality criterion. Section 6.3 presents alternative search algorithms (heuristic search algorithms including hill-climbing and beam search, as well as exhaustive search algorithms).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.95
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 84.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The name Cigol is ‘logic’ spelled backwards.

References

  • Adé, H., De Raedt, L., & Bruynooghe, M. (1995). Declarative bias for specific-to-general ILP systems. Machine Learning, 20(1–2), 119–154. Special Issue on Bias Evaluation and Selection.

    Google Scholar 

  • Agrawal, R., Mannila, H., Srikant, R., Toivonen, H., & Verkamo, A. I. (1995). Fast discovery of association rules. In U. M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, & R. Uthurusamy (Eds.), Advances in knowledge discovery and data mining (pp. 307–328). Menlo Park, CA: AAAI.

    Google Scholar 

  • Bayardo, R. J., Jr. (1997). Brute-force mining of high-confidence classification rules. In Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining (KDD-97) (pp. 123–126). Menlo Park, CA: AAAI.

    Google Scholar 

  • Bergadano, F., Giordana, A., & Saitta, L. (1988). Automated concept acquisition in noisy environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10, 555–578.

    Article  Google Scholar 

  • Botta, M., Giordana, A., & Saitta, L. (1992). Comparison of search strategies in learning relations. In B. Neumann (Ed.), Proceedings of the 10th European Conference on Artificial Intelligence (ECAI-92), Vienna (pp. 451–455). Chichester, UK/New York: Wiley.

    Google Scholar 

  • Clark, P., & Niblett, T. (1989). The CN2 induction algorithm. Machine Learning, 3(4), 261–283.

    Google Scholar 

  • De Raedt, L. (1992). Interactive theory revision: An inductive logic programming approach. London/San Diego, CA/Sydney, NSW: Academic.

    Google Scholar 

  • De Raedt, L. (2008). Logical and relational learning. Berlin, Germany: Springer.

    Book  MATH  Google Scholar 

  • Džeroski, S., & Bratko, I. (1992). Handling noise in inductive logic programming. In S. H. Muggleton & K. Furukawa (Eds.), Proceedings of the 2nd International Workshop on Inductive Logic Programming (ILP-92) (pp. 109–125). No. TM-1182 in ICOT Technical Memorandum, Institute for New Generation Computer Technology, Tokyo, Japan.

    Google Scholar 

  • Džeroski, S., & Lavrač, N. (Eds.). (2001). Relational data mining: Inductive logic programming for knowledge discovery in databases. Berlin, Germany/New York: Springer.

    MATH  Google Scholar 

  • Džeroski, S., Schulze-Kremer, S., Heidtke, K. R., Siems, K., Wettschereck, D., & Blockeel, H. (1998). Diterpene structure elucidation from 13CNMR spectra with inductive logic programming. Applied Artificial Intelligence, 12(5), 363–383. Special Issue on First-Order Knowledge Discovery in Databases.

    Google Scholar 

  • Fensel, D., & Wiese, M. (1993). Refinement of rule sets with JoJo. In P. Brazdil (Ed.), Proceedings of the 6th European Conference on Machine Learning (ECML-93), Vienna (pp. 378–383). Berlin, Germany/New York: Springer.

    Google Scholar 

  • Fensel, D., & Wiese, M. (1994). From JoJo to Frog: Extending a bi-directional strategy to a more flexible three-directional search. In C. Globig & K.-D. Althoff (eds.) Beiträge zum 7. Fachgruppentreffen Maschinelles Lernen (Tech. Rep. LSA-95-01, pp. 37–44). Zentrum für Lernende Systeme und Anwendungen, University of Kaiserslautern.

    Google Scholar 

  • Fürnkranz, J. (2002a). A pathology of bottom-up hill-climbing in inductive rule learning. In N. Cesa-Bianchi, M. Numao, & R. Reischuk (Eds.), Proceedings of the 13th European Conference on Algorithmic Learning Theory (ALT-02), Lübeck, Germany (pp. 263–277). Berlin, Germany/New York: Springer.

    Google Scholar 

  • Giordana, A., & Sale, C. (1992). Learning structured concepts using genetic algorithms. In Sleeman, D., Edwards, P. (eds.) Proceedings of the 9th International Workshop on Machine Learning (ML-92), Edinburgh, UK (pp. 169–178). San Mateo, CA: Morgan Kaufmann.

    Google Scholar 

  • Goethals, B. (2005). Frequent set mining. In O. Maimon & L. Rokach (Eds.), The data mining and knowledge discovery handbook (pp. 377–397). New York: Springer.

    Chapter  Google Scholar 

  • Goldberg, D. E. (1989). Genetic algorithms in search, optimization and machine learning. Reading, MA: Addison-Wesley.

    MATH  Google Scholar 

  • Han, J., Pei, J., Yin, Y., & Mao, R. (2004). Mining frequent patterns without candidate generation: A frequent-pattern tree approach. Data Mining and Knowledge Discovery, 8(1), 53–87.

    Article  MathSciNet  Google Scholar 

  • Hart, P. E., Nilsson, N. J., & Raphael, B. (1968). A formal basis for the heuristic determination of minimum cost paths. IEEE Transactions on Systems Science and Cybernetics, 4(2), 100–107.

    Article  Google Scholar 

  • Hipp, J., Güntzer, U., & Nakhaeizadeh, G. (2000). Algorithms for association rule mining – A general survey and comparison. SIGKDD Explorations, 2(1), 58–64.

    Article  Google Scholar 

  • Janssen, F., & Fürnkranz, J. (2009). A re-evaluation of the over-searching phenomenon in inductive rule learning. In H. Park, S. Parthasarathy, H. Liu, & Z. Obradovic (Eds.), Proceedings of the SIAM International Conference on Data Mining (SDM-09), Sparks, NV (pp. 329–340). Philadelphia: SIAM.

    Google Scholar 

  • Jovanoski, V., & Lavrač, N. (2001). Classification rule learning with APRIORI-C. In P. Brazdil & A. Jorge (Eds.), Proceedings of the 10th Portuguese Conference on Artificial Intelligence (EPIA 2001), Porto, Portugal (pp. 44–51). Berlin, Germany/New York: Springer.

    Google Scholar 

  • Kavšek, B., & Lavrač, N. (2006). Apriori-SD: Adapting association rule learning to subgroup discovery. Applied Artificial Intelligence, 20(7), 543–583.

    Article  Google Scholar 

  • Kirkpatrick, S., Gelatt, C., & Vecchi, M. (1983). Optimization by simulated annealing. Science, 220, 671–680.

    Article  MathSciNet  MATH  Google Scholar 

  • Kononenko, I., & Kovačič, M. (1992). Learning as optimization: Stochastic generation of multiple knowledge. In D. Sleeman & P. Edwards (Eds.), Proceedings of the 9th International Workshop on Machine Learning (ML-92) (pp. 257–262). San Mateo, CA: Morgan Kaufmann.

    Google Scholar 

  • Kovačič, M. (1991). Markovian neural networks. Biological Cybernetics, 64, 337–342.

    Article  MathSciNet  MATH  Google Scholar 

  • Kovačič, M. (1994b). Stochastic inductive logic programming. Ph.D. thesis, Department of Computer and Information Science, University of Ljubljana.

    Google Scholar 

  • Lavrač, N., & Džeroski, S. (1994a). Inductive logic programming: Techniques and applications. New York: Ellis Horwood.

    MATH  Google Scholar 

  • Liu, B., Hsu, W., & Ma, Y. (1998). Integrating classification and association rule mining. In R. Agrawal, P. Stolorz, & G. Piatetsky-Shapiro (Eds.), Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining (KDD-98) (pp. 80–86). Menlo Park, CA: AAAI.

    Google Scholar 

  • Michalski, R. S., Mozetič, I., Hong, J., & Lavrač, N. (1986). The multi-purpose incremental learning system AQ15 and its testing application to three medical domains. In Proceedings of the 5th National Conference on Artificial Intelligence (AAAI-86), Philadelphia (pp. 1041–1045). Menlo Park, CA: AAAI.

    Google Scholar 

  • Mitchell, T. M. (1982). Generalization as search. Artificial Intelligence, 18(2), 203–226.

    Article  MathSciNet  Google Scholar 

  • Mladenić, D. (1993). Combinatorial optimization in inductive concept learning. In Proceedings of the 10th International Conference on Machine Learning (ML-93), Amherst, MA (pp. 205–211). San Mateo, CA: Morgan Kaufmann.

    Google Scholar 

  • Muggleton, S. H. (1987). Structuring knowledge by asking questions. In Bratko, I., & Lavrač, N. (Eds.), Progress in machine learning (pp. 218–229). Wilmslow, England: Sigma Press.

    Google Scholar 

  • Muggleton, S. H. (1988). A strategy for constructing new predicates in first order logic. In Proceedings of the 3rd European Working Session on Learning (EWSL-88) (pp. 123–130). London: Pitman.

    Google Scholar 

  • Muggleton, S. H. (1991). Inverting the resolution principle. In J. E. Hayes, D. Michie, & E. Tyugu (Eds.), Machine intelligence 12 (pp. 93–103). Oxford, UK: Clarendon. Chap. 7

  • Muggleton, S. H. (1995). Inverse entailment and Progol. New Generation Computing, 13(3,4), 245–286. Special Issue on Inductive Logic Programming.

    Google Scholar 

  • Muggleton, S. H., & Buntine, W. L. (1988). Machine invention of first-order predicates by inverting resolution. In Proceedings of the 5th International Conference on Machine Learning (ML-88), Ann Arbor, MI (pp. 339–352). San Mateo, CA: Morgan Kaufmann

    Google Scholar 

  • Muggleton, S. H., & Feng, C. (1990). Efficient induction of logic programs. In Proceedings of the 1st Conference on Algorithmic Learning Theory, Tokyo (pp. 1–14). Tokyo: Japanese Society for Artificial Intelligence.

    Google Scholar 

  • Muggleton, S. H., Santos, J. C. A., & Tamaddoni-Nezhad, A. (2009). ProGolem: A system based on relative minimal generalisation. In L. De Raedt (Ed.), Proceedings of the 19th International Conference on Inductive Logic Programming (ILP-09), Leuven, Belgium (pp. 131–148). Springer.

    Google Scholar 

  • Peña Castillo, L., & Wrobel, S. (2004). A comparative study on methods for reducing myopia of hill-climbing search in multirelational learning. In C. E. Brodley (Ed.), Proceedings of the 21st International Conference on Machine Learning (ICML-2004), Banff, AB. New York: ACM.

    Google Scholar 

  • Plotkin, G. D. (1970). A note on inductive generalisation. In B. Meltzer & D. Michie (Eds.), Machine intelligence 5 (pp. 153–163). New York: Elsevier/North-Holland.

    Google Scholar 

  • Plotkin, G. D. (1971). A further note on inductive generalisation. In B. Meltzer & D. Michie (Eds.), Machine intelligence 6 (pp. 101–124). New York: Elsevier/North-Holland.

    Google Scholar 

  • Pompe, U., Kovačič, M., & Kononenko, I. (1993). SFOIL: Stochastic approach to inductive logic programming. In Proceedings of the 2nd Slovenian Conference on Electrical Engineering and Computer Science (ERK-93), Portorož, Slovenia (Vol. B, pp. 189–192).

    Google Scholar 

  • Quinlan, J. R. (1990). Learning logical definitions from relations. Machine Learning, 5, 239–266.

    Google Scholar 

  • Quinlan, J. R., & Cameron-Jones, R. M. (1995b). Oversearching and layered search in empirical learning. In C. Mellish (Ed.), Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI-95), Montréal, QC (pp. 1019–1024). San Mateo, CA: Morgan Kaufmann.

    Google Scholar 

  • Rivest, R. L. (1987). Learning decision lists. Machine Learning, 2, 229–246.

    MathSciNet  Google Scholar 

  • Rouveirol, C. (1992). Extensions of inversion of resolution applied to theory completion. In S. H. Muggleton (Ed.), Inductive logic programming (pp. 63–92). London: Academic.

    Google Scholar 

  • Rouveirol, C. (1994). Flattening and saturation: Two representation changes for generalization. Machine Learning, 14, 219–232. Special issue on Evaluating and Changing Representation.

    Google Scholar 

  • Rouveirol, C., & Puget, J. F. (1990). Beyond inversion of resolution. In Proceedings of the 7th International Conference on Machine Learning (ML-90), Austin, TX (pp. 122–130). San Mateo, CA: Morgan Kaufmann.

    Google Scholar 

  • Theron, H., & Cloete, I. (1996). BEXA: A covering algorithm for learning propositional concept descriptions. Machine Learning, 24, 5–40.

    Google Scholar 

  • Venturini, G. (1993). SIA: A supervised inductive algorithm with genetic search for learning attributes based concepts. In P. Brazdil (Ed.), Proceedings of the 6th European Conference on Machine Learning (ECML-93), Vienna (pp. 280–296). Berlin, Germany: Springer.

    Google Scholar 

  • Webb, G. I. (1992). Learning disjunctive class descriptions by least generalisation (Tech. Rep. TR C92/9). Geelong, VIC: Deakin University, School of Computing & Mathematics.

    Google Scholar 

  • Webb, G. I. (1993). Systematic search for categorical attribute-value data-driven machine learning. In C. Rowles, H. Liu, & N. Foo (Eds.), Proceedings of the 6th Australian Joint Conference of Artificial Intelligence (AI’93), Melbourne, VIC (pp. 342–347). Singapore: World Scientific.

    Google Scholar 

  • Webb, G. I. (1995). OPUS: An efficient admissible algorithm for unordered search. Journal of Artificial Intelligence Research, 5, 431–465.

    Google Scholar 

  • Webb, G. I. (2000). Efficient search for association rules. In Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2000), Boston (pp. 99–107). New York: ACM.

    Google Scholar 

  • Weiss, S. M., & Indurkhya, N. (1991). Reduced complexity rule induction. In Proceedings of the 12th International Joint Conference on Artificial Intelligence (IJCAI-91), Sydney, NSW (pp. 678–684). San Mateo, CA: Morgan Kaufmann.

    Google Scholar 

  • Widmer, G. (1993). Combining knowledge-based and instance-based learning to exploit qualitative knowledge. Informatica, 17, 371–385. Special Issue on Multistrategy Learning.

    Google Scholar 

  • Wiese, M. (1996). A bidirectional ILP algorithm. In Proceedings of the MLnet Familiarization Workshop on Data Mining with Inductive Logic Programming (ILP for KDD), Bari, Italy (pp. 61–72).

    Google Scholar 

  • Wirth, R. (1988). Learning by failure to prove. In Proceedings of the Third European Working Session on Learning, Glasgow, UK (pp. 237–251). London: Pitman.

    Google Scholar 

  • Zaki, M. J., Parthasarathy, S., Ogihara, M., & Li, W. (1997). New algorithms for fast discovery of association rules. In Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining (KDD-97), Newport, CA (pp. 283–286). Menlo Park, CA: AAAI.

    Google Scholar 

  • Zhang, C., & Zhang, S. (2002). Association rule mining: Models and algorithms. Berlin, Germany/New York: Springer.

    Book  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. FB Informatik, TU Darmstadt, Darmstadt, Germany

    Johannes Fürnkranz

  2. Rudjer Bošković Institute, Zagreb, Croatia

    Dragan Gamberger

  3. Department of Knowledge Technologies, Jožef Stefan Institute, Ljubljana, Slovenia

    Nada Lavrač

Authors
  1. Johannes Fürnkranz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dragan Gamberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nada Lavrač
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Fürnkranz, J., Gamberger, D., Lavrač, N. (2012). Learning Single Rules. In: Foundations of Rule Learning. Cognitive Technologies. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75197-7_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-75197-7_6

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-75196-0

  • Online ISBN: 978-3-540-75197-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation