Abstract
This chapter describes algorithms and strategies for constructing single rules in the concept learning framework. The starting point in Sect. 6.2 is a generic algorithm for finding the best rule which searches the hypothesis space for a rule that optimizes some quality criterion. Section 6.3 presents alternative search algorithms (heuristic search algorithms including hill-climbing and beam search, as well as exhaustive search algorithms).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The name Cigol is ‘logic’ spelled backwards.
References
Adé, H., De Raedt, L., & Bruynooghe, M. (1995). Declarative bias for specific-to-general ILP systems. Machine Learning, 20(1–2), 119–154. Special Issue on Bias Evaluation and Selection.
Agrawal, R., Mannila, H., Srikant, R., Toivonen, H., & Verkamo, A. I. (1995). Fast discovery of association rules. In U. M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, & R. Uthurusamy (Eds.), Advances in knowledge discovery and data mining (pp. 307–328). Menlo Park, CA: AAAI.
Bayardo, R. J., Jr. (1997). Brute-force mining of high-confidence classification rules. In Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining (KDD-97) (pp. 123–126). Menlo Park, CA: AAAI.
Bergadano, F., Giordana, A., & Saitta, L. (1988). Automated concept acquisition in noisy environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10, 555–578.
Botta, M., Giordana, A., & Saitta, L. (1992). Comparison of search strategies in learning relations. In B. Neumann (Ed.), Proceedings of the 10th European Conference on Artificial Intelligence (ECAI-92), Vienna (pp. 451–455). Chichester, UK/New York: Wiley.
Clark, P., & Niblett, T. (1989). The CN2 induction algorithm. Machine Learning, 3(4), 261–283.
De Raedt, L. (1992). Interactive theory revision: An inductive logic programming approach. London/San Diego, CA/Sydney, NSW: Academic.
De Raedt, L. (2008). Logical and relational learning. Berlin, Germany: Springer.
Džeroski, S., & Bratko, I. (1992). Handling noise in inductive logic programming. In S. H. Muggleton & K. Furukawa (Eds.), Proceedings of the 2nd International Workshop on Inductive Logic Programming (ILP-92) (pp. 109–125). No. TM-1182 in ICOT Technical Memorandum, Institute for New Generation Computer Technology, Tokyo, Japan.
Džeroski, S., & Lavrač, N. (Eds.). (2001). Relational data mining: Inductive logic programming for knowledge discovery in databases. Berlin, Germany/New York: Springer.
Džeroski, S., Schulze-Kremer, S., Heidtke, K. R., Siems, K., Wettschereck, D., & Blockeel, H. (1998). Diterpene structure elucidation from 13CNMR spectra with inductive logic programming. Applied Artificial Intelligence, 12(5), 363–383. Special Issue on First-Order Knowledge Discovery in Databases.
Fensel, D., & Wiese, M. (1993). Refinement of rule sets with JoJo. In P. Brazdil (Ed.), Proceedings of the 6th European Conference on Machine Learning (ECML-93), Vienna (pp. 378–383). Berlin, Germany/New York: Springer.
Fensel, D., & Wiese, M. (1994). From JoJo to Frog: Extending a bi-directional strategy to a more flexible three-directional search. In C. Globig & K.-D. Althoff (eds.) Beiträge zum 7. Fachgruppentreffen Maschinelles Lernen (Tech. Rep. LSA-95-01, pp. 37–44). Zentrum für Lernende Systeme und Anwendungen, University of Kaiserslautern.
Fürnkranz, J. (2002a). A pathology of bottom-up hill-climbing in inductive rule learning. In N. Cesa-Bianchi, M. Numao, & R. Reischuk (Eds.), Proceedings of the 13th European Conference on Algorithmic Learning Theory (ALT-02), Lübeck, Germany (pp. 263–277). Berlin, Germany/New York: Springer.
Giordana, A., & Sale, C. (1992). Learning structured concepts using genetic algorithms. In Sleeman, D., Edwards, P. (eds.) Proceedings of the 9th International Workshop on Machine Learning (ML-92), Edinburgh, UK (pp. 169–178). San Mateo, CA: Morgan Kaufmann.
Goethals, B. (2005). Frequent set mining. In O. Maimon & L. Rokach (Eds.), The data mining and knowledge discovery handbook (pp. 377–397). New York: Springer.
Goldberg, D. E. (1989). Genetic algorithms in search, optimization and machine learning. Reading, MA: Addison-Wesley.
Han, J., Pei, J., Yin, Y., & Mao, R. (2004). Mining frequent patterns without candidate generation: A frequent-pattern tree approach. Data Mining and Knowledge Discovery, 8(1), 53–87.
Hart, P. E., Nilsson, N. J., & Raphael, B. (1968). A formal basis for the heuristic determination of minimum cost paths. IEEE Transactions on Systems Science and Cybernetics, 4(2), 100–107.
Hipp, J., Güntzer, U., & Nakhaeizadeh, G. (2000). Algorithms for association rule mining – A general survey and comparison. SIGKDD Explorations, 2(1), 58–64.
Janssen, F., & Fürnkranz, J. (2009). A re-evaluation of the over-searching phenomenon in inductive rule learning. In H. Park, S. Parthasarathy, H. Liu, & Z. Obradovic (Eds.), Proceedings of the SIAM International Conference on Data Mining (SDM-09), Sparks, NV (pp. 329–340). Philadelphia: SIAM.
Jovanoski, V., & Lavrač, N. (2001). Classification rule learning with APRIORI-C. In P. Brazdil & A. Jorge (Eds.), Proceedings of the 10th Portuguese Conference on Artificial Intelligence (EPIA 2001), Porto, Portugal (pp. 44–51). Berlin, Germany/New York: Springer.
Kavšek, B., & Lavrač, N. (2006). Apriori-SD: Adapting association rule learning to subgroup discovery. Applied Artificial Intelligence, 20(7), 543–583.
Kirkpatrick, S., Gelatt, C., & Vecchi, M. (1983). Optimization by simulated annealing. Science, 220, 671–680.
Kononenko, I., & Kovačič, M. (1992). Learning as optimization: Stochastic generation of multiple knowledge. In D. Sleeman & P. Edwards (Eds.), Proceedings of the 9th International Workshop on Machine Learning (ML-92) (pp. 257–262). San Mateo, CA: Morgan Kaufmann.
Kovačič, M. (1991). Markovian neural networks. Biological Cybernetics, 64, 337–342.
Kovačič, M. (1994b). Stochastic inductive logic programming. Ph.D. thesis, Department of Computer and Information Science, University of Ljubljana.
Lavrač, N., & Džeroski, S. (1994a). Inductive logic programming: Techniques and applications. New York: Ellis Horwood.
Liu, B., Hsu, W., & Ma, Y. (1998). Integrating classification and association rule mining. In R. Agrawal, P. Stolorz, & G. Piatetsky-Shapiro (Eds.), Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining (KDD-98) (pp. 80–86). Menlo Park, CA: AAAI.
Michalski, R. S., Mozetič, I., Hong, J., & Lavrač, N. (1986). The multi-purpose incremental learning system AQ15 and its testing application to three medical domains. In Proceedings of the 5th National Conference on Artificial Intelligence (AAAI-86), Philadelphia (pp. 1041–1045). Menlo Park, CA: AAAI.
Mitchell, T. M. (1982). Generalization as search. Artificial Intelligence, 18(2), 203–226.
Mladenić, D. (1993). Combinatorial optimization in inductive concept learning. In Proceedings of the 10th International Conference on Machine Learning (ML-93), Amherst, MA (pp. 205–211). San Mateo, CA: Morgan Kaufmann.
Muggleton, S. H. (1987). Structuring knowledge by asking questions. In Bratko, I., & Lavrač, N. (Eds.), Progress in machine learning (pp. 218–229). Wilmslow, England: Sigma Press.
Muggleton, S. H. (1988). A strategy for constructing new predicates in first order logic. In Proceedings of the 3rd European Working Session on Learning (EWSL-88) (pp. 123–130). London: Pitman.
Muggleton, S. H. (1991). Inverting the resolution principle. In J. E. Hayes, D. Michie, & E. Tyugu (Eds.), Machine intelligence 12 (pp. 93–103). Oxford, UK: Clarendon. Chap. 7
Muggleton, S. H. (1995). Inverse entailment and Progol. New Generation Computing, 13(3,4), 245–286. Special Issue on Inductive Logic Programming.
Muggleton, S. H., & Buntine, W. L. (1988). Machine invention of first-order predicates by inverting resolution. In Proceedings of the 5th International Conference on Machine Learning (ML-88), Ann Arbor, MI (pp. 339–352). San Mateo, CA: Morgan Kaufmann
Muggleton, S. H., & Feng, C. (1990). Efficient induction of logic programs. In Proceedings of the 1st Conference on Algorithmic Learning Theory, Tokyo (pp. 1–14). Tokyo: Japanese Society for Artificial Intelligence.
Muggleton, S. H., Santos, J. C. A., & Tamaddoni-Nezhad, A. (2009). ProGolem: A system based on relative minimal generalisation. In L. De Raedt (Ed.), Proceedings of the 19th International Conference on Inductive Logic Programming (ILP-09), Leuven, Belgium (pp. 131–148). Springer.
Peña Castillo, L., & Wrobel, S. (2004). A comparative study on methods for reducing myopia of hill-climbing search in multirelational learning. In C. E. Brodley (Ed.), Proceedings of the 21st International Conference on Machine Learning (ICML-2004), Banff, AB. New York: ACM.
Plotkin, G. D. (1970). A note on inductive generalisation. In B. Meltzer & D. Michie (Eds.), Machine intelligence 5 (pp. 153–163). New York: Elsevier/North-Holland.
Plotkin, G. D. (1971). A further note on inductive generalisation. In B. Meltzer & D. Michie (Eds.), Machine intelligence 6 (pp. 101–124). New York: Elsevier/North-Holland.
Pompe, U., Kovačič, M., & Kononenko, I. (1993). SFOIL: Stochastic approach to inductive logic programming. In Proceedings of the 2nd Slovenian Conference on Electrical Engineering and Computer Science (ERK-93), Portorož, Slovenia (Vol. B, pp. 189–192).
Quinlan, J. R. (1990). Learning logical definitions from relations. Machine Learning, 5, 239–266.
Quinlan, J. R., & Cameron-Jones, R. M. (1995b). Oversearching and layered search in empirical learning. In C. Mellish (Ed.), Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI-95), Montréal, QC (pp. 1019–1024). San Mateo, CA: Morgan Kaufmann.
Rivest, R. L. (1987). Learning decision lists. Machine Learning, 2, 229–246.
Rouveirol, C. (1992). Extensions of inversion of resolution applied to theory completion. In S. H. Muggleton (Ed.), Inductive logic programming (pp. 63–92). London: Academic.
Rouveirol, C. (1994). Flattening and saturation: Two representation changes for generalization. Machine Learning, 14, 219–232. Special issue on Evaluating and Changing Representation.
Rouveirol, C., & Puget, J. F. (1990). Beyond inversion of resolution. In Proceedings of the 7th International Conference on Machine Learning (ML-90), Austin, TX (pp. 122–130). San Mateo, CA: Morgan Kaufmann.
Theron, H., & Cloete, I. (1996). BEXA: A covering algorithm for learning propositional concept descriptions. Machine Learning, 24, 5–40.
Venturini, G. (1993). SIA: A supervised inductive algorithm with genetic search for learning attributes based concepts. In P. Brazdil (Ed.), Proceedings of the 6th European Conference on Machine Learning (ECML-93), Vienna (pp. 280–296). Berlin, Germany: Springer.
Webb, G. I. (1992). Learning disjunctive class descriptions by least generalisation (Tech. Rep. TR C92/9). Geelong, VIC: Deakin University, School of Computing & Mathematics.
Webb, G. I. (1993). Systematic search for categorical attribute-value data-driven machine learning. In C. Rowles, H. Liu, & N. Foo (Eds.), Proceedings of the 6th Australian Joint Conference of Artificial Intelligence (AI’93), Melbourne, VIC (pp. 342–347). Singapore: World Scientific.
Webb, G. I. (1995). OPUS: An efficient admissible algorithm for unordered search. Journal of Artificial Intelligence Research, 5, 431–465.
Webb, G. I. (2000). Efficient search for association rules. In Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2000), Boston (pp. 99–107). New York: ACM.
Weiss, S. M., & Indurkhya, N. (1991). Reduced complexity rule induction. In Proceedings of the 12th International Joint Conference on Artificial Intelligence (IJCAI-91), Sydney, NSW (pp. 678–684). San Mateo, CA: Morgan Kaufmann.
Widmer, G. (1993). Combining knowledge-based and instance-based learning to exploit qualitative knowledge. Informatica, 17, 371–385. Special Issue on Multistrategy Learning.
Wiese, M. (1996). A bidirectional ILP algorithm. In Proceedings of the MLnet Familiarization Workshop on Data Mining with Inductive Logic Programming (ILP for KDD), Bari, Italy (pp. 61–72).
Wirth, R. (1988). Learning by failure to prove. In Proceedings of the Third European Working Session on Learning, Glasgow, UK (pp. 237–251). London: Pitman.
Zaki, M. J., Parthasarathy, S., Ogihara, M., & Li, W. (1997). New algorithms for fast discovery of association rules. In Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining (KDD-97), Newport, CA (pp. 283–286). Menlo Park, CA: AAAI.
Zhang, C., & Zhang, S. (2002). Association rule mining: Models and algorithms. Berlin, Germany/New York: Springer.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Fürnkranz, J., Gamberger, D., Lavrač, N. (2012). Learning Single Rules. In: Foundations of Rule Learning. Cognitive Technologies. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75197-7_6
Download citation
DOI: https://doi.org/10.1007/978-3-540-75197-7_6
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-75196-0
Online ISBN: 978-3-540-75197-7
eBook Packages: Computer ScienceComputer Science (R0)