Artificial Intelligence Review

, Volume 13, Issue 1, pp 3–54 | Cite as

Separate-and-Conquer Rule Learning

  • Johannes Fürnkranz
Article

Abstract

This paper is a survey of inductive rule learning algorithms that use a separate-and-conquer strategy. This strategy can be traced back to the AQ learning system and still enjoys popularity as can be seen from its frequent use in inductive logic programming systems. We will put this wide variety of algorithms into a single framework and analyze them along three different dimensions, namely their search, language and overfitting avoidance biases.

covering inductive logic programming inductive rule learning 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adé, H., De Raedt, L. & Bruynooghe, M. (1995). Declarative Bias for Specific-to-General ILP Systems. Machine Learning 20(1–2): 119-154. Special Issue on Bias Evaluation and Selection.Google Scholar
  2. Ali, K. M. & Pazzani, M. J. (1993). HYDRA: A Noise-Tolerant Relational Concept Learning Algorithm. In Bajcsy, R. (ed.) Proceedings of the 13th International Joint Conference on Artificial Intelligence (IJCAI-93), 1064-1071. Morgan Kaufmann: Chambèry, France.Google Scholar
  3. Bain, M. (1991). Experiments in Non-Monotonic Learning. In Proceedings of the 8th International Workshop on Machine Learning (ML-91), 380-384. Evanston, Illinois.Google Scholar
  4. Bergadano, F. & Giordana, A. (1988). A Knowledge Intensive Approach to Concept Induction. In Proceedings of the 5th International Conference on Machine Learning (ML-88), 305-317. Ann Arbor, Michigan.Google Scholar
  5. Bergadano, F., Giordana, A. & Ponsero, S. (1989). Deduction in Top-Down Inductive Learning. In Proceedings of the 6th International Workshop on Machine Learning (ML-89), 23-25.Google Scholar
  6. Bergadano, F., Giordana, A. & Saitta, L. (1988). Automated Concept Acquisition in Noisy Environments. IEEE Transactions on Pattern Analysis and Machine Intelligence 10: 555-578.Google Scholar
  7. Bergadano, F. & Gunetti, D. (1993). An Interactive System to Learn Functional Logic Programs. In Bajcsy, R. (ed.) Proceedings of the 13th International Joint Conference on Artificial Intelligence (IJCAI-93), 1044-1049. Morgan Kaufmann.Google Scholar
  8. Bergadano, F. & Gunetti, D. (1995). Inductive Logic Programming — From Machine Learning to Software Engineering. Logic Programming Series. The MIT Press: Cambridge, MA.Google Scholar
  9. Bergadano, F., Matwin, S., Michalski, R. S. & Zhang, J. (1992). Learning Two-Tiered Descriptions of Flexible Concepts: The POSEIDON System. Machine Learning 8: 5-43.Google Scholar
  10. Blockeel, H. & De Raedt, L. (1997). Top-Down Induction of Logical Decision Trees. Tech. rep. CW 247, Katholieke Universiteit Leuven, Department of Computer Science, Leuven, Belgium.Google Scholar
  11. Bloedorn, E. & Michalski, R. S. (1991). Constructive Induction From Data in AQ17-DCI: Further Experiments. Tech. rep. MLI 91-12, Artificial Intelligence Center, George Mason University, Fairfax, VA.Google Scholar
  12. Bloedorn, E., Michalski, R. S. & Wnek, J. (1993). Multistrategy Constructive Induction: AQ17-MCI. In Proceedings of the 2nd International Workshop on Multistrategy Learning, 188-203.Google Scholar
  13. Boström, H. (1995). Covering vs. Divide-and-Conquer for Top-Down Induction of Logic Programs. In Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI-95), 1194-1200.Google Scholar
  14. Botta, M., Giordana, A. & Saitta, L. (1992). Comparison of Search Strategies in Learning Relations. In Neumann, B. (ed.) Proceedings of the 10th European Conference on Artificial Intelligence (ECAI-92), 451-455. John Wiley & Sons: Vienna, Austria.Google Scholar
  15. Botta, M. & Giordana, A. (1993). SMART+: A Multi-Strategy Learning Tool. In Bajcsy, R. (ed.) Proceedings of the 13th International Joint Conference on Artificial Intelligence (IJCAI-93), 937-944. Morgan Kaufmann: Chambèry, France.Google Scholar
  16. Breiman, L., Friedman, J., Olshen, R. & Stone, C. (1984). Classification and Regression Trees. Wadsworth & Brooks: Pacific Grove, CA.Google Scholar
  17. Brunk, C. A. & Pazzani, M. J. (1991). An Investigation of Noise-Tolerant Relational Concept Learning Algorithms. In Proceedings of the 8th International Workshop on Machine Learning (ML-91), 389-393. Morgan Kaufmann: Evanston, Illinois.Google Scholar
  18. Buntine, W. & Niblett, T. (1992). A Further Comparison of Splitting Rules for Decision-Tree Induction. Machine Learning 8: 75-85.Google Scholar
  19. Cameron-Jones, R. M. (1996). The Complexity of Batch Approaches to Reduced Error Rule Set Induction. In Foo, N. & Goebel, R. (eds.) Proceedings of the 4th Pacific Rim International Conference on Artificial Intelligence (PRICAI-96), 348-359. Springer-Verlag: Cairns, Australia.Google Scholar
  20. Cameron-Jones, R. M. & Quinlan, J. R. (1993). Avoiding Pitfalls When Learning Recursive Theories. In Bajcsy, R. (ed.) Proceedings of the 13th International Joint Conference on Artificial Intelligence (IJCAI-93), 1050-1057. Chambèry, France.Google Scholar
  21. Cendrowska, J. (1987). PRISM: An Algorithm for Inducing Modular Rules. International Journal of Man-Machine Studies 27: 349-370.Google Scholar
  22. Clark, P. & Boswell, R. (1991). Rule Induction with CN2: Some Recent Improvements. In Proceedings of the 5th European Working Session on Learning (EWSL-91), 151-163. Springer-Verlag: Porto, Portugal.Google Scholar
  23. Clark, P. & Niblett, T. (1989). The CN2 Induction Algorithm. Machine Learning 3(4): 261-283.Google Scholar
  24. Cohen, W. W. (1993). Efficient Pruning Methods for Separate-and-Conquer Rule Learning Systems. In Proceedings of the 13th International Joint Conference on Artificial Intelligence (IJCAI-93), 988-994. Morgan Kaufmann: Chambèry, France.Google Scholar
  25. Cohen, W. W. (1994). Grammatically Biased Learning: Learning Logic Programs Using an Explicit Antecedent Description Language. Artificial Intelligence 68(2): 303-366.Google Scholar
  26. Cohen, W. W. (1995). Fast Effective Rule Induction. In Prieditis, A. & Russell, S. (eds.) Proceedings of the 12th International Conference on Machine Learning (ML-95), 115-123. Morgan Kaufmann: Lake Tahoe, CA.Google Scholar
  27. De Raedt, L. (1992). Interactive Theory Revision: An Inductive Logic Programming Approach. Academic Press.Google Scholar
  28. De Raedt, L. (ed.) (1995). Advances in Inductive Logic Programming, Vol. 32 of Frontiers in Artificial Intelligence and Applications. IOS Press.Google Scholar
  29. De Raedt, L, & Bruynooghe, M. (1990). Indirect Relevance and Bias in Inductive Concept Learning. Knowledge Acquisition 2: 365-390.Google Scholar
  30. De Raedt, L. & Van Laer, W. (1995). Inductive Constraint Logic. In Proceedings of the 5th Workshop on Algorithmic Learning Theory (ALT-95), 80-94. Springer-Verlag.Google Scholar
  31. Dehaspe, L. & De Raedt, L. (1996). DLAB: A Declarative Language Bias Formalism. In Proceedings of the International Symposium on Methodologies for Intelligent Systems (ISMIS-96), 613-622.Google Scholar
  32. Domingos, P. (1996a). Linear-Time Rule Induction. In Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining (KDD-96), 96-101. AAAI Press.Google Scholar
  33. Domingos, P. (1996b). Unifying Instance-Based and Rule-Based Induction. Machine Learning 24: 141-168.Google Scholar
  34. Džeroski, S. & Bratko, I. (1992). Handling Noise in Inductive Logic Programming. In Proceedings of the International Workshop on Inductive Logic Programming (ILP-92), 109-125. Tokyo, Japan.Google Scholar
  35. Džeroski, S., Schulze-Kremer, S., Heidtke, K. R., Siems, K. & Wettschereck, D. (1996). Applying ILP to Diterpene Structure Elucidation from 13C NMR Spectra. In Proceedings of the MLnet Familiarization Workshop on Data Mining with Inductive Logic Programming (ILP for KDD), 12-24.Google Scholar
  36. Fensel, D. & Wiese, M. (1993). Refinement of Rule Sets with JoJo. In Brazdil, P. (ed.) Proceedings of the 6th European Conference on Machine Learning (ECML-93), No. 667 in Lecture Notes in Artificial Intelligence, 378-383. Springer-Verlag.Google Scholar
  37. Fensel, D. & Wiese, M. (1994). From JoJo to Frog: Extending a Bidirectional Strategy to a More Flexible Three-Directional Search. In Globig, C. & Althoff, K.-D. (eds.) Beiträge zum 7. Fachgruppentreffen Maschinelles Lernen, Forschungbericht No. LSA-95-01, 37-44. University of Kaiserslautern. Zentrum für Lernende Systeme und Anwendungen.Google Scholar
  38. Fürnkranz, J. (1994a). Efficient Pruning Methods for Relational Learning. Ph.D. thesis, Vienna University of Technology.Google Scholar
  39. Fürnkranz, J. (1994b). FOSSIL: A Robust Relational Learner. In Bergadano, F. & De Raedt, L. (eds.) Proceedings of the 7th European Conference on Machine Learning (ECML-94), Vol. 784 of Lecture Notes in Artificial Intelligence, 122-137. Springer-Verlag: Catania, Italy.Google Scholar
  40. Fürnkranz, J. (1994c). Top-Down Pruning in Relational Learning. In Cohn, A. (ed.) Proceedings of the 11th European Conference on Artificial Intelligence (ECAI-94), 453-457. John Wiley & Sons: Amsterdam, The Netherlands.Google Scholar
  41. Fürnkranz, J. (1995). A Tight Integration of Pruning and Learning (Extended Abstract). In Lavrač, N. & Wrobel, S. (eds.) Proceedings of the 8th European Conference on Machine Learning (EMCL-95), Vol. 912 of Lecture Notes in Artificial Intelligence, 291-294. Springer-Verlag: Heraclion, Greece.Google Scholar
  42. Fürnkranz, J. (1997). Pruning Algorithms for Rule Learning. Machine Learning 27(2): 139-171.Google Scholar
  43. Fürnkranz, J. & Widmer, G. (1994). Incremental Reduced Error Pruning. In Cohen W. & Hirsh, H. (eds.) Proceedings of the 11th International Conference on Machine Learning (ML-94), 70-77. Morgan Kaufmann: New Brunswick, NJ.Google Scholar
  44. Georgeff, M. P. & Wallace, C. S. (1984). A General Criterion for Inductive Inference. In O'Shea, T. (ed.) Proceedings of the Sixth European Conference on Artificial Intelligence (ECAI-84), 473-482. Elsevier: Amsterdam.Google Scholar
  45. Giordana, A. & Sale, C. (1992). Learning Structured Concepts Using Genetic Algorithms. In Sleeman, D. & Edwards, P. (eds.) Proceedings of the 9th International Workshop on Machine Learning (ML-92), 169-178. Morgan Kaufmann: Edinburgh.Google Scholar
  46. Goldberg, D. E. (1989). Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley: Reading, MA.Google Scholar
  47. Goodman, R. M. & Smyth, P. (1988). Information-Theoretic Rule Induction. In Kodratoff, Y. (ed.) Proceedings of the 8th European Conference on Artificial Intelligence (ECAI-88), 357-362. Pitman: London.Google Scholar
  48. Grobelnik, M. (1992). Markus — An Optimized Model Inference System. In Rouveirol, C. (ed.) Proceedings of the ECAI-92 Workshop on Logical Approaches to Machine Learning. Vienna, Austria.Google Scholar
  49. Hart, P. E., Nilsson, N. J. & Raphael, B. (1968). A Formal Basis for the Heuristic Determination of Minimum Cost Paths. IEEE Transactions on Systems Science and Cybernetics 4(2): 100-107.Google Scholar
  50. Helft, N. (1989). Induction as Nonmonotonic Inference. In Proceedings of the 1st International Conference on Principles of Knowledge Representation and Reasoning, 149-156.Google Scholar
  51. Holte, R., Acker L. & Porter, B. (1989). Concept Learning and the Problem of Small Disjuncts. In Proceedings of the 11th International Joint Conference on Artificial Intelligence (IJCAI-89), 813-818. Morgan Kaufmann: Detroit, MI.Google Scholar
  52. Karalič, A. (1995). First Order Regression. Ph.D. thesis, University of Ljubljana, Faculty of Electrical Engineering and Computer Science, Slovenia.Google Scholar
  53. Kietz, J.-U. & Wrobel, S. (1992). Controlling the Complexity of Learning in Logic Through Syntactic and Task-Oriented Models. In Muggleton, S. H. (ed.) Inductive Logic Programming, chap. 16, 335-359. Academic Press Ltd.: London.Google Scholar
  54. Kijsirikul, B., Numao, M. & Shimura, M. (1991). Efficient Learning of Logic Programs with Non-Determinate, Non-Discriminating Literals. In Proceedings of the 8th International Workshop on Machine Learning (ML91), 417-421. Evanston, Illinois.Google Scholar
  55. Kijsirikul, B., Numao, M. & Shimura, M. (1992). Discrimination-Based Constructive Induction of Logic Programs. In Proceedings of the 10th National Conference on Artificial Intelligence (AAAI-92), 44-49.Google Scholar
  56. Kirkpatrick, S., Gelatt, C. & Vecchi, M. (1983). Optimization by Simulated Annealing. Science 220: 671-680.Google Scholar
  57. Kohavi, R. (1995). Wrappers for Performance Enhancement and Oblivious Decision Graphs. Ph.D. thesis, Stanford University, Dept. of Computer Science.Google Scholar
  58. Kohavi, R. & John, G. H. (1995). Automatic Parameter Selection by Minimizing Estimated Error. In Prieditis, A. & Russell, S. (eds.) Proceedings of the 12th International Conference on Machine Learning (ICML-95), 304-312. Morgan Kaufmann.Google Scholar
  59. Kononenko, I. & Kovačič, M. (1992). Learning as Optimization: Stochastic Generation of Multiple Knowledge. In Sleeman, D. & Edwards, P. (eds.) Proceedings of the 9th International Workshop on Machine Learning (ML-92), 257-262. Morgan Kaufmann.Google Scholar
  60. Kovačič, M. (1991). Markovian Neural Networks. Biological Cybernetics 64: 337-342.Google Scholar
  61. Kovačič, M. (1994a). MDL-Heuristics in ILP Revised. In Proceedings of the ML-COLT-94 Workshop on Applications of Descriptional Complexity to Inductive, Statistical, and Visual Inference.Google Scholar
  62. Kovačič, M. (1994b). Stochastic Inductive Logic Programming. Ph.D. thesis, Department of Computer and Information Science, University of Ljubljana.Google Scholar
  63. Kramer, S. (1994). CN2-MCI: A Two-Step Method for Constructive Induction. In Proceedings of the ML-COLT-94 Workshop on Constructive Induction and Change of Representation.Google Scholar
  64. Kramer, S. (1996). Structural Regression Trees. In Proceedings of the 13th National Conference on Artificial Intelligence (AAAI-96), 812-819. AAAI Press.Google Scholar
  65. Langley, P., Simon, H. A. & Bradshaw, G. L. (1987). Heuristics for Empirical Discovery. In Bolc, L. (ed.) Computational Models of Learning. Springer-Verlag. Reprinted in Shavlik, J. W. & Dietterich, T. G. (ed.) Reading in Machine Learning. Morgan Kaufmann, 1991.Google Scholar
  66. Lavrač, N., Džeroski, S. & Grobelnik, M. (1991). Learning Nonrecursive Definitions of Relations with LINUS. In Proceedings of the 5th European Working Session on Learning (EWSL-91), 265-281. Springer-Verlag: Porto, Portugal.Google Scholar
  67. Lloyd, J. W. (1987). Foundations of Logic Programming (2nd, extended edition). Springer-Verlag: Berlin.Google Scholar
  68. Matheus, C. J. (1989). A Constructive Induction Framework. In Proceedings of the 6th International Workshop on Machine Learning, 474-475.Google Scholar
  69. Michalski, R. S. (1969). On the Quasi-Minimal Solution of the Covering Problem. In Proceedings of the 5th International Symposium on Information Processing (FCIP-69), Vol. A3 (Switching Circuits), 125-128. Bled, Yugoslavia.Google Scholar
  70. Michalski, R. S. (1973). AQVAL/1 — Computer Implementation of a Variable-Valued Logic System VL1 and Examples of Its Application to Pattern Recognition. In Proceedings of the 1st International Joint Conference on Pattern Recognition, 3-17.Google Scholar
  71. Michalski, R. S. (1980). Pattern Recognition and Rule-Guided Inference. IEEE Transactions on Pattern Analysis and Machine Intelligence 2: 349-361.Google Scholar
  72. Michalski, R. S. (1983). A Theory and Methodology of Inductive Learning. Artificial Intelligence 20(2): 111-162.Google Scholar
  73. Michalski, R. S., Mozetič, I., Hong, J. & Lavrač, N. (1986). The Multi-Purpose Incremental Learning System AQ15 and Its Testing Application to Three Medical Domains. In Proceedings of the 5th National Conference on Artificial Intelligence (AAAI-86), 1041-1045. Philadephia, PA.Google Scholar
  74. Mingers, J. (1989). An Empirical Comparison of Selection Measures for Decision-Tree Induction. Machine Learning 3: 319-342.Google Scholar
  75. Mitchell, T. M. (1980). The Need for Biases in Learning Generalizations. Tech. rep., Computer Science Department, Rutgers University, New Brunswick, MA. Reprinted in Shavlik, J. W. & Dietterich, T. G. (eds.) Readings in Machine Learning. Morgan Kaufmann, 1991.Google Scholar
  76. Mittenecker, E. (1977). Planung und statistische Auswertung von Experimenten (8th edition). Verlag Franz Deuticke: Vienna, Austria. In German.Google Scholar
  77. Mladenić, D. (1993) Combinatorial Optimization in Inductive Concept Learning. In Proceedings of the 10th International Conference on Machine Learning (ML-93), 205-211. Morgan Kaufmann.Google Scholar
  78. Mooney, R. J. (1995). Encouraging Experimental Results on Learning CNF. Machine Learning 19: 79-92.Google Scholar
  79. Mooney, R. J. & Califf, M. E. (1995). Induction of First-Order Decision Lists: Results on Learning the Past Tense of English Verbs. Journal of Artificial Intelligence Research 3: 1-24.Google Scholar
  80. Muggleton, S., Bain, M., Hayes-Michie, J. & Michie, D. (1989). An Experimental Comparison of Human and Machine Learning Formalisms. In Proceedings of the 6th International Workshop on Machine Learning (ML-93), 113-118. Morgan Kaufmann.Google Scholar
  81. Muggleton, S. H. (ed.) (1992). Inductive Logic Programming. Academic Press Ltd.: London.Google Scholar
  82. Muggleton, S. H. (1995). Inverse Entailment and Progol. New Generation Computing 13(3,4): 245-286. Special Issue on Inductive Logic Programming.Google Scholar
  83. Muggleton, S. H. & Feng, C. (1990). Efficient Induction of Logic Programs. In Proceedings of the 1st Conference on Algorithmic Learning Theory, 1-14. Tokyo, Japan.Google Scholar
  84. Nédellec, C., Rouveirol, C., Adé, H., Bergadano, F. & Tausend, B. (1996). Declarative Bias in ILP. In De Raedt, L. (ed.) Advances in Inductive Logic Programming, Vol. 32 of Frontiers in Artificial Intelligence and Applications, 82-103. IOS Press: Amsterdam.Google Scholar
  85. Pagallo, G. & Haussler, D. (1990). Boolean Feature Discovery in Empirical Learning. Machine Learning 5: 71-99.Google Scholar
  86. Pazzani, M. & Kibler, D. (1992). The Utility of Knowledge in Inductive Learning. Machine Learning 9: 57-94.Google Scholar
  87. Pazzani, M., Merz, C., Murphy, P., Ali, K., Hume, T. & Brunk, C. (1994). Reducing Misclassification Costs. In Cohen, W.W. & Hirsh, H. (eds.) Proceedings of the 11th International Conference on Machine Learning (ML-94), 217-225. Morgan Kaufmann: New Brunswick.Google Scholar
  88. Pfahringer, B. (1994a). Controlling Constructive Induction in CiPF: An MDL Approach. In Brazdil, P. B. (ed.) Proceedings of the 7th European Conference on Machine Learning (ECML-94), Lecture Notes in Artificial Intelligence, 242-256. Springer-Verlag, Catania, Sicily.Google Scholar
  89. Pfahringer, B. (1994b). Robust Constructive Induction. In Nebel, B. & Dreschler-Fischer, F. (eds.) Proceedings of the 18th German Annual Conference on Artificial Intelligence (KI-94), Lecture Notes in Artificial Intelligence, 118-129. Springer-Verlag.Google Scholar
  90. Pfahringer, B. (1995a). A New MDL Measure for Robust Rule Induction (Extended Abstract). In Lavrač, N. & Wrobel, S. (eds.), Proceedings of the 8th European Conference on Machine Learning (ECML-95), No. 912 in Lecture Notes in Artificial Intelligence, 331-334. Springer-Verlag: Heraclion, Greece.Google Scholar
  91. Pfahringer, B. (1995b). Practical Uses of the Minimum Description Length Principle in Inductive Learning. Ph.D. thesis, Technische Universität Wien.Google Scholar
  92. Plotkin, G. D. (1970). A Note on Inductive Generalisation. In Meltzer B. & Michie, D. (eds.) Machine Intelligence 5, 153-163. Elsevier North-Holland/New York.Google Scholar
  93. Plotkin, G. D. (1971). A Further Note on Inductive Generalisation. In Meltzer B. & Michie, D. (eds.) Machine Intelligence 6, 101-124. Elsevier North-Holland/New York.Google Scholar
  94. Pompe, U., Kovačič, M. & Kononenko, I. (1993). SFOIL: Stochastic Approach to Inductive Logic Programming. In Proceedings of the 2nd Slovenian Conference on Electrical Engineering and Computer Science (ERK-93), Vol. B, 189-192. Portorož, Slovenia.Google Scholar
  95. Quinlan, J. R. (1983). Learning Efficient Classification Procedures and Their Application to Chess End Games. In Michalski, R. S., Carbonell, J. G. & Mitchell, T. M. (eds.) Machine Learning. An Artificial Intelligence Approach, 463-482. Tioga: Palo Alto, CA.Google Scholar
  96. Quinlan, J. R. (1986). Induction of Decision Trees. Machine Learning 1: 81-106.Google Scholar
  97. Quinlan, J. R. (1987a). Generating Production Rules from Decision Trees. In Proceedings of the 10th International Joint Conference on Artificial Intelligence (IJCAI-87), 304-307. Morgan Kaufmann.Google Scholar
  98. Quinlan, J. R. (1987b). Simplifying Decision Trees. International Journal of Man-Machine Studies 27: 221-234.Google Scholar
  99. Quinlan, J. R. (1990). Learning Logical Definitions from Relations. Machine Learning 5: 239-266.Google Scholar
  100. Quinlan, J. R. (1991). Determinate Literals in Inductive Logic Programming. In Proceedings of the 8th International Workshop on Machine Learning (ML-91), 442-446.Google Scholar
  101. Quinlan, J. R. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann: San Mateo, CA.Google Scholar
  102. Quinlan, J. R. (1994). The Minimum Description Length Principle and Categorical Theories. In Cohen, W. & Hirsh, H. (eds.) Proceedings of the 11th International Conference on Machine Learning (ML-94), 233-241. Morgan Kaufmann: New Brunswick, NJ.Google Scholar
  103. Quinlan, J. R. (1996). Learning First-Order Definitions of Functions. Journal of Artificial Intelligence Research 5: 139-161.Google Scholar
  104. Quinlan, J. R. & Cameron-Jones, R. M. (1995a). Induction of Logic Programs: FOIL and Related Systems. New Generation Computing 13(3,4): 287-312. Special Issue on Inductive Logic Programming.Google Scholar
  105. Quinlan, J. R. & Cameron-Jones, R. M. (1995b). Oversearching and Layered Search in Empirical Learning. In Mellish, C. (ed.) Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI-95), 1019-1024. Morgan Kaufmann.Google Scholar
  106. Rissanen, J. (1978). Modeling by Shortest Data Description. Automatica 14: 465-471.Google Scholar
  107. Rivest, R. L. (1987). Learning Decision Lists. Machine Learning 2: 229-246.Google Scholar
  108. Rouveirol, C. (1992). Extensions of Inversion of Resolution Applied to Theory Completion. In Muggleton, S. H. (ed.) Inductive Logic Programming, 63-92. Academic Press Ltd.: London.Google Scholar
  109. Rouveirol, C. (1994). Flattering and Saturation: Two Representation Changes for Generalization. Machine Learning 14: 219-232. Special Issue on Evaluating and Changing Representation.Google Scholar
  110. Saitta, L. & Bergadano, F. (1993). Pattern Recognition and Valiant's Learning Framework. IEEE Transactions on Pattern Analysis and Machine Intelligence 15(2): 145-154.Google Scholar
  111. Schaffer, C. (1993). Overfitting Avoidance as Bias. Machine Learning 10: 153-178.Google Scholar
  112. Segal, R. & Etzioni, O. (1994). Learning Decision Lists Using Homogeneous Rules. In Proceedings of the 12th National Conference on Artificial Intelligence (AAAI-94), 619-625. AAAI Press: Cambridge, MA.Google Scholar
  113. Shapiro, E. Y. (1981). An Algorithm that Infers Theories from Facts. In Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI-81), 446-451.Google Scholar
  114. Silverstein, G. & Pazzani, M. J. (1991). Relational Clichès: Constraining Constructive Induction During Relational Learning. In Proceedings of the 8th International Workshop on Machine Learning (ML-91), 203-207. Evanston, Illinois.Google Scholar
  115. Silverstein, G. & Pazzani, M. J. (1993). Learning Relational Clichés. In Proceedings of the IJCAI-93 Workshop on Inductive Logic Programming, 71-82.Google Scholar
  116. Srinivasan, A., Muggleton, S. H. & Bain, M. E. (1992). Distinguishing Noise from Exceptions in Non-Monotonic Learning. In Proceedings of the International Workshop on Inductive Logic Programming (ILP-92), 97-107. Tokyo, Japan.Google Scholar
  117. Theron, H. & Cloete, I. (1996). BEXA: A Covering Algorithm for Learning Propositional Concept Descriptions. Machine Learning 24: 5-40.Google Scholar
  118. Utgoff, P. E. (1986). Shift of Bias for Inductive Concept Learning. In Michalski R., Carbonell, J. & Mitchell T. (eds.) Machine Learning: An Artificial Intelligence Approach, Vol. II, 107-148. Morgan Kaufmann: Los Altos, CA.Google Scholar
  119. Van Horn, K. S. & Martinez, T. R. (1993). The BBG Rule Induction Algorithm. In Proceedings of the 6th Australian Joint Conference on Artificial Intelligence, 348-355. Melbourne, Australia.Google Scholar
  120. Vapnik, V. N. & Chervonenkis, Y. A. (1971). On the Uniform Convergence of Relative Frequencies to Their Probabilities. Theory of Probability and Its Applications 16: 264-280.Google Scholar
  121. Vapnik, V. N. & Chervonenkis, Y. A. (1981). Necessary and Sufficient Conditions for the Uniform Convergence of Means to Their Expectations. Theory of Probability and Its Applications 26: 532-553.Google Scholar
  122. Venturini, G. (1993). SIA: A Supervised Inductive Algorithm with Genetic Search for Learning Attributes Based Concepts. In Brazdil P (ed.), Proceedings of the 6th European Conference on Machine Learning (ECML-93), Vol. 667 of Lecture Notes in Artificial Intelligence, 280-296. Springer-Verlag: Vienna, Austria.Google Scholar
  123. Wallace, C. S. & Boulton, D. M. (1968). An Information Measure for Classification. Computer Journal 11: 185-194.Google Scholar
  124. Watanabe, L. & Rendell, L. (1991). Learning Structural Decision Trees from Examples. In Proceedings of the 12th International Joint Conference on Artificial Intelligence (IJCAI-91), 770-776.Google Scholar
  125. Webb, G. I. (1992). Learning Disjunctive Class Descriptions by Least Generalisation. Tech. rep. TR C92/9, Deakin University, School of Computing & Mathematics, Geelong, Australia.Google Scholar
  126. Webb, G. I. (1993). Systematic Search for Categorical Attribute-Value Data-Driven Machine Learning. In Rowles, C., Liu, H. & Foo, N. (eds.) Proceedings of the 6th Australian Joint Conference on Artificial Intelligence (AI '93), 342-347. World Scientific: Melbourne.Google Scholar
  127. Webb, G. I. (1994). Recent Progress in Learning Decision Lists by Prepending Inferred Rules. In Proceedings of the 2nd Singapore International Conference on Intelligent Systems, B280-B285.Google Scholar
  128. Webb, G. I. (1995). OPUS: An Efficient Admissible Algorithm for Unordered Search. Journal of Artificial Intelligence Research 5: 431-465.Google Scholar
  129. Webb, G. I. & Agar, J. W. M. (1992). Inducing Diagnostic Rules for Glomerular Disease with the DLG Machine Learning Algorithm. Artificial Intelligence in Medicine 4: 419-430.Google Scholar
  130. Webb, G. I. & Brkič, N. (1993). Learning Decision Lists by Prepending Inferred Rules. In Proceedings of the AI '93 Workshop on Machine Learning and Hybrid Systems. Melbourne, Australia.Google Scholar
  131. Weiss, S. M. & Indurkhya, N. (1991). Reduced Complexity Rule Induction. In Proceedings of the 12th International Joint Conference on Artificial Intelligence (IJCAI-91), 678-684.Google Scholar
  132. Weiss, S. M. & Indurkhya, N. (1993a). Optimized Rule Induction. IEEE Expert 8(6): 61-69.Google Scholar
  133. Weiss, S. M. & Indurkhya, N. (1993b). Rule-Based Regression. In Bajcsy, R. (ed.) Proceedings of the 13th International Joint Conference on Artificial Intelligence (IJCAI-93), 1072-1078.Google Scholar
  134. Weiss, S. M. & Indurkhya, N. (1995). Rule-Based Machine Learning Methods for Functional Prediction. Journal of Artificial Intelligence Research 3: 383-403.Google Scholar
  135. Widmer, G. (1993). Combining Knowledge-Based and Instance-Based Learning to Exploit Quantitative Knowledge. Informatica 17: 371-385. Special Issue on Multistrategy Learning.Google Scholar
  136. Wiese, M. (1996). A Bidirectional ILP Algorithm. In Proceedings of the MLnet Familiarization Workshop on Data Mining with Inductive Logic Programming (ILP for KDD), 61-72.Google Scholar
  137. Wnek, J. & Michalski, R. S. (1994). Hypothesis-Driven Constructive Induction in AQ17-HCI: A Method and Experiments. Machine Learning 14(2): 139-168. Special Issue on Evaluating and Changing Representation.Google Scholar
  138. Wolpert, D. H. (1993). On Overfitting Avoidance as Bias. Tech. rep. SFI TR 92-03-5001. The Santa Fe Institute, Santa Fe, NM.Google Scholar
  139. Zelle, J. M., Mooney, R. J. & Konvisser, J. B. (1994). Combining Top-Down and Bottom-Up Techniques in Inductive Logic Programming. In Cohen, W. & Hirsh, H. (eds.) Proceedings of the 11th International Conference on Machine Learning (ML-94), 343-351. Morgan Kaufmann: New Brunswick, NJ.Google Scholar

Copyright information

© Kluwer Academic Publishers 1999

Authors and Affiliations

  • Johannes Fürnkranz
    • 1
  1. 1.Austrian Research Institute for Artificial IntelligenceWienAustria

Personalised recommendations