Abstract
We present a new approach, called First Order Regression (FOR), to handling numerical information in Inductive Logic Programming (ILP). FOR is a combination of ILP and numerical regression. First-order logic descriptions are induced to carve out those subspaces that are amenable to numerical regression among real-valued variables. The program FORS is an implementation of this idea, where numerical regression is focused on a distinguished continuous argument of the target predicate. We show that this can be viewed as a generalisation of the usual ILP problem. Applications of FORS on several real-world data sets are described: the prediction of mutagenicity of chemicals, the modelling of liquid dynamics in a surge tank, predicting the roughness in steel grinding, finite element mesh design, and operator's skill reconstruction in electric discharge machining. A comparison of FORS' performance with previous results in these domains indicates that FORS is an effective tool for ILP applications that involve numerical data.
Article PDF
Similar content being viewed by others
References
Bratko, I., & Džeroski, S. (1995). Engineering applications of ILP. New Generation Computing, 13, 313-333.
Camacho, R. (1994). Learning stage transition rules with Indlog. Proceedings of the Fourth International Workshop on Inductive Logic Programming (ILP-94) (pp. 273-290), Bad Honnef/Bonn, Germany: Gesellschaft für Mathematik und Datenverarbeitung Sankt Augustin.
Dolšak, B., Bratko, I., & Jezernik, A. (1994). Finite element mesh design: An engineering domain for ILP application. Proceedings of the Fourth International Workshop on Inductive Logic Programming (ILP-94) (pp. 305-320), Bad Honnef/Bonn, Germany. Gesellschaft für Mathematik und Datenverarbeitung Sankt Augustin.
Dolšak, B. (1996). A Contribution to Mesh Design for Finite Element Method. PhD thesis, University of Maribor, Faculty of Mechanical Engineering, Maribor, Slovenia.
Dolšak, B., Jezernik, A., & Bratko, I. (1994). A knowledge base for finite element mesh design, Artifical Intelligence in Engineering, 9, 19-27.
Dolšak, B., & Muggleton, S. (1992). The application of inductive logic programming to finite-element mesh design. Stephen Muggleton, editor, Inductive Logic Programming: Academic Press.
Džeroski, S. (1991). Handling noise in inductive logic programming. Master's thesis, University of Ljubljana, Faculty for Electrical Engineering and Computer Science, Ljubljana, Slovenia.
Džeroski, S. (1995). Numerical Constraints and Learnability in Inductive Logic Programming, PhD thesis, University of Ljubljana, Faculty for Electrical Engineering and Computer Science, Ljubljana, Slovenia.
Džeroski, S., & Todorovski, L. (1995). Discovering dynamics: from inductive logic programming to machine discovery. Journal of Intelligent Information Systems, 4, 89-108.
Filipič, B., Junkar, M., Bratko, I., & Karalič, A. (1991). An application of machine learning to a metal-working process. Proceedings of ITI-91, 167-172, Cavtat, Croatia.
Hamming, R.W. (1989). Digital Filters: Prentice-Hall.
Junkar, M., Filipič, B., & Bratko, I. (1991). Identifying the Grinding Process by Means of Inductive Machine Learning. Computers in Industry, 17(2-3), 147-153.
Junkar, M., Filipič, B., & Žnidaršič, M. (1993). An AI approach to the selection of dielectricum in electrical discharge machining. Proceedings of Third International Conference on Advanced Manufacturing Systems and Technology AMST'93, Udine, Italy.
Junkar, M., & Komel, I. (1996). Knowledge Acquisition for Adaptive Control of the EDM Process. Proceedings of 15th IAESTED International Conference on Modelling, Identification and Control (pp. 295-297), Innsbruck, Austria.
Karalič, A. (1992). Employing linear regression in regression tree leaves. Proceedings of ECAI'92 (European Conference on Artificial Intelligence)(pp. 440-441), Vienna, Austria.
Karalič, A. (1995a). First Order Regression. PhD thesis, University of Ljubljana, Faculty for Electrical Engineering and Computer Science.
Karalič, A. (1995b). First order regression: Application in real-world domains. Proceedings of Artificial Intelligence Techniques - AIT'95, Brno, Czech Republic.
Karalič, A. (1996). Producing More Comprehensible Models While Retaining Their Performance. Proceedings of Information, Statistics and Induction in Science ISIS '96 (pp. 54-65), Melbourne, Australia.
Komel, I. (1996). Surface Modelling and Expert Knowledge Acquisition for the Control of electrical Discharge Machining Process. Master's Thesis, University of Ljubljana, Faculty of Mechanical Engineering.
Kompare, B. (1995). Faculty of Civil Engineering and Geodesy, Department for Hydroengineering, Institute for Sanitary Engineering. Ljubljana, Slovenia. Personal communication.
Korenjak, B. (1994). Experimental Environment for Modelling of Dynamic Systems Using Artificial Intelligence Methods. Master's thesis, University of Ljubljana, Faculty for Electrical Engineering and Computer Science, Ljubljana, Slovenia. In Slovene.
Kovačič, M. (1995). Stochastic Inductive Logic Programming. PhD thesis, Faculty of electrical Engineering and Computer Science, Ljubljana, Slovenia.
Križman, V. (1993). Handling Noisy Data in Automatic Modelling of Dynamical Systems. Master's thesis, University of Ljubljana, Faculty for Electrical Engineering and Computer Science, Ljubljana, Slovenia. In Slovene.
Križman, V., Džeroski, S., & Kompare, B. (1995). Discovering dynamics from measured data. Working Notes of the MLNet Workshop on Statistics, Machine Learning, and Knowledge Discovery in Databases. Institute of Computer Science, Heraklion, Greece.
Lavrač, N., & Džeroski, S. (1994). Inductive Logic Programming: Techniques and Applications. Ellis Horwood, Chicester.
Li, M., & Vitányi, P. (1993). An Introduction to Kolmogorov Complexity and its Applications. Texts and Monographs in Computer Science. Springer-Verlag.
Mizoguchi, F., & Ohwada, H. (1995). An inductive logic programming approach to constraint acquisition for constraint-based problem solving. Proceedings of the 5th International Workshop on Inductive Logic Programming (ILP-95) (pp. 297-322): Katholieke Universiteit Leuven, Heverlee, Belgium.
Muggleton, S., & Feng, C. (1990). Efficient induction of logic programs. Proceedings of the First Conference on Algorithmic Learning Theory, Tokyo, Japan.
Muggleton, S., Srinivasan, A., Bain., M. (1992). Compression, significance and accuracy. Proceedings of Machine Learning Conference 1992 (pp. 338-347), Aberdeen.
Posel, R. (1995). Expert System for Roughness Prediction During Steel Grinding. Proceedings of Management of Inovative Technologies-MIT'95 (pp. 296-303), Bled, Slovenia.
Quinlan, R. (1990). Learning Logical Definitions From Relations. Machine Learning, 3(5).
Quinlan, R. (1996), University of Sydney, Sydney, Australia, Personal communication.
Quinlan, R., & Cameron-Jones, M. (1993). FOIL: A Midterm Report. Proceedings of Sixth European Conference on Machine Learning, (pp. 3-20), Vienna, Austria.
Quinlan, R., & Rivest., R. L. (1989). Inferring decision trees using the minimum description length principle. Information and Computation, 80, 227-248.
Rissanen, J. (1978). Modelling by shortest data description. Automatica, 14, 465-471.
Rowe, W. B., Yan, L., Inasaki, I., & Malkin, S. (1994). Applications of Artificial Intelligence in Grinding (Keynote paper). Annals of the CIRP, 43(2), 521-531.
Sebag, M., & Rouveirol, C. (1995). Constraint inductive logic programming. Proceedings of the 5th International Workshop on Inductive Logic Programming (ILP-95) (pp. 297-322): Katholieke Universiteit Leuven, Heverlee, Belgium.
Srinivasan, A. (1996). Experiments in numerical reasoning with ILP. Michie, D., Muggleton, S., & Furukawa, K., editors, Machine Intelligence 15. Oxford University Press.
Srinivasan, A., & Muggleton, S. H. (1995). Comparing the use of background knowledge by two inductive logic programming systems. Proceedings of the 5th International Workshop on Inductive Logic Programming (ILP-95) (pp. 199-230): Katholieke Universiteit Leuven, Heverlee, Belgium.
Srinivasan, A., Muggleton, S. H., King, R. D., & Sternberg, M. J. E. (1994). Mutagenesis: ILP experiments in a non-determinate biological domain. Proceedings of the Fourth International Workshop on Inductive Logic Programming (ILP-94), (pp. 217-232): Bad Honnef/Bonn, Germany. Gesellschaft für Mathematik und Datenverarbeitung Sankt Augustin.
Srinivasan, A., Stephen, S. H., Sternberg, M. J. E., & King, R. D. (1995). Theories for mutagenicity: a study in first-order and feature-based induction. Technical Report PRG-TR-8-95, Oxford University Computing Laboratory, Oxford.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Karalič, A., Bratko, I. First Order Regression. Machine Learning 26, 147–176 (1997). https://doi.org/10.1023/A:1007365207130
Issue Date:
DOI: https://doi.org/10.1023/A:1007365207130