Machine Learning

, Volume 26, Issue 2–3, pp 147–176 | Cite as

First Order Regression

  • Aram Karalič
  • Ivan Bratko
Article

Abstract

We present a new approach, called First Order Regression (FOR), to handling numerical information in Inductive Logic Programming (ILP). FOR is a combination of ILP and numerical regression. First-order logic descriptions are induced to carve out those subspaces that are amenable to numerical regression among real-valued variables. The program FORS is an implementation of this idea, where numerical regression is focused on a distinguished continuous argument of the target predicate. We show that this can be viewed as a generalisation of the usual ILP problem. Applications of FORS on several real-world data sets are described: the prediction of mutagenicity of chemicals, the modelling of liquid dynamics in a surge tank, predicting the roughness in steel grinding, finite element mesh design, and operator's skill reconstruction in electric discharge machining. A comparison of FORS' performance with previous results in these domains indicates that FORS is an effective tool for ILP applications that involve numerical data.

machine learning inductive logic programming regression real-valued variables first-order logic applications of machine learning 

References

  1. Bratko, I., & Džeroski, S. (1995). Engineering applications of ILP. New Generation Computing, 13, 313-333.Google Scholar
  2. Camacho, R. (1994). Learning stage transition rules with Indlog. Proceedings of the Fourth International Workshop on Inductive Logic Programming (ILP-94) (pp. 273-290), Bad Honnef/Bonn, Germany: Gesellschaft für Mathematik und Datenverarbeitung Sankt Augustin.Google Scholar
  3. Dolšak, B., Bratko, I., & Jezernik, A. (1994). Finite element mesh design: An engineering domain for ILP application. Proceedings of the Fourth International Workshop on Inductive Logic Programming (ILP-94) (pp. 305-320), Bad Honnef/Bonn, Germany. Gesellschaft für Mathematik und Datenverarbeitung Sankt Augustin.Google Scholar
  4. Dolšak, B. (1996). A Contribution to Mesh Design for Finite Element Method. PhD thesis, University of Maribor, Faculty of Mechanical Engineering, Maribor, Slovenia.Google Scholar
  5. Dolšak, B., Jezernik, A., & Bratko, I. (1994). A knowledge base for finite element mesh design, Artifical Intelligence in Engineering, 9, 19-27.Google Scholar
  6. Dolšak, B., & Muggleton, S. (1992). The application of inductive logic programming to finite-element mesh design. Stephen Muggleton, editor, Inductive Logic Programming: Academic Press.Google Scholar
  7. Džeroski, S. (1991). Handling noise in inductive logic programming. Master's thesis, University of Ljubljana, Faculty for Electrical Engineering and Computer Science, Ljubljana, Slovenia.Google Scholar
  8. Džeroski, S. (1995). Numerical Constraints and Learnability in Inductive Logic Programming, PhD thesis, University of Ljubljana, Faculty for Electrical Engineering and Computer Science, Ljubljana, Slovenia.Google Scholar
  9. Džeroski, S., & Todorovski, L. (1995). Discovering dynamics: from inductive logic programming to machine discovery. Journal of Intelligent Information Systems, 4, 89-108.Google Scholar
  10. Filipič, B., Junkar, M., Bratko, I., & Karalič, A. (1991). An application of machine learning to a metal-working process. Proceedings of ITI-91, 167-172, Cavtat, Croatia.Google Scholar
  11. Hamming, R.W. (1989). Digital Filters: Prentice-Hall.Google Scholar
  12. Junkar, M., Filipič, B., & Bratko, I. (1991). Identifying the Grinding Process by Means of Inductive Machine Learning. Computers in Industry, 17(2-3), 147-153.Google Scholar
  13. Junkar, M., Filipič, B., & Žnidaršič, M. (1993). An AI approach to the selection of dielectricum in electrical discharge machining. Proceedings of Third International Conference on Advanced Manufacturing Systems and Technology AMST'93, Udine, Italy.Google Scholar
  14. Junkar, M., & Komel, I. (1996). Knowledge Acquisition for Adaptive Control of the EDM Process. Proceedings of 15th IAESTED International Conference on Modelling, Identification and Control (pp. 295-297), Innsbruck, Austria.Google Scholar
  15. Karalič, A. (1992). Employing linear regression in regression tree leaves. Proceedings of ECAI'92 (European Conference on Artificial Intelligence)(pp. 440-441), Vienna, Austria.Google Scholar
  16. Karalič, A. (1995a). First Order Regression. PhD thesis, University of Ljubljana, Faculty for Electrical Engineering and Computer Science.Google Scholar
  17. Karalič, A. (1995b). First order regression: Application in real-world domains. Proceedings of Artificial Intelligence Techniques - AIT'95, Brno, Czech Republic.Google Scholar
  18. Karalič, A. (1996). Producing More Comprehensible Models While Retaining Their Performance. Proceedings of Information, Statistics and Induction in Science ISIS '96 (pp. 54-65), Melbourne, Australia.Google Scholar
  19. Komel, I. (1996). Surface Modelling and Expert Knowledge Acquisition for the Control of electrical Discharge Machining Process. Master's Thesis, University of Ljubljana, Faculty of Mechanical Engineering.Google Scholar
  20. Kompare, B. (1995). Faculty of Civil Engineering and Geodesy, Department for Hydroengineering, Institute for Sanitary Engineering. Ljubljana, Slovenia. Personal communication.Google Scholar
  21. Korenjak, B. (1994). Experimental Environment for Modelling of Dynamic Systems Using Artificial Intelligence Methods. Master's thesis, University of Ljubljana, Faculty for Electrical Engineering and Computer Science, Ljubljana, Slovenia. In Slovene.Google Scholar
  22. Kovačič, M. (1995). Stochastic Inductive Logic Programming. PhD thesis, Faculty of electrical Engineering and Computer Science, Ljubljana, Slovenia.Google Scholar
  23. Križman, V. (1993). Handling Noisy Data in Automatic Modelling of Dynamical Systems. Master's thesis, University of Ljubljana, Faculty for Electrical Engineering and Computer Science, Ljubljana, Slovenia. In Slovene.Google Scholar
  24. Križman, V., Džeroski, S., & Kompare, B. (1995). Discovering dynamics from measured data. Working Notes of the MLNet Workshop on Statistics, Machine Learning, and Knowledge Discovery in Databases. Institute of Computer Science, Heraklion, Greece.Google Scholar
  25. Lavrač, N., & Džeroski, S. (1994). Inductive Logic Programming: Techniques and Applications. Ellis Horwood, Chicester.Google Scholar
  26. Li, M., & Vitányi, P. (1993). An Introduction to Kolmogorov Complexity and its Applications. Texts and Monographs in Computer Science. Springer-Verlag.Google Scholar
  27. Mizoguchi, F., & Ohwada, H. (1995). An inductive logic programming approach to constraint acquisition for constraint-based problem solving. Proceedings of the 5th International Workshop on Inductive Logic Programming (ILP-95) (pp. 297-322): Katholieke Universiteit Leuven, Heverlee, Belgium.Google Scholar
  28. Muggleton, S., & Feng, C. (1990). Efficient induction of logic programs. Proceedings of the First Conference on Algorithmic Learning Theory, Tokyo, Japan.Google Scholar
  29. Muggleton, S., Srinivasan, A., Bain., M. (1992). Compression, significance and accuracy. Proceedings of Machine Learning Conference 1992 (pp. 338-347), Aberdeen.Google Scholar
  30. Posel, R. (1995). Expert System for Roughness Prediction During Steel Grinding. Proceedings of Management of Inovative Technologies-MIT'95 (pp. 296-303), Bled, Slovenia.Google Scholar
  31. Quinlan, R. (1990). Learning Logical Definitions From Relations. Machine Learning, 3(5).Google Scholar
  32. Quinlan, R. (1996), University of Sydney, Sydney, Australia, Personal communication.Google Scholar
  33. Quinlan, R., & Cameron-Jones, M. (1993). FOIL: A Midterm Report. Proceedings of Sixth European Conference on Machine Learning, (pp. 3-20), Vienna, Austria.Google Scholar
  34. Quinlan, R., & Rivest., R. L. (1989). Inferring decision trees using the minimum description length principle. Information and Computation, 80, 227-248.Google Scholar
  35. Rissanen, J. (1978). Modelling by shortest data description. Automatica, 14, 465-471.Google Scholar
  36. Rowe, W. B., Yan, L., Inasaki, I., & Malkin, S. (1994). Applications of Artificial Intelligence in Grinding (Keynote paper). Annals of the CIRP, 43(2), 521-531.Google Scholar
  37. Sebag, M., & Rouveirol, C. (1995). Constraint inductive logic programming. Proceedings of the 5th International Workshop on Inductive Logic Programming (ILP-95) (pp. 297-322): Katholieke Universiteit Leuven, Heverlee, Belgium.Google Scholar
  38. Srinivasan, A. (1996). Experiments in numerical reasoning with ILP. Michie, D., Muggleton, S., & Furukawa, K., editors, Machine Intelligence 15. Oxford University Press.Google Scholar
  39. Srinivasan, A., & Muggleton, S. H. (1995). Comparing the use of background knowledge by two inductive logic programming systems. Proceedings of the 5th International Workshop on Inductive Logic Programming (ILP-95) (pp. 199-230): Katholieke Universiteit Leuven, Heverlee, Belgium.Google Scholar
  40. Srinivasan, A., Muggleton, S. H., King, R. D., & Sternberg, M. J. E. (1994). Mutagenesis: ILP experiments in a non-determinate biological domain. Proceedings of the Fourth International Workshop on Inductive Logic Programming (ILP-94), (pp. 217-232): Bad Honnef/Bonn, Germany. Gesellschaft für Mathematik und Datenverarbeitung Sankt Augustin.Google Scholar
  41. Srinivasan, A., Stephen, S. H., Sternberg, M. J. E., & King, R. D. (1995). Theories for mutagenicity: a study in first-order and feature-based induction. Technical Report PRG-TR-8-95, Oxford University Computing Laboratory, Oxford.Google Scholar

Copyright information

© Kluwer Academic Publishers 1997

Authors and Affiliations

  • Aram Karalič
    • 1
  • Ivan Bratko
    • 2
  1. 1.Jožef Stefan InstituteLjubljanaSlovenia
  2. 2.Faculty of Electrical Engineering and Computer ScienceUniversity of Ljubljana, and Jozef Stefan InstituteLjubljanaSlovenia

Personalised recommendations