(Agnostic) PAC Learning Concepts in Higher-Order Logic

  • K. S. Ng
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4212)


This paper studies the PAC and agnostic PAC learnability of some standard function classes in the learning in higher-order logic setting introduced by Lloyd et al. In particular, it is shown that the similarity between learning in higher-order logic and traditional attribute-value learning allows many results from computational learning theory to be ‘ported’ to the logical setting with ease. As a direct consequence, a number of non-trivial results in the higher-order setting can be established with straightforward proofs. Our satisfyingly simple analysis provides another case for a more in-depth study and wider uptake of the proposed higher-order logic approach to symbolic machine learning.


Vertex Cover Inductive Logic Programming Logical Setting Computational Learning Theory Predicate Class 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Anthony, M.: Decision lists and threshold decision lists. Technical Report LSE-CDAM-2002-11, London School of Economics (2002)Google Scholar
  2. 2.
    Anthony, M., Bartlett, P.L.: Neural Network Learning: Theoretical Foundations. Cambridge University Press, Cambridge (1999)MATHCrossRefGoogle Scholar
  3. 3.
    Bowers, A.F., Giraud-Carrier, C., Lloyd, J.W.: A knowledge representation framework for inductive learning (2001),
  4. 4.
    Cohen, W.: Grammatically biased learning: Learning logic programs using an explicit antecedent description language. Artificial Intelligence 68(2), 303–366 (1994)MATHCrossRefGoogle Scholar
  5. 5.
    Cohen, W.W., Page, D.: Polynomial learnability and inductive logic programming: Methods and results. New Generation Computing 13, 369–409 (1995)CrossRefGoogle Scholar
  6. 6.
    De Raedt, L.: Logical settings for concept learning. Artificial Intelligence 95, 187–201 (1997)MATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    De Raedt, L., Džeroski, S.: First order jk-clausal theories are PAC-learnable. Artificial Intelligence 70, 375–392 (1994)MATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Kearns, M., Mansour, Y.: On the boosting ability of top-down decision tree learning algorithms. J. of Computer and System Sciences 58(1), 109–128 (1999)MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Kietz, J.-U., Džeroski, S.: Inductive logic programming and learnability. SIGART Bulletin 5(1), 22–32 (1994)CrossRefGoogle Scholar
  10. 10.
    Kramer, S., Lavrač, N., Flach, P.: Propositionalization approaches to relational data mining. In: Relational data mining, ch.11. Springer, Heidelberg (2001)Google Scholar
  11. 11.
    Lloyd, J.W.: Programming in an integrated functional and logic language. Journal of Functional and Logic Programming 3 (1999)Google Scholar
  12. 12.
    Lloyd, J.W.: Logic for Learning: Learning Comprehensible Theories from Structured Data. In: Cognitive Technologies. Springer, Heidelberg (2003)Google Scholar
  13. 13.
    Muggleton, S., De Raedt, L.: Inductive logic programming: Theory and methods. Journal of Logic Programming 19, 20, 629–679 (1994)CrossRefMathSciNetGoogle Scholar
  14. 14.
    Ng, K.S.: Generalization behaviour of Alkemic decision trees. In: Proc. of the 15th International Conference on Inductive Logic Programming, pp. 246–263 (2005)Google Scholar
  15. 15.
    Ng, K.S.: Learning Comprehensible Theories from Structured Data. PhD thesis, Computer Sciences Laboratory, The Australian National University (2005)Google Scholar
  16. 16.
    Rivest, R.L.: Learning decision lists. Machine Learning 2(3), 229–246 (1987)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • K. S. Ng
    • 1
  1. 1.Symbolic Machine Learning and Knowledge Acquisition, National ICT Australia Limited 

Personalised recommendations