Applied Intelligence

, Volume 11, Issue 1, pp 59–77 | Cite as

The Connectionist Inductive Learning and Logic Programming System

  • Artur S. Avila Garcez
  • Gerson Zaverucha
Article

Abstract

This paper presents the Connectionist Inductive Learning and Logic Programming System (C-IL2P). C-IL2P is a new massively parallel computational model based on a feedforward Artificial Neural Network that integrates inductive learning from examples and background knowledge, with deductive learning from Logic Programming. Starting with the background knowledge represented by a propositional logic program, a translation algorithm is applied generating a neural network that can be trained with examples. The results obtained with this refined network can be explained by extracting a revised logic program from it. Moreover, the neural network computes the stable model of the logic program inserted in it as background knowledge, or learned with the examples, thus functioning as a parallel system for Logic Programming. We have successfully applied C-IL2P to two real-world problems of computational biology, specifically DNA sequence analyses. Comparisons with the results obtained by some of the main neural, symbolic, and hybrid inductive learning systems, using the same domain knowledge, show the effectiveness of C-IL2P.

theory refinement machine learning artificial neural networks logic programming computational biology 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    N. Lavrac and S. Dzeroski, “Inductive logic programming: Techniques and applications,” Ellis Horwood Series in Artificial Intelligence, 1994.Google Scholar
  2. 2.
    T.M. Mitchell, Machine Learning, McGraw-Hill, 1997.Google Scholar
  3. 3.
    S.B. Thrun et al., “The MONK's problems: A performance comparison of different learning algorithms,” Technical Report, Carnegie Mellon University, 1991.Google Scholar
  4. 4.
    R.S. Michalski, “Learning strategies and automated knowledge acquisition,” Computational Models of Learning, Symbolic Computation, Springer-Verlag, 1987.Google Scholar
  5. 5.
    N.K. Bose and P. Liang, Neural Networks Fundamentals with Graphs, Algorithms, and Applications, McGraw-Hill, 1996.Google Scholar
  6. 6.
    F.J. Kurfess, “Neural networks and structured knowledge,” in Knowledge Representation in Neural Networks, edited by Ch. Herrmann, F. Reine, and A. Strohmaier, Logos-Verlag: Berlin, pp. 5-22, 1997.Google Scholar
  7. 7.
    G. Pinkas, “Energy minimization and the satisfiability of propositional calculus,” Neural Computation, vol. 3,no. 2, 1991.Google Scholar
  8. 8.
    G. Pinkas, “Reasoning, nonmonotonicity and learning in connectionist networks that capture propositional knowledge,” Artificial Intelligence, vol. 77, pp. 203-247, 1995.Google Scholar
  9. 9.
    S. Holldobler, “Automated inferencing and connectionist models,” Post Ph. D. Thesis, Intellektik, Informatik, TH Darmstadt, 1993.Google Scholar
  10. 10.
    H.B. Enderton, A Mathematical Introduction to Logic, Academic Press, 1972.Google Scholar
  11. 11.
    S. Holldobler and Y. Kalinke, “Toward a new massively parallel computational model for logic programming,” in Proc. Workshop on Combining Symbolic and Connectionist Processing, ECAI 94, 1994.Google Scholar
  12. 12.
    J.W. Lloyd, Foundations of Logic Programming, Springer-Verlag, 1987.Google Scholar
  13. 13.
    K.R. Apt and D. Pedreschi, “Reasoning about termination of pure prolog programs,” Information and Computation, vol. 106, pp. 109-157, 1993.Google Scholar
  14. 14.
    M. Fitting, “Metric methods—three examples and a theorem,” Journal of Logic Programming, vol. 21, pp. 113-127, 1994.Google Scholar
  15. 15.
    M. Gelfond and V. Lifschitz, “The stable model semantics for logic programming,” in Proc. Fifth International Symposium on Logic Programming, MIT Press: Cambridge, pp. 1070-1080, 1988.Google Scholar
  16. 16.
    G.G. Towell and J.W. Shavlik, “Knowledge-based artificial neural networks,” Artificial Intelligence, vol. 70,no. 1, pp. 119-165, 1994.Google Scholar
  17. 17.
    D.E. Rumelhart, G.E. Hinton, and R.J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, edited by D.E. Rumelhart and J.L. McClelland, MIT Press, vol. 1, pp. 318-363,1986.Google Scholar
  18. 18.
    S. Muggleton and L. Raedt, “Inductive logic programming: Theory and methods,” Journal of Logic Programming, vol. 19, pp. 629-679, 1994.Google Scholar
  19. 19.
    A.S. d'Avila Garcez, K. Broda, and D. Gabbay, “Symbolic knowledge extraction from trained neural networks: A new approach,” Technical Report TR-98-014, Department of Computing, Imperial College, London, 1998.Google Scholar
  20. 20.
    L.M. Fu, Neural Networks in Computer Intelligence, McGraw Hill, 1994.Google Scholar
  21. 21.
    G.G. Towell, “Symbolic knowledge and neural networks: Insertion, refinement and extraction,” Ph.D. Thesis, Computer Sciences Department, University of Wisconsin, Madison, 1991.Google Scholar
  22. 22.
    J. Hertz, A. Krogh, and R.G. Palmer, “Introduction to the theory of neural computation,” Studies in the Science of Complexity, Santa Fe Institute, Addison-Wesley Publishing Company, 1991.Google Scholar
  23. 23.
    K.R. Apt and N. Bol, “Logic programming and negation: A survey,” Journal of Logic Programming, vol. 19, pp. 9-71, 1994.Google Scholar
  24. 24.
    R.M. Karp and V. Ramachandran, “Parallel algorithms for shared-memory machines,” in Handbook of Theoretical Computer Science, edited by J. van Leeuwen, Elsevier Science, vol. 17, pp. 869-941, 1990.Google Scholar
  25. 25.
    K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural Networks, vol. 2, pp. 359-366, 1989.Google Scholar
  26. 26.
    B. DasGupta and G. Schinitger, “Analog versus discrete neural networks,” Neural Computation, vol. 8, pp. 805-818, 1996.Google Scholar
  27. 27.
    M.I. Jordan, “Attractor dynamics and parallelisms in a connectionist sequential machine,” in Proc. Eighth Annual Conference of the Cognitive Science Society, pp. 531-546, 1986.Google Scholar
  28. 28.
    R. Andrews and S. Geva, “Inserting and extracting knowledge from constrained error backpropagation networks,” in Proc. Sixth Australian Conference on Neural Networks, Sydney, 1995.Google Scholar
  29. 29.
    E. Pop, R. Hayward, and J. Diederich, RULENEG: Extracting Rules from a Trained ANN by Stepwise Negation, QUT NRC, 1994.Google Scholar
  30. 30.
    S.B. Thrun, “Extracting provably correct rules from artificial neural networks,” Technical Report, Institut fur Informatik, Universitat Bonn, 1994.Google Scholar
  31. 31.
    M.W. Craven and J.W. Shavlik, “Using sampling and queries to extract rules from trained neural networks,” in Proc. Eleventh International Conference on Machine Learning, pp. 37-45, 1994.Google Scholar
  32. 32.
    G.G. Towell and J.W. Shavlik, “The extraction of refined rules from knowledge based neural networks,” Machine Learning, vol. 13,no. 1, pp. 71-101, 1993.Google Scholar
  33. 33.
    R. Setiono, “Extracting rules from neural networks by pruning and hidden-unit splitting,” Neural Computation, vol. 9, pp. 205-225, 1997.Google Scholar
  34. 34.
    R. Andrews, J. Diederich, and A.B. Tickle, “A survey and critique of techniques for extracting rules from trained artificial neural networks,” Knowledge-based Systems, vol. 8,no. 6, pp. 1-37, 1995.Google Scholar
  35. 35.
    W. Marek and M. Truszczynski, Nonmonotonic Logic: Context Dependent Reasoning, Springer-Verlag, 1993.Google Scholar
  36. 36.
    J.D. Watson, N.H. Hopkins, J.W. Roberts, J.A. Steitz, and A.M. Weiner, Molecular Biology of the Gene, Benjamin Cummings: Menlo Park, vol. 1, 1987.Google Scholar
  37. 37.
    F. Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms, Spartan Books: New York, 1962.Google Scholar
  38. 38.
    J.R. Quinlan, “Induction of decision trees,” Machine Learning, vol. 1, pp. 81-106, 1986.Google Scholar
  39. 39.
    D.H. Fisher, “Knowledge acquisition via incremental conceptual clustering,” Machine Learning, vol. 2, pp. 139-172, 1987.Google Scholar
  40. 40.
    G.D. Stormo, “Consensus patterns in DNA,” Methods in Enzymology, Academic Press: Orlando, vol. 183, pp. 211-221, 1990.Google Scholar
  41. 41.
    D. Ourston and R.J. Mooney, “Theory refinement combining analytical and empirical methods,” Artificial Intelligence, vol. 66, pp. 273-310, 1994.Google Scholar
  42. 42.
    K. Thompson, P. Langley, and W. Iba, “Using background knowledge in concept formation,” in Proc. Eighth International Machine Learning Workshop, Evanston, pp. 554-558, 1991.Google Scholar
  43. 43.
    M. Pazzani and D. Kibler, “The utility of knowledge in inductive learning,” Machine Learning, vol. 9, pp. 57-94, 1992.Google Scholar
  44. 44.
    G.G. Towell and J.W. Shavlik, “Using symbolic learning to improve knowledge-based neural networks,” in Proc. AAAI'94, 1994.Google Scholar
  45. 45.
    M. Gelfond and V. Lifschitz, “Classical negation in logic programs and disjunctive databases,” New Generation Computing, Springer-Verlag, vol. 9, pp. 365-385, 1991.Google Scholar
  46. 46.
    R. Reiter, “A logic for default reasoning,” Artificial Intelligence, vol. 13, pp. 81-132, 1980.Google Scholar
  47. 47.
    A.S. d'Avila Garcez, G. Zaverucha, and V. da Silva, “Applying the connectionist inductive learning and logic programming system to power system diagnosis,” in Proc. IEEE International Joint Conference on Neural Networks IJCNN'97, vol. 1, Houston, USA, pp. 121-126, 1997.Google Scholar
  48. 48.
    G. Zaverucha, “A prioritized contextual default logic: Curing anomalous extensions with a simple abnormality default theory,” in Proc. KI'94, Springer-Verlag: Saarbrucken, Germany, LNAI 861, pp. 260-271, 1994.Google Scholar
  49. 49.
    D.M. Gabbay, LDS—Labelled Deductive Systems—Volume 1 Foundations, Oxford University Press, 1996.Google Scholar
  50. 50.
    N. Hallack, G. Zaverucha, and V. Barbosa, “Towards a hybrid model of first-order theory refinement,” in Neural Information Processing Systems, Workshop on Hybrid Neural Symbolic Integration, Breckenridge, Colorado, USA, 1998.Google Scholar
  51. 51.
    P. Gardenfors (Ed.), “Belief revision,” Cambridge Tracts in Theoretical Computer Science, Cambridge University Press, 1992.Google Scholar
  52. 52.
    P. Gardenfors and H. Rott, “Belief revision,” in Handbook of Logic in Artificial Intelligence and Logic Programming, edited by D. Gabbay, C. Hogger, and J. Robinson, Oxford University Press, vol. 4, pp. 35-132, 1994.Google Scholar
  53. 53.
    M. Hilario, “An overview of strategies for neurosymbolic integration,” in Proc. Workshop on Connectionist-Symbolic Integration: From Unified to Hybrid Approaches, IJCAI 95, 1995.Google Scholar
  54. 54.
    M.C. O'Neill, “Escherichia coli promoters: Consensus as it relates to spacing class, specificity, repeat substructure, and three dimensional organization,” Journal of Biological Chemistry, vol. 264, pp. 5522-5530, 1989.Google Scholar
  55. 55.
    G.G. Towell, J.W. Shavlik, and M.O. Noordewier, “Refinement of approximately correct domain theories by knowledge-based neural networks,” in Proc. AAAI'90, Boston, pp. 861-866, 1990.Google Scholar
  56. 56.
    M.O. Noordewier, G.G. Towell, and J.W. Shavlik, “Training knowledge-based neural networks to recognize genes in DNA sequences,” Advances in Neural Information Processing Systems, Denver, vol. 3, pp. 530-536, 1991.Google Scholar
  57. 57.
    L.M. Fu, “Integration of neural heuristics into knowledge-based inference,” Connection Science, vol. 1, pp. 325-340, 1989.Google Scholar
  58. 58.
    B.F. Katz, “EBL and SBL: A neural network synthesis,” in Proc. Eleventh Annual Conference of the Cognitive Science Society, Ann Arbor, pp. 683-689, 1989.Google Scholar
  59. 59.
    M. Minsky, “Logical versus analogical, symbolic versus connectionist, neat versus scruffy,” AI Magazine, vol. 12,no. 2, 1991.Google Scholar
  60. 60.
    R. Basilio, G. Zaverucha, and A.S. J'Avila Garcez, “Inducing Relational Concepts with Neural Networks via the Linus System,” in Proc. International Conference on Neural Information Processing, vol. 3, pp. 1507-1510, Japan, 1998.Google Scholar

Copyright information

© Kluwer Academic Publishers 1999

Authors and Affiliations

  • Artur S. Avila Garcez
  • Gerson Zaverucha

There are no affiliations available

Personalised recommendations