Predicate invention and learning from positive examples only

  • Henrik Boström
Inductive Logic Programming
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1398)


Previous bias shift approaches to predicate invention are not applicable to learning from positive examples only, if a complete hypothesis can be found in the given language, as negative examples are required to determine whether new predicates should be invented or not. One approach to this problem is presented, MERLIN 2.0, which is a successor of a system in which predicate invention is guided by sequences of input clauses in SLD-refutations of positive and negative examples w.r.t. an overly general theory. In contrast to its predecessor which searches for the minimal finite-state automaton that can generate all positive and no negative sequences, MERLIN 2.0 uses a technique for inducing Hidden Markov Models from positive sequences only. This enables the system to invent new predicates without being triggered by negative examples. Another advantage of using this induction technique is that it allows for incremental learning. Experimental results are presented comparing MERLIN 2.0 with the positive only learning framework of Progol 4.2 and comparing the original induction technique with a new version that produces deterministic Hidden Markov Models. The results show that predicate invention may indeed be both necessary and possible when learning from positive examples only as well as it can be beneficial to keep the induced model deterministic.


Hide Markov Model Logic Program Positive Sequence Inductive Logic Programming Negative Sequence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bain M. and Muggleton S., “Non-Monotonic Learning”, in Muggleton S. (ed.), Inductive Logic Programming, Academic Press (1992) 145–161Google Scholar
  2. 2.
    Bar-Hillel Y., Perles M. and Shamir E., “On formal properties of simple phrase structure grammars”, Zeitschrift für Phonetik, Sprachwissenschaft und Kommunikationsforschung, 14, 1, Akademie Verlag, Berlin (1961) 143–172Google Scholar
  3. 3.
    Baum L., Petrie T, Soules G. and Weiss N., “A maximization technique occurring in the statistical analysis of probabilistic functions in Markov chains”, The Annals of Mathematical Statistics 41 (1970) 164–171Google Scholar
  4. 4.
    Biermann A. W. and Feldman J. A., “On the Synthesis of Finite-State Machines from Samples of Their Behavior”, IEEE Transactions on Computers 21 (1972) 592–597Google Scholar
  5. 5.
    Boström H., “Theory-Guided Induction of Logic Programs by Inference of Regular Languages”, Proc. of the 13th International Conference on Machine Learning, Morgan Kaufmann (1996) 46–53Google Scholar
  6. 6.
    Kijsirikul B., Numao M. and Shimura M., “Discrimination-based constructive induction of logic programs”, Proceedings of the 10th National Conference on Artificial Intelligence, Morgan Kaufmann (1992) 44–49Google Scholar
  7. 7.
    Lewis H. R. and Papadimitriou C. H., Elements of the Theory of Computation, Prentice-Hall (1981)Google Scholar
  8. 8.
    Lapointe S., Ling, C. and Matwin S., “Constructive Inductive Logic Programming”, Proceedings of the 13th International Joint Conference on Artificial Intelligence, Morgan Kaufmann (1993) 1030–1036Google Scholar
  9. 9.
    Lloyd J. W., Foundations of Logic Programming, (2nd edition), Springer-Verlag (1987)Google Scholar
  10. 10.
    Muggleton S., “Inverse entailment and Progol”, New Generation Computing 13 (1995) 245–286Google Scholar
  11. 11.
    Muggleton S., “Learning from positive data”, Proc. of the Sixth International Workshop on Inductive Logic Programming (1996)Google Scholar
  12. 12.
    Muggleton S., “Stochastic Logic Programs”, Advances in Inductive Logic Programming (Ed. L. De Raedt), IOS Press (1996) 254–264Google Scholar
  13. 13.
    Stahl I., “Predicate Invention in Inductive Logic Programming”, Advances in Inductive Logic Programming (Ed. L. De Raedt), IOS Press (1996) 34–47Google Scholar
  14. 14.
    Stolcke A. and Omohundro S., “Best-first Model Merging for Hidden Markov Model Induction”, TR-94-003, International Computer Science Institute, Berkeley, CA (1994)Google Scholar
  15. 15.
    Wirth R. and O'Rorke P., “Constraints on Predicate Invention”, Proceedings of the 8th International Workshop on Machine Learning, Morgan Kaufmann (1991) 457–461Google Scholar
  16. 16.
    Wrobel S., “Concept Formation During Interactive Theory Revision”, Machine Learning Journal 14 (1994) 169–192CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Henrik Boström
    • 1
  1. 1.Dept. of Computer and Systems SciencesStockholm UniversityKistaSweden

Personalised recommendations