MDL learning of unions of simple pattern languages from positive examples

  • Pekka Kilpeläinen
  • Heikki Mannila
  • Esko Ukkonen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 904)


The following learning task is considered: Given a set S of strings consisting of basic symbols and a set C of patterns consisting of basic symbols and variables, compute a concise set \(C \subseteq C\)such that each string in S is obtained from some pattern in C by substituting basic symbols for the variables. We apply Rissanen's MDL principle to the selection of patterns to the result. This leads to a length-minimization problem that we approximately solve in polynomial time (in the length of S and C) with a logarithmic performance guarantee. Our algorithm is based on a greedy solution of a variant of the set covering problem.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. D. Angluin: Finding patterns common to a set of strings. Journal of Computer and System Sciences 21 (1980) 46–62.Google Scholar
  2. H. Arimura, T. Shinohara & S. Otsuki: Finding minimal generalizations for unions of pattern languages and its application to inductive inference from positive data. In: P. Enjalbert, E. W. Mayr & K. W. Wagner (eds.), STAGS 94, Lecture Notes in Computer Science 775, pp. 649–660, Springer-Verlag 1994.Google Scholar
  3. V. Chvátal: A greedy heuristic for the set-covering problem. Mathematics of Operations Research 4 (1979) 233–235.Google Scholar
  4. R. M. Karp: Reducibility among combinatorial problems. In: R. E. Miller & J. W. Thatcher (eds.), Complexity of Computer Computations, pp. 85–103, Plenum Press, New York, 1972.Google Scholar
  5. M. Kearns & L. Pitt: A polynomial-time algorithm for learning k-variable pattern languages from examples. In: R. Rivest, D. Haussler & M. K. Warmuth (eds.): Proc. Second Ann. Workshop on Computational Learning Theory, pp. 57–70. Morgan Kaufmann, San Mateo 1989.Google Scholar
  6. J. Kivinen, H. Mannila & E. Ukkonen: Learning hierarchical rule sets. Proc. Fifth Annual ACM Workshop on Computational Learning Theory, pp. 37–44, ACM Press, New York, 1992.Google Scholar
  7. J. Kivinen, H. Mannila & E. Ukkonen: Learning rules with local exceptions. In: J. Shawe-Taylor & M. Anthony (eds.), Computational Learning Theory: Euro-COLT'93, pp. 35–46, Clarendon Press, Oxford, 1994.Google Scholar
  8. S. Lange & R. Wiehagen: Polynomial-time inference of arbitrary pattern languages. New Generation Computing 8 (1991) 361–370.Google Scholar
  9. M. Li & P.M.B. Vitanyi: An Introduction to Kolmogorov Complexity and Its Applications. Springer-Verlag, New York, 1993.Google Scholar
  10. J. Rissanen: Modeling by shortest data description. Automatica 14 (1978) 465–471.Google Scholar
  11. J. Rissanen: Stochastic Complexity in Statistical Inquiry. World Scientific, Singapore, 1989.Google Scholar
  12. J. Rissanen, T. P. Speed & B. Yu: Density estimation by stochastic conplexity. IEEE Trans. on Information Theory 38 (1992) 315–323.Google Scholar
  13. J. R. Quinlan & R. L. Rivest: Inferring decision trees using the minimum description length principle. Information and Computation 80 (1989) 227–248.Google Scholar
  14. C.S. Wallace & P.R. Freeman: Estimation and inference by compact coding. Journal of the Royal Statistical Society (B) 49 (1987) 240–265.Google Scholar
  15. C.S. Wallace & J.D. Patrick: Coding decision trees. Machine Learning 11 (1993) 7–22.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • Pekka Kilpeläinen
    • 1
  • Heikki Mannila
    • 1
  • Esko Ukkonen
    • 1
  1. 1.Department of Computer ScienceUniversity of HelsinkiFinland

Personalised recommendations