Languages as Hyperplanes: Grammatical Inference with String Kernels

  • Alexander Clark
  • Christophe Costa Florêncio
  • Chris Watkins
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4212)


Using string kernels, languages can be represented as hyperplanes in a high dimensional feature space. We present a new family of grammatical inference algorithms based on this idea. We demonstrate that some mildly context sensitive languages can be represented in this way and it is possible to efficiently learn these using kernel PCA. We present some experiments demonstrating the effectiveness of this approach on some standard examples of context sensitive languages using small synthetic data sets.


Feature Space Regular Language Positive Data Negative Data Context Sensitive 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Bach, E.: Discontinuous constituents in generalized categorial grammars. North East Linguistics Society (NELS 11), 1–12 (1981)Google Scholar
  2. Becker, T., Rambow, O., Niv, M.: The Derivational Generative Power of Formal Systems or Scrambling is Beyond LCFRS (Technical Report 92–38). Institute For Research in Cognitive Science, University of Pennsylvania (1992)Google Scholar
  3. Chalup, S., Blair, A.D.: Hill climbing in recurrent neural networks for learning the a n b n c n language. In: Proceedings of the Sixth International Conference on Neural Information Processing, pp. 508–513 (1999)Google Scholar
  4. Clark, A., Costa Florêncio, C., Watkins, C., Serayet, M.: Planar languages and learnability. In: International Colloquium on Grammatical Inference (ICGI), Tokyo (to appear, 2006)Google Scholar
  5. Gentner, T.Q., Fenn, K.M., Margoliash, D., Nusbaum, H.C.: Recursive syntactic pattern learning by songbirds. Nature 440, 1204–1207 (2006)CrossRefGoogle Scholar
  6. Kearns, M., Valiant, G.: Cryptographic limitations on learning boolean formulae and finite automata. In: 21st annual ACM symposium on Theory of computation, pp. 433–444. ACM Press, New York (1989)Google Scholar
  7. Lodhi, H., Saunders, C., Shawe-Taylor, J., Cristianini, N., Watkins, C.: Text classification using string kernels. JMLR 2, 419–444 (2002)zbMATHCrossRefGoogle Scholar
  8. Parikh, R.J.: On context-free languages. Journal of the ACM 13, 570–581 (1966)zbMATHCrossRefMathSciNetGoogle Scholar
  9. Radzinski, D.: Chinese number-names, tree adjoining languages, and mild context-sensitivity. Comput. Linguist. 17, 277–299 (1991)Google Scholar
  10. Salomaa, A.: On languages defined by numerical parameters (Technical Report 663). Turku Centre for Computer Science (2005)Google Scholar
  11. Schölkopf, B., Smola, A.K. M.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10 (1998)Google Scholar
  12. Shawe-Taylor, J., Christianini, N.: Kernel methods for pattern analysis. Cambridge University Press, Cambridge (2004)Google Scholar
  13. Shieber, S.M.: Evidence against the context-freeness of natural language. Linguistics and Philosophy 8, 333–343 (1985)CrossRefGoogle Scholar
  14. Starkie, B., Coste, F., van Zaanen, M.: The Omphalos context-free grammar learning competition. In: International Colloquium on Grammatical Inference, Athens, Greece, pp. 16–27 (2004)Google Scholar
  15. Vijay-Shanker, K., Weir, D.J., Joshi, A.K.: Characterizing structural descriptions produced by various grammatical formalisms. In: Proceedings of the 25th annual meeting on Association for Computational Linguistics, pp. 104–111. Association for Computational Linguistics, Morristown (1987)CrossRefGoogle Scholar
  16. Watkins, C.: Dynamic alignment kernels. In: Smola, A.J., Bartlette, P.L., Schölkopf, B., Schuurmans, D. (eds.) Advances in large margin classifiers, pp. 39–50. MIT Press, Cambridge (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Alexander Clark
    • 1
  • Christophe Costa Florêncio
    • 1
  • Chris Watkins
    • 1
  1. 1.Department of Computer ScienceUniversity of LondonEgham

Personalised recommendations