Probabilistic Generalization of Simple Grammars and Its Application to Reinforcement Learning

  • Takeshi Shibata
  • Ryo Yoshinaka
  • Takashi Chikayama
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4264)


Recently, some non-regular subclasses of context-free grammars have been found to be efficiently learnable from positive data. In order to use these efficient algorithms to infer probabilistic languages, one must take into account not only equivalences between languages but also probabilistic generalities of grammars. The probabilistic generality of a grammar G is the class of the probabilistic languages generated by probabilistic grammars constructed on G. We introduce a subclass of simple grammars (SGs), referred as to unifiable simple grammars (USGs), which is a superclass of an efficiently learnable class, right-unique simple grammars (RSGs). We show that the class of RSGs is unifiable within the class of USGs, whereas SGs and RSGs are not unifiable within the class of SGs and RSGs, respectively. We also introduce simple context-free decision processes, which are a natural extension of finite Markov decision processes and intuitively may be thought of a Markov decision process with stacks. We propose a reinforcement learning method on simple context-free decision processes, as an application of the learning and unification algorithm for RSGs from positive data.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Angluin, D.: Inductive inference of formal languages from positive data. Information and Control 45, 117–135 (1980)zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Angluin, D.: Inference of reversible languages. Journal of the Association for Computing Machinery 29, 741–765 (1982)zbMATHMathSciNetGoogle Scholar
  3. 3.
    Barto, A.G., Mahadevan, S.: Recent advances in hierarchical reinforcement learning. Discrete Event Dynamic Systems: Theory and Applications 13, 41–77 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Bertsekas, D.P., Tsitsiklis, J.N.: Neuro-dynamic Programming. Athena Scientific, Sec. 5.6 (1996)Google Scholar
  5. 5.
    Hirshfeld, Y., Jerrum, M., Moller, F.: A polynomial algorithm for deciding bisimilarity of normed context-free processes. Theoretical Computer Science 158, 143–159 (1996)zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Kobayashi, S.: Iterated transductions and efficient learning from positive data: A unifying view. In: Oliveira, A.L. (ed.) ICGI 2000. LNCS, vol. 1891, pp. 157–170. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  7. 7.
    Kaelbling, L.P., Littman, M.L., Cassandra, A.R.: Planning and acting in partially observable stochastic domains. Artificial Intelligence 101, 99–134 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Sakakibara, Y.: Recent advances of grammatical inference. Theoretical Computer Science 185, 15–45 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  10. 10.
    Wakatsuki, M., Teraguchi, K., Tomita, E.: Polynomial time identification of strict deterministic restricted one-counter automata in some class from positive data. In: Paliouras, G., Sakakibara, Y. (eds.) ICGI 2004. LNCS, vol. 3264, pp. 260–272. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  11. 11.
    Wetherell, C.S.: Probabilistic languages: A review and some open questions. Computing Surveys 12(4), 361–379 (1980)zbMATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Yokomori, T.: Polynomial-time identification of very simple grammars from positive data. Theoretical Computer Science 298, 179–206 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Yoshinaka, R.: Polynomial-Time Identification of an Extension of Very Simple Grammars from Positive Data. In: Sakakibara, Y., Kobayashi, S., Sato, K., Nishino, T., Tomita, E. (eds.) ICGI 2006. LNCS, vol. 4201, pp. 45–58. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Takeshi Shibata
    • 1
  • Ryo Yoshinaka
    • 2
  • Takashi Chikayama
    • 1
  1. 1.Department of Electronic Engineeringthe University of Tokyo 
  2. 2.Graduate School of Interdisciplinary Information Studiesthe University of Tokyo 

Personalised recommendations