Statistical inductive learning of regular formal languages

  • Juan Andrés Sánchez
  • José Miguel Benedí
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 862)


The estimation problem of probabilistic grammar through the forward-backward algorithm does not guarantee that a global maximum is achieved [12]. In this process, which is based on a gradient descent technique, the initialization is a crucial aspect. In this paper, we show experimentally how the results obtained by this method can be improved when structural information about the task is inductively incorporated in the initial models to be learnt.


Hide Markov Model Relative Entropy Training Corpus Random Initialization Test Corpus 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Trainable Grammars for Speech Recognition” J.K. Baker. 50th Anniversary Celebration of the Acoustical Society of America, pp. 31–35. June 1979.Google Scholar
  2. [2]
    An Inequality and Associated Maximization Technique in Statistical Estimation for Probabilistic Function of a Markov Process” L.E. Baum. Inequalities, Vol. III, 1972.Google Scholar
  3. [3]
    Applying Probability Measures to Abstract Languages” T.L. Booth, R.A. Thompson. IEEE Transactions on Computers, Vol. C-22, No. 5, pp. 442–450. May 1973.Google Scholar
  4. [4]
    Growth Transformations for Probabilistic Functions of Stochastic Grammars” F. Casacuberta. To be published. 1993.Google Scholar
  5. [5]
    Information Theory” T.M. Cover, J.A. Thomas. John Wiley & Sons. 1991.Google Scholar
  6. [6]
    Syntactic Pattern Recognition and Applications” K.S.Fu. Prentice Hall, 1982.Google Scholar
  7. [7]
    Syntactic Pattern Recognition. An Introduction” R.Gonzalez and M.Thomason. Addison-Wesley, 1978.Google Scholar
  8. [8]
    An Inequality for Rational Functions with Applications to some Statistical Estimation Problems” P.S. Gopalakrishnan, D. Kanevsky, A. Nadas, D. Nahamoo. IEEE Transactions on Information Theory, Vol. 37(1), pp. 107–113. 1991.CrossRefGoogle Scholar
  9. [9]
    Introduction to Automata Theory, Languages and Computation” J.E. Hopcroft, J.D. Ullman. Addison-Wesly. 1979.Google Scholar
  10. [10]
    Basic Methods of Probabilistic Context Free Grammars” F. Jelinek, J.D. Lafferty, R.L. Mercer. Speech Recognition and Understanding. Ed. by P. Laface and R. De Mori, pp. 347–360. 1992.Google Scholar
  11. [11]
    The Estimation of Stochastic Context-Free Grammars using the Inside-Outside Algorithm” K. Lari, S. J. Young. Computer Speech and Language Vol. 4, pp. 35–56. 1990.Google Scholar
  12. [12]
    On the Locality of the Forward-Backward Algorithm” B. Merialdo. IEEE Trans. on Speech and Audio Processing, Vol. 1(2), pp. 255–257, 1993.CrossRefGoogle Scholar
  13. [13]
    Hidden Markov chains, the forward-backward algorithm and initial statics” A. Nadas. IEEE Trans. Acoust., Speech and Signal Processing, vol. ASSP-31, pp. 504–506, 1983.Google Scholar
  14. [14]
    Learning Language Models Though the ECGI Method” N. Prieto and E. Vidal. Speech Communications, Vol.11, pp.299–309, 1992.Google Scholar
  15. [15]
    Mathematical Foundations of Hidden Markov Models” L.R.Rabiner. In Recent Advances in Speech Understanding and Dialog Systems. H.Niemann et al (eds). NATO ASI Series, Vo.F46, pp.183–205, 1988.Google Scholar
  16. [16]
    Modeling (Sub)string-length-based Constraints through a Grammatical Inference Method” H. Rulot, E. Vidal. Pattern Recogniton Theory and Applications, Devijver & Kittler Eds. Springer Verlag, pp.451–459. 1987.Google Scholar
  17. [17]
    An Efficient Algorithm for the Inference of Circuit-Free Automata” H. Rulot, E. Vidal. Syntactic and Structural Pattern Recognition. G. Ferrat Ed. Springer Verlag, pp.173–184. 1988.Google Scholar
  18. [18]
    Dynamic Construction of Finite-State Automata from Examples Using Hill-Climbing” M. Tomita. Proc. Fourth Annu. Cogn. Sci. Conf, pp. 105–108. 1982.Google Scholar
  19. [19]
    Application of the Error-Correcting Grammatical Inference Algorithm (ECGI) to Planar Shape Recognition” E. Vidal, H. Rulot, J.M. Valiente, G. Andreu. Grammatical Inference: Theory, Applications and Alternatives. S.24 (1–10), 1993.Google Scholar
  20. [20]
    Probabilistic Languages: A Review and some Open Questions” C.S. Wetherell. Computing Surveys, Vol.12, No.4, pp.361–379, 1980.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1994

Authors and Affiliations

  • Juan Andrés Sánchez
    • 1
  • José Miguel Benedí
    • 1
  1. 1.Departamento de Sistemas Informáticos y ComputaciónUniversidad Politécnica de ValenciaValenciaSpain

Personalised recommendations