Research on Language and Computation

, Volume 6, Issue 2, pp 139–162 | Cite as

Computing Partition Functions of PCFGs



We investigate the problem of computing the partition function of a probabilistic context-free grammar, and consider a number of applicable methods. Particular attention is devoted to PCFGs that result from the intersection of another PCFG and a finite automaton. We report experiments involving the Wall Street Journal corpus.


Equation solving Formal grammars Statistical NLP WSJ corpus 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Abney, S., McAllester, D., & Pereira, F. (1999). Relating probabilistic grammars and automata. In 37th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, Maryland, USA, pp. 542–549.Google Scholar
  2. Baker, J. (1979). Trainable grammars for speech recognition. In J. Wolf & D. Klatt (Eds.), Speech Communication Papers Presented at the 97th Meeting of the Acoustical Society of America, pp. 547–550.Google Scholar
  3. Bar-Hillel Y., Perles M., Shamir E. (1964) On formal properties of simple phrase structure grammars. In: Bar-Hillel Y.(eds) Language and information: Selected essays on their theory and application(Chap 9). Addison-Wesley, Reading Massachusetts, pp 116–150Google Scholar
  4. Chi Z. (1999) Statistical properties of probabilistic context-free grammars. Computational Linguistics, 25(1): 131–160Google Scholar
  5. Chi Z., Geman S. (1998) Estimation of probabilistic context-free grammars. Computational Linguistics, 24(2): 299–305Google Scholar
  6. Corazza A., De Mori R., Gretter R., Satta G. (1991) Computation of probabilities for an island-driven parser. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(9): 936–950CrossRefGoogle Scholar
  7. Cormen, T., Leiserson, C., & Rivest, R. (1990). Introduction to algorithms. The MIT Press.Google Scholar
  8. Etessami, K., & Yannakakis, M. (2005). Recursive Markov chains, stochastic grammars, and Monotone systems of nonlinear equations. In 22nd International Symposium on Theoretical Aspects of Computer Science, Vol. 3404 of Lecture Notes in Computer Science, Stuttgart, Germany (pp. 340–352). Springer-Verlag.Google Scholar
  9. Fred, A. (2000). Computation of substring probabilities in stochastic grammars. In A. Oliveira (Ed.), Grammatical inference: Algorithms and applications, Vol. 1891 of Lecture Notes in Artificial Intelligence (pp. 103–114). Springer-Verlag.Google Scholar
  10. Harrison, M. (1978). Introduction to formal language theory. Addison-Wesley.Google Scholar
  11. Hopcroft, J., & Ullman, J. (1979) Introduction to automata theory, languages, and computation. Addison-Wesley.Google Scholar
  12. Huang T., Fu K. (1971) On stochastic context-free languages. Information Sciences, 3: 201–224CrossRefGoogle Scholar
  13. Jelinek F., Lafferty J. (1991) Computation of the probability of initial substring generation by stochastic context-free grammars. Computational Linguistics, 17(3): 315–323Google Scholar
  14. Johnson M. (1998) PCFG models of linguistic tree representations. Computational Linguistics, 24(4): 613–632Google Scholar
  15. Kelley C. (1995) Iterative methods for linear and nonlinear equations. Society for Industrial and Applied Mathematics, Philadelphia PAGoogle Scholar
  16. Kiefer, S., Luttenberger, M., & Esparza, J. (2007). On the convergence of Newton’s method for Monotone systems of polynomial equations. In Proceedings of the 39th ACM Symposium on Theory of Computing, pp. 217–266.Google Scholar
  17. Luenberger, D. (1973). Introduction to linear and nonlinear programming. Addison-Wesley.Google Scholar
  18. Malouf, R. (2002). A comparison of algorithms for maximum entropy parameter estimation. In Proceedings of the Sixth Conference on Natural Language Learning, Taipei, Taiwan, pp. 49–55.Google Scholar
  19. Manning, C., & Schütze, H. (1999). Foundations of statistical natural language processing. MIT Press.Google Scholar
  20. Nederhof, M.-J., & Satta, G. (2003). Probabilistic parsing as intersection. In 8th International Workshop on Parsing Technologies, LORIA, Nancy, France, pp. 137–148.Google Scholar
  21. Nederhof, M.-J., & Satta, G. (2006). Estimation of consistent probabilistic context-free grammars. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, New York, USA, pp. 343–350.Google Scholar
  22. Persoon E., Fu K. (1975) Sequential classification of strings generated by SCFG’s. International Journal of Computer and Information Sciences, 4(3): 205–217CrossRefGoogle Scholar
  23. Stolcke A. (1995) An efficient probabilistic context-free parsing algorithm that computes prefix probabilities. Computational Linguistics, 21(2): 167–201Google Scholar
  24. Thompson R. (1974) Determination of probabilistic grammars for functionally specified probability-measure languages. IEEE Transactions on Computers, C-23: 603–614CrossRefGoogle Scholar
  25. Vidal E., Thollard F., de la Higuera C., Casacuberta F., Carrasco R. (2005) Probabilistic finite-state machines—part II. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(7): 1026–1039CrossRefGoogle Scholar
  26. Wojtczak, D., & Etessami, K. (2007). PReMo: An analyzer for probabilistic recursive models. In Tools and Algorithms for the Construction and Analysis of Systems, 13th International Conference, Vol. 4424 of Lecture Notes in Computer Science, Braga, Portugal (pp. 66–71). Springer-Verlag.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2008

Authors and Affiliations

  1. 1.School of Computer ScienceUniversity of St AndrewsNorth Haugh, St AndrewsScotland, UK
  2. 2.Department of Information EngineeringUniversity of PaduaPadovaItaly

Personalised recommendations