Advertisement

A Latent Variable Model for Generative Dependency Parsing

  • Ivan Titov
  • James Henderson
Chapter
Part of the Text, Speech and Language Technology book series (TLTB, volume 43)

Abstract

Dependency parsing has been a topic of active research in natural language processing during the last several years. The CoNLL-2006 shared task (Buchholz and Marsi, 2006) made a wide selection of standardized treebanks for different languages available for the research community and allowed for easy comparison between various statistical methods on a standardized benchmark. One of the surprising things discovered by this evaluation is that the best results are achieved by methods which are quite different from state-of-the-art models for constituent parsing, e.g. the deterministic parsing method of Nivre et al. (2006) and the minimum spanning tree parser of McDonald et al.

Keywords

Latent Variable Model Shared Task Current Decision Tune Feature Feature Induction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

This work was funded by Swiss NSF grant 200020-109685, Swiss NSF Fellowship PBGE22-119276, UK EPSRC grant EP/E019501/1, EU FP6 grant 507802 (TALK project), and EU FP7 grant 216594 (CLASSiC project).

References

  1. Abeillé, A. (Ed.) (2003). Treebanks: Building and Using Parsed Corpora. Dordrecht: Kluwer.Google Scholar
  2. Aduriz, I., M. J. Aranzabe, J. M. Arriola, A. Atutxa, A. D. de Ilarraza, A. Garmendia, and M. Oronoz (2003). Construction of a Basque dependency treebank. In Proceedings of the 2nd Workshop on Treebanks and Linguistic Theories (TLT), Växjö, pp. 201–204.Google Scholar
  3. Aho, A. V., R. Sethi, and J. D. Ullman (1986). Compilers: Principles, Techniques and Tools. Reading, MA: Addison Wesley.Google Scholar
  4. Böhmová, A., J. Hajič, E. Hajičová, and B. Hladká (2003). The PDT: a 3-level annotation scenario. See Abeillé (2003),  Chapter 7, pp. 103–127.
  5. Bottou, L. (1991). Une approche théoretique de l’apprentissage connexionniste: Applications à la reconnaissance de la parole. Ph. D. thesis, Université de Paris XI, Paris.Google Scholar
  6. Buchholz, S. and E. Marsi (2006). CoNLL-X shared task on multilingual dependency parsing. In Proceedings of the 10th Conference on Computational Natural Language Learning, New York, NY, pp. 149–164.Google Scholar
  7. Charniak, E. (2000). A maximum-entropy-inspired parser. In Proceedings of the 1st Meeting of North American Chapter of Association for Computational Linguistics, Seattle, WA, pp. 132–139.Google Scholar
  8. Charniak, E. and M. Johnson (2005). Coarse-to-fine n-best parsing and MaxEnt discriminative reranking. In Proceedings of the 43rd Meeting of Association for Computational Linguistics, Ann Arbor, MI, pp. 173–180.Google Scholar
  9. Chen, K., C. Luo, M. Chang, F. Chen, C. Chen, C. Huang, and Z. Gao (2003). Sinica treebank: design criteria, representational issues and implementation. See Abeillé (2003),  Chapter 13, pp. 231–248.
  10. Collins, M. (1999). Head-Driven Statistical Models for Natural Language Parsing. Ph. D. thesis, University of Pennsylvania, Philadelphia, PA.Google Scholar
  11. Collins, M. (2000). Discriminative reranking for natural language parsing. In Proceedings of the 17th International Conference on Machine Learning, Stanford, CA, pp. 175–182.Google Scholar
  12. Csendes, D., J. Csirik, T. Gyimóthy, and A. Kocsor (2005). The Szeged Treebank. Springer, Berlin/ Heidelberg.Google Scholar
  13. Dzeroski, S., T. Erjavec, N. Ledinek, P. Pajas, Z. Zabokrtsky, and A. Zele (2006). Towards a Slovene dependency treebank. In Proceedings of the International Conference on Language Resources and Evaluation (LREC), Genoa, pp. 1388–1391.Google Scholar
  14. Hajič, J., O. Smrž, P. Zemánek, J. Šnaidauf, and E. Beška (2004). Prague Arabic dependency treebank: development in data and tools. In Proceedings of the NEMLAR International Conference on Arabic Language Resources and Tools, Caira, pp. 110–117.Google Scholar
  15. Hall, J., J. Nilsson, J. Nivre, G. Eryigit, B. Megyesi, M. Nilsson, and M. Saers (2007). Single malt or blended? a study in multilingual parser optimization. In Proceedings of the CoNLL Shared Task Session of EMNLP-CoNLL 2007, Prague, pp. 933–939.Google Scholar
  16. Henderson, J. (2003). Inducing history representations for broad coverage statistical parsing. In Proceedings of the Joint Meeting of North American Chapter of the Association for Computational Linguistics and the Human Language Technology Conference, Edmonton, AB, pp. 103–110.Google Scholar
  17. Henderson, J. (2004). Discriminative training of a neural network statistical parser. In Proceedings of the 42nd Meeting of Association for Computational Linguistics, Barcelona, pp. 95–102.Google Scholar
  18. Henderson, J., P. Merlo, G. Musillo, and I. Titov (2008). A latent variable model of synchronous parsing for syntactic and semantic dependencies. In Proceedings of the CoNLL-2008 Shared Task, Manchester, pp. 178–182.Google Scholar
  19. Henderson, J. and I. Titov (2005). Data-defined kernels for parse reranking derived from probabilistic models. In Proceedings of the 43rd Meeting of Association for Computational Linguistics, Ann Arbor, MI, pp. 181–188.Google Scholar
  20. Johansson, R. and P. Nugues (2007). Extended constituent-to-dependency conversion for English. In Proceedings of the 16th Nordic Conference on Computational Linguistics (NODALIDA), Tartu, pp. 105–112.Google Scholar
  21. Jordan, M. I., Z. Ghahramani, T. S. Jaakkola, and L. K. Saul. (1999). An introduction to variational methods for graphical models. In M. I. Jordan (Ed.), Learning in Graphical Models, pp. 183–233. Cambridge, MA: MIT Press.Google Scholar
  22. Koo, T. and M. Collins (2005). Hidden-variable models for discriminative reranking. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Vancouver, BC, pp. 507–514.Google Scholar
  23. Kromann, M. T. (2003). The Danish dependency treebank and the underlying linguistic theory. In Proceedings of the 2nd Workshop on Treebanks and Linguistic Theories (TLT), Vaxjo.Google Scholar
  24. Liang, P., S. Petrov, M. Jordan, and D. Klein (2007). The infinite PCFG using hierarchical dirichlet processes. In Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), Prague, pp. 688–697.Google Scholar
  25. Marcus, M., B. Santorini, and M. Marcinkiewicz (1993). Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics 19(2), 313–330.Google Scholar
  26. Martí, M. A., M. Taulé, L. Màrquez, and M. Bertran (2007). CESS-ECE: a multilingual and multilevel annotated corpus. Available for download from: http://www.lsi.upc.edu/ mbertran/cess-ece/
  27. Matsuzaki, T., Y. Miyao, and J. Tsujii (2005). Probabilistic CFG with latent annotations. In Proceedings of the 43rd Annual Meeting of the ACL, Ann Arbor, MI, pp. 75–82.Google Scholar
  28. McDonald, R., K. Lerman, and F. Pereira (2006). Multilingual dependency analysis with a two-stage discriminative parser. In Proceeding of the 10th Conference on Computational Natural Language Learning, New York, NY, pp. 216–220.Google Scholar
  29. Montemagni, S., F. Barsotti, M. Battista, N. Calzolari, O. Corazzari, A. Lenci, A. Zampolli, F. Fanciulli, M. Massetani, R. Raffaelli, R. Basili, M. T. Pazienza, D. Saracino, F. Zanzotto, N. Nana, F. Pianesi, and R. Delmonte (2003). Building the Italian Syntactic-Semantic Treebank. See Abeillé (2003),  Chapter 11, pp. 189–210.
  30. Murphy, K. P. (2002). Dynamic Belief Networks: Representation, Inference and Learning. Ph. D. thesis, University of California, Berkeley, CA.Google Scholar
  31. Musillo, G. and P. Merlo (2008). Unlexicalised hidden variable models of split dependency grammars. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Columbus, OH, pp. 213–216.Google Scholar
  32. Nakagawa, T. (2007). Multilingual dependency parsing using global features. In Proceedings of the CoNLL Shared Task Session of EMNLP-CoNLL 2007, pp. 952–956.Google Scholar
  33. Neal, R. (1992). Connectionist learning of belief networks. Artificial Intelligence 56, 71–113.CrossRefGoogle Scholar
  34. Nivre, J., J. Hall, and J. Nilsson (2004). Memory-based dependency parsing. In Proceedings of the 8th Conference on Computational Natural Language Learning, Boston, MA, pp. 49–56.Google Scholar
  35. Nivre, J., J. Hall, J. Nilsson, G. Eryigit, and S. Marinov (2006). Pseudo-projective dependency parsing with support vector machines. In Proceedings of the 10th Conference on Computational Natural Language Learning, New York, NY, pp. 221–225.Google Scholar
  36. Oflazer, K., B. Say, D. Z. Hakkani-Tür, and G. Tür (2003). Building a Turkish treebank. See Abeillé (2003),  Chapter 15, pp. 261–277.
  37. Peshkin, L. and V. Savova (2005). Dependency parsing with dynamic Bayesian network. In AAAI, 20th National Conference on Artificial Intelligence, Pittsburgh, PA, pp. 1112–1117.Google Scholar
  38. Petrov, S., L. Barrett, R. Thibaux, and D. Klein (2006). Learning accurate, compact, and interpretable tree annotation. In Proceedings of the Annual Meeting of the ACL and the International Conference on Computational Linguistics, Sydney, pp. 433–44.Google Scholar
  39. Petrov, S. and D. Klein (2007). Improved inference for unlexicalized parsing. In Proceedings of the Conference on Human Language Technology and North American chapter of the Association for Computational Linguistics (HLT-NAACL 2007), Rochester, NY, pp. 404–411.Google Scholar
  40. Prescher, D. (2005). Head-driven PCFGs with latent-head statistics. In Proceedings of the 9th International Workshop on Parsing Technologies, Vancouver, BC, pp. 115–124.Google Scholar
  41. Prokopidis, P., E. Desypri, M. Koutsombogera, H. Papageorgiou, and S. Piperidis (2005). Theoretical and practical issues in the construction of a Greek dependency treebank. In Proceedings of the 4th Workshop on Treebanks and Linguistic Theories (TLT), Barcelona, pp. 149–160.Google Scholar
  42. Riezler, S., T. H. King, R. M. Kaplan, R. Crouch, J. T. Maxwell, and M. Johnson (2002). Parsing the Wall Street Journal using a Lexical-Functional Grammar and discriminative estimation techniques. In Proceedings of the 40th Meeting of Association for Computational Linguistics, Philadelphia, PA, pp. 271–278.Google Scholar
  43. Sallans, B. (2002). Reinforcement Learning for Factored Markov Decision Processes. Ph. D. thesis, University of Toronto, Toronto, ON.Google Scholar
  44. Sha, F. and F. Pereira (2003). Shallow parsing with conditional random fields. In Proceedings of the Joint Meeting of North American Chapter of the Association for Computational Linguistics and the Human Language Technology Conference, Edmonton, AB, pp. 213–220.Google Scholar
  45. Titov, I. and J. Henderson (2007). Constituent parsing with Incremental Sigmoid Belief Networks. In Proc. 45th Meeting of Association for Computational Linguistics (ACL), Prague, pp. 632–639.Google Scholar
  46. van der Beek, L., G. Bouma, J. Daciuk, T. Gaustad, R. Malouf, G. van Noord, R. Prins, and B. Villada (2002). The Alpino dependency treebank. In Computational Linguistic in the Netherlands (CLIN), Enschede, pp. 8–22.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2010

Authors and Affiliations

  1. 1.Cluster of Excellence, MMC, Saarland UniversitySaarbrückenGermany
  2. 2.Department of Computer ScienceUniversity of GenevaGenevaSwitzerland

Personalised recommendations