Advertisement

Improved Character-Based Chinese Dependency Parsing by Using Stack-Tree LSTM

  • Hang Liu
  • Mingtong Liu
  • Yujie ZhangEmail author
  • Jinan Xu
  • Yufeng Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11109)

Abstract

Almost all the state-of-the-art methods for Character-based Chinese dependency parsing ignore the complete dependency subtree information built during the parsing process, which is crucial for parsing the rest part of the sentence. In this paper, we introduce a novel neural network architecture to capture dependency subtree feature. We extend and improve recent works in neural joint model for Chinese word segmentation, POS tagging and dependency parsing, and adopt bidirectional LSTM to learn n-gram feature representation and context information. The neural network and bidirectional LSTMs are trained jointly with the parser objective, resulting in very effective feature extractors for parsing. Finally, we conduct experiments on Penn Chinese Treebank 5, and demonstrate the effectiveness of the approach by applying it to a greedy transition-based parser. The results show that our model outperforms the state-of-the-art neural joint models in Chinese word segmentation, POS tagging and dependency parsing.

Keywords

Chinese word segmentation POS tagging and dependency parsing Dependency subtree Neural network architecture 

Notes

Acknowledgments

The authors are supported by the National Nature Science Foundation of China (Contract 61370130 and 61473294), and the Beijing Municipal Natural Science Foundation (4172047).

References

  1. 1.
    Yamada, H., Matsumoto, Y.: Statistical dependency analysis with support vector machines. In: International Workshop on Parsing Technologies 2003, Nancy, France, pp. 195—206 (2003)Google Scholar
  2. 2.
    Nivre, J.: Incrementality in deterministic dependency parsing. In: Proceedings of the Workshop on Incremental Parsing: Bringing Engineering and Cognition Together, pp. 50–57. Association for Computational Linguistics (2004)Google Scholar
  3. 3.
    Zhang, Y., Clark, S.: A tale of two parsers: investigating and combining graph-based and transition-based dependency parsing using beam search. In: Proceedings of EMNLP, Hawaii, USA (2008)Google Scholar
  4. 4.
    Huang, L., Sagae, K.: Dynamic programming for linear-time incremental parsing. In: Proceedings of ACL, Uppsala, Sweden, pp. 1077–1086, July 2010Google Scholar
  5. 5.
    Hatori, J., Matsuzaki, T., Miyao, Y., Tsujii, J.I.: Incremental joint approach to word segmentation, pos tagging, and dependency parsing in Chinese. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics. Long Papers, vol. 1, pp. 1045–1053. Association for Computational Linguistics (2012)Google Scholar
  6. 6.
    Zhang, M., Zhang, Y., Che, W., Liu, T.: Character-level Chinese dependency parsing. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Long Papers, vol. 1, pp. 1326–1336. Association for Computational Linguistics (2014)Google Scholar
  7. 7.
    Guo, Z., Zhang, Y., Su, C., Xu, J.: Character-level dependency model for joint word segmentation, POS tagging, and dependency parsing in Chinese. J. Chin. Inf. Process. E99.D(1), 257–264 (2014)Google Scholar
  8. 8.
    Kurita, S., Kawahara, D., Kurohashi, S.: Neural joint model for transition-based chinese syntactic analysis. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Long Papers, vol. 1, pp. 1204–1214. Association for Computational Linguistics (2017)Google Scholar
  9. 9.
    Dyer, C., Ballesteros, M., Ling, W., Matthews, A., Smith, N.A.: Transition-based dependency parsing with stack long short-term memory. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Long Papers, vol. 1, pp. 334–343. Association for Computational Linguistics (2015)Google Scholar
  10. 10.
    Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Long Papers, vol. 1, pp. 1556–1566, Beijing, China. Association for Computational Linguistics (2015)Google Scholar
  11. 11.
    Zhu, C., Qiu, X., Chen, X., Huang, X.: A re-ranking model for dependency parser with recursive convolutional neural network. Comput. Sci. (2015)Google Scholar
  12. 12.
    Kingma, D.P., Adam, J.B.: A method for stochastic optimization. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Long Papers, vol. 1 (2015)Google Scholar
  13. 13.
    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. Volume abs/1301.3781 (2013)Google Scholar
  15. 15.
    Jiang, W., Huang, L., Liu, Q., Lu, Y.: A cascaded linear model for joint Chinese word segmentation and part-of-speech tagging. In: Proceedings of ACL-2008: HLT, pp. 897–904. Association for Computational Linguistics (2008)Google Scholar
  16. 16.
    Tseng, H., Chang, P., Andrew, G., Jurafsky, D., Manning, C.: A conditional random field word segmenter for SIGHAN bakeoff 2005. In: Proceedings of the Fourth SIGHAN Workshop on Chinese Language Processing (2005)Google Scholar
  17. 17.
    Dyer, C., Kuncoro, A., Ballesteros, M., Smith, N.A.: Recurrent Neural Network Grammars, pp. 199–209. The North American Chapter of the Association for Computational Linguistics (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Computer and Information TechnologyBeijing Jiaotong UniversityBeijingChina

Personalised recommendations