Advertisement

TBCNN for Dependency Trees in Natural Language Processing

  • Lili MouEmail author
  • Zhi Jin
Chapter
Part of the SpringerBriefs in Computer Science book series (BRIEFSCOMPUTER)

Abstract

This chapter applies tree-based convolution to the dependency parse trees of natural language sentences, resulting in a new variant d-TBCNN. Since dependency trees are different from abstract syntax trees in Chap.  4 and constituency trees in Chap.  5, we need to design new model gadgets for d-TBCNN. The model is evaluated on two sentence classification tasks (sentiment analysis and question classification) and a sentence matching task. In the sentence classification tasks, d-TBCNN outperforms previous state-of-the-art results, whereas in the sentence matching task, d-TBCNN achieves comparable performance to the previous state-of-the-art model, which has a higher matching complexity.

Keywords

Tree-based convolution Dependency parsing Sentence modeling Sentence matching 

References

  1. 1.
    Bos, J., Markert, K.: Combining shallow and deep NLP methods for recognizing textual entailment. In: Proceedings of the First PASCAL Challenges Workshop on Recognising Textual Entailment, pp. 65–68 (2005)Google Scholar
  2. 2.
    Bowman, S.R., Angeli, G., Potts, C., Manning, C.D.: A large annotated corpus for learning natural language inference. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 632–642 (2015)Google Scholar
  3. 3.
    Braud, C., Denis, P.: Comparing word representations for implicit discourse relation classification. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 2201–2211 (2015)Google Scholar
  4. 4.
    Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., Shah, R.: Signature verification using a “ssiamese” time delay neural network. In: Advances in Neural Information Processing Systems, pp. 737–744 (1994)Google Scholar
  5. 5.
    Collobert, R., Weston, J.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, pp. 160–167 (2008)Google Scholar
  6. 6.
    Fox, C.: A stop list for general text. In: ACM SIGIR Forum, pp. 19–21 (1989)CrossRefGoogle Scholar
  7. 7.
    Harabagiu, S., Hickl, A.: Methods for using textual entailment in open-domain question answering. In: Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, pp. 905–912 (2006)Google Scholar
  8. 8.
    Harabagiu, S., Hickl, A., Lacatusu, F.: Negation, contrast and contradiction in text processing. In: Proceedings of AAAI Conference on Artificial Intelligence, pp. 755–762 (2006)Google Scholar
  9. 9.
    He, H., Gimpel, K., Lin, J.: Multi-perspective sentence similarity modeling with convolutional neural networks. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 17–21 (2015)Google Scholar
  10. 10.
    Hu, B., Lu, Z., Li, H., Chen, Q.: Convolutional neural network architectures for matching natural language sentences. In: Advances in Neural Information Processing Systems, pp. 2042–2050 (2014)Google Scholar
  11. 11.
    Irsoy, O., Cardie, C.: Deep recursive neural networks for compositionality in language. In: Advances in Neural Information Processing Systems, pp. 2096–2104 (2014)Google Scholar
  12. 12.
    Jurafsky, D., Martin, J.: Speech and Language Processing. Pearson Education (2000)Google Scholar
  13. 13.
    Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1746–1751 (2014)Google Scholar
  14. 14.
    Lacatusu, F., Hickl, A., Roberts, K., Shi, Y., Bensley, J., Rink, B., Wang, P., Taylor, L.: LCCs GISTexter at DUC 2006: Multi-strategy multi-document summarization. In: Proceedings of DUC 2006 (2006)Google Scholar
  15. 15.
    MacCartney, B., Grenager, T., de Marneffe, M.C., Cer, D., Manning, C.D.: Learning to recognize features of valid textual entailments. In: Proceedings of the Human Language Technology Conference of the NAACL, pp. 41–48 (2006)Google Scholar
  16. 16.
    Mikolov, T., Yih, W.t., Zweig, G.: Linguistic regularities in continuous space word representations. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 746–751 (2013)Google Scholar
  17. 17.
    Mou, L., Peng, H., Li, G., Xu, Y., Zhang, L., Jin, Z.: Discriminative neural sentence modeling by tree-based convolution. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 2315–2325 (2015)Google Scholar
  18. 18.
    Mou, L., Men, R., Li, G., Xu, Y., Zhang, L., Yan, R., Jin, Z.: Natural language inference by tree-based convolution and heuristic matching. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, vol. 2, pp. 130–136 (2016)Google Scholar
  19. 19.
    Rocktäschel, T., Grefenstette, E., Hermann, K.M., Kočiskỳ, T., Blunsom, P.: Reasoning about entailment with neural attention. In: Proceedings of the International Conference on Learning Representations (2016)Google Scholar
  20. 20.
    Silva, J., Coheur, L., Mendes, A., Wichert, A.: From symbolic to sub-symbolic information in question classification. Artif. Intell. Rev. 35(2), 137–154 (2011)CrossRefGoogle Scholar
  21. 21.
    Socher, R., Pennington, J., Huang, E., Ng, A., Manning, C.: Semi-supervised recursive autoencoders for predicting sentiment distributions. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 151–161 (2011)Google Scholar
  22. 22.
    Tai, K., Socher, R., Manning, D.: Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics, pp. 1556–1566 (2015)Google Scholar
  23. 23.
    Vendrov, I., Kiros, R., Fidler, S., Urtasun, R.: Order-embeddings of images and language. In: Proceedings of International Conference on Learning Representations (2016)Google Scholar
  24. 24.
    Yan, R., Song, Y., Wu, H.: Learning to respond with deep neural networks for retrieval-based human-computer conversation system. In: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 55–64 (2016)Google Scholar

Copyright information

© The Author(s) 2018

Authors and Affiliations

  1. 1.AdeptMind ResearchTorontoCanada
  2. 2.Institute of SoftwarePeking UniversityBeijingChina

Personalised recommendations