Logical Parsing from Natural Language Based on a Neural Translation Model

  • Liang Li
  • Yifan Liu
  • Zengchang Qin
  • Pengyu Li
  • Tao Wan
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 781)

Abstract

Semantic parsing has emerged as a powerful paradigm for natural language interface and question answering systems. Traditional methods of building a semantic parser rely on high-quality lexicons, hand-crafted grammars and linguistic features which are limited by applied domain or representation. In this paper, we propose an approach to learn from denotations based on the Seq2Seq model augmented with attention mechanism. We encode input sequence into vectors and use dynamic programming to infer candidate logical forms. We utilize the fact that similar utterances should have similar logical forms to help reduce the searching space. Through learning mechanism of the Seq2Seq model, we can learn mappings gradually with noises. Curriculum learning is adopted to make the learning smoother. We test our model on a small arithmetic domain which shows our model can successfully infer the correct logical forms and learn a meaningful semantic parser.

Keywords

Logical parsing Neural language understanding Seq2Seq Attention model 

Notes

Acknowledgement

This work is supported by the National Science Foundation of China Nos. 61401012 and 61305047.

References

  1. 1.
    Artzi, Y., Zettlemoyer, L.: Weakly supervised learning of semantic parsers for mapping instructions to actions. Trans. Assoc. Comput. Linguist. 1, 49–62 (2013)Google Scholar
  2. 2.
    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate (2014). arXiv preprint arXiv:1409.0473
  3. 3.
    Cho, K., Merrienboer, B.V., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: encoder-decoder approaches (2014). arXiv preprint arXiv:1409.1259
  4. 4.
    Cho, K., Merrienboer, B.V., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. EMNLP, pp. 1724–1734 (2014)Google Scholar
  5. 5.
    Clarke, J., Goldwasser, D., Chang, M.W., Roth, D.: Driving semantic parsing from the world’s response. In: Fourteenth Conference on Computational Natural Language Learning, pp. 18–27 (2010)Google Scholar
  6. 6.
    Dong, L., Lapata, M.: Language to logical form with neural attention. In: Meeting of the Association for Computational Linguistics, pp. 33–43 (2016)Google Scholar
  7. 7.
    Ge, R., Mooney, R.J.: A statistical semantic parser that integrates syntax and semantics. In: Conference on Computational Natural Language Learning, pp. 9–16 (2005)Google Scholar
  8. 8.
    Graves, A., Wayne, G., Danihelka, I.: Neural turing machines (2014). arXiv preprint arXiv:1410.5401
  9. 9.
    Hermann, K.M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., Blunsom, P.: Teaching machines to read and comprehend. In: NIPS, pp. 1693–1701 (2015)Google Scholar
  10. 10.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735 (1997)CrossRefGoogle Scholar
  11. 11.
    Krishnamurthy, J., Mitchell, T.M.: Weakly supervised training of semantic parsers. In: Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 754–765 (2012)Google Scholar
  12. 12.
    Lao, N., Mitchell, T., Cohen, W.W.: Random walk inference and learning in a large scale knowledge base. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pp. 529–539 (2011)Google Scholar
  13. 13.
    Lau, T., Wolfman, S.A., Domingos, P., Weld, D.S.: Programming by demonstration using version space algebra. Mach. Learn. 53(1), 111–156 (2003)CrossRefMATHGoogle Scholar
  14. 14.
    Liang, C., Berant, J., Le, Q., Forbus, K.D., Lao, N.: Neural symbolic machines: Learning semantic parsers on freebase with weak supervision (2016). arXiv preprint arXiv:1611.00020
  15. 15.
    Liang, P., Jordan, M.I., Klein, D.: Learning dependency-based compositional semantics. In: Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 590–599 (2011)Google Scholar
  16. 16.
    Liang, P., Potts, C.: Bringing machine learning and compositional semantics together. Annu. Rev. Linguist. 1(1), 355–376 (2015)CrossRefGoogle Scholar
  17. 17.
    Lu, W., Ng, H.T., Lee, W.S., Zettlemoyer, L.S.: A generative model for parsing natural language to meaning representations. In: EMNLP, pp. 783–792 (2008)Google Scholar
  18. 18.
    Miller, S., Stallard, D., Bobrow, R., Schwartz, R.: A fully statistical approach to natural language interfaces. In: Proceedings of the 34th annual meeting on Association for Computational Linguistics, pp. 55–61 (2002)Google Scholar
  19. 19.
    Neelakantan, A., Le, Q.V., Abadi, M., Mccallum, A., Amodei, D.: Learning a natural language interface with neural programmer (2016). arXiv preprint arXiv:1611.08945
  20. 20.
    Neelakantan, A., Le, Q.V., Sutskever, I.: Neural programmer: inducing latent programs with gradient descent. Computer Science (2015)Google Scholar
  21. 21.
    Pasupat, P., Liang, P.: Compositional semantic parsing on semi-structured tables (2015). arXiv preprint arXiv:1508.00305
  22. 22.
    Pasupat, P., Liang, P.: Inferring logical forms from denotations (2016). arXiv preprint arXiv:1606.06900
  23. 23.
    Reddy, S., Lapata, M., Steedman, M.: Large-scale semantic parsing without question-answer pairs. Trans. Assoc. Comput. Linguist. 2, 377–392 (2014)Google Scholar
  24. 24.
    Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. Adv. Neural Inf. Proc. Syst. 4, 3104–3112 (2014)Google Scholar
  25. 25.
    Vinyals, O., Kaiser, L., Koo, T., Petrov, S., Sutskever, I., Hinton, G.: Grammar as a foreign language. eprint Arxiv (2014)Google Scholar
  26. 26.
    Wang, Y., Berant, J., Liang, P.: Building a semantic parser overnight. In: Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, pp. 1332–1342 (2015)Google Scholar
  27. 27.
    Zettlemoyer, L.: Learning to map sentences to logical form. eprint Arxiv, pp. 658–666 (2010)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  • Liang Li
    • 1
  • Yifan Liu
    • 1
  • Zengchang Qin
    • 1
  • Pengyu Li
    • 2
  • Tao Wan
    • 3
  1. 1.Intelligent Computing and Machine Learning Lab School of ASEEBeihang UniversityBeijingChina
  2. 2.School of Mechanical Engineering and AutomationBeihang UniversityBeijingChina
  3. 3.School of Biological Science and Medical EngineeringBeihang UniversityBeijingChina

Personalised recommendations