TSABCNN: Two-Stage Attention-Based Convolutional Neural Network for Frame Identification

  • Hongyan Zhao
  • Ru Li
  • Fei Duan
  • Zepeng Wu
  • Shaoru Guo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11221)


As an essential sub-task of frame-semantic parsing, Frame Identification (FI) is a fundamentally important research topic in shallow semantic parsing. However, most existing work is based on sophisticated, hand-crafted features which might not be compatible with FI procedure. Besides that, they usually heavily rely on available natural language processing (NLP) toolkits and various lexical resources. Thus existing methods with hand-crafted features may not achieve satisfactory performance. In this paper, we propose a two-stage attention-based convolutional neural network (TSABCNN) to alleviate this problem and capture the most important context features for FI task. In order to dynamically adjust the weight of each feature, we build two levels of attention over instances at input layer and pooling layer respectively. Furthermore, the proposed model is an end-to-end learning framework which does not need any complicated NLP toolkits and feature engineering, and can be applied to any language. Experiments results on FrameNet and Chinese FrameNet (CFN) show the effectiveness of the proposed approach for the FI task.


Frame identification FrameNet Convolutional neural network 



This work is supported by National Natural Science Foundation of China (No. 61772324, No. 61673248), and Shanxi Province Postgraduate Joint Training Base Talent Training Project (No. 2018JD01, No. 2018JD02).


  1. Baker, C.F., Fillmore, C.J., Lowe, J.B.: The Berkeley FrameNet project. In: Meeting of the Association for Computational Linguistics and, International Conference on Computational Linguistics, vol. 47, pp. 86–90 (1998)Google Scholar
  2. Fillmore, C.J.: Frame semantics. In: Linguistics in the Morning Calm, pp. 111–138 (1982)Google Scholar
  3. Hermann, K.M., et al.: Teaching machines to read and comprehend. In: NIPS, pp. 1693–1701 (2015)Google Scholar
  4. Das, D., Schneider, N., Chen, D., Smith, N.A.: Probabilistic frame-semantic parsing. In: Human Language Technologies: The 2010 Conference of the North American Chapter of the Association for Computational Linguistics, vol. 40, pp. 948–956 (2010)Google Scholar
  5. Das, D., Chen, D., Schneider, N., Smith, N.A.: Frame-semantic parsing. Comput. Linguist. 40(1), 9–56 (2014)CrossRefGoogle Scholar
  6. Carreras, X., Litkowski, K.C., Stevenson, S.: Semantic role labeling: an introduction to the special issue. Comput. Linguist. 34(2), 145–159 (2008)CrossRefGoogle Scholar
  7. Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A Convolutional Neural Network for Modelling Sentences. eprint arxiv (2014)Google Scholar
  8. Kim, Y.: Convolutional Neural Networks for Sentence Classification. eprint arxiv (2014)Google Scholar
  9. Zeng, D.J., Liu, K., Lai, S.W., Zhou, G.Y., Zhao, J.: Relation classification via convolutional deep neural network. In: Proceedings of COLING, Dublin, Ireland (2014)Google Scholar
  10. Chen, Y., Xu, L., Liu, K., Zeng, D., Zhao, J.: Event extraction via dynamic multi-pooling convolutional neural networks. In: The Meeting of the Association for Computational Linguistics, Beijing, China, pp. 167–176 (2015)Google Scholar
  11. Dong, L., Wei, F., Zhou, M., Xu, K.: Question answering over freebase with multi-column convolutional neural networks. In: Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, Beijing, China, pp. 260–269 (2015)Google Scholar
  12. Severyn, A., Moschitti, A.: Learning to rank short text pairs with convolutional deep neural networks. In: The International ACM SIGIR Conference, pp. 373–382. ACM (2015)Google Scholar
  13. Yin, W., Schütze, H.: MultiGranCNN: an architecture for general matching of text chunks on multiple levels of granularity. In: Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, Beijing, China, pp. 63–73 (2015)Google Scholar
  14. Gildea, D., Jurafsky, D.: Automatic labeling of semantic roles. Comput. Linguist. 28(28), 245–288 (2002)CrossRefGoogle Scholar
  15. Carreras, X.: Introduction to the CoNLL-2004 shared task: semantic role labeling. In: Proceedings of 8th Conference on Natural Language Learning (CoNLL), Boston, MA, vol. 47, pp. 5–9 (2004)Google Scholar
  16. Carreras, X., Màrquez, L.: Introduction to the CoNLL-2005 shared task: semantic role labeling. In: Proceedings of the 9th Conference on Computational Natural Language Learning (CoNLL), ANN Arbor, MI, pp. 152–164 (2005)Google Scholar
  17. Fleischman, M., Kwon, N., Hovy, E.: Maximum entropy models for FrameNet classification. In: Conference on Empirical Methods in Natural Language Processing, Sapporo, pp. 49–56 (2003)Google Scholar
  18. Erk, K.: Frame assignment as word sense disambiguation. In: proceedings of the 6th International Workshop on Computational Semantic (IWCS-6) (2005)Google Scholar
  19. Johansson, R., Nugues, P.: LTH: semantic structure extraction using nonprojective dependency trees. In: Proceedings of SemEval, vol. 13, no. 4, pp. 227–230 (2007)Google Scholar
  20. Bejan, C.A., Hathaway, C.: UTD-SRL: a pipeline architecture for extracting frame semantic structures. In: International Workshop on Semantic Evaluations, vol. 8764, pp. 460–463 (2007)Google Scholar
  21. Hermann, K.M., Das, D., Weston, J., Ganchev, K.: Semantic frame identification with distributed word representations. In: Meeting of the Association for Computational Linguistics, vol. 1, pp. 1448–1458 (2014)Google Scholar
  22. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv: 1409.0473 (2014)Google Scholar
  23. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12(1), 2493–2537 (2011)zbMATHGoogle Scholar
  24. Yin, W.P., Schütze, H., Xiang, B., Zhou, B.W.: ABCNN: attention-based convolutional neural network for modeling sentence pairs. Action Editor: Brian Roark, vol. 4, pp. 259–272 (2016)Google Scholar
  25. Collobert R., Weston J.: A unified architecture for natural language processing: deep neural networks with multitask learning. International Conference on Machine Learning, pp.160–167. ACM (2008)Google Scholar
  26. Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R. R.: Improving neural networks by preventing co-adaptation of feature detectors. Comput. Sci. vol. 3(4), 212–223 (2012)Google Scholar
  27. Zhang, Y., Wallace, B.: A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification. In: Proceedings of the 8th International Joint Conference on Natural Language Processing, Taipei, Taiwan, pp. 253–263 (2017)Google Scholar
  28. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, vol. 26, pp. 3111–3119 (2013)Google Scholar
  29. Zhou, P., et al.: Attention-based bidirectional long short-term memory networks for relation classification. In: Meeting of the Association for Computational Linguistics, pp. 207–212 (2016)Google Scholar
  30. Li, R., Liu, H.J., Li, S.H.: Chinese frame identification using T-CRF model. In: International Conference on Computational Linguistics, COLING 2010, pp. 674–682 (2010)Google Scholar
  31. Zhao, H.Y., Li, R., Zhang, S., Zhang, L.W.: Chinese frame identification with deep neural network. J. Chin. Inf. Process. 30(6), 75–83 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Computer and Information TechnologyShanxi UniversityTaiyuanChina
  2. 2.School of Computer Science and TechnologyTaiyuan University of Science and TechnologyTaiyuanChina
  3. 3.Key Laboratory of Ministry of Education for Computation Intelligence and Chinese Information ProcessingShanxi UniversityTaiyuanChina

Personalised recommendations