Relation Classification via Target-Concentrated Attention CNNs

  • Jizhao Zhu
  • Jianzhong QiaoEmail author
  • Xinxiao Dai
  • Xueqi Cheng
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10635)


Relation classification is a key natural language processing task that receives much attentions these years. The goal is to assign pre-defined relation labels to the nominal pairs marked in given sentences. It is obvious that different words in a sentence are differentially informative. Moreover, the importance of words is highly relation-dependent, i.e., the same word may be differentially important for different relations. To include sensitivity to this fact, we present a novel model, referred to as TCA-CNN, which takes the attention mechanism at the word level to pay different attention to individual words according to the semantic relation concentrated when constructing the representation of a sentence. Experimental results show that TCA-CNN achieves a comparable performance compared with the state-of-the-art models on the SemEval 2010 relation classification task.


Relation classification Convolutional Neural Networks Attention mechanism 



This work is supported by the 973 Program of China under Grant Nos. 2013CB329606 and 2014CB340405, the National Key Research and Development Program of China under Grant No. 2016YFB1000902, the National Natural Science Foundation of China (NSFC) under Grant Nos. 61272177, 61402442, 61572469, 91646120 and 61572473.


  1. 1.
    Wu, F., Weld, D.S.: Open information extraction using Wikipedia. In: 48th Annual Meeting of the Association for Computational Linguistics, pp. 118–127. ACL Press, Stroudsburg (2010)Google Scholar
  2. 2.
    Golub, D., He, X.: Character-level question answering with attention. arXiv preprint arXiv:1604.00727 (2016)
  3. 3.
    Shin, J., Wu, S., Wang, F., De Sa, C., Zhang, C., Ré, C.: Incremental knowledge base construction using deepdive. Proc. VLDB Endowment 8, 1310–1321 (2015)CrossRefGoogle Scholar
  4. 4.
    Jia, Y., Wang, Y., Lin, H., Jin, X., Cheng, X.: Locally adaptive translation for knowledge graph embedding. In: 30th AAAI Conference on Artificial Intelligence, pp. 992–998. AAAI Press, Menlo Park (2016)Google Scholar
  5. 5.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)CrossRefGoogle Scholar
  6. 6.
    Socher, R., Huval, B., Manning, C.D., Ng, A.Y.: Semantic compositionality through recursive matrix-vector spaces. In: 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1201–1211. ACL Press, Stroudsburg (2012)Google Scholar
  7. 7.
    Xu, Y., Mou, L., Li, G., Chen, Y., Peng, H., Jin, Z.: Classifying relations via long short term memory networks along shortest dependency paths. In: 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1785–1794. ACL Press, Stroudsburg (2015)Google Scholar
  8. 8.
    Wang, L., Cao, Z., de Melo, G., Liu, Z.: Relation classification via multi-level attention cnns. In: 54th Annual Meeting of the Association for Computational Linguistics, pp. 1398–1307. ACL Press, Stroudsburg (2016)Google Scholar
  9. 9.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997)CrossRefGoogle Scholar
  10. 10.
    Cho, K., Van Merriënboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014)
  11. 11.
    Zeng, D., Liu, K., Lai, S., Zhou, G., Zhao, J.: Relation classification via convolutional deep neural network. In: 25th International Conference on Computatinal Linguistics: Technical Papers, pp. 2335–2344. ACM, New York (2014)Google Scholar
  12. 12.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
  13. 13.
    Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12, 2121–2159 (2011)zbMATHMathSciNetGoogle Scholar
  14. 14.
    Hendrickx, I., Kim, S. N., Kozareva, Z., Nakov, P., Ó Séaghdha, D., Padó, S., Pennacchiotti, M., Romano, L., Szpakowicz, S.: Semeval-2010 task 8: multi-way classification of semantic relations between pairs of nominals. In: Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions, pp. 94–99. ACL Press, Stroudsburg (2009)Google Scholar
  15. 15.
    Santos, C.N.D., Xiang, B., Zhou, B.: Classifying relations by ranking with convolutional neural networks. arXiv preprint arXiv:1504.06580 (2015)
  16. 16.
    Liu, Y., Wei, F., Li, S., Ji, H., Zhou, M., Wang, H.: A dependency-based neural network for relation classification. In: 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing(Short Papers), pp. 285–290. ACL Press, Stroudsburg (2015)Google Scholar
  17. 17.
    Xu, K., Feng, Y., Huang, S., Zhao, D.: Semantic relation classification via convolutional neural networks with simple negative sampling. In: 2015 Conference on Empirical Methods in Natural Language Processing, pp. 536–540. ACL Press, Stroudsburg (2015)Google Scholar
  18. 18.
    Xu, Y., Jia, R., Mou, L., Li, G., Chen, Y., Lu, Y., Jin, Z.: Improved relation classification by deep recurrent neural networks with data augmentation. arXiv preprint arXiv:1601.03651 (2016)

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Jizhao Zhu
    • 1
  • Jianzhong Qiao
    • 1
    Email author
  • Xinxiao Dai
    • 2
  • Xueqi Cheng
    • 3
  1. 1.College of Computer Science and EngineeringNortheastern UniversityShenyangChina
  2. 2.Shenyang Open UniversityShenyangChina
  3. 3.CAS Key Laboratory of Network Data Science and TechnologyInstitute of Computing Technology, Chinese Academy of SciencesBeijingChina

Personalised recommendations