Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Encoding syntactic representations with a neural network for sentiment collocation extraction

Abstract

Sentiment collocation refers to the collocation of a target word and a polarity word. Sentiment collocation extraction aims to extract the targets and their modifying polarity words by analyzing the relationships between them. This can be regarded as a basic sentiment analysis task and is relevant in many practical applications. Previous studies relied mainly on the syntactic path, which is used to connect the target word and the polarity word. To deeply exploit the semantic information of the syntactic path, we propose two types of syntactic representation, namely, relation embedding and subtree embedding, to capture the latent semantic features. Relation embedding is used to represent the latent semantics between targets and their corresponding polarity words, and subtree embedding is used to explore the rich syntactic information for each word on the path. To combine the two types of syntactic representations, a neural network is constructed. We use a recursive neural network (RNN) to model the subtree embeddings, and then the subtree embedding and the word embedding are combined as the enhanced word representation for each word in the syntactic path. Finally, a convolutional neural network (CNN) is adopted to integrate the two types of syntactic representations to extract the sentiment collocations from reviews. Our experiments were conducted on six types of reviews, which included product domains (such as cameras and phones) and service domains (such as hotels and restaurants). The experimental results show that our proposed method can accurately capture the latent semantic features hidden behind the syntactic paths that neither the common feature-based methods nor the syntactic-path-based method can handle, and, further, that it significantly outperforms numerous baselines and previous methods.

This is a preview of subscription content, log in to check access.

References

  1. 1

    Pang B, Lee L L. Opinion mining and sentiment analysis. Found Trends Inf Retr, 2008, 2: 1–135

  2. 2

    Liu B. Sentiment analysis and opinion mining. Synth Lect Human Language Tech, 2012, 5: 1–167

  3. 3

    Abbasi A, Chen H, Salem A. Sentiment analysis in multiple languages: feature selection for opinion classification in web forums. ACM Trans Inf Syst, 2008, 26: 1–34

  4. 4

    Duric A, Song F. Feature selection for sentiment analysis based on content and syntax models. Decis Support Syst, 2012, 53: 704–711

  5. 5

    Bloom K, Garg N, Argamon S. Extracting appraisal expressions. In: Proceedings of Human Language Technologies: the Annual Conference of the North American Chapter of the Association for Computational Linguistics, Rochester, 2007. 308–315

  6. 6

    Qiu G, Liu B, Bu J J, et al. Opinion word expansion and target extraction through double propagation. Comput Linguist, 2011, 37: 9–27

  7. 7

    Xu L H, Liu K, Lai S, et al. Mining opinion words and opinion targets in a two-stage framework. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, Sofia, 2013. 1764–1773

  8. 8

    Zhao Y Y, Che W X, Guo H L, et al. Sentence compression for target-polarity word collocation extraction. In: Proceedings of the 25th International Conference on Computational Linguistics: Technical Papers, Dublin, 2014. 1360–1369

  9. 9

    Zhao Y Y, Li S Q, Qin B, et al. Encoding dependency representation with convolutional neural network for targetpolarityword collocation extraction. In: Social Meida Processing. Singapore: Springer, 2016

  10. 10

    Chen Y B, Xu L, Liu K, et al. Event extraction via dynamic multi-pooling convolutional neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, 2015. 167–176

  11. 11

    Liu Y, Wei F R, Li S J, et al. A dependency-based neural network for relation classification. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, 2015. 285–290

  12. 12

    Meng F D, Lu Z D, Wang M X, et al. Encoding source language with convolutional neural network for machine translation. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, 2015. 20–30

  13. 13

    Nguyen T H, Grishman R. Event detection and domain adaptation with convolutional neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, 2015. 365–371

  14. 14

    Vu N T, Adel H, Gupta P, et al. Combining recurrent and convolutional neural networks for relation classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, 2016. 534–539

  15. 15

    Lee J Y, Dernoncourt F. Sequential short-text classification with recurrent and convolutional neural networks. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, 2016. 515–520

  16. 16

    Zhao J, Xu H B, Huang X J, et al. Overview of chinese pinion analysis evaluation 2008. Proceedings of the 1st Chinese Opinion Analysis Evaluation (COAE), 2008

  17. 17

    Che W X, Li Z H, Liu T. LTP: a chinese language technology platform. In: Proceedings of the 23rd International Conference on Computational Linguistics: Demonstrations, Beijing, 2010. 13–16

  18. 18

    Joachims T. Learning to Classify Text Using Support Vector Machines–Methods, Theory, and Algorithms. Norwell: Kluwer Academic Publishers, 2002

  19. 19

    Gui L, Zhou Y, Xu R F, et al. Learning representations from heterogeneous network for sentiment classification of product reviews. Knowledge-Based Syst, 2017, 124: 34–45

  20. 20

    Chen T, Xu R F, He Y L, et al. Learning user and product distributed representations using a sequence model for sentiment analysis. IEEE Comput Intell Mag, 2016, 11: 34–44

  21. 21

    Hu M Q, Liu B. Mining and summarizing customer reviews. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, 2004. 168–177

  22. 22

    dos Santos C, Xiang B, Zhou B. Classifying relations by ranking with convolutional neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, 2015. 626–634

  23. 23

    Kim Y. Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, 2014. 1746–1751

  24. 24

    Ma M B, Huang L, Xiang B, et al. Dependency-based convolutional neural networks for sentence embedding. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, 2015. 174–179

Download references

Acknowledgements

This work was supported by National Basic Research Program of China (973 Program) (Grant No. 2014CB340506) and National Natural Science Foundation of China (Grant Nos. 61632011, 61370164). We thank the anonymous reviewers for their helpful comments.

Author information

Correspondence to Bing Qin.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhao, Y., Qin, B. & Liu, T. Encoding syntactic representations with a neural network for sentiment collocation extraction. Sci. China Inf. Sci. 60, 110101 (2017). https://doi.org/10.1007/s11432-016-9229-y

Download citation

Keywords

  • sentiment collocation extraction
  • sentiment analysis
  • syntactic representation
  • neural network
  • recursive neural network (RNN)
  • convolutional neural network (CNN)