Advertisement

Bidirectional Independently Long Short-Term Memory and Conditional Random Field Integrated Model for Aspect Extraction in Sentiment Analysis

  • Trang Uyen Tran
  • Ha Thi-Thanh HoangEmail author
  • Hiep Xuan Huynh
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1014)

Abstract

Aspect extraction or feature extraction is a crucial and challenging task of opinion mining that aims to identify opinion targets from opinion text. Especially, how to explore these aspects or features from unstructured comments is a matter of concern. In this paper, we propose a novel supervised learning approach using deep learning technique for the above-mentioned aspect extraction task. Our model combines a bidirectional independently long short-term memory (Bi-IndyLSTM) with a Conditional Random Field (CRF). This integrated model is trained on labelled data to extract feature sets in opinion text. We employ a Bi-IndyLSTM with word embeddings achieved by training GloVe on the SemEval 2014 data set. There are 6086 training reviews and 1600 testing reviews on two domains, Laptop and Restaurant of the SemEval 2014 data set. Experimental results showed that our proposed Bi-IndyLSTM-CRF aspect extraction model in sentiment analysis obtained considerably better accuracy than the state-of-the-art methods.

Keywords

Aspect extraction Bi-IndyLSTM CRF 

References

  1. 1.
    Liu, B.: Sentiment Analysis and Opinion Mining. Morgan and Claypool Publishers, San Rafael (2012)CrossRefGoogle Scholar
  2. 2.
    Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: EMNLP’14 Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1532–1543. Doha, 25–29 Oct 2014Google Scholar
  3. 3.
    Hu, M., Liu, B: Mining opinion features in customer reviews. In: AAAI’04 Proceedings of the 19th National Conference on Artificial Intelligence, pp. 755–760, 25–29 July 2004Google Scholar
  4. 4.
    Popescu, A.M., Etzioni O.: Extracting product features and opinions from reviews. In: HLT‘05 Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, pp. 339–346, 6–8 Oct 2005Google Scholar
  5. 5.
    Scaffidi, C., Bierhoff, K., Chang, E., Felker, M., Ng, H., Jin, C.: Red Opal: product-feature scoring from reviews. In: EC’07 Proceedings of the 8th ACM Conference on Electronic Commerce, vol. 2, pp. 182–191, 11–15 June 2007Google Scholar
  6. 6.
    Long, C., Zhang, Z., Zhu, X.: A review selection approach for accurate feature rating estimation. In: COLING’10 Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pp. 766–774. Beijing, 23–27 Aug 2010Google Scholar
  7. 7.
    Kobayashi, N., Inui, K., Matsumoto, Y.: Extracting aspect–evaluation and aspect-of relations in Opinion mining. In: Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1065–1074. Prague, June 2007Google Scholar
  8. 8.
    Wu, Y., Zhang, Q., Huang, X., Wu, L: Phrase dependency parsing for opinion mining. In: EMNLP’09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, vol. 3, pp. 1533–1541. Singapore, 6–7 Aug 2009Google Scholar
  9. 9.
    Qiu, G., Liu, B., Bu, J., Chen, C.: Opinion word expansion and target extraction through double propagation. In: 2011 Association for Computational Linguistics, vol. 37, no. 1 (2011)Google Scholar
  10. 10.
    Lu, Y., Zhai, C.X., Sundaresan, N.: Rated aspect summarization of short comments. In: WWW’09 Proceedings of the 18th ACM International Conference on World Wide Web, pp. 131–140. Madrid, 20–24 Apr 2009Google Scholar
  11. 11.
    Lin, C., He, Y.: Joint sentiment/topic model for sentiment analysis. In: CIKM’09 Proceedings of the 18th ACM Conference on Information and Knowledge Management, pp. 375–384. Hong Kong, 2–6 Nov 2009Google Scholar
  12. 12.
    Brody, S., Elhadad, N.: An unsupervised aspect-sentiment model for online reviews. In: HLT’10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 804–812. Los Angeles, 2–4 June 2010Google Scholar
  13. 13.
    Jakob, N., Gurevych, I.: Extracting opinion targets in a single- and cross-domain setting with conditional random fields. In: EMNLP’10 Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 1035–1045. Cambridge, Massachusetts, 9–11 Oct 2010Google Scholar
  14. 14.
    Li, F., Han, C., Huang, M., Zhu, X., Xia, Y.-J., Zhang, S., Yu, H.: Structure-aware review mining and summarization. In: COLING’10 Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pp. 653–661. Beijing, 23–27 Aug 2010Google Scholar
  15. 15.
    Zhang, M., Zhang, Y., Vo, D.T.: Neural networks for open domain targeted sentiment. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 612–621. Lisbon, 17–21 Sept 2015Google Scholar
  16. 16.
    Wang, W., Pan, S.J., Dahlmeier, D., Xiao, X.: Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis. ArXiv:1603.06679v3 [cs.CL], 19 Sept 2016Google Scholar
  17. 17.
    Katiyar, A., Cardie, C.: Investigating LSTMs for joint extraction of opinion entities and relations. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 919–929. Berlin, Aug 2016Google Scholar
  18. 18.
    Irsoi, O., Cardie, C.: Opinion mining with deep recurrent neural networks. In: EMNLP’14 Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 720–728. Doha, 25–29 Oct 2014Google Scholar
  19. 19.
    Liu, P., Joty, S., Meng, H.: Fine-grained opinion mining with recurrent neural networks and word embeddings. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1433–1443. Lisbon, 17–21 Sept 2015Google Scholar
  20. 20.
    Poria, S., Cambria, E., Gelbukh, A.: Aspect extraction for opinion mining with a deep convolutional neural network. In: Knowledge-Based Systems, vol. 108, pp. 42–49, 15 Sept 2016Google Scholar
  21. 21.
    Giannakopoulos, A., Musat, C., Hossmann, A., Baeriswyl, M.: Unsupervised Aspect Term Extraction with B-LSTM & CRF Using Automatically Labelled Datasets. ArXiv:1709.05094v1 [cs.CL], 15 Sept 2017Google Scholar
  22. 22.
    Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation. ArXiv:1406.1078v3 [cs.CL], 3 Sept 2014Google Scholar
  23. 23.
    Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166. New York, USA (1994)Google Scholar
  24. 24.
    Li, S., Li, W., Cook, C., Zhu, C., Gao, Y.: Independently Recurrent Neural Network (IndRNN): Building a Longer and Deeper RNN. ArXiv: 1803.04831v3 [cs.CV], 22 May 2018Google Scholar
  25. 25.
    Lafferty, J., McCallum, A., Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the 18th ACM International Conference on Machine Learning, pp. 282–289, 28 June 2001Google Scholar
  26. 26.
    Yang, B., Cardie, C.: Extracting opinion expressions with semi-Markov conditional random fields. In: EMNLP-CoNLL’12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1335–1345. Jeju Island, Korea, 12–14 July 2012Google Scholar
  27. 27.
    Rubtsova, Y., Koshelnikov, S.: Aspect extraction from reviews using conditional random fields. In: International Conference on Knowledge Engineering and the Semantic Web, KESW 2015, CCIS, vol. 518, pp. 158–167, 30 Oct 2015Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  • Trang Uyen Tran
    • 1
  • Ha Thi-Thanh Hoang
    • 2
    Email author
  • Hiep Xuan Huynh
    • 3
  1. 1.Faculty of InformaticsUniversity of Education, The Danang UniversityDanangVietnam
  2. 2.Faculty of Statistics and InformaticsDanang University of Economic, The Danang UniversityDanangVietnam
  3. 3.College of Information and Communication TechnologyCan Tho UniversityCanthoVietnam

Personalised recommendations