Skip to main content
SpringerLink
Log in

RecogNet-LSTM+CNN: a hybrid network with attention mechanism for aspect categorization and sentiment classification

  • Published:
Journal of Intelligent Information Systems Aims and scope Submit manuscript

Abstract

Sentiment analysis for user reviews has received substantial heed in recent years. There are many deep learning models for natural language processing (NLP) applications. Long-short term memory (LSTM) and Convolutional neural network (CNN) based models efficiently enhance sentiment accuracy. Aspect-level sentiment analysis involves aspect extraction, aspect categorization, and polarity classification. The aspect sentiments in the dataset are classified as positive, negative, and neutral, depending on the polarity score associated with the aspect emotions. Existing neural architectures combining LSTM and CNN employ only the implicit information from the dataset for sentiment classification. Alternatively, this paper highlights the integration of explicit knowledge from the external database (RecogNet) with the implicit information of the LSTM model to improvise the sentiment accuracy. Incorporating sentic and semantic clues from the RecogNet knowledge base to the LSTM increases aspect extraction and categorization efficiency. Furthermore, we implemented CNN with target and position attention mechanisms over the RecogNet-LSTM layer to further enhance the classification accuracy. Finally, the model evaluations are performed using five online datasets related to the restaurants, laptops, and locations. Among LSTM based hybrid models, our RecogNet-LSTM+CNN model with attention mechanism showed superior performance in aspect categorization and opinion classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Basiri, M. E., Nemati, S., Abdar, M., Cambria, E., & Acharya, U. R. (2021). ABCDM: An attention-based bidirectional CNN-RNN deep model for sentiment analysis. Future Generation Computer Systems, 115, 279–294. [Online]. Available: https://doi.org/10.1016/j.future.2020.08.005.

    Article  Google Scholar 

  • Brun, C., Perez, J., & Roux, C. (2016). XRCE at SemEval-2016 task 5: Feedbacked ensemble modeling on syntactico-semantic knowledge for aspect based sentiment analysis. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016). [Online]. Available: https://doi.org/10.18653%2Fv1%2Fs16-1044 (pp. 277–281). Association for Computational Linguistics.

  • Cambria, E., Das, D., Bandyopadhyay, S., & Feraco, A. (2017). Affective computing and sentiment analysis. In A Practical Guide to Sentiment Analysis. [Online]. Available: https://doi.org/10.1007%2F978-3-319-55394-8_1 (pp. 1–10). Springer International Publishing.

  • Cambria, E., Fu, J., Bisio, F., & Poria, S. (2015). Affectivespace 2: Enabling affective intuition for concept-level sentiment analysis. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. [Online]. Available: https://doi.org/10.5555/2887007.2887078 (pp. 508–514).

  • Cambria, E., Hussain, A., Havasi, C., & Eckl, C. (2009). Common sense computing: From the society of mind to digital intuition and beyond. In Biometric ID Management and Multimodal Communication. [Online]. Available: https://doi.org/10.1007%2F978-3-642-04391-8_33 (pp. 252–259). Springer Berlin Heidelberg.

  • Cambria, E., Li, Y., Xing, F. Z., Poria, S., & Kwok, K. (2020). SenticNet 6: Ensemble application of symbolic and subsymbolic AI for sentiment analysis. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. [Online]. Available: https://doi.org/10.1145%2F3340531.3412003 (pp. 105–114). ACM.

  • Chen, Q., Hu, Q., Huang, J. X., He, L., & An, W. (2017). Enhancing recurrent neural networks with positional attention for question answering. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. [Online]. Available: https://doi.org/10.1145%2F3077136.3080699(pp. 993–996). ACM.

  • Das, S. R., & Chen, M. Y. (2007). Yahoo! for amazon: Sentiment extraction from small talk on the web. Management Science, 53(9), 1375–1388. [Online]. Available: https://doi.org/10.1287/mnsc.1070.0704.

    Article  Google Scholar 

  • Dong, L., Wei, F., Tan, C., Tang, D., Zhou, M., & Xu, K. (2014). Adaptive recursive neural network for target-dependent twitter sentiment classification. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). [Online]. Available: https://doi.org/10.3115%2Fv1%2Fp14-2009 (pp. 49–54). Association for Computational Linguistics.

  • Gehring, J., Auli, M., Grangier, D., Yarats, D., & Dauphin, Y. (2017). Convolutional sequence to sequence learning. In Proceedings of the 34th International Conference on Machine Learning. [Online]. Available: https://proceedings.mlr.press/v70/gehring17a/gehring17a.pdf (pp. 2029–2042).

  • Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. [Online]. Available: https://doi.org/10.1162/neco.1997.9.8.1735.

    Article  Google Scholar 

  • Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014). A convolutional neural network for modelling sentences. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). [Online]. Available: https://doi.org/10.3115%2Fv1%2Fp14-1062 (pp. 655–665). Association for Computational Linguistics.

  • Kang, J., Choi, H., & Lee, H. (2018). Deep recurrent convolutional networks for inferring user interests from social media. Journal of Intelligent Information Systems, 52(1), 191–209. [Online]. Available: https://doi.org/10.1007/s10844-018-0534-3.

    Article  Google Scholar 

  • Khan, M. U., Javed, A. R., Ihsan, M., & Tariq, U. (2020). A novel category detection of social media reviews in the restaurant industry. Multimedia Systems, 1–14. [Online]. Available: https://doi.org/10.1007/s00530-020-00704-2.

  • Kiritchenko, S., Zhu, X., Cherry, C., & Mohammad, S. (2014). NRC-canada-2014: Detecting aspects and sentiment in customer reviews. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). [Online]. Available: https://doi.org/10.3115%2Fv1%2Fs14-2076 (pp. 437–442). Association for Computational Linguistics.

  • Liu, G., & Guo, J. (2019). Bidirectional LSTM with attention mechanism and convolutional layer for text classification. Neurocomputing, 337, 325–338. [Online]. Available: https://doi.org/10.1016/j.neucom.2019.01.078.

    Article  Google Scholar 

  • Ma, Y., Peng, H., Khan, T., Cambria, E., & Hussain, A. (2018). Sentic LSTM: a hybrid network for targeted aspect-based sentiment analysis. Cognitive Computation, 10(4), 639–650. [Online]. Available: https://doi.org/10.1007/s12559-018-9549-x.

    Article  Google Scholar 

  • Mehta, Y., Majumder, N., Gelbukh, A., & Cambria, E. (2019). Recent trends in deep learning based personality detection. Artificial Intelligence Review, 53(4), 2313–2339. [Online]. Available: https://doi.org/10.1007/s10462-019-09770-z.

    Article  Google Scholar 

  • Meyer, B., Bikdash, M., & Dai, X. (2017). Fine-grained financial news sentiment analysis. In SoutheastCon 2017. [Online]. Available: https://doi.org/10.1109%2Fsecon.2017.7925378 (pp. 1–8). IEEE.

  • Mohammadi, A., & Shaverizade, A. (2021). Ensemble deep learning for aspect-based sentiment analysis. International Journal of Nonlinear Analysis and Applications, 12, 29–38. [Online]. Available: https://doi.org/10.22075/ijnaa.2021.4769, https://ijnaa.semnan.ac.ir/article_4769.html.

    Google Scholar 

  • Mousa, A., & Schuller, B. (2017). Contextual bidirectional long short-term memory recurrent neural network language models: A generative approach to sentiment analysis. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. [Online]. Available: https://doi.org/10.18653%2Fv1%2Fe17-1096 (pp. 1023–1032). Association for Computational Linguistics.

  • Nguyen, H. T., & Nguyen, M. L. (2019). An ensemble method with sentiment features and clustering support. Neurocomputing, 370, 155–165. [Online]. Available: https://doi.org/10.1016/j.neucom.2019.08.071.

    Article  Google Scholar 

  • Pennington, J., Socher, R., & Manning, C. (2014). Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). [Online]. Available: https://doi.org/10.3115%2Fv1%2Fd14-1162 (pp. 1532–1543). Association for Computational Linguistics.

  • Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., AL-Smadi, M., Al-Ayyoub, M., Zhao, Y., Qin, B., Clercq, O. D., Hoste, V., Apidianaki, M., Tannier, X., Loukachevitch, N., Kotelnikov, E., Bel, N., Jiménez-Zafra, S. M., & Eryiğit, G. (2016). SemEval-2016 task 5: Aspect based sentiment analysis. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016). [Online]. Available: https://doi.org/10.18653%2Fv1%2Fs16-1002 (pp. 19–30). Association for Computational Linguistics.

  • Pontiki, M., Galanis, D., Papageorgiou, H., Manandhar, S., & Androutsopoulos, I. (2015). SemEval-2015 task 12: Aspect based sentiment analysis. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015). [Online]. Available: https://doi.org/10.18653%2Fv1%2Fs15-2082 (pp. 486–495). Association for Computational Linguistics.

  • Pontiki, M., Galanis, D., Pavlopoulos, J., Papageorgiou, H., Androutsopoulos, I., & Manandhar, S. (2014). SemEval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). [Online]. Available: https://doi.org/10.3115%2Fv1%2Fs14-2004 (pp. 27–35). Association for Computational Linguistics.

  • Poria, S., Cambria, E., & Gelbukh, A. (2016). Aspect extraction for opinion mining with a deep convolutional neural network. Knowledge-Based Systems, 108, 42–49. [Online]. Available: https://doi.org/10.1016/j.knosys.2016.06.009.

    Article  Google Scholar 

  • Rajagopal, D., Cambria, E., Olsher, D., & Kwok, K. (2013). A graph-based approach to commonsense concept extraction and semantic similarity detection. In Proceedings of the 22nd International Conference on World Wide Web - WWW’13 Companion. [Online]. Available: https://doi.org/10.1145%2F2487788.2487995 (pp. 565–570). ACM Press.

  • Saeidi, M., Bouchard, G., Liakata, M., & Riedel, S. (2016). Sentihood: Targeted aspect based sentiment analysis dataset for urban neighbourhoods. In COLING (pp. 1546–1556). The COLING 2016 Organizing Committee.

  • Singh, L. G., & Singh, S. R. (2020). Empirical study of sentiment analysis tools and techniques on societal topics. Journal of Intelligent Information Systems, 56(2), 379–407. [Online]. Available: https://doi.org/10.1007/s10844-020-00616-7.

    Article  Google Scholar 

  • Song, M., Park, H., & shik Shin, K. (2019). Attention-based long short-term memory network using sentiment lexicon embedding for aspect-level sentiment analysis in korean. Information Processing & Management, 56(3), 637–653. [Online]. Available: https://doi.org/10.1016/j.ipm.2018.12.005.

    Article  Google Scholar 

  • Szlam, S. S. A., Weston, J., & Fergus, R. (2015). End-to-end memory networks. In NIPS’15: Proceedings of the 28th International Conference on Neural Information Processing Systems. [Online]. Available: https://doi.org/10.5555/2969442.2969512 (pp. 2440–2448).

  • Tai, K. S., Socher, R., & Manning, C. D. (2015). Improved semantic representations from tree-structured long short-term memory networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). [Online]. Available: https://doi.org/10.3115%2Fv1%2Fp15-1150 (pp. 1556–1566). Association for Computational Linguistics.

  • Tan, X., Cai, Y., Xu, J., Leung, H.-F., Chen, W., & Li, Q. (2020). Improving aspect-based sentiment analysis via aligning aspect embedding. Neurocomputing, 383, 336–347. [Online]. Available: https://doi.org/10.1016/j.neucom.2019.12.035.

    Article  Google Scholar 

  • Tang, D., Qin, B., Feng, X., & Liu, T. (2016). Effective LSTMs for target-dependent sentiment classification. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. [Online]. Available: https://www.aclweb.org/anthology/C16-1311 (pp. 3298–3307). The COLING 2016 Organizing Committee.

  • Tay, Y., Luu, A. T., & Hui, S. C. (2018). Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (pp. 5956–5963).

  • Vaswani, A., Shazeer, N. M., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems. [Online]. Available: https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1%c4a845aa-Paper.pdf (pp. 5999–6009). Curran Associates Inc.

  • Wang, Y., Huang, M., Zhu, X., & Zhao, L. (2016). Attention-based LSTM for aspect-level sentiment classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. [Online]. Available: https://doi.org/10.18653%2Fv1%2Fd16-1058 (pp. 606–615). Association for Computational Linguistics.

  • Xia, M., Li, T., Shu, T., Wan, J., de Silva, C. W., & Wang, Z. (2019). A two-stage approach for the remaining useful life prediction of bearings using deep neural networks. IEEE Transactions on Industrial Informatics, 15 (6), 3703–3711. [Online]. Available: https://doi.org/10.1109/tii.2018.2868687.

    Article  Google Scholar 

  • Xu, J., Chen, D., Qiu, X., & Huang, X. (2016). Cached long short-term memory neural networks for document-level sentiment classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. [Online]. Available: https://doi.org/10.18653%2Fv1%2Fd16-1172 (pp. 1660–1669). Association for Computational Linguistics.

  • Xu, Z., Liu, B., Wang, B., Sun, C., & Wang, X. (2017). Incorporating loose-structured knowledge into conversation modeling via recall-gate LSTM. In 2017 International Joint Conference on Neural Networks (IJCNN). [Online]. Available: https://doi.org/10.1109%2Fijcnn.2017.7966297 (pp. 3506–3513). IEEE.

  • Xue, W., & Li, T. (2018). Aspect based sentiment analysis with gated convolutional networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. [Online]. Available: https://www.aclweb.org/anthology/P18-1234 (pp. 2514–2523). Association for Computational Linguistics.

  • Xue, W., Zhou, W., Li, T., & Wang, Q. (2017). Mtna: A neural multi-task model for aspect category classification and aspect term extraction on restaurant reviews. In Proceedings of the Eighth International Joint Conference on Natural Language Processing. [Online]. Available: https://www.aclweb.org/anthology/I17-2026, (Vol. 2 pp. 151–156). Asian Federation of Natural Language Processing.

  • Zeng, J., Ma, X., & Zhou, K. (2019). Enhancing attention-based LSTM with position context for aspect-level sentiment classification. IEEE Access, 7, 20462–20471. [Online]. Available: https://doi.org/10.1109/access.2019.2893806.

    Article  Google Scholar 

  • Zhang, L., Wang, S., & Liu, B. (2018). Deep learning for sentiment analysis: A survey. WIREs Data Mining and Knowledge Discovery, 8(4), 1–25. [Online]. Available: https://doi.org/10.1002/widm.1253.

    Article  Google Scholar 

  • Zhou, X., Wan, X., & Xiao, J. (2016). Attention-based LSTM network for cross-lingual sentiment classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. [Online]. Available: https://doi.org/10.18653%2Fv1%2Fd16-1024 (pp. 247–256). Association for Computational Linguistics.

Download references

Acknowledgements

The authors would like to acknowledge the Department of Computer science, RMKCET for financial assistance and support. Also, we would like to thank all the reviewers and editors for their feedback and comments.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, R.M.K College of Engineering and Technology, Anna University, Chennai, India

    Srividhya Lakshmi Ramaswamy

  2. Department of Computer Science and Engineering, Sri Venkateswara College of Engineering, Anna University, Chennai, India

    Jayakumar Chinnappan

Authors
  1. Srividhya Lakshmi Ramaswamy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jayakumar Chinnappan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Srividhya Lakshmi Ramaswamy.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ramaswamy, S.L., Chinnappan, J. RecogNet-LSTM+CNN: a hybrid network with attention mechanism for aspect categorization and sentiment classification. J Intell Inf Syst 58, 379–404 (2022). https://doi.org/10.1007/s10844-021-00692-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10844-021-00692-3

Keywords

Search

Navigation