Advertisement

Multi-task Learning for Target-Dependent Sentiment Classification

  • Divam GuptaEmail author
  • Kushagra Singh
  • Soumen Chakrabarti
  • Tanmoy Chakraborty
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11439)

Abstract

Detecting and aggregating sentiments toward people, organizations, and events expressed in unstructured social media have become critical text mining operations. Early systems detected sentiments over whole passages, whereas more recently, target-specific sentiments have been of greater interest. In this paper, we present MTTDSC, a multi-task target-dependent sentiment classification system that is informed by feature representation learnt for the related auxiliary task of passage-level sentiment classification. The auxiliary task uses a gated recurrent unit (GRU) and pools GRU states, followed by an auxiliary fully-connected layer that outputs passage-level predictions. In the main task, these GRUs contribute auxiliary per-token representations over and above word embeddings. The main task has its own, separate GRUs. The auxiliary and main GRUs send their states to a different fully connected layer, trained for the main task. Extensive experiments using two auxiliary datasets and three benchmark datasets (of which one is new, introduced by us) for the main task demonstrate that MTTDSC outperforms state-of-the-art baselines. Using word-level sensitivity analysis, we present anecdotal evidence that prior systems can make incorrect target-specific predictions because they miss sentiments expressed by words independent of target.

Notes

Acknowledgement

The project was partially supported by IBM, Early Career Research Award (SERB, India), and the Center for AI, IIIT Delhi, India.

References

  1. 1.
    Baccianella, S., Esuli, A., Sebastiani, F.: Evaluation measures for ordinal regression. In: Intelligent Systems Design and Applications, Pisa, Italy, pp. 283–287 (2009)Google Scholar
  2. 2.
    Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: ICML, Montreal, Canada, pp. 41–48 (2009)Google Scholar
  3. 3.
    Choi, E., Hewlett, D., Uszkoreit, J., Polosukhin, I., Lacoste, A., Berant, J.: Coarse-to-fine question answering for long documents. In: ACL, pp. 209–220 (2017)Google Scholar
  4. 4.
    Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)
  5. 5.
    Dong, L., Wei, F., Tan, C., Tang, D., Zhou, M., Xu, K.: Adaptive recursive neural network for target-dependent Twitter sentiment classification. In: ACL, Baltimore, Maryland, USA, pp. 49–54 (2014)Google Scholar
  6. 6.
    Esuli, A., Sebastiani, F.: SentiWordNet: a publicly available lexical resource for opinion mining. In: LREC, pp. 417–422 (2006)Google Scholar
  7. 7.
    Go, A., Bhayani, R., Huang, L.: Twitter sentiment classification using distant supervision. CS224N Project Report, Stanford, vol. 1, no. 2009, p. 12 (2009)Google Scholar
  8. 8.
    Godbole, N., Srinivasaiah, M., Skiena, S.: Large-scale sentiment analysis for news and blogs. In: ICWSM, pp. 219–222 (2007)Google Scholar
  9. 9.
    Jiang, L., Yu, M., Zhou, M., Liu, X., Zhao, T.: Target-dependent twitter sentiment classification. In: ACL, Portland, Oregon, USA, pp. 151–160 (2011)Google Scholar
  10. 10.
    Joachims, T.: Optimizing search engines using clickthrough data. In: SIGKDD Conference, pp. 133–142. ACM (2002)Google Scholar
  11. 11.
    Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
  12. 12.
    Liu, B.: Sentiment analysis and opinion mining. Synth. Lect. Hum. Lang. Technol. 5(1), 1–167 (2012)CrossRefGoogle Scholar
  13. 13.
    Maurer, A., Pontil, M., Romera-Paredes, B.: The benefit of multitask representation learning. J. Mach. Learn. Res. 17(81), 1–32 (2016)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Pak, A., Paroubek, P.: Twitter as a corpus for sentiment analysis and opinion mining. In: LREC, Valletta, Malta, pp. 1–3 (2010)Google Scholar
  15. 15.
    Pang, B., Lee, L., et al.: Opinion mining and sentiment analysis. Found. Trends Inf. Retrieval 2(1–2), 1–135 (2008)CrossRefGoogle Scholar
  16. 16.
    Peng, H., Thomson, S., Smith, N.A.: Deep multitask learning for semantic dependency parsing. arXiv preprint arXiv:1704.06855 (2017)
  17. 17.
    Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: EMNLP Conference 2014, pp. 1532–1543 (2014)Google Scholar
  18. 18.
    Ruder, S., Bingel, J., Augenstein, I., Søgaard, A.: Learning what to share between loosely related tasks. arXiv preprint arXiv:1705.08142 (2017)
  19. 19.
    Sanders, N.: Twitter sentiment corpus (2011)Google Scholar
  20. 20.
    Tang, D., Qin, B., Feng, X., Liu, T.: Effective LSTMs for target-dependent sentiment classification. arXiv preprint arXiv:1512.01100 (2015)
  21. 21.
    Teng, Z., Vo, D.-T., Zhang, Y.: Context-sensitive lexicon features for neural sentiment analysis. In: EMNLP, Austin, Texas, USA, pp. 1629–1638 (2016)Google Scholar
  22. 22.
    Wang, B., Liakata, M., Zubiaga, A., Procter, R.: TDParse: multi-target-specific sentiment recognition on Twitter. In: EACL, pp. 483–493 (2017)Google Scholar
  23. 23.
    Wilson, T., Wiebe, J., Hoffmann, P.: Recognizing contextual polarity in phrase-level sentiment analysis. In: EMNLP, Vancouver, BC, Canada, pp. 347–354 (2005)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Divam Gupta
    • 1
    Email author
  • Kushagra Singh
    • 1
  • Soumen Chakrabarti
    • 2
  • Tanmoy Chakraborty
    • 1
  1. 1.IIIT DelhiNew DelhiIndia
  2. 2.IIT BombayMumbaiIndia

Personalised recommendations