Advertisement

Gated Convolutional Encoder-Decoder for Semi-supervised Affect Prediction

  • Kushal ChawlaEmail author
  • Sopan Khosla
  • Niyati Chhaya
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11439)

Abstract

Analyzing human reactions from text is an important step towards automated modeling of affective content. The variance in human perceptions and experiences leads to a lack of uniform, well-labeled, ground-truth datasets, hence, limiting the scope of neural supervised learning approaches. Recurrent and convolutional networks are popular for text classification and generation tasks, specifically, where large datasets are available; but are inefficient when dealing with unlabeled corpora. We propose a gated sequence-to-sequence, convolutional-deconvolutional autoencoding (GCNN-DCNN) framework for affect classification with limited labeled data. We show that compared to a vanilla CNN-DCNN network, gated networks improve performance for affect prediction as well as text reconstruction. We present a regression analysis comparing outputs of traditional learning models with information captured by hidden variables in the proposed network. Quantitative evaluation with joint, pre-trained networks, augmented with psycholinguistic features, reports highest accuracies for affect prediction, namely frustration, formality, and politeness in text.

References

  1. 1.
    Burgoon, J.K., Hale, J.L.: The fundamental topoi of relational communication. Commun. Monogr. 51(3), 193–214 (1984).  https://doi.org/10.1080/03637758409390195CrossRefGoogle Scholar
  2. 2.
    Chung, J., Gülçehre, Ç., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR abs/1412.3555 (2014)Google Scholar
  3. 3.
    Cohen, W.W.: Enron email dataset (2009)Google Scholar
  4. 4.
    Dai, A.M., Le, Q.V.: Semi-supervised sequence learning. In: Advances in Neural Information Processing Systems, pp. 3079–3087 (2015)Google Scholar
  5. 5.
    Danescu-Niculescu-Mizil, C., Sudhof, M., Jurafsky, D., Leskovec, J., Potts, C.: A computational approach to politeness with application to social factors. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (2013)Google Scholar
  6. 6.
    Dauphin, Y.N., Fan, A., Auli, M., Grangier, D.: Language modeling with gated convolutional networks. CoRR abs/1612.08083 (2016)Google Scholar
  7. 7.
    Dieng, A.B., Wang, C., Gao, J., Paisley, J.W.: TopicRNN: a recurrent neural network with long-range semantic dependency. CoRR abs/1611.01702 (2016)Google Scholar
  8. 8.
    Ghosh, S., Chollet, M., Laksana, E., Morency, L.P., Scherer, S.: Affect-LM: a neural language model for customizable affective text generation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (2017)Google Scholar
  9. 9.
    Kalchbrenner, N., Espeholt, L., Simonyan, K., van den Oord, A., Graves, A., Kavukcuoglu, K.: Neural machine translation in linear time. arXiv preprint arXiv:1610.10099 (2016)
  10. 10.
    Ke, Y., Hagiwara, M.: Alleviating overfitting for polysemous words for word representation estimation using lexicons. In: International Joint Conference on Neural Networks (2017)Google Scholar
  11. 11.
    Khosla, S., Chhaya, N., Chawla, K.: Aff2Vec: affect-enriched distributional word representations. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 2204–2218. Association for Computational Linguistics (2018)Google Scholar
  12. 12.
    Lee, D.H.: Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on Challenges in Representation Learning, ICML (2013)Google Scholar
  13. 13.
    Li, J., Luong, M., Jurafsky, D.: A hierarchical neural autoencoder for paragraphs and documents. CoRR abs/1506.01057 (2015)Google Scholar
  14. 14.
    Lin, C.Y.: Rouge: a package for automatic evaluation of summaries. In: Proceedings of ACL Workshop on Text Summarization Branches Out (2004)Google Scholar
  15. 15.
    Mairesse, F., Walker, M.A.: Trainable generation of big-five personality styles through data-driven parameter estimation. In: ACL, pp. 165–173 (2008)Google Scholar
  16. 16.
    Majumder, N., Poria, S., Gelbukh, A., Cambria, E.: Deep learning-based document modeling for personality detection from text. IEEE Intell. Syst. 32(2), 74–79 (2017)CrossRefGoogle Scholar
  17. 17.
    Mikolov, T., Karafiát, M., Burget, L., Černocký, J., Khudanpur, S.: Recurrent neural network based language model. In: Proceedings of the 11th Annual Conference of the International Speech Communication Association, vol. 2010, pp. 1045–1048 (2010)Google Scholar
  18. 18.
    Van den Oord, A., Kalchbrenner, N., Espeholt, L., Vinyals, O., Graves, A., et al.: Conditional image generation with pixelCNN decoders. In: NIPS (2016)Google Scholar
  19. 19.
    Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL 2002, pp. 311–318 (2002)Google Scholar
  20. 20.
    Pavlick, E., Tetreault, J.: An empirical analysis of formality in online communication. Trans. Assoc. Comput. Linguist. 4, 61–74 (2016)CrossRefGoogle Scholar
  21. 21.
    Pennebaker, J.W.: The secret life of pronouns. New Scientist 211(2828), 42–45 (2011)CrossRefGoogle Scholar
  22. 22.
    Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)Google Scholar
  23. 23.
    Preotiuc-Pietro, D., Liu, Y., Hopkins, D.J., Ungar, L.: Personality driven differences in paraphrase preference. In: Proceedings of the Workshop on Natural Language Processing and Computational Social Science (NLP+CSS). ACL (2017)Google Scholar
  24. 24.
    Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)
  25. 25.
    dos Santos, C., Gatti, M.: Deep convolutional neural networks for sentiment analysis of short texts. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp. 69–78 (2014)Google Scholar
  26. 26.
    Severyn, A., Moschitti, A.: Twitter sentiment analysis with deep convolutional neural networks. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2015, pp. 959–962 (2015)Google Scholar
  27. 27.
    Shen, Y., Lin, Z., Huang, C., Courville, A.C.: Neural language modeling by jointly learning syntax and lexicon. CoRR abs/1711.02013 (2017)Google Scholar
  28. 28.
    Subramanian, S., Baldwin, T., Cohn, T.: Content-based popularity prediction of online petitions using a deep regression model. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 182–188 (2018)Google Scholar
  29. 29.
    Titov, I., Klementiev, A.: Semi-supervised semantic role labeling: approaching from an unsupervised perspective. In: Proceedings of COLING 2012, pp. 2635–2652 (2012)Google Scholar
  30. 30.
    Yin, W., Kann, K., Yu, M., Schütze, H.: Comparative study of CNN and RNN for natural language processing. CoRR abs/1702.01923 (2017)Google Scholar
  31. 31.
    Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. In: Advances in Neural Information Processing Systems, pp. 649–657 (2015)Google Scholar
  32. 32.
    Zhang, Y., Shen, D., Wang, G., Gan, Z., Henao, R., Carin, L.: Deconvolutional paragraph representation learning. CoRR abs/1708.04729 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Big Data Experience LabAdobe ResearchBangaloreIndia

Personalised recommendations