Sentiment Analysis on Multi-View Social Data

  • Teng Niu
  • Shiai Zhu
  • Lei Pang
  • Abdulmotaleb El Saddik
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9517)


There is an increasing interest in understanding users’ attitude or sentiment towards a specific topic (e.g., a brand) from the large repository of opinion-rich data on the Web. While great efforts have been devoted on the single media, either text or image, little attempts are paid for the joint analysis of multi-view data which is becoming a prevalent form in the social media. For example, paired with a short textual message on Twitter, an image is attached. To prompt the research on this interesting and important problem, we introduce a multi-view sentiment analysis dataset (MVSA) including a set of image-text pairs with manual annotations collected from Twitter. The dataset can be utilized as a valuable benchmark for both single-view and multi-view sentiment analysis. With this dataset, many state-of-the-art approaches are evaluated. More importantly, the effectiveness of the correlation between different views is also studied using the widely used fusion strategies and an advanced multi-view feature extraction method. Results of these comprehensive experiments indicate that the performance can be boosted by jointly considering the textual and visual views.


Sentiment analysis Multi-View data Social media 


  1. 1.
    Bhattacharya, S., Nojavanasghari, B., Chen, T., Liu, D., Chang, S.F., Shah, M.: Towards a comprehensive computational model for aesthetic assessment of videos. In: ACM MM (2013)Google Scholar
  2. 2.
    Borth, D., Ji, R., Chen, T., Breuel, T., Chang, S.F.: Large-scale visual sentiment ontology and detectors using adjective noun pairs. In: ACM MM (2013)Google Scholar
  3. 3.
    Chen, T., Lu, D., Kan, M.Y., Cui, P.: Understanding and classifying image tweets. In: ACM MM (2013)Google Scholar
  4. 4.
    Chen, T., SalahEldeen, H.M., He, X., Kan, M.Y., Lu, D.: VELDA: Relating an image tweets text and images. In: AAAI (2015)Google Scholar
  5. 5.
    Go, A., Bhayani, R., Huang, L.: Twitter sentiment classification using distant supervision. Processing 150(12), 1–6 (2009)Google Scholar
  6. 6.
    Jiang, Y., Xu, B., Xue, X.: Predicting emotions in user-generated videos. In: AAAI (2014)Google Scholar
  7. 7.
    Li, L.J., Su, H., Xing, E.P., Li, F.F.: Object bank: a high-level image representation for scene classification and semantic feature sparsification. In: NIPS (2010)Google Scholar
  8. 8.
    Maas, A.L., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., Potts, C.: Learning word vectors for sentiment analysis. In: ACL (2011)Google Scholar
  9. 9.
    Pang, B., Lee, L.: Opinion mining and sentiment analysis. Found. Trends Inf. Retr. 2(1–2), 1–135 (2007)Google Scholar
  10. 10.
    Pang, L., Ngo, C.W.: Multimodal learning with deep boltzmann machine for emotion prediction in user generated videos. In: ICMR (2015)Google Scholar
  11. 11.
    Rosenthal, S., Nakov, P., Kiritchenko, S., Mohammad, S.M., Ritter, A., Stoyanov, V.: SemEval-2015 Task 10: sentiment analysis in twitter. In: SemEval 2015 Workshop (2015)Google Scholar
  12. 12.
    Saif, H., Fernandez, M., He, Y., Alani, H.: Evaluation datasets for twitter sentiment analysis: a survey and a new dataset, the STS-Gold. In: ESSEM Workshop (2013)Google Scholar
  13. 13.
    Speriosu, M., Sudan, N., Upadhyay, S., Baldridge, J.: Twitter polarity classification with label propagation over lexical links and the follower graph. In: EMNLP Workshop (2011)Google Scholar
  14. 14.
    Srivastava, N., Salakhutdinov, R.: Multimodal learning with deep boltzmann machines. J. Mach. Learn. Res. 15(1), 2949–2980 (2014)zbMATHMathSciNetGoogle Scholar
  15. 15.
    Torresani, L., Szummer, M., Fitzgibbon, A.: Efficient object category recognition using classemes. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 776–789. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  16. 16.
    Xie, W., Peng, Y., Xiao, J.: Cross-view feature learning for scalable social image analysis. In: AAAI (2014)Google Scholar
  17. 17.
    You, Q., Luo, J.: Towards social imagematics: sentiment analysis in social multimedia. In: MDMKDD (2013)Google Scholar
  18. 18.
    You, Q., Luo, J., Jin, H., Yang, J.: Robust image sentiment analysis using progressively trained and domain transferred deep networks. In: AAAI (2015)Google Scholar
  19. 19.
    Yu, F., Cao, L., Feris, R., Smith, J., Chang, S.F.: Designing category-level attributes for discriminative visual recognition. In: CVPR (2013)Google Scholar
  20. 20.
    Yuan, J., Mcdonough, S., You, Q., Luo, J.: Sentribute: image sentiment analysis from a mid-level perspective. In: WISDOM (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Teng Niu
    • 1
  • Shiai Zhu
    • 1
  • Lei Pang
    • 2
  • Abdulmotaleb El Saddik
    • 1
  1. 1.MCRLabUniversity of OttawaOttawaCanada
  2. 2.Department of Computer ScienceCity University of Hong KongKowloon TongHong Kong

Personalised recommendations