Multimedia Tools and Applications

, Volume 75, Issue 15, pp 8955–8968 | Cite as

Visual sentiment topic model based microblog image sentiment analysis

  • Donglin Cao
  • Rongrong Ji
  • Dazhen Lin
  • Shaozi Li


With a growing number of images being used to express opinions in Microblog, text based sentiment analysis is not enough to understand the sentiments of users. To obtain the sentiments implied in Microblog images, we propose a Visual Sentiment Topic Model (VSTM) which gathers images in the same Microblog topic to enhance the visual sentiment analysis results. First, we obtain the visual sentiment features by using Visual Sentiment Ontology (VSO); then, we build a Visual Sentiment Topic Model by using all images in the same topic; finally, we choose better visual sentiment features according to the visual sentiment features distribution in a topic. The best advantage of our approach is that the discriminative visual sentiment ontology features are selected according to the sentiment topic model. The experiment results show that the performance of our approach is better than VSO based model.


Visual sentiment topic model Visual sentiment ontology Sentiment analysis 



This work was supported by National Nature Science Foundation of China (No.61402386, No. 61305061 and No. 61202143), the Nature Science Foundation of Fujian Province (No. 2014 J01249 and No. 2011 J01367), Doctoral Program Foundation of Institutions of Higher Education of China (No.20090121110032), Shenzhen Science and Technology Research Foundation (No.JC200903180630A) and Special Fund for Developing Shenzhen’s Strategic Emerging Industries (No. JCYJ20120614164600201).


  1. 1.
    Alec Go, Richa Bhayani, Lei Huang (2009) Twitter Sentiment Classification using Distant SupervisionGoogle Scholar
  2. 2.
    Bing Li, Songhe Feng, Weihua Xiong and Weiming Hu (2012) Scaring or Pleasing: Exploit Emotional Impact of an Image. Proceedings of the 20th ACM international conference on Multimedia (MM), Pages: 1365–1366Google Scholar
  3. 3.
    Bing Li, Weihua Xiong, Weiming Hu, and Xinmiao Ding (2012) Context-aware affective images classification based on bilayer sparse representation, ACM MM, Pages: 721–724Google Scholar
  4. 4.
    Chen, Bi, Leilei Zhu, Daniel Kifer, and Dongwon Lee (2010) What is an sentiment about? exploring political standpoints using sentiment scoring model. in Proceeedings of AAAI Conference on Artificial Intelligence (AAAI-2010)Google Scholar
  5. 5.
    Damian Borth, Rongrong Ji, Tao Chen (2013) Thomas Breuel and Shih-Fu Chang. Large-scale Visual Sentiment Ontology and Detectors Using Adjective Noun Pairs. Proceedings of the 21th ACM international conference on Multimedia (MM), Pages: 223–232Google Scholar
  6. 6.
    Datta R, Joshi D, Li J, and Wang J (2006) Studying Aesthetics in Photographic Images using a Computational Approach. ECCV, 2006, Pages: 288–301Google Scholar
  7. 7.
    Feldman, Ronen, Benjamin Rosenfeld, Roy Bar-Haim, and Moshe Fresko (2011) The Stock Sonar - Sentiment Analysis of Stocks Based on a Hybrid Approach. in Proceedings of 23rd IAAI Conference on Artificial Intelligence (IAAI-2011)Google Scholar
  8. 8.
    Gao Y, Tang J, Hong R, Yan S, Dai Q, Zhang N, Chua T-S (2012) Camera constraint-free view-based 3D object retrieval. IEEE Trans Image Process 21(4):2269–2281MathSciNetCrossRefGoogle Scholar
  9. 9.
    Gao Y, Wang M, Zha Z, Shen J, Li X, Xindong W (2013) Visual-textual joint relevance learning for tag-based social image search. IEEE Trans Image Process 22(1):363–376MathSciNetCrossRefGoogle Scholar
  10. 10.
    Gao Y, Wang M, Zha Z, Tian Q, Dai Q, Zhang N (2011) Less is more: efficient 3D object retrieval with query view selection. IEEE Trans Multimedia 11(5):1007–1018CrossRefGoogle Scholar
  11. 11.
    Groh, Georg and Jan Hauffa (2011) Characterizing Social Relations Via NLPbased Sentiment Analysis. in Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media (ICWSM-2011)Google Scholar
  12. 12.
    Jia Jia, Sen Wu, Xiaohui Wang, Peiyun Hu, Lianhong Cai and Jie Tang (2012) Can we understand van Gogh’s Mood?: Learning to infer Affects from Images in Social Networks. Proceedings of the 20th ACM international conference on Multimedia (MM), Pages: 857–860Google Scholar
  13. 13.
    Joshi, Mahesh, Dipanjan Das, Kevin Gimpel, and Noah A. Smith (2010) Movie reviews and revenues: An experiment in text regression. in Proceedings of the North American Chapter of the Association for Computational Linguistics Human Language Technologies Conference (NAACL 2010)Google Scholar
  14. 14.
    Lanjun Zhou, Binyang Li, Wei Gao, Zhongyu Wei and Kam-Fai Wong (2011) Unsupervised discovery of discourse relations for eliminating intra-sentence polarity ambiguities. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Pages: 162–171Google Scholar
  15. 15.
    Liu, Jingjing, Yunbo Cao, Chin-Yew Lin, Yalou Huang, and Ming Zhou (2007) Low-quality product review detection in sentiment summarization. In Proceedings of the Joint Conferenceon Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL-2007)Google Scholar
  16. 16.
    Long Jiang and Mo Yu (2011) Target-dependent Twitter Sentiment Classification. In ACL, 2011, Pages: 151–160Google Scholar
  17. 17.
    Machajdik J, and Hanbury A (2010) Affective Image Classification using Features inspired by Psychology and Art Theory. ACM Multimedia, 2010, Pages: 83–92Google Scholar
  18. 18.
    Marchesotti L, Perronnin F, Larlus D, and Csurka G (2011) Assessing the Aesthetic Quality of Photographs using Generic Image Descriptors. ICCV, 2011, Pages: 1784–1791Google Scholar
  19. 19.
    McGlohon, Mary, Natalie Glance, and Zach Reiter (2010) Star quality: Aggregating reviews to rank products and merchants. in Proceedings of the International Conference on Weblogs and Social Media (ICWSM-2010)Google Scholar
  20. 20.
    Minqing Hu and Bing Liu (2004) Mining and summarizing customer reviews. Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), Pages: 168–177Google Scholar
  21. 21.
    Pang, Bo, Lillian Lee, and Shivakumar Vaithyanathan (2002) Thumbs up? sentiment classification using machine learning techniques. Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP), Pages: 79–86Google Scholar
  22. 22.
    Sicheng Zhao, Hongxun Yao, Fanglin Wang, Xiaolei Jiang, Wei Zhang (2014) Emotion Based Image Musicalizatioin. IEEE International Conference on Multimedia & Expo WorkshopsGoogle Scholar
  23. 23.
    Sicheng Zhao, Hongxun Yao, Xiaoshuai Sun, Pengfei Xu, Xianming Liu, and Rongrong Ji (2011) Video indexing and recommendation based on affective analysis of viewers, ACM MM, Pages: 1473–1476Google Scholar
  24. 24.
    Sicheng Zhao, Hongxun Yao, Xiaoshuai Sun, Xiaolei Jiang, Pengfei Xu (2013) Flexible presentation of videos based on affective content analysis, International Conference on Multimedia ModelingGoogle Scholar
  25. 25.
    Sicheng Zhao, Hongxun Yao, You Yang, Yanhao Zhang (2014) Affective Image Retrieval via Multi-Graph Learning. ACM International Conference on MultimediaGoogle Scholar
  26. 26.
    Sicheng Zhao, Yue Gao, Xiaolei Jiang, Hongxun Yao, Tat-Seng Chua, Xiaoshuai Sun (2014) Exploring Principles-of-Art Features For Image Emotion Recognition. ACM International Conference on MultimediaGoogle Scholar
  27. 27.
    Theresa Wilson, Janyce Wiebe and Paul Hoffmann (2005) Recognizing contextual polarity in phrase-level sentiment analysis. Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing (HLT), Pages: 347–354Google Scholar
  28. 28.
    Tumasjan, Andranik, Timm O. Sprenger, Philipp G. Sandner, and Isabell M. Welpe (2010) Predicting elections with twitter: What 140 characters reveal about political sentiment. in roceedings of the International Conference on Weblogs and Social Media (ICWSM-2010)Google Scholar
  29. 29.
    Vassilios Vonikakis and Stefan Winkler (2012) Emotion-based Sequence of Family Photos. Proceedings of the 20th ACM international conference on Multimedia (MM), Pages: 1371–1372Google Scholar
  30. 30.
    Wilson, Theresa, Janyce Wiebe, and Rebecca Hwa (2004) Just how mad are you? Finding strong and weak sentiment clauses. Proceedings of National Conference on Artificial Intelligence (AAAI), Pages: 761–767Google Scholar
  31. 31.
    Xianming Liu, Hongxun Yao, Rongrong Ji, Pengfei Xu, Xiaoshuai Sun and Qi Tian (2010) Visual Topic Model for Web Image Annotation. ICIMCS’10Google Scholar
  32. 32.
    Xin Lu, Poonam Suryanarayan, Reginald B. Adams, Jr., Jia Li, Michelle G. Newman, and James Z. Wang (2012) On shape and the computability of emotions, ACM MM, Pages: 229–238Google Scholar
  33. 33.
    Yano, Tae and Noah A. Smith (2010) What’s Worthy of Comment? Content and Comment Volume in Political Blogs. in Proceedings of the International AAAI Conference on Weblogs and Social Media (ICWSM 2010)Google Scholar
  34. 34.
    Yanulevskaya V, et al (2008) Emotional Valence Categorization using Holistic Image Features. ICIP, 2008, Pages: 101–104Google Scholar
  35. 35.
    Yanulevskaya V, et al (2012) In the Eye of the Beholder: Employing Statistical Analysis and Eye Tracking for Analyzing Abstract Paintings. ACM MM, 2012, Pages: 349–358Google Scholar
  36. 36.
    Yue Gao, Fanglin Wang, Huabo Luan, Tat-Seng Chua (2014) Brand Data Gathering From Live Social Media Streams, ACM Conference on Multimedia RetrievalGoogle Scholar
  37. 37.
    Zhao S, Yao H, Sun X (2013) Video classification and recommendation based on affective analysis of viewers. Neurocomputing 119:101–110CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Donglin Cao
    • 1
    • 2
  • Rongrong Ji
    • 1
    • 2
  • Dazhen Lin
    • 1
    • 2
  • Shaozi Li
    • 1
    • 2
  1. 1.Cognitive Science DepartmentXiamen UniversityXiamenChina
  2. 2.Fujian Key Laboratory of the Brain-like Intelligent SystemsXiamenChina

Personalised recommendations