Image Sentiment Analysis Using Convolutional Neural Network

  • Akshi Kumar
  • Arunima Jaiswal
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 736)


Visual media is one of the most powerful channel for expressing emotions and sentiments. Social media users are gradually using multimedia like images, videos etc. for expressing their opinions, views and experiences. Sentiment analysis of this vast user generated visual content can aid in better and improved extraction of user sentiments. This motivated us to focus on determining ‘image sentiment analyses’. Significant advancement has been made in this area, however, there is lot more to focus on visual sentiment analysis using deep learning techniques. In our study, we aim to design a visual sentiment framework using a convolutional neural network. For experimentation, we employ the use of Flickr images for training purposes and Twitter images for testing purposes. The results depict that the proposed ‘visual sentiment framework using convolutional neural network’ shows improved performance for analyzing the sentiments associated with the images.


Sentiment analysis Deep learning Convolutional neural network 


  1. 1.
    Pang, B., Lee, L.: Opinion mining and sentiment analysis. Found. Trends Inf. Retr. 2(1–2), 1–135 (2008)CrossRefGoogle Scholar
  2. 2.
    Liu, B.: Sentiment Analysis: Mining Opinions, Sentiments, and Emotions. Cambridge University Press, Chicago (2015)CrossRefGoogle Scholar
  3. 3.
    Kumar, A., Teeja, M.S.: Sentiment analysis: A perspective on its past, present and future. Int. J. Intell. Syst. Appl. 4(10), 1–14 (2012)Google Scholar
  4. 4.
    Esuli, A., Sebastiani, F.: SentiWordnet: a publicly available lexical resource for opinion mining. In: Proceedings of LREC (2006)Google Scholar
  5. 5.
    Thelwall, M., Buckley, K., Paltoglou, G., Cai, D., Kappas, A.: Sentiment strength detection in short informal text. J. Am. Soc. Inf. Sci. Technol. 62(2), 419–442 (2011)CrossRefGoogle Scholar
  6. 6.
    Vonikakis, V., Winkler, S.: Emotion-based sequence of family photos. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 1371–1372. ACM, Japan (2012)Google Scholar
  7. 7.
    Yanulevskaya, V., Uijlings, J., Bruni, E., Sartori, A., Zamboni, E., Bacci, F., Sebe, N.: In the eye of the beholder: employing statistical analysis and eye tracking for analyzing abstract paintings. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 349–358. ACM, Japan (2012)Google Scholar
  8. 8.
    Wang, X., Jia, J., Hu, P., Wu, S., Tang, J., Cai, L.: Understanding the emotional impact of images. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 1369–1370. ACM, Japan (2012)Google Scholar
  9. 9.
    Aradhye, H., Toderici, G., Yagnik, J.: Video2text: Learning to annotate video content. In: IEEE International Conference on Data Mining Workshops, ICDMW 2009, pp. 144–151. IEEE, USA (2009)Google Scholar
  10. 10.
    Borth, D., Ji, R., Chen, T., Breuel, T., Chang, S. F.: Large-scale visual sentiment ontology and detectors using adjective noun pairs. In: Proceedings of the 21st ACM International Conference on Multimedia, pp. 223–232. ACM, Spain (2013)Google Scholar
  11. 11.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp. 1097–1105. ACM, USA (2012)Google Scholar
  12. 12.
    Xu, C., Cetintas, S., Lee, K. C., Li, L. J.: Visual sentiment prediction with deep convolutional neural networks. arXiv preprint arXiv:1411.5731 (2014)
  13. 13.
    Jindal, S., Singh, S.: Image sentiment analysis using deep convolutional neural networks with domain specific fine tuning. In: 2015 International Conference on Information Processing (ICIP), pp. 447–451. IEEE, India (2015)Google Scholar
  14. 14.
    You, Q., Luo, J., Jin, H., Yang, J.: Robust image sentiment analysis using progressively trained and domain transferred deep networks. In: AAAI, pp. 381–388. ACM, USA (2015)Google Scholar
  15. 15.
    Cai, G., Xia, B.: Convolutional neural networks for multimedia sentiment analysis. In: Natural Language Processing and Chinese Computing, pp. 159–167. Springer, Cham (2015)Google Scholar
  16. 16.
    Islam, J., Zhang, Y.: Visual Sentiment Analysis for Social Images Using Transfer Learning Approach. In: IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom) (BDCloud-SocialCom-SustainCom), pp. 124–130. IEEE, USA (2016)Google Scholar
  17. 17.
    Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In: Computer Vision and Pattern Recognition, CVPR 2009, pp. 248–255. IEEE, USA (2009)Google Scholar
  18. 18.
    Zhang, Y., Er, M. J., Venkatesan, R., Wang, N., Pratama, M.: Sentiment classification using comprehensive attention recurrent models. In: Neural Networks (IJCNN), pp. 1562–1569. IEEE, Canada (2016)Google Scholar
  19. 19.
    Jiang, Y. G., Ye, G., Chang, S. F., Ellis, D., Loui, A. C.: Consumer video understanding: A benchmark database and an evaluation of human and machine performance. In: Proceedings of the 1st ACM International Conference on Multimedia Retrieval, p. 29. ACM, Italy (2011)Google Scholar
  20. 20.
    Ain, Q.T., Ali, M., Riaz, A., Noureen, A., Kamran, M., Hayat, B., Rehman, A.: Sentiment analysis using deep learning techniques: a review. Int. J. Adv. Comput. Sci. Appl. 8(6), 424–433 (2017)Google Scholar
  21. 21.
    Jindal, S., Singh, S.: Image sentiment analysis using deep convolutional neural networks with domain specific fine tuning. In: Information Processing (ICIP), pp. 447–451. IEEE, India (2015)Google Scholar
  22. 22.
    Kumar, A., Khorwal, R., Chaudhary, S.: A survey on sentiment analysis using swarm intelligence. Indian J. Sci. Technol. 9(39), 1–7 (2016)Google Scholar
  23. 23.
    Kumar, A., Sebastian, T.M.: Sentiment analysis on twitter. Int. J. Comput. Sci. Issues 9(4), 372–378 (2012)Google Scholar
  24. 24.
    Kumar, A., Sebastian, T. M.: Machine learning assisted sentiment analysis. In: Proceedings of International Conference on Computer Science & Engineering (ICCSE 2012), pp. 123–130. IAENG, UAE (2012)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Delhi Technological UniversityDelhiIndia

Personalised recommendations