Fast Transfer Learning for Image Polarity Detection

  • Edoardo RagusaEmail author
  • Paolo Gastaldo
  • Rodolfo Zunino
Conference paper
Part of the Proceedings of the International Neural Networks Society book series (INNS, volume 1)


Convolutional neural networks (CNNs) provide an effective tool to extract complex information from images. In the area of image polarity detection, CNNs are utilized in combination with transfer learning to tackle a major problem: the unavailability of large sets of labeled data. Accordingly, polarity predictors in general exploit a pre-trained CNN that in turn feeds a classification layer. While the latter layer is trained from scratch, the pre-trained CNN is subject to fine tuning. In the actual implementation of such configuration, the specific CNN architecture indeed sets the performances of the predictor both in terms of generalization abilities and in terms of computational complexity. The latter attribute becomes critical when considering that polarity predictors -in the era of social network and custom profiles- might need to be updated within a short time interval (i.e., hours or even minutes). Thus, the paper proposes a design of experiment that supports a fair comparison between predictors that rely on different architectures.


Convolutional neural networks Transfer learning Image polarity 


  1. 1.
    Borth, D., Ji, R., Chen, T., Breuel, T., Chang, S.F.: Large-scale visual sentiment ontology and detectors using adjective noun pairs. In: Proceedings of the 21st ACM International Conference on Multimedia, pp. 223–232. ACM (2013)Google Scholar
  2. 2.
    Campos, V., Jou, B., Giro-i Nieto, X.: From pixels to sentiment: fine-tuning CNNs for visual sentiment prediction. Image Vis. Comput. 65, 15–22 (2017)CrossRefGoogle Scholar
  3. 3.
    Campos, V., Salvador, A., Giro-i Nieto, X., Jou, B.: Diving deep into sentiment: understanding fine-tuned CNNs for visual sentiment prediction. In: Proceedings of the 1st International Workshop on Affect & Sentiment in Multimedia, pp. 57–62. ACM (2015)Google Scholar
  4. 4.
    Canziani, A., Paszke, A., Culurciello, E.: An analysis of deep neural network models for practical applications. arXiv preprint arXiv:1605.07678 (2016)
  5. 5.
    Chen, T., Borth, D., Darrell, T., Chang, S.F.: DeepSentiBank: visual sentiment concept classification with deep convolutional neural networks. arXiv preprint arXiv:1410.8586 (2014)
  6. 6.
    Fan, S., Jiang, M., Shen, Z., Koenig, B.L., Kankanhalli, M.S., Zhao, Q.: The role of visual attention in sentiment prediction. In: Proceedings of the 2017 ACM on Multimedia Conference, pp. 217–225. ACM (2017)Google Scholar
  7. 7.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  8. 8.
    Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: CVPR, vol. 1, p. 3 (2017)Google Scholar
  9. 9.
    Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., Keutzer, K.: SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and \(<\)0.5 mb model size. arXiv preprint arXiv:1602.07360 (2016)
  10. 10.
    Islam, J., Zhang, Y.: Visual sentiment analysis for social images using transfer learning approach. In: 2016 IEEE International Conferences on BDCloud-SocialCom-SustainCom, pp. 124–130. IEEE (2016)Google Scholar
  11. 11.
    Jou, B., Chen, T., Pappas, N., Redi, M., Topkara, M., Chang, S.F.: Visual affect around the world: a large-scale multilingual visual sentiment ontology. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 159–168. ACM (2015)Google Scholar
  12. 12.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  13. 13.
    Luo, J., Borth, D., You, Q.: Social multimedia sentiment analysis. In: Proceedings of the 2017 ACM on Multimedia Conference, pp. 1953–1954. ACM (2017)Google Scholar
  14. 14.
    Niu, T., Zhu, S., Pang, L., El Saddik, A.: Sentiment analysis on multi-view social data. In: International Conference on Multimedia Modeling, pp. 15–27. Springer (2016)Google Scholar
  15. 15.
    Poria, S., Cambria, E., Bajpai, R., Hussain, A.: A review of affective computing: from unimodal analysis to multimodal fusion. Inf‘. Fusion 37, 98–125 (2017)CrossRefGoogle Scholar
  16. 16.
    Razavian, A.S., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: an astounding baseline for recognition. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 512–519. IEEE (2014)Google Scholar
  17. 17.
    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  19. 19.
    Soleymani, M., Garcia, D., Jou, B., Schuller, B., Chang, S.F., Pantic, M.: A survey of multimodal sentiment analysis. Image Vis. Comput. 65, 3–14 (2017)CrossRefGoogle Scholar
  20. 20.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A., et al.: Going deeper with convolutions. In: CVPR (2015)Google Scholar
  21. 21.
    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)Google Scholar
  22. 22.
    Xu, C., Cetintas, S., Lee, K.C., Li, L.J.: Visual sentiment prediction with deep convolutional neural networks. arXiv preprint arXiv:1411.5731 (2014)
  23. 23.
    Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp. 3320–3328 (2014)Google Scholar
  24. 24.
    You, Q., Luo, J., Jin, H., Yang, J.: Robust image sentiment analysis using progressively trained and domain transferred deep networks. In: AAAI, pp. 381–388 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Edoardo Ragusa
    • 1
    Email author
  • Paolo Gastaldo
    • 1
  • Rodolfo Zunino
    • 1
  1. 1.Department of Electrical, Electronic and Telecommunications Engineering, and Naval Architecture, DITENUniversity of GenoaGenoaItaly

Personalised recommendations