Advertisement

An Efficient and Resource-Aware Hashtag Recommendation Using Deep Neural Networks

  • David Kao
  • Kuan-Ting LaiEmail author
  • Ming-Syan Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11440)

Abstract

The goal of this research is to design a system that can predict and recommend hashtags to users when new images are uploaded. The proposed hashtag recommendation system is called HAZEL (HAshtag ZEro-shot Learning). Selecting right hashtags can increase exposure and attract more fans on a social media platform. With the help of the state-of-the-art deep learning technologies such as Convolutional Neural Network (CNN), the recognition accuracy has improved significantly. However, hashtag prediction is still an open problem due to the large amount of media contents and hashtag categories. Using single machine learning method will not be sufficient. To address this issue, we combine image classification and semantic embedding models to achieve the expansion of recommended hashtags. In this research, we show that not all hashtags are equally meaningful, and some are not suitable in recommendation. In addition, by periodically updating semantic embedding model, we ensure that the hashtags being recommended follow the latest trends. Since the recommended hashtags have not received any training examples in the first place, it fulfills the concept of Zero-shot learning. We demonstrate that our system HAZEL can successfully recommend hashtags that are the most relevant to each image input by applying our design to a larger scale of image-hashtag pairs on Instagram.

Keywords

Hashtag recommendation Zero-shot learning Deep learning 

References

  1. 1.
    Wang, X., et al.: Zero-shot image classification based on deep feature extraction. IEEE Trans. Cogn. Dev. Syst. 10, 432–444 (2016)CrossRefGoogle Scholar
  2. 2.
    Changpinyo, S., et al.: Synthesized classifiers for zero-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  3. 3.
    Xian, Y., Schiele, B., Akata, Z.: Zero-shot learning-the good, the bad and the ugly. arXiv preprint arXiv:1703.04394 (2017)
  4. 4.
    Denton, E., et al.: User conditional hashtag prediction for images. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM (2015)Google Scholar
  5. 5.
    Weston, J., Bengio, S., Usunier, N.: WSABIE: scaling up to large vocabulary image annotation. In: IJCAI, vol. 11 (2011)Google Scholar
  6. 6.
    Frome, A., et al.: DeViSE: a deep visual-semantic embedding model. In: Advances in Neural Information Processing Systems (2013)Google Scholar
  7. 7.
    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10590-1_53CrossRefGoogle Scholar
  8. 8.
    Stutz, D.: Understanding convolutional neural networks. In: Seminar Report, Fakultät für Mathematik, Informatik und Naturwissenschaften Lehr-und Forschungsgebiet Informatik VIII Computer Vision (2014)Google Scholar
  9. 9.
    He, K., Sun, J.: Convolutional neural networks at constrained time cost. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  10. 10.
    He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  11. 11.
    Targ, S., Almeida, D., Lyman, K.: Resnet in resnet: generalizing residual architectures. arXiv preprint arXiv:1603.08029 (2016)
  12. 12.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  13. 13.
    Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  14. 14.
    He, K., et al.: Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: Proceedings of the IEEE International Conference on Computer Vision (2015)Google Scholar
  15. 15.
    Deng, J., et al.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009. IEEE (2009)Google Scholar
  16. 16.
    Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (2012)Google Scholar
  18. 18.
    Zhou, B., et al.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40, 1452–1464 (2017)CrossRefGoogle Scholar
  19. 19.
    Zhou, B., et al.: Places: an image database for deep scene understanding. arXiv preprint arXiv:1610.02055 (2016)
  20. 20.
    Zhou, B., et al.: Learning deep features for scene recognition using places database. In: Advances in Neural Information Processing Systems (2014)Google Scholar
  21. 21.
    Xiao, J., et al.: SUN database: large-scale scene recognition from abbey to zoo. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2010)Google Scholar
  22. 22.
    Everingham, M., et al.: The PASCAL visual object classes challenge: a retrospective. Int. J. Comput. Vis. 111(1), 98–136 (2015)CrossRefGoogle Scholar
  23. 23.
    Girshick, R., et al.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2014)Google Scholar
  24. 24.
    Mikolov, T., et al.: Efficient estimation of word representations in vector space. In: arXiv preprint arXiv:1301.3781 (2013)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.National Taiwan UniversityTaipeiTaiwan
  2. 2.National Taipei University of TechnologyTaipeiTaiwan

Personalised recommendations