Advertisement

A Graph-Based Approach to Explore Relationship Between Hashtags and Images

  • Zhiqiang Zhong
  • Yang Zhang
  • Jun PangEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11881)

Abstract

Online social networks are playing a great role in our daily life by providing a platform for users to present themselves, articulate their social circles, and interact with each other. Posting image is one of the most popular online activities, through which people could share experiences and express their emotions. Intuitively, there must exist a connection between images and their associated hashtags. In this paper, we focus on systematically describing this relationship and using it to improve downstream tasks. First, we use a two-sample Kolmogorov-Smirnov test on an Instagram dataset to show the existence of the relationship at a significance level of \(\alpha =0.001\). Second, in order to comprehensively explore the relationship and quantitatively analyse it, we adopt a graph-based approach, utilising the semantic information of hashtags and graph structure among images, to mine meaningful features for both hashtags and images. At last, we apply the extracted features about the relationship to improve an image multi-label classification task. Compared to a state-of-the-art method, we achieve a \(12.0\%\) overall precision gain.

Notes

Acknowledgements

This work is partially supported by the Luxembourg National Research Fund through grant PRIDE15/10621687/SPsquared.

References

  1. 1.
    Akata, Z., Reed, S., Walter, D., Lee, H., Schiele, B.: Evaluation of output embeddings for fine-grained image classification. In: Proceedings of the 2015 Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2927–2936. IEEE (2015)Google Scholar
  2. 2.
    Backes, M., Humbert, M., Pang, J., Zhang, Y.: walk2friends: Inferring social links from mobility profiles. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS), pp. 1943–1957. ACM (2017)Google Scholar
  3. 3.
    Cheng, R., Pang, J., Zhang, Y.: Inferring friendship from check-in data of location-based social networks. In: Proceedings of the 2015 Workshop on Social Network Analysis in Applications (SNAA), pp. 1284–1291. ACM (2015)Google Scholar
  4. 4.
    Chua, T.S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.: NUS-WIDE: a real-world web image database from National University of Singapore. In: Proceedings of the 2009 International Conference on Image and Video Retrieval (CIVR). ACM (2009)Google Scholar
  5. 5.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: Proceedings of the 2009 Conference on Computer Vision and Pattern Recognition (CVPR). pp. 248–255. IEEE (2009)Google Scholar
  6. 6.
    Denton, E., Weston, J., Paluri, M., Bourdev, L.D., Fergus, R.: User conditional hashtag prediction for images. In: Proceedings of the 2015 ACM Conference on Knowledge Discovery and Data Mining (KDD), pp. 1731–1740. ACM (2015)Google Scholar
  7. 7.
    Godin, F., Slavkovikj, V., Neve, W.D., Schrauwen, B., de Walle, R.V.: Using topic models for Twitter hashtag recommendation. In: Proceedings of the 2013 International Conference on World Wide Web (WWW), pp. 593–596. ACM (2013)Google Scholar
  8. 8.
    Hamilton, W.L., Ying, R., Leskovec, J.: Representation learning on graphs: methods and applications. IEEE Data Eng. Bull. 40, 52–74 (2017)Google Scholar
  9. 9.
    van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. In: Eurographics Conference on Visualization (EuroVis), pp. 2579–2605. Eurographics Association (2008)Google Scholar
  10. 10.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of the 2013 International Conference on Learning Representations (ICLR) (2013)Google Scholar
  11. 11.
    Miller, G.A.: WordNet: a lexical database for english. Commun. ACM 38(11), 39–41 (1995)CrossRefGoogle Scholar
  12. 12.
    Niu, Z., Hua, G., Gao, X., Tian, Q.: Semi-supervised relational topic model for weakly annotated image recognition in social media. In: Proceedings of the 2014 Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4233–4240. IEEE (2014)Google Scholar
  13. 13.
    Perozzi, B., Al-Rfou, R., Skiena, S.: DeepWalk: Online learning of social representations. In: Proceedings of the 2014 ACM Conference on Knowledge Discovery and Data Mining (KDD), pp. 701–710. ACM (2014)Google Scholar
  14. 14.
    Řehůřek, R., Sojka, P.: Software framework for topic modelling with large corpora. In: Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pp. 45–50. ELRA (2010)Google Scholar
  15. 15.
    Richards, D.R., Tunçer, B.: Using image recognition to automate assessment of cultural ecosystem services from social media photographs. Ecosyst. Serv. 31, 318–325 (2018)CrossRefGoogle Scholar
  16. 16.
    Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: A unified embedding for face recognition and clustering. In: Proceedings of the 2015 Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815–823. IEEE (2015)Google Scholar
  17. 17.
    She, J., Chen, L.: TOMOHA: topic model-based hashtag recommendation on twitter. In: Proceedings of the 2014 International Conference on World Wide Web (WWW), pp. 371–372. ACM (2014)Google Scholar
  18. 18.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of the 2018 International Conference on Learning Representations (ICLR) (2015)Google Scholar
  19. 19.
    Tsur, O., Rappoport, A.: What’s in a hashtag? content based prediction of the spread of ideas in microblogging communities. In: Proceedings of the 2012 ACM International Conference on Web Search and Data Mining (WSDM), pp. 643–652. ACM (2012)Google Scholar
  20. 20.
    Wang, J., Yang, Y., Mao, J., Huang, Z., Huang, C., Xu, W.: CNN-RNN: a unified framework for multi-label image classification. In: Proceedings of the 2016 Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2285–2294. IEEE (2016)Google Scholar
  21. 21.
    Wang, X., Wei, F., Liu, X., Zhou, M., Zhang, M.: Topic sentiment analysis in twitter: a graph-based hashtag sentiment classification approach. In: Proceedings of the 2011 ACM International Conference on Information and Knowledge Management (CIKM), pp. 1031–1040. ACM (2011)Google Scholar
  22. 22.
    Wang, X., Ye, Y., Gupta, A.: Zero-shot recognition via semantic embeddings and knowledge graphs. In: Proceedings of the 2018 Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6857–6866. IEEE (2018)Google Scholar
  23. 23.
    Wu, J., Yu, Y., Huang, C., Yu, K.: Deep multiple instance learning for image classification and auto-annotation. In: Proceedings of the 2015 Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3460–3469. IEEE (2015)Google Scholar
  24. 24.
    Zhang, Y.: Language in our time: an empirical analysis of hashtags. In: Proceedings of the 2019 International Conference on World Wide Web (WWW), pp. 2378–2389. ACM (2019)Google Scholar
  25. 25.
    Zhang, Y., Humbert, M., Rahman, T., Li, C.T., Pang, J., Backes, M.: Tagvisor: a privacy advisor for sharing hashtags. In: Proceedings of the 2018 Web Conference (WWW), pp. 287–296. ACM (2018)Google Scholar
  26. 26.
    Zhou, B., Lapedriza, À., Khosla, A., Oliva, A., Torralba, A.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1452–1464 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Faculty of Science, Technology and CommunicationUniversity of LuxembourgEsch-sur-AlzetteLuxembourg
  2. 2.CISPA Helmholtz Center for Information SecuritySaarland Informatics CampusSaarbrückenGermany
  3. 3.Interdisciplinary Centre for Security, Reliability and TrustUniversity of LuxembourgEsch-sur-AlzetteLuxembourg

Personalised recommendations