Abstract
Online social networks are playing a great role in our daily life by providing a platform for users to present themselves, articulate their social circles, and interact with each other. Posting image is one of the most popular online activities, through which people could share experiences and express their emotions. Intuitively, there must exist a connection between images and their associated hashtags. In this paper, we focus on systematically describing this relationship and using it to improve downstream tasks. First, we use a two-sample Kolmogorov-Smirnov test on an Instagram dataset to show the existence of the relationship at a significance level of \(\alpha =0.001\). Second, in order to comprehensively explore the relationship and quantitatively analyse it, we adopt a graph-based approach, utilising the semantic information of hashtags and graph structure among images, to mine meaningful features for both hashtags and images. At last, we apply the extracted features about the relationship to improve an image multi-label classification task. Compared to a state-of-the-art method, we achieve a \(12.0\%\) overall precision gain.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
The dataset was collected in January 2016 when Instagram’s API was still publicly available.
- 3.
- 4.
This explains why we cannot directly use our Instagram dataset as we don’t have such ground truth.
References
Akata, Z., Reed, S., Walter, D., Lee, H., Schiele, B.: Evaluation of output embeddings for fine-grained image classification. In: Proceedings of the 2015 Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2927–2936. IEEE (2015)
Backes, M., Humbert, M., Pang, J., Zhang, Y.: walk2friends: Inferring social links from mobility profiles. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS), pp. 1943–1957. ACM (2017)
Cheng, R., Pang, J., Zhang, Y.: Inferring friendship from check-in data of location-based social networks. In: Proceedings of the 2015 Workshop on Social Network Analysis in Applications (SNAA), pp. 1284–1291. ACM (2015)
Chua, T.S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.: NUS-WIDE: a real-world web image database from National University of Singapore. In: Proceedings of the 2009 International Conference on Image and Video Retrieval (CIVR). ACM (2009)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: Proceedings of the 2009 Conference on Computer Vision and Pattern Recognition (CVPR). pp. 248–255. IEEE (2009)
Denton, E., Weston, J., Paluri, M., Bourdev, L.D., Fergus, R.: User conditional hashtag prediction for images. In: Proceedings of the 2015 ACM Conference on Knowledge Discovery and Data Mining (KDD), pp. 1731–1740. ACM (2015)
Godin, F., Slavkovikj, V., Neve, W.D., Schrauwen, B., de Walle, R.V.: Using topic models for Twitter hashtag recommendation. In: Proceedings of the 2013 International Conference on World Wide Web (WWW), pp. 593–596. ACM (2013)
Hamilton, W.L., Ying, R., Leskovec, J.: Representation learning on graphs: methods and applications. IEEE Data Eng. Bull. 40, 52–74 (2017)
van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. In: Eurographics Conference on Visualization (EuroVis), pp. 2579–2605. Eurographics Association (2008)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of the 2013 International Conference on Learning Representations (ICLR) (2013)
Miller, G.A.: WordNet: a lexical database for english. Commun. ACM 38(11), 39–41 (1995)
Niu, Z., Hua, G., Gao, X., Tian, Q.: Semi-supervised relational topic model for weakly annotated image recognition in social media. In: Proceedings of the 2014 Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4233–4240. IEEE (2014)
Perozzi, B., Al-Rfou, R., Skiena, S.: DeepWalk: Online learning of social representations. In: Proceedings of the 2014 ACM Conference on Knowledge Discovery and Data Mining (KDD), pp. 701–710. ACM (2014)
Řehůřek, R., Sojka, P.: Software framework for topic modelling with large corpora. In: Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pp. 45–50. ELRA (2010)
Richards, D.R., Tunçer, B.: Using image recognition to automate assessment of cultural ecosystem services from social media photographs. Ecosyst. Serv. 31, 318–325 (2018)
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: A unified embedding for face recognition and clustering. In: Proceedings of the 2015 Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815–823. IEEE (2015)
She, J., Chen, L.: TOMOHA: topic model-based hashtag recommendation on twitter. In: Proceedings of the 2014 International Conference on World Wide Web (WWW), pp. 371–372. ACM (2014)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of the 2018 International Conference on Learning Representations (ICLR) (2015)
Tsur, O., Rappoport, A.: What’s in a hashtag? content based prediction of the spread of ideas in microblogging communities. In: Proceedings of the 2012 ACM International Conference on Web Search and Data Mining (WSDM), pp. 643–652. ACM (2012)
Wang, J., Yang, Y., Mao, J., Huang, Z., Huang, C., Xu, W.: CNN-RNN: a unified framework for multi-label image classification. In: Proceedings of the 2016 Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2285–2294. IEEE (2016)
Wang, X., Wei, F., Liu, X., Zhou, M., Zhang, M.: Topic sentiment analysis in twitter: a graph-based hashtag sentiment classification approach. In: Proceedings of the 2011 ACM International Conference on Information and Knowledge Management (CIKM), pp. 1031–1040. ACM (2011)
Wang, X., Ye, Y., Gupta, A.: Zero-shot recognition via semantic embeddings and knowledge graphs. In: Proceedings of the 2018 Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6857–6866. IEEE (2018)
Wu, J., Yu, Y., Huang, C., Yu, K.: Deep multiple instance learning for image classification and auto-annotation. In: Proceedings of the 2015 Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3460–3469. IEEE (2015)
Zhang, Y.: Language in our time: an empirical analysis of hashtags. In: Proceedings of the 2019 International Conference on World Wide Web (WWW), pp. 2378–2389. ACM (2019)
Zhang, Y., Humbert, M., Rahman, T., Li, C.T., Pang, J., Backes, M.: Tagvisor: a privacy advisor for sharing hashtags. In: Proceedings of the 2018 Web Conference (WWW), pp. 287–296. ACM (2018)
Zhou, B., Lapedriza, À., Khosla, A., Oliva, A., Torralba, A.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1452–1464 (2018)
Acknowledgements
This work is partially supported by the Luxembourg National Research Fund through grant PRIDE15/10621687/SPsquared.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhong, Z., Zhang, Y., Pang, J. (2019). A Graph-Based Approach to Explore Relationship Between Hashtags and Images. In: Cheng, R., Mamoulis, N., Sun, Y., Huang, X. (eds) Web Information Systems Engineering – WISE 2019. WISE 2020. Lecture Notes in Computer Science(), vol 11881. Springer, Cham. https://doi.org/10.1007/978-3-030-34223-4_30
Download citation
DOI: https://doi.org/10.1007/978-3-030-34223-4_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-34222-7
Online ISBN: 978-3-030-34223-4
eBook Packages: Computer ScienceComputer Science (R0)