Abstract
Hashing-based cross-modal retrieval maps multi-modal features into binary codes into a common Hamming space. Due to its small storage consumption and high efficiency, hashing has received extensive attention in recent years. However, the current researches have difficulty in constructing a well-defined joint semantic space and conduct more detailed and in-depth learning guidance. In this paper, Graph Attention Hashing via Contrastive Learning (GAHCL) is proposed to address these issues. First, we use the idea of contrastive learning to generate positive samples, and propose a novel contrastive adjacency matrix through a graph attention network. Specifically, this matrix assigns higher weights to node pairs whose source is the same sample, and assigns lower weights to node pairs that do not match each other. The key semantic features can be captured more carefully and accurately under the influence of attention weights. In addition, the contrastive loss function is constructed by taking the output features of different modalities in an instance and its generated positive sample features as a positive sample pair. Extensive experiments on two datasets show that the proposed method can significantly outperform existing competitors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Feng, F., Wang, X., Li, R.: Cross-modal retrieval with correspondence autoencoder. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 7–16 (2014)
Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. Adv. Neural Inf. Process. Syst. 30 (2017)
Hong, D., Gao, L., Yao, J., Zhang, B., Plaza, A., Chanussot, J.: Graph convolutional networks for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 59(7), 5966–5978 (2021). https://doi.org/10.1109/TGRS.2020.3015157
Hu, H., Xie, L., Hong, R., Tian, Q.: Creating something from nothing: unsupervised knowledge distillation for cross-modal hashing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3123–3132 (2020)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Li, M., Wang, H.: Unsupervised deep cross-modal hashing by knowledge distillation for large-scale cross-modal retrieval. In: Proceedings of the 2021 International Conference on Multimedia Retrieval, pp. 183–191 (2021)
Liu, H., Ji, R., Wu, Y., Huang, F., Zhang, B.: Cross-modality binary code learning via fusion similarity hashing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7380–7388 (2017)
Liu, S., Qian, S., Guan, Y., Zhan, J., Ying, L.: Joint-modal distribution-based similarity hashing for large-scale unsupervised deep cross-modal retrieval. In: Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pp. 1379–1388 (2020)
Oord, A.V.D., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)
Radford, A., et al.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning, pp. 8748–8763. PMLR (2021)
Shen, F., Shen, C., Shi, Q., Van Den Hengel, A., Tang, Z.: Inductive hashing on manifolds. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1562–1569 (2013)
Shi, G., Li, F., Wu, L., Chen, Y.: Object-level visual-text correlation graph hashing for unsupervised cross-modal retrieval. Sensors 22(8), 2921 (2022)
Su, S., Zhong, Z., Zhang, C.: Deep joint-semantics reconstructing hashing for large-scale unsupervised cross-modal retrieval. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3027–3035 (2019)
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. STAT 1050, 20 (2017)
Yang, D., Wu, D., Zhang, W., Zhang, H., Li, B., Wang, W.: Deep semantic-alignment hashing for unsupervised cross-modal retrieval. In: Proceedings of the 2020 International Conference on Multimedia Retrieval, pp. 44–52 (2020)
Yu, J., Zhou, H., Zhan, Y., Tao, D.: Deep graph-neighbor coherence preserving network for unsupervised cross-modal hashing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 4626–4634 (2021)
Zhang, J., Peng, Y., Yuan, M.: Unsupervised generative adversarial cross-modal hashing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Zhang, P.F., Li, Y., Huang, Z., Xu, X.S.: Aggregation-based graph convolutional hashing for unsupervised cross-modal retrieval. IEEE Trans. Multimedia 24, 466–479 (2021)
Zhao, Y., Yu, J., Liao, S., Zhang, Z., Zhang, H.: From sparse to dense: semantic graph evolutionary hashing for unsupervised cross-modal retrieval. In: Wang, L., Gall, J., Chin, T., Sato, I., Chellappa, R. (eds.) Computer Vision - ACCV 2022–16th Asian Conference on Computer Vision, Macao, China, 4–8 December 2022, Proceedings, Part IV. Lecture Notes in Computer Science, vol. 13844, pp. 521–536. Springer, Heidelbe (2022). https://doi.org/10.1007/978-3-031-26316-3_31
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yang, C., Ding, S., Li, L., Guo, J. (2024). Graph Attention Hashing via Contrastive Learning for Unsupervised Cross-Modal Retrieval. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1968. Springer, Singapore. https://doi.org/10.1007/978-981-99-8181-6_38
Download citation
DOI: https://doi.org/10.1007/978-981-99-8181-6_38
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8180-9
Online ISBN: 978-981-99-8181-6
eBook Packages: Computer ScienceComputer Science (R0)