Skip to main content
SpringerLink
Log in

Online Discriminative Semantic-Preserving Hashing for Large-Scale Cross-Modal Retrieval

  • Conference paper
  • First Online:
PRICAI 2021: Trends in Artificial Intelligence (PRICAI 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13031))

Included in the following conference series:

Abstract

Cross-modal hashing has drawn increasing attentions for efficient retrieval across different modalities, and existing methods primarily learn the hash functions in a batch based mode, i.e., offline methods. Nevertheless, the multimedia data often comes in a streaming fashion, which makes the batch based learning methods uncompetitive for large-scale streaming data due to the large memory consumption and calculation. To address this problem, we present an Online Discriminative Semantic-Preserving Hashing (ODSPH) method for large-scale cross-modal retrieval, featuring on fast training speed, low memory consumption and high retrieval accuracy. Within the proposed ODSPH framework, we utilize the newly coming data points to learn the hash codes and update hash functions in a stream manner. When new data comes, the corresponding hash codes are obtained by regressing the class label of the training examples. For hash function, we update it with the accumulated information from each round. Besides, we design a novel momentum updating method to adaptively update the hash function and reduce quantization loss, which can produce discriminative hash codes for high retrieval precision. Extensive experiments on three benchmark datasets show that the proposed ODSPH method improves the retrieval performance over the state-of-the-arts.

Supported by the National Science Foundation of Fujian Province (Nos. 2020J01083 and 2020J01084), and Program (JYB202102) of Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, Nanjing University of Science and Technology.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cao, Y., Long, M., Wang, J., Zhu, H.: Correlation autoencoder hashing for supervised cross-modal search. In: Proceedings of the ACM on International Conference on Multimedia Retrieval, pp. 197–204 (2016)

    Google Scholar 

  2. Shen, H.T., et al.: Exploiting subspace relation in semantic labels for cross-modal hashing. IEEE Transactions on Knowledge and Data Engineering (2020)

    Google Scholar 

  3. Ma, X., Zhang, T., Xu, C.: Multi-level correlation adversarial hashing for cross-modal retrieval. IEEE Trans. Multimed. 22(12), 3101–3114 (2020)

    Article  Google Scholar 

  4. Rastegari, M., Choi, J., Fakhraei, S., Hal, D., Davis, L.: Predictable dual-view hashing. In: Proceedings of the International Conference on Machine Learning, pp. 1328–1336 (2013)

    Google Scholar 

  5. Wei, Y., Song, Y., Zhen, Y., Liu, B., Yang, Q.: Heterogeneous translated hashing: a scalable solution towards multi-modal similarity search. ACM Trans. Knowl. Discov. Data 10(4), 36 (2016)

    Article  Google Scholar 

  6. Zhang, L., Zhang, Y., Hong, R., Tian, Q.: Full-space local topology extraction for cross-modal retrieval. IEEE Trans. Image Process. 24(7), 2212–2224 (2015)

    Article  MathSciNet  Google Scholar 

  7. Bronstein, M.M., Bronstein, A.M., Michel, F., Paragios, N.: Data fusion through cross-modality metric learning using similarity-sensitive hashing. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3594–3601 (2010)

    Google Scholar 

  8. Jiang, Q.Y., Li, W.J.: Deep cross-modal hashing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3232–3240 (2017)

    Google Scholar 

  9. Yan, T.K., Xu, X.S., Guo, S., Huang, Z., Wang, X.L.: Supervised robust discrete multimodal hashing for cross-media retrieval. In: Proceedings of the ACM International on Conference on Information and Knowledge Management, pp. 1271–1280 (2016)

    Google Scholar 

  10. Zhang, D., Li, W.J.: Large-scale supervised multimodal hashing with semantic correlation maximization. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 2177–2183 (2014)

    Google Scholar 

  11. Lin, Z., Ding, G., Hu, M., Wang, J.: Semantics-preserving hashing for cross-view retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3864–3872 (2015)

    Google Scholar 

  12. Mandal, D., Chaudhury, K.N., Biswas, S.: Generalized semantic preserving hashing for n-label cross-modal retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4076–4084 (2017)

    Google Scholar 

  13. Liu, H., Ji, R., Wu, Y., Huang, F., Zhang, B.: Cross-modality binary code learning via fusion similarity hashing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7380–7388 (2017)

    Google Scholar 

  14. Xu, X., Shen, F., Yang, Y., Shen, H.T., Li, X.: Learning discriminative binary codes for large-scale cross-modal retrieval. IEEE Trans. Image Process. 26(5), 2494–2507 (2017)

    Article  MathSciNet  Google Scholar 

  15. Liu, X., Hu, Z., Ling, H., Cheung, Y.: Mtfh: a matrix tri-factorization hashing framework for efficient cross-modal retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 43(3), 964–981 (2021)

    Article  Google Scholar 

  16. Xie, L., Shen, J., Zhu, L.: Online cross-modal hashing for web image retrieval. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 294–300 (2016)

    Google Scholar 

  17. Yao, T., Wang, G., Yan, L., Kong, X., Su, Q., Zhang, C., Tian, Q.: Online latent semantic hashing for cross-media retrieval. Pattern Recogn. 89, 1–11 (2019)

    Article  Google Scholar 

  18. Ding, G., Guo, Y., Zhou, J.: Collective matrix factorization hashing for multimodal data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2075–2082 (2014)

    Google Scholar 

  19. Song, J.K., Yang, Y., Yang, Y., Huang, Z., Shen, H.T.: Inter-media hashing for large-scale retrieval from heterogeneous data sources. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, pp. 785–796 (2013)

    Google Scholar 

  20. Tang, J., Wang, K., Shao, L.: Supervised matrix factorization hashing for cross-modal retrieval. IEEE Trans. Image Process. 25(7), 3157–3166 (2016)

    Article  MathSciNet  Google Scholar 

  21. Wang, T., Zhu, L., Cheng, Z., Li, J., Gao, Z.: Unsupervised deep cross-modal hashing with virtual label regression. Neurocomputing 386, 84–96 (2020)

    Article  Google Scholar 

  22. Lin, Q., Cao, W., He, Z., He, Z.: Semantic deep cross-modal hashing. Neurocomputing 396, 113–122 (2020)

    Article  Google Scholar 

  23. Huang, L.K., Yang, Q., Zheng, W.S.: Online hashing. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 1422–1428 (2013)

    Google Scholar 

  24. Jain, P., Kulis, B., Dhillon, I.S., Grauman, K.: Online metric learning and fast similarity search. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 761–768 (2009)

    Google Scholar 

  25. Leng, C., Wu, J., Cheng, J., Bai, X., Lu, H.: Online sketching hashing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2503–2511 (2015)

    Google Scholar 

  26. Cakir, F., He, K., Adel Bargal, S., Sclaroff, S.: Mihash: online hashing with mutual information. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 437–445 (2017)

    Google Scholar 

  27. Wang, D., Wang, Q., An, Y.Q., Gao, X.B., Tian, Y.M.: Online collective matrix factorization hashing for large-scale cross-media retrieval. In: Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1409–1418 (2020)

    Google Scholar 

  28. Gui, J., Liu, T., Sun, Z., Tao, D., Tan, T.: Fast supervised discrete hashing. IEEE Trans. Pattern Anal. Mach. Intell. 40(2), 490–496 (2018)

    Article  Google Scholar 

  29. Huiskes, M.J., Lew, M.S.: The mir flickr retrieval evaluation. In: Proceedings of ACM International Conference on Multimedia Information Retrieval, pp. 39–43 (2008)

    Google Scholar 

  30. Chua, T., Tang, J.H., Hong, R.C., Li, H.J., Luo, Z.P., Tao, Z.Y.: Nus-wide: a real-world web image database from national university of singapore. In: Proceedings of ACM International Conference on Image and Video Retrieval, pp. 1–9 (2009)

    Google Scholar 

  31. Rasiwasia, N., Costa Pereira, J., Coviello, E., Doyle, G., Lanckriet, G.R., Levy, R., Vasconcelos, N.: A new approach to cross-modal multimedia retrieval. In: Proceedings of the ACM International Conference on Multimedia, pp. 251–260 (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Huaqiao University, Xiamen, 361021, China

    Jinhan Yi, Yi He & Xin Liu

  2. Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, Nanjing University of Science and Technology, Nanjing, 210094, People’s Republic of China

    Jinhan Yi & Xin Liu

  3. Xiamen Key Laboratory of Computer Vision and Pattern Recognition, Xiamen, China

    Yi He & Xin Liu

  4. Fujian Key Laboratory of Big Data Intelligence and Security, Xiamen, China

    Jinhan Yi & Yi He

Authors
  1. Jinhan Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xin Liu .

Editor information

Editors and Affiliations

  1. MIMOS Berhad, Kuala Lumpur, Malaysia

    Duc Nghia Pham

  2. Sirindhorn International Institute of Science and Technology, Thammasat University, Mueang Pathum Thani, Thailand

    Thanaruk Theeramunkong

  3. Data61, CSIRO, Brisbane, QLD, Australia

    Guido Governatori

  4. Department of Philosophy, Tsinghua University, Beijing, China

    Fenrong Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yi, J., He, Y., Liu, X. (2021). Online Discriminative Semantic-Preserving Hashing for Large-Scale Cross-Modal Retrieval. In: Pham, D.N., Theeramunkong, T., Governatori, G., Liu, F. (eds) PRICAI 2021: Trends in Artificial Intelligence. PRICAI 2021. Lecture Notes in Computer Science(), vol 13031. Springer, Cham. https://doi.org/10.1007/978-3-030-89188-6_33

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-89188-6_33

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-89187-9

  • Online ISBN: 978-3-030-89188-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation