Advertisement

Towards Joint Multiply Semantics Hashing for Visual Search

  • Yunbo WangEmail author
  • Zhenan Sun
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11903)

Abstract

With the rapid growth of visual data on the web, deep hashing has shown enormous potential in preserving semantic similarity for visual search. Currently, most of the existing hashing methods employ pairwise or triplet-wise constraint to obtain the semantic similarity or relatively similarity among binary codes. However, some potential semantic context cannot be fully exploited, resulting in a suboptimal visual search. In this paper, we propose a novel deep hashing method, termed Joint Multiply Semantics Hashing (JMSH), to learn discriminative yet compact binary codes. In our approach, We jointly learn multiply semantic information to perform feature learning and hash coding. To be specific, the semantic information includes the pairwise semantic similarity between binary codes, the pointwise binary codes semantics and the pointwise visual feature semantics. Meanwhile, three different loss functions are designed to train the JMSH model. Extensive experiments show that the proposed JMSH yields state-of-the-art retrieval performance on representative image retrieval benchmarks.

Keywords

Deep hashing Binary codes Multiply semantics Visual search 

Notes

Acknowledgement

This work is supported by the National Natural Science Foundation of China (Grant No. U1836217, 61427811, 61573360) and the National Key Research and Development Program of China (Grant No. 2017YFC0821602, 2016YFB1001000).

References

  1. 1.
    Cao, Z., Long, M., Wang, J., Philip, S.Y.: Hashnet: deep learning to hash by continuation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5609–5618 (2017)Google Scholar
  2. 2.
    Cao, Z., Sun, Z., Long, M., Wang, J., Yu, P.S.: Deep priority hashing. In: Proceedings of the ACM Multimedia Conference on Multimedia, pp. 1653–1661. ACM (2018)Google Scholar
  3. 3.
    Gionis, A., Indyk, P., Motwani, R., et al.: Similarity search in high dimensions via hashing. In: Proceedings of International Conference on Very Large Data Bases, pp. 518–529 (1999)Google Scholar
  4. 4.
    Gong, Y., Lazebnik, S.: Iterative quantization: a procrustean approach to learning binary codes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 817–824 (2011)Google Scholar
  5. 5.
    Gong, Y., Lazebnik, S., Gordo, A., Perronnin, F.: Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 2916–2929 (2013)CrossRefGoogle Scholar
  6. 6.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  7. 7.
    Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012)
  8. 8.
    Jegou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 117–128 (2011)CrossRefGoogle Scholar
  9. 9.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  10. 10.
    Kulis, B., Grauman, K.: Kernelized locality-sensitive hashing. IEEE Trans. Pattern Anal. Mach. Intell. 34(6), 1092–1104 (2012)CrossRefGoogle Scholar
  11. 11.
    Lai, H., Pan, Y., Liu, Y., Yan, S.: Simultaneous feature learning and hash coding with deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3270–3278 (2015)Google Scholar
  12. 12.
    Li, Q., Sun, Z., He, R., Tan, T.: Deep supervised discrete hashing. In: Advances in Neural Information Processing Systems, pp. 2482–2491 (2017)Google Scholar
  13. 13.
    Li, W.J., Wang, S., Kang, W.C.: Feature learning based deep supervised hashing with pairwise labels. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pp. 1711–1717 (2016)Google Scholar
  14. 14.
    Lin, G., Shen, C., Shi, Q., Van den Hengel, A., Suter, D.: Fast supervised hashing with decision trees for high-dimensional data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1963–1970 (2014)Google Scholar
  15. 15.
    Lin, K., Lu, J., Chen, C.S., Zhou, J.: Learning compact binary descriptors with unsupervised deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1183–1192 (2016)Google Scholar
  16. 16.
    Liu, H., Wang, R., Shan, S., Chen, X.: Deep supervised hashing for fast image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2064–2072 (2016)Google Scholar
  17. 17.
    Liu, W., Mu, C., Kumar, S., Chang, S.F.: Discrete graph hashing. In: Advances in Neural Information Processing Systems, pp. 3419–3427 (2014)Google Scholar
  18. 18.
    Luo, X., Nie, L., He, X., Wu, Y., Chen, Z.D., Xu, X.S.: Fast scalable supervised hashing. In: The International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 735–744 (2018)Google Scholar
  19. 19.
    Norouzi, M., Blei, D.M.: Minimal loss hashing for compact binary codes. In: Proceedings of the International Conference on Machine Learning, pp. 353–360 (2011)Google Scholar
  20. 20.
    Raginsky, M., Lazebnik, S.: Locality-sensitive binary codes from shift-invariant kernels. In: Advances in Neural Information Processing Systems, pp. 1509–1517 (2009)Google Scholar
  21. 21.
    Shen, F., Gao, X., Liu, L., Yang, Y., Shen, H.T.: Deep asymmetric pairwise hashing. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 1522–1530. ACM (2017)Google Scholar
  22. 22.
    Shen, F., Shen, C., Liu, W., Tao Shen, H.: Supervised discrete hashing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 37–45 (2015)Google Scholar
  23. 23.
    Shen, F., Wei, L., Zhang, S., Yang, Y., Shen, H.T.: Learning binary codes for maximum inner product search. In: Proceedings of IEEE International Conference on Computer Vision, pp. 4148–4156. IEEE (2015)Google Scholar
  24. 24.
    Tang, J., Li, Z., Wang, M., Zhao, R.: Neighborhood discriminant hashing for large-scale image retrieval. IEEE Trans. Image Process. 24(9), 2827–2840 (2015)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Wang, J., Zhang, T., Sebe, N., Shen, H.T., et al.: A survey on learning to hash. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 769–790 (2018)CrossRefGoogle Scholar
  26. 26.
    Wang, Y., Liang, J., Cao, D., Sun, Z.: Local semantic-aware deep hashing with hamming-isometric quantization. IEEE Trans. Image Process. 28(6), 2665–2679 (2018)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Weiss, Y., Torralba, A., Fergus, R.: Spectral hashing. In: Advances in Neural Information Processing Systems, pp. 1753–1760 (2009)Google Scholar
  28. 28.
    Xia, R., Pan, Y., Lai, H., Liu, C., Yan, S.: Supervised hashing for image retrieval via image representation learning. In: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, vol. 1, pp. 2156–2162 (2014)Google Scholar
  29. 29.
    Yang, H.F., Lin, K., Chen, C.S.: Supervised learning of semantics-preserving hash via deep convolutional neural networks. IEEE Trans. Pattern Anal. Mach. Intell. 40(2), 437–451 (2018)CrossRefGoogle Scholar
  30. 30.
    Zhang, D., Wang, J., Cai, D., Lu, J.: Self-taught hashing for fast similarity search. In: Proceedings of the ACM SIGIR conference on Research and Development in Information Retrieval, pp. 18–25 (2010)Google Scholar
  31. 31.
    Zhang, P., Zhang, W., Li, W.J., Guo, M.: Supervised hashing with latent factor models. In: Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 173–182. ACM (2014)Google Scholar
  32. 32.
    Zhang, R., Lin, L., Zhang, R., Zuo, W., Zhang, L.: Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Trans. Image Process. 24(12), 4766–4779 (2015)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Zhu, H., Long, M., Wang, J., Cao, Y.: Deep hashing network for efficient similarity retrieval. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp. 2415–2421 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Institute of Automation, Chinese Academy of SciencesBeijingChina
  2. 2.University of Chinese Academy of SciencesBeijingChina

Personalised recommendations