Skip to main content
SpringerLink
Log in

Deep Multi-label Hashing for Large-Scale Visual Search Based on Semantic Graph

  • Conference paper
  • First Online:
Web and Big Data (APWeb-WAIM 2017)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10366))

Abstract

Huge volumes of images are aggregated over time because many people upload their favorite images to various social websites such as Flickr and share them with their friends. Accordingly, visual search from large scale image databases is getting more and more important. Hashing is an efficient technique to large-scale visual content search, and learning-based hashing approaches have achieved great success due to recent advancements of deep learning. However, most existing deep hashing methods focus on single label images, where hash codes cannot well preserve semantic similarity of images. In this paper, we propose a novel framework, deep multi-label hashing (DMLH) based on a semantic graph, which consists of three key components: (i) Image labels, semantically similar in terms of co-occurrence relationship, are classified in such a way that similar labels are in the same cluster. This helps to provide accurate ground truth for hash learning. (ii) A deep model is trained to simultaneously generate hash code and feature vector of images, based on which multi-label image databases are organized by hash tables. This model has excellent capability in improving retrieval speed meanwhile preserving semantic similarity among images. (iii) A combination of hash code based coarse search and feature vector based fine image ranking is used to provide an efficient and accurate retrieval. Extensive experiments over several large image datasets confirm that the proposed DMLH method outperforms state-of-the-art supervised and unsupervised image retrieval approaches, with a gain ranging from 6.25% to 38.9% in terms of mean average precision.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Norouzi, M., Blei, D.M.: Minimal loss hashing for compact binary codes. In: The 28th International Conference on Machine Learning (ICML 2011), pp. 353–360 (2011)

    Google Scholar 

  2. Liu, W., Wang, J., Ji, R., Jiang, Y.-G., Chang, S.-F.: Supervised hashing with kernels. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2074–2081. IEEE (2012)

    Google Scholar 

  3. Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1891–1898 (2014)

    Google Scholar 

  4. Wang, J., Kumar, S., Chang, S.-F.: Semi-supervised hashing for scalable image retrieval. In: 2010 IEEE Conference on CVPR, pp. 3424–3431. IEEE (2010)

    Google Scholar 

  5. Liu, W., Wang, J., Kumar, S., Chang, S.-F.: Hashing with graphs. In: Proceedings of the 28th International Conference on Machine Learning (ICML 2011) (2011)

    Google Scholar 

  6. Gong, Y., Lazebnik, S.: Iterative quantization: a procrustean approach to learning binary codes. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 817–824. IEEE (2011)

    Google Scholar 

  7. Li, W.-J., Wang, S., Kang, W.-C.: Feature learning based deep supervised hashing with pairwise labels. arXiv preprint arXiv:1511.03855 (2015)

  8. Zheng, Y., Guo, Q., Tung, A.K., Wu, S.: LazyLSH: approximate nearest neighbor search for multiple distance functions with a single index. In: Proceedings of the 2016 International Conference on Management of Data. ACM (2016)

    Google Scholar 

  9. Weiss, Y., Torralba, A., Fergus, R.: Spectral hashing. In: Advances in Neural Information Processing Systems, pp. 1753–1760 (2009)

    Google Scholar 

  10. He, K., Wen, F., Sun, J.: K-means hashing: an affinity-preserving quantization method for learning binary compact codes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2938–2945 (2013)

    Google Scholar 

  11. Wan, J., Wang, D., Hoi, S.C.H., Wu, P., Zhu, J., Zhang, Y., Li, J.: Deep learning for content-based image retrieval: a comprehensive study. In: Proceedings of the 22nd ACM international conference on Multimedia, pp. 157–166. ACM (2014)

    Google Scholar 

  12. Lai, H., Pan, Y., Liu, Y., Yan, S.: Simultaneous feature learning and hash coding with deep neural networks. in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3270–3278 (2015)

    Google Scholar 

  13. Erin Liong, V., Lu, J., Wang, G., Moulin, P., Zhou, J.: Deep hashing for compact binary codes learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2475–2483 (2015)

    Google Scholar 

  14. Wang, J., Song, Y., Leung, T.: Learning fine-grained image similarity with deep ranking. In: Proceedings of the IEEE Conference on CVPR, pp. 1386–1393 (2014)

    Google Scholar 

  15. Norouzi, M., Fleet, D.J., Salakhutdinov, R.R.: Hamming distance metric learning. In: Advances in NIPS, pp. 1061–1069 (2012)

    Google Scholar 

  16. Zhang, P., Zhang, W., Li, W.-J., Guo, M.: Supervised hashing with latent factor models. In: Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 173–182. ACM (2014)

    Google Scholar 

  17. Norouzi, M., Punjani, A., Fleet, D.J.: Fast search in hamming space with multi-index hashing. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp. 3108–3115 (2012)

    Google Scholar 

  18. Lin, G., Shen, C., Shi, Q., van den Hengel, A., Suter, D.: Fast supervised hashing with decision trees for high-dimensional data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1963–1970 (2014)

    Google Scholar 

  19. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  20. Gao, J., Jagadish, H.V., Lu, W., Ooi, B.C.: DSH: data sensitive hashing for high-dimensional k-NNsearch. In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data. ACM, pp. 1127–1138 (2014)

    Google Scholar 

  21. Babenko, A., Slesarev, A., Chigorin, A., Lempitsky, V.: Neural codes for image retrieval. In: European Conference on Computer Vision, pp. 584–599 (2014)

    Google Scholar 

  22. Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: OverFeat: integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229 (2013)

  23. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

    Google Scholar 

  24. Zhao, F., Huang, Y., Wang, L., Tan, T.: Deep semantic ranking based hashing for multi-label image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1556–1564 (2015)

    Google Scholar 

  25. Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1717–1724 (2014)

    Google Scholar 

  26. Lin, K., Yang, H.-F., Hsiao, J.-H., Chen, C.-S.: Deep learning of binary hash codes for fast image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 27–35 (2015)

    Google Scholar 

  27. Xia, R., Pan, Y., Lai, H., Liu, C., Yan, S.: Supervised hashing for image retrieval via image representation learning. In: AAAI, vol. 1, p. 2 (2014)

    Google Scholar 

  28. Pazzani, M.J., Billsus, D.: Content-based recommendation systems. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) The Adaptive Web. LNCS, vol. 4321, pp. 325–341. Springer, Heidelberg (2007). doi:10.1007/978-3-540-72079-9_10

    Chapter  Google Scholar 

  29. Chua, T.-S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.-T.: NUS-WIDE: a real-world web image database from national University of Singapore. In: Proceedings of ACM Conference on Image and Video Retrieval (CIVR 2009), 8–10 July 2009

    Google Scholar 

  30. Mark, B.T., Huiskes, J., Lew, M.S.: New trends and ideas in visual concept detection: The MIR flickr retrieval evaluation initiative. In: Proceedings of the 2010 ACM International Conference on Multimedia Information Retrieval, MIR 2010. New York, NY, USA, pp. 527–536. ACM (2010)

    Google Scholar 

  31. Gionis, A., Indyk, P., Motwani, R., et al.: Similarity search in high dimensions via hashing. VLDB 99(6), 518–529 (1999)

    Google Scholar 

  32. Shen, F., Shen, C., Liu, W., Tao Shen, H.: Supervised discrete hashing. In: Proceedings of the IEEE Conference on CVPR, pp. 37–45 (2015)

    Google Scholar 

Download references

Acknowledgments

This research is supported by The National Natural Science Fund under grant 61332004, JSPS KAKENHI grant number 16K16058.

Author information

Authors and Affiliations

  1. University of Science and Technology of China, Hefei, China

    Chunlin Zhong & Kai Xing

  2. National Institute of Informatics, Tokyo, Japan

    Yi Yu & Shin’ichi Satoh

  3. The University of Electro-Communications, Tokyo, Japan

    Suhua Tang

Authors
  1. Chunlin Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Suhua Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shin’ichi Satoh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kai Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chunlin Zhong .

Editor information

Editors and Affiliations

  1. Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong, China

    Lei Chen

  2. Computer Science, Aarhus University, Aarhus N, Denmark

    Christian S. Jensen

  3. Computer Science, University of Southern California, Los Angeles, California, USA

    Cyrus Shahabi

  4. Northeastern University, Shenyang, China

    Xiaochun Yang

  5. Kent State University, Kent, Ohio, USA

    Xiang Lian

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Zhong, C., Yu, Y., Tang, S., Satoh, S., Xing, K. (2017). Deep Multi-label Hashing for Large-Scale Visual Search Based on Semantic Graph. In: Chen, L., Jensen, C., Shahabi, C., Yang, X., Lian, X. (eds) Web and Big Data. APWeb-WAIM 2017. Lecture Notes in Computer Science(), vol 10366. Springer, Cham. https://doi.org/10.1007/978-3-319-63579-8_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-63579-8_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-63578-1

  • Online ISBN: 978-3-319-63579-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation