Abstract
Multi-view hashing has gained considerable research attention in efficient multimedia studies due to its promising performance on heterogeneous data from various sources. However, its application in discriminative hash codes learning remains challenging as it fails to efficiently capture preferable components from multiple representations. In this work, we propose a novel discriminative multi-view hashing framework, dubbed flexible discrete multi-view hashing, in conjunction with collective latent feature learning by combining multiple views of data and consistent hash codes learning by fusing visual features and flexible semantics. Specifically, an adaptive multi-view analysis dictionary learning model is developed to skillfully combine diverse representations into an established common latent feature space where the complementary properties of different views are well explored based on an automatic multi-view weighting strategy. Moreover, we introduce a collaborative learning scheme to jointly encode the visual and semantic embeddings into an aligned consistent Hamming space, which can effectively mitigate the visual-semantic gap. Particularly, we employ the correntropy induced regularization to improve the robustness of the formulated flexible semantics. An efficient learning algorithm is proposed to solve the optimization problem. Extensive experiments show the state-of-art performance of the proposed method on several benchmark datasets.
Similar content being viewed by others
Notes
The codes of this work has been released at https://github.com/DarrenZZhang/FDMH.
References
Aharon M, Elad M, Bruckstein A (2006) K-svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54(11):4311–4322
Ahonen T, Hadid A, Pietikäinen M (2004) Face recognition with local binary patterns. In: European conference on computer vision, pp 469–481
Blei DM, Ng AY, Jordan MI (2003) Latent dirichlet allocation. J. Mach. Learn. Res. 3:993–1022
Chen H, Wang Y (2018) Kernel-based sparse regression with the correntropy-induced loss. Appl Comput Harmon Anal 44(1):144–164
Csurka G, Dance CR, Fan L, Willamowski J, Bray C (2004) Visual categorization with bags of keypoints. In: In ECCV workshop on statistical learning in computer vision, pp 1–22
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision and pattern recognition, pp 886–893
Gionis A, Indyk P, Motwani, R (1999) Similarity search in high dimensions via hashing. In: the 25th international conference on very large data bases, pp 518–529
Gong Y, Lazebnik S, Gordo A, Perronnin F (2013) Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans Pattern Anal Mach Intell 35(12):2916–2929
Gui J, Liu T, Sun Z, Tao D, Tan T (2018) Fast supervised discrete hashing. IEEE Trans Pattern Anal Mach Intell 40(2):490–496
Guillaumin M, Verbeek J, Schmid C (2010) Multiple instance metric learning from automatically labeled bags of faces. Eur Conf Comput Vis 6311:634–647
Guo J, Guo Y, Kong X, Zhang M, He R (2016) Discriminative analysis dictionary learning. In: The thirtieth AAAI conference on artificial intelligence, pp 1617–1623
He R, Zheng WS, Hu BG (2010) Maximum correntropy criterion for robust face recognition. IEEE Trans Pattern Anal Mach Intell 33(8):1561–1576
Hu R, Zhu X, Zhu Y, Gan J (2019) Robust SVM with adaptive graph learning. World Wide Web. https://doi.org/10.1007/s11280-019-00766-x
Huiskes MJ, Lew MS (2008) The MIR flickr retrieval evaluation. In: ACM international conference on multimedia information retrieval, pp 39–43
Jiang Q, Li W (2015) Scalable graph hashing with feature transformation. In: The twenty-fourth international joint conference on artificial intelligence, pp 2248–2254
Kim S, Choi S (2013) Multi-view anchor graph hashing. In: IEEE international conference on acoustics, speech and signal processing, pp 3123–3127
Krizhevsky A (2012) Learning multiple layers of features from tiny images. University of Toronto
Li J, Zhang B, Lu G, Zhang D (2019) Dual asymmetric deep hashing learning. IEEE Access
Li Z, Zhang Z, Qin j, Zhang Z, Shao L (2019) Discriminative fisher embedding dictionary learning algorithm for object recognition. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2019.2910146
Liu L, Du X, Zhu L, Shen F, Huang Z (2018) Discrete binary hashing towards efficient fashion recommendation. In: Database systems for advanced applications—23rd international conference, DASFAA 2018, Gold Coast, QLD, Australia, May 21–24, 2018, Proceedings, Part I, pp 116–132
Liu L, Du X, Zhu L, Shen F, Huang Z (2018) Learning discrete hashing towards efficient fashion recommendation. Data Sci Eng 3(4):307–322
Liu L, Yu M, Shao L (2015) Multiview alignment hashing for efficient image search. IEEE Trans Image Process 24(3):956–966
Liu L, Zhu L, Li Z (2017) Learning robust graph hashing for efficient similarity search. In: Databases theory and applications—28th Australasian database conference, ADC 2017, Brisbane, QLD, Australia, September 25–28, 2017, Proceedings, pp 110–122
Liu W, Pokharel PP, Príncipe JC (2007) Correntropy: properties and applications in non-gaussian signal processing. IEEE Trans Signal Process 55(11):5286–5298
Liu W, Wang J, Kumar S, Chang S (2011) Hashing with graphs. In: the 28th international conference on machine learning, pp 1–8
Liu X, He J, Liu D, Lang B (2012) Compact kernel hashing with multiple features. In: The 20th ACM conference on multimedia, pp 881–884
Lu X, Zhu L, Cheng Z, Nie L, Zhang H (2019) Online multi-modal hashing with dynamic query-adaption. In: The 42nd international ACM SIGIR conference on research and development in information retrieval, pp 715–724
Manning C, Raghavan P, Schütze H (2010) Introduction to information retrieval. Nat Lang Eng 16(1):100–103
Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vis 42(3):145–175
Pokharel R, Príncipe JC (2012) Kernel classifier with correntropy loss. In: The 2012 international joint conference on neural networks (IJCNN), Brisbane, Australia, June 10–15, 2012, pp 1–6
Rasiwasia N, Pereira JC, Coviello E, Doyle G, Lanckriet GRG, Levy R, Vasconcelos N (2010) A new approach to cross-modal multimedia retrieval. In: The 18th international conference on multimedia retrieval, pp 251–260
Shen F, Mu Y, Yang Y, Liu W, Liu L, Song J, Shen HT (2017) Classification by retrieval: Binarizing data and classifiers. In: Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, pp 595–604. ACM
Shen F, Shen C, Liu W, Shen HT (2015) Supervised discrete hashing. In: IEEE conference on computer vision and pattern recognition, pp. 37–45
Shen F, Zhou X, Yu J, Yang Y, Liu L, Shen HT (2019) Scalable zero-shot learning via binary visual-semantic embeddings. IEEE Trans Image Process 28(7):3662–3674
Shen X, Shen F, Liu L, Yuan Y, Liu W, Sun Q (2018) Multiview discrete hashing for scalable multimedia search. ACM Trans Intell Syst Technol 9(5):53:1–53:21
Shen XB, Shen F, Sun Q, Yuan Y (2015) Multi-view latent hashing for efficient multimedia search. In: The 23rd ACM conference on multimedia, pp 831–834
Song J, Yang Y, Huang Z, Shen HT, Luo J (2013) Effective multiple feature hashing for large-scale near-duplicate video retrieval. IEEE Trans Multimed 15(8):1997–2008
Wang H, Nie F, Huang H (2013) Multi-view clustering and feature learning via structured sparsity. In: The 30th international conference on machine learning, pp 352–360
Wang J, Zhang T, Song J, Sebe N, Shen HT (2018) A survey on learning to hash. IEEE Trans Pattern Anal Mach Intell 40(4):769–790
Wang Z, Zhang Z, Luo Y, Huang Z (2019) Deep collaborative discrete hashing with semantic-invariant structure. In: The 42nd international ACM SIGIR conference on research and development in information retrieval, pp 905–908
Weiss Y, Torralba A, Fergus R (2008) Spectral hashing. In: Advances in neural information processing systems, pp 1753–1760
Yang R, Shi Y, Xu XS (2017) Discrete multi-view hashing for effective image retrieval. In: Proceedings of the 2017 ACM on international conference on multimedia retrieval, pp. 175–183
Zhang D, Wang F, Si L (2011) Composite hashing with multiple information sources. In: The 34th international ACM SIGIR conference on research and development in information retrieval, pp 225–234
Zhang P, Zhang W, Li W, Guo M (2014) Supervised hashing with latent factor models. In: The 37th international ACM SIGIR conference on research and development in information retrieval, pp 173–182
Zhang Z, Lai Z, Huang Z, Wong WK, Xie GS, Liu L, Shao L (2019) Scalable supervised asymmetric hashing with semantic and latent factor embedding. IEEE Trans Image Process 28(10):4803–4818
Zhang Z, Liu L, Qin J, Zhu F, Shen F, Xu Y, Shao L, Shen HT (2018) Highly-economized multi-view binary compression for scalable image clustering. In: European conference on computer vision, pp 717–732
Zhang Z, Liu L, Shen F, Shen HT, Shao L (2019) Binary multi-view clustering. IEEE Trans Pattern Anal Mach Intell 41(7):1774–1782
Zheng C, Zhu L, Lu X, Li J, Cheng Z, Zhang H (2019) Fast discrete collaborative multi-modal hashing for large-scale multimedia retrieval. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2019.2913388
Zhu L, Huang Z, Li Z, Xie L, Shen HT (2018) Exploring auxiliary context: discrete semantic transfer hashing for scalable image retrieval. IEEE Trans Neural Netw Learn Syst 29(11):5264–5276
Zhu L, Huang Z, Liu X, He X, Sun J, Zhou X (2017) Discrete multimodal hashing with canonical views for robust mobile landmark search. IEEE Trans Multimed 19(9):2066–2079
Zhu L, Shen J, Xie L, Cheng Z (2017) Unsupervised visual hashing with semantic assistant for content-based image retrieval. IEEE Trans Knowl Data Eng 29(2):472–486
Zhu X, Yang J, Zhang C, Zhang S (2019) Efficient utilization of missing data in cost-sensitive learning. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2019.2956530
Zhu X, Zhang S, Hu R, He W, Lei C, Zhu P (2019) One-step multi-view spectral clustering. IEEE Trans Knowl Data Eng 31(10):2022–2034
Zhu X, Zhu Y, Zheng W (2019) Spectral rotation for deep one-step clustering. Pattern Recognit. https://doi.org/10.1016/j.patcog.2019.107175
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Liu, L., Zhang, Z. & Huang, Z. Flexible Discrete Multi-view Hashing with Collective Latent Feature Learning. Neural Process Lett 52, 1765–1791 (2020). https://doi.org/10.1007/s11063-020-10221-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-020-10221-y