Abstract
To utilize multiple features for clustering, this paper proposes a novel method named as multiview nonnegative matrix factorization with dual HSIC constraints for clustering. The Hilbert-Schmidt independence criterion (HSIC) is employed to measure the correlation(including linear and nonlinear correlation) between the latent representation of each view and the common ones (representation constraint). The independence among the vectors of the basis matrix for each view (basis constraint) is maximized to pursue the discriminant and informative basis. To maintain the nonlinear structure of multiview data, we directly optimize the kernel of the common representation and make its values of the same neighborhood are larger than the others. We adopt partition entropy to constrain the uniformity level of the its values. A novel iterative update algorithm is designed to seek the optimal solutions. We extensively test the proposed algorithm and several state-of-the-art NMF-based multiview methods on four datasets. The clustering results validate the effectiveness of our method.
Similar content being viewed by others
References
Li J, Cheng K, Wang S, Morstatter F, Trevino RP, Tang J, Liu H (2017) Feature selection: a data perspective. ACM Comput Surv 50:1–45
Xue B, Zhang M, Browne WN, Yao X (2016) A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput 20:606–626
Dökeroglu T, Deniz A, Kiziloz HE (2022) A comprehensive survey on recent metaheuristics for feature selection. Neurocomputing 494:269–296
Wang S, Lu J, Gu X, Weyori BA, Yang J-y (2016) Unsupervised discriminant canonical correlation analysis based on spectral clustering. Neurocomputing 171:425–433
Ouadfel S, Abd Elaziz M (2021) A multi-objective gradient optimizer approach-based weighted multi-view clustering. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2021.104480
Liu X, Wang L, Zhu X, Li M, Zhu E, Liu T, Liu L, Dou Y, Yin J (2019) Absent multiple kernel learning algorithms. IEEE Trans Pattern Anal Mach Intell 1–1
Zhang C, Fu H, Liu S, Liu G, Cao X (2015) Low-rank tensor constrained multiview subspace clustering. Proceedings of the IEEE International Conference on Computer Vision 1:1582–1590
Zhang C, Fu H, Hu Q, Cao X, Xie Y, Tao D, Xu D (2020) Generalized latent multi-view subspace clustering. IEEE Trans Pattern Anal Mach Intell 42:86–99
Yin H, Hu W, Li F, Lou J (2021) One-step multi-view spectral clustering by learning common and specific nonnegative embeddings. Int J Mach Learn Cyber 12(7):2121–2134
Mekthanavanh V, Li T, Meng H, Yang Y, Hu J (2019) Social web video clustering based on multi-view clustering via nonnegative matrix factorization. Int J Mach Learn Cyber 10(10):2779–2790
Hotelling H (1936) Relations between two sets of variates. Biometrika 28(3–4):321–377
Guo X, Wang S, Tie Y, Qi L, Guan L (2018) Joint intermodal and intramodal correlation preservation for semi-paired learning. Pattern Recognit 81:36–49
Su S, Fang X, Yang G, Ge B, Zhu Y (2019) Self-balanced multi-view orthogonality correlation analysis for image feature learning. Infrared Phys Technol 100:44–51
Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2013) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184
Cao X, Zhang C, Fu H, L S, ZhangH ( 2015) Diversity-induced Multi-view Subspace Clustering. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 586– 594. IEEE, Boston, MA, USA
Zhou T, Zhang C, Peng X, Bhaskar H, Yang J (2020) Dual shared-specific multiview subspace clustering. IEEE Trans Cybern 50:3517–3530
Yin M, Gao J, Xie S, Guo Y (2019) Multiview subspace clustering via tensorial t-product representation. IEEE Trans Neural Netw Learn Syst 30(3):851–864
Kumar A, Rai P, Daumé H. III( 2011) Co-regularized multi-view spectral clustering. In: Proc Adv Neural Inf Process Syst pp. 1413– 1421. Curran Associates Inc., USA
Liu X, Dou Y, Yin J, Wang L, Zhu E ( 2016) Multiple kernel k-means clustering with matrix-induced regularization. In: Proc AAAI Conf Artif Intell pp. 1888– 1894
Zhan K, Zhang C, Guan J, Wang J (2018) Graph learning for multiview clustering. IEEE Trans Cybern 48:2887–2895
Liang Y, Huang D, Wang C-D, Yu PS (2022) Multi-view graph learning by joint modeling of consistency and inconsistency. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2022.3192445
Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791
Deng C, He X, Han J, Huang TS (2011) Graph regularized non-negative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560
Li X, Cui G, Dong Y (2017) Graph regularized non-negative low-rank matrix factorization for image clustering. IEEE Trans Cybern 47(11):3840–3853
Huang S, Zhao P, Ren Y, Li T, Xu Z (2018) Self-paced and soft-weighted nonnegative matrix factorization for data representation. Knowl Based Syst. https://doi.org/10.1016/j.knosys.2018.10.003
Yi Y, Wang J, Zhou W, Zheng C, Kong J, Qiao S (2020) Non-negative matrix factorization with locality constrained adaptive graph. IEEE Trans Circuits Syst Video Technol 30:427–441
Peng S, Ser W, Chen B, Sun L, Lin Z (2020) Robust nonnegative matrix factorization with local coordinate constraint for image clustering. Eng Appl Artif Intell. https://doi.org/10.1109/ACCESS.2021.3064631
Pascual-Montano A, Carazo JM, Kochi K, Lehmann D, Pascual-Marqui RD (2006) Nonsmooth nonnegative matrix factorization (nsNMF). IEEE Trans Pattern Anal Mach Intell 28(3):403–415
Guo W (2021) Sparse dual graph-regularized deep nonnegative matrix factorization for image clustering. IEEE Access. https://doi.org/10.1109/ACCESS.2021.3064631
Xing Z, Wen M, Peng J, Feng J (2021) Discriminative semi-supervised non-negative matrix factorization for data clustering. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2021.104289
Zhang K, Zhao X, Peng S (2021) Multiple graph regularized semi-supervised nonnegative matrix factorization with adaptive weights for clustering. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2021.104499
Meng Y, Shang R, Jiao L, Zhang W, Yang S (2018) Dual-graph regularized non-negative matrix factorization with sparse and orthogonal constraints. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2017.11.008
Wang C, Song X, Zhang J (2018) Graph regularized nonnegative matrix factorization with sample diversity for image representation. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2017.10.018
Ou W, Yu S, Li G, Lu J, Zhang K, Xie G (2016) Multi-view non-negative matrix factorization by patch alignment framework with view consistency. Neurocomputing 204:116–124
Zong L, Zhang X, Zhao L, Yu H, Zhao Q (2017) Multi-view clustering via multi-manifold regularized non-negative matrix factorization. Neural Netw 88:74–89
Wang X, Zhang T, Gao X (2019) Multiview clustering based on non-negative matrix factorization and pairwise measurements. IEEE Trans Syst Man Cybern 49(9):3333–3346
Wang J, Tian F, Yu H, Liu CH, Zhan K, Wang X (2018) Diverse non-negative matrix factorization for multiview data representation. IEEE Trans Cybern 48(9):2620–2632
Yang Z, Liang N, Yan W, Li Z, Xie S (2020) Uniform distribution non-negative matrix factorization for multiview clustering. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2020.2984552
Khan GA, Hu J, Li T, Diallo B, Wang H (2022) Multi-view data clustering via non-negative matrix factorization with manifold regularization. Int J Mach Learn Cyber 13(3):677–689
Diallo B, Hu J, Li T, Khan GA, Hussein AS (2022) Multi-view document clustering based on geometrical similarity measurement. Int J Mach Learn Cyber 13(3):663–675
Liu J, Wang C, Gao J, Han J ( 2013) Multi-View Clustering via Joint Nonnegative Matrix Factorization. In: Proceedings of International Conference on Data Mining
Li J, Zhou G, Qiu Y, Wang Y, Zhang Y, Xie S (2020) Deep graph regularized non-negative matrix factorization for multi-view clustering. Neurocomputing 390:108–116
Niu D, Jordan MI (2014) Iterative discovery of multiple alternative clustering views. IEEE Trans Pattern Anal Mach Intell 36(7):14
Gretton A, Bousquet O, Smola A, Schölkopf B (2005) Measuring statistical dependence with Hilbert-Schmidt norms. Algorithmic Learning Theory. Springer, Heidelberg, pp 63–77
Duchene J, Leclercq S (1988) An optimal transformation for discriminant and principal component analysis. IEEE Trans Pattern Anal Mach Intell 10(6):978–983
Wang S, Lu J, Gu X, Du H, Yang J (2019) Semi-supervised linear discriminant analysis for dimension reduction and classification. Pattern Recognit 57:179–189
Zhao Y, Wang H, Pei J (2021) Deep non-negative matrix factorization architecture based on underlying basis images learning. IEEE Trans Pattern Anal Mac Intell 43:1897–1913
Yang Z, Li Q, Wenyin L, Lv J (2019) Shared multi-view data representation for multi-domain event detection. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2019.2893953
Lee DD, Seung HS( 2009) Algorithms for non-negative matrix factorization. In: Proc Adv Neural Inf Process Syst p. 7
Song L, Smola A, Gretton A, Borgwardt KM ( 2007) A dependence maximization view of clustering. In: Proc Int Conf Mach Learn pp. 815– 822. ACM Press, Corvalis, Oregon
Song L, Smola A, Gretton A, Bedo J, Borgwardt K (2012) Feature selection via dependence maximization. J Mach Learn Res 13(47):1393–1434
He X, Yan S, Hu Y, Niyogi P, Zhang H-J (2005) Face recognition using Laplacianfaces. IEEE Trans Pattern Anal Machine Intell 27(3):328–340
Sugiyama M (2007) Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis. J Mach Learn Res 8(37):1027–1061
Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416
Liu X, Zhu X, Li M, Wang L, Zhu E, Liu T, Kloft M, Shen D, Yin J, Gao W (2020) Multiple kernel k-means with incomplete kernels. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2019.2892416
Li T, Dou Y, Liu X, Zhao Y, Lv Q (2017) Multiple kernel clustering with corrupted kernels. Neurocomputing 267:447–454
Ping L, Zhan G, He Q, Shi Z, Lu K (2007) On defining partition entropy by inequalities. IEEE Trans Inf Theory 53(7):3233–3239
Simovici DA, Jaroszewicz S (2002) An axiomatization of partition entropy. IEEE Trans Inf Theory 48(7):2138–2142
Greene D, Cunningham P ( 2006) Practical solutions to the problem of diagonal dominance in kernel document clustering. In: Proc Int Conf Mach Learn pp. 377– 384. ACM Press, ???
Wang J, Tian F, Yu H, Liu CH, Zhan K, Wang X (2018) Diverse non-negative matrix factorization for multiview data representation. IEEE Trans Cybern 48:2620–2632
Zhao L, Yang T, Zhang J, Chen Z, Yang Y, Wang ZJ (2021) Co-learning non-negative correlated and uncorrelated features for multi-view data. IEEE Trans Neural Netw Learn Syst 32(4):1486–1496
Liu J, Liu X, Yang Y, Guo X, Kloft M, He L (2021) Multiview subspace clustering via co-training robust data representation. IEEE Trans Neural Netw Learn Syst 33:1–13
Zhang C, Wang S, Liu J, Zhou S, Zhang P, Liu X, Zhu E, Zhang C ( 2021) Multi-view clustering via deep matrix factorization and partition alignment. In: In Proceedings of Multimedia, pp. 4156– 4164
Nie F, Cai G, Li X ( 2017) Multi-View Clustering and Semi-Supervised Classification with Adaptive Neighbours. In: Proceedings of the AAAI Conference on Artificial Intelligence, p. 7
Huang D, Wang C-D, Lai J-H (2018) Locally weighted ensemble clustering. IEEE Trans Cybern 48(5):1460–1473
Zhang G-Y, Zhou Y-R, He X-Y, Wang C-D, Huang D (2020) One-step kernel multi-view subspace clustering. Knowl Based Syst 189:105–126
Acknowledgements
This work is supported in part by the NSF Foundation of China (No.62276162), by Henan Center for Outstanding Overseas Scientists(GZS2022011), by Key R & D and Promotion Project of Henan Province (No.202102210334), by Henan Province Science Foundation for Youths(No.212300410296), by Henan Provincial Department of Science and Technology Research Project(212102310299), by Natural Science Foundation of China(62276162), by Young Scientists Fund of the Natural Science Foundation of Shanxi, China(202103021223464) and Key Research and Development Plan of Shanxi, China(202102070301019).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, S., Chen, L., Sun, Y. et al. Multiview nonnegative matrix factorization with dual HSIC constraints for clustering. Int. J. Mach. Learn. & Cyber. 14, 2007–2022 (2023). https://doi.org/10.1007/s13042-022-01742-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-022-01742-0