Abstract
Nonnegative Matrix Factorization (NMF) has received great attention in the era of big data, owing to its roles in efficiently reducing data dimension and producing feature-based data representation. In this paper, we first propose two new NMF optimization models, called an orthogonal dual graph regularized nonnegative matrix factorization (ODGNMF) method and its modified version: an orthogonal dual graph regularized nonnegative matrix tri-factorization (ODGNMTF) method. Compared with the existing models, our models can preserve the geometrical structures of data manifold and feature manifold by constructing two graphs, and ensure the orthogonality of factor matrices such that they have better NMF performance. Then, two efficient algorithms are developed to solve the models, and the convergence theory of the algorithms is established. Numerical tests by applying our algorithms to mine randomly generated data sets and well-known public databases demonstrate that ODGNMF and ODGNMTF have better numerical performance than the state-of-the-art algorithms in view of computational cost, robustness, sensitivity and sparseness.
Similar content being viewed by others
Data Availability Statement
The data related with the findings of this study are available from the corresponding author upon reasonable requests.
References
Zhang, L., Liu, Z., Pu, J., et al.: Adaptive graph regularized nonnegative matrix factorization for data representation. Appl. Intell. 50, 438–447 (2020). https://doi.org/10.1007/s10489-019-01539-9
Gao, Z., Wang, Y., Wu, Q., et al.: Graph regularized L2,1-nonnegative matrix factorization for miRNA-disease association prediction. BMC Bioinform. 21, 61 (2020). https://doi.org/10.1186/s12859-020-3409-x
Wan, Z., Tang, J., Ren, L., et al.: Optimization techniques to deeply mine the transcriptomic profile of the sub-genomes in hybrid fish lineage. Front. Genet. 10, 911 (2019). https://doi.org/10.3389/fgene.2019.00911
Jolliffe, I.T., Cadima, J.: Principal component analysis: a review and recent developments. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 374, 20150202 (2016). https://doi.org/10.1098/rsta.2015.0202
Comon, P., Jutten, C.: Handbook of Blind Source Separation: Independent Component Analysis and Applications. Academic Press, Cambridge (2010)
Lieven, D.L., Bart, D.M., Joos, V.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21, 1253–1278 (2000). https://doi.org/10.1137/S0895479896305696
Wang, H., Zheng, C., Zhao, X.: jNMFMA: a joint non-negative matrix factorization meta-analysis of transcriptomics data. Bioinformatics 31, 572–580 (2015). https://doi.org/10.1093/bioinformatics/btu679
Shang, R., Song, J., Jiao, L., et al.: Double feature selection algorithm based on low-rank sparse non-negative matrix factorization. Int. J. Mach. Learn. Cybernet. 11, 1891–1908 (2020). https://doi.org/10.1007/s13042-020-01079-6
Belachew, M.T.: Efficient algorithm for sparse symmetric nonnegative matrix factorization. Pattern Recogn. Lett. 125, 735–741 (2019). https://doi.org/10.1016/j.patrec.2019.07.026
Tosyali, A., Kim, J., Choi, J., et al.: Regularized asymmetric nonnegative matrix factorization for clustering in directed networks. Pattern Recogn. Lett. 125, 750–757 (2019). https://doi.org/10.1016/j.patrec.2019.07.005
Peng, S., Ser, W., Chen, B., et al.: Robust nonnegative matrix factorization with local coordinate constraint for image clustering. Eng. Appl. Artif. Intell. 88, 103354 (2020). https://doi.org/10.1016/j.engappai.2019.103354
Chen, G., Xu, C., Wang, J., et al.: Graph regularization weighted nonnegative matrix factorization for link prediction in weighted complex network. Neurocomputing 369, 50–60 (2020). https://doi.org/10.1016/j.neucom.2019.08.068
Hoyer, P.O.: Non-negative matrix factorization with sparseness constraints. J. Mach. Learn. Res. 5, 1457–1469 (2004)
Cai, D., He, X., Han, J., et al.: Graph regularized nonnegative matrix factorization for data representation. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1548–1560 (2011). https://doi.org/10.1109/TPAMI.2010.231
Li, X., Chen, M., Wang, Q.: Discrimination-aware projected matrix factorization. IEEE Trans. Knowl. Data Eng. 32, 809–814 (2020). https://doi.org/10.1109/TKDE.2019.2936855
Wang, Q., He, X., Jiang, X., et al.: Robust Bi-stochastic graph regularized matrix factorization for data clustering. IEEE Trans. Pattern Anal. Mach. Intell. (2020). https://doi.org/10.1109/TPAMI.2020.3007673
Gu, Q., Zhou, J.: Co-clustering on manifolds. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 359–368 (2009). https://doi.org/10.1145/1557019.1557063
Shang, F., Jiao, L.C., Wang, F.: Graph dual regularization non-negative matrix factorization for co-clustering. Pattern Recogn. 45, 2237–2250 (2012). https://doi.org/10.1016/j.patcog.2011.12.015
Wang, Q., Chen, M., Nie, F., et al.: Detecting coherent groups in crowd scenes by multi-view clustering. IEEE Trans. Pattern Anal. Mach. Intell. 42, 46–58 (2020). https://doi.org/10.1109/TPAMI.2018.2875002
Sun, J., Wang, Z., Sun, F., et al.: Sparse dual graph-regularized NMF for image co-clustering. Neurocomputing 316, 156–165 (2018). https://doi.org/10.1016/j.neucom.2018.07.062
Ding, C., Li, T., Peng, W., et al.: Orthogonal nonnegative matrix t-factorizations for clustering. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 126–135 (2006). https://doi.org/10.1145/1150402.1150420
Yoo, J., Choi, S.: Orthogonal nonnegative matrix tri-factorization for co-clustering: Multiplicative updates on Stiefel manifolds. Inf. Process. Manage. 46, 559–570 (2010). https://doi.org/10.1016/j.ipm.2009.12.007
Dhillon, I.S.: Co-clustering documents and words using bipartite spectral graph partitioning. In: Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 269–274 (2001). https://doi.org/10.1145/502512.502550
Dhillon, I.S., Mallela, S., Modha, D.S.: Information-theoretic co-clustering. Proceedings of the 9h ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 89–98 (2003). https://doi.org/10.1145/956750.956764
Wang, S., Chang, T., Cui, Y., et al.: Clustering by orthogonal non-negative matrix factorization: a sequential non-convex penalty approach. In: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5576–5580 (2019). https://doi.org/10.1109/ICASSP.2019.8683466
He, P., Xu, X., Ding, J., et al.: Low-rank nonnegative matrix factorization on Stiefel manifold. Inf. Sci. 514, 131–148 (2020). https://doi.org/10.1016/j.ins.2019.12.004
Abe, H., Yadohisa, H.: Orthogonal nonnegative matrix tri-factorization based on Tweedie distributions. Adv. Data Anal. Classif. 13, 825–853 (2019). https://doi.org/10.1007/s11634-018-0348-8
Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20, 303–353 (1998). https://doi.org/10.1137/S0895479895290954
Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999). https://doi.org/10.1038/44565
Stiefel, E.: Richtungsfelder und Fernparallelismus in n-dimensionalen Mannigfaltigkeiten. Commentarii Mathematici Helvetici 8, 305–353 (1935–1936)
Lv, J., Deng, S., Wan, Z.: An efficient single-parameter scaling memoryless Broyden–Fletcher–Goldfarb–Shanno algorithm for solving large scale unconstrained optimization problems. IEEE Access 8, 85664–85674 (2020). https://doi.org/10.1109/ACCESS.2020.2992340
Li, T., Wan, Z.: New adaptive Barzilar–Borwein step size and its application in solving large scale optimization problems. The ANZIAM J. 61, 76–98 (2019). https://doi.org/10.1017/S1446181118000263
Guo, J., Wan, Z.: A modified spectral PRP conjugate gradient projection method for solving large-scale monotone equations and its application in compressed sensing. Math. Probl. Eng. 2019, 5261830 (2019). https://doi.org/10.1155/2019/5261830
Gaussier, E., Goutte, C.: Relation between PLSA and NMF and implications. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 601–602 (2005). https://doi.org/10.1145/1076034.1076148
Ding, C., Li, T., Peng, W.: On the equivalence between non-negative matrix factorization and probabilistic latent semantic indexing. Comput. Stat. Data Anal. 52, 3913–3927 (2008). https://doi.org/10.1016/j.csda.2008.01.011
Cai, D., He, X., Han, J.: Document clustering using locality preserving indexing. IEEE Trans. Knowl. Data Eng. 17, 1624–1637 (2005). https://doi.org/10.1109/TKDE.2005.198
Huang, S., Wan, Z., Zhang, J.: An extended nonmonotone line search technique for large-scale unconstrained optimization. J. Comput. Appl. Math. 330, 586–604 (2018). https://doi.org/10.1016/j.cam.2017.09.026
Ding, C., He, X., Simon, H.D.: On the equivalence of nonnegative matrix factorization and spectral clustering. In: Proceedings of the 2005 SIAM International Conference on Data Mining, pp. 606–610 (2005). https://doi.org/10.1137/1.9781611972757.70
Sun, J., Cai, X., Sun, F., et al.: Dual graph-regularized constrained nonnegative matrix factorization for image clustering. KSII Trans. Internet Inf. Syst. 11, 2607–2627 (2017). https://doi.org/10.3837/tiis.2017.05.017
Acknowledgements
This research is supported by the National Natural Science Foundation of China (Grant No. 71671190).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
We declare that all the authors have no conflict of interest about submission and publication of this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Tang, J., Wan, Z. Orthogonal Dual Graph-Regularized Nonnegative Matrix Factorization for Co-Clustering. J Sci Comput 87, 66 (2021). https://doi.org/10.1007/s10915-021-01489-w
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10915-021-01489-w