Abstract
High-order tensor data are prevalent in real-world applications, and multiway clustering is one of the most important techniques for exploratory data mining and compression of multiway data. However, existing multiway clustering is based on the K-means procedure and is incapable of addressing the issue of crossed membership degrees. To overcome this limitation, we propose a flexible multiway clustering model called approximately orthogonal nonnegative Tucker decomposition (AONTD). The new model provides extra flexibility to handle crossed memberships while fully exploiting the multilinear property of tensor data. The accelerated proximal gradient method and the low-rank compression tricks are adopted to optimize the cost function. The experimental results on both synthetic data and real-world cases illustrate that the proposed AONTD model outperforms the benchmark clustering methods by significantly improving the interpretability and robustness.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cichocki A, Zdunek R, Phan A H, et al. Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-Way Data Analysis and Blind Source Separation. Piscataway: Wiley Publishing, 2009
Kolda T G, Bader B W. Tensor decompositions and applications. SIAM Rev, 2009, 51: 455–500
Cichocki A, Mandic D, de Lathauwer L, et al. Tensor decompositions for signal processing applications: From two-way to multiway component analysis. IEEE Signal Process Mag, 2015, 32: 145–163
Zhou G, Cichocki A, Zhao Q, et al. Nonnegative matrix and tensor factorizations: An algorithmic perspective. IEEE Signal Process Mag, 2014, 31: 54–65
Bekkerman R, El-Yaniv R, McCallum A. Multi-way distributional clustering via pairwise interactions. In: Proceedings of the 22nd International Conference on Machine Learning. New York, 2005. 41–48
Hore V, Viñuela A, Buil A, et al. Tensor decomposition for multiple-tissue gene expression experiments. Nat Genet, 2016, 48: 1094–1100
Ma P, Zhong W. Penalized clustering of large-scale functional data with multiple covariates. J Am Stat Assoc, 2008, 103: 625–636
Ding C, He X, Simon H D. On the equivalence of nonnegative matrix factorization and spectral clustering. In: Proceedings of the 2005 SIAM International Conference on Data Mining (SDM). Newport Beach, 2005. 606–610
He X, Cai D, Niyogi P. Tensor subspace analysis. In: Proceedings of the 18th International Conference on Neural Information Processing Systems. Vancouver, 2005. 499–506
Kernfeld E, Aeron S, Kilmer M. Clustering multi-way data: A novel algebraic approach. 2014. ArXiv:1412.7056
Long B, Zhang Z M, Wu X, et al. Spectral clustering for multi-type relational data. In: Proceedings of the 23rd International Conference on Machine Learning. Pittsburgh, 2006. 585–592
Damle A, Minden V, Ying L. Simple, direct and efficient multi-way spectral clustering. Inf Inference-A J IMA, 2019, 8: 181–203
Wu T, Benson A R, Gleich D F. General tensor spectral co-clustering for higher-order data. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona, 2016. 2559–2567
Boutalbi R, Labiod L, Nadif M. Tensor latent block model for co-clustering. Int J Data Sci Anal, 2020, 10: 161–175
Hitchcock F L. The expression of a tensor or a polyadic as a sum of products. J Math Phys, 1927, 6: 164–189
Tucker L R. Some mathematical notes on three-mode factor analysis. Psychometrika, 1966, 31: 279–311
Zhou G, Cichocki A, Zhao Q, et al. Efficient nonnegative Tucker decompositions: Algorithms and uniqueness. IEEE Trans Image Process, 2015, 24: 4990–5003
Wu J, Wang Z, Wu Y, et al. A tensor CP decomposition method for clustering heterogeneous information networks via stochastic gradient descent algorithms. Sci Programm, 2017, 2017: 1–13
Wei Z, Zhao H, Zhao L, et al. Multiscale co-clustering for tensor data based on canonical polyadic decomposition and slice-wise factorization. Inf Sci, 2019, 503: 72–91
Qiu Y, Zhou G, Zhang Y, et al. Canonical polyadic decomposition (CPD) of big tensors with low multilinear rank. Multimed Tools Appl, 2020, 98: 1–21
Wang M, Zeng Y. Multiway clustering via tensor block models. In: Proceedings of Advances in Neural Information Processing Systems. Vancouver, 2019. 715–725
Dueck D, Morris Q D, Frey B J. Multi-way clustering of microarray data using probabilistic sparse matrix factorization. Bioinformatics, 2005, 21: i144–i151
Zass R, Shashua A. A unifying approach to hard and probabilistic clustering. In: Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV’05). Beijing, 2005, 1: 294–301
Shashua A, Zass R, Hazan T. Multi-way clustering using super-symmetric non-negative tensor factorization. In: Proceedings of European Conference on Computer Vision. Graz, 2006. 595–608
Li B, Zhou G X, Cichocki A. Two efficient algorithms for approximately orthogonal nonnegative matrix factorization. IEEE Signal Process Lett, 2015, 22: 843–846
Ding C, Li T, Peng W, et al. Orthogonal nonnegative matrix t-factorizations for clustering. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Philadelphia, 2006. 126–135
Qiu Y, Zhou G, Wang Y, et al. A generalized graph regularized nonnegative Tucker decomposition framework for tensor data representation. IEEE Trans Cybern, 2020, doi: https://doi.org/10.1109/TCYB.2020.2979344
Bertsekas D P. Nonlinear Programming. 2nd ed. Nashua: Athena Scientific, 1999
Nesterov Y E. A method of solving a convex programming problem with convergence rate \(o\left( {{1 \over {{k^2}}}} \right)\). Dokl Akad Nauk SSSR, 1983, 269: 543–547
Beck A, Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imag Sci, 2009, 2: 183–202
Guan N, Tao D, Luo Z, et al. NeNMF: An optimal gradient method for nonnegative matrix factorization. IEEE Trans Signal Process, 2012, 60: 2882–2898
Zhang Y, Zhou G, Zhao Q, et al. Fast nonnegative tensor factorization based on accelerated proximal gradient and low-rank approximation. Neurocomputing, 2016, 198: 148–154
Kemp C, Tenenbaum J B, Griffiths T L, et al. Learning systems of concepts with an infinite relational model. In: Proceedings of the 21st National Conference on Artificial Intelligence. Boston, 2006. 381–388
Nickel M, Tresp V, Kriegel H P. A three-way model for collective learning on multi-relational data. In: Proceedings of the 28th International Conference on International Conference on Machine Learning. Madison, 2011. 809–816
Zhou G, Cichocki A, Zhang Y, et al. Group component analysis for multiblock data: Common and individual feature extraction. IEEE Trans Neural Netw Learn Syst, 2016, 27: 2426–2439
Zhou G, Zhao Q, Zhang Y, et al. Linked component analysis from matrices to high-order tensors: Applications to biomedical data. Proc IEEE, 2016, 104: 310–331
Huang Z, Qiu Y, Sun W. Recognition of motor imagery EEG patterns based on common feature analysis. Brain-Comput Interfaces, 2020, 1: 1–9
Yu J, Zhou G, Li C, et al. Low tensor-ring rank completion by parallel matrix factorization. IEEE Trans Neural Netw Learn Syst, 2021, 32: 3020–3033
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by the National Natural Science Foundation of China (Grant Nos. 62073087, 62071132, 61973090 and U1911401) and the Key-Area Research and Development Program of Guangdong Province (Grant Nos. 2019B010154002 and 2019010118001).
Rights and permissions
About this article
Cite this article
Qiu, Y., Sun, W., Zhang, Y. et al. Approximately orthogonal nonnegative Tucker decomposition for flexible multiway clustering. Sci. China Technol. Sci. 64, 1872–1880 (2021). https://doi.org/10.1007/s11431-020-1827-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11431-020-1827-0