Skip to main content
SpringerLink
Log in

Non-negative Tri-factor tensor decomposition with applications

Knowledge and Information Systems volume 34, pages 243–265 (2013)Cite this article

Abstract

Non-negative matrix factorization (NMF) mainly focuses on the hidden pattern discovery behind a series of vectors for two-way data. Here, we propose a tensor decomposition model Tri-ONTD to analyze three-way data. The model aims to discover the common characteristics of a series of matrices and at the same time identify the peculiarity of each matrix, thus enabling the discovery of the cluster structure in the data. In particular, the Tri-ONTD model performs adaptive dimension reduction for tensors as it integrates the subspace identification (i.e., the low-dimensional representation with a common basis for a set of matrices) and the clustering process into a single process. The Tri-ONTD model can also be regarded as an extension of the Tri-factor NMF model. We present the detailed optimization algorithm and also provide the convergence proof. Experimental results on real-world datasets demonstrate the effectiveness of our proposed method in author clustering, image clustering, and image reconstruction. In addition, the results of our proposed model have sparse and localized structures.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  1. Acar E, Yener B (2007) Unsupervised multiway data analysis: a literature survey. Technical report, Computer Science Department, Rensselaer Polytechnic Institute

  2. Acar E, Camtepe SA, Krishnamoorthy M, Yener B (2005) Modeling and multiway analysis of chatroom tensors. In: Proceedings of IEEE international conference on intelligence and security informatics. Lecture Notes in Computer Science

  3. Bader B, Harshman R, Kolda T (2006) Analysis of latent relationships in semantic graphs using DEDICOM invited talk at the workshop on Algorithms for Modern Massive Data Sets

  4. Bock HH (1986) On the interface between cluster analysis, principal components, and multidimensional scaling. In: Proceedings of advances symposium on multivariate modelling and data analysis. Reidel Publishing Co., Dordrecht, pp 17–34

  5. Bolton RJ, Krzanowski WJ (2003) Projection pursuit clustering for exploratory data analysis. J Comput Graph Stat 12: 121–142

    Article  MathSciNet  Google Scholar 

  6. Buntine W, Perttu S (2003) Is multinomial pca multi-faceted clustering or dimensionality reduction. In: Proceedings of 9th international workshop on artificial intelligence and statistics, pp 300–307

  7. Cho H, Dhillon I, Guan Y, Sra S (2004) Minimum sum squared residue co-clustering of gene expression data. In: Proceedings of SIAM data mining conference

  8. De Lathauwer L, De Moor B, Vandewalle J (2000) A multilinear singular value decomposition. SIAM J Matrix Anal Appl 21(4): 1253–1278

    Article  MathSciNet  MATH  Google Scholar 

  9. Ding C, He X, Simon H (2005) On the equivalence of nonnegative matrix factorization and spectral clustering. In: Proceedings of SIAM data mining conference

  10. Ding C, Li T (2007) Adaptive dimension reduction using discriminant analysis and k-means clustering. In: ICML, pp 521–528

  11. Ding C, Li T, Jordan Michael I (2010) Convex and semi-nonnegative matrix factorizations. IEEE Trans Pattern Anal Mach Intell 32(1): 45–55

    Article  Google Scholar 

  12. Ding C, Li T, Peng W, Park H (2006) Orthogonal nonnegative matrix tri-factorizations for clustering. In: SIGKDD, pp 126–135

  13. Ding C, Ye JP (2005) 2-Dimensional singular value decomposition for 2D maps and images. In: Proceedings of SIAM data mining conference

  14. Dhillon IS, Mallela S, Modha DS (2003) Information-theoretical co-clustering. In: SIGKDD, pp 89–98

  15. DeSarbo WS (1982) GENNCLUS: new models for general non-hierarchical clustering analysis. Psychometrika 47: 449–475

    Article  MathSciNet  MATH  Google Scholar 

  16. De Soete G, Carroll JD (1994) K-means Clustering in a Low-dimensional Euclidean Space. In: New approaches in classification and data analysis. Springer, Heidelberg, pp 212–219

  17. Eckart C, Young G (1936) The approximation of one matrix by another of lower rank. Psychometrika 1: 183–187

    Article  Google Scholar 

  18. Golub G, Van Loan C (1996) Matrix computations, 3rd edn. Johns Hopkins, Baltimore

    MATH  Google Scholar 

  19. Govaert G (1995) Simultaneous clustering of rows and columns. Control Cybern 24: 437–458

    MATH  Google Scholar 

  20. Harshman RA (1978) Models for analysis of asymmetrical relationships among N objects or stimuli. In: First joint meeting of the psychometric society for mathematical psychology

  21. Harshman RA (1970) Foundations of the parafac procedure: models and conditions for an ‘explanatory’ multi-modal factor analysis. UCLA working papers in phonetics 16, pp 1–84

  22. Harshman RA, Kolda TG, Bader BW (2007) Temporal analysis of semantic graphs using asalsan. In: Proceedings of IEEE international conference on data mining (ICDM 2007)

  23. Hastie T, Tibshirani R, Friedman JH (2001) The elements of statistical learning. Springer, Berlin

    MATH  Google Scholar 

  24. Hyvarinen A, Karhunen J, Oja E (2001) Independent component analysis. Wiley, London

    Book  Google Scholar 

  25. Kim Y, Choi S (2007) Nonnegative tucker decomposition. In: Proceedings of IEEE conference on computer vision and pattern recognition

  26. Kroonenberg PM, De Leeuw J (1980) Principal component analysis of three-mode data by means of alternating least squares algorithms. Psychometrika 45: 69–97

    Article  MathSciNet  MATH  Google Scholar 

  27. Kolda T (2001) Orthogonal tensor decomposition. SIAM J Matrix Anal Appl 23: 243–255

    Article  MathSciNet  MATH  Google Scholar 

  28. Kolda T, Bader B (2006) The TOPHITS model for higher-order web link analysis. In: Workshop on link analysis, counter terrorism and security

  29. Lee D, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401: 788–791

    Article  Google Scholar 

  30. Lee D, Seung HS (2001) Algorithms for non-negative matrix factorization. In: NIPS

  31. Li T (2008) Clustering based on matrix approximation: a unifying view. Knowl Inf Syst (KAIS) 17(1): 1–15

    Article  MATH  Google Scholar 

  32. Li T, Ding C (2006) The relationships among various nonnegative matrix factorization methods for clustering. In: ICDM, pp 362–371

  33. Li T, Ma S, Ogihara M (2004) Document clustering via adaptive subspace iteration. In: SIGIR, pp 218–225

  34. Mardia KV, Kent JT, Bibby JM (1979) Multivariate analysis. Academic Press, London

    MATH  Google Scholar 

  35. Paatero P (1999) The multilinear engine: a table-driven, least squares program for solving multilinear problems, including the n-way parallel factor analysis model. J Comput Graph Stat 8(4): 854–888

    MathSciNet  Google Scholar 

  36. Paatero P, Tapper U (1994) Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5: 111–126

    Article  Google Scholar 

  37. Peng W, Li T (2011) Temporal relation co-clustering on directional social network and author-topic evolution. Knowl Inf Syst (KAIS) 26(3): 467–486

    Article  Google Scholar 

  38. Peng W, Li T (2011) On the equivalence between nonnegative tensor factorization and tensorial probabilistic latent semantic analysis. Appl Intell 35(2): 285–295

    Article  MathSciNet  MATH  Google Scholar 

  39. Rocci R, Vichi M (2005) Three-mode component analysis with crisp or fuzzy partition of units. Psychometrika 70(4): 715–736

    Article  MathSciNet  Google Scholar 

  40. Shashua A, Hazan T (2005) Non-negative tensor factorization with applications to statistics and computer vision. ICML’05

  41. Smilde A, Bro R, Geladi P (2004) Multi-way analysis: applications in the chemical sciences. Wiley, London

    Book  Google Scholar 

  42. Strehl A, Ghosh J (2002) Cluster ensembles—a knowledge reuse framework for combining multiple partitions. J Mach Learn Res (JMLR) 3: 583–617

    MathSciNet  Google Scholar 

  43. Sun J, Zeng H, Liu H, Lu Y, Chen Z (2005) Cubesvd: a novel approach to personalized web search. In: Proceedings of the 14th international conference on World Wide Web

  44. Tipping M, Bishop C (1999) Probabilistic principal component analysis. J R Stat Soc Ser B 21(3): 611–622

    Article  MathSciNet  Google Scholar 

  45. Thurau C, Kersting K, Wahabzada M, Bauckhage C (2011) Convex non-negative matrix factorization for massive datasets. Knowl Inf Syst (KAIS)

  46. Tucker LR (1966) Some mathematical notes on three-mode factor analysis. Psychometrika 31(3): 279–311

    Article  MathSciNet  Google Scholar 

  47. Yu SX, Shi J (2003) Multiclass spectral clustering. In: Proceedigns of the 9th IEEE international conference on computer vision (ICCV 2003), pp 313–319

  48. Vasilescu MAO, Terzopoulos D (2002) Multilinear analysis of image ensembles: Tensorfaces. In: Proceedings of the 7th European conference on computer vision-part I (ECCV’02), pp 447–460

  49. Vichi M, Kiers HAL (2001) Factorial k-means analysis for two-way data. Comput Stat Data Anal 37: 49–64

    Article  MathSciNet  MATH  Google Scholar 

  50. Vichi M, Rocci R (2008) Two-mode multi-partitioning. Comput Stat Data Anal 52: 1984–2003

    Article  MathSciNet  MATH  Google Scholar 

  51. Vichi M, Rocci R, Kiers HAL (2007) Simultaneous component and clustering models for three-way data: within and between approaches. J Classif 24(1): 71–98

    Article  MathSciNet  MATH  Google Scholar 

  52. Welling M, Weber M (2001) Positive tensor factorization. Pattern Recogn Lett 22(12): 1255–1261

    Article  MATH  Google Scholar 

  53. Wu X, Kumar V, Quinlan JR, Ghosh J, Yang Q, Motoda H, McLachlan GJ, Ng A, Liu B, Yu PS, Zhou Z, Steinbach M, Hand DJ, Steinberg D (2007) Top 10 algorithms in data mining. Knowl Inf Syst (KAIS) 14(1): 1–37

    Article  Google Scholar 

  54. Zhang T, Golub GH (2001) Rank-one approximation to high order tensor. SIAM J Matrix Anal Appl 23: 534–550

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Statistics, Central University of Finance and Economics, Beijing, People’s Republic of China

    Zhong-Yuan Zhang

  2. School of Computing and Information Sciences, Florida International University, 11200 SW 8th Street, Miami, FL, 33199, USA

    Tao Li

  3. Computer Science and Engineering Department, University of Texas at Arlington, Arlington, TX, USA

    Chris Ding

Authors
  1. Zhong-Yuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chris Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao Li.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Zhang, ZY., Li, T. & Ding, C. Non-negative Tri-factor tensor decomposition with applications. Knowl Inf Syst 34, 243–265 (2013). https://doi.org/10.1007/s10115-011-0460-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-011-0460-y

Keywords

search

Navigation