Skip to main content
SpringerLink
Log in

Topological structure regularized nonnegative matrix factorization for image clustering

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Image representation is supposed to reveal the distinguishable feature and be captured in unsupervised fashion. Recently, nonnegative matrix factorization (NMF) has been widely used in capturing the parts-based feature for image representation. Previous NMF variants for image clustering first set the reduced rank as the number of clusters to obtain image representations. Then, K-means technology is adopted for ultimate clustering. However, the setting of rank is too rigid to preserve the geometrical structure of images, leading to unsuitable representation for the following clustering. Moreover, the image representation learning and clustering procedures are conducted individually which is inefficient as well as suboptimal for image clustering. This paper incorporates the constraint of topological structure into the procedure of NMF for simultaneously accomplishing nonnegative image representation and image clustering task. The proposed topological structure regularized NMF (TNMF) makes the rank bigger than the number of clusters, allotting one image more than one coarse clusters to guarantee the geometrical structure of images. In addition, a connected graph constructed by the coefficient matrix is enforced to have the strong connected components whose number equals to the number of clusters. By checking the number of connected components of graph against the number of clusters during the learning procedure, the optimization of TNMF is accomplished using alternative update rules. Extensive experiments on the challenging face, digit, and object data sets demonstrate the effectiveness of TNMF compared with competitive NMF variants for image clustering.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. In this paper, we call the columns of learned basis matrix as coarse centroids when the rank is bigger than the required number of clusters.

  2. The theoretical computation complexity of the eigenvalue decomposition is reported in this paper. However, the eigenvalue decomposition operation can be accelerated due to the property of the Laplacian matrix, symmetric and sparse.

  3. http://www.cad.zju.edu.cn/home/dengcai/Data/Clustering.html.

  4. http://www.cad.zju.edu.cn/home/dengcai/Data/MLData.html.

  5. http://www.cad.zju.edu.cn/home/dengcai/Data/FaceData.html.

  6. http://www.cad.zju.edu.cn/home/dengcai/Data/MLData.html.

  7. http://www.cad.zju.edu.cn/home/dengcai/Data/Clustering.html.

References

  1. Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401:788–791

    Article  Google Scholar 

  2. Lee DD, Seung HS (2000) Algorithms for non-negative matrix factorization. NIPS, Barcelona

    Google Scholar 

  3. Zeng K, Yu J, Li CH, You J, Jin TS (2014) Image clustering by hyper-graph regularized non-negative matrix factorization. Neurocomputing 138(11):209–217

    Article  Google Scholar 

  4. Yi YG, Shi YJ, Zhang HJ, Wang JZ, Kong J (2015) Label propagation based semi-supervised non-negative matrix factorization for feature extraction. Neurocomputing 149(PB):1021–1037

    Article  Google Scholar 

  5. Wang ZF, Kong XW, Fu HY, Li M, Zhang YJ (2015) Feature extraction via multi-view non-negative matrix factorization with local graph regularization. In: ICIP

  6. Yang SZ, Hou CP, Zhang CS, Wu Y (2013) Robust non-negative matrix factorization via joint sparse and graph regularization for transfer learning. Neural Comput Appl 23(2):541–559

    Article  Google Scholar 

  7. Tong M, Bu H, Zhao M, Xi S, Li H (2017) Non-negative enhanced discriminant matrix factorization method with sparsity regularization. Neural Comput Appl 2:1–24

    Google Scholar 

  8. Lu YW, Lai ZH, Xu Y, Li XL, Zhang D, Yuan C (2017) Nonnegative discriminant matrix factorization. IEEE Trans Circuits Syst Video Technol 27(7):1392–1405

    Article  Google Scholar 

  9. Akashi Y, Okatani T (2016) Separation of reflection components by sparse non-negative matrix factorization. Comput Vis Image Underst 146:77–85

    Article  Google Scholar 

  10. He W, Zhang HY, Zhang LP (2016) Sparsity-regularized robust non-negative matrix factorization for hyperspectral unmixing. IEEE J Sel Topics Appl Earth Observ Remote Sens 9(9):4267–4279

    Article  Google Scholar 

  11. Wu ZB, Ye S, Liu JJ, Sun L, Wei ZH (2014) Sparse non-negative matrix factorization on GPUs for hyperspectral unmixing. IEEE J Sel Topics Appl Earth Observ Remote Sens 7(8):3640–3649

    Article  Google Scholar 

  12. Duda RO, Hart PE, Stork DG (2000) Pattern classification. Wiley-Interscience, Hoboken

    MATH  Google Scholar 

  13. Gersho A, Gray RM (1992) Vector quantization and signal compression. Kluwer Academic Press, Norwell

    Book  Google Scholar 

  14. Zhang HJ, Wu QMJ, Chow TWS, Zhao MB (2012) A two-dimensional neighborhood preserving projection for appearance-based face recognition. Pattern Recogn 45(5):1866–1876

    Article  Google Scholar 

  15. Zhu WJ, Yan YH, Peng YS (2017) Pair of projections based on sparse consistence with applications to efficient face recognition. Sig Process Image Commun 55:32–40

    Article  Google Scholar 

  16. Hartigan JA, Wong MA (1979) Algorithm AS 136: a K-means clustering algorithm. J R Stat Soc 28(1):100–108

    MATH  Google Scholar 

  17. Buono ND, Pio G (2015) Non-negative matrix Tri-Factorization for co-clustering: an analysis of the block matrix. Inf Sci 301:13–26

    Article  Google Scholar 

  18. Bampis CG, Maragos P, Bovik AC (2016) Projective non-negative matrix factorization for unsupervised graph clustering. In: ICIP

  19. Dera D, Bouaynaya N, Polikar R, Fathallah-Shaykh HM (2016) Non-negative matrix factorization for non-parametric and unsupervised image clustering and segmentation. In: IJCNN

  20. Li XL, Cui GS, Dong YS (2017) Graph regularized non-negative low-rank matrix factorization for image clustering. IEEE Trans Cybern 47(11):3840–3853

    Article  MathSciNet  Google Scholar 

  21. Li SZ, Hou XW, Zhang HJ, Cheng QS (2003) Learning spatially localized, parts-based representation. In: CVPR

  22. Yang ZR, Yuan ZJ, Laaksonen J (2007) Projective nonnegative matrix factorization with applications to facial image processing. Int J Pattern Recognit Artif Intell 21(8):1353–1362

    Article  Google Scholar 

  23. Yuan ZJ, Yang ZR, Oja E (2005) Projective nonnegative matrix factorization for image compression and feature extraction. In: Scandinavian Conference on Image Analysis, pp 333–342

  24. Choi S (2008) Algorithms for orthogonal nonnegative matrix factorization. In: IJCNN

  25. Ding C, Li T, Peng W, Park H (2006) Orthogonal nonnegative matrix tri-factorizations for clustering. In: ACM Sigkdd International Conference on Knowledge Discovery and Data Mining

  26. Pompili F, Gillis N, Absil PA, Glineur F (2012) Two algorithms for orthogonal nonnegative matrix factorization with application to clustering. Neurocomputing 141(4):15–25

    Google Scholar 

  27. Cai D, He XF, Han JW, Huang TS (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560

    Article  Google Scholar 

  28. Zeng K, Yu J, Li CH, You J, Jin TS (2014) Image clustering by hyper-graph regularized non-negative matrix factorization. Neurocomputing 138(11):209–217

    Article  Google Scholar 

  29. Liu HF, Wu ZH, Cai D, Huang TS (2012) Constrained nonnegative matrix factorization for image representation. IEEE Trans Pattern Anal Mach Intell 34(7):1299–1311

    Article  Google Scholar 

  30. Li GP, Zhang XY, Zheng SY, Li DY (2017) Semi-supervised convex nonnegative matrix factorizations with graph regularized for image representation. Neurocomputing 237:1–11

    Article  Google Scholar 

  31. He XF, Cai D, Yan SC, Zhang HJ (2005) Neighborhood preserving embedding. In: ICCV

  32. Ding C, He XF, Horst DS, Jin R (2005) On the equivalence of nonnegative matrix factorization and K-means-spectral clustering, Factorization

  33. Mohar B, Alavi Y, Chartrand G, Oellermann OR, Schwenk AJ (1991) The Laplacian spectrum of graphs. Graph Theory Comb Appl 18(7):871–898

    MathSciNet  Google Scholar 

  34. Fan K (1949) On a theorem of Weyl concerning eigenvalues of linear transformations II. Proc Natl Acad Sci USA 36(1):652–655

    Article  Google Scholar 

  35. Mairal J, Bach F, Ponce J, Sapiro G (2009) Online dictionary learning for sparse coding. In: ICML

  36. Mairal J, Bach F, Ponce J, Sapiro G (2009) Online learning for matrix factorization and sparse coding. J Mach Learn Res 11(1):19–60

    MathSciNet  MATH  Google Scholar 

  37. Hoyer PO (2002) Non-negative sparse coding. In: IEEE Workshop on Neural Networks for Signal Processing

  38. Yang J, Chu DL, Zhang L, Xu Y, Yang JY (2013) Sparse representation classifier steered discriminative projection with applications to face recognition. IEEE Trans Neural Netw Learn Syst 24(7):1023–1035

    Article  Google Scholar 

  39. Cai D, He XF, Han JW (2005) Document clustering using locality preserving indexing. IEEE Trans Knowl Data Eng 17(12):1624–1637

    Article  Google Scholar 

  40. Chen XL, Cai D (2011) Large scale spectral clustering with landmark-based representation. In: AAAI conference on artificial intelligence

  41. Zhang HJ, Cao X, Ho JKL, Chow TWS (2017) Object-level video advertising: an optimization framework. IEEE Trans Industr Inf 13(2):520–531

    Article  Google Scholar 

  42. Oyedotun OK, Khashman A (2016) Deep learning in vision-based static hand gesture recognition. Neural Comput Appl 28(12):1–11

    Google Scholar 

Download references

Acknowledgements

This work is supported by the National Key Research and Development Program of China (2017YFB0304200), the National Natural Science Foundation (NNSF) (51374063), and the Fundamental Research Funds for the Central Universities (N150308001).

Author information

Authors and Affiliations

  1. School of Mechanical Engineering and Automation, Northeastern University, Shenyang, China

    Wenjie Zhu & Yunhui Yan

  2. School of Mechanical Engineering, Hunan Institute of Science and Technology, Yueyang, China

    Yishu Peng

Authors
  1. Wenjie Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunhui Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yishu Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenjie Zhu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, W., Yan, Y. & Peng, Y. Topological structure regularized nonnegative matrix factorization for image clustering. Neural Comput & Applic 31, 7381–7399 (2019). https://doi.org/10.1007/s00521-018-3572-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-018-3572-4

Keywords

Search

Navigation