Skip to main content
SpringerLink
Log in

Symmetric nonnegative matrix factorization with elastic-net regularized block-wise weighted representation for clustering

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

In unsupervised learning, symmetric nonnegative matrix factorization (NMF) has proven its efficacy for various clustering tasks in recent years, considering both linearly and nonlinearly separable data. On the other hand, block-wise weighted sparse representation-based classification (BW-SRC), a recently proposed sparse representation technique improved sparse coding features for supervised classification. In this work, we take advantage of both techniques to device a new unsupervised image clustering algorithm. A disadvantage of symmetric NMF is its computational burden associated to the quadratic growth of its similarity matrix. We reduce this computation by first working with sub-sampled data and then by working with the full data samples in a second stage elastic-net coefficient estimation problem with previously learned block-wise weights. This decreases both the computational time and the memory requirements when solving for symmetric NMF, but at the same time allows to cluster the full data samples in a robust way. We either outperform or achieve highly competitive results with previous matrix factorization clustering methods on seven benchmark image datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Ramirez I, Sprechmann P, Sapiro G (2010) Classification and clustering via dictionary learning with structured incoherence and shared features. In: 2010 IEEE Computer Society conference on computer vision and pattern recognition, pp 3501–3508

  2. Xu J, Xu K, Chen K, Ruan J (2015) Reweighted sparse subspace clustering. Comput Vis Image Underst 138:25–37. https://doi.org/10.1016/j.cviu.2015.04.003

    Article  Google Scholar 

  3. Zhang R, Lu Z (2016) Large scale sparse clustering. In: Proceedings of the twenty-fifth international joint conference on artificial intelligence, IJCAI’16. AAAI Press, pp 2336–2342

  4. Feng X (2018) Spectral clustering via sparse representation. IntechOpen. https://doi.org/10.5772/intechopen.76586

    Article  Google Scholar 

  5. Dong W, Wu XJ, Kittler J, Yin HF (2019) Sparse subspace clustering via nonconvex approximation. Pattern Anal Appl 22:165–176. https://doi.org/10.1007/s10044-018-00774-z

    Article  MathSciNet  Google Scholar 

  6. Kuang D, Yun S, Park H (2015) SymNMF: nonnegative low-rank approximation of a similarity matrix for graph clustering. J Glob Optim 62:545–574. https://doi.org/10.1007/s10898-014-0247-2

    Article  MathSciNet  MATH  Google Scholar 

  7. Meng Y, Shang R, Jiao L, Zhang W, Yuan Y, Yang S (2018) Feature selection based dual-graph sparse non-negative matrix factorization for local discriminative clustering. Neurocomputing 290:87–99. https://doi.org/10.1016/j.neucom.2018.02.044

    Article  Google Scholar 

  8. Zhu Z, Li X, Liu K, Li Q (2018) Dropping symmetry for fast symmetric nonnegative matrix factorization. In: Proceedings of the 32nd international conference on neural information processing systems, NIPS’18. Curran Associates Inc., Red Hook, NY, USA, pp 5160–5170

  9. Wei L, Zhou R, Zhu C, Zhang X, Yin J (2020) Adaptive graph-regularized fixed rank representation for subspace segmentation. Pattern Anal Appl 23:443–453. https://doi.org/10.1007/s10044-019-00786-3

    Article  MathSciNet  Google Scholar 

  10. Liu H, Shao M, Li S, Fu Y (2016) Infinite ensemble for image clustering. KDD ’16. Association for Computing Machinery, New York, pp 1745–1754. https://doi.org/10.1145/2939672.2939813

    Book  Google Scholar 

  11. Huang D, Wang C, Lai J (2018) Locally weighted ensemble clustering. IEEE Trans Cybern 48(5):1460–1473

    Article  Google Scholar 

  12. Huang D, Wang C, Wu J, Lai J, Kwoh C (2020) Ultra-scalable spectral clustering and ensemble clustering. IEEE Trans Knowl Data Eng 32(6):1212–1226

    Article  Google Scholar 

  13. Zhou W, Zhou Q (2019) Deep embedded clustering with adversarial distribution adaptation. IEEE Access 7:113801–113809

    Article  Google Scholar 

  14. Feng Q, Chen L, Chen CLP, Guo L (2020) Deep fuzzy clustering-a representation learning approach. IEEE Trans Fuzzy Syst 28(7):1420–1433

    Google Scholar 

  15. Chang J, Meng G, Wang L, Xiang S, Pan C (2020) Deep self-evolution clustering. IEEE Trans Pattern Anal Mach Intell 42(4):809–823

    Article  Google Scholar 

  16. Niu C, Zhang J, Wang G, Liang J (2020) Gatcluster: self-supervised gaussian-attention network for image clustering. In: Vedaldi A, Bischof H, Brox T, Frahm JM (eds) Computer vision—ECCV 2020. Springer International Publishing, Cham, pp 735–751

    Chapter  Google Scholar 

  17. Rodríguez-Domínguez U, Dalmau O (2020) Block-wise weighted sparse representation-based classification. Signal Image Video Process. https://doi.org/10.1007/s11760-020-01700-9

    Article  Google Scholar 

  18. Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227. https://doi.org/10.1109/TPAMI.2008.79

    Article  Google Scholar 

  19. Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J R Stat Soc Ser B (Stat Methodol) 67(2):301–320

    Article  MathSciNet  Google Scholar 

  20. Lovász L, Plummer M (2009) Matching theory, vol 367. AMS Chelsea Publishing, New York

    MATH  Google Scholar 

  21. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: 2012 IEEE conference on computer vision and pattern recognition, pp 2066–2073. https://doi.org/10.1109/CVPR.2012.6247911

  22. Samaria FS, Harter AC (1994) Parameterisation of a stochastic model for human face identification. In: Proceedings of 1994 IEEE workshop on applications of computer vision, pp 138–142 https://doi.org/10.1109/ACV.1994.341300

  23. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324. https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  24. Li Fei-Fei, Fergus R, Peron P (2004) Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. In: 2004 conference on computer vision and pattern recognition workshop, pp 178–178 https://doi.org/10.1109/CVPR.2004.383

  25. Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T(2014) Decaf: a deep convolutional activation feature for generic visual recognition. In: Proceedings of the 31st international conference on international conference on machine learning, ICML’14, vol 32. JMLR.org, pp I-647–I-655

  26. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis (IJCV) 115(3):211–252. https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  27. Tian F, Gao B, Cui Q, Chen E, Liu TY (2014) Learning deep representations for graph clustering. In: Proceedings of the twenty-eighth AAAI conference on artificial intelligence, AAAI’14. AAAI Press, pp 1293–1299

  28. Liu H, Liu T, Wu J, Tao D, Fu Y (2015) Spectral ensemble clustering. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’15. Association for Computing Machinery, New York, NY, USA, pp 715–724. https://doi.org/10.1145/2783258.2783287

  29. Shao M, Li S, Ding Z, Fu Y (2015) Deep linear coding for fast graph clustering. In: Proceedings of the 24th international conference on artificial intelligence, IJCAI’15. AAAI Press, pp 3798–3804

Download references

Acknowledgements

This work was supported partly by CONACYT (Mexico) Grant 258033.

Author information

Authors and Affiliations

  1. Mathematics Research Center, Guanajuato, Guanajuato, Mexico

    Ulises Rodríguez-Domínguez & Oscar Dalmau

Authors
  1. Ulises Rodríguez-Domínguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Oscar Dalmau
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ulises Rodríguez-Domínguez.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rodríguez-Domínguez, U., Dalmau, O. Symmetric nonnegative matrix factorization with elastic-net regularized block-wise weighted representation for clustering. Pattern Anal Applic 25, 807–817 (2022). https://doi.org/10.1007/s10044-022-01062-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-022-01062-7

Keywords

Search

Navigation