Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Intelligent Data Analysis

IDA 2018: Advances in Intelligent Data Analysis XVII pp 137–150Cite as

Non-negative Local Sparse Coding for Subspace Clustering

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11191))

Abstract

Subspace sparse coding (SSC) algorithms have proven to be beneficial to the clustering problems. They provide an alternative data representation in which the underlying structure of the clusters can be better captured. However, most of the research in this area is mainly focused on enhancing the sparse coding part of the problem. In contrast, we introduce a novel objective term in our proposed SSC framework which focuses on the separability of data points in the coding space. We also provide mathematical insights into how this local-separability term improves the clustering result of the SSC framework. Our proposed non-linear local SSC algorithm (NLSSC) also benefits from the efficient choice of its sparsity terms and constraints. The NLSSC algorithm is also formulated in the kernel-based framework (NLKSSC) which can represent the nonlinear structure of data. In addition, we address the possibility of having redundancies in sparse coding results and its negative effect on graph-based clustering problems. We introduce the link-restore post-processing step to improve the representation graph of non-negative SSC algorithms such as ours. Empirical evaluations on well-known clustering benchmarks show that our proposed NLSSC framework results in better clusterings compared to the state-of-the-art baselines and demonstrate the effectiveness of the link-restore post-processing in improving the clustering accuracy via correcting the broken links of the representation graph.

This research was supported by the Cluster of Excellence Cognitive Interaction Technology ‘CITEC’ (EXC 277) at Bielefeld University, which is funded by the German Research Foundation (DFG).

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Ana, L., Jain, A.K.: Robust data clustering. In: IEEE Conference on Computer Vision and Pattern Recognition. vol. 2, pp. II-128. IEEE (2003)

    Google Scholar 

  2. Bian, X., Li, F., Ning, X.: Kernelized sparse self-representation for clustering and recommendation. In: SIAM International Conference on Data Mining, pp. 10–17 (2016)

    Google Scholar 

  3. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends\({\textregistered }\) Mach. Learn. 3(1), 1–122 (2011)

    Article  Google Scholar 

  4. Cai, J.F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010)

    Article  MathSciNet  Google Scholar 

  5. Cheng, B., Yang, J., Yan, S., Fu, Y., Huang, T.S.: Learning with l1-graph for image analysis. Trans. Img. Proc. 19(4), 858–866 (2010). Apr

    Article  Google Scholar 

  6. Elhamifar, E., Vidal, R.: Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2765–2781 (2013)

    Article  Google Scholar 

  7. Gao, S., Tsang, I.W.h., Chia, L.t.: Laplacian sparse coding, hypergraph laplacian sparse coding, and applications. IEEE TPAMI 35(1), 92–104 (2012)

    Article  Google Scholar 

  8. Georghiades, A.S., Belhumeur, P.N., Kriegman, D.J.: From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 643–660 (2001)

    Article  Google Scholar 

  9. Hoyer, P.O.: Modeling receptive fields with non-negative sparse coding. Neurocomputing 52, 547–552 (2003)

    Article  Google Scholar 

  10. Kim, T., Shakhnarovich, G., Urtasun, R.: Sparse coding for learning interpretable spatio-temporal primitives. In: NIPS’10, pp. 1117–1125 (2010)

    Google Scholar 

  11. Kirrinnis, P.: Fast algorithms for the sylvester equation ax- xbt= c. Theor. Comput. Sci. 259(1–2), 623–638 (2001)

    Article  MathSciNet  Google Scholar 

  12. Li, C.G., Vidal, R., et al.: Structured sparse subspace clustering: a unified optimization framework. In: CVPR, pp. 277–286 (2015)

    Google Scholar 

  13. Li, X., Cui, G., Dong, Y.: Graph regularized non-negative low-rank matrix factorization for image clustering. IEEE Trans. Cybern. 47(11), 3840–3853 (2017)

    Article  MathSciNet  Google Scholar 

  14. Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y.: Robust recovery of subspace structures by low-rank representation. IEEE TPAMI 35(1), 171–184 (2013)

    Article  Google Scholar 

  15. Martinez, A.M.: The AR face database. CVC Technical Report 24 (1998)

    Google Scholar 

  16. Nene, S.A., Nayar, S.K., Murase, H., et al.: Columbia object image library (coil-20) (1996)

    Google Scholar 

  17. Patel, V.M., Vidal, R.: Kernel sparse subspace clustering. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 2849–2853. IEEE (2014)

    Google Scholar 

  18. Rubinstein, R., Zibulevsky, M., Elad, M.: Efficient implementation of the k-svd algorithm using batch orthogonal matching pursuit. Cs Tech. 40(8), 1–15 (2008)

    Google Scholar 

  19. Tron, R., Vidal, R.: A benchmark for the comparison of 3-d motion segmentation algorithms. In: CVPR’07, pp. 1–8. IEEE (2007)

    Google Scholar 

  20. Vidal, R., Favaro, P.: Low rank subspace clustering (lrsc). Pattern Recog. Lett. 43, 47–61 (2014)

    Article  Google Scholar 

  21. Von Luxburg, U.: A tutorial on spectral clustering. Stat. Comput. 17(4), 395–416 (2007)

    Google Scholar 

  22. Wu, J., Rehg, J.M.: Beyond the Euclidean distance: creating effective visual codebooks using the histogram intersection kernel. In: ICCV 2009. pp. 630–637. IEEE (2009)

    Google Scholar 

  23. Xiao, S., Tan, M., Xu, D., Dong, Z.Y.: Robust kernel low-rank representation. IEEE Trans. Neural Netw. Learn. Syst. 27(11), 2268–2281 (2016)

    Article  MathSciNet  Google Scholar 

  24. Xu, R., Wunsch, D.: Survey of clustering algorithms. IEEE Trans. Neural Netw. 16(3), 645–678 (2005)

    Article  Google Scholar 

  25. Xu, S., Chan, K.S., Zhou, T., Gao, J., Li, X., Hua, X.: A novel cluster ensemble approach effected by subspace similarity. Intell. Data Anal. 20(3), 561–574 (2016)

    Article  Google Scholar 

  26. Yang, Y., Wang, Z., Yang, J., Wang, J., Chang, S., Huang, T.S.: Data clustering by laplacian regularized 11-graph. In: AAAI, pp. 3148–3149 (2014)

    Google Scholar 

  27. You, C., Robinson, D., Vidal, R.: Scalable sparse subspace clustering by orthogonal matching pursuit. In: Computer Vision and Pattern Recognition (CVPR), pp. 3918–3927 (2016)

    Google Scholar 

  28. Zhuang, L., Gao, H., Lin, Z., Ma, Y., Zhang, X., Yu, N.: Non-negative low rank and sparse graph for semi-supervised learning. In: CVPR 2012, pp. 2328–2335. IEEE (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CITEC centre of excellence, Bielefeld University, Bielefeld, Germany

    Babak Hosseini & Barbara Hammer

Authors
  1. Babak Hosseini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Barbara Hammer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Babak Hosseini .

Editor information

Editors and Affiliations

  1. Eindhoven University of Technology, Eindhoven, The Netherlands

    Wouter Duivesteijn

  2. Department of Information and Computing Sciences, University Utrecht, Utrecht, The Netherlands

    Arno Siebes

  3. University of Helsinki, Helsinki, Finland

    Antti Ukkonen

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hosseini, B., Hammer, B. (2018). Non-negative Local Sparse Coding for Subspace Clustering. In: Duivesteijn, W., Siebes, A., Ukkonen, A. (eds) Advances in Intelligent Data Analysis XVII. IDA 2018. Lecture Notes in Computer Science(), vol 11191. Springer, Cham. https://doi.org/10.1007/978-3-030-01768-2_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-01768-2_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-01767-5

  • Online ISBN: 978-3-030-01768-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation