Skip to main content
SpringerLink
Log in

Graph Based Semi-Supervised Learning via Structure Preserving Low-Rank Representation

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Semi-supervised learning works on utilizing both labeled and unlabeled data to improve learning performance, which has been receiving increasing attention in many applications such as clustering and classification. In this paper, we focus on the semi-supervised learning methods developed on data graph whose edge weights are measured by low-rank representation (LRR) coefficients. Specifically, we impose two constraints on LRR when constructing the graph: local affinity and distant repulsion, to preserve the data manifold information. The proposed model, termed structure preserving LRR (SPLRR), can preserve the local geometrical structure and without distorting the distant repulsion property. Using the augmented Lagrange multiplier (ALM) method framework, we derive an efficient approach to optimizing the SPLRR model. Experiments are conducted on four widely used data sets to validate the effectiveness of our proposed SPLRR model and the results demonstrate that SPLRR is an excellent model for graph based semi-supervised learning in comparison with the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. http://www.uk.research.att.com/facedatabase.html.

  2. http://vision.ucsd.edu/~leekc/ExtYaleDatabase/ExtYaleB.html.

  3. http://www.ri.cmu.edu/projects/project_418.html.

  4. http://archive.ics.uci.edu/ml/datasets/ISOLET.

References

  1. Belkin M, Niyogi P (2001) Laplacian eigenmaps and spectral techniques for embedding and clustering. Adv Neural Inf Process Syst 14:585–591

    Google Scholar 

  2. Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396

    Article  MATH  Google Scholar 

  3. Cai D, He X, Han J (2011) Locally consistent concept factorization for document clustering. IEEE Trans Knowl Data Eng 23(6):902–913

    Article  Google Scholar 

  4. Cai D, He X, Han J, Huang TS (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560

    Article  Google Scholar 

  5. Cai JF, Candès EJ, Shen Z (2010) A singular value thresholding algorithm for matrix completion. SIAM J Optim 20(4):1956–1982

    Article  MATH  MathSciNet  Google Scholar 

  6. Candès EJ, Li X, Ma Y, Wright J (2011) Robust principal component analysis? J ACM 58(3):11

    Article  MathSciNet  Google Scholar 

  7. Carreira-Perpinán MA (2010) The elastic embedding algorithm for dimensionality reduction. In: International conference on machine learning, pp 167–174

  8. Cheng B, Yang J, Yan S, Fu Y, Huang TS (2010) Learning with \(\ell ^1\)-graph for image analysis. IEEE Trans Image Process 19(4):858–866

    Article  MathSciNet  Google Scholar 

  9. Cortes C, Mohri M, Mohri M (2006) On transductive regression. In: Proceedings of the second international conference on advances in neural information processing systems, pp 305–312

  10. Hadsell R, Chopra S, LeCun Y (2006) Dimensionality reduction by learning an invariant mapping. IEEE Conf Comput Vision Pattern Recogn 2:1735–1742

    Google Scholar 

  11. He R, Zheng WS, Hu BG, Kong XW (2011) Nonnegative sparse coding for discriminative semi-supervised learning. In: IEEE international conference on computer vision and pattern recognition, pp 2849–2856

  12. He X, Cai D, Shao Y, Bao H, Han J (2011) Laplacian regularized gaussian mixture model for data clustering. IEEE Trans Knowl Data Eng 23(9):1406–1418

    Article  Google Scholar 

  13. Ji S, Ye J (2009) An accelerated gradient method for trace norm minimization. In: Proceedings of the 26th annual international conference on machine learning, pp 457–464

  14. Koh K, Kim S, Boyd S (2008) \(\ell 1\_\)ls: A matlab solver for large-scale \(\ell \)1-regularized least squares problems. Stanford University, Stanford

  15. Lee JM (2003) Smooth manifolds. Springer-Verlag, New York

    Book  Google Scholar 

  16. Li Z, Liu J, Lu H (2013) Structure preserving non-negative matrix factorization for dimensionality reduction. Comput Vision Image Underst 117(9):1175–1189

    Article  Google Scholar 

  17. Lin Z, Chen M, Ma Y (2010) The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv preprint arXiv:1009.5055

  18. Liu G, Lin Z, Yu Y (2010) Robust subspace segmentation by low-rank representation. In: International conference on machine learning, pp 663–670

  19. Lu X, Wang Y, Yuan Y (2013) Graph-regularized low-rank representation for destriping of hyperspectral images. IEEE Trans Geosci Remote Sens 51(7):4009–4018

    Article  Google Scholar 

  20. Peng Y, Wang S, Wang S, Lu BL (2013) Structure preserving low-rank representation for semi-supervised face recognition. In: International conference on neural information processing, pp 148–155

  21. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  22. Tenenbaum JB, De Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323

    Article  Google Scholar 

  23. Wang F, Zhang C (2006) Label propagation through linear neighborhoods. In: International conference on machine learning, pp 985–992

  24. Wang F, Zhang C (2008) Label propagation through linear neighborhoods. IEEE Trans Knowl Data Eng 20(1):55–67

    Article  Google Scholar 

  25. Yang A, Zhou Z, Balasubramanian A, Sastry S, Ma Y (2012) Fast \(\ell \)1-minimization algorithms for robust face recognition. IEEE Trans Image Process 22(8):3234–3246

    Article  Google Scholar 

  26. Yang S, Wang X, Wang M, Han Y, Jiao L (2013) Semi-supervised low-rank representation graph for pattern recognition. IET Image Process 7(2):131–136

    Article  MathSciNet  Google Scholar 

  27. Zhang Z, Zha H (2004) Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. SIAM J Sci Comput 26(1):313–338

    Article  MATH  MathSciNet  Google Scholar 

  28. Zheng M, Bu J, Chen C, Wang C, Zhang L, Qiu G, Cai D (2011) Graph regularized sparse coding for image representation. IEEE Trans Image Process 20(5):1327–1336

    Article  MathSciNet  Google Scholar 

  29. Zheng Z, Zhang H, Jia J, Zhao J, Guo L, Fu F, Yu M (2013) Low-rank matrix recovery with discriminant regularization. In: Proceedings of Pacific-Asia conference on knowledge discovery and data mining, pp 437–448

  30. Zhou D, Bousquet O, Lal TN, Weston J, Schölkopf B (2004) Learning with local and global consistency. Adv Neural Inf Process Syst 16:321–328

    Google Scholar 

  31. Zhu X (2008) Semi-supervised learning literature survey. Technical Report, Department of Computer Science, University of Wiconsin, Madison

Download references

Acknowledgments

This work was supported in part by the National Basic Research Program of China (Grant No. 2013CB329401), the National Natural Science Foundation of China (Grant No. 61272248), the Science and Technology Commission of Shanghai Municipality (Grant No. 13511500200) and the European Union Seventh Framework Program (Grant No. 247619). The first author was supported by China Scholarship Council (Grant No. 201206230012) and this work was mainly done when he was visiting the EECS Department, University of Michigan, Ann Arbor.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China

    Yong Peng & Xianzhong Long

  2. Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Department of Computer Science and Engineering, Center for Brain-like Computing and Machine Intelligence, Shanghai Jiao Tong University, Shanghai, 200240, China

    Bao-Liang Lu

Authors
  1. Yong Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xianzhong Long
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bao-Liang Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bao-Liang Lu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, Y., Long, X. & Lu, BL. Graph Based Semi-Supervised Learning via Structure Preserving Low-Rank Representation. Neural Process Lett 41, 389–406 (2015). https://doi.org/10.1007/s11063-014-9396-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-014-9396-z

Keywords

Search

Navigation