Skip to main content
SpringerLink
Log in

Robust and sparse label propagation for graph-based semi-supervised classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Traditional graph-based semi-supervised classification algorithms are usually composed of two independent parts: graph construction and label propagation. However, the predefined graph may not be optimal for label propagation, and these methods usually use the raw data containing noise directly, which may reduce the accuracy of the algorithm. In this paper, we propose a robust label prediction model called the robust and sparse label propagation (RSLP) algorithm. First, our RSLP algorithm decomposes the raw data into a low-rank clean part and a sparse noise part, and performs graph construction and label propagation in the clean data space. Second, RSLP seamlessly combines the processes of graph construction and label propagation. By jointly minimizing the sample reconstruction error and the label reconstruction error, the resulting graph structure is globally optimal. Third, the proposed RSLP performs l2,1-norm regularization on the predicted label matrix, thereby enhancing the sparsity and discrimination of soft labels. We also analyze the connection between RSLP and other related algorithms, including label propagation algorithms, the robust graph construction method, and principal component analysis. A series of experiments on several benchmark datasets show that our RSLP algorithm achieves comparable and even higher accuracy than other state-of-the-art algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Appice A, Guccione P, Malerba D (2017) A novel spectral-spatial co-training algorithm for the transductive classification of hyperspectral imagery data. Pattern Recogn 63:229–245. https://doi.org/10.1016/j.patcog.2016.10.010

    Article  Google Scholar 

  2. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers

  3. Candès EJ, Li X, Ma Y, Wright J (2011) Robust principal component analysis. J ACM 58(3):11. https://doi.org/10.1145/1970392.1970395

    Article  MathSciNet  Google Scholar 

  4. Chen D, Wang W, Gao W, Zhou Z (2018) Tri-net for semi-supervised deep learning. In: Lang J (ed) Proceedings of the twenty-seventh international joint conference on artificial intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, ijcai.org, 2014-2020. https://doi.org/10.24963/ijcai.2018/278

  5. Cheng B, Yang J, Yan S, Fu Y, Huang TS (2010) Learning with ℓ1-graph for image analysis. IEEE Trans Image Process 19(4):858–866. https://doi.org/10.1109/TIP.2009.2038764

    Article  MathSciNet  Google Scholar 

  6. Cheng G, Han J, Lu X (2017) Remote sensing image scene classification: Benchmark and state of the art. Proc IEEE 105(10):1865–1883. https://doi.org/10.1109/JPROC.2017.2675998

    Article  Google Scholar 

  7. Das D, Lee CSG (2020) A two-stage approach to few-shot learning for image recognition. IEEE Trans Image Process 29:3336–3350. https://doi.org/10.1109/TIP.2019.2959254

    Article  Google Scholar 

  8. Dornaika F, Traboulsi YE (2019) Joint sparse graph and flexible embedding for graph-based semi-supervised learning. Neural Netw 114:91–95. https://doi.org/10.1016/j.neunet.2019.03.002

    Article  Google Scholar 

  9. Dornaika F, Weng L (2019) Sparse graphs with smoothness constraints: Application to dimensionality reduction and semi-supervised classification. Pattern Recogn 95:285–295. https://doi.org/10.1016/j.patcog.2019.06.015

    Article  Google Scholar 

  10. Dornaika F, Dahbi R, Bosaghzadeh A, Ruichek Y (2017) Efficient dynamic graph construction for inductive semi-supervised learning. Neural Netw 94:192–203. https://doi.org/10.1016/j.neunet.2017.07.006

    Article  Google Scholar 

  11. Du B, Xinyao T, Wang Z, Zhang L, Tao D (2018) Robust graph-based semisupervised learning for noisy labeled data via maximum correntropy criterion. IEEE Trans Cybern 49(4):1440–1453. https://doi.org/10.1109/TCYB.2018.2804326

    Article  Google Scholar 

  12. Fang X, Han N, Wong WK, Teng S, Wu J, Xie S, Li X (2019) Flexible affinity matrix learning for unsupervised and semisupervised classification. IEEE Trans Neural Netw 30(4):1133–1149. https://doi.org/10.1109/TNNLS.2018.2861839

    Article  MathSciNet  Google Scholar 

  13. Fei L, Xu Y, Fang X, Yang J (2017) Low rank representation with adaptive distance penalty for semi-supervised subspace classification. Pattern Recogn 67:252–262. https://doi.org/10.1016/j.patcog.2017.02.017

    Article  Google Scholar 

  14. Fox-Roberts P, Rosten E (2014) Unbiased generative semi-supervised learning. J Mach Learn Res 15(1):367–443

    MathSciNet  MATH  Google Scholar 

  15. Gong C, Tao D, Liu W, Liu L, Yang J (2017) Label propagation via teaching-to-learn and learning-to-teach. IEEE Trans Neural Netw Learn Syst 28(6):1452–1465. https://doi.org/10.1109/TNNLS.2016.2514360

    Article  Google Scholar 

  16. Hua Z (2020) Yang, Y. Node influence-based label propagation algorithm for semi-supervised learning. Neural Comput Appl, Qiu, H. https://doi.org/10.1007/s00521-020-05078-0

    Google Scholar 

  17. Kang Z, Pan H, Hoi SCH, Xu Z (2019) Robust graph learning from noisy data. IEEE Trans Syst Man Cybern 1–11. https://doi.org/10.1109/TCYB.2018.2887094

  18. Kingma DP, Mohamed S, Rezende DJ, Welling M (2014) Semi-supervised learning with deep generative models. In: Advances in neural information processing systems, pp 3581–3589

  19. Liu C, Hsaio W, Lee C, Chang T, Kuo T (2016) Semi-supervised text classification with universum learning. IEEE Trans Cybern 46(2):462–473. https://doi.org/10.1109/TCYB.2015.2403573

    Article  Google Scholar 

  20. Liu W, He J, Chang SF (2010) Large graph construction for scalable semi-supervised learning. In: ICML. https://icml.cc/Conferences/2010/papers/16.pdf, pp 679–686

  21. Ma L, Ma A, Ju C, Li X (2016) Graph-based semi-supervised learning for spectral-spatial hyperspectral image classification. Pattern Recognit Lett 83:133–142. https://doi.org/10.1016/j.patrec.2016.01.022, advances in Pattern Recognition in Remote Sensing

    Article  Google Scholar 

  22. Nie F, Xiang S, Liu Y, Zhang C (2010) A general graph-based semi-supervised learning with novel class discovery. Neural Comput and Applic 19(4):549–555. https://doi.org/10.1007/s00521-009-0305-8

    Article  Google Scholar 

  23. Nie F, Xu D, Tsang IWH, Zhang C (2010) Flexible manifold embedding: A framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19(7):1921–1932. https://doi.org/10.1109/TIP.2010.2044958

    Article  MathSciNet  Google Scholar 

  24. Nie F, Shi S, Li X (2020) Semi-supervised learning with auto-weighting feature and adaptive graph. IEEE Trans Knowl Data Eng 32(6):1167–1178. https://doi.org/10.1109/TKDE.2019.2901853

    Article  Google Scholar 

  25. Peng Y, Lu BL, Wang S (2015) Enhanced low-rank representation via sparse manifold adaption for semi-supervised learning. Neural Netw 65:1–17. https://doi.org/10.1016/j.neunet.2015.01.001, https://www.sciencedirect.com/science/article/pii/S0893608015000027

    Article  Google Scholar 

  26. Qiao L, Chen S, Tan X (2010) Sparsity preserving projections with applications to face recognition. Pattern Recognit 43(1):331–341. https://doi.org/10.1016/j.patcog.2009.05.005. https://www.sciencedirect.com/science/article/pii/S0031320309001964

    Article  Google Scholar 

  27. Qiao L, Zhang L, Chen S, Shen D (2018) Data-driven graph construction and graph learning: A review. Neurocomputing 312:336–351. https://doi.org/10.1016/j.neucom.2018.05.084

    Article  Google Scholar 

  28. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326. https://doi.org/10.1126/science.290.5500.2323

    Article  Google Scholar 

  29. Shahid N, Kalofolias V, Bresson X, Bronstein M, Vandergheynst P (2015) Robust principal component analysis on graphs. In: 2015 IEEE international conference on computer vision (ICCV), pp 2812–2820

  30. de Sousa CA (2015) An overview on the gaussian fields and harmonic functions method for semi-supervised learning. In: 2015 international joint conference on neural networks (IJCNN). IEEE, pp 1–8. https://doi.org/10.1109/IJCNN.2015.7280491

  31. Wang F, Zhang C (2008) Label propagation through linear neighborhoods. IEEE Trans Knowl Data Eng 20(1):55–67. https://doi.org/10.1109/TKDE.2007.190672

    Article  MathSciNet  Google Scholar 

  32. Wang J, Wang F, Zhang C, Shen H, Quan L (2009) Linear neighborhood propagation and its applications. IEEE Trans Pattern Anal Mach Intell 31(9):1600–1615. https://doi.org/10.1109/TPAMI.2008.216

    Article  Google Scholar 

  33. Wang M, Fu W, Hao S, Tao D, Wu X (2016) Scalable semi-supervised learning by efficient anchor graph regularization. IEEE Trans Knowl Data Eng 28(7):1864–1877. https://doi.org/10.1109/TKDE.2016.2535367

    Article  Google Scholar 

  34. Wei D, Yang Y, Qiu H (2020) Improving self-training with density peaks of data and cut edge weight statistic. Soft Comput :1–16. https://doi.org/10.1007/s00500-020-04887-8

  35. Weng L, Dornaika F, Jin Z (2016) Graph construction based on data self-representativeness and laplacian smoothness. Neurocomputing 207:476–487. https://doi.org/10.1016/j.neucom.2016.05.021

    Article  Google Scholar 

  36. Wu D, Shang M, Luo X, Xu J, Yan H, Deng W, Wang G (2018) Self-training semi-supervised classification based on density peaks of data. Neurocomputing 275:180–191. https://doi.org/10.1016/j.neucom.2017.05.072

    Article  Google Scholar 

  37. Wu H, Prasad S (2018) Semi-supervised deep learning using pseudo labels for hyperspectral image classification. IEEE Trans Image Process 27(3):1259–1270. https://doi.org/10.1109/TIP.2017.2772836

    Article  MathSciNet  Google Scholar 

  38. Xie Y, Wang H, Yu B, Zhang C (2020) Secure collaborative few-shot learning. Knowl-Based Syst 106157:203. https://doi.org/10.1016/j.knosys.2020.106157

    Google Scholar 

  39. Yan S, Wang H (2009) Semi-supervised learning by sparse representation. pp 792–801. https://doi.org/10.1137/1.9781611972795.68

  40. Yu J, Kim SB (2018) Consensus rate-based label propagation for semi-supervised classification. Inform Sci 465:265–284. https://doi.org/10.1016/j.ins.2018.06.074

    Article  MathSciNet  Google Scholar 

  41. Zhang C, Wang S, Li D, Yang J, Chen H (2015) Prior class dissimilarity based linear neighborhood propagation. https://doi.org/10.1016/j.knosys.2015.03.011, vol 83, pp 58–65

  42. Zhang H, Zhang Z, Li S, Ye Q, Zhao M, Wang M (2018a) Robust adaptive label propagation by double matrix decomposition. In: 2018 24th International conference on pattern recognition (ICPR). pp 2160–2165. https://doi.org/10.1109/ICPR.2018.8545594

  43. Zhang Z, Zhang L, Zhao M, Jiang W, Liang Y, Li F (2015) Semi-supervised image classification by nonnegative sparse neighborhood propagation. In: Proceedings of the 5th ACM on international conference on multimedia retrieval. https://doi.org/10.1145/2671188.2749292, pp 139–146

  44. Zhang Z, Zhang Y, Li F, Zhao M, Zhang L, Yan S (2017) Discriminative sparse flexible manifold embedding with novel graph for robust visual representation and label propagation. Pattern Recogn 61:492–510. https://doi.org/10.1016/j.patcog.2016.07.042

    Article  Google Scholar 

  45. Zhang Z, Li F, Jia L, Qin J, Zhang L, Yan S (2018b) Robust adaptive embedded label propagation with weight learning for inductive classification. IEEE Trans Neural Netw Learn Syst 29(8):3388–3403. https://doi.org/10.1109/TNNLS.2017.2727526

    Article  MathSciNet  Google Scholar 

  46. Zheng Y, Zhang X, Yang S, Jiao L (2013) Low-rank representation with local constraint for graph construction. Neurocomputing 122:398–405

    Article  Google Scholar 

  47. Zhou D, Bousquet O, Lal TN, Weston J, Olkopf BS (2004) Learning with local and global consistency. In: Advances in neural information processing systems, vol 16

  48. Zhou ZH, Li M (2010) Semi-supervised learning by disagreement. Knowl Inf Syst 24(3):415–439. https://doi.org/10.1007/s10115-009-0209-z

    Article  Google Scholar 

  49. Zhu X, Ghahramani Z, Lafferty JD (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: Proceedings of the 20th international conference on machine learning (ICML-03), pp 912–919

  50. Zhu X (2005) Lafferty J, Semi-supervised learning with graphs, Rosenfeld R

  51. Zhuang L, Zhou Z, Gao S, Yin J, Lin Z, Ma Y (2017) Label information guided graph construction for semi-supervised learning. IEEE Trans Image Process 26(9):4182–4192. https://doi.org/10.1109/TIP.2017.2703120

    Article  MathSciNet  Google Scholar 

  52. Zhuang L, Zhou Z, Gao S, Yin J, Lin Z, Ma Y (2017) Label information guided graph construction for semi-supervised learning. IEEE Trans Image Process 26(9):4182–4192. https://doi.org/10.1109/TIP.2017.2703120

    Article  MathSciNet  Google Scholar 

  53. Zoidi O, Tefas A, Nikolaidis N, Pitas I (2018) Positive and negative label propagations. IEEE Trans Circ Syst Video Technol 28(2):342–355. https://doi.org/10.1109/TCSVT.2016.2598671

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China grant 61573266.

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Xidian University, Xi’an, 710071, China

    Zhiwen Hua & Youlong Yang

Authors
  1. Zhiwen Hua
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Youlong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Youlong Yang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hua, Z., Yang, Y. Robust and sparse label propagation for graph-based semi-supervised classification. Appl Intell 52, 3337–3351 (2022). https://doi.org/10.1007/s10489-021-02360-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02360-z

Keywords

Search

Navigation