Knowledge and Information Systems

, Volume 43, Issue 1, pp 81–101 | Cite as

Semi-supervised classification based on subspace sparse representation

  • Guoxian Yu
  • Guoji Zhang
  • Zili Zhang
  • Zhiwen Yu
  • Lin Deng
Regular Paper

Abstract

Graph plays an important role in graph-based semi-supervised classification. However, due to noisy and redundant features in high-dimensional data, it is not a trivial job to construct a well-structured graph on high-dimensional samples. In this paper, we take advantage of sparse representation in random subspaces for graph construction and propose a method called Semi-Supervised Classification based on Subspace Sparse Representation, SSC-SSR in short. SSC-SSR first generates several random subspaces from the original space and then seeks sparse representation coefficients in these subspaces. Next, it trains semi-supervised linear classifiers on graphs that are constructed by these coefficients. Finally, it combines these classifiers into an ensemble classifier by minimizing a linear regression problem. Unlike traditional graph-based semi-supervised classification methods, the graphs of SSC-SSR are data-driven instead of man-made in advance. Empirical study on face images classification tasks demonstrates that SSC-SSR not only has superior recognition performance with respect to competitive methods, but also has wide ranges of effective input parameters.

Keywords

Semi-supervised classification High-dimensional data  Graph construction Subspaces sparse representation 

References

  1. 1.
    Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434MATHMathSciNetGoogle Scholar
  2. 2.
    Breiman L (2001) Random forests. Mach Learn 45(1):5–32CrossRefMATHGoogle Scholar
  3. 3.
    Brown G, Wyatt J, Harris R, Yao X (2005) Diversity creation methods: a survey and categorisation. Inf Fusion 6(1):5–20CrossRefGoogle Scholar
  4. 4.
    Chen L, Liao H, Ko M, Lin J, Yu G (2000) A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recogn 33(10):1713–1726CrossRefGoogle Scholar
  5. 5.
    Chung F (1997) Spectral graph theory. Regional conference series in mathematics, No 92Google Scholar
  6. 6.
    Donoho D (2006) Compressed sensing. IEEE Trans Inf Theory 52(4):1289–1306CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    Duda R, Hart P, Stork D (2001) Pattern classification. Wiley, New YorkMATHGoogle Scholar
  8. 8.
    Elhamifar E, Vidal R (2009) Sparse subspace clustering. Proc IEEE Conf Comput Vis Pattern Recogn 2009:2790–2797Google Scholar
  9. 9.
    Fan M, Gu N, Qiao H, Zhang B (2011) Sparse regularization for semi-supervised classification. Pattern Recogn 44(8):1777–1784CrossRefMATHGoogle Scholar
  10. 10.
    Georghiades A, Belhumeur P, Kriegman D (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660CrossRefGoogle Scholar
  11. 11.
    Ho T (1998) The random subspace method for constructing decision forests. IEEE Trans Pattern Anal Mach Intell 20(8):832–844CrossRefGoogle Scholar
  12. 12.
    Huang K, Aviyente S (2006) Sparse representation for signal classification. In: Proceedings of advances in neural information processing systems 2006, pp 609–616Google Scholar
  13. 13.
    Jebara T, Wang J, Chang S (2009) Graph construction and b-matching for semi-supervised learning. In Proceedings of international conference on machine learning, pp 441–448Google Scholar
  14. 14.
    Kleinberg E (2000) On the algorithmic implementation of stochastic discrimination. IEEE Trans Pattern Anal Mach Intell 22(5):473–490CrossRefGoogle Scholar
  15. 15.
    Kuncheva L, Whitaker C (2003) Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach Learn 51(2):181–207CrossRefMATHGoogle Scholar
  16. 16.
    Liu J, Ye J (2009) Efficient Euclidean projections in linear time. In: Proceedings of international conference on machine learning, pp 657–664Google Scholar
  17. 17.
    Liu W, Chang S (2009) Robust multi-class transductive learning with graphs. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 381–388Google Scholar
  18. 18.
    Maier M, Von Luxburg U, Hein M (2008) Influence of graph construction on graph-based clustering measures. In: Proceedings of advances in neural information processing system, pp 1025–1032Google Scholar
  19. 19.
    Margineantu D, Dietterich T (1997) Pruning adaptive boosting. In: Proceedings of international conference on machine learning, pp 211–218Google Scholar
  20. 20.
    Martinez A (1998) The AR face database. Technical Report 24, CVCGoogle Scholar
  21. 21.
    Olshausen B, Field D (1997) Sparse coding with an overcomplete basis set: a strategy employed by V1? Vis Res 37(23):3311–3325CrossRefGoogle Scholar
  22. 22.
    Parsons L, Haque E, Liu H (2004) Subspace clustering for high dimensional data: a review. ACM SIGKDD Explor Newsl 6(1):90–105CrossRefGoogle Scholar
  23. 23.
    Qiao L, Chen S, Tan X (2010) Sparsity preserving projections with applications to face recognition. Pattern Recogn 43(1):331–341Google Scholar
  24. 24.
    Qiao L, Chen S, Tan X (2010) Sparsity preserving discriminant analysis for single training image face recognition. Pattern Recogn Lett 31(5):422–429Google Scholar
  25. 25.
    Rodriguez J, Kuncheva L, Alonso C (2006) Rotation forest: a new classifier ensemble method. IEEE Trans Pattern Anal Mach Intell 28(10):1619–1630CrossRefGoogle Scholar
  26. 26.
    Roweis S, Saul L (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326CrossRefGoogle Scholar
  27. 27.
    Samaria F, Harter A (1994) Parameterisation of a stochastic model for human face identification. In: Proceedings of the second IEEE workshop on applications of computer vision, pp 138–142Google Scholar
  28. 28.
    Sim T, Baker S, Bsat M (2003) The CMU pose, illumination, and expression (PIE) database. IEEE Trans Pattern Anal Mach Intell 25(12):1615–1618CrossRefGoogle Scholar
  29. 29.
    Wang J, Wang F, Zhang C, Shen H, Quan L (2009) Linear neighborhood propagation and its applications. IEEE Trans Pattern Anal Mach Intell 31(9):1600–1615CrossRefGoogle Scholar
  30. 30.
    Wright J, Yang A, Ganesh A, Sastry S, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227CrossRefGoogle Scholar
  31. 31.
    Wu F, Wang W, Yang Y, Zhuang Y, Nie F (2010) Classification by semi-supervised discriminative regularization. Neurocomputing 73(10):1641–1651CrossRefGoogle Scholar
  32. 32.
    Yan S, Wang H (2009) Semi-supervised learning with sparse representation. In: Proceedings of SIAM conference on data mining, pp 792–801Google Scholar
  33. 33.
    Yan S, Xu D, Zhang B, Zhang HJ, Yang Q, Lin S (2007) Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29(1):40–51CrossRefGoogle Scholar
  34. 34.
    Yang A, Ganesh G, Sastry S, Ma Y (2010) Fast \(\ell \)1-minimization algorithms and an application in robust face recognition: a review. Technical Report 2010–13, Department of Electrical Engineering and Computer Science, University of Carlifornia at BerkeleyGoogle Scholar
  35. 35.
    Yang J, Zhang L, Xu Y, Yang J (2012) Beyond sparsity: the role of L1-optimizer in pattern classification. Pattern Recogn 45(3):1104–1118CrossRefMATHGoogle Scholar
  36. 36.
    Yu G, Zhang G, Domeniconi C, Yu Z, You J (2012) Semi-supervised classification based on random subspace dimensionality reduction. Pattern Recogn 45(3):1119–1135Google Scholar
  37. 37.
    Yu G, Zhang G, Yu Z, Domeniconi C, You J, Han G (2012) Semi-supervised ensemble classification in subspaces. Appl Soft Comput 12(5):1511–1522Google Scholar
  38. 38.
    Zhou D, Bousquet O, Lal TN, Weston J, Schölkopf B (2004) Learning with local and global consistency. In: Proceedings of advances in neural information processing systems, pp 321–328Google Scholar
  39. 39.
    Zhu X (2008) Semi-supervised learning literature survey. Technical Report 1530, Department of Computer Sciences, University of Wisconsin-MadisonGoogle Scholar
  40. 40.
    Zhu X, Ghahramani Z, Lafferty J (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: Proceedings of international conference on machine learning 2003, pp 912–919Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Guoxian Yu
    • 1
    • 2
  • Guoji Zhang
    • 3
  • Zili Zhang
    • 4
  • Zhiwen Yu
    • 2
  • Lin Deng
    • 5
  1. 1.College of Computer and Information ScienceSouthwest UniversityChongqingChina
  2. 2.School of Computer Science and EngineeringSouth China University of TechnologyGuangzhouChina
  3. 3.School of SciencesSouth China University of TechnologyGuangzhouChina
  4. 4.School of Information TechnologyDeakin UniversityGeelongAustralia
  5. 5.Department of Computer ScienceGeorge Mason UniversityFairfaxUSA

Personalised recommendations