Skip to main content
SpringerLink
Log in

Global and local scatter based semi-supervised dimensionality reduction with active constraints selection in ensemble subspaces

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Semi-supervised dimensionality reduction has attracted an increasing amount of attention in this big-data era. Many algorithms have been developed with a small number of pairwise constraints to achieve performances comparable to those of fully su pervised methods. However, one challenging problem with semi-supervised approaches is the appropriate choice of the constraint set, including the cardinality and the composition of the constraint set which, to a large extent, affects the performance of the resulting algorithm. In this work, we address the problem by incorporating ensemble subspaces and active learning into dimensionality reduction and propose a new global and local scatter based semi-supervised dimensionality reduction method with active constraints selection. Unlike traditional methods that select the supervised information in one subspace, we pick up pairwise constraints in ensemble subspaces, where a novel active learning algorithm is designed with both exploration and filtering to generate informative pairwise constraints. The automatic constraint selection approach proposed in this paper can be generalized to be used with all constraint-based semi-supervised learning algorithms. Comparative experiments are conducted on four face database and the results validate the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Averbuch A, Rabin N, Schclar A et al (2012) Dimensionality reduction for detection of moving vehicles. Pattern Anal Appl 15(1):19–27

    Article  MathSciNet  Google Scholar 

  2. Borges HB, Nievola JC (2012) Comparing the dimensionality reduction methods in gene expression databases. Expert Syst Appl 39(12):10780–10795

    Article  Google Scholar 

  3. Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86

    Article  Google Scholar 

  4. He X, Niyogi (2004) Locality preserving projections. In: Proceedings of advances in neural information processing systems, pp 153–160

  5. Martinez AM, Kak AC (2001) PCA versus LDA. IEEE Trans Pattern Anal Mach Intell 23(2):228–233

    Article  Google Scholar 

  6. Li H, Jiang T, Zhang K (2006) Efficient and robust feature extraction by maximum margin criterion. IEEE Trans Neural Netw 17(1):157–165

    Article  Google Scholar 

  7. An S, Liu W, Venkatesh S (2008) Exploiting side information in locality preserving projection. In: International conference on computer vision and pattern recognition, pp 1–8

  8. Hou C, Nie F, Wu Y (2011) Semi-supervised dimensionality reduction via harmonic functions. Modeling decision for artificial intelligence. Springer, Berlin Heidelberg, pp 91–102

    Google Scholar 

  9. Nie F, Xu D, Tsang IWH et al (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19(7):1921–1932

    Article  MathSciNet  Google Scholar 

  10. Zhao M, Zhan C, Wu Z et al (2015) Semi-supervised image classification based on local and global regression. IEEE Signal Process Lett 22(10):1666–1670

    Article  Google Scholar 

  11. Gao Q, Huang Y, Gao X et al (2015) A novel semi-supervised learning for face recognition. Neurocomputing 152:69–76

    Article  Google Scholar 

  12. Xiang S, Nie F, Zhang C (2008) Learning a Mahalanobis distance metric for data clustering and classification. Pattern Recogn 41(12):3600–3612

    Article  MATH  Google Scholar 

  13. Nie F, Xiang S, Jia Y et al (2009) Semi-supervised orthogonal discriminant analysis via label propagation. Pattern Recogn 42(11):2615–2627

    Article  MATH  Google Scholar 

  14. Nie F, Xiang S, Song Y et al (2009) Extracting the optimal dimensionality for local tensor discriminant analysis. Pattern Recogn 42(1):105–114

    Article  MATH  Google Scholar 

  15. Zhang C, Nie F, Xiang S (2010) A general kernelization framework for learning algorithms based on kernel PCA. Neurocomputing 73(4):959–967

    Article  Google Scholar 

  16. Basu S, Banerjee A, Mooney R (2004) Active semi-supervision for pairwise constrained clustering. In: Proceedings of the fourth SIAM international conference on data mining, pp 333–344

  17. Cevikalp H, Verbeek J, Jurie F, Klaser A (2008) Semi-supervised dimensionality reduction using pairwise equivalence constraints. In: Proceedings of the 3rd international conference on computer vision theory and applications, pp 489–496

  18. Mahdieh SB, Saeed BS (2009) Semi-supervised metric learning using pairwise constraints. In: International joint conference on artificial intelligence, pp 1217–1222

  19. Zhang Z, Zhao M, Chow TWS (2012) Marginal semi-supervised sub-manifold projections with informative constraints for dimensionality reduction and recognition. Neural Netw 36:97–111

    Article  MATH  Google Scholar 

  20. Davidson I, Wagstaff K, Basu S (2006) Measuring constraint-set utility for partitional clustering algorithms. In: Proceedings of the 10th European conference on principles and practice of knowledge discovery in databases, pp 115–126

  21. Davidson I (2012) Two approaches to understanding when constraints help clustering. In: Proceedings of the 18th ACM international conference on knowledge discovery and data mining, pp 1312–1320

  22. Wagstaff K (2007) Value, cost, and sharing: open issues in constrained clustering. In: International workshop on knowledge discovery in inductive databases, pp 1–10

  23. Qi M, Xiang Y (2013) Semi-supervised sparsity pairwise constraint preserving projections based on GA. Appl Math 7(3):1065–1075

    MathSciNet  Google Scholar 

  24. Cai X, Wen G, Wei J et al (2014) Relative manifold based semi-supervised dimensionality reduction. Front Comput Sci 8(6):923–932

    Article  MathSciNet  Google Scholar 

  25. Chrysouli C, Tefas A (2015) Spectral clustering and semi-supervised learning using evolving similarity graphs. Appl Soft Comput 34:625–637

    Article  Google Scholar 

  26. Bar-Hillel A, Hertz T, Shental N, Weinshall D (2003) Learning distance functions using equivalence relations. International conference on machine learning, pp 11–18

  27. Hoi SCH, Liu W, Lyu MR, Ma WY (2006) Learning distance metrics with contextual constraints for image retrieval. Conference on computer vision and pattern recognition, pp 2072–2078

  28. Bar-Hillel A, Hertz T, Shental N, Weinshall D (2005) Learning a mahalanobis metric from equivalence constraints. J Mach Learn Res 6(6):937–965

    MathSciNet  MATH  Google Scholar 

  29. Zhang D, Zhou ZH, Chen S (2007) Semi-supervised dimensionality reduction. In: SIAM conference on data Mining, pp 629–634

  30. Wei J, Peng H (2008) Local and global preserving based semi-supervised dimensionality reduction method. J Softw 19(11):2833–2842

    Article  Google Scholar 

  31. Wang N, Li X, Cui Y, Pan JS (2010) Instance-level based discriminative semi-supervised dimensionality reduction with chunklets. Int J Innov Comput Info Control 6(8):3763–3773

    Google Scholar 

  32. Li X, Luo JP, Chen MR, Wang N (2012) An improved shuffled frog-leaping algorithm with extremal optimization for continuous optimization. Info Sci 192(6):143–151

    Article  Google Scholar 

  33. Song Y, Nie F, Zhang C et al (2008) A unified framework for semi-supervised dimensionality reduction. Pattern Recogn 41(9):2789–2799

    Article  MATH  Google Scholar 

  34. Wan J, Yang M, Gao Y et al (2014) Pairwise costs in semisupervised discriminant analysis for face recognition. IEEE Trans Info Forensics Secur 9(10):1569–1580

    Article  Google Scholar 

  35. Yan S, Bouaziz S, Lee D et al (2012) Semi-supervised dimensionality reduction for analyzing high-dimensional data with constraints. Neurocomputing 76(1):114–124

    Article  Google Scholar 

  36. Settles B (2010) Active learning literature survey. Univ Wisconsin Madison 52(55–66):11

    Google Scholar 

  37. Mallapragada PK, Jin R, Jain AK (2008) Active query selection for semi-supervised clustering. In: Proceedings of the 19th international conference on pattern recognition, pp 1–4

  38. Xu Q, Wagstaff KL (2005) Active constrained clustering by examining spectral eigenvectors. Discovery Science. Springer, Berlin Heidelberg, pp 294–307

    Google Scholar 

  39. Greene D, Cunningham P (2007) Constraint selection by committee: an ensemble approach to identifying informative constraints for semi-supervised clustering. In: Proceedings of the 18th European conference on machine learning, pp 17–21

  40. Vu VV, Nicolas L (2012) Improving constrained clustering with active query selection. Pattern Recogn 45(4):1749–1758

    Article  Google Scholar 

  41. Małgorzata C (2012) Boosting, bagging and fixed fusion methods performance for aiding diagnosis. Biocybern Biomed Eng 32(2):17–31

    Article  Google Scholar 

  42. Yu G, Zhang G, Domeniconi C et al (2012) Semi-supervised classification based on random subspace dimensionality reduction. Pattern Recogn 45(3):1119–1135

    Article  MATH  Google Scholar 

  43. Zhang D, Chen S, Zhou ZH, Yang Q (2008) Constraint projections for ensemble learning. In: Proceedings of the 23rd AAAI conference on artificial intelligence, pp 758–763

  44. Sun D, Zhang D (2010) Bagging constraint score for feature selection with pairwise constraints. Pattern Recogn 43(6):2106–2118

    Article  MATH  Google Scholar 

  45. Zhang Z, Zhao M, Chow TWS (2015) Graph based constrained semi-supervised learning framework via label propagation over adaptive neighborhood. IEEE Trans Knowl Data Eng 27(9):2362–2376

    Article  Google Scholar 

  46. Guo YF, Li SJ, Yang JY et al (2003) A generalized Foley-Sammon transform based on generalized fisher discriminant criterion and its application to face recognition. Pattern Recogn Lett 24(1):147–158

    Article  MATH  Google Scholar 

  47. Xiang S, Nie F, Zhang C (2008) Learning a Mahalanobis distance metric for data clustering and classification. Pattern Recogn 41(12):3600–3612

    Article  MATH  Google Scholar 

  48. Wang H, Yan S, Xu D et al (2007) Trace ratio vs. ratio trace for dimensionality reduction. In: IEEE conference on computer vision and pattern recognition, pp 1–8

  49. Jia Y, Nie F, Zhang C (2009) Trace ratio problem revisited. IEEE Trans Neural Netw 20(4):729–735

    Article  Google Scholar 

  50. Zhang Z, Chow TWS, Zhao M (2013) Trace ratio optimization-based semi-supervised nonlinear dimensionality reduction for marginal manifold visualization. IEEE Trans Knowl Data Eng 25(5):1148–1161

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Science Foundation of China under Grant Nos. 60902069, 61171124, 61502315, Supported by Science Technology Planning Project of Shenzhen (Grant Nos. 2011B010200045, JC201105170613A, JCYJ20130329110601621).

Author information

Authors and Affiliations

  1. College of Information Engineering, Shenzhen University, Shenzhen, 518060, China

    Na Wang, Xia Li & Piao Chen

  2. Shenzhen Key Laboratory of Modern Communications and Information Processing, Shenzhen, 518060, China

    Na Wang, Xia Li & Piao Chen

Authors
  1. Na Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xia Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Piao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Na Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, N., Li, X. & Chen, P. Global and local scatter based semi-supervised dimensionality reduction with active constraints selection in ensemble subspaces. Pattern Anal Applic 20, 733–747 (2017). https://doi.org/10.1007/s10044-016-0530-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-016-0530-6

Keywords

Search

Navigation