Skip to main content
SpringerLink
Log in

Adaptive semi-supervised dimensionality reduction based on pairwise constraints weighting and graph optimizing

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

With the rapid growth of high dimensional data, dimensionality reduction is playing a more and more important role in practical data processing and analysing tasks. This paper studies semi-supervised dimensionality reduction using pairwise constraints. In this setting, domain knowledge is given in the form of pairwise constraints, which specifies whether a pair of instances belong to the same class (must-link constraint) or different classes (cannot-link constraint). In this paper, a novel semi-supervised dimensionality reduction method called adaptive semi-supervised dimensionality reduction (ASSDR) is proposed, which can get the optimized low dimensional representation of the original data by adaptively adjusting the weights of the pairwise constraints and simultaneously optimizing the graph construction. Experiments on UCI classification and image recognition show that ASSDR is superior to many existing dimensionality reduction methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. http://archive.ics.uci.edu/ml/.

  2. http://www.cad.zju.edu.cn/home/dengcai/Data/FaceData.html.

  3. http://www4.comp.polyu.edu.hk/~biometrics/.

References

  1. Duda RO, Hart PE, Stork DG (2001) Pattern classification, 2nd edn. Wiley, New York

    MATH  Google Scholar 

  2. Jolliffe IT (2002) Principal component analysis, 2nd edn. Wiley, New York

    MATH  Google Scholar 

  3. Sharma A, Paliwal KK, Imoto S, Miyano S (2013) Principal component analysis using QR decomposition. Int J Mach Learn Cybern (IJMLC) 4:679–683

    Article  Google Scholar 

  4. Sharma A, Paliwal KK (2015) Linear discriminant analysis for the small sample size problem: an overview. Int J Mach Learn Cybern (IJMLC) 6:443–454

    Article  Google Scholar 

  5. Chapelle O, Schölkopf B, Zien A (2006) Semi-supervised learning. MIT Press, Cambridge

    Book  Google Scholar 

  6. Song Y, Nie F, Zhang C, Xiang S (2008) A unified framework for semi-supervised dimensionality reduction. Pattern Recognit 41(9):2789–2799

    Article  MATH  Google Scholar 

  7. Chatpatanasiri R, Kijsirikul B (2010) A unified semi-supervised dimensionality reduction framework for manifold learning. Neurocomputing 73(10–12):1631–1640

    Article  Google Scholar 

  8. Alok AK, Saha S, Ekbal A (2015) Semi-supervised clustering for gene-expression data in multiobjective optimization framework. Int J Mach Learn Cybern (IJMLC). doi:10.1007/s13042-015-0335-8

  9. Wagstaff K, Cardie C (2000) Clustering with instance-level constraints. In: Proceedings of the 17th International Conference on Machine Learning, pp 1003–1110

  10. Chen C, Zhang J, He X, Zhou ZH (2012) Non-parametric kernel learning with robust pairwise constraints. Int J Mach Learn Cybern (IJMLC) 3:83–96

    Article  Google Scholar 

  11. Klein D, Kamvar SD, Manning CD (2002) From instance-level constraints to space-level constraints: making the most of prior knowledge in data clustering. In: Proceedings of the 19th International Conference on Machine Learning, pp 307–314

  12. Bar-Hillel A, Hertz T, Shental N, Weinshall D (2005) Learning a mahalanobis metric from equivalence constraints. J Mach Learn Res 6:937–965

    MathSciNet  MATH  Google Scholar 

  13. Xing EP, Ng AY, Jordan MI, Russell S (2003) Distance metric learning, with application to clustering with side-information. Adv Neural Inf Process Syst 15:505–512

    Google Scholar 

  14. Tang W, Zhong S (2006) Pairwise constraints-guided dimensionality reduction. In: Proceedings of the SDM’06 Workshop on Feature Selection for Data Mining, pp 59–66

  15. Yeung DY, Chang H (2006) Extending the relevant component analysis algorithm for metric learning using both positive and negative equivalence constraints. Pattern Recognit 39(5):1007–1010

    Article  MATH  Google Scholar 

  16. An S, Liu W, Venkatesh S (2008) Exploiting side information in locality preserving projection. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp 1–8

  17. Zhang D, Zhou Z, Chen S (2007) Semi-supervised dimensionality reduction. In: Proceedings of the 7th SIAM International Conference on Data Mining, pp 629–634

  18. Chen S, Zhang D (2011) Semisupervised dimensionality reduction with pairwise constraints for hyperspectral image classification. IEEE Geosci Remote Sens Lett 8(2):369–373

    Article  Google Scholar 

  19. Cevikalp H, Verbeek J, Jurie F, Klaser A (2008) Semi-supervised dimensionality reduction using pairwise equivalence constraints. In: Proceedings of the 2008 International Conference on Computer Vision Theory and Applications, pp 489–496

  20. He X, Niyogi P (2004) Locality preserving projections. Adv in Neural Inf Process Syst 16:153–160

    Google Scholar 

  21. Wei J, Peng H (2008) Neighborhood preserving based semi-supervised dimensionality reduction. Electron Lett 44(20):1190–1191

    Article  Google Scholar 

  22. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2327

    Article  Google Scholar 

  23. Baghshah MS, Shouraki SB (2009) Semi-supervised metric learning using pairwise constraints. In: Proceedings of the 21st International Joint Conference on Artificial Intelligence, pp 1217–1222

  24. Davidson I (2009) Knowledge driven dimension reduction for clustering. In: Proceedings of the 21st International Joint Conference on Artificial Intelligence, pp 1034–1039

  25. Yan S, Bouaziz S, Lee D, Barlow J (2012) Semi-supervised dimensionality reduction for analyzing high-dimensional data with constraints. Neurocomputing 76(1):114–124

    Article  Google Scholar 

  26. Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396

    Article  MATH  Google Scholar 

  27. Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. In: Proceedings of the 22nd International Conference on Machine Learning, pp 57–64

  28. Sim T, Barker S, Bsat M (2003) The CMU pose, illumination, and expression database. IEEE Trans Pattern Anal Mach Intell 25(12):1615–1618

    Article  Google Scholar 

  29. Georghiades AS, Belhumeur PN, Kriegman DJ (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660

    Article  Google Scholar 

  30. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  31. Zhang D, Kong W, You J, Wong M (2003) Online palmprint identification. IEEE Trans Pattern Anal Mach Intell 25(9):1041–1050

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (61402181, 61273363), the Science and Technology Programme of Guangzhou Municipal Government (2014J4100006), the Guangdong Natural Science Foundation (S2012040008022, S2012010009961).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, South China University of Technology, Guangzhou, China

    Meng Meng, Jia Wei, Jiabing Wang & Qianli Ma

  2. Computer Application Research Center, Harbin Institute of Technology Shenzhen Graduate School, Shenzhen, China

    Xuan Wang

Authors
  1. Meng Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiabing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qianli Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jia Wei.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meng, M., Wei, J., Wang, J. et al. Adaptive semi-supervised dimensionality reduction based on pairwise constraints weighting and graph optimizing. Int. J. Mach. Learn. & Cyber. 8, 793–805 (2017). https://doi.org/10.1007/s13042-015-0380-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-015-0380-3

Keywords

Search

Navigation