Abstract
Feature learning is one of the important research trends among researchers in machine learning and other fields, which can select compact representations as feature information from high-dimensional data as well as multi-label data. Discriminative feature learning strengthens discrimination between sample features. Therefore, the feature information of samples can be better discriminated against in algorithms. In this paper, we propose a new unsupervised discriminative feature learning model called UD-LLE (Unsupervised Discriminative Locally Linear Embedding) by the improvement on standard Locally Linear Embedding, which not only maintains the manifold structure of mapping from high-dimensional space to low-dimensional space but also increases the discriminative of features. Specifically, we propose the restructure cost function as an objective function by adding constraint conditions about discrimination to standard function, which is solved by using stochastic gradient descent and momentum gradient descent algorithms combined with standard LLE.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
Wang, H., Zhang, Y., Zhang, J., Li, T., Peng, L.: A factor graph model for unsupervised feature selection. Inf. Sci. 480, 144–159 (2019)
Li, J., et al.: Feature selection: a data perspective. ACM Comput. Surv. (CSUR) 50(6), 1–45 (2017)
Zebari, R., Abdulazeez, A., Zeebaree, D., Zebari, D., Saeed, J.: A comprehensive review of dimensionality reduction techniques for feature selection and feature extraction. J. Appl. Sci. Technol. Trends 1(2), 56–70 (2020)
Jolliffe, I.T., Cadima, J.: Principal component analysis: a review and recent developments. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 374(2065), 20150202 (2016)
Yang, F., Yang, W., Gao, R., Liao, Q.: Discriminative multidimensional scaling for low-resolution face recognition. IEEE Signal Process. Lett. 25(3), 388–392 (2017)
Tenenbaum, J.B., De Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)
Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)
Saul, L.K., Roweis, S.T.: Think globally, fit locally: unsupervised learning of low dimensional manifolds. Departmental Papers (CIS), p. 12 (2003)
Lee, J.: Introduction to Topological Manifolds, vol. 202. Springer, Cham (2010). https://doi.org/10.1007/978-1-4419-7940-7
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., Bordes, A.: Supervised learning of universal sentence representations from natural language inference data. arXiv preprint arXiv:1705.02364 (2017)
Van Engelen, J.E., Hoos, H.H.: A survey on semi-supervised learning. Mach. Learn. 109(2), 373–440 (2019). https://doi.org/10.1007/s10994-019-05855-6
Grill, J.B., et al.: Bootstrap your own latent: a new approach to self-supervised learning. arXiv preprint arXiv:2006.07733 (2020)
Zhang, S., Lei, Y.K.: Modified locally linear discriminant embedding for plant leaf recognition. Neurocomputing 74(14–15), 2284–2290 (2011)
Huang, M., Zhu, Q., Wang, B., Lu, R.: Analysis of hyperspectral scattering images using locally linear embedding algorithm for apple mealiness classification. Comput. Electron. Agric. 89, 175–181 (2012)
Liang, D., Yang, J., Zheng, Z., Chang, Y.: A facial expression recognition system based on supervised locally linear embedding. Pattern Recogn. Lett. 26(15), 2374–2389 (2005)
Luo, J., Xu, T., Pan, T., Sun, W.: An efficient method of hyperspectral image dimension reduction based on low rank representation and locally linear embedding. Integr. Ferroelectr. 208(1), 206–214 (2020)
Yaddaden, Y., Adda, M., Bouzouane, A.: Facial expression recognition using locally linear embedding with LBP and HOG descriptors. In: 2020 2nd International Workshop on Human-Centric Smart Environments for Health and Well-being (IHSH), pp. 221–226. IEEE (2021)
Zhang, Y., Yang, Y., Li, T., Fujita, H.: A multitask multiview clustering algorithm in heterogeneous situations based on LLE and LE. Knowl.-Based Syst. 163, 776–786 (2019)
Xie, Y., Jiang, D., Wang, X., Xu, R.: Robust transfer integrated locally kernel embedding for click-through rate prediction. Inf. Sci. 491, 190–203 (2019)
Li, B., Zhang, Y.: Supervised locally linear embedding projection (SLLEP) for machinery fault diagnosis. Mech. Syst. Signal Process. 25(8), 3125–3134 (2011)
Zhang, Y., Ye, D., Liu, Y.: Robust locally linear embedding algorithm for machinery fault diagnosis. Neurocomputing 273, 323–332 (2018)
Donoho, D.L., Grimes, C.: Hessian eigenmaps: locally linear embedding techniques for high-dimensional data. Proc. Natl. Acad. Sci. 100(10), 5591–5596 (2003)
He, X., Sun, L., Lyu, C., Wang, X.: Quantum locally linear embedding for nonlinear dimensionality reduction. Quantum Inf. Process. 19(9), 1–21 (2020). https://doi.org/10.1007/s11128-020-02818-y
Zhang, Z., Wang, J.: MLLE: modified locally linear embedding using multiple weights. In: Advances in Neural Information Processing Systems, pp. 1593–1600. Citeseer (2007)
Ziegelmeier, L., Kirby, M., Peterson, C.: Sparse locally linear embedding. Procedia Comput. Sci. 108, 635–644 (2017)
Wang, J., Wong, R.K., Lee, T.C.: Locally linear embedding with additive noise. Pattern Recogn. Lett. 123, 47–52 (2019)
Chang, H., Yeung, D.Y.: Robust locally linear embedding. Pattern Recogn. 39(6), 1053–1065 (2006)
Wen, G., Jiang, L., Wen, J.: Dynamically determining neighborhood parameter for locally linear embedding. J. Softw. 19(7), 1666–1673 (2008)
Zhang, S.Q.: Enhanced supervised locally linear embedding. Pattern Recogn. Lett. 30(13), 1208–1218 (2009)
Hettiarachchi, R., Peters, J.F.: Multi-manifold LLE learning in pattern recognition. Pattern Recogn. 48(9), 2947–2960 (2015)
Liu, Y., Hu, Z., Zhang, Y.: Bearing feature extraction using multi-structure locally linear embedding. Neurocomputing 428, 280–290 (2021)
Deng, P., Wang, H., Li, T., Horng, S.J., Zhu, X.: Linear discriminant analysis guided by unsupervised ensemble learning. Inf. Sci. 480, 211–221 (2019)
Jain, A.K.: Data clustering: 50 years beyond K-means. Pattern Recogn. Lett. 31(8), 651–666 (2010)
Chu, R., Wang, H., Yang, Y., Li, T.: Clustering ensemble based on density peaks. Acta Automatica Sinica 42(9), 1401–1412 (2016)
Rodriguez, A., Laio, A.: Clustering by fast search and find of density peaks. Science 344(6191), 1492–1496 (2014)
Frey, B.J., Dueck, D.: Clustering by passing messages between data points. Science 315(5814), 972–976 (2007)
Acknowledgments
This work is supported by the National Natural Science Foundation of China (No. 61806170), the Humanities and Social Sciences Fund of Ministry of Education (No. 18XJC72040001), and the National Key Research and Development Program of China (No. 2019YFB1706104).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, C., Wang, L., Wang, H., Peng, B., Li, T. (2022). Locally Linear Embedding Discriminant Feature Learning Model. In: Sun, Y., et al. Computer Supported Cooperative Work and Social Computing. ChineseCSCW 2021. Communications in Computer and Information Science, vol 1492. Springer, Singapore. https://doi.org/10.1007/978-981-19-4549-6_1
Download citation
DOI: https://doi.org/10.1007/978-981-19-4549-6_1
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-4548-9
Online ISBN: 978-981-19-4549-6
eBook Packages: Computer ScienceComputer Science (R0)
