On Evolutionary Approaches to Unsupervised Nearest Neighbor Regression
The detection of structures in high-dimensional data has an important part to play in machine learning. Recently, we proposed a fast iterative strategy for non-linear dimensionality reduction based on the unsupervised formulation of K-nearest neighbor regression. As the unsupervised nearest neighbor (UNN) optimization problem does not allow the computation of derivatives, the employment of direct search methods is reasonable. In this paper we introduce evolutionary optimization approaches for learning UNN embeddings. Two continuous variants are based on the CMA-ES employing regularization with domain restriction, and penalizing extension in latent space. A combinatorial variant is based on embedding the latent variables on a grid, and performing stochastic swaps. We compare the results on artificial dimensionality reduction problems.
KeywordsDimensionality Reduction Latent Space Locally Linear Embedding Dimensionality Reduction Method Evolution Strategy
Unable to display preview. Download preview PDF.
- 2.Carreira-Perpiñán, M.Á., Lu, Z.: Parametric dimensionality reduction by unsupervised regression. In: Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1895–1902 (2010)Google Scholar
- 4.Jolliffe, I.: Principal component analysis. Springer Series in Statistics. Springer, New York (1986)Google Scholar
- 6.Kramer, O.: Dimensionality reduction by unsupervised k-nearest neighbor regression. In: Proceedings of the International Conference on Machine Learning and Applications (ICMLA), pp. 275–278. IEEE Computer Society Press (2011)Google Scholar
- 7.Kramer, O., Gieseke, F.: A stochastic optimization approach for unsupervised kernel regression. In: Genetic and Evolutionary Methods (GEM), pp. 156–161 (2011)Google Scholar
- 9.Meinicke, P.: Unsupervised Learning in a Generalized Regression Framework. Ph.D. thesis, University of Bielefeld (2000)Google Scholar
- 12.Pearson, K.: On lines and planes of closest fit to systems of points in space. Philosophical Magazine 2(6), 559–572 (1901)Google Scholar