Abstract
Nonlinear Dimensionality Reduction is an important issue in many machine learning areas where essentially low-dimensional data is nonlinearly embedded in some high-dimensional space. In this paper, we show that the existing Laplacian Eigenmaps method suffers from the distortion problem, and propose a new distortion-free dimensionality reduction method by adopting a local linear model to encode the local information. We introduce a new loss function that can be seen as a different way to construct the Laplacian matrix, and a new way to impose scaling constraints under the local linear model. Better low-dimensional embeddings are obtained via constrained concave convex procedure. Empirical studies and real-world applications have shown the effectiveness of our method.
Chapter PDF
Similar content being viewed by others
Keywords
- Loss Function
- Dimensionality Reduction
- Laplacian Matrix
- Locally Linear Embedding
- Nonlinear Dimensionality Reduction
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Jolliffe, I.: Principal Component Analysis. Springer, Heidelberg (2002)
Cox, T., Cox, M.: Multidimensional Scaling. Chapman and Hall, Boca Raton (2001)
Tenenbaum, J., Silva, V., Langford, J.: A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science 290(5500), 2319–2323 (2000)
Roweis, S., Saul, L.: Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science 290(5500), 2323–2326 (2000)
Zhang, Z., Zha, H.: Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. SIAM J. Sci. Comp. 26(1), 313–338 (2005)
Weinberger, K., Saul, L.: Unsupervised Learning of Image Manifolds by Semidefinite Programming. Intl. J. Computer Vision 70(1), 77–90 (2006)
Belkin, M., Niyogi, P.: Laplacian Eigenmaps for Dimensionality Reduction and Data Representation. Neural Computation 15(6), 1373–1396 (2003)
He, X., Niyogi, P.: Locality preserving projections. Advances in Neural Information Processing Systems (2003)
Belkin, M., Niyogi, P.: Towards a theoretical foundation for laplacian-based manifold methods. In: Auer, P., Meir, R. (eds.) COLT 2005. LNCS (LNAI), vol. 3559, pp. 486–500. Springer, Heidelberg (2005)
Ding, C., Simon, H., Jin, R., Li, T.: A learning framework using Green’s function and kernel regularization with application to recommender system. In: Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 260–269 (2007)
Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 888–905 (2000)
Hagen, L., Kahng, A.: New spectral methods for ratio cut partitioning and clustering. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 11(9), 1074–1085 (1992)
Zhang, Z., Wang, J.: MLLE: Modified Locally Linear Embedding Using Multiple Weights. Advances in Neural Information Processing Systems (2007)
Bottou, L., Vapnik, V.: Local learning algorithms. Neural Computation 4(6), 888–900 (1992)
Wu, M., Schölkopf, B.: Transductive classification via local learning regularization. In: Proceedings of the 11th International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 624–631 (2007)
Smola, A., Vishwanathan, S., Hofmann, T.: Kernel methods for missing variables. In: Proc. International Workshop on Artificial Intelligence and Statistics (2005)
Seber, G.: Multivariate Observations. Wiley, Chichester (1984)
LeCun, Y.: The MNIST database of handwritten digits. NEC Research Institute, http://yann.lecun.com/exdb/mnist/index.html
Gashler, M., Ventura, D., Martinez, T.: Iterative non-linear dimensionality reduction with manifold sculpting. In: Advances in Neural Information Processing Systems 20, pp. 513–520 (2008)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jia, Y., Wang, Z., Zhang, C. (2008). Distortion-Free Nonlinear Dimensionality Reduction. In: Daelemans, W., Goethals, B., Morik, K. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2008. Lecture Notes in Computer Science(), vol 5211. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87479-9_55
Download citation
DOI: https://doi.org/10.1007/978-3-540-87479-9_55
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87478-2
Online ISBN: 978-3-540-87479-9
eBook Packages: Computer ScienceComputer Science (R0)