Abstract
We present a new algorithm for manifold learning and nonlinear dimensionality reduction. Based on a set of unorganized data points sampled with noise from a parameterized manifold, the local geometry of the manifold is learned by constructing an approximation for the tangent space at each point, and those tangent spaces are then aligned to give the global coordinates of the data points with respect to the underlying manifold. We also present an error analysis of our algorithm showing that reconstruction errors can be quite small in some cases. We illustrate our algorithm using curves and surfaces both in 2D/3D Euclidean spaces and higher dimensional Euclidean spaces. We also address several theoretical and algorithmic issues for further research and improvements.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Kohonen T. Self-organizing Maps[M]. 3rd Edition, Springer-Verlag, 2000.
Roweis S, Saul L. Nonlinear dimensionality reduction by locally linear embedding[J]. Science, 2000, 290:2323–2326.
Tenenbaum J, de Silva V, Langford J. A global geometric framework for nonlinear dimension reduction [J]. Science, 2000, 290: 2319–2323.
Donoho D, Grimes C. Image manifolds which are isometric to Euclidean space [J]. To appear in Journal Mathematical Imaging and Vision.
Hastie T, Stuetzle W. Principal curves [J]. J. Am. Statistical Assoc., 1988, 84: 502–516.
Martinetz T, Schulten K. Topology representing net-works[J]. Neural Networks, 1994, 7: 507–523.
Saul L, Rowels S. Think globally, fit locally: unsupervised learning of nonlinear manifolds [J]. Journal of Machine Learning Research, 2003, 4: 119–155.
Donoho D, Grimes C. Hessian eigenmaps: new tools for nonlinear dimensionality reduction [A]. Proceedings of National Academy of Science[C]. 2003, 5591–5596.
Munkres J. Analysis on Manifold[M]. Addison Wesley, Redwood City, CA, 1990.
Freedman D. Efficient simplicial reconstructions of manifolds from their samples [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24: 1349–1357.
Teh Y, Rowels S. Automatic alignment of hidden representations[A]. Advances in Neural Information Processing Systems 15[C]. MIT Press, 2003.
Brand M. Charting a manifold[A]. Advances in Neural Information Processing Systems 15 [C]. MIT Press, 2003.
Verbeek J, Vlassis N, Krose B. Procrustes analysis to coordinate mixtures of probabilistic principal component analyzers[R]. Technical report, Computer Science Institute, University of Amsterdam, The Netherlands, IASUVA-02-01, 2002.
Verbeek J, Vlassis N, Krose B. Coordinating principal component analyzers[J]. ICANN, 2002, 914–919.
Venables W N, Ripley B D. Modern Applied Statistics with S-plus[M]. Springer-verlag, 1999.
Golub G H, van Loan C F. Matrix Computations [M]. 3nd Edition, Johns Hopkins University Press, Baltimore, Maryland, 1996.
Bentley J, Friedman J. Data structures for range searching[J]. ACM Computing Surveys, 1979, 11: 397–409.
Berrani S, Amsaleg L, Gros P. Approximate k-nearest-neighbor searches: a new algorithm with probabilistic control of the precision [R]. Technical Report, RR-4675, INRIA, 2002.
Polito M, Perona P. Grouping and Dimensionality reduction by Locally Linear Embedding [A]. Advances in Neural Information Processing Systems 14 [C]. MIT Press, 2002.
Shi J, Malik J. Normalized cuts and image segmentation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 8: 888–905.
Zha H, Ding C, Gu M, et al. Spectral Relaxation for K-means Clustering [A]. Advances in Neural Information Processing Systems 14[C]. MIT Press, 2002.
Kegl B. Intrinsic dimension estimation using packing numbers [A]. Advances in Neural Information Processing Systems 15[C]. MIT Press, 2003.
Costa J, Hero A. Manifold learning with geodesic minimal spanning trees[J]. IEEE Transactions on Signal Processing, 2003.
Belkin M, Niyogi P. Laplacian eigenmaps for dimension reduction and data representation[J]. Neural Computation, 2003, 15: 1373–1396.
Berstein M, de Silva V, Langford J, et al. Graph approximations to geodesies on embedded manifolds [EB/OL]. http://isomap.stanford.edu/BdSLT.pdf, 2000.
Author information
Authors and Affiliations
Additional information
Project supported in part by the Special Funds for Major State Basic Research Projects (Grant No. G19990328) and Foundation for University Key Teacher by the Ministry of Education(Grant No.CCR-9901986)
About this article
Cite this article
Zhang, Zy., Zha, Hy. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. J. of Shanghai Univ. 8, 406–424 (2004). https://doi.org/10.1007/s11741-004-0051-1
Received:
Issue Date:
DOI: https://doi.org/10.1007/s11741-004-0051-1
Key words
- nonlinear dimensionality reduction
- principal manifold
- tangent space
- subspace alignment
- singular value decomposition