LDR-LLE: LLE with Low-Dimensional Neighborhood Representation
The local linear embedding algorithm (LLE) is a non-linear dimension-reducing technique that is widely used for its computational simplicity and intuitive approach. LLE first linearly reconstructs each input point from its nearest neighbors and then preserves these neighborhood relations in a low-dimensional embedding. We show that the reconstruction weights computed by LLE capture the high-dimensional structure of the neighborhoods, and not the low-dimensional manifold structure. Consequently, the weight vectors are highly sensitive to noise. Moreover, this causes LLE to converge to a linear projection of the input, as opposed to its non-linear embedding goal. To resolve both of these problems, we propose to compute the weight vectors using a low-dimensional neighborhood representation. We call this technique LDR-LLE. We present numerical examples of the perturbation and linear projection problems, and of the improved outputs resulting from the low-dimensional neighborhood representation.
KeywordsWeight Vector Reconstruction Error Input Point Linear Projection Lighting Direction
Unable to display preview. Download preview PDF.
- 7.Xu, W., Lifang, X., Dan, Y., Zhiyan, H.: Speech visualization based on locally linear embedding (lle) for the hearing impaired. In: BMEI (2), pp. 502–505 (2008)Google Scholar
- 8.Shi, R., Shen, I.F., Chen, W.: Image denoising through locally linear embedding. In: CGIV 2005: Proceedings of the International Conference on Computer Graphics, Imaging and Visualization, pp. 147–152. IEEE Computer Society, Los Alamitos (2005)Google Scholar
- 12.Xu, X., Wu, F., Hu, Z., Luo, A.: A novel method for the determination of redshifts of normal galaxies by non-linear dimensionality reduction. Spectroscopy and Spectral Analysis 26, 182–186 (2006)Google Scholar
- 13.Hadid, A., Pietikäinen, M.: Efficient locally linear embeddings of imperfect manifolds. Machine Learning and Data Mining in Pattern Recognition, 188–201 (2003)Google Scholar
- 16.Wang, H., Zheng, J., Yao, Z., Li, L.: Improved locally linear embedding through new distance computing. In: Advances in Neural Networks - ISNN 2006, pp. 1326–1333 (2006)Google Scholar
- 17.Zhang, Z., Wang, J.: MLLE: Modified locally linear embedding using multiple weights. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems, vol. 19, pp. 1593–1600. MIT Press, Cambridge (2007)Google Scholar
- 20.Saul, L.K., Roweis, S.T.: Locally linear embedding (LLE) website, http://www.cs.toronto.edu/~roweis/lle/
- 23.Tenenbaum, J.B., de Silva, V., Langford, J.C.: Isomap website, http://isomap.stanford.edu/