Skip to main content
SpringerLink
Log in

Locally linear embedding: a survey

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

As a classic method of nonlinear dimensional reduction, locally linear embedding (LLE) is more and more attractive to researchers due to its ability to deal with large amounts of high dimensional data and its non-iterative way of finding the embeddings. However, several problems in the LLE algorithm still remain open, such as its sensitivity to noise, inevitable ill-conditioned eigenproblems, the lack of how to deal with the novel data, etc. The existing extensions are comprehensively reviewed and discussed classifying into different categories in this paper. Their strategies, advantages/disadvantages and performances are elaborated. By generalizing different tactics in various extensions related to different stages of LLE and evaluating their performances, several promising directions for future research have been suggested.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Balasubramanian M, Schwartz EL, Tenenbaum JB, de Silva V, Langford JC (2002) The isomap algorithm and topological stability. Science 295: 7

    Article  Google Scholar 

  • Beckmann N, Kriegel HP, Schneider R, Seeger B (1990) The R*-tree: an efficient and robust access method for points and rectangles. In: Proceedings of the 1990 ACM SIGMOD international conference on Management of data, pp 322–331

  • Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. Fisherfaces: recognition using class specific linear. IEEE T Pattern Anal 19: 711–720

    Article  Google Scholar 

  • Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6): 1373–1396

    Article  MATH  Google Scholar 

  • Bengio Y, Paiement J, Vincent P, Delalleau O, Roux NL, Ouimet M (2003) Out-of-sample extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral clustering. In: Advances in neural information processing systems, pp 177–184

  • Brand M (2003) Charting a manifold. In: Advances in neural information processing systems, vol 15, pp 961–968

  • Cai D (2009) Spectral regression: a regression framework for efficient regularized subspace learning, UIUC

  • Cai D, He X, Han J (2007) Spectral regression for efficient regularized subspace learning. ICCV, pp 1–8

  • Chang H, Yeung D (2006) Robust locally linear embedding. Pattern Recognit 39(6): 1053–1065

    Article  MATH  Google Scholar 

  • de La Torre F, Black MJ (2003) A framework for robust subspace learning. Int J Computer Vision 54(1–3): 117–142

    Article  MATH  Google Scholar 

  • de Ridder D, Kouropteva O, Okun, Oleg (2003) Supervised locally linear embedding. In: Proceedings of ICANN, pp 333–341

  • Donoho D, Grimes C (2003) Hessian eigenmaps: new tools for nonlinear dimensionality reduction. Proc Natl Acad Sci 100: 5591–5596

    Article  MathSciNet  MATH  Google Scholar 

  • Eftekhari A, Abrishami-Moghaddam H, Babaie-Zadeh M (2009a) k/K-Nearest neighborhood criterion for improvement of locally linear embedding. CAIP, pp 808–815

  • Eftekhari A, Babaie-Zadeh M, Jutten C, Moghaddam HA (2009b) Robust-SL0 for stable sparse representation in noisy settings. ICASSP, pp 3433–3436

  • Fienberq S (1985) The analysis of crossclassified categorical data. MIT press, Cambridge

    Google Scholar 

  • Goldberg Y, Ritov Y (2008) LDR-LLE: LLE with low-dimensional neighborhood representation. ISVC, pp 43–54

  • Guihua W, Lijun J, Jun W (2008) Kernel relative transformation with applications to enhancing locally linear embedding. IJCNN, pp 3401–3406

  • Hadid A, Pietikainen M (2003) Efficient locally linear embeddings of imperfect manifolds. MLDM, pp 188–201

  • Han PY, Beng Jin AT, Kiong WE (2008) Neighbourhood discriminant locally linear embedding in face recognition. CGIV2008, pp 223–228

  • He X, Cai D, Yan S, Zhang H (2005) Neighborhood preserving embedding. IEEE Int Conf Comput Vis 2: 1208–1213

    Google Scholar 

  • Holland PW, Welsch RE (1977) Communications in statistics: theory and methods

  • Hou C, Wang J, Wu Y, Yi D (2009a) Local linear transformation embedding. Neurocomputing 72(10–12): 2368–2378

    Article  Google Scholar 

  • Hou C, Zhang C, Wu Y, Jiao Y (2009b) Stable local dimensionality reduction approaches. Pattern Recognit 42(9): 2054–2066

    Article  MATH  Google Scholar 

  • Hui K, Wang C (2008) Clustering-based locally linear embedding. ICPR

  • Kadoury S, Levine MD (2007) Face detection in gray scale images using locally linear embeddings. Comput Vis Image Underst 105(1): 1–20

    Article  Google Scholar 

  • Karbauskaitė R, Kurasova O, Dzemyda G (2007) Selection of the number of neighbors of each data point for the locally linear embedding algorithm. Inf Technol Control 36(4): 359–364

    Google Scholar 

  • Kouropteva O, Okun O, inen MPA (2002) Selection of the optimal parameter value for the locally linear embedding algorithm. In: Proceedings of the 1st international conference on fuzzy systems and knowledge discovery, pp 359–363

  • Kouropteva O, Okun O, Pietikäinen M (2005) Incremental locally linear embedding. Pattern Recognit 38(10): 1764–1767

    Article  MATH  Google Scholar 

  • Li B, Zheng C, Huang D (2008) Locally linear discriminant embedding: an efficient method for face recognition. Pattern Recognit 41(12): 3813–3821

    Article  MATH  Google Scholar 

  • Li H, Jiang T, Zhang K (2006) Efficient and robust feature extraction by maximum margin criterion. IEEE Trans Neural Netw 17(1): 157–165

    Article  Google Scholar 

  • Li SZ, Lu J (1999) Face recognition using the nearest feature line method. IEEE Trans Neural Netw 10(4): 439–443

    Article  Google Scholar 

  • Lingzhu H, Lingxiang Z, Caiyue C, Min L (2009) Locally linear embedding algorithm with adaptive neighbors. ISA

  • Pan Y, Ge SS, Al Mamun A (2009) Weighted locally linear embedding for dimension reduction. Pattern Recognitn 42(5): 798–811

    Article  MATH  Google Scholar 

  • Pang YH, Teoh ABJ, Wong EK, Abas FS (2008) Supervised locally linear embedding in face recognition. ISBAST’08.

  • Park J, Zhang Z, Zha H, Kasturi R (2004) Local smoothing for manifold learning. CVPR I452-I459

  • Qiu P (2004) The local piecewisely linear kernel smoothing procedure for fitting jump regression surfaces. Technometrics 46: 87–98

    Article  MathSciNet  Google Scholar 

  • Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500): 2323–2326

    Article  Google Scholar 

  • Saul LK, Rowels ST (2004) Think globally, fit locally: unsupervised learning of low dimensional manifolds. J Mach Learn Res 4(2): 119–155

    Article  MathSciNet  MATH  Google Scholar 

  • Tenenbaum JB, De Silva V, Langford JC (2000) A global geometric framework for nonlinear dimension reduction. Science 290(5500): 2319–2323

    Article  Google Scholar 

  • Teng X, Wu B, Yu W, Liu C, (2005) A hand gesture recognition system based on local linear embedding J Visual Lang & Computing Special issue section on Context and Emotion Aware Visual Interaction - Part I 16(5): 442–454

    Google Scholar 

  • Tikhonov AN (1963) Regularization of incorrectly posed problems

  • Valencia-Aguirre J, Álvarez-Mesa A, Daza-Santacoloma G, Castellanos-Domínguez G (2009) Automatic choice of the number of nearest neighbors in locally linear embedding CIARP2009, pp 77–84

  • Varini C, Degenhard A, Nattkemper TW (2006) ISOLLE: LLE with geodesic distance. Neurocomputing 69(13–15): 1768–1771

    Article  Google Scholar 

  • Wang H, Zheng J, Yao Z, Li L (2006) Improved locally linear embedding through new distance computing. ISNN, pp 1326–1333

  • Wang J (2008) Robust and stable locally linear embedding. FSKD, pp 197–201

  • Wang J, Zhang Z (2010) Nonlinear embedding preserving multiple local-linearities. Pattern Recognit 43(4): 1257–1268

    Article  MATH  Google Scholar 

  • Wang Y, Wu Y (2010) Complete neighborhood preserving embedding for face recognition. Pattern Recognit 43(3): 1008–1015

    Article  MATH  Google Scholar 

  • Watts DJ, Strogatz SH (1998) Collective dynamics of small-world networks. Nature 393(6684): 440–442

    Article  Google Scholar 

  • Weinberger KQ, Saul LK (2006) Unsupervised learning of image manifolds by semidefinite programming. Int J Comput Vis 70(1): 77–90

    Article  Google Scholar 

  • Wen G, Jiang L (2006) Clustering-based locally linear embedding. In: Proceedings of 2006 IEEE international conference on systems, man and cybernetics pp 4192–4196

  • Wen G, Jiang L, Wen J, Shadbolt NR (2006) Performing locally linear embedding with adaptable neighborhood size on manifold. J Softw, pp 985–989

  • Wu F, Hu ZY (2006) The LLE and a linear mapping. Pattern Recognit 39(9): 1799–1804

    Article  MATH  Google Scholar 

  • Xia T, Li J, Zhang Y, Tang S (2008) A more topologically stable locally linear embedding algorithm based on R*-tree. PAKDD, pp 803–812

  • Yan Y, Zhang Y (2008) Discriminant projection embedding for face and palmprint recognition. Neurocomputing 71(16–18): 3534–3543

    Article  Google Scholar 

  • Yin J, Hu D, Zhou Z (2008) Noisy manifold learning using neighborhood smoothing embedding. Pattern Recognit Lett 29(11): 1613–1620

    Article  Google Scholar 

  • Ying HP, Andrew Teoh BJ, Wong EK (2008) Neighbourhood discriminant embedding in face recognition. IEICE Electron Express 5(24): 1036–1041

    Article  Google Scholar 

  • Yulin Z, Jian Z, Sun’an W, Xiaohu L (2008) Local linear embedding in dimensionality reduction based on small world principle. CSSE, pp 394–398

  • Zeng X, Luo S (2008) Generalized locally linear embedding based on local reconstruction similarity. FSKD2008, pp 305–309

  • Zhan D, Zhou Z (2006) Neighbor line-based locally linear embedding. PAKDD, pp 806–815

  • Zhang C, Wang J, Zhao N, Zhang D (2004) Reconstruction and analysis of multi-pose face images based on nonlinear dimensionality reduction. Pattern Recognit 37(2): 325–336

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang S (2009) Enhanced supervised locally linear embedding. Pattern Recognit Lett 30(13): 1208–1218

    Article  Google Scholar 

  • Zhang Z, Wang J (2007) MLLE: modified locally linear embedding using multiple weights. Adv Neural Inf Process Syst 19: 1593–1600

    Google Scholar 

  • Zhang X, Liu Y, Gao C, Liu J (2008) An efficient algorithm of learning the parametric map of locally linear embedding. IITA, pp 52–56

  • Zhang Z, Zha H (2005) Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. SIAM J Sci Comput 26(1): 313–338

    Article  MathSciNet  Google Scholar 

  • Zhang Z, Zhao L (2007) Probability-based locally linear embedding for classification. FSKD, pp 243–247

  • Zhao D (2006) Formulating LLE using alignment technique. Pattern Recognit 39: 2233–2235

    Article  MATH  Google Scholar 

  • Zhao L, Zhang Z (2009) Supervised locally linear embedding with probability-based distance for classification. Comput Math Appl 57(6): 919–926

    Article  MATH  Google Scholar 

  • Zhao Q, Zhang D, Lu H (2005) Supervised LLE in ICA space for facial expression recognition. ICNNB’05, pp 1970–1975

  • Zhou CY, Chen YQ (2006) Improving nearest neighbor classification with cam weighted distance. Pattern Recognit 39(4): 635–645

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Physics and Optoelectronic Engineering, Guangdong University of Technology, Guangzhou, 510006, China

    Jing Chen

  2. School of Information Engineering, Guangdong University of Technology, Guangzhou, 510006, China

    Yang Liu

Authors
  1. Jing Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Chen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chen, J., Liu, Y. Locally linear embedding: a survey. Artif Intell Rev 36, 29–48 (2011). https://doi.org/10.1007/s10462-010-9200-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-010-9200-z

Keywords

Search

Navigation