Incremental Locally Linear Embedding Algorithm

  • Olga Kouropteva
  • Oleg Okun
  • Matti Pietikäinen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3540)


A number of manifold learning algorithms have been recently proposed, including locally linear embedding (LLE). These algorithms not only merely reduce data dimensionality, but also attempt to discover a true low dimensional structure of the data. The common feature of the most of these algorithms is that they operate in a batch or offline mode. Hence, when new data arrive, one needs to rerun these algorithms with the old data augmented by the new data. A solution for this problem is to make a certain algorithm online or incremental so that sequentially coming data will not cause time consuming recalculations. In this paper, we propose an incremental version of LLE and experimentally demonstrate its advantages in terms of topology preservation. Also, compared to the original (batch) LLE, the incremental LLE needs to solve a much smaller optimization problem.


Dimensionality Reduction Cost Matrix Locally Linear Embedding Generalization Algorithm Incremental Version 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Technical Report TR-2002-01, University of Chicago, Department of Computer Science (2002)Google Scholar
  2. 2.
    DeCoste, D.: Visualizing Mercer kernel feature spaces via kernelized locally-linear embeddings. In: Proc. of the 8th Int. Conf. on Neural Information Processing, Shanghai, China (2001)Google Scholar
  3. 3.
    Donoho, D.L., Grimes, G.: Hessian eigenmaps: new locally linear embedding techniques for high-dimensional data. Proc. of National Academy of Sciences 100, 5591–5596 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)CrossRefGoogle Scholar
  5. 5.
    Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323 (2000)CrossRefGoogle Scholar
  6. 6.
    Law, M., Zhang, N., Jain, A.: Nonlinear manifold learning for data stream. In: Berry, M., Dayal, U., Kamath, C., Skillicorn, D. (eds.) Proc. of the 4th SIAM International Conference on Data Mining, Lake Buena Vista, Florida, USA, pp. 33–44 (2004)Google Scholar
  7. 7.
    Bengio, Y., Paiement, J.F., Vincent, P., Delalleau, O., Le Roux, N., Ouimet, M.: Out-of-sample extensions for lle, isomap, mds, eigenmaps, and spectral clustering. In: Thrun, S., Saul, L., Schölkopf, B. (eds.) Advances in Neural Information Processing Systems, vol. 16. MIT Press, Cambridge (2004)Google Scholar
  8. 8.
    Kouropteva, O., Okun, O., Hadid, A., Soriano, M., Marcos, S., Pietikäinen, M.: Beyond locally linear embedding algorithm. Technical Report MVG-01-2002, University of Oulu (2002)Google Scholar
  9. 9.
    Saul, L., Roweis, S.: Think globally, fit locally: unsupervised learning of nonlinear manifolds. Journal of Machine Learning Research 4, 119–155 (2003)CrossRefMathSciNetGoogle Scholar
  10. 10.
    Kouropteva, O., Okun, O., Pietikäinen, M.: Selection of the optimal parameter value for the locally linear embedding algorithm. In: Proc. of 2002 Int. Conf. on Fuzzy Systems and Knowledge Discovery, Singapore, pp. 359–363 (2002)Google Scholar
  11. 11.
    de Ridder, D., Duin, R.: Locally linear embedding for classification. Technical Report PH-2002-01, Delft University of Technology (2002)Google Scholar
  12. 12.
    Kouropteva, O.: Unsupervised learning with locally linear embedding algorithm: an experimental study. Master’s thesis, University of Joensuu, Finland (2001)Google Scholar
  13. 13.
    Bai, Z., Demmel, J., Dongorra, J., Ruhe, A., van der Vorst, H.: Templates for the solution of algebraic eigenvalue problems. SIAM, Philadelphia (2000)zbMATHCrossRefGoogle Scholar
  14. 14.
  15. 15.
  16. 16.
  17. 17.

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Olga Kouropteva
    • 1
  • Oleg Okun
    • 1
  • Matti Pietikäinen
    • 1
  1. 1.Machine Vision Group, Infotech Oulu and Department of Electrical and Information EngineeringUniversity of OuluFinland

Personalised recommendations