A Sparse-Grid-Based Out-of-Sample Extension for Dimensionality Reduction and Clustering with Laplacian Eigenmaps

  • Benjamin Peherstorfer
  • Dirk Pflüger
  • Hans-Joachim Bungartz
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7106)


Spectral graph theoretic methods such as Laplacian Eigenmaps are among the most popular algorithms for manifold learning and clustering. One drawback of these methods is, however, that they do not provide a natural out-of-sample extension. They only provide an embedding for the given training data. We propose to use sparse grid functions to approximate the eigenfunctions of the Laplace-Beltrami operator. We then have an explicit mapping between ambient and latent space. Thus, out-of-sample points can be mapped as well. We present results for synthetic and real-world examples to support the effectiveness of the sparse-grid-based explicit mapping.


spectral methods manifold learning clustering sparse grids 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anderson, E., Bai, Z., Bischof, C., Blackford, L.S., Demmel, J., Dongarra, J.J., Croz, J.D., Hammarling, S., Greenbaum, A., McKenney, A., Sorensen, D.: LAPACK Users’ guide, 3rd edn. Society for Industrial and Applied Mathematics, Philadelphia (1999)CrossRefzbMATHGoogle Scholar
  2. 2.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Tech. Rep. TR-2002-01, The University of Chicago CS (January 2002)Google Scholar
  3. 3.
    Bengio, Y., Paiement, J., Vincent, P.: Out-of-Sample extensions for LLE, isomap, MDS, eigenmaps, and spectral clustering. In: Advances in Neural Information Processing Systems, pp. 177–184. MIT Press (2003)Google Scholar
  4. 4.
    Bishop, C.M., James, G.D.: Analysis of multiphase flows using dual-energy gamma densitometry and neural networks. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 327(2-3), 580–593 (1993)CrossRefGoogle Scholar
  5. 5.
    Bungartz, H.-J., Griebel, M.: Sparse grids. Acta Numerica 13 (2004)Google Scholar
  6. 6.
    Fowlkes, C., Belongie, S., Chung, F., Malik, J.: Spectral grouping using the nystrom method. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(2), 214–225 (2004)CrossRefGoogle Scholar
  7. 7.
    Garcke, J., Griebel, M., Thess, M.: Data mining with sparse grids. Computing 67(3), 225–253 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    He, X., Yan, S., Hu, Y., Zhang, H.: Learning a locality preserving subspace for visual recognition. In: Ninth IEEE International Conference on Computer Vision, vol. 1, pp. 385–392 (2003)Google Scholar
  9. 9.
    Heinecke, A., Pflüger, D.: Multi- and many-core data mining with adaptive sparse grids. In: Proceedings of the 2011 ACM International Conference on Computing Frontiers (May 2011)Google Scholar
  10. 10.
    Huang, D., Zhang, X., Huang, G., Pang, Y., Zhang, L., Liu, Z., Yu, N., Li, H.: Neighborhood Preserving Projections (NPP): a Novel Linear Dimension Reduction Method. In: Huang, D.-S., Zhang, X.-P., Huang, G.-B. (eds.) ICIC 2005. LNCS, vol. 3644, pp. 117–125. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  11. 11.
    Hubert, L., Arabie, P.: Comparing partitions. Journal of Classification 2(1), 193–218 (1985)CrossRefzbMATHGoogle Scholar
  12. 12.
    von Luxburg, U.: A tutorial on spectral clustering. Statistics and Computing 17, 395–416 (2007)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Pflüger, D.: Spatially Adaptive Sparse Grids for High-Dimensional Problems. Dissertation, Institut für Informatik, Technische Universität München, München (February 2010)Google Scholar
  14. 14.
    Pflüger, D., Peherstorfer, B., Bungartz, H.J.: Spatially adaptive sparse grids for high-dimensional data-driven problems. Journal of Complexity 26(5), 508–522 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Qiao, H., Zhang, P., Wang, D., Zhang, B.: An explicit nonlinear mapping for manifold learning. CoRR abs/1001.2605 (2010)Google Scholar
  16. 16.
    Schraufstetter, S., Benk, J.: A general pricing technique based on theta-calculus and sparse grids. In: Proceedings of the ENUMATH 2009 Conference, Uppsala (December 2009)Google Scholar
  17. 17.
    Williams, C., Seeger, M.: Using the nyström method to speed up kernel machines. In: Advances in Neural Information Processing Systems, vol. 13, pp. 682–688. MIT Press (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Benjamin Peherstorfer
    • 1
  • Dirk Pflüger
    • 1
  • Hans-Joachim Bungartz
    • 1
  1. 1.Department of InformaticsTechnische Universität MünchenGarchingGermany

Personalised recommendations