Advertisement

Short Review of Dimensionality Reduction Methods Based on Stochastic Neighbour Embedding

  • Diego H. Peluffo-OrdóñezEmail author
  • John A. Lee
  • Michel Verleysen
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 295)

Abstract

Dimensionality reduction methods aimed at preserving the data topology have shown to be suitable for reaching high-quality embedded data. In particular, those based on divergences such as stochastic neighbour embedding (SNE). The big advantage of SNE and its variants is that the neighbor preservation is done by optimizing the similarities in both high- and low-dimensional space. This work presents a brief review of SNE-based methods. Also, a comparative analysis of the considered methods is provided, which is done on important aspects such as algorithm implementation, relationship between methods, and performance. The aim of this paper is to investigate recent alternatives to SNE as well as to provide substantial results and discussion to compare them.

Keywords

Dimensionality reduction divergences similarity stochastic neighbor embedding 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Borg, I.: Modern multidimensional scaling: Theory and applications. Springer (2005)Google Scholar
  2. 2.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation 15(6), 1373–1396 (2003)zbMATHCrossRefGoogle Scholar
  3. 3.
    Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)CrossRefGoogle Scholar
  4. 4.
    Hinton, G.E., Roweis, S.T.: Stochastic neighbor embedding. In: Advances in Neural Information Processing Systems, pp. 833–840 (2002)Google Scholar
  5. 5.
    Van der Maaten, L., Hinton, G.: Visualizing data using t-sne. Journal of Machine Learning Research 9(2579-2605), 85 (2008)Google Scholar
  6. 6.
    Lee, J.A., Renard, E., Bernard, G., Dupont, P., Verleysen, M.: Type 1 and 2 mixtures of kullback-leibler divergences as cost functions in dimensionality reduction based on similarity preservation. Neurocomputing (2013)Google Scholar
  7. 7.
    Carreira-Perpinán, M.A.: The elastic embedding algorithm for dimensionality reduction. In: ICML, vol. 10, pp. 167–174 (2010)Google Scholar
  8. 8.
    Durbin, R., Szeliski, R., Yuille, A.: An analysis of the elastic net approach to the traveling salesman problem. Neural Computation 1(3), 348–358 (1989)CrossRefGoogle Scholar
  9. 9.
    Vladymyrov, M., Carreira-Perpiñán, M.Á.: Partial-hessian strategies for fast learning of nonlinear embeddings. CoRR, abs/1206.4646 (2012)Google Scholar
  10. 10.
    Nene, S.A., Nayar, S.K., Murase, H.: Columbia object image library (coil-20). Dept. Comput. Sci., Columbia Univ., New York, 62 (1996), http://www.cs.columbia.edu/CAVE/coil-20.html
  11. 11.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  12. 12.
    Venna, J., Peltonen, J., Nybo, K., Aidos, H., Kaski, S.: Information retrieval perspective to nonlinear dimensionality reduction for data visualization. The Journal of Machine Learning Research 11, 451–490 (2010)zbMATHMathSciNetGoogle Scholar
  13. 13.
    Nocedal, J., Wright, S.: Numerical optimization. Series in operations research and financial engineering. Springer, New York (2006)zbMATHGoogle Scholar
  14. 14.
    Yu, S.X., Shi, J.: Multiclass spectral clustering. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, pp. 313–319. IEEE (2003)Google Scholar
  15. 15.
    Singer, A., Wu, H.-T.: Vector diffusion maps and the connection Laplacian. Communications on Pure and Applied Mathematics 65(8), 1067–1144 (2012)zbMATHMathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Diego H. Peluffo-Ordóñez
    • 1
    Email author
  • John A. Lee
    • 1
    • 2
  • Michel Verleysen
    • 1
  1. 1.Machine Learning Group - ICTEAMUniversité catholique de LouvainLouvain-la-NeuveBelgium
  2. 2.Molecular Imaging Radiotherapy and Oncology - IRECUniversité catholique de LouvainBruxellesBelgium

Personalised recommendations