Abstract
Many Data Analysis tasks deal with data which are presented in high-dimensional spaces, and the ‘curse of dimensionality’ phenomena is often an obstacle to the use of many methods, including Neural Network methods, for solving these tasks. To avoid these phenomena, various Representation learning algorithms are used, as a first key step in solutions of these tasks, to transform the original high-dimensional data into their lower-dimensional representations so that as much information as possible is preserved about the original data required for the considered task. The above Representation learning problems are formulated as various Dimensionality Reduction problems (Sample Embedding, Data Manifold embedding, Data Manifold reconstruction and newly proposed Tangent Bundle Manifold Learning) motivated by various Data Analysis tasks. A new geometrically motivated algorithm that solves all the considered Dimensionality Reduction problems is presented.
Chapter PDF
Similar content being viewed by others
Keywords
References
Bengio, Y., Courville, A., Vincent, P.: Representation Learning: A Review and New Perspectives. arXiv:1206.5538v2, 1–64 (2012)
Jollie, T.: Principal Component Analysis. Springer, New-York (2002)
Hecht-Nielsen, R.: Replicator neural networks for universal optimal source coding. Science 269, 1860–1863 (1995)
Kramer, M.: Nonlinear Principal Component Analysis using autoassociative neural networks. AIChE Journal 37(2), 233–243 (1991)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Schölkopf, B., Smola, A., Műller, K.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10(5), 1299–1319 (1998)
Cayton, L.: Algorithms for manifold learning. Univ. of California at San Diego (UCSD), Technical Report CS2008-0923, pp. 541–555. Citeseer (2005)
Huo, X., Ni, X., Smith, A.K.: Survey of Manifold-based Learning Methods. In: Liao, T.W., Triantaphyllou, E. (eds.) Recent Advances in Data Mining of Enterprise Data, pp. 691–745. World Scientific, Singapore (2007)
Ma, Y., Fu, Y. (eds.): Manifold Learning Theory and Applications. CRC Press, London (2011)
Bengio, Y., Delalleau, O., Le Roux, N., Paiement, J.-F., Vincent, P., Ouimet, M.: Learning Eigenfunctions Link Spectral Embedding and Kernel PCA. Neural Computation 16(10), 2197–2219 (2004)
Lee, J.A., Verleysen, M.: Quality assessment of dimensionality reduction: Rank-based criteria. Neurocomputing 72(7-9), 1431–1443 (2009)
Chen, J., Deng, S.-J., Huo, X.: Electricity price curve modeling and forecasting by manifold learning. IEEE Transaction on Power Systems 23(3), 877–888 (2008)
Bernstein, A.V., Burnaev, E.V., Chernova, S.S., Zhu, F., Qin, N.: Comparison of Three Geometric Parameterization methods and Their Effect on Aerodynamic Optimization. In: Proceedings of International Conference on Evolutionary and Deterministic Methods for Design, Optimization and Control with Applications to Industrial and Societal Problems, Eurogen 2011, Capua, Italy, September 14-16 (2011)
Gary Wang, G., Shan, S.: Review of Metamodeling Techniques in Support of Engineering Design Optimization. J. Mech. Des. 129(3), 370–381 (2007)
Forrester, A.I.J., Sobester, A., Keane, A.J.: Engineering Design via Surrogate Modelling. A Practical Guide. Wiley, New-York (2008)
Kuleshov, A.P., Bernstein, A.V.: Cognitive Technologies in Adaptive Models of Complex Plants. Information Control Problems in Manufacturing 13(1), 1441–1452 (2009)
Bernstein, A., Kuleshov, A., Sviridenko, Y., Vyshinsky, V.: Fast Aerodynamic Model for Design Technology. In: Proceedings of West-East High Speed Flow Field Conference, WEHSFF-2007. IMM RAS, Moscow (2007), http://wehsff.imamod.ru/pages/s7.html
Bernstein, A.V., Kuleshov, A.P.: Tangent Bundle Manifold Learning via Grassmann & Stiefel Eigenmaps. In arXiv preprint: arXiv:1212.6031v1 [cs.LG], pp. 1–25 (December 2012)
Bernstein, A.V., Kuleshov, A.P.: Manifold Learning: generalizing ability and tangent proximity. International Journal of Software and Informatics 7(3), 359–390 (2013)
Genovese, C.R., Perone-Pacifico, M., Verdinelli, I., Wasserman, L.: Minimax Manifold Estimation. Journal Machine Learning Research 13, 1263–1291 (2012)
Freedman, D.: Efficient simplicial reconstructions of manifold from their samples. IEEE Transaction on Pattern Analysis and Machine Intelligence 24(10), 1349–1357 (2002)
Karygianni, Sofia, Frossard, Pascal. Tangent-based manifold approximation with locally linear models. In: arXiv:1211.1893v1 [cs.LG] (November 6, 2012)
Boissonnat, J.-D., Ghosh, A.: Manifold reconstruction using tangential Delaunay complexes. Discrete & Computational Geometry 51(1), 221–267 (2014)
Canas, G.D., Poggio, T., Rosasco, L.A.: Learning Manifolds with K-Means and K-Flats. In: Advances in Neural Information Processing Systems, NIPS (2012)
Pless, R., Souvenir, R.: A Survey of Manifold Learning for Images. IPSJ Transactions on Computer Vision and Applications 1, 83–94 (2009)
Kuleshov, A.P., Bernstein, A.V.: Tangent Bundle Manifold Learning for Image Analysis. In: Vuksanovic, B., Verikas, A., Zhou, J. (eds.) Proceedings of SPIE, Sixth International Conference on Machine Vision, ICMV 2013, London, The United Kingdom, November 16-17, vol. 9067, pp. 201–205 (2013)
Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation 15, 1373–1396 (2003)
Wolf, L., Shashua, A.: Learning over sets using kernel principal angles. J. Mach. Learn. Res. 4, 913–931 (2003)
Bengio, Y., Monperrus, M.: Non-local manifold tangent learning. In: Saul, L., Weiss, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems, vol. 17, pp. 129–136. MIT Press, Cambridge (2005)
Dollár, P., Rabaud, V., Belongie, S.: Learning to Traverse Image Manifolds. In: Schölkopf, B., Platt, J.C., Hoffman, T. (eds.) Advances in Neural Information Processing Systems, 19, pp. 361–368. MIT Press, Cambridge (2006)
Zhang, Z., Zha, H.: Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment. SIAM Journal on Scientific Computing 26(1), 313–338 (2005)
Kuleshov, A., Bernstein, A., Yanovich, Y.: Asymptotically optimal method in Manifold estimation. In: Márkus, L., Prokaj, V. (eds.) Abstracts of the XXIX-th European Meeting of Statisticians, Budapest, July 20-25, p. 325 (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Bernstein, A., Kuleshov, A. (2014). Low-Dimensional Data Representation in Data Analysis. In: El Gayar, N., Schwenker, F., Suen, C. (eds) Artificial Neural Networks in Pattern Recognition. ANNPR 2014. Lecture Notes in Computer Science(), vol 8774. Springer, Cham. https://doi.org/10.1007/978-3-319-11656-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-11656-3_5
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11655-6
Online ISBN: 978-3-319-11656-3
eBook Packages: Computer ScienceComputer Science (R0)