Nonlinear multi-output regression on unknown input manifold

  • Alexander Kuleshov
  • Alexander BernsteinEmail author


Consider unknown smooth function which maps high-dimensional inputs to multidimensional outputs and whose domain of definition is unknown low-dimensional input manifold embedded in an ambient high-dimensional input space. Given training dataset consisting of ‘input-output’ pairs, regression on input manifold problem is to estimate the unknown function and its Jacobian matrix, as well to estimate the input manifold. By transforming high-dimensional inputs in their low-dimensional features, initial regression problem is reduced to certain regression on feature space problem. The paper presents a new geometrically motivated method for solving both interrelated regression problems.


Manifold learning Regression on manifolds Regression on feature space Manifold estimation Dimensionality reduction 

Mathematics Subject Classification (2010)



Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Vapnik, V.: Statistical Learning Theory. John Wiley, New York (1998)zbMATHGoogle Scholar
  2. 2.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer, New York (2009)CrossRefzbMATHGoogle Scholar
  3. 3.
    James, G., Witten, D., Hastie, T., Tibshirani, R.: An Introduction to Statistical Learning with Applications in R. Springer Texts in Statistics. Springer, New York (2013)zbMATHGoogle Scholar
  4. 4.
    Bishop, C. M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2007)zbMATHGoogle Scholar
  5. 5.
    Deng, L., Yu, D.: Deep Learning: Methods and Applications. NOW Publishers, Boston (2014)zbMATHGoogle Scholar
  6. 6.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefzbMATHGoogle Scholar
  7. 7.
    Friedman, J. H.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 29(5), 1189–1232 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Rasmussen, C. E., Williams, C.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)zbMATHGoogle Scholar
  9. 9.
    Loader, C.: Local Regression and Likelihood. Springer, New York (1999)zbMATHGoogle Scholar
  10. 10.
    Bernstein, A. V., Kuleshov, A. P., Yanovich, Y.A.: Manifold learning in regression tasks. In: Gammerman, A., Vovk, V., Papadopoulos, H (eds.) SLDS 2015. LNCS, vol. 9047, pp 414–423. Springer, Heidelberg (2015)Google Scholar
  11. 11.
    Burnaev, E., Vovk, V.: Efficiency of conformalized ridge regression. JMLR W&,CP 35, 605–622 (2014)Google Scholar
  12. 12.
    Burnaev, E., Panov, M., Zaytsev, A.: Regression on the basis of nonstationary gaussian processes with bayesian regularization. J. Commun. Technol. Electron. 61 (6), 661–671 (2016)CrossRefGoogle Scholar
  13. 13.
    Burnaev, E., Nazarov, I.: Conformalized Kernel Ridge Regression Proceedings of the 15 th IEEE International Conference on Machine Learning and Applications (ICMLA 16), pp 45–52. IEEE Computer Society, USA (2016)Google Scholar
  14. 14.
    Polyak, B. T.: Introduction to optimization. Translations Series in Mathematics and Engineering. Optimization Software Inc. Publications Division, New York (1987)Google Scholar
  15. 15.
    Bertsekas, D. P.: Nonlinear Programming, 2nd edn. Belmont MA, Athena Scientific (1999)zbMATHGoogle Scholar
  16. 16.
    Nesterov, Y.: Introductory Lectures on Convex Optimization. Kluwer, Boston (2004)CrossRefzbMATHGoogle Scholar
  17. 17.
    Wang, G., Gary Shan, S.: Review of metamodeling techniques in support of engineering design optimization. J. Mech. Des. 129(3), 370–381 (2007)CrossRefGoogle Scholar
  18. 18.
    Forrester, A. I. J., Sobester, A., Keane, A. J.: Engineering Design via Surrogate Modelling. A Practical Guide. Wiley, New York (2008)CrossRefGoogle Scholar
  19. 19.
    Kuleshov, A. P., Bernstein, A. V.: Cognitive technologies in adaptive models of complex plants. Inf. Control. Probl. Manuf. 13(1), 1441–1452 (2009)Google Scholar
  20. 20.
    Stone, C. J.: Optimal rates of convergence for nonparametric estimators. Ann. Stat. 8, 1348–1360 (1980)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Stone, C. J.: Optimal global rates of convergence for nonparametric regression. Ann. Stat. 10, 1040–1053 (1982)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Rajaram, D., Pant, R. S.: An improved methodology for airfoil shape optimization using surrogate based design optimization. In: Rodrigues, H. et al. (eds.) Engineering Optimization IV, pp 147–152. CRC Press, Taylor & Francis Group, London (2015)Google Scholar
  23. 23.
    Bernstein, A., Kuleshov, A., Sviridenko, Y., Vyshinsky, V.: Fast aerodynamic model for design technology Proceedings of West-East High Speed Flow Field Conference (WEHSFF-2007), Moscow, Russia (2007). Google Scholar
  24. 24.
    Zhu, F., Qin, N., Burnaev, E. V., Bernstein, A. V., Chernova, S. S.: Comparison of three geometric parameterization methods and their effect on aerodynamic optimization. In: Poloni, C. et al. (eds.) Eurogen 2011, Optimization and Control with Applications to Industrial and Societal Problems International Conference on Proceedings - Evolutionary and Deterministic Methods for Design, pp 758–772. Sira, Capua, Italy (2011)Google Scholar
  25. 25.
    Hu, J., Tian, J., Yang, L.: Functional feature embedded space mapping of fMRI data Proceedings of the 28 th annual IEEE International Conference Engineering in Medicine and Biology Society (EMBS’2006), pp. 1014–1017 (2006)Google Scholar
  26. 26.
    Shen, X., Meyer, F. G.: Analysis of Event-Related fMRI Data Using Diffusion Maps. Lecture Notes in Computer Science, vol. 3565/2005 Information Processing in Medical Imaging, pp. 652–663. Springer, Berlin/Heidelberg (2005)Google Scholar
  27. 27.
    Thirion, B., Faugeras, O.: Nonlinear dimension reduction of fMRI data: The Laplacian embedding approach Proceedings of the IEEE International Symposium on Biomedical Imaging: Nano to Macro, vol. 1, pp 372–375. IEEE, Publisher (2004)Google Scholar
  28. 28.
    Thirion, B., Faugeras, O.: Low Dimensional Embedding of fMRI datasets arXiv:0709.3121v2 [stat.ML], 16 Jan 2008, pp. 1–16 (2008)Google Scholar
  29. 29.
    Mannfolk, P., Wirestam, R., Nilsson, M., Ståhlberg, F., Olsrud, J.: Dimensionality reduction of fMRI time series data using locally linear embedding. Magn. Reson. Mater. Phys., Biol. Med. 23(5), 327–338 (2010)CrossRefGoogle Scholar
  30. 30.
    Gerber, S., Tasdizena, T., Fletcher, P. T., Joshia, S., Whitaker, R.: On the manifold structure of the space of brain images. Lecture Notes in Computer Science, vol. 5761 Medical Image Computing and Computer-Assisted Intervention, pp. 305–312. Springer, Heidelberg (2009)Google Scholar
  31. 31.
    Gerber, S., Tasdizena, T., Fletcher, P. T., Joshia, S., Whitaker, R.: Manifold modeling for brain population analysis. Med. Image Anal. 14(5), 643–653 (2010)CrossRefGoogle Scholar
  32. 32.
    Fletcher, P. T.: Geodesic regression on Riemannian manifolds Proceedings of International Workshop on Mathematical Foundations of Computational Anatomy (MFCA), pp. 75–86 (2011)Google Scholar
  33. 33.
    Fletcher, P. T.: Geodesic regression and the theory of least squares on riemannian manifolds. Int. J. Comput. Vis. 105(2), 171–185 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Banerjee, M., Chakraborty, R., Ofori, E., Vaillancourt, D., Vemuri, B. C.: Nonlinear regression on Riemannian manifolds and its applications to Neuro-image analysis. Lecture Notes in Computer Science, vol. 9349 Medical Image Computing and Computer-Assisted Intervention, Part I, pp 719–727. Springer, Heidelberg (2015)Google Scholar
  35. 35.
    Marcus, D. S., Wang, T. H., et al.: OASIS: Cross-Sectional mri data in young, middle aged, nondemented, and demented older adults. J. Cogn. Neurosci. 19 (9), 1498–1507 (2007)CrossRefGoogle Scholar
  36. 36.
    Seung, H. S., Lee, D. D.: The manifold ways of perception. Science 290 (5500), 2268–2269 (2000)CrossRefGoogle Scholar
  37. 37.
    Huo, X., Ni, X., Smith, A. K.: Survey of manifold-based learning methods. In: Liao, T. W., Triantaphyllou, E. (eds.) Recent Advances in Data Mining of Enterprise Data, pp 691–745. World Scientific, Singapore (2007)Google Scholar
  38. 38.
    Ma, Y., Fu, Y. (eds.): Manifold Learning Theory and Applications. CRC Press, London (2011)Google Scholar
  39. 39.
    Bernstein, A. V., Kuleshov, A. P.: Low-Dimensional Data representation in data analysis. In: EI Gayar, N., Schwenker, F., Suen, C (eds.) ANNPR 2014. LNCS, vol. 8774, pp 47–58. Springer, Heidelberg (2014)Google Scholar
  40. 40.
    Kuleshov, A. P., Bernstein, A. V.: Manifold learning in data mining tasks. In: Perner, P (ed.) MLDM 2014. LNCS, vol. 8556, pp 119–133. Springer, Heidelberg (2014)Google Scholar
  41. 41.
    Pelletier, B.: Nonparametric regression estimation on closed Riemannian manifolds. J. Nonparametric Stat. 18(1), 57–67 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Loubes, J. -M., Pelletier, B.: A kernel-based classifier on a Riemannian manifold. Statistics and Decisions 26(1), 35–51 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Steinke, F., Hein, M., Schölkopf, B.: Nonparametric regression between general Riemannian manifolds. SIAM J. Imaging Sci. 3(3), 527–563 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Levina, E., Bickel, P. J.: Maximum likelihood estimation of intrinsic dimension. In: Saul, L., Weiss, Y., Bottou, L (eds.) Advances in Neural Information Processing Systems, vol. 17, pp 777–784. MIT Press, Cambridge (2005)Google Scholar
  45. 45.
    Fan, M., Qiao, H., Zhang, B.: Intrinsic dimension estimation of manifolds by incising balls. Pattern Recogn. 42, 780–787 (2009)CrossRefzbMATHGoogle Scholar
  46. 46.
    Fan, M., Gu, N., Qiao, H., Zhang, B.: Intrinsic dimension estimation of data by principal component analysis arXiv:1002.2050v1 [cs.CV], 10 Feb 2010, pp 1–8 (2010)Google Scholar
  47. 47.
    Rozza, A., Lombardi, G., Rosa, M., Casiraghi, E., Campadelli, P.: IDEA: Intrinsic Dimension estimation algorithm. In: Maino, G., Foresti, G. (eds.) ICIAP 2011, Part I. LNCS, vol. 6978, pp 433–442. Springer, Heidelberg (2011)Google Scholar
  48. 48.
    Campadelli, P., Casiraghi, E., Ceruti, C., Rozza, A.: Intrinsic Dimension Estimation: Relevant Techniques and a Benchmark Framework. Mathematical Problems in Engineering vol. 2015, Article ID 759567, pp. 1–21 (2015)Google Scholar
  49. 49.
    Banerjee, M., Chakraborty, R., Ofori, E., Okun, M. S., Vaillancourt, D., Vemuri, B. C.: A nonlinear regression technique for manifold valued data with applications to Medical Image Analysis Conference: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 4424–4432 (2016)Google Scholar
  50. 50.
    Hinkle, J., Muralidharan, P., Fletcher, P. T.: Polynomial regression on riemannian manifolds arXiv:1201.2395v2 [math.ST], 1 Mar 2012, pp. 1–14 (2012)Google Scholar
  51. 51.
    Liu, G., Lin, Z., Yu, Y.: Multi-output regression on the output manifold. Pattern Recogn. 42(11), 2737–2743 (2009)CrossRefzbMATHGoogle Scholar
  52. 52.
    Shi, X., Styner, M., Lieberman, J., Ibrahim, J. G., Lin, W., Zhu, H.: Intrinsic regression models for manifold-valued data. J. Amer. Stat. Assoc. 5762(1), 192–199 (2009)Google Scholar
  53. 53.
    Kim, H. J., Bendlin, B. B., et al.: Multivariate general linear models (MGLM) on riemannian manifolds with applications to statistical analysis of diffusion weighted images Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2705–2712 (2014)Google Scholar
  54. 54.
    Kim, H. J., Adluru, N., et al.: Canonical Correlation analysis on Riemannian Manifolds and its Applications. Lecture Notes in Computer Science, vol. 8690 Computer Vision – ECCV 2014, pp. 251–267. Springer, Heidelberg (2014)Google Scholar
  55. 55.
    Bickel, P., Li, B.: Local polynomial regression on unknown manifolds. IMS Lecture notes – Monograph Series, vol. 54 Complex Datasets and Inverse Problems: Tomography Networks and Beyond, pp. 177–186 (2007)Google Scholar
  56. 56.
    Aswani, A., Bickel, P., Tomlin, C.: Regression on manifolds: Estimation of the exterior derivative. Ann. Stat. 39(1), 48–81 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  57. 57.
    Cheng, M. -Y., Wu, H. -T.: Local Linear Regression on Manifolds and its Geometric Interpretation. J. Am. Stat. Assoc. 108(504), 1421–1434 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  58. 58.
    Yang, Y., Dunson, D. B.: Bayesian manifold regression arXiv:1305.0167v2 [math.ST], 16 June 2014, pp. 1–40 (2014)Google Scholar
  59. 59.
    Einbeck, J., Evers, L.: Localized regression on principal manifolds Proceedings of the 25 th International Workshop on Statistical Modeling (IWSM 2010), pp. 1–6 (2010)Google Scholar
  60. 60.
    Niyogi, P., Smale, S., Weinberger, S.: Finding the homology of submanifolds with high confidence from random samples. Discrete Comput. Geom. 39, 419–441 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  61. 61.
    Pennec, X.: Probabilities and statistics on riemannian manifolds: Basic tools for geometric measurements Proceedings of the International Workshop on Nonlinear Signal and Image Processing (NSIP’99), pp. 194–198 (1999)Google Scholar
  62. 62.
    Yanovich, Y.U.: Asymptotic properties of local sampling on manifolds. J. Math. Stat. 12(3), 157–175 (2016)CrossRefGoogle Scholar
  63. 63.
    Adragni, K. P., Cook, R. D.: Sufficient Dimension Reduction and prediction in Regression. Phil. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 367(1906), 4385–4405 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  64. 64.
    Lee, K. -Y., Li, B., Chiaromonte, F.: A general theory for nonlinear sufficient dimension reduction: formulation and estimation. Ann. Stat. 41(1), 221–249 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  65. 65.
    Chen, J., Deng, S. -J., Huo, X.: Electricity price curve modeling and forecasting by manifold learning. IEEE Trans. Power Syst. 23(3), 877–888 (2008)CrossRefGoogle Scholar
  66. 66.
    Lee, J. A., Verleysen, M.: Quality assessment of dimensionality reduction based on k-ary neighborhoods. In: Saeys, Y. et al. (eds.) JMLR Workshop and Conference Proceedings New Challenges for Feature Selection in Data Mining and Knowledge Discovery, vol. 4, pp 21–35. Antwerpen, Belgium (2008)Google Scholar
  67. 67.
    Lee, J. A., Verleysen, M.: Quality assessment of dimensionality reduction: Rank-based criteria. Neurocomputing 72(7-9), 1431–1443 (2009)CrossRefGoogle Scholar
  68. 68.
    Bernstein, A.V., Kuleshov, A.P.: Data-based manifold reconstruction via tangent bundle manifold learning ICML-2014 Workshop Topological Methods for Machine Learning, Beijing, China, June 2014. Scholar
  69. 69.
    Freedman, D.: Efficient simplicial reconstructions of manifold from their samples. IEEE Trans. Pattern Anal. Mach. Intell. 24(10), 1349–1357 (2002)CrossRefGoogle Scholar
  70. 70.
    Levin, D.: Mesh-independent surface interpolation. In: Hamann, Brunnett, Mueller (eds.) Mathematics and Visualization Series, Geometric Modeling for Scientific Visualization, part 1, pp 37–49. Springer, Berlin, Heidelberg (2003)Google Scholar
  71. 71.
    Kolluri, R., Shewchuk, J. R., O’Brien, J. F.: Spectral surface reconstruction from noisy point clouds Proceedings of the 2004 Eurographics/ACM SIGGRAPH Symposium on Geometry Processing (SGP’04), pp 11–21. ACM Press, New York (2004)Google Scholar
  72. 72.
    Boissonnat, J. -D., Ghosh, A.: Manifold reconstruction using tangential Delaunay complexes. Discrete Comput. Geom. 51(1), 221–267 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  73. 73.
    Karygianni, S., Frossard, P.: Tangent-based manifold approximation with locally linear models. Signal Process. 104, 232–247 (2014)CrossRefGoogle Scholar
  74. 74.
    Canas, G. D., Poggio, T., Rosasco, L.: Learning manifolds with K-Means and K-Flats arXiv:1209.1121v4 [cs.LG], 19 Feb 2013, pp. 1–19 (2013)
  75. 75.
    Jollie, T.: Principal component analysis. Springer, New York (2002)Google Scholar
  76. 76.
    Kramer, M.: Nonlinear Principal Component Analysis using autoassociative neural networks. AIChE J 37(2), 233–243 (1991)CrossRefGoogle Scholar
  77. 77.
    Hinton, G. E., Salakhutdinov, R. R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  78. 78.
    Saul, L. K., Roweis, S. T.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)CrossRefGoogle Scholar
  79. 79.
    Saul, L. K., Roweis, S. T.: Think globally, fit locally: unsupervised learning of low dimensional manifolds. J. Mach. Learn. Res. 4, 119–155 (2003)MathSciNetzbMATHGoogle Scholar
  80. 80.
    Zhang, Z., Zha, H.: Principal manifolds and nonlinear dimension reduction via local tangent space alignment. SIAM J. Sci. Comput. 26(1), 313–338 (2005)CrossRefzbMATHGoogle Scholar
  81. 81.
    Bernstein, A. V., Kuleshov, A. P.: Tangent bundle manifold learning via Grassmann&Stiefel eigenmaps arXiv:1212.6031v1 [cs.LG], December 2012, pp. 1–25 (2012)
  82. 82.
    Kuleshov, A., Bernstein, A., Yanovich, Y.U.: Asymptotically optimal method in Manifold estimation. In: Márkus, L., Prokaj, V. (eds.) Abstracts of the 29th European Meeting of Statisticians (EMS-2003), Budapest, Hungary, 20-25 July 2013, p. 325 (2013)Google Scholar
  83. 83.
    Law, M. H. C., Jain, A. K.: Nonlinear manifold learning for data stream. In: Berry, M., Dayal, U., Kamath, C., Skillicorn, D (eds.) Proceedings of the 4 th SIAM International Conference on Data Mining, Like Buena Vista, Florida, USA, 33–44 (2004)Google Scholar
  84. 84.
    Law, M. H. C., Jain, A. K.: Incremental nonlinear dimensionality reduction by manifold learning. IEEE Trans. Pattern Anal. Mach. Intell. 28(3), 377–391 (2006)CrossRefGoogle Scholar
  85. 85.
    Gao, X., Jiye Liang, J.: An improved incremental nonlinear dimensionality reduction for isometric data embedding. Inf. Process. Lett. 115(4), 492–501 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  86. 86.
    Kouropteva, O., Okun, O., Pietikäinen, M.: Incremental locally linear embedding algorithm. In: Kalviainen, H. et al. (eds.) 14th Scandinavian Conference on Image Analysis (SCIA) 2005, LNCS, vol. 3540, pp 521–530. Springer-Verlag, Berlin, Heidelberg (2005)Google Scholar
  87. 87.
    Kouropteva, O., Okun, O., Pietikäinen, M.: Incremental locally linear embedding. Pattern Recogn. 38(10), 1764–1767 (2005)CrossRefzbMATHGoogle Scholar
  88. 88.
    Schuon, S., Ðurković, M., Diepold, K., Scheuerle, J., Markward, S.: Truly Incremental Locally Linear Embedding Proceedings of the CoTeSys 1st International Workshop on Cognition for Technical Systems (6-8 October 2008), munich, Germany, 5 pp (2008)Google Scholar
  89. 89.
    Jia, P., Yin, J., Huang, X., Hu, D.: Incremental Laplacian eigenmaps by preserving adjacent information between data points. Pattern Recogn. Lett. 30(16), 1457–1463 (2009)CrossRefGoogle Scholar
  90. 90.
    Liu, X., Yin, J., Feng, Z., Dong, J.: Incremental Manifold Learning Via Tangent Space Alignment. Schwenker, F., Marinai, S. (eds.) ANNPR 2006, LNAI, Vol. 4087, pp. 107–121. Springer-Verlag. Berlin, Heidelberg (2006)Google Scholar
  91. 91.
    Abdel-Mannan, O., Ben Hamza, A., Youssef, A.: Incremental line tangent space alignment algorithm Proceedings of 2007 Canadian Conference on Electrical and Computer Engineering (CCECE 2007), 22-26 April 2007, Vancouver, pp. 1329–1332. IEEE (2007)Google Scholar
  92. 92.
    Han, Z., Meng, D.-Y., Xu, Z.-B., Gu, N.-N.: Incremental alignment manifold learning. J. Comput. Sci. Technol. 26(1), 153–165 (2011)CrossRefGoogle Scholar
  93. 93.
    Chao Tan, C., Guan, J.: A new manifold learning algorithm based on incremental spectral decomposition. In: Zhou, S., Zhang, S., Karypis, G (eds.) 8th International Conference on Advanced Data Mining and Applications (ADMA) Nanjing, China, December 15-18, 2012, LNCS, vol. 7713, pp 149–160. Springer-Verlag, Berlin, Heidelberg (2012)Google Scholar
  94. 94.
    Kuleshov, A., Bernstein, A.: Incremental construction of low-dimensional data representations. In: Schwenker, F., EI Gayar, N., Abbas, H. M., Trentin, E (eds.) ANNPR 2016. LNCS, vol. 9896, pp 55– 67. Springer, Heidelberg (2016)Google Scholar
  95. 95.
    Golub, G. H., Van Loan, C. F.: Matrix Computation, 3rd edn. Johns Hopkins University Press, Baltimore, MD (1996)Google Scholar
  96. 96.
    Bernstein, A. V., Kuleshov, A. P.: Manifold Learning: generalizing ability and tangent proximity. Int. J. Softw. Eng. Inform. 7(3), 359–390 (2013)Google Scholar
  97. 97.
    Jost, J.: Riemannian Geometry and Geometric Analysis, 6th edn. Springer-Verlag, Berlin, Heidelberg (2011)Google Scholar
  98. 98.
    Lee, J. M.: Manifolds and Differential Geometry Graduate Studies in Mathematics, vol. 107. American Mathematical Society, Providence (2009)Google Scholar
  99. 99.
    Perrault-Joncas, D., Meilă, M.: Non-linear dimensionality reduction: riemannian metric estimation and the problem of geometric recovery arXiv:1305.7255v1 [stat.ML], 30 May 2013, pp. 1–25 (2013)
  100. 100.
    Kuleshov, A. P., Bernstein, A. V.: Extended Regression on Manifolds Estimation. Lecture Notes in Artificial Intelligence, vol. 9653 Conformal and Probabilistic Prediction with Applications, pp. 208–228. Springer, Heidelberg (2016)Google Scholar
  101. 101.
    Singer, A., Wu, H. -T.: Vector diffusion maps and the connection laplacian. Commun. Pure. Appl. Math. 65(8), 1067–1144 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  102. 102.
    Tyagi, H., Vural, E., Frossard, P.: Tangent space estimation for smooth embeddings of Riemannian manifold arXiv:1208.1065v2 [stat.CO], 17 May 2013, pp. 1–35 (2013)
  103. 103.
    Kaslovsky, D. N., Meyer, F. G.: Non-asymptotic analysis of tangent space perturbation. Inf. Inf. J. IMA 3(2), 134–187 (2014)MathSciNetzbMATHGoogle Scholar
  104. 104.
    Hamm, J.: Lee, Daniel D.: Grassmann discriminant analysis: a unifying view on subspace-based learning Proceedings of the 25 th International Conference on Machine Learning (ICML 2008), pp. 376–383 (2008)Google Scholar
  105. 105.
    Wolf, L., Shashua, A.: Learning over sets using kernel principal angles. J. Mach. Learn. Res. 4, 913–931 (2003)MathSciNetzbMATHGoogle Scholar
  106. 106.
    Cormen, T., Leiserson, C., Rivest, R., Stein, C.: Introduction to Algorithms. MIT Press, Cambridge (2001)Google Scholar
  107. 107.
    Fredman, M., Tarjan, R. E.: Fibonacci heaps and their uses in improved network optimization algorithms. J. Assoc. Comput. Mach. 34(3), 596–615 (1987)MathSciNetCrossRefGoogle Scholar
  108. 108.
    Bernstein, A. V., Kuleshov, A. P., Yanovich, Y.A.: Information preserving and locally isometric&conformal embedding via tangent manifold learning Proceedings of the International IEEE Conference on Data Science and Advanced Analytics (DSAA 2015), pp 1–9. IEEE Computer Society, Piscataway, USA (2015)Google Scholar
  109. 109.
    Trefethen, L.N., Bau, D. III: Numerical. Linear Algebra. SIAM, Philadelphia (1997)Google Scholar
  110. 110.
    Wasserman, L.: All of Nonparametric Statistics. Springer Texts in Statistics, Berlin (2007)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  1. 1.Skolkovo Institute of Science and TechnologyMoscowRussia
  2. 2.National Research University Higher School of EconomicsMoscowRussia

Personalised recommendations