Regularized Kernel-Based Reconstruction in Generalized Besov Spaces

  • Michael Griebel
  • Christian Rieger
  • Barbara Zwicknagl
Article
  • 76 Downloads

Abstract

We present a theoretical framework for reproducing kernel-based reconstruction methods in certain generalized Besov spaces based on positive, essentially self-adjoint operators. An explicit representation of the reproducing kernel is given in terms of an infinite series. We provide stability estimates for the kernel, including inverse Bernstein-type estimates for kernel-based trial spaces, and we give condition estimates for the interpolation matrix. Then, a deterministic error analysis for regularized reconstruction schemes is presented by means of sampling inequalities. In particular, we provide error bounds for a regularized reconstruction scheme based on a numerically feasible approximation of the kernel. This allows us to derive explicit coupling relations between the series truncation, the regularization parameters and the data set.

Keywords

Reproducing kernels A priori error analysis Generalized Besov spaces Feasible reconstruction schemes Spline smoothing 

Mathematics Subject Classification

41A17 41A25 41A58 42A82 62G08 

Notes

Acknowledgements

We are grateful for the comments and suggestions of the anonymous referees. The authors acknowledge support of the Deutsche Forschungsgemeinschaft (DFG) through the Sonderforschungsbereich 1060: The Mathematics of Emergent Effects.

References

  1. 1.
    R. A. Adams and J. J. F. Fournier, Sobolev Spaces, Academic Press, Oxford (UK), 2003.MATHGoogle Scholar
  2. 2.
    R. Arcangéli, M. C. L. di Silanes, and J. J. Torrens, An extension of a bound for functions in Sobolev spaces, with applications to (m,s)-spline interpolation and smoothing, Numer. Math., 107(2) (2007), pp. 181–211.Google Scholar
  3. 3.
    R. Arcangéli, M. C. L. di Silanes, and J. J. Torrens, Estimates for functions in Sobolev spaces defined on unbounded domains, J. Approx. Theory, 161 (2009), pp. 198 – 212.MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    R. Arcangéli, M. C. L. di Silanes, and J. J. Torrens, Extension of sampling inequalities to Sobolev semi-norms of fractional order and derivative data, Numer. Math,, 121 (2012), pp. 587–608.MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    L. Boytsov and B. Naidan, Learning to prune in metric and non-metric spaces, in Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States., C. J. C. Burges, L. Bottou, Z. Ghahramani, and K. Q. Weinberger, eds., 2013, pp. 1574–1582.Google Scholar
  6. 6.
    T. Bozkaya and M. Ozsoyoglu, Distance-based indexing for high-dimensional metric spaces, in Proceedings of the 1997 ACM SIGMOD International Conference on Management of Data, SIGMOD ’97, New York, NY, USA, 1997, ACM, pp. 357–368.Google Scholar
  7. 7.
    S. Chandrasekaran, K. R. Jayaraman, and H. N. Mhaskar, Minimum Sobolev norm interpolation with trigonometric polynomials on the torus, J. Comput. Phys, 249 (2013), pp. 96 – 112.MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    S. Chandrasekaran and H. N. Mhaskar, A construction of linear bounded interpolatory operators on the torus, Preprint. https://arxiv.org/pdf/1011.5448.pdf.
  9. 9.
    T. Coulhon and A. Grigor’yan, Random walks on graphs with regular volume growth, Geom. Funct. Anal., 8 (1998), pp. 656–701.MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    T. Coulhon, G. Kerkyacharian, and P. Petrushev, Heat kernel generated frames in the setting of Dirichlet spaces, J. Fourier Anal. Appl., 18 (2012), pp. 995–1066.MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    F. Cucker and D.-X. Zhou, Learning Theory: An Approximation Theory Viewpoint, Cambridge Monographs on Applied and Computational Mathematics, Cambridge University Press, 2007.CrossRefMATHGoogle Scholar
  12. 12.
    P. C. Curtis, \(n\) -parameter families and best approximation, Pac. J. Math., 9 (1959), p. 10131027.MathSciNetCrossRefGoogle Scholar
  13. 13.
    N. Dunford and J. Schwartz, Linear Operators, Part I: General Theory, Interscience Publishers, New York, 1958.Google Scholar
  14. 14.
    N. Dyn, F. J. Narcowich, and J. D. Ward, Variational principles and Sobolev-type estimates for generalized interpolation on a Riemannian manifold, Constr. Approx., 15 (1999), pp. 175–208.Google Scholar
  15. 15.
    P. F. Evangelista, M. J. Embrechts, and B. K. Szymanski, Taming the curse of dimensionality in kernels and novelty detection, in Applied Soft Computing Technologies: The Challenge of Complexity, A. Abraham, B. de Baets, M. Köppen, and B. Nickolay, eds., vol. 34 of Advances in Soft Computing, Springer Berlin Heidelberg, 2006, pp. 425–438.Google Scholar
  16. 16.
    D. Geller and I. Z. Pesenson, Band-limited localized Parseval frames and Besov spaces on compact homogeneous manifolds, J. Geomet. Anal., 21 (2011), pp. 334–371.Google Scholar
  17. 17.
    A. Globerson and S. Roweis, Metric learning by collapsing classes, Adv. Neural Inf. Process Syst., 18 (2006), pp. 451–458.Google Scholar
  18. 18.
    M. Gordina, T. Kumagai, L. Saloff-Coste, and K.-T. Sturm, Heat kernels, stochastic processes and functional inequalities, Oberwolfach Reports, 10 (2013), pp. 1359–1443.MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    M. Griebel, C. Rieger, and B. Zwicknagl, Multiscale approximation and reproducing kernel Hilbert space methods, SIAM J. Numer. Anal., 53 (2015), pp. 852–873.MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    A. Grigor’yan, Heat kernels on weighted manifolds and applications, Cont. Math., 398 (2006), pp. 93–191.MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    A. Grigor’yan, Heat Kernel and Analysis on Manifolds, vol. 47 of AMS/IP Studies in Advanced Mathematics, American Mathematical Society, USA, 2009.Google Scholar
  22. 22.
    T. Hangelbroek, F. J. Narcowich, C. Rieger, and J. D. Ward, An inverse theorem for compact Lipschitz regions in \(\mathbb{R}^d\) using localized kernel bases. Math. Comp., AMS early view version. doi: 10.1090/mcom/3256.
  23. 23.
    T. Hanglebroek, F. J. Narcowich, X. Sun, and J. D. Ward, Kernel approximation on manifolds II: the \(L_{\infty }\) projector, SIAM J. Math. Anal., 43 (2011), pp. 662–684.Google Scholar
  24. 24.
    K. Jetter, J. Stöckler, and J. D. Ward, Norming sets and scattered data approximation on spheres, in Approximation Theory IX, Vol. II: Computational Aspects, Vanderbilt University Press, 1998, pp. 137 – 144.Google Scholar
  25. 25.
    P. Jorgensen and F. Tian, Frames and factorization of graph Laplacians, Opuscula Math., 35 (2015), pp. 293–332.Google Scholar
  26. 26.
    A. Kaenmaki, J. Lehrback, and M. Vuorinen, Dimensions, Whitney covers, and tubular neighborhoods, Indiana Univ. Math. J., 62 (2013), pp. 1861–1889.Google Scholar
  27. 27.
    E. Keogh and A. Mueen, Curse of dimensionality, in Encyclopedia of Machine Learning, C. Sammut and G. I. Webb, eds., Springer US, 2010, pp. 257–258.Google Scholar
  28. 28.
    G. Kerkyacharian and P. Petrushev, Heat kernel based decomposition of spaces of distributions in the framework of Dirichlet spaces, Trans. Amer. Math. Soc., 367 (2015), pp. 121–189.MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    S. Lin, Nonparametric regression using needlet kernels for spherical data. available at arXiv:1502.04168, 2015.
  30. 30.
    W. Madych, An estimate for multivariate approximation II, J. Approx. Theory, 142 (2006), pp. 116–128.MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    M. Maggioni and H. N. Mhaskar, Diffusion polynomial frames on metric measure spaces, Appl. Comp. Harm. Anal., 24 (2008), pp. 329 – 353.MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    J. Mairhuber, On Haar’s theorem concerning Chebysheff problems having unique solutions, Proc. Amer. Math. Soc., 7 (1956), pp. 609–615.Google Scholar
  33. 33.
    H. N. Mhaskar, A Markov- Bernstein inequality for Gaussian networks, in Trends and Applications in Constructive Approximation, vol. 151 of Internat. Ser. Numer. Math., Birkhäuser, Basel, 2005, pp. 165–180.Google Scholar
  34. 34.
    H. N. Mhaskar, Eignets for function approximation on manifolds, Appl. Comp. Harm. Anal., 29 (2010), pp. 63 – 87.MathSciNetCrossRefMATHGoogle Scholar
  35. 35.
    H. N. Mhaskar, F. J. Narcowich, J. Prestin, and J. D. Ward, \(\text{L}^{p}\) Bernstein estimates and approximation by spherical basis functions, Math. Comp., 79 (2010), pp. 1647–1679.Google Scholar
  36. 36.
    F. J. Narcowich, P. Petrushev, and J. D. Ward, Decomposition of Besov and Triebel-Lizorkin spaces on the sphere, J. Funct. Anal., 238 (2006), pp. 530–564.Google Scholar
  37. 37.
    F. J. Narcowich, X. Sun, J. D. Ward, and H. Wendland, Direct and inverse Sobolev error estimates for scattered data interpolation via spherical basis functions, Found. Comput. Math., 7 (2007), pp. 369–390.Google Scholar
  38. 38.
    F. J. Narcowich, J. D. Ward, and H. Wendland, Sobolev error estimates and a Bernstein inequality for scattered data interpolation via radial basis functions, Constr. Approx., 24 (2006), pp. 175–186.Google Scholar
  39. 39.
    F. J. Narcowich, P. Petrushev, and J. D. Ward, Localized Tight Frames on Spheres, SIAM J. Math. Anal., 38 (2006), pp. 574–594.MathSciNetCrossRefMATHGoogle Scholar
  40. 40.
    F. J. Narcowich and J. D. Ward, Scattered-data interpolation on \(\mathbb{R} ^n\): Error estimates for radial basis and band-limited functions, SIAM J. Math. Anal., 36 (2004), pp. 284–300.Google Scholar
  41. 41.
    F. J. Narcowich, J. D. Ward, and H. Wendland, Sobolev bounds on functions with scattered zeros, with applications to radial basis function surface fitting, Math. Comp., 74 (2005), pp. 743–763.Google Scholar
  42. 42.
    R. Opfer, Multiscale kernels, Adv. Comp. Math., 25 (2006), pp. 357–380.MathSciNetCrossRefMATHGoogle Scholar
  43. 43.
    R. Opfer, Tight frame expansions of multiscale reproducing kernels in Sobolev spaces, Appl. Comput. Harm. Anal., 20 (2006), pp. 357–374.CrossRefMATHGoogle Scholar
  44. 44.
    I. Pesenson, A sampling theorem on homogeneous manifolds, Trans. Amer. Math. Soc., 352 (2000), pp. 4257–4269.Google Scholar
  45. 45.
    P. Petrushev and Y. Xu, Decomposition of spaces of distributions induced by Hermite expansions, J. Fourier Anal. Appl., 14 (2008), pp. 371–414.MathSciNetCrossRefMATHGoogle Scholar
  46. 46.
    C. Rieger, Sampling inequalities and applications, PhD thesis, University of Göttingen, 2008. http://hdl.handle.net/11858/00-1735-0000-0006-B3B9-0.
  47. 47.
    C. Rieger, R. Schaback, and B. Zwicknagl, Sampling and stability, Mathematical Methods for Curves and Surfaces, vol. 5862 of Lecture Notes in Computer Science, M. Dæhlen, M. Floater, T. Lyche, J. L. Merrien, K. Mørken, and L. L. Schumaker, eds., Springer, Berlin, Heidelberg, 2010, pp. 347–369.Google Scholar
  48. 48.
    C. Rieger and B. Zwicknagl, Deterministic error analysis of support vector machines and related regularized kernel methods, J. Mach. Learn. Res., 10 (2009), pp. 2115–2132.MathSciNetMATHGoogle Scholar
  49. 49.
    C. Rieger and B. Zwicknagl, Sampling inequalities for infinitely smooth functions, with applications to interpolation and machine learning, Adv. Comp. Math., 32(1) (2010), pp. 103–129.MathSciNetCrossRefMATHGoogle Scholar
  50. 50.
    R. Schaback and H. Wendland, Inverse and saturation theorems for radial basis function interpolation, Math. Comp., 71 (2002), pp. 669–681.MathSciNetCrossRefMATHGoogle Scholar
  51. 51.
    R. Schaback and H. Wendland, Kernel techniques: From machine learning to meshless methods, Acta Numerica, 15 (2006), pp. 543–639.MathSciNetCrossRefMATHGoogle Scholar
  52. 52.
    B. Schölkopf and A. J. Smola, Learning with kernels - Support Vector Machines, Regularisation, and Beyond, MIT Press, Cambridge, Massachusetts, 2002.Google Scholar
  53. 53.
    S. Smale and D.-X. Zhou, Estimating the approximation error in learning theory, Anal. Appl., 01 (2003), pp. 17–41.MathSciNetCrossRefMATHGoogle Scholar
  54. 54.
    B. K. Sriperumbudur, A. Gretton, K. Fukumizu, B. Schölkopf, and G. R. G. Lanckriet, Hilbert space embeddings and metrics on probability measures, J. Mach. Learn. Res., 11 (2010), pp. 1517–1561.Google Scholar
  55. 55.
    H. Triebel, Interpolation theory, function spaces, differential operators, North-Holland Mathematical Library, Amsterdam-New York, 1978.MATHGoogle Scholar
  56. 56.
    H. Triebel, Theory of function spaces, vol. 78 of Monographs in Math., Birkhäuser Verlag, Basel, 1983.Google Scholar
  57. 57.
    F. Utreras, Convergence rates for multivariate smoothing spline functions, J. Approx. Theory, 52 (1988), pp. 1–27.MathSciNetCrossRefMATHGoogle Scholar
  58. 58.
    J. P. Ward, \(\text{ L }^p\) Bernstein inequalities and inverse theorems for RBF approximation on \(\text{ R }^d\) , J. Approx. Theory., 164 (2012), pp. 1577–1593.Google Scholar
  59. 59.
    H. Wendland, Local polynomial reproduction and moving least squares approximation, IMA J. Numer. Anal., 21 (2001), pp. 285–300.MathSciNetCrossRefMATHGoogle Scholar
  60. 60.
    H. Wendland, Scattered Data Approximation, Cambridge Monographs on Applied and Computational Mathematics, Cambridge University Press, Cambridge, 2005.MATHGoogle Scholar
  61. 61.
    H. Wendland and C. Rieger, Approximate interpolation with applications to selecting smoothing parameters, Numer. Math., 101 (2005), pp. 729–748.MathSciNetCrossRefMATHGoogle Scholar
  62. 62.
    J. Werner, Numerische Mathematik 1. Lineare und nichtlineare Gleichungssysteme, Interpolation, numerische Integration, Vieweg, Braunschweig-Wiesbaden, 1992.CrossRefMATHGoogle Scholar
  63. 63.
    W. Zhang, X. Xue, Z. Sun, Y. F. Guo, and H. Lu, Optimal dimensionality of metric space for classification, in ICML ’07: Proceedings of the 24th international conference on Machine learning, New York, NY, USA, 2007, ACM Press, pp. 1135–1142.Google Scholar
  64. 64.
    D.-X. Zhou, The covering number in learning theory, J. of Complexity, 18 (2002), pp. 739 – 767.MathSciNetCrossRefMATHGoogle Scholar
  65. 65.
    B. Zwicknagl, Mathematical analysis of microstructures and low hysteresis shape memory alloys, PhD thesis, University of Bonn, 2011.Google Scholar

Copyright information

© SFoCM 2017

Authors and Affiliations

  • Michael Griebel
    • 1
    • 2
  • Christian Rieger
    • 2
  • Barbara Zwicknagl
    • 3
    • 4
  1. 1.Fraunhofer-Institut für Algorithmen und Wissenschaftliches Rechnen SCAISchloss BirlinghovenSankt AugustinGermany
  2. 2.Institut für Numerische SimulationUniversität BonnBonnGermany
  3. 3.Institut für Angewandte MathematikUniversität BonnBonnGermany
  4. 4.Institut für Mathematik, Universität WürzburgWürzburgGermany

Personalised recommendations