Average Case Approximation: Convergence and Tractability of Gaussian Kernels

  • G. E. Fasshauer
  • F. J. Hickernell
  • H. Woźniakowski
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 23)

Abstract

We study the problem of approximating functions of d variables in the average case setting for a separable Banach space \({\mathcal{F}}_{d}\) equipped with a zero-mean Gaussian measure. The covariance kernel of this Gaussian measure takes the form of a Gaussian that depends on shape parameters \({\gamma }_{\mathcal{l}}\). We stress that d can be arbitrarily large. Our approximation error is defined in the \({\mathcal{L}}_{2}\) norm, and we study the minimal average case error \({e}_{d}^{\mathrm{avg}}(n)\) of algorithms that use at most n linear functionals or function values. For \({\gamma }_{\mathcal{l}} = {\mathcal{l}}^{-\alpha }\) with \(\alpha \geq 0\), we prove that \({e}_{d}^{\mathrm{avg}}(n)\) has a polynomial bound of roughly order \({n}^{-(\alpha -1/2)}\) independent of d iff \(\alpha \,\,>\,\,1/2\). This property is equivalent to strong polynomial tractability and says that the minimal number of linear functionals or function values needed to achieve an average case error \(\epsilon \) has a bound independent of d proportional roughly to \({\epsilon }^{-1/(\alpha -1/2)}\). In the case of algorithms that use only function values the proof is non-constructive. In order to compare the average case with the worst case studied in our earlier paper we specialize the function space \({\mathcal{F}}_{d}\) to a reproducing kernel Hilbert space whose kernel is a Gaussian kernel with shape parameters \({\gamma }_{\mathcal{l}}^{\,\mathrm{rep}}\). To allow for a fair comparison we further equip this space with a zero-mean Gaussian measure whose covariance operator has eigenvalues that depend on a positive parameter q. We prove that the average cases for the whole space and for the unit ball of \({\mathcal{F}}_{d}\) are roughly the same provided the \({\gamma }_{\mathcal{l}}^{\,\mathrm{rep}}\) decay quickly enough. Furthermore, for a particular choice of q the dimension-independent convergence for the worst and average case settings are essentially the same.

Keywords

Unit Ball Gaussian Kernel Average Case Reproduce Kernel Hilbert Space Isotropic Gaussian Kernel 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

This article is dedicated to Stefan Heinrich on the occasion of his \(6{0}^{\text{ th}}\) birthday. We are grateful for many fruitful discussions with several colleagues. The authors were partially supported by the National Science Foundation, the first and second author under DMS-0713848 and DMS-1115392, and the third author under DMS-0914345. The second author was also partially supported by the Department of Energy grant SC0002100.

References

  1. 1.
    Buhmann, M. D. (2003) Radial Basis Functions. Cambridge Monographs on Applied and Computational Mathematics, Cambridge University Press, Cambridge.Google Scholar
  2. 2.
    Fasshauer G. E. (2007) Meshfree Approximation Methods with Matlab, Interdisciplinary Mathematical Sciences, vol 6. World Scientific Publishing Co., Singapore.Google Scholar
  3. 3.
    Fasshauer G. E., Hickernell F.J., Woźniakowski H. (2012) On dimension-independent rates of convergence for function approximation with Gaussian kernels. SIAM J. Numer. Anal. 50(1): 247–271.Google Scholar
  4. 4.
    Fasshauer G. E., McCourt M. J. (2012) Stable evaluation of Gaussian radial basis function interpolants, SIAM J. Sci. Comput. 34(2): A737–A762.Google Scholar
  5. 5.
    Forrester A. I. J., Sóbester A., Keane A. J. (2008) Engineering Design via Surrogate Modelling. Wiley, Chichester.Google Scholar
  6. 6.
    Hickernell F. J., Woźniakowski H. (2000) Integration and approximation in arbitrary dimensions. Adv Comput Math 12:25–58.Google Scholar
  7. 7.
    Hickernell F. J., Wasilkowski G. W., Woźniakowski H. (2008) Tractability of linear multivariate problems in the average case setting. In: Keller A, Heinrich S, Niederreiter H (eds) Monte Carlo and Quasi-Monte Carlo Methods 2006, Springer-Verlag, Berlin, pp 423–452.Google Scholar
  8. 8.
    Kuo F. Y., Sloan I. H., Woźniakowski H. (2008) Lattice rule algorithms for multivariate approximation in the average case setting. J Complexity 24:283–323.Google Scholar
  9. 9.
    Novak E., Woźniakowski H. (2008) Tractability of Multivariate Problems Volume 1: Linear Information. No. 6 in EMS Tracts in Mathematics, European Mathematical Society.Google Scholar
  10. 10.
    Novak E., Woźniakowski H. (2010) Tractability of Multivariate Problems Volume 2: Standard Information for Functionals. No. 12 in EMS Tracts in Mathematics, European Mathematical Society.Google Scholar
  11. 11.
    Papageorgiou A., Wasilkowski G. W. (1990) On the average complexity of multivariate problems. J Complexity 6:1–23.Google Scholar
  12. 12.
    Rasmussen C. E., Williams C. (2006) Gaussian Processes for Machine Learning. MIT Press, Cambridge, Massachusetts, (online version at http://www.gaussianprocess.org/gpml/).
  13. 13.
    Ritter K., Wasilkowski G. W. (1996) On the average case complexity of solving Poisson equations. In: Renegar J., Shub M., Smale S. (eds) The mathematics of numerical analysis, Lectures in Appl. Math., vol 32, American Mathematical Society, Providence, Rhode Island, pp 677–687.Google Scholar
  14. 14.
    Ritter K., Wasilkowski G. W. (1997) Integration and \({L}_{2}\) approximation: Average case setting with isotropic Wiener measure for smooth functions. Rocky Mountain J Math 26:1541–1557.Google Scholar
  15. 15.
    SAS Institute, JMP 9.0, 2010.Google Scholar
  16. 16.
    Schaback, R. (2008) Limit problems for interpolation by analytic radial basis functions. J Comp Appl Math 212:127–149.Google Scholar
  17. 17.
    Stein M. L. (1999) Interpolation of Spatial Data: Some Theory for Kriging. Springer-Verlag, New York.Google Scholar
  18. 18.
    Traub J. F., Wasilkowski G. W., Woźniakowski H. (1988) Information-Based Complexity. Academic Press, Boston.Google Scholar
  19. 19.
    Wahba G. (1990) Spline Models for Observational Data, CBMS-NSF Regional Conference Series in Applied Mathematics, vol 59. SIAM, Philadelphia.Google Scholar
  20. 20.
    Wasilkowski G. W. (1986) Information of varying cardinality. J Complexity 2:204–228.Google Scholar
  21. 21.
    Wasilkowski G. W. (1993) Integration and approximation of multivariate functions: Average case complexity with isotropic Wiener measure. Bull Amer Math Soc 28:308–314.Google Scholar
  22. 22.
    Wasilkowski G. W., Woźniakowski H. (1995) Explicit cost bounds for multivariate tensor product problems. J Complexity 11:1–56.Google Scholar
  23. 23.
    Wasilkowski G. W., Woźniakowski H. (2001) On the power of standard information for weighted approximation. Found Comput Math 1:417–434.Google Scholar
  24. 24.
    Wendland H. (2005) Scattered Data Approximation. No. 17 in Cambridge Monographs on Applied and Computational Mathematics, Cambridge University Press, Cambridge.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • G. E. Fasshauer
    • 1
  • F. J. Hickernell
    • 1
  • H. Woźniakowski
    • 2
    • 3
  1. 1.Department of Applied MathematicsIllinois Institute of TechnologyChicagoUSA
  2. 2.Department of Computer ScienceColumbia UniversityNew YorkUSA
  3. 3.Institute of Applied MathematicsUniversity of WarsawWarszawaPoland

Personalised recommendations