Greedy Kernel Approximation for Sparse Surrogate Modeling

  • Bernard Haasdonk
  • Gabriele Santin


Modern simulation scenarios frequently require multi-query or real-time responses of simulation models for statistical analysis, optimization, or process control. However, the underlying simulation models may be very time-consuming rendering the simulation task difficult or infeasible. This motivates the need for rapidly computable surrogate models. We address the case of surrogate modeling of functions from vectorial input to vectorial output spaces. These appear, for instance, in simulation of coupled models or in the case of approximating general input–output maps. We review some recent methods and theoretical results in the field of greedy kernel approximation schemes. In particular, we recall the vectorial kernel orthogonal greedy algorithm (VKOGA) for approximating vector-valued functions. We collect some recent convergence statements that provide sound foundation for these algorithms, in particular quasi-optimal convergence rates in case of kernels inducing Sobolev spaces. We provide some initial experiments that can be obtained with non-symmetric greedy kernel approximation schemes. The results indicate better stability and overall more accurate models in situations where the input data locations are not equally distributed.



Both authors would like to thank the German Research Foundation (DFG) for financial support within the Cluster of Excellence in Simulation Technology (EXC 310) at the University of Stuttgart.


  1. 1.
    Antoulas, A.: Approximation of Large-Scale Dynamical Systems. SIAM Publications, Philadelphia, PA (2005)CrossRefGoogle Scholar
  2. 2.
    Buhmann, M.D., Dinew, S., Larsson, E.: A note on radial basis function interpolant limits. IMA J. Numer. Anal. 30(2), 543–554 (2010)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Chang, C.-C., Lin, C.-J.: LIBSVM: A Library For Support Vector Machines. Software verfgbar unter. (2001)
  4. 4.
    De Marchi, S., Schaback, R., Wendland, H.: Near-optimal data-independent point locations for radial basis function interpolation. Adv. Comput. Math. 23(3), 317–330 (2005)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Fasshauer, G. E.: Meshfree Approximation Methods with MATLAB, volume 6 of Interdisciplinary Mathematical Sciences. With 1 CD-ROM. Windows, Macintosh and UNIX. World Scientific Publishing Co., Pte. Ltd., Hackensack, NJ (2007)Google Scholar
  6. 6.
    Fasshauer, G.E., McCourt, M.J.: Stable evaluation of Gaussian radial basis function interpolants. SIAM J. Sci. Comput. 34(2), A737–A762 (2012)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Fornberg, B., Larsson, E., Flyer, N.: Stable computations with Gaussian radial basis functions. SIAM J. Sci. Comput. 33(2), 869–892 (2011)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Fornberg, B., Wright, G., Larsson, E.: Some observations regarding interpolants in the limit of flat radial basis functions. Comput. Math. Appl. 47(1), 37–55 (2004)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Haasdonk, B.: Transformation knowledge in pattern analysis with kernel methods, distance and integration kernels. Ph.D. thesis, Albert-Ludwigs-Universität, Freiburg im Breisgau, Fakultät für Angewandte Wissenschaften, Mai. Published 2006 as ISBN-3-8322-5026-3, Shaker-Verlag, Aachen, and Online at (2005)
  10. 10.
    Haasdonk, B.: Reduced basis methods for parametrized PDEs—a tutorial introduction for stationary and instationary problems. In: Benner, M.O.P., Cohen, A., Willcox, K. (eds.) Model Reduction and Approximation: Theory and Algorithms. SIAM, Philadelphia (2017)Google Scholar
  11. 11.
    Larsson, E., Fornberg, B.: Theoretical and computational aspects of multivariate interpolation with increasingly flat radial basis functions. Comput. Math. Appl. 49(1), 103–130 (2005)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Müller, S.: Komplexität und Stabilität von kernbasierten Rekonstruktionsmethoden. Ph.D. thesis, Fakultät für Mathematik und Informatik, Georg-August-Universität Göttingen (2009)Google Scholar
  13. 13.
    Müller, S., Schaback, R.: A Newton basis for kernel spaces. J. Approx. Theory 161(2), 645–655 (2009)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Pazouki, M., Schaback, R.: Bases for kernel-based spaces. J. Comput. Appl. Math. 236(4), 575–588 (2011)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Santin, G., Haasdonk, B.: Non-symmetric kernel greedy interpolation. University of Stuttgart, in preparation (2017)Google Scholar
  16. 16.
    Santin, G., Haasdonk, B.: Convergence rate of the data-independent P-greedy algorithm in kernel-based approximation. Dolomit. Res. Notes Approx. 10, 68–78 (2017)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Schaback, R.: Error estimates and condition numbers for radial basis function interpolation. Adv. Comput. Math. 3(3), 251–264 (1995)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Schaback, R., Wendland, H.: Numerical techniques based on radial basis functions. In: Curve and Surface Fitting: Saint-Malo 1999, Vanderbilt University Press, pp. 359–374 (2000)Google Scholar
  19. 19.
    Schölkopf, B., Smola, A.: Learning with Kernels. The MIT Press (2002)Google Scholar
  20. 20.
    Song, G., Riddle, J., Fasshauer, G.E., Hickernell, F.J.: Multivariate interpolation with increasingly flat radial basis functions of finite smoothness. Adv. Comput. Math. 36(3), 485–501 (2012)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Steinwart, I., Christmann, A.: Support Vector Machines, Information Science and Statistics. Springer, New York (2008)Google Scholar
  22. 22.
    Temlyakov, V.N.: Greedy approximation. Acta Numer. 17, 235–409 (2008)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Wendland, H.: Scattered Data Approximation. Cambridge Monographs on Applied and Computational Mathematics, vol. 17. Cambridge University Press, Cambridge (2005)Google Scholar
  24. 24.
    Wirtz, D., Haasdonk, B.: A vectorial kernel orthogonal greedy algorithm. Dolomites Res. Notes Approx. 6:83–100 (2013). (Proceedings of DWCAA12)Google Scholar
  25. 25.
    Wirtz, D., Karajan, N., Haasdonk, B.: Surrogate modelling of multiscale models using kernel methods. Int. J. Numer. Methods Eng. 101(1), 1–28 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Institute of Applied Analysis and Numerical SimulationUniversity of StuttgartStuttgartGermany

Personalised recommendations