Advertisement

Fundamental Splines on Sparse Grids and Their Application to Gradient-Based Optimization

  • Julian Valentin
  • Dirk PflügerEmail author
Conference paper
Part of the Lecture Notes in Computational Science and Engineering book series (LNCSE, volume 123)

Abstract

Most types of hierarchical basis functions for sparse grids are not continuously differentiable. This can lead to problems, for example, when using gradient-based optimization methods on sparse grid functions. B-splines represent an interesting alternative to conventional basis types since they have displayed promising results for regression and optimization problems. However, their overlapping support impedes the task of hierarchization (computing the interpolant), as, in general, the solution of a linear system is required. To cope with this problem, we propose three general basis transformations. They leave the spanned function space on dimensionally adaptive sparse grids or full grids unchanged, but result in triangular linear systems. One of the transformations, when applied to the B-spline basis, yields the well-known fundamental splines. We suggest a modification of the resulting sparse grid basis to enable nearly linear extrapolation towards the domain’s boundary without the need to spend boundary points. Finally, we apply the hierarchical modified fundamental spline basis to gradient-based optimization with sparse grid surrogates.

Notes

Acknowledgements

This work was financially supported by the Ministry of Science, Research and the Arts of the State of Baden-Württemberg. We thank the referees for their valuable comments.

References

  1. 1.
    H.-J. Bungartz, Finite elements of higher order on sparse grids. Habilitationsschrift, Institut für Informatik, TU München, 1998Google Scholar
  2. 2.
    H.-J. Bungartz, M. Griebel, Sparse grids. Acta Numer. 13, 147–269 (2004)MathSciNetCrossRefGoogle Scholar
  3. 3.
    C.K. Chui, An Introduction to Wavelets (Academic, San Diego, 1992)Google Scholar
  4. 4.
    F. Franzelin, D. Pflüger, From data to uncertainty: an efficient integrated data-driven sparse grid approach to propagate uncertainty, in Sparse Grids and Applications – Stuttgart 2014, ed. by J. Garcke, D. Pflüger. Lecture Notes in Computational Science and Engineering, vol. 109 (Springer, Cham, 2016), pp. 29–49Google Scholar
  5. 5.
    K. Höllig, Finite Element Methods with B-Splines (SIAM, Philadelphia, 2003)Google Scholar
  6. 6.
    K. Höllig, J. Hörner, Approximation and Modeling with B-Splines (SIAM, Philadelphia, 2013)Google Scholar
  7. 7.
    M. Jamil, X.-S. Yang, A literature survey of benchmark functions for global optimisation problems. Int. J. Math. Model. Numer. Optim. 4(2), 150–194 (2013)CrossRefGoogle Scholar
  8. 8.
    J.A. Nelder, R. Mead, A simplex method for function minimization. Comput. J. 7(4), 308–313 (1965)MathSciNetCrossRefGoogle Scholar
  9. 9.
    J. Nocedal, S.J. Wright, Numerical Optimization (Springer, New York, 1999)Google Scholar
  10. 10.
    E. Novak, K. Ritter, Global optimization using hyperbolic cross points, in State of the Art in Global Optimization, ed. by C.A. Floudas, P.M. Pardalos. Nonconvex Optimization and Its Applications, vol. 7 (Springer, Boston, 1996), pp. 19–33zbMATHGoogle Scholar
  11. 11.
    D. Pflüger, Spatially Adaptive Sparse Grids for High-Dimensional Problems (Verlag Dr. Hut, Munich, 2010)Google Scholar
  12. 12.
    D. Pflüger, Spatially adaptive refinement, in Sparse Grids and Applications, ed. by J. Garcke, M. Griebel. Lecture Notes in Computational Science and Engineering (Springer, Berlin, 2012), pp. 243–262Google Scholar
  13. 13.
    E. Polak, G. Ribière, Note sur la convergence de méthodes de directions conjuguées. Rev. Fr. Inf. Rech. Oper. 3(1), 35–43 (1969)zbMATHGoogle Scholar
  14. 14.
    M. Riedmiller, H. Braun, A direct adaptive method for faster backpropagation learning: the RPROP algorithm, in Proceedings of IEEE International Conference on Neural Networks, vol. 1 (1993), pp. 586–591Google Scholar
  15. 15.
    I.J. Schoenberg, Cardinal interpolation and spline functions: II. Interpolation of data of power growth. J. Approx. Theory 6, 404–420 (1972)CrossRefGoogle Scholar
  16. 16.
    I.J. Schoenberg, Cardinal Spline Interpolation (SIAM, Philadelphia, 1973)CrossRefGoogle Scholar
  17. 17.
    W. Sickel, T. Ullrich, Spline interpolation on sparse grids. Appl. Anal. 90(3–4), 337–383 (2011)MathSciNetCrossRefGoogle Scholar
  18. 18.
    R. Storn, K. Price, Differential Evolution – a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997)MathSciNetCrossRefGoogle Scholar
  19. 19.
    J. Valentin, D. Pflüger, Hierarchical gradient-based optimization with B-splines on sparse grids, in Sparse Grids and Applications – Stuttgart 2014, ed. by J. Garcke, D. Pflüger. Lecture Notes in Computational Science and Engineering, vol. 109 (Springer, Cham, 2016), pp. 315–336Google Scholar
  20. 20.
    D. Whitley, S. Rana, J. Dzubera, K.E. Mathias, Evaluating evolutionary algorithms. Artif. Intell. 85(1–2), 245–276 (1996)CrossRefGoogle Scholar
  21. 21.
    X.-S. Yang, Engineering Optimization (Wiley, Hoboken, 2010)CrossRefGoogle Scholar
  22. 22.
    C. Zenger, Sparse Grids. Notes on Numerical Fluid Mechanics, vol. 31 (Vieweg, Braunschweig, 1991), pp. 241–251Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Simulation of Large Systems (SGS), Institute for Parallel and Distributed Systems (IPVS)University of StuttgartStuttgartGermany

Personalised recommendations