Advertisement

Hierarchical Gradient-Based Optimization with B-Splines on Sparse Grids

  • Julian ValentinEmail author
  • Dirk Pflüger
Conference paper
First Online:
Part of the Lecture Notes in Computational Science and Engineering book series (LNCSE, volume 109)

Abstract

Optimization algorithms typically perform a series of function evaluations to find an approximation of an optimal point of the objective function. Evaluations can be expensive, e.g., if they depend on the results of a complex simulation. When dealing with higher-dimensional functions, the curse of dimensionality increases the difficulty of the problem rapidly and prohibits a regular sampling. Instead of directly optimizing the objective function, we replace it with a sparse grid interpolant, saving valuable function evaluations. We generalize the standard piecewise linear basis to hierarchical B-splines, making the sparse grid surrogate smooth enough to enable gradient-based optimization methods. Also, we use an uncommon refinement criterion due to Novak and Ritter to generate an appropriate sparse grid adaptively. Finally, we evaluate the new method for various artificial and real-world examples.

Keywords

Grid Generation Sparse Grid Macro Cell Chebyshev Point Grid Interpolation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

This work was financially supported by the Juniorprofessurenprogramm of the Landesstiftung Baden-Württemberg.

References

  1. 1.
    H. Bachau, E. Cormier, P. Decleva, J.E. Hansen, F. Martín, Applications of B-splines in atomic and molecular physics. Rep. Prog. Phys. 64(12), 1815–1942 (2001)CrossRefGoogle Scholar
  2. 2.
    M.P. Bendsøe, N. Kikuchi, Generating optimal topologies in structural design using a homogenization method. Comput. Methods Appl. Mech. Eng. 71(2) 197–224 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    H.-J. Bungartz, Finite elements of higher order on sparse grids, Habilitationsschrift, Institut für Informatik, TU München, 1998Google Scholar
  4. 4.
    E. Cohen, R.F. Riesenfeld, G. Elber, Geometric Modeling with Splines: An Introduction (A K Peters, Natick, 2001)zbMATHGoogle Scholar
  5. 5.
    M.G. Cox, The numerical evaluation of B-splines. IMA J. Appl. Math. 10(2), 134–149 (1972)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    T.A. Davis, Algorithm 832: UMFPACK V4.3-an unsymmetric-pattern multifrontal method. ACM Trans. Math. Softw. 30(2), 196–199 (2004)Google Scholar
  7. 7.
    C. de Boor, On calculating with B-splines. J. Approx. Theory 6(1), 50–62 (1972)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    C. de Boor, Splines as linear combinations of B-splines. A survey, in Approximation Theory II, ed. by G.G. Lorentz, C.K. Chui, L.L. Schumaker (Academic, New York, 1976), pp. 1–47Google Scholar
  9. 9.
    F. Delbos, L. Dumas, E. Echagüe, Global optimization based on sparse grid surrogate models for black-box expensive functions, http://dumas.perso.math.cnrs.fr/JOGO.pdf, (2016)
  10. 10.
    M.M. Donahue, G.T. Buzzard, A.E. Rundell, Parameter identification with adaptive sparse grid-based optimization for models of cellular processes, in Methods in Bioengineering: Systems Analysis of Biological Networks, ed. by A. Jayaraman, J. Hahn (Artech House, Boston/London, 2009), pp. 211–232Google Scholar
  11. 11.
    I. Ferenczi, Globale Optimierung unter Nebenbedingungen mit dünnen Gittern, Diploma thesis, Department of Mathematics, TU München, 2005Google Scholar
  12. 12.
    P.E. Gill, W. Murray, M.A. Saunders, SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM J. Optim. 12(4), 979–1006 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    G. Guennebaud, B. Jacob et al., Eigen, http://eigen.tuxfamily.org/, (2016)
  14. 14.
    K. Höllig, Finite Element Methods with B-Splines (SIAM, Philadelphia, 2003)CrossRefzbMATHGoogle Scholar
  15. 15.
    K. Höllig, J. Hörner, Approximation and Modeling with B-Splines (SIAM, Philadelphia, 2013)zbMATHGoogle Scholar
  16. 16.
    Y.-K. Hu, Y.P. Hu, Global optimization in clustering using hyperbolic cross points. Pattern Recognit. 40(6), 1722–1733 (2007)CrossRefzbMATHGoogle Scholar
  17. 17.
    D. Hübner, Mehrdimensionale Parametrisierung der Mikrozellen in der Zwei-Skalen-Optimierung, Master’s thesis, Department of Mathematics, FAU Erlangen-Nürnberg, 2014Google Scholar
  18. 18.
    M. Jamil, X.-S. Yang, A literature survey of benchmark functions for global optimisation problems. Int. J. Math. Model. Numer. Optim. 4(2), 150–194 (2013)zbMATHGoogle Scholar
  19. 19.
    Y. Jiang, Y. Xu, B-spline quasi-interpolation on sparse grids. J. Complex. 27(5), 466–488 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    M. Kaltenbacher, Advanced simulation tool for the design of sensors and actuators, in Proceedings of Eurosensors XXIV, Linz, vol. 5, 2010, pp. 597–600Google Scholar
  21. 21.
    F. Lekien, J. Marsden, Tricubic interpolation in three dimensions. Int. J. Numer. Methods Eng. 63(3), 455–471 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    L. Ljung, System Identification: Theory for the User, 2nd edn. (Prentice Hall, Upper Saddle River, 1999)zbMATHGoogle Scholar
  23. 23.
    C.W. McCurdy, F. Martín, Implementation of exterior complex scaling in B-splines to solve atomic and molecular collision problems. J. Phys. B: At. Mol. Opt. Phys. 37(4), 917–936 (2004)CrossRefGoogle Scholar
  24. 24.
    J.A. Nelder, R. Mead, A simplex method for function minimization. Comput. J. 7(4), 308–313 (1965)CrossRefzbMATHGoogle Scholar
  25. 25.
    J. Nocedal, S.J. Wright, Numerical Optimization (Springer, New York, 1999)CrossRefzbMATHGoogle Scholar
  26. 26.
    E. Novak, K. Ritter, Global optimization using hyperbolic cross points, in State of the Art in Global Optimization, ed. by C.A. Floudas, P.M. Pardalos (Springer, Boston, 1996), pp. 19–33CrossRefGoogle Scholar
  27. 27.
    D. Pandey, Regression with spatially adaptive sparse grids in financial applications, Master’s thesis, Department of Informatics, TU München, 2008Google Scholar
  28. 28.
    D. Pflüger, Spatially Adaptive Sparse Grids for High-Dimensional Problems (Verlag Dr. Hut, München, 2010)zbMATHGoogle Scholar
  29. 29.
    D. Pflüger, Spatially adaptive refinement, in Sparse Grids and Applications, ed. by J. Garcke, M. Griebel. Lecture Notes in Computational Science and Engineering (Springer, Berlin/Heidelberg, 2012), pp. 243–262Google Scholar
  30. 30.
    E. Polak, G. Ribière, Note sur la convergence de méthodes de directions conjuguées. Rev. Fr. Inf. Rech. Oper. 3(1), 35–43 (1969)zbMATHGoogle Scholar
  31. 31.
  32. 32.
    M. Riedmiller, H. Braun, A direct adaptive method for faster backpropagation learning: the RPROP algorithm, in Proceedings of 1993 IEEE International Conference on Neural Networks, San Francisco, CA, vol. 1, 1993, pp. 586–591Google Scholar
  33. 33.
    C. Sanderson, Armadillo: an open source C++ linear algebra library for fast prototyping and computationally intensive experiments, Technical report, NICTA, 2010Google Scholar
  34. 34.
    I.J. Schoenberg, Contributions to the problem of approximation of equidistant data by analytic functions. Q. Appl. Math. 4, 45–99, 112–141 (1946)MathSciNetGoogle Scholar
  35. 35.
    R. Storn, K. Price, Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. J. Glob. Optim. 11(4), 341–359 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    J. Valentin, Hierarchische Optimierung mit Gradientenverfahren auf Dünngitterfunktionen, Master’s thesis, IPVS, Universität Stuttgart, 2014Google Scholar
  37. 37.
    D. Whitley, S. Rana, J. Dzubera, K.E. Mathias, Evaluating evolutionary algorithms. Artif. Intel. 85(1–2), 245–276 (1996)CrossRefGoogle Scholar
  38. 38.
    X.-S. Yang, Engineering Optimization (Wiley, Hoboken, 2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Institute for Parallel and Distributed Systems (IPVS)Universität StuttgartStuttgartGermany

Personalised recommendations

Cite paper

Buy options