Sparse Grids and Applications - Stuttgart 2014 pp 315-336 | Cite as
Hierarchical Gradient-Based Optimization with B-Splines on Sparse Grids
Abstract
Optimization algorithms typically perform a series of function evaluations to find an approximation of an optimal point of the objective function. Evaluations can be expensive, e.g., if they depend on the results of a complex simulation. When dealing with higher-dimensional functions, the curse of dimensionality increases the difficulty of the problem rapidly and prohibits a regular sampling. Instead of directly optimizing the objective function, we replace it with a sparse grid interpolant, saving valuable function evaluations. We generalize the standard piecewise linear basis to hierarchical B-splines, making the sparse grid surrogate smooth enough to enable gradient-based optimization methods. Also, we use an uncommon refinement criterion due to Novak and Ritter to generate an appropriate sparse grid adaptively. Finally, we evaluate the new method for various artificial and real-world examples.
Keywords
Grid Generation Sparse Grid Macro Cell Chebyshev Point Grid InterpolationNotes
Acknowledgements
This work was financially supported by the Juniorprofessurenprogramm of the Landesstiftung Baden-Württemberg.
References
- 1.H. Bachau, E. Cormier, P. Decleva, J.E. Hansen, F. Martín, Applications of B-splines in atomic and molecular physics. Rep. Prog. Phys. 64(12), 1815–1942 (2001)CrossRefGoogle Scholar
- 2.M.P. Bendsøe, N. Kikuchi, Generating optimal topologies in structural design using a homogenization method. Comput. Methods Appl. Mech. Eng. 71(2) 197–224 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
- 3.H.-J. Bungartz, Finite elements of higher order on sparse grids, Habilitationsschrift, Institut für Informatik, TU München, 1998Google Scholar
- 4.E. Cohen, R.F. Riesenfeld, G. Elber, Geometric Modeling with Splines: An Introduction (A K Peters, Natick, 2001)zbMATHGoogle Scholar
- 5.M.G. Cox, The numerical evaluation of B-splines. IMA J. Appl. Math. 10(2), 134–149 (1972)MathSciNetCrossRefzbMATHGoogle Scholar
- 6.T.A. Davis, Algorithm 832: UMFPACK V4.3-an unsymmetric-pattern multifrontal method. ACM Trans. Math. Softw. 30(2), 196–199 (2004)Google Scholar
- 7.C. de Boor, On calculating with B-splines. J. Approx. Theory 6(1), 50–62 (1972)MathSciNetCrossRefzbMATHGoogle Scholar
- 8.C. de Boor, Splines as linear combinations of B-splines. A survey, in Approximation Theory II, ed. by G.G. Lorentz, C.K. Chui, L.L. Schumaker (Academic, New York, 1976), pp. 1–47Google Scholar
- 9.F. Delbos, L. Dumas, E. Echagüe, Global optimization based on sparse grid surrogate models for black-box expensive functions, http://dumas.perso.math.cnrs.fr/JOGO.pdf, (2016)
- 10.M.M. Donahue, G.T. Buzzard, A.E. Rundell, Parameter identification with adaptive sparse grid-based optimization for models of cellular processes, in Methods in Bioengineering: Systems Analysis of Biological Networks, ed. by A. Jayaraman, J. Hahn (Artech House, Boston/London, 2009), pp. 211–232Google Scholar
- 11.I. Ferenczi, Globale Optimierung unter Nebenbedingungen mit dünnen Gittern, Diploma thesis, Department of Mathematics, TU München, 2005Google Scholar
- 12.P.E. Gill, W. Murray, M.A. Saunders, SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM J. Optim. 12(4), 979–1006 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
- 13.G. Guennebaud, B. Jacob et al., Eigen, http://eigen.tuxfamily.org/, (2016)
- 14.K. Höllig, Finite Element Methods with B-Splines (SIAM, Philadelphia, 2003)CrossRefzbMATHGoogle Scholar
- 15.K. Höllig, J. Hörner, Approximation and Modeling with B-Splines (SIAM, Philadelphia, 2013)zbMATHGoogle Scholar
- 16.Y.-K. Hu, Y.P. Hu, Global optimization in clustering using hyperbolic cross points. Pattern Recognit. 40(6), 1722–1733 (2007)CrossRefzbMATHGoogle Scholar
- 17.D. Hübner, Mehrdimensionale Parametrisierung der Mikrozellen in der Zwei-Skalen-Optimierung, Master’s thesis, Department of Mathematics, FAU Erlangen-Nürnberg, 2014Google Scholar
- 18.M. Jamil, X.-S. Yang, A literature survey of benchmark functions for global optimisation problems. Int. J. Math. Model. Numer. Optim. 4(2), 150–194 (2013)zbMATHGoogle Scholar
- 19.Y. Jiang, Y. Xu, B-spline quasi-interpolation on sparse grids. J. Complex. 27(5), 466–488 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
- 20.M. Kaltenbacher, Advanced simulation tool for the design of sensors and actuators, in Proceedings of Eurosensors XXIV, Linz, vol. 5, 2010, pp. 597–600Google Scholar
- 21.F. Lekien, J. Marsden, Tricubic interpolation in three dimensions. Int. J. Numer. Methods Eng. 63(3), 455–471 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
- 22.L. Ljung, System Identification: Theory for the User, 2nd edn. (Prentice Hall, Upper Saddle River, 1999)zbMATHGoogle Scholar
- 23.C.W. McCurdy, F. Martín, Implementation of exterior complex scaling in B-splines to solve atomic and molecular collision problems. J. Phys. B: At. Mol. Opt. Phys. 37(4), 917–936 (2004)CrossRefGoogle Scholar
- 24.J.A. Nelder, R. Mead, A simplex method for function minimization. Comput. J. 7(4), 308–313 (1965)CrossRefzbMATHGoogle Scholar
- 25.J. Nocedal, S.J. Wright, Numerical Optimization (Springer, New York, 1999)CrossRefzbMATHGoogle Scholar
- 26.E. Novak, K. Ritter, Global optimization using hyperbolic cross points, in State of the Art in Global Optimization, ed. by C.A. Floudas, P.M. Pardalos (Springer, Boston, 1996), pp. 19–33CrossRefGoogle Scholar
- 27.D. Pandey, Regression with spatially adaptive sparse grids in financial applications, Master’s thesis, Department of Informatics, TU München, 2008Google Scholar
- 28.D. Pflüger, Spatially Adaptive Sparse Grids for High-Dimensional Problems (Verlag Dr. Hut, München, 2010)zbMATHGoogle Scholar
- 29.D. Pflüger, Spatially adaptive refinement, in Sparse Grids and Applications, ed. by J. Garcke, M. Griebel. Lecture Notes in Computational Science and Engineering (Springer, Berlin/Heidelberg, 2012), pp. 243–262Google Scholar
- 30.E. Polak, G. Ribière, Note sur la convergence de méthodes de directions conjuguées. Rev. Fr. Inf. Rech. Oper. 3(1), 35–43 (1969)zbMATHGoogle Scholar
- 31.Y. Renard, J. Pommier, Gmm++, http://download.gna.org/getfem/html/homepage/gmm/index.html, (2016)
- 32.M. Riedmiller, H. Braun, A direct adaptive method for faster backpropagation learning: the RPROP algorithm, in Proceedings of 1993 IEEE International Conference on Neural Networks, San Francisco, CA, vol. 1, 1993, pp. 586–591Google Scholar
- 33.C. Sanderson, Armadillo: an open source C++ linear algebra library for fast prototyping and computationally intensive experiments, Technical report, NICTA, 2010Google Scholar
- 34.I.J. Schoenberg, Contributions to the problem of approximation of equidistant data by analytic functions. Q. Appl. Math. 4, 45–99, 112–141 (1946)MathSciNetGoogle Scholar
- 35.R. Storn, K. Price, Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. J. Glob. Optim. 11(4), 341–359 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
- 36.J. Valentin, Hierarchische Optimierung mit Gradientenverfahren auf Dünngitterfunktionen, Master’s thesis, IPVS, Universität Stuttgart, 2014Google Scholar
- 37.D. Whitley, S. Rana, J. Dzubera, K.E. Mathias, Evaluating evolutionary algorithms. Artif. Intel. 85(1–2), 245–276 (1996)CrossRefGoogle Scholar
- 38.X.-S. Yang, Engineering Optimization (Wiley, Hoboken, 2010)CrossRefGoogle Scholar