Blackbox and derivative-free optimization: theory, algorithms and applications
- 2.6k Downloads
Blackbox optimization refers to problems where the structure of the objective and constraint functions cannot be exploited. This is often the case when their evaluation requires the execution of a (usually time-consuming) simulation using computational models that are typically inaccessible by the user. The term Derivative-Free Optimization refers to the use of algorithms that utilize only function values because their partial derivatives are either not defined or not available; gradient approximations may sometimes be obtained, but the amount of work required to ensure they are dependable may not be worth the effort. Both blackbox and derivative-free optimization have attracted significant, and still increasing, interest from researchers over the last decade. Thus, we felt that it was time to dedicate a special issue of OPTE to this topic.
Blackbox and derivative-free optimization methods are often the only realistic and practical tools available to engineers working on simulation-based design. It is obvious that if the design optimization problem at hand allows an evaluation or reliable approximation of the gradients, then efficient gradient-based methods should be used. Blackbox and derivative-free algorithms are not competitors of gradient-based methods; they are a fallback when gradient-based algorithms cannot be used. The design engineering community is increasingly becoming aware that rigorous blackbox and derivative-free algorithms have made significant advances in the past 20 years and can be much better than heuristics. The latter are popular because they are easy to understand and implement, but their solutions can rarely be characterized in terms of optimality or quality. With only a modest investment in understanding the theoretical background and appropriate use, design engineers may reap large benefits by using blackbox and derivative-free algorithms with convergence properties.
This issue contains a collection of papers devoted to the algorithmic developments of these optimization methods, the analysis of their theoretical foundations, and applications to problems encountered in various engineering disciplines. The first six papers consider derivative-free and blackbox optimization methods for a variety of problems: earth imaging in geophysics, process design in materials science, the coupling of adjacent buildings for seismic retrofitting, multistage wind farm design, radiation signatures in inverse transport problems, and hull form optimization in ship design. The next three papers focus on algorithmic features and particularities: hidden constraints and Monte Carlo simulations, a mixed-integer surrogate framework, and strategies to identify multiple local minima. The issue concludes with a paper that studies the theoretical properties of positive spanning sets, which are at the heart of many blackbox and derivative-free optimization methods.
Algorithms for such problems are now much more elaborate than the 1952 coordinate search algorithm of Fermi and Metropolis and the 1965 Nelder–Mead downhill simplex method. Yet there are still challenges that require further algorithmic development. Glancing at the crystal ball, we anticipate that future research on blackbox and derivative-free optimization will target topics such as stochastic and noisy problems, the exploitation of parallelism for large problems, multifidelity models and surrogates, the explicit use of analytical equality and inequality constraints, and hybrid strategies for escaping local solutions.