Abstract
Multifidelity optimization approaches seek to bring higher-fidelity analyses earlier into the design process by using performance estimates from lower-fidelity models to accelerate convergence towards the optimum of a high-fidelity design problem. Current multifidelity optimization methods generally fall into two broad categories: provably convergent methods that use either the high-fidelity gradient or a high-fidelity pattern-search, and heuristic model calibration approaches, such as interpolating high-fidelity data or adding a Kriging error model to a lower-fidelity function. This paper presents a multifidelity optimization method that bridges these two ideas; our method iteratively calibrates lower-fidelity information to the high-fidelity function in order to find an optimum of the high-fidelity design problem. The algorithm developed minimizes a high-fidelity objective function subject to a high-fidelity constraint and other simple constraints. The algorithm never computes the gradient of a high-fidelity function; however, it achieves first-order optimality using sensitivity information from the calibrated low-fidelity models, which are constructed to have negligible error in a neighborhood around the solution. The method is demonstrated for aerodynamic shape optimization and shows at least an 80% reduction in the number of high-fidelity analyses compared other single-fidelity derivative-free and sequential quadratic programming methods. The method uses approximately the same number of high-fidelity analyses as a multifidelity trust-region algorithm that estimates the high-fidelity gradient using finite differences.
Similar content being viewed by others
Notes
Note that in the multifidelity setting, we use the term “derivative-free” to indicate an absence of derivatives of the high-fidelity model.
Note that an initial feasible point, x 0 could be found directly using the gradients of h(x) and g(x); however, since the penalty method includes the descent direction of the objective function it may better guide the optimization process in the case of multiple feasible regions. Should the initial iterate be feasible, the deficiencies of a quadratic penalty function are not an issue.
References
Aftosmis MJ (1997) Solution adaptive cartesian grid methods for aerodynamic flows with complex geometries. In: 28th computational fluid dynamics lecture series, von Karman Institute for Fluid Dynamics, Rhode-Saint-Genése, Belgium, lecture series 1997-02
Alexandrov N, Lewis R, Gumbert C, Green L, Newman P (1999) Optimization with variable-fidelity models applied to wing design. Tech. Rep. CR-209826, NASA
Alexandrov N, Lewis R, Gumbert C, Green L, Newman P (2001) Approximation and model management in aerodynamic optimization with variable-fidelity models. AIAA J 38(6):1093–1101
Audet C, Dennis Jr JE (2004) A pattern search filter method for nonlinear programming without derivatives. SIAM J Optim 14(4):980–1010
Bertsekas DP (1999) Nonlinear programming, 2nd edn. Athena Scientific
Boggs PT, Dennis Jr JE (1976) A stability analysis for perturbed nonlinear iterative methods. Math Comput 30(134):199–215
Booker AJ, Dennis Jr JE, Frank PD, Serafini DB, Torczon V, Trosset MW (1999) A rigorous framework for optimization of expensive functions by surrogates. Struct Optim 17(1):1–13
Castro J, Gray G, Giunta A, Hough P (2005) Developing a computationally efficient dynamic multilevel hybrid optimization scheme using multifidelity model interactions. Tech. Rep. SAND2005-7498, Sandia
Conn A, Gould N, Toint P (2000) Trust-region methods. MPS/SIAM series on optimization. Society for Industrial and Applied Mathematics, Philadelphia
Conn A, Scheinberg K, Vicente L (2008) Geometry of interpolation sets in derivative free optimization. Math Program 111(1–2):141–172
Conn A, Scheinberg K, Vicente L (2009a) Global convergence of general derivative-free trust-region algorithms to first- and second-order critical points. SIAM J Optim 20(1):387–415
Conn AR, Scheinberg K, Vicente LN (2009b) Introduction to derivative-free optimization. MPS/SIAM series on optimization. Society for Industrial and Applied Mathematics, Philadelphia
Jones D (2001) A taxonomy of global optimization methods based on response surfaces. J Glob Optim 21:345–383
Jones D, Perttunen CD, Stuckmann BE (1993) Lipschitzian optimization without the lipschitz constant. J Optim Theory Appl 79(1):157–181
Jones D, Schonlau M, Welch W (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492
Kennedy M, O’Hagan A (2000) Predicting the output from a complex computer code when fast approximations are available. Biometrika 87(1):1–13
Kolda TG, Lewis RM, Torczon V (2003) Optimization by direct search: new perspectives on classical and modern methods. SIAM Rev 45(3):385–482
Kolda TG, Lewis RM, Torczon V (2006) A generating set direct search augmented lagrangian algorithm for optimization with a combination of general and linear constraints. Tech. Rep. SAND2006-5315, Sandia
Lewis RM, Torczon V (2010) A direct search approach to nonlinear programming problems using an augmented lagrangian method with explicit treatment of linear constraints. Tech. Rep. WM-CS-2010-01, College of William and Mary Department of Computer Science
Liuzzi G, Lucidi S (2009) A derivative-free algorithm for inequality constrained nonlinear programming via smoothing of an l ∞ penalty function. SIAM J Optim 20(1):1–29
March A, Willcox K (2010) A provably convergent multifidelity optimization algorithm not requiring high-fidelity gradients. In: 6th AIAA multidisciplinary design optimization specialist conference, Orlando, FL, AIAA 2010-2912
MathWorks, Inc. (2010) Constrained nonlinear optimization. Optimization toolbox user’s guide, v. 5
Moré JJ, Wild SM (2010) Estimating derivatives of noisy simulations. Tech. Rep. Preprint ANL/MCS-P1785-0810, Mathematics and Computer Science Division
Nelder JA, Mead RA (1965) A simplex method for function minimization. Comput J 7:308–313
Nocedal J, Wright S (2006) Numerical optimization, 2nd edn. Springer, New York
Papalambros PY, Wilde DJ (2000) Principles of optimal design, 2nd edn. Cambridge University Press, Cambridge
Powell MJD (1994) A direct search optimization method that models the objective and constraint functions by linear interpolation. In: Gomez S, Hennart J-P (eds) Advances in optimization and numerical analysis, vol 7. Kluwer Academic, Dordrecht, pp 51–67
Powell MJD (1998) Direct search algorithms for optimization calculations. Acta Numer 7:287–336
Rajnarayan D, Haas A, Kroo I (2008) A multifidelity gradient-free optimization method and application to aerodynamic design. In: 12th AIAA/ISSMO multidisciplinary analysis and optimization conference, Victoria, British Columbia, AIAA 2008-6020
Rvachev VL (1963) On the analytical description of some geometric objects. Tech. Rep. 4, Reports of Ukrainian Academy of Sciences (in Russian)
Sasena MJ, Papalambros P, Goovaerts P (2002) Exploration of metamodeling sampling criteria for constrained global optimization. Eng Optim 34(3):263–278
Wild S (2009) Derivative-free optimization algorithms for computationally expensive functions. PhD thesis, Cornell University
Wild S, Shoemaker CA (2009) Global convergence of radial basis function trust-region algorithms. Tech. Rep. Preprint ANL/MCS-P1580-0209, Mathematics and Computer Science Division
Wild S, Regis R, Shoemaker C (2008) ORBIT: optimization by radial basis function interpolation in trust-regions. SIAM J Sci Comput 30(6):3197–3219
Acknowledgments
The authors gratefully acknowledge support from NASA Langley Research Center contract NNL07AA33C, technical monitor Natalia Alexandrov, and a National Science Foundation graduate research fellowship. In addition, we wish to thank Michael Aftosmis and Marian Nemec for support with Cart3D.
Author information
Authors and Affiliations
Corresponding author
Additional information
A version of this paper was presented as paper AIAA-2010-9198 at the 13th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Fort Worth, TX, 13–15 September 2010.
Rights and permissions
About this article
Cite this article
March, A., Willcox, K. Constrained multifidelity optimization using model calibration. Struct Multidisc Optim 46, 93–109 (2012). https://doi.org/10.1007/s00158-011-0749-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00158-011-0749-1