Journal of Global Optimization

, Volume 31, Issue 1, pp 153–171 | Cite as

Constrained Global Optimization of Expensive Black Box Functions Using Radial Basis Functions

  • Rommel G. RegisEmail author
  • Christine A. Shoemaker


We present a new strategy for the constrained global optimization of expensive black box functions using response surface models. A response surface model is simply a multivariate approximation of a continuous black box function which is used as a surrogate model for optimization in situations where function evaluations are computationally expensive. Prior global optimization methods that utilize response surface models were limited to box-constrained problems, but the new method can easily incorporate general nonlinear constraints. In the proposed method, which we refer to as the Constrained Optimization using Response Surfaces (CORS) Method, the next point for costly function evaluation is chosen to be the one that minimizes the current response surface model subject to the given constraints and to additional constraints that the point be of some distance from previously evaluated points. The distance requirement is allowed to cycle, starting from a high value (global search) and ending with a low value (local search). The purpose of the constraint is to drive the method towards unexplored regions of the domain and to prevent the premature convergence of the method to some point which may not even be a local minimizer of the black box function. The new method can be shown to converge to the global minimizer of any continuous function on a compact set regardless of the response surface model that is used. Finally, we considered two particular implementations of the CORS method which utilize a radial basis function model (CORS-RBF) and applied it on the box-constrained Dixon–Szegö test functions and on a simple nonlinearly constrained test function. The results indicate that the CORS-RBF algorithms are competitive with existing global optimization algorithms for costly functions on the box-constrained test problems. The results also show that the CORS-RBF algorithms are better than other algorithms for constrained global optimization on the nonlinearly constrained test problem.


Black box function Costly function Global optimization Metamodel Radial basis function Response surface Surrogate model 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Björkman, M., Holmström, K. 2000Global optimization of costly nonconvex functions using radial basis functionsOptimization Engineering1373397CrossRefzbMATHGoogle Scholar
  2. Booker, A.J., Dennis, J.E., Frank, P.D., Serafini, D.B., Torczon, V., Trosset, M.W. 1999A rigorous framework for optimization of expensive functions by surrogatesStructural Optimization17113CrossRefGoogle Scholar
  3. Box, G.E.P., Draper, N.R. 1987Empirical Model-Building and Response SurfacesJohn Wiley Sons, Inc.New YorkzbMATHGoogle Scholar
  4. Cressie, N. 1993Statistics for Spatial DataJohn WileyNew YorkGoogle Scholar
  5. Dennis, J.E., Torczon, V. 1991Direct search methods on parallel machinesSIAM Journal on Optimization1448474CrossRefMathSciNetzbMATHGoogle Scholar
  6. Dixon, L.C.W. and Szegö, G. (1978), The global optimization problem: an introduction. In: Dixon, L.C.W. and Szegö, G. (eds.), Towards Global Optimization 2, pp. 1–15, North-Holland, Amsterdam.Google Scholar
  7. Friedman, J.H. 1991Multivariate adaptive regression splines (with discussion)Annals of Statistics191141CrossRefMathSciNetzbMATHGoogle Scholar
  8. Gomez, S. and Levy, A. (1982), The tunneling method for solving the constrained global optimization problem with several non-connected feasible regions. In: Dold, A. and Eckmann, B. (eds.), Numerical Analysis, Lecture Notes in Mathematics 909, pp. 34–47, Springer-Verlag.Google Scholar
  9. Gutmann, H.-M. (2001a), Radial basis function methods for global optimization. Ph.D. Thesis, University of Cambridge.Google Scholar
  10. Gutmann, H.-M. 2001A radial basis function method for global optimizationJournal of Global Optimization19201227CrossRefMathSciNetzbMATHGoogle Scholar
  11. Homström, K. 1999The TOMLAB optimization environment in MatlabAdvanced Modeling and Optimization14769Google Scholar
  12. Horst, R., Pardalos, P.M., Thoai, N.V. 1995Introduction to Global OptimizationKluwerNew YorkzbMATHGoogle Scholar
  13. Ishikawa, T., Matsunami, M. 1997An optimization method based on radial basis functionsIEEE Transactions on Magnetics3318681871CrossRefGoogle Scholar
  14. Ishikawa, T., Tsukui, Y., Matsunami, M. 1999A combined method for the global optimization using radial basis function and deterministic approachIEEE Transactions on Magnetics3517301733CrossRefGoogle Scholar
  15. Jones, D.R. (1996), Global optimization with response surfaces, presented at the Fifth SIAM Conference on Optimization, Victoria, Canada.Google Scholar
  16. Jones, D.R. 2001aA taxonomy of global optimization methods based on response surfacesJournal of Global Optimization21345383CrossRefzbMATHGoogle Scholar
  17. Jones, D.R. (2001b), The DIRECT global optimization algorithm. In: Floudas, C.A. and Pardalos, P.M. (eds.), Encyclopedia of Optimization, Vol. 1, pp. 431–440. Kluwer Academic PublishersGoogle Scholar
  18. Jones, D.R., Perttunen, C.D., Stuckman, B.E. 1993Lipschitz optimization without the lipschitz constantJournal of Optimization Theory and Applications78157181CrossRefMathSciNetGoogle Scholar
  19. Jones, D.R., Schonlau, M., Welch, W.J. 1998Efficient global optimization of expensive black-box functionsJournal of Global Optimization13455492CrossRefMathSciNetzbMATHGoogle Scholar
  20. Khuri, A.I., Cornell, J.A. 1987Response SurfacesMarcel Dekker, Inc.New YorkzbMATHGoogle Scholar
  21. Koehler, J.R. and Owen, A.B. (1996), Computer experiments. In: Ghosh, S. and Rao, C.R. (eds.), Handbook of Statistics, 13: Design and Analysis of Computer Experiments, pp. 261–308. North-Holland, Amsterdam.Google Scholar
  22. McKay, M., Beckman, R., Conover, W. 1979A comparison of three methods for selecting values of input variables in the analysis of output from a computer codeTechnometrics21239246MathSciNetzbMATHGoogle Scholar
  23. Myers, R.H., Montgomery, D.C. 1995Response Surface Methodology: Process and Product Optimization Using Designed ExperimentsJohn Wiley Sons Inc.New YorkzbMATHGoogle Scholar
  24. Nelder, J.A., Mead, R. 1965A simplex method for function minimizationComputer Journal7308313CrossRefzbMATHGoogle Scholar
  25. Nocedal, J., Wright, S.J. 1999Numerical OptimizationSpringerNew YorkCrossRefzbMATHGoogle Scholar
  26. Powell, M.J.D. (1992), The theory of Radial Basis Function Approximation in 1990, in: Light, W. (ed.), Advances in Numerical Analysis, Volume 2: Wavelets, Subdivision Algorithms and Radial Basis Functions, pp. 105–210. Oxford University Press.Google Scholar
  27. Powell, M.J.D. 1994A direct search optimization method that models the objective and constraint functions by linear interpolationGomez, S.Hennart, J.-P. eds. Advances in Optimization and Numerical AnalysisSpringer USNew York5167CrossRefGoogle Scholar
  28. Powell, M.J.D. 1999Recent research at Cambridge on radial basis functionsMüller, M.Buhmann, M.Mache, D.Felten, M. eds. New Developments in Approximation Theory, International Series of Numerical MathematicsBirkhauser VerlagBasel215232CrossRefGoogle Scholar
  29. Powell, M.J.D. 2000UOBYQA: unconstrained optimization by quadratic approximationMathematical Programming92555582CrossRefGoogle Scholar
  30. Powell, M.J.D. 2002On trust region methods for unconstrained minimization without derivatives, Technical Report, Department of Applied Mathematics and Theoretical PhysicsUniversity of CambridgeUK.Google Scholar
  31. Sacks, J., Welch, W.J., Mitchell, T.J., Wynn, H.P. 1989Design and analysis of computer experimentsStatistical Science4409435CrossRefMathSciNetzbMATHGoogle Scholar
  32. Simpson, T.W., Mauery, T.M., Korte, J.J. and Mistree, F. (1998), Comparison of response surface and kriging models for multidisciplinary design optimization. In: Proceedings of the 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Vol. 1, pp. 381–391. St. Louis, MO.Google Scholar
  33. The Mathworks, Inc. (2000), Optimization Toolbox for use with MATLAB: User’s Guide, Version 2.1.Google Scholar
  34. Torczon, V. 1997On the convergence of pattern search algorithms, SIAM Journal on Optimization7125MathSciNetzbMATHGoogle Scholar
  35. Torn, A., Zilinskas, A. 1989Global Optimization, Lecture Notes in Computer Science, Vol. 350Springer-VerlagBerlinGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2005

Authors and Affiliations

  1. 1.School of Operations Research and Industrial EngineeringCornell UniversityIthacaUSA
  2. 2.School of Civil and Environmental EngineeringCornell UniversityIthacaUSA

Personalised recommendations