Structural optimization

, Volume 17, Issue 1, pp 1–13 | Cite as

A rigorous framework for optimization of expensive functions by surrogates

  • A. J. Booker
  • J. E. DennisJr.
  • P. D. Frank
  • D. B. Serafini
  • V. Torczon
  • M. W. Trosset
Research Papers


The goal of the research reported here is to develop rigorous optimization algorithms to apply to some engineering design problems for which direct application of traditional optimization approaches is not practical. This paper presents and analyzes a framework for generating a sequence of approximations to the objective function and managing the use of these approximations as surrogates for optimization. The result is to obtain convergence to a minimizer of an expensive objective function subject to simple constraints. The approach is widely applicable because it does not require, or even explicitly approximate, derivatives of the objective. Numerical results are presented for a 31-variable helicopter rotor blade design example and for a standard optimization test example.


Objective Function Design Problem Optimization Approach Engineering Design Rotor Blade 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Barthelemy, J-F.M.; Haftka, R.T. 1993: Approximation concepts for optimum structural design — a review.Struct. Optim. 5, 129–144Google Scholar
  2. Booker, A.J. 1994: DOE for computer output.Technical Report BCSTECH-94-052 Boeing Computer Services, Research and Technology, M/S 7L-68, Seattle, Washington 98124Google Scholar
  3. Booker, A.J. 1996: Case studies in design and analysis of computer experiments. InProc. Section on Physical and Engineering Sciences, American Statistical AssociationGoogle Scholar
  4. Booker, A.J.; Conn, A.R., Dennis, J.E. Jr.; Frank, P.D.; Trosset, M.W.; Torczon, V. 1995. Global modeling for optimization Boeing/IBM/Rice collaborative project 1995 final report.Technical Report ISSTECH-95-032, Boeing Information & Support Services, Research and Technology, M/S 7L-68, Seattle, Washington 98124Google Scholar
  5. Booker, A.J.; Dennis, J.E., Jr.; Frank, P.D.; Serafini, D.B.; Torczon, V. 1997. Optimization using surrogate objectives on a helicopter test example.Technical Report SSGTECH-97-027, Roeing Shared Services Group, Applied Research & Technology, M/S 7L-68, Seattle, Washington 98124. Also available as Technical Report 97-31, Department of Computational and Applied Mathematics, Rice University, Houston, Texas 77005-1892. To appear in: Borggaard, J.; Burns, J.; Cliff, E.; Schreck, S. (eds.)Optimal design and control. Cambridge, MA: BirkhauserGoogle Scholar
  6. Burgee, S.L.; Giunta, A.A.; Balabanov, V.; Grossman, B.; Mason, W.H.; Narducci, R.; Haftka, R.T.; Watson, L.T. 1996: A coarse-grained parallel variable-complexity multidisciplinary optimization paradigm.Int. J. Supercomputing Appl. & High Performance Computing 10, 269–299Google Scholar
  7. Conn, A.R.; Scheinberg, K.; Toint, Ph.L. 1997: On the convergence of derivative-free methods for unconstrained optimization. In: Iserles, A.; Buhmann, M. (eds.)Approximation theory and optimization: tributes to M.J.D. Powell, pp. 83–108. Cambridge: Cambridge University PressGoogle Scholar
  8. Conn, A.R.; Toint, Ph.L. 1996: An algorithm using quadratic interpolation for unconstrained derivative free optimization. In: Di Pillo, G.; Giannessi, F. (eds.)Nonlinear optimization and applications, pp. 27–47. New York: Plenum PublishingGoogle Scholar
  9. Cox, D.D.; John, S. 1997: SDO: a statistical method for global optimization. In: Alexandrov, N.; Hussaini, M.Y. (eds.)Multidisciplinary design optimization: state of the art, pp. 315–329. Philadelphia: SIAMGoogle Scholar
  10. Currin, C.; Mitchell, T.; Morris, M.; Ylvisaker, D. 1988: A Bayesian approach to the design and analysis of computer experiments.Technical Report ORNL-6498, Oak Ridge National LaboratoryGoogle Scholar
  11. Davis, C. 1954: Theory of positive linear dependence.Amer. J. Math. 76, 733–746Google Scholar
  12. De Boor, C.; Ron, A. 1992: Computational aspects of polynomial interpolation in several variables.Mathematics of Computation 58, 705–727Google Scholar
  13. Dennis, J.E., Jr.; Torczon, V. 1991: Direct search methods on parallel machines.SIAM J. Optimiz. 1, 448–474Google Scholar
  14. Dennis, J.E., Jr.; Torczon, V. 1997: Managing approximation models in optimization. In: Alexandrov, N.; Hussaini, M.Y. (eds.)Multidisciplinary design optimization: state-of-the-art, pp. 330–347. Philadelphia: SIAM. Also available as Technical Report 95-19, Department of Computational and Applied Mathematics, Rice University, Houston, Texas 77005-1892Google Scholar
  15. Dennis, J.E., Jr.; Walker, H.F. 1984: Inaccuracy in quasi-Newton methods: local improvement theorems.Math. Prog. Study 22, 70–85Google Scholar
  16. Dixon, L.C.W.; Szegö G.P. (eds.) 1978:Towards global optimization 2. Amsterdam: North-Holland Pub. Co.Google Scholar
  17. Efron, B.; Stein, C. 1981: The jackknife estimate of variance.Annals of Statistics 9, 586–596Google Scholar
  18. Frank, P.D. 1995: Global modeling for optimization.SIAG/OPT Views-and-News 7, 9–12Google Scholar
  19. Geisser, S.; Eddy, W.F. 1979: A predictive approach to model selection.J. Amer. Statistical Assoc. 74, 153–160Google Scholar
  20. Giunta, A.A. 1997:Aircraft multidisciplinary optimization using design of experiments theory and response surface modeling methods. Ph.D. Thesis, Virginia Tech. Available as MAD 97-05-01, May 1997, Department of Aerospace and Ocean Engineering, Virginia Tech, 215 Randolph Hall, Blacksburg, Virginia 24061Google Scholar
  21. Jones, D.R.; Schonlau, M.; Welch, W.J. 1997: A data analytic approach to Bayesian global optimization.Proc. ASA Google Scholar
  22. Koehler, J.R.; Owen, A.B. 1996: Computer experiments. In: Ghosh, S.; Rao, C.R. (eds.)Handbook of statistics, volume 13, pp. 261–308. New York: Elsevier ScienceGoogle Scholar
  23. Levine, D. 1996: Users guide to the PGAPack parallel genetic algorithm library.Technical Report ANL-95/18, Argonne National Laboratory, 9700 South Cass Avenue, Argonne, Illinois 60439 Available from URL ftp: // Scholar
  24. Lewis, R.M.; Torczon, V. 1996: Pattern search algorithms for bound constrained minimization.Technical Report 96-20 ICASE, NASA Langley Research Center, Hampton, Virginia 23681-2199. To appear inSIAM J. Optimiz.Google Scholar
  25. Lewis, R.M.; Torczon, V. 1996: Rank ordering and positive bases in pattern search algorithms.Technical Report 96-71, ICASE, NASA Langley Research Center, Hampton, Virginia 23681-2199. In revision forMath. Prog. Google Scholar
  26. Lewis, R.M.; Torczon, V. 1997: Pattern search methods for linearly constrained minimization.Technical Report 98-03, ICASE, NASA Langley Research Center, Hampton, Virginia 23681-2199.SIAM J. Optimiz. (to appear)Google Scholar
  27. McKay, M.D.; Conover, W.J.; Beckman, R.J. 1979 A comparison of three methods for selecting values of input variables in the analysis of output from a computer code.Technometrics 21, 239–245Google Scholar
  28. Owen, A.B. 1992: Orthogonal arrays for computer experiments, integration and visualization.Statistica Sinica 2, 439–452Google Scholar
  29. Rinnooy Kan, A.H.G.; Timmer, G.T. 1984: A stochastic approach to global optimization. In: Boggs, P.T.; Byrd, R.H.; Schnabel, R.B. (eds.)Numerical optimization 1984 (Proc. SIAM Conf. on Numerical Optimization), pp. 245–262. Philadelphia: SIAMGoogle Scholar
  30. Ripley, B.D. 1988:Statistical inference for spatial processes. Cambridge: Cambridge University PressGoogle Scholar
  31. Sacks, J.; Welch, W.J.; Mitchell, T.J.; Wynn, H.P. 1989: Design and analysis of computer experiments.Statistical Sci.4, 409–435Google Scholar
  32. Serafini, D.B. 1998:A framework for managing models in nonlinear optimization of computationally expensive functions. Ph.D. Thesis, Rice UniversityGoogle Scholar
  33. Shultz, L.A.; Panda, B.; Tarzanin, F.J.; Derham, R.C.; Oh, B.K.; Dadone, L. 1994: Interdisciplinary analysis for advanced rotors-approach, capabilities and status. AHS Aeromechanics Specialists Conf., PS 4-1-4-15Google Scholar
  34. Snir, M.; Otto S.W.; Huss-Lederman, S.; Walker, D.W.; Dongarra, J. 1996.MPI: The complete reference. Cambridge, MA: The MIT PressGoogle Scholar
  35. Stein, M. 1987: Large sample properties of simulations using latin hypercube sampling.Technometrics 29, 143–151Google Scholar
  36. Tang, B. 1993: Orthogonal array-based latin hypercubes.J. Amer. Statistical Assoc. 88, 1392–1397Google Scholar
  37. Torczon, V. 1992: PDS: direct search methods for unconstrained optimization on either sequential or parallel machines.Technical Report 92-9, Department of Computational and Applied Mathematics, Rice University, Houston, Texas 77005-1892. In revision forACM Trans. Math. Software Google Scholar
  38. Torczon, V. 1995: Pattern search methods for nonlinear optimization.SIAG/OPT Views & News 6, 7–11Google Scholar
  39. Torczon, V. 1997: On the convergence of pattern search algorithms.SIAM J. Optimiz. 7, 1–25Google Scholar
  40. Torczon, V.; Trosset, M.W. 1998: From evolutionary operation to parallel direct search: pattern search algorithms for numerical optimization.Computing Sci. & Statistics 29, 396–401Google Scholar
  41. Trosset, M.W.; Torczon, V. 1997: Numerical optimization using computer experiments.Technical Report 97-38, ICASE, NASA Langley Research Center, Hampton, Virginia 23681-2199Google Scholar
  42. Trosset, M.W. 1997: I know it when I see it: toward a definition of direct search methods.SIAG/OPT Views & News 9, 7–10Google Scholar
  43. Watson, G.S. 1984: Smoothing and interpolation by kriging and with splines.Math. Geology 16, 601–615Google Scholar

Copyright information

© Springer-Verlag 1999

Authors and Affiliations

  • A. J. Booker
    • 1
  • J. E. DennisJr.
    • 2
  • P. D. Frank
    • 3
  • D. B. Serafini
    • 4
  • V. Torczon
    • 5
  • M. W. Trosset
    • 6
  1. 1.Boeing Shared Services Group, Applied Research and TechnologyMathematics & Engineering AnalysisSeattleUSA
  2. 2.Department of Computational and Applied Mathematics & Center for Research on Parallel ComputationRice UniversityHoustonUSA
  3. 3.Mathematics & Engineering Analysis, Boeing Shared Services Group, Applied Research and TechnologySeattleUSA
  4. 4.E.O. Lawrence Berkeley National LaboratoryNational Energy Research Scientific Computing CenterBerkeleyUSA
  5. 5.Department of Computer ScienceCollege of William & MaryWilliamsburgUSA
  6. 6.Department of MathematicsCollege of William & MaryWilliamsburgUSA

Personalised recommendations