Skip to main content
Log in

Update strategies for kriging models used in variable fidelity optimization

  • Research Paper
  • Published:
Structural and Multidisciplinary Optimization Aims and scope Submit manuscript

Abstract

Many optimization methods for simulation-based design rely on the sequential use of metamodels to reduce the associated computational burden. In particular, kriging models are frequently used in variable fidelity optimization. Nevertheless, such methods may become computationally inefficient when solving problems with large numbers of design variables and/or sampled data points due to the expensive process of optimizing the kriging model parameters in each iteration. One solution to this problem would be to replace the kriging models with traditional Taylor series response surface models. Kriging models, however, were shown to provide good approximations of computer simulations that incorporate larger amounts of data, resulting in better global accuracy. In this paper, a metamodel update management scheme (MUMS) is proposed to reduce the cost of using kriging models sequentially by updating the kriging model parameters only when they produce a poor approximation. The scheme uses the trust region ratio (TR-MUMS), which is a ratio that compares the approximation to the true model. Two demonstration problems are used to evaluate the proposed method: an internal combustion engine sizing problem and a control-augmented structural design problem. The results indicate that the TR-MUMS approach is very effective; on the demonstration problems, it reduced the number of likelihood evaluations by three orders of magnitude compared to using a global optimizer to find the kriging parameters in every iteration. It was also found that in trust region-based method, the kriging model parameters need not be updated using a global optimizer—local methods perform just as well in terms of providing a good approximation without affecting the overall convergence rate, which, in turn, results in a faster execution time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Alexandrov NM, Lewis RM (2003) Large-scale PDE-constrained optimization. In: Biegler LT, Ghattas O, Heinkenschloss M, Bloemen Waanders Bv (eds) Lecture notes in computational science and engineering, vol. 30. Springer, Berlin Heidelberg New York

    Google Scholar 

  • Alexandrov NM, Lewis RM, Gumbert CR, Green LL, Newman PA (2000a) Optimization with variable-fidelity models applied to wing design. In: Proceedings of the 38th AIAA Aerospace Sciences Meeting and Exhibit, No. AIAA 2000-0841, Reno, Nevada, 2000

  • Alexandrov NM, Nielsen EJ, Lewis RM, Anderson WK (2000b) First-order model management with variable-fidelity physics applied to multi-element airfoil optimization. In: 8th AIAA/NASA/USAF/ISSMO Symposium on multidisciplinary analysis and optimization, No. AIAA paper 2000-4886, Long Beach, CA, 2000

  • Alexandrov NM, Dennis JE, Lewis RM, Torczon V (2001) A trust region framework for managing the use of approximation models in optimization. ICASE Report 97-50, NASA/CR-201745, October 2001

  • Booker AJ (1998) Design and analysis of computer experiments. In: Proceedings of the 7th AIAA/USAF/NASA ISSMO Symposium on multidisciplinary analysis and optimization, vol 1. St. Louis, MO, 1998, pp 118–128

  • Booker AJ, Dennis JE, Frank PD, Serafini DB, Torczon V, Trosset MW (1999) A rigorous framework for optimization of expensive function by surrogates. Struct Optim 17(1):1–13

    Article  Google Scholar 

  • Chang KJ, Haftka RT, Giles GL, Kao P-J (1993) Sensitivity-based scaling for approximating structural response. J Aircr 30(2):283–288

    Article  Google Scholar 

  • Chung H-S, Alonso JJ (2002a) Using gradients to construct cokriging approximation models for high-dimensional design optimization problems. In: 40th AIAA aerospace sciences meeting and exhibit, No. AIAA 2002-0317, Reno, NV, 2002

  • Chung H-S, Alonso JJ (2002b) Design of a low-boom supersonic business jet using cokriging approximation models. In: 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, No. AIAA 2002-5598, Atlanta, GA, 2002

  • Conn AR, Gould NIM, Toint PL (1988) Global convergence of a class of trust region algorithms for optimization with simple bounds. SIAM J Numer Anal 25(2):433–464

    Article  MathSciNet  MATH  Google Scholar 

  • Currin C, Mitchell T, Morris M, Ylvisaker D (1988) A Bayesian approach to the design and analysis of computer experiments. Technical Report ORNL-6498, Oak Ridge National Laboratory, Oak Ridge, TN

    Google Scholar 

  • Denison DGT, Holmes CC, Mallick BK, Smith AFM (2002) Bayesian methods for nonlinear classification and regression. Wiley, New York

    MATH  Google Scholar 

  • Gano SE, Renaud JE, Martin JD, Simpson TW (2005a) Update strategies for kriging models for use in variable fidelity optimization. In: Proceedings of the 1st multidisciplinary design optimization specialist conference, No. AIAA 2005-2057, Austin, TX, 2005

  • Gano SE, Renuad JE, Sanders B (2005b) Hybrid variable fidelity optimization using a kriging-based scaling function. AIAA J 43(11):2422–2430

    Article  Google Scholar 

  • Giunta AA, Watson LT (1998) A comparison of approximation modeling techniques: polynomial versus interpolating models. In: Proceedings of the 7th AIAA/USAF/NASA/ISSMO Symposium on multidisciplinary analysis and optimization, No. AIAA-98-4758, St. Louis, MO, 1998, pp. 392–404

  • Giunta AA, Eldred MS (2000) Implementation of a trust region model management strategy in the DAKOTA optimization toolkit. In: Proceeedings of the 8th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, No. AIAA-2000-4935, Long Beach, CA

  • Haftka RT (1991) Combining global and local approximations. AIAA J 29(9):1523–1525

    Article  Google Scholar 

  • Iman RL, Helton JC, Cambpell JE (1981a) An approach to sensitivity analysis of computer models, Part I: introduction, input variable selection and preliminary variable assessment. J Qual Technol 13:174–183

    Google Scholar 

  • Iman RL, Helton JC, Cambpell JE (1981b) An approach to sensitivity analysis of computer models, Part II: ranking of input variables, response surface validation, distribution effect and technique synopsis. J Qual Technol 13:232–240

    Google Scholar 

  • Ingber L (1989) Very fast simulated re-annealing. J Math Comput Model 12:967–973

    Article  MathSciNet  MATH  Google Scholar 

  • Ingber L (1992) Genetic algorithms and very fast simulated reannealing: a comparison. J Math Comput Model 16(11):87–100

    Article  MathSciNet  MATH  Google Scholar 

  • Ingber L (1993) Simulated annealing: practice versus theory. J Math Comput Model 18(11):29–57

    Article  MathSciNet  MATH  Google Scholar 

  • Ingber L (1996) Adaptive simulated annealing (ASA): lessons learned. J Control Cybern 25(1):33–54

    MATH  Google Scholar 

  • Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13(4):455–492

    Article  MathSciNet  MATH  Google Scholar 

  • Kowalik J, Osborne MR (1968) Methods for unconstrained optimization problems. American Elsevier, New York

    MATH  Google Scholar 

  • Lewis RM, Nash SG (2000) A multigrid approach to the optimization of systems governed by differential equations In: 8th AIAA/NASA/USAF/ISSMO Symposium on multidisciplinary analysis and optimization, No. AIAA2000-4890, Long Beach, CA, 6–8 September 2000

  • Lim YB, Sacks JJSW, Welch WJ (1997) Design and analysis of computer experiments when the output is highly correlated over the input space. Technical report 62, National Institute of Statistical Sciences, NC, USA

  • Lophaven SN, Nielsen HB, Søndergaard J (2002a) DACE a Matlab kriging toolbox. Technical Report IMM-REP-2002-12, Technical University of Denmark, version 2.0

  • Lophaven SN, Nielsen HB, Søndergaard J (2002b) Aspects of the Matlab toolbox DACE. Technical report IMM-REP-2002-13, Technical University of Denmark

  • Martin JD, Simpson TW (2003) A study on the use of kriging models to approximate deterministic computer models. In: Proceedings of ASME 2003 design engineering technical conferences and computers and information in engineering conference, No. DET2003/DAC-48762, Chicago, IL, 2003

  • Martin JD, Simpson TW (2004) A Monte Carlo simulation of the kriging model. In: Proceedings of the 11th AIAA/ISSMO multidisciplinary analysis & optimization conference, No. AIAA 2004-4483, Albany, NY, 30 September 2004

  • Martin JD, Simpson TW (2005) On the use of kriging models to approximate deterministic computer models. AIAA J 43(4):853–863

    Article  Google Scholar 

  • Matheron G (1971) The theory of regionalized variables and its applications. Ecole des Mines, Fontainebleau

    Google Scholar 

  • McAllister CD, Simpson TW (2003) Multidisciplinary robust design optimization of an internal combustion engine. ASME J Mech Des 125:124–130

    Article  Google Scholar 

  • Myers RH, Montgomery DC (1995) Response surface methodology: process and product optimization using designed experiments. Wiley, New York

    MATH  Google Scholar 

  • Ripley BD (1988) Statistical inference for spatial processes. Cambridge University Press, Cambridge, UK

    Google Scholar 

  • Rodríguez JF, Renaud JE (1998) Convergence of trust region augmented Lagrangian methods using variable fidelity approximation data. Struct Optim 15:141–156

    Article  Google Scholar 

  • Rodriguez JF, Pérez VM, Padmanabhan D, Renaud JE (2001) Sequential approximate optimization using variable fidelity response surface approximations. Struct Multidisc Optim 22(1):24–34

    Article  Google Scholar 

  • Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4(4):409–423

    Article  MathSciNet  MATH  Google Scholar 

  • Sasena MJ, Papalmbros P, Goovaerts P (2002) Exploration of metamodeling sampling criteria for constrained global optimization. Eng Optim 34(3):263–278

    Article  Google Scholar 

  • Simpson TW, Maurey TM, Korte JJ, Mistree F (2001a) Kriging metamodels for global approximation in simulation-based design. Eng Comput 17(2):129–150

    Article  MATH  Google Scholar 

  • Simpson TW, Mauery TM, Korte JJ, Mistree F (2001b) Kriging metamodels for global approximation in simulation-based multidisciplinary design optimization. AIAA J 39(12):2233–2241

    Article  Google Scholar 

  • Simpson TW, Peplinski JD, Koch PN, Allen JK (2001c) Metamodels for computer-based engineering design: survey and recommendations. Eng Comput 17:129–150

    Article  MATH  Google Scholar 

  • Simpson TW, Booker AJ, Gosh D, Gunita AA (2004) Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidisc Optim 27(5):302–313

    Article  Google Scholar 

  • Sobieszczanski-Sobieski J, Bloebaum CL, Hajela P (1991) Sensitivity of control-augmented structure obtained by a system decomposition method. AIAA J 29(2):264–270

    Article  Google Scholar 

  • Stein ML (1999) Interpolation of spatial data: some theory for kriging. Springer, Berlin Heidelberg New York

    MATH  Google Scholar 

  • Wagner TC, Papalambros PY (1991) Optimal engine design using nonlinear programming and the engine system assessment model. Technical report, Ford Motor Company Scientific Research Laboratories, Dearborn, MI and Department of Mechanical Engineering at the University of Michigan, Ann Arbor, MI

  • Warnes JJ, Ripley BD (1987) Problems with likelihood estimation of covariance function of spatial Gaussian processes. Biometrika 74(3):640–642

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to John E. Renaud.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gano, S.E., Renaud, J.E., Martin, J.D. et al. Update strategies for kriging models used in variable fidelity optimization. Struct Multidisc Optim 32, 287–298 (2006). https://doi.org/10.1007/s00158-006-0025-y

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00158-006-0025-y

Keywords

Navigation