Skip to main content
SpringerLink
Log in

Initial sampling methods in metamodel-assisted optimization

  • Original Article
  • Published:
Engineering with Computers Aims and scope Submit manuscript

Abstract

The modern engineering design process often relies on numerical analysis codes to evaluate candidate designs, a setup which formulates an optimization problem which involves a computationally expensive black-box function. Such problems are often solved using a algorithm in which a metamodel approximates the true objective function and provides predicted objective values at a lower computational cost. The metamodel is trained using an initial sample of vectors, and this implies that the procedure by which the initial sample is generated can impact the overall effectiveness of the optimization search. Approaches for generating the initial sample include the statistically based design of experiments, and the more recent search-driven sampling which generates the sample vectors with a direct-search optimizer. This study compares these two approaches in terms of their overall impact on the optimization search and formulates guidelines in which scenario is each approach preferable. An extensive analysis shows that: (a) the main factor affecting search-driven sampling is the size of the initial sample, and such methods performed better in large initial samples, (b) design of experiments methods tended to perform better in lower sample sizes, (c) generating a sample which is space-filling improved the overall search effectiveness

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Acar E (2010) Various approaches for constructing an ensemble of metamodels using local measures. Struct Multidiscip Optim 42(6):879–896

    Article  Google Scholar 

  2. Box GEP, Wilson KB (1951) On the experimental attainment of optimum conditions. J R Stat Soc Ser B (Methodological) 13:1–45

    MathSciNet  MATH  Google Scholar 

  3. Büche D, Schraudolph NN, Koumoutsakos P (2005) Accelerating evolutionary algorithms with Gaussian process fitness function models. IEEE Trans Syst Man Cybernet Part C 35(2):183–194

    Article  Google Scholar 

  4. Chen VP, Barton RR, Meckesheimer M, Tsui KL (2006) A review on design, modeling and applications of computer experiments. IIE Trans 38(4):273–291

    Article  Google Scholar 

  5. Chipperfield A, Fleming P, Pohlheim H, Fonseca C (1994) Genetic algorithm TOOLBOX for use with MATLAB, version 1.2. Department of Automatic Control and Systems Engineering, University of Sheffield, Sheffield

  6. Drela M, Youngren H (2001) XFOIL 6.9 User Primer. Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge

  7. El-Betalgy MA, Keane AJ (2001) Evolutionary optimization for computationally expensive problems using Gaussian processes. In: Arbania H (ed) Proceedings of the International Conference on artificial intelligence-IC-AI’2001. CSREA Press, pp 708–714

  8. Fang KT, Lin DKJ, Winker P, Zhang Y (2000) Uniform design: theory and application. Technometrics 42(2):237–248

    Article  MathSciNet  MATH  Google Scholar 

  9. Finney DJ (1945) Fractional replication of factorial arrangments. Ann Eugen 12:291–301

    Article  MathSciNet  Google Scholar 

  10. Fisher RA (1926) The arrangement of field experiments. J Minist Agric 33:503–513

    Google Scholar 

  11. Forrester AIJ, Keane AJ (2008) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1–3):50–79

    Google Scholar 

  12. Gorissen D, Dhaene T, De Turck F (2009) Evolutionary model type selection for global surrogate modeling. J Mach Learn Res 10:2039–2078

    MathSciNet  MATH  Google Scholar 

  13. Grosso A, Jamali A, Locatelli M (2009) Finding maximin latin hypercube designs by iterated local search heuristics. Eur J Oper Res 197(2):541–547

    Article  MATH  Google Scholar 

  14. Hawe GI, Sykulski JK (2006) Balancing exploration exploitation using Kriging surrogate models in electromagnetic design optimization. In: Proceedings of the 12th Biennial IEEE Conference on electromagnetic field computation. IEEE, p 226

  15. Hicks RM, Henne PA (1978) Wing design by numerical optimization. J Aircraft 15(7):407–412

    Article  Google Scholar 

  16. Husslage BGM, Rennen G, van Dam ER, den Hertog D (2011) Space-filling latin hypercube designs for computer experiments. Optim Eng 12(4):611–630

    Article  MATH  Google Scholar 

  17. Jin R, Chen W, Sudjianto A (2005) An efficient algorithm for constructing optimal design of computer experiments. J Stat Plan Inference 134(1):268–287

    Article  MathSciNet  MATH  Google Scholar 

  18. Jin Y (2005) A comprehensive survey of fitness approximation in evolutionary computation. J Soft Comput 9(1):3–12

    Article  Google Scholar 

  19. Johnson ME, Moore LM, Ylvisaker D (1990) Minimax and maximin distance designs. J Stat Plan Inference 26(2):131–148

    Article  MathSciNet  Google Scholar 

  20. Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Global Optim 13:455–492

    Article  MathSciNet  MATH  Google Scholar 

  21. de Jong KA (2006) Evolutionary computation: a unified approach. MIT Press, Cambridge

    Google Scholar 

  22. Kalagnanam JR, Diwekar UM (1997) An effcient sampling technique for off-line quality control. Technometrics 39(3):308–319

    Article  MATH  Google Scholar 

  23. Krishnakumar K (1989) Micro-genetic algorithms for stationary and non-stationary function optimization. In: Rodriguez GE (ed) Intelligent control and adaptive systems, SPIE, Bellingham, Wash. Proceedings of SPIE-the International Society for optical engineering, USA

  24. Laurenceau J, Sagaut P (2008) Building efficient response surfaces of aerodynamic functions with Kriging and Cokriging. AIAA J 46(2):498–507

    Article  Google Scholar 

  25. Liang KH, Yao X, Newton C (2000) Evolutionary search of approximated N-dimensional landscapes. Int J Knowl Based Intell Eng Syst 4(3):172–183

    Google Scholar 

  26. Lim D, Ong YS, Jin Y (2007) A study on metamodeling techniques, ensembles and multi-surrogates in evolutionary computation. In: Thierens D (ed) Proceedings of the genetic and evolutionary computation conference-GECCO 2007. ACM Press, New York, pp 1288–1295

    Google Scholar 

  27. Martin JD, Simpson TW (2005) Use of Kriging models to approximate deterministic computer models. AIAA J 43(4):853–863

    Article  Google Scholar 

  28. McKay MD, Beckman RJ, Conover WJ (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239–245

    MathSciNet  MATH  Google Scholar 

  29. Metropolis N, Ulam S (1949) The Monte Carlo method. J Am Stat Assoc 44(247):335–341

    Article  MathSciNet  MATH  Google Scholar 

  30. Morris MD, Mitchell TJ (1995) Exploratory designs for computational experiments. J Stat Plan Inference 43:381–402

    Article  MATH  Google Scholar 

  31. Mugunthan P, Shoemaker CA (2006) Assessing the impacts of parameter uncertainty for computationally expensive groundwater models. Water Resourc Res 42(10)

  32. Myers RH, Montgomery DC (1995) Response surface methodology: process and product optimization using designed experiments. Wiley, New York

    MATH  Google Scholar 

  33. Neri F, del Toro Garcia X (2008) Surrogate assisted local search on PMSM drive design. Int J Comput Math Electr Electron Eng 27(3):573–592

    Article  MATH  Google Scholar 

  34. Owen AB (1992) Orthogonal arrays for computer experiments, integration and visualization. Stat Sinica 2:439–452

    MATH  Google Scholar 

  35. Parr JM, Holden CME, Forrester AIJ, Keane AJ (2010) Review of efficient surrogate infill sampling criteria with constraint handling. In: Rodrigues H, Herskovits J, Mota Soares C, Miranda Guedes J, Folgado J, Araujo A, Moleiro F, Kuzhichalil J, Aguilar Madeira J, Dimitrovova Z (eds) Second International Conference on engineering optimization. Technical University of Lisbon

  36. Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker KP (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 41:1–28

    Article  Google Scholar 

  37. Quttineh NH, Holmström K (2009) The influence of experimental designs on the performance of surrogate model based costly global optimization solvers. Stud Inf Control 18(1):87–95

    Google Scholar 

  38. Ratle A (1999) Optimal sampling strategies for learning a fitness model. The 1999 IEEE Congress on evolutionary computation-CEC 1999. IEEE, Piscataway, pp 2078–2085

    Google Scholar 

  39. Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4(4):409–435

    Article  MathSciNet  MATH  Google Scholar 

  40. Senecal PK (2000) Numerical optimization using the GEN4 micro-genetic algorithm code. Tech. rep., Engine Research Center, University of Wisconsin-Madison, Wisconsin

  41. Sherwy MC, Wynn HP (1989) Maximum entropy sampling. J Appl Stat 14(2):165–170

    Google Scholar 

  42. Sheskin DJ (2007) Handbook of parametric and nonparametric statistical procedures, 4th edn. Chapman and Hall, Boca Raton

    MATH  Google Scholar 

  43. Simpson TW, Lin DKJ, Chen W (2001) Sampling strategies for computer experiments: design and analysis. Int J Reliab Appl 2(3):209–240

    Google Scholar 

  44. Sóbester A, Leary SJ, Keane AJ (2005) On the design of optimization strategies based on global response surface approximation models. J Global Optim 33:31–59

    Article  MathSciNet  MATH  Google Scholar 

  45. Suganthan PN, Hansen N, Liang JJ, Deb K, Chen YP, Auger A, Tiwari S (2005) Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. Technical report, KanGAL 2005005, Nanyang Technological University, Singapore and Kanpur Genetic Algorithms Laboratory, Indian Institute of Technology Kanpur, India

  46. Tenne Y, Armfield SW (2009) A framework for memetic optimization using variable global and local surrogate models. J Soft Comput 13(8):781–793

    Article  Google Scholar 

  47. Tenne Y, Goh CK (eds) (2010) Computational intelligence in expensive optimization problems, evolutionary learning and optimization, vol 2. Springer, Berlin

    Google Scholar 

  48. Toal DJJ, Bressloff NW, Keane AJ (2008) Kriging hyperparameter tuning strategies. AIAA J 46(5):1240–1252

    Article  Google Scholar 

  49. Törn A, Žilinskas A (1989) Global optimization. No. 350 in lecture notes in computer science. Springer, Berlin, Heidelberg, New York

    Google Scholar 

  50. Viana FAC, Venter G, Balabanov V (2009) An algorithm for fast optimal latin hypercube design of experiments. Int J Numer Methods Eng 82(2):135–156

    MathSciNet  Google Scholar 

  51. Wang GG, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. J Mech Design 129(4):370–380

    Article  MathSciNet  Google Scholar 

  52. Wu HY, Yang S, Liu F, Tsai HM (2003) Comparison of three geometric representations of airfoils for aerodynamic optimization. In: Proceedings of the 16th AIAA computational fluid dynamics conference. American Institute of Aeronautics and Astronautics, Reston, Virginia, AIAA, pp 2003–4095

  53. You H, Yang M, Wang D, Jia X (2009) Kriging model combined with Latin hypercube sampling for surrogate modeling of analog integrated circuit performance. Proceedings of the Tenth International Symposium on quality electronic design-ISQED 2009. IEEE, Piscataway, pp 554–558

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Ariel University, Ariel, Israel

    Yoel Tenne

Authors
  1. Yoel Tenne
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yoel Tenne.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tenne, Y. Initial sampling methods in metamodel-assisted optimization. Engineering with Computers 31, 661–680 (2015). https://doi.org/10.1007/s00366-014-0372-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00366-014-0372-z

Keywords

Search

Navigation