Abstract
Gaussian-Process based optimization methods have become very popular in recent years for the global optimization of complex systems with high computational costs. These methods rely on the sequential construction of a statistical surrogate model, using a training set of computed objective function values, which is refined according to a prescribed infilling strategy. However, this sequential optimization procedure can stop prematurely if the objective function cannot be computed at a proposed point. Such a situation can occur when the search space encompasses design points corresponding to an unphysical configuration, an ill-posed problem, or a non-computable problem due to the limitation of numerical solvers. To avoid such a premature stop in the optimization procedure, we propose to use a classification model to learn non-computable areas and to adapt the infilling strategy accordingly. Specifically, the proposed method splits the training set into two subsets composed of computable and non-computable points. A surrogate model for the objective function is built using the training set of computable points, only, whereas a probabilistic classification model is built using the union of the computable and non-computable training sets. The classifier is then incorporated in the surrogate-based optimization procedure to avoid proposing new points in the non-computable domain while improving the classification uncertainty if needed. The method has the advantage to automatically adapt both the surrogate of the objective function and the classifier during the iterative optimization process. Therefore, non-computable areas do not need to be a priori known. The proposed method is applied to several analytical problems presenting different types of difficulty, and to the optimization of a fully nonlinear fluid-structure interaction system. The latter problem concerns the drag minimization of a flexible hydrofoil with cavitation constraints. The efficiency of the proposed method compared favorably to a reference evolutionary algorithm, except for situations where the feasible domain is a small portion of the design space.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Simpson T, Poplinski J, Koch NP, Allen J (2001) Metamodels for computer-based engineering design: survey and recommendations. Eng Comput 17(2):129–150
Kleijnen JP (2009) Kriging metamodeling in simulation: a review. Eur J Oper Res 192(3):707–716
Vapnik V (1995) The nature of statistical learning theory. Springer, New York
Marrel A, Iooss B, Laurent B, Roustant O (2009) Calculations of sobol indices for the gaussian process metamodel. Reliab Eng Syst Saf 94(3):742–751
Wang P, Lu Z, Tang Z (2013) An application of the kriging method in global sensitivity analysis with parameter uncertainty. Appl Math Model 37(9):6543–6555
Nickisch H, Rasmussen CE (2008) Approximations for binary gaussian process classification. J Mach Learn Res 9:2035–2078
Liu Y, Shi Y, Zhou Q, Xiu R (2016) A sequential sampling strategy to improve the global fidelity of metamodels in multi-level system design. Struct Multidiscip Optim 53(6):1295–1313
Park C, Haftka RT, Kim NH (2017) Remarks on multi-fidelity surrogates. Struct Multidiscip Optim 55 (3):1029–1050
Dong H, Song B, Wang P, Dong Z (2017) Surrogate-based optimization with clustering-based space exploration for expensive multimodal problems. Struct Multidisc Optim 57(4):1553– 1577
Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive Black-Box functions. J Glob Optim 13(4):455– 492
Liu J, Song WP, Han ZH, Zhang Y (2017) Efficient aerodynamic shape optimization of transonic wings using a parallel infilling strategy and surrogate models. Struct Multidiscip Optim 55(3):925–943
Glaz B, Friedmann PP, Liu L (2008) Surrogate based optimization of helicopter rotor blades for vibration reduction in forward flight. Struct Multidiscip Optim 35(4):341–363
Glaz B, Friedmann PP, Liu L (2009) Helicopter vibration reduction throughout the entire flight envelope using surrogate-based optimization. J Amer Helicopter Soc 54(1):12007
Aghajari N, Schäfer M (2015) Efficient shape optimization for fluid–structure interaction problems. J Fluids Struct 57:298–313
Sacher M, Hauville F, Duvigneau R, Maître OL, Aubin N, Durand M (2017) Efficient optimization procedure in non-linear fluid-structure interaction problem: application to mainsail trimming in upwind conditions. J Fluids Struct 69:209–231
Picheny V, Wagner T, Ginsbourger D (2013) A benchmark of kriging-based infill criteria for noisy optimization. Struct Multidiscip Optim 48(3):607–626
Li Z, Ruan S, Gu J, Wang X, Shen C (2016) Investigation on parallel algorithms in efficient global optimization based on multiple points infill criterion and domain decomposition. Struct Multidiscip Optim 54 (4):747–773
Jain AK, Murty MN, Flynn PJ (1999) Data clustering: a review. ACM computing surveys (CSUR) 31 (3):264–323
Zhang Y, Park C, Kim NH, Haftka RT (2017) Function prediction at one inaccessible point using converging lines. J Mech Des 139(5):051402
Suykens J, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9 (3):293–300
Basudhar A, Dribusch C, Lacaze S, Missoum S (2012) Constrained efficient global optimization with support vector machines. Struct Multidiscip Optim 46(2):201–221
van Gestel T, Suykens JA, Baesens B, Viaene S, Vanthienen J, Dedene G, de Moor B, Vandewalle J (2004) Benchmarking least squares support vector machine classifiers. Mach Learn 54(1):5–32
Hansen N (2006) The CMA Evolution Strategy: a comparing review. In: Towards a new evolutionary computation. Springer, pp 75–102
Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. MIT Press, Cambridge
Hansen N, Arnold DV, Auger A (2015) Evolution strategies. In: Springer handbook of computational intelligence. Springer, pp 871–898
Hansen N (2016) The cma evolution strategy: a tutorial. arXiv:1604.00772
Huang D, Allen TT, Notz WI, Zheng N (2006) Global optimization of stochastic Black-Box systems via sequential kriging meta-models. J Global Optim 34(3):441–466
Schonlau M (1997) Computer Experiments and Global Optimization. PhD thesis, University of Waterloo, Waterloo, Ont., Canada, Canada. AAINQ22234
Cawley GC (2006) Leave-one-out cross-validation based model selection criteria for weighted ls-svms. In: 2006 IEEE international joint conference on neural network proceedings. IEEE, pp 1661–1668
Stone M (1974) Cross-validatory choice and assessment of statistical predictions. J R Stat Soc Ser B Methodol 36(2):111–147
Vapnik VN, Vapnik V (1998) Statistical learning theory, vol 1. Wiley, New York
Cawley GC, Talbot NL (2003) Efficient leave-one-out cross-validation of kernel fisher discriminant classifiers. Pattern Recogn 36(11):2585–2592
Cawley GC, Talbot NL (2004) Fast exact leave-one-out cross-validation of sparse least-squares support vector machines. Neural Netw 17(10):1467–1475
Cawley GC, Talbot NL (2007) Preventing over-fitting during model selection via bayesian regularisation of the hyper-parameters. J Mach Learn Res 8:841–861
Allen DM (1974) The relationship between variable selection and data augmentation and a method for prediction. Technometrics 16(1):125–127
Van Calster B, Luts J, Suykens JAK, Condous G, Bourne T, Timmerman D, Van Huffel S (2007) Comparing methods for multi-class probabilities in medical decision making using LS-SVMs and kernel logistic regression. In: Artificial neural networks – ICANN 2007: 17th international conference, Porto, Portugal, September 9-13, 2007, Proceedings, part II. Springer, Berlin, pp 139–148
Platt JC (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advan Large Margin Classifiers 10(3):61–74
Lin HT, Lin CJ, Weng RC (2007) A note on platt’s probabilistic outputs for support vector machines. Mach Learn 68(3):267– 276
Arnold DV, Hansen N (2012) A (1 + 1)-CMA-ES for constrained optimisation. In: Soule T, Moore JH (eds) GECCO, Philadelphia, United States, ACM. ACM Press, pp 297–304
Tenne Y, Goh C (2010) Computational intelligence in expensive optimization problems. Adaptation, learning, and optimization. Springer, Berlin
Platt JC Schölkopf B, Burges CJC, Smola AJ (eds) (1999) Fast training of support vector machines using sequential minimal optimization. MIT Press, Cambridge
Sacher M, Durand M, Berrini E, Hauville F, Duvigneau R, Le Maître O, Astolfi JA (2017) Flexible hydrofoil optimization for the 35th america’s cup with constrained ego method. In: International Conference on Innovation in High Performance Sailing Yachts, Innov’Sail, pp 193–205
Drela M (1989) XFOIL: an analysis and design system for low Reynolds number airfoils. Springer, Berlin, pp 1–12
Morgado J, Vizinho R, Silvestre M, Ps̈coa J (2016) {XFOIL} vs {CFD} performance predictions for high lift low reynolds number airfoils. Aerosp Sci Technol 52:207–214
Durand M, Leroyer A, Lothodé C, Hauville F, Visonneau M, Floch R, Guillaume L (2014) FSI Investigation on stability of downwind sails with an automatic dynamic trimming. Ocean Eng 90:129–139
Pedersen P (1973) Some properties of linear strain triangles and optimal finite element models. Int J Numer Methods Eng 7(4):415–429
Yang X, Song Q, Cao A (2005) Weighted support vector machine for data classification. In: Proceedings of the 2005 IEEE International Joint Conference on Neural Networks, vol 2, pp 859–864
Author information
Authors and Affiliations
Corresponding author
Additional information
Responsible Editor: Raphael Haftka
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sacher, M., Duvigneau, R., Le Maître, O. et al. A classification approach to efficient global optimization in presence of non-computable domains. Struct Multidisc Optim 58, 1537–1557 (2018). https://doi.org/10.1007/s00158-018-1981-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00158-018-1981-8