Constrained efficient global optimization with support vector machines

Abstract

This paper presents a methodology for constrained efficient global optimization (EGO) using support vector machines (SVMs). While the objective function is approximated using Kriging, as in the original EGO formulation, the boundary of the feasible domain is approximated explicitly as a function of the design variables using an SVM. Because SVM is a classification approach and does not involve response approximations, this approach alleviates issues due to discontinuous or binary responses. More importantly, several constraints, even correlated, can be represented using one unique SVM, thus considerably simplifying constrained problems. In order to account for constraints, this paper introduces an SVM-based “probability of feasibility” using a new Probabilistic SVM model. The proposed optimization scheme is constituted of two levels. In a first stage, a global search for the optimal solution is performed based on the “expected improvement” of the objective function and the probability of feasibility. In a second stage, the SVM boundary is locally refined using an adaptive sampling scheme. An unconstrained and a constrained formulation of the optimization problem are presented and compared. Several analytical examples are used to test the formulations. In particular, a problem with 99 constraints and an aeroelasticity problem with binary output are presented. Overall, the results indicate that the constrained formulation is more robust and efficient.

Appendices

Appendix A: Effect of the design of experiments size

This section presents an example of the effect of initial design of experiments on the optimization results, for Example 2 (Section 5.2). Three sets of initial DOEs are used, consisting of 5, 10 and 15 CVT samples to run the optimization. The percentage relative errors in the objective function value ϵ _k are plotted in Fig. 27 for the three cases. The constrained formulation (update scheme 2) is used. It is seen that for all the cases, the optimization converges within 21 iterations.

Fig. 27

Example 2. Evolution of the relative error ϵ _k with constrained formulation with different initial DOE sizes: 5 (left), 10 (center), 15 (left)

Appendix B: Two constraint problem

This appendix provides another analytical example. The constrained and unconstrained formulations are compared. The example is taken from Sasena (2002), and consists of two variables x ₁ and x ₂ in the range [ − 2, 2]. The feasible space for this problem is bounded by two constraints. The optimization problem is:

$$ \begin{array}{rll} \min\limits_{\mathbf{x}} && \quad f(\mathbf{x})\!=\!(1\!+\!A(x_1+x_2+1)^2)(30+B(2x_1\!-\!3x_2)^2)\\ \text{where} &&\quad A=19-14x_1+3x_1^2-14x_2+6x_1x_2+3x_2^2,\\ \text{and} &&\quad B=18-32x_1+12x_1^2+48x_2-36x_1x_2+27x_2^2\\ \text{s.t.} &&\quad g_1(\mathbf{x})=-3x_1+(-3x_2)^3 \leq 0\\ &&\quad g_2(\mathbf{x})=x_1-x_2-1 \leq 0 \end{array} $$

(25)

The objective function, the constraints and the optimum solution are depicted in Fig. 28. The actual optimum is at (0.5955, − 0.4045) with an objective function value of 289.85.

Fig. 28

Appendix B example. Individual constraints and the resulting feasible and infeasible regions (left). Objective function contours, boundary of feasible space and the the optimum solution (right)

The initial error with 10 CVT samples is 261.0%. The evolution of ϵ _k using the unconstrained and constrained formulations are plotted in Figs. 29 and 30. In order to check the consistency of the results, the algorithm was run 50 times. The mean, median, minimum and maximum errors are provided.

Fig. 29

Appendix B example. Evolution of ϵ _k with unconstrained formulation. 50 runs

Fig. 30

Appendix B example. Evolution of ϵ _k with constrained formulation. 50 runs

As a comparison, this example was also run using a purely Kriging-based approach where the two constraints are also approximated by Kriging. As described in Schonlau (1997) and Forrester et al. (2008), the approach is based on the product of EI times the probabilities of feasibility of each constraint (EI(x) P(g ₁(x) ≤ 0) P(g ₂(x) ≤ 0)). The run was carried out using a freely available online code (Forrester et al. 2008). The statistics of results based on 50 runs are gathered in Fig. 31. It is observed that in the SVM-based unconstrained approach as well as the purely Kriging-based approach, the optimization does not always converge accurately to the optimum. This is clearly demonstrated on Figs. 29 and 31. This is likely due to the fact that the gradient of the objective function is quite large around the optimum. On the other hand, the SVM-based constrained formulation (Fig. 30) does consistently reach the optimum accurately.

Fig. 31

Evolution of ϵ _k using A. Forrester’s code (Forrester et al. 2008). 50 runs