Introduction

Abstract

Many scientific and engineering problems can be formulated as a constrained optimization problem described mathematically as

$$\mathop {\min }\limits_{x \in S} f(x){\text{ }}subject{\text{ }}to{\text{ }}g(x),$$

(1.1)

where S is the solution space, f is the cost function, and g is the set of constraints. Various optimization problems can be categorized based on the characteristics of S, f, and g. If f and g are linear functions, then Equation (1.1) describes a linear optimization problem which can be readily solved. Otherwise, Equation (1.1) becomes a nonlinear optimization problem which is more difficult to solve. A prime example of a linear optimization problem is the linear programming problem where the constraints are in the form of g(x) ≥ 0 or g(x) = 0. The linear programming problem can be solved by the simplex algorithm [125] where the optimal solution can be found in a finite number of steps. However, many optimization problems encountered in engineering and other fields, such as the traveling salesman problem (TSP), various scheduling problems, etc., belong to a class of “difficult to solve” problems where deterministic algorithms are not applicable.