Abstract
Among the possible classifications of the optimization algorithms we decided to divide them into two categories: deterministic and stochastic. By deterministic optimization all the algorithms that follow a rigorous mathematical approach are intended. Strictly speaking this refers to mathematical programming. After introducing the terminology used in this field, line-search and trust region strategies are described. The chapter is further subdivided into two major sections: unconstrained and constrained optimization. In the first several optimization methods are introduced such as simplex, Newton and quasi-Newton, conjugate directions, and Levenberg–Marquardt. In the latter, due to the complexity of the topic only a brief discussion on the main aspects and approaches found in constrained optimization algorithms is given. These are elimination methods, Lagrangian methods, active set methods, 0s methods. Sequential quadratic programming and mixed integer programming are also introduced. In the conclusions, the different algorithms are discussed in terms of their simplicity, reliability, and efficiency.