Unconstrained Optimization Algorithms

Abstract

In this chapter, we will present the most representative algorithms for solving an optimization problem without functional constraints, that is, the problem

$$ \begin{array}{lll} P: &{} \text {Min} &{} f(x) \\ &{} \text {s.t.} &{} x\in C, \end{array} $$

where $\emptyset \ne C\subset \mathop {\mathrm {dom}}f\subset \mathbb {R}^{n}.$ We will usually assume that f is smooth on C, i.e., that $f\in \mathcal {C}^{1}\left( V\right) ,$ where V is an open set such that $C\subset V\subset \mathop {\mathrm {dom}}f$. In fact, in many cases, to simplify, we will assume that $C=\mathbb {R}^{n},$ which means that P is the problem (1.1) with neither functional nor constraint sets. We will present conceptual algorithms that generate infinite sequences, which can be converted into implementable algorithms by adding some stopping criteria as the ones inspired in errors introduced in Subsection 5.2.1 or some approximate optimality conditions.