Second-Order Theory

For functions f:ℝⁿ → ℝ¯, the notions of ‘subderivative’ and ‘subgradient’, along with semidifferentiability and epi-differentiability, have provided a broad and effective generalization of first-order differentiation. What can be said, though, on the level of generalized second-order differentiation? And what might the use of this be?

Classically, second derivatives carry forward the analysis of first derivatives and provide quadratic approximations of a given function, whereas first derivatives by themselves only provide linear approximations. They serve as an intermediate link in an endless chain of differentiation that proceeds to third derivatives, fourth derivatives, and so on. In optimization, derivatives of third order and higher are rarely of importance, but second derivatives help significantly in the understanding of optimality, especially the formulation of sufficient conditions for local optimality in the absence of convexity. Such conditions form the basis for numerical methodology and assist in studies of what happens to optimal solutions when the parameters on which a problem depends are perturbed.