Numerical Solution of Generalized Minimax Problems

Abstract

This contribution contains the description and investigation of three numerical methods for solving generalized minimax problems. These problems consists in the minimization of nonsmooth functions which are compositions of special smooth convex functions with maxima of smooth functions. The most important functions of this type are the sums of maxima of smooth functions. Section 11.2 is devoted to primal interior point methods which use solutions of nonlinear equations for obtaining minimax vectors. Section 11.3 contains investigation of smoothing methods, based on using exponential smoothing terms. Section 11.4 contains short description of primal-dual interior point methods based on transformation of generalized minimax problems to general nonlinear programming problems. Finally the last section contains results of numerical experiments.

Appendix

Algorithm 11.1: Primal interior point method

________________________

 

Data::

A tolerance for the gradient norm of the Lagrange function $\underline {\varepsilon } > 0$. A precision for determination of a minimax vector $\underline {\delta } > 0$. Bounds for a barrier parameter $0 < \underline {\mu } < \overline {\mu }$. Coefficients for decrease of a barrier parameter 0 < λ < 1, σ > 1 (or 0 < 𝜗 < 1). A tolerance for a uniform descent ε ₀ > 0. A tolerance for a step length selection ε ₁ > 0. A maximum step length $\overline {\varDelta } > 0$.

Input.:

A sparsity pattern of the matrix A(x) = [A ₁(x), …, A _m(x)]. A starting point ${\boldsymbol x} \in \mathbb {R}^n$.

Step 1.:

(Initiation) Choose $\mu \leq \overline {\mu }$. Determine a sparse structure of the matrix W = W(x;μ) from the sparse structure of the matrix A(x) and perform a symbolic decomposition of the matrix W (described in [2, Section 1.7.4]). Compute values f _kl(x), 1 ≤ k ≤ m, 1 ≤ l ≤ m _k, values $F_k({\boldsymbol x}) = \max _{1 \leq l \leq m_k} f_{kl}({\boldsymbol x})$, 1 ≤ k ≤ m, and the value of objective function (11.4). Set r = 0 (restart indicator).

Step 2.:

(Termination) Solve nonlinear equations (11.44) with precision $\underline {\delta }$ to obtain a minimax vector z(x;μ) and a vector of Lagrange multipliers u(x;μ). Determine a matrix A = A(x) and a vector g = g(x;μ) = A(x)u(x;μ). If $\mu \leq \underline {\mu }$ and $\|{\boldsymbol g}\| \leq \underline {\varepsilon }$, terminate the computation.

Step 3.:

(Hessian matrix approximation) Set G = G(x;μ) or compute an approximation G of the Hessian matrix G(x;μ) using gradient differences or using quasi-Newton updates (Remark 11.13).

Step 4.:

(Direction determination) Determine a matrix $\nabla ^2 \hat {B}({\boldsymbol x}; \mu )$ by (11.48) and a vector Δx by solving Eq. (11.49) with the right-hand side defined by (11.47).

Step 5.:

(Restart) If r = 0 and (11.54) does not hold, set G = I, r = 1 and go to Step 4. If r = 1 and (11.54) does not hold, set Δx = −g. Set r = 0.

Step 6.:

(Step length selection) Determine a step length α > 0 satisfying inequalities (11.55) (for a barrier function $\hat {B}({\boldsymbol x}; \mu )$ defined by (11.46)) and $\alpha \leq \overline {\varDelta }/\|\varDelta {\boldsymbol x}\|$. Note that nonlinear equations (11.44) are solved at the point x + αΔx. Set x := x + αΔx. Compute values f _kl(x), 1 ≤ k ≤ m, 1 ≤ l ≤ m _k, values $F_k({\boldsymbol x}) = \max _{1 \leq l \leq m_k} f_{kl}({\boldsymbol x})$, 1 ≤ k ≤ m, and the value of objective function (11.4).

Step 7.:

(Barrier parameter update) Determine a new value of a barrier parameter $\mu \geq \underline {\mu }$ using Procedure A or Procedure B. Go to Step 2.

The values $\underline {\varepsilon } = 10^{-6}$, $\underline {\delta } = 10^{-6}$, $\underline {\mu } = 10^{-8}$, $\overline {\mu } = 1$, λ = 0.85, σ = 100, 𝜗 = 0.1, ε ₀ = 10⁻⁸, ε ₁ = 10⁻⁴, and $\overline {\varDelta } = 1000$ were used in our numerical experiments.

Algorithm 11.2: Smoothing method

________________________________

 

Data::

A tolerance for the gradient norm of the smoothing function $\underline {\varepsilon } > 0$. Bounds for a smoothing parameter $0 < \underline {\mu } < \overline {\mu }$. Coefficients for decrease of a smoothing parameter 0 < λ < 1, σ > 1 (or 0 < 𝜗 < 1). A tolerance for a uniform descent ε ₀ > 0. A tolerance for a step length selection ε ₁ > 0. A maximum step length $\overline {\varDelta } > 0$.

Input.:

A sparsity pattern of the matrix A(x) = [A ₁(x), …, A _m(x)]. A starting point ${\boldsymbol x} \in \mathbb {R}^n$.

Step 1.:

(Initiation) Choose $\mu \leq \overline {\mu }$. Determine a sparse structure of the matrix W = W(x;μ) from the sparse structure of the matrix A(x) and perform a symbolic decomposition of the matrix W (described in [2, Section 1.7.4]). Compute values f _kl(x), 1 ≤ k ≤ m, 1 ≤ l ≤ m _k, values $F_k({\boldsymbol x}) = \max _{1 \leq l \leq m_k} f_{kl}({\boldsymbol x})$, 1 ≤ k ≤ m, and the value of objective function (11.4). Set r = 0 (restart indicator).

Step 2.:

(Termination) Determine a vector of smoothing multipliers u(x;μ) by (11.87). Determine a matrix A = A(x) and a vector g = g(x;μ) = A(x)u(x;μ). If $\mu \leq \underline {\mu }$ and $\|{\boldsymbol g}\| \leq \underline {\varepsilon }$, terminate the computation.

Step 3.:

(Hessian matrix approximation) Set G = G(x;μ) or compute an approximation G of the Hessian matrix G(x;μ) using gradient differences or using quasi-Newton updates (Remark 11.13).

Step 4.:

(Direction determination) Determine the matrix W by (11.94) and the vector Δx by (11.93) using the Gill–Murray decomposition of the matrix W.

Step 5.:

(Restart) If r = 0 and (11.54) does not hold, set G = I, r = 1 and go to Step 4. If r = 1 and (11.54) does not hold, set Δx = −g. Set r = 0.

Step 6.:

(Step length selection) Determine a step length α > 0 satisfying inequalities (11.55) (for a smoothing function S(x;μ)) and $\alpha = \overline {\varDelta }/\|\varDelta {\boldsymbol x}\|$. Set x := x + αΔx. Compute values f _kl(x), 1 ≤ k ≤ m, 1 ≤ l ≤ m _k, values $F_k({\boldsymbol x}) = \max _{1 \leq l \leq m_k} f_{kl}({\boldsymbol x})$, 1 ≤ k ≤ m, and the value of the objective function (11.4).

Step 7.:

(Smoothing parameter update) Determine a new value of the smoothing parameter $\mu \geq \underline {\mu }$ using Procedure A or Procedure B. Go to Step 2.

The values $\underline {\varepsilon } = 10^{-6}$, $\underline {\mu } = 10^{-6}$, $\overline {\mu } = 1$, λ = 0.85, σ = 100, 𝜗 = 0.1, ε ₀ = 10⁻⁸, ε ₁ = 10⁻⁴, and $\overline {\varDelta } = 1000$ were used in our numerical experiments.

Algorithm 11.3: Primal-dual interior point method

___________________

Data::

A tolerance for the gradient norm $\underline {\varepsilon } > 0$. A parameter for determination of active constraints $\tilde {\varepsilon } > 0$. A parameter for initiation of slack variables and Lagrange multipliers δ > 0. An initial value of the barrier parameter $\overline {\mu } > 0$. A precision for the direction determination 0 ≤ ω < 1. A parameter for the step length selection 0 < γ < 1. A tolerance for the step length selection ε ₁ > 0. Maximum step length $\overline {\varDelta } > 0$.

Input.:

A sparsity pattern of the matrix A(x) = [A ₁(x), …, A _m(x)]. A starting point ${\boldsymbol x} \in \mathbb {R}^n$.

Step 1.:

(Initialization) Compute values f _kl(x), 1 ≤ k ≤ m, 1 ≤ l ≤ m _k, and set $F_k({\boldsymbol x}) = \max _{1 \leq l \leq m_k} f_{kl}({\boldsymbol x})$, z _k = F _k(x) + δ, 1 ≤ k ≤ m. Compute values $c_{kl}(\tilde {{\boldsymbol x}}) = f_{kl}({\boldsymbol x}) - z_k$, and set $s_{kl} = -c_{kl}(\tilde {{\boldsymbol x}})$, u _kl = δ. Set $\mu = \overline {\mu }$ and compute the value of the barrier function $\tilde {B}_{\mu }(\tilde {{\boldsymbol x}},{\boldsymbol s})$.

Step 2.:

(Termination) Determine a matrix $\tilde {A}(\tilde {{\boldsymbol x}})$ and a vector $\tilde {{\boldsymbol g}}(\tilde {{\boldsymbol x}},{\boldsymbol u}) = \tilde {A}(\tilde {{\boldsymbol x}}) {\boldsymbol u}$ by (11.111). If the KKT conditions $\|\tilde {{\boldsymbol g}}(\tilde {{\boldsymbol x}},{\boldsymbol u})\| \leq \underline {\varepsilon }$, $\|{\boldsymbol c}(\tilde {{\boldsymbol x}}) + {\boldsymbol s}\| \leq \underline {\varepsilon }$, and ${\boldsymbol s}^T {\boldsymbol u} \leq \underline {\varepsilon }$ are satisfied, terminate the computation.

Step 3.:

(Hessian matrix approximation) Set G = G(x, u) or compute an approximation G of the Hessian matrix G(x, u) using gradient differences or utilizing quasi-Newton updates (Remark 11.13). Determine a parameter σ ≥ 0 by (11.121) or set σ = 0. Split the constraints into active if $\hat {s}_{kl} \leq \tilde {\varepsilon } \hat {u}_{kl}$ and inactive if $\check {s}_{kl} > \tilde {\varepsilon } \check {u}_{kl}$.

Step 4.:

(Direction determination) Determine the matrix $\tilde {G} = \tilde {G}(\tilde {{\boldsymbol x}},{\boldsymbol u})$ by (11.111) (where the Hessian matrix G(x, u) is replaced with its approximation G). Determine vectors $\varDelta \tilde {{\boldsymbol x}}$ and $\varDelta \hat {{\boldsymbol u}}$ by solving linear system (11.115), a vector $\varDelta \check {{\boldsymbol u}}$ by (11.114), and a vector Δs by (11.116). Linear system (11.115) is solved either directly using the Bunch–Parlett decomposition (we carry out both the symbolic and the numeric decompositions in this step) or iteratively by the conjugate gradient method with indefinite preconditioner (11.123). Compute the derivative of the augmented Lagrange function by formula (11.120).

Step 5.:

(Restart) If P′(0) ≥ 0, determine a diagonal matrix $\tilde {D}$ by (11.124), set $\tilde {G} = \tilde {D}$, σ = 0, and go to Step 4.

Step 6.:

(Step length selection) Determine a step length parameter α > 0 satisfying inequalities P(α) − P(0) ≤ ε ₁αP′(0) and $\alpha \leq \overline {\varDelta }/\|\varDelta {\boldsymbol x}\|$. Determine new vectors $\tilde {{\boldsymbol x}} := \tilde {{\boldsymbol x}} + \alpha \varDelta \tilde {{\boldsymbol x}}$, s := s(α), u := u(α) by (11.117). Compute values f _kl(x), 1 ≤ k ≤ m, 1 ≤ l ≤ m _k, and set $c_{kl}(\tilde {{\boldsymbol x}}) = f_{kl}({\boldsymbol x}) - z_k$, 1 ≤ k ≤ m, 1 ≤ l ≤ m _k. Compute the value of the barrier function $\tilde {B}_{\mu }(\tilde {{\boldsymbol x}},{\boldsymbol s})$.

Step 7.:

(Barrier parameter update) Determine a new value of the barrier parameter $\mu \geq \underline {\mu }$ using Procedure C. Go to Step 2.

The values $\underline {\varepsilon } = 10^{-6}$, $\tilde {\varepsilon } = 0.1$, δ = 0.1, ω = 0.9, γ = 0.99, $\overline {\mu } = 1$, ε ₁ = 10⁻⁴, and $\overline {\varDelta } = 1000$ were used in our numerical experiments.