A Lower Bound for the Penalty Parameter in the Exact Minimax Penalty Function Method for Solving Nondifferentiable Extremum Problems

Abstract

In the paper, we consider the exact minimax penalty function method used for solving a general nondifferentiable extremum problem with both inequality and equality constraints. We analyze the relationship between an optimal solution in the given constrained extremum problem and a minimizer in its associated penalized optimization problem with the exact minimax penalty function under the assumption of convexity of the functions constituting the considered optimization problem (with the exception of those equality constraint functions for which the associated Lagrange multipliers are negative—these functions should be assumed to be concave). The lower bound of the penalty parameter is given such that, for every value of the penalty parameter above the threshold, the equivalence holds between the set of optimal solutions in the given extremum problem and the set of minimizers in its associated penalized optimization problem with the exact minimax penalty function.

1 Introduction

Considerable attention has been given in recent years to devising methods for solving nonlinear extremum problems using penalty functions. The usual strategy is to convert a constrained nonlinear extremum problem to a sequence of unconstrained minimizations of penalty functions. These penalty functions are constructed so that the sequence of unconstrained optimal solutions approaches the constrained optimal solution in the given optimization problem (see, for example, [1, 2]).

A particular subclass of penalty functions is the so-called exact penalty functions. These functions can be subdivided, in turn, into two main classes: nondifferentiable exact penalty functions and differentiable exact penalty functions. Nondifferentiable exact penalty functions were introduced for the first time by Eremin [3] and by Zangwill [4].

In the exact penalty functions methods, the given constrained optimization problem is replaced by an unconstrained optimization problem, for which the objective function is the sum of a certain “merit” function (which reflects the objective function of the given extremum problem) and a penalty term, which reflects the constraint set. The merit function is chosen in general as the original objective function, while the penalty term is obtained by multiplying a suitable function, which represents the constraints, by a positive parameter c, called the penalty parameter. A given penalty parameter c is called an exact penalty parameter if every solution of the given optimization problem can be found by solving the unconstrained optimization problem with the penalty function associated with c.

In [5], Charalambous introduced a class of nondifferentiable exact penalty functions. One of the subclasses of these nondifferentiable exact penalty function methods is the exact minimax penalty function method. A minimax approach to nonlinear optimization was presented by Bandler and Charalambous in [6]. They formulated, for the given nonlinear optimization problem, an unconstrained minimax optimization problem. Under reasonable restrictions, Bandler and Charalambous showed that a point satisfying the necessary conditions for a minimax optimum also satisfies the Kuhn–Tucker necessary conditions for the given optimization problem.

In the paper, we use the exact minimax penalty function method for solving a nondifferentiable optimization problem with both inequality and equality constraints. The major purpose of this work is to relate an optimal solution of the general nondifferentiable extremum problem involving locally Lipschitz functions to a minimizer in its associated penalized optimization problem with the exact minimax penalty function and, in the context of this analysis, to give a threshold of the penalty parameter such that, for any value of the penalty parameter above this value, this equivalence holds. More specifically, Theorem 4.1 shows that a Karush–Kuhn–Tucker point in the considered nondifferentiable constrained optimization problem is also a minimizer in its associated penalized optimization problem with the exact minimax penalty function. This result is true, under convex assumptions, for all penalty parameters c above the threshold value. We obtain this threshold for the controlling parameter of the penalty function and it is expressed in the function of the Karush–Kuhn–Tucker multipliers. The immediate corollary of this result (Corollary 4.1) is that an optimal solution in the given constrained extremum problem is a minimizer in its penalized optimization problem with the exact minimax penalty function. In Theorem 4.2, we present the converse result. We prove that a minimizer in the penalized optimization problem with the exact minimax penalty function is also an optimal solution in the considered extremum problem. The threshold of the penalty parameter is equal to the sum of absolute values of the Lagrange multipliers satisfying the Karush–Kuhn–Tucker necessary optimality conditions. Thus, we establish that, for all penalty parameters exceeding the threshold, the set of optimal solutions in the given nondifferentiable optimization problem is equal to the set of minimizers in its associated penalized optimization problem with the exact minimax penalty function. This result is established under assumptions that the functions constituting the considered nondifferentiable extremum problem with both inequality and equality constraints, are convex, with the exception of those equality constraints for which the associated Lagrange multipliers are negative—these functions are assumed to be concave. The results established in the paper are illustrated by suitable examples of optimization problems solved by using the exact minimax penalty function method.

2 Preliminary Definitions and Problem Formulation

It is well known that a function $f: X \rightarrow \mathbb{R}$ defined on a convex set $X \subset \mathbb{R}^{n}$ is said to be convex provided that, for all z,x∈X and λ∈[0,1], one has

$$f\bigl(\lambda z + (1 - \lambda )x\bigr) \le \lambda f(z) + (1 - \lambda )f(x). $$

Definition 2.1

The subdifferential of a convex function $f:\mathbb{R}^{n} \to \mathbb{R}$ at $x \in \mathbb{R}^{n}$ is defined as follows:

$$\partial f(x): = \bigl\{ \xi \in \mathbb{R}^{n}:f(z) - f(x) \ge \xi ^{T}(z - x)\ \forall z \in \mathbb{R}^{n}\bigr\}. $$

Definition 2.2

The superdifferential of a concave function $f:\mathbb{R}^{n} \to \mathbb{R}$ at $x \in \mathbb{R}^{n}$ is defined as follows:

$$\partial f(x): = \bigl\{ \xi \in \mathbb{R}^{n}:f(z) - f(x) \le \xi ^{T}(z - x)\ \forall z \in \mathbb{R}^{n}\bigr\}. $$

Remark 2.1

In Definition 2.2, we use, for a concave function, the terminology superdifferential in place of subdifferential. The terminology superdifferential was proposed by Rockafellar in [7] as being more appropriate than the terminology subdifferential. However, he uses the term subdifferential to mean both subdifferential and superdifferential.

Remark 2.2

As follows from the definition of a convex function $f:\mathbb{R}^{n} \to \mathbb{R}$ at x, the following inequality:

$$ f(z) - f(x) \ge \xi ^{T}(z - x),\quad \forall \xi \in \partial f(x) $$

(1)

holds for all $z \in \mathbb{R}^{n}$, where ∂f(x) denotes the subdifferential of f at x. Analogously, for a concave function $f:\mathbb{R}^{n} \to \mathbb{R}$ at x, the following inequality:

$$ f(z) - f(x) \le \xi ^{T}(z - x),\quad \forall \xi \in \partial f(x) $$

(2)

holds for all $z \in \mathbb{R}^{n}$.

Before we prove the main result for the considered optimization problem (P), we need the following useful lemma, the simple proof of which is omitted in this work.

Lemma 2.1

Let φ _k , k=1,…,p, be real-valued-functions defined on $X \subset \mathbb{R}^{n}$. For each x∈X, one has

$$\max_{1 \le k \le p}\varphi _{k}(x) = \max _{\alpha \in \varOmega} \sum_{k = 1}^{p} \alpha _{k} \varphi _{k}(x), $$

where $\varOmega: = \{ \alpha = (\alpha _{1},\ldots,\alpha _{p}) \in \mathbb{R}_{ +} ^{p}:\sum_{k = 1}^{p} \alpha _{k} = 1\}$.

The extremum problem considered in the paper is the general nonlinear optimization problem with both inequality and equality constraints:

$$(P)\quad \min f(x)\quad \mbox{s.t.}\ x \in D = \bigl\{ x \in X:g_{i}(x) \le 0,\ i \in I,\ h_{j}(x) = 0,\ j \in J\bigr\}, $$

where I={1,…,m}, J={1,…,s}, $f:X \to \mathbb{R}$ and $g_{i}:X \to \mathbb{R},i \in I$, $h_{j}:X \to \mathbb{R},j \in J$ are locally Lipschitz functions on a nonempty set $X \subset \mathbb{R}^{n}$ and D is the set of all feasible solutions in problem (P).

For the purpose of simplifying our presentation, we will introduce some notations, which will be used frequently throughout this paper.

We will write $g: = (g_{1},\ldots,g_{m}): X \rightarrow \mathbb{R}^{m}$ and $h: = (h_{1},\ldots, h_{s}): X \rightarrow \mathbb{R}^{s}$ for convenience.

Further, we denote the set of inequality constraint indices that are active at point x∈D by

$$I(\bar{x}): = \bigl\{ i \in I:g_{i}(\bar{x}) = 0\bigr\}. $$

It is well known (see, for example, [7–9]) that the following Karush–Kuhn–Tucker conditions are necessary for optimality of a feasible solution $\bar{x}$ in the considered nonlinear extremum problem.

Theorem 2.1

Let $\bar{x}$ be an optimal solution in problem (P) and a suitable constraint qualification be satisfied at $\bar{x}$. Then there exist the Lagrange multipliers $\bar{\lambda} \in \mathbb{R}^{m}$ and $\bar{\mu} \in \mathbb{R}^{s}$ such that

(3)

(4)

(5)

In the paper, we will assume that a suitable constraint qualification is satisfied at any optimal solution in the considered optimization problem (P).

Definition 2.3

The point $\bar{x} \in D$ is said to be a Karush–Kuhn–Tucker point in problem (P) iff there exist the Lagrange multipliers $\bar{\lambda} \in \mathbb{R}^{m}$ and $\bar{\mu} \in \mathbb{R}^{s}$ such that the Karush–Kuhn–Tucker necessary optimality conditions (3)–(5) are satisfied at this point with these Lagrange multipliers.

3 The Exact Minimax Penalty Function Method

In 1978, Charalambous [5] introduced a class of nondifferentiable exact penalty functions defined as follows:

$$P_{p}(x,\alpha,\beta,c): = f(x) + c \Biggl( \sum _{i = 1}^{m} \bigl[\alpha _{i}g_{i}^{ +} (x)\bigr]^{p} + \sum_{j = 1}^{s} \bigl[\beta _{j}\bigl \vert h_{j}(x) \bigr \vert \bigr]^{p} \Biggr)^{1/p}, $$

where c is a penalty parameter, p≥1, α _i >0, i=1,…,m,β _j >0, j=1,…,s. For a given inequality constraint g _i (x)≤0, the function $g_{i}^{ +} (x)$, defined by

$$ g_{i}^{ +} (x): = \left \{ \begin{array}{l@{\quad }l} 0, & \mbox{if}\ g_{i}(x) \le 0, \\ g_{i}(x), & \mbox{if}\ g_{i}(x) > 0, \end{array} \right . $$

(6)

is equal to zero for all x that satisfy the constraint, and it has a positive value whenever this constraint is violated. Moreover, large violations in the constraint g _i (x)≤0 result in large values for $g_{i}^{ +} (x)$. Thus, the function $g_{i}^{ +} (x)$ has the penalty features relative to the single constraint g _i (x)≤0.

For p=1 and considering all parameters α _i , i=1,…,m,β _j , j=1,…,s equal to 1, we get the most known nondifferentiable exact penalty function, called the exact l ₁ penalty function (also the absolute value penalty function). The exact l ₁ penalty function method has been introduced by Pietrzykowski [10]. Most of the literature on nondifferentiable exact penalty function methods for optimization problems is devoted to the study of conditions ensuring that a (local) optimum of the given constrained optimization problem is also an unconstrained (local) minimizer of the exact penalty function. In the literature, there can be found a lot of research which has been developed on the exactness of the exact l ₁ penalty function method (see, for example, [11–22], and others).

For p=∞, we obtain the so-called exact minimax penalty function. It is given by

$$ P_{\infty} (x,c): = f(x) + c\max_{\substack{1 \le i \le m\\1 \le j \le s}}\bigl\{ g_{i}^{ +} (x),\bigl \vert h_{j}(x) \bigr \vert \bigr\}. $$

(7)

Now, we use the exact minimax penalty function method for solving the considered extremum problem (P). In this method, for the given constrained extremum problem (P), we construct the following penalized optimization problem:

$$ \bigl(P_{\infty} (c)\bigr)\quad P_{\infty} (x,c): = f(x) + c\max _{\substack{1 \le i \le m\\1 \le j \le s}}\bigl\{ g_{i}^{ +} (x),\bigl \vert h_{j}(x) \bigr \vert \bigr\} \to \min. $$

(8)

We call the unconstrained optimization problem defined above the minimax penalized optimization problem or the penalized optimization problem with the exact minimax penalty function.

The idea of the exact minimax penalty function method is to solve the given nonlinear constrained optimization problem (P) by means of a single unconstrained minimization problem (P _∞(c)). Roughly speaking, the exact minimax penalty function for problem (P) is a function P _∞(x,c) given by (5), where c>0 is the penalty parameter, with the property that there exists a lower bound $\bar{c} \geq 0$ such that, for $c > \bar{c}$, any optimal solution of (P) is also a minimizer in the associated penalized optimization problem (P _∞(c)) with the exact minimax penalty function.

4 Main Results

In this section, we completely characterize optimal solutions of the given constrained optimization problem (P) in terms of minimizers of the exact minimax penalty function for a penalty parameter c exceeding some suitable threshold. Thus, we establish the equivalence between an optimal solution in the given constrained extremum problem (P) and a minimizer in its associated penalized optimization problem with the exact minimax penalty function.

First, however, we show that a Karush–Kuhn–Tucker point in the given constrained optimization problem (P) yields a minimizer of the exact minimax penalty function in its associated penalized optimization problem (P _∞(c)) for a penalty parameter c exceeding some suitable threshold, which is expressed in the function of the Lagrange multipliers.

Theorem 4.1

Let $\bar{x}$ be the Karush–Kuhn–Tucker point and the Karush–Kuhn–Tucker necessary optimality conditions (3)–(5) be satisfied at $\bar{x}$ with the Lagrange multipliers $\bar{\lambda} \in \mathbb{R}^{m}$ and $\bar{\mu} \in \mathbb{R}^{s}$. Let $J^{ +} (\bar{x}): = \{ j \in J:\bar{\mu} _{j} > 0\}$ and $J^{ -} (\bar{x}): = \{ j \in J:\bar{\mu} _{j} < 0\}$. Furthermore, assume that the objective function f and the constraint functions $g_{i},i \in I(\bar{x})$, $h_{j},j \in J^{ +} (\bar{x})$ are convex on X, whereas the constraint functions $h_{j},j \in J^{ -} (\bar{x})$ are concave on X. If c is assumed to be sufficiently large (it is sufficient to set $c \ge \sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert$, where $\bar{\lambda} _{i}$, i=1,…,m, $\bar{\mu} _{j}$, j=1,…,s are the Lagrange multipliers associated with the constraints g _i and h _j , respectively), then $\bar{x}$ is also a minimizer in its associated penalized optimization problem (P _∞(c)) with the exact minimax penalty function.

Proof

We consider two cases:

• (i) $\sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert > 0$.

By assumption, the objective function f and the constraint functions $g_{i},i \in I(\bar{x})$, $h_{j},j \in J^{ +} (\bar{x})$ are convex on X and, moreover, the constraint functions $h_{j},j \in J^{ -} (\bar{x})$ are concave on X. Then, by (1) and (2), the inequalities

(9)

(10)

(11)

(12)

hold for all x∈X and any $\xi \in \partial f(\bar{x})$, $\zeta _{i} \in \partial g_{i}(\bar{x}),i \in I(\bar{x})$, $\varsigma _{j} \in \partial h_{j}(\bar{x}),j \in J^{ +} (\bar{x}) \cup J^{ -} (\bar{x})$, respectively. Since $\bar{\lambda} _{i} \ge 0,i \in I,\bar{\mu} _{j} > 0$, $j \in J^{ +} (\bar{x})$, and $\bar{\mu} _{j} < 0$, $j \in J^{ -} (\bar{x})$, (10)–(12) imply, for all x∈X, respectively,

Taking into account the Lagrange multipliers equal to 0, we get

(13)

(14)

Adding both sides of (13) and (14), we obtain

(15)

(16)

Now, adding both sides of (9), (15) and (16), we get

By the Karush–Kuhn–Tucker necessary optimality condition (3), it follows that

$$f(x) - f(\bar{x}) + \sum_{i = 1}^{m} \bar{ \lambda} _{i}g_{i}(x) - \sum_{i = 1}^{m} \bar{\lambda} _{i}g_{i}(\bar{x}) + \sum _{j = 1}^{s} \bar{\mu} _{j}h_{j}(x) - \sum_{j = 1}^{s} \bar{\mu} _{j}h_{j}(\bar{x}) \ge 0. $$

Thus,

$$ f(x) + \sum_{i = 1}^{m} \bar{\lambda} _{i}g_{i}(x) + \sum_{j = 1}^{s} \bar{\mu} _{j}h_{j}(x) \ge f(\bar{x}) + \sum _{i = 1}^{m} \bar{\lambda} _{i}g_{i}( \bar{x}) + \sum_{j = 1}^{s} \bar{\mu} _{j}h_{j}(\bar{x}). $$

(17)

Now, using the Karush–Kuhn–Tucker necessary optimality condition (4) together with the feasibility of $\bar{x}$ in the given optimization problem (P), we find that the inequality

$$f(x) + \sum_{i = 1}^{m} \bar{\lambda} _{i}g_{i}(x) + \sum_{j = 1}^{s} \bar{\mu} _{j}h_{j}(x) \ge f(\bar{x}) $$

holds for all x∈X. Thus, by (6) and the definition of the absolute value, it follows that

$$f(x) + \sum_{i = 1}^{m} \bar{\lambda} _{i}g_{i}^{ +} (x) + \sum _{j = 1}^{s} \vert \bar{\mu} _{j} \vert \bigl \vert h_{j}(x) \bigr \vert \ge f(\bar{x}). $$

We divide both sides of the inequality above by $\sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert > 0$. Thus,

(18)

We denote

(19)

(20)

(21)

(22)

Note that by (19) and (20), it follows that

$$ \bar{\alpha} _{k} \ge 0,\quad k = 1,\ldots,m + s,\ \sum _{k = 1}^{m + s} \bar{\alpha} _{k} = 1. $$

(23)

Combining (18)–(22), we obtain, for all x∈X,

$$\frac{1}{\sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert } f(x) + \sum_{k = 1}^{m + s} \bar{\alpha} _{k}\varphi _{k}(x) \ge \frac{1}{\sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert } f( \bar{x}). $$

Thus, by (19)–(22), it follows that

$$\frac{1}{\sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert } f(x) + \max_{\alpha \in \varOmega} \sum _{k = 1}^{m + s} \alpha _{k}\varphi _{k}(x) \ge \frac{1}{\sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert } f(\bar{x}), $$

where $\varOmega: = \{ \alpha = (\alpha _{1},\ldots,\alpha _{m + s}) \in \mathbb{R}_{ +} ^{m + s}:\sum_{k = 1}^{m + s} \alpha _{k} = 1\}$. By Lemma 2.1, it follows that

$$\frac{1}{\sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert } f(x) + \max_{1 \le k \le m + s}\varphi _{k}(x) \ge \frac{1}{\sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert } f(\bar{x}). $$

Hence, (21) and (22) yield

(24)

By assumption, $\bar{x}$ is optimal in the given optimization problem (P). Therefore, it is feasible in problem (P). Hence, by (6), we have

$$ \max_{\substack{1 \le i \le m\\1 \le j \le s}}\bigl\{ g_{i}^{ +} (\bar{x}), \bigl \vert h_{j}(\bar{x}) \bigr \vert \bigr\} = 0. $$

(25)

Combining (24) and (25), we get

We multiply the inequality above by $\sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert > 0$. Thus,

By assumption, $c \ge \sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert$. Hence, by (25), the following inequality

$$f(x) + c\max_{\substack{1 \le i \le m\\1 \le j \le s}}\bigl\{ g_{i}^{ +} (x), \bigl \vert h_{j}(x) \bigr \vert \bigr\} \ge f(\bar{x}) + c\max _{\substack{1 \le i \le m\\1 \le j \le s}}\bigl\{ g_{i}^{ +} (\bar{x}),\bigl \vert h_{j}(\bar{x}) \bigr \vert \bigr\} $$

holds for all x∈X. By definition of the exact minimax penalty function P _∞(x,c), it follows that the inequality

$$ P_{\infty} (x,c) \ge P_{\infty} (\bar{x},c) $$

(26)

holds for all x∈X. This means that $\bar{x}$ is optimal in the penalized optimization problem (P _∞(c)) with the exact minimax penalty function.

• (ii) $\sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert = 0$.

Then, using (9) together with the Karush–Kuhn–Tucker necessary optimality condition (3), we see that the inequality

$$f(x) \ge f(\bar{x}) $$

holds for all x∈X. Since $\sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert = 0$, then the inequality above yields

By assumption, $c \ge \sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert = 0$. Therefore, combining the above inequality and (25), we see that the inequality

$$f(x) + c\max_{\substack{1 \le i \le m\\1 \le j \le s}}\bigl\{ g_{i}^{ +} (x), \bigl \vert h_{j}(x) \bigr \vert \bigr\} \ge f(\bar{x}) + c\max _{\substack{1 \le i \le m\\1 \le j \le s}}\bigl\{ g_{i}^{ +} (\bar{x}),\bigl \vert h_{j}(\bar{x}) \bigr \vert \bigr\} $$

holds for all x∈X. By definition of the exact minimax penalty function P _∞(x,c), it follows that the following inequality

$$ P_{\infty} (x,c) \ge P_{\infty} (\bar{x},c) $$

(27)

holding for all x∈X. This means that $\bar{x}$ is optimal in the penalized optimization problem (P _∞(c)) with the exact minimax penalty function. Hence, by (26) and (27), it follows that, for all $c \ge \sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert$, the Karush–Kuhn–Tucker point $\bar{x}$ in the given constrained extremum problem is a minimizer in its associated penalized optimization problem (P _∞(c)) with the exact minimax penalty function. This completes the proof. □

Corollary 4.1

Let $\bar{x}$ be an optimal point in the constrained optimization problem (P). Furthermore, assume that the objective function f and the constraint functions $g_{i},i \in I(\bar{x})$, $h_{j},j \in J^{ +} (\bar{x})$ are convex on X, the constraint functions $h_{j},j \in J^{ -} (\bar{x})$ are concave on X. If the penalty parameter c is assumed to be sufficiently large (it is sufficient to set $c \ge \sum_{i = 1}^{m} \bar{\lambda} _{i} + \sum_{j = 1}^{s} \vert \bar{\mu} _{j} \vert$, where $\bar{\lambda} _{i}$, i=1,…,m, $\bar{\mu} _{j}$, j=1,…,s are the Lagrange multipliers associated with the constraints g _i and h _j , respectively), then $\bar{x}$ is also a minimizer in the penalized optimization problem (P _∞(c)) with the exact minimax penalty function.

Proposition 4.1

Let $\bar{x}$ be a minimizer in the penalized optimization problem (P _∞(c)) with the exact minimax penalty function. Then, the inequality

$$f(x) \ge f(\bar{x}) $$

holds for all x∈D.

Proof

Since $\bar{x}$ is optimal in the penalized optimization problem (P _∞(c)) with the exact minimax penalty function, the inequality

$$P_{\infty} (x,c) \ge P_{\infty} (\bar{x},c) $$

holds for all x∈X. By definition of the exact minimax penalty function P _∞(x,c), it follows that the following inequality

$$f(x) + c\max_{\substack{1 \le i \le m\\1 \le j \le s}}\bigl\{ g_{i}^{ +} (x), \bigl \vert h_{j}(x) \bigr \vert \bigr\} \ge f(\bar{x}) + c\max _{\substack{1 \le i \le m\\1 \le j \le s}}\bigl\{ g_{i}^{ +} (\bar{x}),\bigl \vert h_{j}(\bar{x}) \bigr \vert \bigr\} $$

holds for all x∈X. Thus, by (6), for all x∈D,

$$f(x) \ge f(\bar{x}) + c\max_{\substack{1 \le i \le m\\1 \le j \le s}}\bigl\{ g_{i}^{ +} (\bar{x}),\bigl \vert h_{j}(\bar{x}) \bigr \vert \bigr\}. $$

Using (6) again, we find that the inequality

$$f(x) \ge f(\bar{x}) $$

holds for all x∈D. This completes the proof. □

Now, under the suitable convexity assumption imposed on the functions constituting the given extremum problem (P), we prove the converse result. Thus, we show that there exists a threshold $\bar{c}$ such that, for any penalty parameter c exceeding this value, $\bar{x}$, being a minimizer in the penalized optimization problem (P _∞(c)) with the exact minimax penalty function, is also optimal in the given extremum problem (P).

Theorem 4.2

Let $\bar{x}$ be a minimizer in the penalized optimization problem (P _∞(c)) with the exact minimax penalty function and let the penalty parameter c be sufficiently large (that is, $c > \sum_{i = 1}^{m} \tilde{\lambda} _{i} + \sum_{j = 1}^{s} \vert \tilde{\mu} _{j} \vert$, where $\tilde{x}$ is a Karush–Kuhn–Tucker point in problem (P) with the Lagrange multipliers $\tilde{\lambda} \in \mathbb{R}^{m}$ and $\tilde{\mu} \in \mathbb{R}^{s}$). Furthermore, assume that the objective function f and the constraint functions $g_{i},i \in I(\tilde{x})$, $h_{j},j \in J^{ +} (\tilde{x})$ are convex on X, the constraint functions $h_{j},j \in J^{ -} (\tilde{x})$ are concave on X. If the set D of all feasible solutions in problem (P) is compact, then $\bar{x}$ is also an optimal solution in the given extremum problem (P).

Proof

In order to prove that $\bar{x}$ is optimal in the given extremum problem (P), first we show that $\bar{x}$ is feasible in problem (P). By means of contradiction, suppose that $\bar{x}$ is not feasible in problem (P). Since f is a continuous function bounded below on the compact set D, by Weierstrass’ Theorem, f admits its minimum $\tilde{x}$ on D. Hence, the given extremum problem (P) has an optimal solution $\tilde{x}$. Therefore, the Karush–Kuhn–Tucker necessary optimality conditions are satisfied at $\tilde{x}$ with the Lagrange multipliers $\tilde{\lambda} \in \mathbb{R}^{m}$ and $\tilde{\mu} \in \mathbb{R}^{s}$. By assumption, the objective function f and the constraint functions $g_{i},i \in I(\tilde{x})$, $h_{j},j \in J^{ +} (\tilde{x})$ are convex on X and, moreover, the constraint functions $h_{j},j \in J^{ -} (\tilde{x})$ are concave on X. Hence, by (1) and (2), the inequalities

hold for all x∈X and any $\xi \in \partial f(\tilde{x})$, $\zeta _{i} \in \partial g_{i}(\tilde{x}),i \in I(\tilde{x})$, $\varsigma _{j} \in \partial h_{j}(\tilde{x}),j \in J^{ +} (\tilde{x}) \cup J^{ -} (\tilde{x})$, respectively. Therefore, they are also satisfied for $x = \bar{x}$. Thus,

(28)

(29)

(30)

(31)

Since the Karush–Kuhn–Tucker necessary optimality conditions (3)–(5) are satisfied at $\tilde{x}$ with the Lagrange multipliers $\tilde{\lambda} \in \mathbb{R} ^{m}$, $\tilde{\mu} \in \mathbb{R} ^{s}$ and, moreover, $\tilde{\lambda} _{i} \ge 0,i \in I$, $\tilde{\mu} _{j} > 0,j \in J^{ +} (\tilde{x})$, $\tilde{\mu} _{j} < 0,j \in J^{ -} (\tilde{x})$, then, taking into account the Lagrange multipliers equal to 0, we get

(32)

(33)

Adding both sides of (32) and (33), we obtain, respectively,

(34)

(35)

Now, adding both sides of (28), (34) and (35), we get

By the Karush–Kuhn–Tucker necessary optimality condition (3), it follows that

$$f(\bar{x}) - f(\tilde{x}) + \sum_{i = 1}^{m} \tilde{\lambda} _{i}g_{i}(\bar{x}) - \sum _{i = 1}^{m} \tilde{\lambda} _{i}g_{i}( \tilde{x}) + \sum_{j = 1}^{s} \tilde{\mu} _{j}h_{j}(\bar{x}) - \sum_{j = 1}^{s} \tilde{\mu} _{j}h_{j}(\tilde{x}) \ge 0. $$

Thus,

$$f(\bar{x}) + \sum_{i = 1}^{m} \tilde{\lambda} _{i}g_{i}(\bar{x}) + \sum_{j = 1}^{s} \tilde{\mu} _{j}h_{j}(\bar{x}) \ge f(\tilde{x}) + \sum _{i = 1}^{m} \tilde{\lambda} _{i}g_{i}(\tilde{x}) + \sum_{j = 1}^{s} \tilde{\mu} _{j}h_{j}(\tilde{x}). $$

From the feasibility of $\tilde{x}$ in problem (P) and by the Karush–Kuhn–Tucker necessary optimality condition (2), it follows that

$$f(\bar{x}) + \sum_{i = 1}^{m} \tilde{\lambda} _{i}g_{i}(\bar{x}) + \sum_{j = 1}^{s} \tilde{\mu} _{j}h_{j}(\bar{x}) \ge f(\tilde{x}). $$

Using (6) and the definition of the absolute value, we get

$$f(\bar{x}) + \sum_{i = 1}^{m} \tilde{\lambda} _{i}g_{i}^{ +} (\bar{x}) + \sum _{j = 1}^{s} \vert \tilde{\mu} _{j} \vert \bigl \vert h_{j}(\bar{x}) \bigr \vert \ge f(\tilde{x}). $$

Now, we assume that $\sum_{i = 1}^{m} \tilde{\lambda} _{i} + \sum_{j = 1}^{s} \vert \tilde{\mu} _{j} \vert > 0$. We divide the inequality above by $\sum_{i = 1}^{m} \tilde{\lambda} _{i} + \sum_{j = 1}^{s} \vert \tilde{\mu} _{j} \vert > 0$. Thus,

(36)

We denote

(37)

(38)

(39)

(40)

Note that, by (37) and (38), it follows that

$$ \tilde{\alpha} _{k} \ge 0,\quad k = 1,\ldots,m + s,\ \sum _{k = 1}^{m + s} \tilde{\alpha} _{k} = 1. $$

(41)

Combining (36)–(40), we obtain

$$\frac{1}{\sum_{i = 1}^{m} \tilde{\lambda} _{i} + \sum_{j = 1}^{s} \vert \tilde{\mu} _{j} \vert } f(\bar{x}) + \sum_{k = 1}^{m + s} \tilde{\alpha} _{k}\varphi _{k}(\bar{x}) \ge \frac{1}{\sum_{i = 1}^{m} \tilde{\lambda} _{i} + \sum_{j = 1}^{s} \vert \tilde{\mu} _{j} \vert } f(\tilde{x}). $$

Hence, (37)–(40) imply

$$\frac{1}{\sum_{i = 1}^{m} \tilde{\lambda} _{i} + \sum_{j = 1}^{s} \vert \tilde{\mu} _{j} \vert } f(\bar{x}) + \max_{\alpha \in \varOmega} \sum _{k = 1}^{m + s} \alpha _{k}\varphi _{k}(\bar{x}) \ge \frac{1}{\sum_{i = 1}^{m} \tilde{\lambda} _{i} + \sum_{j = 1}^{s} \vert \tilde{\mu} _{j} \vert } f(\tilde{x}), $$

where $\varOmega: = \{ \alpha = (\alpha _{1},\ldots,\alpha _{m + s}) \in \mathbb{R}_{ +} ^{m + s}:\sum_{k = 1}^{m + s} \alpha _{k} = 1\}$. By Lemma 2.1, it follows that

$$\frac{1}{\sum_{i = 1}^{m} \tilde{\lambda} _{i} + \sum_{j = 1}^{s} \vert \tilde{\mu} _{j} \vert } f(\bar{x}) + \max_{1 \le k \le m + s}\varphi _{k}(\bar{x}) \ge \frac{1}{\sum_{i = 1}^{m} \tilde{\lambda} _{i} + \sum_{j = 1}^{s} \vert \tilde{\mu} _{j} \vert } f(\tilde{x}). $$

Thus, (39) and (40) yield

$$\frac{1}{\sum_{i = 1}^{m} \tilde{\lambda} _{i} + \sum_{j = 1}^{s} \vert \tilde{\mu} _{j} \vert } f(\bar{x}) \,{+}\, \max_{\substack{1 \le i \le m\\1 \le j \le s}}\bigl\{ g_{i}^{ +} (\bar{x}),\bigl \vert h_{j}(\bar{x}) \bigr \vert \bigr\} \,{\ge}\, \frac{1}{\sum_{i = 1}^{m} \tilde{\lambda} _{i} + \sum_{j = 1}^{s} \vert \tilde{\mu} _{j} \vert } f(\tilde{x}). $$

Using (6) together with the feasibility of $\tilde{x}$ in the given optimization problem (P), we get

We multiply the inequality above by $\sum_{i = 1}^{m} \tilde{\lambda} _{i} + \sum_{j = 1}^{s} \vert \tilde{\mu} _{j} \vert > 0$. Thus,

(42)

By assumption, $\bar{x} \notin D$. Since $\tilde{x} \in D$, by (6) we have

(43)

(44)

By assumption, $c > \sum_{i = 1}^{m} \tilde{\lambda} _{i} + \sum_{j = 1}^{s} \vert \tilde{\mu} _{j} \vert$. Combining (42), (43), and (44), we get

$$ f(\bar{x}) + c\max_{\substack{1 \le i \le m\\1 \le j \le s}}\bigl\{ g_{i}^{ +} (\bar{x}),\bigl \vert h_{j}(\bar{x}) \bigr \vert \bigr\} > f( \tilde{x}) + c\max_{\substack{1 \le i \le m\\1 \le j \le s}}\bigl\{ g_{i}^{ +} ( \tilde{x}),\bigl \vert h_{j}(\tilde{x}) \bigr \vert \bigr\}. $$

(45)

Hence, by the definition of the exact minimax penalty function P _∞(x,c), it follows that the inequality

$$ P_{\infty} (\bar{x},c) > P_{\infty} (\tilde{x},c) $$

(46)

holds, contradicting the optimality of $\bar{x}$ in the penalized optimization problem (P _∞(c)) with the exact minimax penalty function.

In the case when $\sum_{i = 1}^{m} \tilde{\lambda} _{i} + \sum_{j = 1}^{s} \vert \tilde{\mu} _{j} \vert = 0$, the inequality (42) follows from the Karush–Kuhn–Tucker necessary optimality condition (3) and the inequality (28). Hence, by assumption $c > \sum_{i = 1}^{m} \tilde{\lambda} _{i} + \sum_{j = 1}^{s} \vert \tilde{\mu} _{j} \vert$, we find that the inequality (46) is satisfied also in this case, contradicting the optimality of $\tilde{x}$ in the penalized optimization problem (P _∞(c)) with the exact minimax penalty function.

Hence, we have established that $\bar{x}$ is a feasible solution in the given constrained extremum problem (P).

Thus, the optimality of $\bar{x}$ in the given optimization problem (P) follows directly from Proposition 4.1. This completes the proof. □

From Corollary 4.1 and Theorem 4.2, it follows the following result:

Corollary 4.2

Let the hypotheses of Corollary 4.1 and Theorem 4.2 be satisfied. Then the set of optimal solutions in the given constrained extremum problem (P) and the set of minimizers in its associated minimax penalized optimization problem (P _∞(c)) coincide.

Now, we illustrate the results established in the paper by the help of an example of a nonlinear constrained optimization problem with convex functions. In order to solve it, we use the exact minimax penalty function method.

Example 4.1

Consider the following optimization problem:

Note that $D = \{ (x_{1},x_{2}) \in \mathbb{R}^{2}:1 \le x_{1} \le 2 \wedge (x_{2} = 0 \vee x_{2} = 1)\}$ and $\bar{x} = (1,0)$ is an optimal solution in the considered constrained optimization problem (P1). Further, it is not difficult to show that both the objective function f and the constraint functions g ₁ and h ₁ are convex on $\mathbb{R}^{2}$. In order to solve the considered optimization problem, we use the exact minimax penalty function method. Thus, the following unconstrained optimization problem is constructed in this method:

Note that $\bar{x} = (1,0)$ is feasible in the considered optimization problem (P1) and the Karush–Kuhn–Tucker necessary optimality conditions (3)–(5) are satisfied at this point with the Lagrange multipliers $\bar{\lambda} _{1} = \xi _{1}$, $\bar{\mu} _{1} = \xi _{2}$, where $\xi = (\xi _{1},\xi _{2}) \in \partial f(\bar{x})$. Then, by Theorem 4.1, for any penalty parameter c satisfying $c \ge \bar{\lambda} _{1} + \vert \bar{\mu} _{1} \vert = 2$, the point $\bar{x} = (1,0)$ is also a minimizer in the penalized optimization problem (P1_∞(c)) with the exact minimax penalty function given above.

Conversely, since all hypotheses of Theorem 4.2 are also fulfilled, then $\bar{x} = (1,0)$, being a minimizer in problem (P1_∞(c)), where c>2, is also optimal in the constrained optimization problem (P1).

In the next example, we consider an optimization problem in which not all functions are convex. It turns out that, for such optimization problems, the equivalence might not hold between the set of optimal solutions in the given extremum problem and the set of minimizers in its associated penalized optimization problem constructed in the exact minimax penalty function method.

Example 4.2

We consider the following optimization problem:

Note that $D = \{ x \in \mathbb{R}:x \ge - 1\}$ and x=−1 is an optimal solution in the considered optimization problem (P2). Further, it is not difficult to see that the objective function f and the constraint function g ₂ are not convex on R. However, we use the exact minimax penalty function method for solving the given constrained extremum problem (P2). Then, we construct the following unconstrained optimization problem:

It is not difficult to show that the minimax penalized optimization problem (P2_∞(c)) does not have a minimizer at x=−1 for any penalty parameter c>0. This follows from the fact that the downward order of growth of f exceeds the upward of growth of g at x when moving from x towards smaller values. Indeed, note that inf _x∈R P2_∞(x,c)→−∞ when x→−∞ for any c>0. In this case, there is no equivalence between the sets of optimal solutions in the given extremum problem (P2) and the sets of minimizers in its associated penalized optimization problem (P2_∞(c)) with the exact minimax penalty function for any value of the penalty parameter c.

5 Conclusions

In the paper, the exact minimax penalty function method is used for solving nondifferentiable optimization problems involving both inequality and equality constraints. In this method, for the given nondifferentiable constrained extremum problem (P), its associated penalized optimization problem (P _∞(c)) with the exact minimax penalty function is constructed. A lower bound on the penalty parameter c has been given such that, for every penalty parameter c exceeding this threshold, the minimizer in the minimax penalized optimization problem (P _∞(c)) coincides with the optimum in the given constrained optimization problem (P).

Some interesting topics for further research remain. It would be of interest to investigate whether this result is true also for a larger class of constrained optimization problems, that is, for a class of nonconvex nondifferentiable extremum problems. Thus, further research can focus on the usefulness of the minimax penalty function method in solving various classes of nonconvex optimization problems. Moreover, it seems that the techniques employed in this paper can be used in proving the relationship between saddle points of the Lagrange function defined for the constrained extremum problem and minimizers in its associated penalized optimization problem with the exact minimax penalty function. We shall investigate these questions in subsequent papers.