Equivalent properties of global weak sharp minima with applications

Abstract

In this paper, we study the concept of weak sharp minima using two different approaches. One is transforming weak sharp minima to an optimization problem; another is using conjugate functions. This enable us to obtain some new characterizations for weak sharp minima.

Mathematics Subject Classification (2000): 90C30; 90C26.

1 Introduction

The notion of weak sharp minima plays an important role in the analysis of the perturbation behavior of certain classes of optimization problems as well as in the convergence analysis of algorithms. Of particular note in this fields is the paper by Burke and Ferris [1], which gave an extensive exposition of the notation and its impacted on convex programming and convergence analysis. Since then, this notion was extensively studied by many authors, for example, necessary or sufficient conditions of weak sharp minima for nonconvex programming [2, 3], and necessary and sufficient conditions of local weak sharp minima for sup-type (or lower-C¹) functions [4, 5]. Recent development of weak sharp minima and its related to other issues can be found in [5–8].

A closed set $\bar{S}\subseteq {ℝ}^n$ is said to be a set of weak sharp minima for a function f : ℝⁿ → ℝ relative to a closed set S ⊆ ℝⁿ with $\bar{S}\subseteq S$, if there is an α > 0 such that

$$f\left(x\right)\ge f\left(y\right)+\alpha \mathsf{\text{dist}}\left(x,\bar{S}\right),\forall x\in S\mathsf{\text{and}}y\in \bar{S},$$

(1.1)

where $\mathsf{\text{dist}}\left(x,\bar{S}\right)$ denotes the Euclidean distance from x to $\bar{S}$, i.e.,

$$\mathsf{\text{dist}}\left(x,\bar{S}\right)=\mathsf{\text{inf}}\left\{\parallel x-y\parallel \mid y\in \bar{S}\right\}.$$

An ordinary way to deal with weak sharp minima is using the tools of variational analysis, such as subdifferentials and normal cones or various generalized derivatives and tangent cones. However, we study in this paper the concept of weak sharp minima from a new perspective. The nonconvex and convex cases are treated separately. Specifically, for the nonconvex case, we establish the close relationship between weak sharp minima and the generalized semi-infinite max-min programming (see (2.2) below). To the best of our knowledge, these results do not appear explicitly in the literature. For the convex case, we use conjugate functions to characterize weak sharp minima. This gives a unified way to deal with different problems, such as convex inequality system and affine convex inclusion. Finally, applications of weak sharp minima to algorithm analysis for solving variational inequality problem are given.

We first recall some preliminary notions and results, which will be used throughout this paper. Given a set A ⊂ ℝⁿ, we denote its closure and convex hull as clA and convA, respectively. Denote its polar cone as

$$A^0=\left\{x\in {ℝ}^n\mid ⟨x,y⟩\le 0,\forall y\in A\right\}.$$

The indicator function and support function of A are defined by

$$\delta \left(x\mid A\right)=\left\{\begin{array}{cc}\hfill 0,\hfill & \hfill \mathsf{\text{if}}x\in A,\hfill \\ \hfill +\infty ,\hfill & \hfill \mathsf{\text{otherwise}}\hfill \end{array}\right.$$

and

$$\sigma \left(w\mid A\right)=sup\left\{⟨w,x⟩\mid x\in A\right\}.$$

The conjugate function of a function f : ℝⁿ → ℝ is

$$f^{*}\left(x^{*}\right)=\underset{x\in {ℝ}^n}{sup}\left\{⟨x^{*},x⟩-f\left(x\right)\right\},$$

and the biconjugate function is defined as f**(x) = (f*)*(x), i.e., the conjugate of f*. The inf-convolution operation between f₁ and f₂ is

$$\left(f_1\square f_2\right)\left(x\right)=inf\left\{f_1\left(x_1\right)+f_2\left(x_2\right)\mid x=x_1+x_2\right\}.$$

The rest of the paper is organized as follows. The relationship between weak sharp minima and generalized semi-infinite programming is established in Section 2. In Section 3, we characterize the weak sharpness by using conjugate duality.

2 Nonconvex case

In this section, we show that the concept of weak sharp minima can be translated equivalently to a generalized semi-infinite max-min programming. Given α > 0, define a set-valued mapping as

$$S_{\alpha }\left(x\right)=\left\{y\in S\mid f\left(y\right)+\alpha \parallel x-y\parallel \le f\left(x\right)\right\}.$$

Clearly, this set is nonempty, since x ∈ S _α (x) for all α > 0. Let $\bar{f}$ stand for the optimal value of f over S. Some equivalent expressions of weak sharpness in terms of S _α are given below.

Theorem 2.1. Let f be a lower semi-continuous function. The following statements are equivalent:

(a). $\bar{S}$ is a set of weak sharp minima;

(b). There exists some α > 0 such that $S_{\alpha }\left(x\right)\cap \bar{S}\ne \varnothing$ for all x ∈ S;

(c). There exists some α > 0 such that, for any x ∈ S, one has

$$min\left\{f\left(y\right)\mid y\in S_{\alpha }\left(x\right)\right\}=\bar{f}.$$

Proof. (a) ⇒ (b). If $\bar{S}$ is weak sharpness, it is easy to see that there exists α > 0 such that

$$f\left(x\right)\ge \bar{f}+\alpha \mathsf{\text{dist}}\left(x,\bar{S}\right),\forall x\in S.$$

If $x\in \bar{S}$, then x ∈ S _α (x) for any α > 0 by definition. Thus, the conclusion is true. If $x\in S\backslash \bar{S}$, let $\bar{x}\in P_{\bar{S}}\left(x\right)$, the projection of x onto $\bar{S}$. Then, the above inequality implies that

$$f\left(x\right)\ge f\left(\bar{x}\right)+\alpha \parallel x-\bar{x}\parallel ,$$

i.e, $\bar{x}\in S_{\alpha }\left(x\right)$. Thus, $S_{\alpha }\left(x\right)\cap \bar{S}\ne \varnothing$.

• (b) ⇒ (c). It is elementary.

• (c) ⇒ (a). Choose x ∈ S. The definition of infimum guarantees the existence of a sequence {y _n } ⊆ S _α (x) such that f(y _n ) approaches to $\bar{f}$. Since y _n ∈ S _α (x), then

  $$\bar{f}+\alpha \parallel x-y_n\parallel \le f\left(y_n\right)+\alpha \parallel x-y_n\parallel \le f\left(x\right),$$

  (2.1)

where the first step comes from the fact that y _n ∈ S and $\bar{f}$ is the optimal value. Thus, $\alpha \parallel x-y_n\parallel \le f\left(x\right)-\bar{f}$, which means the boundness of {y _n }. Passing to a subsequence if necessary, we can assume that {y _n } converges to a limit point $ȳ$. We claim that $ȳ\in S_{\alpha }\left(x\right)$, since S _α (x) is closed, due to the lower semi-continuity of f. Using this property again, we have

$$f\left(ȳ\right)\le \underset{n\to +\infty }{lim}f\left(y_n\right)=\bar{f}.$$

On the other hand, since $\bar{f}$ is the optimal value, then $f\left(ȳ\right)\ge \bar{f}$. Hence, $f\left(ȳ\right)=\bar{f}$, i.e., $ȳ\in \bar{S}$. Taking limits in (2.1) yields $\bar{f}+\alpha \parallel x-ȳ\parallel \le f\left(x\right)$. Therefore,

$$\begin{array}{ccc}\hfill \alpha \mathsf{\text{dist}}\left(x\mid \bar{S}\right)\hfill & \hfill \le \hfill & \hfill \alpha \parallel x-ȳ\parallel \hfill \\ \hfill \le \hfill & \hfill f\left(x\right)-\bar{f},\hfill \end{array}$$

where the first step is due to the fact that $ȳ\in \bar{S}$. Since x is an arbitrary element in S, then the above inequality means that $\bar{S}$ is weakly sharp.   □

The foregoing theorem shows that the concept of weak sharp minima can be converted into a class of optimization problems with the same optimal value. Based on this fact, we further derive the following result.

Theorem 2.2. Let f be a lower semi-continuous function. Then, the following statements are equivalent:

(a). $\bar{S}$ is a set of weak sharp minima;

(b). There exists α > 0 such that $\bar{f}$ is the optimal value of the following generalized semi-infinite max-min programming

$$\underset{x\in S}{max}\underset{y\in S_{\alpha }\left(x\right)}{min}f\left(y\right).$$

(2.2)

Proof. It is easy to see that the following estimate

$$\underset{x\in S}{max}\underset{y\in S_{\alpha }\left(x\right)}{min}f\left(y\right)=\bar{f}$$

coincides with

$$\underset{y\in S_{\alpha }\left(x\right)}{min}f\left(y\right)=\bar{f},\forall x\in S,$$

since $\bar{f}$ is the optimal value of f over S. Therefore, the desired result follows from Theorem 2.1.   □

To the best of our knowledge, the connection between weak sharp minima and the generalized semi-infinite programming is not stated explicitly in the literature. This result makes it possible to characterize weak sharpness by using the theory of generalized semi-infinite programming [9–11] and vice verse. In addition, the condition imposed in the foregoing theorem only needs the function to be lower semi-continuous, a rather weak condition in optimization. Hence, our result is applicable even for the case where the subgradient of f does not exist, while in [2–6], f is required, at least, to be subdifferentiable.

3 Convex case

We turn our attention in this section to the case where f and S are convex. In particular, we characterize the concept of weak sharp minima via conjugate function. This way enable us to deal with several different problems, such as convex inequality system and affine convex inclusion. Denote by $B$ the unit ball in ℝⁿ, i.e., $B=\left\{x\in {ℝ}^n\mid \parallel x\parallel \le 1\right\}$. The following simple result can be found in [12]. The proof is given here for completeness.

Lemma 3.1. Let f be a closed convex function and S be a closed convex set. Then, $\bar{S}$ is a set of weak sharp minima if and only if there exists some α > 0 such that

$$\left(f^{*}\square {\sigma }_S\right)\left(x\right)+\bar{f}\le {\sigma }_{\bar{S}}\left(x\right),\forall x\in \alpha B.$$

Proof. Using the indicator function, it is easy to see that (1.1) is equivalent to saying the existence of α > 0 such that

$$f\left(x\right)+{\delta }_S\left(x\right)\ge \bar{f}+\alpha \mathsf{\text{dist}}\left(x,\bar{S}\right),\forall x\in {ℝ}^n.$$

Note that the left function is closed convex, since f is proper closed convex and S is closed convex. Therefore, according to Legendre-Fenchel transform [[13], Theorem 11.1], the above formula can be rewritten equivalently as

$${\left(f+{\delta }_S\right)}^{*}\left(x\right)\le {\left(\bar{f}+\alpha \mathsf{\text{dist}}\left(\cdot \mid \bar{S}\right)\right)}^{*}\left(x\right),\forall x\in {ℝ}^n,$$

which, together with the conjugacy correspondence between support function and indicator function and the fact that $\mathsf{\text{dist}}\left(x,\bar{S}\right)=\left({\sigma }_B\square {\delta }_{\bar{S}}\right)\left(x\right)$ [[14], Section 5], implies

$$\left(f^{*}\square {\sigma }_S\right)\left(x\right)\le \alpha \left({\delta }_B+{\sigma }_{\bar{S}}\right)\left(\frac{x}{\alpha }\right)-\bar{f}.$$

Invoking the positive homogeneity of support function [[14], Theorem 13.2] yields the result as desired.   □

Other deep characterizations of weak sharp minima can be found in [15, 16]. Since the concept of weak sharp minima is closely related to error bounds, we shall use the above result to study the error bounds for convex inequality system and affine convex inclusion, respectively.

3.1 Special cases

3.1.1 Convex inequality system

We first consider a convex inequality system as follows

$$f_i\left(x\right)\le 0,\forall i\in I,$$

(3.1)

where f _i is a closed convex function and I is an arbitrary (possible infinite) index set. Let $f\left(x\right)=\underset{i\in I}{max}{f}_i\left(x\right)$. Then, the solution set of (3.1) is S = {x ∈ ℝⁿ|f(x) ≤ 0}. We say that (3.1) has a global error bound if there exists α > 0 such that

$$\mathsf{\text{dist}}\left(x,S\right)\le \alpha f{\left(x\right)}_{+},\forall x\in {ℝ}^n,$$

(3.2)

where f(x)₊ = max{f (x), 0}.

Theorem 3.2. The system (3.1) has a global error bound if and only if there exists α > 0 such that

$${\sigma }_S\left(x\right)\ge \underset{\lambda \in \left[0,1\right]}{inf}{\left(\lambda f\right)}^{*}\left(x\right),\forall x\in \alpha B.$$

where

$$f^{*}\left(x\right)=cl\left(\mathsf{\text{conv}}\left\{f_i^{*}\mid i\in I\right\}\right)\left(x\right).$$

Proof. Dividing by α in (3.2) and taking the conjugate duality on the both sides yields

$${\sigma }_S(x)\ge {(f{(\cdot )}_{+})}^{*}(x)=(max\{f(x),g(x{)\})}^{*}(x),\forall x\in \alpha \mathbb{B}\mathrm{,}$$

where we let g(x) = 0 for all x. According to [[14], Theorem 16.5], we known that

$$\begin{array}{ll}\hfill {\left(f{\left(\cdot \right)}_{+}\right)}^{*}\left(x\right)& =inf\left\{\lambda f^{*}\left(x_1\right)+\left(1-\lambda \right)g^{*}\left(x_2\right)|x=\lambda x_1+\left(1-\lambda \right)x_2,\lambda \in \left[0,1\right]\right\}\\ =\underset{\lambda \in \left[0,1\right]}{inf}\lambda f^{*}\left(x∕\lambda \right)\\ =\underset{\lambda \in \left[0,1\right]}{inf}{\left(\lambda f\right)}^{*}\left(x\right),\end{array}$$

where the second step follows from the fact g* = δ_{0}. The desired result follows from [[14], Theorem 16.5].   □

The foregoing result is applicable for the case where the algebra interior of the system (3.1) is empty.

3.1.2 Affine convex inclusion

Consider an affine convex inclusion as follows

$$Ax-b\in C,$$

(3.3)

where A is a real systemical matrix in ℝ^n×nand C ⊆ ℝⁿ is a nonempty, closed, and convex set. Denote by S the solution set. The system (3.3) is said to has a global error bound if there exists α > 0 such that

$$\alpha \mathsf{\text{dist}}\left(x,S\right)\le \mathsf{\text{dist}}\left(Ax-b\mid C\right),\forall x\in {ℝ}^n.$$

(3.4)

Theorem 3.3. Let A be an inverse matrix. Then, the affine convex inclusion has a global error bound if and only if there exists α > 0 with α ≤ 1/||A^-1|| such that

$${\sigma }_S\left(x\right)\ge {\sigma }_C\left(A^{-1}x\right)+⟨x,A^{-1}b⟩,\forall x\in \alpha B.$$

Proof. Let f(x) = dist(Ax - b|C). Taking the conjugate duality on both sides of (3.4) yields

$${\sigma }_S\left(x\right)\ge f^{*}\left(x\right),\forall x\in \alpha B.$$

On the other hand, since α ≤ 1/||A^-1||, it then follows that

$$\parallel A^{-1}x\parallel \le \parallel A^{-1}\parallel \parallel x\parallel \le 1,\forall x\in \alpha B.$$

(3.5)

Therefore, for $x\in \alpha B$, we have

$$\begin{array}{ll}\hfill f^{*}\left(x\right)& =\underset{x^{*}}{sup}\left\{⟨x^{*},x⟩-f\left(x^{*}\right)\right\},\mathsf{\text{by}}\mathsf{\text{letting}}y=A{x}^{*}-b\\ =\underset{y}{sup}\left\{⟨x,A^{-1}\left(y+b\right)⟩-\mathsf{\text{dist}}\left(y\mid C\right)\right\}\\ =\underset{y}{sup}\left\{⟨A^{-1}x,y⟩-\mathsf{\text{dist}}\left(y\mid C\right)\right\}+⟨x,A^{-1}b⟩\\ =\left({\delta }_B+{\sigma }_C\right)\left(A^{-1}x\right)+⟨x,A^{-1}b⟩\\ ={\sigma }_C\left(A^{-1}x\right)+⟨x,A^{-1}b⟩,\end{array}$$

where the last step comes from (3.5). This completes the proof.   □

When C is negative orthant, the concept of global error bounds for affine convex inclusion is also referred to as Hoffman bounds in honor of his seminal work [17]. Historically, this is the most intensively studied case. We do not attempt a review of the enormous literature on this case or even on the slightly more general polyhedral case. Rather, our focus is on the case where C is only assumed to be convex.

As mentioned in Introduction, the concept of weak sharp minima plays an important role in the convergence analysis of optimization algorithm. Hence, we investigate the impact of weak sharp minima for solving variational inequality problem (VIP), which is to find a vector x* ∈ X such that

$$⟨F\left(x^{*}\right),x-x^{*}⟩\ge 0,\forall x\in X,$$

where X is a nonempty closed convex set in ℝⁿ and F is a mapping from X into ℝⁿ. Denote by X* the solution set of (VIP). Due to the absence of objective function in (VIP), Marcotte and Zhu [18] adopted the following geometric characterization as the definition of weak sharpness, i.e, the solution set X^_ of VIP is said to be weakly sharp if

$$-F(x^{*})\in \text{int}{\displaystyle \underset{x\in X^{*}}{\cap }(T_X(x){\displaystyle \cap N_{X^{*}}(x))0^{}}},\forall x^{*}\in X^{*}.$$

(3.6)

Here, we further introduce two extended version, uniformly weak sharp minima and locally weakly sharp. More precisely, we say that X* is a uniformly weak sharp minima of VIP if there exists α > 0 such that

$$-F(x)+\alpha \mathbb{B}\subset {\displaystyle \underset{x\in X^{*}}{\cap }(T_X(x){\displaystyle \cap N_{X^{*}}(x))0^{}}},\forall x\in X^{*}.$$

(3.7)

We say that $\bar{x}\in X^{*}$ is locally weakly sharp of VIP if there exists δ > 0 such that

$$-F(\overline{x})\in \text{int}{\displaystyle \underset{x\in X^{*}\cap \mathbb{B}(\overline{x},\delta )}{\cap }(T_X(x){\displaystyle \cap N_{X^{*}}(x))0^{}}}.$$

(3.8)

Clearly, (3.8) is weaker than (3.6), because the latter corresponds to δ = ∞ and $\bar{x}$ must be taken over whole solution set X*.

Theorem 3.4. Let {x^k} ⊂ X be a iterative sequence generalized by some algorithm. If either

(i). X* is uniformly weakly sharp, and

$$\parallel F\left(x^k\right)-F\left(z^k\right)\parallel \to 0\mathsf{\text{as}}k\to \infty$$

(3.9)

where z^k ∈ P _{X *} (x^k); or

(ii). {x^k} converges to some $\bar{x}\in X^{*}$, $\bar{x}$ is locally weakly sharp, and F is continuous over X;

then x^k ∈ X* for all k sufficiently large if and only if

$$\underset{k\to \infty }{lim}{P}_{T_X\left(x^k\right)}\left(-F\left(x^k\right)\right)=0.$$

(3.10)

Proof. The necessity is trial, since x^k ∈ X* is equivalent to saying -F(x^k) ∈ N _X (x^k), which further implies that $P_{T_X\left(x^k\right)}\left(-F\left(x^k\right)\right)=0$.

We now show the sufficiency. First assume that (i) holds. Suppose, on the contrary, that there exists a subsequence ${\left\{x^k\right\}}_{\mathcal{K}}$ such that x^k ∉ X* for all $k\in \mathcal{K}$, where $\mathcal{K}$ is an infinite subset of {1, 2, ...}. For any $k\in \mathcal{K}$, there exists z^k ∈ X* (not necessarily unique) such that ||x^k - z^k|| = dist(x^k, X*), i.e., z^k ∈ P _X* (x^k). Note that $x^k-z^k\in {\widehat{N}}_{X^{*}}\left(z^k\right)$ by [[13], Example 6.16] and that ${\widehat{N}}_{X^{*}}\left(z^k\right)\subseteq N_{X^{*}}\left(z^k\right)$ by [[13], Proposition 6.5]. It then follows that x^k - z^k ∈ N _{X *} (z^k) ∩ T _X (z^k) and z^k - x^k ∈ T _X (x^k).

Invoking (3.7), i.e., there exists α > 0 such that

$$-F\left(z^k\right)+\alpha B\subset {\left(T_X\left(z^k\right)\cap N_{X^{*}}\left(z^k\right)\right)}^0,$$

(3.11)

which further implies

$$⟨-F\left(z^k\right)+\alpha \frac{x^k-z^k}{\parallel x^k-z^k\parallel },x^k-z^k⟩\le 0.$$

Therefore,

$$\begin{array}{ll}\hfill \alpha & \le ⟨F\left(z^k\right),\frac{x^k-z^k}{\parallel x^k-z^k\parallel }⟩\\ =⟨-F\left(x^k\right),\frac{z^k-x^k}{\parallel z^k-x^k\parallel }⟩+⟨F\left(x^k\right)-F\left(z^k\right),\frac{z^k-x^k}{\parallel z^k-x^k\parallel }⟩\\ \le max\left\{⟨-F\left(x^k\right),d⟩\mid d\in T_X\left(x^k\right),\parallel d\parallel \le 1\right\}+\parallel F\left(x^k\right)-F\left(z^k\right)\parallel \\ =\parallel P_{T_X\left(x^k\right)}\left(-F\left(x^k\right)\right)\parallel +\parallel F\left(x^k\right)-F\left(z^k\right)\parallel .\end{array}$$

Taking the limit as $k\in \mathcal{K}$ approaches ∞, it follows from (3.9) and (3.10) that α ≤ 0, which leads to a contradiction.

If the condition (ii) holds, we must have, as shown above, that z^k converges to $\bar{x}$ as well, since $\parallel z^k-\bar{x}\parallel \le \parallel z^k-x^k\parallel +\parallel x^k-\bar{x}\parallel \le 2\parallel x^k-\bar{x}\parallel$. Hence, as k is large enough, we must have $z^k\in B\left(\bar{x},\delta \right)$. Thus, (3.8) means the existence of α > 0 such that

$$-F\left(\bar{x}\right)+\alpha B\subset {\left(T_X\left(z^k\right)\cap N_{X^{*}}\left(z^k\right)\right)}^0.$$

Since F is continuous, then $\parallel F\left(x^k\right)-F\left(z^k\right)\parallel \le \parallel F\left(x^k\right)-F\left(\bar{x}\right)\parallel +\parallel F\left(z^k\right)-F\left(\bar{x}\right)\parallel \to 0$ as k → ∞. Hence, using the argument following (3.11) by replacing z^k by $\bar{x}$ (in the left of (3.11)) yields a contradiction. This completes the proof.   □

Finally, let us compare our result with that given in [18], where the finite termination property is established under the assumption that (i) F is pseudomonotone⁺, i.e., for any x, y ∈ X

$$⟨F\left(x\right),y-x⟩\ge 0\Rightarrow ⟨F\left(y\right),y-x⟩\ge 0,$$

and

$$⟨F\left(x\right),y-x⟩\ge 0\mathsf{\text{and}}⟨F\left(y\right),y-x⟩=0\Rightarrow F\left(y\right)=F\left(x\right),$$

• (ii) X* is weak sharp minima; (iii) dist(x^k|X*) converges to zero, and F is uniformly continuous over some open set containing x^k and X*. Indeed, according to [[18], Theorem 3.1], we know that F is a constant over X* when F is pseudomonotone⁺. Using this fact, the concept of uniformly weak sharp minima reduces to weak sharp minima. Meanwhile, it is easy to see that condition (iii) given in [18] implies (3.9). In addition, we further consider the case when x^k has a limit point under a weaker version of weak sharp minima.