Abstract
The goal of this paper is to discuss iteration complexity of two projection type methods for solving generalized complementarity problems and establish their worst-case sub-linear convergence rates measured by the iteration complexity in both the ergodic and the nonergodic sense.
Similar content being viewed by others
1 Background
The distinguishing feature of a complementarity problem is the set of complementarity conditions. Each of these conditions requires that the product of two or more nonnegative quantities should be zero. Complementarity conditions made their first appearance in the optimality conditions for continuous variable nonlinear programs involving inequality constraints, which were derived by Karush [14] in 1939. But the significance of complementarity conditions goes far beyond this. They appear prominently in the study of equilibrium problems and arise naturally in numerous applications from economics, engineering and the sciences.
The motivation for studying the linear complementarity problem (LCP) was because the KKT optimality conditions for linear and quadratic programs (QPs) constitute an LCP of the form (1.1). The complementarity problem is one of the basic problems in nonlinear analysis along with such problems as optimization, fixed point problems, equilibrium problems and variational inequalities. The nonlinear complementarity problem (NCP) was introduced by Cottle in his Ph.D. thesis in 1964 [5], and the closely related variational inequality problem (VIP) was introduced by Hartman and Stampacchia in 1966 [10], primarily with the goal of computing stationary points for nonlinear programs. While these problems were introduced soon after the LCP, most of the progress in developing algorithms for these problems did not begin until the late 1970s.
Recall that the conventional complementarity problem requires that, for a given single-valued mapping \(F:\mathbb{R}^{n}\longrightarrow \mathbb{R}^{n}\), one finds a point \(x\in \mathbb{R}^{n}\) such that
The bulk of the above publications concern problems of the form (1.1), it is especially true of the work related to numerical methods for searching solutions.
Meanwhile, many applications lead to set-valued mappings. As a result, set-valued complementarity problems must be solved; that is, a point \(x\in \mathbb{R}^{n}\) must be found such that
where \(G:\mathbb{R}^{n}\longrightarrow \prod (\mathbb{R}^{n})\) is a set-valued mapping, hereafter, the symbol \(\prod (S)\) stands for the collection of all subsets of the set S.
The simplest and most widely studied of the complementarity problems is the LCP, which has often been described as a fundamental problem for given \(M \in \mathbb{R}^{n\times n}\), \(q\in \mathbb{R}^{n}\), and \(w = (w_{j})\in \mathbb{R}^{n}\), \(z = (z_{j})\in \mathbb{R}^{n}\) satisfying
We denote this LCP by the symbol \((q,M)\). The LCP \((q,M)\) is said to be monotone if the matrix M is positive semi-definite (PSD). A slight generalization of the LCP is the mixed LCP (mLCP): given \(A\in \mathbb{R}^{n\times n}\), \(B\in \mathbb{R}^{m\times m}\), \(C \in \mathbb{R} ^{n\times m}\), \(D\in \mathbb{R}^{m\times n}\), \(a\in \mathbb{R}^{n}\), \(b \in \mathbb{R}^{m}\), find \(u \in \mathbb{R}^{n}\), \(v\in \mathbb{R}^{m}\) satisfying
Another generalization of the LCP is the horizontal LCP or hLCP: given \(N \in \mathbb{R}^{n\times n}\), \(M \in \mathbb{R}^{n\times n}\), \(q\in \mathbb{R}^{n}\), find \(w \in \mathbb{R}^{n}\), \(z \in \mathbb{R}^{n}\) satisfying
The hLCP (1.5) becomes the standard LCP if \(N =I\). Also, if N is nonsingular, then (1.5) is equivalent to the LCP \((N^{-1}q,N^{-1}M)\). The hLCP is said to be monotone if for any two pairs of points \((w^{1}, z^{1})\) and \((w^{2}, z^{2})\) satisfying \(Nw - Mz = q\) we have
Note that if \(N = I\), this is equivalent to the matrix M being PSD.
Now, we will present nonlinear generalizations of the LCP. The most important of these is the NCP: given a mapping \(F(z)=(F_{i}(z)): \mathbb{R}^{n} \longrightarrow \mathbb{R}^{n}\), and a \(z \in \mathbb{R}^{n}\) satisfying
If \(F(z)\) is the affine function \(q+ Mz\), then (1.6) becomes the LCP \((q,M)\).
A further generalizations of the NCP is the VIP which is defined thus: given a mapping \(F(x)=(F_{i}(x)):\mathbb{R}^{n}\longrightarrow \mathbb{R}^{n}\), and \(\emptyset \neq C\subset \mathbb{R}^{n}\), find a \(x^{\ast }\in C\) satisfying
denoted by \(VIP(C,F)\) where C is a nonempty closed convex subset of \(\mathbb{R}^{n}\). If C is polyhedral and F is affine, it can be verified that \(VIP(C,F)\) is an LCP. C is a rectangular region defined by
This is called the box constrained VIP (BCVIP), which is also commonly referred to as the (nonlinear) mixed complementarity problem (MCP).
Our main goal is to analyse the convergence rates for projection methods under mild assumptions and together with strict contraction property, we establish convergence rates in asymptotical sense under certain error bound condition. More specifically, we want to derive the worst-case sub-linear convergence rate measured by the iteration complexity. This kind of convergence rate analysis based on iteration complexity traces back to [1, 4, 12, 16, 18]. Furthermore, our convergence rate analysis does not need to assume the boundedness of the feasible set which is usually required by iteration complexity analysis of projection methods for generalized complementarity problems; see, [7,8,9, 13, 15].
We assume that \(x^{\ast }\) is a fixed but arbitrary point in the solution set \(C^{\ast }\) of generalized complementarity problems and \(x^{\ell }\) refers to specific vector usually denotes an iteration index. For any matrix M and vector y, we denote their transposes by \(M^{T}\) and \(y^{T}\), respectively. The Euclidean norm is denoted by \(\Vert \cdot \Vert \). Let C be a nonempty closed convex subset of \(\mathbb{R}^{n}, M\in \mathbb{R}^{n\times n}\) and \(q\in \mathbb{R} ^{n}\). We consider the generalized complementarity problems for finding a vector \(x^{\ast }\in C\) such that
We consider the case where the matrix M is positive semi-definite (but not necessary symmetric). Moreover, the solution set of (1.8), denoted by \(C^{\ast }\), is assumed to be nonempty.
Proposition 1.1
([2])
Let \(C^{\ast }\) be a nonempty closed convex subset of \(\mathbb{R}^{n}\) and let G be any \(n\times n\) symmetric positive definite matrix. Then \(x^{\ast }\) solves the problem (1.8) if and only if
where \(G\in \mathbb{R}^{n\times n}\) is a symmetric positive definite matrix, and \(P_{C}^{G}(\cdot )\) denotes the projection onto C with respect to the G-norm:
where \(\Vert x \Vert _{G}=\sqrt{x^{T}Gx}\) for any \(x\in \mathbb{R}^{n}\).
Remark 1.2
If \(G=I\), then the notation \(P_{C}(\cdot )\) becomes \(P_{C}^{I}(\cdot )\) or simply \(P_{C}(\cdot )\).
Moreover, for given \(x\in \mathbb{R}^{n}\), we denote
Hence, we have
We further use the notation
From (1.10), x is a solution of (1.8) if and only if \(x=\bar{x}\). Thus the residual of projection equation \(\Vert e(x^{ \ell }) \Vert ^{2}\) can be used to measure the accuracy of an iterate \(x^{\ell }\) to a solution of (1.8).
More specifically, we proposed the following iterative scheme:
where \(\zeta \in (0,1)\) is a relaxation factor and the step size \(\alpha ^{\ast }_{\ell }\) is determined by
Obviously, for the step size
where \(\Vert \cdot \Vert _{2}\) denotes the spectral norm of a matrix.
The sequence \(\{x^{\ell }\}\) generated by (1.12) satisfies the following inequality:
where \(x^{\ast }\) is an arbitrary solution of (1.8). We note that
if and only if \(x^{\ell }\) is a solution of (1.8).
Now, we consider another iterative scheme for projection method:
where the step size \(\alpha _{\ell }^{\ast }\) is also defined in (1.13).
We note that the sequence \(\{x^{\ell }\}\) is generated by (1.16) also satisfies the property (1.15) and thus its convergence is ensured.
In this work, we establish the worst-case \(O(\frac{1}{t})\)-convergence rate measured by the iteration complexity for (1.12) and (1.16), where t is the iteration counter.
In our analysis, we know that
and
Lemma 1.3
([3])
Let C be a nonempty closed convex subset of a real Hilbert space H. Then, for any \(x,y,z \in H\), the following properties holds:
-
(i)
\(\Vert x-P_{C}(x) \Vert \leq \Vert x-y \Vert \);
-
(ii)
\(\langle x-P_{C}(x), P_{C}(x)-z\rangle \geq 0\), or \(\langle x-P_{C}(x), z-P_{C}(x)\rangle \leq 0\);
-
(iii)
\(\Vert P_{C}(x)-P_{C}(y) \Vert ^{2}\leq \langle x-y, P_{C}(x)-P _{C}(y)\rangle \);
-
(iv)
\(\Vert P_{C}(x)-z \Vert ^{2}\leq \Vert x-z \Vert ^{2}- \Vert x-P_{C}(x) \Vert ^{2}\);
-
(v)
\(\Vert P_{C}(x)-P_{C}(y) \Vert \leq \Vert x-y \Vert \).
Let \(x^{\ast }\) be any fixed solution of generalized complementarity problems. Since \(\bar{x}^{\ell }\in C\), it follows from (1.8) that
Setting \(y=x^{\ell }-(Mx^{\ell }+q)\), \(G=I\) and \(x=x^{\ast }\) in (1.17), because of the notation \(\bar{x}^{\ell }\), we have
Adding Eqs. (1.18) and (1.19) we have
and consequently
2 An ϵ-approximated solution of generalized complementarity problems
To estimate the worst-case convergence rates measured by the iteration complexity for (1.12) and (1.16), we need to precisely define an ϵ-approximated solution of (1.8). We will consider the following two definitions, which are based on the generalized complementarity problems characterization and projection equation residual, respectively.
From [6], we know that \(C^{\ast }\) is a convex and it can be characterized as follows:
Therefore, from [17] \(y\in C\) is an ϵ-approximated solution point of (1.8) if it satisfies
where
Later, we show that, for given \(\epsilon >0\), after at most \(O(\frac{1}{\epsilon })\) iterations, both (1.12) and (1.16) can find y such that
As we mentioned as regards \(e(x)\) defined in (1.11), \(\Vert e(x) \Vert ^{2}\) is a measure of the distance between the iterate x and the solution set \(C^{\ast }\). We call y an ϵ-approximation solution of generalized complementarity problems in the sense of a projection equation residual if \(\Vert e(y) \Vert ^{2}\leq \epsilon \).
Now, we define
where \(\alpha _{\ell }^{\ast }\) is given by (1.13) and use the notation
where M is the matrix in (1.8). We show that the sequence \(\{x^{\ell }\}\) generated by either (1.12) or (1.16) satisfies the inequality
where \(q^{\ell }(\zeta )\) is defined in (2.2).
Lemma 2.1
For given \(x^{\ell }\in \mathbb{R}^{n}\), let \(\bar{x}^{\ell }\) be defined by (1.8) and the new iterate \(x^{\ell +1}\) be generated by (1.12). Then the assertion (2.3) is satisfied.
Proof
Setting \(y=x^{\ell }-(Mx^{\ell }+q)\) in (1.17), and using \(\bar{x}^{\ell }=P_{C}[x^{\ell }-(Mx^{\ell }+q)]\), we have
can be rewritten as
Using the notation \(M+M^{T}=D\) and the Cauchy–Schwarz inequality, we have
Moreover, it follows from (1.12) that
Thus, we obtain
For the crossed term in the right-hand side of (2.4):
it follows from the identity
that
Now, we treat the second part of the right-hand side of (2.5). Using (1.12), we get
Recall the definition of \(\alpha ^{\ast }_{\ell }\) in (1.13). It follows from (2.6) that
and consequently
Substituting it into the right-hand side of (2.5) and using the definition of \(q_{\ell }(\zeta )\), we obtain
Adding (2.4) and (2.7) together, we get the assertion (2.3) and the theorem is proved. □
Now, we prove the assertion (2.3) for (1.16) in the following lemma.
Lemma 2.2
For given \(x^{\ell }\in \mathbb{R}^{n}\), let \(\bar{x}^{\ell }\) be defined by (1.10) and the new iterate \(x^{\ell +1}\) be generated by (1.16). Then the assertion (2.3) is satisfied.
Proof
We have
It follows from the Cauchy–Schwarz inequality that
Consequently, we obtain
Now, we investigate the first term in right-hand side of (2.8) and divide it into the following two terms:
and
First, we deal with term (2.9). Let us set \(y=x^{\ell }-(Mx ^{\ell }+q)\) in (1.17). Since \(\bar{x}^{\ell }=P_{C}^{G}[x ^{\ell }-(Mx^{\ell }+q)]\) and \(x^{\ell +1}\in C\), it follows that
Adding the term
to both sides in the above inequality, we obtain
and it follows that
For the crossed term of right-hand side in (2.11), using the Cauchy–Schwart inequality and (1.13), we get
Substituting it into the right-hand of (2.11) and using the notation \(q^{\ell }(\zeta )\), we obtain
Now, we turn to treating the term (2.10). The update form of (1.16) means that \(x^{\ell +1}\) is the proximal of the vector
onto C. Thus, from (1.17), we have
and consequently
Using the identity
for the right hand side of the last inequality, we obtain
Adding (2.12) and (2.13) together, we get
Finally, substituting it into (2.8), the proof is complete. □
Based on Lemma 2.1 and 2.2, the strict contraction property of the sequences generated by (1.12) and (1.16) can easily be derived. On the basis of the above two lemmas, we have the following corollary.
Corollary 2.3
The sequence \(\{x^{\ell }\}\) generated by either (1.12) or (1.16) is strictly contractive with respect to the solution set \(C^{\ast }\) of (1.8).
Proof
From Lemma 2.1 and Lemma 2.2, we have proved that the sequence \(\{x^{\ell }\}\) generated by either (1.12) or (1.16) satisfies inequality (2.3). Setting \(x=x^{\ast }\) in (2.3), where \(x^{\ast }\in C^{\ast }\) is an arbitrary solution of (1.8), we get
Since
it follows from the last inequality and (2.2) that
which means that the sequence \(\{x^{\ell }\}\) generated by either (1.12) or (1.16) is strictly contractive with respect to the solution set \(C^{\ast }\). The proof is completed. □
3 Iteration complexity in ergodic sense
We first derive the worst-case convergence rates measured by iteration complexity in the ergodic sense. For this purpose, we need the definition of an ϵ-approximated solution of generalized complementarity problems.
Theorem 3.1
Let the sequence \(\{x^{\ell }\}\) be generated by either (1.12) or (1.16) starting from \(x^{0}\) and \(\bar{x}^{\ell }\) be given by (1.10). For any integer \(t>0\), let
Then it satisfies
Proof
Since Lemma 2.1 and Lemma 2.2 satisfied for any \(\zeta >0\) and the strict contraction in Corollary 2.3 is guaranteed for \(\zeta \in (0,2)\). In this proof, we can slightly extend the restriction of ζ to \(\zeta \in (0,2]\). For this case, we still have \(q^{\ell }(\zeta )\geq 0\). It follows from the property of M, (2.2) and (2.3) that
Summarizing the above inequality over \(\ell =0,1,\ldots ,t\), we obtain
Using the notation of \(\lambda _{t}\) and \(\bar{x}^{t}\) in the above inequality, we obtain
Indeed, \(\bar{x}^{t}\in C\), because it is a convex combination of the iterates \(\bar{x}^{0},\bar{x}^{1},\ldots ,\bar{x}^{t}\).
Since
it follows from (3.1) that
Substituting it into (3.3), the proof is completed. □
The next theorem shows that the worst-case \(O(\frac{1}{t})\)-convergence rate measured by the iteration complexity in the ergodic sense for (1.12) and (1.16).
Theorem 3.2
For any \(\epsilon >0\) and \(x^{\ast }\in C^{\ast }\), with an initial iterate \(x^{0}\), either (1.12) or (1.16) requires no more iterations than \(\lceil \frac{d}{2\alpha _{\mathrm{min}}\zeta \epsilon } \rceil \) to produce an ϵ-approximated solution of generalized complementarity problems (2.1), where
Proof
For \(x\in \mathcal{D}(\bar{x}^{t})\), using the Cauchy–Schwarz inequality and the convexity of \(\Vert \cdot \Vert ^{2}_{G}\), we have
On the other hand, it follows from (1.15) that
and
Since
This, together with (3.2), completes the proof of the theorem. □
4 Iteration complexity in nonergodic sense
In this section, we derive the worst-case \(O(\frac{1}{t})\) convergence rates measured by the iteration complexity in a nonergodic sense for (1.12) and (1.16). For this purpose, we need the definition of an ϵ-approximated solution of (1.8) in the sense of a projection equation residual characterization.
Theorem 4.1
For any \(\epsilon >0\) and \(x^{\ast }\in C^{\ast }\), with an initial iterate \(x^{0}\), either (1.12) or (1.16) requires no more iterations than \(\lceil \frac{ \Vert x^{0}-x^{\ast } \Vert ^{2}_{G}}{ \alpha _{\mathrm{min}}\zeta (2-\zeta )\epsilon }\rceil \) to obtain an ϵ-approximated solution of (1.8) in sense of \(\Vert e(x^{\ell }) \Vert ^{2}\leq \epsilon\).
Proof
Summarizing inequality (1.15) over \(\ell =0,1,\ldots , t\) and using the inequality \(\alpha _{\ell }\geq \alpha _{\mathrm{min}}\), we derive that
This implies
which proves the theorem. □
Indeed, the worst-case \(O(\frac{1}{t})\)-convergence rates in nonergodic sense established in Theorem 4.1 can be easily refined as an \(O(\frac{1}{t})\) order. We summarize it in the following.
Corollary 4.2
Let the sequence \(\{x^{\ell }\}\) be generated by either (1.12) or (1.16), and \(e(x^{\ell })\) be defined in (1.11). For any integer \(t>0\), we have
Proof
Notice that
where Eq. (4.3) holds due to (4.1) and the Cauchy principle. The proof is completed. □
References
Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and Distributed Computation, Numerical Methods. Prentice-Hall, Englewood Cliffs (1989)
Billupsa, S.C., Murty, K.G.: Complementarity problems. J. Comput. Appl. Math. 124, 303–318 (2000)
Brezis, H.: Opérateurs Maximaux Monotones et Semi-Groupes de Contractions dans les Espaces de Hilbert. North-Holland, Amsterdam (1973)
Chen, C., Fu, X., He, B.S., Yuan, X.: On the iteration complexity of some projection methods for monotone linear variational inequalities. J. Optim. Theory Appl. 172, 914–928 (2017)
Cottle, R.W., Pang, J.-S., Stone, R.E.: The Linear Complementarity Problem. Academic Press, Boston (1992)
Facchinei, F., Pang, J.S.: Finite Dimensional Variational Inequalities and Complementarity Problems, Vol. I and II, Springer Series in Operations. Springer, New York (2003)
Ferris, M.C., Pang, J.-S. (eds.): Complementarity and Variational Problems: State of the Art. SIAM, Philadelphia (1997)
Ferris, M.C., Pang, J.-S.: Engineering and economic applications of complementarity problems. SIAM Rev. 39, 669–713 (1997)
Harker, P.T., Pang, J.-S.: Finite-dimensional variational inequality and nonlinear complementarity problems: a survey of theory, algorithms and applications. Math. Program. 48, 161–220 (1990)
Hartman, P., Stampacchia, G.: An Introduction to Variational Inequalities and Their Applications. Academic Press, New York (1980)
He, B.S.: A globally linearly convergence projection and contraction method for a class of linear complementarity problems. Schwerpunkprogramm der Deutschen Forschungsgemeinschaft Anwendungsbezogene Optimierung und Steuerung. Report No. 352, (1992)
He, B.S.: A new method for a class of linear variational inequalities. Math. Program. 66, 137–144 (1994)
He, B.S., Yuan, X.M.: On non-ergodic convergence rate of Douglas–Rachford alternating direction method multipliers. Numer. Math. 130(3), 567–577 (2015)
Karush, W.: Minima of functions of several variables with inequalities as side constraints. M.Sc. Dissertation, Department of Mathematics, Univ. Chicago, Chicago, Illinois, USA (1939)
Nemirovski, A.: Pro-method with of convergence \(O(\frac{1}{n})\) for variational inequality with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM J. Optim. 15(2), 229–251 (2005)
Nesterov, Y.E.: A method for solving the convex programming problem with convergence rate \(O(\frac{1}{k^{2}})\). Dokl. Akad. Nauk SSSR 269, 543–547 (1983)
Nesterov, Y.E.: Gradient methods for minimizing composite objective functions. Math. Program. 140(1), 125–161 (2013)
Varga, R.S.: Matrix Iterative Analysis. Prentice-Hall, Englewood Cliffs (1966)
Availability of data and materials
The data sets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Funding
This work was supported by the Department of Mathematics, Jazan University, Jazan-45142, KSA.
Author information
Authors and Affiliations
Contributions
All authors finished this work together. All authors read and approved the final manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Salahuddin, Alesemi, M. Iteration complexity of generalized complementarity problems. J Inequal Appl 2019, 79 (2019). https://doi.org/10.1186/s13660-019-2024-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13660-019-2024-8