Linear complementarity problems on extended second order cones

In this paper, we study the linear complementarity problems on extended second order cones. We convert a linear complementarity problem on an extended second order cone into a mixed complementarity problem on the non-negative orthant. We state necessary and sufficient conditions for a point to be a solution of the converted problem. We also present solution strategies for this problem, such as the Newton method and Levenberg-Marquardt algorithm. Finally, we present some numerical examples.


Introduction
Although research in cone complementarity problems (see the definition in the beginning of the Preliminaries) goes back a few decades only, the underlying concept of complementarity is much older, being firstly introduced by Karush in 1939 [1].It seems that the concept of complementarity problems was first considered by Dantzig and Cottle in a technical report [2], for the non-negative orthant.In 1968, Cottle and Dantzig [3] restated the linear programming problem, the quadratic programming problem and the bimatrix game problem as a complementarity problem, which inspired the research in this field (see [4][5][6][7][8]).
The complementarity problem is a cross-cutting area of research which has a wide range of applications in economics, finance and other fields.Earlier works in cone complementarity problems present the theory for a general cone and the practical applications merely for the non-negative orthant only (similarly to the books [8,9]).These are related to equilibrium in economics, engineering, physics, finance and traffic.Examples in economics are Walrasian price equilibrium models, price oligopoly models, Nash-Cournot production/distribution models, models of invariant capital stock, Markov perfect equilibria, models of decentralised economy and perfect competition equilibrium, models with individual markets of production factors.Engineering and physics applications are frictional contact problems, elastoplastic structural analysis and nonlinear obstacle problems.An example in finance is the discretisation of the differential complementarity formulation of the Black-Scholes models for the American options [10].An application to congested traffic networks is the prediction of steady-state traffic flows.In the recent years several applications have emerged where the complementarity problems are defined by cones essentially different from the non-negative orthant such as positive semidefinite cones, second order cones and direct product of these cones (for mixed complementarity problems containing linear subspaces as well).Recent applications of second order cone complementarity problems are in elastoplasticity [11,12], robust game theory [13,14] and robotics [15].All these applications come from the Karush-Kuhn-Tucker conditions of second order conic optimization problems.
Németh and Zhang extended the concept of second order cone in [16] to the extended second order cone.Their extension seems the most natural extension of second order cones.Sznajder showed that the extended second order cones in [16] are irreducible cones (i.e., they cannot be written as a direct product of simpler cones) and calculated the Lyapunov rank of these cones [17].The applications of second order cones and the elegant way of extending them suggest that the extended second order cones will be important from both theoretical and practical point of view.Although conic optimization problems with respect to extended second order cones can be reformulated as conic optimization problems with respect to second order cones, we expect that for several such problems using the particular inner structure of the second order cones provides a more efficient way of solving them than solving the transformed conic optimization problem with respect to second order cones.Indeed, such a particular problem is the projection onto an extended second order cone which is much easier to solve directly than solving the reformulated second order conic optimization problem [18].
Until now the extended second order cones of Németh and Zhang were used as a working tool only for finding the solutions of mixed complementarity problems on general cones [16] and variational inequalities for cylinders whose base is a general convex set [19].The applications above for second order cones show the importance of these cones and motivates considering conic optimization and complementarity problems on extended second order cones.As another motivation we suggest the application to mean variance portfolio optimization problems [20,21] described in Section 3.
The paper is structured as follows: In Section 2, we illustrate the main terminology and definitions used in this paper.In Section 3 we present an application of extended second order cones to portfolio optimization problems.In Section 4, we introduce the notion of mixed implicit complementarity problem as an implicit complementarity problem on the direct product of a cone and a Euclidean space.In Section 5, we reformulate the linear complementarity problem as a mixed (implicit, mixed implicit) complementarity problem on the non-negative orthant (MixCP).
Our main result is Theorem 1, which discusses the connections between an ESOCLCP and mixed (implicit, mixed implicit) complementarity problems.In particular, under some mild conditions, given the definition of Fischer-Burmeister (FB) regularity and of the stationarity of a point, we prove in Theorem 2 that a point can be the solution of a mixed complementarity problem if it satisfies specific conditions related to FB regularity and stationarity (Theorem 2).This theorem can be used to determine whether a point is a solution of a mixed complementarity problem converted from ESOCLCP.In Section 6, we use Newton's method and Levenberg-Marquardt algorithm to find the solution for the aforementioned MixCP.In Section 7, we provide an example of a linear complementarity problem on an extended second order cone.Based on the above, we convert this linear complementarity problem into a mixed complementarity problem on the non-negative orthant, and use the aforementioned algorithms to solve it.A solution of this mixed complementarity problem will provide a solution of the corresponding ESOCLCP.
As a first step, in this paper, we study the linear complementarity problems on extended second order cones (ESOCLCP).We find that an ESOCLCP can be transformed to a mixed (implicit, mixed implicit) complementarity problem on the non-negative orthant.We will give the conditions for which a point is a solution of the reformulated MixCP problem, and in this way we provide conditions for a point to be a solution of ESOCLCP.

Preliminaries
Let m be a positive integer and F : R m → R m be a mapping and y = F (x).The definition of the classical complementary problem [22] x ≥ 0, y ≥ 0, and x, y = 0, where ≥ denotes the componentwise order induced by the non-negative orthant and •, • is the canonical scalar product in R m , was later extended to more general cones K, as follows: x ∈ K, y ∈ K * , and x, y = 0, where K * is the dual of K [23].
Let k, ℓ, l be non-negative integers such that m = k + ℓ.
Recall the definitions of the mutually dual extended second order cone L(k, ℓ) and where e = (1, . . ., 1) ⊤ ∈ R k .If there is no ambiguity about the dimensions, then we simply denote L(k, ℓ) and M(k, ℓ) by L and M, respectively.Denote by •, • the canonical scalar product in R m and by • the corresponding Euclidean norm.The notation x ⊥ y means that x, y = 0, where x, y ∈ R m .
Let K ⊂ R m be a nonempty closed convex cone and K * its dual.
Definition 2 Let F : R m → R m .Then, the complementarity problem CP(F, K) is defined by: The solution set of CP(F, K) is denoted by SOL-CP(F, K): If T is a matrix, r ∈ R m and F is defined by F (x) = T x + r, then CP(F, K) is denoted by LCP(T, r, K) and is called linear complementarity problem.The solution set of LCP(T, r, K) is denoted by SOL-LCP(T, r, K).
Definition 3 Let G, F : R m → R m .Then, the implicit complementarity problem ICP(F, G, K) is defined by The solution set of ICP(F, G, K) is denoted by SOL-ICP(F, G, K): Let m, k, ℓ be non-negative integers such that m = k + ℓ, Λ ∈ R k be a nonempty closed convex cone and (5) Definition 5 [8, Definition 3.7.29]A matrix Π ∈ R n×n is said to be an S 0 matrix if the system of linear inequalities Πx ≥ 0, 0 = x ≥ 0 has a solution.
The proof of our next result follows immediately from K * = Λ * × {0} and the definitions of CP(F, K) and MixCP(F 1 , F 2 , Λ).

Define the mapping
Then, Definition 6 [24, Schur complement] The notation of the Schur complement for a matrix (ii) We say that f is locally Lipschitz if for every x ∈ I, there exists ε > 0 such that f is Lipschitz on

An Application of Extended Second Order Cones to Portfolio Optimisation Problems
Consider the following Portfolio Optimisation Problem: where Σ ∈ R n×n is the covariance matrix, e = (1, . . ., 1) ⊤ ∈ R n , w ∈ R n is the weight of asset allocation for the portfolio and R is the required return of the portfolio.
In order to guarantee the diversified allocation of the fund into different assets in the market, a new constraint can be reasonably introduced: w ≤ ξ, where ξ is the limitation of the concentration of the fund allocation.If short selling is allowed, then w can be less than zero.The introduction of this constraint can guarantee that the fund will be allocated into few assets only.
Since the covariance matrix Σ can be decomposed into Σ = U ⊤ U, the problem can be rewritten as min w,ξ,y The constraint Uw ≤ y is a relaxation of the constraint U w ≤ y, where U = max x ≤1 Ux .The strengthened problem will become: The minimal value of the objective of the original problem is at most as large as the minimal value of the objective for this latter problem.The second constraint of the latter portfolio optimisation problem means that the point (ξ, y/ U , w) ⊤ belongs to the extended second order cone L(2, n).Hence, the strenghtened problem is a conic optimisation problem with respect to an extended second order cone.

Mixed Implicit Complementarity Problems
Let m, k, ℓ, l be non-negative integers such that m = k + ℓ, Λ ∈ R k be a nonempty, closed, convex cone and Definition 8 Consider the mappings The mixed implicit complementarity problem The solution set of the mixed complementarity problem The proof of our next result follows immediately from K * = Λ * × {0} and the definitions of ICP(F, G, K) and MixICP(F

Main Results
The linear complementarity problem is the dual problem of a quadratic optimisation problem, which has a wide range of applications in various areas.One of the most famous application is the portfolio optimisation problem first introduced by Markowitz [20]; see the application of the extended second order cone to this problem presented in the Introduction.
Proof.Items (i) and (ii) are easy consequence of the definitions of L, M and the complementarity set of a nonempty closed convex cone.
(iv) It is a simple reformulation of item (iii) by using the change of variables (x, u) → (x − u e, u).
(v) Again it is a simple reformulation of item (iv) by using that u C + u ⊤ eA is a nonsingular matrix.
(vi) Suppose that z ∈ SOL-LCP(T, r, L).Then, (x, u, y, v) ∈ C(L), where y = Ax + Bu + p and v = Cx + Du + q.Let t = u , Then, by item (iii) of Proposition 3, we have that ∃λ > 0 such that where z = (x − t, u, t).From equation ( 18), we get t(Cx + Du + q) = −tλu, which, by equation ( 19), implies t(Cx + Du + q) = −ue ⊤ (Ax + Bu + p), which after some algebra gives From equations ( 20) and ( 21) we obtain that z ∈ SOL-MixCP( Note that the item(vi) makes F 1 (x, u, t) and F 2 (x, u, t) become smooth functions by adding the variable t.The smooth functions therefore make the smooth Newton's method applicable to the mixed complementarity problem.The conversion of LCP on extended second order cones to a MixCP problem defined on the non-negative orthant is useful, because it can be studied by using the Fischer-Burmeister function.In order to ensure the existence of the solution of MixCP, we introduce the scalar Fischer-Burmeister C-function (see [26,27]).
Obviously, ψ 2 F B (a, b) is a continuously differentiable function on R 2 .The equivalent FBbased equation formulation for the MixCP problem is: with the associated merit function: We continue by calculating the Jacobian matrix for the associated merit function.
where e i denotes the i-th canonical unit vector.The differential with respect to z j with j = i is Obviously, the differential with respect to z j with j > k, is equal to zero.Note that if (z i , F i 1 ) = (0, 0), then ) i ∂z will be a generalised gradient of a composite function, i.e., a closed unit ball B(0, 1).However, this case will not occur in our paper.As for the term F 2 (x, u, t) with i ∈ (k + 1, ...m + 1), the Jacobian matrix is much more simple, since Therefore, the Jacobian matrix for the associated merit function is: where Define the following index sets: (x, u, t) with respect to x, where and In our case, for the MixCP F 1 , F 2 , R k + , the Jacobian matrices are: In our case, if the Jacobian matrix block Proposition 4 If the matrices A and D are nonsingular for any z ∈ R m+1 , then the Jacobian matrix A for the associated merit function is nonsingular.

Proof. It is easy to check that
A is a nonsingular matrix if and only if the sub-matrix D a + D b A and its Schur complement are nonsingular, and they are nonsingular if and only if the matrices A and D are nonsingular.
The following theorem is [8,Theorem 9.4.4].For the sake of completeness, we provide a proof here.
Then, it follows that z * is a global minimum and hence a stationary point of θ MixCP F B .Thus, (x * , F 1 (z * )) ∈ C(R k + ), and we have P = N = ∅.Therefore, the FB regularity of x * holds since x * = x C , because there is no nonzero vector x satisfying conditions (23).Conversely, suppose that x * is FB regular and . It follows that ∇θ MixCP F B = 0, i.e.: where Hence, for any w ∈ R m+1 , we have Assume that z * is not a solution of MixCP.Then, we have that the index set R is not empty.
. We have Take w with From the definition of D a and D b , we know that have the same sign.Therefore, By the regularity of J F 1 (z) ⊤ , we have The inequalities ( 27) and ( 28) together contradict condition (26).Hence R = ∅.It means that z * is a solution of MixCP( F 1 , F 2 , R k ).

Algorithms
For solving a complementarity problem, there are many different algorithms available.The common algorithms include numerical methods for systems of nonlinear equations (such as Newton's method [28]), the interior point method (Karmarkar's Algorithm [29]), the projection iterative method [30], and the multi-splitting method [31].In the previous sections, we have already provided sufficient conditions for using FB regularity and stationarity to identify a solution of the MixCP problem.In this section, we are trying to find a solution of LCP by finding the solution of MixCP which is converted from LCP.One convenient way to do this is using the Newton's Method as follows: Algorithm (Newton's method): Given initial data z 0 ∈ R m+1 , and r = 10 −7 .
If the Jacobian matrix A ⊤ is nonsingular, then the direction d k ∈ R m+1 for each step can be found.The following theorem, which is based on an idea similar to the one used in [32], proves that such a Newton's Method can efficiently solve the LCP on extended second order cone (i.e.solve the problem within polynomial time), by finding the solution of the MixCP: Theorem 3 Suppose that the Jacobian matrix A is nonsingular.Then, Newton's method for MixCP( F 1 , F 2 , R k + ) converges at least quadratically to if it starts with initial data z 0 sufficiently close to z * .
Proof.Suppose that the starting point z 0 is close to the solution z * , and suppose that A is a Lipschitz function.There are ρ > 0, β 1 > 0, β 2 > 0, such that for all z with z − z * < ρ, there holds A −1 (z) < β 1 , and By the definition of the Newton's method, we have (z * ) = 0 when z * ∈ SOL-MixCP.By Taylor's theorem, we have Also, we have z − z * < ρ, that is, Another widely-used algorithm is presented by Levenberg and Marquardt in [33].Levenberg-Marquardt algorithm can approach second-order convergence speed without requiring the Jacobian matrix to be nonsingular.We can approximate the Hessian matrix by: and the gradient by: G Hence, the upgrade step will be As we can see, Levenberg-Marquardt algorithm is a quasi-Newton's method for an unconstrained problem.When µ equals to zero, the step upgrade is just the Newton's method using approximated Hessian matrix.The number of iterations of Levenberg-Marquardt algorithm to find a solution is higher than that of Newton's method, but it works for singular Jacobian as well.The greater the parameter µ, the slower the calculation speed becomes.Levenberg-Marquardt algorithm is provided as follows: Algorithm (Levenberg-Marquardt): Given initial data z 0 ∈ R m+1 , µ = 0.005, and r = 10 −7 .
Theorem 4 [34] Without the nonsingularity assumption on the Jacobian matrix A, Levenberg-Marquardt Algorithm for MixCP( F 1 , F 2 , R k + ) converges at least quadratically to if it starts with initial data z 0 sufficiently close to z * .
The proof is omitted.

A Numerical Example
In this section, we will provide a numerical example for LCP on extended second order cones.Let L(3, 2) be an extended second order cone defined by (1).Following the notation in Theorem 1, let z = (x, u), ẑ = (x− u , u), z = (x−t, u, t) and r = (p, q) = (−55, −26, 50) ⊤ , (−19, −26) ⊤ with x, p ∈ R 3 , u, q ∈ R 2 , and t ∈ R. Consider  We need to check the FB regularity of z * .It is easy to show that the partial Jacobian matrix of F 1 (z * ) The FB regularity of x * holds as there is no nonzero vector x satisfying conditions (23).Then, we compute the gradient of the merit function, which is

Conclusions
In this paper, we studied the method of solving a linear complementarity problem on an extended second order cone.By checking the stationarity and FB regularity of a point, we can verify whether it is a solution of the mixed complementarity problem.Such conversion of a linear complementarity problem to a mixed complementarity problem reduces the complexity of the original problem.The connection between a linear complementarity problem on an extended second order cone and a mixed complementarity problem on a non-negative orthant will be useful for our further research about applications to practical problems, such us portfolio selection and signal processing problems.
. The square matrices T , A and D are assumed to be nonsingular.(i)Suppose u = 0. We have z ∈ SOL-LCP(T, r, L)⇐⇒ x ∈ SOL-LCP(A, p, R k +) and e ⊤ (Ax + p) ≥ Cx + q .
x − t, F 1 (z * ) = 0.That is, (x, F 1 (z * )) ∈ C(R3+ ), so the index sets P = N = ∅.The matrix A is invertible.In addition, we can calculate that the Schur complement of Π( z * ) with respect to J x F 1 (z * ):Π(z * )/J x F 1 (z * ) = D − C A −1 B = z * is a stationary point of F MixCP F B. By Theorem 2, we conclude that z * is the solution of the MixCP problem.By the item (vi) of Theorem 1, we have that z = (x, u) of LCP(T, r, L) problem.
(22)s easy to show that square matrices T, A and D are nonsingular.By item (vi) of Theorem 1, we can reformulate this LCP problem as a smooth MixCP problem.We will use the Levenberg-Marquardt algorithm to find the solution of the FB-based equation formulation(22)of MixCP problem.The convergence point is: z * = (x − t, u, t) ⊤ ,1271 3582 .