A note on completely positive relaxations of quadratic problems in a multiobjective framework

In a single-objective setting, nonconvex quadratic problems can equivalently be reformulated as convex problems over the cone of completely positive matrices. In small dimensions this cone equals the cone of matrices which are entry wise nonnegative and positive semidefinite, so the convex reformulation can be solved via SDP solvers. Considering multiobjective nonconvex quadratic problems, naturally the question arises, whether the advantage of convex reformulations extends to the multicriteria framework. In this note, we show that this approach only finds the supported nondominated points, which can already be found by using the weighted sum scalarization of the multiobjective quadratic problem, i.e. it is not suitable for nonconvex problems.


Introduction
Many nonconvex, even combinatorial optimization problems can equivalently be reformulated as convex problems over the so-called completely positive matrix cone. Here the reader is referred to [3,7,9] for more details. This reformulation is possible since the whole complexity is moved to the cone constraint. Checking membership to the cone is known to be NP hard, cf. [8]. Nevertheless, especially in small dimensions, an optimal solutions for the nonconvex problem can be obtained by solving the convex reformulation. For larger dimensions approximations of the cone or its dual can be used for deriving lower and upper bounds of the optimal value of the problems, cf. [4,14,16]. In this paper, we consider such a class of nonconvex quadratic problems including binary constraints which can, in the single-objective case, equivalently be reformulated. Thus we especially focus on quadratic objectives and linear constraints as motivated by Burer, cf. [5]. However, we will study the multiobjective case. Multiobjective quadratic problems are of importance in the field of data mining and especially in the context of classification, see for example [6,18]. Classical portfolio optimization, where risk should be minimized and return should be maximized simultaneously, see for example [17], gives rise to another important field of application.
As we will see throughout this note, shifting the complexity of the original problem to the cone constraint comes along with the advantage of a linear objective. Since operating in a convex environment is also a big advantage in a multiobjective framework, naturally the question arises whether convexifying multiobjective nonconvex quadratic binary problems via completely positive reformulations is again a useful tool. In this note, we will show that considering multiobjective completely positive reformulations for this problem returns just a subset of the optimal solution set: only those efficient solutions which can already be found by using a weighted sum scalarization.
Thereby, it is known that the weighted sum scalarization is not an appropriate tool for scalarizing nonconvex problems: only in the case of convexity all efficient solutions of a multiobjective optimization problem can be found by this approach (and by varying the weights), cf. [10, Section 3]. It might even happen that one only finds the individual minima of the individual objective functions, as we will also illustrate with an example in this note. With our results we show that first applying the weighted sum method and then studying its completely positive reformulation, as proposed by [1], finds exactly the same efficient solutions as solving the direct multiobjective completely positive reformulation. Thus this convexifying approach using the cone of completely positive matrices seems not to be a suitable tool for problems with multiple objectives.
The paper is organized as follows: In Section 2, we give the theoretical background based on the results by Burer [5] on how to convexify binary quadratic problems, followed by some basic properties in multiobjective optimization in Section 3. In Section 4, we give our main results in Theorems 4.3 and 4.5 followed by a short illustrative example.

Convex reformulations of nonconvex quadratic problems
Throughout the paper, for given matrices X, Y ∈ R m×n , let X, Y := trace(X T Y ) denote the Frobenius matrix inner product. Moreover, for a given square matrix X, we understand the inequality X ≥ 0 entrywise, and X 0 indicates X to be positive semidefinite. In the following, let Q ∈ R n×n , c ∈ R n , a i ∈ R n , b i ∈ R for every i = 1, . . . , l and B ⊆ {1, . . . , n}. We consider the optimization problem To obtain a convex reformulation of this problem, we make use of the cone of completely positive matrices.
Definition 2.1. A symmetric matrix X of order n is called completely positive if there exists a rank-1 representation of the following type: There exists r ∈ N such that The smallest number r ∈ N for which such a factorization of X exists is called cp-rank of X. We denote by CP n the set of all completely positive matrices of order n.
We have the following properties, cf. [2, Proposition 1.24] and [13]: (a) CP n is a closed, pointed, convex matrix cone with nonempty interior.
In higher dimensions, i.e., for n ≥ 5, to show that a matrix is completely positive is known to be NP-hard, cf. [8]. Nevertheless, any rank-1 representation gives a certificate for the matrix to be completely positive. For a projection-type approach to obtain completely positive factorizations, the reader is referred to [11].
Due to [5], Problem (QP 1 ) can equivalently be reformulated as For showing this equivalence, Burer in [5] makes the following key assumption and shows that this assumption holds without loss of generality. Thus, we assume this assumption to hold for the forthcoming results. The relation of Problem (CP 1 ) to Problem (QP 1 ) is now summarized in the following theorem, cf. [5, Theorem 2.6]: Theorem 2.3. Problems (QP 1 ) and (CP 1 ) are equivalent in the following sense: (a) Assume one of the Problems (QP 1 ) or (CP 1 ) to have an optimal solution, then so has the other and the corresponding optimal objective function values coincide.
(b) Let (x * , X * ) be optimal for (CP 1 ), then x * is in the convex hull of optimal solutions for (QP 1 ).
Moreover, we have the following useful results: Lemma 2.4. If Problem (CP 1 ) is solvable, there always exists an optimal solution of the form (x, xx T ).
Proof. Due to [5,Equation 7], for every optimal solution (x * , X * ) for (CP 1 ), there exist finite subsets of indices K > and K 0 and a factorization and v k j ∈ {0, 1} for every j ∈ B and k ∈ K > , and k∈K> λ k = 1, as well as z k ≥ 0 for all k ∈ K 0 . Thus any v k is also feasible for (QP 1 ).
Letv ∈ argmin {v k |k∈K>} Q, v k v k T +2c T v k . Thenv is also feasible for (QP 1 ). Moreover, by [5, Proof of Theorem 2.6, p.485], any v k and thus in particularv is also optimal for (QP 1 ). Due to Theorem 2.3(a) the optimal value of (CP 1 ) and of (QP 1 ) are the same and we obtain Sincev j = (vv T ) jj for every j ∈ B as well as Lemma 2.5. Let (x, xx T ) be an optimal solution for (CP 1 ). Then x is an optimal solution for (QP 1 ).
Proof. We have that x is also feasible for (QP 1 ). By Theorem 2.3(a) together with Proof. First, observe that x feasible for (QP 1 ) implies (x, xx T ) feasible for (CP 1 ). Moreover, again by Theorem 2.3(a), the optimal values coincide and with To sum up, we have the following relations between (QP 1 ) and (CP 1 ): Corollary 2.7. It holds that:

Multiobjective optimization: efficiency and supported points
In the following, let R m , as well as f : R n → R m and M ⊆ R n , M = ∅. We consider the multiobjective optimization problem . . .
Thus, supported solutions of (MOP) are those which can be obtained by applying the weighted sum scalarization and by solving the resulting single-objective problem. Note that we can equivalently replace the assumption w ∈ R m + with m i=1 w i = 1 in the definition above by w ∈ R m + \ {0}. By [10, Proposition 3.9] any supported solution is also weakly efficient. In a convex setting, we have the following result, cf. [10, Section 3]: In case f i , i = 1, . . . , m and M are convex, then f (M ) + R m + is convex. In the context of efficiency, we further introduce the following definition, see also [15,Definition 3.2.6], where this property is also named external stability.  Proof. First, let x * ∈ M be a supported solution. Thus, there exists w ∈ R m it holds w T y < w T f (x * ) and thus y ∈ K. Hence, f (M ) satisfies the convex inclusion condition at f (x * ). For the reverse direction, let f (M ) satisfy the convex inclusion condition at f (x * ). Thus, there exists a convex set K ⊇ f (M ) such that K ∩ ({f (x * )} − int(R m + )) = ∅. Then, by Eidelheit's separation theorem (see, for example, [12,Theorem 3.16]), there exists a vector d ∈ R m \ {0} and a scalar γ ∈ R with As the assumption d T (−r) < 0 for some r ∈ R m and thus x * is a supported solution.
In [1,Theorem 3.3] it was already stated for quadratic problems with linear constraints that if f (M ) satisfies the convex inclusion condition at f (x * ) with x * being weakly efficient for (MOP), then x * is a supported solution for (MOP). Above we have shown that there is an if and only if characterization between supported solutions and points x * for which the convex inclusion condition at f (x * ) is satisfied. The further examinations in [1] have only been done for supported solutions by using the weighted sum approach.

Extension of the completely positive reformulation to the multiobjective framework
Let Q i ∈ R n×n and c i ∈ R n for every i = 1, . . . , m. A multiobjective extension of Problem (QP 1 ) now reads as: (QP m ) Following the steps from the single-objective reformulation one obtains the completely positive problem In the following, we will show that only one of the relations in Corollary 2.7 does extend to the multicriteria framework. As will be shown in Theorems 4.3 and 4.5, as well as Corollary 4.6, similar results only hold for supported solutions for (QP m ). As a first step to obtain these results, we mention the following convexity property, which follows from the convexity of the cone CP n+1 , see Lemma 2.2: Following Lemma 3.2, we immediately obtain:   (b) By Lemma 4.2 there exists w ∈ R m + with m i=1 w i = 1 such that (x, xx T ) is also an optimal solution for (CP w m ). Lemma 2.5 implies x to be an optimal solution for the problem: Thus, x is a supported solution for (QP m ).
(c) Let x be a supported solution for (QP m ). Thus, there exists w ∈ R m + with m i=1 w i = 1 such that x is an optimal solution for (QP w m ). Due to Lemma 2.6, we know that (x, xx T ) is optimal for (CP w m ). Hence, (x, xx T ) is a supported solution for (CP m ) and by Lemma 4. The only result which transfers from Corollary 2.7 to the multiobjective setting is the following: Proof. Let (x, X) be weakly efficient for (CP m ). Then, by Lemma 4.2 there exists w ∈ R m + with m i=1 w i = 1 such that (x, X) is an optimal solution for (CP w m ). By Theorem 2.3(b), x is then in the convex hull of optimal solutions for (QP w m ). Any optimal solution for (QP w m ) is also a weakly efficient solution for (QP m ) and the proof is complete.
From the proof even the following main characterization follows: This is an important result, as it shows that by looking at weakly efficient solutions (x, X) for (CP m ), we can only find those weakly efficient solutions x for (QP m ) which are in the convex full of supported solutions for (QP m ). Clearly, in general for a nonconvex problem (QP m ), not all weakly efficient solutions are in the convex full of the supported solutions. Here the forthcoming Example 4.8 will provide a concrete instance.
Next to that, we only have the following results, which we collect similarly to Corollary 2.7: The following proposition allows to characterize not only the supported but also the other weakly efficient solutions for (QP m ) and thus adds insights to (3): Proof. Let x be a weakly efficient solution for (QP m ) which is not supported. Then (x, xx T ) is feasible for (CP m ). Due to Theorem 4.3(b), we get that (x, xx T ) is not weakly efficient for (CP m ). To close the proof, it suffices to note that (CP m ) satisfies the domination property, cf. [15,Theorem 3.2.9].
This proposition together with Corollary 4.6 now shows that if Y is compact, it holds that: The following example illustrates our results: We consider the following instance of (QP m ): x ≥ 0.

(EQP )
For the completely positive reformulation of (EQP ), we obtain:

(ECP )
At first, we will have a closer look at the decision space of (EQP ) illustrated in Figure 1. There, the set {x ∈ R 3 | x feasible for (EQP )} is illustrated in dark blue and the set {x | x weakly efficient for (EQP )} is shown in red. In addition, all weakly efficient solutions which are as well supported for (EQP ) are highlighted as green diamonds.
As Figure 1 substantiates, there exist weakly efficient solutions of (EQP ) which are not an element of conv{x | x supported for (EQP )}. Thus, not all weakly efficient solutions x for (EQP ) can be identified, using weakly efficient solutions (x, X) for (ECP ), as Theorem 4.5 shows. In order to solve (ECP ) and since n ≤ 4 holds, we can make use of Lemma 2.2, and thus replace the completely positive cone by the doubly nonnegative cone, which is the cone of symmetric (4 × 4) matrices which are componentwise nonnegative and positive semidefinite in this concrete setting. Note that due to Lemma 4.2, all weakly efficient solutions of Problem (ECP ) are supported. Thus, it is sufficient to apply the weighted sum scalarization with varying weights and to solve the corresponding individual scalar problem of type via semidefinite programming in order to find all weakly efficient solutions for (ECP ). Figure 2 visualizes the objective space of both problems (EQP ) and (ECP ).
To be more precise, the set {f (x) | x feasible for (EQP )} is illustrated in dark blue, the set {f (x) | x weakly efficient for (EQP )} is shown in red, and the two green diamonds represent the set {f (x) | x supported for (EQP )}. Furthermore, the teal dots represent discrete points of the set {F (x, X) | (x, X) feasible for (ECP )} and the set {F (x, X) | (x, X) weakly efficient for (ECP )} is highlighted in orange. We first note that indeed every weakly efficient solution for (ECP ) is as well supported.
Moreover, Figure 2 indicates the images of three weakly efficient solutions for (ECP ) in black, which can be obtained by solving (ECP w ) for different values of w. For instance, by taking w = (0.42857142, 0.57142858), we obtain a weakly efficient solution  Figure 2. Here especially rank(X * ) = 2 holds. Thus, Figure 2 further substantiates the fact that there does not exists (x, xx T ) feasible for (ECP ) such that F (x, xx T ) = F (x * , X * ). Nevertheless, since (x * , X * ) is optimal for (ECP w ) and with the help of Lemma 2.4, there exists a rank-1 optimal solution (x , x x T ) for (ECP w ) such that F w (x * , X * ) = F w (x , x x T ) holds. This very rank-1 optimal solution corresponds to one of the outer black dots in Figure 2. The outer black dots both correspond to so called individual minimizers, i.e., to images of pointsx i withx i ∈ argmin{f i (x) | x feasible for (EQP )}. Also due to Theorem 4.3(b), x is supported for (EQP ). Thus, the green diamonds in Figure

Conclusion
In this note, we proved that the advantages of convexifying quadratic problems via completely positive reformulations do not extend to a multicriteria framework. To be more precise, in the multiobjective completely positive reformulation (CP m ), every weakly efficient solution is already supported. To recover weakly efficient solutions for the multiobjective quadratic problem (QP m ), we need to consider rank-1 weakly efficient solutions for (CP m ). But here we saw that they can only recover the weakly efficient solutions for (QP m ) which are supported. Thus, to obtain these weakly efficient solutions for (QP m ), it would be sufficient to solve the single-objective problem (QP w m ) and to leave the multiobjective framework. Further, a concrete example illustrated the results of this note.