On Indefinite Quadratic Optimization over the Intersection of Balls and Linear Constraints

In this paper, we study the minimization of an indefinite quadratic function over the intersection of balls and linear inequality constraints (QOBL). Using the hyperplanes induced by the intersection of each pair of balls, we show that the optimal solution of QOBL can be found by solving several extended trust-region subproblems (e-TRS). To solve e-TRS, we use the alternating direction method of multipliers approach and a branch and bound algorithm. Numerical experiments show the efficiency of the proposed approach compared to the CVX and the extended adaptive ellipsoid-based algorithm.

and sufficient optimality conditions and exact semidefinite relaxation (SDR) [15,26]. However, for e-TRS, the necessary and sufficient optimality conditions and the SDR hold under certain assumptions [12,13,22]. A variant of QCQO that is minimizing a quadratic function subject to the intersection of the inside and outside of several balls with extra linear constraints is studied in [7]. The authors proposed a Branch and Bound (BB) algorithm to solve it and reported preliminary numerical results.
In this paper, we study a special case of the problem in [7] that minimizes a quadratic function subject to the intersection of several balls and linear inequality constraints (QOBL). Variants of this problem appear for example in solving nonconvex source localization problems and numerical solution of parameter identification [6,8]. As a special case of QCQO, one may apply algorithms such as the Extended Adaptive Ellipsoid-based (EAE) algorithm to solve QOBL [16,23]. We show that QOBL can be reduced to m e-TRS using the hyperplanes induced by the intersection of each pair of balls constraints. To solve e-TRS, we utilize alternating direction method of multipliers (ADMM) and the BB algorithm of [7]. The rest of the paper is organized as follows. In Sect. 2, we give our main results, namely reducing QOBL to m e-TRS. In Sect. 3, we briefly discuss the ADMM [10] for solving e-TRS. Finally, in Sect. 4, numerical results are given to show the efficiency of the proposed approach in comparison with some existing algorithms.

Main Results
Consider the following quadratic optimization problem with ball and linear inequality constraints: where A ∈ R n×n is a symmetric matrix, a, c i , b k ∈ R n , β k ∈ R and δ i ∈ R + . When m = 1 and p = 0, QOBL reduces to the well-known TRS [15] and when m = 1 and p ≥ 1, it reduces to the following e-TRS: that has been widely studied in recent years [1,2,12,13,17,21,22,25,[28][29][30][31].  The feasible  region of QOBL, where the blue  lines are linear constraints The following notations are used throughout this section: For clarity, we have also shown them in Fig. 1. In the following lemma, we discuss a case where QOBL is infeasible. We will discuss other cases where QOBL becomes infeasible in the rest of the paper (see Fig. 2b). The following lemma discusses the redundancy of ball constraints.
Following Lemma 2.2, we make the following assumption for the rest of the paper.
Assumption 1 For all i ∈ I, there is no j ∈ I \ {i} such that B i ⊆ B j and the Slater condition holds for QOBL. Also, we assume m ≥ 3.
As noted earlier, the case with m = 1 corresponds to the well-studied p-eTRS, see for example [2,7,12,22,30] and the case with m = 2 is handled by a similar approach as in [4]. In the following results, our goal is to characterize the feasible region of QOBL as the union of the feasible region of m, (m + p − 1)−eTRS. The first result shows that if M j is nonempty, then it has a point on the boundary of B j .
Then, there exist d ∈ R n and > 0, such that for y = x + d we have Therefore, ||y − c j || 2 = δ 2 j and 2(c j − c i ) T y ≤ α ji ∀i ∈ I \ { j}, which completes the proof.
In the previous lemma, we showed that when M j is nonempty, it intersects the boundary of a ball. In the following lemma, we will show that when p = 0 the intersection of M j with the boundary of a ball is a part of the boundary of the feasible region of QOBL.

Lemma 2.4 Suppose QOBL satisfies Assumption 1 and p
The following theorem enables us to find redundant ball constraints containing the feasible region, but does not completely contain any of the other ball constraints (see Fig. 4). We should note that these types of redundant ball constraints are not of the type discussed in Lemma 2.2.
This means z λ * ∈ M j , which is a contradiction with M j = ∅. Therefore, The following theorem, which is the main result of this paper, shows that the feasible region of QOBL is the union of the feasible region of m, (m + p − 1)−eTRS (see Fig. 5).

Theorem 2.2 Suppose QOBL satisfies Assumption
Without loss of generality, we assume that Thus, This implies m i=1 R i ⊆ R. Therefore, from Theorem 2.2, solving QOBL reduces to solve m, (m+ p−1)−eTRS as follows for all i ∈ I (see Fig. 5): Using Theorem 2.1, in the following lemma, we show that infeasible (P R i ) means a redundant ball constraint (see Fig. 6).
is the optimal solution of the following convex quadratic problem: is empty, and from Theorem 2.1, the ith ball constraint is redundant.
Based on the previous results, the algorithm for solving QOBL can be outlined as follows.

QOBL algorithm
Step 1: If there exist i, j ∈ I such that ||c i − c j || > δ i + δ j , then QOBL is infeasible, stop; else go to Step 2.
Step 2: Step 4: Solve (P R i ) for all i ∈ I, and save x * i , f * i , its optimal solution and optimal objective value.
Step 5: f * k = min i∈I f * i and x * k are the optimal objective value and global optimal solution of QOBL, respectively.
As we see, the main computational costs of algorithm is solving m, (m + p−1)−eTRS.
In the next section, we discuss the solution approach for p-eTRS.

Solving p-eTRS
As mentioned in the introduction, p-eTRS has been widely studied in recent years. The BB algorithm of [7] is a recent efficient algorithm to solve p-eTRS that we use in our numerical experiments. Also, we utilize the ADMM approach that has been widely used to solve various classes of optimization problems [3,10,19,20,30,32]. Consider the following ith (m + p − 1)−eTRS (i ∈ I) that arises in the QOBL algorithm: One can write it in the following equivalent form: Now to define the ADMM steps, consider the augmented Lagrangian of (4) as follows: where λ i 's are Lagrange multipliers and ρ ∈ R + is the appropriate penalty parameter. Let z k be a feasible point for (m + p − 1)−eTRS that is obtained by solving m (C R i ).
The ADMM iterations are as follows: In Step 1, we solve the following TRS: Let x k+1 be the optimal solution of (5). In Step 2, we solve the following problem: As we see, if ρ ≥ −λ min (A), then in Step 1 and Step 2, we have convex optimization problems, where λ min (A) is the smallest eigenvalue of A. It should also be noted that the convergence results for the ADMM algorithms under some mild assumptions are established in [10,20,30,32] for different classes of optimization problems. The convergence of ADMM to the first-order stationary point is given in the following theorem.

Numerical Results
In this section, we compare the QOBL algorithm with CVX [18] (solves the semidefinite programming (SDP) relaxation of QOBL) and the EAE algorithm [16,23]. The SDP relaxation of QOBL is as follows: where I n is the identity matrix and Ker(A) := {d ∈ R n |Ad = 0} [14]. To solve p-eTRS within the QOBL algorithm, we use the BB algorithm of [7] and ADMM. Implementation is done in MATLAB R2017a on a 2.50 GHz laptop with 8 GB of RAM, and the results in tables are the average of 10 runs for each dimension. It is worth noting that p-eTRSs inside the QOBL algorithm can be solved in parallel. We report the results for both parallel and non-parallel implementations. (CPU time in parentheses are for the parallel version.) The used machine allows solving two p-eTRSs in parallel. We generate instances of QOBL such that the Slater condition holds. To this end, first we generate a random matrix C ∈ R n×m . Let c i , i ∈ I be the columns of the matrix C. Then, we set y ∈ R n as the convex combination of the columns of C, i.e., Next, we set δ i = ||c i − y|| + i ∀i ∈ I, where i ∈ (0, 1). "-" in all tables means the algorithm cannot solve the problem -Test class 1: In this class, we compare the QOBL algorithm when using ADMM with CVX (that solves the SDP relaxation). To do so, we consider m < n and generate A ∈ R n×n randomly such that multiplicity of its smallest eigenvalue is greater than m and p = 0. Therefore, the SDP relaxation is exact. The results are reported in Table 2. As we see, for dimensions 50 ≤ n ≤ 300, CVX is better in terms of CPU time, while for the rest of the problems, the QOBL algorithm is faster and CVX cannot solve larger problems. The parallel version of the QOBL algorithm also shows significant CPU time reduction for larger problems. -Test class 2: In this class, we compare the QOBL algorithm and EAE algorithm of [16]. To solve p-eTRS inside the QOBL algorithm, we use the BB algorithm of [7] and ADMM. We generate A ∈ R n×n randomly and we set p = 0. The results are summarized in Table 3. As we see, the EAE algorithm is able to solve problems for n ≤ 100 and except for one instance, the non-parallel QOBL algorithm with ADMM is always faster than it, while they have almost equal objective values. Also, when m ≤ 20, the non-parallel QOBL algorithm with the BB algorithm is faster than the EAE algorithm in terms of CPU time. When the number of ball constraints is increasing, the QOBL algorithm with ADMM is better than the QOBL algorithm with the BB algorithm in terms of CPU time, while having almost equal objective values. Here, also we see significant time reduction of the parallel QOBL algorithm. Also, in both parallel and non-parallel versions, the QOBL algorithm with ADMM is faster than the QOBL algorithm with the BB algorithm for m > 10. -Test class 3: In this class, we compare the QOBL algorithm and EAE algorithm, when p = 0. By considering y as in (8), we add linear inequality constraints as follows: 1-Generate b k ∈ R n for k = 1, . . . , p randomly, 2-Let β k = b T k y + where ∈ (0, 1).     Therefore, y as given in (8) is an interior point of QOBL. The corresponding results are summarized in Table 4. A similar observation as in the previous tables also hold here and the QOBL algorithm in overall, performs better than the EAE algorithm. Also, when the number of ball and linear constraints, and dimensions are increasing, the QOBL algorithm with ADMM is the best among all. -Test class 4: In this class, we apply the parallel QOBL algorithm to instances when m ≥ n. We generate A ∈ R n×n randomly, and we set p = 0. To solve (m + p − 1)−eTRS inside the QOBL algorithm, we use ADMM. The results are summarized in Table 5 that can be further enhanced by running on cluster machines.

Conclusions
In this paper, we studied an indefinite quadratic minimization problem with balls and linear inequality constraints (QOBL). We showed that by solving several extended trust-region subproblems (e-TRS), the optimal solution of QOBL can be found. Our experiments showed that when SDP relaxation is exact, the new approach is better than CVX for larger dimensions. For general instances, our comparison with the EAE algorithm of [7] showed that the new approach is significantly faster. Also using ADMM for solving e-TRS, inside the QOBL algorithm, for majority of problems is faster than the BB algorithm of [7]. Parallelization also is another important feature of the QOBL algorithm.
Funding Open Access funding provided by the Qatar National Library.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.