On convex relaxations for quadratically constrained quadratic programming
Authors
 First Online:
 Received:
 Accepted:
DOI: 10.1007/s1010701206023
 Cite this article as:
 Anstreicher, K.M. Math. Program. (2012) 136: 233. doi:10.1007/s1010701206023
 25 Citations
 1k Views
Abstract
We consider convex relaxations for the problem of minimizing a (possibly nonconvex) quadratic objective subject to linear and (possibly nonconvex) quadratic constraints. Let \(\mathcal{F }\) denote the feasible region for the linear constraints. We first show that replacing the quadratic objective and constraint functions with their convex lower envelopes on \(\mathcal{F }\) is dominated by an alternative methodology based on convexifying the range of the quadratic form \(\genfrac(){0.0pt}{}{1}{x}\genfrac(){0.0pt}{}{1}{x}^T\) for \(x\in \mathcal{F }\). We next show that the use of “\(\alpha \)BB” underestimators as computable estimates of convex lower envelopes is dominated by a relaxation of the convex hull of the quadratic form that imposes semidefiniteness and linear constraints on diagonal terms. Finally, we show that the use of a large class of D.C. (“difference of convex”) underestimators is dominated by a relaxation that combines semidefiniteness with RLT constraints.
Keywords
Quadratically constrained quadratic programmingConvex envelopeSemidefinite programmingReformulationlinearization techniqueMathematics Subject Classification
90C2690C221 Introduction
In Sect. 3 we compare two computable relaxations that can be viewed as tractable approximations of the problems \(\widehat{\mathrm{QCQP}}\) and \(\mathrm{Q}\widetilde{\mathrm{CQ}}\mathrm{P}\). One relaxation utilizes “\({\alpha {\mathrm{BB}}}\)” underestimators [1] for the nonconvex quadratic functions of QCQP, and the other applies semidefinite and diagonal constraints that must hold for matrices in \(\mathcal{C }\). We prove that the latter convexification dominates the former, regardless of the choice of the parameters used to define the underestimators. In Sect. 4 we consider a more general D.C. (for “difference of convex”) underestimation procedure suggested in [22], and a strengthened approximation of \(\mathrm{Q}\widetilde{\mathrm{CQ}}\mathrm{P}\) that combines semidefiniteness with linear constraints from the reformulationlinearization technique (RLT). We again show that the second approach dominates the first, regardless of the parameters used to create the underestimators.
In Sect. 5 we consider particular instances of QCQP that were used as computational examples in [2]. The first of these are indefinite boxconstrained QPs, corresponding to QCQP with \(q=0\) and \(\mathcal{F }=\{ x\,:\,0\le x\le e \}\). For these problems we obtain excellent computational results by further strengthening the approximation of \(\mathcal{C }\) through the addition of triangle inequalities related to the Boolean Quadric Polytope [12]. For the second class of QCQP problems, corresponding to planar circlepacking (or equivalently pointpacking) problems, we prove an interesting theoretical result that relates convex lower envelopes for reverse convex constraints to the use of RLT constraints for \(\mathcal{C }\).
Notation
We use \(X\succeq 0\) to denote that a symmetric matrix \(X\) is positive semidefinite. For \(n\times n\) matrices \(X\) and \(Y\), \(X \bullet Y\) denotes the matrix inner product \(X \bullet Y=\sum _{i,j=1}^n X_{ij}Y_{ij}\). For an \(n\times n\) matrix \(X\), \({\text{ diag}}(X)\) is the vector \(x\) with \(x_i=X_{ii}\), \(i=1,\ldots , n\), and \({\text{ Diag}}(x)\) is the diagonal matrix with \({\text{ diag}}({\text{ Diag}}(x))=x\). We use \(e\) to denote a vector with each component equal to one, and \(e_j\) to denote a vector with all components equal to zero, except the \(j\)th component which is equal to one.
2 Two convex relaxations for QCQP
As in Sect. 1, let \(\mathcal{F }=\{x\ge 0\,:\,Ax\le b\}\) denote the feasible set for the linear constraints of QCQP, and let \(\widehat{\mathrm{QCQP}}\) denote the problem where each function \(f_i(\cdot )\) in QCQP is replaced by \(\hat{f}_i(\cdot )\), its convex lower envelope on \(\mathcal{F }\). Let \(\hat{z}\) denote the solution value in \(\widehat{\mathrm{QCQP}}\). Note that although \(\hat{z}\) is welldefined, in practice \(\hat{z}\) may not be computable because the required convex lower envelopes \(\hat{f}_i(\cdot )\) may be impossible to obtain.
Theorem 1
For \(x\in \mathcal{F }\), let \(f(x)=x^TQx+c^Tx\), and let \(\hat{f}(\cdot )\) be the convex lower envelope of \(f(\cdot )\) on \(\mathcal{F }\). Then \(\hat{f}(x)=c^Tx+{\displaystyle \min \nolimits _X}\{Q \bullet X\,:\,Y(x,X)\in \mathcal{C }\}\).
Proof
For \(x\in \mathcal{F }\), let \(g(x)=c^Tx+{\displaystyle \min \nolimits _X}\{Q\bullet X\,:\,Y(x,X)\in \mathcal{C }\}\). Our goal is to show that \(\hat{f}(x)=g(x)\). To do this we first show that \(g(\cdot )\) is a convex function with \(g(x)\le f(x)\), \(x\in \mathcal{F }\), implying that \(g(x)\le \hat{f}(x)\).
Corollary 1
Let \(\hat{z}\) and \(\tilde{z}\) denote the solution values in the convex relaxations \(\widehat{\mathrm{QCQP}}\) and \(\mathrm{Q}\widetilde{\mathrm{CQ}}\mathrm{P}\), respectively. Then \(\hat{z}\le \tilde{z}\).
3 Two computable relaxations
As mentioned above, in general both \(\mathrm{Q}\widetilde{\mathrm{CQ}}\mathrm{P}\) and \(\widehat{\mathrm{QCQP}}\) are intractable problems due to the complexity of computing a convex lower envelope \(\hat{f}(\cdot )\), or the convex hull \(\mathcal{C }\). In this section we consider the important special case where \(\mathcal{F }\) is the box \(0\le x\le e\), and describe two further relaxations that are computable approximations of \(\mathrm{Q}\widetilde{\mathrm{CQ}}\mathrm{P}\) and \(\widehat{\mathrm{QCQP}}\).
 1.The constraints from the ReformulationLinearization Technique (RLT) [16],$$\begin{aligned} x_{ij} \ge 0,\quad x_{ij} \ge x_i+x_j1,\quad x_{ij} \le x_i,\quad x_{ij} \le x_j. \end{aligned}$$(5)
 2.
The semidefinite programming (SDP) constraint \(Y(x,X)\succeq 0\) [19].
 3.Constraints on the offdiagonal components of \(Y(x,X)\) coming from the Boolean Quadric Polytope (BQP) [6, 21]; for example, the triangle inequalities for \(i\ne j\ne k\),$$\begin{aligned} x_i+x_j+x_k&\le x_{ij}+x_{ik}+x_{jk}+1, \\ x_{ij}+x_{ik}&\le x_i+x_{jk}, \\ x_{ij}+x_{jk}&\le x_j+x_{ik}, \\ x_{ik}+x_{jk}&\le x_k+x_{ij}. \end{aligned}$$
Theorem 2
For \(0\le x\le e\), let \(f_\alpha (x)=x^T(Q+\mathrm{Diag}(\alpha ))x+(c\alpha )^Tx\), where \(\alpha \ge 0\) and \(Q+\mathrm{Diag}(\alpha )\succeq 0\). Assume that \(Y(x,X)\succeq 0\), \(\mathrm{diag}(X)\le x\). Then \(f_\alpha (x)\le Q \bullet X+c^Tx\).
Proof
The following immediate corollary of Theorem 2 confirms a relationship between \({\text{ QCQP}}_{\alpha {\mathrm{BB}}}\) and \({\text{ QCQP}}_\mathrm{SDP}\) first conjectured by Jeff Linderoth (private communication).
Corollary 2
Let \(z_{\alpha {\mathrm{BB}}}\) and \(z_\mathrm{SDP}\) denote the solution values in the convex relaxations \(\mathrm{QCQP}_{\alpha {\mathrm{BB}}}\) and \(\mathrm{QCQP}_\mathrm{SDP}\), respectively. Then \(z_{\alpha {\mathrm{BB}}}\le z_\mathrm{SDP}\).
Note that the example at the end of Sect. 2 has \(\mathcal{F }=\{ x_1\,:\, 0\le x_1\le 1\}\), \(q=1\). For this problem \((\alpha _11)x_1^2\) is convex for \(\alpha _1\ge 1\). Using \(\alpha _1=1\), the problem \({\text{ QCQP}}_{\alpha {\mathrm{BB}}}\) is identical to \(\widehat{\mathrm{QCQP}}\) and has solution value \(z_{\alpha {\mathrm{BB}}}=\hat{z}=\frac{1}{4}\). The problem \({\text{ QCQP}}_\mathrm{SDP}\) is identical to \(\mathrm{Q}\widetilde{\mathrm{CQ}}\mathrm{P}\), and has solution value \(z_\mathrm{SDP}=z^{*}=\frac{1}{2}\).
4 Two stronger relaxations
Theorem 3
For \(x\in \mathcal{F }\), let \(f_\alpha (x)=x^TQ(\alpha )x+c(\alpha )^Tx+p(\alpha )\), where \(\alpha \ge 0\) and \(Q(\alpha )\succeq 0\). Assume that \(Y(x,X)\succeq 0\) and \((x,X)\) satisfy (6). Then \(f_\alpha (x)\le Q \bullet X+c^Tx\).
Proof
Theorem 4
Let \(z_\mathrm{DC}\) and \(z_\mathrm{DNN}\) denote the solution values in the convex relaxations \(\mathrm{QCQP}_\mathrm{DC}\) and \(\mathrm{QCQP}_\mathrm{DNN}\), respectively. Then \(z_\mathrm{DC}\le z_\mathrm{DNN}\).
Proof
In [22] it is shown that if all of the quadratic constraints of QCQP are convex, then for a given set of \(\{ v_j \}_{j=1}^k\) the problem of choosing the vector \(\alpha _0\) that gives the best value of \(z_\mathrm{DC}\) can be formulated as a semidefinite programming problem. Theorem 4 states that regardless of the vectors \(\{v_j\}_{j=1}^k\) and \(\{\alpha _i\}_{i=0}^q\) used to construct the convexifications in \({\text{ QCQP}}_\mathrm{DC}\), the resulting lower bound \(z_\mathrm{DC}\) cannot be better than the bound \(z_\mathrm{DNN}\) obtained from \({\text{ QCQP}}_\mathrm{DNN}\) when the upper and lower bounds \(l\) and \(u\) correspond to the feasible set for the linear constraints \(\mathcal{F }\). However, in the presence of convex quadratic constraints, better values of \(l_j\) and/or \(u_j\) can be obtained by minimizing or maximizing \(v_j^Tx\) over the set \(\mathcal{S }\) corresponding to the feasible region for the linear and convex quadratic constraints, as suggested in [22], and in this case Theorem 4 would no longer apply unless the strengthened linear inequalities were explicitly added to \(\mathcal{F }\). Of course obtaining such improved bounds could entail substantial auxiliary computation. A different approach for utilizing convex quadratic constraints to obtain improved RLT bounds based on the secondorder cone representation of the constraints is suggested in [7, Section 2.3].
5 Applications
Comparison of bounds for indefinite QPB
Problem  Optimum  Cuts added  Relative gaps to optimum  

RLT  TRI  SDP (%)  SDP + RLT (%)  SDP + RLT + TRI (%)  
201001  706.50  197  55  4.655  0.002  0.000 
201002  856.50  184  172  5.102  0.171  0.000 
201003  772.00  168  1.750  0.000  
300601  706.00  371  777  8.799  1.229  0.000 
300602  1,377.17  381  3.614  0.000  
300603  1,293.50  394  288  5.924  0.368  0.000 
300701  654.00  369  784  14.133  3.058  0.000 
300702  1,313.00  449  4.727  0.000  
300703  1,657.40  452  442  3.763  0.010  0.000 
300801  952.73  365  718  10.290  1.315  0.000 
300802  1,597.00  376  1.616  0.000  
300803  1,809.78  317  1.492  0.000  
300901  1,296.50  370  4.009  0.000  
300902  1,466.84  344  4.160  0.000  
300903  1,494.00  420  1.527  0.000  
301001  1,227.13  356  4.777  0.000  
301002  1,260.50  427  465  8.316  0.048  0.000 
301003  1,511.05  377  265  6.622  0.139  0.000 
400301  839.50  656  4.419  0.000  
400302  1,429.00  889  4.747  0.000  
400303  1,086.00  705  6.494  0.000  
400401  837.00  710  1,966  14.228  3.117  0.000 
400402  1,428.00  600  1.718  0.000  
400403  1,173.50  745  1,427  8.209  0.626  0.000 
400501  1,154.50  797  1,608  10.592  0.515  0.000 
400502  1,430.98  788  961  6.047  0.354  0.000 
400503  1,653.63  680  5.665  0.000  
400601  1,322.67  696  1,722  12.043  2.287  0.000 
400602  2,004.23  739  4.758  0.000  
400603  2,454.50  701  2.207  0.000  
400701  1,605.00  584  3.675  0.000  
400702  1,867.50  650  3.418  0.000  
400703  2,436.50  828  3.538  0.000  
400801  1,838.50  615  5.312  0.000  
400802  1,952.50  639  3.094  0.000  
400803  2,545.50  755  742  3.647  0.015  0.000 
400901  2,135.50  763  5.948  0.000  
400902  2,113.00  731  336  7.376  0.035  0.000 
400903  2,535.00  598  2.338  0.000  
401001  2,476.38  673  3.265  0.000  
401002  2,102.50  707  1,251  5.428  0.184  0.000 
401003  1,866.07  664  1,732  9.176  2.257  0.000 
500301  1,324.50  903  4.877  0.000  
500302  1,668.00  831  233  5.257  0.200  0.000 
500303  1,453.61  830  180  7.715  0.087  0.000 
500401  1,411.00  1,017  5.103  0.000  
500402  1,745.76  868  509  7.766  0.212  0.000 
500403  2,094.50  1,081  3.938  0.000  
500501  1,198.41  723  1,531  18.304  8.664  0.144 
500502  1,776.00  867  667  9.377  0.765  0.000 
500503  2,106.10  937  933  7.689  0.752  0.000 
600201  1,212.00  1,199  7.048  0.000  
600202  1,925.50  1,319  4.418  0.000  
600203  1,483.00  1,040  735  8.200  0.543  0.000 
Average 


 5.969  0.499 

The results reported in Table 1 suggest that on QPB problems of these dimensions, the approach based on approximating \(\mathcal{C }\) is highly competitive with other methodologies. The solution process for individual problems in [20] required the solution of up to approximately 28,000 linear programs, with a total of up to approximately 500,000 cuts generated. (The root gaps, after the addition of cuts, ranged from 12.0 % to 168.5 %, with a mean of 66.2 %.) The SDP relaxations used in [8] substantially reduce the amount of enumeration compared to the algorithm of [20], but still required up to \(10^4\) CPU seconds on a 2.7 GHz Linuxbased computer to solve individual problems. Results for the generalpurpose global optimization solver BARON [15] on these problems were also reported in [20]. Of the 51 problems considered, BARON was unable to solve 21 problems within 4,000 CPU seconds on a 1.8 GHz Linuxbased computer, and the problems that were solved required approximately 20 times more computation than that required using the algorithm of [20] running on a slower machine. Good results using a methodology similar to that applied here for indefinite QPB problems of similar dimensions were previously reported in [21]. Yajima and Fujie [21] consider additional valid inequalities for the BQP beyond the triangle inequalities, but only approximate the semidefiniteness condition \(Y(x,X)\succeq 0\) by adding linear inequalities. The advantage of using linear inequalities is that the resulting relaxations are ordinary linear programming (LP) problems that can be solved using an LP solver, as opposed to the conic solver required when \(Y(x,X)\succeq 0\) is directly imposed.
In [2], bounds for the solution value of PP were computed using several combinations of semidefiniteness and RLT constraints. Note that since PP involves no terms of the form \(x_iy_j\), all SDP and RLT constraints can be based on matrices \(X\) and \(Y\) relaxing \(xx^T\) and \(yy^T\), respectively. In addition, it is clear that by symmetry one can assume that \(.5\le x_i\le 1,\; i=1,\ldots , n_x\) and \(.5\le y_i\le 1,\; i=1,\ldots , n_y\) where \(n_x=\lceil n/2 \rceil \) and \(n_y=\lceil n_x/2 \rceil \). We use “SYM” to refer to any problem formulation that uses these more restricted bounds. (Section 5 of [2] considers more elaborate symmetrybreaking using order constraints, but we omit discussion of this topic here.) The computational results obtained in [2] using the SDP, RLT and SYM conditions are summarized in Conjecture 1. (As in Sect. 3, the SDP relaxation includes the diagonal constraints \({\text{ diag}}(X)\le x\) and \({\text{ diag}}(Y)\le y\).) As described in [2], these findings are stated as a conjecture since the solution values given were numerically obtained for instances of size \(n\le 50\).
Conjecture 1
 1.
The optimal value for the RLT relaxation is 2.
 2.
The optimal value for the SDP relaxation is \(1+\frac{1}{n1}\) and adding the RLT constraints does not change this value.
 3.
For \(n\ge 5\) the optimal value for the RLT+SYM relaxation is \(\frac{1}{2}\).
 4.
For \(n\ge 5\) the optimal value for the SDP+SYM relaxation is \(\frac{1}{4}\left(1+\frac{1}{\lfloor (n1)/4 \rfloor }\right)\).
Our interest here is to demonstrate a relationship between the bounds described in Conjecture 1 and bounds that correspond to replacing the quadratic constraints \(f_{ij}(x,y,\theta )\le 0\) with their convex lower envelopes. To do this we will utilize a specialization of Theorem 1 that applies when \(\mathcal{F }=\{ x\,:\,0\le x\le e \}\) and \(f(\cdot )\) is concave.
Theorem 5
Let \(\mathcal{F }=\{ x\,:\,0\le x\le e \}\). For \(x\in \mathcal{F }\), let \(f(x)=x^TQx+c^Tx\), where \(\mathrm{diag}(Q)\le 0\), and let \(\hat{f}(\cdot )\) be the convex lower envelope of \(f(\cdot )\) on \(\mathcal{F }\). Then \(\hat{f}(x)=c^Tx+{\displaystyle \min \nolimits _X}\{Q \bullet X\,:\,(x,X)\in \mathcal{B }_n\}\).
Proof
The proof is similar to that of Theorem 1, but since several steps require modifications we include the details. For \(x\in \mathcal{F }\), let \(g(x)=c^Tx+{\displaystyle \min \nolimits _X}\{Q \bullet X\,:\,(x,X)\in \mathcal{B }_n\}\). Our goal is to show that \(\hat{f}(x)=g(x)\). To do this we first show that \(g(\cdot )\) is a convex function with \(g(x)\le f(x)\), \(x\in \mathcal{F }\), implying that \(g(x)\le \hat{f}(x)\).
Theorem 6
For \(\mathcal{F }=\{ (x,y)\,:\,0\le x\le e,\, 0\le y\le e \}\), let \(\hat{z}\) be the solution value for the relaxation of PP obtained by replacing the constraint functions with their convex lower envelopes on \(\mathcal{F }\), and let \(z_\mathrm{RLT}\) be the solution value for the relaxation that imposes the RLT constraints on \((x,X)\) and \((y,Y)\). Then \(\hat{z}\ge z_\mathrm{RLT}\). Moreover this relationship continues to hold if \(\mathcal{F }\) is replaced by the tighter SYM bounds.
Proof
Since the RLT constraints on \((x,X)\) and \((y,Y)\) are already sufficient to characterize the convex lower envelopes of the quadratic constraints in PP, it would be natural to speculate that adding the semidefiniteness conditions \(X\succeq xx^T\) and \(Y\succeq yy^T\) would have no effect on bounds for the solution value. The values given in Conjecture 1 show that this is not the case. Note, however, that each convex lower envelope \(\hat{f}_{ij}(x)\) requires only values of the variables \(X_{[i,j]}\), and the semidefiniteness condition \(Y(x,X)\succeq 0\) is stronger than the condition that all principal submatrices of \(Y(x,X)\) corresponding to two variables are semidefinite.
In the literature, the convex lower envelope of \(f(\cdot )\) is sometimes called simply the convex envelope of \(f(\cdot )\). We prefer to include the word “lower” as a reminder that the convex (lower) envelope is an underestimator of \(f(\cdot )\).
Acknowledgments
I am grateful to two anonymous referees for corrections and suggestions that have improved the paper.