Skip to main content

On the Convexification of Constrained Quadratic Optimization Problems with Indicator Variables

  • Conference paper
  • First Online:
Integer Programming and Combinatorial Optimization (IPCO 2020)

Abstract

Motivated by modern regression applications, in this paper, we study the convexification of quadratic optimization problems with indicator variables and combinatorial constraints on the indicators. Unlike most of the previous work on convexification of sparse regression problems, we simultaneously consider the nonlinear objective, indicator variables, and combinatorial constraints. We prove that for a separable quadratic objective function, the perspective reformulation is ideal independent from the constraints of the problem. In contrast, while rank-one relaxations cannot be strengthened by exploiting information from k-sparsity constraint for \(k\ge 2\), they can be improved for other constraints arising in inference problems with hierarchical structure or multi-collinearity.

Andrés Gómez is supported, in part, by grant 1930582 of the National Science Foundation. Simge Küçükyavuz is supported, in part, by ONR grant N00014-19-1-2321.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Aktürk, M.S., Atamtürk, A., Gürel, S.: A strong conic quadratic reformulation for machine-job assignment with controllable processing times. Oper. Res. Lett. 37(3), 187–191 (2009)

    Article  MathSciNet  Google Scholar 

  2. Anstreicher, K.M.: On convex relaxations for quadratically constrained quadratic programming. Math. Program. 136(2), 233–251 (2012). https://doi.org/10.1007/s10107-012-0602-3

    Article  MathSciNet  MATH  Google Scholar 

  3. Atamtürk, A., Gómez, A.: Strong formulations for quadratic optimization with M-matrices and indicator variables. Math. Program. 170(1), 141–176 (2018). https://doi.org/10.1007/s10107-018-1301-5

    Article  MathSciNet  MATH  Google Scholar 

  4. Atamtürk, A., Gómez, A.: Rank-one convexification for sparse regression (2019). http://www.optimization-online.org/DB_HTML/2019/01/7050.html

  5. Atamtürk, A., Gómez, A., Han, S.: Sparse and smooth signal estimation: convexification of L0 formulations (2018). http://www.optimization-online.org/DB_HTML/2018/11/6948.html

  6. Bacci, T., Frangioni, A., Gentile, C., Tavlaridis-Gyparakis, K.: New MINLP formulations for the unit commitment problems with ramping constraints. Optimization (2019). http://www.optimization-online.org/DB_FILE/2019/10/7426.pdf

  7. Belotti, P., Góez, J.C., Pólik, I., Ralphs, T.K., Terlaky, T.: A conic representation of the convex hull of disjunctive sets and conic cuts for integer second order cone optimization. In: Al-Baali, M., Grandinetti, L., Purnama, A. (eds.) Numerical Analysis and Optimization. SPMS, vol. 134, pp. 1–35. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-17689-5_1

    Chapter  MATH  Google Scholar 

  8. Bertsimas, D., Cory-Wright, R., Pauphilet, J.: A unified approach to mixed-integer optimization: nonlinear formulations and scalable algorithms. arXiv preprint arXiv:1907.02109 (2019)

  9. Bertsimas, D., King, A.: OR forum - an algorithmic approach to linear regression. Oper. Res. 64(1), 2–16 (2016)

    Article  MathSciNet  Google Scholar 

  10. Bertsimas, D., King, A., Mazumder, R.: Best subset selection via a modern optimization lens. Ann. Stat. 44(2), 813–852 (2016)

    Article  MathSciNet  Google Scholar 

  11. Bien, J., Taylor, J., Tibshirani, R.: A lasso for hierarchical interactions. Ann. Stat. 41(3), 1111 (2013)

    Article  MathSciNet  Google Scholar 

  12. Bienstock, D., Michalka, A.: Cutting-planes for optimization of convex functions over nonconvex sets. SIAM J. Optim. 24(2), 643–677 (2014)

    Article  MathSciNet  Google Scholar 

  13. Burer, S.: On the copositive representation of binary and continuous nonconvex quadratic programs. Math. Program. 120(2), 479–495 (2009). https://doi.org/10.1007/s10107-008-0223-z

    Article  MathSciNet  MATH  Google Scholar 

  14. Burer, S., Kılınç-Karzan, F.: How to convexify the intersection of a second order cone and a nonconvex quadratic. Math. Program. 162(1–2), 393–429 (2016). https://doi.org/10.1007/s10107-016-1045-z

    Article  MathSciNet  MATH  Google Scholar 

  15. Ceria, S., Soares, J.: Convex programming for disjunctive convex optimization. Math. Program. 86, 595–614 (1999). https://doi.org/10.1007/s101070050106

    Article  MathSciNet  MATH  Google Scholar 

  16. Cozad, A., Sahinidis, N.V., Miller, D.C.: Learning surrogate models for simulation-based optimization. AIChE J. 60(6), 2211–2227 (2014)

    Article  Google Scholar 

  17. Cozad, A., Sahinidis, N.V., Miller, D.C.: A combined first-principles and data-driven approach to model building. Comput. Chem. Eng. 73, 116–127 (2015)

    Article  Google Scholar 

  18. Dong, H.: On integer and MPCC representability of affine sparsity. Oper. Res. Lett. 47(3), 208–212 (2019)

    Article  MathSciNet  Google Scholar 

  19. Dong, H., Ahn, M., Pang, J.-S.: Structural properties of affine sparsity constraints. Math. Program. 176(1–2), 95–135 (2019). https://doi.org/10.1007/s10107-018-1283-3

    Article  MathSciNet  MATH  Google Scholar 

  20. Dong, H., Chen, K., Linderoth, J.: Regularization vs. relaxation: a conic optimization perspective of statistical variable selection. arXiv preprint arXiv:1510.06083 (2015)

  21. Dong, H., Linderoth, J.: On valid inequalities for quadratic programming with continuous variables and binary indicators. In: Goemans, M., Correa, J. (eds.) IPCO 2013. LNCS, vol. 7801, pp. 169–180. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36694-9_15

    Chapter  MATH  Google Scholar 

  22. Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32(2), 407–499 (2004)

    Article  MathSciNet  Google Scholar 

  23. Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96(456), 1348–1360 (2001)

    Article  MathSciNet  Google Scholar 

  24. Frangioni, A., Furini, F., Gentile, C.: Approximated perspective relaxations: a project and lift approach. Comput. Optim. Appl. 63(3), 705–735 (2015). https://doi.org/10.1007/s10589-015-9787-8

    Article  MathSciNet  MATH  Google Scholar 

  25. Frangioni, A., Gentile, C.: Perspective cuts for a class of convex 0–1 mixed integer programs. Math. Program. 106, 225–236 (2006). https://doi.org/10.1007/s10107-005-0594-3

    Article  MathSciNet  MATH  Google Scholar 

  26. Frangioni, A., Gentile, C.: SDP diagonalizations and perspective cuts for a class of nonseparable MIQP. Oper. Res. Lett. 35(2), 181–185 (2007)

    Article  MathSciNet  Google Scholar 

  27. Frangioni, A., Gentile, C., Grande, E., Pacifici, A.: Projected perspective reformulations with applications in design problems. Oper. Res. 59(5), 1225–1232 (2011)

    Article  MathSciNet  Google Scholar 

  28. Frangioni, A., Gentile, C., Hungerford, J.: Decompositions of semidefinite matrices and the perspective reformulation of nonseparable quadratic programs. Math. Oper. Res. (2019). https://doi.org/10.1287/moor.2018.0969. Article in Advance (October)

  29. Günlük, O., Linderoth, J.: Perspective reformulations of mixed integer nonlinear programs with indicator variables. Math. Program. 124, 183–205 (2010). https://doi.org/10.1007/s10107-010-0360-z

    Article  MathSciNet  MATH  Google Scholar 

  30. Hastie, T., Tibshirani, R., Wainwright, M.: Statistical Learning with Sparsity: The Lasso and Generalizations. Monographs on Statistics and Applied Probability, vol. 143. Chapman and Hall/CRC, Boca Raton (2015)

    Book  Google Scholar 

  31. Hazimeh, H., Mazumder, R.: Learning hierarchical interactions at scale: a convex optimization approach. arXiv preprint arXiv:1902.01542 (2019)

  32. Hijazi, H., Bonami, P., Cornuéjols, G., Ouorou, A.: Mixed-integer nonlinear programs featuring “on/off” constraints. Comput. Optim. Appl. 52(2), 537–558 (2012). https://doi.org/10.1007/s10589-011-9424-0

    Article  MathSciNet  MATH  Google Scholar 

  33. Huang, J., Breheny, P., Ma, S.: A selective review of group selection in high-dimensional models. Stat. Sci.: Rev. J. Inst. Math. Stat. 27(4), 481–499 (2012)

    Google Scholar 

  34. Jeon, H., Linderoth, J., Miller, A.: Quadratic cone cutting surfaces for quadratic programs with on-off constraints. Discrete Optim. 24, 32–50 (2017)

    Article  MathSciNet  Google Scholar 

  35. Kılınç-Karzan, F., Yıldız, S.: Two-term disjunctions on the second-order cone. In: Lee, J., Vygen, J. (eds.) IPCO 2014. LNCS, vol. 8494, pp. 345–356. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07557-0_29

    Chapter  Google Scholar 

  36. Manzour, H., Küçükyavuz, S., Shojaie, A.: Integer programming for learning directed acyclic graphs from continuous data. arXiv preprint arXiv:1904.10574 (2019)

  37. Miller, A.: Subset Selection in Regression. Chapman and Hall/CRC, Boca Raton (2002). https://doi.org/10.1201/9781420035933

    Book  MATH  Google Scholar 

  38. Modaresi, S., Kılınç, M.R., Vielma, J.P.: Intersection cuts for nonlinear integer programming: convexification techniques for structured sets. Math. Program. 155(1), 575–611 (2015). https://doi.org/10.1007/s10107-015-0866-5

    Article  MathSciNet  MATH  Google Scholar 

  39. Natarajan, B.K.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24(2), 227–234 (1995)

    Article  MathSciNet  Google Scholar 

  40. Richard, J.-P.P., Tawarmalani, M.: Lifting inequalities: a framework for generating strong cuts for nonlinear programs. Math. Program. 121(1), 61–104 (2010). https://doi.org/10.1007/s10107-008-0226-9

    Article  MathSciNet  MATH  Google Scholar 

  41. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc.: Ser. B (Methodol.) 58, 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  42. Vielma, J.P.: Small and strong formulations for unions of convex sets from the Cayley embedding. Math. Program. 177(1–2), 21–53 (2019). https://doi.org/10.1007/s10107-018-1258-4

    Article  MathSciNet  MATH  Google Scholar 

  43. Wang, A.L., Kılınç-Karzan, F.: The generalized trust region subproblem: solution complexity and convex hull results. arXiv preprint arXiv:1907.08843 (2019a)

  44. Wang, A.L., Kılınç-Karzan, F.: On the tightness of SDP relaxations of QCQPs. Optimization Online preprint (2019b). http://www.optimization-online.org/DB_FILE/2019/11/7487.pdf

  45. Wu, B., Sun, X., Li, D., Zheng, X.: Quadratic convex reformulations for semicontinuous quadratic programming. SIAM J. Optim. 27(3), 1531–1553 (2017)

    Article  MathSciNet  Google Scholar 

  46. Xie, W., Deng, X.: The CCP selector: scalable algorithms for sparse ridge regression from chance-constrained programming. arXiv preprint arXiv:1806.03756 (2018)

  47. Zhang, C.-H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38, 894–942 (2010)

    Article  MathSciNet  Google Scholar 

  48. Zheng, X., Sun, X., Li, D.: Improving the performance of MIQP solvers for quadratic programs with cardinality and minimum threshold constraints: a semidefinite program approach. INFORMS J. Comput. 26(4), 690–703 (2014)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Andrés Gómez .

Editor information

Editors and Affiliations

Appendix

Appendix

Proof

(Theorem 2). First, note that the validity of the new inequality defining \(\mathrm {cl \ conv}\left( Z_{Q_1}\right) \) follows from Proposition 1. For \(a, b \in \mathbb R^{p}\) and \(c \in \mathbb {R}\), consider the following two optimization problems:

$$\begin{aligned} \min \quad&a^\top z + b^\top \beta + c t \qquad ~\text{ subject } \text{ to }\;\;\;\; (z, \beta , t) \in Z_{Q_1}. \end{aligned}$$
(14)

and

$$\begin{aligned} \min \quad&a^\top z + b^\top \beta + c t \end{aligned}$$
(15a)
$$\begin{aligned} \text {subject to} \quad&(\mathbf 1^\top \beta )^2 \le t \end{aligned}$$
(15b)
$$\begin{aligned}&\frac{(\mathbf 1^\top \beta )^2}{\sum _{i \in [p]} z_i} \le t \end{aligned}$$
(15c)
$$\begin{aligned}&\sum _{i \in [p]} z_i \le k \end{aligned}$$
(15d)
$$\begin{aligned}&z \in [0,1]^p. \end{aligned}$$
(15e)

The analysis for cases where \(c=0\) and \(c<0\) is similar to the proof of Theorem 1, and we can proceed with assuming \(c=1\) and \(b \in \mathbb R^{p}\). First suppose that b is not a multiple of all-ones vector, then \(\exists b_i < b_j\) for some \(i,j\in [p], i\ne j\). Let \(\bar{z} = e_i + e_j\), \(\bar{\beta }= \tau (e_i - e_j)\) for some scalar \(\tau \), and \(\bar{t}=0\). Note that \((\bar{z},\bar{\beta },\bar{t})\) is feasible for both (14) and (15), and if we let \(\tau \) go to infinity the objective value goes to minus infinity. So (14) and (15) are unbounded.

Now suppose that \(b = \kappa \mathbf 1^\top \) for some \(\kappa \in \mathbb {R}\) and \(c = 1\); in this case both (14) and (15) have finite optimal value. It suffices to show that there exists an optimal solution \((z^{*}, \beta ^{*}, t^{*})\) of (15) that is integral in \(z^{*}\). If \(\sum _{i \in [p]} z^{*}_i = 0\), then we know \(z^{*}_i =\beta _i^*= 0, \forall i \in [p]\) for both (14) and (15), and we are done. If \(0< \sum _{i \in [p]} z^{*}_i < 1\) and the corresponding optimal objective value is 0 (or positive), then by letting \(z^{*} =\mathbf 0\), \(\beta ^{*} =\mathbf 0\) and \(t^{*} = 0\), we get a feasible solution with the same objective value (or better). If \(0< \sum _{i \in [p]} z^{*}_i < 1\) and \((z^{*}, \beta ^{*}, t^{*})\) attains a negative objective value, then let \(\gamma = \frac{1}{\sum _{i \in [p]} z^{*}_i}\): \((\gamma z^{*}, \gamma \beta ^{*}, \gamma t^{*})\) is also a feasible solution of (15) with a strictly smaller objective value, which is a contradiction.

Finally, consider the case where \(\sum _{i \in [p]} z^{*}_i \ge 1\). In this case, the constraint \((\mathbf 1^\top \beta )^2 \le t\) is active and the optimal value is attained when \(\mathbf 1^\top \beta ^{*} = -\frac{\kappa }{2}\) and \(t^{*} = (\mathbf 1^\top \beta ^{*})^2 \), and (15) has the same optimal value as the LP:

$$\begin{aligned} \min \quad&a^\top z - \frac{\kappa ^2}{4} \qquad ~\text{ subject } \text{ to }\;\;\;\; 1 \le \sum _{i \in [p]} z_i \le k, z \in [0,1]^{p}. \end{aligned}$$

The constraint set of this LP is an interval matrix, so the LP has an integral optimal solution, \(z^{*}\), hence, so does (15).    \(\square \)

Proof

(Lemma 1). Suppose \(z^{*}\) is an extreme point of \(Q_g\) and \(z^{*}\) has a fractional entry. If \(\sum _{i \in [p-1]} z^{*}_i - (p-2)z^{*}_p > 1\), let us consider the two cases where \(z^{*}_p = 0\) and \(z^{*}_p >0\). When \(z^{*}_p = 0\) and there exists a fractional coordinate \(z^{*}_i\) where \(i \in [p-1]\), we can perturb \(z^{*}_i\) by a sufficient small quantity \(\epsilon \) such that \(z^{*} + \epsilon e_i\) and \(z^{*} - \epsilon e_i\) are in \(Q_g\). Then, \(z^{*} = \frac{1}{2} (z^{*} + \epsilon e_i) + \frac{1}{2} (z^{*} - \epsilon e_i)\) which contradicts the fact that \(z^{*}\) is an extreme point of \(Q_g\). When \(1> z^{*}_p > 0\) we can perturb \(z^{*}_p\) and all other \(z^{*}_i\) with \(z^{*}_i = z^{*}_p\) by a sufficiently small quantity \(\epsilon \) and stay in \(Q_g\). Similarly, we will reach a contradiction.

Now suppose \(\sum _{i \in [p-1]} z^{*}_i - (p-2)z^{*}_p = 1\), and let us consider again the two cases where \(z^{*}_p = 0\) and \(z^{*}_p >0\). When \(z^{*}_p = 0\), \(z^{*} = z^{*}_1e_1 + \cdots + z^{*}_{(p-1)} e_{(p-1)}\), which is a contradiction since we can write \(z^{*}\) as a convex combination of points \(e_i\in Q_g, i\in [p-1]\) and there exists at least two indices \(i, j \in [p-1], i\ne j\) such that \(1> z^{*}_i , z^{*}_j >0\) by the fact that \(z^{*}\) has a fractional entry and \(\sum _{i \in [p-1]} z^{*}_i = 1, 0 \le z^{*}_i \le 1, \forall i\). When \(1> z^{*}_p > 0\), we first show that there exists at most one 1 in \(z^{*}_1, z^{*}_2, \dots , z^{*}_{(p-1)}\). Suppose we have \(z^{*}_i =1 \) and \(z^{*}_j = 1\) for \(i,j\in [p-1]\) with \(i\ne j\), then \(\sum _{i \in [p-1]} z^{*}_i - (p-2)z^{*}_p = z^{*}_i + \sum _{l \in [p-1], l \ne i} (z^{*}_{l} - z^{*}_p) \ge z^{*}_i + (z^{*}_{j} - z^{*}_p) > z^{*}_i = 1\), which is a contradiction. We now show that we can perturb \(z^{*}_p\) and the \(p-2\) smallest elements in \(z^{*}_i, i \in [p-1]\) by a small quantity \(\epsilon \) and remain in \(Q_g\). The equality \(\sum _{i \in [p-1]} z_i - (p-2) z_p = 1\) clearly holds after the perturbation. And, adding a small quantity \(\epsilon \) to \(z^{*}_p\) and the \(p-2\) smallest elements in \(z^{*}_i, i \in [p-1]\) will not violate the hierarchy constraint since the largest element in \(z^{*}_i, i \in [p-1]\) has to be strictly greater than \(z^{*}_p\). (Note that if \(z^{*}_i = z^{*}_p, \forall i \in [p]\), \(\sum _{i \in [p-1]} z^{*}_i - (p-2)z^{*}_p = z^{*}_p < 1\).) Since \(z^{*}_i \ge z^{*}_p >0, \forall i \in [p-1]\) subtracting a small quantity \(\epsilon \) will not violate the non-negativity constraint. Thus, we can write \(z^{*}\) as a convex combination of two points in \(Q_g\), which is a contradiction.   \(\square \)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wei, L., Gómez, A., Küçükyavuz, S. (2020). On the Convexification of Constrained Quadratic Optimization Problems with Indicator Variables. In: Bienstock, D., Zambelli, G. (eds) Integer Programming and Combinatorial Optimization. IPCO 2020. Lecture Notes in Computer Science(), vol 12125. Springer, Cham. https://doi.org/10.1007/978-3-030-45771-6_33

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-45771-6_33

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-45770-9

  • Online ISBN: 978-3-030-45771-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics