Strong formulations for quadratic optimization with M-matrices and indicator variables

Abstract

We study quadratic optimization with indicator variables and an M-matrix, i.e., a PSD matrix with non-positive off-diagonal entries, which arises directly in image segmentation and portfolio optimization with transaction costs, as well as a substructure of general quadratic optimization problems. We prove, under mild assumptions, that the minimization problem is solvable in polynomial time by showing its equivalence to a submodular minimization problem. To strengthen the formulation, we decompose the quadratic function into a sum of simple quadratic functions with at most two indicator variables each, and provide the convex-hull descriptions of these sets. We also describe strong conic quadratic valid inequalities. Preliminary computational experiments indicate that the proposed inequalities can substantially improve the strength of the continuous relaxations with respect to the standard perspective reformulation.

This is a preview of subscription content, log in to check access.

Fig. 1

Notes

  1. 1.

    The root gap improvements of 95% achieved by Conic indicate that the approximation given in Sect. 5 is strong and considerably better than the natural continuous relaxation.

  2. 2.

    The matrices generated this way have only 20.1% of the off-diagonal entries negative on average – the rest are positive if \(\rho >0\) and 0 if \(\rho =0\). The ratio of the magnitude of the negative entries vs. the total, i.e., \(\frac{\sum _{i\ne j: A_{ij}<0}|A_{ij}|}{\sum _{i\ne j}|A_{ij}|}\), is on average 0.72 if \(\rho =0.1\), 0.57 if \(\rho =0.2\) and 0.34 if \(\rho =0.5\).

References

  1. 1.

    Ahuja, R.K., Hochbaum, D.S., Orlin, J.B.: A cut-based algorithm for the nonlinear dual of the minimum cost network flow problem. Algorithmica 39, 189–208 (2004)

    MathSciNet  Article  MATH  Google Scholar 

  2. 2.

    Aktürk, M.S., Atamtürk, A., Gürel, S.: A strong conic quadratic reformulation for machine-job assignment with controllable processing times. Oper. Res. Lett. 37, 187–191 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  3. 3.

    Anstreicher, K.M.: On convex relaxations for quadratically constrained quadratic programming. Math. Program. 136, 233–251 (2012)

    MathSciNet  Article  MATH  Google Scholar 

  4. 4.

    Atamtürk, A., Bhardwaj, A.: Network design with probabilistic capacities. Networks 71, 16–30 (2018)

    MathSciNet  Article  Google Scholar 

  5. 5.

    Atamtürk, A., Gomez, A.: Submodularity in conic quadratic mixed 0–1 optimization. BCOL Research Report 16.02, IEOR, UC Berkeley. arXiv preprint arXiv:1705.05918 (2017)

  6. 6.

    Atamtürk, A., Jeon, H.: Lifted polymatroid for mean-risk optimization with indicator variables. BCOL Research Report 17.01, IEOR, UC Berkeley. \(\text{arXiv}\,\,\text{ preprint }\) arXiv:1705.05915 (2017)

  7. 7.

    Atamtürk, A., Narayanan, V.: Cuts for conic mixed integer programming. In: Fischetti, M., Williamson, D.P. (eds.) Proceedings of the 12th International IPCO Conference, pp. 16–29 (2007)

  8. 8.

    Balas, E.: Disjunctive programming and a hierarchy of relaxations for discrete optimization problems. SIAM J. Algebr. Discrete Methods 6, 466–486 (1985)

    MathSciNet  Article  MATH  Google Scholar 

  9. 9.

    Belotti, P., Góez, J.C., Pólik, I., Ralphs, T.K., Terlaky, T.: A conic representation of the convex hull of disjunctive sets and conic cuts for integer second order cone optimization. In: Numerical Analysis and Optimization, pp. 1–35. Springer (2015)

  10. 10.

    Bertsimas, D., King, A., Mazumder, R.: Best subset selection via a modern optimization lens. Ann. Stat. 44, 813–852 (2016)

    MathSciNet  Article  MATH  Google Scholar 

  11. 11.

    Bienstock, D.: Computational study of a family of mixed-integer quadratic programming problems. Math. Program. 74, 121–140 (1996)

    MathSciNet  MATH  Google Scholar 

  12. 12.

    Bienstock, D., Michalka, A.: Cutting-planes for optimization of convex functions over nonconvex sets. SIAM J. Optim. 24, 643–677 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  13. 13.

    Boland, N., Dey, S.S., Kalinowski, T., Molinaro, M., Rigterink, F.: Bounding the gap between the McCormick relaxation and the convex hull for bilinear functions. Math. Program. 162, 523–535 (2017a)

    MathSciNet  Article  MATH  Google Scholar 

  14. 14.

    Boland, N., Gupte, A., Kalinowski, T., Rigterink, F., Waterer, H.: Extended formulations for convex hulls of graphs of bilinear functions. arXiv preprint arXiv:1702.04813 (2017b)

  15. 15.

    Bonami, P., Lodi, A., Tramontani, A., Wiese, S.: On mathematical programming with indicator constraints. Math. Program. 151, 191–223 (2015)

    MathSciNet  Article  MATH  Google Scholar 

  16. 16.

    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23, 1222–1239 (2001)

    Article  Google Scholar 

  17. 17.

    Ceria, S., Soares, J.: Convex programming for disjunctive convex optimization. Math. Program. 86, 595–614 (1999)

    MathSciNet  Article  MATH  Google Scholar 

  18. 18.

    Cornuejols, G., Tütüncü, R.: Optimization Methods in Finance, vol. 5. Cambridge University Press, Cambridge (2006)

    Google Scholar 

  19. 19.

    Dong, H., Linderoth, J.: On valid inequalities for quadratic programming with continuous variables and binary indicators. In: Goemans, M., Correa, J. (eds.) Proceedings of IPCO 2013, pp. 169–180. Springer, Berlin (2013)

  20. 20.

    Edmonds, J.: Submodular functions, matroids, and certain polyhedra. In: Guy, R., Hanani, H., Sauer, N., Schönenheim, J. (eds.) Combinatorial Structures and Their Applications, pp. 69–87. Gordon and Breach, Philadelphia (1970)

    Google Scholar 

  21. 21.

    Frangioni, A., Gentile, C.: Perspective cuts for a class of convex 0–1 mixed integer programs. Math. Program. 106, 225–236 (2006)

    MathSciNet  Article  MATH  Google Scholar 

  22. 22.

    Frangioni, A., Gentile, C., Hungerford, J.: Decompositions of semidefinite matrices and the perspective reformulation of nonseparable quadratic programs. Report R-16-10, IASI, Rome (2016)

  23. 23.

    Gao, J., Li, D.: Cardinality constrained linear-quadratic optimal control. IEEE Trans. Autom. Control 56, 1936–1941 (2011)

    MathSciNet  Article  MATH  Google Scholar 

  24. 24.

    Günlük, O., Linderoth, J.: Perspective reformulations of mixed integer nonlinear programs with indicator variables. Math. Program. 124, 183–205 (2010)

    MathSciNet  Article  MATH  Google Scholar 

  25. 25.

    Hijazi, H., Bonami, P., Cornuéjols, G., Ouorou, A.: Mixed-integer nonlinear programs featuring “on/off” constraints. Comput. Optim. Appl. 52, 537–558 (2012)

    MathSciNet  Article  MATH  Google Scholar 

  26. 26.

    Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms I: Fundamentals, vol. 305. Springer, Berlin (2013)

    Google Scholar 

  27. 27.

    Hochbaum, D.S.: Multi-label markov random fields as an efficient and effective tool for image segmentation, total variations and regularization. Numer. Math. Theory Methods Appl. 6, 169–198 (2013)

    MathSciNet  MATH  Google Scholar 

  28. 28.

    Ivănescu, P.L.: Some network flow problems solved with pseudo-boolean programming. Oper. Res. 13, 388–399 (1965)

    MathSciNet  Article  MATH  Google Scholar 

  29. 29.

    Jeon, H., Linderoth, J., Miller, A.: Quadratic cone cutting surfaces for quadratic programs with on–off constraints. Discrete Optim. 24, 32–50 (2017)

    MathSciNet  Article  MATH  Google Scholar 

  30. 30.

    Keilson, J., Styan, G.P.H.: Markov chains and M-matrices: inequalities and equalities. J. Math. Anal. Appl. 41, 439–459 (1973)

    MathSciNet  Article  MATH  Google Scholar 

  31. 31.

    Kılınç-Karzan, F., Yıldız, S.: Two-term disjunctions on the second-order cone. Math. Program. 154, 463–491 (2015)

    MathSciNet  Article  MATH  Google Scholar 

  32. 32.

    Kolmogorov, V., Zabin, R.: What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26, 147–159 (2004)

    Article  Google Scholar 

  33. 33.

    Lobo, M.S., Fazel, M., Boyd, S.: Portfolio optimization with linear and fixed transaction costs. Ann. Oper. Res. 152, 341–365 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  34. 34.

    Lovász, L.: Submodular functions and convexity. In: Bachem, A., Korte, B., Grötschel, M. (eds.) Mathematical Programming The State of the Art: Bonn 1982, pp. 235–257. Springer, Berlin (1983)

    Google Scholar 

  35. 35.

    Luedtke, J., Namazifar, M., Linderoth, J.: Some results on the strength of relaxations of multilinear functions. Math. Program. 136, 325–351 (2012)

    MathSciNet  Article  MATH  Google Scholar 

  36. 36.

    Luedtke, J., D’Ambrosio, C., Linderoth, J., Schweiger, J.: Strong convex nonlinear relaxations of the pooling problem. arXiv preprint arXiv:1803.02955 (2018)

  37. 37.

    Luk, F.T., Pagano, M.: Quadratic programming with M-matrices. Linear Algebra Appl. 33, 15–40 (1980)

    MathSciNet  Article  MATH  Google Scholar 

  38. 38.

    Mahajan, A., Leyffer, S., Linderoth, J., Luedtke, J., Munson, T.: Minotaur: A mixed-integer nonlinear optimization toolkit. ANL/MCS-P8010-0817, Argonne National Lab (2017)

  39. 39.

    Modaresi, S., Kılınç, M.R., Vielma, J.P.: Intersection cuts for nonlinear integer programming: convexification techniques for structured sets. Math. Program. 155, 575–611 (2016)

    MathSciNet  Article  MATH  Google Scholar 

  40. 40.

    Nemhauser, G.L., Wolsey, L.A., Fisher, M.L.: An analysis of approximations for maximizing submodular set functions I. Math. Program. 14, 265–294 (1978)

    MathSciNet  Article  MATH  Google Scholar 

  41. 41.

    Orlin, J.B.: A faster strongly polynomial time algorithm for submodular function minimization. Math. Program. 118, 237–251 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  42. 42.

    Picard, J.C., Ratliff, H.D.: Minimum cuts and related problems. Networks 5, 357–370 (1975)

    MathSciNet  Article  MATH  Google Scholar 

  43. 43.

    Plemmons, R.J.: M-matrix characterizations. I—nonsingular M-matrices. Linear Algebra Appl. 18, 175–188 (1977)

    MathSciNet  Article  MATH  Google Scholar 

  44. 44.

    Poljak, S., Wolkowicz, H.: Convex relaxations of (0,1)-quadratic programming. Math. Oper. Res. 20, 550–561 (1995)

    MathSciNet  Article  MATH  Google Scholar 

  45. 45.

    Stubbs, R.A., Mehrotra, S.: A branch-and-cut method for 0–1 mixed convex programming. Math. Program. 86, 515–532 (1999)

    MathSciNet  Article  MATH  Google Scholar 

  46. 46.

    Vielma, J.P.: Small and strong formulations for unions of convex sets from the cayley embedding. To appear in Mathematical Programming, arXiv preprint arXiv:1704.03954 (2018)

  47. 47.

    Wei, D., Sestok, C.K., Oppenheim, A.V.: Sparse filter design under a quadratic constraint: low-complexity algorithms. IEEE Trans. Signal Process. 61, 857–870 (2013)

    MathSciNet  Article  Google Scholar 

  48. 48.

    Wu, B., Sun, X., Li, D., Zheng, X.: Quadratic convex reformulations for semicontinuous quadratic programming. SIAM J. Optim. 27, 1531–1553 (2017)

    MathSciNet  Article  MATH  Google Scholar 

  49. 49.

    Young, N.: The rate of convergence of a matrix power series. Linear Algebra Appl. 35, 261–278 (1981)

    MathSciNet  Article  MATH  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Alper Atamtürk.

Additional information

A. Atamtürk was supported, in part, by Grant FA9550-10-1-0168 from the Office of the Assistant Secretary of Defense for Research and Engineering.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Atamtürk, A., Gómez, A. Strong formulations for quadratic optimization with M-matrices and indicator variables. Math. Program. 170, 141–176 (2018). https://doi.org/10.1007/s10107-018-1301-5

Download citation

Keywords

  • Quadratic optimization
  • Submodularity
  • Perspective formulation
  • Conic quadratic cuts
  • Convex piecewise nonlinear inequalities

Mathematics Subject Classification

  • 90C11
  • 90C20
  • 90C57