# A conjugate direction based simplicial decomposition framework for solving a specific class of dense convex quadratic programs

- 24 Downloads

## Abstract

Many real-world applications can usually be modeled as convex quadratic problems. In the present paper, we want to tackle a specific class of quadratic programs having a dense Hessian matrix and a structured feasible set. We hence carefully analyze a simplicial decomposition like algorithmic framework that handles those problems in an effective way. We introduce a new master solver, called Adaptive Conjugate Direction Method, and embed it in our framework. We also analyze the interaction of some techniques for speeding up the solution of the pricing problem. We report extensive numerical experiments based on a benchmark of almost 1400 instances from specific and generic quadratic problems. We show the efficiency and robustness of the method when compared to a commercial solver (Cplex).

## Keywords

Simplicial decomposition Convex quadratic programming Dense Hessian matrix Column generation## Mathematics Subject Classification

65K05 90C20 90C25## Notes

## Supplementary material

## References

- 1.Beasley, J.E.: Portfolio optimization data. http://people.brunel.ac.uk/~mastjjb/jeb/orlib/files/ (2016)
- 2.Bertsekas, D.P.: Convex Optimization Algorithms. Athena Scientific, Belmont (2015)zbMATHGoogle Scholar
- 3.Bertsekas, D.P., Yu, H.: A unifying polyhedral approximation framework for convex optimization. SIAM J. Optim.
**21**(1), 333–360 (2011)MathSciNetzbMATHCrossRefGoogle Scholar - 4.Birgin, E.G., Martínez, J.M., Raydan, M.: Nonmonotone spectral projected gradient methods on convex sets. SIAM J. Optim.
**10**(4), 1196–1211 (2000)MathSciNetzbMATHCrossRefGoogle Scholar - 5.Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)zbMATHCrossRefGoogle Scholar
- 6.Buchheim, C., Traversi, E.: Quadratic combinatorial optimization using separable underestimators. INFORMS J. Comput.
**30**(3), 424–437 (2018)MathSciNetCrossRefGoogle Scholar - 7.Cesarone, F., Tardella, F.: Portfolio datasets. http://host.uniroma3.it/docenti/cesarone/datasetsw3_tardella.html (2010)
- 8.Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Rev.
**43**(1), 129–159 (2001)MathSciNetzbMATHCrossRefGoogle Scholar - 9.Chu, P.C., Beasley, J.E.: A genetic algorithm for the multidimensional knapsack problem. J. Heuristics
**4**(1), 63–86 (1998)zbMATHCrossRefGoogle Scholar - 10.Clarkson, K.L.: Coresets, sparse greedy approximation, and the Frank-Wolfe algorithm. ACM Trans. Algorithms (TALG)
**6**(4), 63 (2010)MathSciNetzbMATHGoogle Scholar - 11.Condat, L.: Fast projection onto the simplex and the l1-ball. Math. Program.
**158**(1), 575–585 (2016)MathSciNetzbMATHCrossRefGoogle Scholar - 12.Cristofari, A.: An almost cyclic 2-coordinate descent method for singly linearly constrained problems. Comput. Optim. Appl.
**73**(2), 411–452 (2019)MathSciNetzbMATHCrossRefGoogle Scholar - 13.Cristofari, A., De Santis, M., Lucidi, S., Rinaldi, F.: A two-stage active-set algorithm for bound-constrained optimization. J. Optim. Theory Appl.
**172**(2), 369–401 (2017)MathSciNetzbMATHCrossRefGoogle Scholar - 14.Curtis, F.E., Han, Z., Robinson, D.P.: A globally convergent primal-dual active-set framework for large-scale convex quadratic optimization. Comput. Optim. Appl.
**60**(2), 311–341 (2015)MathSciNetzbMATHCrossRefGoogle Scholar - 15.De Santis, M., Di Pillo, G., Lucidi, S.: An active set feasible method for large-scale minimization problems with bound constraints. Comput. Optim. Appl.
**53**(2), 395–423 (2012)MathSciNetzbMATHCrossRefGoogle Scholar - 16.Demetrescu, C., Goldberg, A.V., Johnson, D.S.: The Shortest Path Problem: Ninth DIMACS Implementation Challenge, vol. 74. American Mathematical Soc., Providence (2009)zbMATHCrossRefGoogle Scholar
- 17.Desaulniers, G., Desrosiers, J., Solomon, M.M.: Column Generation, vol. 5. Springer, Berlin (2006)zbMATHGoogle Scholar
- 18.Djerdjour, M., Mathur, K., Salkin, H.: A surrogate relaxation based algorithm for a general quadratic multi-dimensional knapsack problem. Oper. Res. Lett.
**7**(5), 253–258 (1988)MathSciNetzbMATHCrossRefGoogle Scholar - 19.Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program.
**91**(2), 201–213 (2002)MathSciNetzbMATHCrossRefGoogle Scholar - 20.Drake, J.: Benchmark instances for the multidimensional knapsack problem (2015)Google Scholar
- 21.Elzinga, J., Moore, T.G.: A central cutting plane algorithm for the convex programming problem. Math. Program.
**8**(1), 134–145 (1975)MathSciNetzbMATHCrossRefGoogle Scholar - 22.Ferreau, H.J., Kirches, C., Potschka, A., Bock, H.G., Diehl, M.: qpoases: a parametric active-set algorithm for quadratic programming. Math. Program. Comput.
**6**(4), 327–363 (2014)MathSciNetzbMATHCrossRefGoogle Scholar - 23.Furini, F., Traversi, E., Belotti, P., Frangioni, A., Gleixner, A., Gould, N., Liberti, L., Lodi, A., Misener, R., Mittelmann, H., Sahinidis, N., Vigerske, S., Wiegele, A.: Qplib: a library of quadratic programming instances. Math. Program. Comput.
**11**, 237–265 (2018)MathSciNetCrossRefGoogle Scholar - 24.Glover, F., Kochenberger, G.: Critical event tabu search for multidimensional knapsack problems. In: Meta-Heuristics, pp. 407–427. Springer (1996)Google Scholar
- 25.Glover, F., Kochenberger, G., Alidaee, B., Amini, M.: Solving quadratic knapsack problems by reformulation and tabu search: single constraint case. In: Combinatorial and Global Optimization, pp. 111–121. World Scientific (2002)Google Scholar
- 26.Goffin, J.L., Gondzio, J., Sarkissian, R., Vial, J.P.: Solving nonlinear multicommodity flow problems by the analytic center cutting plane method. Math. Program.
**76**(1), 131–154 (1997)MathSciNetzbMATHCrossRefGoogle Scholar - 27.Goffin, J.L., Vial, J.P.: Cutting planes and column generation techniques with the projective algorithm. J. Optim. Theory Appl.
**65**(3), 409–429 (1990)MathSciNetzbMATHCrossRefGoogle Scholar - 28.Goffin, J.L., Vial, J.P.: On the computation of weighted analytic centers and dual ellipsoids with the projective algorithm. Math. Program.
**60**(1), 81–92 (1993)MathSciNetzbMATHCrossRefGoogle Scholar - 29.Gondzio, J.: Interior point methods 25 years later. Eur. J. Oper. Res.
**218**(3), 587–601 (2012)MathSciNetzbMATHCrossRefGoogle Scholar - 30.Gondzio, J., González-Brevis, P.: A new warmstarting strategy for the primal-dual column generation method. Math. Program.
**152**(1–2), 113–146 (2015)MathSciNetzbMATHCrossRefGoogle Scholar - 31.Gondzio, J., González-Brevis, P., Munari, P.: New developments in the primal-dual column generation technique. Eur. J. Oper. Res.
**224**(1), 41–51 (2013)MathSciNetzbMATHCrossRefGoogle Scholar - 32.Gondzio, J., González-Brevis, P., Munari, P.: Large-scale optimization with the primal–dual column generation method. Math. Program. Comput.
**8**(1), 47–82 (2016)MathSciNetzbMATHCrossRefGoogle Scholar - 33.Gondzio, J., Kouwenberg, R.: High-performance computing for asset-liability management. Oper. Res.
**49**(6), 879–891 (2001)MathSciNetzbMATHCrossRefGoogle Scholar - 34.Gondzio, J., du Merle, O., Sarkissian, R., Vial, J.P.: Accpm—a library for convex optimization based on an analytic center cutting plane method. Eur. J. Oper. Res.
**94**(1), 206–211 (1996)CrossRefGoogle Scholar - 35.Gondzio, J., Sarkissian, R., Vial, J.P.: Using an interior point method for the master problem in a decomposition approach. Eur. J. Oper. Res.
**101**(3), 577–587 (1997)zbMATHCrossRefGoogle Scholar - 36.Gondzio, J., Vial, J.P., et al.: Warm start and -subgradients in a cutting plane scheme for block-angular linear programs. Comput. Optim. Appl.
**14**, 17–36 (1999)MathSciNetzbMATHCrossRefGoogle Scholar - 37.Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for newton’s method. SIAM J. Numer. Anal.
**23**(4), 707–716 (1986)MathSciNetzbMATHCrossRefGoogle Scholar - 38.Grippo, L., Lampariello, F., Lucidi, S.: A truncated newton method with nonmonotone line search for unconstrained optimization. J. Optim. Theory Appl.
**60**(3), 401–419 (1989)MathSciNetzbMATHCrossRefGoogle Scholar - 39.Grippo, L., Lampariello, F., Lucidi, S.: A class of nonmonotone stabilization methods in unconstrained optimization. Numer. Math.
**59**(1), 779–805 (1991)MathSciNetzbMATHCrossRefGoogle Scholar - 40.Hager, W.W., Zhang, H.: A new active set algorithm for box constrained optimization. SIAM J. Optim.
**17**(2), 526–557 (2006)MathSciNetzbMATHCrossRefGoogle Scholar - 41.Hearn, D.W., Lawphongpanich, S., Ventura, J.A.: Restricted simplicial decomposition: computation and extensions. In: Computation Mathematical Programming, pp. 99–118 (1987)Google Scholar
- 42.Holloway, C.A.: An extension of the Frank and Wolfe method of feasible directions. Math. Program.
**6**(1), 14–27 (1974)MathSciNetzbMATHCrossRefGoogle Scholar - 43.IBM: Cplex (version 12.6.3). https://www-01.ibm.com/software/commerce/optimization/cplex-optimizer/ (2015)
- 44.Kiwiel, K.C.: Methods of Descent for Nondifferentiable Optimization, vol. 1133. Springer, Berlin (2006)zbMATHGoogle Scholar
- 45.Levin, A.Y.: On an algorithm for the minimization of convex functions. Sov. Math. Dokl.
**160**, 1244–1247 (1965)Google Scholar - 46.Markowitz, H.: Portfolio selection. J. Finance
**7**(1), 77–91 (1952)Google Scholar - 47.Michelot, C.: A finite algorithm for finding the projection of a point onto the canonical simplex of \({\mathbb{R}}^n\). J. Optim. Theory Appl.
**50**(1), 195–200 (1986)MathSciNetzbMATHCrossRefGoogle Scholar - 48.Munari, P., Gondzio, J.: Using the primal–dual interior point algorithm within the branch-price-and-cut method. Comput. Oper. Res.
**40**(8), 2026–2036 (2013)MathSciNetzbMATHCrossRefGoogle Scholar - 49.Nesterov, Y., Nemirovskii, A.: Interior-Point Polynomial Algorithms in Convex Programming. SIAM, Philadelphia (1994)zbMATHCrossRefGoogle Scholar
- 50.Newman, D.J.: Location of the maximum on unimodal surfaces. J. ACM (JACM)
**12**(3), 395–398 (1965)MathSciNetzbMATHCrossRefGoogle Scholar - 51.Nocedal, J., Wright, S.J.: Conjugate gradient methods. In: Numerical Optimization, pp. 101–134 (2006)Google Scholar
- 52.Nocedal, J., Wright, S.J.: Sequential Quadratic Programming. Springer, Berlin (2006)Google Scholar
- 53.Palomar, D.P., Eldar, Y.C.: Convex Optimization in Signal Processing and Communications. Cambridge University Press, Cambridge (2010)zbMATHGoogle Scholar
- 54.Patriksson, M.: The Traffic Assignment Problem: Models and Methods. Courier Dover Publications, Mineola (2015)Google Scholar
- 55.Rostami, B., Chassein, A., Hopf, M., Frey, D., Buchheim, C., Malucelli, F., Goerigk, M.: The quadratic shortest path problem: complexity, approximability, and solution methods. Eur. J. Oper. Res.
**268**(2), 473–485 (2018)MathSciNetzbMATHCrossRefGoogle Scholar - 56.Rostami, B., Malucelli, F., Frey, D., Buchheim, C.: On the quadratic shortest path problem. In: International Symposium on Experimental Algorithms, pp. 379–390. Springer (2015)Google Scholar
- 57.Syam, S.S.: A dual ascent method for the portfolio selection problem with multiple constraints and linked proposals. Eur. J. Oper. Res.
**108**(1), 196–207 (1998)zbMATHCrossRefGoogle Scholar - 58.Tarasov, S., Khachiian, L., Erlikh, I.: The method of inscribed ellipsoids. Dokl. Akad. Nauk SSSR
**298**(5), 1081–1085 (1988)MathSciNetGoogle Scholar - 59.Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodol.)
**58**(1), 267–288 (1996)MathSciNetzbMATHGoogle Scholar - 60.Ventura, J.A., Hearn, D.W.: Restricted simplicial decomposition for convex constrained problems. Math. Program.
**59**(1), 71–85 (1993)MathSciNetzbMATHCrossRefGoogle Scholar - 61.Von Hohenbalken, B.: Simplicial decomposition in nonlinear programming algorithms. Math. Program.
**13**(1), 49–68 (1977)MathSciNetzbMATHCrossRefGoogle Scholar - 62.WolframAlpha: Mathematica (version 11.3). http://www.wolfram.com/mathematica/ (2018)
- 63.Wright, M.: The interior-point revolution in optimization: history, recent developments, and lasting consequences. Bull. Am. Math. Soc.
**42**(1), 39–56 (2005)MathSciNetzbMATHCrossRefGoogle Scholar - 64.Wright, S.J.: Primal–Dual Interior-Point Methods. SIAM, Philadelphia (1997)zbMATHCrossRefGoogle Scholar
- 65.Ye, Y.: Interior Point Algorithms: Theory and Analysis, vol. 44. Wiley, Hoboken (2011)Google Scholar