pp 1–53 | Cite as

A Multiplicative Weight Updates Algorithm for Packing and Covering Semi-infinite Linear Programs

  • Khaled Elbassioni
  • Kazuhisa Makino
  • Waleed NajyEmail author


We consider the following semi-infinite linear programming problems: \(\max \) (resp., \(\min \)) \(c^Tx\) s.t. \(y^TA_ix+(d^i)^Tx \le b_i\) (resp., \(y^TA_ix+(d^i)^Tx \ge b_i)\), for all \(y \in {{\mathcal {Y}}}_i\), for \(i=1,\ldots ,N\), where \({{\mathcal {Y}}}_i\subseteq {\mathbb {R}}^m_+\) are given compact convex sets and \(A_i\in {\mathbb {R}}^{m_i\times n}_+\), \(b=(b_1,\ldots b_N)\in {\mathbb {R}}_+^N\), \(d^i\in {\mathbb {R}}_+^n\), and \(c\in {\mathbb {R}}_+^n\) are given non-negative matrices and vectors. This general framework is useful in modeling many interesting problems. For example, it can be used to represent a sub-class of robust optimization in which the coefficients of the constraints are drawn from convex uncertainty sets \({{\mathcal {Y}}}_i\), and the goal is to optimize the objective function for the worst case choice in each \({{\mathcal {Y}}}_i\). When the uncertainty sets \({{\mathcal {Y}}}_i\) are ellipsoids, we obtain a sub-class of second-order cone programming. We show how to extend the multiplicative weights update method to derive approximation schemes for the above packing and covering problems. When the sets \({{\mathcal {Y}}}_i\) are simple, such as ellipsoids or boxes, this yields substantial improvements in running time over general convex programming solvers. We also consider the mixed packing/covering problem, in which both packing and covering constraints are given, and the objective is to find an approximately feasible solution.


Multiplicative weights update Robust optimization Second-order cone programming Packing and covering 


Supplementary material


  1. 1.
    Alizadeh, F., Goldfarb, D.: Second-order cone programming. Math. Program. 95(1), 3–51 (2003)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Allen-Zhu, Z., Lee, Y.T., Orecchia, L.: Using optimization to obtain a width-independent, parallel, simpler, and faster positive SDP solver, pp. 1824–1831 (2016).
  3. 3.
    Allen-Zhu, Z., Orecchia, L.: Nearly-linear time positive LP solver with faster convergence rate, pp. 229–236 (2015).
  4. 4.
    Arora, S., Hazan, E., Kale, S.: Fast algorithms for approximate semidefinite programming using the multiplicative weights update method. In: Proceedings of the 46th Symposium on Foundations of Computer Science (FOCS), pp. 339–348 (2005)Google Scholar
  5. 5.
    Arora, S., Kale, S.: A combinatorial, primal–dual approach to semidefinite programs. In: Proceedings of the 39th Symposium on Theory of Computing (STOC), pp. 227–236 (2007)Google Scholar
  6. 6.
    Bartal, Y., Byers, J., Raz, D.: Fast, distributed approximation algorithms for positive linear programming with applications to flow control. SIAM J. Comput. 33(6), 1261–1279 (2004)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Beck, A., Teboulle, M.: Mirror descent and nonlinear projected subgradient methods for convex optimization. Oper. Res. Lett. 31(3), 167–175 (2003). MathSciNetzbMATHGoogle Scholar
  8. 8.
    Ben-Tal, A., Hazan, E., Koren, T., Mannor, S.: Oracle-based robust optimization via online learning. Oper. Res. 63(3), 628–638 (2015)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Ben-Tal, A., Nemirovski, A.: Robust optimization: methodology and applications. Math. Program. 92(3), 453–480 (2002)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Bertsimas, D., Brown, D.B., Caramanis, C.: Theory and applications of robust optimization. SIAM Rev. 53(3), 464–501 (2011)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Bertsimas, D., Thiele, A.: A robust optimization approach to supply chain management. In: Integer Programming and Combinatorial Optimization, 10th International IPCO Conference, New York, NY, USA, 7–11 June 2004, Proceedings, pp. 86–100 (2004)Google Scholar
  12. 12.
    Bhalgat, A., Gollapudi, S., Munagala, K.: Optimal auctions via the multiplicative weight method. In: Proceedings of the Fourteenth ACM Conference on Electronic Commerce, EC ’13, pp. 73–90. ACM, New York (2013)Google Scholar
  13. 13.
    Blum, A.: On-line algorithms in machine learning. In: Developments from a June 1996 Seminar on Online Algorithms: The State of the Art, pp. 306–325. Springer, London (1998)Google Scholar
  14. 14.
    Brönnimann, H., Goodrich, M.T.: Almost optimal set covers in finite VC-dimension. Discrete Comput. Geom. 14(4), 463–479 (1995)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Brown, G.W.: Iterative solution of games by fictitious play. Activity Anal. Prod. Allocation 13(1), 374–376 (1951)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Bubeck, S.: Convex optimization: algorithms and complexity. Found. Trends Mach. Learn. 8(3–4), 231–357 (2015). zbMATHGoogle Scholar
  17. 17.
    Chau, C.K., Elbassioni, K., Khonji, M.: Truthful mechanisms for combinatorial ac electric power allocation. In: Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems, pp. 1005–1012. International Foundation for Autonomous Agents and Multiagent Systems (2014)Google Scholar
  18. 18.
    Chazelle, B.: The Discrepancy Method: Randomness and Complexity. Cambridge University Press, New York (2000)zbMATHGoogle Scholar
  19. 19.
    Constantine, C., Mannor, S., Xu, H.: Robust Optimization in Machine Learning, pp. 369–402. MIT Press, Cambridge (2012)Google Scholar
  20. 20.
    Daskalakis, C., Deckelbaum, A., Kim, A.: Near-optimal no-regret algorithms for zero-sum games. In: Proceedings of the Twenty-Second Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’11, pp. 235–254. SIAM (2011)Google Scholar
  21. 21.
    Diedrich, F., Jansen, K.: Faster and simpler approximation algorithms for mixed packing and covering problems. Theor. Comput. Sci. 377(1–3), 182–204 (2007)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Elbassioni, K., Makino, K., Mehlhorn, K., Ramezani, F.: On randomized fictitious play for approximating saddle points over convex sets. Algorithmica 73(2), 441–459 (2015). MathSciNetzbMATHGoogle Scholar
  23. 23.
    Elbassioni, K., Nguyen, T.T.: Approximation schemes for binary quadratic programming problems with low CP-rank decompositions. arXiv preprint arXiv:1411.5050 (2014)
  24. 24.
    Freund, Y., Schapire, R.: Adaptive game playing using multiplicative weights. Games Econ. Behav. 29(1–2), 79–103 (1999)MathSciNetzbMATHGoogle Scholar
  25. 25.
    Garg, N., Khandekar, R.: Fractional covering with upper bounds on the variables: solving LPs with negative entries. In: Proceedings of the 14th European Symposium on Algorithms (ESA), pp. 371–382 (2004)Google Scholar
  26. 26.
    Garg, N., Könemann, J.: Faster and simpler algorithms for multicommodity flow and other fractional packing problems. In: Proceedings of the 39th Symposium on Foundations of Computer Science (FOCS), pp. 300–309 (1998)Google Scholar
  27. 27.
    Goldfarb, D., Iyengar, G.: Robust portfolio selection problems. Math. Oper. Res. 28(1), 1–38 (2003)MathSciNetzbMATHGoogle Scholar
  28. 28.
    Grant, M., Boyd, S.: Graph implementations for nonsmooth convex programs. In: Blondel, V., Boyd, S., Kimura, H. (eds.) Recent Advances in Learning and Control. Lecture Notes in Control and Information Sciences, pp. 95–110. Springer, Berlin (2008)Google Scholar
  29. 29.
    Grant, M., Boyd, S.: CVX: Matlab software for disciplined convex programming, version 2.1 (2014)Google Scholar
  30. 30.
    Grigoriadis, M., Khachiyan, L.: Approximate solution of matrix games in parallel. In: Advances in Optimization and Parallel Computing, pp. 129–136 (1992)Google Scholar
  31. 31.
    Grigoriadis, M., Khachiyan, L.: A sublinear-time randomized approximation algorithm for matrix games. Oper. Res. Lett. 18(2), 53–58 (1995)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Grigoriadis, M., Khachiyan, L.: Coordination complexity of parallel price-directive de-composition. Math. Oper. Res. 21(2), 321–340 (1996)MathSciNetzbMATHGoogle Scholar
  33. 33.
    Grigoriadis, M.D., Khachiyan, L.G., Porkolab, L., Villavicencio, J.: Approximate max–min resource sharing for structured concave optimization. SIAM J. Optim. 41, 1081–1091 (2001)MathSciNetzbMATHGoogle Scholar
  34. 34.
    Hazan, E.: Efficient algorithms for online convex optimization and their application. Ph.D. thesis, Princeton University, USA (2006)Google Scholar
  35. 35.
    Helmbold, D.P., Schapire, R.E., Singer, Y., Warmuth, M.K.: On-line portfolio selection using multiplicative updates. In: Machine Learning, Proceedings of the Thirteenth International Conference (ICML ’96), Bari, Italy, 3–6 July 1996, pp. 243–251 (1996)Google Scholar
  36. 36.
    Jain, R., Yao, P.: A parallel approximation algorithm for positive semidefinite programming. In: IEEE 52nd Annual Symposium on Foundations of Computer Science, FOCS 2011, Palm Springs, CA, USA, 22–25 October 2011, pp. 463–471 (2011)Google Scholar
  37. 37.
    Kale, S.: Efficient algorithms using the multiplicative weights update method. Ph.D. thesis, Princeton University, USA (2007)Google Scholar
  38. 38.
    Khandekar, R.: Lagrangian relaxation based algorithms for convex programming problems. Ph.D. thesis, Indian Institute of Technology, Delhi (2004)Google Scholar
  39. 39.
    Koufogiannakis, C., Young, N.: Beating simplex for fractional packing and covering linear programs. In: Proceedings of the 48th Symposium on Foundations of Computer Science (FOCS), pp. 494–504 (2007)Google Scholar
  40. 40.
    Littlestone, N., Warmuth, M.: The weighted majority algorithm. Inf. Comput. 108(2), 212–261 (1994)MathSciNetzbMATHGoogle Scholar
  41. 41.
    Lobo, M.S., Vandenberghe, L., Boyd, S., Lebret, H.: Applications of second-order cone programming. Linear Algebra Appl. 284(13), 193–228 (1998)MathSciNetzbMATHGoogle Scholar
  42. 42.
    López, M., Still, G.: Semi-infinite programming. Eur. J. Oper. Res. 180(2), 491–518 (2007)MathSciNetzbMATHGoogle Scholar
  43. 43.
    Lorenz, R., Boyd, S.: Robust minimum variance beamforming. IEEE Trans. Signal Process. 53(5), 1684–1696 (2005)MathSciNetzbMATHGoogle Scholar
  44. 44.
    Lovász, L., Vempala, S.: Fast algorithms for logconcave functions: sampling, rounding, integration and optimization. In: Proceedings of the 47th Symposium on Foundations of Computer Science (FOCS), pp. 57–68 (2006)Google Scholar
  45. 45.
    Luby, M., Nisan, N.: A parallel approximation algorithm for positive linear programming. In: Proceedings of the 25th Symposium on Theory of Computing (STOC), pp. 448–457 (1993)Google Scholar
  46. 46.
    Magaril-Il’yaev, G.G., Tikhomirov, V.M.: Convex Analysis: Theory and Applications, vol. 222. American Mathematical Society, Providence (2003)zbMATHGoogle Scholar
  47. 47.
    Nemirovskii, A., Yudin, D.: Problem Complexity and Method Efficiency in Optimization. A Wiley-Interscience Publication. Wiley, London (1983)Google Scholar
  48. 48.
    Nesterov, Y.: A method of solving a convex programming problem with convergence rate O(1/sqr(k)). Sov. Math. Dokl. 27, 372–376 (1983)zbMATHGoogle Scholar
  49. 49.
    Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course, vol. 87. Springer, Berlin (2013)zbMATHGoogle Scholar
  50. 50.
    Peng, R., Tangwongsan, K.: Faster and simpler width-independent parallel algorithms for positive semidefinite programming. In: Proceedings of the Twenty-Fourth Annual ACM Symposium on Parallelism in Algorithms and Architectures, SPAA ’12, pp. 101–108. ACM, New York (2012)Google Scholar
  51. 51.
    Plotkin, S., Shmoys, D., Tardos, É.: Fast approximation algorithms for fractional packing and covering problems. In: Proceedings of the 32nd Symposium on Foundations of Computer Science (FOCS), pp. 495–504 (1991)Google Scholar
  52. 52.
    Robinson, J.: An iterative method of solving a game. Ann. Math. 54(2), 296–301 (1951)MathSciNetzbMATHGoogle Scholar
  53. 53.
    Shivaswamy, P.K., Bhattacharyya, C., Smola, A.J.: Second order cone programming approaches for handling missing and uncertain data. J. Mach. Learn. Res. 7, 1283–1314 (2006)MathSciNetzbMATHGoogle Scholar
  54. 54.
    Tsang, I.W., Kwok, J.T.: Efficient hyperkernel learning using second-order cone programming. IEEE Trans. Neural Netw. 17(1), 48–58 (2006)zbMATHGoogle Scholar
  55. 55.
    Tulabandhula, T., Rudin, C.: Robust optimization using machine learning for uncertainty sets. In: International Symposium on Artificial Intelligence and Mathematics, ISAIM 2014, Fort Lauderdale, FL, USA, 6–8 January 2014 (2014)Google Scholar
  56. 56.
    Wang, D., Rao, S., Mahoney, M.W.: Unified acceleration method for packing and covering problems via diameter reduction. arXiv preprint arXiv:1508.02439 (2015)
  57. 57.
    Young, N.: Sequential and parallel algorithms for mixed packing and covering. In: Proceedings of the 42nd Symposium on Foundations of Computer Science (FOCS), pp. 538–546 (2001)Google Scholar
  58. 58.
    Zass, R., Shashua, A.: A unifying approach to hard and probabilistic clustering. In: Tenth IEEE International Conference on Computer Vision, 2005. ICCV 2005, vol. 1, pp. 294–301. IEEE (2005)Google Scholar
  59. 59.
    Zhu, Z.A., Orecchia, L.: A novel, simple interpretation of Nesterov’s accelerated method as a combination of gradient and mirror descent. CoRR arXiv:1407.1537 (2014)

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Khaled Elbassioni
    • 1
  • Kazuhisa Makino
    • 2
  • Waleed Najy
    • 3
    Email author
  1. 1.Masdar InstituteKhalifa University of Science and TechnologyAbu DhabiUAE
  2. 2.Research Institute for Mathematical Sciences (RIMS)Kyoto UniversityKyotoJapan
  3. 3.New York UniversityAbu DhabiUAE

Personalised recommendations