Advertisement

An Improved Deterministic Rescaling for Linear Programming Algorithms

  • Rebecca Hoberg
  • Thomas Rothvoss
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10328)

Abstract

The perceptron algorithm for linear programming, arising from machine learning, has been around since the 1950s. While not a polynomial-time algorithm, it is useful in practice due to its simplicity and robustness. In 2004, Dunagan and Vempala showed that a randomized rescaling turns the perceptron method into a polynomial time algorithm, and later Peña and Soheili gave a deterministic rescaling. In this paper, we give a deterministic rescaling for the perceptron algorithm that improves upon the previous rescaling methods by making it possible to rescale much earlier. This results in a faster running time for the rescaled perceptron algorithm. We will also demonstrate that the same rescaling methods yield a polynomial time algorithm based on the multiplicative weights update method. This draws a connection to an area that has received a lot of recent attention in theoretical computer science.

Keywords

Polynomial Time Unit Ball Convex Body Gradient Descent Symmetric Positive Definite Matrix 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. [Agm54]
    Agmon, S.: The relaxation method for linear inequalities. Can. J. Math. 6, 382–392 (1954)MathSciNetCrossRefzbMATHGoogle Scholar
  2. [AHK05]
    Arora, S., Hazan, E., Kale, S.: Fast algorithms for approximate semidefinite programming using the multiplicative weights update method. In: 46th IEEE FOCS, pp. 339–348 (2005)Google Scholar
  3. [AHK12]
    Arora, S., Hazan, E., Kale, S.: The multiplicative weights update method: a meta-algorithm and applications. Theor. Comp. 8, 121–164 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  4. [AS04]
    Alon, N., Spencer, J.H.: The Probabilistic Method. Wiley Series in Discrete Mathematics and Optimization. Wiley, Hoboken (2004)zbMATHGoogle Scholar
  5. [Bal97]
    Ball, K.: An elementary introduction to modern convex geometry. In: Silvio, L. (ed.) Flavors of Geometry, pp. 1–58. University Press, Cambridge (1997)Google Scholar
  6. [Bet04]
    Betke, U.: Relaxation, new combinatorial and polynomial algorithms for the linear feasibility problem. Discrete Comput. Geom. 32(3), 317–338 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  7. [CCZ14]
    Conforti, M., Cornuejols, G., Zambelli, G.: Integer Programming. Springer Publishing Company Inc., Heidelberg (2014)CrossRefzbMATHGoogle Scholar
  8. [Chu12]
    Chubanov, S.: A strongly polynomial algorithm for linear systems having a binary solution. Math. Program. 134(2), 533–570 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  9. [Chu15]
    Chubanov, S.: A polynomial projection algorithm for linear feasibility problems. Math. Program. 153(2), 687–713 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  10. [CKM+11]
    Christiano, P., Kelner, J.A., Madry, A., Spielman, D.A., Teng, S.: Electrical flows, laplacian systems, and faster approximation of maximum flow in undirected graphs. In: Proceedings of the 43rd ACM Symposium on Theory of Computing, New York, NY, USA, pp. 273–282 (2011)Google Scholar
  11. [Dan51]
    Dantzig, G.B.: Maximization of a linear function of variables subject to linear inequalities. In: Activity Analysis of Production and Allocation, Cowles Commission Monograph, vol. 13, pp. 339–347. John Wiley & Sons Inc., Chapman & Hall Ltd., New York (1951)Google Scholar
  12. [DV06]
    Dunagan, J., Vempala, S.: A simple polynomial-time rescaling algorithm for solving linear programs. Math. Program. 114(1), 101–114 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  13. [DVZ16]
    Dadush, D., Végh, L.A., Zambelli, G.: Rescaling algorithms for linear programming - part I: conic feasibility. CoRR, abs/1611.06427 (2016)Google Scholar
  14. [GK07]
    Garg, N., Könemann, J.: Faster and simpler algorithms for multicommodity flow and other fractional packing problems. SIAM J. Comput. 37(2), 630–652 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  15. [Hač79]
    Hačijan, L.G.: A polynomial algorithm in linear programming. Dokl. Akad. Nauk SSSR 244(5), 1093–1096 (1979)MathSciNetzbMATHGoogle Scholar
  16. [Joh48]
    John, F.: Extremum problems with inequalities as subsidiary conditions. In: Friedrichs, K.O., Neugebauer, O.E., Stoker, J.J. (eds.) Studies and Essays presented to R. Courant on his 60th Birthday, pp. 187–204. Interscience Publishers, New York (1948)Google Scholar
  17. [Kar84]
    Karmarkar, N.: A new polynomial-time algorithm for linear programming. Combinatorica 4(4), 373–395 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  18. [KM72]
    Klee, V., Minty, G.: How good is the simplex algorithm? In: Inequalities, III (Proceedings Third Symposium, UCLA, 1969; Dedicated to the Memory of Theodore S. Motzkin), pp. 159–175. Academic Press, New York (1972)Google Scholar
  19. [LS15]
    Lee, Y., Sinford, A.: A new polynomial-time algorithm for linear programming (2015). https://arxiv.org/abs/1312.6677
  20. [Mad10]
    Madry, A.: Faster approximation schemes for fractional multicommodity flow problems via dynamic graph algorithms. In: Proceedings of the 42nd ACM Symposium on Theory of Computing, New York, NY, pp. 121–130 (2010)Google Scholar
  21. [Nes05]
    Nesterov, Y.: Excessive gap technique in nonsmooth convex minimization. SIAM J. Optim. 16(1), 235–249 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  22. [PS12]
    Peña, J., Soheili, N.: A smooth perceptron algorithm. SIAM J. Optim. 22(2), 728–737 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  23. [PS16]
    Peña, J., Soheili, N.: A deterministic rescaled perceptron algorithm. Math. Program. 155(1–2), 497–510 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  24. [PST95]
    Plotkin, S.A., Shmoys, D.B., Tardos, E.: Fast approximation algorithms for fractional packing and covering problems. Math. Oper. Res. 20(2), 257–301 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  25. [Sch86]
    Schrijver, A.: Theory of linear and integer programming. Wiley-Interscience Series in Discrete Mathematics. John Wiley and Sons, Inc., New York (1986)zbMATHGoogle Scholar
  26. [Vaz01]
    Vazirani, V.: Approximation Algorithms. Springer, Heidelberg (2001)zbMATHGoogle Scholar
  27. [WS11]
    Williamson, D.P., Shmoys, D.B.: The Design of Approximation Algorithms. University Press, Cambridge (2011)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.University of WashingtonSeattleUSA

Personalised recommendations