Advertisement

An Improved Deterministic Rescaling for Linear Programming Algorithms

  • Rebecca Hoberg
  • Thomas Rothvoss
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10328)

Abstract

The perceptron algorithm for linear programming, arising from machine learning, has been around since the 1950s. While not a polynomial-time algorithm, it is useful in practice due to its simplicity and robustness. In 2004, Dunagan and Vempala showed that a randomized rescaling turns the perceptron method into a polynomial time algorithm, and later Peña and Soheili gave a deterministic rescaling. In this paper, we give a deterministic rescaling for the perceptron algorithm that improves upon the previous rescaling methods by making it possible to rescale much earlier. This results in a faster running time for the rescaled perceptron algorithm. We will also demonstrate that the same rescaling methods yield a polynomial time algorithm based on the multiplicative weights update method. This draws a connection to an area that has received a lot of recent attention in theoretical computer science.

Keywords

Polynomial Time Unit Ball Convex Body Gradient Descent Symmetric Positive Definite Matrix 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. [Agm54]
    Agmon, S.: The relaxation method for linear inequalities. Can. J. Math. 6, 382–392 (1954)MathSciNetCrossRefMATHGoogle Scholar
  2. [AHK05]
    Arora, S., Hazan, E., Kale, S.: Fast algorithms for approximate semidefinite programming using the multiplicative weights update method. In: 46th IEEE FOCS, pp. 339–348 (2005)Google Scholar
  3. [AHK12]
    Arora, S., Hazan, E., Kale, S.: The multiplicative weights update method: a meta-algorithm and applications. Theor. Comp. 8, 121–164 (2012)MathSciNetCrossRefMATHGoogle Scholar
  4. [AS04]
    Alon, N., Spencer, J.H.: The Probabilistic Method. Wiley Series in Discrete Mathematics and Optimization. Wiley, Hoboken (2004)MATHGoogle Scholar
  5. [Bal97]
    Ball, K.: An elementary introduction to modern convex geometry. In: Silvio, L. (ed.) Flavors of Geometry, pp. 1–58. University Press, Cambridge (1997)Google Scholar
  6. [Bet04]
    Betke, U.: Relaxation, new combinatorial and polynomial algorithms for the linear feasibility problem. Discrete Comput. Geom. 32(3), 317–338 (2004)MathSciNetCrossRefMATHGoogle Scholar
  7. [CCZ14]
    Conforti, M., Cornuejols, G., Zambelli, G.: Integer Programming. Springer Publishing Company Inc., Heidelberg (2014)CrossRefMATHGoogle Scholar
  8. [Chu12]
    Chubanov, S.: A strongly polynomial algorithm for linear systems having a binary solution. Math. Program. 134(2), 533–570 (2012)MathSciNetCrossRefMATHGoogle Scholar
  9. [Chu15]
    Chubanov, S.: A polynomial projection algorithm for linear feasibility problems. Math. Program. 153(2), 687–713 (2015)MathSciNetCrossRefMATHGoogle Scholar
  10. [CKM+11]
    Christiano, P., Kelner, J.A., Madry, A., Spielman, D.A., Teng, S.: Electrical flows, laplacian systems, and faster approximation of maximum flow in undirected graphs. In: Proceedings of the 43rd ACM Symposium on Theory of Computing, New York, NY, USA, pp. 273–282 (2011)Google Scholar
  11. [Dan51]
    Dantzig, G.B.: Maximization of a linear function of variables subject to linear inequalities. In: Activity Analysis of Production and Allocation, Cowles Commission Monograph, vol. 13, pp. 339–347. John Wiley & Sons Inc., Chapman & Hall Ltd., New York (1951)Google Scholar
  12. [DV06]
    Dunagan, J., Vempala, S.: A simple polynomial-time rescaling algorithm for solving linear programs. Math. Program. 114(1), 101–114 (2006)MathSciNetCrossRefMATHGoogle Scholar
  13. [DVZ16]
    Dadush, D., Végh, L.A., Zambelli, G.: Rescaling algorithms for linear programming - part I: conic feasibility. CoRR, abs/1611.06427 (2016)Google Scholar
  14. [GK07]
    Garg, N., Könemann, J.: Faster and simpler algorithms for multicommodity flow and other fractional packing problems. SIAM J. Comput. 37(2), 630–652 (2007)MathSciNetCrossRefMATHGoogle Scholar
  15. [Hač79]
    Hačijan, L.G.: A polynomial algorithm in linear programming. Dokl. Akad. Nauk SSSR 244(5), 1093–1096 (1979)MathSciNetMATHGoogle Scholar
  16. [Joh48]
    John, F.: Extremum problems with inequalities as subsidiary conditions. In: Friedrichs, K.O., Neugebauer, O.E., Stoker, J.J. (eds.) Studies and Essays presented to R. Courant on his 60th Birthday, pp. 187–204. Interscience Publishers, New York (1948)Google Scholar
  17. [Kar84]
    Karmarkar, N.: A new polynomial-time algorithm for linear programming. Combinatorica 4(4), 373–395 (1984)MathSciNetCrossRefMATHGoogle Scholar
  18. [KM72]
    Klee, V., Minty, G.: How good is the simplex algorithm? In: Inequalities, III (Proceedings Third Symposium, UCLA, 1969; Dedicated to the Memory of Theodore S. Motzkin), pp. 159–175. Academic Press, New York (1972)Google Scholar
  19. [LS15]
    Lee, Y., Sinford, A.: A new polynomial-time algorithm for linear programming (2015). https://arxiv.org/abs/1312.6677
  20. [Mad10]
    Madry, A.: Faster approximation schemes for fractional multicommodity flow problems via dynamic graph algorithms. In: Proceedings of the 42nd ACM Symposium on Theory of Computing, New York, NY, pp. 121–130 (2010)Google Scholar
  21. [Nes05]
    Nesterov, Y.: Excessive gap technique in nonsmooth convex minimization. SIAM J. Optim. 16(1), 235–249 (2005)MathSciNetCrossRefMATHGoogle Scholar
  22. [PS12]
    Peña, J., Soheili, N.: A smooth perceptron algorithm. SIAM J. Optim. 22(2), 728–737 (2012)MathSciNetCrossRefMATHGoogle Scholar
  23. [PS16]
    Peña, J., Soheili, N.: A deterministic rescaled perceptron algorithm. Math. Program. 155(1–2), 497–510 (2016)MathSciNetCrossRefMATHGoogle Scholar
  24. [PST95]
    Plotkin, S.A., Shmoys, D.B., Tardos, E.: Fast approximation algorithms for fractional packing and covering problems. Math. Oper. Res. 20(2), 257–301 (1995)MathSciNetCrossRefMATHGoogle Scholar
  25. [Sch86]
    Schrijver, A.: Theory of linear and integer programming. Wiley-Interscience Series in Discrete Mathematics. John Wiley and Sons, Inc., New York (1986)MATHGoogle Scholar
  26. [Vaz01]
    Vazirani, V.: Approximation Algorithms. Springer, Heidelberg (2001)MATHGoogle Scholar
  27. [WS11]
    Williamson, D.P., Shmoys, D.B.: The Design of Approximation Algorithms. University Press, Cambridge (2011)CrossRefMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.University of WashingtonSeattleUSA

Personalised recommendations