Skip to main content
View expanded cover

International Conference on Integer Programming and Combinatorial Optimization

IPCO 2017: Integer Programming and Combinatorial Optimization pp 267–278Cite as

An Improved Deterministic Rescaling for Linear Programming Algorithms

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 10328)

Abstract

The perceptron algorithm for linear programming, arising from machine learning, has been around since the 1950s. While not a polynomial-time algorithm, it is useful in practice due to its simplicity and robustness. In 2004, Dunagan and Vempala showed that a randomized rescaling turns the perceptron method into a polynomial time algorithm, and later Peña and Soheili gave a deterministic rescaling. In this paper, we give a deterministic rescaling for the perceptron algorithm that improves upon the previous rescaling methods by making it possible to rescale much earlier. This results in a faster running time for the rescaled perceptron algorithm. We will also demonstrate that the same rescaling methods yield a polynomial time algorithm based on the multiplicative weights update method. This draws a connection to an area that has received a lot of recent attention in theoretical computer science.

Keywords

  • Polynomial Time
  • Unit Ball
  • Convex Body
  • Gradient Descent
  • Symmetric Positive Definite Matrix

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

T. Rothvoss—Supported by an Alfred P. Sloan Research Fellowship. Both authors supported by NSF grant 1420180 with title “Limitations of convex relaxations in combinatorial optimization”.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-59250-3_22
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   64.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-59250-3
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   84.99
Price excludes VAT (USA)
Learn about institutional subscriptions

Notes

  1. 1.

    The \(\tilde{O}\)-notation suppresses any \({\text {polylog}}(m,n)\) terms.

  2. 2.

    It suffices here to consider the trivial example with \(\lambda _1=\ldots = \lambda _n = \frac{1}{n}\) and \(A_i = e_i\) being the standard basis. Then \(\Vert \sum _{i \in J} \lambda _i A_i\Vert _2 \le \frac{1}{\sqrt{n}}\) for any subset J. The optimality of our rescaling can also be seen since the cone in the last iteration is \(\tilde{O}(n)\)-well rounded, which is optimal up to \(\tilde{O}\)-terms.

  3. 3.

    Recall that a function \(F : {\mathbb {R}}^n \rightarrow {\mathbb {R}}\) is Lipschitz with Lipschitz constant 1 if \(|F(x)-F(y)| \le \Vert x-y\Vert _2\) for all \(x,y \in {\mathbb {R}}^n\). A famous concentration inequality by Sudakov, Tsirelson, Borell states that \(\Pr [|F(g)-\mu | \ge t] \le e^{-t^2/\pi ^2}\), where g is a random Gaussian and \(\mu \) is the mean of F under g.

References

  1. Agmon, S.: The relaxation method for linear inequalities. Can. J. Math. 6, 382–392 (1954)

    MathSciNet  CrossRef  MATH  Google Scholar 

  2. Arora, S., Hazan, E., Kale, S.: Fast algorithms for approximate semidefinite programming using the multiplicative weights update method. In: 46th IEEE FOCS, pp. 339–348 (2005)

    Google Scholar 

  3. Arora, S., Hazan, E., Kale, S.: The multiplicative weights update method: a meta-algorithm and applications. Theor. Comp. 8, 121–164 (2012)

    MathSciNet  CrossRef  MATH  Google Scholar 

  4. Alon, N., Spencer, J.H.: The Probabilistic Method. Wiley Series in Discrete Mathematics and Optimization. Wiley, Hoboken (2004)

    MATH  Google Scholar 

  5. Ball, K.: An elementary introduction to modern convex geometry. In: Silvio, L. (ed.) Flavors of Geometry, pp. 1–58. University Press, Cambridge (1997)

    Google Scholar 

  6. Betke, U.: Relaxation, new combinatorial and polynomial algorithms for the linear feasibility problem. Discrete Comput. Geom. 32(3), 317–338 (2004)

    MathSciNet  CrossRef  MATH  Google Scholar 

  7. Conforti, M., Cornuejols, G., Zambelli, G.: Integer Programming. Springer Publishing Company Inc., Heidelberg (2014)

    CrossRef  MATH  Google Scholar 

  8. Chubanov, S.: A strongly polynomial algorithm for linear systems having a binary solution. Math. Program. 134(2), 533–570 (2012)

    MathSciNet  CrossRef  MATH  Google Scholar 

  9. Chubanov, S.: A polynomial projection algorithm for linear feasibility problems. Math. Program. 153(2), 687–713 (2015)

    MathSciNet  CrossRef  MATH  Google Scholar 

  10. Christiano, P., Kelner, J.A., Madry, A., Spielman, D.A., Teng, S.: Electrical flows, laplacian systems, and faster approximation of maximum flow in undirected graphs. In: Proceedings of the 43rd ACM Symposium on Theory of Computing, New York, NY, USA, pp. 273–282 (2011)

    Google Scholar 

  11. Dantzig, G.B.: Maximization of a linear function of variables subject to linear inequalities. In: Activity Analysis of Production and Allocation, Cowles Commission Monograph, vol. 13, pp. 339–347. John Wiley & Sons Inc., Chapman & Hall Ltd., New York (1951)

    Google Scholar 

  12. Dunagan, J., Vempala, S.: A simple polynomial-time rescaling algorithm for solving linear programs. Math. Program. 114(1), 101–114 (2006)

    MathSciNet  CrossRef  MATH  Google Scholar 

  13. Dadush, D., Végh, L.A., Zambelli, G.: Rescaling algorithms for linear programming - part I: conic feasibility. CoRR, abs/1611.06427 (2016)

    Google Scholar 

  14. Garg, N., Könemann, J.: Faster and simpler algorithms for multicommodity flow and other fractional packing problems. SIAM J. Comput. 37(2), 630–652 (2007)

    MathSciNet  CrossRef  MATH  Google Scholar 

  15. Hačijan, L.G.: A polynomial algorithm in linear programming. Dokl. Akad. Nauk SSSR 244(5), 1093–1096 (1979)

    MathSciNet  MATH  Google Scholar 

  16. John, F.: Extremum problems with inequalities as subsidiary conditions. In: Friedrichs, K.O., Neugebauer, O.E., Stoker, J.J. (eds.) Studies and Essays presented to R. Courant on his 60th Birthday, pp. 187–204. Interscience Publishers, New York (1948)

    Google Scholar 

  17. Karmarkar, N.: A new polynomial-time algorithm for linear programming. Combinatorica 4(4), 373–395 (1984)

    MathSciNet  CrossRef  MATH  Google Scholar 

  18. Klee, V., Minty, G.: How good is the simplex algorithm? In: Inequalities, III (Proceedings Third Symposium, UCLA, 1969; Dedicated to the Memory of Theodore S. Motzkin), pp. 159–175. Academic Press, New York (1972)

    Google Scholar 

  19. Lee, Y., Sinford, A.: A new polynomial-time algorithm for linear programming (2015). https://arxiv.org/abs/1312.6677

  20. Madry, A.: Faster approximation schemes for fractional multicommodity flow problems via dynamic graph algorithms. In: Proceedings of the 42nd ACM Symposium on Theory of Computing, New York, NY, pp. 121–130 (2010)

    Google Scholar 

  21. Nesterov, Y.: Excessive gap technique in nonsmooth convex minimization. SIAM J. Optim. 16(1), 235–249 (2005)

    MathSciNet  CrossRef  MATH  Google Scholar 

  22. Peña, J., Soheili, N.: A smooth perceptron algorithm. SIAM J. Optim. 22(2), 728–737 (2012)

    MathSciNet  CrossRef  MATH  Google Scholar 

  23. Peña, J., Soheili, N.: A deterministic rescaled perceptron algorithm. Math. Program. 155(1–2), 497–510 (2016)

    MathSciNet  CrossRef  MATH  Google Scholar 

  24. Plotkin, S.A., Shmoys, D.B., Tardos, E.: Fast approximation algorithms for fractional packing and covering problems. Math. Oper. Res. 20(2), 257–301 (1995)

    MathSciNet  CrossRef  MATH  Google Scholar 

  25. Schrijver, A.: Theory of linear and integer programming. Wiley-Interscience Series in Discrete Mathematics. John Wiley and Sons, Inc., New York (1986)

    MATH  Google Scholar 

  26. Vazirani, V.: Approximation Algorithms. Springer, Heidelberg (2001)

    MATH  Google Scholar 

  27. Williamson, D.P., Shmoys, D.B.: The Design of Approximation Algorithms. University Press, Cambridge (2011)

    CrossRef  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Washington, Seattle, WA, 98105, USA

    Rebecca Hoberg & Thomas Rothvoss

Authors
  1. Rebecca Hoberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Rothvoss
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rebecca Hoberg .

Editor information

Editors and Affiliations

  1. Ecole Polytechnique Federale de Lausanne - EPFL, Lausanne, Switzerland

    Friedrich Eisenbrand

  2. University of Waterloo, Waterloo, Ontario, Canada

    Jochen Koenemann

Rights and permissions

Reprints and Permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Hoberg, R., Rothvoss, T. (2017). An Improved Deterministic Rescaling for Linear Programming Algorithms. In: Eisenbrand, F., Koenemann, J. (eds) Integer Programming and Combinatorial Optimization. IPCO 2017. Lecture Notes in Computer Science(), vol 10328. Springer, Cham. https://doi.org/10.1007/978-3-319-59250-3_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-59250-3_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-59249-7

  • Online ISBN: 978-3-319-59250-3

  • eBook Packages: Computer ScienceComputer Science (R0)