Skip to main content
SpringerLink
Log in

Smoothed analysis of condition numbers and complexity implications for linear programming

  • Full Length Paper
  • Series A
  • Published:
Mathematical Programming Submit manuscript

Abstract

We perform a smoothed analysis of Renegar’s condition number for linear programming by analyzing the distribution of the distance to ill-posedness of a linear program subject to a slight Gaussian perturbation. In particular, we show that for every n-by-d matrix Ā, n-vector \({\bar{\varvec b}}\), and d-vector \({\bar{\varvec c}}\) satisfying \({{||\bar{\bf A}, \bar{\varvec b}, \bar{\varvec c}||_F \leq 1}}\) and every σ ≤ 1,

$$\mathop{\bf E}\limits_{\bf A,\varvec b,\varvec c }{{[\log C (\bf A,\varvec b,\varvec c)} = O (\log (nd/\sigma)),}$$

where A, b and c are Gaussian perturbations of Ā, \({\bar{\varvec b}}\) and \({\bar{\varvec c}}\) of variance σ 2 and C (A, b, c) is the condition number of the linear program defined by (A, b, c). From this bound, we obtain a smoothed analysis of interior point algorithms. By combining this with the smoothed analysis of finite termination of Spielman and Teng (Math. Prog. Ser. B, 2003), we show that the smoothed complexity of interior point algorithms for linear programming is O (n 3log(nd/σ)).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abramowitz, M., Stegun, I.A. (eds.): Handbook of mathematical functions, 9th edn, volume 55 of Applied Mathematics Series. National Bureau of Standards (1970)

  2. Anstreicher K.M.: Linear programming in O((n 3 / ln n)L) operations. SIAM J. Optim. 9, 803–812 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  3. Anstreicher, K.M., Ji, J., Potra, F.A., Ye, Y.: Average performance of a self-dual interior-point algorithm for linear programming. In: Pardalos, P.M. (ed.) Complexity in Numerical Optimization, pp. 1–15. World Scientific Publishing Co., London (1993)

  4. Anstreicher K.M., Ji J., Potra F.A., Ye Y.: Probabilistic analysis of an infeasible-interior-point algorithm for linear programming. Math. Oper. Res. 24(1), 176–192 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  5. Ball K.: The reverse isoperimetric problem for Gaussian measure. Discret. Comput. Geom. 10(4), 411–420 (1993)

    Article  MATH  Google Scholar 

  6. Bhattacharya, R.N., Rao, R.: Normal Approximation and Asymptotic Expansions. Wiley, New York (1976)

  7. Blum, A., Dunagan, J.: Smoothed analysis of the perceptron algorithm for linear programming. In: Proceedings of the 13th ACM-SIAM Symposium on Discrete Algorithms (SODA ’02) (2002)

  8. Cheung D., Cucker F.: A new condition number for linear programming. Math. Program. Ser. A 91(1), 163–174 (2001)

    MATH  MathSciNet  Google Scholar 

  9. Cheung C.W., Cucker F.: Probabilistic analysis of condition numbers for linear programming. J. Optim. Theory Appl. 114(1), 55–67 (2002)

    Google Scholar 

  10. Cucker F., Peña J.: A primal-dual algorithm for solving polyhedral conic systems with a finite-precision machine. SIAM J. Optim. 12(2), 522–554 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  11. Cucker F., Wschebor M.: On the expected condition number of linear programming problems. Numer. Math. 94, 419–478 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  12. Cheung D., Cucker F., Hauser R.: Tail decay and moment estimates of a condition number for random linear conic systems. SIAM J. Optim. 15(4), 1237–1261 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  13. Deza A., Nematollahi E., Terlaky T.: How good are interior point methods? Klee-Minty cubes tighten iteration-complexity bounds. Math. Program. 113(1), 1–14 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  14. Dunagan, J., Vempala, S.: A simple polynomial-time rescaling algorithm for solving linear programs. In: Proceedings of the 36th Annual ACM Symposium on Theory of Computing, pp. 315–320 (2004)

  15. Epelman M., Freund R.M.: Condition number complexity of an elementary algorithm for computing a reliable solution of a conic linear system. Math. Program. 88(3), 451–485 (2000)

    MATH  MathSciNet  Google Scholar 

  16. Feller, W.: An Introduction to Probability Theory and Its Applications, vol. 1. Wiley, New York (1968)

  17. Freund R.: Complexity of convex optimization using geometry-based measures and a reference point. Math. Program. 99, 197–221 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  18. Freund R., Vera J.: Condition-based complexity of convex optimization in conic linear form via the ellipsoid algorithm. SIAM J. Optim. 10(1), 155–176 (2000)

    Article  MathSciNet  Google Scholar 

  19. Gonzaga, C.C.: An Algorithm for solving linear programming problems in O(n 3 L) operations. In: Megiddo, N. (ed.) Progress in Mathematical Programming, pp. 1–28. Spinger, Berlin (1988)

  20. Gonzaga C.C., Todd M.J.: An \({{O(\sqrt{n}L)}}\)-iteration large-step primal-dual affine algorithm for linear programming. SIAM J. Optim. 2, 349–359 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  21. Khachian, L.L.: A polynomial algorithm in linear programming. Doklady Adademiia Nauk SSSR 244, 1093–1096 (1979) (in Russian)

    Google Scholar 

  22. Grunbaum, B., Danzer, L., Klee, V.: Helly’s theorem and its relatives. In: Convexity. Proceedings of the Symposia on Pure Mathematics 7, pp. 101–180. American Mathematical Society, Providence (1963)

  23. Helly, E.: Über Mengen konvexen Körper mit gemeinschaflichen Punkten. Jber. Deutsch. Math. Verein. 32, 175–176 (1923)

  24. Huhn P., Borgwardt K.H.: Interior-point methods: worst case and average case analysis of a phase-I algorithm and a termination procedure. J. Complex. 18, 833–910 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  25. Karmarkar N.: A new polynomial time algorithm for linear programming. Combinatorica 4, 373–395 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  26. Khachiyan L.G.: A polynomial algorithm in linear programming. Doklady Akademia Nauk SSSR, pp. 1093–1096 (1979)

  27. Lustig, I.J., Marsten, R.E., Shanno, D.F.: The primal-dual interior point method on the Cray supercomputer. In: Coleman, T.F., Li, Y. (eds.) Large-scale Numerical Optimization, Papers from the Workshop held at Cornell University, Ithaca, NY, USA, October 1989, volume 46 of SIAM Proceedings in Applied Mathematics, pp. 70–80. Society of Industrial and Applied Mathematics (SIAM), Philadelphia, USA (1990)

  28. Mezaros, C., Xu, X., Andersen, E., Gondzio, J.: Implementation of interior point methods for large scale linear programming. In: Terlaky, T. (ed.) Interior Point Methods in Mathematical Programming. Kluwer Academic Publisher, Dordrecht (1996)

  29. Mizuno S., Todd M.J., Ye Y.: On adaptive-step primal-dual interior-point algorithms for linear programming. Math. Oper. Res. 18, 964–981 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  30. Nemirovskii, A.S.: An new polynomial algorithm for linear programming. Doklady Akademii Nauk SSSR 298(6), 1321–1325, 1988. Translated in: Soviet Mathematics Doklady 37(1), 264–269 (1988)

  31. Nesterov, J.E., Nemirovskii, A.: Interior Point Polynomial Algorithms in Convex Programming. SIAM, Philadelphia (1994)

  32. Peña J.: Understanding the geometry of infeasible perturbations of a conic linear system. SIAM J. Optim. 10, 534–550 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  33. Renegar J.: Some perturbation theory for linear programming. Math. Program. 65(1, Ser. A), 73–91 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  34. Renegar J.: Incorporating condition measures into the complexity theory of linear programming. SIAM J. Optim. 5(3), 506–524 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  35. Renegar J.: Linear programming, complexity theory and elementary functional analysis. Math. Program. Ser. A 70(3), 279–351 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  36. Sankar A., Spielman D.A., Teng S.-H.: Smoothed analysis of the condition numbers and growth factors of matrices. SIAM J. Matrix Anal. Appl. 28(2), 446–476 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  37. Shanno D., Lustig I., Marsten R.: Interior point methods: computational state of the art. ORSA J. Comput. 6(1), 1–14 (1994)

    MATH  MathSciNet  Google Scholar 

  38. Spielman, D., Teng, S.-H.: Smoothed analysis: motivation and discrete models. In: Proceedings of WADS 2003, Lecture Notes in Computer Science 2748, pp. 256–270 (2003)

  39. Spielman D.A., Teng S.-H.: Smoothed analysis of termination of linear programming algorithms. Math. Program. Ser. B 97, 375–404 (2003)

    MATH  MathSciNet  Google Scholar 

  40. Spielman D., Teng S.-H.: Smoothed analysis of algorithms: why the simplex algorithm usually takes polynomial time. J. ACM 51(3), 385–436 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  41. Todd M.J.: Probabilistic models for linear programming. Math. Oper. Res. 16(4), 671–693 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  42. Todd. M.J.: A lower bound on the number of iterations of primal-dual interior-point methods for linear programming. In: Watson, G.A., Griffiths, D.F. (eds.) Numerical Analysis 1993, pp. 237–259. Longman Press, Harlow (1994)

  43. Todd M.J., Ye Y.: A lower bound on the number of iterations of long-step and polynomial interior-point methods for linear programming. Ann. Oper. Res. 62, 233–252 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  44. Vaidya, P.M.: An algorithm for linear programming which requires O((m + n)n 2 + (m + n)1.5 nL) arithmetic operations. Math. Program. 47:175–201, 1990. Condensed version In: Proceedings of the 19th Annual ACM Symposium on Theory of Computing, pp. 29–38 (1987)

  45. Vaidya, P.M.: Speeding-up linear programming using fast matrix multiplication. In: Proceedings of the 30th Annual Symposium on Foundations of Computer Science, pp. 332–337 (1989)

  46. Vera, J.: Ill-posedness and the complexity of deciding existence of solutions to linear programs. SIAM J. Optim. 6(3) (1996)

  47. Wright, S.: Primal-Dual Interior-Point Method. SIAM Publications, Philadelphia (1997)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Microsoft Research, Bellevue, WA, USA

    John Dunagan

  2. Department of Computer Science, Program in Applied Mathematics, Yale University, New Haven, CT, USA

    Daniel A. Spielman

  3. Department of Computer Science, Boston University, Boston, MA, USA

    Shang-Hua Teng

Authors
  1. John Dunagan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel A. Spielman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shang-Hua Teng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel A. Spielman.

Additional information

This work was supported in part by NSF Grants CCR-9875024, CCR-9972532, CCR-0112487, CCR-0325630, and CCF-0707522, and fellowships from the Alfred P. Sloan Foundation. Part of this work was done while John Dunagan and Daniel A. Spielman were at MIT.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dunagan, J., Spielman, D.A. & Teng, SH. Smoothed analysis of condition numbers and complexity implications for linear programming. Math. Program. 126, 315–350 (2011). https://doi.org/10.1007/s10107-009-0278-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-009-0278-5

Mathematics Subject Classification (2000)

Search

Navigation