Mathematical Programming

, Volume 126, Issue 2, pp 315–350 | Cite as

Smoothed analysis of condition numbers and complexity implications for linear programming

  • John Dunagan
  • Daniel A. Spielman
  • Shang-Hua Teng
Full Length Paper Series A

Abstract

We perform a smoothed analysis of Renegar’s condition number for linear programming by analyzing the distribution of the distance to ill-posedness of a linear program subject to a slight Gaussian perturbation. In particular, we show that for every n-by-d matrix Ā, n-vector \({\bar{\varvec b}}\), and d-vector \({\bar{\varvec c}}\) satisfying \({{||\bar{\bf A}, \bar{\varvec b}, \bar{\varvec c}||_F \leq 1}}\) and every σ ≤ 1,
$$\mathop{\bf E}\limits_{\bf A,\varvec b,\varvec c }{{[\log C (\bf A,\varvec b,\varvec c)} = O (\log (nd/\sigma)),}$$
where A, b and c are Gaussian perturbations of Ā, \({\bar{\varvec b}}\) and \({\bar{\varvec c}}\) of variance σ2 and C (A, b, c) is the condition number of the linear program defined by (A, b, c). From this bound, we obtain a smoothed analysis of interior point algorithms. By combining this with the smoothed analysis of finite termination of Spielman and Teng (Math. Prog. Ser. B, 2003), we show that the smoothed complexity of interior point algorithms for linear programming is O (n3log(nd/σ)).

Mathematics Subject Classification (2000)

90C05 90C51 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Abramowitz, M., Stegun, I.A. (eds.): Handbook of mathematical functions, 9th edn, volume 55 of Applied Mathematics Series. National Bureau of Standards (1970)Google Scholar
  2. 2.
    Anstreicher K.M.: Linear programming in O((n 3 / ln n)L) operations. SIAM J. Optim. 9, 803–812 (1999)MATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Anstreicher, K.M., Ji, J., Potra, F.A., Ye, Y.: Average performance of a self-dual interior-point algorithm for linear programming. In: Pardalos, P.M. (ed.) Complexity in Numerical Optimization, pp. 1–15. World Scientific Publishing Co., London (1993)Google Scholar
  4. 4.
    Anstreicher K.M., Ji J., Potra F.A., Ye Y.: Probabilistic analysis of an infeasible-interior-point algorithm for linear programming. Math. Oper. Res. 24(1), 176–192 (1999)MATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Ball K.: The reverse isoperimetric problem for Gaussian measure. Discret. Comput. Geom. 10(4), 411–420 (1993)MATHCrossRefGoogle Scholar
  6. 6.
    Bhattacharya, R.N., Rao, R.: Normal Approximation and Asymptotic Expansions. Wiley, New York (1976)Google Scholar
  7. 7.
    Blum, A., Dunagan, J.: Smoothed analysis of the perceptron algorithm for linear programming. In: Proceedings of the 13th ACM-SIAM Symposium on Discrete Algorithms (SODA ’02) (2002)Google Scholar
  8. 8.
    Cheung D., Cucker F.: A new condition number for linear programming. Math. Program. Ser. A 91(1), 163–174 (2001)MATHMathSciNetGoogle Scholar
  9. 9.
    Cheung C.W., Cucker F.: Probabilistic analysis of condition numbers for linear programming. J. Optim. Theory Appl. 114(1), 55–67 (2002)Google Scholar
  10. 10.
    Cucker F., Peña J.: A primal-dual algorithm for solving polyhedral conic systems with a finite-precision machine. SIAM J. Optim. 12(2), 522–554 (2001)MATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Cucker F., Wschebor M.: On the expected condition number of linear programming problems. Numer. Math. 94, 419–478 (2003)MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Cheung D., Cucker F., Hauser R.: Tail decay and moment estimates of a condition number for random linear conic systems. SIAM J. Optim. 15(4), 1237–1261 (2005)MATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Deza A., Nematollahi E., Terlaky T.: How good are interior point methods? Klee-Minty cubes tighten iteration-complexity bounds. Math. Program. 113(1), 1–14 (2008)MATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Dunagan, J., Vempala, S.: A simple polynomial-time rescaling algorithm for solving linear programs. In: Proceedings of the 36th Annual ACM Symposium on Theory of Computing, pp. 315–320 (2004)Google Scholar
  15. 15.
    Epelman M., Freund R.M.: Condition number complexity of an elementary algorithm for computing a reliable solution of a conic linear system. Math. Program. 88(3), 451–485 (2000)MATHMathSciNetGoogle Scholar
  16. 16.
    Feller, W.: An Introduction to Probability Theory and Its Applications, vol. 1. Wiley, New York (1968)Google Scholar
  17. 17.
    Freund R.: Complexity of convex optimization using geometry-based measures and a reference point. Math. Program. 99, 197–221 (2004)MATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Freund R., Vera J.: Condition-based complexity of convex optimization in conic linear form via the ellipsoid algorithm. SIAM J. Optim. 10(1), 155–176 (2000)CrossRefMathSciNetGoogle Scholar
  19. 19.
    Gonzaga, C.C.: An Algorithm for solving linear programming problems in O(n 3 L) operations. In: Megiddo, N. (ed.) Progress in Mathematical Programming, pp. 1–28. Spinger, Berlin (1988)Google Scholar
  20. 20.
    Gonzaga C.C., Todd M.J.: An \({{O(\sqrt{n}L)}}\)-iteration large-step primal-dual affine algorithm for linear programming. SIAM J. Optim. 2, 349–359 (1992)MATHCrossRefMathSciNetGoogle Scholar
  21. 21.
    Khachian, L.L.: A polynomial algorithm in linear programming. Doklady Adademiia Nauk SSSR 244, 1093–1096 (1979) (in Russian)Google Scholar
  22. 22.
    Grunbaum, B., Danzer, L., Klee, V.: Helly’s theorem and its relatives. In: Convexity. Proceedings of the Symposia on Pure Mathematics 7, pp. 101–180. American Mathematical Society, Providence (1963)Google Scholar
  23. 23.
    Helly, E.: Über Mengen konvexen Körper mit gemeinschaflichen Punkten. Jber. Deutsch. Math. Verein. 32, 175–176 (1923)Google Scholar
  24. 24.
    Huhn P., Borgwardt K.H.: Interior-point methods: worst case and average case analysis of a phase-I algorithm and a termination procedure. J. Complex. 18, 833–910 (2002)MATHCrossRefMathSciNetGoogle Scholar
  25. 25.
    Karmarkar N.: A new polynomial time algorithm for linear programming. Combinatorica 4, 373–395 (1984)MATHCrossRefMathSciNetGoogle Scholar
  26. 26.
    Khachiyan L.G.: A polynomial algorithm in linear programming. Doklady Akademia Nauk SSSR, pp. 1093–1096 (1979)Google Scholar
  27. 27.
    Lustig, I.J., Marsten, R.E., Shanno, D.F.: The primal-dual interior point method on the Cray supercomputer. In: Coleman, T.F., Li, Y. (eds.) Large-scale Numerical Optimization, Papers from the Workshop held at Cornell University, Ithaca, NY, USA, October 1989, volume 46 of SIAM Proceedings in Applied Mathematics, pp. 70–80. Society of Industrial and Applied Mathematics (SIAM), Philadelphia, USA (1990)Google Scholar
  28. 28.
    Mezaros, C., Xu, X., Andersen, E., Gondzio, J.: Implementation of interior point methods for large scale linear programming. In: Terlaky, T. (ed.) Interior Point Methods in Mathematical Programming. Kluwer Academic Publisher, Dordrecht (1996)Google Scholar
  29. 29.
    Mizuno S., Todd M.J., Ye Y.: On adaptive-step primal-dual interior-point algorithms for linear programming. Math. Oper. Res. 18, 964–981 (1993)MATHCrossRefMathSciNetGoogle Scholar
  30. 30.
    Nemirovskii, A.S.: An new polynomial algorithm for linear programming. Doklady Akademii Nauk SSSR 298(6), 1321–1325, 1988. Translated in: Soviet Mathematics Doklady 37(1), 264–269 (1988)Google Scholar
  31. 31.
    Nesterov, J.E., Nemirovskii, A.: Interior Point Polynomial Algorithms in Convex Programming. SIAM, Philadelphia (1994)Google Scholar
  32. 32.
    Peña J.: Understanding the geometry of infeasible perturbations of a conic linear system. SIAM J. Optim. 10, 534–550 (2000)MATHCrossRefMathSciNetGoogle Scholar
  33. 33.
    Renegar J.: Some perturbation theory for linear programming. Math. Program. 65(1, Ser. A), 73–91 (1994)MATHCrossRefMathSciNetGoogle Scholar
  34. 34.
    Renegar J.: Incorporating condition measures into the complexity theory of linear programming. SIAM J. Optim. 5(3), 506–524 (1995)MATHCrossRefMathSciNetGoogle Scholar
  35. 35.
    Renegar J.: Linear programming, complexity theory and elementary functional analysis. Math. Program. Ser. A 70(3), 279–351 (1995)MATHCrossRefMathSciNetGoogle Scholar
  36. 36.
    Sankar A., Spielman D.A., Teng S.-H.: Smoothed analysis of the condition numbers and growth factors of matrices. SIAM J. Matrix Anal. Appl. 28(2), 446–476 (2006)MATHCrossRefMathSciNetGoogle Scholar
  37. 37.
    Shanno D., Lustig I., Marsten R.: Interior point methods: computational state of the art. ORSA J. Comput. 6(1), 1–14 (1994)MATHMathSciNetGoogle Scholar
  38. 38.
    Spielman, D., Teng, S.-H.: Smoothed analysis: motivation and discrete models. In: Proceedings of WADS 2003, Lecture Notes in Computer Science 2748, pp. 256–270 (2003)Google Scholar
  39. 39.
    Spielman D.A., Teng S.-H.: Smoothed analysis of termination of linear programming algorithms. Math. Program. Ser. B 97, 375–404 (2003)MATHMathSciNetGoogle Scholar
  40. 40.
    Spielman D., Teng S.-H.: Smoothed analysis of algorithms: why the simplex algorithm usually takes polynomial time. J. ACM 51(3), 385–436 (2004)MATHCrossRefMathSciNetGoogle Scholar
  41. 41.
    Todd M.J.: Probabilistic models for linear programming. Math. Oper. Res. 16(4), 671–693 (1991)MATHCrossRefMathSciNetGoogle Scholar
  42. 42.
    Todd. M.J.: A lower bound on the number of iterations of primal-dual interior-point methods for linear programming. In: Watson, G.A., Griffiths, D.F. (eds.) Numerical Analysis 1993, pp. 237–259. Longman Press, Harlow (1994)Google Scholar
  43. 43.
    Todd M.J., Ye Y.: A lower bound on the number of iterations of long-step and polynomial interior-point methods for linear programming. Ann. Oper. Res. 62, 233–252 (1996)MATHCrossRefMathSciNetGoogle Scholar
  44. 44.
    Vaidya, P.M.: An algorithm for linear programming which requires O((m + n)n 2 + (m + n)1.5 nL) arithmetic operations. Math. Program. 47:175–201, 1990. Condensed version In: Proceedings of the 19th Annual ACM Symposium on Theory of Computing, pp. 29–38 (1987)Google Scholar
  45. 45.
    Vaidya, P.M.: Speeding-up linear programming using fast matrix multiplication. In: Proceedings of the 30th Annual Symposium on Foundations of Computer Science, pp. 332–337 (1989)Google Scholar
  46. 46.
    Vera, J.: Ill-posedness and the complexity of deciding existence of solutions to linear programs. SIAM J. Optim. 6(3) (1996)Google Scholar
  47. 47.
    Wright, S.: Primal-Dual Interior-Point Method. SIAM Publications, Philadelphia (1997)Google Scholar

Copyright information

© Springer and Mathematical Programming Society 2009

Authors and Affiliations

  • John Dunagan
    • 1
  • Daniel A. Spielman
    • 2
  • Shang-Hua Teng
    • 3
  1. 1.Microsoft ResearchBellevueUSA
  2. 2.Department of Computer Science, Program in Applied MathematicsYale UniversityNew HavenUSA
  3. 3.Department of Computer ScienceBoston UniversityBostonUSA

Personalised recommendations