A Primal–Dual Smooth Perceptron–von Neumann Algorithm

Part of the Fields Institute Communications book series (FIC, volume 69)


We propose an elementary algorithm for solving a system of linear inequalities A T y>0 or its alternative Ax=0,x≥0,x≠0. Our algorithm is a smooth version of the perceptron and von Neumann algorithms. Our algorithm retains the simplicity of these algorithms but has a significantly improved convergence rate. Our approach also extends to more general conic systems provided a suitable smoothing oracle is available.

Key words

Perceptron algorithm von Neumann algorithm Condition number Smoothing technique 

Subject Classifications

90C05 90C25 90C52 


  1. 1.
    Alizadeh, F., Goldfarb, D.: Second-order cone programming. Math. Program. 95(1), 3–51 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Block, H.D.: The perceptron: a model for brain functioning. Rev. Mod. Phys. 34, 123–135 (1962)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Blum, L., Cucker, F., Shub, M., Smale, S.: Complexity and Real Computation.Springer, New York (1998)CrossRefGoogle Scholar
  4. 4.
    Burgisser, P., Cucker, F.: Condition. (Forthcoming).Google Scholar
  5. 5.
    Cheung, D., Cucker, F.: A new condition number for linear programming. Math. Program. 91, 163–174 (2001)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Dantzig, G.B.: An ε-precise feasible solution to a linear program with a convexity constraint in \({\frac{1} {\epsilon }^{2}}\) iterations independent of problem size. Technical Report, Stanford University (1992)Google Scholar
  7. 7.
    Dunagan, J., Vempala, S.: A simple polynomial-time rescaling algorithm for solving linear programs. In: Proceedings of the Thirty-Sixth Annual ACM Symposium on Theory of Computing, Chicago, ACM, pp. 315–320 (2004)Google Scholar
  8. 8.
    Dunagan, J., Vempala, S.: A simple polynomial-time rescaling algorithm for solving linear programs. Math. Program. 114(1), 101–114 (2006)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Epelman, M., Freund, R.M.: Condition number complexity of an elementary algorithm for computing a reliable solution of a conic linear system. Math. Program. 88, 451–485 (2000)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Freund, R., Vera, J.: Condition-based complexity of convex optimization in conic linear form via the ellipsoid algorithm. SIAM J. Optim. 10, 155–176 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Nemirovski, A.: Prox-method with rate of convergence \(\mathcal{O}(1/t)\) for variational inequalities with Lipschitz-continuous monotone operators and smooth convex-concave saddle point problems. SIAM J. Optim. 15(1), 229–251 (2004)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Nesterov, Y.: A method for unconstrained convex minimization problem with rate of convergence \(\mathcal{O}(1/{k}^{2})\). Doklady AN SSSR (in russian). (English translation. Sov. Math. Dokl.) 269, 543–547 (1983)Google Scholar
  13. 13.
    Nesterov, Y.: Excessive gap technique in nonsmooth convex minimization. SIAM J. Optim. 16(1), 235–249 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Novikoff, A.B.J.: On convergence proofs on perceptrons. In: Proceedings of the Symposium on the Mathematical Theory of Automata, New York, vol. XII, pp. 615–622 (1962)Google Scholar
  15. 15.
    Renegar, J.: Incorporating condition measures into the complexity theory of linear programming. SIAM J. Optim. 5, 506–524 (1995)MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Renegar, J.: Linear programming, complexity theory and elementary functional analysis. Math. Program. 70, 279–351 (1995)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Rosenblatt, F.: The perceptron: A probabilistic model for information storage and organization in the brain. Cornell Aeronautical Laboratory. Psychol. Rev. 65(6), 386–408 (1958)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Soheili, N., Peña, J.: A smooth perceptron algorithm. SIAM J. Optim. 22(2), 728-737 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Tseng, P.: On accelerated proximal gradient methods for convex-concave optimization. SIAM J. Optim., University of Washington (2008, Unpublished Manuscript)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  1. 1.Tepper School of BusinessCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations