Advertisement

Mathematical Programming

, Volume 152, Issue 1–2, pp 381–404 | Cite as

Universal gradient methods for convex optimization problems

  • Yu NesterovEmail author
Full Length Paper Series A

Abstract

In this paper, we present new methods for black-box convex minimization. They do not need to know in advance the actual level of smoothness of the objective function. Their only essential input parameter is the required accuracy of the solution. At the same time, for each particular problem class they automatically ensure the best possible rate of convergence. We confirm our theoretical results by encouraging numerical experiments, which demonstrate that the fast rate of convergence, typical for the smooth optimization problems, sometimes can be achieved even on nonsmooth problem instances.

Keywords

Convex optimization Black-box methods Complexity bounds Optimal methods Weakly smooth functions 

Mathematics Subject Classification (2000)

90C25 90C47 68Q25 

Notes

Acknowledgments

The author is very thankful to three anonymous referees for careful reading and many suggestions, which significantly improved the initial variant of the paper.

References

  1. 1.
    Babonneau, F., Nesterov, Yu., Vial, J.-P.: Design and operating of gas transmission networks. In: Operations Research, pp. 1–14, Feb. 2012. doi: 10.1287/opre.1110.1001
  2. 2.
    Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Devolder, O., Glineur, F., Nesterov, Yu.: First-order methods of smooth convex optimization with inexact oracle. Math. Program. (2013). doi: 10.1007/s10107-013-0677-5
  4. 4.
    Elster, K.-H. (ed.): Modern Mathematical Methods in Optimization. Academie Verlag, Berlin (1993)Google Scholar
  5. 5.
    Lan, G.: Bundle-level methods uniformly optimal for smooth and nonsmooth convex optimization. Math. Program. (2013). doi: 10.1007/s10107-013-0737-x
  6. 6.
    Lemarechal, C., Nemirovskii, A., Nesterov, Yu.: New variants of bundle methods. Math. Program. 69, 111–147 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Nemirovskii, A., Nesterov, Yu.: Optimal methods for smooth convex optimization. Zh. Vychisl. Mat. i Mat. Fiz. 25(3), 356–369 (1985). in RussianMathSciNetGoogle Scholar
  8. 8.
    Nemirovsky, A., Yudin, D.: Problem Complexity and Method Efficiency in Optimization. Wiley, New York (1983)Google Scholar
  9. 9.
    Nesterov, Yu.: Introductory Lectures on Convex Optimization. Kluwer, Boston (2004)CrossRefzbMATHGoogle Scholar
  10. 10.
    Nesterov, Yu.: Smooth minimization of non-smooth functions. Math. Program. A 103(1), 127–152 (2005)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Nesterov, Yu.: Gradient methods for minimizing composite functions. Math. Program. 140(1), 125–161 (2013)Google Scholar
  12. 12.
    Nesterov, Yu.: Primal-dual subgradient methods for convex problems. Math. Program. 120(1), 261–283 (2009)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2014

Authors and Affiliations

  1. 1.Center for Operations Research and Econometrics (CORE)Catholic University of Louvain (UCL)Louvain-la-NeuveBelgium

Personalised recommendations