Smooth minimization of non-smooth functions

Abstract.

In this paper we propose a new approach for constructing efficient schemes for non-smooth convex optimization. It is based on a special smoothing technique, which can be applied to functions with explicit max-structure. Our approach can be considered as an alternative to black-box minimization. From the viewpoint of efficiency estimates, we manage to improve the traditional bounds on the number of iterations of the gradient schemes from keeping basically the complexity of each iteration unchanged.

This is a preview of subscription content, access via your institution.

References

  1. 1.

    Ben-Tal, A., Nemirovskii, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. (SIAM, Philadelphia, 2001)

  2. 2.

    Bertsekas, D.P.: Constrained optimization and Lagrange multiplier methods. (Academic Press, New York, 1982)

  3. 3.

    Goffin, J.-L.: On the convergence rate of subgradient optimization methods. Mathematical Programming 13, 329–347 (1977)

    Google Scholar 

  4. 4.

    Hiriart-Urruty, J.-B., Lemarechal, C.: Convex Analysis and Minimization Algorithms. (Springer-Verlag, 1993)

  5. 5.

    Polyak, B.: On Bertsekas’ method for minimization of composite function. In: Bensoussan, A., Lions, J.L. (eds) Inter. Symp. Systems Opt. 1979, Analysis Springer, pp. 179–186

  6. 6.

    Polyak, R.: Nonlinear rescaling vs. smoothing technique in convex optimization. Mathematical Programming Ser. A 92, 197–235 (2002)

    Article  Google Scholar 

  7. 7.

    Polyak, B.: Introduction to Optimization. (Optimization Software, Inc., Publications Division, New York, 1987)

  8. 8.

    Nemirovsky, A., Yudin, D.: Informational Complexity and Efficient Methods for Solution of Convex Extremal Problems. (J. Wiley & Sons, New York, 1983)

  9. 9.

    Nesterov, Yu.: A method for unconstrained convex minimization problem with the rate of convergence Doklady AN SSSR (translated as Soviet Math. Docl.) 269, 543–547 (1983)

    Google Scholar 

  10. 10.

    Nesterov, Yu.: Introductory Lectures on Convex Optimization: Basic course. (Kluwer, Boston, 2003)

  11. 11.

    Shor, N.: Minimization Methods for Non-Differentiable Functions. (Springer-Verlag, Berlin, 1985)

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Yu. Nesterov.

Additional information

This paper presents research results of the Belgian Program on Interuniversity Poles of Attraction initiated by the Belgian State, Prime Minister’s Office, Science Policy Programming. The scientific responsibility is assumed by the author.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Nesterov, Y. Smooth minimization of non-smooth functions. Math. Program. 103, 127–152 (2005). https://doi.org/10.1007/s10107-004-0552-5

Download citation

Keywords

  • Non-smooth optimization
  • Convex optimization
  • Optimal methods
  • Complexity theory
  • Structural optimization