Skip to main content
SpringerLink
Log in

Convergence Analysis of Some Methods for Minimizing a Nonsmooth Convex Function

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

In this paper, we analyze a class of methods for minimizing a proper lower semicontinuous extended-valued convex function \(f:\Re^{\mathfrak{n}} \to \Re \cup {\infty}\). Instead of the original objective function f, we employ a convex approximation f k + 1 at the kth iteration. Some global convergence rate estimates are obtained. We illustrate our approach by proposing (i) a new family of proximal point algorithms which possesses the global convergence rate estimate \(f\left( {x_k } \right) - \min _{x \in \Re ^n } f\left( x \right) = O\left( {1/\left( {\Sigma _{j = 0}^{k - 1} \sqrt {\lambda _j } } \right)^2 } \right)\) even it the iteration points are calculated approximately, where \({\lambda_k}_{k = 0}^\infty\) are the proximal parameters, and (ii) a variant proximal bundle method. Applications to stochastic programs are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Moreau, J. J., Proximité et Dualité dans un Espace Hilbertien, Bulletin de la Societé Mathematique de France, Vol. 93, pp. 273–299, 1965.

    Google Scholar 

  2. Martinet, B., Regularisation d'Inéquations Variationelles par Approximations Successives, Revenue Francaise d'Informatique et de Recherche Operationelle, Vol. 4, pp. 154–159, 1970.

    Google Scholar 

  3. Rockafellar, R. T., Monotone Operators and Proximal Point Algorithm, SIAM Journal on Control and Optimization, Vol. 14, pp. 877–898, 1976.

    Google Scholar 

  4. Bertsekas, D. P., and Tseng, P., Partial Proximal Minimization Algorithms for Convex Programming, SIAM Journal on Optimization, Vol. 4, pp. 551–572, 1994.

    Google Scholar 

  5. Chen, G., and Teboulle, M., A Proximal-Based Decomposition Method for Convex Minimization Problems, Mathematical Programming, Vol. 64, pp. 81–101, 1994.

    Google Scholar 

  6. Eckstein, J., and Bertsekas, D. P., On the Douglas-Rachford Splitting Method and the Proximal Point Algorithm for Maximal Monotone Operators, Mathematical Programming, Vol. 55, pp. 293–318, 1992.

    Google Scholar 

  7. Fukushima, M., A Descent Algorithm for Nonsmooth Convex Programming, Mathematical Programming, Vol. 30, pp. 163–175, 1984.

    Google Scholar 

  8. Fukushima, M., and Qi, L., A Globally and Superlinearly Convergent Algorithm for Nonsmooth Convex Minimization, SIAM Journal on Optimization, Vol. 6, pp. 463–472, 1996.

    Google Scholar 

  9. GÜler, O., On the Convergence of the Proximal Point Algorithm for Convex Minimization, SIAM Journal on Control and Optimization, Vol. 29, pp. 403–419, 1991.

    Google Scholar 

  10. GÜler, O., New Proximal Point Algorithms for Convex Minimization, SIAM Journal on Optimization, Vol. 4, pp. 649–664, 1994.

    Google Scholar 

  11. Rockafellar, R. T., Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming, Mathematics of Operations Research, Vol. 1, pp. 97–116, 1976.

    Google Scholar 

  12. Zhu, C., Modified Proximal Point Algorithm for Extended Linear-Quadratic Programming, Computational Optimization and Applications, Vol. 1, pp. 185–206, 1992.

    Google Scholar 

  13. Nesterov, Y. E., On an Approach to the Construction of the Optimal Methods of Minimization of Smooth Convex Functions, Ekonomicheskie i Matematicheskie Metody, Vol. 24, pp. 509–517, 1988.

    Google Scholar 

  14. LemarÉchal, C., Nonsmooth Optimization and Descent Methods, Research Report RR-78-4, International Institute of Applied Systems Analysis, Laxenburg, Austria, 1977.

    Google Scholar 

  15. Kiwiel, K. C., Methods of Descent for Nondifferentiable Optimization, Lecture Notes in Mathematics, Springer, Berlin, Germany, Vol. 1133, 1985.

    Google Scholar 

  16. LemarÉchal, C., Nondifferentiable Optimization, Handbooks in Operations Research and Management Science, Edited by G. L. Nemhauser, A. H. G. Rinnooy Kan, and M. J. Todd, North Holland, Amsterdam, Holland, Vol. 1, pp. 529–572, 1989.

    Google Scholar 

  17. Hiriart-Urruty, J., and LemarÉchal, C., Convex Analysis and Minimization Algorithms, Vols. 1–2, Springer Verlag, Berlin, Germany, 1993.

    Google Scholar 

  18. Auslender, A., Numerical Methods for Nondifferentiable Convex Optimization, Mathematical Programming Study, Vol. 30, pp. 102–126, 1986.

    Google Scholar 

  19. Correa, R., and LemarÉchal, C., Convergence of Some Algorithms for Convex Minimization, Mathematical Programming, Vol. 62, pp. 261–275, 1993.

    Google Scholar 

  20. Kiwiel, K. C., An Aggregate Subgradient Method for Nonsmooth Convex Minimization, Mathematical Programming, Vol. 27, pp. 320–341, 1983.

    Google Scholar 

  21. Kiwiel, K. C., Proximity Control in Bundle Methods for Convex Nondifferentiable Minimization, Mathematical Programming, Vol. 46, pp. 105–122, 1990.

    Google Scholar 

  22. LemarÉchal, C., Constructing Bundle Methods for Convex Optimization, Fermat Days 85: Mathematics for Optimization, Edited by J. B. Hiriart-Urruty, North Holland, Amsterdam, Holland, pp. 201–240, 1986.

    Google Scholar 

  23. LemarÉchal, C., and Mifflin, R., A Globally and Superlinearly Convergent Algorithm for One-Dimensional Minimization of Convex Functions, Mathematical Programming, Vol. 24, pp. 241–256, 1982.

    Google Scholar 

  24. LemarÉchal, C., and SagastizÁbal, C., An Approach to Variable-Metric Methods, Proceeding of the 16th IFIP Conference on System Modelling and Optimization, Springer Verlag, Berlin, Germany, pp. 144–162, 1994.

    Google Scholar 

  25. Mifflin, R., A Quasi-Second-Order Proximal Bundle Algorithm, Mathematical Programming, Vol. 73, pp. 51–72, 1996.

    Google Scholar 

  26. Au, K. T., Higle, J. L., and Sen, S., Inexact Subgradient Methods with Applications in Stochastic Programming, Mathematical Programming, Vol. 63, pp. 65–82, 1994.

    Google Scholar 

  27. Birge, J. R., and Qi, L., Subdifferential Convergence in Stochastic Programs, SIAM Journal on Optimization, Vol. 5, pp. 436–453, 1995.

    Google Scholar 

  28. Birge, J. R., and Wets, R. J. B., Designing Approximation Schemes for Stochastic Optimization Problems in Particular Stochastic Programs with Recourse, Mathematical Programming Study, Vol. 27, pp. 54–102, 1986.

    Google Scholar 

  29. Birge, J. R., Chen, X., Qi, L., and Wei, Z., A Stochastic Newton Method for Stochastic Quadratic Programs with Recourse, Applied Mathematics Report AMR 94/9, University of New South Wales, 1994.

  30. Chen, X., Qi, L., and Womersley, R. S., Newton's Method for Quadratic Stochastic Programs with Recourse, Journal of Computational and Applied Mathematics, Vol. 60, pp. 29–46, 1995.

    Google Scholar 

  31. Higle, J. L., and Sen, S., Stochastic Decomposition: An Algorithm for Two-Stage Linear Programs with Recourse, Mathematics of Operations Research, Vol. 3, pp. 650–669, 1991.

    Google Scholar 

  32. Kall, P., and Wallace, S. T., Stochastic Programming, Wiley, New York, New York, 1994.

    Google Scholar 

  33. King, A. J., and Wets, R. J. B., Epi-Consistency of Convex Stochastic Programs, Stochastic and Stochastic Reports, Vol. 34, pp. 83–92, 1990.

    Google Scholar 

  34. Qi, S., and Womersley, R. S., An SQP Algorithm for Extended Linear-Quadratic Problems in Stochastic Programming, Annals of Operation Research, Vol. 56, pp. 251–285, 1995.

    Google Scholar 

  35. Joe, S., and Soan, I. H., Imbedded Lattice Rules for Multidimensional Integration, SIAM Journal on Numerical Analysis, Vol. 29, pp. 1191–1135, 1992.

    Google Scholar 

  36. Niederreiter, H., Random Number Generation and Quasi-Monte Carlo Methods. SIAM, Philadelphia, Pennsylvania, 1992.

    Google Scholar 

  37. Sloan, I. H., and Joe, S., Lattice Methods for Multiple Integration, Clarendon Press, Oxford, 1994.

    Google Scholar 

  38. Spanier, J., and Maize, E. H., Quasi-Random Methods for Estimating Integrals Using Relatively Small Samples, SIAM Review, Vol. 36, pp. 18–44, 1994.

    Google Scholar 

  39. Ortega, J. M., and Rheinboldt, W. C., Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, New York, 1970.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, Michigan

    J. R. Birge (Professor)

  2. School of Mathematics, University of New South Wales, Sydney, NSW, Australia

    L. Qi (Associate Professor)

  3. School of Mathematics, University of New South Wales, Sydney, NSW, Australia

    Z. Wei (PhD Student)

Authors
  1. J. R. Birge
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. L. Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Z. Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Birge, J.R., Qi, L. & Wei, Z. Convergence Analysis of Some Methods for Minimizing a Nonsmooth Convex Function. Journal of Optimization Theory and Applications 97, 357–383 (1998). https://doi.org/10.1023/A:1022630801549

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1022630801549

Search

Navigation