Abstract
We propose an implementable BFGS method for solving a nonsmooth convex optimization problem by converting the original objective function into a once continuously differentiable function by way of the Moreau–Yosida regularization. The proposed method makes use of approximate function and gradient values of the Moreau-Yosida regularization instead of the corresponding exact values. We prove the global convergence of the proposed method under the assumption of strong convexity of the objective function.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Yosida, K., Functional Analysis, Springer Verlag, Berlin, Germany, 1964.
Moreau, J., Proximitéet Dualitédans un Espace Hilbertien, Bulletin de la Socièté Mathématique de France, Vol. 93, pp. 273–299, 1965.
Hiriart-urruty, J., and LemarÉchal, C., Convex Analysis and Minimization Algorithms, Springer Verlag, Berlin, Germany, 1993.
LemarÉchal, C., and SagastizÁbal, C., Practical Aspects of the Moreau- Yosida Regularization, I: Theoretical Preliminaries, SIAM Journal on Optimization, Vol. 7, pp. 367–385, 1997.
qi, L., Second-Order Analysis of the Moreau-Yosida Regularization of a Convex Function, Preprint, School of Mathematics, University of New South Wales, Sydney, Australia, 1995.
LemarÉchal, C., and SagastizÁbal, C., An Approach to Variable-Metric Bundle Methods, Proceedings of the 16th IFIP-TC7 Conference on Systems Modelling and Optimization, Edited by J. Henry and J. P. Yuvor, Springer Verlag, New York, NY, pp. 144–162, 1994.
Bonnans, J. F., Gilbert, J. C., LemarÉchal, C., and SagastizÁbal, C., A Family of Variable-Metric Proximal Methods, Mathematical Programming, Vol. 68, pp. 15–47, 1995.
Mifflin, R., A Quasi-Second-Order Proximal Bundle Algorithm, Mathematical Programming, Vol. 73, pp. 51–72, 1996.
LemarÉchal, C., and SagastizÁbal, C., Variable-Metric Bundle Methods: From Conceptual to Implementable Forms, Mathematical Programming, Vol. 76, pp. 393–410, 1997.
Fukushima, M., and Qi, L., A Globally and Superlinearly Convergent Algorithm for Nonsmooth Convex Minimization, SIAM Journal on Optimization, Vol. 6, pp. 1106–1120, 1996.
Qi, L., and Chen, X., A Preconditioned Proximal Newton Method for Nondifferentiable Convex Optimization, Mathematical Programming, Vol. 76, pp. 411–429, 1997.
Chen, X., and Fukushima, M., Proximal Quasi-Newton Methods for Nondifferentiable Convex Optimization, Mathematical Programming, Vol. 85, pp. 313–334, 1999.
Burke, J. V., and Qian, M., On the Superlinear Convergence of the Variable-Metric Proximal Point-Algorithm Using Broyden and BFGS Matrix Secant Updating, Mathematical Programming (to appear).
Qian, M., and Burke, J. V., On the Local Superlinear Convergence of a Matrix Secant Implementation of the Variable-Metric Proximal-Point Algorithm for Monotone Operators, Reformulation: Nonsmooth, Piecewise Smooth, Semismooth, and Smoothing Methods, Edited by M. Fukushima and L. Qi, Kluwer Academic Publishers, Dordrecht, Holland, pp. 317–334, 1998.
Mifflin, R., Sun, D., and Qi, L., Quasi-Newton Bundle-Type Methods for Nondifferentiable Convex Optimization, SIAM Journal on Optimization, Vol. 8, pp. 583–603, 1998.
Chen, X., Convergence of the BFGS Method for LC 1-Convex Constrained Optimization, SIAM Journal on Control and Optimization, Vol. 34, pp. 2051–2063, 1996.
Ip, C. M., and Kyparisis, J., Local Convergence of Quasi-Newton Methods for B-Differentiable Equations, Mathematical Programming, Vol. 56, pp. 71–89, 1992.
Robinson, S. M., Local Structure of Feasible Sets in Nonlinear Programming, Part 3: Stability and Sensitivity, Mathematical Programming Study, Vol. 30, pp. 45–66, 1987.
Fukushima, M., A Descent Algorithm for Nonsmooth Convex Optimization, Mathematical Programming, Vol. 30, pp. 163–175, 1984.
Auslender, A., Numerical Methods for Nondifferentiable Convex Optimization, Mathematical Programming Study, Vol. 30, pp. 102–126, 1987.
Correa, R., and LemarÉchal, C., Convergence of Some Algorithms for Convex Minimization, Mathematical Programming, Vol. 62, pp. 261–275, 1993.
Dennis, J. E., Jr., and Schnabel, R. B., Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, New Jersey, 1983.
Fletcher, R., Practical Methods of Optimization, 2nd Edition, John Wiley, Chichester, England, 1987.
Ortega, J. M., and Rheinboldt, W. C., Iterative Solutions of Nonlinear Equations in Several Variables, Academic Press, New York, NY, 1970.
Byrd, R. H., and Nocedal, J., A Tool for the Analysis of Quasi-Newton Methods with Application to Unconstrained Optimization, SIAM Journal on Numerical Analysis, Vol. 26, pp. 727–739, 1989.
Byrd, R. H., Nocedal, J., and Yuan, Y., Global Convergence of a Class of Quasi-Newton Methods on Convex Problems, SIAM Journal on Numerical Analysis, Vol. 24, pp. 1171–1190, 1987.
Powell, M. J. D., Some Global Convergence Properties of a Variable Metric Algorithm for Minimization without Exact Line Searches, Nonlinear Programming, Edited by R. W. Cottle and C. E. Lemke, American Mathematical Society, Providence, Rhode Island, pp. 53–72, 1976.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Rauf, A.I., Fukushima, M. Globally Convergent BFGS Method for Nonsmooth Convex Optimization1. Journal of Optimization Theory and Applications 104, 539–558 (2000). https://doi.org/10.1023/A:1004633524446
Issue Date:
DOI: https://doi.org/10.1023/A:1004633524446