Abstract
We consider conceptual optimization methods combining two ideas: the Moreau—Yosida regularization in convex analysis, and quasi-Newton approximations of smooth functions. We outline several approaches based on this combination, and establish their global convergence. Then we study theoretically the local convergence properties of one of these approaches, which uses quasi-Newton updates of the objective function itself. Also, we obtain a globally and superlinearly convergent BFGS proximal method. At each step of our study, we single out the assumptions that are useful to derive the result concerned.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
A. Auslender, “Numerical methods for non-differentiable convex optimization,”Mathematical Programming Study 30 (1987) 102–126.
C.G. Broyden, J.E. Dennis and J.J. Moré, “On the local and superlinear convergence of quasi-Newton methods,”Journal of the Institute of Mathematics and its Applications 12 (1973) 223–245.
R.H. Byrd and J. Nocedal, “A tool for the analysis of quasi-Newton methods with application to unconstrained minimization,”SIAM Journal on Numerical Analysis 26 (1989) 727–739.
F.H. Clarke,Optimization and Non-Smooth Analysis (Wiley, New York, 1983).
R. Correa and C. Lemaréchal, “Convergence of some algorithms for convex minimization,”Mathematical Programming 62 (1993) 261–275.
J.E. Dennis and J.J. Moré, “A characterization of superlinear convergence and its application to quasi-Newton methods,”Mathematics of Computation 28 (1974) 549–560.
J.E. Dennis and J.J. Moré, “Quasi-Newton methods, motivation and theory,”SIAM Review 19 (1977) 46–89.
J.E. Dennis and R.B. Schnabel, “A view of unconstrained optimization,” in: G.L. Nemhauser, A.H.G. Rinnooy Kan and M.J. Todd, eds.,Handbook in Operations Research and Management Science, Vol. 1 (North-Holland, Amsterdam, 1989) 1–72.
M. Fukushima, “A descent algorithm for non-smooth convex programming,”Mathematical Programming 30 (1984) 163–175.
J.Ch. Gilbert and C. Lemaréchal, “Some numerical experiments with variable-storage quasi-Newton algorithms,”Mathematical Programming 45 (1989) 407–435.
W.A. Gruver and E. Sachs,Algorithmic Methods in Optimal Control, Research Notes in Mathematics No. 47 (Pitman, London, 1980).
S.M. Grzegórski, “Orthogonal projections on convex sets for Newton-like methods”,SIAM Journal on Numerical Analysis 22 (1985) 1208–1219.
J.B. Hiriart-Urruty and C. Lemaréchal,Convex Analysis and Minimization Algorithms, Vols. 1 and 2 (Springer-Verlag, Berlin, Heidelberg, New York, 1993).
C.-M. Ip and J. Kyparisis, “Local convergence of quasi-Newton methods for B-differentiable equations,”Mathematical Programming 56 (1992) 71–89.
K.C. Kiwiel, “Proximity control in bundle methods for convex non-differentiable minimization,”Mathematical Programming 46 (1990) 105–122.
B. Kummer, “Newton's method based on generalized derivatives for non-smooth functions: convergence analysis,” in: W. Oettli and D. Pallaschke, eds.,Advances in Optimization, Lecture Notes in Economics and Mathematical Systems No. 382 (Springer-Verlag, Berlin, Heidelberg, New York, 1991) 171–194.
C. Lemaréchal, “A view of line-searches,” in: A Auslender, W. Oettli and J. Stoer, eds.,Optimization and Optimal Control, Lecture Notes in Control and Information Science No. 30 (Springer-Verlag, Berlin, Heidelberg, New York, 1981) 59–78.
C. Lemaréchal and C. Sagastizábal, “An approach to variable metric bundle methods,” in: J. Henry and J.P. Yvor, eds.,Proceedings IFIP Conference Systems Modelling and Optimization, Lecture Notes in Control and Information Sciences No. 197 (Springer-Verlag, New York, 1994).
B. Martinet, “Régularisation d'inéquations variationelles par approximations successives,”Revue Française d'Informatique et Recherche Opérationelle R-3 (1970) 154–179.
R. Mifflin, “A quasi-second-order proximal bundle algorithm,” Technical Report 93-3, University of Washington (Pullman, Washington, 1993).
J.J. Moreau, “Proximité et dualité dans un espace hilbertien,”Bulletin de la Société Mathématique de France 93 (1965) 273–299.
J.M. Ortega and W.C. Rheinboldt,Iterative Solution of Non-linear Equations in Several Variables (Academic Press, New York, 1970).
J.-S. Pang, “Newton's method for B-differentiable equations,”Mathematics of Operations Research 15 (1990) 311–341.
J.S. Pang and L.Q. Qi, “Non-smooth equations: motivation and algorithms,”SIAM Journal on Optimization 3 (1993) 443–465.
M.J.D. Powell, “Some global convergence properties of a variable metric algorithm for minimization without exact line searches,” in: R.W. Cottle and C.E. Lemke, eds.,Non-linear Programming, SIAM—AMS Proceedings No. 9 (American Mathematical Society, Providence, RI, 1976).
L. Qi and J. Sun, “A non-smooth version of Newton's method,”Mathematical Programming 58 (1993) 353–367.
L.Q. Qi, “Convergence analysis of some algorithms for solving non-smooth equations,”Mathematics of Operations Research 18 (1993) 227–244.
M. Qian, “The variable metric proximal point algorithm: global and super-linear convergence,” Manuscript GN-50, University of Washington, Department of Mathematics (Seattle, WA, 1992).
S.M. Robinson, “Local structure of feasible sets in non-linear programming, part III: stability and sensitivity,”Mathematical Programming Study 30 (1987) 45–66.
R.T. Rockafellar,Convex Analysis (Princeton University, Princeton, NJ, 1970).
R.T. Rockafellar, “Monotone operators and the proximal point algorithm,”SIAM Journal on Control and Optimization 14 (1976) 877–898.
C.A. Sagastizábal, “Quelques méthodes numériques d'optimisation. Application en gestion de stocks,” Ph.D. thesis, University of Paris I, Panthéon-Sorbonne (Paris, 1993).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Bonnans, J.F., Gilbert, J.C., Lemaréchal, C. et al. A family of variable metric proximal methods. Mathematical Programming 68, 15–47 (1995). https://doi.org/10.1007/BF01585756
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01585756