Abstract
Many problems in nonlinear analysis and optimization, among them variational inequalities and minimization of convex functions, can be reduced to finding zeros (namely, roots) of set-valued operators. Hence numerous algorithms have been devised in order to achieve this task. A lot of these algorithms are inexact in the sense that they allow perturbations to appear during the iterative process, and hence they enable one to better deal with noise and computational errors, as well as superiorization. For many years a certain fundamental question has remained open regarding many of these known inexact algorithmic schemes in various finite and infinite dimensional settings, namely whether there exist sequences satisfying these inexact schemes when errors appear. We provide a positive answer to this question. Our results also show that various theorems discussing the convergence of these inexact schemes have a genuine merit beyond the exact case. As a by-product we solve the standard and the strongly implicit inexact resolvent inclusion problems, introduce a promising class of functions (fully Legendre functions), establish continuous dependence (stability) properties of the solution of the inexact resolvent inclusion problem and continuity properties of the protoresolvent, and generalize the notion of strong monotonicity.
Similar content being viewed by others
References
Ahmadi, P., Khatibzadeh, H.: On the convergence of inexact proximal point algorithm on Hadamard manifolds. Taiwan. J. Math. 18, 419–433 (2014)
Ambrosetti, A., Prodi, G.: A Primer of Nonlinear Analysis. Cambridge University Press, New York (1993)
Aragón Artacho, F.J., Geoffroy, M.H.: Uniformity and inexact version of a proximal method for metrically regular mappings. J. Math. Anal. Appl. 335, 168–183 (2007)
Auslender, A., Teboulle, M.: Asymptotic cones and functions in optimization and variational inequalities. Springer Monographs in Mathematics. Springer, New York (2003)
Auslender, A., Teboulle, M., Ben-Tiba, S.: Interior proximal and multiplier methods based on second order homogeneous kernels. Math. Oper. Res. 24, 645–668 (1999)
Auslender, A., Teboulle, M., Ben-Tiba, S.: A logarithmic-quadratic proximal method for variational inequalities. Comput. Optim. Appl. 12, 31–40 (1999). Computational optimization—a tribute to Olvi Mangasarian, Part I
Bačák, M.: The proximal point algorithm in metric spaces. Isr. J. Math. 194, 689–701 (2013)
Bačák, M.: Convex Analysis and Optimization in Hadamard Spaces. De Gruyter Series in Nonlinear Analysis and Applications, vol. 22. De Gruyter, Berlin (2014)
Banerjee, A., Merugu, S., Dhillon, I.S., Ghosh, J.: Clustering with Bregman divergences. J. Mach. Learn. Res. 6, 1705–1749 (2005). A preliminary version in Proceedings of the fourth SIAM International Conference on Data Mining, pp. 234–245, Philadelphia, 2004
Bauschke, H.H., Borwein, J.M.: Legendre functions and the method of random Bregman projections. J. Convex Anal. 4, 27–67 (1997)
Bauschke, H.H., Borwein, J.M., Combettes, P.L.: Essential smoothness, essential strict convexity, and Legendre functions in Banach spaces. Commun. Contemp. Math. 3, 615–647 (2001)
Bauschke, H.H., Borwein, J.M., Combettes, P.L.: Bregman monotone optimization algorithms. SIAM J. Control Optim. 42, 596–636 (2003)
Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. CMS Books in Mathematics, 2nd edn. Springer, Cham (2017)
Bauschke, H.H., Wang X., Yao L.: General resolvents for monotone operators: characterization and extension, Biomedical mathematics: Promising directions in imaging, therapy planning and inverse problems (Huangguoshu 2008), Chapter 4, Medical Physics Publishing (2010). arXiv:0810.3905 [math.FA] ([v1], 21 Oct 2008)
Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer Series in Operations Research. Springer, New York (2000)
Borwein, J.M.: Maximal monotonicity via convex analysis. J. Convex Anal. 13, 561–586 (2006)
Borwein, J.M., Lewis, A.L.: Convex Analysis and Nonlinear Optimization: Theory and Examples. CMS Books in Mathematics, 2nd edn. Springer, New York (2006)
Bregman, L.M.: The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. Math. Phys. 7, 200–217 (1967)
Brézis, H.: Opérateurs Maximaux Monotones et Semi-groupes de Contractions dans les Espaces de Hilbert, North-Holland Publishing Co., Amsterdam-London; American Elsevier Publishing Co., Inc., New York, 1973, North-Holland Mathematics Studies, No. 5. Notas de Matemática (50)
Brezis, H.: Functional Analysis, Sobolev Spaces and Partial Differential Equations. Universitext, Springer, New York (2011)
Brézis, H., Lions, P.-L.: Produits infinis de résolvantes. Isr. J. Math. 29, 329–345 (1978)
Bruck, R.E., Reich, S.: Nonexpansive projections and resolvents of accretive operators in Banach spaces. Houston J. Math. 32, 459–470 (1977)
Burachik, R.S., Iusem, A.N.: Set-Valued Mappings and Enlargements of Monotone Operators. Springer Optimization and Its Applications, vol. 8. Springer, New York (2008)
Burachik, R.S., Iusem, A.N., Svaiter, B.F.: Enlargement of monotone operators with applications to variational inequalities. Set Valued Anal. 5, 159–180 (1997)
Burachik, R.S., Lopes, J.O., Da Silva, G.J.P.: An inexact interior point proximal method for the variational inequality problem. Comput. Appl. Math. 28, 15–36 (2009)
Burachik, R.S., Sagastizábal, C.A., Svaiter B.F.: \(\epsilon \)-enlargements of maximal monotone operators: theory and applications. In: Fukushima M., Qi L. (eds.) Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods. (Lausanne, 1997), Applied Optimization, vol. 22. Kluwer Acad. Publ., Dordrecht, pp. 25–43 (1999)
Burachik, R.S., Scheimberg, S.: A proximal point method for the variational inequality problem in Banach spaces. SIAM J. Control Optim. 39, 1633–1649 (2000)
Burachik, R.S., Scheimberg, S., Svaiter, B.F.: Robustness of the hybrid extragradient proximal-point algorithm. J. Optim. Theory Appl. 111, 117–136 (2001)
Burachik, R.S., Svaiter, B.F.: A relative error tolerance for a family of generalized proximal point methods. Math. Oper. Res. 26, 816–831 (2001)
Butnariu, D., Iusem, A.N.: Totally Convex Functions for Fixed Point Computation and Infinite Dimensional Optimization. Applied Optimization. Kluwer Academic Publishers, Dordrecht, The Netherlands (2000)
Butnariu, D., Iusem, A.N., Zălinescu, C.: On uniform convexity, total convexity and convergence of the proximal point and outer Bregman projection algorithms in Banach spaces. J. Convex Anal. 10, 35–61 (2003)
Censor, Y.: Weak and strong superiorization: between feasibility-seeking and minimization, An. Ştiinţ. Univ. “Ovidius” Constanţa Ser. Mat. 23, 41–54 (2015)
Censor, Y.: Superiorization and perturbation resilience of algorithms: a continuously updated bibliography, 2017, http://math.haifa.ac.il/yair/bib-superiorization-censor.html, website last updated: 2 March 2017, arXiv version: arXiv:1506.04219 [math.OC] ([v2], 9 Mar 2017)
Censor, Y., Davidi, R., Herman, G.T.: Perturbation resilience and superiorization of iterative algorithms, Inverse Problems 26 (2010), 065008 (12 pages)
Censor, Y., Lent, A.: An iterative row-action method for interval convex programming. J. Optim. Theory Appl. 34, 321–353 (1981)
Censor, Y., Reem, D.: Zero-convex functions, perturbation resilience, and subgradient projections for feasibility-seeking methods. Math. Program. (Ser. A) 152, 339–380 (2015)
Censor, Y., Zenios, A.: Proximal minimization algorithm with \(D\)-functions. J. Optim. Theory Appl. 73, 451–464 (1992)
Courant, R., Hilbert D.: Methods of mathematical physics. Vol. II: Partial differential equations, (Vol. II by R. Courant.), Interscience Publishers (a division of John Wiley & Sons), New York-London, (1962)
Davidi, R.: Algorithms for superiorization and their applications to image reconstruction, PhD thesis, The City University of New York (CUNY), USA, (2010)
Diestel, J.: Geometry of Banach spaces–selected topics, Lecture Notes in Mathematics, vol. 485. Springer, Berlin (1975)
Djafari Rouhani, B., Khatibzadeh, H.: On the proximal point algorithm. J. Optim. Theory Appl. 137, 411–417 (2008)
Eckstein, J.: Nonlinear proximal point algorithms using Bregman functions, with applications to convex programming. Math. Oper. Res. 18, 202–226 (1993)
Eckstein, J.: Approximate iterations in Bregman-function-based proximal algorithms. Math. Program. 83, 113–123 (1998)
Eckstein, J., Svaiter, B.F.: General projective splitting methods for sums of maximal monotone operators. SIAM J. Control Optim. 48, 787–811 (2009)
Gárciga Otero, R., Iusem, A.N.: Proximal methods with penalization effects in Banach spaces. Numer. Funct. Anal. Optim. 25, 69–91 (2004)
Gárciga Otero, R., Iusem, A.N.: Proximal methods in reflexive Banach spaces without monotonicity. J. Math. Anal. Appl. 330, 433–450 (2007)
Gárciga Otero, R., Iusem, A.N.: Fixed-point methods for a certain class of operators. J. Optim. Theory Appl. 159, 656–672 (2013)
Gárciga Otero, R., Svaiter, B.F.: A strongly convergent hybrid proximal method in Banach spaces. J. Math. Anal. Appl. 289, 700–711 (2004)
Griva, I., Polyak, R.A.: Proximal point nonlinear rescaling method for convex optimization. Numer. Algebra Control Optim. 1, 283–299 (2011)
Ha, C.D.: A generalization of the proximal point algorithm. SIAM J. Control Optim. 28, 503–512 (1990)
Herman, G.T.: Superiorization for image analysis, Combinatorial image analysis, Lecture Notes in Computer Science, vol. 8466, Springer, 2014, p. 1–7
Hiriart-Urruty, J.-B., Lemaréchal, C.: Fundamentals of Convex Analysis, Grundlehren Text Editions, Springer-Verlag, Berlin, 2001. Abridged version of Convex analysis and Minimization Algorithms. I, II, Springer (1993)
Humes Jr., C., Silva, P.J.S., Svaiter, B.F.: Some inexact hybrid proximal augmented Lagrangian algorithms. Numer. Algorithms 35, 175–184 (2004)
Iusem, A.N., Gárciga Otero, R.: Inexact versions of proximal point and augmented Lagrangian algorithms in Banach spaces, Numer. Funct. Anal. Optim. 22, 609–640 (2001). Erratum. Numer. Funct. Anal. Optim. 23(1–2), 227–228 (2002)
Iusem, A.N., Gárciga Otero, R.: Augmented Lagrangian methods for cone-constrained convex optimization in Banach spaces. J. Nonlinear Convex Anal. 3, 155–176 (2002)
Iusem, A.N., Pennanen, T., Svaiter, B.F.: Inexact variants of the proximal point algorithm without monotonicity. SIAM J. Optim. 13, 1080–1097 (2003)
Iusem, A.N., Reem, D., Reich, S.: Fixed points of Legendre-Fenchel type transforms. arXiv:1708.00464 [math.CA] ([v1]: 25 July 2017)
Kassay, G.: The proximal points algorithm for reflexive Banach spaces. Studia Univ. Babeş-Bolyai Math. 30, 9–17 (1985)
Li, C., López G., Martín-Márquez V.: Monotone vector fields and the proximal point algorithm on Hadamard manifolds. J. Lond. Math. Soc. 79, 663–683 (2009)
Li, C., López, G., Martín-Márquez, V., Wang, J.-H.: Resolvents of set-valued monotone vector fields in Hadamard manifolds. Set Valued Var. Anal. 19, 361–383 (2011)
Lindenstrauss, J., Tzafriri, L.: Classical Banach spaces, II: Function spaces, Ergebnisse der Mathematik und ihrer Grenzgebiete (Results in Mathematics and Related Areas). Springer-Verlag, Berlin-New York (1979)
Lotito, P.A., Parente, L.A., Solodov, M.V.: A class of variable metric decomposition methods for monotone variational inclusions. J. Convex Anal. 16, 857–880 (2009)
Martín-Márquez, V., Reich, S., Sabach, S.: Right Bregman nonexpansive operators in Banach spaces. Nonlinear Anal. 75, 5448–5465 (2012)
McCarthy, C.A.: \(c_{p}\). Isr. J. Math. 5, 249–271 (1967)
Minty, G.J.: Monotone (nonlinear) operators in Hilbert space. Duke Math. J. 29, 341–346 (1962)
Monteiro, R.D.C., Svaiter, B.F.: Iteration-complexity of a Newton proximal extragradient method for monotone variational inequalities and inclusion problems. SIAM J. Optim. 22, 914–935 (2012)
Monteiro, R.D.C., Svaiter, B.F.: An accelerated hybrid proximal extragradient method for convex optimization and its implications to second-order methods. SIAM J. Optim. 23, 1092–1125 (2013)
Moreau, J.J.: Fonctions convexes duales et points proximaux dans un espace hilbertien. C.R. Acad. Sci. Paris Sér. I Math. 255, 2897–2899 (1962)
Moreau, J.J.: Proximité et dualité dans un espace hilbertien. Bull. Soc. Math. France 93, 273–299 (1965)
Papa Quiroz, E.A., Oliveira, P.R.: Proximal point methods for quasiconvex and convex functions with Bregman distances on Hadamard manifolds. J. Convex Anal. 16, 49–69 (2009)
Parente, L.A., Lotito, P.A., Solodov, M.V.: A class of inexact variable metric proximal point algorithms. SIAM J. Optim. 19, 240–260 (2008)
Phelps, R.R.: Convex Functions, Monotone Operators and Differentiability, 2 ed., Lecture Notes in Mathematics, vol. 1364, Springer-Verlag, Berlin, 1993, A closely related material can be found in: Lectures on maximal monotone operators, arXiv:math/9302209 [math.FA] ([v1], 4 Feb 1993)
Pinchover, Y., Rubinstein, J.: An Introduction to Partial Differential Equations. Cambridge University Press, Cambridge (2005)
Polyak R.A.: The legendre transformation in modern optimization. In: Goldengorin B. (ed.) Optimization and Its Applications in Control and Data Sciences. Springer Optimization and Its Applications, vol 115, pp 437–507. Springer, Cham (2016)
Reem, D.: The Bregman distance without the Bregman function II, Optimization theory and related topics, Contemp. Math. (Am. Math. Soc., Providence, RI), vol. 568, pp. 213–223 (2012)
Reem, D., De Pierro A.R.: A new convergence analysis and perturbation resilience of some accelerated proximal forward-backward algorithms with errors, Inverse Problems 33 (2017), 044001 (28pp), arXiv:1508.05631 [math.OC] (2015) (current version: [v3], 29 Jun 2016)
Reich, S., Sabach, S.: A strong convergence theorem for a proximal-type algorithm in reflexive Banach spaces. J. Nonlinear Convex Anal. 10, 471–485 (2009)
Reich, S., Sabach, S.: Two strong convergence theorems for a proximal method in reflexive Banach spaces. Numer. Funct. Anal. Optim. 31, 22–44 (2010)
Reich, S., Sabach, S.: Two strong convergence theorems for Bregman strongly nonexpansive operators in reflexive Banach spaces. Nonlinear Anal. 73, 122–135 (2010)
Reich, S., Sabach, S.: Three strong convergence theorems regarding iterative methods for solving equilibrium problems in reflexive Banach spaces, Optimization Theory and Related Topics. Contemp. Math. (Amer. Math. Soc., Providence, RI), vol. 568, 225–240 (2012)
Rockafellar, R.T.: Convex Analysis, Princeton Mathematical Series, No. 28, Princeton University Press, Princeton, NJ, USA, 1970
Rockafellar, R.T.: On the maximal monotonicity of subdifferential mappings. Pac. J. Math. 33, 209–216 (1970)
Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)
Rockafellar, R.T., Wets, R.J.-B.: Variational analysis, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 317. Springer, Berlin (1998)
Silva, P.J.S., Eckstein, J., Humes Jr., C.: Rescaling and stepsize selection in proximal methods using separable generalized distances. SIAM J. Optim. 12, 238–261 (2001)
Simons, S.: From Hahn-Banach to Monotonicity, 2 ed., Lecture Notes in Mathematics, vol. 1693, Springer, New York (2008)
Solodov, M.V.: A class of decomposition methods for convex optimization and monotone variational inclusions via the hybrid inexact proximal point framework. Optim. Methods Softw. 19, 557–575 (2004)
Solodov, M.V., Svaiter, B.F.: A hybrid approximate extragradient-proximal point algorithm using the enlargement of a maximal monotone operator. Set-Valued Anal. 7, 323–345 (1999)
Solodov, M.V., Svaiter, B.F.: A hybrid projection-proximal point algorithm. J. Convex Anal. 6, 59–70 (1999)
Solodov, M.V., Svaiter, B.F.: A comparison of rates of convergence of two inexact proximal point algorithms. In: Pillo, G.D., Giannessi, F. (eds.) Nonlinear Optimization and Related Topics (Erice, 1998). Applied Optimization, vol. 36. Kluwer Acad. Publ., Dordrecht, pp. 415–427 (2000)
Solodov, M.V., Svaiter B.F.: Error bounds for proximal point subproblems and associated inexact proximal point algorithms. Math. Program. Ser. B 88, 371–389 (2000) Error bounds in mathematical programming (Kowloon, 1998)
Solodov, M.V., Svaiter, B.F.: An inexact hybrid generalized proximal point algorithm and some new results in the theory of Bregman functions. Math. Oper. Res. 51, 214–230 (2000)
Solodov, M.V., Svaiter, B.F.: A unified framework for some inexact proximal point algorithms. Numer. Funct. Anal. Optim. 22, 1013–1035 (2001)
Souza, J.C.O., Oliveira, P.R.: A proximal point algorithm for DC fuctions on Hadamard manifolds. J. Global Optim. 63, 797–810 (2015)
Tang, G.-J., Huang, N.-J.: Rate of convergence for proximal point algorithms on Hadamard manifolds. Oper. Res. Lett. 42, 383–387 (2014)
Teboulle, M.: Entropic proximal mappings with applications to nonlinear programming. Math. Oper. Res. 17, 670–690 (1992)
Tomczak-Jaegermann, N.: The moduli of smoothness and convexity and the Rademacher averages of trace classes \(S_{p}(1\le p<\infty )\). Studia Math. 50, 163–182 (1974)
van Tiel, J.: Convex Analysis: An Introductory Text. John Wiley and Sons, Universities Press, Belfast, Northern Ireland (1984)
Walter W.: Ordinary Differential Equations, Graduate Texts in Mathematics, Readings in Mathematics, vol. 182, Springer-Verlag, New York, 1998, Translated by R. Thompson, based on and extends the sixth German edition (from 1996)
Wang, J., Li, C., Lopez, G., Yao, J.C.: Convergence analysis of inexact proximal point algorithms on Hadamard manifolds. J. Global Optim. 61, 553–573 (2015)
Xia, F.Q., Huang, N.J.: An inexact hybrid projection-proximal point algorithm for solving generalized mixed variational inequalities. Comput. Math. Appl. 62, 4596–4604 (2011)
Xu, Z.B., Roach, G.F.: Characteristic inequalities of uniformly convex and uniformly smooth Banach spaces. J. Math. Anal. Appl. 157, 189–210 (1991)
Yin, W., Osher, S., Goldfarb, D., Darbon, J.: Bregman iterative algorithms for \(\ell _1\)-minimization with applications to compressed sensing. SIAM J. Imaging Sci. 1, 143–168 (2008)
Zălinescu, C.: Convex Analysis in General Vector Spaces. World Scientific Publishing, River Edge, NJ (2002)
Zaslavski, A.J.: Inexact proximal point methods in metric spaces. Set-Valued Var. Anal. 19, 589–608 (2011)
Acknowledgements
We would like to express our thanks to Shoham Sabach and Roman Polyak for helpful discussions, and to the referees for considering our paper and for their feedback. Part of the research of the first author was done while he was at the Institute of Mathematical and Computer Sciences (ICMC), University of São Paulo, São Carlos, Brazil (2015) and this is an opportunity for him to thank FAPESP. The second author was partially supported by the Israel Science Foundation (Grant 389/12), by the Fund for the Promotion of Research at the Technion and by the Technion General Research Fund.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Reem, D., Reich, S. Solutions to inexact resolvent inclusion problems with applications to nonlinear analysis and optimization. Rend. Circ. Mat. Palermo, II. Ser 67, 337–371 (2018). https://doi.org/10.1007/s12215-017-0318-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12215-017-0318-6
Keywords
- Algorithmic scheme
- Fully Legendre function
- Inexact
- Inclusion
- Maximally monotone operator
- Protoresolvent
- Resolvent
- Well defined
- Zero