Abstract
The convex feasibility problem (CFP) is at the core of the modeling of many problems in various areas of science. Subgradient projection methods are important tools for solving the CFP because they enable the use of subgradient calculations instead of orthogonal projections onto the individual sets of the problem. Working in a real Hilbert space, we show that the sequential subgradient projection method is perturbation resilient. By this we mean that under appropriate conditions the sequence generated by the method converges weakly, and sometimes also strongly, to a point in the intersection of the given subsets of the feasibility problem, despite certain perturbations which are allowed in each iterative step. Unlike previous works on solving the convex feasibility problem, the involved functions, which induce the feasibility problem’s subsets, need not be convex. Instead, we allow them to belong to a wider and richer class of functions satisfying a weaker condition, that we call “zero-convexity”. This class, which is introduced and discussed here, holds a promise to solve optimization problems in various areas, especially in non-smooth and non-convex optimization. The relevance of this study to approximate minimization and to the recent superiorization methodology for constrained optimization is explained.
Similar content being viewed by others
References
Alber, Y.I., Iusem, A.N., Solodov, M.V.: On the projected subgradient method for nonsmooth convex optimization in a Hilbert space. Math. Program. 81, 23–35 (1998)
Ambrosetti, A., Prodi, G.: A Primer of Nonlinear Analysis. Cambridge University Press, Cambridge (1993)
Amenta, A.B., Bern, M.W., Eppstein, D.: Optimal point placement for mesh smoothing. J. Algorithms 30, 302–322 (1999)
Aurenhammer, F.: Voronoi diagrams—a survey of a fundamental geometric data structure. ACM Comput. Surv. 3, 345–405 (1991)
Avriel, M., Diewert, W.E., Schaible, S., Zang, I.: Generalized concavity, classics in applied mathematics, vol. 63, SIAM, Philadelphia, PA, USA, 2010, an unabridged republication of the work first published by Plenum Press (1988)
Barron, E.N., Liu, W.: Calculus of variations in \(L^\infty \). Appl. Math. Optim. 35, 237–263 (1997)
Barron, E.N., Goebel, R., Jensen, R.R.: Quasiconvex functions and nonlinear PDEs. Trans. Am. Math. Soc. 365, 4229–4255 (2013)
Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Rev. 38, 367–426 (1996)
Bauschke, H.H., Borwein, J.M., Lewis, A.S.: The method of cyclic projections for closed convex sets in Hilbert space. Contemp. Math. 204, 1–38 (1997)
Bauschke, H.H., Combettes, P.L.: A weak-to-strong convergence principle for Fejér-monotone methods in Hilbert spaces. Math. Oper. Res. 26, 248–264 (2001)
Bauschke, H.H., Combettes, P.L., Kruk, S.G.: Extrapolation algorithm for affine-convex feasibility problems. Numer. Algorithms 41, 239–274 (2006)
Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011)
Bazarra, M.S., Sherali, H., Shetty, C.M.: Nonlinear Programming: Theory and Algorithms, 3rd edn. Wiley-Interscience, Hoboken (2006)
Ben-Israel, A., Mond, B.: What is invexity? J. Aust. Math. Soc. Ser. B 28, 1–9 (1986)
Ben-Tal, A., El-Ghaoui, L., Nemirovskii, A.: Robust Optimization. Princeton University Press, Princeton (2009)
Bento, G.C., Melo, J.G.: Subgradient method for convex feasibility on Riemannian manifolds. J. Optim. Theory Appl. 152, 773–785 (2012)
Bertsekas, D.P., Nedić, A., Ozdaglar, A.E.: Convex Analysis and Optimization. Athena Scientific, Belmont (2003)
Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, New York (2000)
Borwein, J.M., Zhu, Q.J.: A survey of subdifferential calculus with applications. Nonlinear Anal.: Theory Methods Appl. 38, 687–773 (1999)
Borwein, J.M., Lewis, A.S.: Convex Analysis and Nonlinear Optimization: Theory and Examples, 2nd edn. Springer, New York (2006)
Browder, F.E.: Convergence theorems for sequences of nonlinear operators in Banach spaces. Math. Zeitschrift 100, 201–225 (1967)
Butnariu, D., Reich, S., Zaslavski, A.J.: Convergence to fixed points of inexact orbits for Bregman-monotone operators and for nonexpansive operators in Banach spaces. In: Fetter Natansky, H., et al. (eds.) Fixed Point Theory and its Applications. Yokohama Publishers, pp. 11–33 (2006)
Butnariu, D., Censor, Y., Reich, S.: Iterative averaging of entropic projections for solving stochastic convex feasibility problems. Comput. Optim. Appl. 8, 21–39 (1997)
Butnariu, D., Iusem, A.N.: Totally Convex Functions for Fixed Point Computation and Infinite Dimensional Optimization. Kluwer, Dordrecht (2000)
Butnariu, D., Davidi, R., Herman, G.T., Kazantsev, I.G.: Stable convergence behavior under summable perturbations of a class of projection methods for convex feasibility and optimization problems. IEEE J. Sel. Top. Signal Process. 1, 540–547 (2007)
Butnariu, D., Reich, S., Zaslavski, A.J.: Asymptotic behavior of inexact orbits for a class of operators in complete metric spaces. J. Appl. Anal. 13, 1–11 (2007)
Butnariu, D., Censor, Y., Gurfil, P., Hadar, E.: On the behavior of subgradient projections methods for convex feasibility problems in Euclidean spaces. SIAM J. Optim. 19, 786–807 (2008)
Cambini, A., Martein, L.: Generalized Convexity and Optimization. Lecture Notes in Economics and Mathematical Systems, vol. 616. Springer, Berlin (2009)
Cegielski, A.: Iterative Methods for Fixed Point Problems in Hilbert Spaces. Springer, Berlin (2012)
Censor, Y., Davidi, R., Herman, G.T., Schulte, R.W., Tetruashvili, L.: Projected subgradient minimization versus superiorization. J. Optim. Theory Appl. 160, 730–747 (2014)
Censor, Y., Davidi, R., Herman, G.T.: Perturbation resilience and superiorization of iterative algorithms. Inverse Probl. 26, 065008 (12pp) (2010)
Censor, Y., Lent, A.: An iterative row-action method for interval convex programming. J. Optim. Theory Appl. 34, 321–353 (1981)
Censor, Y., Lent, A.: Cyclic subgradient projections. Math. Program. 24, 233–235 (1982)
Censor, Y., Zenios, S.A.: Proximal minimization algorithm with \(D\)-functions. J. Optim. Theory Appl. 73, 451–464 (1992)
Censor, Y., Zenios, S.A.: Parallel Optimization: Theory, Algorithms, and Applications. Oxford University Press, New York (1997)
Censor, Y., Segal, A.: Algorithms for the quasiconvex feasibility problem. J. Comput. Appl. Math. 185, 34–50 (2006)
Censor, Y., Chen, W., Pajoohesh, H.: Finite convergence of a subgradient projections method with expanding controls. Appl. Math. Optim. 64, 273–285 (2011)
Censor, Y., Chen, W., Combettes, P.L., Davidi, R., Herman, G.T.: On the effectiveness of projection methods for convex feasibility problems with linear inequality constraints. Comput. Optim. Appl. 51, 1065–1088 (2012)
Chesi, G., Garulli, A., Tesi, A., Vicino, A.: Characterizing the solution set of polynomial systems in terms of homogeneous forms: an LMI approach. Int. J. Robust Nonlinear Control 13, 1239–1257 (2003)
Chiu, S.N., Stoyan, D., Kendall, W.S., Mecke, J.: Stochastic Geometry and Its Applications, 3rd edn. Wiley, Chichester (2013)
Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley Interscience, New York (1983)
Clarke, F.H., Ledyaev, Y.S., Stern, R.J., Wolenski, P.R.: Nonsmooth Analysis and Control Theory. Springer, New York (1998)
Combettes, P.L.: The convex feasibility problem in image recovery. Adv. Imaging Electron Phys. 95, 155–270 (1996)
Combettes, P.L.: Quasi-Fejérian analysis of some optimization algorithms. In: Butnariu, D., Censor, Y., Reich, S. (eds.) Inherently Parallel Algorithms in Feasibility and Optimization and their Applications, pp. 115–152. Elsevier Science Publishers, Amsterdam (2001)
Corvellec, J.-.N., Flåm, S.D.: Non-convex feasibility problems and proximal point methods. Optim. Methods Softw. 19, 3–14 (2004)
Crouzeix, J.-.P., Martinez-Legaz, J.-.E., Volle, M. (eds.): Generalized Convexity, Generalized Monotonicity: Recent Results, Nonconvex Optimization and its Applications, vol. 27, Kluwer, Dordrechtc (1998)
Daniilidis, A., Jules, F., Lassonde, M.: Subdifferential characterization of approximate convexity: the lower semicontinuous case. Math. Program. 116, 115–127 (2009)
Davidi, R.: Algorithms for Superiorization and Their Applications to Image Reconstruction, Ph.D. Thesis, Department of Computer Science, The Graduate Center, The City University of New York (CUNY), New York, NY, USA, 2010. http://www.dig.cs.gc.cuny.edu/rdavidi/Dissertation_RanDavidi.pdf
Davidi, R., Herman, G.T., Censor, Y.: Perturbation-resilient block-iterative projection methods with application to image reconstruction from projections. Int. Trans. Oper. Res. 16, 505–524 (2009)
De Pierro, A.R., Iusem, A.N.: A finitely convergent “row-action” method for the convex feasibility problem. Appl. Math. Optim. 17, 225–235 (1988)
Dembo, R.S., Eisenstat, S.C., Steihaug, T.: Inexact Newton methods. SIAM J. Numer. Anal. 19, 400–408 (1982)
Dixit, A.K.: Optimization in Economic Theory. Oxford University Press, New York (1976)
Eckstein, J.: Approximate iterations in Bregman-function-based proximal algorithms. Math. Program. 83, 113–123 (1998)
Eppstein, D.: Quasiconvex programming. In: Goodman, J.E., Pach, J., Welzl, E., (eds.) Combinatorial and Computational Geometry, MSRI Publications, vol. 52, Cambridge University Press, New York, pp. 287–331 (2005)
Eppstein, D.: Quasiconvex analysis of backtracking algorithms. ACM Trans. Algorithms 2, 492–509 (2006)
Gerstein, M., Tsai, J., Levitt, M.: The volume of atoms on the protein surface: calculated from simulation, using Voronoi polyhedra. J. Mol. Biol. 249, 955–966 (1995)
Goebel, K., Reich, S.: Uniform Convexity, Hyperbolic Geometry, and Nonexpansive Mappings. Marcel Dekker Inc., New York (1984)
Goede, A., Preissner, R., Frömmel, C.: Voronoi cell: new method for allocation of space among atoms: elimination of avoidable errors in calculation of atomic volume and density. J. Comput. Chem. 18, 1113–1123 (1997)
Gohberg, I., Goldberg, S.: Basic Operator Theory. Birkhäuser, Boston (1981)
Gold, C.: The Voronoi Web Site, 2008. http://www.voronoi.com/wiki/index.php?title=Main_Page
Greenberg, H.P., Pierskalla, W.P.: Quasi-conjugate functions and surrogate duality. Cahiers du Centre d‘Etudes de Recherche Operationnelle 15, 437–448 (1973)
Gromicho, J.A.: Quasiconvex Optimization and Location Theory, Applied Optimization, vol. 9. Kluwer, Dordrecht (1998)
Hadjisavvas, N., Komlósi, S., Schaible, S. (eds.): Handbook of Generalized Convexity and Generalized Monotonicity, Nonconvex Optimization and its Applications, vol. 76, Springer, New York (2005)
Hanson, M.A.: On sufficiency of the Kuhn-Tucker conditions. J. Math. Anal. Appl. 80, 545–550 (1981)
Henrion, D., Lasserre, J.B.: Detecting global optimality and extracting solutions in GloptiPoly, In: Henrion, D., Garulli, A., (eds.) Positive Polynomials in Control, Lecture Notes on Control and Information Sciences, vol. 312, Springer, Berlin, pp. 293–310 (2005)
Henrion, D., Lasserre, J.B.: Solving nonconvex optimization problems. IEEE Control Syst. Mag. 24, 72–83 (2004)
Henrion, D., Lasserre, J.B.: Inner approximations for polynomial matrix inequalities and robust stability regions. IEEE Trans. Autom. Control 57, 1456–1467 (2012)
Herman, G.T., Davidi, R.: Image reconstruction from a small number of projections. Inverse Probl. 24, 045011 (2008)
Herman, G.T., Garduño, E., Davidi, R., Censor, Y.: Superiorization: an optimization heuristic for medical physics. Med. Phys. 39, 5532–5546 (2012)
Higham, N.J.: Accuracy and Stability of Numerical Algorithms, Society of Industrial and Applied Mathematics (SIAM), PA, USA, Philadelphia (1996)
Hildebrand, R., Köppe, M.: A new Lenstra-type algorithm for quasiconvex polynomial integer minimization with complexity \(2^{O(n\log n)}\). Discret. Optim. 10, 69–84 (2013)
Horst, R., Pardalos, P.M. (eds.): Handbook of Global Optimization, Nonconvex Optimization and its Applications, vol. 2, Kluwer, Dordrecht (1995)
Horst, R., Tuy, H.: Global Optimization: Deterministic Approaches. Springer, Berlin (1990)
Iusem, A.N., Moledo, L.: A finitely convergent method of simultaneous subgradient projections for the convex feasibility problem. Comput. Appl. Math. 5, 169–184 (1986)
Iusem, A.N., Otero, R.G.: Inexact versions of proximal point and augmented Lagrangian algorithms in Banach spaces. Numer. Funct. Anal. Optim. 22, 609–640 (2001)
Ivanov, M.: Sequential representation formulae for \(G\)-subdifferential and Clarke subdifferential in smooth Banach spaces. J. Convex Anal. 11, 179–196 (2004)
Kim, C.-.M., Won, C.-.I., Cho, Y., Kim, D., Lee, S., Bhak, J., Kim, D.-.S.: Interaction interfaces in proteins via the Voronoi diagram of atoms. Comput.-Aided Des. 38, 1192–1204 (2006)
Kiwiel, K.C.: Generalized Bregman projections in convex feasibility problems. J. Optim. Theory Appl. 96, 139–157 (1998)
Kiwiel, K.C.: Convergence of approximate and incremental subgradient methods for convex optimization. SIAM J. Optim. 14, 807–840 (2004)
Kopecká, E., Reem, D., Reich, S.: Zone diagrams in compact subsets of uniformly convex spaces. Israel J. Math. 188, 1–23 (2012)
Lasserre, J.B.: Global optimization with polynomials and the problem of moments. SIAM J. Optim. 11, 796–817 (2001)
Martínez-Legaz, J.-.E.: Quasiconvex duality theory by generalized conjugation methods. Optimization 19, 603–652 (1988)
Martínez-Legaz, J.E., Sach, P.H.: A new subdifferential in quasiconvex analysis. J. Convex Anal. 6, 1–11 (1999)
Monteiro, R.D.C., Svaiter, B.F.: An accelerated hybrid proximal extragradient method for convex optimization and its implications to second-order methods. SIAM J. Optim. 23, 1092–1125 (2013)
Mordukhovich, B.S.: Maximum principle in problems of time optimal control with nonsmooth constraints. J. Appl. Math. Mech. 40, 960–969 (1976)
Nedić, A., Bertsekas, D.P.: The effect of deterministic noise in subgradient methods. Math. Program. Ser. A 125, 75–99 (2010)
Ngai, H.V., Luc, D.T., Thera, M.: Approximate convex functions. J. Nonlinear Convex Anal. 1, 155–176 (2000)
Okabe, A., Boots, B., Sugihara, K., Chiu, S.N.: Spatial Tessellations: Concepts and Applications of Voronoi Diagrams, 2nd edn. Wiley, Chichester (2000)
Opial, Z.: Weak convergence of the sequence of successive approximations for nonexpansive mappings. Bull. Am. Math. Soc. 73, 591–597 (1967)
Ostrowski, A.M.: The round-off stability of iterations. Zeitschrift für Angewandte Math. und Mech. 47, 77–81 (1967)
Otero, R.G., Iusem, A.: Fixed-point methods for a certain class of operators. J. Optim. Theory Appl. 159, 656–672 (2013)
Pallaschke, D., Rolewicz, S.: Foundations of Mathematical Optimization: Convex Analysis Without Linearity. Kluwer, Norwell (1997)
Penfold, S.N., Schulte, R.W., Censor, Y., Rosenfeld, A.B.: Total variation superiorization schemes in proton computed tomography image reconstruction. Med. Phys. 37, 5887–5895 (2010)
Penot, J.-.P.: Are generalized derivatives useful for generalized convex functions? In: Martinez-Legaz, J.E., Crouzeix, J.-.P., Volle, M. (eds.) Generalized convexity, Generalized Monotonicity: Recent Results. Kluwer, Dordrecht, pp. 3–59 (1998)
Pintér, J.: Global Optimization in Action: Continuous and Lipschitz Optimization-Algorithms, Implementations, and Applications. Kluwer, Dordrecht (1996)
Plastria, F.: Lower subdifferentiable functions and their minimization by cutting planes. J. Optim. Theory Appl. 46, 37–53 (1985)
Pustylnik, E., Reich, S., Zaslavski, A.J.: Inexact orbits of nonexpansive mappings. Taiwan. J. Math. 12, 1511–1523 (2008)
Pustylnik, E., Reich, S., Zaslavski, A.J.: Inexact infinite products of nonexpansive mappings. Numer. Funct. Anal. Optim. 30, 632–645 (2009)
Razumichin, B.S.: Classical Principles and Optimization Problems. D. Reidel Publishing Company, Dordrecht (1987)
Reem, D.: An algorithm for computing Voronoi diagrams of general generators in general normed spaces. In: Proceedings of the Sixth International Symposium on Voronoi Diagrams in Science and Engineering (ISVD 2009), Copenhagen, Denmark, pp. 144–152
Reem, D.: The geometric stability of Voronoi diagrams with respect to small changes of the sites, (2011), Complete version in arXiv 1103.4125 (last updated: April 6: Extended abstract. In: Proceedings of the 27th Annual Symposium on Computational Geometry (SoCG 2011), pp. 254–263. France, Paris (2011)
Reem, D.: The Bregman distance without the Bregman function II. Contemp. Math. 568, 213–223 (2012)
Rockafellar, R.T.: Convex Anal. Princeton University Press, Princeton (1970)
Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)
Rockafellar, R.T.: Directionally Lipschitzian functions and subdifferential calculus. Proc. Lond. Math. Soc. 39, 331–355 (1979)
Ryu, J., Kim, D., Cho, Y., Park, R., Kim, D.-.S.: Computation of molecular surface using Euclidean Voronoi diagram. Comput.-Aided Des. Appl. 2, 439–448 (2005)
Soleimani-damaneh, M.: Nonsmooth optimization using Mordukhovich’s subdifferential. SIAM J. Control Optim. 48, 3403–3432 (2010)
Solodov, M.V., Zavriev, S.K.: Error stability properties of generalized gradient-type algorithms. J. Optim. Theory Appl. 98, 663–680 (1998)
Solodov, M.V., Svaiter, B.F.: An inexact hybrid generalized proximal point algorithm and some new results in the theory of Bregman functions. Math. Oper. Res. 51, 479–494 (2000)
Solodov, M.V., Svaiter, B.F.: A unified framework for some inexact proximal point algorithms. Numer. Funct. Anal. Optim. 22, 1013–1035 (2001)
Svaiter, B.F.: Personal Communication (2011)
Svaiter, B.F.: Personal Communication (2012)
Tuncel, L.: Polyhedral and Semidefinite Programming Methods in Combinatorial Optimization, A co-publication of the American Mathematical Society (AMS) and the Fields Institute for Research in Mathematical Sciences. Providence, RI, USA (2010)
Van Tiel, J.: Convex Analysis: An Introductory Text. Wiley, Chichester (1984)
Zaslavski, A.J.: Subgradient projection algorithms for convex feasibility problems in the presence of computational errors. J. Approx. Theory 175, 19–42 (2013)
Zaslavski, A.J.: Subgradient projection algorithms and approximate solutions of convex feasibility problems. J. Optim. Theory Appl. 157, 803–819 (2013)
Acknowledgments
We thank Luba Tetruashvili for many helpful comments. We thank Benar Svaiter for helpful discussions and suggestions, in particular, for his ideas regarding the alternative proof of Proposition 1c mentioned in Remark 4. Thanks are also due to Jefferson Melo, Alfredo Iusem, Simeon Reich, Alex Segal, and Mikhail Solodov for helpful remarks, in particular regarding some of the references, and to Claudia Sagastizábal for a helpful discussion on some general aspects of the paper. Finally, we greatly appreciate the constructive and insightful comments of three anonymous reviewers and the Associate Editor which helped us improve the paper. This research was supported by the United States-Israel Binational Science Foundation (BSF) Grant Number 200912, by the US Department of Army award number W81XWH-10-1-0170, and by a special postdoc fellowship from IMPA.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was done while the Daniel Reem was at the Department of Mathematics, University of Haifa, Haifa, Israel (2010–2011), and at IMPA—Instituto Nacional de Matemática Pura e Aplicada, Rio de Janeiro, Brazil (2011–2013).
Rights and permissions
About this article
Cite this article
Censor, Y., Reem, D. Zero-convex functions, perturbation resilience, and subgradient projections for feasibility-seeking methods. Math. Program. 152, 339–380 (2015). https://doi.org/10.1007/s10107-014-0788-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-014-0788-7
Keywords
- Feasibility problem
- Perturbation resilience
- Subgradient projection method
- Superiorization
- Voronoi function
- Zero-convexity