Advertisement

Set-Valued and Variational Analysis

, Volume 26, Issue 4, pp 1009–1078 | Cite as

Subgradient Projectors: Extensions, Theory, and Characterizations

  • Heinz H. BauschkeEmail author
  • Caifang Wang
  • Xianfu Wang
  • Jia Xu
Article

Abstract

Subgradient projectors play an important role in optimization and for solving convex feasibility problems. For every locally Lipschitz function, we can define a subgradient projector via generalized subgradients even if the function is not convex. The paper consists of three parts. In the first part, we study basic properties of subgradient projectors and give characterizations when a subgradient projector is a cutter, a local cutter, or a quasi-nonexpansive mapping. We present global and local convergence analyses of subgradent projectors. Many examples are provided to illustrate the theory. In the second part, we investigate the relationship between the subgradient projector of a prox-regular function and the subgradient projector of its Moreau envelope. We also characterize when a mapping is the subgradient projector of a convex function. In the third part, we focus on linearity properties of subgradient projectors. We show that, under appropriate conditions, a linear operator is a subgradient projector of a convex function if and only if it is a convex combination of the identity operator and a projection operator onto a subspace. In general, neither a convex combination nor a composition of subgradient projectors of convex functions is a subgradient projector of a convex function.

Keywords

Approximately convex function Averaged mapping Cutter Essentially strictly differentiable function Fixed point Limiting subgradient Local cutter Local quasi-firmly nonexpansive mapping Local quasi-nonexpansive mapping Local Lipschitz function Linear cutter Linear firmly nonexpansive mapping Linear subgradient projection operator Moreau envelope Projection Prox-bounded Proximal mapping Prox-regular function Quasi-firmly nonexpansive mapping Quasi-nonexpansive mapping (C,ε)-firmly nonexpansive mapping Subdifferentiable function Subgradient projection operator 

Mathematics Subject Classification (2010)

Primary 49J52 Secondary 49J53 47H04 47H05 47H09 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgments

The authors thank two anonymous referees for careful reading and constructive suggestions on the paper. HHB was partially supported by a Discovery Grant of the Natural Sciences and Engineering Research Council of Canada (NSERC) and by the Canada Research Chair Program. CW was partially supported by National Natural Science Foundation of China (11401372). XW was partially supported by a Discovery Grant of the Natural Sciences and Engineering Research Council of Canada (NSERC). JX was supported by by NSERC grants of HHB and XW.

References

  1. 1.
    Adly, S., Nacry, F., Thibault, L.: Preservation of prox-regularity of sets with applications to constrained optimization. SIAM J. Optim. 26(1), 448–473 (2016)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Aussel, D., Daniilidis, A., Thibault, L.: Subsmooth sets: functional characterizations and related concepts. Trans. Amer. Math. Soc. 357(4), 1275–1301 (2005)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bacák, M., Borwein, J.M., Eberhard, A., Mordukhovich, B.S.: Infimal convolutions and Lipschitzian properties of subdifferentials for prox-regular functions in Hilbert spaces. J. Convex Anal. 17(3-4), 737–763 (2010)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Baillon, J.B., Bruck, R.R., Reich, S.: On the asymptotic behavior of nonexpansive mappings and semigroups in Banach spaces. Houston J. Math. 4, 1–9 (1978)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Bauschke, H.H., Borwein, J.M., Combettes, P.L.: Bregman monotone optimization algorithms. SIAM J. Control Optim. 42, 596–636 (2003)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer (2011)Google Scholar
  7. 7.
    Bauschke, H.H., Combettes, P.L.: A weak-to-strong convergence principle for Fejer-monotone methods in Hilbert spaces. Math. Oper. Res. 26, 248–264 (2001)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Bauschke, H.H., Combettes, P.L., Noll, D.: Joint minimization with alternating Bregman proximity operators. Pac. J. Optim. 2(3), 401–424 (2006)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Bauschke, H.H., Chen, J., Wang, X.: A Bregman projection method for approximating fixed points of quasi-Bregman nonexpansive mappings. Appl. Anal. 94, 75–84 (2015)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Bauschke, H.H., Wang, C., Wang, X., Xu, J.: On subgradient projectors. SIAM J. Optim. 25(2), 1064–1082 (2015)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Bauschke, H.H., Wang, C., Wang, X., Xu, J.: On the finite convergence of a projected cutter method. J. Optim. Theory Appl. 165(3), 901–916 (2015)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Bauschke, H.H., Wang, X., Yao, L.: Monotone linear relations: maximality and Fitzpatrick functions. J. Convex Anal. 16, 673–686 (2009)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Bernard, F., Thibault, L.: Prox-regular functions in Hilbert spaces. J. Math. Anal. Appl. 303(1), 1–14 (2005)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Bernard, F., Thibault, L.: Prox-regularity of functions and sets in Banach spaces. Set-Valued Anal. 12(1-2), 25–47 (2004)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Borwein, J.M., Moors, W.B.: Essentially smooth Lipschitz functions. J. Funct. Anal. 149(2), 305–351 (1997)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Borwein, J.M., Moors, W.B., Wang, X.: Generalized subdifferentials: a Baire categorical approach. Trans. Amer. Math. Soc. 353(10), 3875–3893 (2001)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Borwein, J.M., Lewis, A.S.: Convex Analysis and Nonlinear Optimization: Theory and Examples, 2nd edn. Springer, New York (2006)CrossRefGoogle Scholar
  18. 18.
    Borwein, J.M., Zhu, Q.J.: Techniques of Variational Analysis. Springer-Verlag, New York (2005)zbMATHGoogle Scholar
  19. 19.
    Capricelli, T.D.: Algorithmes de Projections Convexes Généralisées et Applications en Imagerie Médicale, Ph.D. dissertation. University Pierre & Marie Curie (Paris 6), Paris (2008)Google Scholar
  20. 20.
    Cegielski, A.: Iterative Methods for Fixed Point Problems in Hilbert Spaces, Lecture Notes in Mathematics, 2057. Springer, Heidelberg (2012)Google Scholar
  21. 21.
    Censor, Y., Chen, W., Pajoohesh, H.: Finite convergence of a subgradient projections method with expanding controls. Appl. Math. Optim. 64(2), 273–285 (2011)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Censor, Y., Lent, A.: Cyclic subgradient projections. Math. Programming 24(1), 233–235 (1982)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Clarke, F.H.: Optimization and Nonsmooth Analysis, Second Edition Classics in Applied Mathematics, 5. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA (1990)Google Scholar
  24. 24.
    Combettes, P.L.: Convex set theoretic image recovery by extrapolated iterations of parallel subgradient projections. IEEE Trans. Image Process. 6(4), 493–506 (1997)CrossRefGoogle Scholar
  25. 25.
    Combettes, P.L., Luo, J.: An adaptive level set method for nondifferentiable constrained image recovery. IEEE Trans. Image Process. 11(11), 1295–1304 (2002)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Daniilidis, A., Georgiev, P.: Approximate convexity and submonotonicity. J. Math. Anal. Appl. 291(1), 292–301 (2004)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Deutsch, F.: Best Approximation in Inner Product Spaces. Springer (2001)Google Scholar
  28. 28.
    Evans, L. C., Gariepy, R. F.: Measure Theory and Fine Properties of Functions, Studies in Advanced Mathematics. CRC Press, Boca Raton FL (1992)zbMATHGoogle Scholar
  29. 29.
    Fukushima, M.: A finitely convergent algorithm for convex inequalities. IEEE Trans. Automat. Control 27(5), 1126–1127 (1982)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Hesse, R., Luke, D.R.: Nonconvex notions of regularity and convergence of fundamental algorithms for feasibility problems. SIAM J. Optim. 23(4), 2397–2419 (2013)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Ioffe, A.D.: Approximate subdifferentials and applications, I: the finite-dimensional theory. Trans. Amer. Math. Soc. 281(1), 389–416 (1984)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Jourani, A., Thibault, L., Zagrodny, D.: Differential properties of the Moreau envelope. J. Funct. Anal. 266(3), 1185–1237 (2014)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Kan, C., Song, W.: The Moreau envelope function and proximal mapping in the sense of the Bregman distance. Nonlinear Anal. 75(3), 1385–1399 (2012)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Kiwiel, K.C.: The efficiency of subgradient projection methods for convex optimization. I. General level methods. SIAM J. Control Optim. 34(2), 660–676 (1996)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Kiwiel, K.C.: The efficiency of subgradient projection methods for convex optimization. II. Implementations and extensions. SIAM J. Control Optim. 34(2), 677–697 (1996)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Kiwiel, K.C.: A Bregman-projected subgradient method for convex constrained nondifferentiable minimization. In: Operations Research Proceedings 1996 (Braunschweig), pp 26–30. Springer, Berlin (1997)zbMATHGoogle Scholar
  37. 37.
    Luke, D.R., Thao, N.H., Tam, M.K.: Quantitative convergence analysis of iterated expansive, set-valued mappings. arXiv:1605.05725 (2016)
  38. 38.
    Meyer, C.: Matrix Analysis and Applied Linear Algebra. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA (2000)Google Scholar
  39. 39.
    Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation I. Springer-Verlag (2006)Google Scholar
  40. 40.
    Mordukhovich, B.S., Shao, Y.: Nonsmooth sequential analysis in Asplund spaces. Trans. Amer. Math. Soc. 348, 1235–1280 (1996)MathSciNetCrossRefGoogle Scholar
  41. 41.
    Ogura, N., Yamada, I.: A deep outer approximating half space of the level set of certain quadratic functions. J. Nonlinear Convex Anal. 6(1), 187–201 (2005)MathSciNetzbMATHGoogle Scholar
  42. 42.
    Pang, C.H.: Finitely convergent algorithm for nonconvex inequality problems, arXiv:1405.7280 (2014)
  43. 43.
    Pauwels, B.: Subgradient projection operators, arXiv:1403.7237v1 (2014)
  44. 44.
    Poliquin, R.A., Rockafellar, R.T.: Prox-regular functions in variational analysis. Trans. Amer. Math. Soc. 348(5), 1805–1838 (1996)MathSciNetCrossRefGoogle Scholar
  45. 45.
    Polyak, B.T.: Minimization of unsmooth functionals. USSR Comput. Math. Math. Phys. 9, 14–29 (1969). (The original version appeared in Akademija Nauk SSSR. žurnal Vyčislitel’ noı̆ Matematiki i Matematičeskoı̆ Fiziki 9 (1969), 509–521.)CrossRefGoogle Scholar
  46. 46.
    Polyak, B.T.: Introduction to Optimization. Optimization Software (1987)Google Scholar
  47. 47.
    Polyak, B.T.: Random algorithms for solving convex inequalities Butnariu, D., Censor, Y., Reich, S. (eds.) Inherently Parallel Algorithms in Feasibility and Optimization and Their Applications, pp. 409–422. Elsevier (2001)Google Scholar
  48. 48.
    Richardson, L.F.: Measure and Integration: A Concise Introduction to Real Analysis. Wiley (2009)Google Scholar
  49. 49.
    Rockafellar, R.T.: Convex Analysis. Princeton University Press (1970)Google Scholar
  50. 50.
    Rockafellar, R.T., Wets, R.: Variational Analysis. Springer corrected 3rd printing (2009)Google Scholar
  51. 51.
    Spingarn, J.E.: Submonotone subdifferentials of Lipschitz functions. Trans. Amer. Math. Soc. 264(1), 77–89 (1981)MathSciNetCrossRefGoogle Scholar
  52. 52.
    Van Ngai, H., Luc, D.T., Théra, M.: Approximate convex functions. J. Nonlinear Convex Anal. 1(2), 155–176 (2000)MathSciNetzbMATHGoogle Scholar
  53. 53.
    Wang, X.: Subdifferentiability of real functions. Real Anal. Exchange 30(1), 137–171 (2004/05)MathSciNetCrossRefGoogle Scholar
  54. 54.
    Xu, J.: Subgradient Projectors: Theory, Extensions, and Algorithms. Ph.D. dissertation, University of British Columbia, Kelowna (2016)Google Scholar
  55. 55.
    Yamada, I., Ogura, N.: Adaptive projected subgradient method for asymptotic minimization of sequence of nonnegative convex functions. Numer. Funct. Anal. Optim. 25(7-8), 593–617 (2004)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2017

Authors and Affiliations

  • Heinz H. Bauschke
    • 1
    Email author
  • Caifang Wang
    • 2
  • Xianfu Wang
    • 1
  • Jia Xu
    • 1
  1. 1.MathematicsUniversity of British ColumbiaKelownaCanada
  2. 2.Department of MathematicsShanghai Maritime UniversityShanghaiChina

Personalised recommendations