Skip to main content
SpringerLink
Log in

Generalized forward–backward splitting with penalization for monotone inclusion problems

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

We introduce a generalized forward–backward splitting method with penalty term for solving monotone inclusion problems involving the sum of a finite number of maximally monotone operators and the normal cone to the nonempty set of zeros of another maximally monotone operator. We show weak ergodic convergence of the generated sequence of iterates to a solution of the considered monotone inclusion problem, provided that the condition corresponding to the Fitzpatrick function of the operator describing the set of the normal cone is fulfilled. Under strong monotonicity of an operator, we show strong convergence of the iterates. Furthermore, we utilize the proposed method for minimizing a large-scale hierarchical minimization problem concerning the sum of differentiable and nondifferentiable convex functions subject to the set of minima of another differentiable convex function. We illustrate the functionality of the method through numerical experiments addressing constrained elastic net and generalized Heron location problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Attouch, H., Czarnecki, M.-O.: Asymptotic behavior of coupled dynamical systems with multiscale aspects. J. Differ. Equ. 248(6), 1315–1344 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  2. Attouch, H., Czarnecki, M.-O., Peypouquet, J.: Prox-penalization and splitting methods for constrained variational problems. SIAM J. Optim. 21(1), 149–173 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  3. Attouch, H., Czarnecki, M.-O., Peypouquet, J.: Coupling forward–backward with penalty schemes and parallel splitting for constrained variational inequalities. SIAM J. Optim. 21(4), 1251–1274 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  4. Attouch, H., Peypouquet, J.: The rate of convergence of Nesterov’s accelerated forward–backward method is actually faster than \(1/k^2\). SIAM J. Optim. 26, 1824–1834 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  5. Baillon, J.-B., Haddad, G.: Quelques propriétés des opérateurs angle-bornés et \(n\)-cycliquement monotones. Israel J. Math. 26, 137–150 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  6. Banert, S., Boţ, R.I.: Backward penalty schemes for monotone inclusion problems. J. Optim. Theory Appl. 166(3), 930–948 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011)

    Book  MATH  Google Scholar 

  8. Bottou, L., Curtis, F.E., Nocedal, J.: Optimization methods for large-scale machine learning. SIAM Rev. 60(2), 223–311 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  9. Boţ, R.I., Csetnek, E.R.: A Tseng’s type penalty scheme for solving inclusion problems involving linearly composed and parallel-sum type monotone operators. Vietnam J. Math. 42(4), 451–465 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  10. Boţ, R.I., Csetnek, E.R.: Forward–backward and Tseng’s type penalty schemes for monotone inclusion problems. Set-Valued Var. Anal. 22, 313–331 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  11. Boţ, R.I., Csetnek, E.R.: An inertial forward–backward–forward primal–dual splitting algorithm for solving monotone inclusion problems. Numer. Algorithms 71, 519–540 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  12. Boţ, R.I., Csetnek, E.R.: Penalty schemes with inertial effects for monotone inclusion problems. Optimization 66, 965–982 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  13. Boţ, R.I., Csetnek, E.R., Nimana, N.: Gradient-type penalty method with inertial effects for solving constrained convex optimization problems with smooth data. Optim. Lett. 12(1), 17–33 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  14. Boţ, R.I., Csetnek, E.R., Nimana, N.: An inertial proximal-gradient penalization scheme for constrained convex optimization problems. Vietnam J. Math. 46(1), 53–71 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  15. Chen, C., Chan, R.H., MA, S., Yang, J.: Inertial proximal ADMM for linearly constrained separable convex optimization. SIAM J. Imaging Sci. 8(4), 2239–2267 (2015)

  16. Chen, C., Ma, S., Yang, J.: A general inertial proximal point algorithm for mixed variational inequality problem. SIAM J. Optim. 25(4), 2120–2142 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  17. Combettes, P.L., Pesquet, J.-C.: Primal–dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators. Set-Valued Var. Anal. 20, 307–330 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  18. Czarnecki, M.-O., Noun, N., Peypouquet, J.: Splitting forward–backward penalty scheme for constrained variational problems. J. Convex Anal. 23, 531–565 (2016)

    MathSciNet  MATH  Google Scholar 

  19. Fitzpatrick, S.: Representing monotone operators by convex functions. In: Workshop/Miniconference on Functional Analysis and Optimization (Canberra, 1988), pp. 59–65, Proc. Centre Math. Anal. Austral. Nat. Univ., 20, Austral. Nat. Univ., Canberra (1988)

  20. Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12, 55–67 (1970)

    Article  MATH  Google Scholar 

  21. Huang, T., Gong, H., Yang, C., He, Z.: ProteinLasso: a Lasso regression approach to protein inference problem in shotgun proteomics. Comp. Biol. Chem. 43, 46–54 (2013)

    Article  Google Scholar 

  22. Mairal, J.: Incremental majorization–minimization optimization with application to large-scale machine learning. SIAM J. Optim. 25(2), 829–855 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  23. Noun, N., Peypouquet, J.: Forward–backward penalty scheme for constrained convex minimization without inf-compactness. J. Optim. Theory Appl. 158(3), 787–795 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  24. Ochs, P., Chen, Y., Brox, T., Pock, T.: iPiano: inertial proximal algorithm for non-convex optimization. SIAM J. Imaging Sci. 7(2), 1388–1419 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  25. Passty, G.B.: Ergodic convergence to a zero of the sum of monotone operators in Hilbert space. J. Math. Anal. Appl. 72, 383–390 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  26. Peypouquet, J.: Coupling the gradient method with a general exterior penalization scheme for convex minimization. J. Optim. Theory Appl. 153(1), 123–138 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  27. Polyak, B.T.: Introduction to Optimization (Translated from the Russian). Translations Series in Mathematics and Engineering. Optimization Software Inc., Publications Division, New York (1987)

    Google Scholar 

  28. Raguet, H., Fadili, J., Peyré, G.: A generalized forward–backward splitting: SIAM. J. Imaging Sci. 6, 1199–1226 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  29. Raguet, H., Landrieu, L.: Preconditioning of a generalized forward–backward splitting and application to optimization on graphs. SIAM J. Imaging Sci. 8, 2706–2739 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  30. Schmidt, M., Le Roux, N., Bach, F.: Minimizing finite sums with the stochastic average gradient. Math. Program. 162(1), 83–112 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  31. Vũ, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. 38, 667–681 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  32. Shalev-Schwartz, S., Zhang, T.: Stochastic dual coordinate ascent methods for regularized loss minimization. J. Mach. Learn. Res. 14, 567–599 (2013)

    MathSciNet  MATH  Google Scholar 

  33. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 58, 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  34. Zălinescu, C.: Convex Analysis in General Vector Spaces. World Scientific, Singapore (2002)

    Book  MATH  Google Scholar 

  35. Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B Stat. Methodol. 67, 301–320 (2005)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors are thankful to two anonymous referees and the Associate Editor for comments and remarks which improved the quality of the paper.

Author information

Authors and Affiliations

  1. Department of Mathematics, Faculty of Science, Khon Kaen University, Khon Kaen, 40002, Thailand

    Nimit Nimana

  2. Department of Mathematics, Faculty of Science, Naresuan University, Phitsanulok, 65000, Thailand

    Narin Petrot

  3. Center of Excellence in Nonlinear Analysis and Optimization, Faculty of Science, Naresuan University, Phitsanulok, 65000, Thailand

    Narin Petrot

Authors
  1. Nimit Nimana
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Narin Petrot
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Narin Petrot.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is partially supported by the Thailand Research Fund under the Project RSA5880028.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nimana, N., Petrot, N. Generalized forward–backward splitting with penalization for monotone inclusion problems. J Glob Optim 73, 825–847 (2019). https://doi.org/10.1007/s10898-018-00730-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-018-00730-5

Keywords

Mathematics Subject Classification

Search

Navigation