Advertisement

Numerical Algorithms

, Volume 78, Issue 3, pp 739–757 | Cite as

l 1-l 2 regularization of split feasibility problems

  • Abdellatif Moudafi
  • Aviv GibaliEmail author
Original Paper

Abstract

Numerous problems in signal processing and imaging, statistical learning and data mining, or computer vision can be formulated as optimization problems which consist in minimizing a sum of convex functions, not necessarily differentiable, possibly composed with linear operators and that in turn can be transformed to split feasibility problems (SFP); see for example Censor and Elfving (Numer. Algorithms 8, 221–239 1994). Each function is typically either a data fidelity term or a regularization term enforcing some properties on the solution; see for example Chaux et al. (SIAM J. Imag. Sci. 2, 730–762 2009) and references therein. In this paper, we are interested in split feasibility problems which can be seen as a general form of Q-Lasso introduced in Alghamdi et al. (2013) that extended the well-known Lasso of Tibshirani (J. R. Stat. Soc. Ser. B 58, 267–288 1996). Q is a closed convex subset of a Euclidean m-space, for some integer m ≥ 1, that can be interpreted as the set of errors within given tolerance level when linear measurements are taken to recover a signal/image via the Lasso. Inspired by recent works by Lou and Yan (2016), Xu (IEEE Trans. Neural Netw. Learn. Syst. 23, 1013–1027 2012), we are interested in a nonconvex regularization of SFP and propose three split algorithms for solving this general case. The first one is based on the DC (difference of convex) algorithm (DCA) introduced by Pham Dinh Tao, the second one is nothing else than the celebrate forward-backward algorithm, and the third one uses a method introduced by Mine and Fukushima. It is worth mentioning that the SFP model a number of applied problems arising from signal/image processing and specially optimization problems for intensity-modulated radiation therapy (IMRT) treatment planning; see for example Censor et al. (Phys. Med. Biol. 51, 2353–2365, 2006).

Keywords

Q-Lasso Split feasibility Soft-thresholding Regularization DCA algorithm Forward-backward iterations Mine-Fukushima algorithm Douglas-Rachford algorithm 

Mathematics Subject Classification (2010)

Primary 49J53, 65K10 Secondary 49M37, 90C25 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgments

We wish to thank the anonymous referee for the thorough analysis and review; all the comments and suggestions helped tremendously in improving the quality of this paper and made it suitable for publication. The first author wish to thank his team “Image & Modèle” and the Computer Science and System Laboratory (L.I.S) of Aix-Marseille University. The second author work is supported by the EU FP7 IRSES program STREVCOMS, grant no. PIRSES-GA-2013-612669.

References

  1. 1.
    Alghamdi, M.A., Ali Alghamdi, M., Shahzad, N., Naseer, H.-K. X.: Properties and Iterative Methods for the Q-Lasso, Abstract and Applied Analysis. Article ID 250943, 8 pages (2013)Google Scholar
  2. 2.
    Byrne, C.: Iterative oblique projection onto convex sets and the split feasibility problem. Inverse Prob. 18, 441–453 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Byrne, C.: A unified treatment of some iterative algorithms in signal processing and image reconstruction. Inverse Prob. 20, 103–120 (2004)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Censor, Y., Bortfeld, T., Martin, B., Trofimov, A.: A unified approach for inversion problems in intensity-modulated radiation therapy. Phys. Med. Biol. 51, 2353–2365 (2006)CrossRefGoogle Scholar
  5. 5.
    Censor, Y., Elfving, T.: A multiprojection algorithm using Bregman projections in a product space. Numer. Algorithms 8, 221–239 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Censor, Y., Gibali, A., Lenzen, F., Schnorr, Ch.: The implicit convex feasibility problem and its application to adaptive image denoising. J. Comput. Math. 34, 610–625 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Chartrand, R.: Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Process Lett. 14, 707–710 (2007)CrossRefGoogle Scholar
  8. 8.
    Chen, S., Donoho, D., Saunders, M.: Atomic decomposition by basis pursuit. SIAM J. Comput. 20, 33–61 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Chaux, C., Pesquet, J.-C., Pustelnik, N.: Nested iterative algorithms for convex constrained image recovery problems. SIAM J. Imag. Sci. 2, 730–762 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20, 33–61 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Combettes, P.L., Pesquet, J.-C.: A Douglas-Rachford splitting approach to nonsmooth convex variational signal recovery. IEEE J. Sel. Top. Sign. Proces. 1, 564–574 (2007)CrossRefGoogle Scholar
  12. 12.
    Condat, L.: A generic proximal algorithm for convex optimization: application to total variation minimization. IEEE Signal Process Lett. 21, 985–989 (2014)CrossRefGoogle Scholar
  13. 13.
    Donoho, D.: Compressed sensing. IEEE Trans. Inf. Theory 52, 1289–1306 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Esser, E., Lou, Y., Xin, J.: A method for finding structured sparse solutions to non-negative least squares problems with applications. SIAM J. Imag. Sci. 6, 2010–2046 (2013)CrossRefzbMATHGoogle Scholar
  15. 15.
    Lou, Y., Yan, M.: Fast l 1l 2 Minimization via a proximal operator. arXiv:1609.09530 (2017)
  16. 16.
    Micchelli, Ch.A., Shen, L., Xu, Y., Zeng, X.: Proximity algorithms for the L 1/TV image denoising model. Adv. Comput. Math. 38, 401–426 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Mine, H., Fukushima, M.: A minimization method for the sum of a convex function and a continuously differentiable function. J. Optim. Theory Appl. 33, 9–23 (1981)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Moudafi, A.: About proximal algorithms for Q-Lasso, Thai Mathematical Journal (2016)Google Scholar
  19. 19.
    Natarajan, B.K.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24, 227–234 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Qu, B., Xiu, N.: A note on the CQ algorithm for the split feasibility problem. Inverse Prob. 21, 1655–1665 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Tang, Y.-C., Liu, L.-W., Gibali, A.: Note on the modified relaxation CQ algorithm for the split feasibility problem. Optim. Lett., 1–14.  https://doi.org/10.1007/s11590-017-1148-3 (2017)
  22. 22.
    Tibshirani, R.: Regression Shrinkage and selection via the Lasso. J. R. Stat. Soc. Ser. B 58, 267–288 (1996)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Yin, P., Lou, Y., He, Q., Xin, J.: Minimization of l 1−2 for compressed sensing. SIAM J. Sci. Comput. 37, 536–563 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Xu, Z., Chang, X., Xu, F., Zhang, H.: L 1−2 regularization: a thresholding representation theory and a fast solver. IEEE Trans. Neural Netw. Learn. Syst. 23, 1013–1027 (2012)CrossRefGoogle Scholar
  25. 25.
    Zeng, X., Figueiredo, M.A.-T.: Solving OSCAR regularization problems by fast approximate proximal splittings algorithms. Digital Signal Process. 31, 124–135 (2014)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.CNRS-L.I.S UMRAix-Marseille UniversitéMarseilleFrance
  2. 2.Domaine Universitaire de Saint-JérômeMarseilleFrance
  3. 3.Department of MathematicsORT Braude CollegeKarmielIsrael

Personalised recommendations