Signal, Image and Video Processing

, Volume 9, Issue 8, pp 1737–1749 | Cite as

Epigraphical projection and proximal tools for solving constrained convex optimization problems

  • G. Chierchia
  • N. Pustelnik
  • J.-C. Pesquet
  • B. Pesquet-Popescu
Invited Paper

Abstract

We propose a proximal approach to deal with a class of convex variational problems involving nonlinear constraints. A large family of constraints, proven to be effective in the solution of inverse problems, can be expressed as the lower-level set of a sum of convex functions evaluated over different blocks of the linearly transformed signal. For such constraints, the associated projection operator generally does not have a simple form. We circumvent this difficulty by splitting the lower-level set into as many epigraphs as functions involved in the sum. In particular, we focus on constraints involving \(\varvec{\ell }_q\)-norms with \(q\ge 1\), distance functions to a convex set, and \(\varvec{\ell }_{1,p}\)-norms with \(p\in \{2,{+\infty }\}\). The proposed approach is validated in the context of image restoration by making use of constraints based on Non-Local Total Variation. Experiments show that our method leads to significant improvements in term of convergence speed over existing algorithms for solving similar constrained problems.

Keywords

Iterative methods Optimization Epigraph Projection Proximal algorithms Restoration Total variation Non-local regularization Patch-based processing 

References

  1. 1.
    Combettes, P.L., Pesquet, J.-C.: Proximal splitting methods in signal processing. In: Bauschke, H.H., Burachik, R.S., Combettes, P.L., Elser, V., Luke, D.R., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pp. 185–212. Springer, New York (2011)CrossRefGoogle Scholar
  2. 2.
    Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends Optim. 1(3), 123–231 (2013)Google Scholar
  3. 3.
    Dupé, F.-X., Fadili, M.J., Starck, J.-L.: A proximal iteration for deconvolving Poisson noisy images using sparse representations. IEEE Trans. Image Process. 18(2), 310–321 (2009)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Aujol, J.-F., Gilboa, G., Chan, T., Osher, S.: Structure-texture image decomposition—modeling, algorithms, and parameter selection. Int. J. Comput. Vis. 67(1), 111–136 (2006)MATHCrossRefGoogle Scholar
  5. 5.
    Brice no-Arias, L.M., Combettes, P.L., Pesquet, J.-C., Pustelnik, N.: Proximal algorithms for multicomponent image recovery problems. J. Math. Imaging Vis. 41(1), 3–22 (Sep.2011)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Theodoridis, S., Slavakis, K., Yamada, I.: Adaptive learning in a world of projections. IEEE Signal Process. Mag. 28(1), 97–123 (2011)CrossRefGoogle Scholar
  7. 7.
    Chaux, C., El Gheche, M., Farah, J., Pesquet, J.-C., Pesquet-Popescu, B.: A parallel proximal splitting method for disparity estimation from multicomponent images under illumination variation. J. Math. Imaging Vis. 47(3), 167–178 (2013)MATHCrossRefGoogle Scholar
  8. 8.
    Mallat, S.: A Wavelet Tour of Signal Processing. Academic Press, San Diego (1997)Google Scholar
  9. 9.
    Jacques, L., Duval, L., Chaux, C., Peyré, G.: A panorama on multiscale geometric representations, intertwining spatial, directional and frequency selectivity. Signal Process. 91(12), 2699–2730 (2011)CrossRefGoogle Scholar
  10. 10.
    Van Den Berg, E., Friedlander, M.P.: Probing the Pareto frontier for basis pursuit solutions. SIAM J. Sci. Comput. 31(2), 890–912 (2008)MATHMathSciNetCrossRefGoogle Scholar
  11. 11.
    Ciak, R., Shafei, B., Steidl, G.: Homogeneous penalizers and constraints in convex image restoration. J. Math. Imaging Vis. 47, 210–230 (2013)Google Scholar
  12. 12.
    Youla, D.C., Webb, H.: Image restoration by the method of convex projections. Part I—theory. IEEE Trans. Med. Imaging 1(2), 81–94 (1982)CrossRefGoogle Scholar
  13. 13.
    Trussell, H.J., Civanlar, M.R.: The feasible solution in signal restoration. IEEE Trans. Acoust. Speech Signal Process. 32(2), 201–212 (1984)CrossRefGoogle Scholar
  14. 14.
    Combettes, P.L.: Inconsistent signal feasibility problems: least-squares solutions in a product space. IEEE Trans. Signal Process. 42(11), 2955–2966 (1994)CrossRefGoogle Scholar
  15. 15.
    Teuber, T., Steidl, G., Chan, R.H.: Minimization and parameter estimation for seminorm regularization models with I-divergence constraints. Inverse Probl. 29(3), 035007 (2013)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Stück, R., Burger, M., Hohage, T.: The iteratively regularized Gauss-Newton method with convex constraints and applications in 4Pi microscopy. Inverse Probl. 28(1), 015012 (2012)CrossRefGoogle Scholar
  17. 17.
    Ono, S., Yamada I.: Poisson image restoration with likelihood constraint via hybrid steepest descent method. In: Proceedings of the International Conference on Acoustics, Speech and Signal Processing, pp. 5929–5933. Vancouver, Canada, May 2013Google Scholar
  18. 18.
    Harizanov, S., Pesquet, J.-C., Steidl, G.: Epigraphical projection for solving least squares anscombe transformed constrained optimization problems. Scale Space Var. Methods Comput. Vis. 7893, 125–136 (2013)CrossRefGoogle Scholar
  19. 19.
    Ono, S., Yamada I.: Second-order total generalized variation constraint. In: Proceedings of the International Conference on Acoustics, Speech and Signal Processing, Florence, Italy, May 4–9 2014Google Scholar
  20. 20.
    Tofighi, M., Kose, K., Cetin, A. E.: Signal reconstruction framework based on Projections onto Epigraph Set of a Convex cost function (PESC) (2014) (preprint)Google Scholar
  21. 21.
    Daubechies, I., Defrise, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. 57(11), 1413–1457 (2004). http://arxiv.org/abs/1402.2088
  22. 22.
    Chaux, C., Combettes, P.L., Pesquet, J.-C., Wajs, V.R.: A variational formulation for frame-based inverse problems. Inverse Probl. 23(4), 1495–1518 (2007)MATHMathSciNetCrossRefGoogle Scholar
  23. 23.
    Combettes, P.L., Pesquet, J.-C.: A Douglas–Rachford splitting approach to nonsmooth convex variational signal recovery. IEEE J. Sel. Top. Signal Process. 1(4), 564–574 (2007)CrossRefGoogle Scholar
  24. 24.
    Figueiredo, M., Nowak, R., Wright S.: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 5981(4), 586–598 (2007)Google Scholar
  25. 25.
    Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)MATHMathSciNetCrossRefGoogle Scholar
  26. 26.
    Fornasier, M., Schönlieb, C.-B.: Subspace correction methods for total variation and \(\ell _1\)-minimization. IMA J. Numer. Anal. 47(8), 3397–3428 (2009)MATHCrossRefGoogle Scholar
  27. 27.
    Steidl, G., Teuber, T.: Removing multiplicative noise by Douglas–Rachford splitting methods. J. Math. Imaging Vis. 36(3), 168–184 (2010)MATHMathSciNetCrossRefGoogle Scholar
  28. 28.
    Pesquet, J.-C., Pustelnik, N.: A parallel inertial proximal optimization method. Pac. J. Optim. 8(2), 273–305 (2012)MATHMathSciNetGoogle Scholar
  29. 29.
    Esser, E., Zhang, X., Chan, T.: A general framework for a class of first order primal–dual algorithms for convex optimization in imaging science. SIAM J. Imaging Sci. 3(4), 1015–1046 (2010)MATHMathSciNetCrossRefGoogle Scholar
  30. 30.
    Chambolle, A., Pock, T.: A first-order primal–dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)MATHMathSciNetCrossRefGoogle Scholar
  31. 31.
    Brice no-Arias, L.M., Combettes, P.L.: A monotone+ skew splitting model for composite monotone inclusions in duality. SIAM J. Optim. 21(4), 1230–1250 (2011)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Combettes, P.L., Pesquet, J.-C.: Primal–dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators. Set-valued Var. Anal. 20(2), 307–330 (2012)MATHMathSciNetCrossRefGoogle Scholar
  33. 33.
    Vũ, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. 38(3), 667–681 (2013)MATHMathSciNetCrossRefGoogle Scholar
  34. 34.
    Condat, L.: A primal–dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2012)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Chen, P., Huang, J., Zhang, X.: A primal–dual fixed point algorithm for convex separable minimization with applications to image restoration. Inverse Probl. 29(2), 025011 (2013)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Alghamdi, M.A., Alotaibi, A., Combettes, P.L., Shahzad, N.: A primal-dual method of partial inverses for composite inclusions. Optim. Lett. (2014). doi:10.1007/s11590-014-0734-x
  37. 37.
    Moreau, J.J.: Proximité et dualité dans un espace hilbertien. Bull. Soc. Math. Fr. 93, 273–299 (1965)MATHMathSciNetGoogle Scholar
  38. 38.
    Bauschke, H. H., Combettes, P. L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer (2011)Google Scholar
  39. 39.
    Combettes, P.L.: A block-iterative surrogate constraint splitting method for quadratic signal recovery. IEEE Trans. Signal Process. 51(7), 1771–1782 (2003)MathSciNetCrossRefGoogle Scholar
  40. 40.
    Kopsinis, Y., Slavakis, K., Theodoridis, S.: Online sparse system identification and signal reconstruction using projections onto weighted \(\ell _{1}\) balls. IEEE Trans. Signal Process. 59(3), 936–952 (2011)MathSciNetCrossRefGoogle Scholar
  41. 41.
    Quattoni, A., Carreras, X., Collins, M., Darrell, T.: An efficient projection for \(\ell _{1,\infty }\) regularization. In: International Conference on Machine Learning, Montreal, Quebec, June 14–18 2009Google Scholar
  42. 42.
    Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)MATHCrossRefGoogle Scholar
  43. 43.
    Gilboa, G., Osher, S.: Nonlocal operators with applications to image processing. Multiscale Model. Simul. 7(3), 1005 (2009)MathSciNetCrossRefGoogle Scholar
  44. 44.
    Gaetano, R., Chierchia, G., Pesquet-Popescu, B.: Parallel implementations of a disparity estimation algorithm based on a proximal splitting method. In Visual Communication and Image Processing, San Diego, USA (2012)Google Scholar
  45. 45.
    Candès, E.J., Wakin, M.B., Boyd, S.: Enhancing sparsity by reweighted \(\ell _1\) minimization. J. Fourier Anal. Appl. 14(5), 877–905 (2008)MATHMathSciNetCrossRefGoogle Scholar
  46. 46.
    Tikhonov, A.: Regularization of incorrectly posed problems. Sov. Math. Dokl. 4, 1624–1627 (1963)MATHGoogle Scholar
  47. 47.
    Combettes, P.L., Pesquet, J.-C.: A proximal decomposition method for solving convex variational inverse problems. Inverse Probl. 24(6), 065014 (2008)MathSciNetCrossRefGoogle Scholar
  48. 48.
    Wu, J., Liu, F., Jiao, L.C., Wang, X., Hou, B.: Multivariate compressive sensing for image reconstruction in the wavelet domain. IEEE Trans. Image Process. 20(12), 3483–3494 (2011)MathSciNetCrossRefGoogle Scholar
  49. 49.
    Turlach, B.A., Venables, W.N., Wright, S.J.: Simultaneous variable selection. Technometrics 47(3), 349–363 (2005)MathSciNetCrossRefGoogle Scholar
  50. 50.
    Chen, Y., Hero, A.O.: Recursive \(\ell _{1,\infty }\) lasso. IEEE Trans. Signal Process. 60(8), 3978–3987 (2012)MathSciNetCrossRefGoogle Scholar
  51. 51.
    Studer, C., Goldstein, T., Yin, W., Baraniuk,, R. G.: Democratic representations. (Jan. 2014). http://arxiv.org/abs/1401.3420
  52. 52.
    Afonso, M.V., Bioucas-Dias, J.M., Figueiredo, M.A.T.: An augmented Lagrangian approach to the constrained optimization formulation of imaging inverse problems. IEEE Trans. Image Process. 20(3), 681–695 (2011)MathSciNetCrossRefGoogle Scholar
  53. 53.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 8(1), 1–122 (2011) Google Scholar
  54. 54.
    Ding, C., Sun, D. F., Toh, K.-C.: An introduction to a class of matrix cone programming. Math. Program. 144(1–2), 141–179 (2014)Google Scholar
  55. 55.
    Donoho, D.L.: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 41(3), 613–627 (1995)MATHMathSciNetCrossRefGoogle Scholar
  56. 56.
    Bach, F., Jenatton, R., Mairal, J., Obozinski, G.: Optimization with sparsity-inducing penalties. Found. Trends Mach. Learn. 4(1), 1–106 (2012)CrossRefGoogle Scholar
  57. 57.
    Cetin, A.E., Tofighi, M.: Denosing using wavelets and projections onto the \(\ell _{1}\)-ball. Jun. (2014). http://arxiv.org/abs/1406.2528
  58. 58.
    Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3(3), 492–526 (2010)MATHMathSciNetCrossRefGoogle Scholar
  59. 59.
    Buades, A., Coll, B., Morel, J.: A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 4(2), 490–530 (2005)MATHMathSciNetCrossRefGoogle Scholar
  60. 60.
    Gilboa, G., Osher, S.: Nonlocal linear image regularization and supervised segmentation. Multiscale Model. Simul. 6(2), 595–630 (2007)MATHMathSciNetCrossRefGoogle Scholar
  61. 61.
    Foi, A., Boracchi, G.: Foveated self-similarity in nonlocal image filtering. In: Proceedings of SPIE Electronic Imaging, vol. 8291. Burlingame, USA, January 2012Google Scholar
  62. 62.
    Li, Y., Osher, S.: A new median formula with applications to PDE based denoising. Commun. Math. Sci. 7(3), 741–753 (2009)MATHMathSciNetCrossRefGoogle Scholar
  63. 63.
    Bresson, X., Chan, T.F.: Fast dual minimization of the vectorial total variation norm and applications to color image processing. Inverse Probl. Imaging 2(4), 455–484 (2008)MATHMathSciNetCrossRefGoogle Scholar
  64. 64.
    Miyata, T., Sakai, Y.: Vectorized total variation defined by weighted L-infinity norm for utilizing inter channel dependency. In: Proceedings of the Intenational Conference on Image Processing, pp. 3057–3060. September 2012Google Scholar
  65. 65.
    Wang, Z., Bovik, A.C.: Mean squared error: love it or leave it? IEEE Signal Process. Mag. 26(1), 98–117 (2009)CrossRefGoogle Scholar
  66. 66.
    Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. B 68, 49–67 (2006)MATHMathSciNetCrossRefGoogle Scholar
  67. 67.
    Ekeland, I., Témam, R.: Convex Analysis and Variational Problems. SIAM, Philadelphia (1999)MATHCrossRefGoogle Scholar
  68. 68.
    Combettes, P.L., Pesquet, J.-C.: Proximal thresholding algorithm for minimization over orthonormal bases. SIAM J. Optim. 18(4), 1351–1376 (2007)MATHMathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2014

Authors and Affiliations

  • G. Chierchia
    • 1
  • N. Pustelnik
    • 2
  • J.-C. Pesquet
    • 3
  • B. Pesquet-Popescu
    • 1
  1. 1.Institut Mines-Télécom, CNRS LTCITélécom ParisTech ParisFrance
  2. 2.Laboratoire de Physique, CNRS-UMR 5672ENS Lyon LyonFrance
  3. 3.LIGM, CNRS-UMR 8049Université Paris-EstMarne-la-ValléeFrance

Personalised recommendations