Journal of Mathematical Imaging and Vision

, Volume 61, Issue 1, pp 122–139 | Cite as

An Iterative Support Shrinking Algorithm for Non-Lipschitz Optimization in Image Restoration

  • Chao ZengEmail author
  • Rui Jia
  • Chunlin Wu


We consider a class of non-Lipschitz regularization problems that include the \(\hbox {TV}^p\) model as a special case. A lower bound theory of the non-Lipschitz regularization is obtained, which inspires us to propose an algorithm guaranteeing the non-expansiveness of the images gradient support set. After being proximally linearized, this algorithm can be easily implemented. Some standard techniques in image processing, like the fast Fourier transform, could be utilized. The global convergence is also established. Moreover, we prove that the restorations by the algorithm have edge preservation property. Numerical examples are given to show good performance of the algorithm and the rationality of the theories.


Non-convex non-smooth optimization Non-Lipschitz optimization Lower bound theory Kurdyka–Łojasiewicz property Total variation regularization Image restoration 



The work of the Chao Zeng was supported by Postdoctoral Science Foundation of China (2016M601248). The work of the Chunlin Wu was supported by National Natural Science Foundation of China (Grants 11301289 and 11531013) and Recruitment Programm of Global Young Expert.


  1. 1.
    Attouch, H., Bolte, J.: On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Program. 116(1), 5–16 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the kurdyka–łojasiewicz inequality. Math. Oper. Res. 35(2), 438–457 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized gauss-seidel methods. Math. Program. 137(1–2), 91–129 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Beck, A., Teboulle, M.: A fast iterative shrinkage–thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Bian, W., Chen, X.: Linearly constrained non-lipschitz optimization for image restoration. SIAM J. Imaging Sci. 8(4), 2294–2322 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Blumensath, T., Davies, M.E.: Iterative hard thresholding for compressed sensing. Appl. Comput. Harmon. Anal. 27(3), 265–274 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Bolte, J., Daniilidis, A., Lewis, A.: The łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J. Optim. 17(4), 1205–1223 (2007)CrossRefzbMATHGoogle Scholar
  8. 8.
    Bolte, J., Daniilidis, A., Lewis, A., Shiota, M.: Clarke subgradients of stratifiable functions. SIAM J. Optim. 18(2), 556–572 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146(1–2), 459–494 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Bondy, J., Murty, U.: Graph theory (graduate texts in mathematics). Springer, New York (2008)Google Scholar
  11. 11.
    Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Candes, E.J., Wakin, M.B., Boyd, S.P.: Enhancing sparsity by reweighted \(\ell_1\) minimization. J. Fourier Anal. Appl. 14(5), 877–905 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Chartrand, R.: Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Process. Lett. 14(10), 707–710 (2007)CrossRefGoogle Scholar
  14. 14.
    Chartrand, R., Staneva, V.: Restricted isometry properties and nonconvex compressive sensing. Inverse Probl. 24(3), 035020 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Chartrand, R., Yin, W.: Iteratively reweighted algorithms for compressive sensing. In: IEEE International Conference on Acoustics, Speech and Signal Processing, 2008. ICASSP 2008, pp. 3869–3872. IEEE (2008)Google Scholar
  16. 16.
    Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Rev. 43(1), 129–159 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Chen, X.: Smoothing methods for nonsmooth, nonconvex minimization. Math. Program. 134, 71–99 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Chen, X., Ng, M.K., Zhang, C.: Non-lipschitz-regularization and box constrained model for image restoration. IEEE Trans. Image Process. 21(12), 4709–4721 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Chen, X., Niu, L., Yuan, Y.: Optimality conditions and a smoothing trust region newton method for nonlipschitz optimization. SIAM J. Optim. 23(3), 1528–1552 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Chen, X., Xu, F., Ye, Y.: Lower bound theory of nonzero entries in solutions of \(\ell _2-\ell _p\) minimization. SIAM J. Sci. Comput. 32(5), 2832–2852 (2010)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Chen, X., Zhou, W.: Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 3(4), 765–790 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Chen, X., Zhou, W.: Convergence of the reweighted \(\ell_1\) minimization algorithm for \(\ell_2-\ell_p\) minimization. Comput. Optim. Appl. 59(1–2), 47–61 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Daubechies, I., DeVore, R., Fornasier, M., Güntürk, C.S.: Iteratively reweighted least squares minimization for sparse recovery. Commun. Pure Appl. Math. 63(1), 1–38 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Donoho, D.L.: For most large underdetermined systems of linear equations the minimal \(\ell _1\)-norm solution is also the sparsest solution. Commun. Pure Appl. Math. 59(6), 797–829 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Foucart, S., Lai, M.-J.: Sparsest solutions of underdetermined linear systems via \(\ell_q\)-minimization for \(0<q<1\). Appl. Comput. Harmon. Anal. 26(3), 395–407 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Gribonval, R., Nielsen, M.: Sparse representations in unions of bases. IEEE Trans. Inf. Theory 49(12), 3320–3325 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Hintermüller, M., Wu, T.: Nonconvex tv\(^q\)-models in image restoration: Analysis and a trust-region regularization-based superlinearly convergent solver. SIAM J. Imaging Sci. 6(3), 1385–1415 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Kurdyka, K.: On gradients of functions definable in o-minimal structures. Annales de l’institut Fourier 48, 769–784 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Lai, M.-J., Wang, J.: An unconstrained \(\ell _q\) minimization with \(0<q \le 1\) for sparse solution of underdetermined linear systems. SIAM J. Optim. 21(1), 82–101 (2011)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Lai, M.-J., Xu, Y., Yin, W.: Improved iteratively reweighted least squares for unconstrained smoothed \(\ell_q\) minimization. SIAM J. Numer. Anal. 51(2), 927–957 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Lanza, A., Morigi, S., Sgallari, F.: Constrained tv\(_p-\ell _2\) model for image restoration. J. Sci. Comput. 68(1), 64–91 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Li, G., Pong, T.K.: Global convergence of splitting methods for nonconvex composite optimization. SIAM J. Optim. 25(4), 2434–2460 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Łojasiewicz, S.: Une propriété topologique des sous-ensembles analytiques réels. Les équations aux dérivées partielles 117, 87–89 (1963)zbMATHGoogle Scholar
  34. 34.
    Lu, Z.: Iterative reweighted minimization methods for \(\ell _p\) regularized unconstrained nonlinear programming. Math. Program. 147(1–2), 277–307 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Natarajan, B.K.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24(2), 227–234 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Ng, M.K., Chan, R.H., Tang, W.-C.: A fast algorithm for deblurring models with neumann boundary conditions. SIAM J. Sci. Comput. 21(3), 851–866 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Nikolova, M.: Analysis of the recovery of edges in images and signals by minimizing nonconvex regularized least-squares. Multiscale Model. Simul. 4(3), 960–991 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Nikolova, M., Ng, M.K., Tam, C.-P.: Fast nonconvex nonsmooth minimization methods for image restoration and reconstruction. IEEE Trans. Image Process. 19(12), 3073–3088 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Nikolova, M., Ng, M.K., Zhang, S., Ching, W.-K.: Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 1(1), 2–25 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Ochs, P., Dosovitskiy, A., Brox, T., Pock, T.: On iteratively reweighted algorithms for nonsmooth nonconvex optimization in computer vision. SIAM J. Imaging Sci. 8(1), 331–372 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  41. 41.
    Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis, vol. 317. Springer, Berlin (2009)zbMATHGoogle Scholar
  42. 42.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Sidky, E.Y., Chartrand, R., Boone, J.M., Pan, X.: Constrained t\(_p\)v minimization for enhanced exploitation of gradient sparsity: application to ct image reconstruction. IEEE J. Transl. Eng. Health Med. 2(6), 1–18 (2014)CrossRefGoogle Scholar
  44. 44.
    Storath, M., Weinmann, A., Frikel, J., Unser, M.: Joint image reconstruction and segmentation using the potts model. Inverse Probl. 31(2), 025003 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Sun, Q.: Recovery of sparsest signals via \(\ell_q\)-minimization. Appl. Comput. Harmon. Anal. 32(3), 329–341 (2012)MathSciNetCrossRefGoogle Scholar
  46. 46.
    Van den Dries, L., Miller, C., et al.: Geometric categories and o-minimal structures. Duke Math. J. 84(2), 497–540 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1(3), 248–272 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  48. 48.
    Wu, C., Tai, X.-C.: Augmented lagrangian method, dual methods, and split bregman iteration for rof, vectorial tv, and high order models. SIAM J. Imaging Sci. 3(3), 300–339 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  49. 49.
    Xu, Z., Chang, X., Xu, F., Zhang, H.: \(l_{1/2}\) regularization: a thresholding representation theory and a fast solver. IEEE Trans. Neural Netw. Learn. Syst. 23(7), 1013–1027 (2012)CrossRefGoogle Scholar
  50. 50.
    Zeng, C., Wu, C.: On the edge recovery property of noncovex nonsmooth regularization in image restoration. SIAM J. Numer. Anal. 56(2), 1168–1182 (2018)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Mathematical SciencesNanKai UniversityTianjinChina

Personalised recommendations