Non-convex Total Variation Regularization for Convex Denoising of Signals

  • Ivan Selesnick
  • Alessandro Lanza
  • Serena Morigi
  • Fiorella SgallariEmail author


Total variation (TV) signal denoising is a popular nonlinear filtering method to estimate piecewise constant signals corrupted by additive white Gaussian noise. Following a ‘convex non-convex’ strategy, recent papers have introduced non-convex regularizers for signal denoising that preserve the convexity of the cost function to be minimized. In this paper, we propose a non-convex TV regularizer, defined using concepts from convex analysis, that unifies, generalizes, and improves upon these regularizers. In particular, we use the generalized Moreau envelope which, unlike the usual Moreau envelope, incorporates a matrix parameter. We describe a novel approach to set the matrix parameter which is essential for realizing the improvement we demonstrate. Additionally, we describe a new set of algorithms for non-convex TV denoising that elucidate the relationship among them and which build upon fast exact algorithms for classical TV denoising.


Signal denoising Total variation regularization Forward-backward splitting algorithm Convex non-convex regularization 



This study was funded by the National Science Foundation (Grant No. CCF-1525398) and University of Bologna (Grant No. ex 60%) and by the National Group for Scientific Computation (GNCS-INDAM), research projects 2018–19.


  1. 1.
    Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, Berlin (2011)CrossRefzbMATHGoogle Scholar
  2. 2.
    Bayram, I.: Correction for On the convergence of the iterative shrinkage/thresholding algorithm with a weakly convex penalty. IEEE Trans. Signal Process. 64(14), 3822–3822 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Bayram, I.: On the convergence of the iterative shrinkage/thresholding algorithm with a weakly convex penalty. IEEE Trans. Signal Process. 64(6), 1597–1608 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Becker, S., Combettes, P.L.: An algorithm for splitting parallel sums of linearly composed monotone operators, with applications to signal recovery. J. Nonlinear Convex Anal. 15(1), 137–159 (2014)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Blake, A., Zisserman, A.: Visual Reconstruction. MIT Press, Cambridge (1987)CrossRefGoogle Scholar
  6. 6.
    Burger, M., Papafitsoros, K., Papoutsellis, E., Schönlieb, C.-B.: Infimal convolution regularisation functionals of BV and Lp spaces. J. Math. Imaging Vis. 55(3), 343–369 (2016)zbMATHCrossRefGoogle Scholar
  7. 7.
    Cai, G., Selesnick, I.W., Wang, S., Dai, W., Zhu, Z.: Sparsity-enhanced signal decomposition via generalized minimax-concave penalty for gearbox fault diagnosis. J. Sound Vib. 432, 213–234 (2018)CrossRefGoogle Scholar
  8. 8.
    Candès, E.J., Wakin, M.B., Boyd, S.: Enhancing sparsity by reweighted l1 minimization. J. Fourier Anal. Appl. 14(5), 877–905 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Carlsson, M.: On convexification/optimization of functionals including an l2-misfit term. arXiv preprint arXiv:1609.09378 (2016)
  10. 10.
    Castella, M., Pesquet, J.-C.: Optimization of a Geman-McClure like criterion for sparse signal deconvolution. In: IEEE International Workshop on Computational Advances Multi-Sensor Adaptive Processing, pp. 309–312 (2015)Google Scholar
  11. 11.
    Chambolle, A., Lions, P.-L.: Image recovery via total variation minimization and related problems. Numerische Mathematik 76, 167–188 (1997)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Chan, R., Lanza, A., Morigi, S., Sgallari, F.: Convex non-convex image segmentation. Numerische Mathematik 138(3), 635–680 (2017)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Chan, T.F., Osher, S., Shen, J.: The digital TV filter and nonlinear denoising. IEEE Trans. Image Process. 10(2), 231–241 (2001)zbMATHCrossRefGoogle Scholar
  14. 14.
    Chartrand, R.: Shrinkage mappings and their induced penalty functions, pp. 1026–1029. In: International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2014)Google Scholar
  15. 15.
    Chouzenoux, E., Jezierska, A., Pesquet, J., Talbot, H.: A majorize-minimize subspace approach for \(\ell _2-\ell _0\) image regularization. SIAM J. Imag. Sci. 6(1), 563–591 (2013)zbMATHCrossRefGoogle Scholar
  16. 16.
    Combettes, P.L., Pesquet, J.-C.: Proximal splitting methods in signal processing. In: Bauschke, H.H., et al. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pp. 185–212. Springer, Berlin (2011)zbMATHCrossRefGoogle Scholar
  17. 17.
    Condat, L.: A direct algorithm for 1-D total variation denoising. IEEE Signal Process. Lett. 20(11), 1054–1057 (2013)CrossRefGoogle Scholar
  18. 18.
    Ding, Y., Selesnick, I.W.: Artifact-free wavelet denoising: non-convex sparse regularization, convex optimization. IEEE Signal Process. Lett. 22(9), 1364–1368 (2015)CrossRefGoogle Scholar
  19. 19.
    Donoho, D., Maleki, A., Shahram, M.: Wavelab 850 (2005).
  20. 20.
    Du, H., Liu, Y.: Minmax-concave total variation denoising. Signal Image Video Process. 12(6), 1027–1034 (2018)CrossRefGoogle Scholar
  21. 21.
    Dümbgen, L., Kovac, A.: Extensions of smoothing via taut strings. Electron. J. Stat. 3, 41–75 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Frecon, J., Pustelnik, N., Dobigeon, N., Wendt, H., Abry, P.: Bayesian selection for the l2-Potts model regularization parameter: 1D piecewise constant signal denoising. IEEE Trans. Signal Process. 65(19), 5215–5224 (2017)MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Friedrich, F., Kempe, A., Liebscher, V., Winkler, G.: Complexity penalized M-estimation: fast computation. J. Comput. Graph. Stat. 17(1), 201–224 (2008)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Huska, M., Lanza, A., Morigi, S., Sgallari, F.: Convex non-convex segmentation of scalar fields over arbitrary triangulated surfaces. J. Comput. Appl. Math. 349, 438–451 (2019)MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Lanza, A., Morigi, S., Selesnick, I., Sgallari, F.: Nonconvex nonsmooth optimization via convex-nonconvex majorization-minimization. Numerische Mathematik 136(2), 343–381 (2017)MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Lanza, A., Morigi, S., Selesnick, I., Sgallari, F.: Sparsity-inducing nonconvex nonseparable regularization for convex image processing. SIAM J. Imag. Sci. 12(2), 1099–1134 (2019)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Lanza, A., Morigi, S., Sgallari, F.: Constrained TVp-l2 model for image restoration. J. Sci. Comput. 68(1), 64–91 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  28. 28.
    Lanza, A., Morigi, S., Sgallari, F.: Convex image denoising via non-convex regularization with parameter selection. J. Math. Imaging Vis. 56(2), 195–220 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Little, M.A., Jones, N.S.: Generalized methods and solvers for noise removal from piecewise constant signals: part I - background theory. Proc. R. Soc. A 467, 3088–3114 (2011)zbMATHCrossRefGoogle Scholar
  30. 30.
    Malek-Mohammadi, M., Rojas, C.R., Wahlberg, B.: A class of nonconvex penalties preserving overall convexity in optimization-based mean filtering. IEEE Trans. Signal Process. 64(24), 6650–6664 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Möllenhoff, T., Strekalovskiy, E., Moeller, M., Cremers, D.: The primal-dual hybrid gradient method for semiconvex splittings. SIAM J. Imag. Sci. 8(2), 827–857 (2015)MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Nikolova, M.: Estimation of binary images by minimizing convex criteria. In: Proceedings of IEEE International Conference on Image Processing (ICIP), vol. 2, pp. 108–112 (1998)Google Scholar
  33. 33.
    Nikolova, M.: Energy minimization methods. In: Scherzer, O. (ed.) Handbook of Mathematical Methods in Imaging, Chapter 5, pp. 138–186. Springer, Berlin (2011)Google Scholar
  34. 34.
    Nikolova, M., Ng, M., Zhang, S., Ching, W.: Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization. SIAM J. Imag. Sci. 1(1), 2–25 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  35. 35.
    Nikolova, M., Ng, M.K., Tam, C.-P.: Fast nonconvex nonsmooth minimization methods for image restoration and reconstruction. IEEE Trans. Image Process. 19(12), 3073–3088 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  36. 36.
    Parekh, A., Selesnick, I.W.: Convex denoising using non-convex tight frame regularization. IEEE Signal Process. Lett. 22(10), 1786–1790 (2015)CrossRefGoogle Scholar
  37. 37.
    Parekh, A., Selesnick, I.W.: Enhanced low-rank matrix approximation. IEEE Signal Process. Lett. 23(4), 493–497 (2016)CrossRefGoogle Scholar
  38. 38.
    Parks, T.W., Burrus, C.S.: Digital Filter Design. Wiley, Hoboken (1987)zbMATHGoogle Scholar
  39. 39.
    Portilla, J., Mancera, L.: L0-based sparse approximation: two alternative methods and some applications. In: Proceedings of SPIE, vol. 6701 (Wavelets XII), San Diego, CA, USA, (2007)Google Scholar
  40. 40.
    Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60, 259–268 (1992)MathSciNetzbMATHCrossRefGoogle Scholar
  41. 41.
    Selesnick, I.: Sparse regularization via convex analysis. IEEE Trans. Signal Process. 65(17), 4481–4494 (2017)MathSciNetzbMATHCrossRefGoogle Scholar
  42. 42.
    Selesnick, I.: Total variation denoising via the Moreau envelope. IEEE Signal Process. Lett. 24(2), 216–220 (2017)MathSciNetCrossRefGoogle Scholar
  43. 43.
    Selesnick, I.W., Bayram, I.: Sparse signal estimation by maximally sparse convex optimization. IEEE Trans. Signal Process. 62(5), 1078–1092 (2014)MathSciNetzbMATHCrossRefGoogle Scholar
  44. 44.
    Selesnick, I.W., Parekh, A., Bayram, I.: Convex 1-D total variation denoising with non-convex regularization. IEEE Signal Process. Lett. 22(2), 141–144 (2015)CrossRefGoogle Scholar
  45. 45.
    Setzer, S., Steidl, G., Teuber, T.: Infimal convolution regularizations with discrete l1-type functionals. Commun. Math. Sci. 9(3), 797–827 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  46. 46.
    Shen, L., Suter, B.W., Tripp, E.E.: Structured sparsity promoting functions. J. Optim. Theory Appl. 183(2), 386–421 (2019)MathSciNetzbMATHCrossRefGoogle Scholar
  47. 47.
    Shen, L., Xu, Y., Zeng, X.: Wavelet inpainting with the l0 sparse regularization. J. Appl. Comp. Harm. Anal. 41(1), 26–53 (2016)zbMATHCrossRefGoogle Scholar
  48. 48.
    Sidky, E.Y., Chartrand, R., Boone, J.M., Xiaochuan, P.: Constrained TpV minimization for enhanced exploitation of gradient sparsity: application to CT image reconstruction. IEEE J. Transl. Eng. Health Med. 2, 1–18 (2014)CrossRefGoogle Scholar
  49. 49.
    Soubies, E., Blanc-Féraud, L., Aubert, G.: A continuous exact \( \ell _0 \) penalty (CEL0) for least squares regularized problem. SIAM J. Imag. Sci. 8(3), 1607–1639 (2015)zbMATHCrossRefGoogle Scholar
  50. 50.
    Storath, M., Weinmann, A., Demaret, L.: Jump-sparse and sparse recovery using Potts functionals. IEEE Trans. Signal Process. 62(14), 3654–3666 (2014)MathSciNetzbMATHCrossRefGoogle Scholar
  51. 51.
    Strang, G.: The discrete cosine transform. SIAM Rev. 41(1), 135–147 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  52. 52.
    Wang, S., Selesnick, I.W., Cai, G., Ding, B., Chen, X.: Synthesis versus analysis priors via generalized minimax-concave penalty for sparsity-assisted machinery fault diagnosis. Mech. Syst. Signal Process. 127, 202–233 (2019)CrossRefGoogle Scholar
  53. 53.
    Zhang, C.-H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38(2), 894–942 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  54. 54.
    Zou, J., Shen, M., Zhang, Y., Li, H., Liu, G., Ding, S.: Total variation denoising with non-convex regularizers. IEEE Access 7, 4422–4431 (2019)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2020

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringNew York UniversityBrooklyn, New YorkUSA
  2. 2.Department of MathematicsUniversity of BolognaBolognaItaly

Personalised recommendations