Skip to main content
SpringerLink
Log in

Convex Non-convex Variational Models

  • Reference work entry
  • First Online:
Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging

Abstract

An important class of computational techniques to solve inverse problems in image processing relies on a variational approach: the optimal output is obtained by finding a minimizer of an energy function or “model” composed of two terms, the data-fidelity term, and the regularization term. Much research has focused on models where both terms are convex, which leads to convex optimization problems. However, there is evidence that non-convex regularization can improve significantly the output quality for images characterized by some sparsity property. This fostered recent research toward the investigation of optimization problems with non-convex terms. Non-convex models are notoriously difficult to handle as classical optimization algorithms can get trapped at unwanted local minimizers. To avoid the intrinsic difficulties related to non-convex optimization, the convex non-convex (CNC) strategy has been proposed, which allows the use of non-convex regularization while maintaining convexity of the total cost function. This work focuses on a general class of parameterized non-convex sparsity-inducing separable and non-separable regularizers and their associated CNC variational models. Convexity conditions for the total cost functions and related theoretical properties are discussed, together with suitable algorithms for their minimization based on a general forward-backward (FB) splitting strategy. Experiments on the two classes of considered separable and non-separable CNC variational models show their superior performance than the purely convex counterparts when applied to the discrete inverse problem of restoring sparsity-characterized images corrupted by blur and noise.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 899.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 949.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011)

    Book  MATH  Google Scholar 

  • Bayram, I.: On the convergence of the iterative shrinkage/thresholding algorithm with a weakly convex penalty. IEEE Trans. Signal Process. 64(6), 1597–1608 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  • Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Becker, S., Combettes, P.L.: An algorithm for splitting parallel sums of linearly composed monotone operators, with applications to signal recovery. J. Nonlinear Convex Anal. 15(1), 137–159 (2014)

    MathSciNet  MATH  Google Scholar 

  • Bello Cruz, J.Y.: On proximal subgradient splitting method for minimizing the sum of two nonsmooth convex functions. Set-Valued Var. Anal 25, 245–263 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Blake, A., Zisserman, A.: Visual Reconstruction. MIT Press, Cambridge, MA (1987)

    Book  Google Scholar 

  • Bruckstein, A., Donoho, D., Elad, M.: From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev. 51(1), 34–81 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Burger, M., Papafitsoros, K., Papoutsellis, E., Schönlieb, C.B.: Infimal convolution regularisation functionals of BV and Lp spaces. J. Math. Imaging Vis. 55(3), 343–369 (2016)

    Article  MATH  Google Scholar 

  • Cai, G., Selesnick, I.W., Wang, S., Dai, W., Zhu, Z.: Sparsity enhanced signal decomposition via generalized minimax-concave penalty for gearbox fault diagnosis. J. Sound Vib. 432, 213–234 (2018)

    Article  Google Scholar 

  • Candés, E.J., Wakin, M.B., Boyd, S.: Enhancing sparsity by reweighted l1 minimization. J. Fourier Anal. Appl.14(5), 877–905 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  • Carlsson, M.: On convexification/optimization of functionals including an l2-misfit term. arXiv preprint arXiv:1609.09378 (2016)

    Google Scholar 

  • Castella, M., Pesquet, J.C.: Optimization of a Geman-McClure like criterion for sparse signal deconvolution. In: IEEE International Workshop on Computational Advances Multi-sensor Adaptive Processing, pp. 309–312 (2015)

    Google Scholar 

  • Chambolle, A., Lions, P.L.: Image recovery via total variation minimization and related problems. Numerische Mathematik 76, 167–188 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  • Chan, R., Lanza, A., Morigi, S., Sgallari, F.: Convex non-convex image segmentation. Numerische Mathematik 138(3), 635–680 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Chartrand, R.: Shrinkage mappings and their induced penalty functions. In: International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1026–1029 (2014)

    Google Scholar 

  • Chen, P.Y., Selesnick, I.W.: Group-sparse signal denoising: non-convex regularization, convex optimization. IEEE Trans. Signal Proc. 62, 3464–3478 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Chouzenoux, E., Jezierska, A., Pesquet, J., Talbot, H.: A majorize-minimize subspace approach for l2–l0 image regularization. SIAM J. Imag. Sci. 6(1), 563–591 (2013)

    Article  MATH  Google Scholar 

  • Ding, Y., Selesnick, I.W.: Artifact-free wavelet denoising: nonconvex sparse regularization, convex optimization. IEEE Signal Process. Lett. 22(9), 1364–1368 (2015)

    Article  Google Scholar 

  • Du, H., Liu, Y.: Minmax-concave total variation denoising. Signal Image Video Process. 12(6), 1027–1034 (2018)

    Article  Google Scholar 

  • Geiger, D., Girosi, F.: Parallel and deterministic algorithms from MRF’s: surface reconstruction. IEEE Trans. Pattern Anal. Mach. Intell. 13(5), 410–412 (1991)

    Article  Google Scholar 

  • Geman, S., Geman, D.: Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images. IEEE PAMI 6(6), 721–741 (1984)

    Article  MATH  Google Scholar 

  • Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1997)

    MATH  Google Scholar 

  • Hartman, P.: On functions representable as a difference of convex functions. Pac. J. Math. 9(3), 707–713 (1959)

    Article  MathSciNet  MATH  Google Scholar 

  • Hastie, T., Tibshirani, R., Wainwright, M.: Statistical Learning with Sparsity: The Lasso and Generalizations. CRC Press, Boca Raton (2015)

    Book  MATH  Google Scholar 

  • Huska, M., Lanza, A., Morigi, S., Sgallari, F.: Convex non-convex segmentation of scalar fields over arbitrary triangulated surfaces. J. Comput. Appl. Math. 349, 438–451 (2019a)

    Article  MathSciNet  MATH  Google Scholar 

  • Huska, M., Lanza, A., Morigi, S., Selesnick, I.W.: A convex-nonconvex variational method for the additive decomposition of functions on surfaces. Inverse Problems 35, 124008–124041 (2019b)

    Article  MathSciNet  MATH  Google Scholar 

  • Jensen, J.B., Nielsen, M.: A simple genetic algorithm applied to discontinuous regularization. In: Proceedings IEEE workshop on NNSP, Copenhagen (1992)

    Google Scholar 

  • Lanza, A., Morigi, S., Sgallari, F.: Convex image denoising via non-convex regularization. Scale Space Variat. Methods Comput. Vis. 9087, 666–677 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Lanza, A., Morigi, S., Sgallari, F.: Convex image denoising via nonconvex regularization with parameter selection. J. Math. Imaging Vis. 56(2), 195–220 (2016a)

    Article  MATH  Google Scholar 

  • Lanza, A., Morigi, S., Sgallari, F.: Constrained TVp-ℓ2 model for image restoration. J. Sci. Comput. 68, 64–91 (2016b)

    Article  MathSciNet  MATH  Google Scholar 

  • Lanza, A., Morigi, S., Selesnick, I.W., Sgallari, F.: Nonconvex nonsmooth optimization via convex-nonconvex majorization minimization. Numerische Mathematik 136(2), 343–381 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Lanza, A., Morigi, S., Sgallari, F.: Automatic parameter selection based on residual whiteness for convex non-convex variational restoration. In: Mathematical Methods in Image Processing and and Inverse Problems (eds) Tai XC, Wei S, Liu H. Springer, Singapore, 360, (2021). https://doi.org/10.1007/978-981-16-2701-9

  • Lanza, A., Morigi, S., Selesnick, I.W., Sgallari, F.: Sparsity-inducing nonconvex nonseparable regularization for convex image processing. SIAM J. Imag. Sci. 12(2), 1099–1134 (2019)

    Article  MathSciNet  Google Scholar 

  • Lanza, A., Pragliola, M., Sgallari, F.: Residual whiteness principle for parameter-free image restoration. Electron. Trans. Numer. Anal. 53, 329–351 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  • Lefkimmiatis, S., Ward, J., Unser, M.: Hessian Schatten-Norm regularization for linear inverse problems. IEEE Trans. Image Process. 22, 1873–1888 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Malek-Mohammadi, M., Rojas, C.R., Wahlberg, B.: A class of nonconvex penalties preserving overall convexity in optimization based mean filtering. IEEE Trans. Signal Process. 64(24), 6650–6664 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  • Nikolova, M.: Estimation of binary images by minimizing convex criteria. Proc. IEEE Int. Conf. Image Process. 2, 108–112 (1998)

    Google Scholar 

  • Nikolova, M.: Energy minimization methods. In: Scherzer, O. (ed.) Handbook of Mathematical Methods in Imaging, Chapter 5, pp. 138–186. Springer, Berlin (2011)

    Google Scholar 

  • Nikolova, M., Ng, M.K., Tam, C.P.: Fast nonconvex nonsmooth minimization methods for image restoration and reconstruction. IEEE Trans. Image Process. 19(12), 3073–3088 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Parekh, A., Selesnick, I.W.: Convex denoising using non-convex tight frame regularization. IEEE Signal Process. Lett. 22(10), 1786–1790 (2015)

    Article  Google Scholar 

  • Parekh, A., Selesnick, I.W.: Enhanced low-rank matrix approximation. IEEE Signal Process. Lett. 23(4), 493–497 (2016)

    Article  Google Scholar 

  • Park, T.W., Burrus, C.S.: Digital Filter Design. Wiley, New York (1987)

    MATH  Google Scholar 

  • Portilla, J., Mancera, L.: L0-based sparse approximation: two alternative methods and some applications. In: Proceedings of SPIE, San Diego, vol. 6701 (Wavelets XII) (2007)

    Google Scholar 

  • Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physics D 60(1–4), 259–268 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  • Selesnick, I.W.: Sparse regularization via convex analysis. IEEE Trans. Signal Process. 65(17), 4481–4494 (2017a)

    Article  MathSciNet  MATH  Google Scholar 

  • Selesnick, I.W.: Total variation denoising via the Moreau envelope. IEEE Signal Process. Lett. 24(2), 216–220 (2017b)

    Article  MathSciNet  Google Scholar 

  • Selesnick, I.W., Bayram, I.: Sparse signal estimation by maximally sparse convex optimization. IEEE Trans. Signal Proc. 62(5), 1078–1092 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Selesnick, I.W., Parekh, A., Bayram, I.: Convex 1-D total variation denoising with non-convex regularization. IEEE Signal Process. Lett. 22, 141–144 (2015)

    Article  Google Scholar 

  • Selesnick, I.W., Lanza, A., Morigi, S., Sgallari, F.: Non-convex total variation regularization for convex denoising of signals. J. Math. Imag. Vis. 62, 825–841 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  • Setzer, S., Steidl, G., Teuber, T.: Infimal convolution regularizations with discrete l1-type functionals. Commun. Math. Sci. 9(3), 797–827 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Shen, L., Xu, Y., Zeng, X.: Wavelet inpainting with the l0 sparse regularization. J. Appl. Comp. Harm. Anal. 41(1), 26–53 (2016)

    Article  MATH  Google Scholar 

  • Sidky, E.Y., Chartrand, R., Boone, J.M., Pan, X.: Constrained TpV–minimization for enhanced exploitation of gradient sparsity: application to CT image reconstruction. IEEE J. Trans. Eng. Health Med. 2, 1–18 (2014)

    Article  Google Scholar 

  • Soubies, E., Blanc-Féraud, L., Aubert, G.: A continuous exact L0 penalty (CEL0) for least squares regularized problem. SIAM J. Imag. Sci.8(3), 1607–1639 (2015)

    Article  MATH  Google Scholar 

  • Tipping, M.E.: Sparse Bayesian learning and the relevance vector machine. J. Mach. Learn. Res. 1, 211–244 (2001)

    MathSciNet  MATH  Google Scholar 

  • Tuy, H.: DC optimization: theory, methods and algorithms. In: Handbook of Global Optimization, pp. 149–216. Springer, Boston, (1995)

    Google Scholar 

  • Wang, S., Selesnick, I.W., Cai, G., Ding, B., Chen, X.: Synthesis versus analysis priors via generalized minimax-concave penalty for sparsity-assisted machinery fault diagnosis. Mech. Syst. Signal Process. 127, 202–233 (2019)

    Article  Google Scholar 

  • Wipf, D.P., Rao, B.D., Nagarajan, S.: “Latent variable Bayesian models for promoting sparsity. In: IEEE Trans. Inf. Theory 57(9), 6236–6255 (2011)

    MathSciNet  MATH  Google Scholar 

  • Yuille, A.L., Rangarajan, A.: The concave-convex procedure. Neural Comput. 15(4), 915–936 (2003)

    Article  MATH  Google Scholar 

  • Zhang, C.-H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38(2), 894–942 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Zou, J., Shen, M., Zhang, Y., Li, H., Liu, G., Ding, S.: Total variation denoising with non-convex regularizers. IEEE Access 7, 4422–4431 (2019)

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported in part by the National Group for Scientific Computation (GNCS-INDAM), Research Projects 2019/2020.

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Bologna, Bologna, Italy

    Alessandro Lanza, Serena Morigi & Fiorella Sgallari

  2. Department of Electrical and Computer Engineering, New York University, New York, NY, USA

    Ivan W. Selesnick

Authors
  1. Alessandro Lanza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Serena Morigi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ivan W. Selesnick
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fiorella Sgallari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fiorella Sgallari .

Editor information

Editors and Affiliations

  1. Department of Mathematical Sciences, The University of Liverpool, Liverpool, UK

    Ke Chen

  2. Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, UK

    Carola-Bibiane Schönlieb

  3. Hong Kong Center for Cerebrocardiovascular Health Engineering (COCHE), Shatin, Hong Kong

    Xue-Cheng Tai

  4. Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD, USA

    Laurent Younes

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Lanza, A., Morigi, S., Selesnick, I.W., Sgallari, F. (2023). Convex Non-convex Variational Models. In: Chen, K., Schönlieb, CB., Tai, XC., Younes, L. (eds) Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging. Springer, Cham. https://doi.org/10.1007/978-3-030-98661-2_61

Download citation

Publish with us

Policies and ethics

Search

Navigation