Journal of Mathematical Imaging and Vision

, Volume 47, Issue 3, pp 210–230 | Cite as

Homogeneous Penalizers and Constraints in Convex Image Restoration



Recently convex optimization models were successfully applied for solving various problems in image analysis and restoration. In this paper, we are interested in relations between convex constrained optimization problems of the form \(\operatorname{argmin}\{ \varPhi(x) \mbox{ subject to } \varPsi(x) \le\tau\}\) and their penalized counterparts \(\operatorname{argmin}\{\varPhi(x) + \lambda\varPsi(x)\}\). We recall general results on the topic by the help of an epigraphical projection. Then we deal with the special setting Ψ:=∥L⋅∥ with L∈ℝ m,n and Φ:=φ(H⋅), where H∈ℝ n,n and φ:ℝ n →ℝ∪{+∞} meet certain requirements which are often fulfilled in image processing models. In this case we prove by incorporating the dual problems that there exists a bijective function such that the solutions of the constrained problem coincide with those of the penalized problem if and only if τ and λ are in the graph of this function. We illustrate the relation between τ and λ for various problems arising in image processing. In particular, we point out the relation to the Pareto frontier for joint sparsity problems. We demonstrate the performance of the constrained model in restoration tasks of images corrupted by Poisson noise with the I-divergence as data fitting term φ and in inpainting models with the constrained nuclear norm. Such models can be useful if we have a priori knowledge on the image rather than on the noise level.


Convex minimization ROF functional Homogeneous function Pareto frontiers Pareto curves Kullback-Leibler divergence I-divergence Primal-dual methods Epigraphical projection Sparsity 


  1. 1.
    Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011) MATHCrossRefGoogle Scholar
  2. 2.
    Beck, A., Teboulle, M.: Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans. Image Process. 18(11), 2419–2434 (2009) MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bertero, M., Boccacci, P.: Introduction to Inverse Problems in Imaging. IOP Publishing, Bristol (1998) MATHCrossRefGoogle Scholar
  4. 4.
    Bertsekas, D.P., Nedić, A., Ozdaglar, A.E.: Convex Analysis and Optimization. Athena Scientific, Belmont (2003) MATHGoogle Scholar
  5. 5.
    Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer Series in Operations Research, vol. 7. Springer, Berlin (2000) MATHGoogle Scholar
  6. 6.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2011) CrossRefGoogle Scholar
  7. 7.
    Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3(3), 492–526 (2010) MathSciNetMATHCrossRefGoogle Scholar
  8. 8.
    Cai, J.-F., Candés, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010) MathSciNetMATHCrossRefGoogle Scholar
  9. 9.
    Cai, J.-F., Osher, S.: Fast singular value thresholding without singular value decomposition. UCLA CAM Report 10-24 (2010) Google Scholar
  10. 10.
    Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20, 89–97 (2004) MathSciNetCrossRefGoogle Scholar
  11. 11.
    Chambolle, A., Lions, P.-L.: Image recovery via total variation minimization and related problems. Numer. Math. 76, 167–188 (1997) MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011) MathSciNetMATHCrossRefGoogle Scholar
  13. 13.
    Chan, T.F., Esedoglu, S., Park, F.: A fourth order dual method for staircase reduction in texture extraction and image restoration problems. CAM Report 05-28, UCLA (2005) Google Scholar
  14. 14.
    Chan, T.F., Shen, J.: Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastical Methods. SIAM, Philadelphia (2005) CrossRefGoogle Scholar
  15. 15.
    Chen, D.-Q., Cheng, L.-Z.: Spatially adapted regularization parameter selection based on the local discrepancy function for Poissonian image deblurring. Inverse Probl. 28(1), 015004 (2012) MathSciNetCrossRefGoogle Scholar
  16. 16.
    Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20, 33–61 (1998) MathSciNetCrossRefGoogle Scholar
  17. 17.
    Cherchia, G., Pustelnik, N., Pesquet, J.-C., Pesquet-Popescu, B.: A proximal approach for constrained cosparse modelling. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Kyoto, Japan (2012) Google Scholar
  18. 18.
    Ciak, R.: Homogeneous penalizers and constraints in convex problems. University of Kaiserslautern (2012). Preprint Google Scholar
  19. 19.
    Combettes, P.L., Pesquet, J.-C.: Proximal splitting methods in signal processing. In: Bauschke, H.H., Burachik, R.S., Combettes, P.L., Elser, V., Luke, D.R., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pp. 185–212. Springer, New York (2011) CrossRefGoogle Scholar
  20. 20.
    Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. 4, 1168–1200 (2005) MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Daubechies, I., Fornasier, M., Loris, I.: Accelerated projected gradient methods for linear inverse problems with sparsity constraints. J. Fourier Anal. Appl. 14(5–6), 764–792 (2008) MathSciNetMATHCrossRefGoogle Scholar
  22. 22.
    Dong, Y., Hintermüller, M., Rincon-Camacho, M.M.: Automated regularization parameter selection in multi-scale total variation models for image restoration. J. Math. Imaging Vis. 40(1), 82–104 (2011) MATHCrossRefGoogle Scholar
  23. 23.
    Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52, 1289–1306 (2006) MathSciNetCrossRefGoogle Scholar
  24. 24.
    Duchi, J., Shalev-Shwartz, S., Singer, Y., Chandra, T.: Efficient projections onto the l1-ball for learning in high dimensions. In: ICML’08 Proceedings of the 25th International Conference on Machine Learning. ACM, New York (2008) Google Scholar
  25. 25.
    Eckstein, J., Bertsekas, D.B.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992) MathSciNetMATHCrossRefGoogle Scholar
  26. 26.
    Esser, E.: Applications of Lagrangian based alternating direction methods and connections to split Bregman. CAM Report 09-31, UCLA (2009) Google Scholar
  27. 27.
    Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vols. I, II. Springer, New York (2003) Google Scholar
  28. 28.
    Fadili, J., Peyré, G.: Total variation projection with first order schemes. IEEE Trans. Image Process. 20(3), 657–669 (2011) MathSciNetCrossRefGoogle Scholar
  29. 29.
    Figueiredo, M.A.T., Bioucas-Dias, J.M.: Deconvolution of Poissonian images using variable splitting and augmented Lagrangian optimization. In: IEEE Workshop on Statistical Signal Processing, Cardiff (2009) Google Scholar
  30. 30.
    Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Solution of Boundary Value Problems, pp. 299–340. North-Holland, Amsterdam (1983). Chap. IX CrossRefGoogle Scholar
  31. 31.
    Gilboa, G., Darbon, J., Osher, S., Chan, T.: Nonlocal convex functionals for image regularization. UCLA Cam Report, 06-57 (2006) Google Scholar
  32. 32.
    Gilboa, G., Osher, S.: Nonlocal operators with applications to image processing. Multiscale Model. Simul. 7(3), 1005–1028 (2008) MathSciNetMATHCrossRefGoogle Scholar
  33. 33.
    Goldstein, T., Osher, S.: The split Bregman method for l1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009) MathSciNetMATHCrossRefGoogle Scholar
  34. 34.
    Hiriart-Urruty, J.-B., Lemarechal, C.: Convex Analysis and Minimization Algorithms vol. 1. Springer, Berlin (1993) Google Scholar
  35. 35.
    Ivanov, K.V., Vasin, V.V., Tanana, V.P.: Theory of Linear Ill-Posed Problems and Its Applications. Brill, Utrecht (2002) MATHCrossRefGoogle Scholar
  36. 36.
    Kunisch, K., Pock, T.: A bilevel optimization approach for parameter learning in variational models (2012). Preprint Google Scholar
  37. 37.
    Lefkimmiatis, S., Bourquard, A., Unser, M.: Hessian-based norm regularization for image restoration with biomedical applications. IEEE Trans. Image Process. 21(3), 983–995 (2012) MathSciNetCrossRefGoogle Scholar
  38. 38.
    Lewis, A.S.: Convex analysis on the Hermitian matrices. SIAM J. Optim. 6, 164–177 (1996) MathSciNetMATHCrossRefGoogle Scholar
  39. 39.
    Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values in visual data. In: Proceedings of ICCV’2009, pp. 2114–2121 (2009) Google Scholar
  40. 40.
    Lorenz, D.A.: Constructing test instances for basis pursuit denoising. Technical report, TU Braunschweig (2011).
  41. 41.
    Lysaker, M., Lundervold, A., Tai, X.: Noise removal using fourth-order partial differential equations with applications to medical magnetic resonance images in space and time. IEEE Trans. Image Process. 12(12), 1579–1590 (2003) CrossRefGoogle Scholar
  42. 42.
    Mallat, S.: A Wavelet Tour of Signal Processing, 2nd edn. Academic Press, San Diego (1999) MATHGoogle Scholar
  43. 43.
    Morozov, V.A.: Methods for Solving Incorrectly Posed Problems. Springer, New York (1984) CrossRefGoogle Scholar
  44. 44.
    Nesterov, Y.: Smooth minimization of non-smooth functions. Math. Program. 103(1), 127–152 (2005) MathSciNetMATHCrossRefGoogle Scholar
  45. 45.
    Ng, M.K., Weiss, P., Yuan, X.: Solving constrained total-variation image restoration and reconstruction problems via alternating direction methods. SIAM J. Sci. Comput. 32(5), 2710–2736 (2010) MathSciNetMATHCrossRefGoogle Scholar
  46. 46.
    Peyré, G., Fadili, J.: Learning analysis sparsity priors. In: Proceedings of Sampta’11 (2011) Google Scholar
  47. 47.
    Rao, K.R., Yip, P.: Discrete Cosine Transform. Academic Press, New York (1990) MATHGoogle Scholar
  48. 48.
    Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970) MATHGoogle Scholar
  49. 49.
    Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis, 2nd edn. A Series of Comprehensive Studies in Mathematics, vol. 317. Springer, Berlin (2004) Google Scholar
  50. 50.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica A 60, 259–268 (1992) MATHGoogle Scholar
  51. 51.
    Samuel, K., Tappen, M.: Learning optimized map estimates in continuously-valued mrf models. In: Proceedings of CVPR’09, pp. 477–484. IEEE Press, New York (2009) Google Scholar
  52. 52.
    Setzer, S.: Operator splittings, Bregman methods and frame shrinkage in image processing. Int. J. Comput. Vis. 92, 265–280 (2011) MathSciNetMATHCrossRefGoogle Scholar
  53. 53.
    Setzer, S., Steidl, G., Teuber, T.: Deblurring Poissonian images by split Bregman techniques. J. Vis. Commun. Image Represent. 21, 193–199 (2010) CrossRefGoogle Scholar
  54. 54.
    Setzer, S., Steidl, G., Teuber, T.: On vector and matrix median computation. J. Comput. Appl. Math. 236(8), 2200–2222 (2012). doi: 10.1016/ MathSciNetMATHCrossRefGoogle Scholar
  55. 55.
    Steidl, G.: A note on the dual treatment of higher order regularization functionals. Computing 76, 135–148 (2006) MathSciNetMATHCrossRefGoogle Scholar
  56. 56.
    Steidl, G., Teuber, T.: Removing multiplicative noise by Douglas-Rachford splitting methods. J. Math. Imaging Vis. 36(3), 168–184 (2010) MathSciNetCrossRefGoogle Scholar
  57. 57.
    Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B 58, 267–288 (1994) MathSciNetGoogle Scholar
  58. 58.
    Tikhonov, A.N., Arsenin, V.Y.: Solutions of Ill-Posed Problems. Wiley, Washington (1977) MATHGoogle Scholar
  59. 59.
    Toh, K.-C., Yun, S.: An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems. Pac. J. Optim. 6, 615–640 (2010) MathSciNetMATHGoogle Scholar
  60. 60.
    Tropp, J.: Just relax: convex programming methods for identifying sparse signals in noise. IEEE Trans. Inf. Theory 52, 1030 (2006) MathSciNetCrossRefGoogle Scholar
  61. 61.
    van den Berg, E., Friedlander, M.P.: Probing the Pareto frontier for basis pursuit solutions. SIAM J. Sci. Comput. 31(2), 890–912 (2008) MathSciNetMATHCrossRefGoogle Scholar
  62. 62.
    Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1(3), 248–272 (2008) MathSciNetMATHCrossRefGoogle Scholar
  63. 63.
    Watson, G.A.: Characterization of the subdifferential of some matrix norms. Linear Algebra Appl. 170, 33–45 (1992) MathSciNetMATHCrossRefGoogle Scholar
  64. 64.
    Wen, Y., Chan, R.: Parameter selection for total variation based image restoration using discrepancy principle. IEEE Trans. Image Process. 21(4), 1770–1781 (2012). doi: 10.1109/TIP.2011.2181401 MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  1. 1.Dept. of MathematicsUniversity KaiserslauternKaiserslauternGermany
  2. 2.Fraunhofer ITWMKaiserslauternGermany

Personalised recommendations