Journal of Mathematical Imaging and Vision

, Volume 59, Issue 3, pp 534–566 | Cite as

Bias Reduction in Variational Regularization

  • Eva-Maria Brinkmann
  • Martin Burger
  • Julian Rasch
  • Camille SutourEmail author


The aim of this paper was to introduce and study a two-step debiasing method for variational regularization. After solving the standard variational problem, the key idea is to add a consecutive debiasing step minimizing the data fidelity on an appropriate set, the so-called model manifold. The latter is defined by Bregman distances or infimal convolutions thereof, using the (uniquely defined) subgradient appearing in the optimality condition of the variational method. For particular settings, such as anisotropic \(\ell ^1\) and TV-type regularization, previously used debiasing techniques are shown to be special cases. The proposed approach is, however, easily applicable to a wider range of regularizations. The two-step debiasing is shown to be well-defined and to optimally reduce bias in a certain setting. In addition to visual and PSNR-based evaluations, different notions of bias and variance decompositions are investigated in numerical studies. The improvements offered by the proposed scheme are demonstrated, and its performance is shown to be comparable to optimal results obtained with Bregman iterations.


Variational regularization Bias Debiasing Bregman distances 



This work was supported by ERC via Grant EU FP 7-ERC Consolidator Grant 615216 LifeInverse. MB acknowledges support by the German Science Foundation DFG via EXC 1003 Cells in Motion Cluster of Excellence, Münster, Germany.


  1. 1.
    Benning, M., Burger, M.: Ground states and singular vectors of convex variational regularization methods. Methods Appl. Anal. 20, 295–334 (2013)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Borwein, J., Zhu, Q.: Techniques of variational analysis. Springer Science & Business Media (2006)Google Scholar
  3. 3.
    Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3, 492–526 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bredies, K., Pikkarainen, H.: Inverse problems in spaces of measures. ESAIM Control Optim. Calc. Var. 9(1), 190–218 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Burger, M.: Bregman distances in inverse problems and partial differential equations. In: Hiriard-Urrurty, J., Korytowski, A., Maurer, H., Szymkat, M. (eds.) Advances in Mathematical Modeling, Optimization, and Optimal Control. Springer, New York (2016)Google Scholar
  6. 6.
    Burger, M., Gilboa, G., Moeller, M., Eckardt, L., Cremers, D.: Spectral decompositions using one-homogeneous functionals. arxiv 1601.02912 (2016)Google Scholar
  7. 7.
    Burger, M., Gilboa, G., Osher, S., Xu, J.: Nonlinear inverse scale space methods. Commun. Math. Sci. 4, 178–212 (2006)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Burger, M., Moeller, M., Benning, M., Osher, S.: An adaptive inverse scale space method for compressed sensing. Math. Comput. 82, 269–299 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Burger, M., Osher, S.: Convergence rates of convex variational regularization. Inverse Probl. 20, 1411–1421 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Burger, M., Osher, S.: A guide to the TV zoo. In: Level Set and PDE Based Reconstruction Methods in Imaging, pp. 1 70. Springer International Publishing (2013)Google Scholar
  11. 11.
    Burger, M., Resmerita, E., He, L.: Error estimation for Bregman iterations and inverse scale space methods in image restoration. Computing 81, 109–135 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Candes, E.J., Plan, Y.: A probabilistic and RIPless theory of compressed sensing. IEEE Trans. Inf. Theory 57(11), 7235–7254 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Caselles, V., Chambolle, A., Novaga, M.: Total variation in imaging. In: Handbook of Mathematical Methods in Imaging, pp. 1016–1057. Springer Science & Business Media (2011)Google Scholar
  14. 14.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40, 120–145 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Combettes, P.L., Salzo, S., Villa, S.: Regularized learning schemes in Banach space. Preprint, arXiv:1410.6847 (2014)
  16. 16.
    Deledalle, C.A., Papadakis, N., Salmon, J.: On debiasing restoration algorithms: applications to total-variation and nonlocal-means. In: Scale Space and Variational Methods in Computer Vision 2015, Lecture Notes in Computer Science, 9087, pp. 129–141 (2015)Google Scholar
  17. 17.
    Deledalle, C.A., Papadakis, N., Salmon, J., Vaiter, S.: CLEAR: Covariant LEAst-Square Re-fitting with Applications to Image Restoration. Preprint, arXiv:1606.05158 (2016)
  18. 18.
    Ekeland, I., Temam, R.: Convex Analysis and Variational Problems. SIAM, Philadelphia (1999)CrossRefzbMATHGoogle Scholar
  19. 19.
    Esser, E., Zhang, X., Chan, T.: A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science. SIAM J. Imaging Sci. 3(4), 1015–1046 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Figueiredo, M., Nowak, R., Wright, S.: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 1(4), 586–598 (2007)CrossRefGoogle Scholar
  21. 21.
    Hintermüller, M., Papafitsoros, K., Rautenberg, C.: Analytical aspects of spatially adapted total variation regularisation. Preprint, arXiv:1609.01074 (2016)
  22. 22.
    Lederer, J.: Trust, but verify: benefits and pitfalls of least-squares refitting in high dimensions. Preprint, arXiv:1306.0113v1 (2013)
  23. 23.
    Min, J., Vonesch, C., Kirshner, H., Carlini, L., Olivier, N., Holden, S., Manley, S., Ye, J.C., Unser, M.: FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data. Sci. Rep. 4, 4577 (2014)CrossRefGoogle Scholar
  24. 24.
    Moeller, M., Brinkmann, E.-M., Burger, M., Seybold, T.: Color Bregman TV. SIAM J. Imaging Sci. 7(4), 2771–2806 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Moeller, M., Burger, M.: Multiscale methods for polyhedral regularizations. SIAM J. Optim. 23, 1424–1456 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Osher, S., Ruan, F., Xiong, J., Yao, Y., Yin, W.: Sparse recovery via differential inclusions. Appl. Comput. Harmon. Anal. 41(2), 436–469 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Osher, S., Burger, M., Goldfarb, D., Xu, J., Yin, W.: An iterative regularization method for total variation-based image restoration. SIAM Multiscale Model. Simul. 4, 460–489 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Pock, T., Cremers, D., Bischof, H., Chambolle, A.: An algorithm for minimizing the Mumford-Shah functional. In: IEEE 12th International Conference on Computer Vision 2009, pp. 1133–1140 (2009)Google Scholar
  29. 29.
    Resmerita, E.: Regularization of ill-posed problems in Banach spaces: convergence rates. Inverse Probl. 21(4), 1303 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Rockafellar, R.T.: Convex Analysis. Princeton Landmarks in Mathematics. Princeton University Press, Princeton (1997)Google Scholar
  31. 31.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D Nonlinear Phenom. 60, 259–268 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Scherzer, O., Groetsch, C.: Inverse scale space theory for inverse problems. Scale-space Springer LNCS 2106, pp. 317–325 (2001)Google Scholar
  33. 33.
    Shapiro, A.: On concepts of directional differentiability. J. Optim. Theory Appl. 66(3), 477–487 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Schuster, T., Kaltenbacher, B., Hofmann, B., Kazimierski, K.S.: Regularization Methods in Banach Spaces. DeGruyter, Berlin (2012)CrossRefzbMATHGoogle Scholar
  35. 35.
    Tadmor, E., Nezzar, S., Vese, L.: A multiscale image representation using hierarchical (BV, L2) decompositions. Multiscale Model. Simul. 2, 554–579 (2004)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Institut für Numerische und Angewandte MathematikWestfälische Wilhelms-Universität (WWU) MünsterMünsterGermany

Personalised recommendations