Bregman Distances in Inverse Problems and Partial Differential Equations

  • Martin BurgerEmail author
Part of the Springer Optimization and Its Applications book series (SOIA, volume 109)


The aim of this paper is to provide an overview of recent development related to Bregman distances outside its native areas of optimization and statistics. We discuss approaches in inverse problems and image processing based on Bregman distances, which have evolved to a standard tool in these fields in the last decade. Moreover, we discuss related issues in the analysis and numerical analysis of nonlinear partial differential equations with a variational structure. For such problems Bregman distances appear to be of similar importance, but are currently used only in a quite hidden fashion. We try to work out explicitly the aspects related to Bregman distances, which also lead to novel mathematical questions and may also stimulate further research in these areas.


Planck Equation Nonlinear Evolution Equation Optimal Transport Lyapunov Functional Convex Functional 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was partially supported by ERC via Grant EU FP 7 - ERC Consolidator Grant 615216 LifeInverse and by the German Science Foundation DFG via BU 2327/6-1 and EXC 1003 Cells in Motion Cluster of Excellence, Münster, Germany


  1. 1.
    Ambrosio, L., Gigli, N., Savare, G.: Gradient Flows in Metric Spaces and in the Space of Probability Measures. Birkhaeuser, Basel/Boston/Berlin (2005)zbMATHGoogle Scholar
  2. 2.
    Arnold, A., Unterreiter, A.: Entropy decay of discretized Fokker-Planck equations I: temporal semidiscretization. Comput. Math. Appl. 46, 1683–1690 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Arnold, A., Markowich, P., Toscani, G., Unterreiter, A.: On convex Sobolev inequalities and the rate of convergence to equilibrium for Fokker-Planck type equations. Commun. Partial Differ. Equ. 26, 43–100 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Arnold, A., Carrillo, J.A., Desvillettes, L., Dolbeault, J., Jüngel, A., Lederman, C., Markowich, P., Toscani, G., Villani, C.: Entropies and equilibria of many-particle systems: an essay on recent research. Mon. Hefte Math. 142, 35–43 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Arnold, A., Carlen, E., Ju, Q.: Large-time behavior of non-symmetric Fokker-Planck type equations. Commun. Stoch. Anal. 2, 153–175 (2008)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Bartels, S.: Error control and adaptivity for a variational model problem defined on functions of bounded variation. Math. Comput. 84, 1217–1240 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Benamou, J.-D., Carlier, G., Cuturi, M., Nenna, L.: Peyre, G., Iterative Bregman projections for regularized transportation problems. SIAM J. Sci. Comput. 37, A1111–A1138 (2015)Google Scholar
  8. 8.
    Benning, M., Burger, M.: Error estimates for general fidelities. Electron. Trans. Numer. Anal. 38, 44–68 (2011)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Benning, M., Burger, M.: Ground states and singular vectors of convex variational regularization methods. Methods Appl. Anal. 20, 295–334 (2013)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Bodineau, T., Lebowitz, J., Mouhot, C., Villani, C.: Lyapunov functionals for boundary-driven nonlinear drift-diffusion equations. Nonlinearity 27, 2111 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Bregman, L.M.: The relaxation method for finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. Math. Phys. 7, 200–217 (1967)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Brune, C., Sawatzky, A., Burger, M.: Primal and dual Bregman methods with application to optical nanoscopy. Int. J. Comput. Vis. 92, 211–229 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Burger, M., Lucka, F.: Maximum-a-posteriori estimates in linear inverse problems with log-concave priors are proper Bayes estimators. Inverse Prob. 30, 114004 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Burger, M., Osher, S.: Convergence rates of convex variational regularization. Inverse Prob. 20, 1411–1421 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Burger, M., Osher, S.: A guide to the TV zoo. In: Burger, M., Osher, S. (eds.) Level Set and PDE-Based Reconstruction Methods in Imaging. Springer, Berlin (2013)CrossRefGoogle Scholar
  16. 16.
    Burger, M., Gilboa, G., Osher, S., Xu, J.: Nonlinear inverse scale space methods. Commun. Math. Sci. 4, 179–212 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Burger, M., Frick, K., Osher, S., Scherzer, O.: Inverse total variation flow. Multiscale Model. Simul. 6, 366–395 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Burger, M., Resmerita, E., He, L.: Error estimation for Bregman iterations and inverse scale space methods in image restoration. Computing 81, 109–135 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Burger, M., Di Francesco, M., Pietschmann, J.F., Schlake, B.: Nonlinear cross-diffusion with size exclusion. SIAM J. Math. Anal. 42, 2842–2871 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Burger, M., Moeller, M., Benning, M., Osher, S.: An adaptive inverse scale space method for compressed sensing. Math. Comput. 82, 269–299 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Burger, M., Eckardt, L., Gilboa, G., Moeller, M.: Spectral representation of one-homogeneous functionals. In: Aujol, J.F., Niholovq, M., Pzpadikis, N. (eds.) Scale Space and Variation Methods in Computer Vision, Springer, Berlin, Heibelly, 16–27Google Scholar
  22. 22.
    Cai, J., Osher, S., Shen, Z.: Linearized Bregman iterations for compressed sensing. Math. Comput. 78, 1515–1536 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Carlier, G., Jimenez, C.: On Monge’s problem for Bregman-like cost functions. J. Convex Anal. 14, 647–655 (2007)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Carrillo, J.A., Jüngel, A., Markowich, P., Toscani, G., Unterreiter, A.: Entropy dissipation methods for degenerate parabolic problems and generalized Sobolev inequalities. Mon. Hefte Math. 133, 1–82 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Censor, Y., Lent, A.: An iterative row-action method for interval convex programming J. Optim. Theory Appl. 34, 321–353 (1981)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Censor, Y., Zenios, S.: Parallel Optimization: Theory, Algorithms, and Applications. Oxford University Press, Oxford (1998)zbMATHGoogle Scholar
  27. 27.
    Chavent, G., Kunisch, K.: Regularization of linear least squares problems by total bounded variation. ESAIM: Control Optim. Calc. Var. 2, 359–376 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Choi, K., Fahimian, B.P., Li, T., Suh, T.S., Lei, X.: Enhancement of four-dimensional cone-beam computed tomography by compressed sensing with Bregman iteration. J. X-Ray Sci. Technol. 21, 177–192 (2013)Google Scholar
  29. 29.
    Di Francesco, M., Fellner, K., Markowich, P.: The entropy dissipation method for inhomogeneous reaction-diffusion systems. Proc. R. Soc. A 464, 3272–3300 (2008)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Diening, L., Kreuzer, C.: Linear convergence of an adaptive finite element method for the p-Laplacian equation. SIAM J. Numer. Anal. 46, 614–638 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Droniou, J., Vazquez, J.L.: Noncoercive convection-diffusion elliptic problems with Neumann boundary conditions. Calc. Var. Partial Differ. Equ. 34, 413–434 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Ekeland, I., Temam, R.: Convex Analysis and Variational Problems. SIAM, Philadelphia (1999)CrossRefzbMATHGoogle Scholar
  33. 33.
    Engl, H., Hanke-Bourgeois, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer, Dordrecht (1996)CrossRefzbMATHGoogle Scholar
  34. 34.
    Flemming, J.: Theory and examples of variational regularization with non-metric fitting functionals. J. Inverse III-Posed Probl. 18, 677–699 (2010)MathSciNetzbMATHGoogle Scholar
  35. 35.
    Goldstein, T., Osher, S.: The split Bregman method for l 1 regularized problems. SIAM J. Imaging Sci. 2, 323–343 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Grasmair, M.: Generalized Bregman distances and convergence rates for non-convex regularization methods. Inverse Problems 11, 115014 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Grasmair, M.: Variational inequalities and higher order convergence rates for Tikhonov regularisation on Banach spaces. J. Inverse III-Posed Probl. 21, 379–394 (2013)MathSciNetzbMATHGoogle Scholar
  38. 38.
    Hein, T.: Tikhonov regularization in Banach spaces - improved convergence rates results. Inverse Prob. 25, 035002 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Helin, T., Burger, M.: Maximum a posteriori probability estimates in infinite-dimensional Bayesian inverse problems. Preprint arXiv:1412.5816 (2015)Google Scholar
  40. 40.
    Jüngel, A.: The boundedness-by-entropy method for cross-diffusion systems. Nonlinearity 28, 1963–2001 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  41. 41.
    Jüngel, A., Matthes, D.: Entropiemethoden für nichtlineare partielle Differentialgleichungen. Int. Math. Nachr. 209, 1–14 (2008)Google Scholar
  42. 42.
    Jüngel, A., Stelzer, I.V.: Entropy structure of a cross-diffusion tumor-growth model. Math. Mod. Meth. Appl. Sci. 22, 1250009, (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Kaipio, J., Somersalo, E.: Statistical and Computational Inverse Problems. Springer, New York (2005)zbMATHGoogle Scholar
  44. 44.
    Kiwiel, K.C.: Proximal minimization methods with generalized Bregman functions. SIAM J. Control Optim. 35, 1142–1168 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Mielke, A., Haskovec, J., Markowich, P.A.: On uniform decay of the entropy for reaction-diffusion systems. J. Dyn. Differ. Equ. 27, 897–928 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    Mielke, A., Rossi, R., Savare, G.: Nonsmooth analysis of doubly nonlinear evolution equations. Calc. Var. Partial Differ. Equ. 46, 253–310 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Moeller, M.: Multiscale methods for polyhedral regularizations and applications in high dimensional imaging. Ph.D. thesis, University of Muenster (2012)Google Scholar
  48. 48.
    Moeller, M., Burger, M.: Multiscale methods for polyhedral regularizations. SIAM J. Optim. 23, 1424–1456 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  49. 49.
    Müller, J., Brune, C., Sawatzky, A., Kösters, T., Schäfers, K.P., Burger, M.: Reconstruction of short time PET scans using Bregman iterations. In: Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC), pp. 2383–2385. IEEE, New York (2011)Google Scholar
  50. 50.
    Osher, S., Burger, M., Goldfarb, D., Xu, J., Yin, W.: An iterative regularization method for total variation-based image restoration. SIAM Multiscale Model. Simul. 4, 460–489 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  51. 51.
    Otto, F.: The geometry of dissipative evolution equations: the porous medium equation. Commun. Partial Differ. Equ. 26, 101–174 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  52. 52.
    Pöschl, C.: An overview on convergence rates for Tikhonov regularization methods for non-linear operators. J. Inverse III-posed Probl. 17, 77–83 (2009)MathSciNetzbMATHGoogle Scholar
  53. 53.
    Reid, M.: Meet the Bregman divergences. (2013)
  54. 54.
    Resmerita, E.: Regularization of ill-posed problems in Banach spaces: convergence rates. Inverse Problems 21, 1303 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  55. 55.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D: Nonlinear Phenom. 60, 259–268 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  56. 56.
    Saint-Raymond, L.: Convergence of solutions to the Boltzmann equation in the incompressible Euler limit. Arch. Ration. Mech. Anal. 166, 47–80 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  57. 57.
    Scherzer, O., Groetsch, C.: Inverse scale space theory for inverse problems. In: Scale-Space and Morphology in Computer Vision, pp. 317–325. Springer, Berlin/Heidelberg (2001)Google Scholar
  58. 58.
    Werner, F., Hohage, T.: Convergence rates in expectation for Tikhonov-type regularization of inverse problems with Poisson data. Inverse Problems 28, 104004 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  59. 59.
    Witkin, A.P.: Scale-space filtering: a new approach to multi-scale description. In: IEEE International Conference on ICASSP’84 Acoustics, Speech, and Signal Processing, vol. 9, pp. 150–153 (1984)Google Scholar
  60. 60.
    Xu J., Osher, S.: Iterative regularization and nonlinear inverse scale space applied to wavelet-based denoising. IEEE Trans. Image Process. 16, 534–544 (2007)MathSciNetCrossRefGoogle Scholar
  61. 61.
    Yin, W.: Analysis and generalizations of the linearized Bregman method. SIAM J. Imaging Sci. 3, 856–877 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  62. 62.
    Zamponi, N., Jüngel, A.: Analysis of degenerate cross-diffusion population models with volume filling, TU Vienna. Preprint (2015)zbMATHGoogle Scholar
  63. 63.
    Zhang, X., Burger, M., Bresson, X., Osher, S.: Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM J. Imaging Sci. 3, 253–276 (2010)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Institut für Numerische und Angewandte MathematikWestfälische Wilhelms-Universität (WWU) Münster. Einsteinstr. 62MünsterGermany

Personalised recommendations