Regularization of Ill-Posed Problems with Non-negative Solutions

  • Christian ClasonEmail author
  • Barbara Kaltenbacher
  • Elena Resmerita


This survey reviews variational and iterative methods for reconstructing non-negative solutions of ill-posed problems in infinite-dimensional spaces. We focus on two classes of methods: variational methods based on entropy-minimization or constraints, and iterative methods involving projections or non-negativity-preserving multiplicative updates. We summarize known results and point out some open problems.


Convex optimization Fenchel duality Entropy Regularization Sparsity Signal processing 

AMS 2010 Subject Classification

49M20 65K10 90C30 


  1. 1.
    Amato, U., Hughes, W.: Maximum entropy regularization of Fredholm integral equations of the first kind. Inverse Problems 7, 793–803 (1991)CrossRefMathSciNetzbMATHGoogle Scholar
  2. 2.
    Boţ, R.I., Hein, T.: Iterative regularization with a general penalty term—theory and application to L 1 and TV  regularization. Inverse Problems 28(10), 104010, 19 (2012)Google Scholar
  3. 3.
    Borwein, J.: On the failure of maximum entropy reconstruction for Fredholm equations and other infinite systems. Math Program 61, 251–261 (1993)CrossRefMathSciNetzbMATHGoogle Scholar
  4. 4.
    Borwein, J., Goodrich, R., Limber, M.: A comparison of entropies in the underdetermined moment problem (1993). URL Technical report
  5. 5.
    Borwein, J., Lewis, A.: Convergence of best entropy estimates. SIAM Journal on Optimization 1, 191–205 (1991)CrossRefMathSciNetzbMATHGoogle Scholar
  6. 6.
    Borwein, J., Lewis, A.: Duality relationships for entropy-like minimization problems. SIAM Journal on Control and Optimization 29, 325–338 (1991)CrossRefMathSciNetzbMATHGoogle Scholar
  7. 7.
    Borwein, J., Limber, M.: On entropy maximization via convex programming (1996). URL Technical report
  8. 8.
    Bredies, K., Pikkarainen, H.K.: Inverse problems in spaces of measures. ESAIM: Control, Optimisation and Calculus of Variations 19(1), 190–218 (2013)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Brègman, L.M.: The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. and Math. Phys. 7, 200–217 (1967)CrossRefMathSciNetzbMATHGoogle Scholar
  10. 10.
    Byrne, C.: Applied Iterative Methods. A K Peters, Ltd., Wellesley, MA (2008)zbMATHGoogle Scholar
  11. 11.
    Byrne, C.: EM algorithms from a non-stochastic perspective. In: O. Scherzer (ed.) Handbook of Mathematical Methods in Imaging, second edn. Springer New York (2015)Google Scholar
  12. 12.
    Byrne, C., Eggermont, P.P.B.: EM algorithms. In: O. Scherzer (ed.) Handbook of Mathematical Methods in Imaging, second edn., pp. 305–388. Springer New York (2015)Google Scholar
  13. 13.
    Chavent, G., Kunisch, K.: Convergence of Tikhonov regularization for constrained ill-posed inverse problems. Inverse Problems 10(1), 63 (1994)CrossRefMathSciNetzbMATHGoogle Scholar
  14. 14.
    Clason, C., Kunisch, K.: A measure space approach to optimal source placement. Computational Optimization and Applications 53(1), 155–171 (2012)CrossRefMathSciNetzbMATHGoogle Scholar
  15. 15.
    Clason, C., Schiela, A.: Optimal control of elliptic equations with positive measures. Control, Optimisation and Calculus of Variations (ESAIM-COCV) 23, 217–240 (2017)CrossRefMathSciNetzbMATHGoogle Scholar
  16. 16.
    Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting. Multiscale Modeling & Simulation 4(4), 1168–1200 (2005)CrossRefMathSciNetzbMATHGoogle Scholar
  17. 17.
    Dombrovskaja, I., Ivanov, V.K.: On the theory of certain linear equations in abstract spaces. Sibirsk. Mat. Z. 6, 499–508 (1965)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Eggermont, P.P.B.: Maximum entropy regularization for Fredholm integral equations of the first kind. SIAM Journal of Mathematical Analysis 24, 1557–1576 (1993)CrossRefMathSciNetzbMATHGoogle Scholar
  19. 19.
    Eggermont, P.P.B.: Nonlinear smoothing and the EM algorithm for positive integral equations of the first kind. Applied Mathematics and Optimization 39(1), 75–91 (1999)CrossRefMathSciNetzbMATHGoogle Scholar
  20. 20.
    Eggermont, P.P.B., LaRiccia, V.N.: Maximum penalized likelihood estimation and smoothed EM algorithms for positive integral equations of the first kind. Numer. Funct. Anal. Optim. 17, 737–754 (1996)CrossRefMathSciNetzbMATHGoogle Scholar
  21. 21.
    Eicke, B.: Iteration methods for convexly constrained ill-posed problems in Hilbert space. Numerical Functional Analysis and Optimization 13(5–6), 413–429 (1992)CrossRefMathSciNetzbMATHGoogle Scholar
  22. 22.
    Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems, Mathematics and its Applications, vol. 375. Kluwer Academic Publishers Group, Dordrecht (1996)CrossRefzbMATHGoogle Scholar
  23. 23.
    Engl, H.W., Kunisch, K., Neubauer, A.: Convergence rates for Tikhonov regularisation of non-linear ill-posed problems. Inverse Problems 5, 523–540 (1989)CrossRefMathSciNetzbMATHGoogle Scholar
  24. 24.
    Engl, H.W., Landl, G.: Convergence rates for maximum entropy regularization. SIAM J. Num. Anal. 30, 1509–1536 (1993)CrossRefMathSciNetzbMATHGoogle Scholar
  25. 25.
    Engl, H.W., Landl, G.: Maximum entropy regularization of nonlinear ill-posed problems. In: World Congress of Nonlinear Analysts ’92, Vol. I–IV (Tampa, FL, 1992), pp. 513–525. de Gruyter, Berlin (1996)Google Scholar
  26. 26.
    Flemming, J.: Generalized Tikhonov regularization: Basic theory and comprehensive results on convergence rates. Ph.D. thesis, TU Chemnitz (2011). URL
  27. 27.
    Flemming, J., Hofmann, B.: Convergence rates in constrained Tikhonov regularization: equivalence of projected source conditions and variational inequalities. Inverse Problems 27(8), 085001 (2011)CrossRefMathSciNetzbMATHGoogle Scholar
  28. 28.
    Haltmeier, M., Leitão, A., Resmerita, E.: On regularization methods of EM-Kaczmarz type. Inverse Problems 25, 075008 (2009)CrossRefMathSciNetzbMATHGoogle Scholar
  29. 29.
    Hämarik, U., Kaltenbacher, B., Kangro, U., Resmerita, E.: Regularization by discretization in Banach spaces. Inverse Problems 32, 035004 (2016)CrossRefMathSciNetzbMATHGoogle Scholar
  30. 30.
    Hanke, M., Groetsch, C.: Nonstationary iterated Tikhonov regularization. Journal of Optimization Theory and Applications 98, 37–53 (1998)CrossRefMathSciNetzbMATHGoogle Scholar
  31. 31.
    Hofmann, B., Kaltenbacher, B., Pöschl, C., Scherzer, O.: A convergence rates result for Tikhonov regularization in Banach spaces with non-smooth operators. Inverse Problems 23(3), 987–1010 (2007)CrossRefMathSciNetzbMATHGoogle Scholar
  32. 32.
    Hudson, H.M., Larkin, R.S.: Accelerated image reconstruction using ordered subsets of projection data. IEEE Trans. Med. Imaging 13, 601–609 (1994)CrossRefGoogle Scholar
  33. 33.
    Iusem, A.: A short convergence proof of the EM algorithm for a specific Poisson model. Brazilian Journal of Probability and Statistics 6, 57–67 (1992). URL
  34. 34.
    Iusem, A.: Metodos de Pontos Proximal EM Optimizacao. IMPA, Rio de Janeiro (1995)Google Scholar
  35. 35.
    Ivanov, V.K.: On linear problems which are not well-posed. Dokl. Akad. Nauk SSSR 145, 270–272 (1962)MathSciNetGoogle Scholar
  36. 36.
    Ivanov, V.K.: On ill-posed problems. Mat. Sb. (N.S.) 61 (103), 211–223 (1963)Google Scholar
  37. 37.
    Ivanov, V.K., Vasin, V.V., Tanana, V.P.: Theory of Linear Ill-posed Problems and Its Applications. Inverse and Ill-posed Problems Series. VSP, Utrecht (2002)Google Scholar
  38. 38.
    Jin, Q., Wang, W.: Landweber iteration of Kaczmarz type with general non-smooth convex penalty functionals. Inverse Problems 29(8), 085011, 22 (2013)Google Scholar
  39. 39.
    Kaltenbacher, B., Klassen, A.: On convergence and convergence rates for Ivanov and Morozov regularization and application to some parameter identification problems in elliptic PDEs. Inverse Problems 34(5), 055008 (2018)CrossRefMathSciNetzbMATHGoogle Scholar
  40. 40.
    Kondor, A.: Method of convergent weights – an iterative procedure for solving Fredholm’s integral equations of the first kind. Nuclear Instruments and Methods in Physics Research 216, 177–181 (1983)CrossRefGoogle Scholar
  41. 41.
    Lellmann, J., Lorenz, D.A., Schönlieb , C., Valkonen, T.: Imaging with Kantorovich–Rubinstein discrepancy. SIAM Journal on Imaging Sciences 7(4), 2833–2859 (2014)CrossRefMathSciNetzbMATHGoogle Scholar
  42. 42.
    Lorenz, D., Worliczek, N.: Necessary conditions for variational regularization schemes. Inverse Problems 29(7), 075016 (2013)CrossRefMathSciNetzbMATHGoogle Scholar
  43. 43.
    Lucy, L.: An iterative technique for the rectification of observed distributions. Astron. J. 7, 81–92 (1975)Google Scholar
  44. 44.
    Mülthei, H.N.: Iterative continuous maximum-likelihood reconstruction methods. Math. Methods Appl. Sci. 15, 275–286 (1992)CrossRefMathSciNetzbMATHGoogle Scholar
  45. 45.
    Mülthei, H.N., Schorr, B., Törnig, W.: On an iterative method for a class of integral equations of the first kind. Math. Methods Appl. Sci. 9, 137–168 (1987)CrossRefMathSciNetzbMATHGoogle Scholar
  46. 46.
    Mülthei, H.N., Schorr, B., Törnig, W.: On properties of the iterative maximum likelihood reconstruction method. Math. Methods Appl. Sci. 11, 331–342 (1989)CrossRefMathSciNetzbMATHGoogle Scholar
  47. 47.
    Neubauer, A.: Tikhonov-regularization of ill-posed linear operator equations on closed convex sets. Journal of Approximation Theory 53(3), 304–320 (1988)CrossRefMathSciNetzbMATHGoogle Scholar
  48. 48.
    Neubauer, A.: Tikhonov regularisation for non-linear ill-posed problems: optimal convergence rates and finite-dimensional approximation. Inverse Problems 5(4), 541 (1989)CrossRefMathSciNetzbMATHGoogle Scholar
  49. 49.
    Neubauer, A., Ramlau, R.: On convergence rates for quasi-solutions of ill-posed problems. Electron. Trans. Numer. Anal. 41, 81–92 (2014). URL
  50. 50.
    Piana, M., Bertero, M.: Projected Landweber method and preconditioning. Inverse Problems 13(2), 441–463 (1997)CrossRefMathSciNetzbMATHGoogle Scholar
  51. 51.
    Pöschl, C.: Tikhonov regularization with general residual term. Ph.D. thesis, University of Innsbruck (2008)Google Scholar
  52. 52.
    Resmerita, E., Anderssen, R.S.: A joint additive Kullback–Leibler residual minimization and regularization for linear inverse problems. Math. Methods Appl. Sci. 30, 1527–1544 (2007)CrossRefMathSciNetzbMATHGoogle Scholar
  53. 53.
    Resmerita, E., Engl, H.W., Iusem, A.N.: The expectation-maximization algorithm for ill-posed integral equations: a convergence analysis. Inverse Problems 23(6), 2575 (2007)CrossRefMathSciNetzbMATHGoogle Scholar
  54. 54.
    Resmerita, E., Engl, H.W., Iusem, A.N.: Corrigendum. The expectation-maximization algorithm for ill-posed integral equations: a convergence analysis. Inverse Problems 24(5), 059801 (2008)zbMATHGoogle Scholar
  55. 55.
    Richardson, W.H.: Bayesian-based iterative method of image restoration. J. Opt. Soc. Am. 62, 55–59 (1972)CrossRefGoogle Scholar
  56. 56.
    Seidman, T.I., Vogel, C.R.: Well posedness and convergence of some regularisation methods for non-linear ill posed problems. Inverse Problems 5(2), 227 (1989)CrossRefMathSciNetzbMATHGoogle Scholar
  57. 57.
    Shepp, L.A., Vardi, Y.: Maximum likelihood reconstruction in positron emission tomography. IEEE Trans. Medical Imaging 1, 113–122 (1982)CrossRefGoogle Scholar
  58. 58.
    Silverman, B.W., Jones, M.C., Nychka, D.W., Wilson, J.D.: A smoothed EM approach to indirect estimation problems, with particular reference to stereology and emission tomography. J. Roy. Statist. Soc. B 52, 271–324 (1990). URL
  59. 59.
    Stummer, W., Vajda, I.: On Bregman distances and divergences of probability measures. IEEE Trans. Information Theory 58(3), 1277–1288 (2012)CrossRefMathSciNetzbMATHGoogle Scholar
  60. 60.
    Tikhonov, A.N., Arsenin, V.Y.: Solutions of Ill-Posed Problems. Wiley, New York (1977)zbMATHGoogle Scholar
  61. 61.
    Vardi, Y., Shepp, L.A., Kaufmann, L.: A statistical model for positron emission tomography. J. Am. Stat. Assoc 80, 8–37 (1985)CrossRefMathSciNetzbMATHGoogle Scholar
  62. 62.
    Werner, F.: Inverse problems with Poisson data: Tikhonov-type regularization and iteratively regularized Newton methods. Ph.D. thesis, University of Göttingen (2012). URL
  63. 63.
    Werner, F., Hohage, T.: Convergence rates in expectation for Tikhonov-type regularization of inverse problems with Poisson data. Inverse Problems 28(10), 104004 (2012)CrossRefMathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Christian Clason
    • 1
    Email author
  • Barbara Kaltenbacher
    • 2
  • Elena Resmerita
    • 2
  1. 1.Faculty of MathematicsUniversity Duisburg-EssenEssenGermany
  2. 2.Institute of MathematicsAlpen-Adria Universität KlagenfurtKlagenfurtAustria

Personalised recommendations