Skip to main content
SpringerLink
Log in
Book cover

Handbook of Mathematical Methods in Imaging pp 345–384Cite as

Iterative Solution Methods

  • Reference work entry

Abstract

This chapter deals with iterative methods for nonlinear ill-posed problems. We present gradient and Newton type methods as well as nonstandard iterative algorithms such as Kaczmarz, expectation maximization, and Bregman iterations.Our intention here is to cite convergence results in the sense of regularization and to provide further references to the literature.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   679.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References and Further Reading

  1. Akcelik V, Biros G, Draganescu A, Hill J, Ghattas O, Van Bloemen Waanders B (2005) Dynamic data-driven inversion for terascale simulations: Real-time identification of airborne contaminants. In: Proceedings of SC05. IEEE/ACM, Seattle

    Google Scholar 

  2. Bachmayr M, Burger M (2009) Iterative total variation schemes for nonlinear inverse problems. Inverse Prob 25, 105004

    Article  MathSciNet  Google Scholar 

  3. Bakushinsky AB (1992) The problem of the convergence of the iteratively regularized Gauss-Newton method. Comput Math Math Phys 32:1353–1359

    MathSciNet  Google Scholar 

  4. Bakushinsky AB (1995) Iterative methods without degeneration for solving degenerate nonlinear operator equations. Dokl Akad Nauk 344:7–8

    MathSciNet  Google Scholar 

  5. Bakushinsky AB, Kokurin MY (2004) Iterative methods for approximate solution of inverse problems. Vol. 577 of Mathematics and its applications. Springer, Dordrecht

    Google Scholar 

  6. Bauer F, Hohage T (2005) A Lepskij-type stopping rule for regularized Newton methods. Inverse Prob 21:1975–1991

    Article  MathSciNet  MATH  Google Scholar 

  7. Bauer F, Hohage T, Munk A (2009) Iteratively regularized Gauss-Newton method for nonlinear inverse problems with random noise. SIAM J Numer Anal 47:1827–1846

    Article  MathSciNet  MATH  Google Scholar 

  8. Baumeister J, De Cezaro A, Leitao A (2009) On iterated Tikhonov-Kaczmarz regularization methods for ill-posed problems, ICJV (2010), doi: 10.1007/s11263-010-0339-5

    Google Scholar 

  9. Baumeister J, Kaltenbacher B, Leitao A (2010) On Levenberg-Marquardt-Kaczmarz iterative methods for solving systems of nonlinear ill-posed equations. Inverse Problems and Imaging, (to appear)

    Google Scholar 

  10. Bertero M, Boccacci P (1998) Introduction to inverse problems in imaging. Institute of Physics, Bristol

    Book  MATH  Google Scholar 

  11. Bissantz N, Hohage T, Munk A, Ruymgaart F (2007) Convergence rates of general regularization methods for statistical inverse problems and applications. SIAM J Numer Anal 45:2610–2636

    Article  MathSciNet  MATH  Google Scholar 

  12. Bissantz N, Mair B, Munk A (2008) A statistical stopping rule for mlem reconstructions in pet. IEEE Nucl Sci Symp Conf Rec 8:4198–4200

    Google Scholar 

  13. Blaschke B, Neubauer A, Scherzer O (1997) On convergence rates for the iteratively regularized Gauss–Newton method. IMA J Numer Anal 17:421–436

    Article  MathSciNet  MATH  Google Scholar 

  14. Bregman LM (1967) The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comp Math Math Phys 7:200–217

    Article  Google Scholar 

  15. Brune C, Sawatzky A, Burger M (2009) Bregman-EM-TV methods with application to optical nanoscopy. In: Tai X-C et al (ed) Proceedings of the 2nd International Conference on Scale Space and Variational Methods in Computer Vision. Vol. 5567 of LNCS, Springer, pp. 235–246

    Google Scholar 

  16. Brune C, Sawatzky A, Burger M (2009) Primal and dual Bregman methods with application to optical nanoscopy. Preprint, Submitted to IJCV: Special Issue SSVM09, Institute of Computational and Applied Mathematics

    Google Scholar 

  17. Burger M, Kaltenbacher B (2006) Regularizing Newton–Kaczmarz methods for nonlinear ill-posed problems. SIAM J Numer Anal 44:153–182

    Article  MathSciNet  MATH  Google Scholar 

  18. Cai JF, Osher S, Shen Z (2009) Convergence of the linearized Bregman iteration for l1-norm minimization. Math Comp 78:2127–2136

    Article  MathSciNet  MATH  Google Scholar 

  19. Cai JF, OSher S, Shen Z (2009) Linearized Bregman iterations for compressed sensing. Math Comp 78:1515–1536

    Google Scholar 

  20. Cai JF, OSher S, Shen Z (2009) Linearized Bregman iterations for frame-based image deblurring. SIAM J Imaging Sci 2:226–252

    Google Scholar 

  21. Ciarlet PG (1978) The finite element method for elliptic problems. North Holland, Amsterdam

    MATH  Google Scholar 

  22. Colonius F, Kunisch K (1989) Output least squares stability in elliptic systems. Appl Math Optim 19:33–63

    Article  MathSciNet  MATH  Google Scholar 

  23. Combettes PL, Pesquet J-C (2008) A proximal decomposition method for solving convex variational inverse problems, Inverse Prob 24, 065014 (27pp)

    Google Scholar 

  24. Daubechies I, Defrise M, De Mol C (2004) An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Comm Pure Appl Math 57:1413–1457

    Article  MathSciNet  MATH  Google Scholar 

  25. De Cezaro A, Haltmeier M, Leitao A, Scherzer O (2008) On steepest-descent-Kaczmarz methods for regularizing systems of nonlinear ill-posed equations. Appl Math Comp 202:596–607

    Article  MATH  Google Scholar 

  26. Dembo RS, Eisenstat SC, Steihaug T (1982) Inexact Newton’s method. SIAM J Numer Anal 14:400–408

    Article  MathSciNet  Google Scholar 

  27. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B 39

    Google Scholar 

  28. Deuflhard P, Engl HW, Scherzer O (1998) A convergence analysis of iterative methods for the solution of nonlinear ill-posed problems under affinely invariant conditions. Inverse Prob 14:1081–1106

    Article  MathSciNet  MATH  Google Scholar 

  29. Douglas J, Rachford HH (1956) On the numerical solution of heat conduction problems in two and three space variables. Trans Am Math Soc 82:421–439

    Article  MathSciNet  MATH  Google Scholar 

  30. Egger H (2007) Fast fully iterative Newton-type methods for inverse problems. J Inverse Ill-Posed Prob 15:257–275

    Article  MathSciNet  MATH  Google Scholar 

  31. Egger H (2008) Y-Scale regularization. SIAM J Numer Anal 46:419–436

    Article  MathSciNet  MATH  Google Scholar 

  32. Egger H, Neubauer A (2005) Preconditioning Landweber iteration in Hilbert scales. Numer Math 101:643–662

    Article  MathSciNet  MATH  Google Scholar 

  33. Eicke B, Louis AK, Plato R (1990) The instability of some gradient methods for ill-posed problems. Numer Math 58:129–134

    Article  MathSciNet  MATH  Google Scholar 

  34. Engl HW, Hanke M, Neubauer A (1996) Regularization of inverse problems. Kluwer, Dordrecht

    Book  MATH  Google Scholar 

  35. Engl HW, Zou J (2000) A new approach to convergence rate analysis of Tiknonov regularization for parameter identification in heat conduction. Inverse Prob 16:1907–1923

    Article  MathSciNet  MATH  Google Scholar 

  36. Glowinski R, Le Tallec P (1989) Augmented lagrangian and operator splitting methods in nonlinear mechanics. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  37. Green PJ (1990) Bayesian reconstructions from emission tomography data using a modified EM algorithm. IEEE Trans Med Imaging 9:84–93

    Article  Google Scholar 

  38. Green PJ (1990) On use of the EM algorithm for penalized likelihood estimation. J R Stat Soc Ser B (Methodological) 52:443–452

    MATH  Google Scholar 

  39. Haber E (2005) Quasi-Newton methods for large-scale electromagnetic inverse problems. Inverse Prob 21:305–323

    Article  MathSciNet  MATH  Google Scholar 

  40. Haber E, Ascher U (2001) A multigrid method for distributed parameter estimation problems. Inverse Prob 17:1847–1864

    Article  MathSciNet  MATH  Google Scholar 

  41. Haltmeier M, Kowar R, Leitao A, Scherzer O (2007) Kaczmarz methods for regularizing nonlinear ill-posed equations II: applications. Inverse Prob Imaging 1:507–523

    Article  MathSciNet  MATH  Google Scholar 

  42. Haltmeier M, Leitao A, Scherzer O (2007) Kaczmarz methods for regularizing nonlinear ill-posed equations I: convergence analysis. Inverse Prob Imaging 1:289–298

    Article  MathSciNet  MATH  Google Scholar 

  43. Hanke M (1997) A regularization Levenberg–arquardt scheme, with applications to inverse groundwater filtration problems. Inverse Prob 13:79–95

    Article  MathSciNet  MATH  Google Scholar 

  44. Hanke M (1997) Regularizing properties of a truncated Newton-CG algorithm for nonlinear inverse problems. Numer Funct Anal Optim 18:971–993

    Article  MathSciNet  MATH  Google Scholar 

  45. Hanke M (2009) The regularizing Levenberg-Marquardt scheme is of optimal order, J. Integral Equations Appl. 22, (2010), 259–283

    MathSciNet  Google Scholar 

  46. Hanke M, Neubauer A, Scherzer O (1995) A convergence analysis of the Landweber iteration for nonlinear ill-posed problems. Numer Math 72:21–37

    Article  MathSciNet  MATH  Google Scholar 

  47. He L, Burger M, Osher S (2006) Iterative total variation regularization with non-quadratic fidelity. J Math Imaging Vision 26:167–184

    Article  MathSciNet  Google Scholar 

  48. Hochbruck M, Hönig M, Ostermann A (2009) A convergence analysis of the exponential Euler iteration for nonlinear ill-posed problems. Inverse Prob 25:075009 (18pp)

    Google Scholar 

  49. Hochbruck M, Hönig M, Ostermann A (2009) Regularization of nonlinear ill-posed problems by exponential integrators. Math Mod Numer Anal 43:709–720

    Article  MATH  Google Scholar 

  50. Hohage T (1997) Logarithmic convergence rates of the iteratively regularized Gauß-Newton method for an inverse potential and an inverse scattering problem. Inverse Prob 13: 1279–1299

    Article  MathSciNet  MATH  Google Scholar 

  51. Hohage T (1999) Iterative methods in inverse obstacle scattering: regularization theory of linear and nonlinear exponentially ill-posed problems. PhD thesis, University of Linz

    Google Scholar 

  52. Hohage T (2000) Regularization of exponentially ill-posed problems. Numer Funct Anal Optim 21:439–464

    Article  MathSciNet  MATH  Google Scholar 

  53. Jin Q, Tautenhahn U (2009) On the discrepancy principle for some Newton type methods for solving nonlinear ill-posed problems. Numer Math 111:509–558

    Article  MathSciNet  MATH  Google Scholar 

  54. Kaltenbacher B (1997) Some Newton type methods for the regularization of nonlinear ill-posed problems. Inverse Prob 13:729–753

    Article  MathSciNet  MATH  Google Scholar 

  55. Kaltenbacher B (1998) On Broyden’s method for ill-posed problems. Numer Funct Anal Optim 19:807–833

    Article  MathSciNet  MATH  Google Scholar 

  56. Kaltenbacher B (1998) A posteriori parameter choice strategies for some Newton type methods for the regularization of nonlinear ill-posed problems. Numer Math 79:501–528

    Article  MathSciNet  MATH  Google Scholar 

  57. Kaltenbacher B (2000) A projection-regularized Newton method for nonlinear ill-posed problems and its application to parameter identification problems with finite element discretization. SIAM J Numer Anal 37:1885–1908

    Article  MathSciNet  MATH  Google Scholar 

  58. Kaltenbacher B (2001) On the regularizing properties of a full multigrid method for ill-posed problems. Inverse Prob 17:767–788

    Article  MathSciNet  MATH  Google Scholar 

  59. Kaltenbacher B, Hofmann B (2009) Convergence rates for the iteratively regularized Gauss-Newton method in Banach spaces, Inverse Problems 26 (2010), 035007

    MathSciNet  Google Scholar 

  60. Kaltenbacher B, Neubauer A (2006) Convergence of projected iterative regularization methods for nonlinear problems with smooth solutions. Inverse Prob 22:1105–1119

    Article  MathSciNet  MATH  Google Scholar 

  61. Kaltenbacher B, Neubauer A, Ramm AG (2002) Convergence rates of the continuous regularized Gauss–Newton method. J Inverse Ill-Posed Prob 10:261–280

    MathSciNet  MATH  Google Scholar 

  62. Kaltenbacher B, Neubauer A, Scherzer O (2008) Iterative regularization methods for nonlinear ill-posed problems. Radon Series on Computational and Applied Mathematics, de Gruyter, Berlin

    Google Scholar 

  63. Kaltenbacher B, Schicho J (2002) A multi-grid method with a priori and a posteriori level choice for the regularization of nonlinear ill-posed problems. Numer Math 93:77–107

    MathSciNet  MATH  Google Scholar 

  64. Kaltenbacher B, Schöpfer F, Schuster T (2009) Iterative methods for nonlinear ill-posed problems in Banach spaces: convergence and applications to parameter identification problems. Inverse Prob 25:065003 (19pp)

    Google Scholar 

  65. King JT (1992) Multilevel algorithms for ill-posed problems. Numer Math 61:311–334

    Article  MathSciNet  MATH  Google Scholar 

  66. Kowar R, Scherzer O (2002) Convergence analysis of a Landweber-Kaczmarz method for solving nonlinear ill-posed problems. In: Romanov VG, Kabanikhin SI, Anikonov YuE, Bukhgeim AL (eds) Ill-posed and inverse problems. Zeist, VSP, pp 69–90

    Google Scholar 

  67. Krein SG, Petunin JI (1966) Scales of Banach spaces. Russian Math Surveys 21:85–160

    Article  MathSciNet  Google Scholar 

  68. Kügler P (2003) A derivative free Landweber iteration for parameter identification in certain elliptic PDEs. Inverse Prob 19:1407–1426

    Article  MATH  Google Scholar 

  69. Kügler P (2003) A derivative free landweber method for parameter identification in elliptic partial differential equations with application to the manufacture of car wind-shields. PhD thesis, Johannes Kepler University, Linz, Austria

    Google Scholar 

  70. Langer S (2007) Preconditioned Newton methods for ill-posed problems. PhD thesis, University of Göttingen

    Google Scholar 

  71. Langer S, Hohage T (2007) Convergence analysis of an inexact iteratively regularized Gauss-Newton method under general source conditions. J Inverse Ill-Posed Prob 15:19–35

    Article  MathSciNet  Google Scholar 

  72. Lions P-L, Mercier B (1979) Splitting algorithms for the sum of two nonlinear operators. SIAM J Numer Anal 16:964–979

    Article  MathSciNet  MATH  Google Scholar 

  73. Mülthei HN, Schorr B (1989) On properties of the iterative maximum likelihood reconstruction method. Math Methods Appl Sci 11:331–342

    Article  MathSciNet  MATH  Google Scholar 

  74. Natterer F, Wübbeling F (2001) Mathematical methods in image reconstruction. Society for Industrial and Applied Mathematics (SIAM), Philadelphia

    Google Scholar 

  75. Neubauer A (1992) Tikhonov regularization of nonlinear ill-posed problems in Hilbert scales. Appl Anal 46:59–72

    Article  MathSciNet  MATH  Google Scholar 

  76. Neubauer A (2000) On Landweber iteration for nonlinear ill-posed problems in Hilbert scales. Numer Math 85:309–328

    Article  MathSciNet  MATH  Google Scholar 

  77. Neubauer A, Scherzer O (1995) A convergent rate result for a steepest descent method and a minimal error method for the solution of nonlinear ill-posed problems. ZAA 14:369–377

    MathSciNet  MATH  Google Scholar 

  78. Osher S, Burger M, Goldfarb D, Xu J, Yin W (2005) An iterative regularization method for total variation based image restoration. SIAM Multiscale Mod Simul 4:460–489

    Article  MathSciNet  MATH  Google Scholar 

  79. Press WH, Teukolsky SA, Vetterling WT, Flannery BP (2007) Numerical recipes: the art of scientific computing, 3rd edn. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  80. Resmerita E, Engl HW, Iusem AN (2007) The expectation-maximization algorithm for ill-posed integral equations: a convergence analysis. Inverse Prob 23:2575–2588

    Article  MathSciNet  MATH  Google Scholar 

  81. Rieder A (1999) On the regularization of nonlinear ill-posed problems via inexact Newton iterations. Inverse Prob 15:309–327

    Article  MathSciNet  MATH  Google Scholar 

  82. Rieder A (2001) On convergence rates of inexact Newton regularizations. Numer Math 88: 347–365

    Article  MathSciNet  MATH  Google Scholar 

  83. Rieder A (2005) Inexact Newton regularization using conjugate gradients as inner iteration. SIAM J Numer Anal 43:604–622

    Article  MathSciNet  MATH  Google Scholar 

  84. Sawatzky A, Brune C, Wübbeling F, Kösters T, Schäfers K, Burger M (2008) Accurate EM-TV algorithm in PET with low SNR. Nuclear Science Symposium Conference Record. NSS’08. IEEE, pp 5133–5137

    Google Scholar 

  85. Scherzer O (1998) A modified Landweber iteration for solving parameter estimation problems. Appl Math Optim 38:45–68

    Article  MathSciNet  MATH  Google Scholar 

  86. Schöpfer F, Louis AK, Schuster T (2006) Nonlinear iterative methods for linear ill-posed problems in Banach spaces. Inverse Prob 22:311–329

    Article  MATH  Google Scholar 

  87. Schöpfer F, Schuster T, Louis AK (2008) An iterative regularization method for the solution of the split feasibility problem in banach spaces. Inverse Prob 24:055008 (20pp)

    Google Scholar 

  88. Vardi Y, Shepp LA, Kaufman L (1985) A statistical model for positron emission tomography with discussion. J Am Stat Assoc 80:8–37

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Mainz, Mainz, Germany

    Martin Burger

  2. University of Graz, Graz, Austria

    Barbara Kaltenbacher

  3. Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany

    Andreas Neubauer

Authors
  1. Martin Burger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Barbara Kaltenbacher
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andreas Neubauer
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Computational Science Center, University of Vienna, Nordbergstrasse 15, Vienna, Austria

    Otmar Scherzer

  2. RICAM, Austrian Academy of Sciences, Linz, Austria

    Otmar Scherzer

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this entry

Cite this entry

Burger, M., Kaltenbacher, B., Neubauer, A. (2011). Iterative Solution Methods. In: Scherzer, O. (eds) Handbook of Mathematical Methods in Imaging. Springer, New York, NY. https://doi.org/10.1007/978-0-387-92920-0_9

Download citation

Publish with us

Policies and ethics

Search

Navigation