Skip to main content
SpringerLink
Log in

Large-Scale Inverse Problems in Imaging

  • Reference work entry
Handbook of Mathematical Methods in Imaging

Abstract

Large-scale inverse problems arise in a variety of significant applications in image processing, and efficient regularization methods are needed to compute meaningful solutions. This chapter surveys three common mathematical models including a linear, a separable nonlinear, and a general nonlinear model. Techniques for regularization and large-scale implementations are considered, with particular focus on algorithms and computations that can exploit structure in the problem. Examples from image deconvolution, multi-frame blind deconvolution, and tomosynthesis illustrate the potential of these algorithms. Much progress has been made in the field of large-scale inverse problems, but many challenges still remain for future research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 679.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References and Further Reading

  1. Andrews HC, Hunt BR (1977) Digital image restoration. Prentice-Hall, Englewood Cliffs

    Google Scholar 

  2. Bachmayr M, Burger M (2009) Iterative total variation schemes for nonlinear inverse problems. Inverse Prob 25:105004

    Article  MathSciNet  Google Scholar 

  3. Bardsley JM (2008) An efficient computational method for total variation-penalized Poisson likelihood estimation. Inverse Prob Imaging 2(2):167–185

    Article  MathSciNet  MATH  Google Scholar 

  4. Bardsley JM (2008) Stopping rules for a nonnegatively constrained iterative method for illposed Poisson imaging problems. BIT 48(4):651–664

    Article  MathSciNet  MATH  Google Scholar 

  5. Bardsley JM, Vogel CR (2003) A nonnegatively constrained convex programming method for image reconstruction. SIAM J Sci Comput 25(4):1326–1343

    Article  MathSciNet  MATH  Google Scholar 

  6. Barzilai J, Borwein JM (1988) Two-point step size gradient methods. IMA J Numer Anal 8(1):141–148

    Article  MathSciNet  MATH  Google Scholar 

  7. Björck Å (1988) A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations. BIT, 28(3):659–670

    Article  MathSciNet  MATH  Google Scholar 

  8. Björck Å (1996) Numerical methods for least squares problems. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  9. Björck Å, Grimme E, van Dooren P (1994) An implicit shift bidiagonalization algorithm for ill-posed systems. BIT 34(4):510–534

    Article  MathSciNet  MATH  Google Scholar 

  10. Brakhage H (1987) On ill-posed problems and the method of conjugate gradients. In: Engl HW, Groetsch CW (eds) Inverse and ill-posed problems. Academic, Boston, pp 165–175

    Google Scholar 

  11. Calvetti D, Reichel L (2003) Tikhonov regularization of large linear problems. BIT 43(2):263–283

    Article  MathSciNet  MATH  Google Scholar 

  12. Calvetti D, Somersalo E (2007) Introduction to Bayesian scientific computing. Springer, New York

    MATH  Google Scholar 

  13. Candès EJ, Romberg JK, Tao T (2006) Stable signal recovery from incomplete and inaccurate measurements. Commun Pure Appl Math 59(8):1207–1223

    Article  MATH  Google Scholar 

  14. Carasso AS (2001) Direct blind deconvolution. SIAM J Appl Math 61(6):1980–2007

    Article  MathSciNet  MATH  Google Scholar 

  15. Chadan K, Colton D, Päivärinta L, Rundell W (1997) An introduction to inverse scattering and inverse spectral problems. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  16. Chan TF, Shen J (2005) Image processing and analysis: variational, PDE, wavelet, and stochastic methods. SIAM, Philadelphia

    MATH  Google Scholar 

  17. Cheney M, Borden B (2009) Fundamentals of radar imaging. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  18. Chung J, Haber E, Nagy J (2006) Numerical methods for coupled super-resolution. Inverse Prob 22(4):1261–1272

    Article  MathSciNet  MATH  Google Scholar 

  19. Chung J, Nagy J (2010) An efficient iterative approach for large-scale separable nonlinear inverse problems. SIAM J Sci Comput 31(6):4654–4674

    Article  MathSciNet  MATH  Google Scholar 

  20. Chung J, Nagy J, Sechopoulos I (2010) Numerical algorithms for polyenergetic digital breast tomosynthesis reconstruction. SIAM J Imaging Sci 3(1):133–152

    Article  MathSciNet  MATH  Google Scholar 

  21. Chung J, Nagy JG, O’Leary DP (2008) A weighted GCV method for Lanczos hybrid regularization. Elec Trans Numer Anal 28: 149–167

    MathSciNet  Google Scholar 

  22. Chung J, Sternberg P, Yang C (2010) High performance 3-d image reconstruction for molecular structure determination. Int J High Perform Comput Appl 24(2):117–135

    Article  Google Scholar 

  23. De Man B, Nuyts J, Dupont P, Marchal G, Suetens P (2001) An iterative maximumlikelihood polychromatic algorithm for CT. IEEE Trans Med Imaging 20(10):999–1008

    Article  Google Scholar 

  24. Diaspro A, Corosu M, Ramoino P, Robello M (1999) Two-photon excitation imaging based on a compact scanning head. IEEE Eng Med Biol 18(5):18–30

    Article  Google Scholar 

  25. Dobbins JT III, Godfrey DJ (2003) Digital X-ray tomosynthesis: current state of the art and clinical potential. Phys Med Biol 48(19):R65–R106

    Article  Google Scholar 

  26. Easley GR, Healy DM, Berenstein CA (2009) Image deconvolution using a general ridgelet and curvelet domain. SIAM J Imaging Sci 2(1):253–283

    Article  MATH  Google Scholar 

  27. Elad M, Feuer A (1997) Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans Image Process 6(12):1646–1658

    Article  Google Scholar 

  28. Engl HW, Hanke M, Neubauer A (2000) Regularization of inverse problems. Kluwer, Dordrecht

    Google Scholar 

  29. Engl HW, Kügler P (2005) Nonlinear inverse problems: theoretical aspects and some industrial applications. In: Capasso V, Périaux J (eds) Multidisciplinary methods for analysis optimization and control of complex systems. Springer, Berlin, pp 3–48

    Chapter  Google Scholar 

  30. Engl HW, Kunisch K, Neubauer A (1989) Convergence rates for Tikhonov regularisation of nonlinear ill-posed problems. Inverse Prob 5(4):523–540

    Article  MathSciNet  MATH  Google Scholar 

  31. Engl HW, Louis AK, Rundell W (eds) (1996) Inverse problems in geophysical applications. SIAM, Philadelphia

    Google Scholar 

  32. Eriksson J, Wedin P (2004) Truncated Gauss-Newton algorithms for ill-conditioned nonlinear least squares problems. Optim Meth Softw 19(6):721–737

    Article  MathSciNet  MATH  Google Scholar 

  33. Faber TL, Raghunath N, Tudorascu D, Votaw JR (2009) Motion correction of PET brain images through deconvolution: I. Theoretical development and analysis in software simulations. Phys Med Biol 54(3):797–811

    Article  Google Scholar 

  34. Figueiredo MAT, Nowak RD, Wright SJ (2007) Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. IEEE J Sel Top Signal Process 1(4):586–597

    Article  Google Scholar 

  35. Frank J (2006) Three-dimensional electron microscopy of macromolecular assemblies. Oxford University Press, New York

    Book  Google Scholar 

  36. Golub GH, Heath M, Wahba G (1979) Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21(2):215–223

    MathSciNet  MATH  Google Scholar 

  37. Golub GH, Luk FT, Overton ML (1981) A block Lanczos method for computing the singular values and corresponding singular vectors of a matrix. ACM Trans Math Softw 7(2): 149–169

    Article  MathSciNet  MATH  Google Scholar 

  38. Golub GH, Pereyra V (1973) The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate. SIAM J Numer Anal 10(2):413–432

    Article  MathSciNet  MATH  Google Scholar 

  39. Golub GH, Pereyra V (2003) Separable nonlinear least squares: the variable projection method and its applications. Inverse Prob 19: R1–R26

    Article  MathSciNet  MATH  Google Scholar 

  40. Haber E, Ascher UM, Oldenburg D (2000) On optimization techniques for solving nonlinear inverse problems. Inverse Prob 16(5):1263–1280

    Article  MathSciNet  MATH  Google Scholar 

  41. Haber E, Oldenburg D (2000) A GCV based method for nonlinear ill-posed problems. Comput Geosci 4(1):41–63

    Article  MathSciNet  MATH  Google Scholar 

  42. Hammerstein GR, Miller DW, White DR, Masterson ME, Woodard HQ, Laughlin JS (1979) Absorbed radiation dose in mammography. Radiology 130(2):485–491

    Google Scholar 

  43. Hanke M (1995) Conjugate gradient type methods for ill-posed problems. Pitman research notes in mathematics, Longman Scientific & Technical, Harlow

    Google Scholar 

  44. Hanke M (1996) Limitations of the L-curve method in ill-posed problems. BIT 36(2):287–301

    Article  MathSciNet  MATH  Google Scholar 

  45. Hanke M (2001) On Lanczos based methods for the regularization of discrete ill-posed problems. BIT 41(5):1008–1018

    Article  MathSciNet  Google Scholar 

  46. Hansen PC (1992) Analysis of discrete ill-posed problems by means of the L-curve. SIAM Rev 34(4):561–580

    Article  MathSciNet  MATH  Google Scholar 

  47. Hansen PC (1992) Numerical tools for analysis and solution of Fredholm integral equations of the first kind. Inverse Prob 8(6):849–872

    Article  MATH  Google Scholar 

  48. Hansen PC (1994) Regularization tools: a MATLAB package for analysis and solution of discrete ill-posed problems. Numer Algorithms 6(1):1–35

    Article  MathSciNet  MATH  Google Scholar 

  49. Hansen PC (1998) Rank-deficient and discrete ill-posed problems. SIAM, Philadelphia

    Book  Google Scholar 

  50. Hansen PC (2010) Discrete inverse problems: insight and algorithms. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  51. Hansen PC, Nagy JG, O’Leary DP (2006) Deblurring images: matrices, spectra and filtering. SIAM, Philadelphia

    MATH  Google Scholar 

  52. Hansen PC, O’Leary DP (1993) The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J Sci Comput 14(6):1487–1503

    Article  MathSciNet  MATH  Google Scholar 

  53. Hardy JW (1994) Adapt Opt Sci Am 270(6): 60–65

    Google Scholar 

  54. Hofmann B (1993) Regularization of nonlinear problems and the degree of ill-posedness. In: Anger G, Gorenflo R, Jochmann H, Moritz H, Webers W (eds) Inverse problems: principles and applications in geophysics, technology, and medicine. Akademie Verlag, Berlin

    Google Scholar 

  55. Hohn M, Tang G, Goodyear G, Baldwin PR, Huang Z, Penczek PA, Yang C, Glaeser RM, Adams PD, Ludtke SJ (2007) SPARX, a new environment for Cryo-EM image processing. J Struct Biol 157(1):47–55

    Article  Google Scholar 

  56. Jain AK (1989) Fundamentals of digital image processing. Prentice-Hall, Englewood Cliffs

    MATH  Google Scholar 

  57. Kang MG, Chaudhuri S (2003) Super-resolution image reconstruction. IEEE Signal Process Mag 20(3):19–20

    Article  Google Scholar 

  58. Kaufman L (1975) A variable projection method for solving separable nonlinear least squares problems. BIT 15(1):49–57

    Article  MathSciNet  MATH  Google Scholar 

  59. Kilmer ME, Hansen PC, Español MI (2007) A projection-based approach to general-form Tikhonov regularization. SIAM J Sci Comput 29(1):315–330

    Article  MathSciNet  MATH  Google Scholar 

  60. Kilmer ME, O’Leary DP (2001) Choosing regularization parameters in iterative methods for ill-posed problems. SIAM J Matrix Anal Appl 22(4):1204–1221

    Article  MathSciNet  MATH  Google Scholar 

  61. Landweber L (1951) An iteration formula for Fredholm integral equations of the first kind. Am J Math 73(3):615–624

    Article  MathSciNet  MATH  Google Scholar 

  62. Larsen RM (1998) Lanczos bidiagonalization with partial reorthogonalization. PhD thesis, Department of Computer Science, University of Aarhus, Denmark

    Google Scholar 

  63. Lawson CL, Hanson RJ (1995) Solving least squares problems. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  64. Löfdahl MG (2002) Multi-frame blind deconvolution with linear equality constraints. In: Bones PJ, Fiddy MA, Millane RP (eds) Image reconstruction from incomplete data II, vol 4792-21. SPIE, pp 146–155

    Google Scholar 

  65. Lohmann AW, Paris DP (1965) Space-variant image formation. J Opt Soc Am 55(8):1007–1013

    Google Scholar 

  66. Marabini R, Herman GT, Carazo JM (1998) 3D reconstruction in electron microscopy using ART with smooth spherically symmetric volume elements (blobs). Ultramicroscopy 72(1–2):53–65

    Article  Google Scholar 

  67. Matson CL, Borelli K, Jefferies S, Beckner CC Jr, Hege EK, Lloyd-Hart M (2009) Fast and optimal multiframe blind deconvolution algorithm for high-resolution groundbased imaging of space objects. Appl Opt 48(1):A75–A92

    Article  Google Scholar 

  68. McNown SR, Hunt BR (1994) Approximate shift-invariance by warping shift-variant systems. In: Hanisch RJ, White RL (eds) The restoration of HST images and spectra II. Space Telescope Science Institute, Baltimore, MD, pp 181–187

    Google Scholar 

  69. Miller K (1970) Least squares methods for ill-posed problems with a prescribed bound. SIAM J Math Anal 1(1):52–74

    Article  MathSciNet  MATH  Google Scholar 

  70. Modersitzki J (2004) Numerical methods for image registration. Oxford University Press, Oxford

    MATH  Google Scholar 

  71. Morozov VA (1966) On the solution of functional equations by the method of regularization. Sov Math Dokl 7:414–417

    MATH  Google Scholar 

  72. Nagy JG, O’Leary DP (1997) Fast iterative image restoration with a spatially varying PSF. In: Luk FT (ed) Advanced signal processing: algorithms, architectures, and implementations VII, vol 3162. SPIE, pp 388–399

    Google Scholar 

  73. Nagy JG, O’Leary DP (1998) Restoring images degraded by spatially-variant blur. SIAM J Sci Comput 19(4):1063–1082

    Article  MathSciNet  MATH  Google Scholar 

  74. Natterer F (2001) The mathematics of computerized tomography. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  75. Natterer F, Wübbeling F (2001) Mathematical methods in image reconstruction. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  76. Nguyen N, Milanfar P, Golub G (2001) Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Trans Image Process 10(9):1299–1308

    Article  MathSciNet  MATH  Google Scholar 

  77. Nocedal J, Wright S (1999) Numerical optimization. Springer, New York

    Book  MATH  Google Scholar 

  78. O’Leary DP, Simmons JA (1981) A bidiagonali- zation-regularization procedure for large scale discretizations of ill-posed problems. SIAM J Sci Stat Comput 2(4):474–489

    Article  MathSciNet  MATH  Google Scholar 

  79. Osborne MR (2007) Separable least squares, variable projection, and the Gauss-Newton algorithm. Elec Trans Numer Anal 28:1–15

    MathSciNet  MATH  Google Scholar 

  80. Paige CC, Saunders MA (1982) Algorithm 583 LSQR: Sparse linear equations and least squares problems. ACM Trans Math Softw 8(2): 195–209

    Article  MathSciNet  Google Scholar 

  81. Paige CC, Saunders MA (1982) LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Trans Math Softw 8(1): 43–71

    Article  MathSciNet  MATH  Google Scholar 

  82. Penczek PA, Radermacher M, Frank J (1992) Three-dimensional reconstruction of single particles embedded in ice. Ultramicroscopy 40(1):33–53

    Article  Google Scholar 

  83. Phillips DL (1962) A technique for the numerical solution of certain integral equations of the first kind. J Assoc Comput Mach 9(1): 84–97

    Article  MathSciNet  MATH  Google Scholar 

  84. Raghunath N, Faber TL, Suryanarayanan S, Votaw JR (2009) Motion correction of PET brain images through deconvolution: II. Practical implementation and algorithm optimization. Phys Med Biol 54(3):813–829

    Article  Google Scholar 

  85. Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Physica D 60:259–268

    Article  MATH  Google Scholar 

  86. Ruhe A, Wedin P (1980) Algorithms for separable nonlinear least squares problems. SIAM Rev 22(3):318–337

    Article  MathSciNet  MATH  Google Scholar 

  87. Saad Y (1980) On the rates of convergence of the Lanczos and the block-Lanczos methods. SIAM J Numer Anal 17(5):687–706

    Article  MathSciNet  MATH  Google Scholar 

  88. Saban SD, Silvestry M, Nemerow GR, Stewart PL (2006) Visualization of α-helices in a 6-Ångstrom resolution cryoelectron microscopy structure of adenovirus allows refinement of capsid protein assignments. J Virol 80(24): 49–59

    Article  Google Scholar 

  89. Tikhonov AN (1963) Regularization of incorrectly posed problems. Sov Math Dokl 4:1624–1627

    MATH  Google Scholar 

  90. Tikhonov AN (1963) Solution of incorrectly formulated problems and the regularization method. Sov Math Dokl 4:1035–1038

    Google Scholar 

  91. Tikhonov AN, Arsenin VY (1977) Solutions of ill-posed problems. Winston, Washington

    MATH  Google Scholar 

  92. Tikhonov AN, Leonov AS, Yagola AG (1998) Nonlinear ill-posed problems, vol 1–2. Chapman and Hall, London

    MATH  Google Scholar 

  93. Trussell HJ, Fogel S (1992) Identification and restoration of spatially variant motion blurs in sequential images. IEEE Trans Image Process 1(1):123–126

    Article  Google Scholar 

  94. Tsaig Y, Donoho DL (2006) Extensions of compressed sensing. Signal Process 86(3): 549–571

    Article  MATH  Google Scholar 

  95. Varah JM (1983) Pitfalls in the numerical solution of linear ill-posed problems. SIAM J Sci Stat Comput 4(2):164–176

    Article  MathSciNet  MATH  Google Scholar 

  96. Vogel CR (1986) Optimal choice of a truncation level for the truncated SVD solution of linear first kind integral equations when data are noisy. SIAM J Numer Anal 23(1):109–117

    Article  MathSciNet  MATH  Google Scholar 

  97. Vogel CR (1987) An overview of numerical methods for nonlinear ill-posed problems. In: Engl HW, Groetsch CW (eds) Inverse and ill-posed problems. Academic Press, Boston, pp 231–245

    Google Scholar 

  98. Vogel CR (1996) Non-convergence of the L-curve regularization parameter selection method. Inverse Prob 12(4):535–547

    Article  MATH  Google Scholar 

  99. Vogel CR (2002) Computational methods for inverse problems. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  100. Wagner FC, Macovski A, Nishimura DG (1988) A characterization of the scatter pointspread-function in terms of air gaps. IEEE Trans Med Imaging 7(4):337–344

    Article  Google Scholar 

Download references

Authors
  1. Julianne Chung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sarah Knepper
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. James G. Nagy
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Computational Science Center, University of Vienna, Nordbergstrasse 15, Vienna, Austria

    Otmar Scherzer

  2. RICAM, Austrian Academy of Sciences, Linz, Austria

    Otmar Scherzer

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this entry

Cite this entry

Chung, J., Knepper, S., Nagy, J.G. (2011). Large-Scale Inverse Problems in Imaging. In: Scherzer, O. (eds) Handbook of Mathematical Methods in Imaging. Springer, New York, NY. https://doi.org/10.1007/978-0-387-92920-0_2

Download citation

Publish with us

Policies and ethics

Search

Navigation