Skip to main content
SpringerLink
Log in

Large-Scale Inverse Problems in Imaging

  • Reference work entry
Handbook of Mathematical Methods in Imaging

  • 6208 Accesses

Abstract

Large-scale inverse problems arise in a variety of significant applications in image processing, and efficient regularization methods are needed to compute meaningful solutions. This chapter surveys three common mathematical models including a linear model, a separable nonlinear model, and a general nonlinear model. Techniques for regularization and large-scale implementations are considered, with particular focus on algorithms and computations that can exploit structure in the problem. Examples from image deconvolution, multi-frame blind deconvolution, and tomosynthesis illustrate the potential of these algorithms. Much progress has been made in the field of large-scale inverse problems, but many challenges still remain for future research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 1,200.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 549.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Andrews, H.C., Hunt, B.R.: Digital Image Restoration. Prentice-Hall, Englewood Cliffs (1977)

    Google Scholar 

  2. Bachmayr, M., Burger, M.: Iterative total variation schemes for nonlinear inverse problems. Inverse Prob. 25, 105004 (2009)

    Article  MathSciNet  Google Scholar 

  3. Bardsley, J.M.: An efficient computational method for total variation-penalized Poisson likelihood estimation. Inverse Prob. Imaging 2(2), 167–185 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  4. Bardsley, J.M.: Stopping rules for a nonnegatively constrained iterative method for illposed Poisson imaging problems. BIT 48(4), 651–664 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  5. Bardsley, J.M., Vogel, C.R.: A nonnegatively constrained convex programming method for image reconstruction. SIAM J. Sci. Comput. 25(4), 1326–1343 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  6. Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8(1), 141–148 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  7. Björck, Å.: A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations. BIT 28(3), 659–670 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  8. Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)

    Book  MATH  Google Scholar 

  9. Björck, Å., Grimme, E., van Dooren, P.: An implicit shift bidiagonalization algorithm for ill-posed systems. BIT 34(4), 510–534 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  10. Brakhage, H.: On ill-posed problems and the method of conjugate gradients. In: Engl, H.W., Groetsch, C.W. (eds.) Inverse and Ill-Posed Problems, pp. 165–175. Academic, Boston (1987)

    Chapter  Google Scholar 

  11. Calvetti, D., Reichel, L.: Tikhonov regularization of large linear problems. BIT 43(2), 263–283 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  12. Calvetti, D., Somersalo, E.: Introduction to Bayesian Scientific Computing. Springer, New York (2007)

    MATH  Google Scholar 

  13. Candès, E.J., Romberg, J.K., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)

    Article  MATH  Google Scholar 

  14. Carasso, A.S.: Direct blind deconvolution. SIAM J. Appl. Math. 61(6), 1980–2007 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  15. Chadan, K., Colton, D., Päivärinta, L., Rundell, W.: An Introduction to Inverse Scattering and Inverse Spectral Problems. SIAM, Philadelphia (1997)

    Book  MATH  Google Scholar 

  16. Chan, T.F., Shen, J.: Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastic Methods. SIAM, Philadelphia (2005)

    Book  Google Scholar 

  17. Cheney, M., Borden, B.: Fundamentals of Radar Imaging. SIAM, Philadelphia (2009)

    Book  MATH  Google Scholar 

  18. Chung, J., Haber, E., Nagy, J.: Numerical methods for coupled super-resolution. Inverse Prob. 22(4), 1261–1272 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  19. Chung, J., Nagy, J.: An efficient iterative approach for large-scale separable nonlinear inverse problems. SIAM J. Sci. Comput. 31(6), 4654–4674 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  20. Chung, J., Nagy, J., Sechopoulos, I.: Numerical algorithms for polyenergetic digital breast tomosynthesis reconstruction. SIAM J. Imaging Sci. 3(1), 133–152 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  21. Chung, J., Nagy, J.G., O’Leary, D.P.: A weighted GCV method for Lanczos hybrid regularization. Elec. Trans. Numer. Anal. 28, 149–167 (2008)

    MathSciNet  Google Scholar 

  22. Chung, J., Sternberg, P., Yang, C.: High performance 3-d image reconstruction for molecular structure determination. Int. J. High Perform. Comput. Appl. 24(2), 117–135 (2010)

    Article  Google Scholar 

  23. De Man, B., Nuyts, J., Dupont, P., Marchal, G., Suetens, P.: An iterative maximumlikelihood polychromatic algorithm for CT. IEEE Trans. Med. Imaging 20(10), 999–1008 (2001)

    Article  Google Scholar 

  24. Diaspro, A., Corosu, M., Ramoino, P., Robello, M.: Two-photon excitation imaging based on a compact scanning head. IEEE Eng. Med. Biol. 18(5), 18–30 (1999)

    Article  Google Scholar 

  25. Dobbins, J.T., III, Godfrey, D.J.: Digital X-ray tomosynthesis: current state of the art and clinical potential. Phys. Med. Biol. 48(19), R65–R106 (2003)

    Article  Google Scholar 

  26. Easley, G.R., Healy, D.M., Berenstein, C.A.: Image deconvolution using a general ridgelet and curvelet domain. SIAM J. Imaging Sci. 2(1), 253–283 (2009)

    Article  MATH  Google Scholar 

  27. Elad, M., Feuer, A.: Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans. Image Process. 6(12), 1646–1658 (1997)

    Article  Google Scholar 

  28. Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer, Dordrecht (2000)

    Google Scholar 

  29. Engl, H.W., Kügler, P.: Nonlinear inverse problems: theoretical aspects and some industrial applications. In: Capasso, V., Périaux, J. (eds.) Multidisciplinary Methods for Analysis Optimization and Control of Complex Systems, pp. 3–48. Springer, Berlin (2005)

    Chapter  Google Scholar 

  30. Engl, H.W., Kunisch, K., Neubauer, A.: Convergence rates for Tikhonov regularisation of nonlinear ill-posed problems. Inverse Prob. 5(4), 523–540 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  31. Engl, H.W., Louis, A.K., Rundell, W. (eds.): Inverse Problems in Geophysical Applications. SIAM, Philadelphia (1996)

    Google Scholar 

  32. Eriksson, J., Wedin, P.: Truncated Gauss-Newton algorithms for ill-conditioned nonlinear least squares problems. Optim. Meth. Softw. 19(6), 721–737 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  33. Faber, T.L., Raghunath, N., Tudorascu, D., Votaw, J.R.: Motion correction of PET brain images through deconvolution: I. Theoretical development and analysis in software simulations. Phys. Med. Biol. 54(3), 797–811 (2009)

    Google Scholar 

  34. Figueiredo, M.A.T, Nowak, R.D., Wright, S.J.: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 1(4), 586–597 (2007)

    Article  Google Scholar 

  35. Frank, J.: Three-Dimensional Electron Microscopy of Macromolecular Assemblies. Oxford University Press, New York (2006)

    Book  Google Scholar 

  36. Golub, G.H., Heath, M., Wahba, G.: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21(2), 215–223 (1979)

    Article  MATH  MathSciNet  Google Scholar 

  37. Golub, G.H., Luk, F.T., Overton, M.L.: A block Lanczos method for computing the singular values and corresponding singular vectors of a matrix. ACM Trans. Math. Softw. 7(2), 149–169 (1981)

    Article  MATH  MathSciNet  Google Scholar 

  38. Golub, G.H., Pereyra, V.: The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate. SIAM J. Numer. Anal. 10(2), 413–432 (1973)

    Article  MATH  MathSciNet  Google Scholar 

  39. Golub, G.H., Pereyra, V.: Separable nonlinear least squares: the variable projection method and its applications. Inverse Prob. 19, R1–R26 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  40. Haber, E., Ascher, U.M., Oldenburg, D.: On optimization techniques for solving nonlinear inverse problems. Inverse Prob. 16(5), 1263–1280 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  41. Haber, E., Oldenburg, D.: A GCV based method for nonlinear ill-posed problems. Comput. Geosci. 4(1), 41–63 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  42. Hammerstein, G.R., Miller, D.W., White, D.R., Masterson, M.E., Woodard, H.Q., Laughlin, J.S.: Absorbed radiation dose in mammography. Radiology 130(2), 485–491 (1979)

    Article  Google Scholar 

  43. Hanke, M.: Conjugate gradient type methods for ill-posed problems. Pitman research notes in mathematics, Longman Scientific & Technical, Harlow (1995)

    MATH  Google Scholar 

  44. Hanke, M.: Limitations of the L-curve method in ill-posed problems. BIT 36(2), 287–301 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  45. Hanke, M.: On Lanczos based methods for the regularization of discrete ill-posed problems. BIT 41(5), 1008–1018 (2001)

    Article  MathSciNet  Google Scholar 

  46. Hansen, P.C.: Analysis of discrete ill-posed problems by means of the L-curve. SIAM Rev. 34(4), 561–580 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  47. Hansen, P.C.: Numerical tools for analysis and solution of Fredholm integral equations of the first kind. Inverse Prob. 8(6), 849–872 (1992)

    Article  MATH  Google Scholar 

  48. Hansen, P.C.: Regularization tools: a MATLAB package for analysis and solution of discrete ill-posed problems. Numer. Algorithms 6(1), 1–35 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  49. Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1998)

    Book  Google Scholar 

  50. Hansen, P.C.: Discrete Inverse Problems: Insight and Algorithms. SIAM, Philadelphia (2010)

    Book  Google Scholar 

  51. Hansen, P.C., Nagy, J.G., O’Leary, D.P.: Deblurring Images: Matrices, Spectra and Filtering. SIAM, Philadelphia (2006)

    Book  Google Scholar 

  52. Hansen, P.C., O’Leary, D.P.: The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J Sci Comput 14(6):1487–1503 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  53. Hardy, J.W.: Adaptive optics. Sci. Am. 270(6), 60–65 (1994)

    Article  Google Scholar 

  54. Hofmann, B.: Regularization of nonlinear problems and the degree of ill-posedness. In: Anger, G., Gorenflo, R., Jochmann, H., Moritz, H., Webers, W. (eds.) Inverse Problems: Principles and Applications in Geophysics, Technology, and Medicine. Akademie Verlag, Berlin (1993)

    Google Scholar 

  55. Hohn, M., Tang, G., Goodyear, G., Baldwin, P.R., Huang, Z., Penczek, P.A., Yang, C., Glaeser, R.M., Adams, P.D., Ludtke, S.J.: SPARX, a new environment for Cryo-EM image processing. J. Struct. Biol. 157(1), 47–55 (2007)

    Article  Google Scholar 

  56. Jain, A.K.: Fundamentals of Digital Image Processing. Prentice-Hall, Englewood Cliffs (1989)

    MATH  Google Scholar 

  57. Kang, M.G., Chaudhuri, S.: Super-resolution image reconstruction. IEEE Signal Process. Mag. 20(3), 19–20 (2003)

    Article  Google Scholar 

  58. Kaufman, L.: A variable projection method for solving separable nonlinear least squares problems. BIT 15(1), 49–57 (1975)

    Article  MATH  MathSciNet  Google Scholar 

  59. Kilmer, M.E., Hansen, P.C., Español, M.I.: A projection-based approach to general-form Tikhonov regularization. SIAM J. Sci. Comput. 29(1), 315–330 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  60. Kilmer, M.E., O’Leary, D.P.: Choosing regularization parameters in iterative methods for ill-posed problems. SIAM J. Matrix. Anal. Appl. 22(4), 1204–1221 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  61. Landweber, L.: An iteration formula for Fredholm integral equations of the first kind. Am. J. Math. 73(3), 615–624 (1951)

    Article  MATH  MathSciNet  Google Scholar 

  62. Larsen, R.M.: Lanczos bidiagonalization with partial reorthogonalization. PhD thesis, Department of Computer Science, University of Aarhus, Denmark (1998)

    Google Scholar 

  63. Lawson, C.L., Hanson, R.J.: Solving Least Squares Problems. SIAM, Philadelphia (1995)

    Book  MATH  Google Scholar 

  64. Löfdahl, M.G.: Multi-frame blind deconvolution with linear equality constraints. In: Bones, P.J., Fiddy, M.A., Millane, R.P. (eds.) Image Reconstruction from Incomplete Data II, Seattle, vol. 4792–21, pp. 146–155. SPIE (2002)

    Chapter  Google Scholar 

  65. Lohmann, A.W., Paris, D.P.: Space-variant image formation. J. Opt. Soc. Am. 55(8), 1007–1013 (1965)

    Google Scholar 

  66. Marabini, R., Herman, G.T., Carazo, J.M.: 3D reconstruction in electron microscopy using ART with smooth spherically symmetric volume elements (blobs). Ultramicroscopy 72(1–2), 53–65 (1998)

    Article  Google Scholar 

  67. Matson, C.L., Borelli, K., Jefferies, S., Beckner, C.C., Jr., Hege, E.K., Lloyd-Hart, M.: Fast and optimal multiframe blind deconvolution algorithm for high-resolution groundbased imaging of space objects. Appl. Opt. 48(1), A75–A92 (2009)

    Article  Google Scholar 

  68. McNown, S.R., Hunt, B.R.: Approximate shift-invariance by warping shift-variant systems. In: Hanisch, R.J., White, R.L. (eds.) The Restoration of HST Images and Spectra II, pp. 181–187. Space Telescope Science Institute, Baltimore (1994)

    Google Scholar 

  69. Miller, K.: Least squares methods for ill-posed problems with a prescribed bound. SIAM J. Math. Anal. 1(1), 52–74 (1970)

    Article  MATH  MathSciNet  Google Scholar 

  70. Modersitzki, J.: Numerical Methods for Image Registration. Oxford University Press, Oxford (2004)

    MATH  Google Scholar 

  71. Morozov, V.A.: On the solution of functional equations by the method of regularization. Sov. Math. Dokl. 7, 414–417 (1966)

    MATH  Google Scholar 

  72. Nagy, J.G., O’Leary, D.P.: Fast iterative image restoration with a spatially varying PSF. In: Luk, F.T. (ed.) Advanced Signal Processing: Algorithms, Architectures, and Implementations VII, San Diego, vol. 3162, pp. 388–399. SPIE (1997)

    Chapter  Google Scholar 

  73. Nagy, J.G., O’Leary, D.P.: Restoring images degraded by spatially-variant blur. SIAM J. Sci. Comput. 19(4), 1063–1082 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  74. Natterer, F.: The Mathematics of Computerized Tomography. SIAM, Philadelphia (2001)

    Book  MATH  Google Scholar 

  75. Natterer, F., Wübbeling, F.: Mathematical Methods in Image Reconstruction. SIAM, Philadelphia (2001)

    Book  MATH  Google Scholar 

  76. Nguyen, N., Milanfar, P., Golub, G.: Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Trans. Image Process. 10(9), 1299–1308 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  77. Nocedal, J., Wright, S.: Numerical Optimization. Springer, New York (1999)

    Book  MATH  Google Scholar 

  78. O’Leary, D.P., Simmons, J.A.: A bidiagonali-zation-regularization procedure for large scale discretizations of ill-posed problems. SIAM J. Sci. Stat. Comput. 2(4), 474–489 (1981)

    Article  MATH  MathSciNet  Google Scholar 

  79. Osborne, M.R.: Separable least squares, variable projection, and the Gauss-Newton algorithm. Elec. Trans. Numer. Anal. 28, 1–15 (2007)

    MATH  MathSciNet  Google Scholar 

  80. Paige, C.C., Saunders, M.A.: Algorithm 583 LSQR: sparse linear equations and least squares problems. ACM Trans. Math. Softw. 8(2), 195–209 (1982)

    Article  MathSciNet  Google Scholar 

  81. Paige, C.C., Saunders, M.A.: LSQR: an algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Softw. 8(1), 43–71 (1982)

    Article  MATH  MathSciNet  Google Scholar 

  82. Penczek, P.A., Radermacher, M., Frank, J.: Three-dimensional reconstruction of single particles embedded in ice. Ultramicroscopy 40(1), 33–53 (1992)

    Article  Google Scholar 

  83. Phillips, D.L.: A technique for the numerical solution of certain integral equations of the first kind. J. Assoc. Comput. Mach. 9(1), 84–97 (1962)

    Article  MATH  MathSciNet  Google Scholar 

  84. Raghunath, N., Faber, T.L., Suryanarayanan, S., Votaw, J.R.: Motion correction of PET brain images through deconvolution: II. Practical implementation and algorithm optimization. Phys. Med. Biol. 54(3), 813–829 (2009)

    Google Scholar 

  85. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)

    Article  MATH  Google Scholar 

  86. Ruhe, A., Wedin, P.: Algorithms for separable nonlinear least squares problems. SIAM Rev. 22(3), 318–337 (1980)

    Article  MATH  MathSciNet  Google Scholar 

  87. Saad, Y.: On the rates of convergence of the Lanczos and the block-Lanczos methods. SIAM J. Numer. Anal. 17(5), 687–706 (1980)

    Article  MATH  MathSciNet  Google Scholar 

  88. Saban, S.D., Silvestry, M., Nemerow, G.R., Stewart, P.L.: Visualization of α-helices in a 6-Ångstrom resolution cryoelectron microscopy structure of adenovirus allows refinement of capsid protein assignments. J. Virol. 80(24), 49–59 (2006)

    Article  Google Scholar 

  89. Tikhonov, A.N.: Regularization of incorrectly posed problems. Sov. Math. Dokl. 4, 1624–1627 (1963)

    MATH  Google Scholar 

  90. Tikhonov, A.N.: Solution of incorrectly formulated problems and the regularization method. Sov. Math. Dokl. 4, 1035–1038 (1963)

    Google Scholar 

  91. Tikhonov, A.N., Arsenin, V.Y.: Solutions of Ill-Posed Problems. Winston, Washington (1977)

    MATH  Google Scholar 

  92. Tikhonov, A.N., Leonov, A.S., Yagola, A.G.: Nonlinear Ill-Posed Problems, vol. 1–2. Chapman and Hall, London (1998)

    Book  MATH  Google Scholar 

  93. Trussell, H.J., Fogel, S.: Identification and restoration of spatially variant motion blurs in sequential images. IEEE Trans. Image Process. 1(1), 123–126 (1992)

    Article  Google Scholar 

  94. Tsaig, Y., Donoho, D.L.: Extensions of compressed sensing. Signal Process. 86(3), 549–571 (2006)

    Article  MATH  Google Scholar 

  95. Varah, J.M.: Pitfalls in the numerical solution of linear ill-posed problems. SIAM J. Sci. Stat. Comput. 4(2), 164–176 (1983)

    Article  MATH  MathSciNet  Google Scholar 

  96. Vogel, C.R.: Optimal choice of a truncation level for the truncated SVD solution of linear first kind integral equations when data are noisy. SIAM J. Numer. Anal. 23(1), 109–117 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  97. Vogel, C.R.: An overview of numerical methods for nonlinear ill-posed problems. In: Engl, H.W., Groetsch, C.W. (eds.) Inverse and Ill-Posed Problems, pp. 231–245. Academic, Boston (1987)

    Chapter  Google Scholar 

  98. Vogel, C.R.: Non-convergence of the L-curve regularization parameter selection method. Inverse Prob. 12(4), 535–547 (1996)

    Article  MATH  Google Scholar 

  99. Vogel, C.R.: Computational Methods for Inverse Problems. SIAM, Philadelphia (2002)

    Book  MATH  Google Scholar 

  100. Wagner, F.C., Macovski, A., Nishimura, D.G.: A characterization of the scatter pointspread-function in terms of air gaps. IEEE Trans. Med. Imaging 7(4), 337–344 (1988)

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank Eldad Haber, University of British Columbia, and Per Christian Hansen, Technical University of Denmark, for carefully reading the first draft of this chapter. Their comments and suggestions helped to greatly improve our presentation. The research of J. Chung is supported by the US National Science Foundation (NSF) under grant DMS-0902322. The research of J. Nagy is supported by the US National Science Foundation (NSF) under grant DMS-0811031, and by the US Air Force Office of Scientific Research (AFOSR) under grant FA9550-09-1-0487.

Author information

Authors and Affiliations

  1. Virginia Tech, Blacksburg, VA, USA

    Julianne Chung

  2. Emory University, Atlanta, GA, USA

    Sarah Knepper & James G. Nagy

Authors
  1. Julianne Chung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sarah Knepper
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. James G. Nagy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Julianne Chung .

Editor information

Editors and Affiliations

  1. Computational Science Center, University of Vienna, Vienna, Austria

    Otmar Scherzer

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science+Business Media New York

About this entry

Cite this entry

Chung, J., Knepper, S., Nagy, J.G. (2015). Large-Scale Inverse Problems in Imaging. In: Scherzer, O. (eds) Handbook of Mathematical Methods in Imaging. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-0790-8_2

Download citation

Publish with us

Policies and ethics

Search

Navigation