Large-Scale Inverse Problems in Imaging

  • Julianne Chung
  • Sarah Knepper
  • James G. Nagy
Living reference work entry

Latest version View entry history


Large-scale inverse problems arise in a variety of significant applications in image processing, and efficient regularization methods are needed to compute meaningful solutions. This chapter surveys three common mathematical models including a linear model, a separable nonlinear model, and a general nonlinear model. Techniques for regularization and large-scale implementations are considered, with particular focus on algorithms and computations that can exploit structure in the problem. Examples from image deconvolution, multi-frame blind deconvolution, and tomosynthesis illustrate the potential of these algorithms. Much progress has been made in the field of large-scale inverse problems, but many challenges still remain for future research.


Inverse Problem Singular Value Decomposition Regularization Parameter Point Spread Function Tikhonov Regularization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We would like to thank Eldad Haber, University of British Columbia, and Per Christian Hansen, Technical University of Denmark, for carefully reading the first draft of this chapter. Their comments and suggestions helped to greatly improve our presentation. The research of J. Chung is supported by the US National Science Foundation (NSF) under grant DMS-0902322. The research of J. Nagy is supported by the US National Science Foundation (NSF) under grant DMS-0811031, and by the US Air Force Office of Scientific Research (AFOSR) under grant FA9550-09-1-0487.


  1. 1.
    Andrews, H.C., Hunt, B.R.: Digital Image Restoration. Prentice-Hall, Englewood Cliffs (1977)Google Scholar
  2. 2.
    Bachmayr, M., Burger, M.: Iterative total variation schemes for nonlinear inverse problems. Inverse Prob. 25, 105004 (2009)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bardsley, J.M.: An efficient computational method for total variation-penalized Poisson likelihood estimation. Inverse Prob. Imaging 2(2), 167–185 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bardsley, J.M.: Stopping rules for a nonnegatively constrained iterative method for illposed Poisson imaging problems. BIT 48(4), 651–664 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Bardsley, J.M., Vogel, C.R.: A nonnegatively constrained convex programming method for image reconstruction. SIAM J. Sci. Comput. 25(4), 1326–1343 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8(1), 141–148 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Björck, Å.: A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations. BIT 28(3), 659–670 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)CrossRefzbMATHGoogle Scholar
  9. 9.
    Björck, Å., Grimme, E., van Dooren, P.: An implicit shift bidiagonalization algorithm for ill-posed systems. BIT 34(4), 510–534 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Brakhage, H.: On ill-posed problems and the method of conjugate gradients. In: Engl, H.W., Groetsch, C.W. (eds.) Inverse and Ill-Posed Problems, pp. 165–175. Academic, Boston (1987)Google Scholar
  11. 11.
    Calvetti, D., Reichel, L.: Tikhonov regularization of large linear problems. BIT 43(2), 263–283 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Calvetti, D., Somersalo, E.: Introduction to Bayesian Scientific Computing. Springer, New York (2007)zbMATHGoogle Scholar
  13. 13.
    Candès, E.J., Romberg, J.K., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)CrossRefzbMATHGoogle Scholar
  14. 14.
    Carasso, A.S.: Direct blind deconvolution. SIAM J. Appl. Math. 61(6), 1980–2007 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Chadan, K., Colton, D., Päivärinta, L., Rundell, W.: An Introduction to Inverse Scattering and Inverse Spectral Problems. SIAM, Philadelphia (1997)CrossRefzbMATHGoogle Scholar
  16. 16.
    Chan, T.F., Shen, J.: Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastic Methods. SIAM, Philadelphia (2005)CrossRefGoogle Scholar
  17. 17.
    Cheney, M., Borden, B.: Fundamentals of Radar Imaging. SIAM, Philadelphia (2009)CrossRefzbMATHGoogle Scholar
  18. 18.
    Chung, J., Haber, E., Nagy, J.: Numerical methods for coupled super-resolution. Inverse Prob. 22(4), 1261–1272 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Chung, J., Nagy, J.: An efficient iterative approach for large-scale separable nonlinear inverse problems. SIAM J. Sci. Comput. 31(6), 4654–4674 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Chung, J., Nagy, J., Sechopoulos, I.: Numerical algorithms for polyenergetic digital breast tomosynthesis reconstruction. SIAM J. Imaging Sci. 3(1), 133–152 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Chung, J., Nagy, J.G., O’Leary, D.P.: A weighted GCV method for Lanczos hybrid regularization. Elec. Trans. Numer. Anal. 28, 149–167 (2008)MathSciNetGoogle Scholar
  22. 22.
    Chung, J., Sternberg, P., Yang, C.: High performance 3-d image reconstruction for molecular structure determination. Int. J. High Perform. Comput. Appl. 24(2), 117–135 (2010)CrossRefGoogle Scholar
  23. 23.
    De Man, B., Nuyts, J., Dupont, P., Marchal, G., Suetens, P.: An iterative maximumlikelihood polychromatic algorithm for CT. IEEE Trans. Med. Imaging 20(10), 999–1008 (2001)CrossRefGoogle Scholar
  24. 24.
    Diaspro, A., Corosu, M., Ramoino, P., Robello, M.: Two-photon excitation imaging based on a compact scanning head. IEEE Eng. Med. Biol. 18(5), 18–30 (1999)CrossRefGoogle Scholar
  25. 25.
    Dobbins, J.T., III, Godfrey, D.J.: Digital X-ray tomosynthesis: current state of the art and clinical potential. Phys. Med. Biol. 48(19), R65–R106 (2003)CrossRefGoogle Scholar
  26. 26.
    Easley, G.R., Healy, D.M., Berenstein, C.A.: Image deconvolution using a general ridgelet and curvelet domain. SIAM J. Imaging Sci. 2(1), 253–283 (2009)CrossRefzbMATHGoogle Scholar
  27. 27.
    Elad, M., Feuer, A.: Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans. Image Process. 6(12), 1646–1658 (1997)CrossRefGoogle Scholar
  28. 28.
    Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer, Dordrecht (2000)Google Scholar
  29. 29.
    Engl, H.W., Kügler, P.: Nonlinear inverse problems: theoretical aspects and some industrial applications. In: Capasso, V., Périaux, J. (eds.) Multidisciplinary Methods for Analysis Optimization and Control of Complex Systems, pp. 3–48. Springer, Berlin (2005)CrossRefGoogle Scholar
  30. 30.
    Engl, H.W., Kunisch, K., Neubauer, A.: Convergence rates for Tikhonov regularisation of nonlinear ill-posed problems. Inverse Prob. 5(4), 523–540 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Engl, H.W., Louis, A.K., Rundell, W. (eds.): Inverse Problems in Geophysical Applications. SIAM, Philadelphia (1996)Google Scholar
  32. 32.
    Eriksson, J., Wedin, P.: Truncated Gauss-Newton algorithms for ill-conditioned nonlinear least squares problems. Optim. Meth. Softw. 19(6), 721–737 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Faber, T.L., Raghunath, N., Tudorascu, D., Votaw, J.R.: Motion correction of PET brain images through deconvolution: I. Theoretical development and analysis in software simulations. Phys. Med. Biol. 54(3), 797–811 (2009)Google Scholar
  34. 34.
    Figueiredo, M.A.T, Nowak, R.D., Wright, S.J.: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 1(4), 586–597 (2007)CrossRefGoogle Scholar
  35. 35.
    Frank, J.: Three-Dimensional Electron Microscopy of Macromolecular Assemblies. Oxford University Press, New York (2006)CrossRefGoogle Scholar
  36. 36.
    Golub, G.H., Heath, M., Wahba, G.: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21(2), 215–223 (1979)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Golub, G.H., Luk, F.T., Overton, M.L.: A block Lanczos method for computing the singular values and corresponding singular vectors of a matrix. ACM Trans. Math. Softw. 7(2), 149–169 (1981)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Golub, G.H., Pereyra, V.: The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate. SIAM J. Numer. Anal. 10(2), 413–432 (1973)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Golub, G.H., Pereyra, V.: Separable nonlinear least squares: the variable projection method and its applications. Inverse Prob. 19, R1–R26 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Haber, E., Ascher, U.M., Oldenburg, D.: On optimization techniques for solving nonlinear inverse problems. Inverse Prob. 16(5), 1263–1280 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  41. 41.
    Haber, E., Oldenburg, D.: A GCV based method for nonlinear ill-posed problems. Comput. Geosci. 4(1), 41–63 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Hammerstein, G.R., Miller, D.W., White, D.R., Masterson, M.E., Woodard, H.Q., Laughlin, J.S.: Absorbed radiation dose in mammography. Radiology 130(2), 485–491 (1979)CrossRefGoogle Scholar
  43. 43.
    Hanke, M.: Conjugate gradient type methods for ill-posed problems. Pitman research notes in mathematics, Longman Scientific & Technical, Harlow (1995)zbMATHGoogle Scholar
  44. 44.
    Hanke, M.: Limitations of the L-curve method in ill-posed problems. BIT 36(2), 287–301 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Hanke, M.: On Lanczos based methods for the regularization of discrete ill-posed problems. BIT 41(5), 1008–1018 (2001)MathSciNetCrossRefGoogle Scholar
  46. 46.
    Hansen, P.C.: Analysis of discrete ill-posed problems by means of the L-curve. SIAM Rev. 34(4), 561–580 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Hansen, P.C.: Numerical tools for analysis and solution of Fredholm integral equations of the first kind. Inverse Prob. 8(6), 849–872 (1992)CrossRefzbMATHGoogle Scholar
  48. 48.
    Hansen, P.C.: Regularization tools: a MATLAB package for analysis and solution of discrete ill-posed problems. Numer. Algorithms 6(1), 1–35 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  49. 49.
    Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1998)CrossRefGoogle Scholar
  50. 50.
    Hansen, P.C.: Discrete Inverse Problems: Insight and Algorithms. SIAM, Philadelphia (2010)CrossRefGoogle Scholar
  51. 51.
    Hansen, P.C., Nagy, J.G., O’Leary, D.P.: Deblurring Images: Matrices, Spectra and Filtering. SIAM, Philadelphia (2006)CrossRefGoogle Scholar
  52. 52.
    Hansen, P.C., O’Leary, D.P.: The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J Sci Comput 14(6):1487–1503 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  53. 53.
    Hardy, J.W.: Adaptive optics. Sci. Am. 270(6), 60–65 (1994)CrossRefGoogle Scholar
  54. 54.
    Hofmann, B.: Regularization of nonlinear problems and the degree of ill-posedness. In: Anger, G., Gorenflo, R., Jochmann, H., Moritz, H., Webers, W. (eds.) Inverse Problems: Principles and Applications in Geophysics, Technology, and Medicine. Akademie Verlag, Berlin (1993)Google Scholar
  55. 55.
    Hohn, M., Tang, G., Goodyear, G., Baldwin, P.R., Huang, Z., Penczek, P.A., Yang, C., Glaeser, R.M., Adams, P.D., Ludtke, S.J.: SPARX, a new environment for Cryo-EM image processing. J. Struct. Biol. 157(1), 47–55 (2007)CrossRefGoogle Scholar
  56. 56.
    Jain, A.K.: Fundamentals of Digital Image Processing. Prentice-Hall, Englewood Cliffs (1989)zbMATHGoogle Scholar
  57. 57.
    Kang, M.G., Chaudhuri, S.: Super-resolution image reconstruction. IEEE Signal Process. Mag. 20(3), 19–20 (2003)CrossRefGoogle Scholar
  58. 58.
    Kaufman, L.: A variable projection method for solving separable nonlinear least squares problems. BIT 15(1), 49–57 (1975)MathSciNetCrossRefzbMATHGoogle Scholar
  59. 59.
    Kilmer, M.E., Hansen, P.C., Español, M.I.: A projection-based approach to general-form Tikhonov regularization. SIAM J. Sci. Comput. 29(1), 315–330 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  60. 60.
    Kilmer, M.E., O’Leary, D.P.: Choosing regularization parameters in iterative methods for ill-posed problems. SIAM J. Matrix. Anal. Appl. 22(4), 1204–1221 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  61. 61.
    Landweber, L.: An iteration formula for Fredholm integral equations of the first kind. Am. J. Math. 73(3), 615–624 (1951)MathSciNetCrossRefzbMATHGoogle Scholar
  62. 62.
    Larsen, R.M.: Lanczos bidiagonalization with partial reorthogonalization. PhD thesis, Department of Computer Science, University of Aarhus, Denmark (1998)Google Scholar
  63. 63.
    Lawson, C.L., Hanson, R.J.: Solving Least Squares Problems. SIAM, Philadelphia (1995)CrossRefzbMATHGoogle Scholar
  64. 64.
    Löfdahl, M.G.: Multi-frame blind deconvolution with linear equality constraints. In: Bones, P.J., Fiddy, M.A., Millane, R.P. (eds.) Image Reconstruction from Incomplete Data II, Seattle, vol. 4792–21, pp. 146–155. SPIE (2002)Google Scholar
  65. 65.
    Lohmann, A.W., Paris, D.P.: Space-variant image formation. J. Opt. Soc. Am. 55(8), 1007–1013 (1965)Google Scholar
  66. 66.
    Marabini, R., Herman, G.T., Carazo, J.M.: 3D reconstruction in electron microscopy using ART with smooth spherically symmetric volume elements (blobs). Ultramicroscopy 72(1–2), 53–65 (1998)CrossRefGoogle Scholar
  67. 67.
    Matson, C.L., Borelli, K., Jefferies, S., Beckner, C.C., Jr., Hege, E.K., Lloyd-Hart, M.: Fast and optimal multiframe blind deconvolution algorithm for high-resolution groundbased imaging of space objects. Appl. Opt. 48(1), A75–A92 (2009)CrossRefGoogle Scholar
  68. 68.
    McNown, S.R., Hunt, B.R.: Approximate shift-invariance by warping shift-variant systems. In: Hanisch, R.J., White, R.L. (eds.) The Restoration of HST Images and Spectra II, pp. 181–187. Space Telescope Science Institute, Baltimore (1994)Google Scholar
  69. 69.
    Miller, K.: Least squares methods for ill-posed problems with a prescribed bound. SIAM J. Math. Anal. 1(1), 52–74 (1970)MathSciNetCrossRefzbMATHGoogle Scholar
  70. 70.
    Modersitzki, J.: Numerical Methods for Image Registration. Oxford University Press, Oxford (2004)zbMATHGoogle Scholar
  71. 71.
    Morozov, V.A.: On the solution of functional equations by the method of regularization. Sov. Math. Dokl. 7, 414–417 (1966)zbMATHGoogle Scholar
  72. 72.
    Nagy, J.G., O’Leary, D.P.: Fast iterative image restoration with a spatially varying PSF. In: Luk, F.T. (ed.) Advanced Signal Processing: Algorithms, Architectures, and Implementations VII, San Diego, vol. 3162, pp. 388–399. SPIE (1997)Google Scholar
  73. 73.
    Nagy, J.G., O’Leary, D.P.: Restoring images degraded by spatially-variant blur. SIAM J. Sci. Comput. 19(4), 1063–1082 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  74. 74.
    Natterer, F.: The Mathematics of Computerized Tomography. SIAM, Philadelphia (2001)CrossRefzbMATHGoogle Scholar
  75. 75.
    Natterer, F., Wübbeling, F.: Mathematical Methods in Image Reconstruction. SIAM, Philadelphia (2001)CrossRefzbMATHGoogle Scholar
  76. 76.
    Nguyen, N., Milanfar, P., Golub, G.: Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Trans. Image Process. 10(9), 1299–1308 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  77. 77.
    Nocedal, J., Wright, S.: Numerical Optimization. Springer, New York (1999)CrossRefzbMATHGoogle Scholar
  78. 78.
    O’Leary, D.P., Simmons, J.A.: A bidiagonali-zation-regularization procedure for large scale discretizations of ill-posed problems. SIAM J. Sci. Stat. Comput. 2(4), 474–489 (1981)MathSciNetCrossRefzbMATHGoogle Scholar
  79. 79.
    Osborne, M.R.: Separable least squares, variable projection, and the Gauss-Newton algorithm. Elec. Trans. Numer. Anal. 28, 1–15 (2007)MathSciNetzbMATHGoogle Scholar
  80. 80.
    Paige, C.C., Saunders, M.A.: Algorithm 583 LSQR: sparse linear equations and least squares problems. ACM Trans. Math. Softw. 8(2), 195–209 (1982)MathSciNetCrossRefGoogle Scholar
  81. 81.
    Paige, C.C., Saunders, M.A.: LSQR: an algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Softw. 8(1), 43–71 (1982)MathSciNetCrossRefzbMATHGoogle Scholar
  82. 82.
    Penczek, P.A., Radermacher, M., Frank, J.: Three-dimensional reconstruction of single particles embedded in ice. Ultramicroscopy 40(1), 33–53 (1992)CrossRefGoogle Scholar
  83. 83.
    Phillips, D.L.: A technique for the numerical solution of certain integral equations of the first kind. J. Assoc. Comput. Mach. 9(1), 84–97 (1962)MathSciNetCrossRefzbMATHGoogle Scholar
  84. 84.
    Raghunath, N., Faber, T.L., Suryanarayanan, S., Votaw, J.R.: Motion correction of PET brain images through deconvolution: II. Practical implementation and algorithm optimization. Phys. Med. Biol. 54(3), 813–829 (2009)Google Scholar
  85. 85.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)CrossRefzbMATHGoogle Scholar
  86. 86.
    Ruhe, A., Wedin, P.: Algorithms for separable nonlinear least squares problems. SIAM Rev. 22(3), 318–337 (1980)MathSciNetCrossRefzbMATHGoogle Scholar
  87. 87.
    Saad, Y.: On the rates of convergence of the Lanczos and the block-Lanczos methods. SIAM J. Numer. Anal. 17(5), 687–706 (1980)MathSciNetCrossRefzbMATHGoogle Scholar
  88. 88.
    Saban, S.D., Silvestry, M., Nemerow, G.R., Stewart, P.L.: Visualization of α-helices in a 6-Ångstrom resolution cryoelectron microscopy structure of adenovirus allows refinement of capsid protein assignments. J. Virol. 80(24), 49–59 (2006)CrossRefGoogle Scholar
  89. 89.
    Tikhonov, A.N.: Regularization of incorrectly posed problems. Sov. Math. Dokl. 4, 1624–1627 (1963)zbMATHGoogle Scholar
  90. 90.
    Tikhonov, A.N.: Solution of incorrectly formulated problems and the regularization method. Sov. Math. Dokl. 4, 1035–1038 (1963)Google Scholar
  91. 91.
    Tikhonov, A.N., Arsenin, V.Y.: Solutions of Ill-Posed Problems. Winston, Washington (1977)zbMATHGoogle Scholar
  92. 92.
    Tikhonov, A.N., Leonov, A.S., Yagola, A.G.: Nonlinear Ill-Posed Problems, vol. 1–2. Chapman and Hall, London (1998)zbMATHGoogle Scholar
  93. 93.
    Trussell, H.J., Fogel, S.: Identification and restoration of spatially variant motion blurs in sequential images. IEEE Trans. Image Process. 1(1), 123–126 (1992)CrossRefGoogle Scholar
  94. 94.
    Tsaig, Y., Donoho, D.L.: Extensions of compressed sensing. Signal Process. 86(3), 549–571 (2006)CrossRefzbMATHGoogle Scholar
  95. 95.
    Varah, J.M.: Pitfalls in the numerical solution of linear ill-posed problems. SIAM J. Sci. Stat. Comput. 4(2), 164–176 (1983)MathSciNetCrossRefzbMATHGoogle Scholar
  96. 96.
    Vogel, C.R.: Optimal choice of a truncation level for the truncated SVD solution of linear first kind integral equations when data are noisy. SIAM J. Numer. Anal. 23(1), 109–117 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  97. 97.
    Vogel, C.R.: An overview of numerical methods for nonlinear ill-posed problems. In: Engl, H.W., Groetsch, C.W. (eds.) Inverse and Ill-Posed Problems, pp. 231–245. Academic, Boston (1987)Google Scholar
  98. 98.
    Vogel, C.R.: Non-convergence of the L-curve regularization parameter selection method. Inverse Prob. 12(4), 535–547 (1996)CrossRefzbMATHGoogle Scholar
  99. 99.
    Vogel, C.R.: Computational Methods for Inverse Problems. SIAM, Philadelphia (2002)CrossRefzbMATHGoogle Scholar
  100. 100.
    Wagner, F.C., Macovski, A., Nishimura, D.G.: A characterization of the scatter pointspread-function in terms of air gaps. IEEE Trans. Med. Imaging 7(4), 337–344 (1988)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Virginia TechBlacksburg, VAUSA
  2. 2.Emory UniversityAtlanta, GAUSA
  3. 3.Emory UniversityAtlanta, GAUSA

Personalised recommendations