Numerical Algorithms

, Volume 29, Issue 4, pp 323–378 | Cite as

Deconvolution and Regularization with Toeplitz Matrices

  • Per Christian Hansen


By deconvolution we mean the solution of a linear first-kind integral equation with a convolution-type kernel, i.e., a kernel that depends only on the difference between the two independent variables. Deconvolution problems are special cases of linear first-kind Fredholm integral equations, whose treatment requires the use of regularization methods. The corresponding computational problem takes the form of structured matrix problem with a Toeplitz or block Toeplitz coefficient matrix. The aim of this paper is to present a tutorial survey of numerical algorithms for the practical treatment of these discretized deconvolution problems, with emphasis on methods that take the special structure of the matrix into account. Wherever possible, analogies to classical DFT-based deconvolution problems are drawn. Among other things, we present direct methods for regularization with Toeplitz matrices, and we show how Toeplitz matrix–vector products are computed by means of FFT, being useful in iterative methods. We also introduce the Kronecker product and show how it is used in the discretization and solution of 2-D deconvolution problems whose variables separate.

deconvolution regularization Toeplitz matrix Kronecker product SVD analysis image deblurring 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    H.-M. Adorf, Hubble space telescope image restoration in its fourth year, Inverse Problems 11 (1995) 639–653.Google Scholar
  2. [2]
    C.T.H. Baker, The Numerical Treatment of Integral Equations (Clarendon Press, Oxford, 1977).Google Scholar
  3. [3]
    M.R. Banham and A.K. Katsaggelos, Digital image restoration, IEEE Signal Proc. Mag. 14 (1997) 24–41.Google Scholar
  4. [4]
    M. Bertero, P. Brianzi and E.R. Pike, Super-resolution in confocal scanning microscopy, Inverse Problems 3 (1987) 195–212.Google Scholar
  5. [5]
    Å. Björck, Numerical Methods for Least Squares Problems (SIAM, Philadelphia, PA, 1996).Google Scholar
  6. [6]
    A.W. Bojanczyk, R.P. Brent and F.R. de Hoog, QR factorization of Toeplitz matrices, Numer. Math. 49 (1986) 210–221.Google Scholar
  7. [7]
    W.L. Briggs and V.E. Henson, The DFT. An Owner's Manual for the Discrete Fourier Transform (SIAM, Philadelphia, PA, 1995).Google Scholar
  8. [8]
    R.H. Chan, J.G. Nagy and R.J. Plemmons, FFT-based preconditioners for Toeplitz-block least squares problems, SIAM J. Numer. Anal. 30 (1993) 1740–1768.Google Scholar
  9. [9]
    T. Chan, An optimal circulant preconditioner for Toeplitz systems, SIAM J. Sci. Statist. Comput. 9 (1988) 766–771.Google Scholar
  10. [10]
    T.F. Chan and P.C. Hansen, A look-ahead Levinson algorithm for general Toeplitz systems, IEEE Trans. Signal Processing 40 (1992) 1079–1090.Google Scholar
  11. [11]
    L. Eldén, An efficient algorithm for the regularization of ill-conditioned least squares problems with triangular Toeplitz matrices, SIAM J. Sci. Statist. Comput. 5 (1984) 229–236.Google Scholar
  12. [12]
    L. Eldén, Numerical solution of the sideways heat equation, in: Inverse Problems in Diffusion Processes, eds. H.W. Engl and W. Rundell (SIAM, Philadelphia, PA, 1995).Google Scholar
  13. [13]
    L. Eldén and I. Skoglund, Algorithms for the regularization of ill-conditioned least squares problems with tensor product structure, and applications to space-invariant image restoration, Report LiTH-MAT-R-1982-48, Department of Mathematics, Linköping University, Sweden (1982).Google Scholar
  14. [14]
    J.N. Franklin, Minimum principles for ill-posed problems, SIAM J. Math. Anal. 9 (1978) 638–650.Google Scholar
  15. [15]
    G.H. Golub and C.F. Van Loan, Matrix Computations, 3rd ed. (Johns Hopkins Univ. Press, Baltimore, MD, 1996).Google Scholar
  16. [16]
    M. Hanke and J.G. Nagy, Restoration of atmospherically blurred images by symmetric indefinite conjugate gradient techniques, Inverse Problems 12 (1996) 157–173.Google Scholar
  17. [17]
    M. Hanke and C.R. Vogel, Two-level preconditioners for regularized inverse problems I: Theory, Numer. Math. 83 (1999) 385–402.Google Scholar
  18. [18]
    P.C. Hansen, Truncated SVD solutions to discrete ill-posed problems with ill-determined numerical rank, SIAM J. Sci. Statist. Comput. 11 (1990) 503–518.Google Scholar
  19. [19]
    P.C. Hansen, Regularization tools: A Matlab package for analysis and solution of discrete ill-posed problems, Numer. Algorithms 6 (1994) 1–35.Google Scholar
  20. [20]
    P.C. Hansen, Rank-Deficient and Discrete Ill-Posed Problems. Numerical Aspects of Linear Inversion (SIAM, Philadelphia, PA, 1998).Google Scholar
  21. [21]
    P.C. Hansen, Regularization tools version 3.0 for Matlab 5.2, Numer. Algorithms 20 (1999) 195–196; software and manual is available from Netlib in directory numeralgo/na4.Google Scholar
  22. [22]
    M.R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Nat. Bur. Standards 49 (1952) 409–436.Google Scholar
  23. [23]
    P.A. Jansson, ed. Deconvolution of Images and Spectra, 2nd ed. (Academic Press, San Diego, 1997).Google Scholar
  24. [24]
    T. Kailath and A.H. Sayed, eds., Fast Reliable Algorithms for Matrices with Structure (SIAM, Philadelphia, PA, 1999).Google Scholar
  25. [25]
    N. Kalauptsidis, D. Manolakis and G. Carayannis, A family of computationally efficient algorithms for multichannel signal processing-a tutorial review, Signal Processing 5 (1983) 5–19.Google Scholar
  26. [26]
    M. Kilmer and D.P. O'Leary, Pivoted Cauchy-like preconditioners for regularized solution of ill-posed problems, SIAM J. Sci. Comput. 21 (1999) 88–110.Google Scholar
  27. [27]
    M. Kilmer and G.W. Stewart, Iterative regularization and MINRES, SIAM J. Matrix Anal. Appl. 21 (1999) 613–628.Google Scholar
  28. [28]
    J.G. Nagy, R.J. Plemmons and T.C. Torgersen, Iterative image restoration using approximate inverse preconditioning, IEEE Trans. Image Processing 5 (1996) 1151–1162.Google Scholar
  29. [29]
    H. Park and L. Eldén, Stability analysis and fast algorithms for triangularization of Toeplitz matrices, Numer. Math. 76 (1997) 383–402.Google Scholar
  30. [30]
    H. Park and L. Eldén, Schur-type methods for solving least squares problems with Toeplitz structure, SIAM J. Sci. Comput. 22 (2000) 406–430.Google Scholar
  31. [31]
    D.L. Phillips, A technique for the numerical solution of certain integral equations of the first kind, J. Assoc. Comput. Mach. 9 (1962) 84–97.Google Scholar
  32. [32]
    B.J. Sullivan and B. Liu, On the use of singular value decomposition and decimation in discrete-time band-limited signal extrapolation, IEEE Trans. Acoust. Speech Signal Processing 32 (1984) 1201–1212.Google Scholar
  33. [33]
    A.N. Tikhonov, Solution of incorrectly formulated problems and the regularization method, Soviet Math. Dokl. 4 (1963) 1035–1038; English translation of Dokl. Akad. Nauk. SSSR 51 (1963) 501-504.Google Scholar
  34. [34]
    C.F. Van Loan, Computational Frameworks for the Fast Fourier Transform (SIAM, Philadelphia, PA, 1992).Google Scholar
  35. [35]
    G.M. Wing and J.D. Zahrt, A Primer on Integral Equations of the First Kind (SIAM, Philadelphia, PA, 1991).Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Per Christian Hansen
    • 1
  1. 1.Department of Mathematical ModellingTechnical University of DenmarkLyngbyDenmark

Personalised recommendations