Skip to main content

Relations Between Regularization and Diffusion Filtering

Abstract

Regularization may be regarded as diffusion filtering with an implicit time discretization where one single step is used. Thus, iterated regularization with small regularization parameters approximates a diffusion process. The goal of this paper is to analyse relations between noniterated and iterated regularization and diffusion filtering in image processing. In the linear regularization framework, we show that with iterated Tikhonov regularization noise can be better handled than with noniterated. In the nonlinear framework, two filtering strategies are considered: the total variation regularization technique and the diffusion filter technique of Perona and Malik. It is shown that the Perona-Malik equation decreases the total variation during its evolution. While noniterated and iterated total variation regularization is well-posed, one cannot expect to find a minimizing sequence which converges to a minimizer of the corresponding energy functional for the Perona–Malik filter. To overcome this shortcoming, a novel regularization technique of the Perona–Malik process is presented that allows to construct a weakly lower semi-continuous energy functional. In analogy to recently derived results for a well-posed class of regularized Perona–Malik filters, we introduce Lyapunov functionals and convergence results for regularization methods. Experiments on real-world images illustrate that iterated linear regularization performs better than noniterated, while no significant differences between noniterated and iterated total variation regularization have been observed.

This is a preview of subscription content, access via your institution.

References

  1. R. Acar and C.R. Vogel, “Analysis of bounded variation penalty methods for ill-posed problems,” Inverse Problems, Vol. 10, pp. 1217–1229, 1994.

    Google Scholar 

  2. R.A. Adams, Sobolev Spaces, Academic Press: New York, San Francisco, London, 1975.

    Google Scholar 

  3. G.I. Barenblatt, M. Bertsch, R. Dal Passo, and M. Ughi, “A degenerate pseudoparabolic regularization of a nonlinear forward-backward heat equation arising in the theory of heat and mass exchange in stably stratified turbulent shear flow,” SIAM J. Math. Anal., Vol. 24, pp. 1414–1439, 1993.

    Google Scholar 

  4. M. Bertero, T.A. Poggio, and V. Torre, “Ill-posed problems in early vision,” Proc. IEEE, Vol. 76, pp. 869–889, 1988.

    Google Scholar 

  5. F. Catté, P.L. Lions, J.M. Morel, and T. Coll, “Image selective smoothing and edge detection by nonlinear diffusion,” SIAM J. Numer. Anal., Vol. 32, pp. 1895–1909, 1992.

    Google Scholar 

  6. A. Chambolle and P.L. Lions, “Image recovery via total variation minimization and related problems,” Numer. Math., Vol. 76, pp. 167–188, 1995.

    Google Scholar 

  7. T.F. Chan, G.H. Golub, and P. Mulet, “A nonlinear primal-dual method for total-variation based image restoration,” in ICAOS '96: Images, Wavelets and PDEs, M.O. Berger, R. Deriche, I. Herlin, J. Jaffré, and J.M. Morel (Eds.), Lecture Notes in Control and Information Sciences, Vol. 219, Springer, London, 1996, pp. 241–252.

    Google Scholar 

  8. T.F. Chan and C.K. Wong, “Total variation blind deconvolution,” IEEE Trans. Image Proc., Vol. 7, pp. 370–375, 1998.

    Google Scholar 

  9. P. Charbonnier, L. Blanc-Féraud, G. Aubert, and M. Barlaud, “Two deterministic half-quadratic regularization algorithms for computed imaging,” in Proc. IEEE Int. Conf. Image Processing (ICIP-94, Austin, Nov. 13–16, 1994), IEEE Computer Society Press, Los Alamitos, 1994, Vol. 2, pp. 168–172.

    Google Scholar 

  10. B. Dacorogna, Weak Continuity and Weak Lower Semicontinuity of Non-Linear Functionals, Lecture Notes in Mathematics, Springer, Berlin, 1982.

    Google Scholar 

  11. B. Dacorogna, Direct Methods in the Calculus of Variations, Springer: Berlin, 1989.

    Google Scholar 

  12. D. Dobson and O. Scherzer, “Analysis of regularized total variation penalty methods for denoising,” Inverse Problems, Vol. 12, pp. 601–617, 1996.

    Google Scholar 

  13. D.C. Dobson and C.R. Vogel, “Convergence of an iterative method for total variation denoising,” SIAM J. Numer. Anal., Vol. 34, pp. 1779–1791, 1997.

    Google Scholar 

  14. H.W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems, Kluwer, Dordrecht, 1996.

    Google Scholar 

  15. L.C. Evans and R.F. Gariepy, Measure Theory and Fine Properties of Functions, CRC Press: Boca Raton, 1992.

    Google Scholar 

  16. L. Florack, Image Structure, Kluwer: Dordrecht, 1997.

    Google Scholar 

  17. D. Geman and C. Yang, “Nonlinear image recovery with half-quadratic regularization,” IEEE Transactions on Image Processing, Vol. 4, pp. 932–945, 1995.

    Google Scholar 

  18. P.J. Green, “Bayesian reconstructions from emission tomography data using a modified EM algorithm,” IEEE Trans. Medical Imaging, Vol. 9, pp. 84–93, 1990.

    Google Scholar 

  19. C.W. Groetsch, The Theory of Tikhonov regularization for Fredholm Equations of the First Kind, Pitman: Boston, 1984.

    Google Scholar 

  20. C.W. Groetsch, “Spectral methods for linear inverse problems with unbounded operators,” J. Approx. Th., Vol. 70, pp. 16–28, 1992.

    Google Scholar 

  21. C.W. Groetsch and O. Scherzer, “Optimal order of convergence for stable evaluation of differential operators,” Electronic Journal of Differential Equations (http://ejde.math. unt.edu), No. 4, pp. 1–10, 1993.

  22. D. Gilbarg and N. Trudinger, Elliptic Partial Differential Equations of Second Order. Grundlehren der Mathematischen Wissenschaften, Vol. 224, Springer, Berlin, 2nd ed., 1994.

    Google Scholar 

  23. E. Giusti, Minimal Surfaces and Functions of Bounded Variation, Birkhäuser: Basel, 1984.

    Google Scholar 

  24. B.M. ter Haar Romeny (Ed.), Geometry-Driven Diffusion in Computer Vision, Kluwer: Dordrecht, 1994.

    Google Scholar 

  25. M. Hanke and C.W. Groetsch, “Nonstationary iterated Tikhonov regularization,” J. Optim. Theory and Applications, Vol. 98, pp. 37–53, 1998.

    Google Scholar 

  26. G. Helmberg, Introduction to Spectral Theory in Hilbert Space, Amsterdam, London, 1969.

  27. K. Ito, and K. Kunisch, “An active set strategy based on the augmented Lagrangian formulation for image restoration,” RAIRO Math. Mod. and Num. Analysis, Vol. 33, pp. 1–21, 1999.

    Google Scholar 

  28. Y. Li and F. Santosa, “A computational algorithm for minimizing total variation in image enhancement,” IEEE Trans. Image Proc., Vol. 5, pp. 987–995, 1996.

    Google Scholar 

  29. T. Lindeberg, Scale-Space Theory in Computer Vision, Kluwer: Boston, 1994.

    Google Scholar 

  30. J.M. Morel and S. Solimini, Variational Methods in Image Segmentation, Birkhäuser: Boston, 1995.

    Google Scholar 

  31. V.A. Morozov, Methods for Solving Incorrectly Posed Problems, Springer: New York, 1984.

    Google Scholar 

  32. M.Z. Nashed and O. Scherzer, “Least squares and bounded variation regularization with nondifferentiable functionals,” Numer. Funct. Anal. and Optimiz., Vol. 19, pp. 873–901, 1998.

    Google Scholar 

  33. M. Nielsen, L. Florack, and R. Deriche, “Regularization, scale-space and edge detection filters,” J. Math. Imag. Vision, Vol. 7, pp. 291–307, 1997.

    Google Scholar 

  34. M. Nitzberg and T. Shiota, “Nonlinear image filtering with edge and corner enhancement,” IEEE Trans. Pattern Anal. Mach. Intell., Vol. 14, pp. 826–833, 1992.

    Google Scholar 

  35. N. Nordström, “Biased anisotropic diffusion-a unified regularization and diffusion approach to edge detection,” Image and Vision Computing, Vol. 8, pp. 318–327, 1990.

    Google Scholar 

  36. S. Osher and L.I. Rudin, “Feature-oriented image enhencement using shock filters,” SIAM J. Numer. Anal., Vol. 27, pp. 919–940, 1990.

    Google Scholar 

  37. P. Perona and J. Malik, “Scale space and edge detection using anisotropic diffusion,” IEEE Trans. Pattern Anal. Mach. Intell., Vol. 12, pp. 629–639, 1990.

    Google Scholar 

  38. E. Radmoser, O. Scherzer, and J. Weickert, “Scale-space properties of regularization methods,” in M. Nielsen, P. Johansen, O.F. Olsen, and J. Weickert (Eds.), Scale-Space Theories in Computer Vision, Lecture Notes in Computer Science, Springer, Berlin, Vol. 1682, pp. 211–222, 1999.

    Google Scholar 

  39. F. Rieszm and B. Sz-Nagy, Functional Analysis, Ungar, New York, 1955.

  40. L.I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D, Vol. 60, pp. 259–268, 1992.

    Google Scholar 

  41. O. Scherzer, “Stable evaluation of differential operators and linear and nonlinear multi-scale filtering,” Electronic Journal of Differential Equations (http://ejde.math.unt.edu), No. 15, pp. 1–12, 1997.

  42. O. Scherzer, “Denoising with higher order derivatives of bounded variation and an application to parameter estimation,” Computing, Vol. 60, pp. 1–27, 1998.

    Google Scholar 

  43. C. Schnörr, “Unique reconstruction of piecewise smooth images by minimizing strictly convex non-quadratic functionals,” J. Math. Imag. Vision, Vol. 4, pp. 189–198, 1994.

    Google Scholar 

  44. J. Sporring, M. Nielsen, L. Florack, and P. Johansen (Eds.), Gaussian Scale-Space Theory, Kluwer: Dordrecht, 1997.

    Google Scholar 

  45. D.M. Strong and T.F. Chan, “Relation of regularization parameter and scale in total variation based image denoising,” CAM Report 96–7, Dept. of Mathematics, Univ. of California, Los Angeles, CA 90024, U.S.A., 1996.

    Google Scholar 

  46. A.N. Tikhonov and V.Y. Arsenin, Solutions of Ill-Posed Problems, John Wiley & Sons, Washington, D.C., 1977, Translation editor Fritz John (English).

    Google Scholar 

  47. V. Torre and T.A. Poggio, “On edge detection,” IEEE Trans. Pattern Anal. Mach. Intell., Vol. 8, pp. 148–163, 1986.

    Google Scholar 

  48. C.R. Vogel and M.E. Oman, “Fast, robust total variation-based reconstruction of noisy, blurred images,” IEEE Trans. Image Proc., Vol. 7, pp. 813–824, 1998.

    Google Scholar 

  49. J. Weickert, “Anisotropic diffusion filters for image processing based quality control,” in Proc. Seventh European Conf. on Mathematics in Industry, A. Fasano and M. Primicerio (Eds.), Teubner, Stuttgart, 1994, pp. 355–362.

    Google Scholar 

  50. J. Weickert, “A review of nonlinear diffusion filtering,” in Scale-Space Theory in Computer Vision, B. ter Haar Romeny, L. Florack, J. Koenderink and M. Viergever (Eds.), Lecture Notes in Computer Science, Vol. 1252, Springer, Berlin, 1997, pp. 3–28.

    Google Scholar 

  51. J. Weickert, Anisotropic Diffusion in Image Processing, Teubner: Stuttgart, 1998.

    Google Scholar 

  52. J. Weickert and B. Benhamouda, “A semidiscrete nonlinear scale-space theory and its relation to the Perona-Malik paradox,” in Advances in Computer Vision, F. Solina, W.G. Kropatsch, R. Klette, and R. Bajcsy (Eds.), Springer, Wien, 1997, pp. 1–10.

    Google Scholar 

  53. J. Weidmann, Lineare Operatoren in Hilberträumen, Teubner: Stuttgart, 1976.

    Google Scholar 

  54. J. Wloka, Partielle Differentialgleichungen, Teubner: Stuttgart, 1982.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Scherzer, O., Weickert, J. Relations Between Regularization and Diffusion Filtering. Journal of Mathematical Imaging and Vision 12, 43–63 (2000). https://doi.org/10.1023/A:1008344608808

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008344608808

  • regularization
  • diffusion filtering
  • image restoration
  • inverse problems
  • total variation denoising