Learning PDEs for Image Restoration via Optimal Control

  • Risheng Liu
  • Zhouchen Lin
  • Wei Zhang
  • Zhixun Su
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6311)

Abstract

Partial differential equations (PDEs) have been successfully applied to many computer vision and image processing problems. However, designing PDEs requires high mathematical skills and good insight into the problems. In this paper, we show that the design of PDEs could be made easier by borrowing the learning strategy from machine learning. In our learning-based PDE (L-PDE) framework for image restoration, there are two terms in our PDE model: (i) a regularizer which encodes the prior knowledge of the image model and (ii) a linear combination of differential invariants, which is data-driven and can effectively adapt to different problems and complex conditions. The L-PDE is learnt from some input/output pairs of training samples via an optimal control technique. The effectiveness of our L-PDE framework for image restoration is demonstrated with two exemplary applications: image denoising and inpainting, where the PDEs are obtained easily and the produced results are comparable to or better than those of traditional PDEs, which were elaborately designed.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

978-3-642-15549-9_9_MOESM1_ESM.pdf (45 kb)
Electronic Supplementary Material (45 KB)

References

  1. 1.
    Gabor, D.: Information theory in electron microscopy. Laboratory Investigation 14, 801–807 (1965)Google Scholar
  2. 2.
    Jain, A.: Partial differential equations and finite-difference methods in image processing, part 1. Journal of Optimization Theory and Applications 23, 65–91 (1977)MATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Koenderink, J.: The structure of images. Biological Cybernetics 50, 363–370 (1984)MATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Witkin, A.: Scale-space filtering. In: International Joint Conference on Artificial Intelligence, IJCAI (1983)Google Scholar
  5. 5.
    Pietro, P., Jitendra, M.: Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 629–639 (1990)CrossRefGoogle Scholar
  6. 6.
    Alvarez, L., Lions, P.L., Morel, J.M.: Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis 29, 845–866 (1992)MATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Chen, T., Shen, J.: Image processing and analysis: variational, PDE, wavelet, and stochastic methods. SIAM Publisher, Philadelphia (2005)Google Scholar
  8. 8.
    Tikhonov, A., Arsenin, V.: Solutions of ill-posed problems. Halsted Press (1977)Google Scholar
  9. 9.
    Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)MATHCrossRefGoogle Scholar
  10. 10.
    Osher, S., Rudin, L.: Feature-oriented image enhancement using shock filters. SIAM Journal on Numerical Analysis 27, 919–940 (1990)MATHCrossRefGoogle Scholar
  11. 11.
    Olver, P.: Applications of Lie groups to differential equations. Springer, Heidelberg (1993)MATHGoogle Scholar
  12. 12.
    Hinze, M., Pinnau, R., Ulbrich, M., Ulbrich, S.: Optimization with PDE contraints. Springer, Heidelberg (2009)Google Scholar
  13. 13.
    Papadakis, N., Corpetti, T., Memin, E.: Dynamically consistent optical flow estimation. In: International Conference on Computer Vision, ICCV (2007)Google Scholar
  14. 14.
    Papadakis, N., Memin, E.: Variational optimal control technique for the tracking of deformable objects. In: International Conference on Computer Vision, ICCV (2007)Google Scholar
  15. 15.
    Florack, L., Romeny, B., Koenderink, J., Viergever, M.: Scale and the differential structure of image. Image and Vision Computing 10, 376–388 (1992)CrossRefGoogle Scholar
  16. 16.
    Lindeberg, T.: Discrete derivative approximations with scale-space properties: a basis for low-level feature extraxtion. Journal of Mathematical Imaging and Vision 3, 349–376 (1993)CrossRefGoogle Scholar
  17. 17.
    Chan, T., Shen, J.: Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics 62, 1019–1043 (2002)MATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Stoer, J., Bulirsch, R.: Introduction to numerical analysis, 2nd edn. Springer, Heidelberg (1998)Google Scholar
  19. 19.
    Lions, J.: Optimal control systems governed by partial differential equations. Springer, Heidelberg (1971)MATHGoogle Scholar
  20. 20.
    Lin, Z., Zhang, W., Tang, X.: Learning partial differential equations for computer vision. Technical report, Microsoft Research, MSR-TR-2008-189 (2008)Google Scholar
  21. 21.
    Lin, Z., Zhang, W., Tang, X.: Designing partial differential equations for image processing by combining differental invariants. Technical report, Microsoft Research, MSR-TR-2009-192 (2009)Google Scholar
  22. 22.
    Chan, T., Esedoglu, S.: Aspects of total variation regularizaed L 1 function approximation. SIAM Journal on Applied Mathematics 65, 1817–1837 (2005)MATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Martin, D.R., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: International Conference on Computer Vision, ICCV (2001)Google Scholar
  24. 24.
    Roth, S., Black, M.J.: Fields of experts: a framework for learning image priors. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2005)Google Scholar
  25. 25.
    Weickert, J., Hagen, H.: Visualization and processing of tensor fields. Springer, Heidelberg (2009)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Risheng Liu
    • 1
  • Zhouchen Lin
    • 2
  • Wei Zhang
    • 3
  • Zhixun Su
    • 1
  1. 1.Dalian University of TechnologyDalianP.R. China
  2. 2.Microsoft Research AsiaBeijingP.R. China
  3. 3.The Chinese University of Hong Kong, ShatinHong KongP.R. China

Personalised recommendations