Learning PDEs for Image Restoration via Optimal Control

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6311)


Partial differential equations (PDEs) have been successfully applied to many computer vision and image processing problems. However, designing PDEs requires high mathematical skills and good insight into the problems. In this paper, we show that the design of PDEs could be made easier by borrowing the learning strategy from machine learning. In our learning-based PDE (L-PDE) framework for image restoration, there are two terms in our PDE model: (i) a regularizer which encodes the prior knowledge of the image model and (ii) a linear combination of differential invariants, which is data-driven and can effectively adapt to different problems and complex conditions. The L-PDE is learnt from some input/output pairs of training samples via an optimal control technique. The effectiveness of our L-PDE framework for image restoration is demonstrated with two exemplary applications: image denoising and inpainting, where the PDEs are obtained easily and the produced results are comparable to or better than those of traditional PDEs, which were elaborately designed.


Optimal Control Problem Image Restoration Noisy Image Image Denoising Total Variation Regularization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

978-3-642-15549-9_9_MOESM1_ESM.pdf (45 kb)
Electronic Supplementary Material (45 KB)


  1. 1.
    Gabor, D.: Information theory in electron microscopy. Laboratory Investigation 14, 801–807 (1965)Google Scholar
  2. 2.
    Jain, A.: Partial differential equations and finite-difference methods in image processing, part 1. Journal of Optimization Theory and Applications 23, 65–91 (1977)zbMATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Koenderink, J.: The structure of images. Biological Cybernetics 50, 363–370 (1984)zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Witkin, A.: Scale-space filtering. In: International Joint Conference on Artificial Intelligence, IJCAI (1983)Google Scholar
  5. 5.
    Pietro, P., Jitendra, M.: Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 629–639 (1990)CrossRefGoogle Scholar
  6. 6.
    Alvarez, L., Lions, P.L., Morel, J.M.: Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis 29, 845–866 (1992)zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Chen, T., Shen, J.: Image processing and analysis: variational, PDE, wavelet, and stochastic methods. SIAM Publisher, Philadelphia (2005)Google Scholar
  8. 8.
    Tikhonov, A., Arsenin, V.: Solutions of ill-posed problems. Halsted Press (1977)Google Scholar
  9. 9.
    Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)zbMATHCrossRefGoogle Scholar
  10. 10.
    Osher, S., Rudin, L.: Feature-oriented image enhancement using shock filters. SIAM Journal on Numerical Analysis 27, 919–940 (1990)zbMATHCrossRefGoogle Scholar
  11. 11.
    Olver, P.: Applications of Lie groups to differential equations. Springer, Heidelberg (1993)zbMATHGoogle Scholar
  12. 12.
    Hinze, M., Pinnau, R., Ulbrich, M., Ulbrich, S.: Optimization with PDE contraints. Springer, Heidelberg (2009)Google Scholar
  13. 13.
    Papadakis, N., Corpetti, T., Memin, E.: Dynamically consistent optical flow estimation. In: International Conference on Computer Vision, ICCV (2007)Google Scholar
  14. 14.
    Papadakis, N., Memin, E.: Variational optimal control technique for the tracking of deformable objects. In: International Conference on Computer Vision, ICCV (2007)Google Scholar
  15. 15.
    Florack, L., Romeny, B., Koenderink, J., Viergever, M.: Scale and the differential structure of image. Image and Vision Computing 10, 376–388 (1992)CrossRefGoogle Scholar
  16. 16.
    Lindeberg, T.: Discrete derivative approximations with scale-space properties: a basis for low-level feature extraxtion. Journal of Mathematical Imaging and Vision 3, 349–376 (1993)CrossRefGoogle Scholar
  17. 17.
    Chan, T., Shen, J.: Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics 62, 1019–1043 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Stoer, J., Bulirsch, R.: Introduction to numerical analysis, 2nd edn. Springer, Heidelberg (1998)Google Scholar
  19. 19.
    Lions, J.: Optimal control systems governed by partial differential equations. Springer, Heidelberg (1971)zbMATHGoogle Scholar
  20. 20.
    Lin, Z., Zhang, W., Tang, X.: Learning partial differential equations for computer vision. Technical report, Microsoft Research, MSR-TR-2008-189 (2008)Google Scholar
  21. 21.
    Lin, Z., Zhang, W., Tang, X.: Designing partial differential equations for image processing by combining differental invariants. Technical report, Microsoft Research, MSR-TR-2009-192 (2009)Google Scholar
  22. 22.
    Chan, T., Esedoglu, S.: Aspects of total variation regularizaed L 1 function approximation. SIAM Journal on Applied Mathematics 65, 1817–1837 (2005)zbMATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Martin, D.R., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: International Conference on Computer Vision, ICCV (2001)Google Scholar
  24. 24.
    Roth, S., Black, M.J.: Fields of experts: a framework for learning image priors. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2005)Google Scholar
  25. 25.
    Weickert, J., Hagen, H.: Visualization and processing of tensor fields. Springer, Heidelberg (2009)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  1. 1.Dalian University of TechnologyDalianP.R. China
  2. 2.Microsoft Research AsiaBeijingP.R. China
  3. 3.The Chinese University of Hong Kong, ShatinHong KongP.R. China

Personalised recommendations