Deinterlacing with Motion-Compensated Anisotropic Diffusion

  • Matthias Ghodstinat
  • Andrés Bruhn
  • Joachim Weickert
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5604)

Abstract

We present a novel deinterlacing scheme that makes consequent use of discontinuity-preserving partial differential equations (PDEs). It combines the accuracy of recent variational motion estimation techniques with the directional interpolation qualities of anisotropic diffusion filters. Our algorithm proceeds in three steps: First, we interpolate the interlaced images by means of a spatial edge enhancing diffusion process (EED). Then we apply the variational optic flow technique of Brox et al. (2004) in order to obtain a precise interframe registration. Finally we use a spatiotemporal generalisation of EED for motion-compensated inpainting of the missing data in the original sequence. Experiments demonstrate that the proposed method outperforms not only classical deinterlacing schemes, but also a recent PDE-based approach.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Almog, A., Levi, A., Bruckstein, A.M.: Spatial de-interlacing using dynamic time warping. In: Proc. 2005 IEEE International Conference on Image Processing, Genova, Italy, vol. 2, pp. 1010–1013 (2005)Google Scholar
  2. 2.
    Ballester, C., et al.: An inpainting-based deinterlacing method. IEEE Transactions on Image Processing 16(10), 2476–2491 (2007)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bellers, E.B., de Haan, G.: A Key Technology for Scanline Conversion. Elsevier, Amsterdam (2000)Google Scholar
  4. 4.
    Biswas, M., Kumar, S., Nguyen, T.: Performance analysis of motion-compensated de-interlacing systems. IEEE Transactions on Image Processing 15(9), 2596–2609 (2006)CrossRefGoogle Scholar
  5. 5.
    Brox, T., Bruhn, A., Papenberg, N., Weickertw, J.: High accuracy optic flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Bruhn, A., Weickert, J.: Towards ultimate motion estimation: Combining highest accuracy with real-time performance. In: Proc. Tenth International Conference on Computer Vision, Beijing, China, vol. 1, pp. 749–755. IEEE Computer Society Press, Los Alamitos (2005)Google Scholar
  7. 7.
    Bruhn, A., Weickert, J., Kohlberger, T., Schnörr, C.: A multigrid platform for real-time motion computation with discontinuity-preserving variational methods. International Journal of Computer Vision 70(3), 257–277 (2006)CrossRefGoogle Scholar
  8. 8.
    Capodiferro, L.: Interlaced to progressive conversion by median filtering. In: Chiarglione, L. (ed.) Proc. 3rd International Workshop on HDTV, vol. 3, pp. 677–684 (1989)Google Scholar
  9. 9.
    Charbonnier, P., Blanc-Féraud, L., Aubert, G., Barlaud, M.: Two deterministic half-quadratic regularization algorithms for computed imaging. In: Proc. 1994 IEEE International Conference on Image Processing, Austin, TX, pp. 168–172. IEEE Computer Society Press, Los Alamitos (1994)CrossRefGoogle Scholar
  10. 10.
    Cocquerez, J.P., Chanas, L., Blanc-Talon, J.: Simultaneous inpainting and motion estimation of highly degraded video-sequences. In: Bigun, J., Gustavsson, T. (eds.) SCIA 2003. LNCS, vol. 2749, pp. 685–692. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  11. 11.
    D’Amore, L., Marcellino, L., Murli, A.: Image sequence inpainting: a numerical approach for blotch detection and removal via motion estimation. Journal of Computational and Applied Mathematics 2, 396–413 (2007)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Di Zenzo, S.: A note on the gradient of a multi-image. Computer Vision, Graphics and Image Processing 33, 116–125 (1986)MATHCrossRefGoogle Scholar
  13. 13.
    Elad, M., Feuer, A.: Superresolution restoration of an image sequence: adpative filering approach. IEEE Transactions on Pattern Analysis and Machine Intelligence 21, 817–834 (1999)CrossRefGoogle Scholar
  14. 14.
    Förstner, W., Gülch, E.: A fast operator for detection and precise location of distinct points, corners and centres of circular features. In: Proc. ISPRS Intercommission Conference on Fast Processing of Photogrammetric Data, Interlaken, Switzerland, June 1987, pp. 281–305 (1987)Google Scholar
  15. 15.
    Galić, I., Weickert, J., Welk, M., Bruhn, A., Belyaev, A., Seidel, H.-P.: Towards PDE-based image compression. In: Paragios, N., Faugeras, O., Chan, T., Schnörr, C. (eds.) VLSM 2005. LNCS, vol. 3752, pp. 37–48. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  16. 16.
    Galić, I., Weickert, J., Welk, M., Bruhn, A., Belyaev, A., Seidel, H.-P.: Image compressions with anisotropic diffusion. Journal of Mathematical Imaging and Vision 31, 255–269 (2008)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Gillies, D., Plantholt, M., Westerkamp, D.: Motion adaptive field rate upconversion algorithms for 900 lines/100 hz/2:1 displays. IEEE Transactions on Consumer Electronics 36, 149–160 (1990)CrossRefGoogle Scholar
  18. 18.
    Grossauer, H., Thoman, P.: GPU-based multigrid: Real-time performance in high resolution nonlinear image processing. In: Gasteratos, A., Vincze, M., Tsotsos, J.K. (eds.) ICVS 2008. LNCS, vol. 5008, pp. 141–150. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  19. 19.
    Gwosdek, P., Bruhn, A., Weickert, J.: High performance parallel optical flow algorithms on the Sony Playstation 3. In: Deussen, O., Keim, D., Saupe, D. (eds.) VMV 2008: Proc. Vision, Modeling, and Visualization, Konstanz, Germany, AKA, October 2008, pp. 253–262 (2008)Google Scholar
  20. 20.
    Gwosdek, P., Bruhn, A., Weickert, J.: Variational optic flow on the Sony Playstation 3 – acurate dense flow fields for real-time applications. Technical Report 233, Department of Mathematics, Saarland University, Germany (April 2009)Google Scholar
  21. 21.
    He, H., Kondi, L.P.: A regularization framework for joint blur estimation and super-resolution of video sequences. In: Proc. 2005 IEEE International Conference on Image Processing, Genova, Italy, September 2005, vol. 3, pp. 329–332 (2005)Google Scholar
  22. 22.
    Keller, S., Lauze, F., Nielsen, M.: A total variation motion adaptive deinterlacing scheme. In: Kimmel, R., Sochen, N.A., Weickert, J. (eds.) Scale-Space 2005. LNCS, vol. 3459, pp. 408–418. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  23. 23.
    Keller, S., Lauze, F., Nielsen, M.: Deinterlacing using variational methods. IEEE Transactions on Image Processing 17(11), 2015–2028 (2008)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Kokaram, A.: Motion Picture Restoration. Springer, London (1998)Google Scholar
  25. 25.
    Köstler, H., Stürmer, M., Freundl, C., Rüde, U.: PDE based video compression in real time. Technical Report 07-11, Institut für Informatik, Univ. Erlangen-Nürnberg, Germany (2007)Google Scholar
  26. 26.
    Kovačević, J., Safranek, R.J., Yeh, E.M.: Deinterlacing by succesive approximation. IEEE Transactions on Image Processing 6(2), 339–344 (1997)CrossRefGoogle Scholar
  27. 27.
    Li, M., Nguyen, T.: A de-intelacing algorithm using Markov random field model. IEEE Transactions on Image Processing 16(11), 2633–2648 (2007)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Nguyen, A., Dubois, E.: Spatio-temporal adaptive interlaced-to-progressive conversion. In: Dubois, E., Chiarglione, L. (eds.) Proc. 4th International Workshop on HDTV, Kawasaki, Japan, pp. 749–756 (1992)Google Scholar
  29. 29.
    Patti, A.J., Tekalp, A.M., Sezan, M.I.: Image sequence restoration and de-interlacing by motion compensated kalman filtering. In: Image and Video Processing. Proceedings of SPIE, vol. 1903, pp. 59–70. SPIE Press, Bellingham (1993)Google Scholar
  30. 30.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)Google Scholar
  31. 31.
    Tekalp, M.: Digital Video Processing. Prentice Hall, Englewood Cliffs (1995)Google Scholar
  32. 32.
    Wang, Y., Ostermann, J., Zhang, Y.: Video Processing and Communications. Prentice Hall, Upper Saddle River (2002)Google Scholar
  33. 33.
    Wedel, A., Pock, T., Zach, C., Bischof, H., Cremers, D.: An improved algorithm for TV-L 1 optical flow. In: Rosenhahn, B., Cremers, D., Yuille, A. (eds.) Visual Motion Analysis 2008. LNCS, vol. 5604, pp. 23–45. Springer, Heidelberg (2009)Google Scholar
  34. 34.
    Weickert, J.: Theoretical foundations of anisotropic diffusion in image processing. Computing Suppl 11, 221–236 (1996)CrossRefGoogle Scholar
  35. 35.
    Weickert, J.: Anisotropic Diffusion in Image Processing. Teubner, Stuttgart (1998)Google Scholar
  36. 36.
    Woods, N.A., Galatsanos, N.P.: Non-stationary approximate Bayesian super-resolution using a hierarchical prior model. In: Proc. 2005 IEEE International Conference on Image Processing, Genova, Italy, September 2005, vol. 1, pp. 37–40 (2005)Google Scholar
  37. 37.
    Yoo, H., Jeong, J.: Direction-oriented interpolation and its application to de-interlacing. IEEE Transactions on Consumer Electronics 48(4), 954–962 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Matthias Ghodstinat
    • 1
  • Andrés Bruhn
    • 1
  • Joachim Weickert
    • 1
  1. 1.Mathematical Image Analysis Group, Faculty of Mathematics and Computer ScienceSaarland UniversitySaarbrückenGermany

Personalised recommendations