Advertisement

International Journal of Computer Vision

, Volume 92, Issue 2, pp 162–176 | Cite as

Partial Differential Equations for Zooming, Deinterlacing and Dejittering

  • Frank Lenzen
  • Otmar Scherzer
Article

Abstract

In this paper, for imaging applications, we introduce partial differential equations (PDEs), which allow for correcting displacement errors, for dejittering, and for deinterlacing, respectively, in multi-channel data. These equations are derived via semi-groups for non-convex energy functionals. As a particular example, for gray valued data, we find the mean curvature equation and the corresponding non-convex energy functional. As a further application for correction of displacement errors we study image interpolation, in particular zooming, of digital color images. For actual image zooming, the solutions of the proposed PDEs are projected onto a space of functions satisfying interpolation constraints. A comparison of the test results with standard and state-of-the-art interpolation algorithms shows the competitiveness of this approach.

Keywords

Non-convex semigroups Partial differential equations Dejittering Deinterlacing Zooming 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aubert, G., Deriche, R., & Kornprobst, P. (1999). Computing optical flow via variational techniques. The SIAM Journal on Applied Mathematics, 60, 156–182. MathSciNetzbMATHCrossRefGoogle Scholar
  2. Ballester, C., Bertalmio, M., Caselles, V., Garrido, L., Ranchin, A., & Marques, V. (2007). An inpainting-based deinterlacing method. IEEE Transactions on Image Processing, 16(10), 2476–2491. MathSciNetCrossRefGoogle Scholar
  3. Belahmidi, A., & Guichard, F. (2004). A partial differential equation approach to image zoom. In Proc. of the 2004 int. conf. on image processing (pp. 649–652). Google Scholar
  4. Bertalmio, M., Sapiro, G., Caselles, V., & Ballester, C. (2000). Image inpainting. In S. Hoffmeyer (Ed.), Proceedings of the computer graphics conference 2000 (SIGGRAPH-00) (pp. 417–424). New York: ACM Press. Google Scholar
  5. Brézis, H. (1973). Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert. Amsterdam: North-Holland. North-Holland Mathematics Studies, No. 5. Notas de Matemática (50). zbMATHGoogle Scholar
  6. Burger, W., & Burge, M. (2005). Digitale Bildverarbeitung. Berlin: Springer. Google Scholar
  7. Chambolle, A. (1994). Partial differential equations and image processing. In Proceedings 1994 international conference on image processing (Vol. 1, pp. 16–20). Google Scholar
  8. Chan, R., Setzer, S., & Steidl, G. (2008). Inpainting by flexible Haar wavelet shrinkage. The SIAM Journal on Imaging Sciences, 1(3), 273–293. MathSciNetzbMATHCrossRefGoogle Scholar
  9. Chan, T., Kang, S., & Shen, J. (2002). Euler’s elastica and curvature based inpaintings. The SIAM Journal on Applied Mathematics, 63(2), 564–592. MathSciNetzbMATHGoogle Scholar
  10. Dacorogna, B. (1982). Lecture notes in computer science : Vol. 922. Weak continuity and weak lower semicontinuity of non-linear functionals. Berlin/Heidelberg/New York: Springer. Google Scholar
  11. Dacorogna, B. (1989). Applied mathematical sciences : Vol. 78. Direct methods in the calculus of variations. Berlin: Springer. zbMATHGoogle Scholar
  12. De Haan, G., & Bellers, E. B. (1997). De-interlacing of video data. IEEE Transactions on Consumer Electronics, 43, 819–825. CrossRefGoogle Scholar
  13. De Haan, G., & Bellers, E. B. (1998). Deinterlacing—an overview. Proceedings of the IEEE, 86, 1839–1857. CrossRefGoogle Scholar
  14. Deriche, R., Kornprobst, P., & Aubert, G. (1995). Optical flow estimation while preserving its discontinuities: a variational approach. In Proceedings of the 2nd Asian conference on computer vision (Vol. 2, pp. 71–80). Google Scholar
  15. Elbau, P., Grasmair, M., Lenzen, F., & Scherzer, O. (2008). Evolution by non-convex energy functionals (Reports of FSP S092—“Industrial Geometry” 75). University of Innsbruck, Austria. Submitted. Google Scholar
  16. Ghodstinat, M., Bruhn, A., & Weickert, J. (2009). Deinterlacing with motion-compensated anisotropic diffusion. In D. Cremers, B. Rosenhahn, A. Yuille, & F. Schmidt (Eds.), Lecture notes in computer science : Vol. 5604. Statistical and geometrical approaches to visual motion analysis (pp. 91–106). Berlin: Springer. CrossRefGoogle Scholar
  17. Grasmair, M., Lenzen, F., Obereder, A., Scherzer, O., & Fuchs, M. (2005). A non-convex PDE scale space. In R. Kimmel, N. A. Sochen, & J. Weickert (Eds.), Lecture notes in computer science : Vol. 3459. Scale space and pde methods in computer vision (pp. 303–315). Berlin: Springer. CrossRefGoogle Scholar
  18. Guichard, F., & Malgouyres, F. (1998). Total variation based interpolation. In Proceedings of the European signal processing conference (Vol. 3, pp. 1741–1744). Google Scholar
  19. Guichard, F., Morel, J. M., & Ryan, R. (2007). textitContrast invariant image analysis and PDE’s. Preprint, Paris. Work in preparation. Google Scholar
  20. Jähne, B. (2002). Digitale Bildverarbeitung (5th ed.). Berlin: Springer. Google Scholar
  21. Kang, S.-H., & Shen, J. (2006). Video dejittering by Bake and Shake. Image and Vision Computing, 24(2), 143–152. CrossRefGoogle Scholar
  22. Keller, S., Lauze, F., & Nielsen, M. (2005). A total variation motion adaptive deinterlacing scheme. In R. Kimmel, N. A. Sochen, & J. Weickert (Eds.), Lecture notes in computer science : Vol. 3459. Scale space and PDE methods in computer vision (pp. 408–419). Berlin: Springer. CrossRefGoogle Scholar
  23. Keller, S. H., Lauze, F., & Nielsen, M. (2008). Deinterlacing using variational methods IEEE. Transactions on Image Processing, 17(11), 2015–2028. MathSciNetCrossRefGoogle Scholar
  24. Lenzen, F., & Scherzer, O. (2009). A geometric PDE for interpolation of m-channel data. In X. C. Tai, K. Mørken, M. Lysaker, & K. A. Lie (Eds.), Lecture notes in computer science : Vol. 5567. Scale space and variational methods in computer vision (pp. 413–425). Second international conference, SSVM 2009, Voss, Norway, June 1–5. Berlin: Springer. CrossRefGoogle Scholar
  25. Malgouyres, F., & Guichard, F. (2001). Edge direction preserving image zooming: a mathematical and numerical analysis. SIAM Journal on Numerical Analysis, 39, 1–37. MathSciNetzbMATHCrossRefGoogle Scholar
  26. Nashed, M. (Ed.) (1976). Generalized inverses and applications. New York: Academic Press [Harcourt Brace Jovanovich Publishers]. zbMATHGoogle Scholar
  27. Nesi, P. (1993). Variational approach to optical flow estimation managing discontinuities. Image and Vision Computing, 11(7), 419–439. CrossRefGoogle Scholar
  28. Nikolova, M., (2009). Fast dejittering for digital video frames. In X. C. Tai, K. Mørken, M. Lysaker, & K. A. Lie (Eds.), Lecture notes in computer science : Vol. 5567. Scale space and variational methods in computer vision (pp. 439–451). Second International Conference, SSVM 2009, Voss, Norway, June 1-5. Berlin: Springer. CrossRefGoogle Scholar
  29. Papenberg, N., Bruhn, A., Brox, T., Didas, S., & Weickert, J. (2006). Highly accurate optic flow computation with theoretically justified warping. International Journal of Computer Vision, 67(2), 141–158. CrossRefGoogle Scholar
  30. Roussos, A., & Maragos, P. (2007). Vector-valued image interpolation by an anisotropic diffusion-projection PDE. In F. Sgallari, A. Murli, & N. Paragios (Eds.), Lecture notes in computer science : Vol. 4485. Scale space and variational methods in computer vision (pp. 104–115). First International Conference, SSVM 2007, Ischia, Italy, May 30–June 2. Berlin: Springer. CrossRefGoogle Scholar
  31. Roussos, A., & Maragos, P. (2009). Reversible interpolation of vectorial images by an anisotropic diffusion-projection PDE. International Journal of Computer Vision, 84(2), 130–145. CrossRefGoogle Scholar
  32. Scharstein, D., & Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1), 7–42. zbMATHCrossRefGoogle Scholar
  33. Scherzer, O., & Weickert, J. (2000). Relations between regularization and diffusion filtering. Journal of Mathematical Imaging and Vision, 12(1), 43–63. MathSciNetzbMATHCrossRefGoogle Scholar
  34. Shen, J. (2004). Bayesian video dejittering by the BV image model. The SIAM Journal on Applied Mathematics, 64(5), 1691–1708. zbMATHCrossRefGoogle Scholar
  35. Szu, H. H., Buss, J. R., Garcia, J. P., Breaux, N. A. et al. (2004). Subpixel jitter video restoration on board of micro-UAV. Proceedings of SPIE, 5439(183). Google Scholar
  36. Thevenaz, P., Blu, T., & Unser, M. (2000). Image interpolation and resampling. In I. N. Bankman (Ed.), Handbook of medical imaging, processing and analysis (pp. 393–420). San Diego: Academic Press. Chap. 25. CrossRefGoogle Scholar
  37. Tschumperlé, D. (2006). Fast anisotropic smoothing of multi-valued images using curvature-preserving PDEs. International Journal of Computer Vision, 68, 65–82. CrossRefGoogle Scholar
  38. Tschumperle, D., & Besserer, B. (2004). High quality deinterlacing using inpainting and shutter-model directed temporal interpolation. In Int. conf. on computer vision and graphics (pp. 301–307). Google Scholar
  39. Tschumperlé, D., & Deriche, R. (2005). Vector valued image regularization with PDEs: a common framework for different applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 506–517. CrossRefGoogle Scholar
  40. Wang, Z., Glazowski, C. E., & Zavislan, J. M. (2007). Modulation transfer function measurement of scanning reflectance microscopes. Journal of Biomedical Optics, 12(5), 051802. CrossRefGoogle Scholar
  41. Weickert, J., & Welk, M. (2006). Tensor field interpolation with PDEs. In H. Hagen, & J. Weickert (Eds.), Mathematics and visualization. Visualization and processing of tensor fields (pp. 315–325). Berlin: Springer. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Heidelberg Collaboration for Image ProcessingUniversity of HeidelbergHeidelbergGermany
  2. 2.Computational Science CenterUniversity of ViennaViennaAustria
  3. 3.Johann Radon Institute for Computational and Applied Mathematics (RICAM)Austrian Academy of SciencesLinzAustria

Personalised recommendations