Advertisement

Reconstructing Optical Flow Fields by Motion Inpainting

  • Benjamin Berkels
  • Claudia Kondermann
  • Christoph Garbe
  • Martin Rumpf
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5681)

Abstract

An edge-sensitive variational approach for the restoration of optical flow fields is presented. Real world optical flow fields are frequently corrupted by noise, reflection artifacts or missing local information. Still, applications may require dense motion fields. In this paper, we pick up image inpainting methodology to restore motion fields, which have been extracted from image sequences based on a statistical hypothesis test on neighboring flow vectors. A motion field inpainting model is presented, which takes into account additional information from the image sequence to improve the reconstruction result. The underlying functional directly combines motion and image information and allows to control the impact of image edges on the motion field reconstruction. In fact, in case of jumps of the motion field, where the jump set coincides with an edge set of the underlying image intensity, an anisotropic TV-type functional acts as a prior in the inpainting model. We compare the resulting image guided motion inpainting algorithm to diffusion and standard TV inpainting methods.

Keywords

Descent Direction Motion Field Error Concealment Image Inpainting Total Variation Minimization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bruhn, A., Weickert, J.: In: A Confidence Measure for Variational Optic Flow Methods, pp. 283–298. Springer, Heidelberg (2006)Google Scholar
  2. 2.
    Kondermann, C., Kondermann, D., Jähne, B., Garbe, C.S.: An adaptive confidence measure for optical flows based on linear subspace projections. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 132–141. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  3. 3.
    Kondermann, C., Mester, R., Garbe, C.: A statistical confidence measure for optical flows. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 290–301. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  4. 4.
    Masnou, S., Morel, J.: Level lines based disocclusion. In: Proceedings of the ICIP 1998, vol. 3, pp. 259–263 (1998)Google Scholar
  5. 5.
    Chan, T.F., Shen, J.: Mathematical models for local nontexture inpaintings. SIAM J. Appl. Math. 62, 1019–1043 (2001)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Chan, T.F., Shen, J.: Non-texture inpainting by curvature-driven diffusions. J. Visual Comm. Image Rep. 12, 436–449 (2001)CrossRefGoogle Scholar
  7. 7.
    Telea, A.: An image inpainting technique based on the fast marching method. Journal of graphics tools 9(1), 25–36 (2003)Google Scholar
  8. 8.
    Bornemann, F., März, T.: Fast image inpainting based on coherence transport. Journal of Mathematical Imaging and Vision 28(3), 259–278 (2007)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Bertalmio, M., Bertozzi, A., Sapiro, G.: Navier-stokes, fluid dynamics, and image and video inpainting. In: IEEE Proceedings of the International Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 355–362 (2001)Google Scholar
  10. 10.
    Chan, T., Shen, J.: Mathematical models for local non-texture inpaintings. SIAM J. Appl. Math. 62(3), 1019–1043 (2002)CrossRefzbMATHGoogle Scholar
  11. 11.
    Nitzberg, M., Shiota, T., Mumford, D.: Filtering, Segmentation and Depth. LNCS, vol. 662. Springer, Heidelberg (1993)CrossRefzbMATHGoogle Scholar
  12. 12.
    Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and gray levels. IEEE Transactions on Image Processing 10(8), 1200–1211 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Ambrosio, L., Masnou, S.: A direct variational approach to a problem arising in image reconstruction. Interfaces and Free Boundaries 5, 63–81 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Chan, T.F., Shen, J., Zhou, H.M.: Total variation wavelet inpainting. Journal of Mathematical Imaging and Vision 25(1), 107–125 (2006)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Patwardhan, K.A., Sapiro, G., Bertalmio, M.: Video inpainting of occluding and occluded objects. IMA Preprint Series 2016 (January 2005)Google Scholar
  16. 16.
    Matsushita, Y., Ofek, E., Weina, G., Tang, X., Shum, H.: Full-frame video stabilization with motion inpainting. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(7), 1150–1163 (2006)CrossRefGoogle Scholar
  17. 17.
    Chen, L., Chan, S., Shum, H.: A joint motion-image inpainting method for error concealment in video coding. In: IEEE International Conference on Image Processing, ICIP (2006)Google Scholar
  18. 18.
    Kondermann, C., Kondermann, D., Garbe, C.: Postprocessing of optical flows via surface measures and motion inpainting. In: Rigoll, G. (ed.) DAGM 2008. LNCS, vol. 5096, pp. 355–364. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  19. 19.
    Ambrosio, L., Fusco, N., Pallara, D.: Functions of bounded variation and free discontinuity problems. Oxford Mathematical Monographs. Oxford University Press, New York (2000)zbMATHGoogle Scholar
  20. 20.
    Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence 12(7), 629–639 (1990)CrossRefGoogle Scholar
  21. 21.
    Nagel, H.H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 8(5), 565–593 (1986)CrossRefGoogle Scholar
  22. 22.
    Alvarez, L., Monreal, L.J.E., Lefebure, M., Perez, J.S.: A pde model for computing the optical flow. In: Proc. XVI Congreso de Ecuaciones Diferenciales Aplicaciones, Universidad de Las Palmas de Gran Canaria, September 1999, pp. 1349–1356 (1999)Google Scholar
  23. 23.
    Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Transactions on Image Processing 10(2), 266–277 (2001)CrossRefzbMATHGoogle Scholar
  24. 24.
    Braess, D.: Finite Elemente, 2nd edn. Springer, Heidelberg (1997); Theorie, schnelle Löser und Anwendungen in der ElastizitätstheorieCrossRefzbMATHGoogle Scholar
  25. 25.
    Schaback, R., Werner, H.: Numerische Mathematik, 4th edn. Springer, Berlin (1992)CrossRefGoogle Scholar
  26. 26.
    Kosmol, P.: Methoden zur numerischen Behandlung nichtlinearer Gleichungen und Optimierungsaufgaben. 2nd edn. Teubner, Stuttgart (1993)Google Scholar
  27. 27.
    Sundaramoorthi, G., Yezzi, A., Mennucci, A.: Sobolev active contours. International Journal of Computer Vision 73(3), 345–366 (2007)CrossRefzbMATHGoogle Scholar
  28. 28.
    Chan, T., Shen, J.: The role of the bv image model in image restoration. AMS Contemporary Mathematics (2002)Google Scholar
  29. 29.
    Chambolle, A.: An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision 20(1-2), 89–97 (2004)MathSciNetGoogle Scholar
  30. 30.
    Baker, S., Roth, S., Scharstein, D., Black, M., Lewis, J., Szeliski, R.: A database and evaluation methodology for optical flow. In: Proceedings of the International Conference on Computer Vision, pp. 1–8 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Benjamin Berkels
    • 1
  • Claudia Kondermann
    • 2
  • Christoph Garbe
    • 2
  • Martin Rumpf
    • 1
  1. 1.Institute for Numerical SimulationUniversität BonnBonnGermany
  2. 2.IWRUniversität HeidelbergHeidelbergGermany

Personalised recommendations

Cite paper

Buy options