Skip to main content
SpringerLink
Log in

Reconstructing Optical Flow Fields by Motion Inpainting

  • Conference paper
Book cover Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR 2009)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 5681))

Abstract

An edge-sensitive variational approach for the restoration of optical flow fields is presented. Real world optical flow fields are frequently corrupted by noise, reflection artifacts or missing local information. Still, applications may require dense motion fields. In this paper, we pick up image inpainting methodology to restore motion fields, which have been extracted from image sequences based on a statistical hypothesis test on neighboring flow vectors. A motion field inpainting model is presented, which takes into account additional information from the image sequence to improve the reconstruction result. The underlying functional directly combines motion and image information and allows to control the impact of image edges on the motion field reconstruction. In fact, in case of jumps of the motion field, where the jump set coincides with an edge set of the underlying image intensity, an anisotropic TV-type functional acts as a prior in the inpainting model. We compare the resulting image guided motion inpainting algorithm to diffusion and standard TV inpainting methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bruhn, A., Weickert, J.: In: A Confidence Measure for Variational Optic Flow Methods, pp. 283–298. Springer, Heidelberg (2006)

    Google Scholar 

  2. Kondermann, C., Kondermann, D., Jähne, B., Garbe, C.S.: An adaptive confidence measure for optical flows based on linear subspace projections. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 132–141. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  3. Kondermann, C., Mester, R., Garbe, C.: A statistical confidence measure for optical flows. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 290–301. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  4. Masnou, S., Morel, J.: Level lines based disocclusion. In: Proceedings of the ICIP 1998, vol. 3, pp. 259–263 (1998)

    Google Scholar 

  5. Chan, T.F., Shen, J.: Mathematical models for local nontexture inpaintings. SIAM J. Appl. Math. 62, 1019–1043 (2001)

    MathSciNet  MATH  Google Scholar 

  6. Chan, T.F., Shen, J.: Non-texture inpainting by curvature-driven diffusions. J. Visual Comm. Image Rep. 12, 436–449 (2001)

    Article  Google Scholar 

  7. Telea, A.: An image inpainting technique based on the fast marching method. Journal of graphics tools 9(1), 25–36 (2003)

    Google Scholar 

  8. Bornemann, F., März, T.: Fast image inpainting based on coherence transport. Journal of Mathematical Imaging and Vision 28(3), 259–278 (2007)

    Article  MathSciNet  Google Scholar 

  9. Bertalmio, M., Bertozzi, A., Sapiro, G.: Navier-stokes, fluid dynamics, and image and video inpainting. In: IEEE Proceedings of the International Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 355–362 (2001)

    Google Scholar 

  10. Chan, T., Shen, J.: Mathematical models for local non-texture inpaintings. SIAM J. Appl. Math. 62(3), 1019–1043 (2002)

    Article  MATH  Google Scholar 

  11. Nitzberg, M., Shiota, T., Mumford, D.: Filtering, Segmentation and Depth. LNCS, vol. 662. Springer, Heidelberg (1993)

    Book  MATH  Google Scholar 

  12. Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and gray levels. IEEE Transactions on Image Processing 10(8), 1200–1211 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  13. Ambrosio, L., Masnou, S.: A direct variational approach to a problem arising in image reconstruction. Interfaces and Free Boundaries 5, 63–81 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  14. Chan, T.F., Shen, J., Zhou, H.M.: Total variation wavelet inpainting. Journal of Mathematical Imaging and Vision 25(1), 107–125 (2006)

    Article  MathSciNet  Google Scholar 

  15. Patwardhan, K.A., Sapiro, G., Bertalmio, M.: Video inpainting of occluding and occluded objects. IMA Preprint Series 2016 (January 2005)

    Google Scholar 

  16. Matsushita, Y., Ofek, E., Weina, G., Tang, X., Shum, H.: Full-frame video stabilization with motion inpainting. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(7), 1150–1163 (2006)

    Article  Google Scholar 

  17. Chen, L., Chan, S., Shum, H.: A joint motion-image inpainting method for error concealment in video coding. In: IEEE International Conference on Image Processing, ICIP (2006)

    Google Scholar 

  18. Kondermann, C., Kondermann, D., Garbe, C.: Postprocessing of optical flows via surface measures and motion inpainting. In: Rigoll, G. (ed.) DAGM 2008. LNCS, vol. 5096, pp. 355–364. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  19. Ambrosio, L., Fusco, N., Pallara, D.: Functions of bounded variation and free discontinuity problems. Oxford Mathematical Monographs. Oxford University Press, New York (2000)

    MATH  Google Scholar 

  20. Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence 12(7), 629–639 (1990)

    Article  Google Scholar 

  21. Nagel, H.H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 8(5), 565–593 (1986)

    Article  Google Scholar 

  22. Alvarez, L., Monreal, L.J.E., Lefebure, M., Perez, J.S.: A pde model for computing the optical flow. In: Proc. XVI Congreso de Ecuaciones Diferenciales Aplicaciones, Universidad de Las Palmas de Gran Canaria, September 1999, pp. 1349–1356 (1999)

    Google Scholar 

  23. Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Transactions on Image Processing 10(2), 266–277 (2001)

    Article  MATH  Google Scholar 

  24. Braess, D.: Finite Elemente, 2nd edn. Springer, Heidelberg (1997); Theorie, schnelle Löser und Anwendungen in der Elastizitätstheorie

    Book  MATH  Google Scholar 

  25. Schaback, R., Werner, H.: Numerische Mathematik, 4th edn. Springer, Berlin (1992)

    Book  Google Scholar 

  26. Kosmol, P.: Methoden zur numerischen Behandlung nichtlinearer Gleichungen und Optimierungsaufgaben. 2nd edn. Teubner, Stuttgart (1993)

    Google Scholar 

  27. Sundaramoorthi, G., Yezzi, A., Mennucci, A.: Sobolev active contours. International Journal of Computer Vision 73(3), 345–366 (2007)

    Article  MATH  Google Scholar 

  28. Chan, T., Shen, J.: The role of the bv image model in image restoration. AMS Contemporary Mathematics (2002)

    Google Scholar 

  29. Chambolle, A.: An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision 20(1-2), 89–97 (2004)

    MathSciNet  Google Scholar 

  30. Baker, S., Roth, S., Scharstein, D., Black, M., Lewis, J., Szeliski, R.: A database and evaluation methodology for optical flow. In: Proceedings of the International Conference on Computer Vision, pp. 1–8 (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Numerical Simulation, Universität Bonn, Endenicher Allee 60, 53115, Bonn, Germany

    Benjamin Berkels & Martin Rumpf

  2. IWR, Universität Heidelberg, Im Neuenheimer Feld 368, 69120, Heidelberg, Germany

    Claudia Kondermann & Christoph Garbe

Authors
  1. Benjamin Berkels
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Claudia Kondermann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christoph Garbe
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Martin Rumpf
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institut für Informatik III, Rö, Universität Bonn, merstraße 164, 53117, Bonn, Germany

    Daniel Cremers

  2. Computer Science Department, University of Western Ontario, N6A 5A5, London, ON, Canada

    Yuri Boykov

  3. Microsoft Research, J.J. Thomson Avenue, CB3 OFB, Cambridge, United Kingdom

    Andrew Blake

  4. Institut für Informatik III, Universität Bonn, Römerstraße 164, 53117, Bonn, Germany

    Frank R. Schmidt

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Berkels, B., Kondermann, C., Garbe, C., Rumpf, M. (2009). Reconstructing Optical Flow Fields by Motion Inpainting. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds) Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2009. Lecture Notes in Computer Science, vol 5681. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03641-5_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-03641-5_29

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-03640-8

  • Online ISBN: 978-3-642-03641-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation