Optical Flow Estimation with Channel Constancy

  • Laura Sevilla-Lara
  • Deqing Sun
  • Erik G. Learned-Miller
  • Michael J. Black
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8689)

Abstract

Large motions remain a challenge for current optical flow algorithms. Traditionally, large motions are addressed using multi-resolution representations like Gaussian pyramids. To deal with large displacements, many pyramid levels are needed and, if an object is small, it may be invisible at the highest levels. To address this we decompose images using a channel representation (CR) and replace the standard brightness constancy assumption with a descriptor constancy assumption. CRs can be seen as an over-segmentation of the scene into layers based on some image feature. If the appearance of a foreground object differs from the background then its descriptor will be different and they will be represented in different layers. We create a pyramid by smoothing these layers, without mixing foreground and background or losing small objects. Our method estimates more accurate flow than the baseline on the MPI-Sintel benchmark, especially for fast motions and near motion boundaries.

Keywords

Optical flow channel representation pyramids large motions 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. IJCV 92(1) (March 2011), http://dx.doi.org/10.1007/s11263-010-0390-2
  2. 2.
    Berg, A.C., Malik, J.: Geometric blur for template matching. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 1. IEEE, pp. I–607 (2001)Google Scholar
  3. 3.
    Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding 63(1), 75–104 (1996)CrossRefGoogle Scholar
  4. 4.
    Brox, T., Malik, J.: Large displacement optical flow: Descriptor matching in variational motion estimation. PAMI 33(3) (2011), http://lmb.informatik.uni-freiburg.de//Publications/2011/Bro11a
  5. 5.
    Bruhn, A., Weickert, J., Schnörr, C.: Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. IJCV 61(3), 211–231 (2005)CrossRefGoogle Scholar
  6. 6.
    Burt, P.J., Adelson, E.H.: The Laplacian pyramid as a compact image code. IEEE Transactions on Communications 31(4), 532–540 (1983)CrossRefGoogle Scholar
  7. 7.
    Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  8. 8.
    Charbonnier, P., Blanc-Feraud, L., Aubert, G., Barlaud, M.: Two deterministic half-quadratic regularization algorithms for computed imaging. In: IEEE Int. Conf. Image Proc. (ICIP), vol. 2, pp. 168–172 (1994)Google Scholar
  9. 9.
    Felsberg, M.: Spatio-featural scale-space. In: Tai, X.-C., Mørken, K., Lysaker, M., Lie, K.-A. (eds.) SSVM 2009. LNCS, vol. 5567, pp. 808–819. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  10. 10.
    Felsberg, M.: Adaptive filtering using channel representations. In: Mathematical Methods for Signal and Image Analysis and Representation, pp. 31–48. Springer (2012)Google Scholar
  11. 11.
    Felsberg, M., Forssén, P.E., Scharr, H.: Channel smoothing: Efficient robust smoothing of low-level signal features. PAMI 28(2), 209–222 (2006)CrossRefGoogle Scholar
  12. 12.
    Granlund, G.H.: An associative perception-action structure using a localized space variant information representation. In: Sommer, G., Zeevi, Y.Y. (eds.) AFPAC 2000. LNCS, vol. 1888, pp. 48–68. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  13. 13.
    Haussecker, H.W., Fleet, D.J.: Computing optical flow with physical models of brightness variation. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 661–673 (2001), http://dx.doi.org/10.1109/34.927465 CrossRefGoogle Scholar
  14. 14.
    Horn, B.K., Schunck, B.G.: Determining optical flow. Tech. rep., Massachusetts Institute of Technology, Cambridge, MA, USA (1980)Google Scholar
  15. 15.
    Jonsson, E., Felsberg, M.: Accurate interpolation in appearance-based pose estimation. In: Ersbøll, B.K., Pedersen, K.S. (eds.) SCIA 2007. LNCS, vol. 4522, pp. 1–10. Springer, Heidelberg (2007), http://dx.doi.org/10.1007/978-3-540-73040-8_1 CrossRefGoogle Scholar
  16. 16.
    Jonsson, E., Felsberg, M.: Efficient computation of channel-coded feature maps through piecewise polynomials. Image and Vision Computing 27(11) (2009)Google Scholar
  17. 17.
    Koenderink, J.J., Van Doorn, A.J.: The structure of locally orderless images. International Journal of Computer Vision 31(2-3), 159–168 (1999)CrossRefGoogle Scholar
  18. 18.
    Liu, C., Yuen, J., Torralba, A., Sivic, J., Freeman, W.T.: SIFT flow: Dense correspondence across different scenes. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 28–42. Springer, Heidelberg (2008), http://dx.doi.org/10.1007/978-3-540-88690-7_3 CrossRefGoogle Scholar
  19. 19.
    Mears, B., Sevilla-Lara, L., Learned-Miller, E.: Distribution fields with adaptive kernels for large displacement image alignment. In: BMVC. IEEE (2013)Google Scholar
  20. 20.
    Nordberg, K., Granlund, G., Knutsson, H.: Representation and Learning of Invariance. Report LiTH-ISY-I-1552, Computer Vision Laboratory, SE-581 83 Linköping, Sweden (1994)Google Scholar
  21. 21.
    Oron, S., Bar-Hillel, A., Levi, D., Avidan, S.: Locally orderless tracking. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1940–1947. IEEE (2012)Google Scholar
  22. 22.
    Sevilla-Lara, L., Learned-Miller, E.: Distribution fields. Tech. rep., UMass Amherst (2011)Google Scholar
  23. 23.
    Sevilla-Lara, L., Learned-Miller, E.: Distribution fields for tracking. In: CVPR (2012)Google Scholar
  24. 24.
    Snippe, H.P., Koenderink, J.J.: Discrimination thresholds for channel-coded systems. Biological Cybernetics 66(6), 543–551 (1992)CrossRefMATHGoogle Scholar
  25. 25.
    Steinbrucker, F., Pock, T., Cremers, D.: Large displacement optical flow computation without warping. In: ICCV (2009)Google Scholar
  26. 26.
    Steinbruecker, F., Pock, T., Cremers, D.: Advanced data terms for variational optic flow estimation. In: Proceedings Vision, Modeling and Visualization (2009)Google Scholar
  27. 27.
    Sun, D., Roth, S., Black, M.J.: A quantitative analysis of current practices in optical flow estimation and the principles behind them. International Journal of Computer Vision (IJCV) 106(2), 115–137 (2014)CrossRefGoogle Scholar
  28. 28.
    Sun, D., Roth, S., Lewis, J.P., Black, M.J.: Learning optical flow. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 83–97. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  29. 29.
    van Ginneken, B., ter Haar Romeny, B.M.: Applications of locally orderless images. In: Nielsen, M., Johansen, P., Fogh Olsen, O., Weickert, J. (eds.) Scale-Space 1999. LNCS, vol. 1682, pp. 10–21. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  30. 30.
    Weber, J., Malik, J., Devadas, S., Michel, P.: Robust computation of optical flow in a multi-scale differential framework. IJCV 14 (1994)Google Scholar
  31. 31.
    Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: Deepflow: Large displacement optical flow with deep matching. In: ICCV, pp. 1385–1392 (2013)Google Scholar
  32. 32.
    Werlberger, M.: Convex Approaches for High Performance Video Processing. Ph.D. thesis, Institute for Computer Graphics and Vision, Graz University of Technology, Graz, Austria (June 2012), http://gpu4vision.icg.tugraz.at/papers/2012/werlberger_phd.pdf

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Laura Sevilla-Lara
    • 1
  • Deqing Sun
    • 2
  • Erik G. Learned-Miller
    • 1
  • Michael J. Black
    • 3
  1. 1.School of Computer ScienceUniversity of MassachusettsAmherstUSA
  2. 2.School of Engineering and Applied SciencesHarvard UniversityCambridgeUSA
  3. 3.Max Planck Institute for Intelligent SystemsTübingenGermany

Personalised recommendations