Abstract
Assumptions of brightness constancy and spatial smoothness underlie most optical flow estimation methods. In contrast to standard heuristic formulations, we learn a statistical model of both brightness constancy error and the spatial properties of optical flow using image sequences with associated ground truth flow fields. The result is a complete probabilistic model of optical flow. Specifically, the ground truth enables us to model how the assumption of brightness constancy is violated in naturalistic sequences, resulting in a probabilistic model of “brightness inconstancy”. We also generalize previous high-order constancy assumptions, such as gradient constancy, by modeling the constancy of responses to various linear filters in a high-order random field framework. These filters are free variables that can be learned from training data. Additionally we study the spatial structure of the optical flow and how motion boundaries are related to image intensity boundaries. Spatial smoothness is modeled using a Steerable Random Field, where spatial derivatives of the optical flow are steered by the image brightness structure. These models provide a statistical motivation for previous methods and enable the learning of all parameters from training data. All proposed models are quantitatively compared on the Middlebury flow dataset.
Keywords
- Ground Truth
- Optical Flow
- Data Term
- Spatial Term
- Brightness Constancy
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download conference paper PDF
References
Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. CVIU 63, 75–104 (1996)
Roth, S., Black, M.J.: On the spatial statistics of optical flow. IJCV 74, 33–50 (2007)
Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. In: ICCV (2007)
Nagel, H.H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE TPAMI 8, 565–593 (1986)
Roth, S., Black, M.J.: Steerable random fields. In: ICCV (2007)
Wainwright, M.J., Simoncelli, E.P.: Scale mixtures of Gaussians and the statistics of natural images. In: NIPS, pp. 855–861 (1999)
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)
Bruhn, A., Weickert, J., Schnörr, C.: Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. IJCV 61, 211–231 (2005)
Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 16, 185–203 (1981)
Fermüller, C., Shulman, D., Aloimonos, Y.: The statistics of optical flow. CVIU 82, 1–32 (2001)
Gennert, M.A., Negahdaripour, S.: Relaxing the brightness constancy assumption in computing optical flow. Technical report, Cambridge, MA, USA (1987)
Haussecker, H., Fleet, D.: Computing optical flow with physical models of brightness variation. IEEE TPAMI 23, 661–673 (2001)
Toth, D., Aach, T., Metzler, V.: Illumination-invariant change detection. In: 4th IEEE Southwest Symposium on Image Analysis and Interpretation, pp. 3–7 (2000)
Adelson, E.H., Anderson, C.H., Bergen, J.R., Burt, P.J., Ogden, J.M.: Pyramid methods in image processing. RCA Engineer 29, 33–41 (1984)
Alvarez, L., Deriche, R., Papadopoulo, T., Sanchez, J.: Symmetrical dense optical flow estimation with occlusions detection. IJCV 75, 371–385 (2007)
Fleet, D.J., Black, M.J., Nestares, O.: Bayesian inference of visual motion boundaries. In: Exploring Artificial Intelligence in the New Millennium, pp. 139–174. Morgan Kaufmann Pub., San Francisco (2002)
Black, M.J.: Combining intensity and motion for incremental segmentation and tracking over long image sequences. In: Sandini, G. (ed.) ECCV 1992. LNCS, vol. 588, pp. 485–493. Springer, Heidelberg (1992)
Simoncelli, E.P., Adelson, E.H., Heeger, D.J.: Probability distributions of optical flow. In: CVPR, pp. 310–315 (1991)
Black, M.J., Yacoob, Y., Jepson, A.D., Fleet, D.J.: Learning parameterized models of image motion. In: CVPR, pp. 561–567 (1997)
Freeman, W.T., Pasztor, E.C., Carmichael, O.T.: Learning low-level vision. IJCV 40, 25–47 (2000)
Scharstein, D., Pal, C.: Learning conditional random fields for stereo. In: CVPR (2007)
Roth, S., Black, M.J.: Fields of experts: A framework for learning image priors. In: CVPR, vol. II, pp. 860–867 (2005)
Stewart, L., He, X., Zemel, R.: Learning flexible features for conditional random fields. IEEE TPAMI 30, 1145–1426 (2008)
Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Comput 14, 1771–1800 (2002)
Blake, A., Zisserman, A.: Visual Reconstruction. The MIT Press, Cambridge, Massachusetts (1987)
Zhu, S., Wu, Y., Mumford, D.: Filters random fields and maximum entropy (FRAME): To a unified theory for texture modeling. IJCV 27, 107–126 (1998)
Lempitsky, V., Roth, S., Rother, C.: FusionFlow: Discrete-continuous optimization for optical flow estimation. In: CVPR (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sun, D., Roth, S., Lewis, J.P., Black, M.J. (2008). Learning Optical Flow. In: Forsyth, D., Torr, P., Zisserman, A. (eds) Computer Vision – ECCV 2008. ECCV 2008. Lecture Notes in Computer Science, vol 5304. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88690-7_7
Download citation
DOI: https://doi.org/10.1007/978-3-540-88690-7_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88689-1
Online ISBN: 978-3-540-88690-7
eBook Packages: Computer ScienceComputer Science (R0)