Learning for Optical Flow Using Stochastic Optimization

  • Yunpeng Li
  • Daniel P. Huttenlocher
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5303)


We present a technique for learning the parameters of a continuous-state Markov random field (MRF) model of optical flow, by minimizing the training loss for a set of ground-truth images using simultaneous perturbation stochastic approximation (SPSA). The use of SPSA to directly minimize the training loss offers several advantages over most previous work on learning MRF models for low-level vision, which instead seek to maximize the likelihood of the data given the model parameters. In particular, our approach explicitly optimizes the error criterion used to evaluate the quality of the flow field, naturally handles missing data values in the ground truth, and does not require the kinds of approximations that current methods use to address the intractable nature of maximum-likelihood estimation for such problems. We show that our method achieves state-of-the-art results and requires only a very small number of training images. We also find that our method generalizes well to unseen data, including data with quite different characteristics than the training set.


  1. 1.
    Nagel, H.H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. PAMI 8(5), 565–593 (1986)CrossRefGoogle Scholar
  2. 2.
    Anandan, P.: A computational framework and an algorithm for the measurement of visual motion. IJCV 2(3), 283–310 (1989)CrossRefGoogle Scholar
  3. 3.
    Bergen, J.R., Anandan, P., Hanna, K.J., Hingorani, R.: Hierarchical model-based motion estimation. In: ECCV, London, UK, pp. 237–252. Springer, Heidelberg (1992)Google Scholar
  4. 4.
    Cohen, I.: Nonlinear variational method for optical flow computation. In: Scandinavian Conference on Image Analysis (1993)Google Scholar
  5. 5.
    Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. CVIU 63(1), 75–104 (1996)Google Scholar
  6. 6.
    Szeliski, R., Coughlan, J.: Spline-based image registration. IJCV 22(3) (1997)Google Scholar
  7. 7.
    Alvarez, L., Weickert, J., Sanchez, J.: Reliable estimation of dense optical flow fields with large displacements. International Journal of Computer Vision 39(1), 41–56 (2000)CrossRefzbMATHGoogle Scholar
  8. 8.
    Ye, M., Haralick, R.M., Shapiro, L.G.: Estimating piecewise-smooth optical flow with global matching and graduated optimization. PAMI 25(12), 1625–1630 (2003)CrossRefGoogle Scholar
  9. 9.
    Bruhn, A., Weickert, J., Schnörr, C.: Lucas/kanade meets horn/schunck: combining local and global optic flow methods. IJCV 61(3), 211–231 (2005)CrossRefGoogle Scholar
  10. 10.
    Roth, S., Black, M.J.: On the spatial statistics of optical flow. IJCV 74(1), 33–50 (2007)CrossRefGoogle Scholar
  11. 11.
    Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. In: ICCV (2007)Google Scholar
  12. 12.
    Besag, J.E.: Spatial interaction and the statistical analysis of lattice systems. J. Royal Stat. Soc., B 36(2) (1974)Google Scholar
  13. 13.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. PAMI 6(6), 721–741 (1984)CrossRefzbMATHGoogle Scholar
  14. 14.
    Szeliski, R.: Bayesian modeling of uncertainty in low-level vision. IJCV 5(3) (1990)Google Scholar
  15. 15.
    Spall, J.C.: Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Transactions on Automatic Control 37, 332–341 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Otte, M., Nagel, H.H.: Optical flow estimation: Advances and comparisons. In: ECCV, Secaucus, NJ, USA, pp. 51–60. Springer, New York (1994)Google Scholar
  17. 17.
    Scharstein, D., Pal, C.: Learning conditional random fields for stereo. CVPR (2007)Google Scholar
  18. 18.
    Sutton, C., McCallum, A.: Piecewise training of undirected models. In: UAI (2005)Google Scholar
  19. 19.
    LeCun, Y., Huang, F.J.: Loss functions for discriminative training of energy-based models. In: AIStats (2005)Google Scholar
  20. 20.
    Tappen, M.F.: Utilizing variational optimization to learn Markov random fields. CVPR (2007)Google Scholar
  21. 21.
    Spall, J.C.: Overview of the simultaneous perturbation method for efficient optimization. Hopkins APL Technical Digest 19, 482–492 (1998)Google Scholar
  22. 22.
    Bertero, M., Poggio, T., Torre, V.: Ill-posed problems in early vision. In: Proceedings of IEEE (1988)Google Scholar
  23. 23.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: IJCAI, pp. 674–679 (1981)Google Scholar
  24. 24.
    Jepson, A., Black, M.J.: Mixture models for optical flow computation. CVPR, 760–761 (1993)Google Scholar
  25. 25.
    Horn, B.K.P., Schunk, B.G.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)CrossRefGoogle Scholar
  26. 26.
    Geman, S., Graffigne, C.: Markov random field image models and their applications to computer vision. In: Intl. Congress of Mathematicians (1986)Google Scholar
  27. 27.
    Simoncelli, E.P., Adelson, E.H., Heeger, D.J.: Probability distributions of optical flow. In: CVPR, Mauii, Hawaii, pp. 310–315. IEEE Computer Society, Los Alamitos (1991)Google Scholar
  28. 28.
    Fleet, D.J., Black, M.J., Yacoob, Y., Jepson, A.D.: Design and use of linear models for image motion analysis. IJCV 36(3), 171–193 (2000)CrossRefGoogle Scholar
  29. 29.
    Roth, S., Black, M.J.: Fields of experts: A Framework for Learning Image Priors. CVPR (2005)Google Scholar
  30. 30.
    Kiefer, J., Wolfowitz, J.: Stochastic estimation of the maximum of a regression function. Annuals of Mathematical Statistics 23(3), 462–466 (1952)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Weiss, Y., Freeman, W.F.: What makes a good model of natural images? CVPR (2007)Google Scholar
  32. 32.
    Nocedal, J.: Updating quasi-newton matrices with limited storage. Mathematics of Computation 35, 773–782 (1980)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Mathematical Programming 45(3, (Ser. B)), 503–528 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Spall, J.C.: Implementation of the simultaneous perturbation algorithm for stochastic optimization. IEEE Trans. Aerosp. Electon. Syst. 34(3), 817–823 (1998)CrossRefGoogle Scholar
  35. 35.
    Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. IJCV 12(1), 43–77 (1994)CrossRefGoogle Scholar
  36. 36.
    Fleet, D.J., Jepson, A.D.: Computation of component image velocity from local phase information. IJCV 5(1), 77–104 (1990)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Yunpeng Li
    • 1
  • Daniel P. Huttenlocher
    • 1
  1. 1.Department of Computer ScienceCornell UniversityIthaca

Personalised recommendations