International Journal of Computer Vision

, Volume 14, Issue 1, pp 67–81 | Cite as

Robust computation of optical flow in a multi-scale differential framework

  • Joseph Weber
  • Jitendra Malik


We have developed a new algorithm for computing optical flow in a differential framework. The image sequence is first convolved with a set of linear, separable spatiotemporal filter kernels similar to those that have been used in other early vision problems such as texture and stereopsis. The brightness constancy constraint can then be applied to each of the resulting images, giving us, in general, an overdetermined system of equations for the optical flow at each pixel. There are three principal sources of error: (a) stochastic error due to sensor noise (b) systematic errors in the presence of large displacements and (c) errors due to failure of the brightness constancy model. Our analysis of these errors leads us to develop an algorithm based on a robust version of total least squares. Each optical flow vector computed has an associated reliability measure which can be used in subsequent processing. The performance of the algorithm on the data set used by Barron et al. (IJCV 1994) compares favorably with other techniques. In addition to being separable, the filters used are also causal, incorporating only past time frames. The algorithm is fully parallel and has been implemented on a multiple processor machine.


Optical Flow Constancy Model Vision Problem Robust Computation Flow Vector 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Barron, J., Fleet, D., and Beauchemin, S. 1993. “ Performance of optical flow techniques,” Tech. Rep. RPL-TR-9107, Queen's University, Ontario. Revised version of U. Western Ontario TR 299.Google Scholar
  2. Barron, J., Fleet, D., Beauchemin, S., and Burkitt, T. 1994. “Performance of optical flow techniques,”International Journal of Computer Vision, vol. 12, pp. 43–77.Google Scholar
  3. Fleet, D. and Jepson, A. 1990. “Computation of component image velocity from local phase information,”International Journal of Computer Vision, vol. 5, pp. 77–104.Google Scholar
  4. Bergen, J., and Adelson, E. 1988. “Early vision and texture perception,”Nature, vol. 333, pp. 363–364.Google Scholar
  5. Canny, J. 1986. “A computational approach to edge detection,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 8, pp. 679–698.Google Scholar
  6. Jones, D. and Malik, J. 1992. “Computational framework for determining stereo correspondence from a set of linear spatial filters,”Image and Vision Computing, vol. 10, no. 10, pp. 699–708.Google Scholar
  7. Jones, D. and Malik, J. 1992. “Determining three-dimensional shape from orientation and spatial frequency disparities,” inProceedings of the Second European Conference on Computer Vision, pp. 661–669.Google Scholar
  8. Malik, J. and Perona, P. 1990. “Preattentive texture discrimination with early vision mechanisms,” Journal of the Optical Society of America A, vol. 7, no. 5, pp. 923–932.Google Scholar
  9. Turner, M. 1986. “Texture discrimination by Gabor functions,”Biological Cybernetics, vol. 55, pp. 71–82.Google Scholar
  10. Heeger, D.J. 1988. “Optical flow using spatiotemporal filters,”International Journal of Computer Vision, vol. 1, pp. 279–302.Google Scholar
  11. Fennema, C. and Thompson, W. 1979. “Velocity determination in scenes containing several moving objects,”Computer Graphics and Image Processing, vol. 9, pp. 301–315.Google Scholar
  12. Horn, B. and Schunck, B. 1981. “Determining optical flow,”Artificial Intelligence, no. 17, pp. 185–203.Google Scholar
  13. Tretiak, O. and Pastor, L. 1984. “Velocity estimation from image sequences with second order differential operators,” inProceedings of the International Conference on Pattern Recognition, (Montreal).Google Scholar
  14. Nagel, H.-H. 1987. “On the estimation of optical flow: relations between different approaches and some new results,”Artificial Intelligence, vol. 33, pp. 299–324.Google Scholar
  15. Uras, S., Girosi, F., Verri, A. and Torre, V. 1988. “A computational approach to motion perception,”Biological Cybernetics, vol. 60, pp. 79–87.Google Scholar
  16. Verri, A., Girosi, F. and Torre, V. 1990. “Differential techniques for optical flow,”Journal of the Optical Society of America A, vol. 5, pp. 912–922.Google Scholar
  17. Srinivasan, M. 1990. “Generalized gradient schemes for the measurement of two-dimensional image motion,”Biological Cybernetics, vol. 63, pp. 421–431.Google Scholar
  18. Lucas, B. and Kanade, T. 1981. “An iterative image restoration technique with an application to stereo vision,” inProceedings of the DARPA IU Workshop, pp. 121–130.Google Scholar
  19. Campani, M. and Verri, A. 1990. “Computing optical flow from an overconstrained system of linear algebraic equations,” inProceedings of the 3rd International Conference on Computer Vision, (Osaka), pp. 22–26.Google Scholar
  20. Wang, S., Markandey, V. and Reid, A. 1992. “Total least squares fitting spatiotemporal derivatives to smooth optical flow fields,” inProceedings of the SPIE: Signal and Data Processing of Small Targets, vol. 1698, pp. 42–55.Google Scholar
  21. VanHuffel, S. and Vandewalle, J. 1991.The Total Least Squares Problem: Computational Aspects and Analysis. Frontiers in Applied Mathematics, Philadelphia: SIAM.Google Scholar
  22. Duda, R. and Hart, P. 1973.Pattern Classification and Scene Analysis, New York, Chichester, Brisbane, Toronto, Singapore: John Wiley & Sons.Google Scholar
  23. Pearson, K. 1901. “On lines and planes of closest fit to points in space,”Philos. Mag., vol. 2, pp. 559–572.Google Scholar
  24. Madansky, A. 1959. “The fitting of straight lines when both variables are subject to error,”/. Amer. Statist. Assoc., vol. 54, pp. 173–205.Google Scholar
  25. Sprent, P. 1969. Models in Regression and Related Topics. London: Methuen.Google Scholar
  26. Golub, G. and VanLoan, C. 1980. “An analysis of the total least squares problem,”SIAM Journal Numer. Anal., vol. 17, pp. 883–893.Google Scholar
  27. Shizawa, M. and Mase, K. 1990. “Simultaneous multiple optical flow estimation,” inProceedings of the 10th International Conference on Pattern Recognition (Atlantic City, New Jersey), pp. 274–278.Google Scholar
  28. Simoncelli, E., Adelson, E. and Heeger, D. 1991. “Probability distributions of optical flow,” inProceedings of the IEEE Computer Vision and Pattern Recognition Conference, pp. 310–315.Google Scholar
  29. Cleary, R. and Braddick, O. 1990. “Directional discrimination for band-pass filtered random dot kinematograms,”Vision Research, vol. 30, pp. 303–316.Google Scholar
  30. Battiti, R., Amaldi, E. and Koch, C. 1991. “Computing optical flow across multiple scales: An adaptive coarse-to-fine strategy,“International Journal of Computer Vision, vol. 6, no. 2, pp. 133–145.Google Scholar
  31. Young, R. 1985. “The gaussian derivative theory of spatial vision: Analysis of cortical cell receptive field line-weighting profiles,” Technical Report GMR-4920, General Motors Research.Google Scholar
  32. Black, M.J. and Anandan, P. 1993. “A framework for the robust estimation of optical flow,” inProceedings of the Fourth ICCV, (Berlin), pp. 231–236.Google Scholar
  33. Huber, P.J. 1981.Robust Statistics. Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons.Google Scholar
  34. Weber, J. and Malik, J. 1992. “Robust computation of optical flow in a multi-scale differential framework,” Tech. Rep. UCB/CSD 92/709, Computer Science Division (EECS), University of California, Berkeley.Google Scholar

Copyright information

© Kluwer Academic Publishers 1995

Authors and Affiliations

  • Joseph Weber
    • 1
  • Jitendra Malik
    • 1
  1. 1.Department of Electrical Engineering and Computer ScienceUniversity of California at BerkeleyBerkeley

Personalised recommendations