International Journal of Computer Vision

, Volume 12, Issue 1, pp 43–77 | Cite as

Performance of optical flow techniques

  • J. L. Barron
  • D. J. Fleet
  • S. S. Beauchemin
Systems And Experiment

Abstract

While different optical flow techniques continue to appear, there has been a lack of quantitative evaluation of existing methods. For a common set of real and synthetic image sequences, we report the results of a number of regularly cited optical flow techniques, including instances of differential, matching, energy-based, and phase-based methods. Our comparisons are primarily empirical, and concentrate on the accuracy, reliability, and density of the velocity measurements; they show that performance can differ significantly among the techniques we implemented.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adelson, E.H., and Bergen, J.R. 1985. Spatiotemporal energy models for the perception of motion,J. Opt. Soc. Amer. A 2: 284–299.Google Scholar
  2. Adelson, E.H., and Bergen, J.R. 1986. The extraction of spatiotemporal energy in human and machine vision,Proc. IEEE Workshop on Visual Motion, Charleston, pp. 151–156.Google Scholar
  3. Aloimonos, J., and Brown, C.M. 1984. Direct processing of curvilinear sensor motion from a sequence of perspective images.Proc. 2nd Workshop on Computer Vision, Annapolis, pp. 72–77.Google Scholar
  4. Aloimonos, Y., and Duric, Z. 1992. Active egomotion estimation: a qualitative approach,Proc. Europ. Conf. Comput. Vis., Ligure, Italy, pp. 497–510.Google Scholar
  5. Anandan, P. 1987. Measuring Visual Motion from Image Sequences. Ph.D. dissertation, COINS TR 87-21, Univ. of Massachusetts, Amherst, MA.Google Scholar
  6. Anandan, P. 1989. A computational framework and an algorithm for the measurement of visual motion,Intern. J. Comput. Vis. 2: 283–310.Google Scholar
  7. Barman, H., Haglund, L., Knutsson, H., and Granlund, G. 1991. Estimation of velocity, acceleration and disparity in time sequences,Proc. IEEE Workshop on Visual Motion, Princeton, pp. 44–51.Google Scholar
  8. Barron, J.L., Fleet, D.J., and Beauchemin, S.S. 1993. Performance of optical flow techniques, Tech. Rept. TR299, Dept. of Computer Science, University of Western Ontario; and RPL-TR-9107, Dept. of Computing Science, Queens University, July 1992 (revised July 1993).Google Scholar
  9. Barron, J.L., Fleet, D.J., Beauchemin, SS., and Burkitt, T. 1992. Performance of optical flow techniques,Proc. Conf. Comp. Vis. Patt. Recog., Champaign, June, pp. 236–242.Google Scholar
  10. Barron, J.L., Jepson, A.D., and Tsotsos, J.K. 1990. The feasibility of motion and structure from noisy time-varying image velocity information,Intern. J. Comput. Vis. 5: 239–269.Google Scholar
  11. Beaudet, P.R. 1978. Rotationally invariant image operators.Proc. 4th Intern. Conf. Patt. Recog., Tokyo, pp. 579–583.Google Scholar
  12. Bigun, J., Granlund, G., and Wiklund, J. 1991. Multidimensional orientation estimation with applications to texture analysis and optical flow.IEEE Trans. Patt. Anal. Mach. Intell. 13: 775–790.Google Scholar
  13. Burt, P.J., and Adelson, E.H. 1983. The Laplacian pyramid as a compact image code,IEEE Trans. Communications 31, pp. 532–540.Google Scholar
  14. Burt, P.J., Yen, C., and Xu, X. 1983. Multiresolution flow-through motion analysis,Proc. Conf. Comput. Vis. Patt. Recog., Washington, pp. 246–252.Google Scholar
  15. Buxton, B., and Buxton, H. 1984. Computation of optic flow from the motion of edge features in image sequences,Image Vis. Comput. 2: 59–74.Google Scholar
  16. Cippola, R., and Blake, A. 1992. Surface orientation and time to contact from image divergence and deformation,Proc. 2nd Europ. Conf. Comput. Vis., Ligure, Italy, pp. 187–202.Google Scholar
  17. Duncan, J.H., and Chou, T.C. 1988. Temporal edges: The detection of motion and the computation of optical flow,Proc. 2nd Intern. Conf. Comput. Vis., Tampa, pp. 374–382.Google Scholar
  18. Dutta, R., Manmatha, R., Williams, L., and Riseman, E.M. 1989. A data set for quantitative motion analysis,Proc. Conf. Comput. Vis. Patt. Recog., San Diego, pp. 159–164.Google Scholar
  19. Fennema, C., and Thompson, W. 1979. Velocity determination in scenes containing several moving objects,Comput. Graph. Image Process. 9: 301–315.Google Scholar
  20. Fleet, D.J. 1992.Measurement of Image Velocity. Kluwer Academic Publishers: Norwell, MA.Google Scholar
  21. Fleet, D.J., and Jepson, A.D. 1990. Computation of component image velocity from local phase information,Intern. J. Comput. Vis. 5: 77–104.Google Scholar
  22. Fleet, D.J., and Jepson, A.D. 1993. Stability of phase information,IEEE Trans. Patt. Anal. Mach. Intell. (in press).Google Scholar
  23. Fleet, D.J., and Langley, K. 1993. Toward real-time optical flow,Proc. Vision Interface, Toronto, pp. 116–124 (also see Tech. Rept.: RPL-TR-9308, Robotics and Perception Laboratory, Queen's University).Google Scholar
  24. Girosi, F., Verri, A., and Torre, V. 1989. Constraints for the computation of optical flow,Proc. IEEE Workshop on Visual Motion, Irvine, pp. 116–124.Google Scholar
  25. Glazer, F., Reynolds, G., and Anandan, P. 1983. Scene matching through hierarchical correlation,Proc. Conf. Comput. Vis. Patt. Recog., Washington, pp. 432–441.Google Scholar
  26. Grzywacz, N.M., and Yuille, A.L. 1990. A model for the estimation of local image velocity by cells in the visual cortex,Proc. Roy. Soc. London B 239: 129–161.Google Scholar
  27. Haglund, L. 1992. Adaptive Multidimensional Filtering. Ph.D. dissertation, Dept. Electrical Engineering, Univ. of Linkoping (ISSN 0345-7524).Google Scholar
  28. Heeger, D.J. 1987. Model for the extraction of image flow,J. Opt. Soc. Amer. A 4: 1455–1471.Google Scholar
  29. Heeger, D.J. 1988. Optical flow using spatiotemporal filters,Intern. J. Comput. Vis. 1: 279–302.Google Scholar
  30. Hildreth, E.C. 1984. The computation of the velocity field,Proc. Roy. Soc. London B 221: 189–220.Google Scholar
  31. Horn, B.K.P. 1986.Robot Vision. MIT Press: Cambridge, MA.Google Scholar
  32. Horn, B.K.P., and Schunck, B.G. 1981. Determining optical flow,Artificial Intelligence 17: 185–204.Google Scholar
  33. Horn, B.K.P., and Weldon, Jr., E.J. 1988. Direct methods for recovering motion,Intern. J. Comput. Vis. 2: 51–76.Google Scholar
  34. Jahne, B. 1987. Image sequence analysis of complex physical objects: nonlinear small scale water surface waves,Proc. 1st Intern. Conf. Comput. Vis., London, pp. 191–200.Google Scholar
  35. Jepson, A.D., and Fleet, D.J. 1991. Phase singularities in scale-space,Image Vis. Comput. 9: 338–343.Google Scholar
  36. Jepson, A.D., and Heeger, D.J. 1990. Subspace methods for recovering rigid motion II: Theory, Tech. Rept. RBCV-TR-90-36, Dept. of Computer Science, University of Western Ontario. (To appear inInt. J. Comput. Vis.).Google Scholar
  37. Kearney, J.K., Thompson, W.B., and Boley, D.L. 1987. Optical flow estimation: An error analysis of gradient-based methods with local optimization,IEEE Trans. Patt. Anal. Artif. Mach. Intell. 9:229–244.Google Scholar
  38. Little, J.J., and Verri, A. 1989. Analysis of differential and matching methods for optical flow,IEEE Workshop on Visual Motion, Irvine, CA, pp. 173–180.Google Scholar
  39. Little, J.J., Bulthoff, H.H., and Poggio, T.A. 1988. Parallel optical flow using local voting.Proc. 2nd Intern. Conf. Comput. Vis., Tampa, pp. 454–459.Google Scholar
  40. Lucas, B.D. 1984. Generalized Image Matching by the Method of Differences. Ph.D. dissertation, Dept. of Computer Science, Carnegie-Mellon University.Google Scholar
  41. Lucas, B., and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision.Proc. DARPA Image Understanding Workshop, pp. 121–130.Google Scholar
  42. Marr, D., and Hildreth, E.C. 1980. Theory of edge detection,Proc. Roy. Soc. London, B 207: 187–217.Google Scholar
  43. Nagel, H.H. 1983. Displacement vectors derived from second-order intensity variations in image sequences,Comput. Graph. Image Process. 21: 85–117.Google Scholar
  44. Nagel, H.-H. 1987. On the estimation of optical flow: Relations between different approaches and some new results,Artificial Intelligence 33: 299–324.Google Scholar
  45. Nagel, H.-H. 1989. On a constraint equation for the estimation of displacement rates in image sequences,IEEE Trans. Patt. Anal. Mach. Intell. 11: 13–30.Google Scholar
  46. Nagel, H.H., and Enkelmann, W. 1986. An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences.IEEE Trans. Patt. Anal. Mach. Intell. 8: 565–593.Google Scholar
  47. Negahdaripour, S., and Horn, B.K.P. 1987. Direct passive navigation,IEEE Trans. Patt. Anal. Mach. Intell. 9: 168–176.Google Scholar
  48. Santen, J.P.H. van, and Sperling, G. 1985. Elaborated Reichardt detectors,J. Opt. Soc. Amer. A 2: 300–321.Google Scholar
  49. Schunck, B.G. 1984. The motion constraint equation for optical flow,Proc. 7th Intern. Conf. Patt. Recog., Montreal, pp. 20–22.Google Scholar
  50. Schunck, B.G. 1986. Image flow continuity equations for motion and density,Proc. IEEE Workshop on Visual Motion, Charleston, pp. 89–94.Google Scholar
  51. Simoncelli, E.P. 1993. Distributed Representation and Analysis of Visual Motion. Ph.D. dissertation, Dept. of Electrical Engineering and Computer Science, MIT.Google Scholar
  52. Simoncelli, E.P., Adelson, E.H., and Heeger, D.J. 1991. Probability distributions of optical flow.Proc. Conf. Comput. Vis. Patt. Recog., Maui, pp. 310–315.Google Scholar
  53. Singh, A. 1990. An estimation-theoretic framework for image-flow computation,Proc. 3rd Intern. Conf. Comput. Vis., Osaka, pp. 168–177.Google Scholar
  54. Singh, A. 1992.Optic Flow Computation: A Unified Perspective. IEEE Computer Society Press.Google Scholar
  55. Tistarelli, M., and Sandini, G. 1990. Estimation of depth from motion using anthropomorphic visual sensor,Image Vis. Comput. 8: 271–278.Google Scholar
  56. Tretiak, O., and Pastor, L. 1984. Velocity estimation from image sequences with second order differential operators,Proc. 7th Intern. Conf. Patt. Recog., Montreal, pp. 20–22.Google Scholar
  57. Uras, S., Girosi, F., Verri, A., and Torre, V. 1988. A computational approach to motion perception,Biol. Cybern. 60: 79–97.Google Scholar
  58. Verri, A., and Poggio, T. 1987. Against quantitative optical flow,Proc. 1st Intern. Conf. Comput. Vis., London, pp. 171–180.Google Scholar
  59. Watson, A.B., and Ahumada, A.J. 1985. Model of human visual-motion sensing,J. Opt. Soc. Amer. A 2: 322–342.Google Scholar
  60. Waxman, A.M., and Wohn, K. 1985. Contour evolution, neighbourhood deformation and global image flow: Planar surfaces in motion, Intern.J. Robotics Res. 4: 95–108.Google Scholar
  61. Waxman, A.M., Wu, J., and Bergholm, F. 1988. Convected activation profiles and receptive fields for real time measurement of short range visual motion,Proc. Conf. Comput. Vis. Patt. Recog., Ann Arbor, pp. 771–723.Google Scholar
  62. Willick, D., and Yang, Y.H. 1991. Experimental evaluation of motion constraints equations.Comput. Vis. Graph. Image Process.: Image Understanding 54: 206–214.Google Scholar

Copyright information

© Kluwer Academic Publishers 1994

Authors and Affiliations

  • J. L. Barron
    • 1
  • D. J. Fleet
    • 2
  • S. S. Beauchemin
    • 1
  1. 1.Department of Computer ScienceUniversity of Western OntarioLondon
  2. 2.Department of Computing ScienceQueen's UniversityKingston

Personalised recommendations