Advertisement

International Journal of Computer Vision

, Volume 7, Issue 2, pp 95–117 | Cite as

Subspace methods for recovering rigid motion I: Algorithm and implementation

  • David J. Heeger
  • Allan D. Jepson
Article

Abstract

As an observer moves and explores the environment, the visual stimulation in his/her eye is constantly changing. Somehow he/she is able to perceive the spatial layout of the scene, and to discern his/her movement through space. Computational vision researchers have been trying to solve this problem for a number of years with only limited success. It is a difficult problem to solve because the optical flow field is nonlinearly related to the 3D motion and depth parameters.

Here, we show that the nonlinear equation describing the optical flow field can be split by an exact algebraic manipulation to form three sets of equations. The first set relates the flow field to only the translational component of 3D motion. Thus, depth and rotation need not be known or estimated prior to solving for translation. Once the translation has been recovered, the second set of equations can be used to solve for rotation. Finally, depth can be estimated with the third set of equations, given the recovered translation and rotation.

The algorithm applies to the general case of arbitrary motion with respect to an arbitrary scene. It is simple to compute, and it is plausible biologically. The results reported in this article demonstrate the potential of our new approach, and show that it performs favorably when compared with two other well-known algorithms.

Keywords

Flow Field Computer Vision Computer Image Nonlinear Equation Difficult Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adelson, E.H., and Bergen, J.R., 1986. The extraction of spatiotemporal energy in human and machine vision. Proc. IEEE Workshop on Motion: Representation and Analysis, Charleston, S. Carolina, pp. 151–156.Google Scholar
  2. AdivG., 1985. Determining three-dimensional motion and structure from optical flow generated by several moving objects. IEEE Trans. Patt. Anal. Mach. Intell. 7:384–401.Google Scholar
  3. AdivG., 1989. Inherent ambiguities in recovering 3D motion and structure from a noisy flow field. IEEE Trans. Anal. Mach. Intell. 11:477–489.Google Scholar
  4. AggarwalJ.K. and NandhakumarN., 1988. On the computation of motion from sequences of images—a review. Proc. IEEE. 76: 917–935.Google Scholar
  5. AnandanP., 1989. A computational framework and an algorithm for the measurement of visual motion. Intern. J. Comput. Vision. 2:283–310.Google Scholar
  6. BallardD.H. and KimballO.A., 1983. Rigid body motion from depth and optical flow. Comput. Vision, Graph. Image Process 22:95–115.Google Scholar
  7. Barron, J., 1984. A survey of approaches for determining optic flow, environmental layout and egomotion. Techn. Rept. RBCV-TR-84-5, Department of Computer Science, University of Toronto.Google Scholar
  8. BarronJ.L., JepsonA.D., and TsotsosJ.K., 1990. The feasibility of motion and structure from noisy time-varying image velocity information. Intern. J. Comput. Vision 5:239–269.Google Scholar
  9. BollesR.C., BakerH.H., and MarimontD.H., 1987. Epipolarplane image analysis: An approach to determining structure from motion. Intern. J. Comput. Vision 1:7–55.Google Scholar
  10. BroidaT.J., and ChellappaR., 1986. Estimation of object motion parameters from noisy images. IEEE Trans. Patt. Anal. Mach. Intell. 8:90–99.Google Scholar
  11. BrussA.R., and HornB.K.P., 1983. Passive navigation. Comput. Vision, Graph. Image Process. 21:3–20.Google Scholar
  12. Faugeras, O.D., Lustman, F., and Toscani, G., 1987. Motion and structure from motion from point and line matches. Proc. 1st Intern. Conf. Comput. Vision, London, June, pp. 25–34.Google Scholar
  13. FleetD.J. and JepsonA.D., 1990. Computation of component image velocity from local phase information. Intern. J. Comput. Vision. 5:77–104.Google Scholar
  14. GibsonJ.J. 1950. The Perception of the Visual World. Houghton Mifflin, Boston.Google Scholar
  15. GibsonJ.J., and GibsonE.J., 1957. Continuous perspective transformation and the perception of rigid motions. J. Exp. Psychol. 54: 129–138.Google Scholar
  16. GrzywaczN.M., and YuilleA.L., 1990. A model for the estimate of local image velocity by cells in the visual cortex. Proc. Roy. Soc. London A, 239:129–161.Google Scholar
  17. HayJ.C., 1966. Optical motions and space perception: An extension of Gibson's analysis. Psychological Review, 73:550–565.Google Scholar
  18. HeegerD.J., 1987. Model for the extraction of image flow. J. Opt. Soc. Amer. A 4:1455–1471.Google Scholar
  19. HeegerD.J., 1988. Optical flow using spatiotemporal filters. Intern. J. Comput. Vision 1:279–302.Google Scholar
  20. HeegerD.J., and JepsonA., 1990a. Visual perception of three-dimensional motion. Neural Computation 2:129–137.Google Scholar
  21. HeegerD.J., and JepsonA., 1990b. Visual perception of 3D motion and depth. Invest. Opthal. Vis. Sci. Suppl. 31:173.Google Scholar
  22. Heeger, D.J., and Jepson, A., 1990c. Simple method for computing 3D motion and depth. Proc. 3rd. Intern. Conf. Comput. Vision, Osaka, Japan, December, pp. 96–100.Google Scholar
  23. HeegerD.J., and JepsonA., 1991. Recovering observer translation with center-surround motion-opponent mechanisms. Invest. Opthal. Vis. Sci. Suppl. 32:823.Google Scholar
  24. Heel, J., 1989a. Direct estimation of structure and motion for multiple frames. Tech. Rep. 1190, MIT AI Lab.Google Scholar
  25. Heel, J., 1989b. Dynamic motion vision. Proc. SPIE. Philadelphia.Google Scholar
  26. Heel, J., 1990. Direct dynamic motion vision. Proc. IEEE Conf. Robot. Autom. Cincinnati.Google Scholar
  27. HornB.K.P., 1986. Robot Vision. MIT Press: Cambridge, Ma.Google Scholar
  28. HornB.K.P., 1987. Motion fields are hardly ever ambiguous. Intern. J. Comput. Vision, 1:259–274.Google Scholar
  29. HornB.K.P., and NegahdaripourS., 1987. Direct passive navigation: Analytical solution for planes. IEEE Trans. Patt. Anal. Mach. Intell. 9:168–176.Google Scholar
  30. HornB.K.P., and SchunkB.G., 1981. Determining optical flow. Artificial Intelligence 17:185–203.Google Scholar
  31. HornB.K.P., and WeldonE.J., 1988. Direct methods for recovering motion. Intern. J. Comput. Vision 2:51–76.Google Scholar
  32. Jepson, A., and Heeger, D.J., 1989. Egomotion without depth estimation. Optics News 15:A-20.Google Scholar
  33. Jepson, A., and Heeger, D.J., 1990. Subspace methods for recovering rigid motion II: Theory Submitted to International Journal of Computer Vision, available as Tech. Rept. RBCV-TR-90-36, Department of Computer Science, University of Toronto.Google Scholar
  34. Jepson, A. and Heeger, D.J. 1991. A fast subspace algorithm for recovering rigid motion. Proc. IEEE Workshop on Visual Motion, Princeton, N.J., pp. 124–131.Google Scholar
  35. JohanssonG., 1975. Visual motion perception. Scientific American 232: 76–88.Google Scholar
  36. KoenderinkJ.J. and vanDornA.J. 1975. Invariant properties of the motion parallax field due to the movement of rigid bodies relative to an observer. Optica Acta 22: 773–791.Google Scholar
  37. KoenderinkJ.J., and vanDornA.J. 1976. Local structure of movement parallax of the plane. J. Opt. Soc. Amer. 66:717–723.Google Scholar
  38. KoenderinkJ.J., and vanDornA.J. 1981. Exterospecific component of the motion parallax field. J. Opt. Soc. Amer. 71:953–957.Google Scholar
  39. Longuet-HigginsH.C., 1981. A computer algorithm for reconstructing a scene from two projections. Nature 293:133–135.Google Scholar
  40. Longuet-HigginsH.C., 1984. The visual ambiguity of a moving plane. Proc. Roy. Soc. London B 223:165–175.Google Scholar
  41. Longuet-HigginsH.C., 1988. Multiple interpretations of a pair of images of a surface. Proc. Roy. Soc. London B 418:1–15.Google Scholar
  42. Longuet-Higgins, and PrazdnyK. 1980. The interpretation of a moving retinal image. Proc. Roy. Soc. London B 208:385–397.Google Scholar
  43. Lucas, B.D., and Kanade, T., 1981. An iterative image registration technique with an application to stereo vision. Proc. 7th Intern. Joint Conf. Artif. Intell. Vancouver, pp. 674–679.Google Scholar
  44. MaloneyL.T., and WandellB.A., 1986. Color constancy: a method for recovering surface spectral reflectance. J. Opt. Soc. Amer. A 1:29–33.Google Scholar
  45. MatthiesL., SzeliskiR., and KanadeT., 1989. Kalman filter-based algorithms for estimating depth from image sequences. Intern. J. Comput. Vision. 3:209–238.Google Scholar
  46. MaybankS.J., 1985. The angular velocity associated with the optical flow field arising from motion through a rigid environment. Proc. Roy. Soc. London A 410:317–326.Google Scholar
  47. Maybank, S.J., 1987. A Theoretical Study of Optical flow. Ph.D. thesis, University of London.Google Scholar
  48. NagelH.H., 1987. On the estimation of optical flow: relations between different approaches and some new results. Artificial Intelligence 33:299–324.Google Scholar
  49. NakayamaK., 1985. Biological image motion processing: A review. Vision Research 25:625–660.Google Scholar
  50. NegahdaripourS., and HornB.K.P., 1989. A direct method for locating the focus of expansion. Comput. Vision, Graph. Image Process. 46:303–326.Google Scholar
  51. PrazdnyK., 1980. Egomotion and relative depth from optical flow, Biological Cybermetics 36:87–102.Google Scholar
  52. PrazdnyK., 1981. Determining the instantaneous direction of motion from optical flow generated by a curvilinearly moving observer. Comput. Graph. Image Process. 17:238–248.Google Scholar
  53. PrazdnyK., 1983. On the information in optical flows. Comput. Graph. Image Process. 22:239–259.Google Scholar
  54. ReigerJ.H., and LawtonD.T., 1985. Processing differential image motion. J. Opt. Soc. Amer. A. 2:354–359.Google Scholar
  55. RoachJ.W., and AggarwalJ.K., 1980. Determining the movement of objects from a sequence of images. IEEE Trans. Patt. Anal. Mach. Intell. 2:554–562.Google Scholar
  56. SimoncelliE.P., and AdelsonE.H., 1991. Relationship between gradient, spatio-temporal energy, and regression models for motion perception. Invest. Opthal. Vis. Sci. Suppl. 32:893.Google Scholar
  57. Simoncelli, E.R., Adelson, E.H., and Heeger, D.J., 1991. Probability distributions of optical flow. Proc. Comput. Vision Patt. Recog., Maui, HI, June, pp. 310–315.Google Scholar
  58. Southall, J.P.C., editor, 1962. Helmholtz's Treatise on Physiological Optics. Dover Publications: NY Originally published by the Optical Society of America in 1925.Google Scholar
  59. StrangG. 1980. Linear Algebra and Its Applications. Academic Press: New York.Google Scholar
  60. Sundareswaran, V. 1991. Egomotion from global flow field data. Proc. IEEE Workshop on Visual Motion, Princeton, N.J., pp. 140–145.Google Scholar
  61. TsaiR.Y., and HuangT.S., 1984. Uniqueness and estimation of three-dimensional motion parameters of rigid objects with curved surfaces. IEEE, Trans. Patt. Anal. Mach. Intell. 6:13–27.Google Scholar
  62. UllmanS., 1979. The Interpretation of Visual Motion. MIT Press: Cambridge, MAGoogle Scholar
  63. UllmanS., 1984. Maximizing rigidity: the incremental recovery of 3-D structure from rigid and rubbery motion. Perception 13:255–274.Google Scholar
  64. WallachH., and O'ConnellD.N., 1953. The kinetic depth effect. J. Exp. Psychol. 45:205–217.Google Scholar
  65. WandellB.A., 1987. The synthesis and analysis of color images. IEEE Trans. Patt. Anal. Mach. Intell. 9:2–13.Google Scholar
  66. WarrenW.H., and HannonD.J., 1988. Direction of self-motion is perceived from optical flow. Nature 336:162–163.Google Scholar
  67. WarrenW.H., and HannonD.J., 1990. Eye movements and optical flow. J. Opt. Soc. Amer. A 7:160–169.Google Scholar
  68. WatsonA.B., and AhumadaA.J., 1985. Model of human visualmotion sensing. J. Opt. Soc. Amer. A 2:322–342.Google Scholar
  69. WaxmanA.M., and UllmanS. 1985. Surface structure and three-dimensional motion from image flow kinematics. Intern. J. Robot. Res. 4:72–94.Google Scholar
  70. WaxmanA.M., and WohnK., 1985. Contour evolution, neighborhood deformation, and global image flow: planar surfaces in motion. Intern J. Robot. Res. 4:95–108.Google Scholar
  71. WaxmanA.M., and WohnK., 1988. Image flow theory: A framework for 3-D inference from time-varying imagery. In Advances in Computer Vision. vol. 1, pp. 165–224. Lawrence Erlbaum Assoc.: Hillsdale, NJ.Google Scholar
  72. WaxmanA.M., Kamgar-ParsiB., and SubbaraoM., 1987. Closed-form solutions to image flow equations and 3D structure and motion. Intern. J. Comput. Vision 1:239–258.Google Scholar
  73. WengJ., HuangT.S., and AhujaN., 1989. Motion and structure from two perspective views: Algorithms, error analysis, and error estimation. IEEE Trans. Patt. Anal. Mach. Intell. 11:451–476.Google Scholar

Copyright information

© Kluwer Academic Publishers 1992

Authors and Affiliations

  • David J. Heeger
    • 1
    • 2
  • Allan D. Jepson
    • 3
  1. 1.NASA-Ames Research CenterMoffett Field
  2. 2.Psychology DepartmentStanford UniversityStanford
  3. 3.Computer Science DepartmentUniversity of TorontoToronto

Personalised recommendations