International Journal of Computer Vision

, Volume 7, Issue 2, pp 95–117

# Subspace methods for recovering rigid motion I: Algorithm and implementation

• David J. Heeger
• Allan D. Jepson
Article

## Abstract

As an observer moves and explores the environment, the visual stimulation in his/her eye is constantly changing. Somehow he/she is able to perceive the spatial layout of the scene, and to discern his/her movement through space. Computational vision researchers have been trying to solve this problem for a number of years with only limited success. It is a difficult problem to solve because the optical flow field is nonlinearly related to the 3D motion and depth parameters.

Here, we show that the nonlinear equation describing the optical flow field can be split by an exact algebraic manipulation to form three sets of equations. The first set relates the flow field to only the translational component of 3D motion. Thus, depth and rotation need not be known or estimated prior to solving for translation. Once the translation has been recovered, the second set of equations can be used to solve for rotation. Finally, depth can be estimated with the third set of equations, given the recovered translation and rotation.

The algorithm applies to the general case of arbitrary motion with respect to an arbitrary scene. It is simple to compute, and it is plausible biologically. The results reported in this article demonstrate the potential of our new approach, and show that it performs favorably when compared with two other well-known algorithms.

## Keywords

Flow Field Computer Vision Computer Image Nonlinear Equation Difficult Problem
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

## References

1. Adelson, E.H., and Bergen, J.R., 1986. The extraction of spatiotemporal energy in human and machine vision. Proc. IEEE Workshop on Motion: Representation and Analysis, Charleston, S. Carolina, pp. 151–156.Google Scholar
2. AdivG., 1985. Determining three-dimensional motion and structure from optical flow generated by several moving objects. IEEE Trans. Patt. Anal. Mach. Intell. 7:384–401.Google Scholar
3. AdivG., 1989. Inherent ambiguities in recovering 3D motion and structure from a noisy flow field. IEEE Trans. Anal. Mach. Intell. 11:477–489.Google Scholar
4. AggarwalJ.K. and NandhakumarN., 1988. On the computation of motion from sequences of images—a review. Proc. IEEE. 76: 917–935.Google Scholar
5. AnandanP., 1989. A computational framework and an algorithm for the measurement of visual motion. Intern. J. Comput. Vision. 2:283–310.Google Scholar
6. BallardD.H. and KimballO.A., 1983. Rigid body motion from depth and optical flow. Comput. Vision, Graph. Image Process 22:95–115.Google Scholar
7. Barron, J., 1984. A survey of approaches for determining optic flow, environmental layout and egomotion. Techn. Rept. RBCV-TR-84-5, Department of Computer Science, University of Toronto.Google Scholar
8. BarronJ.L., JepsonA.D., and TsotsosJ.K., 1990. The feasibility of motion and structure from noisy time-varying image velocity information. Intern. J. Comput. Vision 5:239–269.Google Scholar
9. BollesR.C., BakerH.H., and MarimontD.H., 1987. Epipolarplane image analysis: An approach to determining structure from motion. Intern. J. Comput. Vision 1:7–55.Google Scholar
10. BroidaT.J., and ChellappaR., 1986. Estimation of object motion parameters from noisy images. IEEE Trans. Patt. Anal. Mach. Intell. 8:90–99.Google Scholar
11. BrussA.R., and HornB.K.P., 1983. Passive navigation. Comput. Vision, Graph. Image Process. 21:3–20.Google Scholar
12. Faugeras, O.D., Lustman, F., and Toscani, G., 1987. Motion and structure from motion from point and line matches. Proc. 1st Intern. Conf. Comput. Vision, London, June, pp. 25–34.Google Scholar
13. FleetD.J. and JepsonA.D., 1990. Computation of component image velocity from local phase information. Intern. J. Comput. Vision. 5:77–104.Google Scholar
14. GibsonJ.J. 1950. The Perception of the Visual World. Houghton Mifflin, Boston.Google Scholar
15. GibsonJ.J., and GibsonE.J., 1957. Continuous perspective transformation and the perception of rigid motions. J. Exp. Psychol. 54: 129–138.Google Scholar
16. GrzywaczN.M., and YuilleA.L., 1990. A model for the estimate of local image velocity by cells in the visual cortex. Proc. Roy. Soc. London A, 239:129–161.Google Scholar
17. HayJ.C., 1966. Optical motions and space perception: An extension of Gibson's analysis. Psychological Review, 73:550–565.Google Scholar
18. HeegerD.J., 1987. Model for the extraction of image flow. J. Opt. Soc. Amer. A 4:1455–1471.Google Scholar
19. HeegerD.J., 1988. Optical flow using spatiotemporal filters. Intern. J. Comput. Vision 1:279–302.Google Scholar
20. HeegerD.J., and JepsonA., 1990a. Visual perception of three-dimensional motion. Neural Computation 2:129–137.Google Scholar
21. HeegerD.J., and JepsonA., 1990b. Visual perception of 3D motion and depth. Invest. Opthal. Vis. Sci. Suppl. 31:173.Google Scholar
22. Heeger, D.J., and Jepson, A., 1990c. Simple method for computing 3D motion and depth. Proc. 3rd. Intern. Conf. Comput. Vision, Osaka, Japan, December, pp. 96–100.Google Scholar
23. HeegerD.J., and JepsonA., 1991. Recovering observer translation with center-surround motion-opponent mechanisms. Invest. Opthal. Vis. Sci. Suppl. 32:823.Google Scholar
24. Heel, J., 1989a. Direct estimation of structure and motion for multiple frames. Tech. Rep. 1190, MIT AI Lab.Google Scholar
26. Heel, J., 1990. Direct dynamic motion vision. Proc. IEEE Conf. Robot. Autom. Cincinnati.Google Scholar
27. HornB.K.P., 1986. Robot Vision. MIT Press: Cambridge, Ma.Google Scholar
28. HornB.K.P., 1987. Motion fields are hardly ever ambiguous. Intern. J. Comput. Vision, 1:259–274.Google Scholar
29. HornB.K.P., and NegahdaripourS., 1987. Direct passive navigation: Analytical solution for planes. IEEE Trans. Patt. Anal. Mach. Intell. 9:168–176.Google Scholar
30. HornB.K.P., and SchunkB.G., 1981. Determining optical flow. Artificial Intelligence 17:185–203.Google Scholar
31. HornB.K.P., and WeldonE.J., 1988. Direct methods for recovering motion. Intern. J. Comput. Vision 2:51–76.Google Scholar
32. Jepson, A., and Heeger, D.J., 1989. Egomotion without depth estimation. Optics News 15:A-20.Google Scholar
33. Jepson, A., and Heeger, D.J., 1990. Subspace methods for recovering rigid motion II: Theory Submitted to International Journal of Computer Vision, available as Tech. Rept. RBCV-TR-90-36, Department of Computer Science, University of Toronto.Google Scholar
34. Jepson, A. and Heeger, D.J. 1991. A fast subspace algorithm for recovering rigid motion. Proc. IEEE Workshop on Visual Motion, Princeton, N.J., pp. 124–131.Google Scholar
35. JohanssonG., 1975. Visual motion perception. Scientific American 232: 76–88.Google Scholar
36. KoenderinkJ.J. and vanDornA.J. 1975. Invariant properties of the motion parallax field due to the movement of rigid bodies relative to an observer. Optica Acta 22: 773–791.Google Scholar
37. KoenderinkJ.J., and vanDornA.J. 1976. Local structure of movement parallax of the plane. J. Opt. Soc. Amer. 66:717–723.Google Scholar
38. KoenderinkJ.J., and vanDornA.J. 1981. Exterospecific component of the motion parallax field. J. Opt. Soc. Amer. 71:953–957.Google Scholar
39. Longuet-HigginsH.C., 1981. A computer algorithm for reconstructing a scene from two projections. Nature 293:133–135.Google Scholar
40. Longuet-HigginsH.C., 1984. The visual ambiguity of a moving plane. Proc. Roy. Soc. London B 223:165–175.Google Scholar
41. Longuet-HigginsH.C., 1988. Multiple interpretations of a pair of images of a surface. Proc. Roy. Soc. London B 418:1–15.Google Scholar
42. Longuet-Higgins, and PrazdnyK. 1980. The interpretation of a moving retinal image. Proc. Roy. Soc. London B 208:385–397.Google Scholar
43. Lucas, B.D., and Kanade, T., 1981. An iterative image registration technique with an application to stereo vision. Proc. 7th Intern. Joint Conf. Artif. Intell. Vancouver, pp. 674–679.Google Scholar
44. MaloneyL.T., and WandellB.A., 1986. Color constancy: a method for recovering surface spectral reflectance. J. Opt. Soc. Amer. A 1:29–33.Google Scholar
45. MatthiesL., SzeliskiR., and KanadeT., 1989. Kalman filter-based algorithms for estimating depth from image sequences. Intern. J. Comput. Vision. 3:209–238.Google Scholar
46. MaybankS.J., 1985. The angular velocity associated with the optical flow field arising from motion through a rigid environment. Proc. Roy. Soc. London A 410:317–326.Google Scholar
47. Maybank, S.J., 1987. A Theoretical Study of Optical flow. Ph.D. thesis, University of London.Google Scholar
48. NagelH.H., 1987. On the estimation of optical flow: relations between different approaches and some new results. Artificial Intelligence 33:299–324.Google Scholar
49. NakayamaK., 1985. Biological image motion processing: A review. Vision Research 25:625–660.Google Scholar
50. NegahdaripourS., and HornB.K.P., 1989. A direct method for locating the focus of expansion. Comput. Vision, Graph. Image Process. 46:303–326.Google Scholar
51. PrazdnyK., 1980. Egomotion and relative depth from optical flow, Biological Cybermetics 36:87–102.Google Scholar
52. PrazdnyK., 1981. Determining the instantaneous direction of motion from optical flow generated by a curvilinearly moving observer. Comput. Graph. Image Process. 17:238–248.Google Scholar
53. PrazdnyK., 1983. On the information in optical flows. Comput. Graph. Image Process. 22:239–259.Google Scholar
54. ReigerJ.H., and LawtonD.T., 1985. Processing differential image motion. J. Opt. Soc. Amer. A. 2:354–359.Google Scholar
55. RoachJ.W., and AggarwalJ.K., 1980. Determining the movement of objects from a sequence of images. IEEE Trans. Patt. Anal. Mach. Intell. 2:554–562.Google Scholar
56. SimoncelliE.P., and AdelsonE.H., 1991. Relationship between gradient, spatio-temporal energy, and regression models for motion perception. Invest. Opthal. Vis. Sci. Suppl. 32:893.Google Scholar
57. Simoncelli, E.R., Adelson, E.H., and Heeger, D.J., 1991. Probability distributions of optical flow. Proc. Comput. Vision Patt. Recog., Maui, HI, June, pp. 310–315.Google Scholar
58. Southall, J.P.C., editor, 1962. Helmholtz's Treatise on Physiological Optics. Dover Publications: NY Originally published by the Optical Society of America in 1925.Google Scholar
59. StrangG. 1980. Linear Algebra and Its Applications. Academic Press: New York.Google Scholar
60. Sundareswaran, V. 1991. Egomotion from global flow field data. Proc. IEEE Workshop on Visual Motion, Princeton, N.J., pp. 140–145.Google Scholar
61. TsaiR.Y., and HuangT.S., 1984. Uniqueness and estimation of three-dimensional motion parameters of rigid objects with curved surfaces. IEEE, Trans. Patt. Anal. Mach. Intell. 6:13–27.Google Scholar
62. UllmanS., 1979. The Interpretation of Visual Motion. MIT Press: Cambridge, MAGoogle Scholar
63. UllmanS., 1984. Maximizing rigidity: the incremental recovery of 3-D structure from rigid and rubbery motion. Perception 13:255–274.Google Scholar
64. WallachH., and O'ConnellD.N., 1953. The kinetic depth effect. J. Exp. Psychol. 45:205–217.Google Scholar
65. WandellB.A., 1987. The synthesis and analysis of color images. IEEE Trans. Patt. Anal. Mach. Intell. 9:2–13.Google Scholar
66. WarrenW.H., and HannonD.J., 1988. Direction of self-motion is perceived from optical flow. Nature 336:162–163.Google Scholar
67. WarrenW.H., and HannonD.J., 1990. Eye movements and optical flow. J. Opt. Soc. Amer. A 7:160–169.Google Scholar
68. WatsonA.B., and AhumadaA.J., 1985. Model of human visualmotion sensing. J. Opt. Soc. Amer. A 2:322–342.Google Scholar
69. WaxmanA.M., and UllmanS. 1985. Surface structure and three-dimensional motion from image flow kinematics. Intern. J. Robot. Res. 4:72–94.Google Scholar
70. WaxmanA.M., and WohnK., 1985. Contour evolution, neighborhood deformation, and global image flow: planar surfaces in motion. Intern J. Robot. Res. 4:95–108.Google Scholar
71. WaxmanA.M., and WohnK., 1988. Image flow theory: A framework for 3-D inference from time-varying imagery. In Advances in Computer Vision. vol. 1, pp. 165–224. Lawrence Erlbaum Assoc.: Hillsdale, NJ.Google Scholar
72. WaxmanA.M., Kamgar-ParsiB., and SubbaraoM., 1987. Closed-form solutions to image flow equations and 3D structure and motion. Intern. J. Comput. Vision 1:239–258.Google Scholar
73. WengJ., HuangT.S., and AhujaN., 1989. Motion and structure from two perspective views: Algorithms, error analysis, and error estimation. IEEE Trans. Patt. Anal. Mach. Intell. 11:451–476.Google Scholar

## Authors and Affiliations

• David J. Heeger
• 1
• 2
• Allan D. Jepson
• 3
1. 1.NASA-Ames Research CenterMoffett Field
2. 2.Psychology DepartmentStanford UniversityStanford
3. 3.Computer Science DepartmentUniversity of TorontoToronto