International Journal of Computer Vision

, Volume 44, Issue 3, pp 219–249 | Cite as

Optimization Criteria and Geometric Algorithms for Motion and Structure Estimation

  • Yi Ma
  • Jana Košecká
  • Shankar Sastry


Prevailing efforts to study the standard formulation of motion and structure recovery have recently been focused on issues of sensitivity and robustness of existing techniques. While many cogent observations have been made and verified experimentally, many statements do not hold in general settings and make a comparison of existing techniques difficult. With an ultimate goal of clarifying these issues, we study the main aspects of motion and structure recovery: the choice of objective function, optimization techniques and sensitivity and robustness issues in the presence of noise.

We clearly reveal the relationship among different objective functions, such as “(normalized) epipolar constraints,” “reprojection error” or “triangulation,” all of which can be unified in a new “optimal triangulation” procedure. Regardless of various choices of the objective function, the optimization problems all inherit the same unknown parameter space, the so-called “essential manifold.” Based on recent developments of optimization techniques on Riemannian manifolds, in particular on Stiefel or Grassmann manifolds, we propose a Riemannian Newton algorithm to solve the motion and structure recovery problem, making use of the natural differential geometric structure of the essential manifold.

We provide a clear account of sensitivity and robustness of the proposed linear and nonlinear optimization techniques and study the analytical and practical equivalence of different objective functions. The geometric characterization of critical points and the simulation results clarify the difference between the effect of bas-relief ambiguity, rotation and translation confounding and other types of local minima. This leads to consistent interpretations of simulation results over a large range of signal-to-noise ratio and variety of configurations.

motion and structure recovery optimal triangulation essential manifold Riemannian Newton's algorithm Stiefel manifold 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Adiv, G. 1989. Inherent ambiguities in recovering 3-D motion and structure from a noisy flow field. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(5):477-489.Google Scholar
  2. Boothby, W.M. 1986. An Introduction to Differential Manifolds and Riemannian Geometry. 2nd edn. Academic Press: San Diego.Google Scholar
  3. Danilidis, K. 1997. Visual Navigation. Lawrence Erlbaum Associates.Google Scholar
  4. Danilidis, K. and Nagel, H.-H. 1990. Analytical results on error sensitivity of motion estimation from two views. Image and Vision Computing, 8:297-303.Google Scholar
  5. Edelman, A., Arias, T., and Smith, S.T. 1998. The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Analysis Applications, 20(2):303-353.Google Scholar
  6. Faugeras, O. 1993. Three-dimensional Computer Vision: A Geometric Viewpoint. The MIT Press: Cambridge, MA, USA.Google Scholar
  7. Hartley, R. and Sturm, P. 1997. Triangulation. Computer Vision and Image Understanding, 68(2):146-157.Google Scholar
  8. Horn, B. 1990. Relative orientation. International Journal of Computer Vision, 4:59-78.Google Scholar
  9. Jepson, A.D. and Heeger, D.J. 1993. Spatial Vision in Humans and Robots. Cambridge University Press: Cambridge. pp. 39-62.Google Scholar
  10. Kanatani, K. 1993. Geometric Computation for Machine Vision. Oxford Science Publications: Oxford.Google Scholar
  11. Kobayashi, S. and Nomizu, T. 1996. Foundations of Differential Geometry: Volume I. John Wiley & Sons, Inc.: New York.Google Scholar
  12. Longuet-Higgins, H.C. 1981. A computer algorithm for reconstructing a scene from two projections. Nature, 293:133-135.Google Scholar
  13. Luong, Q.-T. and Faugeras, O. 1996. The fundamental matrix: Theory, algorithms, and stability analysis. International Journal of Computer Vision, 17(1):43-75.Google Scholar
  14. Ma, Y., Košeckà, J., and Sastry, S. 1998. A mathematical theory of camera self-calibration. Electronic Research Laboratory Memorandum, UC Berkeley, UCB/ERL M98/64.Google Scholar
  15. Ma, Y., Košeckà, J., and Sastry, S. 2000. Linear differential algorithm for motion recovery: A geometric approach. IJCV, 36(1):71-89.Google Scholar
  16. Maybank, S. 1993. Theory of Reconstruction from Image Motion. Springer-Verlag: Berlin.Google Scholar
  17. Milnor, J. 1969. Morse Theory. Annals of Mathematics Studies no. 51. Princeton University Press: Princeton.Google Scholar
  18. Murray, R.M., Li, Z., and Sastry, S.S. 1994. A Mathematical Introduction to Robotic Manipulation. CRC press Inc.: Boca Raton, FL.Google Scholar
  19. Oliensis, J. 1999. A new structure from motion ambiguity. In IEEE Proceedings from CVPR, pp. 185-191.Google Scholar
  20. Sastry, S.S. 1999. Nonlinear Systems: Analysis, Stability and Control. Springer-Verlag: Berlin.Google Scholar
  21. Smith, S.T. 1993. Geometric optimization methods for adaptive filtering. Ph.D. Thesis, Harvard University, Cambridge, Massachusetts.Google Scholar
  22. Soatto, S. and Brockett, R. 1998. Optimal and suboptimal structure from motion. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.Google Scholar
  23. Soatto, S. and Perona, P. 1996. Motion estimation via dynamic vision. IEEE Transactions on Automatic Control, 41(3):393-413.Google Scholar
  24. Spetsakis, M. 1994. Models of statistical visual motion estimation. CVIPG: Image Understanding, 60(3):300-312.Google Scholar
  25. Spivak, M. 1979. A Comprehensive Introduction to Differential Geometry: 2nd ed. Publish or Perish, Inc. Berkeley.Google Scholar
  26. Taylor, C.J. and Kriegman, D.J. 1995. Structure and motion from line segments in multiple images. IEEE Transactions on PAMI, 17(11):1021-1032.Google Scholar
  27. Thomas, I. and Simoncelli, E. 1995. Linear structure from motion. Ms-cis-94-61, Grasp Laboratory, University of Pennsylvania.Google Scholar
  28. Tian, T.Y., Tomasi, C., and Heeger, D. 1996. Comparison of approaches to egomotion computation. In CVPR.Google Scholar
  29. Weng, J., Huang, T.S., and Ahuja, N. 1989. Motion and structure from two perspective views: Algorithms, error analysis, and error estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(5):451-475.Google Scholar
  30. Weng, J., Huang, T.S., and Ahuja, N. 1993a. Motion and Structure from Image Sequences. Springer Verlag: Berlin.Google Scholar
  31. Weng, J., Huang, T.S., and Ahuja, N. 1993b. Optimal motion and structure estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(9):864-884.Google Scholar
  32. Zhang, T. and Tomasi, C. 1999. Fast, robust and consistent camera motion estimation. In Proceedings of CVPR.Google Scholar
  33. Zhang, Z. 1998. Understanding the relationship between the optimization criteria in two-view motion analysis. In Proceedings of International Conference on Computer Vision, Bombay, India, pp. 772-777.Google Scholar

Copyright information

© Kluwer Academic Publishers 2001

Authors and Affiliations

  • Yi Ma
    • 1
  • Jana Košecká
    • 2
  • Shankar Sastry
    • 3
  1. 1.Department of Electrical and Computer EngineeringUniversity of Illinois at Urbana-ChampaignUrbanaUSA
  2. 2.Department of Computer ScienceGeorge Mason UniversityFairfaxUSA
  3. 3.Department of Electrical Engineering and Computer SciencesUniversity of California at BerkeleyBerkeleyUSA

Personalised recommendations