Generalized Rank Conditions in Multiple View Geometry with Applications to Dynamical Scenes

  • Kun Huang
  • Robert Fossum
  • Yi Ma
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2351)


In this paper, the geometry of a general class of projections from ℝn to ℝk (k < n) is examined, as a generalization of classic multiple view geometry in computer vision. It is shown that geometric constraints that govern multiple images of hyperplanes in ℝn, as well as any incidence conditions among these hyperplanes (such as inclusion, intersection, and restriction), can be systematically captured through certain rank conditions on the so-called multiple view matrix. All constraints known or unknown in computer vision for the projection from ℝ3 to ℝ2 are simply instances of this result. It certainly simplifies current efforts to extending classic multiple view geometry to dynamical scenes. It also reveals that since most new constraints in spaces of higher dimension are nonlinear, the rank conditions are a natural replacement for the traditional multilinear analysis. We also demonstrate that the rank conditions encode extremely rich information about dynamical scenes and they give rise to fundamental criteria for purposes such as stereopsis in n-dimensional space, segmentation of dynamical features, detection of spatial and temporal formations, and rejection of occluding T-junctions.


multiple view geometry rank condition multiple view matrix dynamical scenes segmentation formation detection occlusion structure from motion 


  1. 1.
    S. Avidan and A. Shashua. Novel view synthesis by cascading trilinear tensors. IEEE Transactions on Visualization and Computer Graphics (TVCG), 4(4), pp. 293–306, 1998.CrossRefGoogle Scholar
  2. 2.
    O. Faugeras, Q.-T. Luong, and T. Papadopoulo. Geometry of Multiple Images. The MIT Press, 2001.Google Scholar
  3. 3.
    R. Fossum, K. Huang, Y. Ma General rank conditions in multiple view geometry. UIUC, CSL Technical Report, UILU-ENG 01-2222 (DC-203), October 8, 2001.Google Scholar
  4. 4.
    R. Hartley. Lines and points in three views-a unified approach. In Proceedings of 1994 Image Understanding Workshop, pp. 1006–1016, Monterey, CA USA, 1994. OMNIPRESS.Google Scholar
  5. 5.
    R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge, 2000.Google Scholar
  6. 6.
    A. Heyden and K. Åström. Algebraic properties of multilinear constraints. Mathematical Methods in Applied Sciences, 20(13), pp. 1135–1162, 1997.zbMATHCrossRefGoogle Scholar
  7. 7.
    Y. Liu and T.S. Huang. Estimation of rigid body motion using straight line correspondences IEEE Workshop on Motion: Representation and Analysis, Kiawah Island, SC, May 1986.Google Scholar
  8. 8.
    H. C. Longuet-Higgins. A computer algorithm for reconstructing a scene from two projections. Nature, 293, pp. 133–135, 1981.CrossRefGoogle Scholar
  9. 9.
    Y Ma, K. Huang, and J. Košecká. New rank deficiency condition for multiple view geometry of line features. UIUC, CSL Technical Report, UILU-ENG 01-2209 (DC-201), May 8, 2001.Google Scholar
  10. 10.
    Y Ma, K. Huang, R. Vidal, J. Košecká, and S. Sastry. Rank conditions of multiple view matrix in multiple view geometry. UIUC, CSL Technical Report, UILU-ENG 01-2214 (DC-220), submitted to IJCV, June 18, 2001.Google Scholar
  11. 11.
    M. Spetsakis and Y Aloimonos. Structure from motion using line correspondences. International Journal of Computer Vision, 4(3): 171–184, 1990.CrossRefGoogle Scholar
  12. 12.
    B. Triggs. Matching constraints and the joint image. In Proceedings of Fifth International Conference on Computer Vision, pp. 338–343, Cambridge, MA, USA, 1995. IEEE Comput. Soc. Press.Google Scholar
  13. 13.
    L. Wolf and A. Shashua. On projection matrices P kP 2, k = 3,..., 6, and their applications in computer vision. In Proceedings of the Eighth International Conference on Computer Vision, pp. 412–419, Vancouver, Canada, 2001.Google Scholar
  14. 14.
    G. Sparr. A common framework for kinetic depth, reconstruction and motion for deformable objects. Proceedings of the Fourth European Conference on Computer Vision, pp. 471–482, Cambridge, England, 1994.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Kun Huang
    • 1
  • Robert Fossum
    • 2
  • Yi Ma
    • 1
  1. 1.Electrical & Computer Engineering Dept., and Coordinated Science Lab.USA
  2. 2.Mathematics Department, and Beckman InstituteUniversity of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations