Advertisement

International Journal of Computer Vision

, Volume 17, Issue 1, pp 43–75 | Cite as

The fundamental matrix: Theory, algorithms, and stability analysis

  • Quan-Tuan Luong
  • Olivier D. Faugeras
Article

Abstract

In this paper we analyze in some detail the geometry of a pair of cameras, i.e., a stereo rig. Contrarily to what has been done in the past and is still done currently, for example in stereo or motion analysis, we do not assume that the intrinsic parameters of the cameras are known (coordinates of the principal points, pixels aspect ratio and focal lengths). This is important for two reasons. First, it is more realistic in applications where these parameters may vary according to the task (active vision). Second, the general case considered here, captures all the relevant information that is necessary for establishing correspondences between two pairs of images. This information is fundamentally projective and is hidden in a confusing manner in the commonly used formalism of the Essential matrix introduced by Longuet-Higgins (1981). This paper clarifies the projective nature of the correspondence problem in stereo and shows that the epipolar geometry can be summarized in one 3×3 matrix of rank 2 which we propose to call the Fundamental matrix.

After this theoretical analysis, we embark on the task of estimating the Fundamental matrix from point correspondences, a task which is of practical importance. We analyze theoretically, and compare experimentally using synthetic and real data, several methods of estimation. The problem of the stability of the estimation is studied from two complementary viewpoints. First we show that there is an interesting relationship between the Fundamental matrix and three-dimensional planes which induce homographies between the images and create unstabilities in the estimation procedures. Second, we point to a deep relation between the unstability of the estimation procedure and the presence in the scene of so-called critical surfaces which have been studied in the context of motion analysis. Finally we conclude by stressing the fact that we believe that the Fundamental matrix will play a crucial role in future applications of three-dimensional Computer Vision by greatly increasing its versatility, robustness and hence applicability to real difficult problems.

Keywords

Aspect Ratio Computer Vision Focal Length Estimation Procedure Motion Analysis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adiv, G. 1989. Inherent ambiguities in recovering 3-D motion and structure from a noisy flow field. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11:477–489.Google Scholar
  2. Barrett, E.B., Brill, M.H., Haag, N.N., and Payton, P.M. 1992. Invariant linear methods in photogrammetry and model-matching. In J.L. Mundy and A. Zisserman (Eds.), Geometric Invariance in Computer Vision, Chap. 14, pp. 277–292, MIT Press.Google Scholar
  3. Beardsley, P.A. 1992. Applications of Projective Geometry to Computer vision. Ph.D. Thesis, University of Oxford.Google Scholar
  4. Beardsley, P.A., Zisserman, A., and Murray, D.W. 1994. Navigation using affine structure from motion. In Proc. European Conference on Computer Vision, Stockholm, Sweden, pp. 85–96.Google Scholar
  5. Bookstein, F.L. 1979. Fitting conic sections to scattered data. Computer Graphics and Image Processing, 9(1):56–71.Google Scholar
  6. Coxeter, H.S.M. 1987. Projective Geometry. Springer Verlag (Second Edition).Google Scholar
  7. Daniilidis, K. 1992. Zur Fehlerempfindlichkeit in der Ermittlung von Objektbeschreibungen und relativen Bewegugen aus monokularen Bildfolgen. Ph.D. Thesis, University of Karsruhe.Google Scholar
  8. Daniilidis, K. and Nagel, H.-H. 1990. Analytical results on error sensitivity of motion estimation from two views. Image and Vision Computing, 8:297–303.Google Scholar
  9. Demey, S., Zisserman, A., and Beardsley, P.A. 1992. Affine and projective structure from motion. In Proc. British Machine Vision Conference, Leeds, UK, pp. 49–58.Google Scholar
  10. Deriche, R., Zhang, Z., Luong, Q.-T., and Faugeras, O.D. 1994. Robust recovery of the epipolar geometry for an uncalibrated stereo rig. In Proc. European Conference on Computer Vision, Stockholm, Sweden, pp. 567–576.Google Scholar
  11. Devernay, F. and Faugeras, O.D. 1994. Computing differential properties of 3-D shapes from stereoscopic images without 3-D models. In cvpr, Seattle, WA, pp. 208–213.Google Scholar
  12. Fang, J.Q. and Huang, T.S. 1984. Some experiments on estimating the 3D motion parameters of a rigid body from two consecutive image frames. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6:545–554.Google Scholar
  13. Faugeras, O.D. 1992. What can be seen in three dimensions with an uncalibrated stereo rig. In Proc. European Conference on Computer Vision, pp. 563–578.Google Scholar
  14. Faugeras, O.D. 1993. Three-Dimensional Computer Vision: A Geometric Viewpoint. MIT Press.Google Scholar
  15. Faugeras, O.D. Hotz, B. 1994. Real time correlation-based stereo: algorithm, implementations and applications. The International Journal of Computer Vision. To appear.Google Scholar
  16. Faugeras, O.D., Luong, Q.-T., and Maybank, S.J. 1992. Camera self-calibration: theory and experiments. In Proc. European Conference on Computer Vision, Santa-Margerita, Italy, pp. 321–334.Google Scholar
  17. Faugeras, O.D. and Lustman, F. 1988. Motion and structure from motion in a piecewise planar environment. International Journal of Pattern Recognition and Artificial Intelligence, 2(3):485–508.Google Scholar
  18. Faugeras, O.D., Lustman, F., and Toscani, G. 1987. Motion and structure from point and line matches. In Proc. International Conference on Computer Vision, pp. 25–34.Google Scholar
  19. Faugeras, O.D. and Robert, L. 1993. What can two images tell us about a third one? Technical Report RR-2018, INRIA. To appear in IJCV.Google Scholar
  20. Faugeras, O.D. and Toscani, G. 1986. The calibration problem for stereo. In Proceedings of CVPR'86, pp. 15–20.Google Scholar
  21. Garner, L.E. 1981. An Outline of Projective Geometry. Elsevier: North Holland.Google Scholar
  22. Gros, P. and Quan, L. 1993. 3D projective invariants from two images. In Geometric Methods in Computer Vision II, SPIE Optical Instrumentation and Applied Science, San Diego.Google Scholar
  23. Hartley, R.I. 1992. Estimation of relative camera positions for uncalibrated cameras. In Proc. European Conference on Computer Vision, pp. 579–587.Google Scholar
  24. Hartley, R.I. 1993. Cheirality invariants. In Proc. DARPA Image Understanding Workshop, University of Maryland, pp. 745–753.Google Scholar
  25. Hartley, R.I. 1994a. An algorithm for self calibration from several views. In Proc. Conference on Computer Vision and Pattern Recognition, Seattle, WA, pp. 908–912.Google Scholar
  26. Hartley, R.I. 1994b. Projective reconstruction from line correspondences. In Proc. Conference on Computer Vision and Pattern Recognition, Seattle, WA, pp. 903–907.Google Scholar
  27. Hartley, R.I. and Gupta R. 1993. Computing matched-epipolar projections. In Proc. Conference on Computer Vision and Pattern Recognition, New York, pp. 549–555.Google Scholar
  28. Horn, B.K.P. 1990. Relative orientation. The International Journal of Computer Vision, 4(1):59–78.Google Scholar
  29. Horn, B.K.P. and Weldon, E.J. 1988. Direct methods for recovering motion. The International Journal of Computer Vision, 2(1):51–76.Google Scholar
  30. Huang, T.S. and Faugeras, O.D. 1989. Some properties of the E-matrix in two view motion estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11:1310–1312.Google Scholar
  31. Jerian, C.P. and Jain, R. 1991. Structure from motion. A critical analysis of methods. IEEE Transactions on Systems, Man and Cybernetics, 21(3):572–587.Google Scholar
  32. Jones, D.G. and Malik, J. 1992. A computational framework for determining stereo correspondence from a set of linear spatial filters. In Proc. European Conference on Computer Vision, pp. 395–410.Google Scholar
  33. Kanatani, K. 1991. Computational projective geometry. Computer Vision, Graphics, and Image Processing. Image Understanding, 54(3).Google Scholar
  34. Kanatani, K. 1992. Geometric Computation for Machine Vision. Oxford University Press.Google Scholar
  35. Laveau, S. and Faugeras, O.D. 1994. 3-D scene representation as a collection of images. In Proc. International Conference on Pattern Recognition, Jerusalem, Israel. To appear.Google Scholar
  36. Lawn, J.M. and Cipolla, R. 1994. Robust egomotion estimation from affine motion parallax. Technical Report CUED/F-INFENG/TR 160, University of Cambridge. A shorter version appeared at ECCV'94.Google Scholar
  37. Lee, C.H. 1991. Time-varying images: the effect of finite resolution on uniqueness. Computer Vision, Graphics, and Images Processing. Image Understanding, 54(3):325–332.Google Scholar
  38. Longuet-Higgins, H.C. 1981. A computer algorithm for reconstructing a scene from two projections. Nature, 293:133–135.Google Scholar
  39. Longuet-Higgins, C. 1984. The reconstruction of a scene from two projections: configurations that defeat the 8-point algorithm. In Proc. 1st Conf. on Artificial Intelligence Applications, Denver, pp. 395–397.Google Scholar
  40. Longuet-Higgins, H.C. 1988. Multiple interpretations of a pair of images of a surface. Proc. of the Royal Society London A, 418:1–15.Google Scholar
  41. Longuet-Higgins, H.C. and Prazdny, K. 1980. The interpretation of moving retinal images. Proceedings of the Royal Society of London B, 208:385–387.Google Scholar
  42. Luong, Q.-T. 1992. Matrice fondamentale et auto-calibration en vision par ordinateur. Ph.D. Thesis, Universite de Paris-Sud, Orsay.Google Scholar
  43. Luong, Q.-T. and Faugeras, O.D. 1993a. Camera calibration, scene motion and structure recovery from point correspondences and fundamental matrices. Submitted to IJCV.Google Scholar
  44. Luong, Q.-T. and Faugeras, O.D. 1993b. Determining the fundamental matrix with planes: unstability and new algorithms. In Proc. Conference on Computer Vision and Pattern Recognition, New York, pp. 489–494.Google Scholar
  45. Luong, Q.-T., Deriche, R., Faugeras, O.D., and Papadopoulo, T. 1993. On determining the fundamental matrix: analysis of different methods and experimental results. Technical Report RR-1894, INRIA. A shorter version appeared in the Israelian Conf. on Artificial Intelligence and Computer Vision.Google Scholar
  46. Luong, Q.-T. and Faugeras, O.D. 1994. Stratified projective motion analysis: Fundamental matrix and self-calibration. In preparation.Google Scholar
  47. Luong, Q.-T. and Vieville, T. 1994. Canonical representations for the geometries of multiple projective views. CVGIP: image understanding. To appear.Google Scholar
  48. Maybank, S.J. 1985. The angular velocity associated with the optical flow field arising from motion through a rigid environment. Proc. of the Royal Society London A, 401:317–326.Google Scholar
  49. Maybank, S.J. 1990a. The projective geometry of ambiguous surfaces. Proc. of the Royal Society London A, 332:1–47.Google Scholar
  50. Maybank, S.J. 1990b. Properties of essential matrices. International Journal of Imaging Systems and Technology, 2:380–384.Google Scholar
  51. Maybank, S.J. and Faugeras, O.D. 1992. A theory of self-calibration of a moving camera. The International Journal of Computer Vision, 8(2):123–151.Google Scholar
  52. McLauchlan, P.F., Reid, I.D., and Murray, D.W. 1994. Recursive affine structure and motion from image sequences. In Proc. European Conference on Computer Vision, Stockholm, Sweden, pp. 217–224.Google Scholar
  53. Mitiche, A., Zhuang, X., and Haralick, R. 1987. Interpretation of optical flow by rotation decoupling. In Proc IEEE Workshop on Computer Vision, Miami Beach, FL, pp. 195–200.Google Scholar
  54. Mundy, J.L. and Zisserman, A. (Eds.) 1992, Geometric Invariance in Computer Vision. MIT Press.Google Scholar
  55. Mundy, J., Welty, R.P., Brill, M.H., Payton, P.M., and Barrett. 1992. 3-D model alignment without computing pose. In Proc. DARPA Image Understanding Workshop, San Mateo, CA, pp. 727–735.Google Scholar
  56. Nishimura, E., Xu, G., and Tsuji, S. 1993. Motion segmentation and correspondence using epipolar constraint. In Proc. 1st Asian Conf. Computer Vision, Osaka, Japan, pp. 199–204.Google Scholar
  57. Olsen, S.I. 1992. Epipolar line estimation. In Proc. European Conference on Computer Vision, pp. 307–311.Google Scholar
  58. Philip, J. 1991. Estimation of three-dimensional motion of rigid objects from noisy observations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(1):61–66.Google Scholar
  59. Ponce, J., Marimont, D.H., and Cass, T.A. 1994. Analytical methods for uncalibrated stereo and motion reconstruction. In Proc. European Conference on Computer Vision, Stockholm, Sweden. pp. 463–470.Google Scholar
  60. Quan, L. and Mohr, R. 1992. Affine shape representation from motion through reference points. Journal of Mathematical Imaging and Vision, 1:145–151.Google Scholar
  61. Robert, L. 1993. Reconstruction de Courbs et de Surfaces par Vision Stéréoscopique. Applications a la Robotique Mobile. Ph.D. Thesis, Ecole Polytechnique.Google Scholar
  62. Robert, L. and Faugeras, O.D. 1993. Relative 3D positioning and 3D convex hull computation from a weakly calibrated Stereo pair. In Proc. International Conference on Computer Vision, Berlin, Germany, pp. 540–543.Google Scholar
  63. Robert, L. and Hebert, M. 1994. Deriving orientation cues from stereo images. In Proc. European Conference on Computer Vision, Stockholm, Sweden, pp. 377–388.Google Scholar
  64. Sampson, P.D. 1982. Fitting conic sections to very scattered data: an iterative refinement of the Bookstein algorithm. Computer Graphics and Image Processing, 18(1):97–108.Google Scholar
  65. Semple, J.G. and Kneebone, G.T. 1952. Algebraic Projective Geometry. Clarendon Press: Oxford. Reprinted 1979.Google Scholar
  66. Shapiro, L.S., Zisserman, A., and Brady, M. 1993. Motion from point matches using affine epipolar geometry. Technical Report OUEL 1994/93, Oxford University. A shorter version appeared at ECCV'94.Google Scholar
  67. Shashua, A. 1993a. On geometric and algebraic aspects of 3D affine and projective structures from perspective 2D views. In Proceedings of the 2nd European Workshop on Invariants, Ponta Delagada, Azores. Also MIT AI Memo No. 1405.Google Scholar
  68. Shashua, A. 1993b. Projective depth: a geometric invariant for 3d reconstruction from two perspective/orthographic views and for visual recognition. In Proc. International Conference on Computer Vision, Berlin, Germany, pp. 583–590.Google Scholar
  69. Shashua, A. and Navab, N. 1994. Relative affine structure: theory and application to 3D reconstruction from perspective views. In Proc. Conference on Computer Vision and Pattern Recognition, Seattle, WA, pp. 483–489.Google Scholar
  70. Sinclair, D., Blake, A., Smith, S., and Rothwell, C. 1992. Planar region detection and motion recovery. In Proc. British Machine Vision Conf., pp. 59–68.Google Scholar
  71. Spetsakis, M.E. 1992. A linear algorithm for point and line-based structure from motion. Computer Vision, Graphics, and Image Processing. Image Understanding, 56(2):230–241.Google Scholar
  72. Torr, P.H.S. and Murray, D.W. 1994. Stochastic motion clustering. In Proc. European Conference on Computer Vision, Stockholm, Sweden, pp. 328–337.Google Scholar
  73. Tsai, R.Y. 1986. An efficient and accurate camera calibration technique for 3D machine vision. In IEEE Proceedings CVPR'86. Miami Beach, Florida, pp. 364–374.Google Scholar
  74. Tsai, R.Y. and Huang, T.S. 1982. Estimating three-dimensional motion parameters of a rigid planar patch, II: singular value decomposition. IEEE Transactions on Acoustic, Speech and Signal Processing, 30.Google Scholar
  75. Tsai, R.Y. and Huang, T.S. 1984. Uniqueness and estimation of three-dimensional motion parameters of rigid objects wirth curved surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6:13–27.Google Scholar
  76. Viéville, T. and Sander, P. 1992. Using pseudo kalman-filters in the presence of constraints. Technical Report RR-1669, INRIA.Google Scholar
  77. Viéville, T., Luong, Q.-T., and Faugeras, O.D. 1994. Motion of points and lines in the uncalibrated case. Intl. Journal of Computer Vision. To appear.Google Scholar
  78. Weber, J. and Malik, J. 1994. Rigid body segmentation and shape description from dense optical flow under weak perspective. Dept. of EECS, University of California at Berkeley.Google Scholar
  79. Weng, J., Huang, T.S., and Ahuja, N., 1989. Motion and structure from two perspective views: algorithms, error analysis and error estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(5):451–476.Google Scholar
  80. Zhang, Z. and Faugeras, O.D. 1992. 3D Dynamic Scene Analysis. Springer-Verlag.Google Scholar

Copyright information

© Kluwer Academic Publishers 1996

Authors and Affiliations

  • Quan-Tuan Luong
    • 1
  • Olivier D. Faugeras
    • 1
  1. 1.I.N.R.I.A.Sophia-AntipolisFrance

Personalised recommendations