International Journal of Computer Vision

, Volume 27, Issue 2, pp 161–195

Determining the Epipolar Geometry and its Uncertainty: A Review

  • Zhengyou Zhang
Article

Abstract

Two images of a single scene/object are related by the epipolar geometry, which can be described by a 3×3 singular matrix called the essential matrix if images' internal parameters are known, or the fundamental matrix otherwise. It captures all geometric information contained in two images, and its determination is very important in many applications such as scene modeling and vehicle navigation. This paper gives an introduction to the epipolar geometry, and provides a complete review of the current techniques for estimating the fundamental matrix and its uncertainty. A well-founded measure is proposed to compare these techniques. Projective reconstruction is also reviewed. The software which we have developed for this review is available on the Internet.

epipolar geometry fundamental matrix calibration reconstruction parameter estimation robust techniques uncertainty characterization performance evaluation software 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aggarwal, J. and Nandhakumar, N. 1988. On the computation of motion from sequences of images-A review. In Proc. IEEE, Vol. 76,No. 8, pp. 917-935.CrossRefGoogle Scholar
  2. Aloimonos, J. 1990. Perspective approximations. Image and Vision Computing, 8(3):179-192.CrossRefGoogle Scholar
  3. Anderson, T. 1958. An Introduction to Multivariate Statistical Analysis. John Wiley & Sons, Inc.Google Scholar
  4. Ayache, N. 1991.Artificial Vision for Mobile Robots. MIT Press.Google Scholar
  5. Ayer, S., Schroeter, P., and Bigün, J. 1994. Segmentation of moving objects by robust motion parameterestimation over multiple frames. In Proc. of the 3rd European Conf. on Computer Vision, J.-O. Eklundh (Ed.), Vols. 800-801 of Lecture Notes in Computer Science, Springer-Verlag: Stockholm, Sweden, Vol. II, pp. 316- 327.Google Scholar
  6. Beardsley, P., Zisserman, A., and Murray, D. 1994. Navigation using affine structure from motion. In Proc. of the 3rd European Conf. on Computer Vision, J.-O. Eklundh (Ed.), Vol. 2 of Lecture Notes in Computer Science, Springer-Verlag: Stockholm, Sweden, pp. 85- 96.Google Scholar
  7. Boufama, B. and Mohr, R. 1995. Epipole and fundamental matrix estimation using the virtual parallax property. In Proc. of the 5th Int. Conf. on Computer Vision, IEEE Computer Society Press: Boston, MA, pp. 1030-1036.Google Scholar
  8. Carlsson, S. 1994. Multiple image invariance using the double algebra. In Applications of Invariance in Computer Vision, J.L. Mundy, A. Zissermann, and D. Forsyth (Eds.), Vol. 825 of Lecture Notes in Computer Science, Springer-Verlag, pp. 145-164.Google Scholar
  9. Csurka, G. 1996. Modélisation projective des objets tridimensionnels en vision par ordinateur. Ph.D. Thesis, University of Nice, Sophia-Antipolis, France.Google Scholar
  10. Csurka, G., Zeller, C., Zhang, Z., and Faugeras, O. 1996. Characterizing the uncertainty of the fundamental matrix. Computer Vision and Image Understanding, 68(1):18-36, 1997. Updated version of INRIA Research Report 2560, 1995.CrossRefGoogle Scholar
  11. Deriche, R., Zhang, Z., Luong, Q.-T., and Faugeras, O. 1994. Robust recovery of the epipolar geometry for an uncalibrated stereo rig. In Proc. of the 3rd European Conf. on Computer Vision, J.-O. Eklundh (Ed.), Vols. 800-801 of Lecture Notes in Computer Science, Springer Verlag: Stockholm, Sweden, Vol. 1, pp. 567-576.Google Scholar
  12. Enciso, R. 1995. Auto-calibration des capteurs visuels actifs. Reconstruction 3D active. Ph.D. Thesis, University Paris XI Orsay.Google Scholar
  13. Faugeras, O. 1992. What can be seen in three dimensions with an uncalibrated stereo rig. In Proc. of the 2nd European Conf. on Computer Vision, G. Sandini (Ed.), Vol. 588 of Lecture Notes in Computer Science, Springer-Verlag: Santa Margherita Ligure, Italy, pp. 563-578.Google Scholar
  14. Faugeras, O. 1993. Three-Dimensional Computer Vision: A Geometric Viewpoint. The MIT Press.Google Scholar
  15. Faugeras, O. 1995. Stratification of 3-D vision: Projective, affine, and metric representations. Journal of the Optical Society of America A, 12(3):465-484.Google Scholar
  16. Faugeras, O. and Lustman, F. 1988. Motion and structure from motion in a piecewise planar environment. International Journal of Pattern Recognition and Artificial Intelligence, 2(3):485- 508.Google Scholar
  17. Faugeras, O., Luong, T., and Maybank, S. 1992. Camera selfcalibration: Theory and experiments. In Proc. 2nd ECCV, G. Sandini (Ed.), Vol. 588 of Lecture Notes in Computer Science, Springer-Verlag: Santa Margherita Ligure, Italy, pp. 321-334.Google Scholar
  18. Faugeras, O. and Robert, L. 1994. What can two images tell us about a third one?. In Proc. of the 3rd European Conf. on Computer Vision, J.-O. Eklundh (Ed.), Vols. 800-801 of Lecture Notes in Computer Science, Springer-Verlag: Stockholm, Sweden. Also INRIA Technical report 2018.Google Scholar
  19. Faugeras, O. and Mourrain, B. 1995. On the geometry and algebra of the point and line correspondences between nimages. In Proc. of the 5th Int. Conf. on Computer Vision, IEEE Computer Society Press: Boston, MA, pp. 951-956.Google Scholar
  20. Fischler, M. and Bolles, R. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24:381- 385.CrossRefGoogle Scholar
  21. Golub, G. and van Loan, C. 1989. Matrix Computations. The John Hopkins University Press.Google Scholar
  22. Haralick, R. 1986. Computer vision theory: The lack thereof. Computer Vision, Graphics, and Image Processing, 36:372-386.Google Scholar
  23. Hartley, R. 1993. Euclidean reconstruction from uncalibrated views. In Applications of Invariance in Computer Vision, J. Mundy and A. Zisserman (Eds.), Vol. 825 of Lecture Notes in Computer Science, Springer-Verlag: Berlin, pp. 237-256.Google Scholar
  24. Hartley, R. 1994. Projective reconstruction and invariants from multiple images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(10):1036-1040.CrossRefGoogle Scholar
  25. Hartley, R. 1995. In defence of the 8-point algorithm. In Proc. of the 5th Int. Conf. on Computer Vision, IEEE Computer Society Press: Boston, MA, pp. 1064-1070.Google Scholar
  26. Hartley, R., Gupta, R., and Chang, T. 1992. Stereo from uncalibrated cameras. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Urbana Champaign, IL, pp. 761-764.Google Scholar
  27. Hartley, R. and Sturm, P. 1994. Triangulation. In Proc. of the ARPA Image Understanding Workshop, Defense Advanced Research Projects Agency, Morgan Kaufmann Publishers, Inc., pp. 957- 966.Google Scholar
  28. Heeger, D.J. and Jepson, A.D. 1992. Subspace methods for recovering rigid motion I: Algorithm and implementation. The International Journal of Computer Vision, 7(2):95-117.Google Scholar
  29. Hesse, O. 1863. Die cubische gleichung, von welcher die Lösung des problems der homographie von M. Chasles Abhängt. J. Reine Angew. Math., 62:188-192.Google Scholar
  30. Huang, T. and Netravali, A. 1994. Motion and structure from feature correspondences: A review. In Proc. IEEE, 82(2):252-268.CrossRefGoogle Scholar
  31. Huber, P. 1981. Robust Statistics. John Wiley & Sons: New York.Google Scholar
  32. Laveau, S. 1996. Géométrie d'un système de Ncaméras. Théorie. Estimation. Applications. Ph.D. Thesis, École Polytechnique.Google Scholar
  33. Longuet-Higgins, H. 1981. A computer algorithm for reconstructing a scene from two projections. Nature, 293:133-135.Google Scholar
  34. Luong, Q.-T. 1992. Matrice Fondamentale et CalibrationVisuelle sur l'Environnement-Vers une plus grande autonomie des systèmes robotiques. Ph.D. Thesis, Université de Paris-Sud, Centre d'Orsay.Google Scholar
  35. Luong, Q.-T. and Viéville, T. 1994. Canonic representations for the geometries of multiple projective views. In Proc. of the 3rd European Conf. on Computer Vision, J.-O. Eklundh (Ed.), Vols. 800- 801 of Lecture Notes in Computer Science, Springer-Verlag: Stockholm, Sweden, Vol. 1, pp. 589-599.Google Scholar
  36. Luong, Q.-T. and Faugeras, O.D. 1996. The fundamental matrix: Theory, algorithms and stability analysis. The International Journal of Computer Vision, 1(17):43-76.Google Scholar
  37. Maybank, S. 1992. Theory of Reconstruction from Image Motion. Springer-Verlag.Google Scholar
  38. Maybank, S.J. and Faugeras, O.D. 1992. A theory of self-calibration of a moving camera. The International Journal of Computer Vision, 8(2):123-152.Google Scholar
  39. Mohr, R., Boufama, B., and Brand, P. 1993a. Accurate projective reconstruction. In Applications of Invariance in Computer Vision, J. Mundy and A. Zisserman (Eds.), Vol. 825 of Lecture Notes in Computer Science, Springer-Verlag: Berlin, pp. 257-276.Google Scholar
  40. Mohr, R., Veillon, F., and Quan, L. 1993b. Relative 3d reconstruction using multiple uncalibrated images. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 543-548.Google Scholar
  41. More, J. 1977. The levenberg-marquardt algorithm, implementation and theory. In Numerical Analysis, G.A. Watson (Ed.), Lecture Notes in Mathematics 630, Springer-Verlag.Google Scholar
  42. Mundy, J.L. and Zisserman, A. (Eds.) 1992.Geometric Invariance in Computer Vision. MIT Press.Google Scholar
  43. Odobez, J.-M. and Bouthemy, P. 1994. Robust multiresolution estimation of parametric motion models applied to complex scenes. Publication Interne 788, IRISA-INRIA Rennes, France.Google Scholar
  44. Olsen, S. 1992. Epipolar line estimation. In Proc. of the 2nd European Conf. on Computer Vision, Santa Margherita Ligure, Italy, pp. 307-311.Google Scholar
  45. Ponce, J. and Genc, Y. 1996. Epipolar geometry and linear subspace methods: A new approach to weak calibration. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, San Francisco, CA, pp. 776-781.Google Scholar
  46. Press, W.H., Flannery, B.P., Teukolsky, S.A., and Vetterling, W.T. 1988. Numerical Recipes in C. Cambridge University Press.Google Scholar
  47. Quan, L. 1993. Affine stereo calibration for relative affine shape reconstruction. In Proc. of theFourth BritishMachineVision Conf., Surrey, England, pp. 659-668.Google Scholar
  48. Quan, L. 1995. Invariants of six points and projective reconstruction from three uncalibrated images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(1).Google Scholar
  49. Rey W.J. 1983. Introduction to Robust and Quasi-Robust Statistical Methods. Springer: Berlin, Heidelberg.Google Scholar
  50. Robert, L. and Faugeras, O. 1993. Relative 3d positioning and 3d convex hull computation from a weakly calibrated stereo pair. In Proc. of the 4th Int. Conf. on Computer Vision, IEEE Computer Society Press: Berlin, Germany, pp. 540-544. Also INRIA Technical Report 2349.Google Scholar
  51. Rothwell, C., Csurka, G., and Faugeras, O. 1995. A comparison of projective reconstruction methods for pairs of views. In Proc. of the 5th Int. Conf. on Computer Vision, IEEE Computer Society Press: Boston, MA, pp. 932-937.Google Scholar
  52. Rousseeuw, P. and Leroy, A. 1987. Robust Regression and Outlier Detection. John Wiley & Sons: New York.Google Scholar
  53. Shapiro, L. 1993. Affine analysis of image sequences. Ph.D. Thesis, University of Oxford, Department of Engineering Science, Oxford, UK.Google Scholar
  54. Shapiro, L., Zisserman, A., and Brady, M. 1994. Motion from point matches using affine epipolar geometry. In Proc. of the 3rd European Conf. on Computer Vision, J.-O. Eklundh (Ed.), Vol. II of Lecture Notes in Computer Science, Springer-Verlag: Stockholm, Sweden, pp. 73-84.Google Scholar
  55. Shapiro, L. and Brady, M. 1995. Rejecting outliers and estimating errors in an orthogonal-regression framework. Phil. Trans. Royal Soc. of Lon. A, 350:407-439.Google Scholar
  56. Shashua, A. 1994a. Projective structure from uncalibrated images: structure from motion and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(8):778-790.CrossRefGoogle Scholar
  57. Shashua, A. 1994b. Trilinearity in visual recognition by alignment. In Proc. of the 3rd European Conf. on Computer Vision, J.-O. Eklundh (Ed.), Vols. 800-801 of Lecture Notes in Computer Science, Springer-Verlag: Stockholm, Sweden, pp. 479-484.Google Scholar
  58. Spetsakis, M. and Aloimonos, J. 1989. A unified theory of structure from motion. Technical Report CAR-TR-482, Computer Vision Laboratory, University of Maryland.Google Scholar
  59. Sturm, R. 1869. Das problem der projektivitt und seine anwendung auf die flächen zweiten grades. Math. Ann., 1:533-574.Google Scholar
  60. Torr, P. 1995. Motion segmentation and outlier detection. Ph.D. Thesis, Department of Engineering Science, University of Oxford.Google Scholar
  61. Torr, P. and Murray, D. 1993. Outlier detection and motion segmentation. In Sensor Fusion VI, SPIE Vol. 2059, P. Schenker (Ed.), Boston, pp. 432-443.Google Scholar
  62. Torr, P., Beardsley, P., and Murray, D. 1994. Robust vision. British Machine Vision Conf., University of York, UK, pp. 145-154.Google Scholar
  63. Torr, P., Zisserman, A., and Maybank, S. 1995. Robust detection of degenerate configurations for the fundamental matrix. In Proc. of the 5th Int. Conf. on Computer Vision, IEEE Computer Society Press: Boston, MA, pp. 1037-1042.Google Scholar
  64. Torr, P., Zisserman, A., and Maybank, S. 1996. Robust detection of degenerate configurations whilst estimating the fundamental matrix. Technical Report OUEL 2090/96, Oxford University, Dept. of Engineering Science.Google Scholar
  65. Triggs, B. 1995. Matching constraints and the joint image. In Proc. of the 5th Int. Conf. on Computer Vision, IEEE Computer Society Press: Boston, MA, pp. 338-343.Google Scholar
  66. Tsai, R. and Huang, T. 1984. Uniqueness and estimation of three-dimensional motion parameters of rigid objects with curved surface. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(1):13-26.Google Scholar
  67. Viéville, T., Faugeras, O.D., and Luong, Q.-T. 1996. Motion of points and lines in the uncalibrated case. The International Journal of Computer Vision, 17(1):7-42.Google Scholar
  68. Weinshall, D., Werman, M., and Shashua, A. 1995. Shape tensors for efficient and learnable indexing. IEEEWorkshop on Representation of Visual Scenes, IEEE, pp. 58-65.Google Scholar
  69. Xu, G. and Zhang, Z. 1996. Epipolar Geometry in Stereo, Motion and Object Recognition: A Unified Approach. Kluwer Academic Publishers.Google Scholar
  70. Zeller, C. 1996. Calibration projective affine et euclidienne en vision par ordinateur. Ph.D. Thesis, École Polytechnique.Google Scholar
  71. Zeller, C. and Faugeras, O. 1994. Applications of non-metric vision to some visual guided tasks. In Proc. of the Int. Conf. on Pattern Recognition, Computer Society Press: Jerusalem, Israel, pp. 132-136. A longer version in INRIA Tech. Report RR2308.Google Scholar
  72. Zhang, Z. 1995. Motion and structure of four points from one motion of a stereo rig with unknown extrinsic parameters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(12):1222- 1227.CrossRefGoogle Scholar
  73. Zhang, Z. 1996a. A new multistage approach to motion and structure estimation: From essential parameters to euclidean motion via fundamental matrix. Research Report 2910, INRIA Sophia-Antipolis, France. Also appeared in Journal of the Optical Society of AmericaA, 14(11):2938-2950, 1997.Google Scholar
  74. Zhang, Z. 1996b. On the epipolar geometry between two images with lens distortion. International Conferences on Pattern Recognition, Vienna, Austria, Vol. I, pp. 407-411.CrossRefGoogle Scholar
  75. Zhang, Z. 1996c. Parameter estimation techniques: A tutorial with application to conic fitting. Image and Vision Computing, 15(1):59-76, 1997. Also INRIA Research Report No. 2676, Oct. 1995.CrossRefGoogle Scholar
  76. Zhang, Z. and Faugeras, O.D. 1992. 3D Dynamic Scene Analysis: A Stereo Based Approach. Springer: Berlin, Heidelberg.Google Scholar
  77. Zhang, Z., Deriche, R., Luong, Q.-T., and Faugeras, O. 1994. A robust approach to image matching: Recovery of the epipolar geometry. In Proc. International Symposium of Young Investigators on Information\Computer\Control, Beijing, China, pp. 7-28.Google Scholar
  78. Zhang, Z., Deriche, R., Faugeras, O., and Luong, Q.-T. 1995a. A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artificial Intelligence Journal, 78:87-119.CrossRefGoogle Scholar
  79. Zhang, Z., Faugeras, O., and Deriche, R. 1995b. Calibrating a binocular stereo through projective reconstruction using both a calibration object and the environment. In Proc. Europe-China Workshop on Geometrical Modelling and Invariants for Computer Vision, R. Mohr and C. Wu (Eds.), Xi'an, China, pp. 253-260. Also appeared in Videre: A Journal of Computer Vision Research, 1(1):58-68, Fall 1997.Google Scholar
  80. Zhang, Z., Luong, Q.-T., and Faugeras, O. 1996. Motion of an uncalibrated stereo rig: Self-calibration and metric reconstruction. IEEE Trans. Robotics and Automation, 12(1):103-113.CrossRefGoogle Scholar
  81. Zhuang, X., Wang, T., and Zhang, P. 1992. A highly robust estimator through partially likelihood function modeling and its application in computer vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(1):19-34.CrossRefGoogle Scholar
  82. Zisserman, A. 1992. Notes on geometric invariants in vision. BMVC92 Tutorial.Google Scholar

Copyright information

© Kluwer Academic Publishers 1998

Authors and Affiliations

  • Zhengyou Zhang
    • 1
  1. 1.INRIASophia-Antipolis CedexFrance. E-mail

Personalised recommendations