Advertisement

International Journal of Computer Vision

, Volume 32, Issue 1, pp 27–44 | Cite as

The Problem of Degeneracy in Structure and Motion Recovery from Uncalibrated Image Sequences

  • Philip H.S. Torr
  • Andrew W. Fitzgibbon
  • Andrew Zisserman
Article

Abstract

The aim of this work is the recovery of 3D structure and camera projection matrices for each frame of an uncalibrated image sequence. In order to achieve this, correspondences are required throughout the sequence. A significant and successful mechanism for automatically establishing these correspondences is by the use of geometric constraints arising from scene rigidity. However, problems arise with such geometry guided matching if general viewpoint and general structure are assumed whilst frames in the sequence and/or scene structure do not conform to these assumptions. Such cases are termed degenerate.

In this paper we describe two important cases of degeneracy and their effects on geometry guided matching. The cases are a motion degeneracy where the camera does not translate between frames, and a structure degeneracy where the viewed scene structure is planar. The effects include the loss of correspondences due to under or over fitting of geometric models estimated from image data, leading to the failure of the tracking method. These degeneracies are not a theoretical curiosity, but commonly occur in real sequences where models are statistically estimated from image points with measurement error.

We investigate two strategies for tackling such degeneracies: the first uses a statistical model selection test to identify when degeneracies occur: the second uses multiple motion models to overcome the degeneracies. The strategies are evaluated on real sequences varying in motion, scene type, and length from 13 to 120 frames.

projective reconstruction multiview geometry match moving 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Akaike, H. 1974. A new look at the statistical model identification. IEEE Trans. on Automatic Control, AC-19(6):716–723.Google Scholar
  2. Ayache, N. 1991. Artificial Vision for Mobile Robots. MIT Press: Cambridge.Google Scholar
  3. Bar-Shalom, Y. and Fortmann, T.E. 1988. Tracking and Data Association. Academic Press.Google Scholar
  4. Beardsley, P., Torr, P., and Zisserman, A. 1996. 3D model acquisition from extended image sequences. In Proc. European Conference on Computer Vision, LNCS 1064/1065, Springer-Verlag, pp. 683–695.Google Scholar
  5. Beardsley, P., Zisserman, A., and Murray, D.W. 1994. Navigation using affine structure and motion. In Proc. European Conference on Computer Vision, LNCS 800/801, Springer-Verlag, pp. 85–96.Google Scholar
  6. Capel, D. and Zisserman, A. 1998. Automated mosaicing with superresolution zoom. In Proceedings of the Conference on Computer Vision and Pattern Recognition, Santa Barbara, pp. 885–891.Google Scholar
  7. Cox, I.J., Rehg, J.M., and Hingorani, S. 1992. A Bayesian multiple hypothesis approach to contour grouping. In Proc. European Conference on Computer Vision, LNCS 588, Springer-Verlag, pp. 72–77.Google Scholar
  8. Faugeras, O. 1992. What can be seen in three dimensions with an uncalibrated stereo rig? In Proc. European Conference on Computer Vision, LNCS 588, Springer-Verlag, pp. 563–578.Google Scholar
  9. Faugeras, O. 1993. Three-Dimensional Computer Vision: A Geometric Viewpoint. MIT Press.Google Scholar
  10. Fitzgibbon, A.W. and Zisserman, A. 1998. Automatic camera recovery for closed or open image sequences. In Proc. European Conference on Computer Vision, Springer-Verlag, pp. 311–326.Google Scholar
  11. Harris, C.J. 1987. Determination of ego-motion from matched points. In Third Alvey Vision Conference, pp. 189–192.Google Scholar
  12. Harris, C.J. and Stephens, M. 1988. A combined corner and edge detector. In Proc. 4th Alvey Vision Conference, Manchester, pp. 147–151.Google Scholar
  13. Hartley, R.I. 1994a. Euclidean reconstruction from uncalibrated views. In Applications of Invariance in Computer Vision, J. Mundy, A. Zisserman, and D. Forsyth (Eds.), LNCS 825, Springer-Verlag, pp. 237–256.Google Scholar
  14. Hartley, R.I. 1994b. Self-calibration from multiple views with a rotating camera. In Proc. European Conference on Computer Vision, LNCS 800/801, Springer-Verlag, pp. 471–478.Google Scholar
  15. Hartley, R.I. 1995. A linear method for reconstruction from lines and points. In Proc. International Conference on Computer Vision, pp. 882–887.Google Scholar
  16. Hartley, R.I. 1997. Lines and points in three views and the trifocal tensor. International Journal of Computer Vision, 22(2):125–140.Google Scholar
  17. Hartley, R.I., Gupta, R., and Chang, T. 1992. Stereo from uncalibrated cameras. In Proceedings of the Conference on Computer Vision and Pattern Recognition.Google Scholar
  18. Huber, P.J. 1981. Robust Statistics. John Willey and Sons.Google Scholar
  19. Irani, M. and Anandan, P. 1996. Parallax geometry of pairs of points for 3d scene analysis. In Proc. 4th European Conference on Computer Vision, B. Buxton and R. Cipolla (Eds.), LNCS 1064, Cambridge, Springer, pp. 17–30.Google Scholar
  20. Kanatani, K. 1996. Automatic singularity test for motion analysis by an information criterion. In Proc. 4th European Conference on Computer Vision, B. Buxton and R. Cipolla (Eds.), LNCS 1064, Cambridge, Springer-Verlag, pp. 697–708.Google Scholar
  21. Laveau, S. 1996. Géométrie d'un système de N caméras. Théorie, estimation et applications. Ph.D. Thesis, INRIA.Google Scholar
  22. Maybank, S.J. 1993. Theory of Reconstruction from Image Motion. Springer-Verlag: Berlin.Google Scholar
  23. McLauchlan, P.F. and Murray, D.W. 1995. A unifying framework for structure from motion recovery from image sequences. In Proc. International Conference on Computer Vision, pp. 314–320.Google Scholar
  24. Milgram, P., Shumin, S., Drascic, D., and Grodski, J. 1993. Applications of augmented reality for human-robot communication. In International Conference on Intelligent Robots and Systems Proceedings, Yokohama, Japan, pp. 1467–1472.Google Scholar
  25. Pollefeys, M., Koch, R., and Van Gool, L. 1998. Self calibration and metric reconstruction in spite of varying and unknown internal camera parameters. In Proc. 6th International Conference on Computer Vision, Bombay, India, pp. 90–96.Google Scholar
  26. Press, W., Flannery, B., Teukolsky, S., and Vetterling, W. 1988. Numerical Recipes in C. Cambridge University Press.Google Scholar
  27. Rao, B. 1992. Data association methods for tracking systems. In Active Vision, A. Blake and A. Yuille (Eds.), MIT Press, pp. 91–105.Google Scholar
  28. Schwarz, G. 1978. Estimating dimension of a model. Ann. Stat., 6:461–464.Google Scholar
  29. Shashua, A. 1994. Trilinearity in visual recognition by alignment. In Proc. 3rd European Conference on Computer Vision, Stockholm, vol. 1, pp. 479–484.Google Scholar
  30. Spetsakis, M.E. and Aloimonos, J. 1991. A multi-frame approach to visual motion perception. International Journal of Computer Vision, 16(3):245–255.Google Scholar
  31. Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization approach. International Journal of Computer Vision, 9(2):137–154.Google Scholar
  32. Torr, P.H.S. 1997. An assessment of information criteria for motion model selection. In Proceedings of the Conference on Computer Vision and Pattern Recognition, Puerto Rico. To appear in CVIU.Google Scholar
  33. Torr, P.H.S., Fitzgibbon, A.W., and Zisserman, A. 1998. Maintaining multiple motion model hypotheses over many views to recover matching and structure. In Proc. 6th International Conference on Computer Vision, Bombay, India, pp. 485–491.Google Scholar
  34. Torr, P.H.S. and Murray, D.W. 1993. Statistical detection of independent movement from a moving camera. Image and Vision Computing, 1(4):180–187.Google Scholar
  35. Torr, P.H.S. and Zisserman, A. 1997. Robust parameterization and computation of the trifocal tensor. Image and Vision Computing, 15:591–605.Google Scholar
  36. Torr, P.H.S. and Zisserman, A. 1998. Robust computation and parameterization of multiple view relations. In Proc. 6th International Conference on Computer Vision, Bombay, India, pp. 727–732.Google Scholar
  37. Triggs, W. 1998. Autocalibration from planar scenes. In Proc. 5th European Conference on Computer Vision, Freiburg, Germany.Google Scholar
  38. Zeller, C. 1996. Projective, Affine and Euclidean Calibration in Compute Vision and the Application of Three Dimensional Perception. Ph.D. Thesis, RobotVis Group, INRIA Sophia-Antipolis.Google Scholar
  39. Zhang, Z., Deriche, R., Faugeras, O., and Luong, Q. 1995. A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artificial Intelligence, 78:87–119.Google Scholar
  40. Zhang, Z. and Faugeras, O. 1992. 3D Dynamic Scene Analysis. Springer-Verlag.Google Scholar

Copyright information

© Kluwer Academic Publishers 1999

Authors and Affiliations

  • Philip H.S. Torr
    • 1
  • Andrew W. Fitzgibbon
    • 2
  • Andrew Zisserman
    • 2
  1. 1.Microsoft Research, One Microsoft WayRedmondUSA
  2. 2.Robotics Research Group, Department of Engineering ScienceOxford UniversityUK

Personalised recommendations