# Reconstruction from Uncalibrated Sequences with a Hierarchy of Trifocal Tensors

## Abstract

This paper considers projective reconstruction with a hierarchical computational structure of trifocal tensors that integrates feature tracking and geometrical validation of the feature tracks. The algorithm was embedded into a system aimed at completely automatic Euclidean reconstruction from uncalibrated handheld amateur video sequences. The algorithm was tested as part of this system on a number of sequences grabbed directly from a low-end video camera without editing. The proposed approach can be considered a generalisation of a scheme of [Fitzgibbon and Zisserman, ECCV ’98]. The proposed scheme tries to adapt itself to the motion and frame rate in the sequence by finding good triplets of views from which accurate and unique trifocal tensors can be calculated. This is in contrast to the assumption that three consecutive views in the video sequence are a good choice. Using trifocal tensors with a wider span suppresses error accumulation and makes the scheme less reliant on bundle adjustment. The proposed computational structure may also be used with fundamental matrices as the basic building block.

## Keywords

Line Triple Bundle Adjustment Sweet Spot Structure From Motion Projective Reconstruction## References

- [1]A. Fitzgibbon, A. Zisserman, Automatic camera recovery for closed or open image sequences,
*Proc. ECCV 98*, pp. 311–326.Google Scholar - [2]P. Beardsley, A. Zisserman, D. Murray, Sequential updating of projective and affine structure from motion,
*1JCV*, 23(3), pp. 235–259, 1997.Google Scholar - [3]P. Beardsley, P. Torr, A. Zisserman, 3D model acquisition from extended image sequences,
*Proc. ECCV 96*, pp. 683–695.Google Scholar - [4]R. Cipolla, E. Boyer, 3D model acquisition from uncalibrated images,
*Proc. IAPR Workshop on Machine Vision Applications*, Chiba Japan, pp. 559–568, Nov 1998.Google Scholar - [5]P. Debevec, C. Taylor, J. Malik, Modeling and rendering architecture from photographs: a hybrid geometry-and image-based approach,
*SIGGRAPH 96*, pp. 11–20.Google Scholar - [6]O. Faugeras, What can be seen in three dimensions with an uncalibrated stereo rig?,
*Proc. ECCV 92*, pp. 563–578.Google Scholar - [7]P. Fua, Reconstructing complex surfaces from multiple stereo views,
*Proc. ICCV 95*, pp. 1078–1085.Google Scholar - [8]K. Hanna, N. Okamoto, Combining stereo and motion for direct estimation of scene structure,
*Proc. ICCV 93*, pp. 357–365.Google Scholar - [9]R. Hartley, Euclidean reconstruction from uncalibrated views,
*Applications of Invariance in Computer Vision*, LNCS825, pp. 237–256, Springer-Verlag, 1994.Google Scholar - [10]R. Hartley, E. Hayman, L. de Agapito, I. Reid, Camera calibration and the search for infinity,
*Proc. ICCV 99*, pp. 510–517.Google Scholar - [11]R. Hartley, Estimation of relative camera positions for uncalibrated cameras,
*Proc. ECCV 92*, pp. 579–587.Google Scholar - [12]R. Hartley, P. Sturm, Triangulation,
*American Image Understanding Workshop*, pp. 957–966, 1994.Google Scholar - [13]A. Heyden, K. Astrom, Euclidean reconstruction from image sequences with varying and unknown focal length and principal point,
*Proc. CVPR 97*, pp. 438–443.Google Scholar - [14]M. Irani, P. Anandan, M. Cohen, Direct recovery of planar-parallax from multiple frames,
*Proc. Vision Algorithms Workshop (1CCV 99)*, Corfu, pp. 1–8, September 1999.Google Scholar - [15]K. Kutulakos, S. Seitz, A theory of shape by space carving,
*Proc ICCV 99*, pp. 307–314.Google Scholar - [16]S. Maybank, O. Faugeras, A theory of self-calibration of a moving camera, International Journal of Computer Vision, 8(2), pp. 123–151, 1992.CrossRefGoogle Scholar
- [17]P. McLauchlan, D. Murray, A unifying framework for structure from motion recovery from image sequences,
*Proc. ICCV 95*, pp. 314–320.Google Scholar - [18]D. Nister, Frame decimation for structure and motion,
*Submitted to SMILE 2000*.Google Scholar - [19]C. Poelman, T. Kanade, A paraperspective factorization method for shape and motion recovery,
*Proc. ECCV 94*, pp. 97–108.Google Scholar - [20]M. Pollefeys, R. Koch, L. Van Gool, Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters,
*IJCV*, 32(1), pp. 7–26, Aug, 1999.Google Scholar - [21]L. Quan, Invariants of 6 points from 3 uncalibrated images,
*Proc. ECCV 94*, LNCS 800/801, Stockholm, pp. 459–469. Springer Verlag, 1994.CrossRefGoogle Scholar - [22]L. Robert, O. Faugeras, Relative 3D positioning and 3D convex hull computation from a weakly calibrated stereo pair,
*Proc. ICCV 93*, pp. 540–544.Google Scholar - [23]A. Shashua, Trilinear tensor: the fundamental construct of multiple-view geometry and its applications’,
*Proc. AFPAC 97*, Kiel Germany, pp. 190–206, Sep 8–9, 1997.Google Scholar - [24]P. Torr, An assessment of information criteria for motion model selection,
*Proc. CVPR 97*, pp. 47–53.Google Scholar - [25]P. Torr, A. Zisserman, Robust parametrization and computation of the trifocal tensor,
*Image and Vision Computing*, Vol. 15, p. 591–605, 1997.CrossRefGoogle Scholar - [26]M. Spetsakis, J. Aloimonos, Structure from motion using line correspondences,
*IJCV*, pp. 171–183, 1990.Google Scholar - [27]P. Sturm, W. Triggs, A factorization based algorithm for multi-image projective structure and motion,
*Proc. ECCV 96*, pp. 709–720.Google Scholar - [28]C. Tomasi, T. Kanade, Shape and motion from image streams under orthography: a factorization approach,
*IJCV*, 9(2), pp. 137–154. November 1992.Google Scholar - [29]P. Torr, D. Murray, The development and comparison of robust methods for estimating the fundamental matrix,
*IJCV*, pp. 1–33, 1997.Google Scholar - [30]B. Triggs, Autocalibration and the absolute quadric,
*Proc. CVPR 97*, pp. 609–614.Google Scholar - [31]L. Van Gool, A. Zisserman, Automatic 3D model building from video sequences,
*Proc. ECMAST 96*, pp. 563–582.Google Scholar - [32]Z. Zhang, Determining the epipolar geometry and its uncertainty: a review,
*IJCV*, 27(2), pp. 161–195, 1997.CrossRefGoogle Scholar - [33]J. Canny, A computational approach to edge detection,
*IEEE Transactions on Pattern Analysis and Machine Intelligence*, Vol. PAMI-8, No. 6, Nov 1986.Google Scholar - [34]C. Harris, M. Stephens, A combined corner and edge detector,
*Proc. 4*^{th}*Alvey Vision Conference*, pp. 147–151, 1988.Google Scholar - [35]O. Faugeras,
*Three-Dimensional Computer Vision: a Geometric Viewpoint*, MIT Press, 1993.Google Scholar - [36]H. Longuet-Higgins, A computer algorithm for reconstructing a scene from two projections,
*Nature*, vol. 293, pp. 133–135, 1981.CrossRefGoogle Scholar - [37]M. Fischler, R. Bolles, Random sample consencus: a paradigm for model fitting with application to image analysis and automated cartography,
*Commun. Assoc. Comp. Mach.*, vol 24, pp. 381–395, 1981.MathSciNetGoogle Scholar - [38]C. Schmid and A. Zisserman, Automatic line matching across views,
*Proc. CVPR 97*, pp. 666–671.Google Scholar