Frame Decimation for Structure and Motion

  • David Nistér
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2018)


A frame decimation scheme is proposed that makes automatic extraction of Structure and Motion (SaM) from handheld sequences more practical. Decimation of the number of frames used for the actual SaM calculations keeps the size of the problem manageable, regardless of the input frame rate. The proposed preprocessor is based upon global motion estimation between frames and a sharpness measure. With these tools, shot boundary detection is first performed followed by the removal of redundant frames. The frame decimation makes it feasible to feed the system with a high frame rate, which in turn avoids loss of connectivity due to matching difficulties. A high input frame rate also enables robust automatic detection of shot boundaries. The development of the preprocessor was prompted by experience with a number of test sequences, acquired directly from a handheld camera. The preprocessor was tested on this material together with a SaM algorithm. The scheme is conceptually simple and still has clear benefits.


Video Sequence Frame Rate High Frame Rate Shot Boundary Lower Frame Rate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    P. Beardsley, P. Torr, A. Zisserman, 3D model acquisition from extended image sequences, Proc. ECCV 96, pp. 683–695.Google Scholar
  2. 2.
    P. Beardsley, A. Zisserman, D. Murray, Sequential updating of projective and affine structure from motion, IJCV, 23(3), pp. 235–259, 1997.CrossRefGoogle Scholar
  3. 3.
    D. Capel, A. Zisserman, Automated mosaicing with super-resolution zoom, Proc. CVPR 98, pp. 885–891.Google Scholar
  4. 4.
    R. Cipolla, E. G. Boyer, 3D model acquisition from uncalibrated images, Proc IAPR Workshop on Machine Vision Applications, Chiba Japan, pp. 559–568, November 1998.Google Scholar
  5. 5.
    P. Debevec, C. Taylor, J. Malik, Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach, Proc. SIGGRAPH’96, pp. 11–20.Google Scholar
  6. 6.
    O. Faugeras, What can be seen in three dimensions with an uncalibrated stereo rig?, Proc.ECCV 92, pp. 563–578.Google Scholar
  7. 7.
    P. Fua, Reconstructing complex surfaces from multiple stereo views, Proc. ICCV 95, pp. 1078–1085.Google Scholar
  8. 8.
    A. W. Fitzgibbon, A. Zisserman, Automatic camera recovery for closed or open image sequences, Proc. ECCV 98, pp. 311–326.Google Scholar
  9. 9.
    K. Hanna, N. Okamoto, Combining stereo and motion for direct estimation of scene structure, Proc. ICCV 93, pp. 357–365.Google Scholar
  10. 10.
    R. Hartley, Euclidean reconstruction from uncalibrated views, Applications of Invariance in Computer Vision, LNCS 825, pp. 237–256, Springer-Verlag, 1994.Google Scholar
  11. 11.
    A. Heyden, K. Åström, Euclidean reconstruction from image sequences with varying and unknown focal length and principal point, Proc. CVPR 97, pp. 438–443.Google Scholar
  12. 12.
    M. Irani, P. Anandan, M. Cohen, Direct recovery of planar-parallax from multiple frames, Proc. Vision Algorithms Workshop (ICCV 99), Corfu Greece, pp. 1–8, September 1999.Google Scholar
  13. 13.
    K. Kanatani, N. Ohta, Accuracy bounds and optimal computation of homography for image mosaicing applications, Proc. ICCV 99, pp. 73–78.Google Scholar
  14. 14.
    P. McLauchlan, D. Murray, A unifying framework for structure from motion recovery from image sequences, Proc. ICCV 95, pp. 314–320.Google Scholar
  15. 15.
    R. Mohr, F. Veillon, L. Quan, Relative 3D reconstruction using multiple uncalibrated images, Proc. CVPR 93, pp. 543–548.Google Scholar
  16. 16.
    D. Nistér, Reconstruction from uncalibrated sequences with a hierarchy of trifocal tensors, Accepted to ECCV 2000.Google Scholar
  17. 17.
    S. Peleg, J. Herman, Panoramic mosaics by manifold projection, Proc. CVPR 97, pp. 338–343.Google Scholar
  18. 18.
    M. Pollefeys, R. Koch, L. Van Gool, Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters, IJCV, 32(1), pp. 7–26, Aug, 1999.CrossRefGoogle Scholar
  19. 19.
    W. Press, S. Teukolsky, W. Vetterling, B. Flannery, Numerical recipes in C, ISBN 0-521-43108-5, Cambridge University Press, 1988.Google Scholar
  20. 20.
    L. Robert, O. Faugeras, Relative 3D positioning and 3D convex hull computation from a weakly calibrated stereo pair, Proc. ICCV 93, pp. 540–544.Google Scholar
  21. 21.
    H. Sawhney, S. Hsu, R. Kumar, “Robust video mosaicing through topology inference and local to global alignment”, Proc. ECCV 98, pp.103–119.Google Scholar
  22. 22.
    A. Shashua, Trilinearity in visual recognition by alignment, Proc. ECCV 94, pp. 479–484.Google Scholar
  23. 23.
    M. Spetsakis, J. Aloimonos, Structure from motion using line correspondences, IJCV, pp. 171–183, 1990.Google Scholar
  24. 24.
    P. Sturm, W. Triggs, A factorization based algorithm for multi-image projective structure and motion, Proc. ECCV 96, pp. 709–720.Google Scholar
  25. 25.
    R. Szeliski, H.-Y. Shum, Creating full view panoramic image mosaics and environment maps, Proc. SIGGRAPH’97, pp. 251–258.Google Scholar
  26. 26.
    C. Tomasi, T. Kanade, Shape and motion from image streams under orthography: a factorization approach, IJCV, 9(2), pp. 137–154. November 1992.CrossRefGoogle Scholar
  27. 27.
    L. Van Gool, A. Zisserman, Automatic 3D model building from video sequences, Proc. ECMAST 96, pp. 563–582.Google Scholar


  1. 1.
    B. Triggs. Plane + Parallax, Tensors and Factorization. In Proc. European Conference on Computer Vision, pages 522–538, 2000.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • David Nistér
    • 1
  1. 1.Visual Technology, Ericsson ResearchEricsson Radio SystemsStockholmSWEDEN

Personalised recommendations