Non-Parametric Sequential Frame Decimation for Scene Reconstruction in Low-Memory Streaming Environments

  • Daniel Knoblauch
  • Mauricio Hess-Flores
  • Mark A. Duchaineau
  • Kenneth I. Joy
  • Falko Kuester
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6938)

Abstract

This paper introduces a non-parametric sequential frame decimation algorithm for image sequences in low-memory streaming environments. Frame decimation reduces the number of input frames to increase pose and structure robustness in Structure and Motion (SaM) applications. The main contribution of this paper is the introduction of a sequential low-memory work-flow for frame decimation in embedded systems where memory and memory traffic come at a premium. This approach acts as an online preprocessing filter by removing frames that are ill-posed for reconstruction before streaming. The introduced sequential approach reduces the number of needed frames in memory to three in contrast to global frame decimation approaches that use at least ten frames in memory and is therefore suitable for low-memory streaming environments. This is moreover important in emerging systems with large format cameras which acquire data over several hours and therefore render global approaches impossible.

In this paper a new decimation metric is designed which facilitates sequential keyframe extraction fit for reconstruction purposes, based on factors such as a correspondence-to-feature ratio and residual error relationships between epipolar geometry and homography estimation. The specific design of the error metric allows a local sequential decimation metric evaluation and can therefore be used on the fly. The approach has been tested with various types of input sequences and results in reliable low-memory frame decimation robust to different frame sampling frequencies and independent of any thresholds, scene assumptions or global frame analysis.

Keywords

Camera Movement Fundamental Matrix Epipolar Geometry Input Frame Reprojection Error 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Pollefeys, M., Van Gool, L., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. International Journal of Computer Vision 59, 207–232 (2004), doi:10.1023/B:VISI.0000025798.50602.3aCrossRefGoogle Scholar
  2. 2.
    Nistér, D.: Reconstruction from uncalibrated sequences with a hierarchy of trifocal tensors. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 649–663. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  3. 3.
    Lowe, D.: Object recognition from local scale-invariant features. In: ICCV, pp. 1150–1157. IEEE Computer Society, Los Alamitos (1999)Google Scholar
  4. 4.
    Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: Speeded-up robust features (SURF). Computer Vision and Image Understanding 110, 346–359 (2008); Similarity Matching in Computer Vision and MultimediaCrossRefGoogle Scholar
  5. 5.
    Nistér, D.: Frame decimation for structure and motion. In: Pollefeys, M., Van Gool, L., Zisserman, A., Fitzgibbon, A.W. (eds.) SMILE 2000. LNCS, vol. 2018, pp. 17–34. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  6. 6.
    Ahmed, M., Dailey, M., Landabaso, J., Herrero, N.: Robust key frame extraction for 3D reconstruction from video streams. In: International Conference on Computer Vision Theory and Applications (VISAPP), pp. 231–236 (2010)Google Scholar
  7. 7.
    Torr, P.H.: Geometric motion segmentation and model selection. Philosophical Transactions: Mathematical, Physical and Engineering Sciences 356, 1321–1340 (1998)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Royer, E., Lhuillier, M., Dhome, M., Lavest, J.M.: Monocular vision for mobile robot localization and autonomous navigation. International Journal of Computer Vision 74, 237–260 (2007), doi:10.1007/s11263-006-0023-yCrossRefMATHGoogle Scholar
  9. 9.
    Torr, P.H., Fitzgibbon, A.W., Zisserman, A.: The problem of degeneracy in structure and motion recovery from uncalibrated image sequences. International Journal of Computer Vision 32, 27–44 (1999), doi:10.1023/A:1008140928553CrossRefGoogle Scholar
  10. 10.
    Beder, C., Steffen, R.: Determining an initial image pair for fixing the scale of a 3d reconstruction from an image sequence. In: Franke, K., Müller, K.-R., Nickolay, B., Schäfer, R. (eds.) DAGM 2006. LNCS, vol. 4174, pp. 657–666. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  11. 11.
    Strecha, C., von Hansen, W., Van Gool, L., Fua, P., Thoennessen, U.: On benchmarking camera calibration and multi-view stereo for high resolution imagery. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8 (2008)Google Scholar
  12. 12.
    Fitzgibbon, A.W., Cross, G., Zisserman, A.: Automatic 3D model construction for turn-table sequences. In: Koch, R., Van Gool, L. (eds.) SMILE 1998. LNCS, vol. 1506, pp. 155–170. Springer, Heidelberg (1998)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Daniel Knoblauch
    • 1
  • Mauricio Hess-Flores
    • 2
  • Mark A. Duchaineau
    • 3
  • Kenneth I. Joy
    • 2
  • Falko Kuester
    • 1
  1. 1.University of CaliforniaSan DiegoUSA
  2. 2.University of CaliforniaDavisUSA
  3. 3.Lawrence Livermore National LaboratoryLivermoreUSA

Personalised recommendations