Advertisement

The Visual Computer

, 25:997 | Cite as

Video stabilization based on a 3D perspective camera model

  • Guofeng Zhang
  • Wei Hua
  • Xueying QinEmail author
  • Yuanlong Shao
  • Hujun BaoEmail author
Original Article

Abstract

This paper presents a novel approach to stabilize video sequences based on a 3D perspective camera model. Compared to previous methods which are based on simplified models, our stabilization system can work in situations where significant depth variations exist in the scenes and the camera undergoes large translational movement. We formulate the stabilization problem as a quadratic cost function on smoothness and similarity constraints. This allows us to precisely control the smoothness by solving a sparse linear system of equations. By taking advantage of the sparseness, our optimization process is very efficient. Instead of recovering dense depths, we use approximate geometry representation and analyze the resulting warping errors. We show that by appropriately constraining warping error, visually plausible results can be achieved even using planar structures. A variety of experiments have been implemented, which demonstrates the robustness and efficiency of our approach.

Keywords

Video stabilization Structure from motion Optimization View warping Warping error 

References

  1. 1.
  2. 2.
    Barrett, R., Berry, M., Chan, T.F., Demmel, J., Donato, J., Dongarra, J., Eijkhout, V., Pozo, R., Romine, C., der Vorst, H.V.: Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, 2nd edn. SIAM, Philadelphia (1994) Google Scholar
  3. 3.
    Bhat, P., Zitnick, C.L., Snavely, N., Agarwala, A., Agrawala, M., Curless, B., Cohen, M., Kang, S.B.: Using photographs to enhance videos of a static scene. In: Kautz, J., Pattanaik, S. (eds.) Rendering Techniques 2007, Proceedings Eurographics Symposium on Rendering, pp. 327–338. Eurographics (2007) Google Scholar
  4. 4.
    Buehler, C., Bosse, M., McMillan, L.: Non-metric image-based rendering for video stabilization. In: CVPR, pp. 609–614 (2001) Google Scholar
  5. 5.
    Davis, L.S., Bajcsy, R., Herman, M.: RSTA on the move. In: Proceedings of the ARPA Image Understanding Workshop, pp. 435–456 (1994) Google Scholar
  6. 6.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. In: CommACM, vol. 24, p. 381–395 (1981) Google Scholar
  7. 7.
    Fitzgibbon, A., Zisserman, A.: Automatic camera tracking. In: Shah, M., Kumar, R. (eds.) Video Registration, pp. 18–35. Kluwer, Dordrecht (2003) Google Scholar
  8. 8.
    Hansen, M., Anadan, P., Dana, K., van de Wal, G., Burt, P.: Realtime scene stabilization and mosaic construction. In: Proc. IEEE Image Understanding Workshop, pp. 457–463 (1994) Google Scholar
  9. 9.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000) zbMATHGoogle Scholar
  10. 10.
    Litvin, A., Konrad, J., Karl, W.: Probabilistic video stabilization using Kalman filtering and mosaicking. In: IS&T/SPIE Symposium on Electronic Imaging, Image and Video Communications, pp. 663–674 (2003) Google Scholar
  11. 11.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004) CrossRefGoogle Scholar
  12. 12.
    Mark, W.R., McMillan, L., Bishop, G.: Post-rendering 3D warping. In: SI3D, pp. 7–16, 180 (1997) Google Scholar
  13. 13.
    Matsushita, Y., Ofek, E., Tang, X., Shum, H.Y.: Full-frame video stabilization. In: CVPR (1), pp. 50–57 (2005) Google Scholar
  14. 14.
    Morimoto, C., Chellappa, R.: Fast 3D stabilization and mosaic construction. In: CVPR, pp. 660–665 (1997) Google Scholar
  15. 15.
    Morimoto, C., Chellappa, R.: Evaluation of image stabilization algorithms. In: International Conference on Acoustics, Speech and Signal Processing, vol. 5, pp. 2789–2792 (1998) Google Scholar
  16. 16.
    Nistér, D.: Preemptive ransac for live structure and motion estimation. Mach. Vis. Appl. 16(5), 321–329 (2005) CrossRefGoogle Scholar
  17. 17.
    Pilu, M.: Video stabilization as a variational problem and numerical solution with the Viterbi method. In: CVPR (1), pp. 625–630 (2004) Google Scholar
  18. 18.
    Pollefeys, M., Gool, L.J.V., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. Int. J. Comput. Vis. 59(3), 207–232 (2004) CrossRefGoogle Scholar
  19. 19.
  20. 20.
    Shade, J., Gortler, S.J., wei He, L., Szeliski, R.: Layered depth images. In: SIGGRAPH, pp. 231–242 (1998) Google Scholar
  21. 21.
    Snavely, N., Seitz, S.M., Szeliski, R.: Photo-tourism: exploring photo-collections in 3D. ACM Trans. Graph. 25(3), 835–846 (2006) CrossRefGoogle Scholar
  22. 22.
  23. 23.
    Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment—a modern synthesis. In: Workshop on Vision Algorithms, pp. 298–372 (1999) Google Scholar
  24. 24.
    Yao, Y., Burlina, P., Chellappa, R., Wu, T.: Electronic image stabilization using multiple visual cues. In: Proceedings of International Conference on Image Processing, pp. 191–194 (1995) Google Scholar
  25. 25.
    Zhang, G., Qin, X., Hua, W., Wong, T.T., Heng, P.A., Bao, H.: Robust metric reconstruction from challenging video sequences. In: CVPR (2007) Google Scholar
  26. 26.
    Zhang, Z.: Determining the epipolar geometry and its uncertainty: A review. Int. J. Comput. Vis. 27(2), 161–195 (1998) CrossRefGoogle Scholar
  27. 27.
    Zhu, Z., Xu, G., Yang, Y., Jin, J.S.: Camera stabilization based on 2.5D motion model estimation and inertial motion filtering. In: IEEE Conference on Intelligent Vehicles, pp. 329–334 (1998) Google Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  1. 1.State Lab of CAD&CGZhejiang UniversityZhejiangPeople’s Republic of China
  2. 2.School of Computer Science & TechnologyShandong UniversityJinanPeople’s Republic of China

Personalised recommendations