Advertisement

Towards Semantic Fast-Forward and Stabilized Egocentric Videos

  • Michel Melo SilvaEmail author
  • Washington Luis Souza Ramos
  • Joao Pedro Klock Ferreira
  • Mario Fernando Montenegro Campos
  • Erickson Rangel Nascimento
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9913)

Abstract

The emergence of low-cost personal mobiles devices and wearable cameras and the increasing storage capacity of video-sharing websites have pushed forward a growing interest towards first-person videos. Since most of the recorded videos compose long-running streams with unedited content, they are tedious and unpleasant to watch. The fast-forward state-of-the-art methods are facing challenges of balancing the smoothness of the video and the emphasis in the relevant frames given a speed-up rate. In this work, we present a methodology capable of summarizing and stabilizing egocentric videos by extracting the semantic information from the frames. This paper also describes a dataset collection with several semantically labeled videos and introduces a new smoothness evaluation metric for egocentric videos that is used to test our method.

Keywords

Semantic information First-person video Fast-forward Egocentric stabilization 

Notes

Acknowledgments

The authors would like to thank the agencies CAPES, CNPq, FAPEMIG, ITV (Vale Institute of Technology) and Petrobras for funding different parts of this work.

Supplementary material

Supplementary material 1 (mp4 7759 KB)

References

  1. 1.
    Dollár, P.: Piotr’s Computer Vision Matlab Toolbox (PMT). https://github.com/pdollar/toolbox
  2. 2.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381395 (1981)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Gong, Y., Liu, X.: Video summarization using singular value decomposition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 174–180, Hilton Head Island, SC, USA, June 2000Google Scholar
  4. 4.
    Gygli, M., Grabner, H., Riemenschneider, H., Van Gool, L.: Creating summaries from user videos. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part VII. LNCS, vol. 8695, pp. 505–520. Springer, Heidelberg (2014)Google Scholar
  5. 5.
    Halperin, T., Poleg, Y., Arora, C., Peleg, S.: Egosampling: wide view hyperlapse from single and multiple egocentric videos. CoRR abs/1604.07741 (2016). http://arxiv.org/abs/1604.07741
  6. 6.
    Hari, R., Roopesh, C., Wilscy, M.: Human face based approach for video summarization. In: 2013 IEEE Recent Advances in Intelligent Computational Systems (RAICS), Trivandrum, India, pp. 245–250, December 2013Google Scholar
  7. 7.
    Hsu, Y.F., Chou, C.C., Shih, M.Y.: Moving camera video stabilization using homography consistency. In: 2012 19th IEEE International Conference on Image Processing, Orlando, FL, USA. IEEE, September 2012Google Scholar
  8. 8.
    Joshi, N., Kienzle, W., Toelle, M., Uyttendaele, M., Cohen, M.F.: Real-time hyperlapse creation via optimal frame selection. ACM Trans. Graph. 34(4), 63:1–63:9 (2015)CrossRefGoogle Scholar
  9. 9.
    Kopf, J., Cohen, M.F., Szeliski, R.: First-person hyper-lapse videos - supplemental material. http://research.microsoft.com/en-us/um/redmond/projects/hyperlapse/supplementary/index.html. Accessed 26 July 2016
  10. 10.
    Kopf, J., Cohen, M.F., Szeliski, R.: First-person hyper-lapse videos. ACM Trans. Graph. 33(4), 78:1–78:10 (2014)CrossRefGoogle Scholar
  11. 11.
    Lee, Y.J., Ghosh, J., Grauman, K.: Discovering important people and objects for egocentric video summarization. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, pp. 1346–1353, June 2012Google Scholar
  12. 12.
    Liao, S., Jain, A., Li, S.: A fast and accurate unconstrained face detector. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 211–223 (2016)CrossRefGoogle Scholar
  13. 13.
    Lu, Z., Grauman, K.: Story-driven summarization for egocentric video. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA, pp. 2714–2721, June 2013Google Scholar
  14. 14.
    Ngo, C.W., Ma, Y.F., Zhang, H.: Automatic video summarization by graph modeling. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, Nice, France, vol. 1, pp. 104–109, October 2003Google Scholar
  15. 15.
    Poleg, Y., Ephrat, A., Peleg, S., Arora, C.: Compact cnn for indexing egocentric videos. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Placid, NY, USA, pp. 1–9, March 2016Google Scholar
  16. 16.
    Poleg, Y., Halperin, T., Arora, C., Peleg, S.: Egosampling: fast-forward and stereo for egocentric videos. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, pp. 4768–4776, June 2015Google Scholar
  17. 17.
    Potapov, D., Douze, M., Harchaoui, Z., Schmid, C.: Category-specific video summarization. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part VI. LNCS, vol. 8694, pp. 540–555. Springer, Heidelberg (2014)Google Scholar
  18. 18.
    Ramos, W.L.S., Silva, M.M., Campos, M.F.M., Nascimento, E.R.: Fast-forward video based on semantic extraction. In: 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AR, USA. IEEE, September 2016Google Scholar
  19. 19.
    Yu, J., Kankanhalli, M., Mulhen, P.: Semantic video summarization in compressed domain MPEG video. In: Proceedings of the 2003 International Conference on Multimedia and Expo, ICME 2003, Baltimore, MD, USA, vol. 3, pp. III–329–332, July 2003Google Scholar
  20. 20.
    Zhuang, Y., Xiao, R., Wu, F.: Key issues in video summarization and its application. In: Proceedings of the 2003 Joint Conference of the Fourth International Conference on Information, Communications and Signal Processing and Fourth Pacific Rim Conference on Multimedia, Singapore, vol. 1, pp. 448–452, December 2003Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Michel Melo Silva
    • 1
    Email author
  • Washington Luis Souza Ramos
    • 1
  • Joao Pedro Klock Ferreira
    • 1
  • Mario Fernando Montenegro Campos
    • 1
  • Erickson Rangel Nascimento
    • 1
  1. 1.Departamento de Ciência da ComputaçãoUniversidade Federal de Minas GeraisBelo HorizonteBrazil

Personalised recommendations