Advertisement

Automatic View Tracking in 360º Multimedia Using xAPI

  • Antoni Oliver
  • Javier del Molino
  • Antoni Bibiloni
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 813)

Abstract

360º video can consume up to six times the bandwidth of a regular video by delivering the entire frames instead of just the current viewport, introducing an additional difficulty to the delivery of this kind of multimedia. Many authors address this challenge by narrowing the delivered viewport using knowledge of where the user is likely to look at. To address this, we propose an automatic view tracking system based in xAPI to collect the data required to create the knowledge required to decide the viewport that is going to be delivered. We present a use case of an interactive 360º video documentary around the migratory crisis in Greece. In this case, omnidirectional content is recorded using a 6-camera array, rendered in an equirectangular projection and played later by an HTML5 web application. Interactive hotspots are placed on specific coordinates of the space-time of the production, introducing a connection of the viewer with the story by playing additional multimedia content. The current view and other usage data are recorded and permit us to obtain metrics on the user behavior, like most watched areas, with the goal to obtain that required knowledge.

Keywords

Hypervideo 360º video Interactive documentary Video tiling View-adaptive streaming xAPI 

Notes

Acknowledgements

The authors want to thank the people that appear in the documentary for sharing their stories with us. We also want to thank PROEM-AID for their work where they are much needed, and especially Manuel Elviro Vidal for recording the scenes shown in the documentary in situ. A. Oliver was supported by a predoctoral contract with the Universitat de les Illes Balears (UIB).

References

  1. 1.
    Sawhney, N., Balcom, D., Smith, I.: HyperCafe. In: Proceedings of the Seventh ACM Conference on Hypertext – HYPERTEXT 1996, pp. 1–10. ACM Press, New York (1996)Google Scholar
  2. 2.
    Gaudenzi, S.: The living documentary: from representing reality to co-creating reality in digital interactive documentary (2013)Google Scholar
  3. 3.
    Bleumers, L., Van den Broeck, W., Lievens, B., Pierson, J.: Seeing the bigger picture. In: Proceedings of the 10th European Conference on Interactive TV and Video – EuroiTV 2012, p. 115 (2012)Google Scholar
  4. 4.
    Sherman, W.R., Craig, A.B.: Understanding Virtual Reality: Interface, Application, and Design. Morgan Kaufmann, Los Altos (2003)Google Scholar
  5. 5.
    Neumann, U., Pintaric, T., Rizzo, A.: Immersive panoramic video. In: Proceedings of the Eighth ACM International Conference on Multimedia - MULTIMEDIA 2000, pp. 493–494. ACM Press, New York (2000)Google Scholar
  6. 6.
    Lantz, E.: The future of virtual reality. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques – SIGGRAPH 1996, pp. 485–486. ACM Press, New York (1996)Google Scholar
  7. 7.
    Kavanagh, S., Luxton-Reilly, A., Wüensche, B., Plimmer, B.: Creating 360° educational video. In: Proceedings of the 28th Australian Conference on Computer-Human Interaction – OzCHI 2016, pp. 34–39. ACM Press, New York (2016)Google Scholar
  8. 8.
    Neng, L.A.R., Chambel, T.: Get around 360° hypervideo. In: Proceedings of the 14th International Academic MindTrek Conference on Envisioning Future Media Environments – MindTrek 2010, p. 119. ACM Press, New York (2010)Google Scholar
  9. 9.
    Berning, M., Yonezawa, T., Riedel, T., Nakazawa, J., Beigl, M., Tokuda, H.: pARnorama. In: Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication – UbiComp 2013 Adjunct, pp. 1471–1474. ACM Press, New York (2013)Google Scholar
  10. 10.
    Kasahara, S., Nagai, S., Rekimoto, J.: JackIn Head: immersive visual telepresence system with omnidirectional wearable camera. IEEE Trans. Vis. Comput. Graph. 23, 1222–1234 (2017)CrossRefGoogle Scholar
  11. 11.
    Williamson, J.R., Sundén, D., Bradley, J.: GlobalFestival: evaluating real world interaction on a spherical display. In: Proceedings of the Joint International Conference on Pervasive and Ubiquitous Computing, International Symposium on Wearable Computers, pp. 1251–1261 (2015)Google Scholar
  12. 12.
    Sodagar, I.: The MPEG-DASH standard for multimedia streaming over the internet. IEEE Multimed. 18, 62–67 (2011)CrossRefGoogle Scholar
  13. 13.
    Bao, Y., Wu, H., Zhang, T., Ramli, A.A., Liu, X.: Shooting a moving target: motion-prediction-based transmission for 360-degree videos. In: Joshi, J., Karypis, G., Liu, L., Hu, X., Ak, R., Xia, Y., Xu, W., Sato, A.H., Rachuri, S., Ungar, L., Yu, P.S., Govindaraju, R., Suzumura, T. (eds.) 2016 IEEE International Conference on Big Data (Big Data), pp. 1161–1170. IEEE, New York (2016)Google Scholar
  14. 14.
    Bao, Y., Wu, H., Ramli, A.A., Wang, B., Liu, X.: Viewing 360 degree videos: motion prediction and bandwidth optimization. In: 2016 IEEE 24th International Conference on Network Protocols (ICNP) (2016)Google Scholar
  15. 15.
    Qian, F., Ji, L., Han, B., Gopalakrishnan, V.: Optimizing 360 video delivery over cellular networks. In: Proceedings of the 5th Workshop on All Things Cellular Operations, Applications and Challenges – ATC 2016, pp. 1–6. ACM Press, New York (2016)Google Scholar
  16. 16.
    Hosseini, M., Swaminathan, V.: Adaptive 360 VR video streaming: divide and conquer (2016)Google Scholar
  17. 17.
    Niamut, O.A., Thomas, E., D’Acunto, L., Concolato, C., Denoual, F., Lim, S.Y.: MPEG DASH SRD. In: Proceedings of the 7th International Conference on Multimedia Systems – MMSys 2016, pp. 1–8. ACM Press, New York (2016)Google Scholar
  18. 18.
    Corbillon, X., Simon, G., Devlic, A., Chakareski, J.: Viewport-adaptive navigable 360-degree video delivery. In: 2017 IEEE International Conference on Communications (ICC), pp. 1–7. IEEE (2017)Google Scholar
  19. 19.
    Kuzyakov, E., Pio, D.: Next-generation video encoding techniques for 360 video and VR. https://code.facebook.com/posts/1126354007399553/next-generation-video-encoding-techniques-for-360-video-and-vr/
  20. 20.
  21. 21.
    Kuzyakov, E., Chen, S., Peng, R.: Enhancing high-resolution 360 streaming with view prediction. https://code.facebook.com/posts/118926451990297/enhancing-high-resolution-360-streaming-with-view-prediction/
  22. 22.
    Bibiloni, T., Oliver, A., del Molino, J.: Automatic collection of user behavior in 360° multimedia. Multimed. Tools Appl. 1–18 (2017).  https://doi.org/10.1007/s11042-017-5510-3
  23. 23.
    Experience API (xAPI) - Video Vocabulary. http://xapi.vocab.pub/datasets/video/
  24. 24.

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Laboratorio de Tecnologías de la Información Multimedia (LTIM), Departamento de Matemáticas e InformáticaUniversitat de les Illes BalearsPalmaSpain

Personalised recommendations