Multi-device Content Based on Video. A Practical Toolset for Creation and Delivery

  • Joan LloberaEmail author
  • Isaac Fraile
  • Juan A. Núñez
  • Szymon Malewski
  • Xavier Artigas
  • Sergi Fernandez
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 813)


The arrival of head mounted displays (HMDs) to the contemporary living room extends the need for producing content that works for the television and companion screen devices. In this work we introduce a set of three tools that can be used for the production and delivery of synchronous multi-device content, across the TV, companion screens and HMDs. The production tool is implemented as a custom Adobe Premiere Pro plugin. The publication and delivery process is implemented as an online service controlled through a web application. The content playout is implemented with a multi-device video player that combines video decoding and playout. In this article we introduce the design choices guiding our software development and the different tools we developed to realize it. We also detail some basic measures of system performance on different devices, and propose further steps towards the easy production and delivery of multi-device content.


Multi-device synchronization Multi-device content Virtual reality Omnidirectional video 



This work has been developed with the support of the European Union’s Horizon 2020 programme under grant agreement No. 688619 ( Developing an end-to-end pipeline requires competent and motivated programmers. In the work here introduced we acknowledge the contributions of Wojciech Kapsa and Daniel Piesik, from PSNC, as well as Ibai Jurado, David Gómez, Einar Meyerson, David Cassany and Juan Gordo from i2cat Foundation.


  1. 1.
    Boronat, F., Montagud, M., Marfil, D., Luzón, C.: Hybrid broadcast/broadband TV services and media synchronization: demands, preferences and expectations of spanish consumers. IEEE Trans. Broadcast. (2017)Google Scholar
  2. 2.
    Carranza, J., Theobalt, C., Magnor, M.A., Seidel, H.-P.: Free-viewpoint video of human actors. In: ACM Transactions on Graphics (TOG), pp. 569–577 (2003)CrossRefGoogle Scholar
  3. 3.
    Collet, A., Chuang, M., Sweeney, P., Gillett, D., Evseev, D., Calabrese, D., Hoppe, H., Kirk, A., Sullivan, S.: High-quality streamable free-viewpoint video. ACM Trans. Graph. (TOG) 34(4), 69 (2015)CrossRefGoogle Scholar
  4. 4.
    Courtois, C., D’heer, E.: Second screen applications and tablet users: constellation, awareness, experience, and interest. In: Proceedings of the 10th European Conference on Interactive TV and Video, pp. 153–156 (2012)Google Scholar
  5. 5.
    D’heer, E., Courtois, C.: The changing dynamics of television consumption in the multimedia living room. Convergence 22(1), 3–17 (2016)CrossRefGoogle Scholar
  6. 6.
    Gómez, D., Boronat, F., Montagud, M., Luzón, C.: End-to-end DASH platform including a network-based and client-based adaptive quality switching module. In: Proceedings of the 7th International Conference on Multimedia Systems, p. 38 (2016)Google Scholar
  7. 7.
    Huang, J., Chen, Z., Ceylan, D., Jin, H.: 6-DOF VR videos with a single 360-camera. In: Virtual Reality (VR), 2017 IEEE, pp. 37–44 (2017)Google Scholar
  8. 8.
    Meixner, B., Glancy, M., Rogers, M., Ward, C., Röggla, T., Cesar, P.: Multi-screen director: a new role in the TV production workflow? In: Adjunct Publication of the 2017 ACM International Conference on Interactive Experiences for TV and Online Video, pp. 57–62 (2017)Google Scholar
  9. 9.
    Montagud, M., Boronat, F., Stokking, H., van Brandenburg, R.: Inter-destination multimedia synchronization: schemes, use cases and standardization. Multimed. Syst. 18(6), 459–482 (2012)CrossRefGoogle Scholar
  10. 10.
    Niamut, O.A., Thomas, E., D’Acunto, L., Concolato, C., Denoual, F., Lim, S.Y.: MPEG DASH SRD: spatial relationship description. In: Proceedings of the 7th International Conference on Multimedia Systems, p. 5 (2016)Google Scholar
  11. 11.
    Rainer, B., Timmerer, C.: Self-organized inter-destination multimedia synchronization for adaptive media streaming. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 327–336 (2014)Google Scholar
  12. 12.
    Sheikh, A., Brown, A., Watson, Z., Evans, M.: Directing attention in 360-degree video, pp. 43–47 (2016).
  13. 13.
    Skupin, R., Sanchez, Y., Hellge, C., Schierl, T.: Tile based HEVC video for head mounted displays (2016)Google Scholar
  14. 14.
    Slater, M.: Place illusion and plausibility in virtual environments. Philos. Trans. R. Soc. B: Biol. Sci. 364(1535), 3549 (2009)CrossRefGoogle Scholar
  15. 15.
    Starck, J., Hilton, A.: Spherical matching for temporal correspondence of non-rigid surfaces. In: Tenth IEEE International Conference on Computer Vision (ICCV 2005), Volume 1, vol. 2, pp. 1387–1394 (2005).
  16. 16.
    Starck, J., Miller, G., Hilton, A.: Video-based character animation. In: Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation - SCA 2005, p. 49 (2005).
  17. 17.
    Veenhuizen, A., van Brandenburg, R.: Frame accurate media synchronization of heterogeneous media sources in an HBB context. In: Media Synchronization Workshop (2012)Google Scholar
  18. 18.
    Vinayagamoorthy, V., Ramdhany, R., Hammond, M.: Enabling frame-accurate synchronised companion screen experiences. In: Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video, pp. 83–92 (2016)Google Scholar
  19. 19.
    The ImmersiaTV Project. Accessed 08 Jan 2018
  20. 20.
  21. 21.
    Description of the Precision Time Protocol, as implemented in the Gstreamer framework. Accessed 08 Jan 2018
  22. 22.
    The Portal Game. Accessed 08 Jan 2018
  23. 23.
    VideoStitch Studio. Accessed 21 Feb 2018
  24. 24.
  25. 25.
    Mettle. Accessed 21 Feb 2018
  26. 26.
    Gpac. Accessed 21 Feb 2018
  27. 27.
  28. 28.
  29. 29.
    Gstreamer Movie Texture.!/content/59897. Accessed 21 Feb 2018

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.i2cat FoundationBarcelonaSpain
  2. 2.PSNCPoznańPoland

Personalised recommendations