Multimedia Tools and Applications

, Volume 76, Issue 1, pp 373–396 | Cite as

Advanced free viewpoint video streaming techniques

Article
  • 164 Downloads

Abstract

Free-viewpoint video is a new type of interactive multimedia service allowing users to control their viewpoint and generate new views of a dynamic scene from any perspective. The uniquely generated and displayed views are composed from two or more high bitrate camera streams that must be delivered to the users depending on their continuously changing perspective. Due to significant network and computational resource requirements, we proposed scalable viewpoint generation and delivery schemes based on multicast forwarding and distributed approach. Our aim was to find the optimal deployment locations of the distributed viewpoint synthesis processes in the network topology by allowing network nodes to act as proxy servers with caching and viewpoint synthesis functionalities. Moreover, a predictive multicast group management scheme was introduced in order to provide all camera views that may be requested in the near future and prevent the viewpoint synthesizer algorithm from remaining without camera streams. The obtained results showed that even 42 % traffic decrease can be realized using distributed viewpoint synthesis and the probability of viewpoint synthesis starvation can be also significantly reduced in future free viewpoint video services.

Keywords

Free viewpoint video Multicast streaming Viewpoint prediction Distributed networks 

References

  1. 1.
    Adams A, Nicholas J, Siadak W (2005) Protocol independent multicast - dense mode (PIM-DM). RFC 3973Google Scholar
  2. 2.
    Chakareski J (2013) Adaptive multiview video streaming: challenges and opportunities. IEEE Commun Mag 51(5):94–100CrossRefGoogle Scholar
  3. 3.
    Chiariglione L, Szabó CA (2014) Multimedia communications: technologies, services, perspectives, part II: applications, services and future directions. Infocommunications Journal VI(3):51–59Google Scholar
  4. 4.
    Chuang J, Sirbu M (1998) Pricing multicast communication: a costbased approach, presented at the INETGoogle Scholar
  5. 5.
    Domańsk M, Gotfryd M, Wegner K (2009) View synthesis for multiview video transmission. In: The 2009 International conference on image processing, computer vision, and pattern recognition, Las Vegas, 1–4Google Scholar
  6. 6.
    Fehn C (2004a) Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV. Proc of SPIE, Stereoscopic Displays Virtual Reality Syst 5291:93–104Google Scholar
  7. 7.
    Fehn C (2004b) 3D-TV using depth-image-based rendering (DIBR). Proceedings of picture coding symposium, San FranciscoGoogle Scholar
  8. 8.
    Fenner B et al. (2006) Protocol independent multicast - sparse mode (PIM-SM): protocol specification. RFC 4601Google Scholar
  9. 9.
    Guan-Ming S, Lai Y-C, Kwasinski A, Wang H (2011) 3D video communications: challenges and opportunities. Int J Commun Syst 24(10):1261–1281Google Scholar
  10. 10.
    Gurler CG, Görkemli B, Saygili G, Tekalp AM (2011) Flexible transport of 3-D video over networks. Proc IEEE 99(4):694–707CrossRefGoogle Scholar
  11. 11.
    Han Z, Dai Q (2007) A new scalable free viewpoint video streaming system over IP network. IEEE international conference on acoustics, speech and signal processing, 2007. ICASSP 2007. 2:II-773–II-776Google Scholar
  12. 12.
    Ho T-Y, Yeh Y-N, Yang D-N (2014) Multi-view 3D video multicast for broadband IP networks. arXiv preprint arXiv:1410.3977Google Scholar
  13. 13.
    Jo S, Lee D, Kim Y, Yoo C (2008) Development of a simple viewpoint video system. IEEE Int Conf Multimedia and Expo, Hannover, pp 1577–1580Google Scholar
  14. 14.
    Kellerer H, Pferschy U, Pisinger D (2004) Knapsack problems. SpringerGoogle Scholar
  15. 15.
    Kimata H, Shimizu S, Kunita Y, Isogai M, Kamikura K, Yashima Y (2008) Real-time MVC viewer for free viewpoint navigation. IEEE Int Conf MultimediaGoogle Scholar
  16. 16.
    Kurutepe E, Civanlar MR, Tekalp AM (2007a) Client-driven selective streaming of multiview video for interactive 3DTV. IEEE Trans Circuits Syst Video Technol 17(11):1558–1565CrossRefGoogle Scholar
  17. 17.
    Kurutepe E, Aksay A, Bilen C, Gurler CG, Sikora T, Akar GB, Tekalp AM (2007b) BA standards-based, flexible, end-to-end multi-view video streaming architecture. In: Proceedings of the international packet video workshop, Lausanne, pp 302–307Google Scholar
  18. 18.
    Levoy M, Hanrahan P (1996) Light field rendering. In: Proceedings of Computer graphics, SIGGRAPH96Google Scholar
  19. 19.
    Miao D et al (2011) Resource allocation for cloud-based free viewpoint video rendering for mobile phones. In: Proceedings of the 19th ACM international conference on multimedia, ACMGoogle Scholar
  20. 20.
    Ni Z, Dong T, Bhagavathy S, Llach J, Manjunath BS (2009) Improving the quality of depth image based rendering for 3D Video systems. Conf Image Processing (ICIP) 2009:7–10Google Scholar
  21. 21.
    Ns-2 – Network Simulator http:///www.isi.edu/nsnam/ns/index.html
  22. 22.
    Petrovic G, de With PHN (2006) Near-future streaming framework for 3D-TVapplications. ICMEGoogle Scholar
  23. 23.
    Petrovic G , Farin D (2007) A distributed delivery model for 3D-video streams. In: Proceedings of the first international conference on immersive telecommunications, ICSTGoogle Scholar
  24. 24.
    Phillips G, Shenker S, Tangmunarunkit H (1999) Scaling of multicast trees: comments on the chuang-sirbu scaling law. SIGCOMM ‘99, New YorkGoogle Scholar
  25. 25.
    Smolic A (2011) 3D video and free viewpoint video-from capture to display. Pattern Recogn 44(9):1958–1968MathSciNetCrossRefGoogle Scholar
  26. 26.
    Starck J, Kilner J, Hilton A (2009) A free-viewpoint video renderer. J Graphics, GPU, Game Tools 14(3):57–72CrossRefGoogle Scholar
  27. 27.
    TanimotoM TMP, Fujii T, Yendo T (2011) Free-Viewpoint TV. IEEE Signal Process Mag 28(1):67–76CrossRefGoogle Scholar
  28. 28.
    Toni L, Cheung G, Frossard P (2015) In-network view synthesis for interactive multiview video systems. arXiv preprint arXiv:1509.00464Google Scholar
  29. 29.
    Zuo L, Lou JG, Cai H, Li J (2006) Multicast of real-time multi-view video. Multimedia and Expo, 2006 I.E. Int Constr 1225(1228):9–12Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Department of Networked Systems and ServicesBudapest University of Technology and EconomicsBudapestHungary

Personalised recommendations