Advertisement

Journal of Real-Time Image Processing

, Volume 12, Issue 2, pp 329–342 | Cite as

A DASH-based HEVC multi-view video streaming system

  • Tianyu SuEmail author
  • Ashkan Sobhani
  • Abdulsalam Yassine
  • Shervin Shirmohammadi
  • Abbas Javadtalab
Special Issue Paper

Abstract

Recent advancement in cameras and image processing technology has generated a paradigm shift from traditional 2D and 3D video to multi-view video (MVV) technology, while at the same time improving video quality and compression through standards such as high efficiency video coding (HEVC). In multi-view, cameras are placed in predetermined positions to capture the video from various views. Delivering such views with high quality over the Internet is a challenging prospect, as MVV traffic is several times larger than traditional video, since it consists of multiple video sequences, each captured from a different angle, requiring more bandwidth than single-view video to transmit MVV. In addition, the Internet is known to be prone to packet loss, delay, and bandwidth variation, which adversely affect MVV transmission. Another challenge is that end users’ devices have different capabilities in terms of computing power, display, and access link capacity, requiring MVV to be adapted to each user’s context. In this paper, we propose an HEVC multi-view system using Dynamic Adaptive Streaming over HTTP to overcome the above-mentioned challenges. Our system uses an adaptive mechanism to adjust the video bit rate to the variations of bandwidth in best effort networks. We also propose a novel scalable way for the multi-view video and depth content for 3D video in terms of the number of transmitted views. Our objective measurements show that our method of transmitting MVV content can maximize the perceptual quality of virtual views after the rendering and hence increase the user’s quality of experience.

Keywords

High efficiency video coding (HEVC) Dynamic Adaptive Streaming over HTTP (DASH) Multi-view plus depth coding (MVD) Multi-view video coding video scalability and adaptation Quality of experience Depth image-based rendering Rate-distortion optimization 

References

  1. 1.
    Benzie, P., Watson, J., Member, S., Surman, P., Rakkolainen, I., Hopf, K., Urey, H., Sainov, V., Von Kopylow, C.: A survey of 3dtv displays : techniques and technologies. IEEE Trans. Circuit Syst. Video Technol. 17(11), 1647–1658 (2007)CrossRefGoogle Scholar
  2. 2.
    Buchowicz, A.: Video coding and transmission standards for 3D television—a survey. Opto Electron. Rev. 21(1), 39–51 (2012)Google Scholar
  3. 3.
    Tanimoto, M.: Free-Viewpoint Television. In: R. Ronfard, R., Taubin, G., (eds.) Image and Geometry Processing for 3-D Cinematography, vol. 5, pp. 53–76. Springer, Berlin (2010)Google Scholar
  4. 4.
    Kauff, P., Atzpadin, N., Fehn, C., Müller, M., Schreer, O., Smolic, A., Tanger, R.: Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability”. Signal Process. Image Commun. 22(2), 217–234 (2007)CrossRefGoogle Scholar
  5. 5.
    Müller, K., Schwarz, H., Marpe, D., Bartnik, C., Bosse, S., Brust, H., Hinz, T., Lakshman, H., Merkle, P., Rhee, F.H., Tech, G., Winken, M., Wiegand, T.: 3D high-efficiency video coding for multi-view video and depth data. IEEE Trans. Image Process. 22(9), 3366–3378 (2013)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Ozcinar, C.: Ekmekcioglu, E., Kondoz, A.: Dynamic adaptive 3D multi-view video streaming over the internet. In: Proceedings of the 2013 ACM international workshop on Immersive media experiences—ImmersiveMe’13, pp. 51–56. (2013)Google Scholar
  7. 7.
    Sullivan, G.J., Ohm, J., Han, W., Wiegand, T.: Overview of the high efficiency video coding. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1668 (2012)CrossRefGoogle Scholar
  8. 8.
    Müller, K., Schwarz, H., Marpe, D., Bartnik, C., Bosse, S., Brust, H., Hinz, T., Lakshman, H., Merkle, P., Rhee, F.H., Tech, G., Winken, M., Wiegand, T.: 3D high-efficiency video coding for multi-view video and depth data. IEEE Trans. Image Process. 22(9), 3366–3378 (2013)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Ohm, J., Sullivan, G.J., Schwarz, H., Tan, T.K., Member, S., Wiegand, T.: Comparison of the coding efficiency of video coding standards—Including high efficiency video coding (HEVC). IEEE Trans. Circuits Syst. Video Technol. 22(12), 1669–1684 (2012)CrossRefGoogle Scholar
  10. 10.
    H.-H.-I. Fruanhofer, “HEVC 3D extension Test Model(3DV HTM) version 11.0,” 2013. [Online]. Available: https://hevc.hhi.fraunhofer.de/svn/svn_3DVCSoftware/tags/HTM-11.0/
  11. 11.
    (MPEG) IJSW. Dynamic adaptive streaming over http. w11578, CD 23001-6, w11578, CD 23001-6. ISO/IEC JTC 1/SC 29/WG 11 (MPEG), Guangzhou, China, (2010)Google Scholar
  12. 12.
    Stockhammer, T.: Dynamic adaptive streaming over HTTP—Standards and design principles. In: Proceedings of the Second Annual ACM Conference on Multimedia Systems (MMSYS 2011), no. i, pp. 133–143 (2011)Google Scholar
  13. 13.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Member, S., Simoncelli, E.P., Member, S.: Image quality assessment : from error visibility to structural similarity. IEEE Trans. Circuits Syst. Video Technol. 13(4), 600–612 (2004)Google Scholar
  14. 14.
    Wiegand, T., Schwarz, H., Joch, A., Kossentini, F., Member, S., Sullivan, G.J.: Rate-constrained coder control and comparison of video coding standards. IEEE Trans. Circuits Syst. Video Technol. 13(7), 688–703 (2003)CrossRefGoogle Scholar
  15. 15.
    Begen, A., Akgul, T., Baugher, M.: Watching video over the web: part 1: streaming protocols. J. IEEE Internet Comput. 15(2), 54–63 (2011)CrossRefGoogle Scholar
  16. 16.
    Kuschnig, R., Kofler, I., Hellwagner, H.: Evaluation of HTTP-based request-response streams for internet video streaming. In: Proceedings of the second annual ACM conference on Multimedia systems. (San Jose, California, USA, February 23–25, 2011) MMSys `11. pp. 245–256. ACM, New York (2011)Google Scholar
  17. 17.
    Pantos, R., May, W.: HTTP Live Streaming. Internet Draft IETF Draft. IETF Tools. http://tools.ietf.org/html/draft-pantos-http-live-streaming-04 (2010)
  18. 18.
    Zambelli, A.: IIS smooth streaming technical overview. Microsoft Corporation (2009)Google Scholar
  19. 19.
    Hassoun, D.: Dynamic streaming in flash media server 3.5. Adobe. http://www.adobe.com/devnet/adobe-media-server/articles/dynstream_advanced_pt1.html (2010)
  20. 20.
  21. 21.
    Lohmar, T., Einarsson, T., Frojdh, P., Gabin, F., Kampmann, M.: Dynamic adaptive HTTP streaming of live content. In: Proceedings of the 12th IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks (Lucca, Italy, 20–24 June, 2011) WoWMoM ‘11. pp. 1–8 (2011)Google Scholar
  22. 22.
    Javadtalab, A., Semsarzadeh, M., Khanchi, A., Shirmohammandi, S., Yassine, A.: Continuous one-way detection of available bandwidth changes for video streaming over best-effort networks. IEEE Trans. Instrum. Meas. 64(1), 190–203 (2015)CrossRefGoogle Scholar
  23. 23.
    Ba, K.T., Tekalp, A.M., Adaptive stereoscopic 3D video streaming. In: Image Processing (ICIP), 2010 17th IEEE International Conference on, pp. 2409–2412 (2010)Google Scholar
  24. 24.
    Hamza A, Hefeeda, M.: A DASH-based free viewpoint video streaming system. In: Proceedings of Network and Operating System Support on Digital Audio and Video Workshop, p. 55. (2013)Google Scholar
  25. 25.
    Calagari, K.: Anahita: A system for 3D Video streaming with depth customization categories and subject descriptors. In: Proceedings of the ACM International Conference on Multimedia, pp. 337–346 (2014)Google Scholar
  26. 26.
    Psannis, K.E., Hadjinicolaou, M., Krikelis, A.: MPEG-2 streaming of full interactive content. IEEE Trans. Circuits Syst. Video Technol. 16(2), 280–285 (2006)CrossRefGoogle Scholar
  27. 27.
    Schierl, T., Hannuksela, M.M., Wang, Y.-K., Wenger, S.: System layer integration of high efficiency video coding (HEVC). IEEE Trans. Circuits Syst. Video Technol. 22(12), 1871–1884 (2012)CrossRefGoogle Scholar
  28. 28.
    Wenger, S.: H.264/AVC over IP, IEEE. Trans. Circuits Syst. 13(7), (2003)Google Scholar
  29. 29.
    Roodaki, H., Hashemi, M.R., Shirmohammadi, S.: A new methodology to derive objective quality assessment metrics for scalable multi-view 3D video coding. ACM Trans. Multimed. Comput. Commun. Appl. 8(3s), 25 (2012). (Article 44). doi: 10.1145/2348816.2348823
  30. 30.
    Roodaki, H., Hashemi, M. R., Shirmohammadi, S.: Rate-Distortion optimization for scalable multi-view video coding. In: Proceedings IEEE International Conference on Multimedia and Expo, p. 6. Chengdu (2014). doi: 10.1109/ICME.2014.6890275
  31. 31.
    Oztas, B., Pourazad, M.T., Nasiopoulos, P., Sodagar, I., Leung, V.C.M.: A rate adaptation approach for streaming multiview plus depth content. In: Computing, Networking and Communications (ICNC), 2014 International Conference on, no. Mvd, pp. 1006–1010 (2013)Google Scholar
  32. 32.
    Bosc, E., Racapé, F., Jantet, V., Riou, P., Pressigout, M., Morin, L.: A study of depth/texture bit-rate allocation in multi-view video plus depth compression. Ann. Telecommun. Ann. Des Télécommun. 68(11–12), 615–625 (2013)CrossRefGoogle Scholar
  33. 33.
    Li, B., Li, H., Li, L., Jinlei, Z.: Rate control by R-lambda model for HEVC. Jt. Collab. Team Video Coding(JCT-VC)of ITU-T SG 16 WP 3 ISO/IEC JTC 1/SC 29/WG 11, pp. 1–11 (2012)Google Scholar
  34. 34.
    Roodaki, H., Hashemi, M.R., Shirmohammadi, S.: New scalable modalities in multi-view 3D video. In: Proceedings ACM Workshop on Mobile Video, Oslo, Norway, pp. 25–30 (2013) doi: 10.1145/2457413.2457420
  35. 35.
    Ndjiki-nya, P., Köppel, M., Doshkov, D., Lakshman, H., Merkle, P., Muller, K., Wiegand, T.: Depth image-based rendering with advanced texture synthesis for 3-D video. IEEE Trans. Multimed. 13(3), 453–465 (2011)CrossRefGoogle Scholar
  36. 36.
    Dimenco: Non-glass 3D displayer. http://www.dimenco.eu/3d-displays/displays/65-inch-4k/(2014)
  37. 37.
    Alioscopy: Alioscopy 3D HD 55” LV data sheet. http://www.alioscopy.com/en/datasheet.php?model=Alioscopy 3D HD 47%22 LV. (2010)
  38. 38.
    Tanimoto Lab at Nagoya University: Kendo Test sequences. http://www.tanimoto.nuee.nagoya-u.ac.jp/
  39. 39.
    Ho, Y.-S., Lee, E.-K., Cheon, L.: Newspaper, multiview video test sequence and camera parameters. In: International Organisation For Standardisation Organisation I ISO/IEC JTC1/SC29/WG11 Coding of Moving Pictures and Audio, pp. 1–6. (2008)Google Scholar
  40. 40.
    Call for Proposals on 3D Video Coding Technology. ISO/IEC JTC1/SC29/WG11 MPEG2011/N12036, Geneva, (2011)Google Scholar
  41. 41.
    Rizzo, L.: Dummynet : a simple approach to the evaluation of network protocols. ACM SIGCOMM Comput. Commun. Rev. 27(1997), 31–41 (1997)CrossRefGoogle Scholar
  42. 42.
    Internet Information Service. http://www.iis.net (2014)
  43. 43.
    Thang, T.C., Le, H.T., Member, S., Pham, A.T., Member, S., Ro, Y.M.: An evaluation of bitrate adaptation methods for HTTP live streaming. IEEE J. Sel. ATREAS Commun. 32(4), 693–705 (2014)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Distributed and Collaborative Virtual Environments Research Laboratory (DISCOVER)University of OttawaOttawaCanada

Personalised recommendations