Advertisement

Hybrid Lossless-Lossy Compression for Real-Time Depth-Sensor Streams in 3D Telepresence Applications

  • Yunpeng Liu
  • Stephan Beck
  • Renfang Wang
  • Jin Li
  • Huixia Xu
  • Shijie Yao
  • Xiaopeng Tong
  • Bernd Froehlich
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9314)

Abstract

We developed and evaluated different schemes for the real-time compression of multiple depth image streams. Our analysis suggests that a hybrid lossless-lossy compression approach provides a good tradeoff between quality and compression ratio. Lossless compression based on run length encoding is used to preserve the information of the highest bits of the depth image pixels. The lowest 10-bits of a depth pixel value are directly encoded in the Y channel of a YUV image and encoded by a x264 codec. Our experiments show that the proposed method can encode and decode multiple depth image streams in less than 12 ms on average. Depending on the compression level, which can be adjusted during application runtime, we are able to achieve a compression ratio of about 4:1 to 20:1. Initial results indicate that the quality for 3D reconstructions is almost indistinguishable from the original for a compression ratio of up to 10:1.

Keywords

Depth map Compression x264 3D Tele-immersion 

Notes

Acknowledgments

The author gratefully acknowledges the support of K. C. Wong Education Foundation and DAAD, NSFC (Grant No. 61073074, 61303144), Projects in Science and Technique of Ningbo Municipal (Grant No. 2012B82003), Zhejiang Higher Education Reform Project (Grant No. jg2013135), National Students’ Innovation and Entrepreneurship Project (Grant No. 201410876012).

References

  1. 1.
    Bal, C., Nguyen, T.: Multiview video plus depth coding with depth-based prediction mode. IEEE Trans. Circuits Syst. Video Technol. 24(6), 995–1005 (2014)CrossRefGoogle Scholar
  2. 2.
    Beck, S., Kunert, A., Kulik, A., Froehlich, B.: Immersive group-to-group telepresence. IEEE Trans. Visual Comput. Graphics 19(4), 616–625 (2013)CrossRefGoogle Scholar
  3. 3.
    Champawat, Y., Kumar, S.: Online point-cloud transmission for tele-immersion. In: Proceedings of VRCAI 2012, pp. 79–82. ACM, New York (2012)Google Scholar
  4. 4.
    Cho, T., Lee, Y., Shin, J.: A homogenizing filter for depth map compressive sensing using edge-awarded method. In: Proceedings of ICT Convergence 2013, pp. 591–595, October 2013Google Scholar
  5. 5.
    Choi, J.-A., Ho, Y.-S.: Improved near-lossless hevc codec for depth map based on statistical analysis of residual data. In: Proceedings of ISCS 2012, pp. 894–897, May 2012Google Scholar
  6. 6.
    FFMpeg. Ffmpeg (2014, nov. version 2.6.1): audio and video codec library. https://www.ffmpeg.org/
  7. 7.
    Fu, J., Miao, D., Yu, W., Wang, S., Lu, Y., Li, S.: Kinect-like depth data compression. IEEE Trans. Multimedia 15(6), 1340–1352 (2013)CrossRefGoogle Scholar
  8. 8.
    Gautier, J., Le Meur, O., Guillemot, C.: Efficient depth map compression based on lossless edge coding and diffusion. In: Proceedings of PCS 2012, pp. 81–84, May 2012Google Scholar
  9. 9.
    Hoffmann, S., Mainberger, M., Weickert, J., Puhl, M.: Compression of depth maps with segment-based homogeneous diffusion. In: Pack, T. (ed.) SSVM 2013. LNCS, vol. 7893, pp. 319–330. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  10. 10.
    Ko, H., Kuo, C.-C.: A new in-loop filter for depth map coding in hevc. In: Proceedings of APSIPA ASC 2012, pp. 1–7, December 2012Google Scholar
  11. 11.
    Lan, C., Xu, J., Wu, F.: Object-based coding for kinect depth and color videos. In: Proceedings of VCIP 2012, pp. 1–6, November 2012Google Scholar
  12. 12.
    Mamou, K., Zaharia, T., Prêteux, F.: Tfan: a low complexity 3d mesh compression algorithm. Comput. Animat. Virtual Worlds 20(23), 343–354 (2009)CrossRefGoogle Scholar
  13. 13.
    Mekuria, R., Sanna, M., Asioli, S., Izquierdo, E., Bulterman, D.C.A., Cesar, P.: A 3d tele-immersion system based on live captured mesh geometry. In: Proceedings of MMSys 2013, pp. 24–35. ACM, New York (2013)Google Scholar
  14. 14.
    Oh, B.T.: An adaptive quantization algorithm without side information for depth map coding. Image Commun. 29(9), 962–970 (2014)Google Scholar
  15. 15.
    Pece, F., Kautz, J., Weyrich, T.: Adapting standard video codecs for depth streaming. In: Proceedings of EGVE-JVRC 2011, pp. 59–66, Aire-la-Ville, Switzerland. Eurographics Association (2011)Google Scholar
  16. 16.
    Raghuraman, S., Venkatraman, K., Wang, Z., Prabhakaran, B., Guo, X.: A 3d tele-immersion streaming approach using skeleton-based prediction. In: Proceedings of MM 2013, pp. 721–724. ACM, New York (2013)Google Scholar
  17. 17.
    Schwarz, H., Bartnik, C., Bosse, S., Brust, H., Hinz, T., Lakshman, H., Merkle, P., Muller, K., Rhee, H., Tech, G., Winken, M., Marpe, D., Wiegand, T.: Extension of high efficiency video coding (hevc) for multiview video and depth data. In: Proceedings of ICIP 2012, pp. 205–208, September 2012Google Scholar
  18. 18.
    Schwarz, S., Olsson, R., Sjostrom, M., Tourancheau, S.: Adaptive depth filtering for hevc 3d video coding. In: Proceedings of PCS 2012, pp. 49–52, May 2012Google Scholar
  19. 19.
    Van Wallendael, G., Van Leuven, S., De Cock, J., Bruls, F., Van de Walle, R.: Multiview and depth map compression based on hevc. In: Proceedings of ICCE 2012, pp. 168–169, January 2012Google Scholar
  20. 20.
    Videolan. x264 (2014, sep. version 142) : a free h264/avc encoder. http://www.videolan.org/developers/x264.html
  21. 21.
    Yuan, H., Kwong, S., Liu, J., Sun, J.: A novel distortion model and lagrangian multiplier for depth maps coding. IEEE Trans. Circuits Syst. Video Technol. 24(3), 443–451 (2014)CrossRefGoogle Scholar
  22. 22.
    Zamarin, M., Salmistraro, M., Forchhammer, S., Ortega, A.: Edge-preserving intra depth coding based on context-coding and h.264/avc. In: Proceedings of ICME 2013, pp. 1–6, July 2013Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Yunpeng Liu
    • 1
    • 2
  • Stephan Beck
    • 1
  • Renfang Wang
    • 2
  • Jin Li
    • 2
  • Huixia Xu
    • 2
  • Shijie Yao
    • 2
  • Xiaopeng Tong
    • 2
  • Bernd Froehlich
    • 1
  1. 1.Virtual Reality Systems GroupBauhaus-Universität WeimarWeimarGermany
  2. 2.College of Computer Science and Information TechnologyZhejiang Wanli UniversityNingboChina

Personalised recommendations