Skeleton-Based Data Compression for Multi-camera Tele-Immersion System

  • Jyh-Ming Lien
  • Gregorij Kurillo
  • Ruzena Bajcsy
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4841)


Image-based full body 3D reconstruction for tele-immersive applications generates large amount of data points, which have to be sent through the network in real-time. In this paper we introduce a skeleton-based compression method using motion estimation where kinematic parameters of the human body are extracted from the point cloud data in each frame. First we address the issues regarding the data capturing and transfer to a remote site for the tele-immersive collaboration. We compare the results of the existing compression methods and the proposed skeleton-based compression technique. We examine robustness and efficiency of the algorithm through experimental results with our multi-camera tele-immersion system. The proposed skeleton-based method provides high and flexible compression ratios (from 50:1 to 5000:1) with reasonable reconstruction quality (peak signal-to-noise ratio from 28 to 31 dB).


Point Cloud Compression Ratio Motion Estimation Iterative Close Point Compression Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lanier, J.: Virtually there. Scientific American 4, 52–61 (2001)Google Scholar
  2. 2.
    Yang, Z., Cui, Y., Anwar, Z., Bocchino, R., Kiyanclar, N., Nahrstedt, K., Campbell, R., Yurcik, W.: Real-time 3d video compression for tele-immersive environments. In: MMCN 2006. Proceedings of SPIE/ACM Multimedia Computing and Networking, San Jose, CA, ACM Press, New York (2006)Google Scholar
  3. 3.
    Kalra, P., Magnenat-Thalman, N., Moccozet, L., Sannier, G., Aubel, A., Thalman, D.: Real-time animation of realistic virtual humans. IEEE Computer Graphics and Applications 18, 42–56 (1998)CrossRefGoogle Scholar
  4. 4.
    Mulligan, J., Daniilidis, K.: Real time trinocular stereo for tele-immersion. In: Proceedings of 2001 International Conference on Image Processing, Thessaloniki, Greece, pp. 959–962 (2001)Google Scholar
  5. 5.
    Baker, H., Tanguay, D., Sobel, I., Gelb, D., Gross, M., Culbertson, W., Malzenbender, T.: The coliseum immersive teleconferencing system. In: Proceedings of International Workshop on Immersive Telepresence, Juan-les-Pins, France (2002)Google Scholar
  6. 6.
    Wrmlin, S., Lamboray, E., Gross, M.: 3d video fragments: dynamic point samples for real-time free-viewpoint video. Computers and Graphics 28, 3–14 (2004)CrossRefGoogle Scholar
  7. 7.
    Jung, S., Bajcsy, R.: A framework for constructing real-time immersive environments for training physical activities. Journal of Multimedia 1, 9–17 (2006)CrossRefGoogle Scholar
  8. 8.
    Patel, K., Bailenson, J.N., Hack-Jung, S., Diankov, R., Bajcsy, R.: The effects of fully immersive virtual reality on the learning of physical tasks. In: Proceedings of the 9th Annual International Workshop on Presence, Ohio, USA, pp. 87–94 (2006)Google Scholar
  9. 9.
    Piccardi, M.: Background subtraction techniques: a review. In: Proceedings of IEEE International Conference on Systems, Man and Cybernetics, Hague, Netherlands, pp. 3099–3104. IEEE Computer Society Press, Los Alamitos (2004)Google Scholar
  10. 10.
    Zhao, W., Nandhakumar, N.: Effects of camera alignment errors on stereoscopic depth estimates. Pattern Recognition 29, 2115–2126 (1996)CrossRefGoogle Scholar
  11. 11.
    Zhang, D., Nomura, Y., Fujii, S.: Error analysis and optimization of camera calibration. In: IROS 1991. Proceedings of IEEE/RSJ International Workshop on Intelligent Robots and Systems, Osaka, Japan, pp. 292–296 (1991)Google Scholar
  12. 12.
    Tsai, R.: A versatile camera calibration technique for high-accuracy 3d machine vision metrology using off-the-shelf tv cameras and lenses. IEEE Journal of Robotics and Automation RA3, 323–344 (1987)CrossRefGoogle Scholar
  13. 13.
    Zlib: Compression library (2005)Google Scholar
  14. 14.
    Besl, P., McKay, N.: A method for registration of 3-d shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence 14, 239–256 (1992)CrossRefGoogle Scholar
  15. 15.
    Herda, L., Fua, P., Plankers, R., Boulic, R., Thalman, D.: Skeleton-based motion capture for robust reconstruction of human motion. In: Proceedings of Computer Animation Conference, pp. 77–93 (2000)Google Scholar
  16. 16.
    Theobalt, C., Magnor, M., Schuler, P., Seidel, H.: Multi-layer skeleton fitting for online human motion capture. In: VMV 2002. Proceedings of 7th International Workshop on Vision, Modeling and Visualization, Erlangen, Germany, pp. 471–478 (2002)Google Scholar
  17. 17.
    Aggarwal, J.K., Cai, Q.: Human motion analysis: a review. Comput. Vis. Image Underst. 73, 428–440 (1999)CrossRefGoogle Scholar
  18. 18.
    Gavrila, D.M.: The visual analysis of human movement: a survey. Comput. Vis. Image Underst. 73, 82–98 (1999)zbMATHCrossRefGoogle Scholar
  19. 19.
    Lien, J.M., Bajcsy, R.: Skeleton-based compression of 3-d tele-immersion data. In: ICDSC 2007. Proceedings of the ACM/IEEE International Conference on Distributed Smart Cameras, IEEE Computer Society Press, Los Alamitos (2007)Google Scholar
  20. 20.
    Besl, P.J., McKay, N.D.: A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14, 239–256 (1992)CrossRefGoogle Scholar
  21. 21.
    Wiegand, T., Sulivan, G., Bjntegaard, G., Luthra, A.: Overview of the h. 264/avc video coding standard. IEEE Transactions on Circuits and Systems for Video Technology 13, 560–576 (2003)CrossRefGoogle Scholar
  22. 22.
    Adobe: Qicktime 7.0 h.264 implementation (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Jyh-Ming Lien
    • 1
  • Gregorij Kurillo
    • 2
  • Ruzena Bajcsy
    • 2
  1. 1.George Mason University, Fairfax, VA 
  2. 2.University of California, Berkeley, CA 

Personalised recommendations