Multi-user Live Video in a Shared Virtual World for Enhanced Group Collaboration

  • Seon-Min Rhee
  • Jiyoung Park
  • Myoung-Hee Kim
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3942)


We present a novel method for merging a live video stream of multiple users into a shared virtual world in a large VR display environment. Collaborators at a remote site can be displayed in a virtual world as video textures to support visual communication more efficiently. To generate the video texture of a user in a dynamic background environment such as a CAVETM-like system, we use a camera pair which consists of two gray-scale cameras with band-pass filters, and two color cameras. The gray-scale cameras capture infrared images in which the dynamic background on the screen is eliminated, having only the infrared light reflected by the person. This makes it easy to extract the user’s silhouette robustly. Using projection histogram we found rough position of the all users’ head in the image. We can then find exact corresponding points using cross-correlation between two user silhouette images to determine the 3D position of each user. The extracted user silhouette is warped by rectification and translated to fit into a color image. We can now generate a video texture of the user by masking the background of the user in the color image. Each video texture is then transmitted to a remote site and displayed on a billboard in virtual space.


Virtual World Multiple User Search Area Remote Site Stereo Pair 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ogi, T., Yamada, T., Tamagawa, K., Kano, M., Hirose, M.: Immersive telecommunication using stereo video avatar. In: Proc. Virtual Reality 2001 Conference (VR 2001). IEEE Computer Society Press, Los Alamitos (2001)Google Scholar
  2. 2.
    Ogi, T., Yamada, T., Hirose, M., Fujita, M., Kuzuu, K.: High presence remote presentation in the shared immersive virtual world. In: Proc. IEEE Virtual Reality 2003 Conference (VR 2003). IEEE Computer Society Press, Los Alamitos (2003)Google Scholar
  3. 3.
    Ogi, T., Yamada, T., Hattori, Y., Kurita, Y., Hirose, M.: Usage of video avatar technology for immersive communication. In: Proc. First Intl Workshop on Language Understanding and Agents for Real World Interaction 2003 (2003)Google Scholar
  4. 4.
    Gross, M., Wuermlin, S., Naef, M., Lamboray, E., Spagno, C., Kunz, A., K-Meier, E., Svoboda, T., Gool, L., Lang, S., Strehlke, K., Moere, A.V., Staadt, O.: blue-c: a spatially immersive display and 3d video portal for telepresence. ACM Trans. Graphics 22(3), 819–827 (2003)CrossRefGoogle Scholar
  5. 5.
    Wuemlin, S., Lamboray, E., Gross, M.: 3d video fragments: Dynamic point samples for real-time free-viewpoint video. Computers and Graphics 28(1), 3–14 (2004)CrossRefGoogle Scholar
  6. 6.
    Vorozcovs, A., Hogue, A., Stuerzlinger, W.: The hedgehog: a novel optical tracking method for spatially immersive displays. In: Proc. IEEE VR 2005. IEEE Computer Society press, Los Alamitos (2005)Google Scholar
  7. 7.
    Matusik, W.: Image-based visual hulls. Master’s thesis, Massachusetts Institute of Technology (2001)Google Scholar
  8. 8.
    Haritaoglu, I., Harwood, D., Davis, L.S.: Hydra: Multiple people detection and tracking using silhouettes. In: Proc.IEEE International Workshop on Visual Surveillance. IEEE Computer Society press, Los Alamitos (1999)Google Scholar
  9. 9.
    Tsai, R.Y.: An efficient and accurate camera calibration technique for 3d machine vision. In: Proc. IEEE Computer Vision and Pattern Recognition. IEEE Computer Society Press, Los Alamitos (1986)Google Scholar
  10. 10.
    Fusiello, A., Trucco, E., Verri, A.: A compact algorithm for rectification of stereo pairs. Machin Vision and Applications 12(1), 16–22 (2000)CrossRefGoogle Scholar
  11. 11.
    Naef, M., Staadt, O., Gross, M.: Multimedia integration into the blue-c api. Computers and Graphics 29(1), 3–15 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Seon-Min Rhee
    • 1
  • Jiyoung Park
    • 1
  • Myoung-Hee Kim
    • 1
    • 2
  1. 1.Dept. of Computer Science & EngineeringEwha Womans UniversitySeoulKorea
  2. 2.Center for Computer Graphics & Virtual Reality(CCGVR)Ewha Womans UniversitySeoulKorea

Personalised recommendations