MultiStage: Acting across Distance

  • Fei Su
  • Giacomo Tartari
  • John Markus Bjørndalen
  • Phuong Hoai Ha
  • Otto J. Anshus
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7990)


We report on a prototype system helping actors on a stage to interact and perform with actors on other stages as if they were on the same stage. At each stage four 3D cameras tiled back to back for an almost 360 degree view, continuously record actors. The system processes the recorded data on-the-fly to discover actions by actors that it should react to, and it streams data about actors and their actions to remote stages where each actor is represented by a remote presence, a visualization of the actor. When the remote presences lag behind too much because of network and processing delays, the system applies various techniques to hide this, including switching rapidly to a pre-recorded video or animations of individual actors. The system amplifies actors’ actions by adding text and animations to the remote presences to better carry the meaning of actions across distance. The system currently scales across the Internet with good performance to three stages, and comprises in total 15 computers, 12 cameras, and several projectors.


Temporal Synchronization Remote Interaction Computer Mediated Collaboration 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Sawchuk, A., Chew, E., Zimmermann, R., Papadopoulos, C., Kyriakakis, C.: From remote media immersion to distributed immersive performance. In: Proceedings of the 2003 ACM SIGMM Workshop on Experiential Telepresence, pp. 110–120. ACM (2003)Google Scholar
  2. 2.
    Chew, E., Kyriakakis, C., Papadopoulos, C., Sawchuk, A., Zimmermann, R.: Distributed immersive performance: Enabling technologies for and analyses of remote performance and collaboration. In: NIME 2006 (2006)Google Scholar
  3. 3.
    Zimmermann, R., Chew, E., Ay, S., Pawar, M.: Distributed musical performances: Architecture and stream management. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP) 4(2), 14 (2008)Google Scholar
  4. 4.
    Sato, Y., Hashimoto, K., Shibata, Y.: A new remote camera work system for teleconference using a combination of omni-directional and network controlled cameras. In: 22nd International Conference on Advanced Information Networking and Applications, AINA 2008, pp. 502–508. IEEE (2008)Google Scholar
  5. 5.
    Tang, A., Pahud, M., Inkpen, K., Benko, H., Tang, J., Buxton, B.: Three’s company: understanding communication channels in three-way distributed collaboration. In: Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work, pp. 271–280. ACM (2010)Google Scholar
  6. 6.
    Sakamoto, D., Kanda, T., Ono, T., Ishiguro, H., Hagita, N.: Android as a telecommunication medium with a human-like presence. In: 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 193–200. ACM (2007)Google Scholar
  7. 7.
    Dou, M., Shi, Y., Frahm, J., Fuchs, H., Mauchly, B., Marathe, M.: Room-sized informal telepresence system. In: 2012 IEEE Virtual Reality Workshops (VR), pp. 15–18. IEEE (2012)Google Scholar
  8. 8.
    Petit, B., Lesage, J., Menier, C., Allard, J., Franco, J., Raffin, B., Boyer, E., Faure, F.: Multicamera real-time 3d modeling for telepresence and remote collaboration. International Journal of Digital Multimedia Broadcasting 2010 (2009)Google Scholar
  9. 9.
    Essid, S., Lin, X., Gowing, M., Kordelas, G., Aksay, A., Kelly, P., Fillon, T., Zhang, Q., Dielmann, A., Kitanovski, V., et al.: A multi-modal dance corpus for research into interaction between humans in virtual environments. Journal on Multimodal User Interfaces, 1–14 (2012)Google Scholar
  10. 10.
    Elgendi, M., Picon, F., Magnenat-Thalmann, N.: Real-time speed detection of hand gesture using kinect. In: Springer (ed.) Proceedings of the Autonomous Social Robots and Virtual Humans Workshop, 25th Annual Conference on Computer Animation and Social Agents (2012)Google Scholar
  11. 11.
    Stodle, D., Troyanskaya, O., Li, K., Anshus, O.: Tech-note: Device-free interaction spaces. In: IEEE Symposium on 3D User Interfaces, 3DUI 2009, pp. 39–42. IEEE (2009)Google Scholar
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Fei Su
    • 1
  • Giacomo Tartari
    • 1
  • John Markus Bjørndalen
    • 1
  • Phuong Hoai Ha
    • 1
  • Otto J. Anshus
    • 1
  1. 1.Department of Computer ScienceUniversity of TromsøNorway

Personalised recommendations