Multimedia Tools and Applications

, Volume 51, Issue 2, pp 593–623 | Cite as

Next generation session management for 3D teleimmersive interactive environments

  • Klara Nahrstedt
  • Zhenyu Yang
  • Wanmin Wu
  • Ahsan Arefin
  • Raoul Rivas
Article

Abstract

In the recent past we have seen a boom of distributed interactive multimedia environments which use multiple correlated media sensors, multi-view displays, and advanced haptic-audio-visual user interfaces for teleimmersive gaming, business meetings and other collaborative activities. However, when we investigate the emerging teleimmersive environments closer, we realize that their overall session management, including end-to-end session setup, establishment and run-time management are not appropriate for the new demands that these environments present. These environments are cyber-physical rooms that demand (a) large scale of multi-sensory devices across geographically-distributed locations and (b) interaction with each other in synchronous and real-time manner. To deal with the new teleimmersive demands, we present a new session management design with (a) session initiation protocol(s) that understand media correlations, (b) view-based multi-stream topology establishment among multiple parties, (c) efficient, light-weight and distributed session monitoring with querying and debugging capabilities, (d) flexible view-based session adaptation with efficient topology adjustments, and (e) light-weighted and consistent session tear-down protocols. The presented design of the next generation session management protocols, services, algorithms and data structures is based on our extensive experiences with building 3D teleimmersive interactive systems, experimenting with high impact teleimmersive applications and deploying such environments at various venues.

Keywords

Session management 3D teleimmersive environments Interactive multimedia systems 

References

  1. 1.
    Agrawal M, Davis L (2002) Trinocular stereo using shortest path and the ordering constraint. Int J Comput Vision 47:43–50MATHCrossRefGoogle Scholar
  2. 2.
    Arefin A, Sarwar Y, Gupta I, Nahrstedt K (2009) “Q-Tree: a multi-attribute based range query solution for teleimmersive framework,” IEEE International Conference on Distributed Computing SystemsGoogle Scholar
  3. 3.
    Bajcsy P, McHenry K, Na H-J, Malik R, Spencer A, Lee S-K, Kooper R, Frogley M (2009) “Immersive environments for rehabilitation activities”, In the Proceedings of ACM International Conference on Multimedia, Beijing, China, October 19–24Google Scholar
  4. 4.
    Baker H, Bhatti N, Tanguay D, Sobel I, Gelb D, Goss M, Culbertson W, Malzbender T (2005) “Understanding performance in coliseum, an immersive videoconferencing system,” ACM Transactions on Multimedia Computing, Communications, and ApplicationsGoogle Scholar
  5. 5.
  6. 6.
    Corradi A, Leonardi L, Zambonelli F (1999) “Diffusive load-balancing policies for dynamic applications,” In the Proceedings of IEEE Concurrency, vol. 7, no. 1Google Scholar
  7. 7.
    Dabek F, Cox R, Kaashoek F, Morris R (2004) “Vivaldi: a decentralized network coordinate system”, In the Proceedings of the ACM SIGCOMM ‘04 ConferenceGoogle Scholar
  8. 8.
    Daniilidis F, Mulligan J, Mckendall R, Majumder A, Kamberova G, Schid D, Bajcsy R, Fuchs H (1999) “Towards the Holodeck: an initial testbed for real-time 3D teleimmersion”, ACM SIGGRAPHGoogle Scholar
  9. 9.
  10. 10.
    Goemans M (2003) “Minimum bounded degree spanning trees” In the Proceedings of IEEE Symposium on Foundations of Computer Science, pp. 273–282Google Scholar
  11. 11.
    Gross M, Würmlin S, Naef M, Lamboray E, Spagno C, Kunz A, Koller-Meier E, Svoboda T, Gool LV, Lang S, Strehlke K, Moere AV, Staadt O (2003)“Blue-c: a spatially immersive display and 3d video portal for telepresence,” ACM Trans GraphGoogle Scholar
  12. 12.
  13. 13.
    International Telecommunication Union (2003) “Packet based multimedia communication systems,” Recommendation H.323Google Scholar
  14. 14.
    Jain M, Dovrolis C (2002) “End-to-end available bandwidth: Measurement methodology, dynamics, and relation with tcp throughput,” In the Proceedings of ACM SIGCOMM, pages 295–308Google Scholar
  15. 15.
    Kurillo G, Vasudevan R, Lobaton E, Bajcsy R (2008) “A framework for collaborative real-time 3D teleimmersion in a geographically distributed environment,” In the Proceedings of the 10th IEEE International Symposium on MultimediaGoogle Scholar
  16. 16.
    Magor Telecollaboration, http://www.magorcorp.com
  17. 17.
    Microsoft NetMeeting Protocol Specification, Microsoft Corp., June 2010Google Scholar
  18. 18.
    Microsoft NetMeeting, http://www.microsoft.com
  19. 19.
    Mitel Telecollaboration, http://www.mitel.com
  20. 20.
    Ott DE, Mayer-Patel K (2004) “Coordinated multi-streaming for 3d teleimmersion”, In the Proceedings of the 12th annual ACM International Conference on MultimediaGoogle Scholar
  21. 21.
    QQ Messenger. http://www.qq.com
  22. 22.
    Rosenberg J, Schulzrinne H, Camarillo G, Johnston A, Peterson J, Sparks R, Handley M Schooler E (2002) “SIP: Session Initiation Protocol (RFC 3261)”Google Scholar
  23. 23.
    Sat B, Huang Z, Wah BW (2007) “The design of a multi-party VoIP conferencing system over the internet,” In the Proceedings of IEEE International Symposium on MultimediaGoogle Scholar
  24. 24.
    Schulzrinne H, Casner S, Frederick R, Jacobson V (2003) “RTP: A transport protocol for real-time applications (RFC 3550)”Google Scholar
  25. 25.
    Schulzrinne H, Rao A, Lanphier R (1998) “Real time streaming protocol (RFC 2326)”Google Scholar
  26. 26.
    Sheppard R, Kamali M, Rivas R, Tamai M, Yang Z, Wu W, Nahrstedt K (2008) “Distributed virtual collaboration through Teleimmersive Dance (TED): A symbiotic creativity and design environment for art and computer science,” In the Proceedings of ACM International Conference on Multimedia, Vancouver, BC, CanadaGoogle Scholar
  27. 27.
    Shi S, Nahrstedt K, Campbell RH (2008) “View-dependent real-time 3D video compression for mobile devices” In the Proceedings of ACM International Conference on MultimediaGoogle Scholar
  28. 28.
  29. 29.
    Svoboda T, Martinec D, Pajdla T (2005) “A convenient multi-camera self-calibration for virtual environments,” In PRESENCE: Teleoperators and Virtual Environments 14:407–422Google Scholar
  30. 30.
    Tandberg Telepresence, http://www.tandberg.com
  31. 31.
    Teliris Telepresence, http://www.teliris.com
  32. 32.
    Vidyo Telepresence, http://www.vidyo.com
  33. 33.
    Wang Z, Crowcroft J (1996) Qos routing for supporting resource reservation. IEEE J Sel Areas Commun 14:1228–1234CrossRefGoogle Scholar
  34. 34.
    Windows Live Messenger, http://messenger.live.com.
  35. 35.
    Wu W, Yang Z, Gupta I, Nahrstedt K (2008) “Towards multi-site collaboration in 3D teleimmersive environments” IEEE International Conference on Distributed Computing SystemsGoogle Scholar
  36. 36.
    Wu W, Yang Z, Nahrstedt K (2008) “Implementing a distributed 3D Teleimmersive System,” In the Proceedings of the 10th IEEE International Symposium on Multimedia, Berkeley, CA, USA, December 15–17Google Scholar
  37. 37.
    Wu W, Yang Z, Nahrstedt K (2008) “A study of visual context representation and control for remote sport learning tasks”, AACE World Conference on Educational Multimedia, Hypermedia and Telecommunications (ED-MEDIA), Vienna, AustriaGoogle Scholar
  38. 38.
    Yahoo Instant Messenger, http://messenger.yahoo.com
  39. 39.
    Yang Z, Cui Y, Yu B, Liang J, Nahrstedt K, Jung S-H, Bajcsy R (2005) “TEEVE: the next generation architecture for teleimmersive environments,” In the Proceedings of the 7th IEEE International Symposium on Multimedia, Irvine, CA, USAGoogle Scholar
  40. 40.
    Yang Z, Wu W, Nahrstedt K, Kurillo G, Bajcsy R (2007) “Viewcast: view dissemination and management for multi-party 3d teleimmersive environments,” In the Proceedings of the 15th ACM International Conference on MultimediaGoogle Scholar
  41. 41.
    Yang Z, Wu W, Nahrstedt K, Kurillo G, Bajcsy R (2010) “Enabling multi-party 3D teleimmersive environments with ViewCast,” ACM Transactions on Multimedia Computing, Communications, and Applications, vol. 6, no. 2Google Scholar
  42. 42.
    Z. Yang, B. Yu, K. Nahrstedt and R. Bajcsy, “A multi-stream adaptation framework for bandwidth management in 3D teleimmersion,” In the Proceedings of the 16th International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV), 2006Google Scholar
  43. 43.
    Yang Z, Yu B, Wu W, Danikov R, Nahrstedt K, Bajcsy R (2006) Study of collaborative dancing in tele-immersive environment, IEEE International Symposium on Multimedia (ISM) 2006, San Diego, USAGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Klara Nahrstedt
    • 1
  • Zhenyu Yang
    • 2
  • Wanmin Wu
    • 1
  • Ahsan Arefin
    • 1
  • Raoul Rivas
    • 1
  1. 1.Department of Computer ScienceUniversity of Illinois at Urbana-ChampaignUrbanUSA
  2. 2.School of Computing and Information SciencesFlorida International UniversityMiamiUSA

Personalised recommendations