Simultaneous Output-Timing Control in Networked Games and Virtual Environments
In this chapter, we make a survey of techniques for simultaneous output-timing control, which adjusts the output timing of media streams among multiple terminals in networked games and virtual environments. When media units (MUs, each of which is the information unit, such as a video frame and a voice packet for media synchronization) are transmitted over non-guaranteed Quality of Service (QoS) networks like the Internet, the receiving times of each MU at the terminals may be different from each other owing to network delays and delay jitters. Therefore, for example, the fairness among players may be damaged in networked games, and collaborative work may not be done efficiently among users in virtual environments. It is important for multiple players/users to play/do networked games/collaborative work while watching the same displayed images simultaneously. To solve the problems, the simultaneous output-timing control, such as media synchronization control and causality control is needed. In this chapter, as the control, we mainly handle the group (or inter-destination) synchronization control, which is a type of media synchronization control, the adaptive Δ-causality control, and the dynamic local lag control. We also discuss the similarities and differences among the three types of control. Generally, the group synchronization control or adaptive Δ-causality control can be employed to keep the fairness and/or the consistency in good conditions among multiple terminals in networked games and virtual environments, and the dynamic local lag control is used for sound synchronization in networked virtual ensembles. However, the interactivity may seriously be deteriorated under such types of control. Therefore, we introduce prediction control to improve the interactivity. As a result of Quality of Experience (QoE) assessment, we demonstrate that the prediction control improves the interactivity and there is the optimum prediction time according to the network delay. Finally, we discuss the future directions of simultaneous output-timing control in networked games and virtual environments.
KeywordsQoS control Simultaneous output-timing control Networked game Virtual environment
The authors thank Prof. Hitoshi Watanabe of Tokyo University of Science for his valuable discussions.
QoS is the quality of service which is provided from a layer to its upper layer in network layer model and it is defined as how much the service is faithful to the ideal situation.
QoE is also called the user-level QoS. QoE is the quality which is perceived subjectively and/or experienced objectively by end-users.
MOS is a scale of the rating scale method and is obtained by averaging scores of all the subjects for each stimulus. MOS is one of subjective QoE measures. In the rating scale method for example, the five-grade quality scale or impairment scale is employed.
A network which does not guarantee the Quality of Service (QoS).
Therefore when packets are transmitted over the network, large network delays, delay jitter, and packet loss may occur.
- 1.Smed, J., Kaukoranta, T., Hakonen, H.: A review on networking and multiplayer computer games. Technical Report 454. Turku Center for Computer Science, Apr 2002Google Scholar
- 2.Chiueh, T.: Distributed systems support for networked games. In: IEEE 6th Workshop on Hot Topics in Operating Systems, pp. 99–104, May 1997Google Scholar
- 3.ITU-T Rec. I. 350: General aspects of quality of service and network performance in digital networks, including ISDNs, Mar 1993Google Scholar
- 5.Ishibashi, Y., Tsuji, A., Tasaka, S.: A group synchronization mechanism for stored media in multicast communications. In: Proceedings of the IEEE INFOCOM, pp. 693–701, April 1997Google Scholar
- 6.Yavatkar, R.: MCP: a protocol for coordination and temporal synchronization in multimedia collaborative applications. In: Proceedings of the ICDS, pp. 606–613, June 1992Google Scholar
- 7.Ishibashi, Y., Tasaka, S.: A group synchronization mechanism for live media in multicast communications. In: Conference Rec. IEEE GLOBECOM, pp. 746–752, Nov 1997Google Scholar
- 8.Ishibashi, Y., Tasaka, S.: A distributed control scheme for group synchronization in multicast communications. In: Proceedings of the ISCOM, pp. 317–323, Nov 1999Google Scholar
- 9.Ishibashi, Y., Tasaka, S., Tachibana, Y.: Adaptive causality and media synchronization control for networked multimedia applications. In: Conference Rec. IEEE ICC, pp. 952–958, June 2001Google Scholar
- 10.Sithu, M., Ishibashi, Y., Fukushima, N.: Effects of dynamic local lag control on sound synchronization and interactivity in joint musical performance. ITE Trans Media Technol. Appl. Spec. Sect. Multimedia Transm. Syst. Serv. 2(4), 299–309 (2014)Google Scholar
- 11.Ishibashi, Y., Tasaka, S.: A synchronization mechanism for continuous media in multimedia communications. In: Proceedings of the IEEE INFOCOM, pp. 1010–1019, April 1995Google Scholar
- 12.Ishibashi, Y., Tasaka, S.: A comparative survey of synchronization algorithms for continuous media in network environments. In: Proceeding of the IEEE LCN, pp. 337–348, Nov 2000Google Scholar
- 14.Ishibashi, Y., Tasaka, S., Hasegawa, T.: The virtual-time rendering algorithm for haptic media synchronization in networked virtual environments. In: Proceedings of the CQR, pp. 213–217, May 2002Google Scholar
- 15.Ishibashim, Y., Tomaru, K., Tasaka, S., Inazumi, K.: Group synchronization in networked virtual environments. In: Proceedings of the IEEE ICC, pp. 885–890, May 2003Google Scholar
- 17.Sithu, M., Ishibashi, Y., Fukushima, N.: Dynamic local lag control for sound synchronization in joint musical performance. In: Proceedings of the NetGames, Dec 2013Google Scholar
- 19.Sithu, M., Ishibashi, Y., Fukushima, N.: Enhancement of dynamic local lag control for networked musical performance. In: Proceeding of the IEEE GCCE, Oct 2014Google Scholar
- 20.Ishibashi, Y., Tasaka, S.: Causality and media synchronization control for networked multimedia games: centralized versus distributed. In: Proceedings of the NetGames, pp. 42–51, May 2003Google Scholar
- 21.Ishibashi, Y., Hasegawa, T., Tsaka, S.: Group synchronization control for haptic media in networked virtual environments. In: Proceedings of the Haptics, pp. 106–113, Mar 2004Google Scholar
- 22.Kaneoka, H., Ishibashi, Y.: Effects of group synchronization control over haptic media in collaborative work. In: Proceedings of the ICAT, pp. 138–145, Nov/Dec 2004Google Scholar
- 23.Ishibashi, Y., Kaneoka, H.: Group synchronization for haptic media in a networked real-time game. IEICE Trans. Commun. E89-B(2), 313–319 (2006)Google Scholar
- 24.Hashimoto, T., Ishibashi, Y.: Group synchronization control over haptic media in a networked real-time game with collaborative work. In: Proceeding of the NetGames, Oct 2006Google Scholar
- 25.Ida, Y., Ishibashi, Y., Fukushima, N., Sugawara, S.: QoE assessment of interactivity and fairness in first person shooting with group synchronization control. In: Proceeding of the NetGames, Nov 2010Google Scholar
- 26.ITU-T Rec. P. 10/G. 100 Amendment 1: New appendix I—definition of quality of experience (QoE), Jan 2007Google Scholar
- 27.Hosoya, K., Ishibashi, Y., Sugawara, S., Psannis, K.E.: Effects of group synchronization control in networked virtual environments with avatars. In: Proceedings of IEEE/ACM DS-RT, pp. 119–127, Oct 2008Google Scholar
- 28.Ishibashi, Y., Hashimoto, Y., Ikedo, T., Sugawara, S.: Adaptive—causality control with adaptive dead-reckoning in networked games. In: Proceedings of NetGames, pp. 75–80, Sept 2007Google Scholar
- 29.Kusunose, Y., Ishibashi, Y., Fukushima, N., Sugawara, S.: QoE comparison of competition avoidance methods for management of shared object in networked real-time game with haptic media. In: Proceedings of ICAT’11, Nov 2011Google Scholar
- 30.Ikedo, T., Ishibashi, Y.: An adaptive scheme for consistency among players in networked racing games. In: Proceedings of FMUIT’06, pp. 154–157, May 2006Google Scholar
- 31.Tsumaki, Y., Yokohama, M.: Predictive motion display for acceleration based teleoperation. In: Proceedings of IEEE ICRA, pp. 2927–2932, May 2006Google Scholar
- 33.Asano, T., Ishibashi, Y., Kameyama, S.: Interactive haptic transmission for remote control system. In: Proceedings of IEEE ICME, pp. 2113–2116, July 2006Google Scholar
- 34.Huang, P., Ishibashi, Y., Fukushima, N., Sugawara, S.: Collaborative haptic play with building blocks. In: Proceedings of ACM SIGCHI ACE, Oct 2009Google Scholar
- 36.ITU-R BT. 500-12: Methodology for the subjective assessment of the quality of television pictures. International Telecommunication Union, Sept 2009Google Scholar
- 37.Kusunose, Y., Ishibashi, Y., Fukushima, N., Sugawara, S.: Adaptive delta-causality control scheme with prediction in networked real-time game using haptic media. In: Proceedings of APCC, pp. 800–805, Oct 2012Google Scholar
- 38.Hara, Y., Ishibashi, Y., Fukushima, N., Sugawara, S.: Adaptive delta-causality control scheme with dynamic control of prediction time in networked haptic game. In: Proceedings of NetGames, Nov 2012Google Scholar
- 39.Sithu, M., Ishibashi, Y., Fukushima, N.: Effect of dynamic local lag control with dynamic control of prediction time in joint haptic drum performance. In: Proceedings of ICCA, pp. 343–350, Feb 2014Google Scholar
- 40.Ishibashi, Y., Nagasaka, M.: An adaptive QoS control scheme of avatars in distributed virtual environments. J. Inst. Image Inf. Tele. Eng. 60(11), 1835–1839, Nov 2006 (in Japanese)Google Scholar
- 41.Sannomiya, H., Osada, J., Ishibashi, Y., Fukushima, N., Sugawara, S.: Inter-stream synchronization control with group synchronization algorithm. In: Proceeedings of IEEE GCCE, pp. 520–524, Oct 2013Google Scholar
- 42.Ishibashi, Y., Kanbara, T., Tasaka, S.: Inter-stream synchronization between haptic media and voice in collaborative virtual environments. In: Proceedings of ACM Multimedia, pp. 604–611, Oct 2004Google Scholar