Lag Compensation for First-Person Shooter Games in Cloud Gaming
- 11k Downloads
Cloud gaming is an emerging technology that combines cloud computing with computer games. Compared to traditional gaming, its core advantages include ease of development/deployment for developers, and lower technology costs for users given the potential to play on thin client devices. In this chapter, we firstly describe the approach, and then focus on the impact of latency, known as lag, on Quality of Experience, for so-called First Person Shooter games. We outline our approach to lag compensation whereby we equalize within reason the up and downlink delays in real-time for all players. We describe the testbed in detail, the open source Gaming Anywhere platform, the use of NTP to synchronise time, the network emulator and the role of the centralized log server. We then present results that firstly validate the mechanism and also use small scale and preliminary subjective tests to assess and prove its performance. We conclude the chapter by outlining ongoing and future work.
KeywordsCloud gaming Quality of experience Network delay Lag compensation
The gaming industry plays an important role in the entertainment and software industries. According to “Video Game Revenue Forecast: 2017-22”, it is expected that the global market of video games will grow up to $174 billion by 2022 .
Traditionally, computer games are downloaded from the Internet and installed on a PC or other end user device allowing players to run the corresponding game. With game sizes running into multiple gigabytes, the installation process may take the order of hours, with perhaps additional time required to install patches of new game versions. Furthermore, when players wish to play newly released games, they may require a higher specified hardware configuration to enable all the visual effects, and so they have to upgrade their computers to meet the particular specification. Both of these factors can result in frustration and may result in gamers give up the game .
For these reasons, game developers and users/gamers are paying more attention to cloud gaming systems . From the developer perspective, the benefits include the potential of reaching out to more gamers, easier testing of ideas to improve cloud gaming systems, avoiding piracy due to the fact that games are not being downloaded to client devices .
From the gamer perspective, the benefits include access to games anytime (on demand), without the need to download games, reduced costs due to the fact that the computer hardware does not need to be upgraded frequently, and ability to play games on different platforms, such as PC, smartphone, tablet and so on.
Although cloud games open up a new direction for the video games industry, it is not without its challenges . According to previous research, not only the bandwidth but also the CPU has a significant influence on cloud gamers’ Quality of Experience (QoE) . Ideally, gamers would like to play games with both high quality videos, and where games are delay sensitive, low latency. Latency, also known as lag in gaming, is especially important in the First Person Shooter (FPS) game genre. However - high quality videos, for example, 720p/1080p at 50 fps, can make cloud gaming systems vulnerable to a high network latency  as much more network capacity is needed than in the case of conventional video games (e.g., 5000 kb/s vs. 50 kb/s) . To meet the needs of cloud gamers, network service providers thus have to take network latency, efficiency, high video quality, and error resiliency into consideration . These factors represent significant challenges to the roll out of large scale cloud gaming services. Without adequate infrastructure that meets the specific needs of cloud gaming, the potential and benefits will not be realised.
In this chapter, we focus on the impact of lag on QoE for so-called FPS games. In multi-player scenarios, different lag values experienced among players can lead to unfair game play and frustration among players . In , the authors report findings that state that different QoS leads to unfairness or imbalanced games when there are no mechanisms for mitigating the QoS differences. Previous research has analysed the potential of achieving fairness in multi-player networked games through automated latency balancing . We further tackle these challenges in the context of cloud gaming. We describe in detail our approach to lag compensation whereby we equalise within reason the up and downlink delays in real-time for all players, aiming to achieve fairness among players and consequently improve QoE.
At present, several cloud service providers have developed cloud gaming platforms, most of which are closed source (e.g., Sony PlayStation Now, NVIDIA GeForce Now). Therefore, game developers cannot test and fine-tune their games on them  and they have to do the tests and fine-tuning of the games on emulators. This fact increases the difficulty of improving cloud gaming systems to better reflect gamers’ needs and expectations.
In response to this, Gaming Anywhere (GA) was designed and developed. It is the first open-source cloud gaming platform that allows researchers to quickly explore their ideas. More importantly, GA has greatly promoted the evolution of cloud gaming within the video game industry .
The remainder of this chapter is structured as follows. Section 2 provides a literature survey on gaming QoE. It firstly deals with the impact of latency and packet loss on game QoE, before focusing specifically on latency. It then examines the impact of delay for different game genres. Section 3 introduces the concept of lag compensation and outlines our two research objectives. Section 4 outlines in detail how lag equalization was implemented on the Gaming Anywhere platform in order to address both research objectives. Section 5 presents results that firstly validate the lag compensation mechanism and then outlines the results of preliminary subjective tests. Section 6 concludes the paper by outlining ongoing and future work.
2 Literature Review
A key research challenge has been to determine the impact of a wide range of influence factors on gaming QoE, including a wide range of human, system, and context factors [10, 11, 12]. Focusing on system influence factors, and as outlined in the previous section, cloud gaming demands a high level of network Quality of Service (QoS) to deliver acceptable user perceived quality (QoE) to players. Key QoS-related factors include packet loss and delay, with their impact on QoE differing for different types (genres) of games.
In this section, we review relevant literature that outlines the impact that both roundtrip delay (latency), also known as Response Delay (RD) or lag, and packet loss have on the end user experience in general, and also for different game genres. As the main focus of this chapter is on lag compensation, we focus more on lag in the literature review.
Given the inherently interactive nature of gaming in general, a key challenge is meeting delay requirements. This involves the delivery of both user control inputs (mouse, keyboard strokes) to the game server and uninterrupted presentation of continuous game content to players, transmitted in the form of a video stream. For this reason, conventional methods for diminishing the effects of poor network conditions and consequent jitter on streaming media, such as buffering data for display, are not readily applicable in the context of cloud gaming. Moreover, lag compensation techniques applicable in “traditional” gaming, such as client-side prediction , are not applicable in the context of cloud gaming, where the client is simply decoding and portraying the stream received from the server. Consequently, numerous studies have addressed the impact of latency due to heterogeneous and variable network conditions on the end user QoE of cloud gaming.
As reported in [5, 14, 15, 16], packet loss and delay have a significant impact on cloud gaming QoE. Basically, network congestion results in network delay jitter and when queue size is exceeded, packet loss occurs. Delay and jitter impact both on uplink time between the player sending input events to the server, and downlink transmission of game scenes that are eventually displayed on the screen. Moreover, as it has been shown in , a high network delay disrupts an interaction between server and players and negatively influence players’ QoE.
It is important to note that not all cloud games are equally sensitive to latency, as is of course also the case for “traditional” networked games . For Real-time Strategy (RTS) games, the process of constructing buildings or moving troops towards a battlefield is unaffected by latency as high as 1000 ms . However, First Person Shooter (FPS) games, where users are shooting at a moving target tend to be more sensitive to latency with delays of over 100 ms seen as unacceptable . Moreover, the effects of latency are based on two action properties: precision and deadline. Precision refers to the accuracy of actions, whereas deadline refers to the timeliness of events. Games with higher precision and tight deadline are more sensitive to latency. For this reason, FPS players always emphasis precision and deadline .
As it has been reported above, latency plays a very important role when it comes to cloud gaming. Despite this fact, there is no work, to the best of our knowledge, dealing specifically with the impact of delay compensation on QoE in the context of cloud gaming. Therefore, we have decided to focus on this issue in this chapter. More specifically, we showcase lag compensation impact on QoE in a case study involving an FPS cloud game.
3 Lag Compensation
The game server is located in Dublin, Ireland, P1 plays in London with an average lag of say 100 ms (equal delay in both directions) while P2 is in Galway, Ireland with an average lag of say 10 ms, again with equal delay in both directions. Since P1 has a longer RD than P2, P1 will have a relatively bad game experience as P2 has an inherent advantage.
As shown, equal delays are added to both the uplink (client to server or c2s) and downlink (server to client or s2c) traffic. The assumption of equal delays in both directions is in reality rarely true, largely due to asymmetry in traffic flows.
3.1 Research Objectives
How feasible is it to implement a real-time lag compensation strategy for cloud gaming?
Will the lag compensation approach result in improved QoE for FPS gamers?
In this section, we describe the implementation details related to the testbed used to address both research objectives. We firstly describe the cloud gaming platform Gaming Anywhere that was used, followed by the game that was chosen for the platform. Other testbed requirements and tools such as time synchronization and network emulator are also briefly described.
Processing delay (PD): it represents the time between when the server receives the control events and sends the encoded frames to the client.
Playout delay (OD): it is the time to decode and render the decoded frames on the screen on the client side.
Network delay (ND): it is the time required for a round trip data exchange between the client and the server.
With the platform chosen, the next step was to choose a suitable game with which to test/validate our approach. As outlined in the Literature Review, an FPS game was most suited as these have the tightest lag constraints, i.e., are more sensitive to lag than other game genres. For this reason, the game Assault Cube was chosen.
4.2 Game Setup
4.3 Assault Cube Configuration
Assault Cube (AC) is an FPS game which is based on the CUBE engine and available for free on Windows, Linux and OS X. It supports single player and multi-player game mode. AC is launched in Server1 first, and this becomes the master/host for a multi-player game scenario. AC is then launched in Server2, by selecting multi-player mode and joining the game created by Server1 as a slave. Figure 6 illustrates the connection between the two servers and also between the two GA clients/players and their respective GA servers. As shown in Figs. 1 and 2, the actual data flow is bidirectional. Once set up, each GA client sends its game actions to its server and receives video feed from its own server.
4.4 Time Synchronization
Network Time Protocol (NTP) is designed to synchronise system time of computers across IP networks to Universal Coordinated Time (UTC), achieving millisecond (ms) level synch or better across well provisioned wired LANs and single ms level synch across well configured WANs. In this experiment, NTP plays a key role in synchronising time across the GA servers and GA clients. This then facilitates accurate delay measurements as outlined in the next section. In our university LAN, the server to client (s2c) delay and client to server (c2s) are minimal with RTT measured via the ping utility of the order of a few ms or less. Due to NTP tolerances when running on Windows platform and resulting clock offsets, the s2c and c2s delays occasionally are determined as less than 0 or greater than the RTT.
4.5 Each Way Delay Measurement
The GA platform utilizes the Live555 Realtime Transport Protocol RTP library to transport audio/video from server to client. RTP and its companion control protocol RTCP are very widely used in VoIP conferencing software such as WebRTC and Facebook, WhatsApp voice clients. For our purposes, the RTCP library source code was modified to enable calculation of each way delay. By default, RTCP traffic, which runs in parallel to the media RTP flows for both audio and video, enable the calculation of Round Trip Delay, i.e., RTT minus residency time on remote host (described above as Network Delay in ). By adding code to also return the local timestamp when the RTCP receiver report packet is sent back, this facilitates the calculation of both upstream and downstream delay for each stream, once NTP is correctly implemented.
4.6 Network Emulator
In a real Cloud Gaming environment, the players are often at different geographic locations with different network latency, resulting in an unfair game for the player with longer RD. In order to emulate this scenario and thus test the implementation of lag equalization, Network Emulator for Windows Toolkit (NEWT) (available from https://blogs.technet.microsoft.com/juanand/2010/03/05/standalone-network-emulator-tool/) was used on the server side to emulate different network environments for both uplink (c2s) and downlink (s2c) traffic for each player.
4.7 Centralized Log Server
UDP sockets are used instead of TCP sockets to minimize delay and maximize responsiveness of system – it does not need any connection setup such as three-way-handshake, and ignores lost packets which would otherwise incur more delay when sending data. UDP sockets are thus an appealing choice for non-critical delay-sensitive applications.
Once determined, the centralized log server sends recommended compensation delay data back to each GA server based on IP address. The respective GA servers then introduce these additional delays on upstream control traffic (player actions) and downstream data (audio/video).
As shown above, we introduce emulated delays of 20 (10/10) and 40 (20/20) ms to Client1 and Client2 respectively. This results in lag compensation of 10 ms on both up and down link traffic for Client2 – shown in grey above so that c2s and s2c are equal for both clients. Note that the delays shown here are symmetric, which is rarely the case in reality. Further details on setting up the testbed and background work can be found in .
In this section, we outline results that address both research objectives from Sect. 3.1. We firstly present a range of results for the 2-player scenario in order to validate the performance of the lag compensation mechanism. We present baseline delays with no emulated delays to get a sense for delays as measured across the university LAN network and the possible need for lag compensation to cope with inherent delay asymmetries. We then outline a range of tests whereby we introduce both up and downlink delays for different players using the network emulator and see how the lag compensation mechanism performs. Moving to objective 2, we then present results of preliminary subjective tests that exposed players to a range of delays with and without lag compensation and captured the resulting QoE scores. The section concludes with a discussion of ongoing and future work.
5.1 Baseline Results
The output shows how the lag compensating mechanism is achieving the goal of equalising the delays for both players. For example, Player 1 measured s2c delays are 7/5 ms for v/a respectively whereas Player 2 has 6/1 ms for v/a respectively. Therefore, in order to equalise this, compensating delays for Player 2 s2c of 1/4 ms are added for v/a so that totals for Player 2 are 7 (1 + 6) and 5 (4 +1) i.e. same as Player 1.
Moving to c2s delays, Player 1 has measured c2s delays of 4/4 ms for v/a respectively whereas Player 2 has 6/6 ms for v/a respectively. Therefore, in order to equalise this, Player 1 c2s added delays are 2 for v/a so that total for Player 1 is 6 (4 + 2) i.e. same as Player 2.
5.2 Emulated Delays
additional delay of 11 ms on downstream s2c v/a stream so that Player 1 has a total of 26 ms (15 + 11) for video and 30 (19 + 11) for audio – same as Player 2,
additional delay of 12 ms on upstream so that totals are 25 ms – same as Player 2.
As above, the log server detects the changed network conditions and communicates the appropriate changes to respective streams back to GA servers in order to equalise delays.
delay added to downstream s2c,
total s2c delay after lag equalization,
avg c2s delay,
delay added to downstream c2s,
total c2s delay after lag equalization and
In each case, the mechanism is seen to work correctly by firstly detecting the impact of the emulated delays (including noise) typically within one second and then implementing lag compensation so that total c2s and s2c delays for player 1 and 2 are equal.
5.3 Preliminary Subjective Testing
More comprehensive and rigorous tests are planned to fully evaluate the effectiveness of our approach. More detailed results from the above preliminary tests are available in .
5.4 Next Step – Virtualization
In this chapter, we examine cloud gaming from the delay (lag) perspective, and in particular the impact of lag on QoE of gamers. Cloud gaming is an emerging service, which combines cloud computing and online gaming. It opens a promising direction for the computer games industry but several challenges still remain to provide good QoE for every player. We review the literature and then analyze certain QoS-related key factors and characteristics, which influence the QoE for cloud gaming, especially for so-called FPS games. The conclusion is that for FPS games, network latency presents one of the most important QoE factors. In this context, we propose a lag equalization strategy to level the playing field in the context of QoE and outline two research objectives. The first objective is to examine the feasibility of implementing a real-time lag compensation mechanism for cloud gaming. To meet this objective, we implemented a cloud gaming system with both up and downlink lag compensation based on the Gaming Anywhere platform, an open platform for researchers. The FPS game Assault Cube was used to showcase the implementation. The mechanism uses a modified version of the RTCP protocol along with NTP to ensure adequate time synchronization to yield accurate each way delays. These are communicated to a centralized monitoring service that then determines and communicates back to each server the necessary up and downlink compensation delays. The lag compensation approach was evaluated in an emulated environment whereby a series of tests were carried out with differing uplink and downlink emulated delays to emulate differing network conditions. The results validate the mechanism by successfully implementing real-time delay equalization. To meet objective 2, we then carried out preliminary subjective tests with a small group of participants. Results firstly confirmed the impact of lag on QoE as detailed in the literature review and then validated our lag compensation approach whereby the reported QoE remained high for high delay values once equalization was implemented. Our future research will firstly optimize the experimental cloud gaming environment by introducing virtual machines for scalability. More comprehensive subjective QoE tests using the proposed lag compensation approach will then be undertaken to more rigorously evaluate its effectiveness.
This work has been partially supported by the ICT COST Action IC1304 - Autonomous Control for a Reliable Internet of Services (ACROSS), November 14, 2013–November 13, 2017, funded by European Union.
Andrej Zgank’s work was partially funded by the Slovenian Research Agency (research core funding No. P2-0069).
We would also like to acknowledge the technical support received by the core GA developer Chun-Ying Huang.
- 1.Jackson, P.: Video Game Revenue Forecast: 2017–22 (2017). https://www.ovum.com/research/video-game-revenue-forecast-2017-22/. Accessed 26 Apr 2017
- 2.Huang, C.Y., Chen, D.Y., Hsu, C.H., Chen, K.T.: GamingAnywhere: an open-source cloud gaming testbed. In: Proceedings of the 2013 ACM Multimedia conference (Open Source Software Competition Track), pp. 827–830. ACM, Barcelona (2013). http://dx.doi.org/10.1145/2502081.2502222
- 3.Claypool, M., Finkel, D.: The effects of latency on player performance in cloud-based games. In Proceedings of the 13th Annual Workshop on Network and Systems Support for Games (NetGames), pp. 1–6. IEEE, Nagoya (2014). https://doi.org/10.1109/NetGames.2014.7008964
- 5.Wen, Z.-Y., Hsiao, H.-F.: QoE-driven performance analysis of cloud gaming services. In: Proceedings of the 16th International Workshop on Multimedia Signal Processing (MMSP), pp. 1–6. IEEE, Jakarta (2014). https://doi.org/10.1109/MMSP.2014.6958835
- 6.Amiri, M., Al Osman, H., Shirmohammadi, S.: Datacenter traffic shaping for delay reduction in cloud gaming. In: Proceedings of the International Symposium on Multimedia (ISM), pp. 569–574. IEEE, San Jose (2016). https://doi.org/10.1109/ISM.2016.0124
- 7.Brun, J., Safaei, F., Boustead, P.: Fairness and playability in online multiplayer games. In: Proceedings of the 3rd IEEE Consumer Communications and Networking Conference (CCNC 2006), pp. 1199–1203. IEEE, Las Vegas (2006). https://doi.org/10.1109/CCNC.2006.1593228
- 8.Zander, S., Armitage, G.: Empirically measuring the QoS sensitivity of interactive online game players. In: Proceedings of the Australian Telecommunications Networks and Applications Conference (ATNAC 2004), pp. 511–518. ATNAC, Sydney (2004)Google Scholar
- 9.Zander, S., Leeder, I., Armitage, G.: Achieving fairness in multiplayer network games through automated latency balancing. In; Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, pp. 117–124. ACM, Valencia (2005). https://doi.org/10.1145/1178477.1178493
- 11.Möller, S., Pommer, D., Beyer, J., Rake-Revelant, J.: Factors influencing gaming QoE: lessons learned from the evaluation of cloud gaming services. In: Proceedings of the 4th International Workshop on Perceptual Quality of Systems (PQS), TU-Berlin, Vienna, Austria, pp. 1–5 (2013)Google Scholar
- 12.Slivar, I., Skorin-Kapov, L., Suznjevic, M.: Cloud gaming QoE models for deriving video encoding adaptation strategies. In: Proceedings of the 7th International Conference on Multimedia Systems (MMSys 2016), pp. 18:1–18:12. ACM, Klagenfurt (2016). https://doi.org/10.1145/2910017.2910602
- 13.Bernier, Y.W.: Latency compensating methods in client/server in-game protocol design and optimization. In: Proceedings of the Game Developers Conference, vol. 98033, no. 425 (2001)Google Scholar
- 14.Slivar, I., Suznjevic, M., Skorin-Kapov, L., Matijasevic, M.: Empirical QoE study of in-home streaming of online games. In: Proceedings of the 14th Annual Workshop on Network and Systems Support for Games (NetGames), pp. 1–6. IEEE, Nagoya (2014)Google Scholar
- 15.Clincy, V., Wilgor, B.: Subjective evaluation of latency and packet loss in a cloud-based game. In: Proceedings of the 10th International Conference on Information Technology: New Generations (ITNG), pp. 473–476. IEEE, Las Vegas (2013). https://doi.org/10.1109/ITNG.2013.79
- 16.Jarschel, M., Schlosser, D., Scheuring, S., Hoßfeld, T.: An evaluation of QoE in cloud gaming based on subjective tests. In: Proceedings of the 2011 Fifth International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS 2011), pp. 330–335. IEEE Computer Society, Washington, DC (2011). http://dx.doi.org/10.1109/IMIS.2011.92
- 17.Amiri, M., Malik, K.P.S., Al Osman, H., Shirmohammadi, S.: Game-aware resource manager for home gateways. In: Proceedings of the International Symposium on Multimedia (ISM), pp. 403–404. IEEE, San Jose (2016). https://doi.org/10.1109/ISM.2016.0091
- 19.Beyer, J., Varbelow, R., Antons, J.N., Zander, S.: A method for feedback delay measurement using a low-cost arduino microcontroller: lesson learned: delay influenced by video bitrate and game-level. In: Proceedings of the 7th International Workshop on Quality of Multimedia Experience (QoMEX), pp. 1–2. IEEE, Pylos-Nestoras (2013). https://doi.org/10.1109/QoMEX.2015.7148095
- 22.Li, Z.: Time Aware Gaming – Levelling the playing field for everyone. In: MSc thesis, National University of Ireland, Galway, Ireland, August 2017. Available on RequestGoogle Scholar
<SimplePara><Emphasis Type="Bold">Open Access</Emphasis> This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.</SimplePara> <SimplePara>The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.</SimplePara>