We present approaches to the generation of synthetic workloads for benchmarking multiplayer online gaming infrastructures. Existing techniques, such as mobility or traffic models, are often either too simple to be representative for this purpose or too specific for a particular network structure. Desirable properties of a workload are reproducibility, representativeness, and scalability to any number of players. We analyze different mobility models and AI-based workload generators. Real gaming sessions with human players using the prototype game Planet PI4 serve as a reference workload. Novel metrics are used to measure the similarity between real and synthetic traces with respect to neighborhood characteristics. We found that, although more complicated to handle, AI players reproduce real workload characteristics more accurately than mobility models.
This is a preview of subscription content,to check access.
Access this article
Similar content being viewed by others
This categorization is inspired by Fan et al. , who divide the tasks of a P2P game into six issues, of which five also apply to client/server infrastructures.
The relation between quality of service and quality of experience is far from trivial and a field of research on its own. Specifically for games, there has been research on player performance depending on network properties .
Using dead reckoning techniques , position updates might not be sent in a precisely fixed frequency, but instead to a certain degree depend on the players' activities. Still, there is usually a minimum and maximum rate at which these updates are transmitted. On average, they are thus expected to show a more regular behavior.
Beigbeder, T., Coughlan, R., Lusher, C., Plunkett, J., Agu, E., Claypool, M.: The effects of loss and latency on user performance in unreal tournament 2003. In: Proceedings of 3rd ACM SIGCOMM Workshop on Network and System Support for Games (NetGames’04), ACM, pp. 144–151 (2004)
Bharambe, A., Pang, J., Seshan, S.: Colyseus: A distributed architecture for online multiplayer games. In: Proceedings of the 3rd conference on Networked Systems Design & Implementation (NSDI’06), USENIX Association, Berkeley, pp. 155–168 (2006)
Bharambe, A., Douceur, J.R., Lorch, J.R., Moscibroda, T., Pang, J., Seshan, S., Zhuang, X.: Donnybrook: enabling large-scale, high-speed, peer-to-peer games. In: Proceedings of the ACM SIGCOMM Conference on Data Communication (SIGCOMM’08), ACM, pp. 389–400 (2008)
Champandard, A.: Behaviour trees for next gen game ai (video part1). http://aigamedev.com/open/articles/behavior-trees-part1/ (2007)
Claypool, M., Claypool, K.: Latency and player actions in online games. Commun. ACM 49(11), 40–45 (2006)
Claypool, M., Claypool, K.: Latency can kill: precision and deadline in online games. In: Proceedings of the First Annual ACM SIGMM Conference on Multimedia Systems (MMSys’10), ACM, pp. 215–222 (2010)
Fan, L., Trinder, P., Taylor, H.: Design issues for peer-to-peer massively multiplayer online games. Int. J. Adv. Media Commun. 4(2), 108–125 (2010)
Gross, C., Lehn, M., Muenker, C., Buchmann, A., Steinmetz, R.: Towards a comparative performance evaluation of overlays for networked virtual environments. In: Proceedings of the IEEE International Conference on Peer-to-Peer Computing (P2P’11), IEEE, pp. 34–43 (2011)
Hu, S.Y., Liao, G.M.:Von: A scalable peer-to-peer network for virtual environments. IEEE Netw. 20(4), 22–31 (2006)
Lang, T., Branch, P., Armitage, G.: A synthetic traffic model for Quake3. In: Proceedings of the 2004 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (ACE’04), ACM, pp. 233–238 (2004)
Lehn, M., Triebel, T., Gross, C., Stingl, D., Saller, K., Effelsberg, W., Kovacevic, A., Steinmetz, R.: Designing benchmarks for p2p systems. In: Sachs K, Petrov I, Guerrero P (eds) From Active Data Management to Event-Based Systems and More, Lecture Notes in Computer Science, vol 6462, Springer, pp. 209–229 (2010)
Lehn, M., Leng, C., Rehner, R., Triebel, T., Buchmann, A.: An online gaming testbed for peer-to-peer architectures. In: Proceedings of the ACM SIGCOMM 2011 Conference (SIGCOMM’11), ACM, demo (2011)
Pantel, L., Wolf, L.: On the suitability of dead reckoning schemes for games. In: Proceedings of the 1st Workshop on Network and System Support for Games, ACM, pp. 79–84 (2002)
Schmieg, A., Stieler, M., Jeckel, S., Kabus, P., Kemme, B., Buchmann, A.: pSense—maintaining a dynamic localized peer-to-peer structure for position based multicast in games. In: IEEE International Conference on Peer-to-Peer Computing (P2P’08) (2008)
Svoboda, P., Karner, W., Rupp, M.: Traffic analysis and modeling for World of Warcraft. In: Proceedings of the IEEE International Conference on Communications (ICC’07), pp. 1612–1617 (2007)
Terpstra, W.W., Kangasharju, J., Leng, C., Buchmann, A.P.: Bubblestorm: resilient, probabilistic, and exhaustive peer-to-peer search. In: Proceedings of the conference on Applications, technologies, architectures, and protocols for computer communications (SIGCOMM’07), ACM, pp. 49–60 (2007)
Triebel, T., Lehn, M., Rehner, R., Guthier, B., Kopf, S., Effelsberg, W.: Generation of synthetic workloads for multiplayer online gaming benchmarks. In: International Workshop on Network and Systems Support for Games (NetGames’12), IEEE (2012)
Yahyavi, A., Kemme, B.: Peer-to-peer architectures for massively multiplayer online games: a survey. ACM Comput. Surv. 46(1), 9:1–9:51 (2013). doi:10.1145/2522968.2522977
Yu, A.P., Vuong, S.T.: Mopar: a mobile peer-to-peer overlay architecture for interest management of massively multiplayer online games. In: Proceedings of the international Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV’05), ACM, pp. 99–104 (2005)
This work has been co-funded by the German Research Foundation (DFG) in the collaborative research center (SFB) 1053 "MAKI: Multi-Mechanism Adaptation for the Future Internet" and the research training group (GRK) 1343 "Topology of Technology".
About this article
Cite this article
Lehn, M., Triebel, T., Rehner, R. et al. On synthetic workloads for multiplayer online games: a methodology for generating representative shooter game workloads. Multimedia Systems 20, 609–620 (2014). https://doi.org/10.1007/s00530-014-0359-z