Skip to main content

On synthetic workloads for multiplayer online games: a methodology for generating representative shooter game workloads

Abstract

We present approaches to the generation of synthetic workloads for benchmarking multiplayer online gaming infrastructures. Existing techniques, such as mobility or traffic models, are often either too simple to be representative for this purpose or too specific for a particular network structure. Desirable properties of a workload are reproducibility, representativeness, and scalability to any number of players. We analyze different mobility models and AI-based workload generators. Real gaming sessions with human players using the prototype game Planet PI4 serve as a reference workload. Novel metrics are used to measure the similarity between real and synthetic traces with respect to neighborhood characteristics. We found that, although more complicated to handle, AI players reproduce real workload characteristics more accurately than mobility models.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Notes

  1. 1.

    This categorization is inspired by Fan et al. [7], who divide the tasks of a P2P game into six issues, of which five also apply to client/server infrastructures.

  2. 2.

    The relation between quality of service and quality of experience is far from trivial and a field of research on its own. Specifically for games, there has been research on player performance depending on network properties [1].

  3. 3.

    Using dead reckoning techniques [13], position updates might not be sent in a precisely fixed frequency, but instead to a certain degree depend on the players' activities. Still, there is usually a minimum and maximum rate at which these updates are transmitted. On average, they are thus expected to show a more regular behavior.

References

  1. 1.

    Beigbeder, T., Coughlan, R., Lusher, C., Plunkett, J., Agu, E., Claypool, M.: The effects of loss and latency on user performance in unreal tournament 2003. In: Proceedings of 3rd ACM SIGCOMM Workshop on Network and System Support for Games (NetGames’04), ACM, pp. 144–151 (2004)

  2. 2.

    Bharambe, A., Pang, J., Seshan, S.: Colyseus: A distributed architecture for online multiplayer games. In: Proceedings of the 3rd conference on Networked Systems Design & Implementation (NSDI’06), USENIX Association, Berkeley, pp. 155–168 (2006)

  3. 3.

    Bharambe, A., Douceur, J.R., Lorch, J.R., Moscibroda, T., Pang, J., Seshan, S., Zhuang, X.: Donnybrook: enabling large-scale, high-speed, peer-to-peer games. In: Proceedings of the ACM SIGCOMM Conference on Data Communication (SIGCOMM’08), ACM, pp. 389–400 (2008)

  4. 4.

    Champandard, A.: Behaviour trees for next gen game ai (video part1). http://aigamedev.com/open/articles/behavior-trees-part1/ (2007)

  5. 5.

    Claypool, M., Claypool, K.: Latency and player actions in online games. Commun. ACM 49(11), 40–45 (2006)

    Article  Google Scholar 

  6. 6.

    Claypool, M., Claypool, K.: Latency can kill: precision and deadline in online games. In: Proceedings of the First Annual ACM SIGMM Conference on Multimedia Systems (MMSys’10), ACM, pp. 215–222 (2010)

  7. 7.

    Fan, L., Trinder, P., Taylor, H.: Design issues for peer-to-peer massively multiplayer online games. Int. J. Adv. Media Commun. 4(2), 108–125 (2010)

    Article  Google Scholar 

  8. 8.

    Gross, C., Lehn, M., Muenker, C., Buchmann, A., Steinmetz, R.: Towards a comparative performance evaluation of overlays for networked virtual environments. In: Proceedings of the IEEE International Conference on Peer-to-Peer Computing (P2P’11), IEEE, pp. 34–43 (2011)

  9. 9.

    Hu, S.Y., Liao, G.M.:Von: A scalable peer-to-peer network for virtual environments. IEEE Netw. 20(4), 22–31 (2006)

    MathSciNet  Article  Google Scholar 

  10. 10.

    Lang, T., Branch, P., Armitage, G.: A synthetic traffic model for Quake3. In: Proceedings of the 2004 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (ACE’04), ACM, pp. 233–238 (2004)

  11. 11.

    Lehn, M., Triebel, T., Gross, C., Stingl, D., Saller, K., Effelsberg, W., Kovacevic, A., Steinmetz, R.: Designing benchmarks for p2p systems. In: Sachs K, Petrov I, Guerrero P (eds) From Active Data Management to Event-Based Systems and More, Lecture Notes in Computer Science, vol 6462, Springer, pp. 209–229 (2010)

  12. 12.

    Lehn, M., Leng, C., Rehner, R., Triebel, T., Buchmann, A.: An online gaming testbed for peer-to-peer architectures. In: Proceedings of the ACM SIGCOMM 2011 Conference (SIGCOMM’11), ACM, demo (2011)

  13. 13.

    Pantel, L., Wolf, L.: On the suitability of dead reckoning schemes for games. In: Proceedings of the 1st Workshop on Network and System Support for Games, ACM, pp. 79–84 (2002)

  14. 14.

    Schmieg, A., Stieler, M., Jeckel, S., Kabus, P., Kemme, B., Buchmann, A.: pSense—maintaining a dynamic localized peer-to-peer structure for position based multicast in games. In: IEEE International Conference on Peer-to-Peer Computing (P2P’08) (2008)

  15. 15.

    Svoboda, P., Karner, W., Rupp, M.: Traffic analysis and modeling for World of Warcraft. In: Proceedings of the IEEE International Conference on Communications (ICC’07), pp. 1612–1617 (2007)

  16. 16.

    Terpstra, W.W., Kangasharju, J., Leng, C., Buchmann, A.P.: Bubblestorm: resilient, probabilistic, and exhaustive peer-to-peer search. In: Proceedings of the conference on Applications, technologies, architectures, and protocols for computer communications (SIGCOMM’07), ACM, pp. 49–60 (2007)

  17. 17.

    Triebel, T., Lehn, M., Rehner, R., Guthier, B., Kopf, S., Effelsberg, W.: Generation of synthetic workloads for multiplayer online gaming benchmarks. In: International Workshop on Network and Systems Support for Games (NetGames’12), IEEE (2012)

  18. 18.

    Yahyavi, A., Kemme, B.: Peer-to-peer architectures for massively multiplayer online games: a survey. ACM Comput. Surv. 46(1), 9:1–9:51 (2013). doi:10.1145/2522968.2522977

    Google Scholar 

  19. 19.

    Yu, A.P., Vuong, S.T.: Mopar: a mobile peer-to-peer overlay architecture for interest management of massively multiplayer online games. In: Proceedings of the international Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV’05), ACM, pp. 99–104 (2005)

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Robert Rehner.

Additional information

This work has been co-funded by the German Research Foundation (DFG) in the collaborative research center (SFB) 1053 "MAKI: Multi-Mechanism Adaptation for the Future Internet" and the research training group (GRK) 1343 "Topology of Technology".

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Lehn, M., Triebel, T., Rehner, R. et al. On synthetic workloads for multiplayer online games: a methodology for generating representative shooter game workloads. Multimedia Systems 20, 609–620 (2014). https://doi.org/10.1007/s00530-014-0359-z

Download citation

Keywords

  • Mobility Model
  • Game World
  • Game Session
  • Gaming Behavior
  • Game Event