Empirical Software Engineering

, Volume 22, Issue 4, pp 2095–2126 | Cite as

Studying the urgent updates of popular games on the Steam platform

  • Dayi LinEmail author
  • Cor-Paul Bezemer
  • Ahmed E. Hassan


The steadily increasing popularity of computer games has led to the rise of a multi-billion dollar industry. This increasing popularity is partly enabled by online digital distribution platforms for games, such as Steam. These platforms offer an insight into the development and test processes of game developers. In particular, we can extract the update cycle of a game and study what makes developers deviate from that cycle by releasing so-called urgent updates. An urgent update is a software update that fixes problems that are deemed critical enough to not be left unfixed until a regular-cycle update. Urgent updates are made in a state of emergency and outside the regular development and test timelines which causes unnecessary stress on the development team. Hence, avoiding the need for an urgent update is important for game developers. We define urgent updates as 0-day updates (updates that are released on the same day), updates that are released faster than the regular cycle, or self-admitted hotfixes. We conduct an empirical study of the urgent updates of the 50 most popular games from Steam, the dominant digital game delivery platform. As urgent updates are reflections of mistakes in the development and test processes, a better understanding of urgent updates can in turn stimulate the improvement of these processes, and eventually save resources for game developers. In this paper, we argue that the update strategy that is chosen by a game developer affects the number of urgent updates that are released. Although the choice of update strategy does not appear to have an impact on the percentage of updates that are released faster than the regular cycle or self-admitted hotfixes, games that use a frequent update strategy tend to have a higher proportion of 0-day updates than games that use a traditional update strategy.


Update cycle Update strategy Urgent update Computer games Steam 


  1. Ampatzoglou A, Stamelos I (2010) Software engineering research for computer games: a systematic review. Inf Softw Technol 52(9):888–901CrossRefGoogle Scholar
  2. Arora A, Caulkins J P, Telang R (2006) Research note: sell first, fix later: Impact of patching on software quality. Manag Sci 52(3):465–471CrossRefGoogle Scholar
  3. Arora A, Krishnan R, Telang R, Yang Y (2010) An empirical analysis of software vendors’ patch release behavior: impact of vulnerability disclosure. Inf Syst Res 21(1):115–132CrossRefGoogle Scholar
  4. Becker R, Chernihov Y, Shavitt Y, Zilberman N (2012) An analysis of the Steam community network evolution. In: Proceedings of the 27th convention of electrical & electronics engineers in Israel (IEEEI). IEEE, pp 1–5Google Scholar
  5. Blackburn J, Simha R, Kourtellis N, Zuo X, Long C, Ripeanu M, Skvoretz J, Iamnitchi A (2011) Cheaters in the Steam community gaming social network. arXiv preprint arXiv:11124915
  6. Bohn R (2000) Stop fighting fires. Harv Bus Rev 78(4):82–91Google Scholar
  7. Chambers C, Feng WC, Sahu S, Saha D (2005) Measurement-based characterization of a collection of on-line games, USENIX AssociationGoogle Scholar
  8. da Costa DA, McIntosh S, Kulesza U, Hassan AE (2016) The impact of switching to a rapid release cycle on the integration delay of addressed issues: an empirical study of the Mozilla Firefox project. In: Proceedings of the 13th international workshop on mining software repositories (MSR). ACM, pp 374–385Google Scholar
  9. Galyonkin S (2016) SteamSpy - All the data and stats about Steam games., (last visited: Jul 16, 2016)
  10. Gray J (2016) Steam Charts - Tracking What’s Played., (last visited: Jul 16, 2016)
  11. Hassan S, Shang W, Hassan A E (2016) An empirical study of emergency updates for top Android mobile apps. Empir Softw Eng:1–42Google Scholar
  12. Huang J, Zimmermann T, Nagapan N, Harrison C, Phillips B C (2013) Mastering The art of war: how patterns of gameplay influence skill in Halo. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI). ACM, pp 695–704Google Scholar
  13. Kerzazi N, Adams B (2016) Botched releases: do we need to roll back? Empirical study on a commercial web app. In: Proceedings of the 23rd international conference on software analysis, evolution, and reengineering (SANER), vol 1. IEEE, pp 574–583Google Scholar
  14. Khomh F, Dhaliwal T, Zou Y, Adams B (2012) Do faster releases improve software quality? an empirical case study of mozilla firefox. In: 2012 9th IEEE working conference on mining software repositories (MSR). IEEE, pp 179–188Google Scholar
  15. Khomh F, Adams B, Dhaliwal T, Zou Y (2015) Understanding the impact of rapid releases on software quality. Empir Softw Eng 20(2):336–373CrossRefGoogle Scholar
  16. Kim B C, Chen P Y, Mukhopadhyay T (2011) The effect of liability and patch release on software security: the monopoly case. Prod Oper Manag 20(4):603–617CrossRefGoogle Scholar
  17. Lesensmer (2013) Hotfix - League of Legends Wiki - wikia., (last visited: Jul 16, 2016)
  18. Lewis C, Whitehead J, Wardrip-Fruin N (2010) What went wrong: a taxonomy of video game bugs. In: Proceedings of the 5th international conference on the foundations of digital games (FDG). ACM, pp 108–115Google Scholar
  19. Leys C, Ley C, Klein O, Bernard P, Licata L (2013) Detecting outliers: do not use standard deviation around the mean, use absolute deviation around the median. J Exp Soc Psychol 49(4):764–766CrossRefGoogle Scholar
  20. Long J D, Feng D, Cliff N (2003) Ordinal analysis of behavioral data. John Wiley & Sons, IncGoogle Scholar
  21. Mäntylä M V, Khomh F, Adams B, Engström E, Petersen K (2013) On rapid releases and software testing. In: Proceedings of the 29th international conference on software maintenance (ICSM). IEEE, pp 20–29Google Scholar
  22. McGraw G, Hoglund G (2007) Online games and security. IEEE Secur Priv 5(5):76–79CrossRefGoogle Scholar
  23. McIlroy S, Ali N, Hassan AE (2016) Fresh apps: an empirical study of frequently-updated mobile apps in the Google Play store. Empir Softw Eng 21 (3):1346–1370CrossRefGoogle Scholar
  24. Microsoft (2003) Understanding patch and update management: Microsoft’s software update strategy., (last visited: Jul 16,2016)
  25. Microsoft (2015) Xbox Game Store., (last visited: Jul 16, 2016)
  26. Miller J (1991) Short report: Reaction time analysis with outlier exclusion: bias varies with sample size. Q J Exp Psychol 43(4):907–912CrossRefGoogle Scholar
  27. Motoyama M, McCoy D, Levchenko K, Savage S, Voelker G M (2011) An analysis of underground forums. In: Proceedings of the 2011 SIGCOMM conference on internet measurement conference (IMC). ACM, pp 71–80Google Scholar
  28. Murphy-Hill E, Zimmermann T, Nagappan N (2014) Cowboys, ankle sprains, and keepers of quality: how is video game development different from software development?. In: Proceedings of the 36th international conference on software engineering. ACM, pp 1–11Google Scholar
  29. Nayebi M, Adams B, Ruhe G (2016) Release practices for mobile apps – what do users and developers think?. In: Proceedings of the 23rd international conference on software analysis, evolution and reengineering (SANER). IEEE, pp 552–562Google Scholar
  30. Romano J, Kromrey J D, Coraggio J, Skowronek J, Devine L (2006) Exploring methods for evaluating group differences on the NSSE and other surveys: are the t-test and Cohen’s d indices the most appropriate choices. In: Annual meeting of the Southern association for institutional researchGoogle Scholar
  31. Rosenmai P (2013) Using the median absolute deviation to find outliers., (last visited: Jul 16, 2016)
  32. RUBAT (2013) Censorship update., (last visited: Jul 16, 2016)
  33. Seaman CB (1999) Qualitative methods in empirical studies of software engineering. IEEE Trans Softw Eng 25(4):557–572CrossRefGoogle Scholar
  34. Seaman C B, Shull F, Regardie M, Elbert D, Feldmann R L, Guo Y, Godfrey S (2008) Defect categorization: making use of a decade of widely varying historical data, ACMGoogle Scholar
  35. Sifa R, Drachen A, Bauckhage C (2015) Large-scale cross-game player behavior analysis on Steam. In: Proceedings of the 11th artificial intelligence and interactive digital entertainment conference (AIIDE). AAAIGoogle Scholar
  36. Sinclair B (2015) Gaming will hit $91.5 billion this year., (last visited: Jul 16, 2016)
  37. Souza R, Chavez C, Bittencourt R A (2015) Rapid releases and patch backouts: a software analytics approach. IEEE Softw 32(2):89–96CrossRefGoogle Scholar
  38. Stone J (2016) New ’Steam stealer’ malware gives hackers access to 77k users’ games, credit card numbers every month., (last visited: Jul 16, 2016)
  39. SuperData (2015) Worldwide digital games market., (last visited: Jul 16, 2016)
  40. Tassey G (2002) The economic impacts of inadequate infrastructure for software testing. National Institute of Standards and TechnologyGoogle Scholar
  41. Valve (2016a) Steam community., (last visited: Jul 16, 2016)
  42. Valve (2016b) Steam Store., (last visited: Jul 16, 2016)
  43. Washburn Jr M, Sathiyanarayanan P, Nagappan M, Zimmermann T, Bird C (2016) “What went right and what went wrong”: an analysis of 155 postmortems from game development. In: Proceedings of the 38th international conference on software engineering (ICSE). IEEE/ACM, pp 280–289Google Scholar
  44. Wiki W (2009) Hotfix - Vanilla WoW Wiki - Wikia., (last visited: Jul 16, 2016)
  45. Zwillinger D, Kokoska S (1999) CRC standard probability and statistics tables and formulae. Crc PressGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Queen’s UniversityKingstonCanada

Personalised recommendations