Skip to main content
SpringerLink
Log in

Studying the urgent updates of popular games on the Steam platform

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

The steadily increasing popularity of computer games has led to the rise of a multi-billion dollar industry. This increasing popularity is partly enabled by online digital distribution platforms for games, such as Steam. These platforms offer an insight into the development and test processes of game developers. In particular, we can extract the update cycle of a game and study what makes developers deviate from that cycle by releasing so-called urgent updates. An urgent update is a software update that fixes problems that are deemed critical enough to not be left unfixed until a regular-cycle update. Urgent updates are made in a state of emergency and outside the regular development and test timelines which causes unnecessary stress on the development team. Hence, avoiding the need for an urgent update is important for game developers. We define urgent updates as 0-day updates (updates that are released on the same day), updates that are released faster than the regular cycle, or self-admitted hotfixes. We conduct an empirical study of the urgent updates of the 50 most popular games from Steam, the dominant digital game delivery platform. As urgent updates are reflections of mistakes in the development and test processes, a better understanding of urgent updates can in turn stimulate the improvement of these processes, and eventually save resources for game developers. In this paper, we argue that the update strategy that is chosen by a game developer affects the number of urgent updates that are released. Although the choice of update strategy does not appear to have an impact on the percentage of updates that are released faster than the regular cycle or self-admitted hotfixes, games that use a frequent update strategy tend to have a higher proportion of 0-day updates than games that use a traditional update strategy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. E.g., related news for Dota 2: http://store.steampowered.com/news/?appids=570

  2. We attempted to extend this regular expression with more terms such as ‘patch’ and ‘emergency’, however, we found that these terms incorrectly match too many update notes that are not for hotfixes.

  3. http://store.steampowered.com/earlyaccessfaq/

  4. 4 Median Absolute Deviation, see Section 3.

References

  • Ampatzoglou A, Stamelos I (2010) Software engineering research for computer games: a systematic review. Inf Softw Technol 52(9):888–901

    Article  Google Scholar 

  • Arora A, Caulkins J P, Telang R (2006) Research note: sell first, fix later: Impact of patching on software quality. Manag Sci 52(3):465–471

    Article  Google Scholar 

  • Arora A, Krishnan R, Telang R, Yang Y (2010) An empirical analysis of software vendors’ patch release behavior: impact of vulnerability disclosure. Inf Syst Res 21(1):115–132

    Article  Google Scholar 

  • Becker R, Chernihov Y, Shavitt Y, Zilberman N (2012) An analysis of the Steam community network evolution. In: Proceedings of the 27th convention of electrical & electronics engineers in Israel (IEEEI). IEEE, pp 1–5

  • Blackburn J, Simha R, Kourtellis N, Zuo X, Long C, Ripeanu M, Skvoretz J, Iamnitchi A (2011) Cheaters in the Steam community gaming social network. arXiv preprint arXiv:11124915

  • Bohn R (2000) Stop fighting fires. Harv Bus Rev 78(4):82–91

    Google Scholar 

  • Chambers C, Feng WC, Sahu S, Saha D (2005) Measurement-based characterization of a collection of on-line games, USENIX Association

  • da Costa DA, McIntosh S, Kulesza U, Hassan AE (2016) The impact of switching to a rapid release cycle on the integration delay of addressed issues: an empirical study of the Mozilla Firefox project. In: Proceedings of the 13th international workshop on mining software repositories (MSR). ACM, pp 374–385

  • Galyonkin S (2016) SteamSpy - All the data and stats about Steam games. http://steamspy.com/, (last visited: Jul 16, 2016)

  • Gray J (2016) Steam Charts - Tracking What’s Played. http://steamcharts.com/, (last visited: Jul 16, 2016)

  • Hassan S, Shang W, Hassan A E (2016) An empirical study of emergency updates for top Android mobile apps. Empir Softw Eng:1–42

  • Huang J, Zimmermann T, Nagapan N, Harrison C, Phillips B C (2013) Mastering The art of war: how patterns of gameplay influence skill in Halo. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI). ACM, pp 695–704

  • Kerzazi N, Adams B (2016) Botched releases: do we need to roll back? Empirical study on a commercial web app. In: Proceedings of the 23rd international conference on software analysis, evolution, and reengineering (SANER), vol 1. IEEE, pp 574–583

  • Khomh F, Dhaliwal T, Zou Y, Adams B (2012) Do faster releases improve software quality? an empirical case study of mozilla firefox. In: 2012 9th IEEE working conference on mining software repositories (MSR). IEEE, pp 179–188

  • Khomh F, Adams B, Dhaliwal T, Zou Y (2015) Understanding the impact of rapid releases on software quality. Empir Softw Eng 20(2):336–373

    Article  Google Scholar 

  • Kim B C, Chen P Y, Mukhopadhyay T (2011) The effect of liability and patch release on software security: the monopoly case. Prod Oper Manag 20(4):603–617

    Article  Google Scholar 

  • Lesensmer (2013) Hotfix - League of Legends Wiki - wikia. http://leagueoflegends.wikia.com/wiki/Hotfix/, (last visited: Jul 16, 2016)

  • Lewis C, Whitehead J, Wardrip-Fruin N (2010) What went wrong: a taxonomy of video game bugs. In: Proceedings of the 5th international conference on the foundations of digital games (FDG). ACM, pp 108–115

  • Leys C, Ley C, Klein O, Bernard P, Licata L (2013) Detecting outliers: do not use standard deviation around the mean, use absolute deviation around the median. J Exp Soc Psychol 49(4):764–766

    Article  Google Scholar 

  • Long J D, Feng D, Cliff N (2003) Ordinal analysis of behavioral data. John Wiley & Sons, Inc

  • Mäntylä M V, Khomh F, Adams B, Engström E, Petersen K (2013) On rapid releases and software testing. In: Proceedings of the 29th international conference on software maintenance (ICSM). IEEE, pp 20–29

  • McGraw G, Hoglund G (2007) Online games and security. IEEE Secur Priv 5(5):76–79

    Article  Google Scholar 

  • McIlroy S, Ali N, Hassan AE (2016) Fresh apps: an empirical study of frequently-updated mobile apps in the Google Play store. Empir Softw Eng 21 (3):1346–1370

    Article  Google Scholar 

  • Microsoft (2003) Understanding patch and update management: Microsoft’s software update strategy. https://msdn.microsoft.com/en-us/library/cc768045.aspx, (last visited: Jul 16,2016)

  • Microsoft (2015) Xbox Game Store. http://marketplace.xbox.com, (last visited: Jul 16, 2016)

  • Miller J (1991) Short report: Reaction time analysis with outlier exclusion: bias varies with sample size. Q J Exp Psychol 43(4):907–912

    Article  Google Scholar 

  • Motoyama M, McCoy D, Levchenko K, Savage S, Voelker G M (2011) An analysis of underground forums. In: Proceedings of the 2011 SIGCOMM conference on internet measurement conference (IMC). ACM, pp 71–80

  • Murphy-Hill E, Zimmermann T, Nagappan N (2014) Cowboys, ankle sprains, and keepers of quality: how is video game development different from software development?. In: Proceedings of the 36th international conference on software engineering. ACM, pp 1–11

  • Nayebi M, Adams B, Ruhe G (2016) Release practices for mobile apps – what do users and developers think?. In: Proceedings of the 23rd international conference on software analysis, evolution and reengineering (SANER). IEEE, pp 552–562

  • Romano J, Kromrey J D, Coraggio J, Skowronek J, Devine L (2006) Exploring methods for evaluating group differences on the NSSE and other surveys: are the t-test and Cohen’s d indices the most appropriate choices. In: Annual meeting of the Southern association for institutional research

  • Rosenmai P (2013) Using the median absolute deviation to find outliers. http://eurekastatistics.com/using-the-median-absolute-deviation-to-find-outliers/, (last visited: Jul 16, 2016)

  • RUBAT (2013) Censorship update. http://steamcommunity.com/games/252490/announcements/detail/1478602529560858502, (last visited: Jul 16, 2016)

  • Seaman CB (1999) Qualitative methods in empirical studies of software engineering. IEEE Trans Softw Eng 25(4):557–572

    Article  Google Scholar 

  • Seaman C B, Shull F, Regardie M, Elbert D, Feldmann R L, Guo Y, Godfrey S (2008) Defect categorization: making use of a decade of widely varying historical data, ACM

  • Sifa R, Drachen A, Bauckhage C (2015) Large-scale cross-game player behavior analysis on Steam. In: Proceedings of the 11th artificial intelligence and interactive digital entertainment conference (AIIDE). AAAI

  • Sinclair B (2015) Gaming will hit $91.5 billion this year. http://www.gamesindustry.biz/articles/2015-04-22-gaming-will-hit-usd91-5-billion-this-year-newzoo, (last visited: Jul 16, 2016)

  • Souza R, Chavez C, Bittencourt R A (2015) Rapid releases and patch backouts: a software analytics approach. IEEE Softw 32(2):89–96

    Article  Google Scholar 

  • Stone J (2016) New ’Steam stealer’ malware gives hackers access to 77k users’ games, credit card numbers every month. http://www.ibtimes.com/new-steam-stealer-malware-gives-hackers-access-77k-users-games-credit-card-numbers-2337423/, (last visited: Jul 16, 2016)

  • SuperData (2015) Worldwide digital games market. https://www.superdataresearch.com/blog/us-digital-games-market/, (last visited: Jul 16, 2016)

  • Tassey G (2002) The economic impacts of inadequate infrastructure for software testing. National Institute of Standards and Technology

  • Valve (2016a) Steam community. http://steamcommunity.com/, (last visited: Jul 16, 2016)

  • Valve (2016b) Steam Store. http://store.steampowered.com/, (last visited: Jul 16, 2016)

  • Washburn Jr M, Sathiyanarayanan P, Nagappan M, Zimmermann T, Bird C (2016) “What went right and what went wrong”: an analysis of 155 postmortems from game development. In: Proceedings of the 38th international conference on software engineering (ICSE). IEEE/ACM, pp 280–289

  • Wiki W (2009) Hotfix - Vanilla WoW Wiki - Wikia. http://vanilla-wow.wikia.com/wiki/Hotfix, (last visited: Jul 16, 2016)

  • Zwillinger D, Kokoska S (1999) CRC standard probability and statistics tables and formulae. Crc Press

Download references

Author information

Authors and Affiliations

  1. Queen’s University, Kingston, ON, Canada

    Dayi Lin, Cor-Paul Bezemer & Ahmed E. Hassan

Authors
  1. Dayi Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cor-Paul Bezemer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ahmed E. Hassan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dayi Lin.

Additional information

Communicated by: Emerson Murphy-Hill

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, D., Bezemer, CP. & Hassan, A.E. Studying the urgent updates of popular games on the Steam platform. Empir Software Eng 22, 2095–2126 (2017). https://doi.org/10.1007/s10664-016-9480-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-016-9480-2

Keywords

Search

Navigation