Skip to main content
SpringerLink
Log in

Building the perfect game – an empirical study of game modifications

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Prior work has shown that gamer loyalty is important for the sales of a developer’s future games. Therefore, it is important for game developers to increase the longevity of their games. However, game developers cannot always meet the growing and changing needs of the gaming community, due to the often already overloaded schedules of developers. So-called modders can potentially assist game developers with addressing gamers’ needs. Modders are enthusiasts who provide modifications or completely new content for a game. By supporting modders, game developers can meet the rapidly growing and varying needs of their gamer base. Modders have the potential to play a role in extending the life expectancy of a game, thereby saving game developers time and money, and leading to a better overall gaming experience for their gamer base. In this paper, we empirically study the metadata of 9,521 mods that were extracted from the Nexus Mods distribution platform. The Nexus Mods distribution platform is one of the largest mod distribution platforms for PC games at the time of our study. The goal of our paper is to provide useful insights about mods on the Nexus Mods distribution platform from a quantitative perspective, and to provide researchers a solid foundation to further explore game mods. To better understand the potential of mods to extend the longevity of a game we study their characteristics, and we study their release schedules and post-release support (in terms of bug reports) as a proxy for the willingness of the modding community to contribute to a game. We find that providing official support for mods can be beneficial for the perceived quality of the mods of a game: games for which a modding tool is provided by the original game developer have a higher median endorsement ratio than mods for games that do not have such a tool. In addition, mod users are willing to submit bug reports for a mod. However, they often fail to do this in a systematic manner using the bug reporting tool of the Nexus Mods platform, resulting in low-quality bug reports which are difficult to resolve. Our findings give the first insights into the characteristics, release schedule and post-release support of game mods. Our findings show that some games have a very active modding community, which contributes to those games through mods. Based on our findings, we recommend that game developers who desire an active modding community for their own games provide the modding community with an officially-supported modding tool. In addition, we recommend that mod distribution platforms, such as Nexus Mods, improve their bug reporting system to receive higher quality bug reports.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. https://elderscrolls.bethesda.net/en/skyrim

  2. https://www.dota2.com/play/

  3. http://us.blizzard.com/en-us/games/war3/

  4. https://www.nexusmods.com/

  5. https://www.nexusmods.com/news/12474

  6. https://help.nexusmods.com/article/45-what-are-file-endorsements

  7. https://linux.die.net/man/1/file

  8. https://www.nexusmods.com/skyrim/mods/56086

  9. https://www.nexusmods.com/witcher3/mods/836?tab=posts

  10. https://www.nexusmods.com/Core/Libs/Common/Widgets/ModBugReplyList?issue_id=18914

  11. https://www.planetminecraft.com/resources/mods/

References

  • Ahmed F, Zia M, Mahmood H, Al Kobaisi S (2017) Open source computer game application: an empirical analysis of quality concerns. Entertain Comput 21:1–10

    Article  Google Scholar 

  • Ampatzoglou A, Stamelos I (2010) Software engineering research for computer games: a systematic review. Inf Softw Technol 52(9):888–901

    Article  Google Scholar 

  • Anvik J, Hiew L, Murphy GC (2005) Coping with an open bug repository. In: OOPSLA workshop on Eclipse technology eXchange. ACM, pp 35–39

  • Arakji RY, Lang KR (2007) Digital consumer networks and producer-consumer collaboration: Innovation and product development in the video game industry. J Manag Inf Syst 24(2):195–219

    Article  Google Scholar 

  • Ash0821 (2017) The Essential Skyrim Mods You Should Have Installed Right Now. https://www.gamepedia.com/blogs/1574-the-essential-skyrim-mods-you-should-have, (last visited: February 16, 2019)

  • Bécares JH, Valero LC, Martín PPG (2017) An approach to automated videogame beta testing. Entertain Comput 18:79–92

    Article  Google Scholar 

  • Bettenburg N, Just S, Schröter A, Weiss C, Premraj R, Zimmermann T (2008) What makes a good bug report? In: 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, pp 308–318

  • Blackburn J, Kourtellis N, Skvoretz J, Ripeanu M, Iamnitchi A (2014) Cheating in online games: a social network perspective. ACM Trans Internet Technol (TOIT) 13(3):9

    Article  Google Scholar 

  • BlindJudge (2018) Witcher 3 modding tools released with NMM support. https://www.nexusmods.com/witcher3/news/12550, (last visited: February 16, 2019)

  • Bostan B, Kaplancali U (2010) Explorations in player motivations: Game mods. In: GAMEON-ASIA. EUROSIS, pp 1–9

  • camycamera (2016) Why are Bethesda games so mod-popular? https://www.reddit.com/r/Fallout/comments/4454ig/why_are_bethesda_games_so_modpopular/, (last visited: February 16, 2019)

  • Chambers C, Feng Wc, Sahu S, Saha D (2005) Measurement-based characterization of a collection of on-line games. In: 5th ACM SIGCOMM conference on Internet Measurement. USENIX Association, pp 1–14

  • Cheung GK, Zimmermann T, Nagappan N (2014) The first hour experience: how the initial play can engage (or lose) new players. In: First ACM SIGCHI Annual Symposium on Computer-Human Interaction in Play. ACM, pp 57–66

  • Consalvo M (2007) Cheating : Gaining Advantage in Videogames. The MIT Press, Cambridge

  • Daneva M (2017) Striving for balance: a look at gameplay requirements of massively multiplayer online role-playing games. J Syst Softw 134:54–75

    Article  Google Scholar 

  • Dey T, Massengill JL, Mockus A (2016) Analysis of popularity of game mods: A case study. In: Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts. ACM, pp 133–139

  • Gallagher R, Jong C, Sinervo KA (2017) Who wrote the elder scrolls?: modders, developers, and the mythology of Bethesda softworks. Loading 10(16):1–21

  • Game Lengths (2009) Game Lengths. http://www.gamelengths.com/, (last visited: February 16, 2019)

  • Graham TN, Roberts W (2006) Toward quality-driven development of 3d computer games. In: International workshop on design, Specification, and Verification of Interactive Systems. Springer, pp 248–261

  • Guana V, Stroulia E, Nguyen V (2015) Building a game engine: A tale of modern model-driven engineering. In: IEEE/ACM 4th International Workshop on Games and Software Engineering. IEEE, pp 15–21

  • Hassan S, Shang W, Hassan AE (2017) An empirical study of emergency updates for top android mobile apps. Empir Softw Eng 22(1):505–546

    Article  Google Scholar 

  • Hansen M (2018a) Fallout 76 is already being fixed by modders. https://www.techradar.com/news/fallout-76-is-already-getting-fixed-by-modders, (last visited: February 22, 2019)

  • Hassan S, Bezemer CP, Hassan AE (2018b) Studying bad updates of top free-to-download apps in the Google Play store. IEEE Transactions on Software Engineering (In press)

  • Hirvonen E (2017) Improving the game with user generated content: an overview of Skyrim mod users. Master’s thesis, University of Jyväskylä, Finland

    Google Scholar 

  • Hruska J (2016) Comparing Skyrim: Enhanced Edition With the classic version. https://www.extremetech.com/gaming/238558-comparing-skyrim-enhanced-edition-classic-version (last visited: February 16 2019)

  • Jeppesen LB (2004) Profiting from innovative user communities: How firms organize the production of user modifications in the computer games industry. Technical report. Department of Industrial Economics and Strategy, Copenhagen Business School

  • Kanode CM, Haddad HM (2009) Software engineering challenges in game development. In: Sixth international conference on information technology, New Generations. IEEE, pp 260–265

  • Kasurinen J, Palacin-Silva M, Vanhala E (2017) What concerns game developers?: A study on game development processes, sustainability and metrics. In: 8th Workshop on Emerging Trends in Software Metrics. IEEE, pp 15–21

  • Köhler B, Haladjian J, Simeonova B, Ismailović D (2012) Feedback in low vs. high fidelity visuals for game prototypes. In: Second international workshop on games and software engineering, Realizing User Engagement with Game Engineering Techniques. IEEE, pp 42–47

  • Koleva B, Tolmie P, Brundell P, Benford S, Rennick Egglestone S (2015) From front-end to back-end and everything in-between: work practice in game development. In: Annual Symposium on Computer-Human Interaction in Play. ACM, pp 141–150

  • Laukkanen T (2005) Modding scenes-introduction to user-created content in computer gaming

  • Lee D, Lin D, Bezemer CP, Hassan AE (2019) Supplementary material for our paper,. https://github.com/SAILResearch/suppmaterial-19-danlee-nexusmods, (last visited: September 23, 2019)

  • Lewis C, Whitehead J, Wardrip-Fruin N (2010) What went wrong: a taxonomy of video game bugs. In: Fifth International Conference on the Foundations of Digital Games. ACM, pp 108–115

  • Lewis C, Whitehead J (2011) The whats and the whys of games and software engineering. In: 1st International Workshop on Games and Software Engineering. ACM, pp 1–4

  • Lin D, Bezemer CP, Hassan AE (2017) Studying the urgent updates of popular games on the Steam platform. Empir Softw Eng 22(4):2095–2126

    Article  Google Scholar 

  • Lin D, Bezemer CP, Hassan AE (2018) An empirical study of early access games on the Steam platform. Empir Softw Eng 23(2):771–799

    Article  Google Scholar 

  • Lin D, Bezemer CP, Hassan AE (2019a) Identifying gameplay videos that exhibit bugs in computer games. Empirical Software Engineering: 1–28

  • Lin D, Bezemer CP, Zou Y, Hassan AE (2019b) An empirical study of game reviews on the Steam platform. Empir Softw Eng 24(1):170–207

    Article  Google Scholar 

  • Long JD, Feng D, Cliff N (2003) Ordinal analysis of behavioral data. In: Schinka J. A., Velicer W. F. (eds) Handbook of psychology: Research methods in psychology, vol 2. Wiley, pp 635–661

  • Martin W, Sarro F, Jia Y, Zhang Y, Harman M (2017) A survey of app store analysis for software engineering. IEEE Trans Softw Eng 43(9):817–847

    Article  Google Scholar 

  • McIlroy S, Ali N, Hassan A E (2016) Fresh apps: an empirical study of frequently-updated mobile apps in the Google play store. Empir Softw Eng 21(3):1346–1370

    Article  Google Scholar 

  • Murphy-Hill E, Zimmermann T, Nagappan N (2014) Cowboys, ankle sprains, and keepers of quality: How is video game development different from software development?. In: 36Th international conference on software engineering. ACM, New York, pp 1–11

  • Nieborg DB, Van der Graaf S (2008) The mod industries? the industrial logic of non-market game production. Eur J Cult Stud 11(2):177–195

    Article  Google Scholar 

  • Pascarella L, Palomba F, Di Penta M, Bacchelli A (2018) How is video game development different from software development in open source? In: 2018 IEEE/ACM 15th International Conference on Mining Software Repositories (MSR), pp 392–402

  • Peterson S (2014) Gamer loyalty plays into profits. https://www.gamesindustry.biz/articles/2014-08-29-gamer-loyalty-plays-into-profits, (last visited: February 16, 2019)

  • Politowski C, Fontoura L, Petrillo F, Guéhéneuc YG (2016) Are the old days gone?: A survey on actual software engineering processes in video game industry. In: 5th International Workshop on Games and Software Engineering. ACM, pp 22–28

  • Poretski L, Arazy O (2017) Placing value on community co-creations: A study of a video game ‘modding’ community. In: ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, pp 480–491

  • Postigo H (2007) Of mods and modders: Chasing down the value of fan-based digital game modifications. Games Cult 2(4):300–313

    Article  Google Scholar 

  • Ramadan R (2016) Does game modding require programming? https://www.quora.com/Does-game-modding-require-programming, (last visited: February 16, 2019)

  • Romano J, Kromrey JD, Coraggio J, Skowronek J, Devine L (2006) Exploring methods for evaluating group differences on the nsse and other surveys: Are the t-test and cohen’sd indices the most appropriate choices. In: Annual Meeting of the Southern Association for Institutional Research. Citeseer

  • Scacchi W (2010) Computer game mods, modders, modding, and the mod scene. First Monday 15(5)

  • Scacchi W (2011a) Modding as a basis for developing game systems. In: 1st International Workshop on Games and Software Engineering. ACM, pp 5–8

  • Scacchi W (2011b) Modding as an open source approach to extending computer game systems. In: IFIP International Conference on Open Source Systems. Springer, pp 62–74

  • Scacchi W, Cooper KM (2015) Research challenges at the intersection of computer games and software engineering. In: Conference on Foundations of Digital Games

  • Sifa R, Bauckhage C, Drachen A (2014) The playtime principle: Large-scale cross-games interest modeling. In: IEEE Conference on Computational Intelligence and Games. IEEE, pp 1–8

  • Sotamaa O (2003) Computer game modding, intermediality and participatory culture. New Media: 1–5

  • Sotamaa O (2010) When the game is not enough: Motivations and practices among computer game modding culture. Games Cult 5(3):239–255

    Article  Google Scholar 

  • Steamchartscom (2018) Dota 2 Steam Charts. https://steamcharts.com/app/570 (last visited: February 16, 2019)

  • tinydanser (2017) List of performance mods that do more harm than good? https://bethesda.net/community/topic/72261/list-of-performance-mods-that-do-more-harm-than-good/2?lang=en-US&page=1, (last visited: February 16, 2019)

  • Villarroel L, Bavota G, Russo B, Oliveto R, Di Penta M (2016) Release planning of mobile apps based on user reviews. In: 2016 IEEE/ACM 38Th international conference on software engineering (ICSE). IEEE, pp 14–24

  • Washburn Jr, M, Sathiyanarayanan P, Nagappan M, Zimmermann T, Bird C (2016) What went right and what went wrong: an analysis of 155 postmortems from game development. In: 38th International Conference on Software Engineering Companion. ACM, pp 280–289

  • Wijman T (2018) Mobile Revenues Account for More Than 50% of the Global Games Market as It Reaches $137.9 Billion in 2018. https://newzoo.com/insights/articles/global-games-market-reaches-137-9-billion-in-2018-mobile-games-take-half/, (last visited: February 16, 2019)

  • Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bullet 1(6):80–83

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Software Analysis and Intelligence Lab (SAIL), Queen’s University, Kingston, ON, Canada

    Daniel Lee, Dayi Lin & Ahmed E. Hassan

  2. Analytics of Software, Games and Repository Data (ASGAARD) Lab, University of Alberta, Edmonton, AB, Canada

    Cor-Paul Bezemer

Authors
  1. Daniel Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dayi Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cor-Paul Bezemer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ahmed E. Hassan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cor-Paul Bezemer.

Additional information

Communicated by: Emerson Murphy-Hill

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, D., Lin, D., Bezemer, CP. et al. Building the perfect game – an empirical study of game modifications. Empir Software Eng 25, 2485–2518 (2020). https://doi.org/10.1007/s10664-019-09783-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-019-09783-w

Keywords

Search

Navigation