Skip to main content

Advertisement

SpringerLink
Log in

An empirical study of the characteristics of popular Minecraft mods

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

It is becoming increasingly difficult for game developers to manage the cost of developing a game, while meeting the high expectations of gamers. One way to balance the increasing gamer expectation and development stress is to build an active modding community around the game. There exist several examples of games with an extremely active and successful modding community, with the Minecraft game being one of the most notable ones. This paper reports on an empirical study of 1,114 popular and 1,114 unpopular Minecraft mods from the CurseForge mod distribution platform, one of the largest distribution platforms for Minecraft mods. We analyzed the relationship between 33 features across 5 dimensions of mod characteristics and the popularity of mods (i.e., mod category, mod documentation, environmental context of the mod, remuneration for the mod, and community contribution for the mod), to understand the characteristics of popular Minecraft mods. We firstly verify that the studied dimensions have significant explanatory power in distinguishing the popularity of the studied mods. Then we evaluated the contribution of each of the 33 features across the 5 dimensions. We observed that popular mods tend to have a high quality description and promote community contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. https://mojang.com/

  2. https://www.paypal.com/

  3. https://www.patreon.com/

  4. https://minecraft.curseforge.com/projects/fossils

  5. Bukkit is a Minecraft Server mod that helps in the running and modification of a Minecraft server. See https://bukkit.org/pages/about-us/ for more details.

  6. https://worldofwarcraft.com/en-us/

  7. https://www.rdocumentation.org/packages/stats/versions/3.6.1/topics/glm

  8. https://www.rdocumentation.org/packages/stats/versions/3.6.1

  9. https://www.rdocumentation.org/packages/ScottKnottESD/versions/1.2.2/topics/%22sk_esd%22

  10. https://www.rdocumentation.org/packages/ScottKnottESD/versions/1.2.2

  11. https://www.rdocumentation.org/packages/Hmisc/versions/4.2-0/topics/varclus

  12. https://cran.r-project.org/web/packages/Hmisc/index.html

  13. https://www.rdocumentation.org/packages/Hmisc/versions/4.2-0/topics/redun

  14. https://www.rdocumentation.org/packages/car/versions/3.0-3/topics/Anova

  15. https://www.rdocumentation.org/packages/car/versions/3.0-3

  16. https://www.rdocumentation.org/packages/lmtest/versions/0.9-37/topics/lrtest

  17. https://www.rdocumentation.org/packages/lmtest/versions/0.9-37

  18. https://www.rdocumentation.org/packages/rms/versions/5.1-3.1/topics/fastbw

  19. https://www.rdocumentation.org/packages/rms/versions/5.1-3.1/topics/nomogram

  20. https://www.curseforge.com/minecraft/mc-mods/top-addons

  21. https://www.patreon.com/lexmanos

References

  • Ahmed F, Zia M, Mahmood H, Al Kobaisi S (2017) Open source computer game application: an empirical analysis of quality concerns. Entertain Comput 21:1–10

    Google Scholar 

  • Ahn S, Kang J, Park S (2017) What makes the difference between popular games and unpopular games? analysis of online game reviews from steam platform using word2vec and bass model. ICIC Express Letters,pp 1729–1737

  • Al-Washmi R, Bana J, Knight I, Benson E, Kerr OAA, Blanchfield P, Hopkins G (2014) Design of a math learning game using a Minecraft mod. In: European conference on games based learning, academic conferences international limited, vol 1, p 10

  • Ampatzoglou A, Stamelos I (2010) Software engineering research for computer games: a systematic review. Inf Softw Technol 52(9):888–901

    Google Scholar 

  • Arakji RY, Lang KR (2007) Digital consumer networks and producer-consumer collaboration: Innovation and product development in the video game industry. J Manag Inf Syst 24(2):195–219

    Google Scholar 

  • Balogh G, Beszédes Á (2013) CodeMetrpolis–A Minecraft based collaboration tool for developers. In: 2013 First IEEE working conference on software visualization VISSOFT, IEEE. pp 1–4

  • Bavota G, Linares-Vasquez M, Bernal-Cardenas CE, Di Penta M, Oliveto R, Poshyvanyk D (2014) The impact of api change-and fault-proneness on the user ratings of android apps. IEEE Trans Softw Eng 41(4):384–407

    Google Scholar 

  • Bayliss JD (2012) Teaching game AI through Minecraft mods. In: International Games Innovation Conference, IEEE. pp 1–4

  • Bebbington S (2014) A case study of the use of the game Minecraft and its affinity spaces for information literacy development in teen gamers. PhD thesis, Université d’Ottawa/University of Ottawa

  • Bécares JH, Valero LC, Martín PPG (2017) An approach to automated videogame beta testing. Entertain Comput 18:79–92

    Google Scholar 

  • Beggs B (2012) Minecraft, it’s a mod, mod, modder’s world: Computer game modifications as civic discourse. Reconstruction:, Studies in Contemporary Culture, 12(2)

  • Blackburn J, Kourtellis N, Skvoretz J, Ripeanu M, Iamnitchi A (2014) Cheating in online games: a social network perspective. ACM Trans Internet Technol (TOIT) 13(3):9

    Google Scholar 

  • Blake V (2019) Minecraft might be the biggest-selling video game of all time now. https://www.eurogamer.net/articles/2019-05-19-minecraft-might-be-the-biggest-selling-video-game-of-all-time-now, (last visited: July 11, 2019)

  • Blincoe K, Sheoran J, Goggins S, Petakovic E, Damian D (2016) Understanding the popular users: Following, affiliation influence and leadership on github. Inf Softw Technol 70:30–39

    Google Scholar 

  • Borges H, Valente MT (2018) What’s in a github star? understanding repository starring practices in a social coding platform. J Syst Softw 146:112–129

    Google Scholar 

  • Borges H, Hora A, Valente MT (2016a) Predicting the popularity of github repositories. In: Proceedings of the The 12th International Conference on Predictive Models and Data Analytics in Software Engineering, pp 1–10

  • Borges H, Hora A (2016b) Understanding the factors that impact the popularity of github repositories. In: 2016 IEEE International conference on software maintenance and evolution (ICSME), IEEE pp. 334–344

  • Brand J, Kinash S (2013) Crafting minds in Minecraft. Educ Technol Solut 55:56–58

    Google Scholar 

  • Canossa A, Martinez JB, Togelius J (2013) Give me a reason to dig Minecraft and psychology of motivation. In: 2013 IEEE Conference on Computational Inteligence in Games (CIG), IEEE. pp 1–8

  • Cheung GK, Zimmermann T, Nagappan N (2014) The first hour experience: how the initial play can engage (or lose) new players. In: First ACM SIGCHI Annual Symposium on Computer-Human Interaction in Play, ACM, pp 57–66

  • Chia PH, Yamamoto Y, Asokan N (2012) Is this app safe? a large scale study on application permissions and risk signals. In: Proceedings of the 21st international conference on World Wide Web, pp 311–320

  • Chun FKH, Karakiewicz PI, Briganti A, Walz J, Kattan MW, Huland H, Graefen M (2007) A critical appraisal of logistic regression-based nomograms, artificial neural networks, classification and regression-tree models, look-up tables and risk-group stratification models for prostate cancer. BJU international 99(4):794–800

    Google Scholar 

  • CurseForge (2006). CurseForge https://minecraft.curseforge.com/. (last visited: March 12, 2019)

  • Dey T, Massengill JL, Mockus A (2016) Analysis of popularity of game mods: A case study. In: Annual symposium on computer-human interaction in play companion extended abstracts, ACM, pp 133–139

  • Duncan SC (2011) Minecraft, beyond construction and survival. Well Played:, a journal on video games, value and meaning 1(1):1–22

    Google Scholar 

  • Ekaputra G, Lim C, Eng KI (2013) Minecraft: A game as an education and scientific learning tool. Information Systems International Conference (ISICO)

  • Fabric development team (2018). Fabric Announcement. http://fabricmc.net/2018/12/10/announcement.html, (last visited: August 30, 2019)

  • Finley K (2014) New Minecraft Mod teaches you code as you play. https://www.wired.com/2014/08/learntomod/, (last visited: July 11, 2019

  • Fox J, Monette G (2002) An R and S-Plus companion to applied regression. Sage

  • Geere D, Copeland W (2019) The best Minecraft mods. https://www.pcgamer.com/best-minecraft-mods/, (last visited: July 11 2019)

  • Graham TN, Roberts W (2006) Toward quality-driven development of 3D computer games. In: International workshop on design, Specification, and Verification of Interactive Systems. Springer, Berlin, pp 248–261

  • Guana V, Stroulia E, Nguyen V (2015) Building a game engine: A tale of modern model-driven engineering. In: IEEE/ACM 4th International Workshop on Games and Software Engineering, IEEE. pp 15–21

  • Guerrouj L, Azad S (2015) the influence of app churn on app success and stackoverflow discussions. In: 2015 IEEE 22Nd international conference on software analysis, evolution, and reengineering (SANER), IEEE. pp 321–330

  • Hackman E, Björkqvist U (2014) Modders of skyrim: Motivations and modifications: A qualitative study of what motivations and modifications the modders of Elder scrolls: Skyrim exhibit. Master’s thesis, Södertörn University

  • Hanghøj T, Hautopp H, Jessen C, Denning RC (2014) Redesigning and reframing educational scenarios for Minecraft within mother tongue education. In: European conference on games based learning, academic conferences international limited, vol 1. pp 182

  • Harman M, Jia Y, Zhang Y (2012) App store mining and analysis: Msr for app stores. In: 2012 9Th IEEE working conference on mining software repositories (MSR), IEEE. pp 108–111

  • Harrell FE Jr (2001) Lecture Notes in Biostatistical Modeling. http://biostat.mc.vanderbilt.edu/wiki/pub/Main/BioMod/notes.pdf. Last visited:, September 2, 2019

  • Harrell FE Jr, Slaughter JC (2001) Introduction to Biostatistics for Biomedical Research. http://hbiostat.org/doc/bbr.pdf, Last visited:, September 2, 2019

  • Harrell FE Jr, Lee KL, Califf RM, Pryor DB, Rosati RA (1984) Regression modelling strategies for improved prognostic prediction. Stat Med 3(2):143–152

    Google Scholar 

  • Jeppesen LB (2004) Profiting from innovative user communities: How firms organize the production of user modifications in the computer games industry. Tech rep, Department of Industrial Economics and Strategy Copenhagen Business School

  • Kalliamvakou E, Gousios G, Blincoe K, Singer L, German DM, Damian D (2014) The promises and perils of mining github. In: Proceedings of the 11th working conference on mining software repositories, pp 92–101

  • Köhler B, Haladjian J, Simeonova B, Ismailović D (2012) Feedback in low vs. high fidelity visuals for game prototypes. In: Second international workshop on games and software engineering: realizing user engagement with game engineering techniques, IEEE, pp 42–47

  • Lane HC, Yi S, Guerrero B, Comins NF (2017) Minecraft as a sandbox for stem interest development: Preliminary results. In: 25th International conference on computers in education proceedings

  • Leavitt A (2013) The source of open-source culture: Participation in the production of an open media artifact, minecraft. AoIR Selected Papers of Internet Research, pp 3

  • Lee D, Lin D, Bezemer CP, Hassan AE (2018) Building the perfect game - an empirical study of game modifications. Empirical Software Engineering Under review, pp 1–23

  • Lenig S, Caporusso N (2018) Minecrafting virtual education. In: International Conference on Applied Human Factors and Ergonomics. Springer, Berlin, pp 275–282

  • Lewis C, Whitehead J (2011) The whats and the whys of games and software engineering. In: 1st International Workshop on Games and Software Engineering, ACM, pp 1–4

  • Lewis C, Whitehead J, Wardrip-Fruin N (2010) What went wrong: a taxonomy of video game bugs. In: Fifth International Conference on the Foundations of Digital Games, ACM, pp 108–115

  • Lin D, Bezemer CP, Hassan AE (2017) Studying the urgent updates of popular games on the Steam platform. Empir Softw Eng 22(4):2095–2126

    Google Scholar 

  • Lin D, Bezemer CP, Hassan AE (2018) An empirical study of early access games on the Steam platform. Empir Softw Eng 23(2):771–799

    Google Scholar 

  • Lin D, Bezemer CP, Hassan AE (2019a) Identifying gameplay videos that exhibit bugs in computer games. Empir Softw Eng 24(115):1573–7616

  • Lin D, Bezemer CP, Zou Y, Hassan AE (2019b) An empirical study of game reviews on the Steam platform. Empir Softw Eng 24(1):170–207

  • Linares-Vásquez M, Bavota G, Bernal-Cárdenas C, Di Penta M, Oliveto R, Poshyvanyk D (2013) Api change and fault proneness: a threat to the success of android apps. In: Proceedings of the 2013 9th joint meeting on foundations of software engineering, pp 477–487

  • Long JD, Feng D, Cliff N (2003) Ordinal analysis of behavioral data. Handbook of psychology, pp 635–661

  • McIntosh S, Kamei Y, Adams B, Hassan AE (2016) An empirical study of the impact of modern code review practices on software quality. Empir Softw Eng 21(5):2146–2189

    Google Scholar 

  • Midi H, Sarkar SK, Rana S (2010) Collinearity diagnostics of binary logistic regression model. J Interdiscip Math 13(3):253–267

    MATH  Google Scholar 

  • Mills A (2015) A Brief History of Minecraft Modding. https://hub.packtpub.com/brief-history-minecraft-modding/https://hub.packtpub.com/brief-history-minecraft-modding/, (last visited: July 11, 2019)

  • Mojang (2019) What is Minecraft?. https://www.minecraft.net/en-us/what-is-minecraft/, (last visited: July 11, 2019)

  • Molnar C (2018) Interpretable machine learning. A Guide for Making Black Box Models Explainable 7. https://christophm.github.io/interpretable-ml-book/

  • Morrison A (2014) How To Make Minecraft A Survival Game. https://www.rockpapershotgun.com/2014/10/24/minecraft-survival-mods/, (last visited: March 12, 2019)

  • Müller S, Kapadia M, Frey S, Klinger S, Mann RP, Solenthaler B, Sumner RW, Gross M (2015) Statistical analysis of player behavior in Minecraft. In: Proceedings of the 10th International Conference on the Foundations of Digital Games, Society for the Advancement of the Science of Digital Games

  • Murphy-Hill E, Zimmermann T, Nagappan N (2014) Cowboys, ankle sprains, and keepers of quality: How is video game development different from software development?, ACM, USA

  • Nagappan M, Shihab E (2016) Future trends in software engineering research for mobile apps. In: 2016 IEEE 23Rd international conference on software analysis, evolution, and reengineering (SANER), IEEE, vol 5, pp 21–32

  • Nebel S, Schneider S, Rey GD (2016) Mining learning and crafting scientific experiments: a literature review on the use of minecraft in education and research. J Educ Technol & Soc 19(2):355–366

    Google Scholar 

  • Nguyen J (2016) Minecraft and the building blocks of creative individuality. Configurations 24(4):471–500

    Google Scholar 

  • Nieborg DB, Van der Graaf S (2008) The mod industries? the industrial logic of non-market game production. Eur J Cult Stud 11(2):177–195

    Google Scholar 

  • O’Brien C (2013) How Minecraft became one of the biggest video games in history. https://www.latimes.com/business/la-xpm-2013-sep-03-la-fi-tn-how-minecraft-video-games-20130822-story.html, (last visited: July 11, 2019

  • Pascarella L, Palomba F, Di Penta M, Bacchelli A (2018) How is video game development different from software development in open source?. In: 2018 IEEE/ACM 15th International Conference on Mining Software Repositories (MSR), pp 392–402

  • Petrillo F, Pimenta M, Trindade F, Dietrich C (2008) Houston, we have a problem... a survey of actual problems in computer games development. In: Proceedings of the 2008 ACM symposium on Applied computing, pp 707–711

  • Petrillo F, Pimenta M, Trindade F, Dietrich C (2009) What went wrong? a survey of problems in game development. Computers in Entertainment (CIE) 7(1):1–22

    Google Scholar 

  • Petrov A (2014) Using Minecraft in education: A qualitative study on benefits and challenges of Game-Based Education. Unpublished master’s thesis, University of Toronto, Ontario, Canada https://tspace.library.utoronto.ca/bitstream/1807/67048/1/Petrov_Anton_201406_MT_MTRP.pdf

  • Phillips T (2018) The human cost of Red Dead Redemption 2. https://www.eurogamer.net/articles/2018-10-25-the-human-cost-of-red-dead-redemption-2, (last visited: March 12, 2019)

  • Politowski C, Fontoura L, Petrillo F, Guéhéneuc YG (2016) Are the old days gone?: A survey on actual software engineering processes in video game industry. In: In: 5th International Workshop on Games and Software, Engineering, ACM, pp 22–28

  • Poretski L, Arazy O (2017) Placing value on community co-creations: A study of a video game ‘modding’ community. In: ACM Conference on Computer Supported Cooperative Work and Social Computing, ACM, pp 480–491

  • Postigo H (2007) Of mods and modders: Chasing down the value of fan-based digital game modifications. Games and Culture 2(4):300–313

    Google Scholar 

  • Quiring T (2015) From voxel vistas: Place-making in minecraft. J Virtual Worlds Res 8(1)

  • Romano J, Kromrey JD, Coraggio J, Skowronek J, Devine L (2006) Exploring methods for evaluating group differences on the nsse and other surveys: Are the t-test and cohen’sd indices the most appropriate choices. In: Annual Meeting of the Southern Association for Institutional Research, Citeseer

  • Ruiz A, Villa N (2008) Storms prediction: Logistic regression vs random forest for unbalanced data. arXiv:08040650

  • Saito D, Takebayashi A, Yamaura T (2014) Minecraft-based preparatory training for software development project. In: 2014 IEEE International Professional Communication Conference (IPCC). IEEE, pp 1–9

  • Scacchi W, Cooper KM (2015) Research challenges at the intersection of computer games and software engineering. In: Conference on Foundations of Digital Games

  • Scott R (2007). Nexus Mods. https://www.nexusmods.com/, (last visited: August 30 2019)

  • Shariat SF, Karakiewicz PI, Godoy G, Lerner SP (2009) Use of nomograms for predictions of outcome in patients with advanced bladder cancer. Ther Adv Urol 1(1):13–26

    Google Scholar 

  • Short D (2012) Teaching scientific concepts using a virtual world–Minecraft. Teac Sci J Aust Sc Teach Ass 58(3):55

    Google Scholar 

  • Shumovsky Y (2018) How much does it cost to make a video game?. https://vironit.com/how-much-does-it-cost-to-make-a-video-game/, (last visited: March 12 2019)

  • Sifa R, Bauckhage C, Drachen A (2014) The playtime principle: Large-scale cross-games interest modeling. In: IEEE Conference on Computational Intelligence and Games, IEEE, pp 1–8

  • Siko J, Barbour M, Toker S (2011) Beyond Jeopardy and lectures: Using Microsoft PowerPoint as a game design tool to teach science. J Comput Math Sc Teach 30(3):303–320

    Google Scholar 

  • Stone BG, Mills KA, Saggers B (2019) Online multiplayer games for the social interactions of children with autism spectrum disorder: a resource for inclusive education. Int J Incl Educ 23(2):209–228

    Google Scholar 

  • Taba SES, Keivanloo I, Zou Y, Ng J, Ng T (2014) An exploratory study on the relation between user interface complexity and the perceived quality. In: International Conference on Web Engineering. Springer, Berlin, pp 370–379

  • Tantithamthavorn C, McIntosh S, Hassan AE, Matsumoto K (2016) An empirical comparison of model validation techniques for defect prediction models. IEEE Trans Softw Eng 43(1):1–18

    Google Scholar 

  • Targett S, Verlysdonk V, Hamilton HJ, Hepting D (2012) A study of user interface modifications in World of Warcraft. Game Studies 12(2)

  • Thongtanunam P, Hassan AE (2018) Review dynamics and its impact on software quality. IEEE Trans Softw Eng, pp 1–13

  • Tian Y, Nagappan M, Lo D, Hassan AE (2015) What are the characteristics of high-rated apps? A case study on free Android applications. In: Software maintenance and evolution (ICSME) IEEE International Conference on, IEEE pp 301–310

  • Washburn M Jr, Sathiyanarayanan P, Nagappan M, Zimmermann T, Bird C (2016) What went right and what went wrong: an analysis of 155 postmortems from game development. In: 38th International Conference on Software Engineering Companion, ACM, pp 280–289

  • Wilcoxon F (1945) Individual comparisons by ranking methods. Biometrics bulletin 1(6):80–83

    Google Scholar 

  • Wu HA (2016) Video game prosumers: Case study of a minecraft affinity space. Visual Arts Research 42(1):22–37

    Google Scholar 

  • Zhu J, Zhou M, Mockus A (2014) Patterns of folder use and project popularity: A case study of github repositories. In: Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, pp 1–4

  • Zorn C, Wingrave CA, Charbonneau E, LaViola JJ Jr (2013) Exploring Minecraft as a conduit for increasing interest in programming. In: FDG, International Conference on the Foundations of Digital Games, pp 352–359

Download references

Author information

Authors and Affiliations

  1. Software Analysis and Intelligence Lab (SAIL), Queen’s University Kingston, Kingston, ON, Canada

    Daniel Lee, Gopi Krishnan Rajbahadur, Dayi Lin, Mohammed Sayagh & Ahmed E. Hassan

  2. Analytics of Software, Games and Repository Data (ASGAARD) Lab, University of Alberta Edmonton, Edmonton, AB, Canada

    Cor-Paul Bezemer

Authors
  1. Daniel Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gopi Krishnan Rajbahadur
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dayi Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohammed Sayagh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cor-Paul Bezemer
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ahmed E. Hassan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gopi Krishnan Rajbahadur.

Additional information

Communicated by: Emerson Murphy-Hill

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, D., Rajbahadur, G.K., Lin, D. et al. An empirical study of the characteristics of popular Minecraft mods. Empir Software Eng 25, 3396–3429 (2020). https://doi.org/10.1007/s10664-020-09840-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-020-09840-9

Keywords

Search

Navigation