What Empirically Based Research Tells Us About Game Development
- 132 Downloads
This paper reviews empirically grounded research on practices in game development with the intent to give a comprehensive overview of contemporary development practices used in the video game industry. While there are many intangible elements that inform game development processes, this review specifically covers the more immediate practical challenges. The review covers a total of 48 papers published between 2006 and 2016, which were all subjected to thematic analysis by three reviewers. The results of the review show that an almost universal characteristic of game development is that it is almost impossible to accurately plan a development project in detail, largely due to the soft requirements inherent in game production which emerge mid-process during development projects, during when testing is coupled with continuous ideation and refinement. Practicing game developers have created their own frameworks that accommodate for this lack of planning. They include flat hierarchies, democratic decision-making, creative autonomy, and informal communication, which are designed to create an environment that maintains creativity and openness to product changes long into the production process. These frameworks vary significantly between studios and often between individual projects. This review also shows that the term ‘Agile’, while often used by both researchers and developers to characterize the process of game development, is not an apt descriptor of how game developers actually work. Agile is used as shorthand for unstructured and flexible development, rather than serving as a descriptor of a definable or unified work method. Finally, as companies develop more complicated hierarchies of stakeholders and staff, the desired flexibility and autonomy of game development becomes increasingly complicated to maintain, and often necessitates more formalized management processes and company structures. In these cases, inherent tensions of game development become more pronounced, and continuous creativity is hard to maintain due to a growing need to formalize processes.
KeywordsGame development Empirics Management Development processes Game industry Literature review
In this paper, we present a literature review which addresses that particular potential gap, and surveys the development and production processes that underpin the games industry. Game development is often described in general terms with vague definitions, such as being subjective, flexible, and agile (Kortmann and Harteveld 2009; O’Hagan et al. 2014; Petrillo and Pimenta 2010). While these descriptors are not necessarily incorrect—in fact, as they are so often used they are likely to be fairly accurate—they are not particularly beneficial in helping the reader understanding what is actually happening during game development projects. As games are a complicated intermingling of various crafts and disciplines (e.g. audiovisual arts, design and user experience, genre conventions and tradition, software and hardware engineering, management, business and marketing, etc.) each game, and each studio developing it, is bound to have some unique quirks. Even though a universal answer is unlikely to exist, this review is an attempt to identify and highlight some of the patterns and commonalities that unify development practices.
… potential gaps in game research. […] While other authors may occasionally discuss the game industries, no other authors have this as their main research topic. Furthermore, none of the non-game authors cited are experts on business or industry. (Martin 2018)
An important distinction made in this review is that it is not concerned with hypothetical best practice prescriptions. There are many examples of research studies that describe how games should be developed: correlations between game design and models from UX and heuristics are made (e.g. Bernhaupt 2015; Koeffel et al. 2010); the viability of software development and standardizations on game development are discussed (e.g. Dormans 2012; Srisuriyasavad and Prompoon 2013); and project management and planning research are argued to be suitable approaches for solving challenges inherent in game development (e.g. McGregor 2013; Trantow et al. 2013; Vallance 2014; Vanhala and Kasurinen 2014). A persistent issue within this research field, however, is that such studies are rarely carried out by, or with input from, practitioners and experts from the games industry, and a large portion of literature on game development thus rarely takes everyday realities of practices into account (Martin 2018). With this in mind, this literature review specifically aims to examine how game development is described by individuals involved in game development, and thus this review exclusively focuses on empirically grounded research conducted on game developers and game development studios in order to present a picture of how games are developed.
The review presented in this article is part of a comprehensive literature study aiming to explore game development from a broad perspective, covering a wide set of disciplines. The details of the fundamental method of the study are described in Engström et al. (2018). To briefly summarize the literature selection process, a series of keywords related to game development, software engineering, and creative industries were used in several wide-reaching databases (Scopus, Springer, ACM, and DiGRA’s digital archives) to generate an initial set of 2278 publications. Through title and abstract analysis, this set was reduced to a set of 488 papers that were reviewed and coded. Two particular outcomes of this process are a research quality evaluation, and a case description that identifies whether a paper’s outcomes were based on empirical data from the games industry. The study presented in this article addresses one of four research questions identified in Engström et al. (2018), namely, “What are the current development practices used in the video games industry?” (p. 12).
It should be noted that the selection process was not limited to a pre-defined set of geographic regions. However, European and North American research studies dominate this area, so papers by authors from these regions are prominent in this review. The selection process involved identifying only papers written in English. It should also be noted that the vast majority of articles relating to game development have taken little or no consideration of regional aspects in their research approaches, and regional aspects that might affect game development are seldom discussed.
From the larger set of 488 articles, by applying the aforementioned method classifications, quality evaluations, and case descriptions, we have filtered out papers to produce a subset of 48 papers that fulfil the criteria that provide the foundation for this review: they contain empirical data from industry practitioners (i.e. not development conducted under academic auspices); they display a high quality of research (in terms of clearly stated research question, method description, and results); and, they are relevant to our understanding to the practices of game development.
The selected papers were subjected to a thematic analysis of the papers’ contents by the three reviewers. Thematic analysis is, as described by Braun and Clarke (2006, p. 6), “a method for identifying, analysing, and reporting patterns (themes) within data. It minimally organises and describes your data set in (rich) detail. However, it also often goes further than this, and interprets various aspects of the research topic…” As this review aims to identify patterns and themes which can describe game development, the qualitative-oriented nature of thematic analysis is suitable for doing in-depth content analysis of the sizeable amount of data which this review entails. The process was conducted using the MAXQDA qualitative analysis software program (VERBI GmbH 1995) which allowed the three reviewers to analyse and keep personal notes on the papers independently of one another, and to later merge their work to see if there were obvious points of disagreements or agreements regarding the papers’ content.
The entire process can be divided into three distinct phases: a preparation phase, a content processing phase, and an analysis phase. These phases entailed a total of 6 steps.
Phase 1: Preparation
Familiarization and initial coding the full set of papers was quickly and manually surveyed by the three reviewers. Not only did this serve to familiarize the reviewers with the material under review, but each reviewer also wrote down suggestions of code categories, which were then discussed in a reviewer meeting. In this initial coding step, all reviewers devised their own set of codes independently from one another, which ultimately resulted in an amount of codes that would be unfeasible to process in a reasonable timeframe. Many of these codes were also overlapping or semantically identical, even though the reviewers phrased them slightly differently, compounding the codes’ unsuitability for a continued reviewing process.
Establishing a unified code vocabulary in order to establish a more unified code vocabulary between the reviewers, two papers (Musil et al. 2010; Vallance 2014) were subjected to a round of test-coding. After using the initial code set on the first paper (Musil et al. 2010) the reviewers met to discuss the definitions, usefulness, accuracy, and potential interpretations of the different codes. After refining the code set based on these discussions, the process was repeated once more with the second paper (Vallance 2014). These trial runs served the purpose of ensuring that reviewers agreed upon code phrasings that risked having multiple interpretations, as having reviewers interpret codes in their own different ways would complicate later analysis stages significantly. The established code set was imported into the MAXQDA program, providing each reviewer with the same baseline template for the content analysis and coding that was to follow.
Phase 2: Processing the Full Set of 48 Papers
Secondary content processing and coding each of the three reviewers analysed the complete set of 48 papers by reading through them and marking text segments with codes from the established code categories. Every reviewer was given a different starting point in the data set, in order to minimize the potential impact of reviewers’ fatigue affecting the coding of a few particular papers too heavily. This process resulted in a total of 5190 coded segments.
Reviewer code synthesis with all the material coded, the three reviewers’ coding projects in MAXQDA were merged to create a unified team version of the coded material. This version, which included all 48 papers with all reviewers’ codes, provided the primary foundation for the next phase of the thematic analysis, in which the material was analysed to see which themes and patterns praxis emerged from the coded material.
Phase 3: Code Analysis, Theme Creation, and Report Compilation
Searching for themes the compiled and coded materials were analysed by the three reviewers to search for themes. While some themes emerged from the material rather quickly, the reviewers did not discriminate between themes of varied weight. The result from the initial analysis was a highlighting of various themes and the codes associated with them, which was discussed during reviewer meetings and subsequently used as the foundation for the final step of the process.
Defining and naming themes after establishing an initial set of themes, the review group took them into consideration in preparation for another set of meetings to better define the roots of the themes in coded material, and to name the themes appropriately to make them easier to present and describe in this paper. The various connections between coded segments and the themes were mapped out more clearly, and the reviewers ensured that the various nuances of positives and negatives surrounding each theme (as stated by developers in the empirical data) were properly represented.
3 Review Results
Overview of the themes, and the papers from which they emerged
Creating an experience
Creativity and ideation
Alves et al. (2007), Cohendet and Simon (2007), Hagen (2012), Hodgson and Briand (2013), Hotho and Champion (2011), Kultima (2010), Lê et al. (2013), Llerena et al. (2009), Musial et al. (2015), Simon (2006), Stacey et al. (2007), Stacey and Nandhakumar (2008, 2009), Tschang and Szczypula (2006), Wang and Nordmark (2015)
Testing and player experience
Cohendet and Simon (2007), Hodgson and Briand (2013), Kasurinen et al. (2013), Kasurinen and Smolander (2014), Koutonen and Leppänen (2013), Lê et al. (2013), Petrillo et al. (2009), Schmalz et al. (2014), Stacey and Nandhakumar (2008), Walfisz et al. (2006), Zackariasson et al. (2006)
Requirements and subjectivity
Alves et al. (2007), Bryant et al. (2010), Cohendet and Simon (2007), Daneva (2014), Kasurinen et al. (2014), Land and Wilson (2006), Murphy-Hill et al. (2014), O’Hagan and O’Connor (2015), Schmalz et al. (2014), Tran and Biddle (2008), Walfisz et al. (2006), Wang and Nordmark (2015), Zackariasson et al. (2006)
The technology and creativity schism
Creating a product
Methods in theory and in execution
Hodgson and Briand (2013), Kasurinen et al. (2014), Kasurinen and Smolander (2014), Koutonen and Leppänen (2013), Kultima (2010), Lê et al. (2013), Murphy-Hill et al. (2014), Nelson and Palumbo (2014), O’Hagan and O’Connor (2015), Schmalz et al. (2014), Stacey and Nandhakumar (2008), Walfisz et al. (2006), Wang and Nordmark (2015), Zackariasson et al. (2006)
Documentation and shared objects for collaboration
Alves et al. (2007), Cohendet and Simon (2007), Kasurinen et al. (2013, 2014), Murphy-Hill et al. (2014), Stacey and Nandhakumar (2009), Tran and Biddle (2008), Wang and Nordmark (2015), Wimmer and Sitnikova (2011)
The tools used in game development
Kasurinen et al. (2013, 2014), Kasurinen and Smolander (2014), Llerena et al. (2009), McAllister and White (2015), Murphy-Hill et al. (2014), O’Donnell (2011), O’Hagan and O’Connor (2015), Petrillo et al. (2009), Stacey and Nandhakumar (2009), Wang and Nordmark (2015), Vanhala and Kasurinen (2014)
Requirements, structure and formalization
Alves et al. (2007), Cohendet and Simon (2007), Hodgson and Briand (2013), McAllister and White (2015), Myllärniemi et al. (2006), Nelson and Palumbo (2014), Stacey et al. (2007), Stacey and Nandhakumar (2008), Walfisz et al. (2006), Wang and Nordmark (2015)
4 Research Overview
As per the criteria of the paper sampling process, all papers included empirical findings from game developers. This naturally meant that the reviewed papers were highly reliant on case studies (e.g. Amanatiadou and Van De Weerd 2009; Cohendet and Simon 2007; McAllister and White 2015; Myllärniemi et al. 2006; O’Hagan and O’Connor 2015; Vanhala and Kasurinen 2014). There were, however, some examples of research conducted on individual developers independently of any ongoing projects or studio work that dealt with experiences and attitudes towards their craft in a more general sense (e.g. Murphy-Hill et al. 2014; Wang and Nordmark 2015). As for data gathering methods, most of the reviewed papers relied primarily on qualitative methods, and semi-structured interviews were particularly common (Kasurinen et al. 2013, 2014; Kultima 2010). Some papers also used surveys, often to supplement or support the material gathered through interviews (Koutonen and Leppänen 2013; Murphy-Hill et al. 2014; Wang and Nordmark 2015). There were also a couple of examples of ethnographic research, in which the authors either recounted their own experiences working in development teams in the past (Walfisz et al. 2006) or kept journaled accounts of on-going development processes (Bryant et al. 2010; Cohendet and Simon 2007; Nelson and Palumbo 2014).
The vast majority of studied cases focused on what could be classified as more “straight forward” entertainment products for various platforms. There were a few exceptions, however, that focused on development of serious games (Ruggiero and Watson 2014; Tran and Biddle 2008). In this review, we do not make a particular distinction between the genres of games being developed, and we consider serious game developers to also be part of the broader games industry.
The investigated game companies in these papers have a varied international spread. Research studies in northern Europe and Canada constitute the largest portion of the reviewed set, while other regions are more sparsely represented. For example, no papers mention anything about Australian or African game development, making them the only excluded continents. Several papers investigate several game studios from different countries simultaneously (Chung and Fung 2013; Musial et al. 2015; Stacey and Nandhakumar 2008), while the most common approach seems to be to focus on one particular region; this is likely due to the researchers’ chosen methods (interviews) making it preferable to confine their study to one region (Hotho and Champion 2011; Zackariasson et al. 2006). Some of the papers do not include any descriptions of where the study was conducted (Drachen et al. 2013; O’Hagan et al. 2014; Tran and Biddle 2008).
The game studios studied in the reviewed papers differ from one another both in terms of size and organizational structure, ranging from small startup companies with a few people working on a single project (e.g. Llerena et al. 2009; O’Hagan and O’Connor 2015; Tran and Biddle 2008) to large established AAA studios with hundreds of employees working on multiple projects, sometimes across international borders (Cohendet and Simon 2007; Drachen et al. 2013; Walfisz et al. 2006); a number of studios fell somewhere in-between the two extremes (Hotho and Champion 2011; Myllärniemi et al. 2006; Nelson and Palumbo 2014). The studied game studios also worked with different platforms, such as PC (Kasurinen et al. 2013; Walfisz et al. 2006) mobile (Kultima 2010; Llerena et al. 2009; Myllärniemi et al. 2006) console (McAllister and White 2015) and web-browsers (Tran and Biddle 2008). It is difficult to determine whether there is a significant weighting towards any particular platform as many papers did not clearly state the examined studio’s target platform, and many studios also worked across many different platforms simultaneously. Thus, the set of reviewed papers seem to represent many different types of game development situations.
4.1 Creating an Experience
The first main theme in the studied material relates to how the game experience is created. This includes questions such as: how game ideas are born; when and how is the game design made; and what is the role of game testing. Testing, design, and ideation may not be exclusively relevant to game development as they, for example, happen in software and information system development as well. They are, however, uniquely approached in game development in that they are considerations that extend beyond functionality and effectivity. Many of the reviewed papers aimed to understand this distinctive characteristic of game development, making it a frequently recurring and nuanced theme in this analysis.
4.1.1 Creativity and Ideation
Creativity is an aspect of game development that is addressed in a majority of the papers. In many of them, there is an explicit focus on creativity and innovation in game development (Hagen 2012; Hodgson and Briand 2013; Hotho and Champion 2011; Kultima 2010; Lê et al. 2013; Llerena et al. 2009; Musial et al. 2015; Tschang and Szczypula 2006).
Not only are interpersonal relationships important for this type of creative ideation, but several studies (Lê et al. 2013; Stacey and Nandhakumar 2008, 2009; Tschang and Szczypula 2006; Wang and Nordmark 2015) also report how the interplay between ideas and the technology used to realize them affect the process. New technological possibilities, as well as limitations in the technology and even bugs, can give rise to new game concepts. The creative game development process is non-linear, whereby ideas evolve during the development and test process: “Game ideas are prone to be altered in one way or another during the design process. This was emphasized by virtually all of the interviewees” (Kultima 2010, p. 36).
The sources of creativity as well as efficiency at [Company] rely on a subtle alchemy among communities of scriptwriters, game-designers, graphic artists, sound designers, software programmers and even testers. The team is important for the creative process. (Cohendet and Simon 2007, p. 591)
A related issue observed in a large number of papers (Cohendet and Simon 2007; Hotho and Champion 2011; Kultima 2010; Musial et al. 2015; Simon 2006) is how creativity affects management of the game development process. These studies give a uniform message that creativity implies the need for more flexible processes, room for collaboration, and openness to change. As stated in one paper, “Management of creativity at work is merely seen as the art of setting the material, social and symbolic limits of the workspace collectively experienced as a creative play-ground” (Simon 2006, p. 121).
The auteur tradition, which is strong in the movie industry, has very limited support in the game industry. Only one study (Hotho and Champion 2011) reports on a case where one person tried to control the whole creative process. This study reports on how this effort had several negative consequences. In summary, the empirical studies on game development give a relatively uniform picture: that creativity is achieved through a collaborative, test-driven process where structure, documentation and control are de-emphasized.
4.1.2 Testing and Player Experience
Despite the lack of standard development methods and loose requirement specifications, game features, feature changes, and development goals still have to originate from somewhere. Play testing is an activity that is highlighted in many studies as one of these origins, and is often described as tightly connected to the “player experience” goal that many game developers pursue. As summarized by one interviewee in Kasurinen and Smolander (2014), “You can plan for a large number of things, but ultimately the final decision is made when actual people try out the idea.” Overall, developers are highly aware of, and accepting towards, change, and it is treated as an inevitable part of the process and crucial for honing the experience: “The game experience rules. Change is imperative” (Walfisz et al. 2006, p. 492).
The role of testing and player experience strongly influences the way game development is organised. Many studies report that companies adopt some kind of staged process. A common structure is to have the following four stages: ideation, pre-production, production, and post-production. A stage-gate methodology is discussed in some cases (Cohendet and Simon 2007; Hodgson and Briand 2013; Zackariasson et al. 2006), but it appears that the nature of game production makes it hard for developers to adhere to it even at its most general formulation. Play testing is conducted in all stages of the development process, and the result of these tests can have an impact on the product, irrespective of phase. As stated by an interviewee: “If a tester comes to say that this does not work, there is not fun in it, you really cannot leave that in the game, you have to fix it” (Kasurinen et al. 2013, p. 14).
Further, the first prototypes produced by the R&D team are replaced by others (including the one working with the SDK), which will subsequently be replaced by game prototypes. In a sense, there are only prototypes in videogame development. (Lê et al. 2013, p. 55)
The testing of the game may also lead to the identification of new ideas (see Creativity), which were not possible to foresee at an early ideation stage. Ideas can appear in all stages of game production, and they can originate from any of the involved actors, including testers: “You never know where the next great idea is coming from—as we saw at Goo, some even came from secretaries” (Stacey and Nandhakumar 2008, p. 145).
Interestingly, only a few studies (Koutonen and Leppänen 2013; Petrillo et al. 2009; Walfisz et al. 2006) discuss the concept of ‘feature creep’, which is a risk that arises when changes are allowed during all stages of production. Since it is difficult to clearly identify which features (e.g. themes, game mechanics, aesthetics, technologies, etc.) will ultimately result in the desired player experience, game developers often remain open to new ideas late in the production process (Petrillo et al. 2009). The main downside of this open-ended ideation is the high risk of feature creep, which is present in game development to a larger degree than in, for example, software development (Wang and Nordmark 2015). Developers try to minimize this risk by emphasizing a flexible and comprehensive early stage of game production (Cohendet and Simon 2007; Schmalz et al. 2014). Companies can have several potential products in the ideation and pre-production phases, and early and frequent playtesting of these prototypes will determine which should go into production. One of the papers conclude that the open-ended nature of game development is inherently risky, but that “the project structure itself often mitigated risk by emphasizing prototyping, pre-production and the early jettison of troubled projects” (Schmalz et al. 2014, p. 4332).
4.1.3 Requirements, Subjectivity and Flexibility
The vast majority of papers that discuss requirements as an aspect of game development characterize them as being highly subjective (Alves et al. 2007; Kasurinen et al. 2014; Murphy-Hill et al. 2014; O’Hagan and O’Connor 2015), unpredictable (Kasurinen et al. 2014; Tran and Biddle 2008), and flexible (Daneva 2014; Schmalz et al. 2014). The subjectivity is often tied to the artefact itself, as gameplay goals (e.g. aesthetic goals, or gameplay goals such as being fun or thrilling) are mentioned as something that is difficult to approach in a formalized and well-defined way (Kasurinen et al. 2014; Murphy-Hill et al. 2014). While the subjective concept of “fun” has often been described as something defined by the developers themselves in smaller studios (Zackariasson et al. 2006), larger studios have used identified subjective preferences and usability concerns of their target audiences as a guiding requirement (Alves et al. 2007; Bryant et al. 2010; Murphy-Hill et al. 2014). The unpredictability and changing nature of requirements are mostly discussed in terms of how they necessitate iterative working processes (Schmalz et al. 2014). Constant team communication (Land and Wilson 2006; Tran and Biddle 2008) and testing (Cohendet and Simon 2007; Kasurinen et al. 2014; Tran and Biddle 2008; Walfisz et al. 2006) are often mentioned as efficient means in scoping out and identifying requirements as production progresses. In essence, the nature of requirements in game development is often seen as one of the primary causes for why game development processes turn out the way they do.
When requirements are mentioned, they are often seen as a necessity due to their prevalence in traditional software development (Kasurinen et al. 2014; Land and Wilson 2006; Murphy-Hill et al. 2014); however, their merits and actual practical application in game development is debatable. In one of the reviewed papers, for example, developers were asked to evaluate how applicable ISO standardized software development processes were to their own working processes, to which they responded: “The first thing I see here is that requirement analysis is completely done before construction. So design is finished before anything is implemented… that’s just not the way it happens” (Kasurinen et al. 2014, p. 13). For most game developers, requirements are rarely seen as an identified goal that must be fulfilled, but rather something that is to be discovered during the development process (Daneva 2014; Tran and Biddle 2008).
… in an e-commerce application, a user has a task to complete that typically takes only a few minutes. … the requirement for games is that the user should be able to stay engaged on multiple timescales, and the mechanism to achieve that will vary from game to game. (Murphy-Hill et al. 2014, p. 4)
4.1.4 The Technology and Creativity Schism
A constant thread is the strong emphasis on the importance of a communal, democratic, and flexible production pipeline (e.g. role overlap, informal communication, team-based decisions, and shared knowledge architecture). Creativity was also equally emphasized as the driving force, and ultimate outcome, of game production (Kasurinen and Smolander 2014). There were, however, a few notable exceptions to this openness and flexibility: developers working with the software and technology aspects of game production had a comparatively protective approach to their work and contributions. Programmers in game studios could, for example, regard management with various levels of distrust if they were perceived to lack the necessary technological know-how to make realistic decisions (Murphy-Hill et al. 2014; Wang and Nordmark 2015).
We also found that conflicts may occur between designers and developers. This happens because game designers, who are especially creative individuals, generally try to include features that are very hard to implement. They argue that such features may improve the gameplay and the overall look and feel of the game. However, there is no formal process to assess that; just common-sense is used. Normally, the final game is the result of trade-offs among creative design, technical constraints and platforms constraints. (Alves et al. 2007, p. 279)
In essence, programming and technological knowledge seem to be under-appreciated and misunderstood in terms of their contribution to creativity (Kasurinen and Smolander 2014; Musial et al. 2015; Simon 2006; Wang and Nordmark 2015). In a study by Kasurinen and Smolander (2014), this was coupled with a sentiment that programming was the linchpin making the creation of games fundamentally possible: “…In their perspective the software development work was the actual requirement to create a game; without programming skills there would be no game product, but even without a competent artist you could create at least something.”
When a game-designer asks a programmer to design an animated “rope” as a decorative object in a virtual setting, he thinks its a very simple task and does not understand the rebuttal from the programmer, rather promoting a stick. (Simon 2006, p. 120)
While input and ideas from designers and artists are of course seen as an important part of the production process, they could be regarded with increased skepticism if they came from colleagues with little programming knowledge.
4.2 Creating a Product
The second main theme in the studied material concerns the industrialization of game production. This theme is focused on how production aspects, applicable in most production processes, are approached in the games industry. This includes project management, documentation, planning and the role of tools used during production.
4.2.1 Methods in Theory and Execution
One theme that emerges clearly from the studied papers is that there is no standard for game development, and that developments methods are applied differently from how they are prescribed (Kasurinen et al. 2014; Kasurinen and Smolander 2014; Koutonen and Leppänen, 2013; Lê et al. 2013; Murphy-Hill et al. 2014; O’Hagan and O’Connor 2015; Schmalz et al. 2014; Stacey and Nandhakumar 2008; Walfisz et al. 2006; Zackariasson et al. 2006). Developers’ reasons for deviating from established project management and software engineering methods mainly stem from the focus on player experience (Kultima 2010; Murphy-Hill et al. 2014; Walfisz et al. 2006; Wang and Nordmark 2015). As stated by Hodgson and Briand (2013): “Our results suggest that game developers focus on soft values such as game content or user experience, instead of more traditional objectives such as reliability or efficiency.”
The Agile methods are thus applied unorthodoxly, compared to the “regular” software industry, which regards Agile, and associated processes such as Scrum and eXtreme Programming, as being flexible but relying on an underlying structure and framework of planning, iterations, and backlogs (Hodgson and Briand 2013). This is, again, exemplified by studies which have analysed the applicability of ISO development standards in game development, and which conclude that the standard is difficult to implement in game companies (Kasurinen et al. 2013, 2014). Interviewed developers in other studies state, in clear terms, that Agile is unsuitable for their working processes: “We have a problem because the artists aren’t Agile. They detest it! … That’s a problem. There’s a dual system happening here.” (Hodgson and Briand 2013, p. 320).
We’ve got so many specialists on the team, so the kind of planning that you usually do in Agile doesn’t work quite so well… You know [specialists] are more concerned about the creative process than an engineering process. (Murphy-Hill et al. 2014, p. 7)
The challenge of controlling and standardizing the process of creating a product that is largely unpredictable and open to creative input until a very late stage of development is an issue that has been discussed in many papers (Cohendet and Simon 2007; Murphy-Hill et al. 2014; O’Hagan et al. 2014; Tschang and Szczypula 2006; Wang and Nordmark 2015), and these discussions are anchored in many different parts of the development ecosystem. It has been described as an issue with requirements, creative autonomy, technical constraints, or as a management issue, which might suggest that game development does not easily adhere to one particular methodological framework.
4.2.2 Documentation and Shared Objects for Collaborations
A strong trend in the game development literature is a very low focus on documentation and a strong focus on playable prototypes. The most frequently mentioned documentation (Alves et al. 2007; Kasurinen et al. 2014; Stacey and Nandhakumar 2009; Wimmer and Sitnikova 2011) is the game design document. It is worth noting that several of these studies more often identify shortcomings than benefits with the documental approach to game design. For example: “Even an apparently complete [game design document] is likely to change during the development process” (Alves et al. 2007, p. 278). Or, as stated by an interviewee: “When the production started, the specifications went out of the window… There simply is not enough knowledge to make a full design at the early stage.” (Kasurinen et al. 2013, p. 14).
It is also important to note that the producer is sometimes highlighted as having an important role in facilitating these processes; however, almost none of the reviewed papers focused on describing the role of the producer in game development, with only Schmalz et al. (2014) being a notable exception.
It’s very much a dialogue, we try not to have too formal split between tech and creative team when thinking about this, but prioritize what the user experience should be and when we can ship at target quality. (Wang and Nordmark 2015, p. 279)
Learning about experiences from others exposes each member to the different aspects of the game development process. As a result, the team is more empathetic to different disciplinary perspectives and approaches. (Tran and Biddle 2008, p. 51)
4.2.3 The Tools Used in Game Development
Software tools within game development are sparingly discussed in the reviewed literature. The papers that discuss software tools to a larger degree (Kasurinen et al. 2013, 2014; Kasurinen and Smolander 2014; Llerena et al. 2009; O’Donnell 2011; Wang and Nordmark 2015; Vanhala and Kasurinen 2014) clearly highlight the differences between game studios and how they use them. Some game studios create their own development tools from scratch, some modify existing ones, and some use third-party tools to create games. There is a difference between how these tools are selected and used, depending on company structure and size (Kasurinen et al. 2013, 2014; Stacey and Nandhakumar 2009). Small-to-medium sized game studios in particular outsource their game engine development (Kasurinen et al. 2013, 2014; Kasurinen and Smolander 2014; Wang and Nordmark 2015; Vanhala and Kasurinen 2014). The tools used in larger organizations reflect a more structured and formalized development process. Development support tools such as version control and file-sharing services, however, are commonly used in game development organizations, irrespective of size.
The articles that do discuss software tools often focus on the importance of an effective tool pipeline, tool selection and usage, and game developers’ experience. The use of software tools is very dependent on organization and surrounding circumstances. Therefore, they are often used differently.
4.2.4 Requirements, Structure and Formalization
There are a few specific areas in game development in which requirements are described in a more “traditional” manner as a list of fixed, objective, necessary goals that developers need to achieve. In papers dealing with large development projects, for example, AAA studios working with external stakeholders such as publishers, platform holders or investors, technical requirements seem to become significantly more explicit and static (Cohendet and Simon 2007; Hodgson and Briand 2013; Walfisz et al. 2006). For example, games released on major hardware-specific platforms (e.g. Microsoft, Sony, or Nintendo consoles, or different brands of phones) need to adhere to rigorous lists of requirements on performance, compatibility, and usability; in essence, software architecture seems to be one of the main areas of games that have rigorous requirements (Alves et al. 2007; McAllister and White 2015; Myllärniemi et al. 2006; Stacey et al. 2007; Wang and Nordmark 2015).
In situations where spontaneous communication is difficult, there is a need for formalized documentation and requirements (Alves et al. 2007; Schmalz et al. 2014; Stacey et al. 2007). In companies that did extensive outsourcing, or engaged in other forms of cross-company collaborations, requirements are often described as a crucial tool to ensure that the—normally quite messy and informal—act of game development could be channelled towards a clear goal (Hodgson and Briand 2013; Stacey and Nandhakumar 2008). In these cases, requirements seem to be used as a form of risk management, as it gives stakeholders an opportunity to present their expectations on developers’ performances, which developers then become beholden to (Hodgson and Briand 2013; Stacey et al. 2007; Walfisz et al. 2006).
In summary, there is no real consensus on the exact requirements for game development. The use of formalized requirements in development seems to be closely tied to a game studio’s size and “maturity”.
5 Discussion and Conclusion
The picture that the 48 reviewed papers paints of game development is a complex and ambivalent one: games are created in an entangled web of content development intertwined with production processes, in which technological and creative requirements may clash but also give rise to new opportunities.
There are some themes that emerged from this review that are prevalent enough across cases that they can be considered general truisms of game development. There is a conspicuous avoidance of firm methods and explicitly unified language among developers, and ad-hoc development driven by subjective experience requirements is the most prevalent praxis across the industry. The outcome of this review puts into question whether ‘Agile’ is actually an apt description of the development model that game projects employ. Several of the reviewed papers stated that while Agile is a term often used to describe game development due to its association with flexibility, game developers rarely use actual Agile methods (Hodgson and Briand 2013; Koutonen and Leppänen 2013; Murphy-Hill et al. 2014; Schmalz et al. 2014; Stacey and Nandhakumar 2008). As phrased in one of the papers, “Interviewees [implied] that Agile is sometimes a euphemism for a lack of process.” (Murphy-Hill et al. 2014, p. 6) Some developers also more or less explicitly stated that Agile is not a good fit for their working processes (Hodgson and Briand 2013; Koutonen and Leppänen 2013).
Practicing game developers have created frameworks that accommodate for this lack of planning, including flat hierarchies, democratic decision-making, creative autonomy, and informal communication, which create an environment that maintains creativity and openness to product changes long into the production process (Cohendet and Simon 2007; Llerena et al. 2009; Tran and Biddle 2008). A prevalent theme in the reviewed studies is that play-testing has a central role in all phases of game development (Kasurinen and Smolander 2014).
Some of the reviewed papers seem to be focused on making game development more “mature” by arguing for ways of standardizing (e.g. employing ISO standards) game development practices (e.g. Kasurinen et al. 2013; McAllister and White 2015; O’Hagan and O’Connor 2015; Seung et al. 2006). While the findings of these types of papers were often that standardization is unfeasible due to the unpredictable requirements inherent in game production, the conclusions were often that formalization needed to be pursued with different semantics so that developers understood their values better (Kasurinen et al. 2013; O’Hagan and O’Connor 2015), rather than acknowledging that the fluid practices that game developers use might be deliberate and carefully honed in spite of their sometimes chaotic appearance. In short, researchers’ ways of considering a practice as being “mature” might be at odds with game developers’ ways of working.
That being said, however, one interesting outcome from this research is that there is an uncertainty regarding whether even developers’ stated impression of what game development is, matches what they actually do. The general axiom of games being subjective, and game development thriving under flexibility and autonomous work, might be so strong as to be a self-fulfilling prophecy. One paper in particular featured an interesting case where an increase in studio autonomy in relation to creative direction (i.e. dropping external stakeholders to focus on the development team’s own intellectual properties) led to a more complicated and unhealthy working climate due to increasing economic uncertainty and changed power dynamics (Hotho and Champion 2011). Another paper raised the issue that, despite the conviction that Agile work processes facilitate autonomy and prevent the emergence of stifling power structures, empirical studies of developers’ day-to-day work have “highlighted the persistence of power hierarchies within and around the project team” (Hodgson and Briand 2013). Yet another paper brought up the issue of developers’ perceptions and expectations of their colleagues’ and their own passion for development leading to a form of peer pressure, which subsequently worked to maintain and exacerbate unhealthy working climates (Marie-Josée and Kathleen 2012). These types of findings suggest that the processes that developers pursue and value might not always be positive. While this review does not intend to provide prescriptions for how the craft of game development should evolve in the future, the outcomes do hint at a need to examine whether or not these universally agreed-upon practices exist because of the inherent values they bring to development projects, or if they primarily persist due to tradition.
Open access funding provided by University of Skövde. This research was funded by the University of Skövde, the Game Hub Scandinavia (NYPS 20200428) and the Game Hub Scandinavia 2.0 (NYPS 20201849) projects under the EU regional development fund Interreg Öresund-Kattegat-Skagerrak.
Compliance with Ethical Standards
Conflict of interest
The corresponding author states that there is no conflict of interest.
- Alves, C., Ramalho, G., & Damasceno, A. (2007). Challenges in requirements engineering for mobile games development: The meantime case study. In Proceedings-15th IEEE international requirements engineering conference, RE 2007 (pp. 275–280).Google Scholar
- Amanatiadou, A., & Van De Weerd, I. (2009). Extending the reference method for game production: A situational approach. In Proceedings of the 2009 conference in games and virtual worlds for serious applications, VS-GAMES 2009 (pp. 20–27).Google Scholar
- Bryant, J. A., Akerman, A., & Drell, J. (2010). Diminutive subjects, design strategy, and driving sales: Preschoolers and the nintendo DS. The International Journal of Computer Game Research, 10(1).Google Scholar
- Daneva, M. (2014). How practitioners approach gameplay requirements? An exploration into the context of massive multiplayer online role-playing games. In 2014 IEEE 22nd international requirements engineering conference, RE 2014-Proceedings (pp. 3–12).Google Scholar
- Dezso, C. L., Grohsjean, T., & Kretschmer, T. (2010). The what, the who, and the how: Coordination experience and team performance in the electronic game industry. In Academy of management 2010 annual meeting-dare to care: Passion and compassion in management practice and research, AOM 2010.Google Scholar
- Dormans, J. (2012). The effectiveness and efficiency of model driven game design. In M. Herrlich, R. Malaka, & M. Masuch (Eds.), Entertainment computing-ICEC 2012: 11th international conference, ICEC 2012, Proceedings, Bremen, Germany, September 26–29, 2012 (pp. 542–548). Berlin: Springer.Google Scholar
- Hagen, U. (2012). Lodestars for player experience: Ideation in videogame design. Stockholm: Stockholm University.Google Scholar
- Kasurinen, J., Laine, R., & Smolander, K. (2013). How applicable is ISO/IEC 29110 in game software development? In J. Heidrich, M. Oivo, A. Jedlitschka & M. T. Baldassarre (Eds.), Product-focused software process improvement: 14th international conference, PROFES 2013, Paphos, Cyprus, June 12–14, 2013. Proceedings (pp. 5–19). Berlin: Springer.Google Scholar
- Kasurinen, J., Maglyas, A., & Smolander, K. (2014) Is requirements engineering useless in game development? In Paper presented to the lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics).Google Scholar
- Kasurinen, J., & Smolander, K. (2014). What do game developers test in their products? In International symposium on empirical software engineering and measurement.Google Scholar
- Kasurinen, J., Strandén, J. P., & Smolander, K. (2013). What do game developers expect from development and design tools? In ACM International conference proceeding series (pp. 36–41).Google Scholar
- Koeffel, C., Hochleitner, W., Leitner, J., Haller, M., Geven, A., & Tscheligi, M. (2010). Using heuristics to evaluate the overall user experience of video games and advanced interaction games. In R. Bernhaupt (Ed.), Evaluating user experience in games: Concepts and methods (pp. 233–256). London: Springer.CrossRefGoogle Scholar
- Kortmann, L. J., & Harteveld, C. (2009). Agile game development: Lessons learned from software engineering. In Learn to game, game to learn; The 40th conference ISAGA.Google Scholar
- Koutonen, J., & Leppänen, M. (2013). How are agile methods and practices deployed in video game development? A survey into finnish game studios. In H. Baumeister, & B. Weber (Eds.), Agile processes in software engineering and extreme programming: 14th international conference, XP 2013, Vienna, Austria, June 3–7, 2013. Proceedings (pp. 135–149). Berlin: Springer.Google Scholar
- Kultima, A. (2010). The organic nature of game ideation: Game ideas arise from solitude and mature by bouncing. In Paper presented to the proceedings of the international academic conference on the future of game design and technology, Vancouver, British Columbia, Canada.Google Scholar
- Lê, P. L., Massé, D., & Paris, T. (2013). Technological change at the heart of the creative process: Insights from the videogame industry. International Journal of Arts Management, 15(2), 45–59.Google Scholar
- Llerena, P., Burger-Helmchen, T., & Cohendet, P. (2009). Division of labor and division of knowledge: A case study of innovation in the video game industry. In U. Cantner, J.-L. Gaffard, & L. Nesta (Eds.), Schumpeterian perspectives on innovation, competition and growth (pp. 315–333). Berlin: Springer.CrossRefGoogle Scholar
- Marie-Josée, L., & Kathleen, O. (2012). So into it they forget what time it is? Video game designers and unpaid overtime. In J. Dariusz & M. Abigail (Eds.), Managing dynamic technology-oriented businesses: High-tech organizations and workplaces (pp. 82–102). Hershey: IGI Global.Google Scholar
- Martin, P. (2018). The intellectual structure of game research. Game Studies, 18(1).Google Scholar
- McGregor, N. (2013). Business growth, the internet and risk management in the computer games industry. In S. Hotho & N. McGregor (Eds.), Changing the rules of the game: Economic, management and emerging issues in the computer games industry (pp. 65–81). London: Palgrave Macmillan.CrossRefGoogle Scholar
- Murphy-Hill, E., Zimmermann, T., & Nagappan, N. (2014). Cowboys, ankle sprains, and keepers of quality: How is video game development different from software development? In Paper presented to the proceedings of the 36th international conference on software engineering, Hyderabad, India. Google Scholar
- Musial, M., Kauppinen, A., & Puhakka, V. (2015). Recognised creativity: The influence of process, social needs, and the third drive on creative individuals’ work through social media. In Social media and networking: Concepts, methodologies, tools, and applications (Vol. 3–4, pp. 1249–1280).Google Scholar
- Musil, J., Schweda, A., Winkler, D., & Biffl, S. (2010). Improving video game development: Facilitating heterogeneous team collaboration through flexible software processes. In Paper presented to the communications in computer and information science.Google Scholar
- Myllärniemi, V., Raatikainen, M., & Männistö, T. (2006). Inter-organisational approach in rapid software product family development: A case study. In: M. Morisio (Ed.), Reuse of off-the-shelf components: 9th international conference on software reuse, ICSR 2006 Turin, Italy, June 12–15, 2006 Proceedings (pp. 73–86). Berlin: Springer.Google Scholar
- Nelson, W. A., & Palumbo, D. B. (2014). When design meets hollywood: Instructional design in a production studio environment. In B. Hokanson & A. Gibbons (Eds.), Design in educational technology: Design thinking, design process, and the design studio (pp. 75–88). Cham: Springer International Publishing.CrossRefGoogle Scholar
- O’Hagan, A. O., & O’Connor, R. V. (2015). Towards an understanding of game software development processes: A case study. In Paper presented to the communications in computer and information science.Google Scholar
- O’Hagan, A. O., Coleman, G., & O’Connor, R. V. (2014). Software development processes for games: A systematic literature review. In Paper presented to the communications in computer and information science.Google Scholar
- Petrillo, F., & Pimenta, M. (2010). Is agility out there? Agile practices in game development. In SIGDOC 2010-Proceedings of the 28th ACM international conference on design of communication (pp. 9–15).Google Scholar
- Petrillo, F., Pimenta, M., Trindade, F., & Dietrich, C. (2008). Houston, we have a problem…: A survey of actual problems in computer games development. In Proceedings of the ACM symposium on applied computing (pp. 707–711).Google Scholar
- Schmalz, M., Finn, A., & Taylor, H. (2014). Risk management in video game development projects. In Proceedings of the annual hawaii international conference on system sciences (pp. 4325–4334).Google Scholar
- Seung, H. L., Gum, H. L., Hyun, H. C., Doo, H. S., & Sung, Y. R. (2006). An empirical model of the game software development processes. In Proceedings-fourth international conference on software engineering research, management and applications, SERA 2006 (pp. 371–377).Google Scholar
- Srisuriyasavad, A., & Prompoon, N. (2013). Defining usability quality metric for game prototype using software attributes. Lecture Notes in Engineering and Computer Science, 2202, 522–528.Google Scholar
- Stacey, P., Brown, A., & Nandhakumar, J. (2007). Making sense of stories: The development of a new mobile computer game. In Proceedings of the annual hawaii international conference on system sciences.Google Scholar
- Tran, M. Q., & Biddle, R. (2008). Collaboration in serious game development: A case study. In Paper presented to the proceedings of the 2008 conference on future play: Research, play, share, Toronto, Ontario, Canada.Google Scholar
- Trantow, S., Hansen, A., Richert, A., & Jeschke, S. (2013). Emergence of innovation: Eleven strategies to increase innovative capability. In S. Jeschke, I. Isenhardt, F. Hees, & K. Henning (Eds.), Automation, communication and cybernetics in science and engineering 2011/2012 (pp. 161–189). Berlin: Springer.CrossRefGoogle Scholar
- VERBI GmbH (1995) MAXQDA, [Software], 18.0.7 edn.Google Scholar
- Wang, A. I., & Nordmark, N. (2015). Software architectures and the creative processes in game development. In K. Chorianopoulos, M. Divitini, J. Baalsrud Hauge, L. Jaccheri, & R. Malaka (Eds.), Entertainment computing-ICEC 2015: 14th international conference, ICEC 2015, Trondheim, Norway, September 29–Ocotober 2, 2015, Proceedings (pp. 272–285). Springer International Publishing, Cham.Google Scholar
- Wimmer, J., & Sitnikova, T. (2011). The professional identity of gameworkers revisited. A qualitative inquiry on the case study of German professionals. In Proceedings of DiGRA 2011 conference: Think design play.Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.