1 Introduction

Video game development has grown into a major global industry and source of economic growth in a number of countries. There are approximately 2.3 billion gamers across the globe and the global games market is expected to grow from $137.9 billion in 2018 to more than $180.1 billion in 2021.Footnote 1 Due to the international and highly competitive nature of the video game business, game companies have been in the forefront of adopting new technologies in their business. As a result, using the game industry as a benchmark can help to spur innovation among other businesses as well as public sector organizations. Furthermore, game-like elements have been successfully implemented to a wide range of services outside the games sector. The term gamification refers to the use of video game elements outside the gaming context to improve user experience and user engagement (Deterding et al. 2011; Hamari et al. 2014). For example, gamification has been employed in smart city design to collect citizen’s requirements (Zica et al. 2018). As a result, examining the game industry can provide valuable to insights to other sectors.

Against this backdrop, this study focuses on the use of big data and analytics in the games industry particularly in freemium game development. The use of big data and business analytics has become pervasive across industries (e.g. Grover and Kar 2017; Pousttchi and Hufenbach 2014; Bradlow et al. 2017) including game development (Bauckhage et al. 2012). A key driver in the diffusion of analytics among game developers has been the rapid proliferation of the freemium business model that has created the increased need to accurately measure, predict, and intervene in player behavior. By utilizing analytics, the video game industry is transferring from a product-based business logic towards software-as-a-service (SAAS) business logic, referred to as game-as-a-service (GAAS) (Clark 2014). This means continually developing and optimizing the game to maximize its revenues based on the data obtained from players’ behaviors. For example, Supercell, the company behind the top-grossing freemium games Clash of Clans and Clash Royale generated revenues of ca. $2 billion in 2017 without introducing any new game.Footnote 2 The second driver of the diffusion of game analytics has been the fast development and introduction of cost-effective ‘off-the-shelf’ analytics tools that make basic analytics accessible even for startup-sized game developers.

Business analytics refers to “the extensive use of data, statistical and quantitative analysis, explanatory and predictive models, and fact-based management to drive decisions and actions” (Davenport and Harris 2007, p. 7). Game analytics in turn refers to applying analytics and big data (cf. Wedel and Kannan 2016) in the gaming context (El-Nasr et al. 2013). Prior research has focused on describing methods of data gathering and analyzing (El-Nasr et al. 2013) as well as the role of analytics in game development (Canossa et al. 2013; Gómez-Maureira et al. 2014). Game analytics can be used to improve the players’ gaming experiences as well as to maximize in-game purchases by tweaking different elements of the game (Drachen et al. 2013a, b; McAllister et al. 2013). For example, game analytics can be used to optimize the time needed to complete a specific task, the price of a specific virtual item or the power of a specific weapon (Zoeller 2013; Mellon 2009).

Yet it is broadly known that the leading game developers utilize analytics extensively, game analytics as an area of academic study has thus far received relatively little coverage (Drachen and Canossa 2009). The limited amount of prior literature can be partly explained by the fact that game developers very often regard their use of analytics as a confidential issue and are thus reluctant to share information on their analytics processes (Wallner et al. 2014). As a result, the business aspects of game analytics are not well understood. Since the freemium model is widely adopted by game developers, examining the use of analytics by freemium game developers is of particular practical relevance.

While there is nascent research on the use of analytics in development of freemium games (Koskenvoima and Mäntymäki 2015), a gap in the literature exists with respect to the use of game analytics by small and medium-sized freemium game developers. Small and medium-sized game developers are an insightful research context since operate with scant resources and typically need funding and distribution partners to grow.

The present study therefore seeks to answer the following question: how do small and medium-sized freemium game developers use game analytics? To answer this research question, we conducted a set of in-depth expert interviews among small and medium-sized freemium game developers. The results show that the various uses of game analytics can be summarized as performing one of four main roles: 1) as a sense-making device, 2) as a decision-support system, 3) as a communication tool, and 4) as a hygiene factor. We employ the attention-based view (ABV) of the firm (Ocasio 1997) as the theoretical lens through which to discuss how these roles diverge and converge in terms of organizational attention. The study advances the understanding of the uses and business value of analytics in the game and software industry.

The remainder of this paper proceeds as follows. After the introduction, we review prior research on freemium models in the video game business and present the theoretical underpinnings of our study. The third section covers the research methodology, and the fourth section the results. In the fifth and final section, we discuss the main findings, summarize the theoretical and practical contributions, discuss the study’s limitations, and suggest areas for future research.

2 Background

2.1 Freemium Models in the Video Game Business

In the freemium business model, a basic or downgraded version of the service is offered free of charge; however, users who wish to access more features can purchase a premium version (Anderson, 2009; Liu et al. 2015). The freemium model is widely used in games (e.g. Candy Crush Saga, Clash of Clans) but also across utilitarian (e.g. Dropbox and LinkedIn) and hedonic online services (e.g. Spotify and YouTube). The business logic behind the freemium model is to attract a large customer base with the free offering and to then monetize this customer base by converting the non-paying customers into paying customers, for example via premium user accounts with additional benefits or featuring micro-transactions (Koch and Benlian 2017; Kumar 2014; Liu et al. 2015; Mäntymäki and Islam 2015). The freemium model can be employed in a number of different ways. The literature distinguishes between fixed freemium and flexible freemium. In the fixed freemium model, the consumer makes a choice between a free basic (or downgraded) version and a full-feature (upgraded) premium version. In the flexible freemium model, there are multiple premium versions and the consumer can switch back to the free basic version at any time (Shi et al. 2015).

Freemium games employ various monetization strategies (Fields and Cotton 2012), such as premium user accounts micro-transactions, and in-game advertising. Premium user account denote a model in which the player can purchase a premium subscription that provides an upgraded user experience compared to the free basic version. Micro-transactions in turn refer to buying virtual items or services that can be used and have value only inside a specific gaming environment (cf. Mäntymäki and Salo 2015). Second, the players can be provided with the opportunity to buy “time”, either to bypass waiting enforced by the game mechanics or to skip repetitive “grinding” phases (Fields and Cotton 2012), or to unlock additional levels or areas of the game. Third, virtual items and benefits compared to the non-paying users can be bundled into a premium user account or a subscription (Mäntymäki and Salo 2013). Fourth, popular games can attract millions of players and are thus attractive as advertising media. In freemium games, in-game advertising (cf. Salo and Karjaluoto 2007) can be used to monetize the user base. Moreover, an ad-free, uninterrupted gameplay experience, can be provided as a part of the premium offering. Fifth and finally, game operators can sell information about the players to third parties, typically marketers (Clemons 2009) similarly than e.g. social media platform operators.

Freemium games are typically designed to engage the player immediately since there is no initial cost causing a lock-in effect (Zauberman 2003). In addition, the game mechanics used in freemium games motivate the players to make in-game purchases (Hamari and Lehdonvirta 2010). As a result, sustained play and customer lifetime value are critical for the economic success of freemium games. This has led to the emergence of the GAAS concept (Clark 2014). GAAS highlights fostering player engagement and in-game purchasing as well as continuous development with incremental updates over the lifespan of the game (Clark 2014). This in turn requires detailed information about player behavior and product lifecycle. Game analytics is an important means to address this challenge.

2.2 Game Analytics

Game analytics is a subset of analytics applied to game development (Drachen et al. 2013b). Analytics in turn refers to using business intelligence in the process of discovering and communicating patterns from data and using recognized patterns in solving business problems (Davenport and Harris 2007). The need for game analytics has increased as games have become more sophisticated and complex (Hullett et al. 2011), and the rise of mobile gaming and the freemium business model has also had its effect.

Today’s video games, particularly online games and mobile games, are capable of collecting data about almost all players’ in-game activities (Drachen et al. 2013b). This data can be used to, for instance, to better understand the sources of player frustration (Pruett 2010). A well-known example of detecting player frustration is Replica Island, a mobile game for Android devices that employed a player tracking system to identify instances where players were facing difficulties such as deaths (Pruett 2010). Interestingly, the whole metrics system was implemented by a single-person analytics team at practically no monetary cost. Since its inception, Replica Island’s analytics and has proven to be very useful and valuable in detecting problems with the game design (Pruett 2010). However, according to Drachen et al. (2013c), analytics has not been very highly prioritized in the game industry. This is due to the fact that analytics and GAAS-based business logic differs considerably from the traditional value chain approach and product-oriented business logic of the video game development adopted by the leading global players of the industry (Drachen et al. 2013c; Clark 2014).

Prior game analytics research has employed the purchase funnel concept to illustrate the challenges related to the freemium model (El-Nasr et al. 2013; Fields & Cotton, 2011; Moreira et al. 2014). For example, the well-known AIDA model (awareness, interest, desire, and action) from consumer behavior literature describes the process of new product adoption (Webster Jr 1969). In application marketplaces where people typically download freemium games the customer can go through the process from awareness to action in seconds. However, out of the people who have downloaded a freemium game, only a small fraction, for example 5 %, will pay anything (Fields & Cotton, 2011). As a result, successful employment of the freemium model requires sustained player engagement and efficiently managing a large pool of non-paying players toward conversion during the course of play. Since a freemium game can be very popular without ever becoming profitable, a monetization strategy is a sine qua non in the freemium game business (Davidovici-Nora 2014).

2.3 Attention-Based View of the Firm

We adopt the attention-based view (ABV) of the firm as the theoretical lens through which to study freemium game developers’ utilization of game analytics. According to ABV, the context in which cognition and action are situated determines which aspects of the environment managers attend to and which opportunities are retained within the firm (Ocasio 1997; Ocasio and Joseph 2005). In line with Ocasio (1997), we define attention as the noticing, encoding, interpreting, and focusing of time and effort by organizational decision makers on both problems and solutions.

ABV recognizes that attention within an organization is not always uniform and that people may have both imperfect and divergent understandings of environmental signals (Rerup 2009; Weick and Sutcliffe 2006). While utilization of game analytics apparently increases an organization’s capacity to process information, the ABV perspective places specific emphasis on the issues decision makers consider important and on which they focus their attention at a specific time and place. Recent advances in ABV (Ocasio 2011) have emphasized the role of communication channels in “attentional engagement,” defined as the process of intentional, sustained allocation of cognitive resources to guide problem solving, planning, sensemaking, and decision making (Ocasio 2011). As a result, allocation of attention may help explain the different ways freemium game developers utilize game analytics (cf. Salvato 2009).

Previous ABV literature has examined the influence of attention structures on decision making (e.g., Bouquet and Birkinshaw 2008; Joseph and Ocasio 2012; Maula et al. 2013; Ocasio and Joseph 2005; Salvato 2009; Tuggle et al. 2010). Moreover, prior ABV studies have also investigated the top-down and/or bottom-up attentional processing in support of firm behavior (Kaplan and Tripsas 2008; Nadkarni and Narayanan 2007; Rerup 2009; Shepherd et al. 2007, 2017). As a result, ABV provides a relevant theoretical explanation of the differences in how game analytics is utilized by small and medium-sized game developers as well as how the members of the organizations perceive the role and value of game analytics.

3 Methodology

3.1 Data Collection

The empirical data was collected using in-depth interviews (see Miles et al. 2013) from 10 experienced game industry professionals. We chose interviews as the method to collect the empirical data since the purpose of the study is to obtain an in-depth understanding of the phenomenon under investigation (see Miles et al. 2013) as well as due to the confidentiality issues surrounding analytics. The analytics tools and processes employed in game analytics are often considered company confidential, particularly among game companies that consider analytics as a potential source of competitive advantage. Thus, obtaining in-depth insights into the specific ways game companies use analytics with e.g. a survey would be very challenging.

All interviewees had first-hand experience either from doing analytics themselves or utilizing analytics in their work. All informants had at least five years of relevant work experience for the games industry. In addition, all respondents held at least a master’s level degree from a field either related to analytics in particular or games industry in general.Footnote 3 Considering the background of the informants as well as the companies they represented, our study and the qualitative approach adopted is positioned to provide insights that are very difficult if not impossible to obtain with other research methods, for example a survey. Appendix Table 4 provides additional information about the informants.

The interviews were conducted face-to-face in sessions lasting from 50 min to 80 min. At the time of the interview, each of the interviewees from the companies were in the process of developing a freemium game and using software to collect and analyze gameplay data. All interviews were recorded and transcribed. In addition, notes were taken during the interviews.

When recruiting the informants, it appeared that game analytics, particularly when employed in an advanced manner, is often considered a sensitive topic with high levels of confidentiality. Because of confidentiality, a number of potential informants declined to be interviewed. However, a few of them shared some of their views and experiences “off-the-record.” The information obtained from these informal discussions was used as background material but is not reported here.

Furthermore, publicly available financial information, materials from company webpages, newspaper and magazine articles as well as blog posts and tweets were used to obtain contextual information about the companies the informants worked for. Finally, to obtain a players’ perspective and to gain additional insight into the phenomenon of interest the authors played the games by the interviewed companies and made notes about the gameplay experiences as well as thoughts regarding potential areas where game analytics could be used to assist game development.

3.2 Analysis

We used the Gioia method (Gioia et al. 2013) to guide our data analysis. According to Gioia et al. (2013), novel insights can often be obtained by carefully investigating how different actors of an organizational process experience events. Gioia et al. (2013) further suggest certain practices that bring “qualitative rigor.” As is typical of inductive research, the analytical process was iterative and partially overlapped with data collection. Nevertheless, certain phases in the analytical process can be recognized. During these phases, we iterated and refined inferences of theoretical mechanisms from the empirical material.

We started with open coding (Strauss and Corbin 1997). The first stage of the analysis process included reading the interview transcripts and marking codes to describe the content of the interviews. We used the research question to guide the first round of coding. Beyond coding, we identified differences and similarities among different segments of empirical data. This practice was similar to constant comparisons in grounded theory research (Corbin & Strauss 2008; Strauss and Corbin 1997). Table 1 provides a summary of the codes relevant for the present study and their related content.

Table 1 Summary of the codes used in the analysis and their related content
Full size table

In the second stage of the analysis process, we further categorized the first-order codes into more abstract concepts, while writing numerous brief notes and memos throughout the process to document the choices made and further develop our insights. As is typical for an iterative research process, we refined our coding procedures according to our evolving understanding (Strauss and Corbin 1997). In the third stage of the analysis process, we incorporated the ABV perspective into the analysis and specifically looked at how attention manifests itself in the categories identified in the previous stage. This led to the emergence of four theory-guided aggregate themes that will be discussed in the next section. The three stages of the analysis process are summarized in Fig. 1.

Fig. 1
figure 1

The three stages of the analysis process employed in the research

Full size image

We took several measures to ensure the rigor of the research process and the trustworthiness of our interpretations. First, we chose the Gioia method (Gioia et al. 2013) to guide the analysis process. The Gioia method has been developed and widely used to systematize the process of conducting and reporting qualitative research. Second, we followed the well-established guidelines for qualitative inquiry by Lincoln and Guba (1985) to maximize the trustworthiness of our findings in terms of credibility, transferability, dependability and confirmability. Table 2 below provides a summary of the measures taken to ensure different dimensions of trustworthiness.

Table 2 Measures taken to ensure the trustworthiness of results (Lincoln and Guba 1985)
Full size table

4 Results

Four aggregate themes portraying different uses of game analytics emerged as a result of the analysis. These four themes describe the role of game analytics 1) as a sense-making device, 2) as a decision-support system, 3) as a communication tool, and 4) as a hygiene factor. In this section we elaborate on these four themes and interpret them through the ABV lens (see Table 2 for a summarized description of the four themes and their interpretation).

4.1 Game Analytics as a Sense-Making Device

The first category, sense-making, refers to the use of analytics to better understand (i.e., make sense of) player behavior and preferences. For example, using player telemetry data to identify specific points in the game where players spend excessive time or quit playing, in order to help developers in quality assurance, for example.

“Data can help understand what is happening in the player’s head. Often you start with a hunch and then iterate it toward a better formulated question.” (P7)

“There is no other way (except analytics) to obtain data about the players. Of course we can ask feedback but I feel it is often biased in so many ways.” (P9)

Particularly the informants who had been actively involved in game design, highlighted that the role of analytics is not, and should not be, to drive game design. The informants with design and development background maintained that analytics can be very useful in, for example, questioning, and occasionally disqualifying, the intuition of the designers. Nevertheless, there was a clear consensus among the informants that analytics as such is not a silver bullet.

“You cannot make a good game with just analytics, it’s very challenging – [everything new] comes from the creative side.” (P2)

Further underscoring that the design philosophy is not data-driven but rather data-supported, the informants pointed out that while analytics activities such as collecting, storing, and analyzing data can be outsourced, game design is essentially an in-house activity. In other words, analytics and metrics do not make the games, but they can be very useful and valuable in making games better. The following quotation illustration the role of game analytics in relation to game design:

“Maybe the larger benefit, at least in our case, is that instead of driving development, they [game analytics] are used to spot errors in the code and clear design mistakes. It’s more about monitoring – first you design, then you code and then you monitor how well did it go.” (P5)

The data-supported design process was considered strongly iterative. First, a visible problem is noticed in high-level metrics. Then the designers seek possible causes of the problem by drilling down into the data. Thereafter, specific changes are made to the game, and then the effects are measured. As a result, there is a constant loop for validating design decisions as depicted in the following quotation:

“[We tend to find a] high-level issue and then try to find one specific user experience issue that you think you change, and then you iterate the process.” (P4)

When interpreting these observations through ABV, using game analytics as a sense-making device represents a bottom-up perspective to direct the organization’s attention. In other words, identifying the important issues and directing attention takes place at a “shop floor level” instead of at the level of management (cf. Shepherd et al. 2017).

4.2 Game Analytics as Decision-Support System

The second aggregate theme, decision-making, refers to the use of analytics for finding answers to more clearly predefined questions. Thus, compared to sense-making, where the focus is on identifying the unknowns, decision-making revolves around more concrete issues. From an ABV perspective, decision-making represents a top-down approach in directing attention within an organization.

“It is clear that – especially in this free-to-play model – you cannot operate under the mentality that you just launch a game and hope for the best.” (P1)

Typical examples of decision-making include predicting revenues generated from a game schedule, new product launches or ramp-down of existing products. Similarly, analytics is typically used in making decisions regarding which game mechanics to use in new products as well as estimating the typical points of purchase within the game and during the player lifecycle.

“There are companies that operate by replicating the mechanics and core ideas of top-grossing games. Of course, their games don’t become as popular as the ‘original’ counterparts but it doesn’t matter that much as the idea is to have a portfolio of these copycat games. So, when revenues from one game start to decline they start ramping up a new game ... the margins are thin when you do it this way so it’s really about analytics.” (P8)

With respect to decision-making, our interviews illustrate an interesting controversy in how the role of game analytics is viewed. All our informants generally acknowledged the value-added of analytics and some even openly advocated tracking as much as possible to maximize the amount of data available. Others in turn preferred a stricter focus on certain key metrics in order to avoid information overload and collecting what they referred as “vanity metrics.” These informants also more deliberately highlighted the importance of evaluating the value of game analytics against the respective costs.

A recurring theme in the interviews was that players are surprisingly unpredictable and that they often behave differently than the designers expected, often in a seemingly irrational way. Analytics can thus assist game design in providing a better understanding of players’ gaming experiences. For instance, one of our informants described a mobile game with design mechanic that punished the player for erroneous behavior by deducting his/her points. A competing solution was to terminate the session immediately after an error was made and force the players to start over. Somewhat counter-intuitively, the latter version, which the designers felt was more “hardcore-oriented”, was more appealing to players and also yielded to superior retention rates. Thus, game analytics proved that, contrary to certain conventions of game development and designers’ expectations, the harder and more punishing version was more popular.

“We don’t do analytics because they are cheaper, but because they are better. I never trust people who say, ‘If you would develop this, I would use it all the time.’ Only when I can measure that they really use it, will I believe it.” (P3)

The analysis tools and methods in use were largely uniform among informants. Maybe for this reason the informants did not consider game analytics as a genuine source of competitive advantage. As all our informants worked for small or medium-sized companies with very limited human and financial resources to allocate to analytics, they had mostly adopted ‘off-the-shelf’ analytics tools. Since most small and medium-sized game companies use a similar set of tools, the informants considered expertise and proficiency in utilizing these tools and ability to draw the relevant creative conclusions the best ways to extract value from game analytics.

There was a strong consensus among the informants that data and analytics general provide realistic and accurate insights about player behavior. All of the interviewees had experience from primarily quantitative analysis and metrics. The informants’ view was that game companies seldom utilize qualitative data, as it was considered more taxing, less effective, and harder to implement with third-party solutions. The informants also stated that interviewing players about their gameplay habits can lead to misleading results since people can seldom tell what they really want and would use.

All informants maintained that the results obtained from analytics are actively shared and communicated within the development team. At the same time, however, the interviews also implied that game companies rarely share their sales figures or metrics data with external parties. Due to the scarcity of reference data, the companies rely quite heavily on comparing the KPIs of new launches with historical data from their prior games. As a result, game analytics also assists in informing game publishers’ portfolio management.

A typical way to tackle the issues that are found during the development process is to create two or more different versions of the game. This procedure is referred to as A/B testing. The different versions of the game are randomly distributed to players and their respective performance is measured. A/B testing cannot be used extensively for every decision since developing each alternative consumes resources and the inferior versions are essentially a wasted effort. Moreover, obtaining relatively reliable results from the A/B tests requires a considerable amount time.

“When something new is added to the game, it is done in two different ways and half of the players get version A and the other half get version B and then the metrics are compared. There may be a hypothesis behind the test, but it is more about trying to find what works and then developing the better version further, leaving it as is or abandoning both.” (P1)

Interpreted through the ABV lens, with respect to game analytics as a decision support system, the direction of attention is top down as the focus is on finding answers to rather clearly formulated questions that often come from outside the game development team. In the first theme, game analytics as a sense-making device, the questions are open-ended and often emerge as a result of intuition and/or intellectual curiosity of the development team. As a result, in ABV terminology, with game analytics as a sense-making device the focus of attentional engagement (Ocasio 2011) is on the issues while with game analytics as a decision-support system the focus is on answers (Ocasio 1997).

4.3 Game Analytics as a Communication Tool

“[Analytics] is a kind of tool for studios to justify their decisions, for example why a certain game is not ready for launch yet, because we need to improve this metric. The investors are more willing to give extra time when they can see that in the long run the game will make more money if improved.” (P4)

As exemplified by the quotation above, the informants held that investors and publishers are very interested in the insights provided by game analytics. According to the interviewees, certain key performance indicators have to be reported frequently to investors and publishers. Furthermore, the negative trajectories in these metrics often lack of improvement in these metrics are very likely to raise draw publishers’ and investors’ attention and raise questions. Then again, designers use the figures provided by analytics to articulate their case in when for example there is a need to postpone the launch of a promising game that has fallen behind schedule.

“If I were a game publisher, I would ask teams to soft launch and provide retention and ARPDAU [average revenue per daily active users] numbers before I would invest anything.” (P3)

“Businesswise, you need to provide something concrete, some evidence that the things you do actually make sense. There are certain KPIs that for example venture capital investors are quite keen on. For instance, how many people continue playing after 1, 7 days and 14 days, and so on, tells a lot how well the game engages the player.” (P10)

Freemium games typically undergone very frequent scheduled updates. Hence, developers are very keen on knowing the impact of changes as soon as possible. Sizeable and potentially risky changes in the game are often tested on a small subgroup of players in order to avoid disturbing the regular service. Similarly, before the global launch, a new game can be ‘soft launched’, i.e. released in a smaller market (e.g., Canada or Finland). The aim of the soft launch is to ensure that the game will attract the target audience and has the potential to become profitable by collecting data about players’ behavior.

“We used to have a thing where once a week every studio [under that publisher] would report to the headquarters in California in an hour-long conference call and give a preformatted presentation in which the key metrics were analyzed, future plans to improve them laid out, etc.” (P4)

The immediate business benefits from employing game analytics include better informed financial decisions, such as budgeting of new product launches. The interviewees acknowledged that the history of video game industry contains a number of famous and very costly flops, where the projects have gone over-time and exceeded their budgets, resulting in significant financial losses. The interviewees also contemplated on whether some of these flops could have been fully avoided or at least mitigated by employing analytics rigorously. Furthermore, the informants acknowledged that game companies face pressure from diverse stakeholders, competitors, players, and publishers to adopt and employ game analytics.

“Retention is the most important metric in the game industry” (P2)

Among our informants, retention rate was unanimously regarded as the most important individual metric of commercial success in freemium game development. All the informants also indicated that retention rates are being monitored in their companies. Retention rate refers to the amount of time players keep playing a game for subsequent sessions. For example, the number of game-overs per player can be used as a metric to measure player retention. Retention rate is typically measured over a certain period of time such as seven days or a month. In addition to being a key metric to evaluate the monetization potential of games, retention rate can be used as the basis of funnel analysis. Furthermore, game developers use retention rate to enhance first impressions and the tutorials. Interestingly, yet there was a consensus on the importance and value-added of game analytics, the interviewees also described how increasing volumes of data make extracting relevant insights more challenging. Potentially due to this reason, the interviewees highlighted the need to keep the analytics process as simple and systematic as possible by e.g. focusing on certain KPIs and detecting their trends over time and over different versions.

“The most common wisdom in this free-to-play model is that if you don’t have retention you are never going to make money. Retention stems exactly from that the game itself has some interesting aspects and is in some way fun.” (P4)

Metrics focusing on monetization as well as retention-specific metrics such as average revenue per user (ARPU), conversion rates, tutorial funnels and day-1, day-7, day-14 and day-30 retention rates, were also mentioned as examples of metrics that most game developers monitor, or at least should monitor. Additionally, customer lifetime value is used to measure to what extent the costs related to customer acquisition are covered. Customer lifetime value is an aggregate of other metrics, namely cost-per-install and ARPU, and is used to guide marketing spending. The following quotation depicts the role of customer lifetime value:

“It is mostly based on things like ‘is it profitable to invest in marketing the game?’ LTV [Lifetime Value] will tell you that.” (P1)

Another commonly used and frequently mentioned analysis method was funnel analysis. Tutorials represent a typical example of a funnel. Throughout the tutorial phase gradually fewer and fewer players reach each subsequent step. Since the players have not necessarily paid anything for the game yet, it is pivotal that they do not churn out this early. Measuring the return rate or retention of players in subsequent steps (or levels) in the game was the most common way of analyzing the quality of the funnel. When the designers note a drop in retention, they know that there is a problem in that part of the funnel.

“When you do freemium games, the mechanics you can choose from are different than if you do only premium. If there is the front payment, people are more patient as otherwise it [buying the game] would be just lost money.” (P8)

4.4 Game Analytics as a Hygiene Factor

In addition to the three aggregate themes described above, the findings also strongly suggest that game analytics have other than functional purposes. Interestingly, the informants largely perceived game analytics as a necessity. In fact, operating without utilizing analytics was considered “flying blind” and analytics was seen as a means to reduce the risk of failures. The interviewees also repeatedly stated that the benefits were so clear that there had been no need to state an explicit business case to justify investments in game analytics.

“In my view, this data science thing, I mean using data and analytics and things like that…I feel it has become kind of. institutionalized” (P10)

Analytics was also considered as a means to signal a certain level of professionalism and organizational maturity as the following quotations exemplify:

“You know there are as many game companies as there are people in this business. Only part of the players really do this thing seriously. Analytics is kind of a way to show that you do your homework … that you are doing this thing with a professional mindset.” (P6)

“Typically, analytics comes on board more seriously when the company reaches a certain maturity point … when investors and publishers start to come on board. If you have no idea about basic player telemetry and monetization-related KPIs it shows that you either have some deep artistic motives or are just a plain amateur. … So, I would say that analytics helps you to speak a common language with them [investors and publishers].” (P8)

Small and medium-sized games companies have very limited resources to invest in analytics compared to larger players such as Supercell or King. This potentially explains why game analytics was seen as something that is important but not as a differentiating factor or a potential source of competitive advantage.

“If you would ask some of the big freemium houses about their analytics … well first of all they wouldn’t tell you much ... they have their own AI [artificial intelligence] running there to start with.” (P8)

Due to the resource constraints, small players cannot compete on analytics. One respondent even explicitly pointed out that other business models than freemium might allow a small game developer to be better positioned among the competition.

“In my view, if you’re a small player, if you do premium you have kind of more room for your own ideas and for the creative side … for small game companies this freemium thing … well … I think it’s really tough.” (P7)

Interpreted through the ABV lens, seeing game analytics as a hygiene factor demonstrates that the primary focus of attentional engagement lies outside reaping the business benefits but rather in engaging in communicative practices aimed at signaling professionalism and organizational maturity (Ocasio 2011; Ocasio et al. 2018). Perceiving game analytics as a necessity and/or a means to signal maturity might also reflect that analytics is becoming an institutionalized component of the games business. This may lead into what institutional theory (DiMaggio & Powell, 1983) describes as isomorphism, i.e. a situation where the majority of the players within an industry increasingly use the same tools and follow the same operational procedures. Table 3 below provides a summary of the four roles of analytics and their interpretation through the ABV lens.

Table 3 Roles of game analytics and their interpretation through the attention-based view (ABV)
Full size table

5 Discussion

5.1 Key Findings

This study was set out to examine how do small and medium-sized freemium game developers use game analytics. The findings imply that game analytics can play multiple roles for games companies. Besides the operational benefits obtained from improving the player experience, optimizing monetization as well as assisting in making data-informed product ramp-up and ramp-down decisions, game analytics has an important communication function in providing key stakeholders such as investors and publishers with the information they need as well as helping to find a common language between people focusing on the creative processes and those with more business-focused roles.

Retention rate was clearly considered the most important metric extracted from analytics. Retention was also systematically measured and used with A/B tests and gameplay funnel analyses. These approaches resemble the split testing presented by Andersen et al. (2011) to detect the most engaging design choices from a set of alternative versions. While the prior literature has advocated utilizing a combination of multiple data sources such as gameplay data, interviews, biometric data to obtain more insightful results (Gómez-Maureira et al. 2014), our results show that small and medium-sized freemium game developers tend to rely on basic analytics tools and a few key metrics such as retention rate. The lack of interest in qualitative data reported by our informants further highlights that when the resources allocated to the analysis processes are very limited, they must be very time-efficient and the data must be easy to interpret (cf. Xie et al. 2014).

The results further indicate that among small and medium-sized game developers, instead of being considered a potential source of competitive advantage, game analytics can help to achieve competitive parity (Barney 1991). This is essentially due to the limited resources allocated to analytics and due to the fact that small game developers typically rely on off-the-shelf third-party analytics tools. The results indicate on the one hand a broad interest in adopting more sophisticated analytical procedures, but on the other hand they also indicate a lack of resources to implement them. In particular, skills in data management were seen as a bottleneck for adopting advanced analytics. In this realm, it is hardly surprising that the role of analytics essentially relates to risk reduction.

5.2 Research Implications

Considering the growing economic significance of the games industry and the pervasiveness of the freemium model beyond, the results from this study illustrates the diverse roles of analytics in game development. Beyond the game industry context, the present study advances the understanding of the utilization of big data and analytics (Popovič et al. 2018; Gupta et al. 2018; Bharati and Chaudhury 2018). To this end, our study provides a theoretically-driven empirical account of the differences in the infusion of analytics in organizational practices (Karimikia et al. 2018).

Overall, our study portrays a different picture of game analytics than the prior research that has listed numerous opportunities and introduced sophisticated technological solutions for analyzing gameplay data (Bauckhage et al. 2012; El-Nasr et al. 2013). Our analysis also reveals two different approaches for collecting gameplay data. The first approach highlights that data cannot be collected retroactively and thus collecting as much data as possible ensures that “you have it when you need it.” The other approach emphasizes more selective data collection and is somewhat critical about collecting the so-called “vanity metrics” that do not contribute to the actual game design but may confuse the analysis process (see Canossa 2013a). Moreover, our study adds on the prior research on game analytics by demonstrating that at least for small and medium-sized game developers, the role of analytics goes beyond game development. To this end, our findings imply that analytics serves important communicative purposes as it helps to communicate with the key stakeholders and to signal professionalism and certain organizational maturity.

A combination of designer creativity and effective utilization of analytics is considered the winning formula as the potential competitive advantage from game analytics was viewed to emerge from synergistically utilizing the experience and creativity of both data analysts and designers. At the same time, however, tensions between the “creative side” and “business side” were a recurring theme in the data. Based on our analysis we interpret that there are two narratives among games industry professionals used to describe the nature and the essence of the business. The first narrative, primarily espoused by informants with creative design and development backgrounds, describes the games business as a creative industry and game development as a creative process. The second narrative emphasizes the role of analytics in driving business decisions and views game development predominantly as a process of implementing game mechanics and subsequently optimizing not only the software product but also the whole product portfolio for maximum profits. These tensions between the “creative side” and “business side” and the resulting two narratives must be acknowledged when evaluating the informants’ potential biases and preconceptions concerning the topic of the study.

Our study contributes to the development of ABV (Ocasio 2011; Ocasio et al. 2018; Vuori and Huy 2016) by providing a contextualized account of attentional engagement. Ocasio (2011) viewed attentional engagement as the process of intentional, sustained allocation of cognitive resources to guide problem solving, planning, sensemaking, and decision making and further highlighted the role of communication channels in allocating organizational attention. Our findings imply game analytics can be considered a communication channel as it not only serves problem solving, planning, sensemaking, and decision making (Ocasio 2011) but also has communicative functions and allocates within and beyond organizational boundaries.

5.3 Managerial Implications

Our results imply that risk reduction is an important operational benefit of game analytics that can be obtained with even very limited resources and small investments in analytics tools. In particular, using analytics during the development stage can decrease the risk of total failure in the launch stage. In addition, even the basic level of analytics with the standard key performance indicators can assist companies in, for example, terminating projects that are unlikely to generate sufficient revenue. In this respect, game analytics seems to be able to respond to the calls for insights that can be turned into concrete managerial actions made in the game analytics literature (Canossa and Cheong 2011). Our results add on this particular body of research by demonstrating that from the perspective of small and medium-sized freemium game developers even rather simple analytics processes can provide substantial business benefits.

Furthermore, compared to past reports on the game industry’s prior reluctance to adopt analytics (Mellon 2009), our results portray a profound change in the mindsets of professionals in the industry. Since tools for basic analytics are now available even for startup companies, the initial cost of implementing baseline game analytics is relatively low. For example, applying basic telemetry data analysis is one the most cost-efficient ways to do user research is and available even for one-person teams (Canossa 2013b).

Finally, our results demonstrate that for freemium game developers, game analytics provides important tools for communicating with investors and negotiating with publishers. An interesting observation was also that certain metrics have become close to industry standard. Key performance indicators that measure monetization and retention, such as ARPU, conversion rates, tutorial funnels and day-1, day-7, day-14 and day-30 retention and customer lifetime value were mentioned as examples of the key metrics expected by publishers and investors. These metrics are also used to evaluate and predict the financial success of the game.

Collectively, our findings suggest that analytics can be driver of new, data-driven business models (cf. Hartmann et al. 2016) in the game industry. While new product development has traditionally been the core business process in the game industry, the proliferation of analytics may challenge the dominant logic of the industry. For example, adopting the game mechanics and imitating the user interfaces of top-grossing benchmark games to cut the costs of new product development while focusing analytics to optimize the product portfolio could be a business model in which data and analytics plays a focal role. The emergence of data-driven business models in turn calls for increased understanding of how to form a strategic position in terms of utilization of big data and analytics (Wirén and Mäntymäki 2018).

5.4 Limitations and Future Research

As is the case with any empirical research, our results are subject to interpretation and are limited to the data available. At the same time, the limitations of this study provide avenues for future research. Against this background, we discuss the limitations of the present study in conjunction with potential areas for future research in terms of the empirical data used, the research methodology adopted, and the research context.

First, we focused on small and medium-sized freemium game developers located in Northern Europe. Had this study been conducted among large freemium games companies, the results would arguably have been considerably different. For example, resource constraints and the resulting imperative to focus on the key activities were frequently mentioned to inhibit adopting and employing advanced analytics particularly among the smaller end of the companies interviewed. As a result, future research examining how leading companies with global coverages such as Supercell and King utilize analytics would be highly insightful to make comparisons between small and large games companies, future studies and thus provide a more comprehensive picture of the field. Furthermore, the fact that several of the companies studied here had obtained venture capital is clearly visible in the data and in the way analytics was used to communicate with the current and prospective investors. Since games companies are also financed by e.g. banks, as well as governmental institutions, additional research addressing this limitation of the present study is needed.

Second, our reliance on interviews as the method to collect empirical data makes our results bound to the specific informants. Adopting a case study approach would have allowed us to more extensively utilize and report the voluminous additional empirical data accrued from our prolonged engagement with the field and phenomenon under investigation. However, presenting more detailed information about the companies would have ultimately compromised the anonymity of the informants. As a result, the role of our other empirical material and engagement with the companies was to obtain a strong preunderstanding of the phenomenon and to reinforce the credibility of our findings. Nevertheless, we recommend future research with e.g. a multiple-case study approach to investigate the use of game analytics at a company level. Moreover, we recommend future studies with a quantitively-oriented approach such as survey methodology to empirically test the findings of the present study.

Third, since there are a number of variations of the freemium model adopted by game companies, future research could focus on the specific analytical techniques and processes employed in games that employ different types of the freemium model. Furthermore, future studies could also extend the investigative locus beyond to games to other online services that employ the freemium model to identify best practices that could be applicable across industries.