Keywords

1 Introduction

Organizations continuously need to adapt to economic, technical and social changes that challenge established structures and its individuals alike [1]. Organizational development (OD) describes attempts of guided changes in organizations, which enable the personnel to cope with such environmental-based changes [2]. Typical goals are increases in performances, satisfaction or problem-solving [3]. Establishing organizational learning cultures is considered a constant contribution to OD and potential mechanism to achieve these goals [4].

Fig. 1.
figure 1

Success-factor model

Full size image

Applied games [5] are still relatively new in organizational development and organizational learning. The original idea of using electronic and digital games for purposes other than entertainment roots in the 70 s. The early 2000 s then mark the beginning of the new generation of technically and educationally advanced, and heavily diversifying forms of applied games. Gameful applications [6] mark but the latest branch. A prominent European example was the serious game “Airline Company”. The game transports the business model of Lufthansa AG to its employees. “Module X” is a gamification learning platform incorporating applied gaming and web-based trainings at Bayer AG.

Multisensory stimulation, experiences of continual improvement and perceived proficiency are but a few of the many attributes applied games potentially inherit from their entertainment counterparts [7], which can be utilized in OD. Most research focuses on positive characteristics of games [7], best practices [8], frameworks guiding a successful design [9, 10] and questions of evaluation end effect [11]. Literature on how to successfully set up and conduct applied games projects is comparatively scarce. Therefore the aim of this study was to explore success factors described by experts in the field and to condense those into a model for practitioners.

2 Methodology

This study is based on an exploratory, qualitative approach. First experts with experience in conducting applied game projects needed to be identified. The search radius was narrowed to persons with a recent history (approximately 2010) of 10+ projects within an organizational context in the DACH-region (Germany, Austria, and Switzerland) to reflect probable cultural specialties. The search criteria resulted in a total of nine experts between 30 and 61 years with a history of 10 to 50 projects within search radius. Four held an academic title (4S). Backgrounds ranged from an explicit vitae in game design and development (2S), media and arts (1S) over economics and informatics (2S), to pedagogy (1S) and management (3S).

To solidify and ground previous findings an extensive literature research for success factors was conducted as a second step. It included relevant databases such as ACM DL, IEEE Xplore, ScienceDirect or SpringerLink. A first iteration concentrated on literature on applied games, such as gamification, game-based learning and serious games developed for organizational contexts. As the results were not extensive, a second iteration was conducted, to see what could be learned from other recent professions that share attributes with applied games. Those were: Relatively novel and innovative to organizations; a potential to contribute to organizational challenges mentioned above; a focus on the design of experiences; respecting the end user’s needs. The selection resulted in the professions of user experience design [12] and design thinking [13].

Third a half-standardized interview questionnaire was developed based on the literature research and the underlying structure of a six-phase procedure model [14]. The questionnaire was assembled by all authors of the study.

Two types of interviews were conducted. The first series of seven interviews resulted in a wide range of statements that was processed according to the qualitative content analysis of Mayring [15]. The results were clustered distinctively to the phases of the process model as a horizontal structure. Next a type-structuring was conducted, clustering and counting similar expressions which resulted in a first iteration of the success-factors. A second series of four interviews, including two interviewees from the first series, aimed to revise and enhance the model. The interviewees were asked to comment on the success-factors found so far which resulted in some rearrangements, condensation and changes in wording of the factors.

Last the factors were classified along organizational abstraction layers, as defined below. The layers represent the second, vertical dimension concluding the two-dimensional success-factor model (Fig. 1).

Abbreviations. Throughout the further paper following abbreviations are used: “AG” = Applied Games; “SP” = Service Provider; User Experience = UX; Design Thinking = DT. A number followed by the letter “S” (e.g. 5S) highlights the collated number of subjects associated with a statement.

2.1 Two-Dimensional Framework

The horizontal dimension follows a six-phases procedure model [14]. It consists distinct and sequentially organized phases typical for applied game projects. For each phase input, throughput and output definitions are given. Iterations within each phase are strongly encouraged and common. Each end of a phase marks a possible exit of the procedure if results so far are not convincing or reasons demand ending the project. The idea of having closed phases while still iterating increases the plannability of an AG project. The model encourages a strict user-centered approach to AGs. It further aims at reducing known complexity regarding the solution and costs, with a close-to-100 % predictability at the end of the design phase. The vertical organization of the success factors follows three self-defined abstraction layers with an ascending level of influence by a SP.

First the organizational layer summarized factors, which are related to the culture, structure and external influences of an organization. The properties of these factors are rather consistent and hard to grasp, such as culture. Organizational factors exert an implicit influence on the AG project. Second the operational layer describes factors that frame the AG project, such as organizational goals, decrees, processes and workflows, teams and responsibilities. These factors follow mostly fixed rules and are usually well documented. Third the project layer refers directly to the AG-project and is about processes, human factors, methods, etc.

3 Literature Review

3.1 Applied Games

Michael & Chen [16] cite Quinn’s most important preconditions for a project: A specific need for training, the perception and believe that a game approach could help and financial resources. Prensky, among other things argues for an openness and acceptance of the younger generation’s play attitude towards serious contexts, such as work and learning [8]. Usually an organization needs to be convinced of a game-based solution and need to discuss an exit strategy for players who successfully finished a game [17]. Nicholson suggests to start a project with the desired outcome and a set of open questions to get there [18]. He and others also argue for a user-centered approach that addresses the target-user’s needs and goals [10]. Other factors are facilitating a common ground among stakeholders about audiovisual quality and costs [16]. Academics again stress the necessity but also complexity of an early planning and implementations of evaluations [16, 19, 20] as the overall evidence of successful applied games remains rare [7, 21]. There is much more literature on the actual design of games for learning [7, 16], but that is outside scope of this paper.

3.2 User Experience

Likewise applied games, convincing customers is an important first step [22] in UX-projects [12]. Both is often new to organizations. A transparent and clear communication throughout the project avoids mistrust and gains the crucial management support [23]. Scholz and Wallach propose a five-step process for user-centered design [24]. It includes a thorough scoping, analysis of users and products, iterative design, evaluation and delivery of the product. Abele et al. focus on a systemic approach and a strict user-focus and propose communication of successful projects within the organization to advertise and establish the approach [25].

3.3 Design Thinking

Design Thinking is more a philosophy than an exact process [13, 26]. It describes a shift in one’s attitude and thinking to approach a problem, likewise Schmidt et al. [14]. However, there are a number of rules, resources and methods such as ways for organizing information, working in small, mixed teams, well defined project scopes and a structured, time-bound and moderated process [13]. Flexibility, tangibility of ideas and prototypes and putting people first are other, very important constants of DT [13]. Others authors stress the importance of management commitment and clear communication to guide stakeholders through the probably unaccustomed way of thinking [26]. That includes the possibility of failure [27]. The idea of design thinking will feel less uncomfortable if an organization already established and innovative culture [28].

4 General Project Characteristics

All projects were declared innovation-projects and took place across industries (9S). Mid- to large-size organizations dominate the customers, presumably because of an earlier perception of the idea and larger personal and financial resources (9S). The experts did not disclose exact information on budgets but from the information given, ranges are between lower four-figure to mid six-figure amounts with a presumable typical range between 30.000–50.000 € and a project duration between three to six months. Interestingly, gamification budgets were rated 10-20 % cheaper in comparison to serious games projects (2S), probably because of smaller expected costs for audio-visual content. The most typical project-goal seems to be the increase of the overall performance and efficiency regarding learning goals. The designs therefore address motivation and behaviour of the learners to increase their engagement in the learning activity (8S). Most projects address younger people and are offered on top existing learning settings. Only rarely the organizations aim to replace a classic setting, such as seminars, completely. Six experts mentioned the additional political use of applied game projects, to support the company’s public image (6S).

5 Success Factors

5.1 Overall Success Factors

Organizational layer – A culture of Innovation and Learning (9S).

Similar to Prensky [8] for game-based learning and Hilbrecht and Kempkens for DT [28], all interviewees noted that a culture of innovation and learning in a customer’s organization is advantageous (9S) on several levels. There seems to be a greater openness towards new ideas and ways to tackle problems and present learning scenarios frequently integrate new media. Most importantly those organizations follow an established innovation process that provides guidance to plan and conduct the project.

Operational layer – Managerial Support (9S).

All interviewees and most literature stress the importance of managerial support of department and company heads to ensure the allocation of resources for the project (9S) [17, 26]. SPs must be aware that in a number of cases the contact person only explores the possibility of applied games and needs a business case before addressing superiors. Managerial support for AG projects cannot be taken for a fact within the organization and usually SPs are to earn and keep that support. Showstoppers, apart from missing managerial support, are a loss of interest in the topic or changes within organizational strategy and focus (3P).

Operational layer – Flexibility of Resources (4S).

Fixed budgets are quite common with smaller organizations (2S) while bigger companies tend to allow flexibility (2S). There is a preference towards flexible budgets and other resources among the SPs while especially enterprises naturally prefer predictability. Two SPs (2S) try to negotiate each phase of the project separately and sequentially as suggested by Schmidt et al. [14]. Such practice may also foster the trust-building process but will sometimes be prohibited by organizational policy.

Operational layer – Transparency and Trust (5).

Building and keeping up trust towards the idea of AGs and a SPs ability is indispensable across hierarchies. AGs are usually cooperative projects and therefore need a common sense among stakeholders, developed by the SPs. The overall project flow and next steps should be transparent at all times and invite customers to take part. Similar suggestions are found in [26, 29].

Operational layer – Moral and Ethics (2S).

AGs are often criticized because of their manipulative nature [30]. Despite only explicitly mentioned by two interviewees, it was decided to include this factor as everyone working with persuasive designs should know about its advantages and downsides and therefore develop a clear position regarding project goals and morality.

5.2 Phase 1: Exploration

Both partners aim for a common understanding of the problem, project goals and success criteria. They discuss the motivation and chances for AGs as a possible solution to the problem and agree on milestones and financial parameters. A vision statement is a recommended outcome of this phase and foundation to a contract [14].

Organizational layer – Communication and Persuasion (5S).

Quite often SPs must argue for AGs as a possible solution for learning and change in organizations. Interviewees stressed the sensibility of the word “game” and suggested to use other terms, such as motivational design or else (3S). Games and play are strongly connected with leisure time and work is attributed with seriousness and effectiveness in Germany (and elsewhere) (2S). Citing positive and competitor examples as well as stressing the idea of pleasure being a good precursor of job satisfaction and work performance, are common among SPs to argue for AGs (e.g. Schubert et al. [22]). However, some inevitable uncertainties about conventional problem solutions and cost pressure will naturally support the SPs argumentation (2S).

Operational layer – Consulting and Expectation Management (4S/6S).

Consultancy is needed regarding chances, challenges and risks of a project and its coherence to company culture (4S) [28]. Moreover, most interviewees notice misleading thoughts about the complexity of game design and the expectable audiovisual quality of a product in the light of the budget ranges outlined above (6S). At last, two interviewees also discuss the necessity of an iterative process that accepts unconventional ideas and failing of ideas, likewise design thinking [13, 27] (2S).

Project layer – Definition of Scope and Goals (5S).

Most interviewees discussed the relative vagueness of goals in the early stages of a project. The definition of distinct and measurable goals [16] within a specific project scope is accordingly part of the consulting and expectation management and will often differ from original ideas. For example, the project lifetime has an important influence on the design of AGs (1S). Then again, goals need to be readjusted when new information is available (5S). Specifically referring to gamification, two SPs do not believe in cross-departmental projects (2S). Overall interviewees report a certain realism regarding goals and expectations towards behavior change and performance goals.

5.3 Phase 2: Analysis

A thorough context and target user-group analysis is conducted using qualitative and sometimes quantitative methods. The overall goal is to outline multiple perspectives from different stakeholders towards the problem and current solution and document the results with suitable artifacts such as user-journeys, use-case descriptions or technical specifications [14].

Project layer – Accurate User and Case-Analysis (9S).

All SPs ensured to follow a user-centered approach. However, in most cases budgets and logistics do not allow for a thorough user-research process. Instead domain experts such as department managers provide contextual facts and information about users (9S). One interviewee explicitly highlighted the risk of inaccuracy of second-hand information for the success of the project (1S). Another mentioned the importance of choosing efficient methods, such as contextual inquiry and job shadowing, if given the chance (1S). First hand context and user information is a key factor in user-centered design and design thinking, too [13, 24, 25]. Standardized quantitative methods on the other hand appear not existent as these are thought to be less efficient.

5.4 Phase 3: Ideation (Idea + Creation)

The ideation uses the information of exploration and analysis to review and probably adjust the projects goals. A creative team incorporating multiple professions and domain experts aims to create a number of gameful solutions to the problem in workshop and creative sessions. Ideas are mixed, further elaborated and documented in a structured and comparable format, such as a canvas or exposé sheet, including visuals and first hand low-fi prototypes [14].

Organizational layer – Understanding of Culture and Law (4S).

SPs need a thorough understanding of culture and law, both rural and organizational wise and a sensibility for the context and codes of conduct (4S). For example, in Germany strict data security rules mostly forbid ideas with a competitive nature. The following, non-extensive list of legal-fields is relevant to AGs in enterprises: (i) The Right to informational self-determination concerning the survey and use of personal data and questions of data ownership; (ii) Workplace-related rights usually enforced by employee-organizations and unions, such as rights of co-determination; (iii) IT-regulations and employee-agreements on ownership and use of outcomes conducted with a software; and (iv) Usage agreements and contractual rights between organization and SP.

Operational layer – Incorporation and Competence (4S).

Besides finding a common ground like discussed above, SPs seek to raise competence levels regarding AGs on the customer side (4S). They do so to incorporate and partner up with the customer during the following project phases for purposes of easier mutual agreements on decisions and their consequences, likewise suggested by Wilke et al. [29]. Interviewees preferred to have at least one domain expert on their team who also may represent the target group. Workshops during which both parties ideate together are common. Some SPs are more cautious about incorporating customers, especially during the creativity processes, for effectiveness reasons (2S). DT likewise [13] argues for variety on the team and domain experts are requested in user experience projects [25].

Project layer – Creativity and Tools (5S).

Interviewees suggested workshops as a primary format to iterate and ideate. These follow common rules for creativity sessions, such as no early critic, time-boxing and early visualization (5S). However, crazy ideas too early in the process could create doubts within conservative organizations. Warm ups exercises and discussing personal game experiences may help to start the activity and build a gameful attitude, which is important. (1S) [8, 13, 14].

5.5 Phase 4: Design

The design phase starts with a decision for one or more ideation results. The ideas are further elaborated and tested in a number of prototypes to test function, effectiveness, affective reactions of users and the overall experience they provide with respect to the projects goals. This phase includes feedback with target users and aims for a fully documented design concept, prototypes, content plan and technical specification that lays ground for a viable production planning [14].

Operational layer – Incorporation of Specialists (5S).

When going further with one or more selected ideas, specialists should be added to the design team. On an administrative level the conformity of an idea to rules and regulations is required (employee organizations, stakeholders from IT, compliance and legal) (5S). Domain experts and pedagogical experts consult the team on a content level (4S).

Project layer – Iteration and Prototyping (4S).

Likewise ideation the team should plan for a number of iterations during the design phase. Early low-fi and later hi-fi prototypes ensure the communication of functionality and emotionality of an idea (4S). It is important to regularly review the vision and goals of the project and respect common guidelines for usability and accessibility (1S). Other than suggested by literature, but coherent with the critic of phase two, target-users are only introduced late or not at all until the design is fixed (2S) [13, 17, 18, 25].

5.6 Phase 5: Production

The production is separated from the former creation phases. It is a predictive phase in terms of time and necessary resources where the product is actually implemented. Parallel to production the product’s launch is being prepared [14].

Organizational layer – Communication and Advertisement (6S).

Starting early to advertise the project across the target group and organization is a crucial factor for its later success [25] (4S), but regularly neglected from the experience of the SPs (2S). It is suggested to address management and set up a core communication team. Despite the importance of this aspect SPs usually are not incorporated (2S).

Project layer – Feasibility and Extensibility (3S).

Overall the interviewees did not consider technical feasibility as a problem or risk (3S). A careful decision for a specific technology is of course important (2S). Requests for mobile or cross-platform development increase (1S). Three SPs stressed the need for a solid IT-architecture that allows for easy extension of features and contents after launch (3S).

5.7 Phase 6: Go-Live and Maintenance

The launch of the product concludes the project and should be accompanied by strong communication within the organization. An evaluation is conducted to control and review goals the contractors agreed on during project exploration and ideation. Probably maintenance and update conditions are negotiated [14].

Operational layer – Communication and Lead by Example (4S).

Most AG projects are an additional offer above standard procedures, such as e-learning or seminars. When going live, department managers and core users are important multiplicators for project advertisement and role models (4S).

Project layer – Test and Review (5S).

The majority of interviewees report tests of direct measurable data only, such as attendance or test results (5S). Other measures on soft factors, such as motivation or satisfaction are usually not applied. Academic methods, especially quantitative ones, can become costly. Organizations rather rely on informal indicators such as sensed satisfaction, which was criticized (3S).

Project layer – Update and Maintenance (4S).

A periodically data collection and processing is common in the games world. The information suggests improvements and updates necessary to keep a product alive. The same would count for an AG that also wears of over time, like all technical products. Because of the nature of innovative projects unfortunately no budget is planned for maintaining the product (4S).

6 Discussion

The success-factor model reflects first hand practical experiences of conducting an AG project across a wide range of organizations and budgets. The findings show that project goals mostly seem concerned with organizational learning, probably followed by product marketing and issues of human resources [17]. However, AGs are far from established yet in the DACH-region and SPs mostly offer such services complementary to others, such as UX or e-learning. All projects were declared as innovation projects and show some typical characteristics accordingly [29].

Managerial support, communications and clear goal definitions are preconditions to a project in general. But SPs for applied games should take transparency and communications even more serious to gain trust and understanding towards the process and the only gradually revealing outcome of an AG project. It is suggested to educate customers towards the unfamiliar game-perspective and to incorporate them in creation and decision making. Times and budgets are relatively small and often do not allow for a thorough analysis on case and users. Without, it is difficult to design meaningful experiences and motivate on a deeper psychological level which fosters shallow designs as rightfully criticized [30]. Applied Games tend to process user data. Legal issues and company specific rules and culture therefore can have serious implications and might interfere with design ideas. Overall most of the qualitative factors presented could be backed up with literature, supporting the notion of a high-level comparability of UX, DT, and AG projects as presumed. However, the complexity of AG-projects suggests experience to be an additional important success-factor.

Since this is an exploratory study some limitations need to be addressed. First this kind of study is not able to and was not planned to infer causality. Second the number of expert interviewees (n = 9) is relatively small. This can partly be explained by the relative novelty of applied games and consequently a rare number of projects and partly by the narrow search radius of highly experienced experts. Therefore, we are confident to have identified a relevant sample for the DACH-region. Clearly, an international search would yield more representative results. Third no objective success criteria were mentioned as these differ widely across the projects. Standardized, practicable methods are not available apart from individual and indirect indicators, such as attendance. Fourth interviewees of the second series generally agreed on the two-dimensional framework but mentioned individual differences in the definition of the ideation and design phases.

AG projects remain a promising endeavor to organizational development, potentially improving the attractiveness and effectiveness of applications and processes of organizational learning, guided changes and enterprise software. Expectations regarding their quality and power to foster motivation and create meaningful experiences and outcome [7, 18] are high. When conducting such projects, much can already be learned from experts and other disciplines as referenced here. But budgets and circumstances need further adaption in some projects to actually transfer game-like experiences to the enterprise context. To contribute the further success of AGs, the results of this explorative research need further empirical and practical validation. A prioritization of the validated factors then eventually would allow for the design of a maturity model that helps enterprises and service providers to identify risks and areas of development of future projects. Cross-country studies would further help to differentiate general from rural and cultural depended aspects. Outside scope, this study suggests further research on design practices and standardized evaluation methods [11, 20] as well as on reasons and counterstrategies of weary effects.