Gamification of Electronic Negotiation Training: Effects on Motivation, Behaviour and Learning

Abstract

Organisations are involved in various types of negotiation. As digitalisation advances, such business negotiations are to a large extent electronic negotiations. Consequently, dedicated training for such electronic negotiations is important for mastering negotiation skills. We designed a gamified negotiation system used in e-negotiation training to increase participants’ motivation, engagement, use of the system’s negotiation support features and to improve their decision making. The quantitative evaluation using students as subjects shows higher motivation, engagement and better system and decision-making skills for participants in the gamified training compared to a conventional training. Furthermore, female participants show higher engagement in the gamified training than males. An analysis of the individual elements in the system provides insights into participants’ perceptions and shows that the inclusion of a domain-specific feedback element yields motivational results that are almost similar compared to those using traditional game elements. Organisations can employ the designed artefact for fundamental and effective e-negotiation training.

1 Introduction

Communication processes in business organisations have become increasingly digitalised shaping various forms of social interactions. Negotiations as one important form of business interaction including communication and decision-making, are nowadays conducted electronically via asynchronous media such as email or negotiation systems (Schoop et al. 2008). The required skills for electronic negotiations (e-negotiations) can be obtained by participating in dedicated negotiation training, involving negotiation theory and practical tasks. In e-negotiation training, participants engage in realistic negotiation simulations and use web-based negotiation support systems (NSSs) that provide several features to support communication and decision-making tasks (Köszegi and Kersten 2003; Melzer and Schoop 2016; Schoop 2020; Vetschera et al. 2006). However, despite the participation in e-negotiation training, the negotiators still settle on inefficient agreements (Gettinger et al. 2016) and the features of NSSs are not always used to their fullest extent (Druckman et al. 2012). Potential reasons for the observed problems are the participants’ lack of motivation to engage deeply with the practical negotiation tasks and the NSS, and a lack of feedback in current forms of negotiation training (Schmid and Schoop 2019).

In recent years, the need to sustain learners’ attention, facilitate their motivation and support their learning has been recognised as a major challenge in education (Dichev and Dicheva 2017). Game-based approaches have become popular to improve students learning due to their motivational power and feedback provided. In the area of negotiation training, game-based approaches have been introduced by means of agents as virtual characters (Gratch et al. 2016; Kim et al. 2009), by means of virtual reality training (Ding et al. 2020), or by means of full-fledged games such as Merchants or Reign of Aquaria. These approaches target a face-to-face negotiation training and simulate real-time interactions with the negotiation partner, whereas e-negotiations conducted via asynchronous media do not use virtual characters but involve the (written) exchange of formal offers and longer text messages.

Another promising approach to enhance learners’ motivation is gamification, defined as the use of game design elements in non-game contexts (Deterding et al. 2011). When gamification is applied to an information system, all of the system’s functionalities are retained and game elements are additionally incorporated (Liu et al. 2017). To date, the results of gamified learning interventions on learners’ motivation, engagement and cognitive outcomes are mixed but predominantly positive (Dichev and Dicheva 2017; Sailer and Homner 2020). Gamification also seems suitable to improve social and practical skills such as problem-solving, decision-making, communication and collaboration (Putz et al. 2020). These skills and tasks are particularly important for e-negotiations. Since NSSs are often used for e-negotiation training (Vetschera et al. 2006), gamifying of an NSS is a promising solution for the afore-mentioned problems. It has been shown that e-negotiations have several inherent game-like elements such as feedback scores (i.e. utility values) and negotiations of varying difficulty (Schmid and Schoop 2018), and, therefore, provide an interesting basis for adding game design elements.

Our research goal is to design a gamified NSS to be used in e-negotiation training to improve participants’ motivation, engagement and learning outcomes. A first evaluation demonstrated the effectiveness of the designed artefact in enhancing motivation and engagement; however, learning outcomes could not be improved (Schmid et al. 2020). The current paper reports the first large-scale quantitative evaluation of the revised artefact and also investigates effects beyond the training phase. In particular, we have chosen students as subjects (for reasons to be discussed later) to answer the following research questions:

• 1. Which effect does a gamified e-negotiation training have on participants’ motivation?

• 2. Which effect does a gamified e-negotiation training have on participants’ engagement?

• 3. Which effect does a gamified e-negotiation training have on participants’ learning outcomes?

• 4. What is the participants’ perception of the integrated elements regarding their support for motivation and learning?

• 5. Which effect does a gamified e-negotiation training have on the participants’ use of an NSS in a follow-up negotiation?

234 students from three universities participated in this study. We compare our gamified e-negotiation training with an established, conventional e-negotiation training (Melzer and Schoop 2016). The central theories and concepts of motivation, gamification and e-negotiation training are reviewed in the following section. The designed system (Sect. 3) and the research design (Sect. 4) are introduced, followed by the results of the evaluation in Sect. 5. Finally, we discuss our results, limitations and contributions to practice and provide an outlook on future research directions.

2 Theoretical Background

The current work integrates theories and concepts of motivation, feedback, gamified education and training and negotiation research, which will now be described.

2.1 Motivation and Feedback

Motivation can be intrinsic (i.e. performing an activity for its inherent satisfaction) or extrinsic (i.e. performing an activity for a separable outcome or to avoid negative consequences) (Ryan and Deci 2000a). In education, intrinsic motivation is the desirable motivation and results in high-quality learning (Ryan and Deci 2000a). According to self-determination theory (SDT)—a macro-theory of motivation—intrinsic motivation flourishes once an individual’s basic psychological needs for autonomy, competence and social relatedness are fulfilled (Ryan and Deci 2000b). Autonomy is defined as the extent to which an individual perceives an action as self-determined. Furthermore, individuals master an activity and thereby feel competent. Finally, individuals require a secure social basis and/or need to feel connected with others (Ryan and Deci 2000b).

Individuals are further motivated to strive towards goals (Locke and Latham 2002). Feedback is an essential part of learning, as it helps individuals to pursue their goals, informs them about their progress and shows which steps to perform next (Hattie and Timperley 2007). The way feedback is presented and formulated has direct effects on the motivation of an individual. Feedback perceived as controlling undermines autonomy, whereas feedback perceived as informational positively affects autonomy and intrinsic motivation (Deci et al. 1999). Furthermore, positive and constructive feedback and challenges that are feasible for the individuals facilitate their feelings of competence and in turn increase intrinsic motivation (Ryan and Deci 2000b).

2.2 Gamification in Education and Training

Deterding et al. (2011) define gamification as the use of game design elements in non-game contexts, while not transforming the non-game context into a full-fledged game. In addition, other researchers explicitly highlight the goals of gamification. Landers (2014) defines gamification of education as a means to facilitate learning and related outcomes. Gamification studies report predominantly positive effects on the motivation, engagement and cognitive outcomes of learners (Dichev and Dicheva 2017; Sailer and Homner 2020). Research is often based on SDT (Ryan and Deci 2000b) to explain the effects of gamification (Tyack and Mekler 2020; Xi and Hamari 2019), and therefore gamified interventions focus on integrating game elements that are expected to fulfil a learner’s basic psychological needs. Still, the results of a gamified intervention depend on the context and the perception of its users (Hamari et al. 2014) and further depend on the combination of and interaction between the game elements (Dicheva et al. 2019; Liu et al. 2017).

The game elements used in gamification can be categorised according to their abstraction level into components, mechanics and dynamics (Werbach and Hunter 2012). The most abstract form are dynamics, representing the overall objectives of the gamified intervention such as creating emotions or progression. Mechanics represent the means to realise these objectives, e.g. in the form of competitions, challenges, rewards or by providing feedback. The most concrete form are the components, which contribute to the mechanics. Examples of components are avatars, badges, levels, and quests (Werbach and Hunter 2012). In the education domain, points, quests, badges, rankings, and levels are frequently used (Majuri et al. 2018). Such elements provide clear goals to work for and provide feedback about the progression towards these goals (Mekler et al. 2017). However, the frequent use of points, badges, and rankings has been criticised because these components work as extrinsic motivators by inducing and rewarding an activity (Liu et al. 2017; van Roy and Zaman 2017). Crucially, it depends on whether an individual perceives a reward as controlling, which undermines autonomy and intrinsic motivation, or as informational feedback towards the achievement of a goal (Deci et al. 1999).

The context-dependent effects of gamification (Hamari et al. 2014) have been demonstrated in a number of studies. Using badges and rankings led to less motivation and lower exam scores in Hanus and Fox (2015). Sailer et al. (2017) found that badges, rankings, and performance graphs positively affect participants’ autonomy and competence need satisfaction, which facilitate intrinsic motivation. A study by Xi and Hamari (2019) showed that badges, points, status bars, and rankings positively affect all three basic psychological needs.

Gamified learning interventions should implement game elements that set challenging but attainable goals and satisfy all three basic psychological needs in order to engage the learners (van Roy and Zaman 2017) and facilitate intrinsic motivation in the long run (Sailer and Homner 2020). To choose appropriate game elements, an analysis of the application context is necessary (Morschheuser et al. 2018).

2.3 Electronic Negotiation Training

A negotiation is conducted by at least two negotiation parties dealing with interdependent tasks, who continually engage in communication and decision-making tasks to search for a consensus (Bichler et al. 2003). For several years, business negotiations have been conducted electronically, especially using asynchronous media such as e-mail (Schoop et al. 2008). Electronic negotiations are conducted with the primary objective of saving transaction costs, finding agreements in less time, and reaching agreements of higher quality (Bichler et al. 2003). Negotiators must not only possess the required negotiation skills and the required ICT skills but also specific digital negotiation skills for conducting electronic negotiations. For example, communication in electronic negotiations differs from communication in face-to-face negotiations due to missing cues such as mimics, gestures, and tone of voice. Instead, electronic negotiation communication must convey semantics by different patterns through communication quality (Schoop et al. 2010; Schoop 2021), e.g. ensuring grounding and coherence. These skills require extensive training.

Dedicated negotiation training is offered in organisations such as companies and universities to facilitate the development of the relevant communication and decision-making skills. Negotiations require individuals to claim their own positions behaving competitively as well as acting collaboratively in search of win–win agreements (Lewicki et al. 2010). As discussed above, e-negotiations require dedicated negotiation systems to improve communication and decision making (Köszegi and Kersten 2003; Schoop 2010). The asynchronous mode of e-negotiations allows for more preparation time to define and implement negotiation strategies and tactics. E-negotiation training facilitates the development of these skills and often uses negotiation support systems (NSSs) such as Inspire (Köszegi and Kersten 2003; Vetschera et al. 2006) or Negoisst (Melzer and Schoop 2016).

NSSs provide various types of support for communication and decision-making whilst leaving the final decision with the negotiation party (Schoop et al. 2003). A bilateral e-negotiation process may look as follows: In the preparation phase, both parties separately define the preferred values for each of the negotiation issues as well as a ranking of these issues according to their importance.

One party initiates the negotiation and prepares a first message, which looks similar to an email. NSSs such as Negoisst require the selection of a message type such as a request, offer, or counteroffer to explicate the sender’s intention (Schoop 2010, 2020). Some NSSs also offer informal message exchange through means of questions and clarifications (Schoop 2010). Once a message type and consequently the mode of the message is determined, the sender’s communication and decision-making skills are required to select the preferred values for the negotiation issues, provide reasonable arguments for their selection, and establish a relationship with the negotiation partner (Schoop et al. 2010). When the first message was sent, the recipient can evaluate the received request using a utility value computed by the NSS ranging between 0 and 100%. This value represents the extent to which the received request or the own offer under construction corresponds to the negotiator’s preferences and helps to assess appropriate concessions for one’s own next step (Schoop 2010, 2020).

The message composition is exemplified in Fig. 1, where the user is sending a reply to a negotiation partner with the fictitious role name “Alexander Iwanow”. The user has chosen a counteroffer as the message type and selected the values for the three negotiation issues about the printing ink procurement in the agenda on the right. The currently selected values result in a utility value of 86% for the user. In addition, the user writes a textual message in natural language to present own arguments to the negotiation partner, who has sent a counteroffer that yields a utility value of 25% for the current user. The negotiation ends once a party agrees to accept a received offer or once a party decides to finally reject the offers and thus to end the negotiation without a deal.

Fig. 1

Writing a message in negoisst

Consequently, an e-negotiation training requires (1) an end-user training for the NSS to be used and (2) a training for the development of e-negotiation skills (Melzer and Schoop 2016). Negotiation training usually follows the experiential learning methodology by Kolb (1984). As part of an e-negotiation training, participants engage hands-on in bilateral negotiation simulations either with another participant or with a software agent as part of the experience (Köszegi and Kersten 2003; Melzer et al. 2012). For actual learning to take place, participants need feedback to reflect on their experience and draw conclusions for future behaviour. A recent study shows that participants want to receive particular feedback to improve their negotiation skills such as preparedness, effectiveness, rationality, strategy and problems-solving (Meyer et al. 2020). Current e-negotiations training supports negotiators with their individual reflections solely offline through debriefings or in-class discussions (Köszegi and Kersten 2003; Melzer and Schoop 2016). However, the use of gamification and of gamified feedback elements integrated in an NSS might facilitate the reflection phase and could improve participants’ motivation, engagement, and negotiation skills (Schmid and Schoop 2019). In the following chapter, we will present our system design for a new e-negotiation training.

3 System Design

Improving e-negotiation training with game design and feedback elements requires choosing an existing NSS used in e-negotiation training and integrate these elements into the NSS. The NSS chosen for this study is Negoisst (Schoop 2010, 2020) due to the following reasons. Negoisst is a research prototype that can be used to conduct business negotiations, has been used for almost two decades to train future negotiators, has been used to conduct international negotiation experiments, and is one of the most comprehensive NSSs including communication and decision support as well as document management and conflict management (Schoop 2010, 2020). We will use the negotiation process as described in Sect. 2.3 and present the newly added feedback and gamification components briefly and justify their choice.

Based on previously derived requirements (Schmid and Schoop 2019), our system design is in line with the method for gameful design by Deterding (2015) stating that game elements are centred around the inherent challenge of a user’s pursued action. This inherent challenge in the context of e-negotiation training is to reach a good agreement with the negotiation partner. Its difficulty depends to a large extent on the complexity of the negotiation and on the behaviour of the negotiation partner (Lewicki et al. 2010). Various tactical and strategic actions can be performed to finally establish an agreement.

Our overall learning goal is to engage participants in continuous learning through participation in realistic bilateral e-negotiation simulations. Dichev and Dicheva (2017) emphasize the necessity for safe learning places, in which participants can gain experience without fearing negative consequences. Therefore, all tasks (in this case all e-negotiation simulation tasks) can be repeated during the training phase. During the training negotiation, the participant requires feedback for the performed actions and will need additional feedback once the agreement was settled, i.e. whether a distributive, integrative and/or fair agreement was achieved. The latter feedback could serve as an incentive to repeat a negotiation simulation, to experiment with different negotiation strategies and to improve logrolling behaviour (Schmid and Schoop 2019). The provision of feedback is essential as it serves as a trigger for the reflection phase in the experiential learning methodology. We expect increased hands-on experience and feedback on negotiation performance to facilitate the development of important e-negotiation skills such as preparedness, rationality, and strategic behaviour (Lewicki et al. 2010).

To avoid overburdening demands and tailor the e-negotiations to the current skills of the participants we chose the mechanics of increasing challenges. These are realised through the implementation of five levels, each corresponding to one bilateral negotiation simulation that the user needs to complete successfully in order to unlock the next more difficult level (see Fig. 2). The levels are connected through a continuous story, with the participant being the responsible negotiator for a procurement department. In the first level, participants face a simple single-issue negotiation about the price of a product and learn to exchange messages in the system. In level two, the decision support by means of the utility value and a visualisation of one’s preferences is introduced and helps the participants in their first multi-issue negotiation. Level three includes the history graph as additional visualisation supporting decision-making and an enhanced communication support feature called semantic enrichment (Schoop 2010). Level four is designed as a complex negotiation with several issues and level five represents a competitive negotiation. The features in the first three levels are presented using a guided tour before participants start negotiating the corresponding case allowing them to apply these features. The increasing challenges and their implementation using levels is, therefore, realised by making the negotiation simulations and the NSS more complex. Furthermore, levels provide the learners with clear goals to work for and visualise the progress towards these goals (Mekler et al. 2017).

Fig. 2

Level overview with levels 1 to 3

Negotiation tasks are always interdependent and a negotiator’s behaviour depends on the negotiation partner (Bichler et al. 2003). In order to keep the difficulty in the levels consistent and to train communication behaviour, the participants negotiate with an automated software agent called Tactical Negotiation Trainer (TNT) (Melzer et al. 2012). Based on predefined preferences and a strategy, the TNT creates a new counteroffer corresponding to its concession strategy and generates a matching text message presenting its arguments and requests using a sentence recommender. Since the TNT replies within a few seconds, immediate feedback is provided to the human participant on whether the negotiation behaviour, i.e. strategies and tactics, turn out to be successful or not. In all levels, the preferences of the human negotiator are already defined and cannot be modified. Therefore, once the trainee is familiar with the case and the given preferences, they initiate the negotiation and send a first message.

When an agreement has been found, utility rankings provide feedback. Three rankings display the individual performance as well as the sum of the negotiators’ individual performances (i.e. the joint utility) and the contract imbalance of the agreement. Rankings provide informational feedback (Mekler et al. 2017); however, this feedback is relative and depends on the performances of others. A top position in the ranking does not necessarily mean that an excellent agreement has been found; there might still be potential for improvement on both negotiation sides. Whilst the joint utility ranking over all participants shown in Fig. 3 suggests that the maximum joint utility that was reached is 121.50%, this value is not necessarily the highest possible utility; joint utility could thus be further maximised.

Fig. 3

Anonymised joint utility ranking

Therefore, a classic feedback visualisation from the negotiation literature was added in this study, namely the Pareto graph (Tripp and Sondak 1992). In contrast to the Pocket Negotiator by Jonker et al. (2017), that displays the graph during the negotiation based on the assumed preferences of the negotiation partner, our graph is displayed after an agreement has been found. The graph has two axes which display the real utilities of partner 1 and of partner 2 (both represented by their role names in the negotiation and not by their real names), each ranging between 0 and 100 (see Fig. 4). It displays all Pareto-optimal agreements as small red points indicating the Pareto frontier; the settled agreement is displayed as a big blue dot. Consequently, the user can clearly see how close the settled agreement is to the Pareto-optimal agreements. In contrast to the utility rankings, the graph offers absolute feedback about one’s performance. Although the graph is not a typical game design element at the component level in terms of the game element hierarchy by Werbach and Hunter (2012) and certainly domain-specific, the provision of absolute feedback at the level of mechanics comparing a user’s performance to an absolute standard (in this case: the Pareto frontier) is a common game element (Burgers et al. 2015). A discrepancy between the settled agreement and the Pareto-optimal agreements can be expected to motivate users to repeat the negotiation simulation and to find a better agreement.

Fig. 4

Pareto graph display

Based on the individual utility in the utility rankings, participants are provided with information about how well they are currently performing during their ongoing negotiation. To this end, the current individual utility is compared to the utility values of all other participants that are currently negotiating or have already reached an agreement. A small textual information was added which we call process feedback, that gives the participants the information that they are currently among the top 10%, the top 25%, the upper half or the lower half of all negotiators in this level. The process feedback is immediate and is refreshed each time a new message is received.

Last, two reward components are present to induce positive emotions and to lead to intensified system use. Experience points serve as an immediate feedback for performed actions (Sailer et al. 2013). Several actions are rewarded, e.g. sending messages, viewing one’s preferences during the negotiation, or using other system features. Users can compare their obtained experience points in a ranking. Furthermore, a page displaying the recently earned experience points is available to ensure transparency about the point rewarding mechanism. The second implemented reward mechanism are badges (see Fig. 5). A separate badge page lists all 25 badges and their corresponding unlock instructions since badges should have a clear goal setting function to be motivating (Hamari 2017). On the one hand, the badges award desirable and intensified system use, e.g. the “Process Analyser (Bronze)” for analysing the history graph (Schoop 2010) for the first time. On the other hand, the badges also include more difficult goals to work for, such as “The Maximiser” for finding an agreement which is very close to the maximum joint utility that can be achieved in the negotiation. Users receive a notification on their screen once they unlock a new badge. Rewards are considered to be effective for short-term or intermittent system use facilitating extrinsic motivation (Liu et al. 2017). In our context, we expect them to increase in-depth negotiation training and desirable system use.

Fig. 5

Badge page showing all unlocked badges

The gamified Negoisst provides several components for the participants with goals to work for; most of them prominently displayed on the home screen (see Fig. 6). All components show either immediate or delayed feedback. Whilst the rewards and rankings may be perceived as extrinsically motivating components first, intrinsic motivation might also be facilitated on the long run: A participant has the freedom to define the negotiation strategy for each level and can repeat a level, e.g. when the negotiation was unsuccessful or the outcome turned out to be inefficient. Freedom of strategic choices, feedback components and the option to repeat levels reduces thoughts about negative consequences and can incentivise participants to experiment with different negotiation approaches. Participants’ autonomy is further facilitated through the freedom to decide which goals they would like to pursue (e.g. which badge to unlock, which position in the ranking to strive for).

Fig. 6

Home screen of the gamified negoisst system

4 Research Design

To answer the research questions and to evaluate our artefact, we performed a quasi-experimental study in November 2019.

4.1 Participants and Setting

The evaluation of the training was conducted involving 234 students from four universities in Austria, the Netherlands and Germany, 218 of whom were graduate students. Each university offered a negotiation course for their students participating in the study. The courses taught negotiation theory and practice. 81 students participated in course 1 (Germany), 24 students participated in course 2 (Austria), 54 students participated in course 3 (Austria) and 75 students participated in course 4 (Netherlands). 221 of the participants studied management, business administration, information systems, or business communication and digital media. All students gave their consent before participating in the study. Regardless of their training performance, the students received credit points for the participation in the training and for completing all online surveys, which are described in Sect. 4.2.

As part of the negotiation lecture course at each university, the students participated in an international negotiation simulation. To prepare, all students took part in an e-negotiation training with Negoisst once they had gained fundamental face-to-face negotiation skills. After the students were trained to negotiate electronically using the Negoisst system, they conducted bilateral negotiations with students from the other universities. The data obtained during the training phase and the international negotiation have been used to answer important research questions over many years (e.g. Filzmoser et al. 2016; Melzer and Schoop 2016) as this setting has been in place for over ten years. The current paper reports on the 2019 experiment.

The conventional e-negotiation training (c-training), i.e. the non-gamified existing training, is conducted by the same instructors that also teach their students during the regular negotiation lecture course. All instructors have decades of experience in conducting such a training and frequently exchange the contents and the pedagogical methods in their courses, thus, ensuring that all students obtain the same knowledge in an identical way. The c-training is compared to the new gamified training which was developed as part of the research reported on in this paper.

4.2 Experiment Procedure

The experiment included the training to be evaluated, in which the students learned to use the NSS and to negotiate electronically, followed by the five-day international e-negotiation. In the international e-negotiation participants negotiated in a bilateral setting. We focus on the training phase and will not analyse the negotiation outcomes of the five-day e-negotiations, as these are strongly influenced by the negotiation partner and their behaviour, which might interfere with our experimental manipulation. However, we can analyse the participants’ system use during these e-negotiations, which is less dependent on and influenced by the negotiation partner as the negotiations had to be conducted via the system.

The negotiation courses were assigned a-priori to the control group or to the gamified group using the designed system. We chose this approach instead of a randomised assignment, as students of the same university could become aware about the different system features within their course, i.e. the included game components, which would confound our experimental setting and results. Students in courses 1 & 2 were assigned to the gamified training group (g-training) and students in courses 3 & 4 were assigned to the control group participating in a conventional training (c-training). In a first online survey before the training, students’ demographics and their intrinsic motivation for the overall negotiation course were assessed. Afterwards, the two types of training took place within a timeframe of nine days. The c-training was conducted during the regular lectures of the courses by their instructors. G-training participants could choose their preferred time to complete the training. Both types of training are expected to require about 90 min for each student.

In the g-training participants had to complete the first three levels successfully. The slides from the c-training were the basis for the contents of these levels presented in the guided tours, ensuring the same effectiveness. Its effectiveness and contents were furthermore evaluated by one instructor of a c-training. In the g-training all previously described game components were present. The c-training was conducted in a face-to-face setting and followed an enactive method (Melzer and Schoop 2016). The instructor presented the NSS and its features first followed by the students implementing their chosen negotiation strategy in a negotiation using a trial-and-error approach. During that training, students prepared for and completed a multi-issue negotiation task. In the present experiment, this task was identical to the third level in the g-training. With the instructor acting as a moderator, students discussed their results after the preparation phase and after the negotiation (Melzer and Schoop 2016). Similar to the g-training, the participants negotiated with the TNT (Melzer et al. 2012) and received immediate responses. However, their version of Negoisst did not contain any of the gamified components.

In both training settings, participants could continue to work with the system and practice e-negotiations as often as they liked during the allotted nine days. As a measure for voluntary engagement all concluded negotiations were counted. After the training was conducted, participants had to fill in a second online survey measuring their intrinsic motivation for the training. The survey further included quiz questions to assess their learning outcomes regarding their understanding of the NSS and their ability to perform e-negotiations, as well as an evaluation of the training and the integrated components.

4.3 Data Collection and Analysis

The students’ intrinsic motivation was measured using the Intrinsic Motivation Inventory (IMI) by Ryan et al. (1983), which is rooted in the self-determination theory (Ryan and Deci 2000b) and is an established measurement for intrinsic motivation for gamification (Seaborn and Fels 2015). We used the IMI’s subscales “interest/enjoyment” and “perceived competence”. The interest/enjoyment subscale is the self-report of intrinsic motivation. Perceived competence is a positive predictor of intrinsic motivation (Ryan and Deci 2000b) and particularly interesting for gamification research, as several game elements provide competence-confirming feedback (Sailer et al. 2017). We assessed intrinsic motivation and perceived competence before the training (i.e. their motivation and competence for the negotiation lecture course) and after the e-negotiation training. The analysis follows a repeated measure design, which allows to adjust for any differences in motivation between the training groups and—since instructors of the c-training are also the instructors of their negotiation lecture course—enables analysing motivational changes for the training. All variables were measured using five items and a 7-point Likert scale from “Strongly disagree” to “Strongly agree”. Cronbach’s alpha shows good reliability for the four measured variables “intrinsic motivation for the negotiation course” (α = 0.86), “perceived competence for the negotiation course” (α = 0.86), “intrinsic motivation after the training” (α = 0.90) and “perceived competence after the training” (α = 0.86).

Learning outcomes were measured using multiple-choice quiz questions with four answers each in the second survey. In the following, we distinguish between learning outcomes for the system (LO system), i.e. participants’ knowledge about the NSS, and learning outcomes for e-negotiations (LO e-negotiation) referring to their ability to conduct e-negotiations, especially relating to their decision making (Schmid et al. 2020). Four questions measured LO system and three questions LO e-negotiation (see Appendix). Each correct answer per question was awarded with one point, whereas wrong answers led to a subtraction of one point. Additionally, participants evaluated their training using a 5-point Likert scale with scores from “1 very good” to “5 poor”. They were asked to evaluate how well the training helped them to get used to the system, how well it helped them to learn to negotiate electronically and how they would evaluate the overall feedback gathered during the training. Finally, the students’ engagement was analysed using one objective and one subjective measure. As objective measure, we use the participants’ voluntary engagement to further their skills. Participants in both types of training had the option to practice by conducting additional negotiations in the system. Therefore, all additional negotiations were counted. If participants in the g-training failed to pass one of the first three levels and had to repeat the negotiation again, this negotiation was not counted. As a subjective measure, we included the effort subscale of the IMI (Ryan et al. 1983). This variable was measured using five items and the same 7-point Likert scale used for the previously presented IMI variables. Cronbach’s alpha reveals good reliability (α = 0.83).

To evaluate our design in detail, the participants were asked about their perceptions of the integrated game components. We used seven items for each component which cover a broad spectrum of desirable and undesirable gamified education. We asked participants, whether they felt generally motivated by the element, whether the element made them strive to be the best or better themselves, and whether they consider its use to be enjoyable. For effective learning we have argued for the importance of feedback. Two questions assessed whether participants perceive the element to have helped them in their learning tasks and to have provided valuable feedback. To identify potentially negative effects of the elements, the participants stated whether they perceived the element to be demotivating and distracting (Blohm and Leimeister 2013), which e.g. could potentially occur for the badge notifications. Answers were given on a 7-point Likert scale from “Strongly disagree” to “Strongly agree”, and also included “not applicable” if participants could not answer.

We complement and improve our data analysis using log file analysis. Every HTTP-request in Negoisst has been tracked with a timestamp. The collected data allows us to analyse which participant has used which feature or component in the system and how often these were used. This enables us to exclude ratings of components that were not used at all. Furthermore, while negotiation outcomes in any negotiation are highly dependent on the negotiation partner, we will analyse the impact of the training on the use of three NSS features in the international e-negotiation. The features were presented in both types of training and are expected to improve participants’ rationality, decision-making, and communication quality, namely the visualisation of their preferences, the history graph, and semantic enrichment (Schoop 2010).

Statistical analysis was performed using IBM SPSS Statistics 27. From the original data set of 234 participants several participants had to be removed from the data analysis. Participants were excluded from analysis if they had not answered both surveys or did not participate in the training. Three participants were excluded due to contradictory answers for the reverse items in the IMI, i.e. for strongly agreeing on finding the training very interesting and very boring at the same time. In total, 201 participants remained for the analysis.

5 Results

After cleansing, 91 participants in the g-training and 110 in the c-training remained for statistical analysis. Participants in the c-training group are slightly older (M = 24.62, SD = 2.72) than the g-training participants (M = 24.01, SD = 2.20). Gender distribution is unequal between the groups: While female (43) and male (48) participants in the g-training are almost balanced, the c-training includes more female (78) than male participants (32). To ensure that there is no selection bias and that the groups have similar relevant characteristics for an electronic negotiation training (Shadish and Cook 2009), participants were asked about the frequency of using electronic devices within a month (from “1—once a month” to “5—several times a day”) as well as their average duration of daily use (from “1—less than 2 h” to “5—more than 8 h”). Table 1 shows the mean values per group and the test statistics, revealing that there are no significant differences between the groups.

Table 1 Descriptive and test statistics for participants’ use of electronic devices

5.1 Impact on Participants’ Motivation

To answer the first research question regarding the impact of the training on participants’ motivation, their intrinsic motivation and perceived competence for the negotiation course before the training (pre-score) and their intrinsic motivation and perceived competence after the training were measured. The means, standard deviations, and correlations for the four variables are shown in Table 2. Before the training, participants in the c-training reported slightly higher intrinsic motivation. In both types of training, intrinsic motivation decreased, with the g-training participants reporting slightly higher intrinsic motivation after the training. Perceived competence was higher in the c-training group before and after the training. However, in both types of training the perceived competence slightly improved. Participants’ pre-score of intrinsic motivation is significantly correlated with the intrinsic motivation after the training (r = 0.30). According to SDT, perceived competence is a positive predictor of intrinsic motivation. Therefore, the significant correlations between perceived competence after the training and intrinsic motivation after the training (r = 0.42) and between perceived competence (pre-score) and intrinsic motivation (pre-score) (r = 0.38) make sense. Additionally, perceived competence (pre-score) significantly correlated with the perceived competence after the training (r = 0.33).

Table 2 Descriptive statistics and correlations for motivation variables

To analyse the effect of the training on participants’ intrinsic motivation, we conducted a repeated measures ANOVA. Our experimental design can be described as a “one between” and “one within” factor design (Stevens 2009), where time is the within-subjects factor and the participation in the training the between-subjects factor. Since we only compare two settings at two points of time, the assumption of sphericity is not relevant (Field 2018). Applying the repeated measure ANOVA, the analysis of the effect on intrinsic motivation revealed a significant effect of time, F(1, 199) = 6.79, p = 0.010. On average, the reported intrinsic motivation decreased for participants of both types of training. However, intrinsic motivation in the c-training decreased more drastically than in the g-training. The main interaction effect between time and training had a significant effect on intrinsic motivation, F(1, 199) = 4.05, p = 0.045. Consequently, participants in the g-training could maintain their intrinsic motivation level over time, whilst participants in the c-training report lower intrinsic motivation after the training. No significant time or interaction effects between time and training were found for perceived competence.

5.2 Impact on Engagement and Learning

The self-reported effort and the voluntary engagement of participants to conclude additional negotiations were analysed to answer research question 2; and the learning outcomes of participants were analysed for the multiple-choice questions to answer research question 3. According to an a priori conducted Kolmogorov–Smirnov test, normality distribution cannot be assumed for the learning outcome (LO) variables and the additional negotiations. We will, therefore, use a non-parametric Mann–Whitney U test for comparison between the groups. Regarding the effect on engagement, participants in the g-training reported higher effort than c-training participants (see Table 3). An independent samples t-test revealed no significant differences (t (199) = − 1.333, p = 0.18). The number of additional completed negotiations was higher in the g-training group (Mdn = 2) than in the c-training (Mdn = 0). This difference was highly significant (U = 2443.50, z = − 7.12, p < 0.001) and resulted in a large effect (r = − 0.50). The learning outcomes for the system could range between − 16 and + 16 and were higher in the g-training group (Mdn = 6) compared to the c-training group (Mdn = 2). System learning outcomes differed with a high significance (U = 2088.50, z = − 7.19, p < 0.001) and also resulted in a large effect (r = − 0.51). The learning outcomes for e-negotiations could range between − 12 and + 12. Again, the g-training group performed better (Mdn = 10) than the c-training group (Mdn = 8), resulting in a significant difference and a small effect (U = 3816, z = − 3.00, p = 0.003, r = − 0.21).

Table 3 Descriptive statistics for engagement and learning outcomes

Since gender distribution is unequal between the two training groups, gender differences were assessed. While gender had no significant impact on participants’ intrinsic motivation, some interesting results exist for participants’ engagement and learning outcomes. For both types of training, females reported higher invested effort (see Table 4). In the g-training group, there was a significant difference between male and female participants (t(89) = 2.112, p = 0.038). Female participants in the g-training conducted more additional negotiations than males, also resulting in a significant difference (U = 778.5, z = − 2.07, p = 0.038). However, male participants in the c-training were more likely to perform additional negotiations than females. Regarding the learning outcomes, male participants achieved better system and e-negotiation learning outcomes in both types of training. These differences were significant for the system learning outcomes in the g-training (U = 766, z = − 2.14, p = 0.032) and the c-training (U = 740.5, z = − 3.41, p = 0.001).

Table 4 Mean values per gender for engagement and learning outcome variables

The participants also evaluated the training regarding its subjective impact on their learning using scores from “1 very good” to “5 poor”. G-training participants evaluated their training to have helped them getting used to the system significantly better than c-training participants (see Table 5). The same pattern with slightly lower scores can be found for the question whether the training helped to negotiate electronically: G-training participants rated their training better than the c-training participants, but no significant effect was found. The overall feedback received during the training was rated the worst in both types of training but was significantly better evaluated in the g-training than in the c-training.

Table 5 Participants’ evaluation of the training using scores

5.3 Evaluation of Integrated Components

To evaluate our design in detail and answer the fourth research question, the participants were asked about their perceptions of the included components. The means for each of the components and the measured seven items are shown in Table 6. Note that not every participant evaluated every component. Furthermore, we removed ratings for the utility rankings and the Pareto graph from the analysis, if the log file showed that the participants had not viewed the component at all. The use of the other components was either obligatory (i.e. the levels) or could be seen on the start-page or as notifications in the system.

Table 6 Mean values for the evaluation of the integrated components

In sum, the levels as the most central part of our gamified system were perceived most positively. They were perceived as motivating, enjoyable and helpful for learning tasks. The utility rankings also revealed motivational power, and their feedback on negotiation performance was considered to be valuable. Interestingly, the Pareto graph as a negotiation specific feedback component still possesses much motivational power and made participants strive to better their outcome. The graph’s feedback was perceived as the most valuable feedback component. The most controversial component was the process feedback. It received the lowest scores in five of seven categories. The high standard deviations further indicate that this component was perceived differently. The two reward components, i.e. experience points and badges, revealed quite similar results and were considered to be particularly motivating and enjoyable, and scored only average regarding their feedback and impact on learning. In general, the badges were perceived more positively than the experience points. Overall, this analysis confirms large parts of our system design.

Splitting the participants by gender reveals some interesting but not significant tendencies regarding the perceptions. The levels were more motivating for females (M_female = 5.54; M_male = 5.36) and made them strive harder to better their performance (M_female = 5.58; M_male = 5.29). The Pareto graph was more enjoyable to females (M_female = 5.03; M_male = 4.52) and made them strive harder to better their performance (M_female = 5.28; M_male = 5.09). In contrast, males perceived the utility rankings to be more enjoyable (M_female = 4.81; M_male = 5.15) and helped them strive harder to be the best (M_female = 5.71; M_male = 5.96). Male participants also found the badges to be more motivating (M_female = 5.40; M_male = 5.70) and enjoyable (M_female = 5.05; M_male = 5.34) than female participants.

5.4 Impact on Subsequent System Use

After the training, all but three participants engaged in a five-day international e-negotiation. The participants continued to use the gamified or non-gamified system they used in the training. We only analysed the participants’ individual system use by analysing our log files and the messages exchanged to answer our fifth research question for reasons discussed before. First, we assessed how often the participants analysed their preferences, which is useful for the preparation and offer construction. The second feature is the history graph (Schoop 2010). Both features are expected to increase participants’ rationality and improve their decision-making. Since participants’ need to grasp the displayed information, only those data sets were considered that showed a time of use of at least 10 s. Finally, the exchanged messages and how often the communication feature semantic enrichment were analysed (Schoop 2010). The semantic enrichment is used in the textual messages to avoid misunderstandings and to improve communication quality.

The results in Table 7 show that on average participants of the gamified training used all of the system features more often. Since there is a non-normal data distribution, a non-parametric independent samples Mann–Whitney U test was conducted. In general, the participants of the g-training analysed their preferences more often, resulting in a statistically significant effect (p < 0.001) and a medium effect (r = − 0.38). Also, g-training participants analysed the negotiation process more often (which is statistically relevant with p < 0.001) which resulted in a small effect (r = − 0.27). Last, participants of the g-training also used the semantic enrichment feature more frequently, which yields another statistically significant difference (p < 0.001) and a medium effect (r = − 0.41).

Table 7 System feature use after the training

6 Discussion

In the present study, we report on the design of a gamified NSS used in e-negotiation training to improve participants’ motivation, engagement and resulting learning outcomes. Using a quantitative study, we evaluated our designed artefact by comparing it with a conventional e-negotiation training (Melzer and Schoop 2016). We further provided an analysis of the included components on how they support participants’ motivation and learning.

For the first research question, the effects of the gamified NSS on participants’ motivation were analysed. The participants’ intrinsic motivation before and after the training was measured. A repeated measure ANOVA revealed a significant interaction effect between time and training. While participants’ intrinsic motivation in the c-training decreases, intrinsic motivation for g-training participants is on an almost constant level. After the training, the c-training participants report lower intrinsic motivation than g-training participants. In general, negotiations as a soft-skill topic build upon intrinsic motivation to learn about it (Melzer 2018). Traditional negotiation training creates large involvement with the learners through role plays, cases, and discussions (Lewicki 1997). Prior research assumes that intrinsic motivation of participants before and during an e-negotiation training is also very high and facilitates self-regulated learning (Melzer and Schoop 2015). However, negotiation training systems have recently been criticised for neglecting the importance of facilitating motivation for the training tasks (Ding et al. 2020). The results support our assumption that these conventional forms of e-negotiation training do not facilitate intrinsic motivation sufficiently, as intrinsic motivation decreases during the training. Using the gamified artefact, participants’ intrinsic motivation for the course could be maintained throughout the e-negotiation training.

According to the results and in line with SDT, perceived competence is a strong predictor of intrinsic motivation (Ryan and Deci 2000b). However, in this study perceived competence was not affected by the participation in the training. On the one hand, this is surprising, as several components implemented in the NSS such as badges, points, and rankings provide competence-confirming feedback (Sailer et al. 2017). On the other hand, the controversial process feedback might have diminished perceived competence. Potential reasons for the difference in intrinsic motivation between the groups might be related to the satisfaction of the psychological needs for autonomy and relatedness, which have not been measured in this study.

Regarding our research questions two and three, we were interested in the effects on participants’ engagement and their learning outcomes. Although participants in the g-training already had to complete the first three level negotiations, they were motivated to further engage in more additional negotiations than those in the c-training group. Qualitative interviews in a previous study revealed that the relative feedback on their negotiation performance through the utility rankings motivated them to repeat a level and experiment with other negotiation strategies (Schmid et al. 2020). In this study, we additionally included the Pareto graph to provide an absolute feedback, allowing to assess missed negotiation potential, and to facilitate the crucial reflections for the experiential learning methodology (Kolb 1984). In general, participants’ need for social comparison to assess one’s performance differs (Schöbel et al. 2017). Offering both absolute and relative feedback through rankings and the Pareto graph provides motivating and informative feedback for all participants.

Significantly better learning outcomes—relating to participants’ decision-making and important negotiation skills such as preparedness, rationality, and strategic behaviour—were obtained by the participants in the g-training. Whilst negotiators so far often agree on inefficient agreements (Gettinger et al. 2016), we expect our participants in the g-training to settle on better or less inefficient agreements. An even stronger effect on participants’ learning outcomes exists for the system skills. The participation in increasingly more complex levels in combination with the feedback provided through game elements and an improved intrinsic motivation appear to be an effective mechanism for participants to be deeply engaged in the system. For novice users of an NSS, the cognitive burdens are very high (Schmid and Schoop 2019). Intrinsic motivation is likely to occur when the task at hand is considered to be both challenging and attainable, matching the current skills of an individual (Csikszentmihalyi 1990). This is done in our artefact by providing increasingly more complex levels and system features on each level. The c-training, presenting all of the features at once and starting with multi-issue negotiations, might overwhelm the participants and is less effective for learning according to the results. An analysis of the game elements in this study confirm that participants liked learning with the levels. While Urh et al. (2015) recommend to divide the main learning task into smaller sub-tasks, Alcivar and Abad (2016) and the results of this study particularly show the effectiveness of structuring the tasks using levels for system training.

Within the g-training a significant impact of gender was found. While females and males reported similar intrinsic motivation, females’ subjective and objective engagement was higher than for males. So far, prior research revealed only perceptual differences regarding social benefits of gamification (Koivisto and Hamari 2014) or different perceptions of the game components (Codish and Ravid 2017). The higher engagement of females might stem from the perception of levels to be more motivating for female participants than for their male colleagues. The role of gender remains to be explored in more detail, as our findings also suggest the badges to be more motivating for males, which contradicts the findings of Codish and Ravid (2017). Differences between the implementations and goal setting of the badges in these two studies might cause different perceptions, which need to be further investigated.

To further analyse research questions two and three, the participants assigned scores for their perception of the system training, e-negotiation training, and overall feedback. These scores reflect the results of the learning outcomes very well, i.e. the participants perceived the g-training to be very good (which is the highest score) for system training and much better than their c-training counterparts. The impact on e-negotiation training was also evaluated better for the g-training than for the c-training but the difference is smaller. The overall feedback in both types of training was characterised as satisfactory and better in the g-training, but still improvable. We acknowledge, that most of the feedback in the gamified artefact is centred around the participants’ learning progress and their decision-making in e-negotiations and less around their communication behaviour. The TNT replying to the messages of the human negotiator is limited in its ability to reflect human communication behaviour, e.g. to detect and reflect emotions or different levels of politeness. One potential technology to improve the TNT is bot technology, which is capable of realistically imitating human behaviour (Ferrara et al. 2016) and might improve realism in e-negotiations (Schmid et al. 2021). A recent study on feedback in e-negotiation training shows that participants want additional features such as an expert review or the possibility to set and track their negotiation goals, all of which improve negotiation skills such as preparedness, effectiveness, goal-orientation, rationality, strategic and problems-solving (Meyer et al. 2020). Including such elements could further improve the acquisition of e-negotiation learning outcomes in this complex application domain. In its current form, the artefact might also be used by business organisations for e-negotiation training. When the TNT is capable of answering in a more realistic way, further levels including different negotiation simulations can be added for a more extensive e-negotiation training.

The current components included in our gamified system and their perceptions of the participants have been investigated as part of research question four. Similar to other gamified learning interventions (e.g. Buckley and Doyle 2017; Putz et al. 2020) we have integrated several game elements contributing to different parts of the learning experience. According to Dicheva et al. (2019) it is important to focus on the holistic learning experience provided by the use and combination of different components. These different components contribute to different objectives of a gamified learning intervention, i.e. to pedagogical objectives and/or the facilitation of psychological needs. Our results do not allow to draw direct conclusions regarding their individual impact on the measured variables. However, they provide an evaluation for each component, i.e. whether they are perceived as intended or need to be revised, and give a rough estimation on how they might contribute to the results. The levels—the only component whose use was enforced—were very positively perceived among all categories, i.e. they are perceived as being very helpful for learning while motivating the participants through clear intermediate checkpoints and feedback on their progress (Glover 2013). The rewards and the feedback by badges and experience points were perceived as less helpful for learning but make the learning experience more enjoyable. The utility rankings together with the badges are the most motivating component. Utility rankings provide informational performance feedback, motivated the participants to do their best and to improve their performances.

As another performance feedback visualisation, the Pareto graph was included as a domain-specific feedback component. Surprisingly, the Pareto graph was perceived to be motivating in general and less demotivating than the utility rankings. Furthermore, the Pareto graph was evaluated as being more helpful for learning than the utility rankings and scored as the best component regarding its valuable feedback. In fact, absolute feedback can be more powerful than relative feedback (Moore and Klein 2008) and evaluating a score against an absolute standard is also common in several games. The review by Koivisto and Hamari (2019) shows that points, badges, and leaderboards are still most frequently used in gamified systems. They also observe that several studies incorporate other gamification elements as well. Our results suggest that using a domain-specific feedback component can also yield motivational power. We would, therefore, encourage other gamification designers to broaden their point of view by searching for such feedback components, as they can serve as an important addition for a gamification design.

The most controversial component was the process feedback displayed during the negotiation process. It was perceived to be the least motivating and most demotivating component. If comparative feedback is present, participants are more oriented towards social comparison behaviour and will experience greater pressure (Huschens et al. 2019). In contrast to the informational feedback providing utility rankings, the process feedback might be perceived as much more comparative and competitive. Another issue explaining the rather negative evaluation of this component might be the design of the process feedback messages themselves, which confront low performers with the coarse feedback that they belong to the lower half. More detailed feedback as to whether these low performers belong to the third quarter or the last quarter could be more informative and motivating. Based on the results, we will question the use of this component in a further design iteration. Additionally, we encourage other researchers designing and evaluating complex gamified learning interventions to adapt the seven suggested items, helping them to detect weaknesses and strengths in the design. Mixed-method studies in early design stages might further help to improve the gamification design and gather insights into participants’ perceptions.

Finally, the g-training also had a positive effect on participants’ subsequent use of the system in an international negotiation (RQ 5). All three system features improving participants’ decision making or communication behaviour had a significantly higher frequency of use by participants of the g-training. However, all participants used the system only for the duration of three weeks which is rather short. Two reward components, i.e. the experience points and the badges, awarded desirable system use. The use of rewards is often debated, as they lead to rather extrinsically motivated behaviour and may undermine intrinsic motivation (Deci et al. 1999). Liu et al. (2017) consider the use of rewards to be effective for short-term or intermittent system use. At least for short-term usage, we successfully demonstrated that employing rewards as an incentive for the use of support features of the NSS works. The insufficient use of NSS features might, therefore, be solved by the use of game elements instead of designing these systems more proactively (Druckman et al. 2012). Based on the internalisation process in SDT, Schmid and Schoop (2019) suggest that the originally extrinsically motivated behaviour (i.e. through rewards) will become more self-determined, as soon as the benefits of the features for one’s negotiation performance are recognised. However, the effects of gamification in NSS for the long-term remain unknown, and, therefore, need to be investigated as one important area for future gamification research (Nacke and Deterding 2017).

Our study includes several limitations. First, we conducted a quasi-experimental study and no randomised study to evaluate our artefact. While it was necessary to run a quasi-experimental study to avoid confusion among the students within the same course about the presence of game elements, there might be differences between the two training groups affecting our results. The results might be further biased by the online vs offline setting and due to the asynchronous vs. synchronous setting, leading to effects that cannot be attributed to the gamification design only. However, a prior study showed that given the choice between an asynchronous and synchronous training, students choose their training for opportunistic reasons. Furthermore, the students participating in the asynchronous training particularly liked the game elements (Schmid et al. 2020). Nonetheless, a bias cannot be completely ruled out. Last, our learning outcomes for e-negotiations cover only the decision-making skills of the participants. An additional, more detailed analysis of their communication behaviour is required to retrieve an overall picture regarding the training’s effectiveness.

7 Conclusion and Future Research

The present study reports the design and evaluation of a gamified negotiation support system, which is commonly used in e-negotiation training (Melzer and Schoop 2016). Our evaluation using a quasi-experimental design with a conventional training as control group reveals a positive effect of gamification on participants’ intrinsic motivation, enhanced engagement, and better learning outcomes. The artefact was particularly beneficial for system training, which is also manifested in improved system use by the participants of the gamified training. The effect on the acquisition of e-negotiation skills was also positive. However, we have also seen that the feedback during the training for participants’ e-negotiation skills could be further improved, as this complex task includes problem-solving, decision-making, communication, and collaboration. Overall, we successfully applied gamification in the domain of e-negotiations to provide an artefact that can be used for a motivating and engaging e-negotiation training. For both system and negotiation training, we recommend the use of increasingly more challenging tasks (e.g. in the form of levels), as they present clear proximate sub-goals to attain (Glover 2013; Urh et al. 2015) and match participants’ current skills (Lee and Hammer 2011).

Our study provides several opportunities for future research. First, our research shows the effects of the individual game elements without measuring their direct impact on motivation and learning, different combinations of game elements can be tested and their effects measured (Dicheva et al. 2019). For example, the Pareto graph and the utility rankings are two competing feedback alternatives for participants’ negotiation performance. It would be interesting to see whether the use of a domain-specific feedback element such as the Pareto graph, which was evaluated as being motivating, yields the same effects as the use of the utility rankings. Second, the artefact itself provides various opportunities to include new negotiation feedback and further improve the learning process as well, as has been investigated by Meyer et al. (2020). Last, having collected more data and log files of users, we might derive certain usage patterns and game element preferences of the participants. Classifying participants according to their gamification user types (e.g. Tondello et al. 2016) and deriving usage patterns and preferences for these user types might reveal interesting results for the area of tailored gamification research, and might also help to explain the higher engagement of females in greater detail. Tailored gamification might be used to provide suitable game elements for each type of user and to avoid negative effects on their motivation.

Appendix

System Learning Outcome Questions:

• (1) Which of the following statements are true regarding the negotiation agenda for multiple issues when writing a formal message?

  • a. For each issue a value has to be defined.

  • b. For several or all issues the values can be left empty. (R)

  • c. One cannot select values for the issues while writing a formal message.

  • d. Each time the value of an issue is changed, the utility value is updated. (R)

• (2) What kinds of issues are represented in the Negoisst preferences?

  • a. Numeric issues (R)

  • b. Compatible issues

  • c. Non compatible issues

  • d. Categorial issues (R)

• (3) Which of the following statements are true for the negotiation protocol?

  • a. I can send a Reject to end the negotiation at any time.

  • b. I can only send a message when it’s my turn to do so. (R)

  • c. I can send an informal message, if I’d like my negotiation partner to clarify something. (R)

  • d. I can only accept my negotiation partner's counteroffer, if he/she defined values for all negotiation issues. (R)

• (4) What is NOT a purpose of semantic enrichment?

  • a. Clearly indicating the intention of the negotiation message. (R)

  • b. Improving English grammar. (R)

  • c. Consistency between the written message and the values in the agenda.

  • d. Suggesting optimal counteroffers. (R)

E-Negotiation Outcome Questions:

• (1) Given the preferences above, which negotiation issue or which negotiation issues are your most important ones?

  • a. Price

  • b. Guarantee

  • c. Delivery (R)

  • d. 4 years

• (2) Given the preferences above, which statements are correct regarding the best cases?

  • a. The best case for the issue price is 5000. (R)

  • b. The best case for the issue guarantee is 3 years.

  • c. The best case for the issue delivery is overnight. (R)

  • d. The best case for the issue guarantee is 2 years.

    

• (3) Which of the statements are true for the History Graph depicted above?

  • a. My last offer had a utility of 50% and my partner should have a utility of 25%.

  • b. My last offer had a utility of 50%, my partner’s utility value is unknown. (R)

  • c. My partner’s last offer had a utility of 25% for me, so my partner should have a utility value of 75%.

  • d. My partner’s last offer had a utility of 25% for me, my partner’s utility value is unknown. (R)

Note: Answers marked with (R) are right answers.

The quiz questions included the following instruction: “Note that for each question one or multiple answers can be correct”.