Engineering digital motivation in businesses: a modelling and analysis framework

Abstract

Digital motivation refers to the use of software-based solutions to change, enhance, or maintain people’s attitude and behaviour towards specific tasks, policies, and regulations. Gamification, persuasive technology, and entertainment computing are example strands of such a paradigm. Digital motivation has unique properties which necessitate careful consideration of its analysis design methods. This stems from the strong human factor involvement, and if it is not implemented effectively, it can result in digital motivation being perceived negatively or leading to reduced motivation. The emerging literature on the topic includes approaches for creating digital motivation solutions. However, their primary focus is on specifying its operation, for example, the design of feedback, rewards and levels. In this paper, we propose a novel modelling language which enables capturing digital motivation as an integral part of the organisational and social structure of a business, captured via goal models. We also demonstrate how modelling of motivational techniques at this level, the goal level, enables a more powerful analysis that informs the introduction, design and management of digital motivation. Finally, we evaluate the language and its analysis using different perspectives and quality measures and report the results.

1 Introduction

Digital motivation (hereafter DM) centres on the use of software solutions to increase the will of people to follow certain behaviours and prevent others [25]. For example, it is used to encourage adherence to fitness programs [32] and to assist smoking cessation [40]. DM builds on the well-established motivation theory, widely defined as the “psychological processes that cause the arousal, direction, and persistence of behaviour” [31]. The element which facilitates an increase in the will of a person to follow certain behaviours is called a “motive” [24]. Gamification [15] and persuasive technology [16] are examples of paradigms that employ DM and use software-based motives.

Enterprises embed various motivational strategies within their management practices. These include appraisals and bonuses offered as encouragement to employees to perform tasks more efficiently and boost the attainment of both business goals and quality outcomes. In this regard, motivation can be seen as a supplementary requirement that an enterprise can employ to support the fulfilment of other functional and non-functional requirements [48]. Also, integrating DM into the fabric of a business can contribute to the fulfilment of other social requirements, such as providing a sense of belongingness or loyalty to the enterprise.

We previously argued that the design of DM mostly follows approaches highly reliant on design creativity, with limited engineering principles and life-cycle support [46]. Such digital incarnation of motivation, however, brings new characteristics and abilities to it, such as capturing data with higher frequency and granularity, which amongst other things, enable more precision in the monitoring, accuracy, and transparency features of the rewarding system. However, an ad hoc introduction of DM to a business may be detrimental and pose adverse side effects as well, such as increased pressure and stress within the workplace [46]. This calls for the consideration of the creation and introduction of DM as a software engineering problem rather than a creative design issue. Similar to other categories of requirements such as security [36] and regulatory requirements [18], a rigorous modelling of DM within its socio-technical system environment will help capture and manage system requirements more effectively.

The existing approaches for engineering DM mainly specify its operation. They are heavily reliant on concepts from games design. For example, the game description language (GDL) [52] is a modelling language for digital games. Despite its capabilities, GDL is limited to describing motivation requirements in a business context. It focuses on play as the main goal, whereas in a business context, play is a secondary goal which should help the fulfilment of other business goals and desired behaviours. Another example is Agent Modelling Language (AML) [13] which enables designers to develop a behaviour model of the players. However, AML can only consider human behaviour while participating in a digital game context and does not consider a non-game context which requires considering the player’s role and interactions within an organisation, e.g. the interdependencies amongst stakeholders and their available strategies to achieve their requirements within the constraints and strategic business interests.

A further example in this area is GaML, proposed by Herzig et al. [20], which is designed specifically for gamification development. GaML divides motivation requirements into basic concepts and gamification rules. The basic concepts are the atomic motivational elements based on the taxonomy proposed by [15] and visual elements, such as an avatar. Despite the power of GaML in formalising the design and specification of a gamification solution, it still needs to cater for the social and organisational structure of the business and the fact that gamification has an intense human factor requiring a holistic socio-technical view.

There has been an effort to model maintainable goals by [7], noting the importance of data collection, the presence of extrinsic and intrinsic motivation, and allowing the users to control the maintenance of their goals. Despite the evaluation result of the study, in this case, it can be argued that the model provided relies on personal goals and their application, such as living a healthier life. In this context, users are expected to be more open towards data collection, monitoring, and surveillance. However, in a business context, the opposite behaviour may be observed as the nature of the surveillance and work progress tracking varies significantly.

According to [35], DM design needs to be integrated at the early stages of software design projects, a process which requires iterative testing of DM ideas and significant involvement of the users. They emphasise the priority of user goals over organisational goals, due to the users’ motivation and engagement with a system stemming from their personal needs and preferences, rather than organisational and business goals.

In this paper, we provide a systematic approach to model and analyse DM solutions aiming to ensure their fitness to the social and organisational structure of a business. To this end, we propose Digital Motivation Modelling Language (DMML), a language specialising in capturing DM and its relationship with requirements and organisational elements of a business captured through Goal Model [57]. DMML builds on the empirical studies presented in our previous works [45,46,47] which identified various facets of DM and issues potentially caused by its ad hoc introduction to a business. Our consideration of DM as a system requirement underpinned by an intense human factor is the main driver for DMML. Besides its usage as a documentation and representation tool, the language is augmented with a set of automated reasoning for processing its models and detecting conflicts that DM can introduce to a business. We present five of these reasonings in Sects. 5.2–5.6. We also evaluate DM and its reasoning toolkit from different perspectives and for different quality criteria and report the results.

2 Background

The literature on developing DM solutions provides principles and best practices for practical design solutions with a maximised chance of leading to change in users perception and behavioural towards business tasks, e.g. [12, 42]. The development of integrated methods for gamification design which are evaluated and established is still in its early stages. Methods available in the literature are often domain specific and tightly connected to the nature of the application areas such as gamifying education and learning online platforms [10, 29]. The focus is mainly on deriving heuristics and lessons from the application of specific configurations and modalities of gamification techniques and on providing evidence of their effectiveness in behaviour change and motivation.

Current gamification design methods inherit elements and characteristics from game design methods [5, 23] and are mainly focused on the implementation and testing stages, e.g. [19], with limited tools and concepts that suit the analysis and requirements stages. However, as gamification is a secondary system, i.e. operating on top of a serious business objective such as learning and answering customers calls, its design shall consider both the underlying system and the game mechanics augmenting that system. Omitting the nature of that primary system, the intentions and capabilities of personnel, as well as the social structure of the organisation in terms of roles and hierarchy, introduces risks of inefficiency and determinability. It can also lead to game mechanics that conflict with the primary goals and preferences of their subjects. This introduces severe side effects on staff work style and well-being [45,46,47]. Our approach provides constructs and mechanisms to detect these potential risks early in the development life cycle and filter the space of alternatives at the strategic level of goals and recommend viable design options which are consistent with their socio-technical systems. This includes the detection of a conflict of interest and potential for social loafing when a gamification technique is introduced to a task performed by individual actors.

Morschheuser et al. [34] proposed a method of seven stages for developing gamified systems aiming to address the lack of methods in developing DM solutions systematically. Interviews with domain experts and literature on gamification design are used as foundations for the method. The method is informed by various disciplines including psychology and behavioural science, user experience, and software analysis and design. It aims at addressing the whole cycle of developing gamification from the preparation till evaluation and monitoring stages. DMML and its algorithms support the stags of preparation and analysis in particular by allowing decision-making on design options to be known early and concerning a level of abstraction suitable to these stages of the development, i.e. the level of organisational goals and strategies.

Motivation and its various facets, such as gamification, have been of particular interest in the requirements engineering for some years now. For example, the use of gamification in requirements acceptance was studied, and a generic gamification model that captures possible gamified operationalisations of acceptance requirements was proposed [39]. Similarly, gamification has been studied as a means to increase stakeholders’ engagement during requirements engineering processes in order to improve performance [27]. Eliciting security requirements was augments via serious games in [8]. The authors in [53] propose gamification for inclusive and maximised collaboration and knowledge sharing amongst programmers and their adherence to the requirements specified.

The authors in [51] also propose a method for eliciting ‘soft issues’ in requirements engineering. Part of their taxonomy for establishing their method revolves around motivational processes and their consequences. These motivations, such as self-esteem or achievement, are in direct association with the capture and analysis of motivation as a requirement in software systems. However, while the paper goes through motivational approaches and their implications, it does not elaborate on how these motivations should be designed and implemented in the workplace together with other functional and non-functional requirements. Sutcliffe in [49] provides a review of applications of psychological theories into requirements engineering. This user-oriented RE can rely on the application of such theories in various modalities including scenario-based processes for the psychological state of the user.

Emotions and their formalisations have been another topic of research in requirements engineering. Sometimes referred to as affective computing [38], it has been noticed by some scholars that emotions have been little if at all, discussed in the process of requirements elicitation and specification [9]. However, emotions can usually have a direct relation with intrinsic motivations, i.e. activities which are conducted through intrinsic motivations can often lead to emotions such as fun and enjoyment [28]. Such emotions can also affect the work patterns of employees of an organisation, e.g. when a new software system is introduced in their workplace [41]. Therefore, some scholars have tried to incorporate the elicitation and specification of emotions into requirements engineering practices via RE-specific tool, such as the one proposed in [14].

In [54], the authors provided a set of characteristics together with their interrelationships for games success. A concept map of non-functional requirements and a set of questions to assess them are proposed to assist systems engineers in designing and validating a game. A systematic mapping study around the research in this area can be found in [37]. The conclusion was that the use of gamification was mainly focused on software development where few works tackled the phase of requirements. The authors also demonstrated that only simple gamification mechanics such as points and badges were used and that there is a general lack of evidence of their effectiveness.

We have highlighted the many interpersonal and intrapersonal psychological factors relating to motivation; if these are ignored or incorrectly implemented, this can lead to serious issues in the system, e.g. DM could cause reduced morale or destructive behaviours. For situations where competition or collaboration is encouraged (either between individuals or groups), the effectiveness of a reward will depend on the individual personalities of employees and group factors (e.g. the cohesiveness of groups within an organisation). Yee [56] similarly categorised ways to motivate video game-players into interpersonal and intrapersonal factors. Interpersonal motivation related to the role of social factors (in game design) to motivate players. Such as group strategies (e.g. players motivated other group members and encouraged group loyalty) and encouraging player interaction as a key motivation to continue playing. Intrapersonal motivation related to personal achievement and immersion. A sense of personal control can enhance self-esteem by gaining respect. Similarly, DM has been used to motivate individual student engagement in learning using digital games [55]. In both contexts, entertainment and education, incorrect use of DM can lead to negative results. Therefore, it needs to be incorporated based on psychological research and understanding of motivation.

A review of theories of motivation and motivating factors by Franken [17] highlights the roles of positive and negative reinforcement and extrinsic and intrinsic motivation. A goal for many motivators is to reduce harmful behaviours (such as smoking) and enhance positive behaviours (such as healthy eating). This can be achieved through positive reinforcement (rewards) or negative reinforcement (such as a reduction in benefits or withdraw existing positive rewards). Again though, if these are not operationalised or managed correctly within DM, they can lead to the opposite behaviour. Individuals differ in their preference for extrinsic or intrinsic motivation. Extrinsic motivation occurs when individuals are motivated to perform a behaviour to earn a reward or avoid punishment (pupils studying for a good grade, cleaning a room to avoid parents nagging, taking part in a competition to achieve prizes). Intrinsic motivation occurs when engaging in a behaviour because it is personally rewarding (sport enjoyable, solving a puzzle). Psychological research on these factors needs to be considered as, for example, it is known that offering excessive external rewards for an already internally rewarding behaviour leads to a reduction in intrinsic motivation. Consideration of such individual differences will be an important area for further research involving DM.

3 Methodology

Our research has adopted a mixed method approach towards conceptualising and formalising DM, with the aim of engineering DM requirements. The general phases of the methodology we followed are depicted in Fig. 1. The mixed method approach consisted of interviews with six experts, and surveys with a further forty experts from relevant fields of study. The survey consisted of multiple-choice questions with an open-ended text box for the participants to provide any additional comments. The qualitative responses underwent content analysis by three researchers, while the quantitative results were statistically analysed. The overall results of this phase of the study were validated by another study which involved interviews with ten participants (employees and managers) with experience in digital motivation within their workplace. Another ten employees were interviewed to identify more concepts which shape DM and have an impact in the context of a business environment. The studies and the material used them can be found in [44]. We will summarise these studies in the rest of this section.

Fig. 1

Research methodology adopted to create DMML foundations

3.1 Exploratory phase

Based on the literature review, a certain amount of ambiguity, and a lack of rigour in the design and implementation of DM in enterprises was identified. To investigate these issues, we used semi-structured interviews, allowing for additional flexibility in the interviewing process. The interview flexibility related to the order in which the questions were asked, and the option to ask supplementary questions, if necessary. Any new question would be added to the overall list of interview questions and become part of the next interview.

3.1.1 Identification of experts for interviews

To identify appropriate experts for interviews, we considered the finding of high impact peer-reviewed publications. In addition, experts with different affiliations, from various fields of expertise and background were invited to participate. The selected interviewees possessed both practical and expert academic experience, without any past common collaborative activities.

Of the six experts agreed to participate in the interview phase of this research, four were drawn from academia (with one of them collaborating closely with industry), and the remaining two from the industry. In terms of experience, three had engaged in developing theoretical frameworks for DM in the past, and three others had developed and applied DM in practice. Those with more focus on academic and theoretical aspects of DM had also implemented DM in practice as part of their research projects; hence, they also had experience in issues relating to DM. The experts came from different countries and had varying levels of exposure to DM; UK 4 years, South Africa 3 years, USA 4 years, Portugal 3 years, Germany 4 years, and Canada 10 years of expertise.

3.1.2 Interview process

To maximise the effectiveness of the interview sessions, interviewees were sent the set of interview questions in advance. The duration of the interviews ranged from 27 min to a maximum of 50 min, averaging on 39 min. Consent was obtained to record the interviews. Before the interviews, pilot studies were used to test and refine the interview questions.

3.1.3 Data analysis

The interviews were transcribed and the text was content-analysed to identify the important aspects of DM. The findings were grouped in several sub-themes. To content analyse the qualitative results, two researchers performed the analysis, while a third independent researcher was used to resolve any conflicting results. The survey questions were formed based on the agreed themes.

3.2 Enhancement phase

To confirm and enhance the findings of the interview session, a survey was conducted. The survey, as a quantitative phase, was designed to confirm and enhance the findings of the first qualitative phase, i.e. the interviews. The questionnaire included multiple-choice questions and an open text box at the end of each general question for participants to add further comments. The questionnaire was piloted on two participants before being sent to the experts.

3.2.1 Identification of participants

Authors of peer-reviewed publications were invited to take part in the survey. The study was designed to identify issues or discrepancies in the application of DM amongst the experts. A total of forty were sent a private link to the questionnaire.

Of the forty participants, seven considered themselves to be experts, eighteen identified themselves as having a high level of practical knowledge, 14 stated to have a medium level of practical expertise knowledge, and one expressed to have a low level of practical experience. The latter possessed an extensive theoretical and research background in DM. Contributors who specified their level of practical experience with DM as a medium were considered experts in areas which are core to the design of DM, such as incentive-centred design, cyber-psychology, and human–computer interaction (HCI).

Table 1 provides the distribution of participants based on their field of study and country.

Forty of the 48 experts who began the survey completed it successfully. In addition to statistical information, the comments given by the experts at the end of each question were collected and analysed to aid further statistical analysis.

Table 1 Distribution of participants

3.2.2 Confirmatory phase

At this stage of the study, interview sessions with 12 managers and employees were conducted to confirm the current findings from the quantitative phase, to provide the research with more in-depth details with regard to DM, and to inform the study about aspects of DM which can influence the well-being of employees within workplaces. To keep the opinions of the participant diverse and relevant, seven employees and five managers all familiar with DM in their workplace were involved in this phase of the study.

3.2.3 Personification study

To study the impact DM can have on the staff working in an enterprise, another set of interviews were arranged and conducted using ten different employees with experience of working in a DM (or gamified) environment. Participants at this stage aged between 24 and 37 years old, consisting of four females and six males with a balanced academic and industrial experience. This phase of the study was focused on eliciting the aspect of DM which can influence the preferences of users about DM settings. Participants provided their opinions and priorities on different settings of DM, and the actions they would take in various contextual situations, e.g. decreasing the quality of work for the sake of just gaining the points.

4 Digital Motivation Modelling Language (DMML)

The results of the studies described in the previous section were published in [45,46,47,48] providing a solid background for creating a theory-informed modelling language and analysis toolkit which put together the various concepts and issues we identified. In other words, our modelling language and analysis framework are a concretisation of the results obtained through the analysis of these studies.

In [47], we explore the elements which describe DM and its relation to the organisational models. For example, we derive essential characteristics of the motive to consider such as the chance of winning and the reward assignment strategy (individually or collectively assigned), and of the tasks and goals such as their ownership, the genuine interest of their actors in them and needed dependencies on other actors to achieve them. This is reflected in the meta-model for the environment (Table 2) and the meta-model for the motive (Table 3).

The study in [46] highlighted negative work ethics stemming from particular combinations of such characteristics. For example, a reward given individually on a competitive basis to an actor to perform a task which requires a resource from other actors may create a pressure on the staff when colleagues are late in providing that resource especially if colleagues are not given any reward for that. The findings of the study are reflected in the reasoning proposed in Sects. 5.2–5.6. Also, the problematic cases discovered in this study highlighted the need to model the interrelationships between actors within the organisation so that we can detect related cases, e.g. two actors competing on one task and collaborating on the other and being motivated through DM on one of them only. Such relationships are reflected in the augmentation to Goal Model proposed in Table 4.

As we recognise the individual differences and the fact that an organisational role is not enough to describe a person, we introduced the concept of personas in [45] where we elicit the elements which differentiate people in their preferences and issue with DM in the business environment. Such characteristics and personas are part of the environment meta-model presented in Table 2.

In this section, we put together the findings of the previous studies in a more formal presentation and create Digital Motivation Modelling Language (DMML) as a language that enables the capturing and management of motivation requirements in a business. The language considers the organisational structure and business activities in an enterprise as core aspects for achieving a holistic DM modelling. Hence, DMML positions software-based motives within their social and organisational structure. This consideration of DM makes goal-oriented modelling languages such as the i* [57] a suitable starting point. The i* modelling framework provides elementary concepts to model DM, e.g. goals, tasks, and actors to motivate. However, several concepts and attributes are still needed in i* modelling framework which would enable a more specialised, expressive, and comprehensive modelling of DM. Such augmentation enables a more powerful and accurate presentation and, hence, reasoning and automated analysis expressly tailored for DM and its properties. In the following subsections, we will describe the modelling constructs and meta-model for DMML and its graphical notation. The full description of DMML including its formal mathematical specifications and a set of automated reasonings and scenarios can be found as supplementary material at [44].

4.1 Modelling constituents and representations

The creation of DMML is based on the premise that the business environment and the use of digital motives are the factors most relevant to the design of DM in a business ecosystem, for which a specialised modelling language will be beneficial.

DMML is based on the application of DM elements (motives) to an organisational information system (environment). Table 2 concretises the elements which describe the business environment and relevant to DM ecosystem. For example, the actors’ relations on a task, e.g. dependency or collaboration, affect whether applying a reward with collective or individual performance measurement is introduced to that task. Tasks characteristics within the work environment such as uniformity, being quantity based and subjectivity are needed to detect cases when performance measurement can cause negative work ethics such as cheating, e.g. by increasing the quantity of tasks performance and neglecting quality to win a quantity-based reward. Figures 3, 4 and 5 are examples of modelling artefacts built based on this meta-model.

Motives such as leaderboards, badges, points and avatars are different regarding the reward policy, elements, nature and strategy. Table 3 presents taxonomy of motives characteristics. For example, some motive can have a strategy of high-value reward but a low chance of winning. This may appeal to some personas but not others in the working environment. Also, we discovered from our findings in [45, 46] that staff pay much attention to privacy and the performance information being captured about them when the DM systems and managers want to decide the reward. There could be a risk that DM is seen as spyware or as an ‘exploitation-ware’. Hence, we introduced the specification of the captured information regarding visibility and what is being stored as this will enable the detection of such risks when modelled together with the personas representation. Figure 6 is an example of an instantiation of a leaderboard as a motive according to this meta-model presented in Table 3.

4.1.1 Environment

In light of our previous studies, six main constituents can describe a working environment concerning DM—agents, personas, actors, values, tasks, and the relationships amongst these constituents. In Table 2, we sketch a meta-model which describes these constituents in details. These various constituents and their relationships are described as follows:

• Actors are active entities which should achieve specific sets of goals by performing certain tasks, and are the main pillar for the social structure of businesses. Actors are either an abstract representation or a class of agents.

• Agents refer to the individuals in a business who perform tasks to fulfil certain goals. Agents representation is important as their preferences on the design of the DM can ensure that DM is motivational and not a source of tension or pressure for individuals. Eliciting agents preferences on various settings of DM is a challenging task, as the number of agents may be large, and they may not share the same preferences. To address this challenge, we propose the use of personas [45].

• Values refer to the cultural and environmental values of business. Values could range from encouraging competition between the sub-groups of the workforce to encouraging collaboration between them. The values have different facets in relation to employee satisfaction and the importance given to achieving high performance in task quantity and task quality versus workforce well-being.

• Tasks are important in three different ways. The employees’ perception about Measurability and quantifiability of the outcome, their subjectivity of performance, and the perception of the quality orientation of tasks, indicating whether a task is quality or quantity focused. Hence, assigning a motive with pre-defined rewards may dissatisfy agents performing tasks with non-measurable and subjective outcomes, or even persuade them to reduce quality and focus on quantity when doing a quality-oriented task.

• Relations exist between constituents in a business environment. There are seven relations of importance to modelling DM:

  • Relations between actors and a task There can be a collaboration relation or a competition relation between actors with respect to a specific task, meaning that actors collaborate or compete with each other to perform the task.

  • Relations between two tasks Two tasks can be interdependent, meaning that the operation of one task is reliant on the input from another. Cross-actor task dependency leads to agents dependencies and this can, amongst other things, lead to negative work ethics which delay or prevent dependent actors from winning the reward.

  • Relations between a task and an actor Three relations between a task and an actor are of importance in modelling DM. First is the delegation relation which represents the passing of responsibility for performing a task from one actor to another. Second is the ownership relation, which represents whether an actor owns a task or just the responsibility of executing it. The third relation is the genuine interest of an actor in performing a task. Normally, this relations arises when a task is delegated from an actor to another.

  • Relations between two actors There are two relations between actors which are important for DM modelling:: supervision and promotion. A reward system shall consider that to avoid conflict of interest and be aware of the aspiration of agents.

  • Relations between agents and personas Agents can be assigned to a persona to represent them. It should be noted that based on the situation, for example, when adopting multiple actors, agents may switch between personas.

  • Relations between actors and personas This relation denotes the coverage of the persona across the agents adopting the role of that actor. It can be calculated using surveys with the agents adopting the role of certain actors as participants, providing their preferences on the created and shaped personas.

  • Relations between agents and tasks This relation specifies which task is delegated from one agent to another. Mapping the relations between an agent that is delegating a task to another agent provides designers with useful information which aids the prevention of probable conflicts and issues.

Table 2 Concepts and constituents of the environment

• Personas provide the designers with stereotypes regarding agents preferences. They are built using five different aspects: incentives, performance and feedback, privacy, goal setting, and collaboration nature [45].

  • Incentives have four characteristics describing agents preferences about the rewarding strategy. The consideration of the quality of output, not only the quantity, is the first. The second refers to the availability of the incentive. This indicates the knowledge of agents about the existence of an incentive, which in some cases may not be present. The third and fourth characteristics refer to the probability of winning and the value of the incentive which are coupled together, often meaning that an increase in one leads to a decrease in another. Agents may have different and often conflicting preferences on the chance of winning and the value of the incentive.

  • Performance and feedback reports are important for agents from two perspectives. One is the frequency of receiving the reports and the second is whether they are generated by either a computer or by a human as each can judge in a different way.

  • Privacy is a main concern of DM agents. DM relies on capturing and processing performance information about agents and providing it to a specific audience, e.g. a leaderboard available to everyone within the workplace. Different agents may have dissimilar views on who the audience should be and what type of information should be available.

  • Goal setting involves breaking down the tasks for agents into smaller tasks, and providing agents with the steps necessary to achieve their goals. Having control over defining these steps and being able to opt-out of implemented DM systems are two important aspects of this strategy.

  • Collaboration nature of the agents should be known prior to DM design. Introducing a motive that is not aligned with the collaborative nature of an agent, e.g. group-based motivation to a self-centred agent, may fail.

4.1.2 Motives

A motive model has two elements: the reward provided and the information captured. The meta-model for the motives is presented in Table 3. The elements of the meta-model are described in the following:

• Reward The reward introduced through a motive to a business is modelled in four facets; its policy, its persuasion element, the nature of its reward, and its rewarding strategy.

  • Policy Motives can employ three rewarding policies. The policy can encourage (i) competition, (ii) collaboration within and between groups. Also, the reward can be derived from the (iii) individual or groups performance.

  • Element The element of persuasion implemented in the reward could be social recognition, for example, giving agents a chance to become well known for their performance using the leaderboard. It could also be communication, facilitating the collaboration between the agents, using forums. Persuasion could create a sense of accomplishment, like virtual badges and levelling up, providing a feeling of achievement for the agents. In addition, constraints and coercion are also elements of the reward, such as deadlines and time pressure, warning employees about the time left to achieve their goals.

  • Nature The nature of a reward could be tangible, for instance, extra paid holidays or intangible, such as virtual badges. People may find these rewards motivational, as a result of their personal preferences.

  • Strategy Transparency is one of the strategy components. As a strategy, transparency can increase the acceptability of DM in the business, allowing employees to understand how decisions are made. However, this may not be an option for some businesses depending on their business plans. Another strategy element relates to the value of the reward and the likelihood of an agent to win the reward. Despite the appeal a high-value reward may have, generally, the value of a reward depends upon its scarcity. There will be different preferences for reward value or reward scarcity, or a reward setting can be provided which satisfies both views, such as providing scarce high-value rewards and easier to achieve lower value rewards.

  • Points Points could be given to agents in a pre-defined manner, meaning that agents receive a certain number of points for certain tasks. However, for quality-oriented tasks, human intervention could be necessary.

  • Reinforcement A motive can have positive reinforcement (such as rewards), negative reinforcement (such as demotion or sacking lower performer employees), or a combination of both. Negative reinforcement might not be obvious but, instead, it can be realised when others receive positive reinforcement. This can be detrimental and drive agents to perform unethically just to avoid the negative consequences of failing to meet desired behaviours.

• Captured Information DM relies on information from the environment. There are three important aspects of captured information, what information is stored, who can access it, and the frequency of its collection. Since DM often relies on captured information through sensors, cameras, and social surveillance, it can be intrusive in terms of what can be derived from it, for example, detecting the mood of individual agents using cameras, and showing it to them as an avatar representing their mood. Agents have different tolerance on the frequency of capturing the information. This could vary from real-time to low frequency data collection, which could be, for instance, at the end of each week.

Table 3 Meta-model for motives

4.2 Formal specification

This section provides the mathematical definitions for the properties that are needed for modelling DM as a system-to-be or as a system-as-is:

4.2.1 Environmental properties

Let $Ac=\{ac_1, ac_2, ac_3, \ldots , ac_n\}$ be a set of Actors, $P = \{p_1, p_2, p_3, \ldots , p_n\}$ be a set of identified Personas, $Ag = \{ag_1, ag_2. ag_3, \ldots , ag_n\}$ and $T=\{t_1, t_2, t_3, \ldots , t_n\}$ be the set of Agents and Tasks in the environment.

• Definition 1: Tasks

  $\forall t \in T, t = <Uniformity, Measurability, Quality-oriented|Uniformity, Measurability, Quality-oriented \in \{true, false\}>$

• Definition 2: Relation between Agent and Persona

  $AgP = \{agp_1, agp_2, agp_3, \ldots , agp_n\}$ is defined as a set of relations available between the agents and the personas present in the environment. Then,

  $\forall agp \in AgP, agp=<ag_i, ac_i, p_i, rel|ag_i \in Ag, ac_i \in Ac, p_i \in P, rel = Has>$

• Definition 3: Relation between Agent and Actor

  $AR=\{agac_1, agac_2, agac_3, \ldots , agac_n\}$ is defined as a set of relations available between agents and actors in an environment. Then,

  $\forall agac \in AgAc, agac=<ag_i, ac_i, rel|ag_i \in Ag, ac_i \in Ac, rel=Plays>$

• Definition 4: Relation between two Agents

  $AgAg = \{agag_1, agag_2, agag_3, \ldots , agag_n\}$ is defined as a set of relations available between two agents in the environment. Then,

  $\forall agag \in AgAg, agag = \{ag_i, ag_j, rel|ag_i, ag_j \in Ag, rel \subset \{Acquaintance, Close\} \}$

• Definition 5: Relations between two Agents and a Task

  $AgAgT = \{agagt_1, agagt_2, agagt_3, \ldots , agagt_n\}$ is defined as a set of relations available between two agents and a task in the environment. Then,

  $\forall agagt \in AgAgT, agagt = <ag_i, ag_j, t_i|ag_i, ag_j \in Ag, t_i \in T, rel = Delegated>$

• Definition 6: Relation between two Actors and a Task

  $AcAcT = \{acact_1, acact_2, acact_3, \ldots , acact_n\}$ is defined as a set of relations available between two actors and a task in the environment. Then,

  $\forall acact \in AcAcT, acact=<ac_i, ac_j, t_i, rel|ac_i, ac_j \in Ac, t_i \in T, rel \subset \{Competition, Collaboration\} \& rel \ne \{Competition, Collaboration\}>$

• Definition 7: Relation between Persona and Actor

  $PAc = \{pac_1, pac_2, pac_3, \ldots , pac_n\}$ is defined as a set of available relations between actors and present personas in the environment. Then,

  $\forall pac \in PAc, pac=<p_i, ac_i, rel|p_i \in P, ac_i \in Ac, rel \subset \{Plays, Weight\}>$

• Definition 8: Relations between Actors

  $RR=\{acac_1, acac_2, acac_3, \ldots , acac_n\}$ is defined as a set of available relations between actors in the environment. Then,

  $\forall acac \in AcAc, acac = <ac_i, ac_j, rel| ac_i, ac_j \in Ac, rel \subset \{Supervision, Next Role\}>$

• Definition 9: Relations between Tasks

  $TT=\{tt_1, tt_2, tt_3, \ldots , tt_n\}$ is defined as a set of relations between tasks in the environment. Then, $\forall tt \in TT, tt=<t_i, t_j, rel| t_i, t_j \in T, rel = Dependency>$

• Definition 10: Relations between Tasks and Actors$TAc=\{tac_1, tac_2, tac_3, \ldots , tac_n\}$ is defined as a set of available relations between the actors and the tasks. Then,

  $\forall tac \in TAc, tac = <t_i, ac_i, rel|t_i \in T, ac_i \in Ac, rel \subset \{Performs, Owns, No Interest\}>$

4.3 Motives

• Definition 11: Reward

  $RW=\{rw_1, rw_2, rw_3, \ldots , rw_n\}$ is defined as a set of rewards that motives can have. Then,

  $\forall rw \in RW, rw=<Policy, Nature, Strategy, Elements>$ where: $Policy=<type, value, performance|type \in \{competition, collaboration, combined\}, performance \in \{individual, group, both\}>$and$Nature=<type|type \in \{tangible, intangible, combined\}>$and$Strategy=<transparency, value, chance of winning, points, reinforcement| transparency \in \{true, false\}, value \in \{high, low, balance\}, chance of winning \in \{high, low, balanced\}, points \in \{pre-defined, calculated\_by = \{ac_i \in Ac\}\}, reinforcement \in \{positive, negative, combined\}>$and$elements = \big \{ e_1^n|e \subseteq \{social recognition, communication, accomplishment, time pressure, \ldots \}\big \}$

• Definition 12: Captured Information

  $CI=\{ci|_1, ci_2, ci_3, \ldots , ci_i\}$ is defined as a set of possible ways that motives can capture information. Then, $\forall ci \in CI, ci=\big \langle visibility, what\, is\, stored|visibility \in \{everyone, relevant, managers, self-only\}$and$what\, is\, stored=\langle personal\, information, frequency, work\, information| personal\, information \in \{true, false\}, frequency \in \{low, medium, high\}, work\, information \in \{detialed information, general information\} \rangle \big \rangle$

• Definition 13: Motives

  $M=\{m_1^n|\forall m, m=<t,rw,tl,ci>\}$ is defined as a set of motives available in the environment based on the values of all constructs of each motive and the task each motive is added to.

4.4 DMML graphical notation

This section describes the graphical notation of DMML and elaborates on how it can be used to model and represent the environment and the motives. DMML builds on the standard goal-oriented modelling language and therefore uses the same notation for the shared elements.

4.4.1 Modelling counterparts

DMML consists of three parts, the environment at the abstract level, the environment at the instance level, and the motive. The combination of these three parts enables various kinds of analyses of the impact of DM on a business.

• Environment—organisational level Environment is represented through nodes and links. Nodes can be tasks, goals, soft-goals, actors, personas, or motives in the environment. Links can be dependency, delegation, supervision, notification, promotion, ownership, no genuine interest, collaboration, or competition of a task. These constituents are described in Table 4, and a full legend of the notations is presented in Fig. 2.

• Environment– -personal level At the instance level, there are three relations to depict. The first is related to the mapping between actors and agents. The second relates to the mapping between personas, agents, and actors. The third related to the relation regarding agents and the delegation of tasks or goals to other agents. An actor may be played by various agents, and an agent may act as different actors in the environment. A full description of the constituent mapping and the required procedure is in supplementary material available in [44].

• Motives To model the motives, the meta-model provided should be used to facilitate the relevant information for each motive to shape its settings. We use a UML-like static structure diagram to describe motives. A configuration of a motive can be defined using its set of all possible attributes described in the meta-model. Each configuration can inherit the settings and have its own values for the attributes. For the purpose of readability, only a graphical notation representing the motive (a trapezoid) will be used to depict the model.

Fig. 2

DMML notation

Table 4 Description of DMML constituents

5 Motivation requirements analysis

The use of DMML can provide businesses with practical solutions for addressing difficulties which may emerge as a result of introducing DM. This is possible since DMML enables an automated motivation requirements analysis and software tool support. Facilitating an automated analysis via algorithmic investigation enables the identification of problems with the design of DM, such as conflict of interest or sabotage. This section provides an illustrative example of the use of DMML to capture DM in conjunction with the business goal model 5.1, and we also explain the use of it and our automated analysis to identify and detect potential risks of DM on the organisational goals and staff well-being in Sects. 5.2–5.6.

5.1 Illustrative example

To illustrate the capabilities of DMML, we use an example which is based on an IT department of a university, derived from publicly available documents, organisational structure and hierarchy of the university, and job descriptions related to the roles in the IT department. The motives added to the IT department do not represent the actual system and are added to the model for descriptive purposes. For the purpose of this study, only four of the jobs in the IT department are considered: help desk support analyst, help desk support supervisor, user support analyst, and user support supervisor. The following explains the description of the responsibilities of each job.

5.1.1 Help desk support analyst

A help desk support analyst’s main duty is to resolve incidents which are reported to the IT department via the website, email, or calls. They are required to maintain records of the incidents and log the person responsible for resolving the issues. Additionally, a help desk support analyst is expected to inform the users on the resolution progress of incidents via frequent updates and timely reports. They are required to assess incidents and decide whether to resolve the problem using administrative rights, remote desktop tools, or diagnostic tools, or to escalate the incident to the help desk support supervisor or to the user support analysts. In each case, the help desk support analyst is responsible for resolving the incident and needs to follow due process and inform the users of any progress. To keep user satisfaction at an acceptable level, help desk support analysts are expected to be knowledgeable, participate in regular training sessions, be aware of relevant IT policies, and be professional when dealing with users.

5.1.2 Help desk support supervisor

A help desk support supervisor’s main duty is to ensure that the help desk support analysts are performing well, they are provided with the necessary resources, and resolve the escalated incidents by either delegating them to the user support team or providing second line support. The second line support can be delivered through direct communication with the user, using remote desktop tools, diagnostic tools, or administrative rights. To ensure the integrity of the system as a whole, the help desk support supervisor is expected to escalate any issue that is identified by escalating to the correct team. To make sure that the help desk support analysts are performing well, the supervisor is expected to monitor the progress and state of the incidents by consulting the records created by the help desk support analysts. The supervisor is also responsible for keeping the help desk support analyst team resourced and up to date with new policies. This is usually achieved by recruiting new staff if required, providing training sessions, and communicating updated policies.

5.1.3 User support analyst

A user support analyst is tasked to make sure that the computer system of the organisation performs properly. As part of their daily responsibilities, they need to ensure the integrity and security of the system. Once a security incident is reported, they will need to determine its priority and react to it quickly. In case the issue occurring is classified as major, it needs escalation to a higher team. User support analysts are responsible for maintaining policies up to date by regular reviews. Participating in training sessions is also necessary to ensure their expertise is kept up to date. In addition, they are responsible for policy dissemination to other teams. Lastly, the user support analyst team is expected to provide first line support on a rotational basis. This is mainly with regard to the issues that the help desk support analysts cannot handle, but hold minor importance and do not require escalation to a higher level.

5.1.4 User support supervisor

A user support supervisor is responsible for making sure that the support analysts are performing well, and liaising with other teams to ensure the integrity of the computer system by delegating tasks when necessary. One major responsibility of the user support team is to implement and integrate projects to the current computer system which requires liaising with other managers, allocating tasks to team members, and forwarding appropriate tasks to other teams. Also, in order to ensure the satisfactory performance of the team, the supervisor is expected to communicate policies, recruit new staff when required, and provide training for staff. To ensure the integrity of the system, the supervisor is expected to respond to incidents escalated from the user support team and delegate them to the appropriate team or department. Lastly, the user support supervisor works closely with the help desk support supervisor and allocates some first line support tasks to the team members on a rotational basis. These incidents are mainly the responsibility of the help desk support analysts which they cannot solve.

5.1.5 Augmenting with DM

The management team of the university realised that the IT department lacks motivation in their work, resulting in low user satisfaction. To tackle this issue, it was decided to add DM features to the existing system with the aim of increasing employee engagement and motivation.

As a result, it was decided to provide a leaderboard to track the main task of resolving incidents raised by users. The leaderboard would be visible to all employees, with points given to employees for solving incidents and receiving satisfactory feedback from users. At the end of each month, a £25 voucher would be awarded to the employee at the top of the leaderboard. Since employees receive calls on a rotational basis, all employees would an equal number of calls, and hence, the possibility of winning would be equal to all. In addition to the leaderboard, employees who receive high satisfaction points from the users, would receive different tiers of the solver badge. The badges would be awarded to anyone who meets the criteria and they are unlimited. They, too, are visible to all employees.

The management team also decided to add a progress bar to track feedback to the users. The progress bar, visible to all employees, would help the managers and employees to keep track of the current incident resolution progress, and act accordingly. The progress bar would capture team level performance, representing the overall progress of a team, and the remaining tasks for the team as a whole. The management team hoped to create a collaborative culture by introducing the progress bar, which would motive employees can support supervisors to monitor the performance of the team.

As part of an effort to encourage employees to remain up to date with new developments and benefit from the training sessions, managers added a leaderboard for monitoring the uptake of training sessions by help desk support analysts and user support analysts. The leaderboard would display the top performers in the training sessions and use this information as the means to determine who the top performers are.

To keep track of the progress of the projects dedicated to the user support analyst team, a progress bar was also added to the tasks. This progress bar helps the managers and the employees to keep track of their progress and make decisions when necessary. The progress bar is at a team level, and only the supervisors and the team members have access to this progress bar.

A progress bar is also added to the review documentation task to provide a source of information with regard to which documents require reviewing and what the status of the progress is. Since the task is highly qualitative, the information provider is the employee, and it is used as a form of information tracker and provider for the supervisors and the employees. However, the managers will provide a badge for the active knowledge updaters according to their efforts. The badges will have tiers and levels, representing the effort required to achieve each.

The model in Fig. 3 represents the goal model for the organisational system together with the DM elements applied to its tasks and the personas associated with the actors. The model is complemented by the models presented in Figs. 4, 5 and 6 as well as Tables 5 and 6. Figure 4, similar to i* models, represents the social structure between actors. Figure 5 represents relationships between actors on the task as this is hard to visualise in classic i*, e.g. two actors collaborate on a task or compete on a task. Such information is important in DM as it would decide how rewards are allocated to performing tasks which are subject to dependencies on others. The model in Fig. 6 provides the detailed specification of the DM elements depicted in Fig. 3 according to various attributes such as chance of winning, performance metrics and reward nature. This means that two leaderboards can be configured differently, e.g. one with individual performance measurement and rewarding strategy and another with group performance measurement. Tables 5 and 6 provide information about the actual staff playing the roles and also the delegation of tasks amongst them. This helps to detect cases in which staff play more than one role and have different interdependencies according to the role they play.

In this paper, we mainly use the model as a knowledge base to enable the automated reasoning explained later in this section. The full description of the algorithms and the tools used for that can be found in [44]. Other uses of the models would be to simulate certain organisational roles aiming to detect negative scenarios where DM solutions may affect quality and well-being. The investigation of such risks and their mitigation strategies has started, and taxonomy of the risks and their mitigation strategies can be found in our work in [1, 2].

Fig. 3

DMML for IT Department

Fig. 4

DMML for actors’ relation

Fig. 5

DMML for actors’ relation on a task

5.2 Conflict of interest

Conflict of interest exists when an agent has the opportunity to hinder something in the environment in order to gain an advantage over another agent. It can occur when a competition is introduced to a certain task/goal, and there is a relation of dependency between this task/goal and another one. If one agent participating in the competition can interfere with the dependee task, such a disturbance can lower the dependee’s performance and benefit the agent in their competition. A healthy design of DM should be able to detect such conflict of interest and resolve this issue before introducing the DM to the business.

5.2.1 Description

Focusing on the help desk support analyst team and the user support analyst team, the two teams are set to compete on the task provide 1st line support, as illustrated in Fig. 5. The competition is introduced using a leaderboard (Fig. 6) with a low chance of winning, scoring calculated by humans, focusing on the individual performance of the users, and providing a high-value reward. As depicted in Fig. 3, it is also noticeable that there is a dependency link between the ensure 1st line support integrity of the user support analyst team and the resolve incident task.

Fig. 6

DMML for motives—leaderboards

The problems arising by this situation are discussed here. A leaderboard is not compatible with the collaborative environmental value of the organisation. The university wants all staff to collaborate with each other to achieve their goals. However, a leaderboard is competitive by nature, and thus not compatible with the values of the organisation. Moreover, there is a risk of conflict of interest as there exists a dependency link between the resolve incident task and ensuring 1st line support integrity. The problem is raised when an actor in the competition has control over the integrity of the work of another actor. The conflict of interest is more aggravated as there is no control over the performance of user support team on ensuring integrity goal, allowing them to hinder the task with minimum risk of getting caught.

5.3 Bribe for an exchange

A bribe for an exchange can occur when one agent can allow another to win by asking for a favour in return, or similarly when asking another agent to let them win as a favour. Bribe for an exchange can be very detrimental from a business point of view, especially in the case of quality-oriented tasks where an agent may be offered a win as a bribe. The likelihood of bribe for an exchange is more likely to occur when there is a competition in place with a high-value reward involved, and a task delegation with no reward is happening. A healthy design of DM should be able to detect the happening likelihood of this issue and introduce preventive measures.

5.3.1 Description

Keeping with the same example, a leaderboard is added to the training sessions to increase the motivation of the staff to participate in training initiatives. Training sessions take place on a regular basis and give the winners of the leaderboard social recognition. On the other hand, the user support analyst team needs to make sure that the knowledge is up to date. As illustrated in Table 5, Conor has responsibilities in both teams. Conor is behind his tasks in reviewing the documentations and asks his supervisor Kevin to delegate his share of responsibility in maintaining records to another employee in the help desk support team. Kevin agrees to this with the condition of an employee voluntarily accepting to cover for Conor. The mapping of the task delegation and the agents involved in this is presented in Table 6 in case Kieran agrees to cover for Conor.

Table 5 Mapping of actors with the agents

This scenario may seem reasonable at first glance; however, there is a risk hidden in the setting. Since there is an element of competition in the training sessions for the leaderboard, Conor may offer to let a team member win if he/she agrees to perform some of Conor’s tasks. Also, there is a risk of another team member asking for the same from Conor. In either case, the team member has an increased chance of winning in the leaderboard and may reduce the expected effort. This situation may result in a decrease in the quality of learning which is undesirable from the perspective of the management.

Table 6 DMML task delegation mapping

5.4 Free-riding

Free-riding occurs when a group member performs less than they are required to, on the assumption that others will do their job. Free-riding can increase the tension in group activities and decrease the performance of the group. It is likely to take place when there is a group activity and individual contributions are not captured by DM. In addition, it is more likely to occur if an individual agent has no genuine interest in the goal to be fulfilled. Such is the case where an agent has a task delegated to them from another group, and performing that task will offer no personal rewards to the agent.

5.4.1 Description

Continuing with the same example, a progress bar is added to the task keep users informed. The progress bar is visible by the supervisor and the team members, allowing them to track the performance towards completing a task. To encourage collaboration, the progress bar collects the performance of the group without relying on the individual performance of the employees involved in this task. Andrew from the help desk support analyst team calls in sick. Since this goal has a direct impact on user satisfaction, the managers delegate the task to Joe from user support analyst team to cover for Andrew.

Although this simple scenario may seem fine, a problem arises because the monitoring system for the progress bar relies on the group performance of the employees. With individual contributions not acknowledged, Joe may not commit fully and rely on other team members to do the task. Table 7 illustrates the mapping between the agents involved in this situation.

Table 7 Task delegation to agents mapping

5.5 Secrecy

In this context, secrecy occurs when agents find a solution which can aid in achieving the business goals with a higher quality or performance, but they decide to keep it as a secret since it provides them with work-related benefits. This is likely to happen in cases where groups or agents are competing for high-value prizes, and the sharing of information has lower value or even no value. This will prevent all the other agents and groups from using the found solution and will prevent the organisation from higher performance or quality. A design of DM should be able to detect and prevent secrecy from happening.

5.5.1 Description

In the running example, the user support analyst team and help desk support analyst team are set to compete in the 1st line support task. This is a critical task with a direct impact on user satisfaction. Hence, all teams in the IT department are involved in this task. The university values collaboration and the sharing of information between teams, which can lead to better resolving issues and in less time. It is in the interest of the organisation for its employees to be active in finding innovative and novel solutions and sharing them with other teams.

Analysis on the current setting of DM in the given environment shows that there is a risk of secrecy. The reason for this risk is the incompatibility of having a competitive element, very high-value reward, and the expectation of collaboration and information sharing. Teams may find solutions that resolve issues more efficiently, but they may opt to keep them secret to increase their chance of winning the rewards.

5.6 Workplace intimidation

Several circumstances can lead to workplace intimidation and bullying. An example of workplace intimidation is when agents with higher performance, group together and put pressure on agents with lower performance. The likelihood of workplace intimidation increases when the agents are compared with each other, or they have access to each other’s performance achievements, weaknesses, or strengths. A healthy design of DM should consider the possibility of work intimidation taking place and provide preventive measures.

5.6.1 Description

A solver badge is given to employees who solve issues effectively. To keep the badges diverse and sustain their attractiveness, they are given to specific skills employees hold and have different tiers showcasing different levels of expertise. This can also allow managers to understand the strengths of employees and assign tasks that are relevant to them and suits their level of expertise.

6 Evaluating DMML

This section aims to empirically investigate the usefulness and effectiveness of using DMML in the engineering processes of embedding digital motivation into work practices of enterprises. To achieve this, DMML is evaluated from the perspective of both experts and novice software engineers. This evaluation can help to identify the strengths of DMML and the aspects of it which require improvement. For the full description of the evaluation, please refer to [44].

6.1 Phase A: DMML evaluation with novice software engineers

In order to evaluate DMML and the reasoning framework, a two-phase study was performed, a focus group followed by lab sessions performing modelling tasks, which are described in the following subsections in details.

6.1.1 Study planning and participants

The PICOC technique was followed for this phase of the study. Table 8 describes each step of this technique in more details.

The participants of this study had to comply with the following requirements: (1) being a final-year undergraduate student in computer science; (2) having industrial work experience through work placement or internship; and (3) having knowledge of requirements engineering and requirements analysis using a goal-oriented modelling language. Choosing final-year undergraduate students can help to ensure a low deviation in participants’ knowledge. A low standard deviation means that any findings or results are not affected by a delta in the general requirements engineering knowledge of participants.

Table 8 Description of the PICOC for the evaluation of DMML—novice software system modellers’ view

6.1.2 Study design

An initial DMML training session was provided to all participants, followed by a focus group. The participants were provided with training materials and a full description of DMML. They were invited again after 2 weeks for a second training session. All training sessions including the focus group were conducted using three facilitators recording the participants’ findings. The study design is summarised in Fig. 7.

Fig. 7

Phase A: study design

6.1.3 Distributed group session

Participants were provided with five scenarios that had DM implemented in them with intentional design issues. They were asked if they could detect any issues in a 5-min time frame. Next, they were provided with a representative model of one scenario depicted using DMML and were asked if they could identify any issues using the provided models. They were given 5 min and were asked to record any issues on the answer sheets.

6.1.4 Lab sessions

In the training sessions, the participants were asked to read through a short scenario of DM implemented within a workplace. They have been invited to draw a model using i* modelling framework, and another one using DMML. Then, they were asked to analyse their models and find any issues with the design of DM in the business environment for the given scenario. Next, they were asked to read through the reasoning and re-analyse the models, for detecting any issues. Finally, they were asked to fill in the design perception questionnaire provided for them.

6.2 Data analysis

During each of the organised training sessions, the participants’ interactions with each other and the facilitators were observed and documented. Participants also filled in a questionnaire which helped to elicit their feedback with regard to various aspects of DMML, provided in the following:

• DMML learnability and understandability DMML provides more concepts and relations which need to be learnt in comparison with the i* framework. Hence, it requires higher cognitive load from its users to understand it and learn it adequately. Participants found DMML to have clear and understandable concepts, which are easy to learn. Despite the extra effort to learn additional concepts, DMML provides easier modelling of DM and the environment in comparison with classic goal modelling. The chief difficulty expressed was in the definition of motives. Participants indicated that this required them a thorough examination of the motives’ attributes in the meta-model to configure them. This task required a deeper understanding of DM and knowing the characteristics of motives jointly with human behaviour. However, with the help of facilitators, participants were successful in defining motives provided in the scenario with correct settings and values for the attributes. The added value of that is the educational power of DMML as the list of attributes act as a checklist to go through, and this helps the analysis and detects ambiguity and obscurity in the specification.

  Moreover, participants raised the question of the meaning and usage of informs links, particularly where more than one informs link from different actors could end in the same, single, motive. This ambiguity was clarified by explaining that motives rely on performance information from the tasks and all the tasks which are involved in a motive will inform that motive. Some participants stated that modelling the relations between actors, which are the supervision and promotion links may make the models less readable. Clearly, this is a potential issue, however, it was explained that the whole system does not need to be modelled in a single model, rather there can be various complementary parts to support readability.

• DMML expressiveness Several participants were able to use the i* framework to model DM in the business scenario given to them. They added motives to an environment as i* tasks. They also used the positive and negative contributions to motivation goals or soft-goals. However, they found i* to be less expressive and harder to model and analyse DM in comparison with DMML, as their models lacked relations and attributes specific to DMML, such as tasks’ measurability, subjectivity, or quality orientation. Moreover, the use of i* did not allow them to depict the wide variety of attributes of motives which define motives’ setting. The participants indicated that i* basic model fits the initial stages of modelling where major business decisions are made. DMML should be used after those decisions are made so that it does not confuse the core requirements with DM as a supplementary one.

• DMML efficiency and areas of difficulty Participants found it difficult to identify issues with the design of DM when they were given a textual scenario. However, after providing a DMML model of the scenario, some participants could detect DM-related issues. One reason for this may be the emphasis that DMML has on attributes and relations, which may be missed when reading the textual description. A number of participants enriched the reasoning already considered by the authors with new insights, such as the detection of the extra burden added by delegation and its effects on the other tasks of an agent. All participants who performed the analysis following the algorithms were able to detect the DM-related issues. When asked to model a scenario using i* modelling framework and DMML and run the reasoning, most of the participants found it difficult to use i*, stating that DMML complements i* modelling framework, making it more expressly tailored to DM and enabling more in-depth and less subjective analysis concerning DM.

• Intention to use DMML All participants were asked to model a gamified call centre as part of their training. They were invited to use the classic versions of goal modelling, e.g. i* modelling framework, and given a chance to use DMML as an auxiliary aid. At the end of the training sessions, most of the participants indicated that they intended to use DMML. The reasons provided by the participants were mainly with regard to DMML providing a detailed and specialised conceptualising of DM, such as modelling motives as a separate entity in the system, enabling different reasoning based on various characteristics of motivational techniques that would be implemented in a business.

As a general limitation, it has been noticed that DMML does not cater for the sustainability of motivation and its evolution needs. Even a successful design of DM may become outdated, and users may lose their interest in what DM offers. This problem is valid, and the evolution of DM is crucial for sustaining employees’ motivation and engagement which must be addressed in DMML’s next version.

In addition, few frequent and recurring mistakes have been identified during this phase of the evaluation, presented in Table 9. One of the frequent mistakes that the novice system modellers made was not providing the detailed settings of the motives they had provided in the model. DMML provides a UML-like class diagram which allows the definition of the motives in granular detail. However, the participants neglected to provide this information in their models. This negligence prevented full use of DMML, and limited a considerable proportion of properties for analysis.

Another frequent mistake that the participants made was mixing DMML with other modelling languages and frameworks. This mixture may not necessarily result in an incorrect model, though it is most likely to produce a disorderly model which is not easy to read and follow. Another issue was the negligence of task attributes by participants. Analysing the models produced by novice software modellers, it is noticeable that it is likely for them to ignore defining the attributes of the tasks. Lack of the attributes of the tasks prevents a complete analysis of the model, and in some cases may also result in an incorrect analysis, e.g. allowing a competitive motive for a quality-oriented task, that is hard to measure and also is subject to human judgement.

Another mistake which was observed during this phase of the study was that participants occasionally did not draw the relations between the roles. This may not have a significant impact on the analysis of the model and the design but again prevented a complete analysis. In addition, participants proposed a number of personas in their designs, but did not provide the description of the personas. Not providing the description of the personas would invalidate their use in the model.

Furthermore, it was noted that some participants used incorrect input values, damaging the consistency of the models, and preventing a proper automated analysis. Some participants did not use the proper notation. This mistake relates mainly to the visual aspect of using DMML, i.e. forgetting the “I” in the informs link from a task to a motive. This mistake will not have a significant impact on the final model as the correct notation is inferable.

The last frequent mistake that was identified during this phase of the research was the lack of instance level modelling by participants. Although the abstract level modelling of DMML allows the analysis of the model at the organisational level, providing the instance level information would allow more detailed and accurate analysis of the system as a whole. These mistakes were mainly identified by analysing the assignments that the participants have submitted as their final works.

Table 9 Frequent mistakes made by novice system modellers while using DMML

6.3 Phase B: evaluation with expert software system modellers

Despite all of these errors the evaluation clearly provided an insight into the extent to which the approach could be learned and used, by what were relatively inexperienced modellers, and showed that even this group were able to understand the advantages offered by the approach. To evaluate the usefulness and effectiveness of DMML from the perspective of experienced software system modellers, empirical qualitative research methodology approach was followed, conducting interview sessions leading to two parallel focus groups, which are described in the following subsections.

6.3.1 Study planning and participants

For planning the evaluation study for this phase, the PICOC technique [33] was employed. Table 10 describes each of the steps for this technique in details.

Table 10 Description of the PICOC for the evaluation of DMML—expert software system modellers’ view

This study required two types of participants; managers of a business and experienced software system modellers. The selection criteria for the managers was: (1) being employed by a common business; (2) having executive and decision-making responsibilities. The software system modellers were required to have: (1) a minimum of 5 years of experience in software system modelling; (2) minimum M.Sc. in computing; (3) familiarity with DM; (4) and complete understanding of the i* modelling framework.

6.3.2 Study design

To evaluate DMML, it was decided to apply DMML in a business environment which intended to integrate DM within its workplace. As a result, this study was performed in three main steps: motivation requirements elicitation, modelling and design, and manager opinion elicitation. The study design is summarised in Fig. 8. Each of these steps are described as follows:

Fig. 8

Phase B: study design

Motivation Requirements Elicitation To perform this step, a set of requirements elicitation questions were used to collect motivation requirements of the enterprise, the stakeholders, the business goals, priority of the requirements, and monetary expenses which could be used in the rewarding system. Three of the executive managers in the IT department of the business were approached for the interview sessions. All the managers held a degree in computing and had knowledge with regard to software systems modelling and software design life cycle. The privacy policies of the business environment did not allow for any video or voice recordings and the research was limited to taking notes during the interview sessions. Prior to the interview sessions, the participants were provided with a research information sheet and they signed the consent form, allowing their anonymised data to be used in this research. The interview sessions were conducted in a semi-structured manner, allowing flexibility in the order of the questions being asked and discussing the situation where necessary. The sessions were limited to 30 min each from the managerial team. To ensure the correctness and integrity of the notes taken during the interview sessions, all three participants were provided with the final notes for their approval.

Modelling Sessions To evaluate DMML, it has been decided to compare its usefulness and effectiveness in modelling and designing DM for a business with the i* modelling frameworks. To achieve this aim, ten experts in software systems modelling have been invited to take part in this research. The experts were divided into two separate groups, each performing different modelling tasks. This helped in eliminating the learning effect bias towards the latter sessions. To remove the bias in assigning the experts in the sessions, the participants were given a number and a randomised algorithm was used to create an order of these numbers. The first five numbers shaped the focus group which would work on goal-oriented requirements engineering (GORE) and specifically the i* modelling framework. The second group would use DMML as the modelling language to model and design a DM system for the intended business. All participants were provided with the research information sheet, and they provided the research with their consent to use their anonymised data. The performed sessions are described as follows:

Focus Groups—GORE

This session was focused on modelling and designing a DM for the intended business guided by the requirements document. All participants in this focus group were provided with a set of guidelines providing the common practices used in the industry 2 weeks prior to the session. After 1 week, a 2 h long tutorial session was conducted, allowing the participants to gain the same minimum level of understanding with regard to DM and ask questions related to the guideline and remove ambiguities in their understanding.

In the focus group session, all participants were provided with a set of questions which guided them through modelling the requirements document and design a DM system that could address the motivation requirements. Participants discussed this amongst themselves and modelled the given business using the i* modelling framework. Once the modelling was finished, participants started to design a DM for the given requirements document. The participants documented the model and the design of the DM during the session. In case an idea was controversial, the decision with the higher agreement was considered as the final decision. The facilitator would intervene to disambiguate the understanding from the requirements document where necessary. The session lasted for 2 h, and the final model and the design of the DM using the i* modelling framework was approved by all participants (Fig. 9).

Focus Groups—DMML

This session was focused on modelling and designing a DM for the business using the requirements document. To ensure the familiarity of all participants with DMML, the full guideline of the modelling language was provided to the participants 2 week before the focus group session. After 1 week, all participants of the DMML session were invited to take part in a 2 h long tutorial session on DMML. This session has provided the participants with opportunities to learn DMML in more details and also ask any questions with regard to the language to remove ambiguities in their understanding.

In the focus group session, the participants were provided with a set of questions which were designed to guide them through the steps required to model the requirements document and design a DM to address the set of requirements. Participants discussed amongst each other and modelled the given scenario using DMML, allowing them to start the design phase of the study. Participants were asked to use the model depicted using DMML and come up with a DM design which can address the requirements provided in the document. In case a unanimous decision could not be made, participants voted on the ideas and the idea with the higher vote would be accepted as the final answer. Where there was an ambiguity in the requirements document, the facilitator would intervene and help removing the ambiguity. The session lasted 2 h, and the final model and design of the DM for the business was approved by all participants (Figs. 10, 11).

Design Approval The managers who participated in the requirements elicitation phase were approached for their opinions on the models and the designs produced in the two focus group sessions. The models and designs from the two focus group sessions were further analysed using DMML for issues with the designs. First, managers were provided with the results from the GORE focus group and their opinions and issues with the models and design were documented. Second, the results from the DMML focus group session were provided to them and their opinions and issues with the models and design were documented. Lastly, the reasoning using the DMML was provided to them and their opinion was documented. Each session was limited to 30 min and due to the organisational privacy policies and restrictions, no video or voice recording could be performed. However, the participants notes were documented carefully and at the end of the interview sessions, the managers asked for their approval of the notes taken to ensure correctness of the elicited opinions.

6.3.3 Data analysis

During the focus group sessions, all participants’ interaction with each other and with the facilitator were observed and documented. Participants also enriched this study with their observation about modelling motivation requirements using DMML and GORE frameworks. Also, the result of the investigation was provided to the managers, and their opinion was elicited and analysed. The results of these observations and analysis are presented in the following:

GORE Session The modelling of the given environment using the i* framework was very straight forward for the modellers. They unanimously identified the actors, functional requirements, non-functional requirements, and tasks for all motivation requirements. However, one challenge to tackle for them was the addition of motives to the model. It was stated that the addition of the motives would be very descriptive, resulting in an unreadable final model, or a large supplementary document describing how each added motive should behave. The modellers stated that there should be an “easier way” to model the motives. The addition of the motives “are very descriptive and confusing at the end”. In addition, it was observed that a considerable amount of time was being spent on deciding how to depict the digital motivation solution and embedding that in the model instead of deciding the compatibility of the motive for the given situation and context. One participant mentioned that it was challenging to define the “behaviour of the motives”. There have been some discussions on possible design issues and conflicts which may be introduced by the addition of the motives to the environment. These discussions stemmed from the digital motivation tutorial, where a number of issues with regard to the introduction of digital motivation to an environment were listed.

For instance, one of the participants mentioned that involving the managers and staff in the “same leaderboard” is problematic based on the “personal experience” the participant had. However, the participant mentioned that using the i* framework, there is no easy solution to depict, represent, and detect this situation. It was also stated that despite their “feeling” of incompatibility of some of the motives with the environment, it was not clear to them that which characteristics of the motives could cause those issues and how these issues could be detected or prevented. Moreover, it was mentioned that digital motivation could be perceived differently depending on the various personality of its users. Participants agreed that “no single person” in the focus group session would have “similar preferences” with regard to the DM. As a result, a user-centred design approach should be adopted to enable the consideration of end-user preferences and “adaptability” to user requirements.

Fig. 9

Digital motivation model developed in the GORE session

DMML Session In comparison with the GORE focus group session, the modellers had to put more effort in modelling the environment as the first stage of the session. Since DMML relies on more data from the environment and the motives, hence a higher level of effort towards modelling digital motivation using DMML was expected. Moreover, despite the straightforward identification of the actors, functional requirements, non-functional requirements, tasks, and relations between the constituents in the environment, there has been less unanimity in defining the attributes of the tasks. In several instances, there was a need for voting and a collective decision-making process. Due to the qualitative nature of this process, a debate on defining a number of attributes was expected prior to the session.

Despite the higher effort and cognitive load required by DMML, richer and more in-depth discussions on finding a digital motivation solution were observed. Modelling the environment using DMML and providing the constituents of motives to the modellers allowed a richer discussion on the compatibility of each motive they planned to add to the task and environment. The quality orientation of tasks had started discussions whether pressurising motives such as “time limits” should be introduced. The measurability of tasks enabled discussions on how the points produced by DM should be calculated. Also, it was discussed whether it was wise to add competition for tasks which are challenging to measure or are subject to human judgement. Moreover, there were discussions about the visibility of each motive and the information gathered by each motive which played a critical role in deciding who the audience of a motive could be. There have been some discussions on the relevance of actors’ relations, especially the competition or collaboration relation on tasks and their compatibility with the motives added to those tasks.

It was raised during the session that it would have been useful if DMML could consider the time constraint on the tasks where relevant. Despite the presence of this element on some tasks, its consideration would not benefit the modelling of motivation. Tasks which have their time constraints at an organisational and business level should be performed within the given time regardless of digital motivation. A solution to embedding the time constraint in the design of digital motivation is to add the time constraint as an individual motive to the task, and designing it using the provided meta-model. One other participant stated that the consideration of resource dependency could be beneficial for DMML, allowing it to capture more properties.

Fig. 10

Digital motivation model developed in the DMML session—environment part

Fig. 11

Digital motivation model designed in the DMML session—actors relation

Managers Assessment Both models created in the focus group sessions were presented to the managers for their opinions to evaluate the usefulness and effectiveness of DMML. The aim of this process was to identify which model provides better analysis of the situation and leads to a better design of digital motivation. All the three managers are proficient in computer science and hold a degree in computing, with one holding an MSc in Computer Science and two holding BSc in Software Engineering.

The managers were asked to carefully study the models, provide their opinions, and see if they can find any flaws with the designs. They have been invited to choose one of the models as the candidate design for the final implementation of digital motivation in their business. Moreover, they were asked to provide improvements to the candidate model with reasons to why they made those changes. The result of this stage is as follows:

Goal Modelling for DM

All the managers found the models created using the i* framework to be useful, informative, and easy to understand. They mentioned that it divides the business goals and assigns tasks which are required to achieve the goals distinctively. The model provided a general understanding of the proposed DM system. However, the managers found it difficult to imagine the final design. It was stated that one “has to look for the designs and map them to the tasks in mind”, which requires “a lot of brain work” if the intended system is large. Moreover, it was pointed out that the motives which were added to the tasks did not provide very informative information. The first comment that was made by all managers was “how the points are calculated”, emphasising that it is difficult to quantify various educational tasks. Also, it was added that there is an excessive use of leaderboards in the design. They stated that there is not sufficient information with regard to how the leaderboards would behave in the environment. Nevertheless, considering the general understanding of a leaderboard, all the managers disagreed with this setting. They believed that leaderboards were not suitable for an educational environment. They reasoned that shifting the main focus of the environment from collaboration to competition was against the nature of an educational organisation, hindering the learning experience of the learners. This competitiveness could damage the reputation of the organisation in the long term. Another issue which was mentioned during the interview sessions was the audience for the collected information, emphasising that the learners “must not” have access to any part of the data. The managers stated that they did not find any section of the model allowing them to understand whether the students would have access to this information.

DMML Modelling for DM

Managers provided positive feedback to the additional constituents and their attributes that DMML introduces in comparison with the i* framework, believing that these additions would make the models more effective. DMML was perceived to be more flexible regarding defining the behaviour of each motive, allowing them to understand the settings of motives in detail. Also, the attributes added to the tasks were regarded positively by the managers. However, they did not agree with all assigned values of those attributes. They believed that some of the values were not correct and the values for the attributes should be decided by the Quality department of the organisation, and not the software modellers and designers. The reason behind this was the complexity of the tasks and lack of a comprehensive understanding of the software system modellers with regard to these tasks. It was noted that the marking task is not a quality task and the managers disagreed with this assignment. Nevertheless, having attributes was “helpful” in deciding whether a motive was suitable for the intended task if the values were set correctly. For instance, the managers agreed on adding a badge for the marking task. The reason for this decision was made based on the information which was available on the model, declaring that a badge would promote collaboration and not competition. Since marking is quality based, competition is “best to be avoided”. However, there was a unanimous view on the wall of shame leaderboard, stating that “marking is a very important part of every teacher’s job here, we want everyone to do this properly, no additional pressure is required. We do not want to persuade the teachers to just mark for the sake of not appearing in the leaderboard. It will create a lot of unfairness to some students.”

It was mentioned that the relations mapping of the DMML provided useful information, helping in making decisions with regard to the suitability of a motive for a task. It is not advised to involve two ’roles’ in a similar “scoreboard” if there is a hierarchy between the two. Also, another issue that the managers mentioned was that two roles should not be involved in a rewarding mechanism if one of the roles can make decisions with regard to the rewards. These two issues were pertinent to the case for the leaderboard which was designed for increasing the knowledge, and the reward assigned for learning and embedding mathematics in lessons. This issue with the design was detected using the mapping between the agents and the roles, enabling the detection of the same agent playing various roles; in this case, the head of the department role and the teacher role. One flaw with the modelling language that was detected by one of the managers was that although the modelling language clarifies how the points are calculated, it does not provide the role responsible for calculating the points, in case the points are not pre-defined. Defining the person in charge of this calculation helps to identify whether an agent is involved in the motive as well as responsible for assigning points to other agents. It was added that this is very easy to happen as educational organisations have numerous tasks which are shared amongst all staff, regardless of their role. Therefore, the presence of these mappings can help detecting and preventing settings which may cause these issues.

It was also mentioned that the use of personas would be very beneficial as the settings are very diverse, different staff may have conflicting preferences. The use of personas can help in finding the most common settings and reduce the possibility of the conflicts. The managers found the visibility attribute of the motives to be interesting, as this was deemed to be a critical aspect of integrating DM in a business. Table 11 provides a comparision between GORE and DMML according to our evaluation study.

Table 11 GORE vs. DMML for engineering digital motivation—weaknesses and strengths

In general, the managers found the model created by DMML to be more expressive, flexible, and scalable. DMML provided more details and allowed in-depth understanding of the designs. It allowed for focus on a specific area of the organisation and to make decisions for that specific area. It provided useful relations and information which allowed more, and richer, discussions, of the various options to enhance the design of the DM. It had clear characteristics and attributes which enabled a better understanding of the behaviour of the DM and allowed for better analysis and decision-making. All of these features of the DMML made it the choice of the managers for analysis purposes and decision-making. However, the i* modelling framework was of the interest of the managers as well. Although the design of the DM from the GORE focus group was not very appealing to the managers, it was stated that the use of this could be a useful tool for presenting purposes. DMML could be very large and provide much information which may not be of interest of the senior management team. DMML could be used as the design tool and enable very detailed information, and the i* modelling framework could provide a higher abstract level of the design, provided with a more general information about the design of the DM.

In the end, the managers have decided to study the design of the DM from the DMML focus group in more details and enhance it further. The enhanced design will be presented to the senior management team of the organisation for their approval. There have been a number of issues with the design which should be fixed prior to implementation. A number of tasks were given incorrect values for their quality orientation, subjectivity, or measure-ability. Also, some dependency relations were missing as a result of not considering the learners as a part of the system. It was mentioned that although students are not part of the business side of the system, however, there are some dependency relations which require them to be a part of the system.

7 Discussion

We provide a modelling language to help the automatic detection of potential issues about applying DM within an organisational information system. The models generated through DMML can be subject to other types of reasoning including their use as a basis for simulation and test cases generation. Some DM-related faults are highly subjective and require iterative feedback from the personnel in an organisation and hence our need for a mixture between automated and social validation processes. We nominate techniques like role-playing, simulation, personas and other methods used in usability and human factors to augment the requirements engineering practices for DM. These techniques might be applied iteratively to detect emerging issues and be seen as risk assessment and resolution methods [1, 2].

DMML models are the basis for automated reasoning to detect properties like a bribe for exchange and workplace intimidation described earlier and implemented in [44]. We note here that the construction of the DMML models can also be an expensive process. For example, the assessment of the weight of a persona playing a certain role in the Goal Model can require surveying the staff playing that role within the organisation. However, as noted in the previous section, personnel who participated in our evaluation found the process of constructing the model itself useful to have a richer discussion around DM and its managed introduction to the workspace, and this would be seen as an added value.

The automated reasoning proposed in this paper is mainly meant for the detection of potential issues. Since it is about human behaviour and motivation, we emphasise that the algorithms are meant to generate cases for further checking, perhaps done through additional focus groups and role-playing with staff representing the roles captured in the model and relevant to the conflict and issue detected. Furthermore, our automated reasoning would need further extension to propose alternatives when an issue is detected. For example, when a reward can facilitate a bribe for exchange, a contingency plan would be to assure that the two personnel involved would not be allocated the reward simultaneously to minimise the effect. When a potential social loafing is detected, the algorithm may suggest additional rewards for the individual performance, e.g. a leaderboard, in a way which also avoids creating other consequences such as work intimidation, e.g. through semi-anonymous settings.

As demonstrated in Sect. 2, approaches to build DM solutions are mainly driven by game design methods. DM was indeed used in the literature of Requirements Engineering but mainly to promote and strengthen activities like requirements elicitation and stakeholders involvement. To the best of our knowledge, DMML is the first modelling effort which treats motivation as a special kind of requirements and delves into the details of the relationship between DM and the rest of the system constituents such as goals, social dependencies and tasks. Our evaluation demonstrated that such an extension, despite the need for a considerable effort, helped the various stakeholders to have a richer discussion around DM and its introduction and that they found it more natural to depict motivational requirements than GORE. The properties provided and the algorithms automating them proved to be also useful for large models. In future work, we will work more on the automation part so that the construction process and error detection are assisted progressively in minimising effort and increasing efficiency.

While DDML can be seen as an extension to GORE to tackle the peculiarities of motivational requirements, its constructs can be potentially applied widely. For example, the concept of personas and the meta-model of motives and actors relationship couple be replicated to model DM in modelling languages meant for depicting business processes. Indeed, the time aspect in languages like Business Process Modelling Notation [4] would allow additional properties to be detected noting that some conflicts may arise amongst roles due to the simultaneous application of rewards. We leave that investigation and development for a future work.

Our work in this paper is meant to facilitate the specification of DM elements and relate them to other business requirements and the detection of their potential risks on the business goals and staff relations and well-being. We made the argument that DM can be counterproductive and a risk assessment has to be conducted to detect its side effects and we proposed a set of mitigation strategies in our work in [3]. Our future work will augment DMML with a further layer of recommendation which will enrich the fault detection with solutions proposals. For example, strategies like external auditing can fit risks of the type of social loafing and diffusion of responsibility when DM uses a collective performance measure.

7.1 Threats to validity

It can be argued that the use of students as the participants in the first phase of our evaluation may not lead to a perfect outcome. While we recognise this as a threat to validity, it has been shown in [22, 50] that there is only a minor difference in the result of using students for software engineering studies in comparison with professional software developers. Another threat to validity would relate to the limitation in covering a wide range of scenarios due to time and resource constraints and to avoid participants fatigue. Such diversity could have revealed additional problems of adequacy and expressiveness. To address this limitation, we have written and provided the scenarios in a flexible style and encouraged participants to create their own variations of them when needed. Also, participants were instructed to share experiences and thoughts with others only after they did their individual thinking allowing different aspects, interpretation and cases of the same scenario to emerge. Measuring learnability and efficiency in depth would need dedicated studies for a relatively extended period. Our evaluation would be seen as an initial proof of concept rather than a confirmation of DMML quality in that aspect.

8 Conclusion

In this paper, we argued the need for a specialised and systematic development of DM in businesses. Lack of such approaches may lead to incompatible DM designs, imposing detrimental outcomes. This research builds on previous study findings and proposes a language for DM based on shared concepts with the i* modelling framework. DMML considers the organisational structure, human factors, and business activities within a business to be core aspects necessary for modelling DM. DMML provides a meta-model and graphical notations which enables modelling and automated analysis of DM, which can be used to detect conflicts and issues in the design, helping in better management of DM. We evaluated DMML and concluded that DMML is reasonably easy to learn and understand for novice modellers, and complements the i*, by providing a more expressive and specialised modelling of DM and enabling, therefore, more specialised reasoning about DM and its risks.

It is known that humans tend to demand more rewards over time to keep improving the work and maintain the quality of their performance. Also, group dynamics can play an essential role in the acceptance of feedback and rewards, e.g. the existence of dominant characters and the social norms of the organisation [11]. Cultural dimensions [21, 43] especially those related to collectivism and masculinity play a role in group work and performance and, consequently, their acceptance of the individual rewarding system and its competitive nature and chance of winning. Hence, we would expect our reasoning to need further contextualisation to the nature of the environment and its culture and norms. Personality traits [6] and personality tests would also help that contextualisation and personalisation process. In DMML underpinning research, we have provided a set of personals and personality identifiers concerning people acceptance of DM [45].

Our future work will focus on the method of eliciting and validating motivational requirements within a particular business. This includes the risk assessment process for DM solutions and their chance of occurrence and severity. Our proposed reasoning will be augmented to become more probabilistic and predictive. Eliciting the right DM for a particular context would need advanced techniques which go beyond the classic methods of requirements elicitations mainly because of the volatile nature of motivation and the difficulty of speculating that. Techniques like motivational interviewing [30] and goal setting [26] would inspire our future solutions.