Model-Based and Agile Development

Abstract

This chapter introduces the main concepts of the software development lifecycle, and describes software specification techniques and development approaches which can be used for financial applications.

This chapter introduces the main concepts of the software development lifecycle, and describes software specification techniques and development approaches which can be used for financial applications:

• Software modelling using UML

• Model-based development (MBD)

• Domain-specific modelling

• Agile development methods: Scrum, Kanban and XP.

The chapter will give a non-technical overview of these topics, and they will be developed in more detail in subsequent chapters.

3.1 The Software Development Lifecycle

The following activities take place in any software development process, whether organised into strict sequential stages (as with the classical ‘waterfall’ process) or iterated in repeated cycles (as in agile methods such as Scrum). The software lifecycle stages are:

• Feasibility analysis

• Requirements analysis

• Specification

• Design

• Implementation

• Testing

• Maintenance and Decommissioning.

3.1.1 Feasibility Analysis

This stage asks whether there is a business case for the system and if it will actually be used. It considers (i) Technical feasibility—is it possible to build the system with the available technology? (ii) Financial feasibility—can it be developed with the available budget? (iii) Time—is it possible to develop in a useful time-frame? (iv) Resources—are the necessary resources (people, tools) available for the development?

Fig. 3.1

‘Onion model’ of stakeholders [1]

3.1.2 Requirements Analysis

In this stage the requirements analyst:

• Identifies the stakeholders of the system: these may be customers, users, regulators, or anyone with an interest in or impacted by the system. Some stakeholders are more directly involved in the system than others—we can represent this situation by the ‘onion model’ of different categories of stakeholders (Fig. 3.1).

• Systematically identifies and records the requirements of stakeholders of the system, and constraints imposed on the system by existing systems it operates with, or by existing work practices and regulations.

Requirements divide into functional requirements (the services and functions provided by the system) and non-functional requirements (its efficiency, usability, extensibility, etc). There may be conflicts between different requirements and ambiguities in the informal requirements. Such conflicts and ambiguities should be resolved before the definitive specification is constructed.

For example, we may have an accuracy requirement for a financial computation, and a timeliness requirement, and these may be in conflict: it may be necessary to settle for lower accuracy in order to achieve the timeliness requirement, if this is the more critical requirement.

There are four main phases in requirements analysis:

• Domain analysis and requirements elicitation: identify stakeholders, gather information on domain and requirements from users, customers, and from other stakeholders and sources. Classify requirements as functional or non-functional, and decompose requirements into sub-requirements using a notation such as SysML [2].

• Evaluation and negotiation: identify conflicts, imprecision, omissions, and redundancies in requirements; consult and negotiate with stakeholders to agree resolutions to these issues.

• Specification and documentation: systematically document the requirements as a system specification, in a precise notation (which is not necessarily machine-readable): this specification represents an agreement between developers and stakeholders on what will be delivered.

• Validation and verification: check the formalised requirements for consistency, completeness and correctness wrt stakeholder requirements.

For example, in the case of a financial system the initial requirements specification could be expressed in terms of formulae defining the required properties in mathematical notation.

There are many possible requirements elicitation techniques that can be used to obtain requirements from stakeholders:

• Interviews with stakeholders

• Brainstorming sessions

• Observation of existing processes/work practices

• Scenario analysis—model specific scenarios of use of the system, e.g., as UML sequence diagrams

• Document mining

• Goal decomposition

• Exploratory prototyping.

Thorough requirements analysis can reduce errors and costs later in a development.

3.1.3 Specification

Based on the requirements specification, a detailed machine-readable model of a system is constructed, using graphical or textual notations to define required data and behaviour in an explicit but platform-independent manner. UML notations such as class diagrams and OCL can be used to define the model. It is important to avoid implementation details, in order that the specification can be translated into a wide range of different implementation platforms. Only sufficient detail should be included to specify the logical properties of the system.

Validation and verification techniques include inspection of the specification text, including formal reviews, structured walkthroughs of its behaviour in particular scenarios, and execution/testing, in the case of an executable specification language.

3.1.4 Design

Based on the specification, the design defines an architecture and structure for the system, dividing it into subsystems/modules responsible for parts of the system functionality and data. The design process includes:

• 1. Architectural design: define the global architecture of system, as a set of major subsystems, and the dependencies between them.

• 2. Subsystem design: decompose the global subsystems into smaller subsystems. Continue until clearly identified modules are obtained (subsystems which cannot be further divided).

• 3. Module design: define each module, in terms of:

  • a. the data it encapsulates—attributes/associations;

  • b. the operations it provides (external services)—e.g.: their names, input and output data, and specifications. This is called the interface of the module.

• 4. Detailed design: for each operation of a module, identify the steps of its processing.

The structure of a design may evolve as experience with prototypes of the system grows, for example.

3.1.5 Implementation

In this stage code is produced from the design in one or more programming languages. Traditionally, this is done manually by programmers. In model-driven engineering (MDE)/model-based development (MBD) approaches, code production is automated. Automated coding potentially reduces implementation time and cost, but problems can arise if the generated code needs to be manually adapted. The code-generation process should preserve the semantics of the specification, so that the code is correct-by-construction (assuming that the specification is correct). Likewise, the code generation should not introduce additional quality flaws such as duplicated code or excessive numbers of operations in a class.

3.1.6 Testing

The aim of testing is to discover errors in a system (before the customers or users discover them). Testing can be either white-box: based on the internal code structure, and designed to test each program path; or black-box: based on requirements independently of code structure.

Testing can be applied at several levels:

• Code/Unit testing: testing of each component separately (this is mainly white-box)

• Integration testing: test that components interact correctly

• System testing: test entire system

• Acceptance tests: test the system against requirements (mainly black-box).

3.1.7 Maintenance

This includes all post-delivery activities, including:

• Correction: bug-fixing and correcting defects.

• Adaption: changing the system to operate in a new or updated environment.

• Prevention: refactoring and reorganisation of a system to improve the structure of the system and facilitate its future evolution.

• Enhancement: extension of the system to handle new requirements.

• Decommissioning: ensure use of the system is phased-out in a controlled manner, and securely dispose of data held by the system.

These activities can consume far more resources than the development of entirely new systems.

3.2 Software Modelling Using UML

The Unified Modelling Language (UML), was introduced in 1997 as a unification of different object-oriented modelling approaches and methods such as OMT and the Booch method. By this time, the large number of different modelling notations was becoming an obstacle to the use of object-oriented methods, and the leading experts and companies in the field agreed to collaborate to produce a single authoritative approach. The UML is now an international standard controlled by the OMG industry consortium (including most leading software companies such as IBM, Microsoft, Oracle, etc): www.omg.org/uml. UML has become the most widely-used modelling notation in industry, and many hundreds of tools and books have been produced for UML. The language has been through two main versions 1.* and 2.*, of which version 2.* is a major extension of 1.* with richer modelling notations. Specialised versions of UML, such as Foundational UML (fUML) have also been developed, to support executable modelling or modelling in specialised domains.

The main motivations for UML modelling are:

• to precisely define the requirements of a software application, in a System Requirements Specification (SRS);

• as a tool for thought, to work towards an understanding of a system and to facilitate communication between developers and stakeholders;

• to define reusable models capturing domain concepts;

• to support model-based development, including code generation from models.

The main notations of UML are: class diagrams, use case diagrams, state machines, sequence diagrams, Object Constraint Language (OCL), activities, and deployment diagrams. Surveys of users have shown that these notations are the most frequently used, and particularly class diagrams, use case diagrams and sequence diagrams.

Class diagrams are the primary notation of UML, and define the data of an application: information that it processes or is aware of, and the structure of this data and its internal relationships. Figure 3.2 shows an example class diagram.

Other UML notations include:

• Use case diagrams: these describe the functionalities of an application from the user perspective. They show the functional services provided, and link these to the actors (roles of users) who may interact with the services.

• State machines: these define the life histories of objects in terms of states and events and the transitions that events produce between the states of objects. They can also be used to define the stages of execution of an operation or of use cases.

• Sequence diagrams: these show examples of interactions (communications) between objects, and between users and the system.

• Object Constraint Language (OCL): a textual specification notation that can be used to precisely define operations and use cases, and class invariants.

• Activity diagrams/textual activity notation: these show the compositions of actions as sequences, choices, iterations, etc., to form operation or use case behaviours.

• Deployment diagrams: these show physical configurations of devices, communication links, and the distribution of software across devices.

Different models are emphasised in different domains. Interaction diagrams are used particularly in telecoms system specification, and for embedded systems. Activities can show workflows for business processes. Class diagrams and use case diagrams are widely used across many application domains. In the financial domain, class diagrams can be used to specify the financial concepts and their inter-relations, as in Fig. 3.2. OCL can be used to give precise functional specifications of operations, corresponding to function definitions in Excel/VBA. Activities can show the algorithms and procedures used for particular financial processes.

3.2.1 Class Diagrams

Figure 3.2 shows an example of a class diagram, for a conceptual model of derivative securities.

Fig. 3.2

Example UML class diagram

Class diagrams show the entity types of system as class rectangles (e.g., Investor) enclosing their internal data, or attributes. For example, each Investor has a name, which is a string, and each Asset has a price, name, dividendRate and volatility. This defines the structure of the application data, which can then be relied upon by application processing (e.g., the asset data can be used in calculations of the price of a derivative security based upon the asset). Relationships are shown as lines between classes: for example, any DerivativeSecurity has a specific underlyingAsset, which is an Asset. Relationships (associations) normally have two ends, each end may have a rolename to indicate the meaning of that end, and a multiplicity to indicate how many objects can be related: for example any number (the * symbol) of derivative securities may depend upon the same underlying asset. In addition, one investor may have any number of contracts, involving different (or the same) assets.

Specialisations between one entity and another are shown using an inheritance arrow (with an unfilled triangle at the superclass/generalisation end). For example, Commodity is a special case of Asset, and PutOption and CallOption are two alternative specialisations of Option. Subclasses inherit all the features of their superclasses, thus a PutOption has attributes of maturity, maturityPrice, volatility, price, etc (but not amount or quality, which are specific to Commodity objects).

In cases where a small number of alternative values are needed, an enumerated type or enumeration can be used. For example, OptionKind declares that there are two kinds of option, european or american. This type could be used to declare an attribute kind : OptionKind of the Option class. Further class diagram elements are operations, which can define query or update processing on individual classes, and specialised forms of relationship, such as ordered associations and aggregations (whole/part relationships).

3.2.2 Use Case Diagrams

Use cases are used during requirements analysis of a system, to identify the ways in which the system is intended to be used, and the services it is expected to provide. A use case expresses a particular unit of an application’s behaviour, encompassing a family of similar scenarios of use of the system. The users of the system are also shown, they are grouped into actors and linked to the use cases that they have the capability/authorisation to perform.

For example, for a derivative securities trading application based on Fig. 3.2, there could be use cases to buy a futures contract, to sell a futures contract and to buy and sell call options (Fig. 3.3). The actors involved in the use cases are shown as stick figures linked to their use cases. Each use case in this example involves interaction with a trader who wants to trade the asset, and with an exchange which lists these assets as tradeable.

Fig. 3.3

Use case diagram example

Typically, the use cases depend upon the class diagram data and involve processing upon this data, using the data directly or by means of operations of the classes.

3.2.3 State Machines

State machines define the dynamic behaviour of objects and operations, they can also be used to specify operation or use case behaviour as a series of steps, and to express life histories of objects.

For example, the procedure to obtain a yield curve from bond market data could be specified as in Fig. 3.4. From the initial state (shown as a black circle) the process moves through states (processing stages) of obtaining market data, converting this data to zero-coupon bond data, then using an optimisation procedure to fit a Nelson–Siegal curve to the data, and finally extracting the parameters of this curve. The termination state is shown as a ‘bullseye’ symbol; this marks the termination of the process.

Fig. 3.4

Example state machine diagram

3.2.4 Interactions

Interactions, expressed as UML sequence diagrams, describe examples of system behaviour in terms of object communications. The diagrams show object instances obj of classes Entity as vertical lifelines, and the messages exchanged between objects as arrows between lifelines. Time increases from the top to the bottom of the diagram. Figure 3.5 shows an example interaction, in which an Application object a requests a FIXEngine object f to send an order request to set up a futures contract.

Fig. 3.5

Example interaction diagram

The sequence diagram shows an example (a scenario) of execution of the buy futures contract use case of Fig. 3.3, and details of how this use case is carried out within the system. The vertical lines show the timelines of particular objects, with time increasing from the top to the bottom of the lines. Boxes on the lines represent operation executions. An arrow with an open arrowhead, as in the example, represents an asynchronous message from one object to another: the caller does not wait for the called object to respond. An arrow with a filled arrowhead represents a synchronous message: the caller’s flow of control is suspended until the called operation is completed.

Unlike class diagrams and formally-specified use case diagrams, interaction diagrams do not provide sufficient information to support code generation from the models, they are instead used to help the developers understand the required behaviour of the system. In our development process they are an optional model.

3.2.5 Object Constraint Language (OCL)

The OCL was added to UML in order to provide more precise textual specifications to describe UML elements such as operations, classes and use cases. OCL includes data types of integers and real numbers, and collection data such as sequences and sets. The usual programming language expressions such as $a = b$, $a < b$, $a ~{/\;{=}}~ b$, $a*b$, a.pow(b), etc are supported, in addition various operators on objects and collections are defined, which provide a powerful facility to specify behaviours declaratively:

• Navigation through a class diagram using the x.f operator to refer to feature f of object or collection x.

  For example, in the model of Fig. 3.2, inv.contracts is the set of contracts of an investor inv : Investor, and d.underlyingAsset.price is the price of the underlying asset of a derivative security d.

• $s{\rightarrow }collect( x\, | \, e )$ is the sequence of values of e for objects x in a collection s.

  For example, $inv.contracts{\rightarrow }collect( c\,| \,c.holding.price )$ is the sequence of prices of the assets held via the contracts of investor inv.

• $s{\rightarrow }select$ $( x\, | \,P )$ is the subcollection of collection s that consists of the x : s that satisfy P.

• $s{\rightarrow }reject( x\, | \,P )$ is the subcollection of collection s that consists of the x : s that do not satisfy P.

• $s{\rightarrow }sum()$ is the sum of the values in a non-empty collection s of numbers or strings.

• $s{\rightarrow }prd()$ is the product of the values in a non-empty collection s of numbers.

• $s{\rightarrow }sortedBy(e)$ produces a copy of s sorted in ascending order of the e-values of its elements.

These and other operators correspond to programming library operations, as in the C++ Standard Template Library (STL): $s{\rightarrow }sum()$ corresponds to std::accumulate(s.begin(), s.end(), 0) in C++ STL. $s{\rightarrow }collect(e)$ corresponds to std::transform(s.begin(), s.end(), result.begin(), e).

The operators can be combined to define the effect of operations in a class diagram, or of use cases. For example, an operation to compute a sum of squared values could be:

3.2.6 Activity Diagrams

Activity diagrams provide a means to describe behaviours that are composed of collections of tasks (such as the algorithms of operations, or the workflows of business processes), in a graphical manner. They consist of:

Activities:

An activity is the specification of behaviour as the coordinated sequencing of subordinate units whose individual elements are actions.

Actions:

An action represents a single step within an activity, that is, one that is not further decomposed within the activity. An action may be complex in its effect and not atomic.

Activities are generalisations of sequential programming constructs such as sequencing, conditionals and loops.

Activity diagrams show:

• Actions (as state boxes)

• Arrowed lines denoting control flows (sequencing of actions)

• Conditional choice point branching and joining (diamonds)

• Parallel flows (starting and ending at vertical bars).

Figure 3.6 summarises these notations. Parallel flows mean that multiple separate threads of control execute together, whilst in a conditional choice only one flow of control is taken.

Structured activities can also be expressed as pseudocode using generic program statements:

• variable  :=  expression

• if E then S1 else S2

• while E do S

• S1 ; S2

• for x : s do S

• S1 parallel S2

• var x : T

• execute E

where S, S1 and S2 are statements, and E is an expression.

Fig. 3.6

Activity diagram notations

For example, the create yield curve workflow can be written more explicitly as the pseudocode statements:

Activities are the central modelling notation for behaviours in executable UML languages such as fUML [3]. However, activities and pseudocode can be too procedural and low-level to be used for specification, and it is preferable to use OCL constraints for the declarative description of a system where possible.

3.3 Model-Based Development (MBD)

MBD is a form of software development which uses UML or another modelling language to develop systems based on models, instead of code. In MDD (Model-driven development), and MDA (Model-driven architecture) models are the primary artifact in software development, with code as secondary. Typically, executable code is automatically generated from models. MBD has the benefits that models are simpler to review and modify than code, and automated code generation can raise productivity substantially. Model repositories and libraries can be established for the rapid production of systems in a common ‘product line’ family. MBD may also reduce the need for outsourcing. MBD has been widely used in specialised industry sectors (such as automobile systems), with generally positive results [4]. Typically a domain specific language (DSL) is defined for some specific application or technical domain, and used to write models for that domain. Tools specialised for the DSL can then generate code from the DSL models, or perform analysis of the DSL models. Negative aspects of MBD are the training and adoption costs needed, and the generally poor level of tool support.

3.3.1 Models and Metamodels

Class diagrams and other visual and textual descriptions of a system are termed models of the system; they define the required and expected properties of the system using the graphical/textual notations. Such models are at a higher abstraction level than program code (in, for example, Java or ANSI C): they abstract away details of how data is arranged in computer memory or how iterations through data collections are to be performed.

At the specification level an expression such as

$$ inv.contracts{\rightarrow }collect( c \,| \,c.holding.price ) $$

can be written, without any details of how this sequence of values is to be computed or represented in a program. In a Java implementation for example, any sequential data structure such as a Vector or ArrayList could be used to implement the expression. The specification avoids fixing such a particular data structure representation, and hence permits a wide range of possible implementations in many different programming languages.

Models such as Fig. 3.2 correspond to programs in Java, C#, Python, etc: the class diagram classes will usually be represented by classes in these languages (or by structs in C), their features by fields/instance variables, and inheritance by an inheritance mechanism in these languages.

Class diagrams can also describe languages themselves, e.g., a language of activities (Fig. 3.7). Such diagrams are termed metamodels: they are models defining the permitted structure of other models, in this case of the activity language statements. Metamodels can be defined to represent software languages such as Java, C, Python etc, as well as the entirety of UML itself.

Fig. 3.7

Activity language as a metamodel

3.3.2 Model Transformations

Model transformations are procedures which map one model into another model, or into text. An example is a code generator transformation which takes as input a class diagram model, and produces as output Java code. Other forms of transformation include refactorings, to restructure a model in-place, or migrations, to map a model in one modelling language to a corresponding model in a related modelling language. In this book we will be mainly concerned with the use of transformations, rather than their definition, but the UML notations of class diagrams (used as metamodels) and use cases can also be used to specify model transformations in addition to conventional software applications.

3.3.3 Domain-Specific Modelling

A domain-specific language (DSL) is a notation (graphical, textual or a combination), together with a precise definition of the notation, intended to represent concepts and elements of specific application domains. A DSL has:

• An identified application domain where it will be used.

• A set of concepts with properties and relationships, forming the ontology of the domain.

• Representations that are appropriate, visually and conceptually, for modellers in the domain, and for stakeholders who need to review such models. This is termed the concrete syntax of the DSL.

• A precise abstract grammar, defining the way DSL elements can be combined. This is called the abstract syntax of the DSL. It can be defined by a metamodel, e.g., Fig. 3.2 considered as a metamodel. This fixes the structure of models of the DSL.

• A precise way, usually by means of model transformations or templates, to map DSL models to implementations.

DSLs encode domain knowledge about a specific technical or application domain, and the DSL transformations encode knowledge and expertise about the implementation of the domain concepts in code. Domains are typically quite narrow, such as a domain for derivative security pricing. The restriction of the domain enables more focussed and accurate modelling, and specifically optimised code to be generated, however this also means that multiple DSLs may need to be constructed for a single application.

3.3.4 The State of Practice of MBD in Industry

The research of [4] surveyed over 450 practitioners, and found that some MBD is used across many sectors, including finance. But it is usually used selectively, for parts of systems. Commonly, a DSL is built for a narrow domain (a family of closely-related applications), together with code-generators and templates for this domain. This automates the previously manual coding of applications in the domain. Productivity increases of 20–30% were obtained from code generation. It was concluded that MBD can reduce effort in developing and maintaining software—enabling businesses to concentrate on their core business, not IT. Numerous companies reported that they reduced offshoring, as MBD automated work that would previously have been outsourced.

The survey also found that there were barriers to MBD adoption, particularly from middle-managers and from code gurus (who may oppose MBD because of their fear of their skills becoming redundant). To be successful, MBD projects need at least one ‘MBD guru’ in a development team. MBD seems more appropriate for specific domains, not for general purpose software.

Some examples of substantial MBD projects include:

• SunGard Financial Systems carried out a MBD modernisation project of Front Arena/AMS (an application concerned with the management of trades and orders). This project used MDA and Scrum—but with XML, not UML [5].

• Motorola use the MDD-SLAP process for telecomms systems development [6].

• Tata Consultancy Services defined the MasterCraft toolset for business systems development, using MBD and UML [7].

• Volvo Cars use a systematic MBD process for vehicle software construction, using Simulink models of vehicle components [8].

Companies such as Volvo and Tata have invested heavily in MBD and have used it in many projects. Microsoft have used an MBD approach called Software Factories, for software product line development using DSLs [9].

3.3.5 MBD Using UML-RSDS

UML-RSDS is a lightweight MBD approach, in which platform-independent specifications are defined using UML class diagrams and use cases. These are analysed for internal quality and correctness. Platform-independent designs are then synthesised in a mainly automated manner from the specifications. From the designs, executable code in an OO programming language (currently, Java, C# or C++) or in C or Python can then be mainly automatically synthesised.

This approach is more restricted in scope than more elaborate MBD approaches such as MDA, but it is more automated (Fig. 3.8). Thus it provides greater agility and ability to respond to changes. In contrast to fUML, applications are defined in terms of declarative use case and operation specifications, instead of detailed procedural activities. Thus specification construction and modification is simpler and less costly.

Fig. 3.8

UML-RSDS software production process

DSLs can be defined in UML-RSDS using class diagram metamodels to define the domain ontology and DSL abstract syntax. A simple concrete syntax is used, based on OCL, and transformations defined using UML-RSDS can be used to map DSL models to code or to other representations.

3.4 Agile Development Methods

The concept of agile software development was introduced c. 2000 to correct drawbacks of conventional plan-based development of software, which was viewed as a heavyweight and inflexible process, unable to respond to changing requirements [10]. Conventional plan-based development emphasises the gathering and formalising of all requirements before starting coding, in contrast, agile development emphasises incremental work on requirements, coding, and integration in short cycles. Thus it can be responsive to changing requirements.

In conventional development, the development team is often isolated from stakeholders: this results in delays in obtaining information or feedback. In contrast, agile development emphasises close collaboration between the team and stakeholders.

The key principles of agile development include (agilemanifesto.org): to satisfy the customer through early and continuous software delivery; to welcome changing requirements; to deliver working software frequently (every 2 weeks to every 2 months); business people and developers to work together daily; rely on face-to-face communication to convey information; continuous attention to software quality; simplicity is essential.

Agile development is now widely adopted in industry. The main approaches are eXtreme Programming (XP) [10, 11], Kanban and Scrum [12].

3.4.1 Agile Development Techniques

Agile techniques include:

• Iterations or sprints: development work which implements specific user requirements, in a short time frame to produce new releases.

• Refactoring: Regularly restructure code to improve it, to remove redundancy and other flaws [13].

Figure 3.9 shows the agile development process used in the Scrum method, with a product backlog of work items to be worked on in the project, a subset selected for work in the current iteration, the sprint backlog, and an iteration or sprint which implements these work items to produce a deliverable increment of the software. The iteration should not normally last longer than one month, and a daily review cycle is maintained via ‘daily scums’ or ‘standup meetings’ of the team.

Fig. 3.9

Scrum process

Sprints are regular re-occuring iterations in which project work is completed. These produce deliverables that contribute to the overall project and yield an increment of the system. Each iteration involves a set of work items or tasks (‘user stories’ in Scrum) to be implemented. Tasks can be classified by their business value to the customer (high, medium, low), and by the development risk or the development effort. High priority and high risk tasks should be dealt with first. The project velocity is the amount of developer-time available per iteration. Taking these factors into account, it is possible to define an initial release plan: identifying which tasks will be delivered by which iteration and by which developers. This plan will be revised as development proceeds.

3.4.2 Agile Methods: Scrum

The Scrum method is now the most widely-used agile approach. It involves the following key elements:

• User stories: requirements expressed in terms of capabilities needed by users of the system in specific roles. The general template for a user story is

  “As a [user role], I want to [goal], so that I can [achieve business value]”

  E.g., “As a customer of the bank, I want to view the balances of my bank accounts, so that I can manage my finances”. User stories are essentially use cases, and can be decomposed into subtasks/subcases.

• Product Backlog: an ordered (in terms of priority) list of user stories/tasks relevant to the project.

• Sprint Backlog: an ordered list of user stories/tasks to be completed in a sprint.

• Sprint planning: performed by the Scrum team before a sprint, the team agrees the tasks to be worked on in the sprint, that is, the subset of the project backlog to include in the current sprint backlog. The duration of this meeting is normally 2 h per each week of the sprint.

• Daily scrum: this daily meeting organises the activities of the team, reviews sprint progress, and identifies issues. It is time limited (e.g., 15 min) and often takes place at the start of each day. It raises the key questions for developers: (i) what did I achieve yesterday? (ii) what do I plan to achieve today? (iii) is there anything blocking me from achieving my work? The meeting is also called the ‘daily standup’ meeting. It is not for detailed discussion, problems identified should be dealt with by a designated person following the meeting.

• Definition of done: a criteria to state when a task is completed. For example, that the testing process for it has been completed and all bugs detected have been fixed.

• Sprint review: a review conducted by the team at the end of the sprint. It identifies what work was completed and planned work that was not completed, demonstrates completed work to the stakeholders, and involves collaboration with the stakeholders to decide what should be in the following programme of work—for example in which iteration remaining uncompleted tasks should be completed. A review is recommended to last 1 hour for each week of the sprint it reviews.

• Sprint retrospective: after the sprint review, before the next sprint planning. It analyses the achievements of the sprint, and considers ideas for improvement of the development process. It is facilitated by the Scrum master and has a duration of approximately 1.5 h for each 2 weeks of the sprint.

Additional optional elements, such as an impediment backlog can also be included.

During a sprint, the team uses aScrum board showing the tasks to do, in progress and completed (for example, Table 3.1).

Table 3.1 Schematic Scrum board structure

A Burndown Chart shows a graph of the estimated remaining work against time.

The key roles of the Scrum team members are:

• Product owner: this team member is a stakeholder representative in the team, and is responsible for liaising between the technical staff and the stakeholders. The product owner identifies required work items, identifies their priority, and adds these to the product backlog.

• Development team: the workers who perform the technical work. The team should have all needed skills and be self-organising. Typically there will be between 3 to 9 team members in this role.

• Scrum master: the Scrum master facilitates the Scrum process and events, and the self-organisation of the team. This role is not a project manager role and does not have personnel management responsibilities. They ensure that the Scrum process is correctly followed in the development.

3.4.3 Agile Methods: Extreme Programming (XP)

XP is an alternative agile approach, with a strong focus on the coding activity, in contrast to Scrum, which emphasises the organisation of the development team and process.

XP advocates techniques such as pair programming, where two programmers work at one terminal, one having the role of reviewing the code of the other. Code refactoring is also a key process in XP, to achieve the agile principle of “continuous attention to software quality”.

XP consists of:

• 5 Values: communication, simplicity, feedback, courage, respect.

• 3 Principles: Feedback—via customer interaction and fine-grained testing; Assuming simplicity—code that’s just good enough, small changes; Embracing change.

• 12 Practices: pair programming; planning game; test-driven development; whole team; refactoring; small releases; system metaphor; simple design; continuous integration; collective code ownership; sustainable pace; coding standards.

XP has been found more appropriate for small teams, including single-programmer developments.

3.4.4 Agile Methods: Kanban

Kanban is a general manufacturing concept originating in the Japanese car industry: the principle is only to produce what is needed when it is needed, not before; demand is tracked through all production stages in order to manufacture to meet demand. Adapted to software, it is an agile approach oriented to a continuous delivery software production process. The approach uses demand-led production, the demand (such as customer or user requests) determines the priority for work items.

An important principle is to limit the work-in-progress (WIP): developers work only on few tasks—often just one—at a time. When the current task is finished, they then start on the next highest priority task from the backlog.

Table 3.2 compares Kanban and Scrum.

Table 3.2 Kanban compared to Scrum

Kanban uses Kanban boards, which are similar to Scrum boards, and may have columns such as Backlog (tasks to be done next); In Development; Testing; Customer Acceptance; Done.

With Kanban it is possible to have separate teams for separate development stages (e.g., a specification and design team, and a coding and delivery team). The output of one team is fed into the backlog of the next.

3.4.5 Benefits and Disadvantages of Agile Development

The State of Agile Survey of the software industry (versionone.com, 2017) has consistently identified that the main benefits of agile development are: (i) the ability to manage changing priorities; (ii) increased team productivity; (iii) improved project visibility.

The majority of survey respondents found that agile projects were mainly successful. Scrum is consistently the main method used, with iteration planning, daily standups, retrospectives, reviews, short iterations, release planning the most common techniques.

The following disadvantages of agile development have been identified:

• That it is focussed on manual coding—and hence is resource intensive.

• It is focussed on functional requirements and does not explicitly address non-functional requirements.

• It does not emphasise reuse.

Generally, agile approaches minimise the use of documentation and formal architecture descriptions, and verification is carried out only by testing or by code inspection.

The state of agile survey found that barriers to the adoption of Agile development included:

• Organisational culture in conflict with agile principles.

• Lack of experience with agile methods.

• Lack of management support.

• Lack of access to users/customers.

3.4.6 Can Agile and MBD be Combined?

Both agile and MBD aim to accelerate development. Whilst agile focusses upon rapid response to changing requirements, MBD focusses upon software correctness and adding value over a longer term. In principle, the combination of agile and MBD approaches could be effective and provide complementary strengths:

• MBD provides modelling, verification and reuse support lacking in Agile.

• An agile MBD approach would use models as the primary artifact, not code, reducing development costs and time.

But MBD tends to be a ‘heavyweight’ process with substantial use of documentation and multiple process steps. In particular, the use of multiple models in UML and MDA hinder rapid specification change, since a change to one model may impact others, and these inconsistencies need to be resolved before a new executable version is generated. The agile emphasis on simplicity can be applied to reduce the complexity of MBD, in particular to reduce the number of parallel models being maintained for a system.

Several agile MBD approaches have been created [14, 15], including MDD-SLAP at Motorola [6] and xUML/fUML/Alf from the OMG [3]. In this book we will follow an agile/MBD process based on the UML-RSDS subset of UML and its supporting tools.

Summary

In this chapter, we:

• Introduced the concepts of UML, MBD, DSLs and Agile development

• Gave examples of their use in industry, and rationales for/against their use.

In subsequent chapters, we will use an agile MBD approach for developing financial systems, based on UML.