A novel model-based testing approach for software product lines

Abstract

Model-based testing relies on a model of the system under test. FineFit is a framework for model-based testing of Java programs. In the FineFit approach, the model is expressed by a set of tables based on Parnas tables. A software product line is a family of programs (the products) with well-defined commonalities and variabilities that are developed by (re)using common artifacts. In this paper, we address the issue of using the FineFit approach to support the development of correct software product lines. We specify a software product line as a specification product line where each product is a FineFit specification of the corresponding software product. The main challenge is to concisely specify the software product line while retaining the readability of the specification of a single system. To address this, we used delta-oriented programming, a recently proposed flexible approach for implementing software product lines, and developed: (1) delta tables as a means to apply the delta-oriented programming idea to the specification of software product lines; and (2) DeltaFineFit as a novel model-based testing approach for software product lines.

Appendices

Appendix 1: The structure and semantics of operation tables

A FineFit operation table is a predicate that specifies the behavior of an operation as a relation between the model’s state variables before the operation starts (the pre-state) and after the operation completes (the post-state). It consists of two major areas: an expression table and a precondition tree. The expression table is a set of columns, where each column defines the values of the state variables in the post-state, given their values in the pre-state. The predcondition tree consists of predicates that determine which columns to use in the definition of the post-state. For example, consider the following operation table:

The operation R has a single input parameter c? and determines the value of its two state variables x and y as follows: If c? is positive, then the operation must set x to $x+c?$ and y to $x+y$ (the right most column), otherwise, if c? is not positive, the effect of the operation depends on the current value of x. When x is not negative x and y must be set to zero and when x is negative, then x and y do not change (their value must be the same as it was when the operation started). Note that the predicate for determining whether columns 1 and 2¹⁷ are relevant for the specification is the conjunction of the child predicates $x<0$ and $x\ge 0$ with their parent predicate $c? \le 0$.

We now define the conditions necessary for the precondition part of the table to form a tree. Consider the precondition part of the table as a matrix of cells. Then, each predicate spans one or more consecutive cells (in the same row) and the space that each predicate occupies must be included below the space that its parent (the predicate above) occupies. For example, the predicate $x \ge 0$ occupies cell (2, 2) and its parent, the predicate $c? \le 0$ occupies cells (1, 1), (1, 2).

Formally, consider a precondition part that is arranged in a matrix of n rows by m columns of cells. The i-th row contains a sequence of predicates, each occupying a span of one or more cells. Let $k_i$ indicate the sequence of spans that each predicate occupies (and therefore $k_{ij}$ is the j-th span in the i-th row). For example, in the operation above, $k_1 = 2,1$ and $k_2 = 1,1,1$. In order for the spans to describe a valid precondition tree, three conditions must be met:

• 1. The sum of spans in each row must be equal to the number of columns in the matrix:

  $$\begin{aligned} \sum _{j=1}^{|k_i|}k_{ij} = m \end{aligned}$$

  where $|k_i|$ is the number of elements in the ith sequence of spans.

• 2. The cells that each predicate spans must be included below the span of cells of its parent:

  $$\begin{aligned} S_{ij} \subseteq S_{(i-1)l} \qquad \hbox {for some }1\le l\le m \end{aligned}$$

  where the span of cells of the j-th predicate in the i-th row is

  $$\begin{aligned} S_{ij} = \sum _{l=1}^{j-1}k_{il} + 1, \ldots , \sum _{l=1}^j k_{il} \end{aligned}$$

• 3. The last row (n) must consist of individual cells, that is

  $$\begin{aligned} k_{nj} = 1 \quad \hbox {for all }1 \le j \le m. \end{aligned}$$

This arrangement ensures that the predicates form a tree structure¹⁸ with m leaves, all occupying the last row. The following matrix illustrates this structure:

There are three span sequences, one for each row:

$$\begin{aligned} k_{3}= & {} 1,1,1,1,1,1,1 \end{aligned}$$

(3)

$$\begin{aligned} k_{2}= & {} 3,2,2 \end{aligned}$$

(4)

$$\begin{aligned} k_{1}= & {} 5,2 \end{aligned}$$

(5)

We can see that each span is included below the span of its parent. For example,

$$\begin{aligned} S_{2,2} = \{4,5\} \subseteq S_{1,1} = \{1,2,3,4,5\} \end{aligned}$$

We say that the precondition tree is well formed when it satisfies the three conditions we have defined above. In the rest of the discussion, we always assume that we are working with well-formed precondition trees. Given a column i, we define the guard for this column as the conjunction of the leaf that occupies the i-th column and all its ancestors. Let $\bar{v}' = v'_1,\ldots ,v'_l$ be the state space vector. Let $\bar{c_i}$ be the i-th column of the expression table. Then, the meaning of the operation specification is:

$$\begin{aligned} \bar{v}' \in \{ \bar{c_i} : 1 \le i \le m \wedge guard(i) \} \end{aligned}$$

That is, the value of the state variables vector in the post-state can be equal to the value of any of the expression table columns whose guard was true in the pre-state.

The semantics of individual predicates and expressions is explained in [3].

Appendix 2: The algorithm that computes the $apply$ function

We provide (in Sect. “Executable specification in Haskell”) a Haskell executable specification of the $apply$ function (cf. Sect. 4.2.1) and present (in Sect. “The Java implementation”) its Java implementation. Note that in this paper all the tables resulting from delta-table application have been generated using this implementation.

1.1 Executable specification in Haskell

The executable specification of the algorithm that computes the $apply$ function is provided as a program written in Haskell (Listing 13).

Haskell is a functional programming language with a syntax that strongly resembles the usual mathematical notation for defining function by cases, via pattern matching. The Haskell code and the comments in Listing 13 should be almost self-explanatory. In the following, we shortly explain some technicalities. The expression contained in an ordinary table cell and in a delta-table cell containing the insert or replace operators is represented using the standard library type String (Line 3). Both ordinary tables and delta tables are represented as trees, where each node corresponds to a cell. A tree that represents a table (ordinary or delta) is expressed as a value of the recursive data type Table (Line 10)—a value of type Table describes the root cell and the list of its immediate subtrees. The data type Table has five data constructors (each describes a different kind of node): Basic, Insert, and Replace (all have arity two), Match and Remove (arity one).¹⁹ Ordinary tables are represented by using only the data constructor Basic, while delta tables are represented by using only the other data constructors, which correspond to the delta-table operators.

Example 3

(Representation of ordinary tables and delta tables in Haskell) Since ordinary tables may have more than one cell in their first row, they are always encoded by adding a top node containing the string “ROOT”. Delta tables are therefore encoded by adding a top node containing the match operator (matching the top node in the ordinary tables). The following ordinary and delta tables

are respectively represented by the following values of type Table:

The Haskell function apply (Lines 15–64) corresponds to the $apply$ function (introduced in Sect. 4.2.1) that executes the delta operations of the delta table. To improve readability, we have not modeled the behavior described in Remark 2 of Sect. 4.2.1.

Line 30 declares the type of the apply function. The first argument of the function apply has type Maybe Table. The standard library Maybe data type has two constructors: the unary data constructor Just and the constant Nothing. It is used to specify optional values: A value of type Maybe Table either contains a table t (represented as Just t), or it is empty (represented as Nothing).

The standard library function concat takes two lists and returns their concatenation. The standard library function zipWith calls a given function pairwise on each member of both lists, returning a list. For the convenience of readers, zipWith’s code is as follows:

Example 4

(An execution of the Haskell function apply) Consider the following delta-table application, which already has been presented at the end of Sect. 4.2.1 (recall that the numbers to the left of the cells denote the recursion step):

The above delta-table application can be defined by appending to the code in Listing 13 the following three lines:²⁰

Executing the main program yields

1.2 The Java implementation

The Java implementation of the algorithm that computes the $apply$ function is given in Listings 14 and 15. A cell of a table is represented by the Node class which extends the Vector class. Tables are represented as trees, and an object of class Node represents a cell, which is the root of a subtree, and contains (in the Vector’s elements) the reference to the roots of the immediate subtrees.

The applyPrime method (Lines 16–23 in Listing 14) corresponds to the $apply '$ function (introduced in Sect. 4.3). It first calls (Line 19) the prepare method (which applies the rules described in Sect. 4.3) and then invokes (Line 21) the apply method (Lines 30–64 in Listing 14). The apply method corresponds to the $apply$ function (introduced in Sect. 4.2.1 and specified in Sect. 1). When calling the apply method, the this object is the current node of the delta table, and orig and res are the current nodes of the original table and of the resulting table, respectively.

The body of the apply method implements the recursive walking of the parse tree, which is controlled by the operations in the cells of the delta table and, therefore, describes how each operation works. The first part of the method (Lines 36–50) reads the operation of the current delta node and updates the content of the resulting node; Lines 40–46 implement the behavior described in Remark 2 of Sect. 4.2.1. The second part of the method (Lines 52–62) is responsible for the recursive call.²¹ At each recursive call (Lines 43, 56, and 60), the method getOrCreateChild(i), which creates the nodes of the resulting table, is called; it either returns the i-th subnode if it already exists, or it creates the i-th subnode and returns it. We now continue illustrating the apply method using small examples. (An example of an execution trace of the method apply is provided in Example 5.)

The loop that begins in Line 52 iterates over the siblings of the current delta-table node. This means that if at the current level the original table has more cells than the delta table, then the additional cells are ignored. For example:

If the delta table has more cells than the original table at the current level, the last cell of the original table is repeatedly used when processing the additional operations of the delta table (Line 53). For example:

If the original table contains no cells at the current level and the current operation of the delta table is $*$ or $-$, the remaining operations of the delta table on the current branch are ignored (Line 52).

If the original table contains no cells at the current level and the current operation of the delta table is $\blacktriangleright$ or $*\blacktriangleright$, a cell with the value of the insert or replace operations is inserted (Line 50) in the resulting table. Line 47 maps in this case the replace operation to an insert operation. For example:

It is therefore possible to extend a table with new variable and value rows by using either the insert or replace operation.

The algorithm traverses the original table and the delta table in parallel. The recursive call in Line 54 is responsible for the operations match, remove, and replace.²² The recursive call in Line 50 handles the insert operation, where the current node of the original table is passed as the first argument rather than the current child of the original table. Delaying the recursive step on the original table results in the described semantics of the insert operation.

The recursive call in Line 37 is used for convenience. If either the remove or the match operation occurs as the last condition in a condition hierarchy, but the condition cell c of the original table has additional subconditions (this is checked by isLastConditionNode()), the remove or the match operation is applied to all subcondition of c. For example, let A, B, and C be condition cells, then

The reader may have noticed that unlike the other operations, the remove operation is not handled in the loop in Lines 52–62. When a remove operation occurs, a temporary cell is created in the resulting table that is marked to be removed (Line 38). The marked cells are removed in Line 57. Removing nodes at the end of the recursion simplifies the implementation, because removing a node c pulls up its children $c_1,\ldots ,c_n$ to the current level. Hence, by removing a node the number of siblings at the current level may decrease, stay constant, or increase. In the following example, we assume that A is not a condition cell.

Example 5

(An execution trace of the method apply) For a complete example consider the following delta-table application, which already has been presented at the end of Sect. 4.2.1 and in Example 4 of “Executable specification in Haskell” of appendix:

Executing the method apply in (Lines 34–64) will result in the following recursive invocations of the method:

At the end of the last recursion step of the apply algorithm (Line 57 in Listing 14), the method deleteTemporaryRemoveNodes is called which removes the temporary “$-$” node from the resulting tree res that is introduced at recursion step 5.