As mentioned in the previous section, our technique consists of two steps, where we determine (trace-)aberrant statements in the first step and compute their suspiciousness ranks in the next one.
3.1 Programming Language
To precisely describe our technique, we introduce a small programming language, shown in Fig. 2. As shown in the figure, a program consists of a statement, and statements include sequencing, assignments, assertions, assumptions, conditionals, and loops. Observe that conditionals and loops have non-deterministic predicates, but note that, in combination with assumptions, they can express any conditional or loop with a predicate p. To simplify the discussion, we do not introduce additional constructs for procedure definitions and calls.
We assume that program execution terminates as soon as an assertion or assumption violation in encountered (that is, when the corresponding predicate evaluates to false). For simplicity, we also assume that our technique is applied to one failing assertion at a time.
3.2 Trace-Aberrant Statements
Recall from the previous section that trace-aberrant statements are assignments along the error trace for which there exists a local fix that makes the trace verify (that is, the error becomes unreachable along this trace):
Definition 1
(Trace aberrance). Let \(\tau \) be a feasible error trace and s an assignment statement of the form \(v := \star \) or \(v := e\) along \(\tau \). Statement s is trace-aberrant iff there exists an expression \(e'\) that may be assigned to variable v such that the trace verifies.
To determine which assignments along an error trace are trace-aberrant, we first compute, in the post-state of each assignment, the weakest condition that ensures that the trace verifies. We, therefore, define a predicate transformer \( WP \) such that, if \( WP(S, Q) \) holds in a state along the error trace, then the error is unreachable and Q holds after executing statement S. The definition of this weakest-precondition transformer is standard [9] for all statements that may appear in an error trace:
-
\( WP (s_1;s_2, Q) \equiv WP(s_1, WP(s_2, Q)) \)
-
\( WP (v:=\star , Q) \equiv \forall v'.Q[v:=v'], \text {where } v' \notin freeVars (Q)\)
-
\( WP (v:=e, Q) \equiv Q[v:=e]\)
-
-
In the weakest precondition of the non-deterministic assignment, \(v'\) is fresh in Q and \(Q[v:=v']\) denotes the substitution of v by \(v'\) in Q.
To illustrate, we compute this condition in the post-state of each assignment along the error trace of Sect. 2. Weakest precondition
should hold in the pre-state of statement
, and thus in the post-state of assignment x := 0, for the trace to verify. Similarly, \(3 \le \texttt {y}\) and \( false \) should hold in the post-state of assignments \(\texttt {y = }\star \) and x := 2, respectively. Note that condition \( false \) indicates that the error is always reachable after assignment x := 2.
Second, we compute, in the pre-state of each assignment along the error trace, the strongest condition that holds when executing the error trace until that state. We define a predicate transformer \( SP \) such that condition \( SP(P, S) \) describes the post-state of statement S for an execution of S that starts from an initial state satisfying P. The definition of this strongest-postcondition transformer is also standard [10] for all statements that may appear in an error trace:
-
\( SP (P, s_1;s_2) \equiv SP(SP(P, s_1), s_2) \)
-
\( SP (P, v:=\star ) \equiv \exists v'.P[v:=v'], \text {where } v' \notin freeVars (P) \)
-
\( SP (P, v:=e) \equiv \exists v'.P[v:=v'] \wedge v=e[v:=v'],\)
\(\text {where } v' \notin freeVars (P) \cup freeVars (e)\)
-
-
In the strongest postcondition of the assignment statements, \(v'\) represents the previous value of v.
For example, strongest postcondition
$$ SP ( true ,\texttt {x := 2}) \equiv \texttt {x} = 2 $$
holds in the post-state of assignment x := 2, and therefore in the pre-state of \(\texttt {y := }\star \). Similarly, the strongest such conditions in the pre-state of assignments x := 2 and x := 0 along the error trace are \( true \) and \(\texttt {x} = 2 \wedge \texttt {y} < 3\), respectively.
Third, our technique determines if an assignment a (of the form \(v := \star \) or \(v := e\)) along the error trace is trace-aberrant by checking whether the Hoare triple [17] \(\{\phi \} \; v := \star \; \{\psi \}\) is valid. Here, \(\phi \) denotes the strongest postcondition in the pre-state of assignment a, v the left-hand side of a, and \(\psi \) the negation of the weakest precondition in the post-state of a. If this Hoare triple is invalid, then assignment statement a is trace-aberrant, otherwise it is not.
Intuitively, the validity of the Hoare triple implies that, when starting from the pre-state of a, the error is always reachable no matter which value is assigned to v. In other words, there is no local fix for statement a that would make the trace verify. Consequently, assignment a is not trace-aberrant since it cannot possibly be the cause of the error. As an example, consider statement x := 2. For this assignment, our technique checks the validity of the Hoare triple \(\{ true \} \; \texttt {x := } \star \; \{ true \}\). Since any value for x satisfies the true postcondition, assignment x := 2 is not trace-aberrant.
If, however, the Hoare triple is invalid, there exists a value for variable v such that the weakest precondition in the post-state of a holds. This means that there is a local fix for a that makes the error unreachable. As a result, statement a is found to be trace-aberrant. For instance, for statement x := 0, we construct the following Hoare triple: \(\{\texttt {x} = 2 \wedge \texttt {y} < 3\} \; \texttt {x := } \star \; \{\texttt {x} \le 0\}\). This Hoare triple is invalid because there are values that may be assigned to x such that \(\texttt {x} \le 0\) does not hold in the post-state. Assignment x := 0 is, therefore, trace-aberrant. Similarly, for \(\texttt {y := }\star \), the Hoare triple \(\{\texttt {x} = 2\} \; \texttt {y := } \star \; \{\texttt {y} < 3\}\) is invalid.
3.3 Program-Aberrant Statements
We now define program-aberrant statements; these are assignments for which there exists a local fix that makes every trace through them verify:
Definition 2
(Program aberrance). Let \(\tau \) be a feasible error trace and s an assignment statement of the form \(v := \star \) or \(v := e\) along \(\tau \). Statement s is program-aberrant iff there exists an expression \(e'\) that may be assigned to variable v such that all traces through s verify.
Based on the above definition, the trace-aberrant assignments in the program of Fig. 1 are also program-aberrant. This is because there is only one error trace through these statements.
As another example, let us replace the assignment on line 8 of the program by x := -1. In this modified program, the assertion on line 10 fails when program execution takes either branch of the if statement. Now, assume that a static analyzer, which clearly fails to verify the assertion, generates the same error trace that we described in Sect. 2 (for the then branch of the if statement). Like before, our technique determines that statements \(\texttt {y := }\star \) and x := 0 along this error trace are trace-aberrant. However, although there is still a single error trace through statement x := 0, there are now two error traces through \(\texttt {y := }\star \), one for each branch of the conditional. We, therefore, know that x := 0 is program-aberrant, but it is unclear whether assignment \(\texttt {y := }\star \) is.
To determine which trace-aberrant assignments along an error trace are also program-aberrant, one would need to check if there exists a local fix for these statements such that all traces through them verify. Recall that there exists a fix for a trace-aberrant assignment if there exists a right-hand side that satisfies the weakest precondition in the post-state of the assignment along the error trace. Therefore, checking the existence of a local fix for a program-aberrant statement involves computing the weakest precondition in the post-state of the statement in the program, which amounts to program verification and is undecidable.
Identifying which trace-aberrant statements are also program-aberrant is desirable since these are precisely the statements that can be fixed for the program to verify. However, determining these statements is difficult for the reasons indicated above. Instead, our technique uses the previously-computed weakest preconditions to decide which trace-aberrant assignments must also be program-aberrant, in other words, it can under-approximate the set of program-aberrant statements. In our experiments, we find that many trace-aberrant statements are must-program-aberrant.
To compute the must-program-aberrant statements, our technique first identifies the trace-aberrant ones, for instance, \(\texttt {y := }\star \) and x := 0 for the modified program. In the following, we refer to the corresponding error trace as \(\epsilon \).
As a second step, our technique checks whether all traces through the trace-aberrant assignments verify with the most permissive local fix that makes \(\epsilon \) verify. To achieve this, we instrument the faulty program as follows. We replace a trace-aberrant statement \(a_i\) of the form \(v := e\) by a non-deterministic assignment \(v := \star \) with the same left-hand side v. Our technique then introduces an assume statement right after the non-deterministic assignment. The predicate of the assumption corresponds to the weakest precondition that is computed in the post-state of the assignment along error trace \(\epsilon \)Footnote 1. We apply this instrumentation separately for each trace-aberrant statement \(a_i\), where \(i = 0, \dots , n\), and we refer to the instrumented program that corresponds to trace-aberrant statement \(a_i\) as \(P_{a_i}\). (Note that our technique uses this instrumentation for ranking aberrant statements in terms of suspiciousness, as we explain in Sect. 4.) Once we obtain a program \(P_{a_i}\), we instrument it further to add a flag that allows the error to manifest itself only for traces through statement \(a_i\) (as per Definition 2). We denote each of these programs by \(P_{a_i}^\dag \).
For example, the light gray boxes on the right show the instrumentation for checking whether statement x := 0 of the modified program is program-aberrant (the darker boxes should be ignored). Lines 8–9 constitute the instrumentation that generates program \(P_{\texttt {x := 0}}\). As explained in Sect. 3.2, when the computed weakest precondition holds on line 9, it implies that trace \(\epsilon \) verifies. Consequently, this instrumentation represents a hypothetical local fix for assignment x := 0. Lines 1, 10, and 15 block any program execution that does not go through statement x := 0. As a result, the assertion may fail only due to failing executions through this statement. Similarly, when considering the dark gray boxes in addition to lines 1 and 15 (and ignoring all other light boxes), we obtain \(P_{\texttt {y := }\star }^\dag \). Line 4 alone constitutes the instrumentation that generates program \(P_{\texttt {y := }\star }\).
Third, our technique runs the static analyzer on each of the n instrumented programs \(P_{a_i}^\dag \). If the analyzer does not generate a new error trace, then statement \(a_i\) must be program-aberrant, otherwise we do not know. For instance, when running the static analyzer on \(P_{\texttt {x := 0}}^\dag \) from above, no error is detected. Statement x := 0 is, therefore, program-aberrant. However, an error trace is reported for program \(P_{\texttt {y := }\star }^\dag \) (through the else branch of the conditional). As a result, our technique cannot determine whether \(\texttt {y := }\star \) is program-aberrant. Notice, however, that this statement is, in fact, not program-aberrant because there is no fix that we can apply to it such that both traces verify.
3.4 k-Aberrance
So far, we have focused on (trace- or program-) aberrant statements that may be fixed to single-handedly make one or more error traces verify. The notion of aberrance, however, may be generalized to sets of statements that make the corresponding error traces verify only when fixed together:
Definition 3
(k-Trace aberrance). Let \(\tau \) be a feasible error trace and \(\bar{s}\) a set of assignment statements of the form \(v := \star \) or \(v := e\) along \(\tau \). Statements \(\bar{s}\) are \(|\bar{s}|\)-trace-aberrant, where \(|\bar{s}|\) is the cardinality of \(\bar{s}\), iff there exist local fixes for all statements in \(\bar{s}\) such that trace \(\tau \) verifies.
Definition 4
(k-Program aberrance). Let \(\bar{\tau }\) be the set of all feasible error traces through any assignment statement s in a set \(\bar{s}\). Each statement s is of the form \(v := \star \) or \(v := e\) along an error trace \(\tau \) in \(\bar{\tau }\). Statements \(\bar{s}\) are \(|\bar{s}|\)-program-aberrant, where \(|\bar{s}|\) is the cardinality of \(\bar{s}\), iff there exist local fixes for all statements in \(\bar{s}\) such that all traces \(\bar{\tau }\) verify.
For example, consider the modified version of the program in Fig. 1 that we discussed above. Assignments x := 0 and x := -1 are 2-program-aberrant because their right-hand side may be replaced by a positive value such that both traces through these statements verify.
Our technique may be adjusted to compute k-aberrant statements by exploring all combinations of k assignments along one or more error traces.