Incremental Rewriting Modulo SMT

Abstract

Rewriting Modulo SMT combines two powerful automated deduction techniques (1) rewriting and (2) SMT-solving. Rewriting enables the specification of behavior of systems using rewriting rules, while SMT theories specify system properties. Rewriting Modulo SMT is enabled by combining existing tools, such as Maude and SMT solvers. Search algorithms used for carrying out Rewriting Modulo SMT, however, cannot exploit the incremental solving features available in SMT solvers as they are based on breadth-first search. This paper addresses this limitation by proposing Incremental Rewriting Modulo SMT Theories, which is a syntactical restriction to rewriting rules. This restriction turns out to naturally be used in several applications of Rewriting Modulo SMT, including the verification of algorithms, cyber-physical systems, and security protocols. Moreover, we propose a Hybrid-Search algorithm for Incremental Rewriting Modulo SMT Theories that combines breadth-first search and depth-first search, thus enabling incremental SMT-solving. We demonstrate through a collection of existing benchmarks that the Hybrid-Search algorithm can achieve a 10 times performance improvement in verification times.

1 Introduction

Rewriting modulo SMT [14] is the result of the combination of two powerful automated deduction methods: rewriting logic and SMT-solving. It is supported by the integration [11] of powerful tools, such as Maude [6] and Z3 [8]. During rewriting, a set of constraints on the symbols appearing in a term are generated. These constraints can be, for example, non-linear arithmetic constraints that specify possible values that can be assumed by the configuration parameters. Demonstrating properties of such specifications amounts to search using these rewrite rules and satisfiability checking of the accumulated constraints using SMT solvers. Rewriting modulo SMT has been successfully applied in several case-studies from several domains, including safety of cyber-physical systems (CPSes) [13]; verification of algorithms [2]; and for network security analysis [16].

One important aspect that has not been addressed until now is how to exploit an SMT solver’s capability of incrementally solving problems. In this solving method, instead of checking for the satisfiability of a formula from scratch, it re-uses data previously computed by prior checks. For example, if the satisfiability of a formula $\textsf{b}$ has been checked, the check on $\textsf{b} \wedge \textsf{b}_I$ may re-use the intermediate results obtained while checking for the satisfiability of $\textsf{b}$. It has been shown that incremental solving can greatly improve performance by a factor of 2–5 times [10].¹

The search algorithms used to implement rewriting modulo SMT are similar to those implemented in the Maude search engine [6]. They use a breadth-first search (BFS) algorithm with memoization techniques in order to improve performance. This type of search seems incompatible with incremental solving as constraints appearing in different branches of the search tree are generated under different conditions. Thus, it is hard to define what the increment ($\textsf{b}_I$ mentioned above) would be.

This paper’s goal is to enable rewriting modulo SMT that can exploit incremental solving. To achieve this, we make the following contributions:

• Incremental Rewriting Modulo SMT by identifying a class of rewrite rules that are amenable to incremental solving. More specifically, rewrite rules are applied to terms containing symbols paired with a set of boolean terms constraining the values of these symbols. Moreover, any rewrite rule can only add new constraints, i.e., not change the existing set of constraints on the term that is being rewritten. We show that a variety of theories used in published case studies can be seen to be amenable to incremental solving.

• A Hybrid Search Algorithm for Incremental Theories which combines breadth and depth-first search (DFS) strategies. The combination is parameterized by a level of depth parameter which specifies how many depth-first search steps shall be performed before switching to a breath-first search. The proposed hybrid search algorithm enjoys the benefits of BFS, namely better coverage as it alternates through different branches of the search tree, and the benefits of DFS, namely incremental solving.

We carried out a collection of experiments (the case studies mentioned above) on algorithm verification, cyber-physical systems verification, and network security analysis. The experiments show that in all these benchmarks, the hybrid search algorithm outperforms current BFS techniques, in some experiments achieving a 10 factor performance improvement.

Section 2 illustrates the problems of existing BFS methods for Rewriting Modulo SMT and proposes Incremental Rewriting Theories which formalizes the notion of increments. Section 3 describes the Hybrid algorithm proposed illustrating how it enables incremental SMT solving. Section 4 describes experiments that compare different search mechanisms (BFS, DFS, and Hybrid) on existing benchmarks from the literature. Finally, we conclude by discussing Related Work in Sect. 5 and Future Work in Sect. 6.

2 Incremental Rewriting Modulo SMT

Rewriting logic [12] is a logical formalism that is based on two ideas: states of a system are represented as elements of an algebraic data type, specified in an equational theory, and the behavior of a system is given by local transitions between states described by rewrite rules. A rewrite rule has the form $\textsf{t}\rightarrow \textsf{t}'$ if $\textsf{b}$, where $\textsf{t}$ and $\textsf{t}'$ are terms possibly containing variables and $\textsf{b}$ is a condition (a boolean term). Such a rule applies to a system in state $\textsf{s}$ (a ground term) if $\textsf{t}$ can be matched to a part of $\textsf{s}$ by supplying the right values for the variables, and if the condition $\textsf{b}$ holds when supplied with those values. In this case, the rule can be applied by replacing the part of $\textsf{s}$ matching $\textsf{t}$ by $\textsf{t}'$ using the matching values for the variables in $\textsf{t}'$.

Maude is a language and tool based on rewriting logic [6]. Maude provides a high performance rewriting engine featuring matching modulo associativity, commutativity, and identity axioms; and search and model-checking capabilities. Thus, given a specification S of a concurrent system, one can execute S to find one possible behavior; use search to see if a state meeting a given condition can be reached; or model-check S to see if a temporal property is satisfied, and if not, to see a computation that is a counterexample.

Symbolic rewriting modulo SMT [13, 14] allows rewriting symbolic states $(\textsf{t},\textsf{b})$, where $\textsf{t}$ is a term possibly containing variables and $\textsf{b}$ a boolean term constraining the allowed values of variables of $\textsf{t}$. The symbolic state $(\textsf{t},\textsf{b})$ represents the set of (concrete) states that are instances of $\textsf{t}$ such that the instantiating substitution satisfies $\textsf{b}$. Thus a rewrite to a symbolic state $\textsf{t}',\textsf{b}')$ such that $\textsf{b}'$ is not satisfiable represents the empty set of concrete rewrites and satisfiability can be checked at each step to avoid useless work. This independent of checking that a goal is satisfied by a symbolic state. To implement symbolic rewriting in Maude, variables are replaced by symbols, treated as constants by Maude, and translated as variables when using an SMT solver to check satisfiability of the constraint. Symbolic rewriting allows us to reason about open systems, and to reason about all (possibly infinitely many) instances of a configuration.

Verification problems are expressed as reachability problems expressed as statements for the form

$$ \textsf {search}(\textsf{t}_0,\textsf{b}_0) \Rightarrow (\textsf{t}',\textsf{b}') \textsf { such that }\textsf {goalCond}(\textsf{t}',\textsf{b}') $$

where $(\textsf{t}',\textsf{b}')$ is a pattern and $\textsf {goalCond}$ is a boolean function that checks whether a state satisfies some condition. Typically, $\textsf {goalCond}(\textsf{t}',\textsf{b}')$ also makes calls to the SMT solver to check whether some constraints derived from $\textsf{b}'$ are satisfiable.

As illustrated by Fig. 1, Rewriting Modulo SMT implementations [11] traverse the search tree derived from the rewrite rules using BFS-based algorithms. At each step, e.g., $(\textsf{t}_0,\textsf{b}_0) \rightarrow (\textsf{t}_1,\textsf{b}_1)$, the engine checks for the satisfiability of the condition $\textsf{b}_1$. If the check fails, then search backtracks following BFS strategy. Otherwise, if the check succeeds, then the engine checks (1) whether $(\textsf{t}_1,\textsf{b}_1)$ matches the pattern $(\textsf{t}',\textsf{b}')$ and (2) if this is the case, it checks the condition $\textsf {goalCond}(\textsf{t}_1,\textsf{b}_1)$, which may make further calls to the SMT solver, written as $\textsf{SMT}(\textsf {goalCond}(\textsf{t}_1,\textsf{b}_1))$. If $\textsf {goalCond}$ returns true, then a solution for the reachability problem is found. Otherwise, the algorithm continues search following BFS.

Fig. 1.

Illustration of the search tree and SMT-calls when using Rewriting Modulo SMT following a BFS algorithm. The sequence of SMT-calls of a BFS algorithm is depicted to the left, where $\textsf{SMT}(\textsf {goalCond}(\textsf{t}_i,\textsf{b}_i))$ denotes possible SMT-calls required by the goal condition $\textsf {goalCond}$. The numbers inside the circles specify the order in which nodes are traversed.

From the sequence of calls to the SMT solver, one can observe the following difficulties of exploiting incremental SMT solving when using BFS based search strategy:

• Definitions of Increments: Given the generality of the accepted theory, it is not possible for the search engine to determine whether constraints, $\textsf{b}_1$ and $\textsf{b}_2$, used in subsequent calls to the SMT, $\textsf{SMT}(b_1)$ and $\textsf{SMT}(b_2)$, are constructed using some increment, i.e., whether $\textsf{b}_2 = \textsf{b}_1 \wedge \textsf{b}_{1,2}$. This is because $\textsf{b}_1$ and $\textsf{b}_2$ are derived by applying different instances of rules which normally add/modify constraints in different ways.

• Not possible to chain incremental calls: As it is not possible to define increments when using rewrites rules in general, it is not possible to effectively use incremental solving by chaining calls, such as in $\textsf{SMT}(\textsf{b}_1); \textsf{SMT}(\textsf{b}_1 \wedge \textsf{b}_{1,2}); \textsf{SMT}(\textsf{b}_{1} \wedge \textsf{b}_{1,2} \wedge \textsf{b}_{2,3})\ldots$.

To address this problem, we introduce a special class of rewrite theories, called Incremental Rewrite Theories.

Definition 1

An incremental rewrite theory is a rewrite theory specification $\langle \varSigma , \mathcal {E}, \mathcal {R}\rangle$ where $\varSigma$ is a typed alphabet; $\mathcal {E}$ is an equational theory; and $\mathcal {R}$ is a set of rewrite rules of the forms:

$$ \begin{array}{l} (\textsf{t},\textsf{b}) \rightarrow (\textsf{t}_1,\textsf{b}\wedge \textsf{b}_I) \qquad \text { and } \qquad (\textsf{t},\textsf{b}) \rightarrow (\textsf{t}_1,\textsf{b}\wedge \textsf{b}_I) \textsf { if } \textsf {cond}\end{array} $$

where $\textsf{t}, t_I$ are well-formed terms; $\textsf{b},\textsf{b}_I$ are boolean formulas (in a given theory); and $\textsf {cond}$ is a conjunction of equations.²

The verification problem for incremental problems is a specialized reachability problem as defined below.

Definition 2

Let $\mathcal {T}$ be an incremental rewrite theory. An incremental reachability problem over $\mathcal {T}$ is of the form:

$$ \textsf {search}(\textsf{t}_0,\textsf{b}_0) \Rightarrow (\textsf{t}',\textsf{b}') \textsf { such that }\textsf {goalTerm}(\textsf{t}') \textsf { and } \textsf{SMT}(\textsf{b}' \wedge \textsf{b}_I) $$

where $\textsf {goalTerm}$ is a function that takes a term and returns a boolean value and $\textsf{b}_I = \textsf{goal}(\textsf{t}')$ is a boolean formula constructed from $\textsf{t}'$.

The following three examples illustrate how incremental theories can model different types of systems. These examples are based on specifications from the literature [2, 13, 16]. For ease of exposition, we simplify the rules in the description below. In Sect. 4, the full specifications from the literature are used in our experiments.

Example 1

This example is based on the work [2] for verification of the CASH scheduling algorithm [4]. In this algorithm, each task has a worst-case execution time. Whenever a task is completed before its deadline, the unused processing time is added to a global queue of unused budget, which can then be used by other tasks. Rewriting modulo SMT has been used to verify whether it is possible for a task to miss its deadline [2]. In particular, constraints keep track of the processing times and the available time budgets.

It turns out that the specification of this algorithm as rewrite rules and the verification problem are an incremental rewrite theory and an incremental reachability problem, respectively. For example, the following rule specifies when a deadline is missed:

$$ \begin{array}{l} (\langle \textsf{id}_1 : \textsf{global}\mid \textsf{deadlineMiss}: missStat, \textsf{Ats}\rangle ,\\ \langle \textsf{id}_0 : \textsf{server}\mid \textsf{state}: st, \textsf{usedBudget}: t, \textsf{timeDeadline}: t_1, \textsf{maxBudget}: n\rangle ~ \textsf{rest},\textsf{b})\\ \rightarrow (\langle \textsf{id}_1 : \textsf{global}\mid \textsf{deadlineMiss}: \textsf{true}, \textsf{Ats}\rangle , \\ \langle \textsf{id}_0 : \textsf{server}\mid \textsf{state}: st, \textsf{usedBudget}: t, \textsf{timeDeadline}: t_1, \textsf{maxBudget}: n\rangle ~ \textsf{rest}, \\ \textsf{b}\wedge \textsf{b}_I ) ~\textsf{if}~ (st = \textsf{waiting}\vee st = \textsf{executing}) \end{array} $$

where $\textsf{rest}$ is the specification of the remaining tasks, $\textsf{Ats}$ are other attributes of the server, $\textsf{b}_I$ is the set of constraints $t \ge 0 \wedge t_1 \ge 0 \wedge n> 0 \wedge (n - t) > t_1$. This rule specifies that the deadline is missed if there is a task $\textsf{id}_0$ that is not finished, i.e., either waiting or executing, such that the time to finish ($t_1$) cannot be met by the available time budget $n - t$ required by the task.

The verification problem of checking whether for some given configuration $(\textsf{t}_0,\textsf{b}_0)$ of server and tasks, a task can miss its deadline is specified by the following search command which is an incremental reachability problem

$$ \begin{array}{l} \textsf {search}(\textsf{t}_0,\textsf{b}_0) \Rightarrow (\langle \textsf{id}_1 : \textsf{global}\mid \textsf{deadlineMiss}: \textsf{true}, \textsf{Ats}\rangle ~ \textsf{rest},\textsf{b}') \textsf { such that }\textsf{SMT}(\textsf{b}') \end{array} $$

Example 2

Rewriting Modulo SMT has been used for verifying whether resource bounded intruders can slowly deny access to webservers [16]. This type of attack was inspired by application layer DDoS attacks such as Slowloris [7] where the attacker attempts to exhaust all the resources of a webserver by periodically sending bursts of multiple requests. When receiving such bursts of requests, the webserver has to allocate resources for at least some period of time, called timeout. As the webserver has limited resources, the attacker is capable of denying service to legitimate users by sending enough bursts.

Constraints were used in previous work [16] to keep track of (1) the number of resources available by the webservers, and (2) the timeout period of bursts. While we refer to the previous work [16] for the complete formalization, we illustrate the incrementality of such specifications with a simplified version of the protocol initialization rule from reference [16].

$$ \begin{array}{l} ([\textsf{iid}\mid \textsf{pxs}\mid \textsf{ri}\mid \textsf{Trec}] ~ [\textsf{sid}\mid \textsf{pxs}' \mid \textsf{rs}], \textsf{b}) \rightarrow \\ ([\textsf{iid}\mid \textsf{px}(\textsf{num},\textsf{rp})~ \textsf{pxs}\mid \textsf{ri}^\nu \mid \textsf{Trec}] ~ [\textsf{sid}\mid \textsf{px}(\textsf{num},\textsf{rp})~ \textsf{pxs}' \mid \textsf{rs}^\nu ], \textsf{b}\wedge \textsf{b}_I)\\ \end{array} $$

This rule specifies that the intruder $\textsf{iid}$ with $\textsf{ri}$ resources creates a new burst of protocol session instances $\textsf{px}(\textsf{num},\textsf{rp})$ with $\textsf{num}$ instances each using $\textsf{rp}$ resources, where $\textsf{num}$ is a symbol. These instance requests are received by the server $\textsf{sid}$ which has $\textsf{rs}$ resources. The resources of the intruder, $\textsf{ri}$, and the resources of the server $\textsf{rs}$ are updated to the fresh symbols $\textsf{ri}^\nu$ and $\textsf{rs}^\nu$. These symbols are constrained by the boolean increment $\textsf{b}_I$ defined as $\textsf{ri}^\nu = (\textsf{ri}- \textsf{num}\times \textsf{rp}) \wedge \textsf{rs}^\nu = (\textsf{rs}- \textsf{num}\times \textsf{rp}) \wedge \textsf{num}> 0 \wedge \textsf{ri}^\nu \ge 0$. Similar rules specify when the protocol sessions timeout and are cleaned up by the server thus releasing resources.

The verification property is to check whether a bounded intruder with some limited number of resources $\textsf{ri}$ can deny service by consuming the server $\textsf{sid}$’s resources. This can be expressed by an incremental reachability property as follows where $(\textsf{t}_0,\textsf{b}_0)$ specifies the initial condition when all intruder and server resources are free:

$$ \begin{array}{l} \textsf {search}(\textsf{t}_0,\textsf{b}_0) \Rightarrow ([\textsf{iid}\mid \textsf{pxs}\mid \textsf{ri}\mid \textsf{Trec}] ~ [\textsf{sid}\mid \textsf{pxs}' \mid \textsf{rs}],\textsf{b}') \textsf { such that }\textsf{SMT}(\textsf{b}' \wedge \textsf{b}_I) \end{array} $$

where $\textsf{b}_I$ is the constraint $\textsf{rs}\le 0$ specifying that the resources of the server $\textsf{sid}$ are depleted.

Example 3

This example of verification of cyber-physical systems (CPSes) is based on reference [13]. A CPS is represented by a set of agents ($\textsf{ag}_1, \ldots , \textsf{ag}_n$) that interact with the environment ($\textsf{env}$) to achieve some goal while not violating properties, such as the minimum distance to other objects.

Constraints are used to specify agent’s physical attributes, such as its position, $\textsf{at} (\textsf{ag},(x,y))$, speed, $\textsf{spd} (\textsf{ag},v)$, acceleration, $\textsf{acc} (\textsf{ag},acc)$, and direction $\textsf{dir} (\textsf{ag},dir)$ of an agent $\textsf{ag}$. The evolution of a system with one agent can be specified by the following incremental rule when assuming, for simplicity, that the agent’s direction is on the x-axis.

$$ \begin{array}{l} ([\textsf{env}\mid \textsf{at} (\textsf{ag},(x,y)), \textsf{spd} (\textsf{ag},v), \textsf{acc} (\textsf{ag},acc), \textsf{dir} (\textsf{ag},dir), \textsf{kb} ]~ \textsf{conf}, \textsf{b}) \rightarrow \\ \qquad ([\textsf{env}\mid \textsf{at} (\textsf{ag},(x_1,y_1)), \textsf{spd} (\textsf{ag},v_1), \textsf{acc} (\textsf{ag},acc), \textsf{dir} (\textsf{ag},dir), \textsf{kb} ]~ \textsf{conf}, \textsf{b}\wedge \textsf{b}_I) \end{array} $$

Here $\textsf{kb}$ is the set of other knowledge-base elements, $\textsf{conf}$ contains the agent’s internal representation, $x_1,y_1,v_1$ are fresh symbols and $\textsf{b}_I$ is set of constraints: $x_1 = (x + (v + v_1) \times \textsf{dt}/ 2) \wedge y_1 = y \wedge v_1 = v + \textsf{acc} \times \textsf{dt}$. These constraints specify the agent’s new position and speed using classical physics equations.

The verification property $\textsf{bad}$ where an agent is too close to an obstacle, such as a pedestrian, is specified by the search command:

$$ \begin{array}{l} \textsf {search}(\textsf{t}_0,\textsf{b}_0) \Rightarrow \\ \qquad ([\textsf{env}\mid \textsf{at} (\textsf{ag}_1,(x_1,y_1)), \textsf{at} (\textsf{ag}_2,(x_2,y_2)), \textsf{kb} ]~ \textsf{conf},\textsf{b}') \textsf { such that }\textsf{SMT}(\textsf{b}' \wedge \textsf{b}_I) \end{array} $$

where $\textsf{b}_I$ is the set of constraints: $x_1 = x_2 \wedge y_1 = y_2$, specifying that two agents $\textsf{ag}_1$ and $\textsf{ag}_2$ are in the same position, i.e., colliding.

3 Hybrid BFS-DFS Algorithm

The definition of Incremental Rewrite Theories addresses the problem of the Definition of Increments discussed above. The second problem (Not possible to chain incremental calls) still needs to be addressed. Indeed, BFS procedures do not enable the chaining of incremental calls. To illustrate this, consider again the search tree and BFS execution in Fig. 1. Assume that $\textsf{b}_1 = \textsf{b}_0 \wedge \textsf{b}_{0,1}, \textsf{b}_2 = \textsf{b}_0 \wedge \textsf{b}_{0,2}$ and that $\textsf {goalCond}(\textsf{t},\textsf{b})$ has the form $\textsf{b}\wedge \textsf{b}_I$ as one would expect when using incremental rewrite theories. It is possible to call the SMT solver incrementally during the sequence of calls $\textsf{SMT}(\textsf{b}_1)$ and $\textsf{SMT}(\textsf {goalCond}(\textsf{t}_1,\textsf{b}_1))$, but not chain incrementally the call $\textsf{SMT}(\textsf{b}_2)$. This is because it is not possible to define an increment between $\textsf{b}_1$ and $\textsf{b}_2$ as they lie in different branches of the search tree.

The first obvious alternative is using Depth-First Search (DFS) instead of BFS. This would indeed lead to an execution that could chain incremental calls to the SMT solver. For example, in the tree depicted in Fig. 1, the sequence of calls would be

$$ \begin{array}{l} \textsf{SMT}(\textsf{b}_0) ; \textsf{SMT}(\textsf {goalCond}(\textsf{t}_0,\textsf{b}_0)) ; \textsf{SMT}(\textsf{b}_1) ; \textsf{SMT}(\textsf {goalCond}(\textsf{t}_1,\textsf{b}_1)) ; \\ \textsf{SMT}(\textsf{b}_3) ; \textsf{SMT}(\textsf {goalCond}(\textsf{t}_3,\textsf{b}_3)) \ldots \end{array} $$

Since $\textsf{b}_3$ is of the form $\textsf{b}_0 \wedge \textsf{b}_{0,1} \wedge \textsf{b}_{1,3}$, we know the increment is $\textsf{b}_{1,3}$. There are, however, two problems with DFS. The first problem is that DFS may not find a solution that could be found using BFS due to an infinite branch. The second problem is that the sequence of call using $\textsf {goalCond}(\textsf{t},\textsf{b})$ appears in between the increments, e.g., $\textsf{SMT}(\textsf{b}_0) ; \textsf{SMT}(\textsf {goalCond}(\textsf{t}_0,\textsf{b}_0)) ; \textsf{SMT}(\textsf{b}_1)$.

Fig. 2.

Pseudo-code of the Hybrid Search Algorithm $\mathsf {hybrid\_search}$.

We propose the algorithm $\mathsf {hybrid\_search}$ described in Fig. 2 that addresses these two problems of DFS by combining BFS and DFS and using the $\textsf{PUSH}$ and $\textsf{POP}$ features of SMT solvers for incremental solving. These features enable the creation of backtracking scopes of learned clauses. By default, sequential calls to $\textsf{SMT}$ will attempt to use incremental solving based on the constraints solved in previous calls. A call to $\textsf{PUSH}$ will add to the solver stack any learned clauses from calls to $\textsf{SMT}$ while a call to $\textsf{POP}$ will remove any learned clauses since the last $\textsf{PUSH}$.

The $\mathsf {hybrid\_search}$ algorithm takes as input the search tree T³, a non-negative natural number d, and a goal condition g. Intuitively, the parameter d specifies the depth to which the algorithm shall perform DFS before switching to BFS.

We start with $Queue$ empty and a $Solver$. $\mathsf {hybrid\_search}$ starts at line 4 with the next few lines initializing $found$ to be NULL and pushing the root of T onto $Queue$. The while loop starts with line 7 continuing while $Queue$ is non empty and no solution has been found. It pops the next node off the $Queue$ on line 8, then calls $\mathsf {dfs\_bounded}$ on the next line using this node as the root starting on line 12. $\mathsf {dfs\_bounded}$ is a modified depth-bounded depth-first search. It starts with creating a backtracking scope on $Solver$ by calling $\textsf{PUSH}$ and storing the result $\textsf{SMT}(b)$ where b is the boolean constraint of the current node.

Subsequently, in line 16, it checks if $\textsf{SMT}(b)$ returned UNSAT, and if so, we $\textsf{POP}$ and return immediately and not explore any children of this node. Any descendent nodes would have a boolean constraint of the form $b \wedge b_I$ for some $b_I$, and since $\textsf{SMT}(b)$ is UNSAT it must be the case that $b \wedge b_I$ is also UNSAT. Otherwise, we continue with checking if goal(node) is true on line 20 and if so setting $found$ to this node and then terminating $\mathsf {dfs\_bounded}$ and $\mathsf {hybrid\_search}$. If $found$ is not set, then line 24 checks when the current depth is equal to the depth parameter d and if it is we add all of the children nodes, i.e., all the nodes that are $d+1$ depth away from the initial root node called from line 9, to $Queue$ and no more nodes at a lower depth are visited for now. After all such nodes are added, the execution returns to line 7 to start another $\mathsf {dfs\_bounded}$ from the next element in $Queue$. Until then, it continues traversing the tree in a DFS-like manner on line 30 ensuring that when $\mathsf {dfs\_bounded}$ backtracks, we call $\textsf{POP}$ for each node, and hence it backtracks such that $Solver$ can properly unlearn clauses that it no longer needs.

Fig. 3.

Illustration of an $\mathsf {hybrid\_search}$ algorithm execution using the goal condition $\textsf{g}$ and depth two. The $\textsf{POP}$ surrounded by a box indicates the points when the algorithm back-tracks in the search tree. The numbers inside the circles specify the order in which nodes are traversed.

We illustrate the execution of $\mathsf {hybrid\_search}$ with the tree shown in Fig. 3. It also contains the sequence of calls to $\textsf{PUSH},\textsf{POP}$ and $\textsf{SMT}$ due to the initial call to $\mathsf {dfs\_bounded}$. The sequence of calls illustrates the chaining of incremental calls to the SMT solver. For example, the data-structures constructed in the call $\textsf{SMT}(\textsf{b}_1)$ are used in the SMT calls for $\textsf{b}_3, \textsf{b}_4$, including the calls $\textsf{goal}(\textsf{b}_3)$ and $\textsf{goal}(\textsf{b}_4)$. This makes sense as $\textsf{b}_1$ is sub-formula of $\textsf{b}_3$, $\textsf{b}_4$, $\textsf{goal}(\textsf{b}_3)$ and $\textsf{goal}(\textsf{b}_4)$. However, the data-structures constructed in the SMT call for $\textsf{goal}(\textsf{b}_1)$ are not stored due to the subsequent $\textsf{POP}$ call, as $\textsf{goal}(\textsf{b}_1)$ is not necessarily a subformula of $\textsf{b}_3$, $\textsf{b}_4$, $\textsf{goal}(\textsf{b}_3)$ and $\textsf{goal}(\textsf{b}_4)$. The second observation is the combination of DFS and BFS. While the subtree of depth $d = 2$ is traversed, the algorithm removes the data-structures constructed during the call of $\textsf{SMT}(\textsf{b}_1)$, indicated by the $2 \times \textsf{POP}$ in Fig. 3, as $\textsf{b}_1$ is not necessarily a subformula of $\textsf{b}_2$.

Notice that the depth parameter (d) plays the role of specifying how much incremental solving one is willing to use with the risk of traversing longer a branch of the search tree that may not have a solution. For example, in the tree and execution shown in Fig. 3, the algorithm will traverse the node $(\textsf{t}_7,\textsf{b}_7)$ and will call $\textsf{SMT}(\textsf{b}_7)$, but without using the data-structures constructed previously for $\textsf{b}_3$, that is, it will not solve it incrementally.

The following results relate $\mathsf {hybrid\_search}$ with BFS and with DFS.

Proposition 1

Let T be a tree and g be a decidable goal condition. Then, $\mathsf {hybrid\_search}(T, 0, g)$ will traverse T in the same order as BFS.

Proof Sketch. A DFS search bounded by depth 0 will only traverse a single node, the node it starts at. Then, it adds nodes to a FIFO queue in the same manner as BFS. Hence, $\mathsf {hybrid\_search}(T, 0, g)$ will traverse T in the same order as BFS. QED.

Proposition 2

Let T be a tree and g be a decidable goal condition. Suppose the depth of T is d. Then, for any $k \ge d$, $\mathsf {hybrid\_search}(T, k, g)$ will traverse T in the same order as DFS.

Proof Sketch. If k is greater than or equal to the depth of T, then a k depth-bounded DFS from the root node would traverse all of T. Hence, $\mathsf {hybrid\_search}(T, k, g)$ traverses the T in the same order as DFS. QED.

The following statement provides coverage guarantees.

Proposition 3

Let $d>0$, T be a tree of finite branching, and g be a decidable goal condition. Then, $\mathsf {hybrid\_search}(T, d, g)$ finds a solution in finite time, i.e., some node n in T such that g(n) is true, if such a solution exists.

Proof. Let $B_i$ be the number of nodes in T at depth i. Suppose that the solution node n exists at depth r and no solutions exist at a lower depth. Let $0 \le r \le qd$ for some q. The first depth-bounded DFS will traverse all nodes up to depth d. This then adds $B_{d+1}$ nodes to Queue. Running the depth-bounded DFS run these nodes will traverse all the nodes to 2d. Traversing all nodes up to qd would take $1 + B_{d+1} + B_{d+2} + ... + B_{qd}$ iterations of depth-bounded depth first searches. Since n exists at depth $r \le qd$ and each $B_i$ is finite since T has finite branching, n would be found in finite time. QED.

To address the fact that search trees may have infinite depth, often one uses bounded search that searches the tree until only some given depth d. The following proposition states that in these cases it is best to deploy $\mathsf {hybrid\_search}$ with depth d to search through all nodes of the sub-tree, provided incremental SMT calls are more efficient than SMT calls from scratch.

Proposition 4

Let T be a tree of finite branching with branching factor b and g be a decidable goal condition. Let T(d) be the sub-tree of T of depth d with $d > 0$. Assume that incremental SMT calls, i.e., using $\textsf{PUSH}$, take less time than calls from scratch, i.e., without using $\textsf{PUSH}$. Then for any $d' \ge 0$ such that $d' \ne d$, the time required by $\mathsf {hybrid\_search}(T,d,g)$ to traverse all nodes in T(d) is less than the time of $\mathsf {hybrid\_search}(T,d',g)$ to traverse all nodes in T(d).

Proof. Let $0< r < 1$ be the average performance benefit from incremental SMT calls and t be the time it takes for non-incremental SMT calls. Let $B_i$ be the number of nodes at depth i. Since b is finite, each $B_i$ is finite. The time required by $\mathsf {hybrid\_search}(T, d, g)$ to traverse all nodes in T(d) is $t + rtB_1 + rtB_2 + ... + rtB_d$. Suppose that $0< d' < d$. Let $pd' < d \le (p+1)d'$ for some p. For $\mathsf {hybrid\_search}(T, d', g)$ to traverse all nodes in T(d), it must traverse all nodes in $T((p+1)d')$ because each $\mathsf {dfs\_bounded}$ must travel exactly $d'$ depth, $\mathsf {hybrid\_search}(T, d', g)$ will traverse only depths that are multiples of $d'$. Then, the time required for $\mathsf {hybrid\_search}(T, d', g)$ is $t + rtB_1 + ... + rtB_{d'} + tB_{d'+1} + rtB_{d'+2} + ... rtB_{2d'} + ... + tB_{pd'} + rtB_{pd'+1} + ... + rtB_{(p+1)d'}$. There are $p+1$ terms that do not get the benefit from incremental SMT calls for $\mathsf {hybrid\_search}(T, d', g)$ while there is 1 term that does not get this benefit for $\mathsf {hybrid\_search}(T, d, g)$. Hence, the time required for $\mathsf {hybrid\_search}(T, d, g)$ to traverse all nodes in T(d) is less than the time required for $\mathsf {hybrid\_search}(T, d', g)$ to traverse all nodes in T(d). Now, suppose that $d' > d$. Then, for $\mathsf {hybrid\_search}(T, d', g)$ to traverse all nodes in T(d), it must traverse all nodes in $T(d')$. The time required for $\mathsf {hybrid\_search}(T, d', g)$ is $t + rtB_1 + rtB_2 + ... + rtB_{d'}$. But, because $d' > d$ and each $rtB_i > 0$ the time required for $\mathsf {hybrid\_search}(T, d, g)$ is less than $\mathsf {hybrid\_search}(T, d', g)$. Hence, the time required for $\mathsf {hybrid\_search}(T, d, g)$ to traverse all nodes in T(d) is less than the time required for $\mathsf {hybrid\_search}(T, d', g)$ to traverse all nodes in T(d). Therefore, for any $d' \ne d$ the the time required for $\mathsf {hybrid\_search}(T, d, g)$ to traverse all nodes in T(d) is less than the time required for $\mathsf {hybrid\_search}(T, d', g)$ to traverse all nodes in T(d). QED.

4 Implementation and Experiments

Our implementation is based on Python with the Z3 SMT solver and Maude integrated using Python bindings [15] as depicted in Fig. 4. The Z3 Solver is responsible for checking the incremental satisfiability of constraints using $\textsf{SMT}$, $\textsf{PUSH}$ and $\textsf{POP}$, while Maude is responsible for executing rewriting rules. The Maude bindings allow for loading Maude files into the Python implementation of $\mathsf {hybrid\_search}$. The search is done with a Python function that repeatedly calls the Maude search with one step (Search1) so that the traversal of the search space can be controlled. The original Maude specifications were modified to replace calls to $\textsf{SMT}$ with calls to functions defined using the Maude hook mechanism for attaching external code to function symbols. This mechanism is exposed by the Maude Python bindings. There are two types of function, one that checks satisfiability while keeping any learned clauses from the check, and one that just checks without adding any learned clauses. The functions keep track of the SMT solver state using appropriate calls to $\textsf{PUSH}$ and $\textsf{POP}$. The implementation is available at [17].

Fig. 4.

Overview of the implementation used for the experiments using $\mathsf {hybrid\_search}$, the SMT solver Z3 and the rewriting tool Maude.

Figures 5, 6 and 7 summarize the experiments carried out using implementations available in the literature [3, 13, 16] for the verification of the systems described in Examples 1, 2, and 3. All experiments were run on a Windows 10 machine, Intel Core i7-10700J, 16 GB of RAM, on Python 3.10.2, using Maude Python bindings 1.1.2 and Z3 4.11.2.0. We measure the runtime for these three applications of rewriting modulo SMT to determine the performance gain from using hybrid search at various depth parameters compared to BFS and DFS. Each table shows the initial configuration for the system, then statistics for searches for BFS, DFS, and using $\mathsf {hybrid\_search}$ at various depths terminating when finding a single goal node. The statistics have the form n/m/p which specify the time n in seconds to perform verification, the number of states m traversed, and the percentage p of verification time required by SMT-solving. DNF indicates that no solution was found within 30 min. For example, the first row for $\textsf {cashOK}_1$ using the BFS mechanism for instance, the execution time was 6.9 seconds, requiring 91 state traversals while spending $77\%$ of execution time in Z3.

For our experiments, we used the same subsets of the verification problems used in references [3, 13, 16]:

• $\textsf {cashOK}(I_0,I_1,I_2,I_3,\textsf{b})$ and $\textsf {cashBad}(I_0,I_1,I_2,I_3,\textsf{b})$ correspond to symbolic initial configurations of a CASH scheduling problem with two servers (see Example 1). $I_0$ and $I_1$ specify, respectively, the maximum budget and the period of the first server, while $I_2$ and $I_3$ specify, respectively, the maximum budget and period of the second server. $\textsf{b}$ is a constraint on the values of $I_1,I_2,I_3,$ and $I_4$. $\textsf {cashOK}$ uses a correct implementation of the scheduler, while $\textsf {cashBad}$ uses an incorrect specification.

• $\textsf {Slowloris}(P_1,P_2,\textsf {DoSDur})$ corresponds to symbolic initial configurations of a Slowloris verification problem (see Example 2). $P_1$ specifies the bound on the number of parallel bursts of symbolic protocols, and $P_2$ specifies the bound on the number of different types of messages sent in parallel, where $P_2 = 0$ denotes no bound. Moreover, $\textsf {DoSDur}$ specifies the minimum duration for which the server’s resources are depleted in order to consider the DoS attack successful.

• $\textsf {pedestrian}(t,\textsf {Safer},\textsf {Safe},\textsf {Unsafe})$ specifies a pedestrian crossing scenario problem where an autonomous vehicle is approaching a pedestrian crossing. The verification problem is to avoid an unsafe situation. The three levels of safety are defined according to the parameters $\textsf {Safer}> \textsf {Safe}> \textsf {Unsafe}$ specifying bounds on the distance to between the vehicle and the pedestrian measured in terms of time to travel. The verification problem is to determine whether a given vehicle controller cannot reach an unsafe situation within t time units when starting at a safe situation. The size of a time unit is 0.1s.

Fig. 5.

CASH Verification Experiments. $\textsf {cashOK}_1 = \textsf {cashOK}(I0, I1, I2, I3, true)$, $\textsf {cashOK}_2 = \textsf {cashOK}(I0, I1, I2, I3, I0 + I3 > I1 + I2)$, and $\textsf {caseOK}_3 = \textsf {caseOK}(I0, I1, I2, I1, I0 + I2 > I1)$, and mutatis mutandis for $\textsf {cashBad}_1$, $\textsf {cashBad}_2$ and $\textsf {cashBad}_3$.

Fig. 6.

Slowloris Experiments. $\textsf {Slow}_1 = \textsf {Slowloris}(1, 0, 24)$, $\textsf {Slow}_2 = \textsf {Slowloris}(1, 0, 36)$, $\textsf {Slow}_3 = \textsf {Slowloris}(1, 1, 12)$, $\textsf {Slow}_4 = \textsf {Slowloris}(1, 1, 24)$, $\textsf {Slow}_5 = \textsf {Slowloris}(1, 1,36)$.

Fig. 7.

Cyber-Physical System Verification Experiments, where $\textsf {cps}_1 = \textsf {pedestrian}(3,3,2,1)$, $\textsf {cps}_2 = \textsf {pedestrian}(4,3,2,1)$, $\textsf {cps}_3 = \textsf {pedestrian}(5,3,2,1)$, $\textsf {cps}_4 = \textsf {pedestrian}(3,4,2,1)$, $\textsf {cps}_5 = \textsf {pedestrian}(4,4,2,1)$, $\textsf {cps}_6 = \textsf {pedestrian}(5,4,2,1)$. The bound t, $2\times t$ and $3 \times t$ is determined according to the t parameter of the scenario.

The results for the CASH verification experiments show that $\mathsf {hybrid\_search}$ finishes up to about 10 times faster than BFS and terminates in all cases as opposed to two of the DFS cases where it does not finish within 30 min. The overhead of Z3 is reduced from about 70% to 80% down to 6% to 25% from BFS to $\mathsf {hybrid\_search}$. This indicates the effectiveness of the incremental SMT solving for the types of constraints used in this example.

Similarly, in the Slowloris examples, $\mathsf {hybrid\_search}$ finishes up to 10 times faster than BFS with termination while two of the DFS cases do not finish within 30 min. In these cases the overhead of Z3 goes from about 80% to 90% in BFS while it goes from about 30% to 60% in $\mathsf {hybrid\_search}$, demonstrating the effectiveness of the incremental solving. Interestingly, even when there is a much larger number of states traversed, e.g., in case $\textsf {Slow}_2$ and HYBRID d = 4 with 3314 states traversed as opposed to 775 states traversed by BFS, the verification time is one third, from about 40s to 13s. This indicates that the main overhead of BFS is indeed SMT solving.

For the Cyber-Physical System (CPS) Verification experiments, $\mathsf {hybrid\_search}$ completes up to about 5 times faster than BFS. The overhead of Z3 does not change significantly in these experiments, which indicates that the incremental solving is not as effective as in the other two examples (CASH and Slowloris). The reason for this may be the non-linear nature of the constraints for CPS systems which contrast with the former two examples that use linear arithmetic constraints. Despite this, $\mathsf {hybrid\_search}$ and DFS still outperform BFS because they need to traverse fewer nodes before finding a goal node.

5 Related Work

We consider three related areas of work in optimizing symbolic execution modulo SMT, hybrid search strategies, incremental constraint solving methods, and tradeoffs between search space and constraint complexity.

Hybrid Search Strategies. There have been others that have previously explored techniques of combining BFS and DFS so to take advantage of both of their benefits while reducing the drawbacks of each.

Reference [5] proposes a hybrid algorithm for Binary Decision Diagrams (BDDs). BDDs are are often used to represent and manipulate boolean functions symbolically. Traditionally, depth-first approaches were used in the construction of BDDs as it had relatively low memory overhead. Though, it had been discovered that using a breadth-first approach instead had better performance due to better memory access locality at the cost of larger memory overhead. To improve upon both approaches a hybrid of the two is used. Essentially, the algorithm switches between the two techniques based on its memory overhead. When the memory overhead is computed to be low, a breadth-first search is used and when it is high a depth-first search is used.

Reference [1] constructs a “breadth-first, depth-next” algorithm for building Random Forest (RF) models. An RF model is a machine learning model that uses decision trees. Both DFS and BFS approaches are used in machine learning frameworks. They observe that BFS has memory efficient access patterns at lower depths. As the depth increases it loses this benefit and virtually has random access to memory. At this point, DFS performs better. As a result, their algorithm starts with a breadth-first approach until it is computed that is no longer has efficient access pattern, switching to a depth-first approach.

Reference [9] introduces “depth-first iterative-deepening (DFID).” One of the issues with BFS is that it has exponential memory complexity. DFS can circumvent this drawback as its memory complexity is linear, but comes with its own problems. It generally requires some depth bound and check for repeated nodes, otherwise the search may not terminate. The actual depth bound needed may not be knowable at runtime and choosing a bound too low may result in the search ending without finding the solution. To counteract the downsides of BFS and DFS, DFID is used. DFID starts with DFS bounded by depth one, then performs a DFS bounded by depth two, and continue this process with incrementally larger bounded depths until a solution is found. It must visit the same nodes multiple times, but it is shown that the runtime complexity is not effected by it.

Unfortunately, none of these algorithms seem particularly helpful with respect to rewriting modulo SMT. For example, prior algorithms [1, 5] attempt to take advantage of memory locality as much as possible. In our case, it would not give us much performance increase. Reference [9] requires nodes to be visited multiple times. This would lead to duplicate calls the SMT solver, only increasing the bottleneck.

Incremental Solving. In reference [10] the authors compare cache-based and stack-based incremental constraint solving methods in the context of symbolic execution for test generation. Cached-based incrementality works outside the solver to cache results and attempt to reuse them. Stack-based incrementality uses a solvers ability to reuse information learned when solving a subproblem and the associated push/pop interface. Implementations of the two methods and a baseline (no incrementality) were compare on large benchmark set of C programs and on randomly generated programs. The space of symbolic execution paths was searched using bounded depth first search. The authors found that caching generally increased average solving time over baseline (by a factor of 2–5 depending on code size), while stack-based methods decreased average solving time by roughly a factor of 20. This is consistent with our observations even though the source of search tree is different and the class of constraints is different.

Trading Search Space for Constraint Complexity. A notion of guarded term is introduced in reference [2] as a method to reduce the search state space in symbolic rewriting modulo SMT by replacing non-determinism by disjunction. The effect of using guarded terms is demonstrated in a study of the CASH algorithm for task scheduling. Many properties that could not be checked using symbolic execution modulo SMT (due to size of search space and timeout) became tractable using guarded terms.

A study of the tradeoff between search space size and constraint size using symbolic execution modulo SMT in the context of analyzing safety of autonomous systems such as platooning scenarios is presented in reference [13]. The results in that paper suggest that not only the size of state space matters for automation, but also the size of constraints that are sent to the SMT solver as many searches fail to terminate due to non-termination of constraint solving when constraints get large, while the same searches terminate with disjunctions are turned into branching in the search space.

None of these approaches, however, investigate the use of incremental SMT solving for improving performance of Rewriting Modulo SMT.

6 Conclusions and Future Work

This paper proposes Incremental Rewrite Theories that enable incremental SMT solving for rewriting modulo SMT. This is accomplished by the search procedure $\mathsf {hybrid\_search}$ which combines BFS and DFS. The effectiveness of $\mathsf {hybrid\_search}$ is demonstrated by using a collection of verification problems taken from the literature, including algorithm verification, network security analysis, and cyber-physical systems safety verification. In all examples, the time taken to verify by $\mathsf {hybrid\_search}$ improved by a factor between 5–10 when compared to traditional BFS approaches, showing the great benefits of using incremental solving.

The current notion of incremental rewrite theory is essentially a syntactic notion although equational theories are used to reduce terms and matching may be modulo axioms, such as associativity and commutativity. This makes identifing the boolean increment efficient and thus well suited for the hybrid algorithm. An interesting direction for future work is to investigate less restrictive notions of incremental and indentify more general classes of rewrite theories where incremental solving is effective. Another direction of future work we are investigating is the trade-offs of incremental solving and the shape of constraints, e.g., use disjunctions to reduce search space versus split disjunctions to reduce SMT solving time. We also are investigating the incorporation of incremental solving algorithms in tool implementations such as Maude.