In the following we describe our extension of the formalization of Spasić and Marić through the integration of minimal unsatisfiable cores (Sect. 3.1), the integration of an optimization during Phase 2 (Sect. 3.2) and the development of an incremental interface to the simplex algorithm (Sect. 3.3).
3.1 Minimal Unsatisfiable Cores
Our first extension is the integration of the functionality for producing unsatisfiable cores, i.e., given a set of unsatisfiable constraints, we seek a subset of the constraints which is still unsatisfiable. Small unsatisfiable cores are crucial for a DPLL(T)-based SMT solver in order to derive small conflict clauses, hence it is desirable to obtain minimal unsatisfiable cores, of which each proper subset is satisfiable. For example, in Fig. 2, \(\{A,B,C\}\) is a minimal unsatisfiable core. We will refer to this example throughout this section.
Internally, the formalized simplex algorithm represents the data available on Layer 3 in a data structure called a state, which contains the current tableau, valuation, the set of asserted atoms,Footnote 2 and an unsatisfiability flag. Unsatisfiability is detected by the check operation in Phase 3, namely if the current valuation of a state does not satisfy the atoms, and pivoting is not possible.Footnote 3 For instance, in step 7 unsatisfiability is detected as follows: The valuation \(v_3\) does not satisfy the atom \(x \ge 5 + \delta \) since \(v_3(x) = \frac{9}{2}\). The pivoting procedure looks at the tableau equation for x,
$$\begin{aligned} x = \frac{1}{2}s - \frac{1}{2}y, \end{aligned}$$
(1)
and checks whether it is possible to increase the value of x. This is only possible if the valuation of s in increased (since s occurs with positive coefficient in (1)), or if y is decreased (since y occurs with a negative coefficient). Neither is possible, because \(v_3(s)\) is already at its maximum (\(s \le 12\)) and \(v_3(y)\) at its minimum (\(y \ge 3\)). Hence, in order prove unsatisfiability on Layer 3, it suffices to consider the tableau and the atoms \(\{x \ge 5 + \delta , s \le 12, y \ge 3\}\).
We formally verify that this kind of reasoning works in general: Given the fact that some valuation v of a state does not satisfy an atom \(x \ge c\) for some left-hand side variable x, we can obtain the corresponding equation \(x = p\) of the tableau T, and take the unsatisfiable core as the set of atoms formed of: \(x \ge c\), all atoms \(y \ge v(y)\) for variables y of p with coefficient \(<0\), and all atoms \(s \le v(s)\) for variables s of p with coefficient \(>0\). The symmetric case \(x \le c\) is handled similarly by flipping signs.
We further prove that the generated cores are minimal w.r.t. the subset relation: Let A be a proper subset of an unsatisfiable core. There are two cases. If A does not contain the atom of the left-hand side variable x, then all atoms in A only contain right-hand side variables. Then by the invariant of the simplex algorithm, the current valuation satisfies both the tableau T and A. In the other case, some atom with a variable z of p is dropped. But then it is possible to apply pivoting for x and z. Let \(T'\) be the new tableau and v be the new valuation after pivoting. At this point we use the formalized fact that pivoting maintains the invariant. In particular, \(v \models T'\) and \(v \models A\), where the latter follows from the fact that A only contains right-hand side variables of the new tableau \(T'\) (note that x and z switched sides in the equation following pivoting). Since T and \(T'\) are equivalent, we conclude that v satisfies both T and A.
In the formalization, the corresponding lemma looks as follows:
The assumptions in the lemma express precisely the invariant of the simplex algorithm, and the lemma states that whenever the check operation sets the unsatisfiability flag \(\mathcal U\), then indeed a minimal unsatisfiable core is stored in the new state
. Whereas the assumptions have been taken unmodified from the existing simplex formalization, we needed to modify the formalized definition of the check operation and the datatype of states, so that
can compute and store the unsatisfiable core in the resulting state.
At this point, we have assembled a verified simplex algorithm for Layer 3 that will either return satisfying valuations or minimal unsatisfiable cores. The next task is to propagate the minimal unsatisfiable cores upwards to Layer 2 and 1, since, initially, the unsatisfiable cores are defined in terms of the data available at Layer 3, which is not meaningful when speaking about the first two layers. A question that arises here is how to represent unsatisfiable cores. Taking the constraints literally is usually not a desirable solution, as then we would have to convert the atoms \(\{x \ge 5 + \delta , s \le 12,y \ge 3\}\) back to the non-strict constraints \(\{x \ge 5 \,+\, \delta , 2x \,+\, y \le 12, 2y \ge 6\}\) and further into \(\{x > 5, 2x \,+\, y \le 12, 2y \ge 6\}\), i.e., we would have to compute the inverses of the transformations in Phases 2 and 1. A far more efficient and simple solution is to use indexed constraints in the same way, as they already occur in the running example. Hence, the unsatisfiable core is just a set of indices (\(\{A,B,C\}\) in our example). These indices are then valid for all layers and do not need any conversion.
Since the formalization of Spasić and Marić does not contain indices at all, we modify large parts of the source code so that it now refers to indexed constraints, i.e., we integrate indices into algorithms, data structures, definitions, locales, properties and proofs. For instance, indexed constraints ics are just sets of pairs, where each pair consists of an index and a constraint, and satisfiability of indexed constraints is defined as
$$\begin{aligned} (I,v) \models ics \qquad \text { if and only if }\qquad v \models \{ c \mid (i,c) \in ics \wedge i \in I\}, \end{aligned}$$
where I is an arbitrary set of indices.
In order to be able to lift the unsatisfiable core from Layer 3 to the upper layers, we have to prove that the two transformations (elimination of strict inequalities and introduction of slack variables) maintain minimal unsatisfiable cores. To this end, we modify existing proofs for these transformation, since they are not general enough initially. For instance, the soundness statement for the introduction of slack variables in Phase 2 states that if the transformation on non-strict constraints N produces the tableau T and atoms A, then N and the combination of T and A are equisatisfiable, i.e.,
$$\begin{aligned} (\exists v.\ v \models N) \longleftrightarrow (\exists v.\ v \models T \wedge v \models A). \end{aligned}$$
However, for lifting minimal unsatisfiable cores we need a stronger property, namely that the transformation is also sound for arbitrary indexed subsets I:Footnote 4
$$\begin{aligned} (\exists v.\ (I,v) \models N) \longleftrightarrow (\exists v.\ v \models T \wedge (I,v) \models A). \end{aligned}$$
(2)
Here, the indexed subsets in (2) are needed for both directions: given a minimal unsatisfiable core I of T and A, by the left-to-right implication of (2) we conclude that I is an unsatisfiable core of N, and it is minimal because of the right-to-left implication of (2). Note that tableau satisfiability (\(v \models T\)) is not indexed, since the tableau equations are global.
Our formalization therefore contains several new generalizations, e.g., the following lemma is the formal analogue to (2), where
is the function that introduces slack variables. In addition to the tableau
and the indexed atoms
, it also provides a computable function
to convert satisfying valuations for
and
into satisfying valuations for
.
After all these modifications we obtain a simplex implementation that indeed provides minimal unsatisfiable cores. The corresponding function
returns a sum type, which is either a satisfying valuation or an unsatisfiable core represented by a set of indices.
Here, the minimality of the unsatisfiable cores can only be ensured if the indices in the input constraints are distinct. That distinctness is essential can easily be seen: Consider the following indexed constraints \(\{(E, x \le 3), (F, x \le 5), (F, x \ge 10)\}\) where index F refers to two different constraints. If we invoke the verified simplex algorithm on these constraints, it detects that \(x \le 3\) is in conflict with \(x \ge 10\) and hence produces \(\{E,F\}\) as an unsatisfiable core. This core is clearly not minimal, however, since \(\{F\}\) by itself is already unsatisfiable.
Some technical problems arise, regarding distinctness in combination with constraints involving equality. For example, the Layer 1-constraint \((G, p = c)\) will be translated into the two constraints \((G, p \ge c)\) and \((G, p \le c)\) on Layer 2,Footnote 5 violating distinctness. These problems are solved by weakening the notion of distinct constraints on Layers 2 and 3, and strengthening the notion of a minimal unsatisfiable core for these layers: For each proper subset J of the unsatisfiable subset, each inequality has to be satisfied as if it were an equality, i.e., whenever there is some constraint \((j, p \le c)\) or \((j, p \ge c)\) with \(j \in J\), the satisfying valuation must fulfill \(p = c\).
3.2 Elimination of Unused Variables in Phase 2
Directly after creating the tableau and the set of atoms from non-strict constraints in Phase 2, it can happen that there are unused variables, i.e., variables in the tableau for which no atoms exist.
Dutertre and de Moura propose to eliminate unused variables by Gaussian elimination [10, end of Section 3] in order to reduce the size of the tableau. We integrate this elimination of variables into our formalization. However, instead of using Gaussian elimination, we implement the elimination via pivoting. To be more precise, for each unused variable x we perform the following steps.
-
If x is not already a left-hand side variable of the tableau, find any equation \(y = p\) in the tableau that contains x, and perform pivoting of x and y, so that afterwards x is a left-hand side variable of the tableau.
-
Drop the unique equation from the tableau that has x on its left-hand side, but remember the equation for reconstructing satisfying valuations.
Example 1
Consider the non-strict constraints \(\{x + y \ge 5, x + 2y \le 7, y \ge 2\}\) on Layer 2. These are translated to the atoms \(\{s \ge 5, t \le 7, y \ge 2\}\) in combination with the tableau \(\{s = x + y, t = x + 2y\}\), so x becomes an unused variable. Since x is not a left-hand side variable, we perform pivoting of x and s and obtain the new tableau \(\{x = s - y, t = s + y\}\). Then we drop the equation \(x = s - y\) resulting in the smaller tableau \(\{t = s + y\}\). Moreover, any satisfying valuation v for the variables \(\{y,s,t\}\) will be extended to \(\{x,y,s,t\}\) by defining \(v(x) := v(s) - v(y)\).
In the formalization, the elimination has been integrated into the
function of Sect. 3.1. In fact,
just executes both preprocessing steps sequentially: first, the conversion of non-strict constraints into tableau and atoms, and afterwards the elimination of unused variables as described in this section. Interestingly, we had to modify the locale-structure of Spasić and Marić at this point, since preprocessing now depends on pivoting.
3.3 Incremental Simplex
The previous specifications of the simplex algorithm are monolithic: even if two (consecutive) inputs differ only in a single constraint, the functions
(in Sect. 2) and
(in Sect. 3.1) will start the computation from scratch. Hence, they do not specify an incremental simplex algorithm, despite the fact that an incremental interface is provided on Layer 3 via assert and check.
Since the incrementality of a theory solver is a crucial requirement for developing a DPLL(T)-based SMT solver, we will provide a formalization of the simplex algorithm that provides an incremental interface at each layer. Our design closely follows Dutertre and de Moura, who propose the following operations.
-
Initialize the solver by providing the set of all possible constraints. This will return a state where none of these constraints have been asserted.
-
Assert a constraint. This invokes a computationally inexpensive deduction algorithm and returns an unsatisfiable core or a new state.
-
Check a state. Performs an expensive computation that decides satisfiability of the set of asserted constraints; returns an unsat core or a checked state.
-
Extract a solution of a checked state.
-
Compute some checkpoint information for a checked state.
-
Backtrack to a state with the help of some checkpoint information.
Since a DPLL(T)-based SMT solver basically performs an exhaustive search, its performance can be improved considerably by having it keep track of checked states from which the search can be restarted in a different direction. This is why the checkpointing and backtracking functionality is necessary.
In Isabelle/HOL we specify this informal interface for each layer as a locale, which fixes the operations and the properties of that layer. For instance, the locale
is for Layer 1, where the type-variable \(\sigma \) represents the internal state for the layer, and \(\gamma \) is the checkpoint information.
The interface consists of the six operations
to invoke the algorithm, and the two invariants
and
, the latter of which entails the former.
Both invariants
and
take the three arguments
,
and
. Here,
is the global set of indexed constraints that is encoded in the state
. It can only be set by invoking
and is kept constant otherwise.
indicates the set of all constraints that have been asserted in the state
.
We briefly explain the specification of
and
and leave the usage of the remaining functionality to the reader.
For the
operation there are two possible outcomes. If the assertion of index
was successful, it returns a new state
which satisfies the same invariant as
, and whose set of indices of asserted constraints contains
, and is otherwise the same as the corresponding set in
. Otherwise, the operation returns a set of indices
, which is a subset of the set of indices of asserted constraints (including
), such that the set of all
-indexed constraints is a minimal unsatisfiable core.
The backtracking facility works as follows. Assume that one has computed the checkpoint information
in a state
, which is only permitted if
satisfies the stronger invariant for some set of indices
. Afterwards, one may have performed arbitrary operations and transitioned to a state
corresponding to a superset of indices
. Then, solely from
and
, one can compute via
a new state
that corresponds to the old set of indices
. Of course, the implementation should be done in such a way that the size of
is small in comparison to the size of
; in particular,
should not be
itself. And, indeed, our implementation behaves in the same way as the informally described algorithm by Dutertre and de Moura: for a checkpoint
of state
we store the asserted atoms of the state
, but neither the valuation nor the tableau. These are taken from the state
when invoking
.
In order to implement the incremental interface, we take the same modular approach as Spasić and Marić, namely that for each layer and its corresponding Isabelle locale, we rely upon the existing functionality of the various phases, together with the interface of the lower layers, to implement the locale.
In our case, a significant part of the work has already been done via the results described in Sect. 3.1: most of the generalizations that have been performed in order to support indexed constraints, play a role in proving the soundness of the incremental simplex implementation. In particular, the generalizations for Phases 1 and 2 are vital. For instance, the set of indices
in lemma
on page 9 can not only be interpreted as an unsatisfiable core, but also as the set of currently asserted constraints. Therefore,
allows us to convert a satisfying valuation on Layer 2 into a satisfying valuation on Layer 1 for the currently asserted constraints that are indexed by
. Consequently, the internal state of the simplex algorithm on Layer 1 not only stores the state of Layer 3 as it is described at the beginning of Sect. 3.1, but additionally stores the function
, in order to compute satisfying valuations on Layer 1.
We further integrate and prove the correctness of the functionality of checkpointing and backtracking on all layers, since these features have not been formalized by Spasić and Marić. For instance, when invoking
on Layer 3 with
, we obtain a new state that contains the tableau
and valuation
of state
, but the asserted atoms
of state
. Hence, we need to show that
satisfies those asserted atoms of
that correspond to right-hand side variables of
. To this end, we define the invariant on Layer 3 in a way that permits us to conclude that the tableaux
and
are equivalent. Using this equivalence, we then formalize the desired result for Layer 3. Checkpointing and backtracking on the other layers is just propagated to the next-lower layers, i.e., no further checkpointing information is required on Layers 1 and 2.
Finally, we combine the implementations of all phases and layers to obtain a fully verified implementation of the simplex algorithm w.r.t. the specification defined in the locale
.
Note that the incremental interface does not provide a function to assert constraints negatively. However, this limitation is easily circumvented by passing both the positive and the negative constraint with different indices to the
function. For example, instead of using \((A, x > 5)\) as in Fig. 2, one can use the two constraints \((+A, x > 5)\) and \((-A, x \le 5)\). Then one can assert both the original and the negated constraint via indices \(+A\) and \(-A\), respectively. Only the negation of equations is not possible in this way, since this would lead to disjunctions. However, each equation can easily be translated into the conjunction of two inequalities on the formula-level, i.e., they can be eliminated within a preprocessing step of the SMT-solver.