Abstract
We implement a decision procedure for linear mixed integer arithmetic and formally verify its soundness in Isabelle/HOL. We further integrate this procedure into one application, namely into CeTA, a formally verified certifier to check untrusted termination proofs. This checking involves assertions of unsatisfiability of linear integer inequalities; previously, only a sufficient criterion for such checks was supported. To verify the soundness of the decision procedure, we first formalize the proof that every satisfiable set of linear integer inequalities also has a small solution, and give explicit upper bounds. To this end we mechanize several important theorems on linear programming, including statements on integrality and bounds. The procedure itself is then implemented as a branchandbound algorithm, and is available in several languages via Isabelle’s code generator. It internally relies upon an adapted version of an existing verified incremental simplex algorithm.
Keywords
 Branchandbound
 Isabelle/HOL
 Linear programming
 Polyhedra
 Simplex algorithm
This research was supported by the Austrian Science Fund (FWF) project Y757. The authors are listed in alphabetical order regardless of individual contributions or seniority.
Download conference paper PDF
1 Introduction
The computational problem of deciding whether a system of linear inequalities with integer coefficients has an integral solution arises in many practical situations. Since it is NPcomplete, no currently known algorithm can in general avoid searches of exponential length. Furthermore, while satisfiable instances always have short solutions that can be efficiently checked, there need not be short, efficientlycheckable proofs for the fact that an instance is unsatisfiable, unless \(\textsc {NP}=\textsc {coNP}\). (Contrast this with the related problem of deciding whether a system of linear inequalities with integer coefficients has a rational solution – this problem is in P, and Farkas’ lemma provides a short and efficiently checkable certificate that an unsatisfiable instance indeed has no solution.) Thus, if a solver declares that a given instance is unsatisfiable over the integers, the length of any proof for this fact may be exponential in the size of the input instance, in which case the computational effort required to check such a proof would be exponential as well.
Instead of repeatedly performing certification tasks that require immense amounts of data and computational effort, it may be more fruitful to formally verify the soundness of a solver once, so that it can then be trusted without instancebyinstance certification of its output. The implementation of such a solver, together with a formal proof of its soundness, is the goal of the present work. Specifically, we use Isabelle/HOL [19] to implement and prove the correctness of a branchandbound algorithm [21, Chapter 24.1], and then use Isabelle’s code generator [11] to obtain verified executable code. Along the way, we also give the first formalized proofs for several important results on integer programming.
A concrete example of an application for our solver comes from termination analysis, where a program is given as input to a termination tool that tries to determine whether the given program terminates on all inputs. Since termination tools get patched and improved repeatedly, maintaining an uptodate formal proof of soundness would be extremely difficult. Therefore, the approach that is typically used is to have the (unverified) termination tool output a certificate for its analysis, which can then be checked by a verified certificate checker. One such certificate checker is CeTA [5, 24]. It has been verified in Isabelle/HOL, so that whenever it accepts a proof of termination for some program, the formal proof of CeTA ’s soundness ensures that the program does indeed terminate.
As an example, consider a program to compute the binary logarithm.
This program can be translated into an integer transition system and termination can be proved by showing that the value of x is decreased by at least 1 in every loop iteration. This property can be expressed in linear integer arithmetic (LIA): it is equivalent to the validity of formula (1), where \(x'\) and \(n'\) represent the new values of x and n, respectively, after an iteration of the loop.
Validity of (1) is equivalent to unsatisfiability of the negated formula, which is simply a conjunction of linear inequalities:
A sufficient condition for the unsatisfiability of (2) over the integers (LIA) is the unsatisfiability of the same system over the rationals (LRA); the latter can be shown, for instance, via the simplex algorithm [9]. Indeed, a verified implementation [23] of the simplex algorithm is currently integrated into CeTA [5]. However, whereas (2) is unsatisfiable over the integers, it has a rational solution \(x = n' = 1\), \(x' = \frac{1}{2}\), \(n = 0\). For such examples, considering the problem over the rationals may prohibit CeTA from detecting unsatisfiability over the integers.
Therefore, in this paper we develop a verified theory solver for LIA (in fact, for linear mixed integer arithmetic, where only a userspecified part of the solution is required to be integral). The verified solver takes a conjunction of strict and nonstrict linear inequalities as input, and decides whether they are simultaneously solvable. We fully integrate the LIA solver into CeTA, so that the new version can handle all instances that are unsatisfiable over the integers and not only those that are unsatisfiable over the rationals as well. Of course, the LIA solver can also be used as a standalone theory solver, e.g., to perform verified SMT solving.
We verify our LIA solver in two major steps.

1.
First, we show that for every set of LIA constraints it suffices to search for small solutions. To this end, we formally verify an a priori bound in the style of Papadimitriou [20]: If there is an integer solution to a set of LIA constraints, then there is also one that is bounded by \(b := n(ma)^{2m+1}\), where n is the number of variables, m the number of inequalities, and a the largest absolute value of any number occurring in the inequalities. To be more precise, the small solution satisfies \(x \le b\) for each variable x.
Our verified upper bound matches the one given in a textbook [21, Thm. 17.1] (which is considerably lower than the one by Papadimitriou).^{Footnote 1} Specifically, we establish a bound of \((n+1)!a^n\) (with no dependence on m). To prove this bound in Isabelle/HOL we mostly follow the textbook proofs and formalize several important results from linear programming, often with additional statements on bounds and integrality. These results include: the fundamental theorem of linear inequalities, the Farkas–Minkowski–Weyl theorem, Carathéodory’s theorem, and the decomposition theorem for polyhedra. Note that the bound on the size of solutions also implies the fact that the problem of deciding satisfiability for linear integer inequalities is in NP.

2.
Using the upper bound, we can decide satisfiability via a finite search. For instance, for formula (2) we have \(n = 4\), \(a = 2\) and \(m = 6\) (the equality counts as two inequalities), and we know that if (2) is satisfiable, then there is an integer solution with absolute values at most 1920.
To perform this search, we implement and verify a basic branchandbound algorithm. It is based on an incremental version of the simplex algorithm by Dutertre and de Moura [10], which is used to deliver candidate solutions and to prune the search tree by detecting unsatisfiability in LRA. Although the incremental simplex algorithm has recently been verified in Isabelle/HOL [3], its integration into the branchandbound algorithm is not immediate: the branchandbound algorithm requires frequent updates of bounds on variables, and this operation is not supported by the existing verified incremental simplex algorithm.
Note that our verified LIA solver is missing several possible optimizations [6, 7, 14], some of which might be integrated in future work. Therefore, it clearly cannot compete with stateoftheart (unverified) solvers. Still, our experimental results show that there are some examples from SMTLIB where our solver is successful, but both CVC4 [2] and Z3 [16] fail.
Structure. We give some preliminaries on linear (integer) programming and Isabelle in Sect. 2. Afterwards, we present our formalization of linear programming and the mentioned bound in Sect. 3. The branchandbound algorithm with the adaptation of the incremental simplex algorithm are covered in Sect. 4. We provide experimental results in Sect. 5 and conclude in Sect. 6.
The collection of theorems on polyhedra and small solutions is available as part of the archive of formal proofs (AFP) in the entry on linear inequalities [4], and the branchandbound algorithm is part of IsaFoR/CeTA [24]. All of the theorems of this paper are linked to the formalization on an accompanying website. It also provides details on the experiments.
Related Work. Allamigeon and Katz [1] have implemented the simplex algorithm in Coq and used it to give constructive proofs of a number of important theorems about convex polyhedra. The overlap between our work and [1] consists of formalizations of basic facts concerning cones and polyhedra, the fundamental theorem of linear inequalities, and Farkas’ lemma. However, whereas in [1] a simplex algorithm for optimization problems is implemented in order to be used in constructive mathematical proofs, we formalize theorems concerning integer programming, including bounds on the size of solutions, and use these together with the previously Isabelleverified simplex algorithm to obtain formally verified, yet efficient, software.
There is also a formalization of theorems about polyhedra in HOL Light, due to Harrison [12], but it contains neither a formalization of the simplex algorithm nor does it cover integer programming.
Cooper’s algorithm has been formalized by Nipkow [18] in Isabelle/HOL. Although this algorithm also solves linear integer arithmetic, it internally works completely differently and its formalization requires different proofs; therefore, we do not see any overlap between the two works. We nevertheless consider the verified version of Cooper’s algorithm in our experiments.
Finally, we mention two generalpurpose verified solvers. Carlier et al. [8] used Coq to implement and verify an algorithm for solving constraint satisfaction problems over finite domains. As with [1], the resulting implementation can be used in principle, but is not efficient enough to compete with unverified implementations of the same algorithm. Narkawicz and Muñoz [17] used PVS to verify a general branchandbound algorithm; a C++ implementation of this algorithm is described in [22]. In contrast to our work, this implementation was not automatically generated from a formal, verified algorithm specification, but was coded separately. Furthermore, in order to use the general branchandbound algorithm, one must first tailor it to an application domain by specifying a number of functions that must respect certain specifications, whereas every part of our LIA solver (both branchandbound and simplex) has been formally verified. Thus, while the algorithm we verify lacks the generality of the one in [17], our implementation retains a higher degree of reliability than the one in [22], due to being entirely generated from a formally verified algorithm, and it is nevertheless reasonably efficient.
2 Preliminaries
We briefly review some linear programming and Isabelle background.
2.1 Linear Programming
We assume familiarity with vector spaces. Although our Isabelle theorems use a more general type, here we present our results in the context of Euclidean spaces (\(\mathbb {R}^n\)). We denote the usual inner product in \(\mathbb {R}^n\) by ‘\(\cdot \)’.
A (nonstrict) linear inequality is an inequality of the form \(a\cdot x \le b\), where \(a,x\in \mathbb {R}^n\) (a a row vector, x a column vector) and \(b\in \mathbb {R}\). A system of linear inequalities can therefore be written as \(Ax\le b\), with \(A\in \mathbb {R}^{m\times n}\) and \(b\in \mathbb {R}^m\) a column vector. A system of linear inequalities is a mixed integer system if, for some \(I\subseteq \{1,\ldots ,n\}\), it is required that \(x_i\in \mathbb {Z}\) for all \(i\in I\). We also define strict linear inequalities to be inequalities of the form \(ax<b\), with a, x and b as before.
In this work we consider mixed integer systems of linear inequalities containing both nonstrict and strict inequalities.
For reference, we collect below the definitions of several important concepts from linear algebra that are needed in order to state the theorems that we formalize. These definitions can be found in textbooks on linear programming such as [21, Chapters 7.1–2 and 16.2].
Definition 1
(Halfspaces, hyperplanes, polyhedra). For \(c\in \mathbb {R}^n\setminus \{0_n\}\) (a row vector) and \(d\in \mathbb {R}\), we say that the set \(H = \{x \mid c\cdot x \le d\}\) is an affine halfspace, and that c is its normal vector. If \(d = 0\), then H is called a linear halfspace (or just a halfspace). The set \(\{x \mid c \cdot x = 0\}\) is called a hyperplane (of which c is a normal vector).
A set \(P\subseteq \mathbb {R}^n\) is called a (convex) polyhedron if \(P = \{x\mid Ax \le b\}\), for some matrix \(A\in \mathbb {R}^{m\times n}\) and \(b\in \mathbb {R}^m\). In words, a polyhedron is the intersection of a finite collection of affine halfspaces.
Definition 2
(Cones). A nonempty set \(C\subseteq \mathbb {R}^n\) is a cone if, for all \(x,y\in C\) and \(\lambda ,\mu \ge 0\), we have \(\lambda x + \mu y\in C\). A cone C is generated by the set of vectors X if \(C = \left\{ \lambda _1v_1+\ldots +\lambda _m v_m\mid \lambda _1,\ldots ,\lambda _m\ge 0,\{v_1,\ldots ,v_m\}\subseteq X\right\} \), and C is finitely generated if it is generated by a finite set of vectors. A cone is polyhedral if it is the intersection of finitely many (linear) halfspaces.
Definition 3
(Convex hull, polytopes, integer hull). The convex hull of a vector set X is the set of all convex linear combinations of vectors from X. More precisely,
The convex hull of a finite set of vectors is called a (convex) polytope.
Finally, if P is a polyhedron, then the integer hull of P, denoted \(P_{I}\), is the convex hull of the set of integral vectors of P. (Integral vectors are vectors whose coordinates with respect to the standard basis are integers.)
2.2 Isabelle
For our formalization work we use the theorem prover Isabelle. Knowledge of Isabelle will be helpful, but is not necessary in order to read the paper, as we have tried to make the formal source listings accessible even to a reader with a purely mathematical background.
Nevertheless, we briefly explain the meaning of some important notation here. First, we have , , and denote the zerovector of dimension n by . Often, the statement that a vector or a matrix has a certain property will be expressed as membership in the set of all vectors or matrices with that property: is the set of vectors (of finite dimension) with entries bounded in absolute value by (similarly ), is the set of vectors v with \(v_i\in \mathbb {Z}\) for all , and, finally, \(\mathbb {Z}_{v}\) is the set of vectors (of finite dimension) with integer entries (similarly, \(\mathbb {Z}_m\) is a set of matrices). We also have a notation for sets defined by some set of vectors or by a matrix: denotes the cone generated by the finite set X; other examples are , and , all with the obvious meanings.
3 MixedInteger Linear Problems
3.1 The Main Formalized Theorems
We discuss our formalization of several results that are needed in order to formally prove the soundness of a branchandboundbased solver for mixedinteger linear systems of inequalities. The main theorem for this purpose states that if a mixed integer system of linear inequalities can be described using only integers, then it has a solution if and only if it also has a solution involving only numbers of bounded size.
Theorem 4
In order to derive this result, we require formalizations of several results from the theory of linear inequalities, beginning with the fundamental theorem of linear inequalities. This theorem states that for any finite set of vectors A and vector b, either b is in the cone generated by a subset of A, or there exists a hyperplane \(\{x\mid c\cdot x=0\}\) separating b from A and containing some number of vectors of A.
Theorem 5
^{Footnote 2}
To prove the theorem, one first considers an algorithm that iteratively applies a procedure that takes a subset of vectors from A and produces either the cone containing b from the theorem statement, or the separating hyperplane, or a new set of vectors from A. In case of the third outcome, the output set is used as the input for the next iteration. Thus, starting from some valid set of vectors, the above algorithm either never terminates (if the third outcome occurs in every iteration), or it produces an object satisfying the theorem statement. The proof is completed by showing that an infinite execution cannot occur.
The above argument could in principle be formalized in Isabelle by defining a function that incorporates the algorithm, and then proving that the function is welldefined (which implies the termination of the algorithm on all inputs). However, we are really only interested in the algorithm’s termination; the fact that some input is mapped to a certain output is irrelevant for the proof of the theorem. Furthermore, we only need that the algorithm terminates when the set of input vectors is valid (i.e., of the right cardinality and linearly independent), but, due to the limitations of the Isabelle functionpackage [13], the domain of a function cannot be restricted in this manner. Consequently, we formalize the proof without modeling the algorithm as an Isabelle function. Instead, we define a relation on pairs of valid subsets of A: The pair \((J',J)\) is in the relation if and only if, starting with J as input, one iteration of the algorithm produces output \(J'\). In other words, the relation encodes all iterations of the algorithm where the third outcome occurs. Since A is finite, termination is equivalent to the fact that the above relation has no cycles. The latter fact is established by a proof by contradiction (here, our formalization closely follows the textbook proof [21, Chapter 7.1]).
We also need to formalize three corollaries of Theorem 5. First, we have the theorem of Carathéodory, which follows directly.
Theorem 6
Next, we have the FarkasMinkowskiWeyl theorem, which states that a cone is polyhedral if and only if it is finitely generated.
Theorem 7
The proofs of Theorems 7 and 5 in [21] contain some simplifying assumptions that can be made without loss of generality. Of course, in Isabelle we must provide the full details of every proof, which often entails a nontrivial amount of additional formalization work. For example, the textbook proof of the “\(\longrightarrow \)”implication of Theorem 7 only covers the case where X spans \(\mathbb {R}^n\). One way to recover this part of the theorem in full generality is to identify the span of X with \(\mathbb {R}^m\) for some \(m < n\), apply the “\(\longrightarrow \)”implication for dimension m, and then extend the halfspaces (of \(\text {span }X\)) that define the polyhedral cone, into \(\mathbb {R}^n\). In fact, this argument is essentially the justification for the wlog that is given in the book. Unfortunately, the Isabelle vector/matrix library we use does not support identifying an arbitrary proper subspace of \(\mathbb {R}^n\) with a Euclidean subspace of lower dimension: Even if we prove some statement for , we cannot apply it to some arbitrary mdimensional subspace of \(\mathbb {R}^n\). Instead, our formalization of the general case involves adding suitable dummy vectors to X until the set spans all of \(\mathbb {R}^n\), so that we can apply the fulldimension implication for . This is one of several situations where filling in the “obvious” steps of a proof in a way that can be formally expressed in Isabelle requires some creativity.
The third corollary is the decomposition theorem for polyhedra, stating that every polyhedron can be written as the sum of a polytope and a polyhedral cone:
Theorem 8
For both FarkasMinkowskiWeyl (Theorem 7) and the decomposition theorem, the fact that we used a setbased matrix/vector library proved to be beneficial. To show the “\(\longrightarrow \)”implication of Theorem 7, one defines a matrix, the dimension of which is a function of X (and can therefore not be independently fixed just by the type of X). Constructing matrices of dimensions that depend on the value of some variable is easy when using , but would be very difficult with matrix libraries which utilize Harrison’s encoding of dimensions in types [12]. In the case of the decomposition theorem for polyhedra, the proof involves adding a new component to each vector from a set of ndimensional vectors and then reasoning about the resulting set of \((n+1)\)dimensional vectors, while maintaining the correspondence between the two sets. Here, the use of makes it possible to easily switch between dimensions and reason about objects such as “the vector formed of the first n components of some \((n+1)\)dimensional vector”.
Since the set of (real) solutions of a system of linear inequalities is a polyhedron, the decomposition theorem for polyhedra allows us to write any solution vector x as \(y+z\), with y an element of a polytope (and therefore bounded), and z an element of a finitely generated cone. This suggests the following approach to proving Theorem 4 ( ): If x is such that \(x_i\in \mathbb {Z}\) for all \(i\in I\), we may try to replace z with a vector \(z'\) of the same cone, with bounded entries, such that \((y+z')_i\in \mathbb {Z}\) for all \(i\in I\) (thus, \(y+z'\) would be the desired bounded solution). This approach does in fact work, but it clearly requires a more powerful version of the decomposition theorem, since the one we have shown so far says nothing about bounds or integrality. The proof of the new decomposition theorem also requires a bounded integer version of Theorem 7. This latter theorem in turn is based on a modified version of Theorem 5 which describes more precisely how separating hyperplanes can be computed so that the normal vectors are integral and with components of bounded size.
Theorem 9
The ‘\(\longrightarrow \)’implication of this stronger version of the decomposition theorem for polyhedra states that if A and b have bounded integer entries, then the finite sets Q and X can be chosen such that they contain only bounded vectors and, furthermore, such that X contains only integral vectors. The integrality of the vectors in X is the crucial ingredient necessary for constructing the vector \(z'\) as required and completing the proof of Theorem 4.
In [21], only a weaker version of Theorem 4 is proved; it covers only the case of nonstrict linear inequalities with integral solutions. Although our result trivially implies this weaker form, we have formalized the proof from the textbook as well, for the sake of completeness.
This proof relies on a decomposition theorem for the integer hull of a polyhedron, which also requires bounded integer versions of Theorem 7 and the decomposition theorem for polyhedra. Only a rough sketch is given in the book as to how the bounded integer versions of these theorems can be obtained. When formalizing this part, however, we encounter the following issue: In the course of a proof, it will be necessary to add new vectors to a set until it has a certain property, or to add halfspaces to a collection until its intersection coincides with some polyhedron. This suffices if we only wish to prove the existence of a set of vectors with some property, or of a specific representation of a polyhedron, but if we also need to prove bounds on the numbers needed to describe these objects, it becomes crucial which vectors or halfspaces are chosen, because some choices, while valid, will lead to results that do not respect the desired bounds.
For a concrete example, we return to the “\(\longrightarrow \)”implication of Theorem 7 (FarkasMinkowskiWeyl), this time in its bounded integer version:
Theorem 10
As mentioned earlier in this section, this implication is proved for the case where the span of X is \(\mathbb {R}^n\), which is then used to prove the general implication, but the switch from the special to the general case involves adding vectors to X until the set spans the entire space, and then applying the fulldimension statement to obtain the halfspaces that define the polyhedral cone. Now, the vectors that are added to X can affect the size of the entries of the resulting matrix A, and the fact that these vectors can also be chosen in such a way that the entries of A are bounded in terms of only and n, is not obvious, and in fact requires a careful construction. Whereas such matters are simply glossed over in the textbook, resolving the wlogs in the proof of the bounded version of Theorem 5 and of Theorem 7 resulted in Isabelle proofs of 176 lines and 110 lines, respectively.
In the end, we achieve the following formalized version of the textbook theorem [21, Thm. 17.1].
Theorem 11
3.2 Additional Formalized Theorems
In order to formalize the proofs of the main theorems, we collect a number of basic lemmas concerning cones, convex hulls, integer hulls, normal vectors and bases of vector spaces. On the one hand, these lemmas include very basic statements that would not normally require separate proofs, but were needed for the formalization, such as the fact that a set of vectors is a subset of the cone it generates, or that a convex combination of two vectors of a cone belongs to the cone. On the other hand, our supporting lemmas include statements that appear in standard mathematical texts, such as the fact stated in Lemma 12 that any linearly independent set of vectors can be extended to a basis of the vector space. We mention that we have proved all of these facts only for Euclidean vector spaces, making heavy use of the fact that the dimension is finite, because this case suffices for our application.
Lemma 12
We note that in Lemma 12, is list concatenation and refers to the standard basis of \(\mathbb {R}^n\). Of course, a linearly independent set can be extended in many other ways, but we use vectors from the standard basis because they allow us to obtain the same number bounds as for the original linearly independent set. Adding the standard basis vectors is also the reason for using instead of in many theorems that mention upper bounds. Indeed, the “ ”operation often cannot be dropped. For instance, consider the “\(\longleftarrow \)”implication of Theorem 7 and the degenerate case where the matrix A is empty or just contains zeros. Then the entries of A are bounded by 0 and the cone is the whole space. Thus, for generating this cone one needs at least n nonzero vectors, e.g., the unit vectors. And these do not have all their entries bounded by 0, but by .
A notable exception, without “ ”, is our main Theorem 4 ( ). This result is first proved with the “ ” expression in the bounds. The version without the operation is then established by proving that the theorem also holds in all degenerate cases (where the bound is 0).
Aside from the main theorems and supporting lemmas, we also formally prove two variants of Farkas’ lemma. We do not need these for our work on the verified linear arithmetic solver, but obtaining them did not entail a prohibitively large additional effort, and they may be useful for other formalizations.
Although there already exists an entry for Farkas’ lemma in the AFP, its proof there is based not on the fundamental theorem of linear inequalities (Theorem 5), but on a separate formalization of the simplex algorithm (one that has been formalized solely for rational numbers). Since here we use Theorem 5, we obtain a version of a lemma that allows for the use of a more general type than just the rationals. (In Isabelle, type annotation is denoted by . Below, is a type variable that stands for the type of the entries of a matrix/vector; it can be any type with the suitable algebraic properties.)
Lemma 13
Lemma 14
Finally, we remark that, while the first of the two variants of Farkas’ lemma follows easily from Theorem 5, the second variant (which, in [21], has a threeline proof that is based on the first variant) is somewhat more difficult to formalize. This is because its proof involves concatenating matrices and deducing inequalities involving the resulting matrix from facts about its components. Such operations require laborious lowlevel manipulations of vector inequalities, turning a threeline textbook proof into 102 lines of Isabelle code.
4 A Verified BranchandBound Algorithm
4.1 The BranchandBound Algorithm
Algorithm 1 shows the Isabelle/HOL function , which is our implementation of a branchandbound algorithm for solving LIA problems. It takes as parameters a list of constraints , the list of variables that should get an integer assignment and (total) functions and that map the variables in to their lower and upper integer bounds. returns either a satisfying assignment which maps variables to rational numbers and all variables in to integers, or , if the mixed integer problem is unsatisfiable within the bounds and . first uses the simplex algorithm to find a rational solution of the constraints within the bounds. If the constraints are already unsatisfiable in the rational numbers or if the solution is already integral for all values in , then terminates accordingly. Otherwise, there exists an where (the value assigned to in the rational solution ) is not an integer. We update the bounds on once in and once in and branch by running with the new upper bound and then with the new lower bound.
To verify in Isabelle/HOL we have to show that it always terminates. Note that in every recursive call, we either decrease one of the upper bounds or increase one of the lower bounds . This fact is used to show that in every recursive call, the range of possible values decreases for some , and, hence, so does the search space. Thus, we use the following measure (of the size of the search space) to prove termination in Isabelle/HOL:
We then prove two theorems about : any detected solution is valid, and whenever delivers , no solution exists within the range that is specified by the lower and upperbounds. The expression means that the solution satisfies all of the constraints in and that all are assigned integer values by .
At this point we connect the branchandbound algorithm with the bounds from Sect. 3 to obtain a decision procedure for linear (mixed) integer arithmetic:
Here, is an algorithm that extracts the relevant parameters (number of variables, maximal absolute value in constraints) and then calculates the upper bound as in Sect. 3. One complication comes from the fact that there are two different representations of constraints: the statements regarding bounds have been proved for constraints given in matrixvector form, \(Ax \le b\) or \(Ax < b\) with integral matrix A and integral vector b, whereas the input to the branchandbound algorithm is a set of constraints, where each constraint is represented via a (sparse) linear polynomial with rational entries, e.g., \(x_5 + \frac{1}{10}x_{1041} \le \frac{7}{3}\). Hence, internally also normalizes the constraints, e.g., by canceling fractions, and by renaming the variables so that the indices of variables with nonzero coefficients form a contiguous block: \(x_0,\ldots ,x_{n1}\). The normalized constraints can then easily be translated into matrixvectorform, which enables a lifting of Theorem 4 ( ) to constraints that are represented via sparse polynomials.
At this point, it is easy to combine the results of with to finally show that is a complete and sound decision procedure. Either it returns some assignment, which is then a solution to the mixed integer problem; or it returns , and the mixed integer problem is unsatisfiable.
4.2 Using the Incremental Version of Simplex
One problem of the branchandbound algorithm from the previous section is in the way it invokes the simplex algorithm: although in every iteration only a single constraint changes, the simplex algorithm is always started from scratch.
Therefore, in this section we optimize the branchandbound algorithm to use an already existing verified incremental version of the simplex algorithm [3, 15], which returns a state instead of only returning a satisfying assignment or stating unsatisfiability. The state contains for instance a tableau, i.e., a list of equations which is essential for the simplex algorithm. By reusing the state, expensive operations like creating the tableau can be avoided, making the incremental simplex very attractive to be used within the branchandbound algorithm.
A complication arises, since the verified incremental simplex algorithm was developed to be used in a DPLL(T)solver, where all constraints are known beforehand and the constraints are not changed throughout one DPLL(T) run. Therefore, the incremental interface does not allow for changing constraints or adding new ones. As a consequence, an integration of the incremental simplex into the branchandbound algorithm is not immediate, since there the bounds are changed in every iteration.
Our solution is a slight extension of the incremental simplex algorithm. To be more precise, we write a function which changes exactly one constraint in the state in a way that the relevant invariants of the incremental interface still hold. This extension allows us to reuse all the existing soundness properties and proofs of the incremental simplex algorithm without modifications. It is specifically tailored for running the branchandbound algorithm. We choose this approach instead of adding a feature to change arbitrary constraints in the incremental simplex interface, since such a feature would require a major rewrite.
Since the algorithmic structure and the soundness statement of the modified branchandbound algorithm is completely identical to the one of Sect. 4.1, we just refer to the formalization for further details.
5 Benchmarking
We tested two versions of our solver (based on incremental/nonincremental simplex) by comparing them with two wellestablished SMTsolvers, Z3 and CVC4. Testing was done on a subset of the nonincremental^{Footnote 3} QF_LIA (quantifierfree linear integer arithmetic)^{Footnote 4} benchmark set from SMTLIB. For this experiment we had two goals in mind: 1. to see whether it is worthwhile to use the nonincremental version of simplex as a subroutine in the branchandbound algorithm, and 2. to get an idea about the extent to which our verified, nonoptimized solver can handle practical examples.
We did not go through the effort of making our solver compliant with the language of SMTLIB, as we felt that for the above two goals, it would suffice to write a simple converter that could handle a reasonable portion of the QF_LIA benchmarks. Thus, we obtained a dataset of 1192 benchmarks, comprising 18% of the 6489 benchmarks in QF_LIA. (More specifically, the following subfolders were fully converted to a format that is readable by our solver: 20180326Bromberger, miplib2003, pb2010, pidgeons, primecone, and slacks.) All solvers were tested on this dataset, on the same hardware, with a 60stimeout per benchmark. Z3 version 4.4.0pre2, CVC4 version 1.54, and the 20190509 release of SMTLIB were used.
The only other verified LIA solver that we are aware of is an Isabelle formalization of Cooper’s algorithm in the AFP. This algorithm solves a more general problem than linear integer arithmetic (namely linear arithmetic with arbitrary quantifiers over integer variables). We obtained an implementation with minimal changes to make code generation possible (just as we produced executables for our own solver).
Evaluation. Our branchandbound implementation solves 37% of the dataset with incremental simplex as a subroutine, and 31% with nonincremental simplex (Table 1). Since we have only implemented a naive branchandbound algorithm, without any additional heuristics for pruning the search space, it is unsurprising that its performance cannot match that of more mature solvers. Somewhat surprising is the fact that some benchmarks are solved by our solvers but not by Z3 or CVC4: of the benchmarks solved by incremental bnb, 28 are not solved by Z3, 29 are not solved by CVC4, and 8 are solved by neither Z3 nor CVC4.
Interestingly, the nonincremental simplexbased solver can handle a few instances that the incremental simplexbased solver does not. Although using an incremental simplex leads to better overall results, it appears that reusing valuations from previous simplex runs can sometimes lead the search astray in such a way that simple solutions are missed. The phenomenon of a search proceeding in the wrong direction and missing a simple solution may also be the reason why some instances cannot be handled by either Z3 or CVC4, despite being solved by our solver.
Cooper’s algorithm is known to have a very high asymptotic complexity, which means that its performance is not a matter of optimizing an implementation. As such, the outcome of our experiments with regards to Cooper’s algorithm is as expected, showing that this algorithm is not usable on mediumsized examples in practice.
6 Conclusion and Future Work
We have developed a verified solver for linear mixed integer arithmetic, and have formalized important results on linear integer programming that were needed in order to prove the soundness of the solver. To the extent of our knowledge, the main mathematical theorems of which we formalized proofs had not been previously verified in any formal system, and our solver is the first verified LIA solver that is also usable in practice. The two parts of our formalization amount to 9813 lines of Isabelle code and took roughly 10 personmonths to implement.
Currently, our solver is essentially “proof of concept” software, and there are a number of known optimizations that could improve it, e.g., preprocessing of constraints, integration of cutting planes, unitcubetests, etc. [6, 7, 14]. We have also used runtime profiling in order to establish which subroutines our solver spends most time on, and have identified parts of the incremental simplex algorithm that we could further modify in order to improve running times.
Change history
10 August 2020
The original versions of this book and Chapter 14 were revised. The following was corrected:
Dimitra Giannakopoulou, the General Chair of the NFM 2020 conference, was inadvertently forgotten and, therefore, added as a volume editor.
Chapter 14 was retrospectively made available open access under a CC BY 4.0 license at link.springer.com.
Notes
 1.
The textbook bound is somewhat more precise than ours, as it is phrased in terms of subdeterminants, whereas we use a generic bound on subdeterminants.
 2.
The Isabelle statement given here matches the presentation of the theorem in [21]; in our formalization, the equivalence is written as an equality of sets.
 3.
Here, “nonincremental” means that the tests are simply sets of constraints, as opposed to constraints together with an assert/check script that a solver must execute.
 4.
QF_LIRA (quantifierfree mixed integer real arithmetic) contains only 8 tests.
References
Allamigeon, X., Katz, R.D.: A formalization of convex polyhedra based on the simplex method. In: AyalaRincón, M., Muñoz, C.A. (eds.) ITP 2017. LNCS, vol. 10499, pp. 28–45. Springer, Cham (2017). https://doi.org/10.1007/9783319661070_3
Barrett, C., et al.: CVC4. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 171–177. Springer, Heidelberg (2011). https://doi.org/10.1007/9783642221101_14
Bottesch, R., Haslbeck, M.W., Thiemann, R.: Verifying an incremental theory solver for linear arithmetic in Isabelle/HOL. In: Herzig, A., Popescu, A. (eds.) FroCoS 2019. LNCS (LNAI), vol. 11715, pp. 223–239. Springer, Cham (2019). https://doi.org/10.1007/9783030290078_13
Bottesch, R., Reynaud, A., Thiemann, R.: Linear inequalities. Archive of Formal Proofs, June 2019. http://isaafp.org/entries/Linear_Inequalities.html. Formal proof development
Brockschmidt, M., Joosten, S.J.C., Thiemann, R., Yamada, A.: Certifying safety and termination proofs for integer transition systems. In: de Moura, L. (ed.) CADE 2017. LNCS (LNAI), vol. 10395, pp. 454–471. Springer, Cham (2017). https://doi.org/10.1007/9783319630465_28
Bromberger, M.: A reduction from unbounded linear mixed arithmetic problems into bounded problems. In: Galmiche, D., Schulz, S., Sebastiani, R. (eds.) IJCAR 2018. LNCS (LNAI), vol. 10900, pp. 329–345. Springer, Cham (2018). https://doi.org/10.1007/9783319942056_22
Bromberger, M., Weidenbach, C.: New techniques for linear arithmetic: cubes and equalities. Form. Methods Syst. Des. 51(3), 433–461 (2017). https://doi.org/10.1007/s1070301702787
Carlier, M., Dubois, C., Gotlieb, A.: A certified constraint solver over finite domains. In: Giannakopoulou, D., Méry, D. (eds.) FM 2012. LNCS, vol. 7436, pp. 116–131. Springer, Heidelberg (2012). https://doi.org/10.1007/9783642327599_12
Dantzig, G.B.: Linear Programming and Extensions. Princeton University Press, Princeton (1963)
Dutertre, B., de Moura, L.: A fast lineararithmetic solver for DPLL(T). In: Ball, T., Jones, R.B. (eds.) CAV 2006. LNCS, vol. 4144, pp. 81–94. Springer, Heidelberg (2006). https://doi.org/10.1007/11817963_11
Haftmann, F., Nipkow, T.: Code generation via higherorder rewrite systems. In: Blume, M., Kobayashi, N., Vidal, G. (eds.) FLOPS 2010. LNCS, vol. 6009, pp. 103–117. Springer, Heidelberg (2010). https://doi.org/10.1007/9783642122514_9
Harrison, J.: The HOL light theory of Euclidean space. J. Autom. Reasoning 50, 173–190 (2013). https://doi.org/10.1007/s1081701292509
Krauss, A.: Partial and nested recursive function definitions in higherorder logic. J. Autom. Reasoning 44(4), 303–336 (2010). https://doi.org/10.1007/s1081700991572
Marchand, H., Martin, A., Weismantel, R., Wolsey, L.A.: Cutting planes in integer and mixed integer programming. Discrete Appl. Math. 123(1–3), 397–446 (2002). https://doi.org/10.1016/S0166218X(01)003481
Marić, F., Spasić, M., Thiemann, R.: An incremental simplex algorithm with unsatisfiable core generation. Archive of Formal Proofs, August 2018. http://isaafp.org/entries/Simplex.html. Formal proof development
de Moura, L., Bjørner, N.: Z3: an efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008). https://doi.org/10.1007/9783540788003_24
Narkawicz, A., Muñoz, C.: A formally verified generic branching algorithm for global optimization. In: Cohen, E., Rybalchenko, A. (eds.) VSTTE 2013. LNCS, vol. 8164, pp. 326–343. Springer, Heidelberg (2014). https://doi.org/10.1007/9783642541087_17
Nipkow, T.: Linear quantifier elimination. J. Autom. Reasoning 45(2), 189–212 (2010). https://doi.org/10.1007/s1081701091830
Nipkow, T., Wenzel, M., Paulson, L.C. (eds.): Isabelle/HOL – A Proof Assistant for HigherOrder Logic. LNCS, vol. 2283. Springer, Heidelberg (2002). https://doi.org/10.1007/3540459499
Papadimitriou, C.H.: On the complexity of integer programming. J. ACM 28(4), 765–768 (1981). https://doi.org/10.1145/322276.322287
Schrijver, A.: Theory of Linear and Integer Programming. Wiley, Hoboken (1999)
Smith, A., Muñoz, C., Narkawicz, A., Markevicius, M.: A rigorous generic branch and bound solver for nonlinear problems. In: 2015 17th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), pp. 71–78. IEEE (2015). https://doi.org/10.1109/SYNASC.2015.20
Spasić, M., Marić, F.: Formalization of incremental simplex algorithm by stepwise refinement. In: Giannakopoulou, D., Méry, D. (eds.) FM 2012. LNCS, vol. 7436, pp. 434–449. Springer, Heidelberg (2012). https://doi.org/10.1007/9783642327599_35
Thiemann, R., Sternagel, C.: Certification of termination proofs using CeTA. In: Berghofer, S., Nipkow, T., Urban, C., Wenzel, M. (eds.) TPHOLs 2009. LNCS, vol. 5674, pp. 452–468. Springer, Heidelberg (2009). https://doi.org/10.1007/9783642033599_31
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2020 The Author(s)
About this paper
Cite this paper
Bottesch, R., Haslbeck, M.W., Reynaud, A., Thiemann, R. (2020). Verifying a Solver for Linear Mixed Integer Arithmetic in Isabelle/HOL. In: Lee, R., Jha, S., Mavridou, A., Giannakopoulou, D. (eds) NASA Formal Methods. NFM 2020. Lecture Notes in Computer Science(), vol 12229. Springer, Cham. https://doi.org/10.1007/9783030557546_14
Download citation
DOI: https://doi.org/10.1007/9783030557546_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 9783030557539
Online ISBN: 9783030557546
eBook Packages: Computer ScienceComputer Science (R0)