Program Verification in the Presence of I/O

Abstract

Software verification tools that build machine-checked proofs of functional correctness usually focus on the algorithmic content of the code. Their proofs are not grounded in a formal semantic model of the environment that the program runs in, or the program’s interaction with that environment. As a result, several layers of translation and wrapper code must be trusted. In contrast, the CakeML project focuses on end-to-end verification to replace this trusted code with verified code in a cost-effective manner.

In this paper, we present infrastructure for developing and verifying impure functional programs with I/O and imperative file handling. Specifically, we extend CakeML with a low-level model of file I/O, and verify a high-level file I/O library in terms of the model. We use this library to develop and verify several Unix-style command-line utilities: cat, sort, grep, diff and patch. The workflow we present is built around the HOL4 theorem prover, and therefore all our results have machine-checked proofs.

A Appendix: Further Example Programs

For the benefit of readers, we describe our verified implementations of the grep, sort, and cat command-line utilities.

1.1 A.1 Cat

A verified cat implementation was presented in our previous work on CF [13]. The cat implementation presented here differs in two respects: first, it is verified with respect to a significantly more low-level file system model (see Sect. 3.1). Second, it has significantly improved performance, since it is implemented in terms of more low-level I/O primitives. Hence this example demonstrates that reasonably performant I/O verified with respect to a low-level I/O model is feasible in our setting. Here is the code:

The difference over the previous implementation is pipe_2048, which gains efficiency by requesting 2048 characters at a time from the input stream, rather than single characters as previously. We elide its straightforward CF specification, which essentially states that the output produced on stdout is the concatenation of the file contents of the filenames given as command line arguments. The cat implementation above does not handle exceptions thrown by TextIO.openIn; hence the specification assumes that all command line arguments are valid names of existing files.

1.2 A.2 Sort

The sort program reads all of the lines in from a list of files given on the command-line, puts the lines into an array, sorts them using Quicksort, and then prints out the contents of the array. The proof that the printed output contains all of the lines of the input files, and in sorted order, is tedious, but straightforward.

We do not use an existing Quicksort implementation, but write and verify one from scratch. Unlike the various list-based Quicksort algorithms found in HOL, Coq, and Isabelle, we want an efficient array-based implementation of pivoting. Hence we implement something more akin to Hoare’s original algorithm. We sweep two pointers inward from the start and end of the array, swapping elements when they are on the wrong side of the pivot. We stop when the pointers pass each other. Note that we pass in a comparison function: our Quicksort is parametric in the type of array elements.

Because this is intrinsically imperative code, we do not use the synthesis tool, but instead verify it with CF directly. The only tricky thing about the proof is working out the invariants for the various recursive functions, which are surprisingly subtle, for an algorithm so appealingly intuitive.

Our approach to verifying the algorithm is to assume a correspondence between the CakeML values in the array, and HOL values that have an appropriate ordering on them. The Quicksort algorithm needs that ordering to be a strict weak order. This is a less restrictive assumption than requiring it to be a linear order (strict or otherwise). Roughly speaking, this will allow us to assume that unrelated elements are equivalent, even when they are not equal. Hence, we can sort arrays that hold various kinds of key/value pairs, where there are duplicate keys which might have different values.

Even though we are not using the synthesis tool, we do use its refinement invariant combinators to maintain the CakeML/HOL correspondence. This enforces a mild restriction that our comparison function must be pure, but greatly simplifies the proof by allowing us to reason about ordering and permutation naturally in HOL.

The following is our correctness theorem for partition. We assume that there is a strick weak order ${\textsf {cmp}}$ that corresponds to the CakeML value passed in as the comparison. We also assume some arbitrary refinement invariant $a$ on the elements of the array. The combinator lifts refinement invariants to functions.

We can read the above as follows, starting in the conclusion of the theorem. Partition takes 5 arguments $cmp\texttt {\_}{}v$, $arr\texttt {\_}{}v$, $pivot\texttt {\_}{}v$, $lower\texttt {\_}{}v$, and $upper\texttt {\_}{}v$, all of which are CakeML values. As a precondition, the array’s contents can be split into 3 lists of CakeML values $elems\texttt {\_}{}vs_{\mathrm {1}}$, $elems\texttt {\_}{}vs_{\mathrm {2}}$, and $elems\texttt {\_}{}vs_{\mathrm {3}}$.⁹ Now looking at the assumptions, the length of $elem\texttt {\_}{}vs_{\mathrm {1}}$ must be the integer value for the lower pointer. A similar relation must hold for the upper pointer, so that $elem\texttt {\_}{}vs_{\mathrm {2}}$ is the list of elements in-between the pointers, inclusive. We also must assume that the pivot element is in segment to be partitioned (excluding the last element).

The postcondition states that the partition code will terminate, and that there exists two partitions. The array in the heap now contains the two partitions instead of $elem\texttt {\_}{}vs_{\mathrm {2}}$. The ${\textsf {partition\_pred}}$ predicate (definition omitted), ensures that the two partitions are non-empty, permute $elem\texttt {\_}{}vs_{\mathrm {2}}$, and that the elements of the first are not greater than the pivot, and the elements of the second are not less. These last two points use the shallowly embedded ${\textsf {cmp}}$ and $elems_{\mathrm {2}}$, rather than $cmp\texttt {\_}{}v$ and $elems\texttt {\_}{}vs_{\mathrm {2}}$.

1.3 A.3 grep

grep  prints to stdout every line from the files that matches the regular expression . Unlike sort, diff and patch which need to see the full file contents before producing output, grep can process lines one at a time and produce output after each line. The main loop of grep reads a line, and prints it if it satisfies the predicate m:

For each filename, we run the above loop if the file can be opened, and print an appropriate error message to stderr otherwise:

The latter function satisfies the following CF specification (eliding stderr output):

The postcondition states that the output to stdout is precisely those lines in $f$ that satisfy $m$, with $f$ and a colon prepended to each line. The three assumptions mean, respectively: that $f$ is a string without null characters, and $fv$ is its corresponding deeply embedded CakeML value; that our view of the file system has a free file descriptor; and that $m$ is a fully specified (i.e., lacking preconditions) function of type $\texttt {char}\;\texttt {lang}$ and $mv$ is the corresponding CakeML closure value.

The main function of grep is as follows:

parse_regexp and build_matcher are synthesised from a previous formalisation of regular expressions by Slind [31], based on Brzozowski derivatives [29].

The semantics of grep is given by the function ${\textsf {grep\_sem}}$, which returns a tuple of output for stdout and stderr, respectively.

regexp_lang is a specification of build_matcher due to Slind, and grep_sem_file is a semantics definition for print_matching_lines_in_file. The final CF specification states that the output to the std* streams are as in ${\textsf {grep\_sem}}$, and has two premises: that there is an unused file descriptor, and that Brzozowski derivation terminates on the given regular expression¹⁰.