Keywords

figure a
figure b

1 Verification Approach

Korn is a verifier for C programs that is based on a translation into systems of constrained Horn clauses [5, 12]. Therein, each program location is abstracted by a second-order predicate over the program variables which are active at that point. The system of Horn clauses has a (second-order) solution if and only if the program is correct. Horn clauses encodings are a convenient intermediate representation that is linear in the size of the program and that is inherently modular, such that loops, procedure contracts, and non-local control flow like gotos and labels can be easily abstracted (see Sect. 3 wrt. category Recursive).

Korn uses state-of-the-art solvers to determine the satisfiability of the generated Horn clause system (cf. Sect. 2), specifically for SV-COMP it uses Z3 [6] and Eldarica [15]. Both solvers generate evidence for correctness of a given program in terms of models that describe how the unknown predicates need to be instantiated. Moreover, Eldarica can generate counterexample traces, and Korn instruments the Horn clause system to get the concrete values returned by the __VERIFIER_nondet_*() functions on an error path. For these reasons, Korn tends to produce detailed correctness and violation witnesses.

The different solvers have different strengths and weaknesses. To that end, Korn implements a portfolio approach with several sequential stages. The configuration for SV-COMP 2023 [2] is as follows, where the specific timeouts for the individual tools are chosen heuristically based on prior experiments:

  1. 1.

    Initially, 10s of random sampling with small values is performed. It picks for each input value uniformly between number 0, and values of 2, 5, and 10 bits respectively, possibly with a sign. Absense of too large values avoids very long running loops when the counter is nondeterministic. There is no particular justification for the sampling scheme, but it is effective.

  2. 2.

    Next, Z3 is executed on the verification problem, translated from C to Horn clauses for 20s. Usually, Z3 finds solutions very quickly if it succeeds at all, specifically on those benchmarks where Z3 succeeds but not Eldarica.

  3. 3.

    Finally, Eldarica is executed for the remaining time. From past experience, it should be slightly better in comparison to Z3 in the long run on this specific set of tasks [10]. The generated invariants from Eldarica tend to be simpler and avoid the existential quantifiers often introduced by Z3, which improves witness generation. To prevent spurious counterexamples, Korn reports a violations only if it can be confirmed by executing the program natively.

Korn is overall similar to SeaHorn [13] but it operates on the C source level instead of LLVM. Korn aims at a rather different design point, namely to favor simplicity over features, therefore offering a good platform for experiments. Eldarica has its own C frontend that supports a different set of features, recently published as TriCera [11]. Here the main distinction is that Korn uses a large block encoding, such that the verification conditions closely reflect the structure of the program. Korn offers a second verification approach with loop contracts [7, 14, 16]. This was the original motivation to develop the tool, and neither SeaHorn nor TriCera supports this feature, albeit it was not used for SV-COMP because it offers no advantages [10] and because the encoding of loop contracts into loop invariants would require quantifiers in the witnesses format.

2 Software Architecture

Korn is mainly written in the JVM language Scala.Footnote 1 The front-end uses a custom parser, generated with jFlex and Beaver. The random sampler relies on native execution which links the benchmark task with a C file __VERIFIER_random.c that implements the _VERIFIER_nondet_* functions. Verification conditions are generated in the fragment of SMT-LIB of the HORN logic.Footnote 2 Korn can invoke any compliant solver as a backend either using its standard input or a file to communicate the verification task. There is explicit support for Z3 [12], Eldarica [15] to pass e.g. timeouts with tool-specific options or to produce models resp. counterexamples. Currently, Korn use the theories of integers and arrays.

In order to produce SV-COMP correctness witnesses, Korn can read the models generated by the backend-solvers, and translate them back into C expressions. The correctness witnesses produced currently are derived from the invariants that are reported back by the Horn solvers (get-model resp. -ssol flag of Eldarica). Violation witnesses are either read off the output of Eldarica (-cex flag), or from the output of the random sampler, as a sequence of nondeterministic choices. When a counterexample is found, a test harness is compiled to confirm whether reach_error() is in fact called.

3 Discussion: Strengths and Weaknesses

Korn supports a substantial fraction of the C language, with the greatest limitation being the lack of support for dynamic data structures (see website for a detailed account), which means that currently any task which requires a memory model is out of scope. The translation supports most control structures, including goto and labels. With respect to solving verification tasks, Korn inherits the strenths and limitations of the underlying solvers. Tasks that for which invariants and procedure contracts are expressible in linear integer arithmetic are typically proved quickly by the solvers, whereas they struggle on tasks with arrays and quantified invariants. Honoring these aspects, Korn participated in four categories, ControlFlow, Loops, Recursive, XCSP for property ReachSafety.

The theoretical approach used by Korn is sound and complete relative to the solver capabilities. Korn produced no incorrect result in SV-COMP 2023, but there are circumstances which could lead to wrong verdicts. With respect to C semantics, Korn currently makes the following trade-offs:

  • Integer types are treated as unbounded and arithmetic overflows are not modeled at all. This affects a single task, nla-digbench/geo1-u.c, which contains an error caused by an unsigned integer overflow. This error is fortunately caught by random sampling— Korn would otherwise wrongly prove this task safe. We aim to experiment with a bitvector encoding eventually, which would allow Korn to tackle tasks involving bitwise operations.

  • Arrays are currently modeled as value types. Benchmarks in which tracking aliases is relevant may not be solved correctly, but that does not occur in the categories in which Korn participates.

  • By confirming counterexamples via native execution, each bug reported is necessarily a true bug. This safety net catches two incorrect error verdicts on loops-crafted/theatreSquare.c and recursive/Primes.c, the reason for this unsoundness is under investigation. However, counterexample confirmation prevents Korn from rightfully reporting 50 error verdicts found by Z3 in category XCSP which are missed by Eldarica (\(\dagger \) in Sect. 1). It is unclear how to get usable counterexample traces from Z3 to resolve this dilemma.

  • Differently from most other SV-COMP tools, Korn fixes the evaluation order of function arguments to be right-to-left which matches the order typically used by C compilers. This is not faithful to C semantics as Korn potentially misses bugs due to side-effects for some specific evaluation order.

The random sampler is very effective—in SV-COMP 2023 it discovered all 210 violations reported by Korn, of which 204 are found within 2 seconds. Sampling of small non-zero values is crucial, e.g., Ackermann02.c falsifies with input vector [2,0]; using all zero inputs still finds 57 of these 210 violations.

A key strength of Horn clause encodings is that they are inherently modular. This means that loops and recursion are abstracted by invariants resp. pre-/postcondition pairs. The latter enable Korn to significantly outperform all other tools in category Recursive. Plausible explanations are that classic state-space exploration techniques struggle to abstract call stacks or maybe that techniques developed for loops like k-induction have simply not been adapted well to recursive procedures. For Horn clause encodings on the other hand both abstractions are uniform and solvers are largely agnostic to the purpose of predicates. As a downside of enforcing modular proofs, Korn is currently unable to compete in category Arrays, where finding the quantified invariants is hard but state-space exploration succeeds on tasks with fixed loop bounds.

Table 1. Comparison of official results (number of tasks solved) in comparison to result of the best-scoring other tool in that category and post-competition experiments after fixing an issue with the submitted Korn verifier archive which did not run Eldarica at all. # Tasks is the number of tasks supported by Korn vs. category size. The result marked by \(\dagger \) is without counterexample confirmation. The official results can be found at https://sv-comp.sosy-lab.org/2023/results/results-verified/
Full size table

Unfortunately, in the 2023 competition, Eldarica did not run at all due to some unknown problem with the verifier archive, such that Korn terminated way too early and missed out on many results. Table 1 presents results from re-running the evaluation on the competition hardware. This produces 208 additional proofs from Eldarica in category Loops with a hypothetical score of 755 wrt. 323 in SV-COMP 2023, albeit the actual score would be lower than that because usually not all witnesses are confirmed.

4 Software Project, Configuration & Participation

The implementation of Korn is available at https://github.com/gernst/korn under the MIT license, installation instructions are part of the README. The SV-COMP 2023 submission was packaged from commit 8e968dd and shows version 0.4. The included solvers are Z3 4.11.2 64 bit (default configuration) and Eldarica v2.0.8 (using -portfolio). The command line in SV-COMP 2023 is

figure c

Participation: ControlFlow, Loops, Recursive, XCSP for ReachSafety.

Contributors. Korn is developed and maintained by the author. G. Alexandru [1] and J. Blau have contributed insights to approach of loop contracts [7].

5 Data Availability Statement

The tool archive packaged for SV-COMP 2023 is part of the official tools artifact [4] and also available separately [9]. The official competition results [3] are complemented with our post-competition evaluation, based on commit 92e6732 and are available at [8].