figure a
figure b

1 Validation Approach

Goblint Validator is an extension of the Goblint verifier [14,15,16] for validation of correctness witnesses in the YAML format [1], consisting of location and loop invariants. The extension involves two related but independent components: witness invariants are checked for correctness and unassumed for speedup. We present here a high-level overview of our recently-published approach to abstract-interpretation–powered witness validation [17].

Correctness of witness invariants is determined by treating them as additional proof obligations. However, instead of inserting assert statements into the program, the validator uses the Goblint verifier as a black box to check whether its computed abstract states satisfy the witness invariants. Hence, invalid witness invariants cannot undermine soundness of the verification process via refinement.

Speedup from witness invariants is attained by incorporating novel unassume statements with the invariants into the program. As opposed to refining the abstract state like assume operations, these relax the state instead. Doing so in a controlled manner, fixpoint iteration can converge faster, i.e., in fewer iterations. In the best case, the witness invariant precisely characterizes the fixpoint, avoiding further iteration. Unassuming can also make the abstract interpreter more precise, without requiring more expressive abstract domains, by leading the solver to a more precise fixpoint, which widening would otherwise extrapolate over [17].

Sound unassume operators must preserve all reaching concrete states, thus preserving soundness of the entire analysis. Goblint Validator implements two different unassume operators:

  1. 1.

    For non-relational domains (e.g., numeric intervals or points-to sets), a classic propagating algorithm for assume operators [4, 7] is adapted with minimal modifications. This admits relaxing abstract values in dynamically allocated memory through pointers.

  2. 2.

    For relational domains (e.g., octagons), dual-narrowing [8] is employed to retain more relations than a generic unassume operator definition [17].

2 Software Architecture

Goblint Validator builds on the Goblint verifier [14,15,16] which is implemented in OCaml, uses an updated fork of CIL [12] as its frontend and Apron [9] for relational domains.

Instead of altering the control-flow graphs, unassume statements are inserted implicitly as events that activated analyses can handle. In the modular architecture of Goblint [2] the unassume analysis is responsible for emitting these events after transfer functions corresponding to witness invariants. Widening tokens [10] are used to delay widening and allow the invariants to be incorporated without immediate precision loss. The solution of a side-effecting constraint system [3, 18] is post-processed to validate witness invariants and determine the verdict.

3 Strengths and Weaknesses

Overall, Goblint Validator inherits the strengths and weaknesses of Goblint, which are described in its tool papers [14,15,16]. Thanks to the generic validation approach, the validator works in all SV-COMP categories as the Goblint verifier, including those that are currently excluded from correctness witness validation, e.g., concurrency. Due to over-approximation, the verifier can only prove the absence of bugs, but not their presence. Consequently, the validator can currently only confirm correctness witnesses. However, it could be extended to reject violation witnesses in the future.

Table 1. Number of tasks verified by Goblint and their witness validation verdicts by Goblint Validator, grouped by property.
Fig. 1.
figure 1

CPU time scatter plot where each mark (in ) indicates a task verified by Goblint and whose witness was confirmed by Goblint Validator. Ordinary least squares (OLS) regression (in ) follows \(y = 0.76x + 0.14\) (\(r^2 = 0.94\)).

Table 2. Percentage of witnesses from other verifiers confirmed by Goblint Validator.

We evaluate our validator according to the same three aspects considered by Beyer et al. [6]: same-framework consistency, content-effort dependence and cross-framework validation. The first two only focus on witnesses produced by the Goblint verifier.

Regarding same-framework consistency, table 1 lists how many tasks with each property it can verify and how many of those witnesses Goblint Validator can confirm. The overall average confirmation rate of 78% is lower than the 90% Beyer et al. [6] report for CPAchecker and UAutomizer with GraphML witnesses. Reasons for unconfirmed witnesses range from excessive precision loss by unassuming to validator crashes. In some cases, the validator exceeds resource limits, likely due to large witnesses with many unhelpful invariants. A handful of instances indicate mismatches between witness generation and their interpretation due to implementation errors in either the verifier or the validator. Fixing such issues could improve the overall quality of the framework [6].

Regarding content-effort dependence, fig. 1 plots the corresponding verification and validation times in the 7,088 confirmed cases. While the results at the low end (< 1 s) are noisy, the results at the high end (> 5 s) show the benefit of witness validation, with up to \(10\times \) improvements. Regression analysis estimates an average speedup of 24%, which matches our previous results [17], albeit with greater variance. This is unlike CPAchecker and UAutomizer for which no general performance improvement from consuming witnesses was observed [6].

Regarding cross-framework validation, table 2 presents the confirmation rate of Goblint Validator of correctness witnesses from other tools. For the Ultimate tool family, the percentage is between 46% and 60%, which is similar to what Beyer et al. [6] observed. We have a high ratio for the Mopsa abstract interpreter [11], although it only produces trivial witnesses containing no invariants, on which Goblint Validator effectively reduces to the Goblint verifier. Nevertheless, overwhelming success of Mopsa in the SoftwareSystems category warrants independent validation of abstract interpretation results.

4 Tool Setup and Configuration

Goblint Validator version svcomp24-0-gc2e9465a7 took part in all categories except FalsificationOverall of SV-COMP 2024 [5, 13]. It is available in both binary (Ubuntu 22.04) and source code form at our GitHub repository.Footnote 1 Instructions for building from source can be found in the README.

The tool-info module for BenchExec is named goblint and the benchmark definition for SV-COMP is goblint-validate-correctness-witnesses-2.0. They correspond to running the tool as follows:

figure e

5 Software Project and Contributors

Goblint Validator development takes place alongside Goblint on GitHub, while related publications are listed on its website.Footnote 2 It is an MIT-licensed project initiated by Technische Universität München and the University of Tartu.