figure a
figure b

1 Verification approach

Bubaak  [7] is a program analysis tool that runs multiple verifiers at the same time, and uses ideas from runtime monitoring and enforcement [5, 10] to mediate the communication of useful information between the verifiers, such as invariants or already explored parts of the program. As of this year, the verifiers can be executed in an arbitrary combination of sequential and parallel portfolio, fully dynamically based on the information learned during the verification process.

With Bubaak-SpLit , we explore program splitting [12, 13] as a way to improve the scalability of the verification process. The main idea behind program splitting is to split a given program P into multiple subprograms \(P_1, \dots , P_n\) which then can be analyzed in parallel. As a result, Bubaak-SpLit can verify multiple subprograms with multiple verifier instances at the same time.

Control-flow Splitting. Bubaak-SpLit adopts control-flow splittingFootnote 1 [13] for splitting programs into subprograms. Control-flow splitting splits a program P at the first branching point B creating two subprograms \(P^+\) and \(P^-\). \(P^+\) and \(P^-\) each represent the program P when assuming that the branching condition at B is evaluated to true or false respectively. For example, Figure 2 depicts \(P^+\) and \(P^-\) when splitting the program in Figure 1 at the first branching point in Line 3. Syntactically splitting a program might result in suboptimal splits [12] where one part of the split is easy-to-verify and the other remains hard-to-verify. To mitigate the problem of suboptimal splits, Bubaak-SpLit implements a dynamic splitting strategy: (1) we first check if the given program (or split) is hard-to-verify by running a weak verifier, (2) if it is hard-to-verify we split the program and repeat the process on the generated splits, (3) if it is not hard-to-verify we record the result of the weak verifier and continue with the other splits (if any). We continue this process until a fixed number of hard-to-verify splits is generated or a splitting limit is reached. If the problem is solved during the splitting process, we report the results of the weak verifiers.

Figure 1 provides an example of the splitting process. After splitting two times, Bubaak-SpLit identifies two hard-to-verify splits which are then verified by two verifiers in parallel in the main verification phase. Existing static splitting strategies for C programs [12] might stop after the first split, resulting in a suboptimal split (with little to no benefits for the verification process).

Fig. 1.
figure 1

Overview over the verification process of Bubaak-SpLit for the given example. Bubaak-SpLit splits program that are too hard to be verified by a weak verifier (gray nodes), stops for easy-to-verify nodes (crossed out nodes) and it proceeds until n hard-to-verify splits are found (green nodes).

Full size image

Verification technology. Bubaak-SpLit in SV-COMP 2024 utilizes verifiers based on forward and backward symbolic execution.

(Forward) symbolic execution (SE) [14] systematically explores program’s executions from the initial location. Backward symbolic execution (BSE) [8] explores executions that reach a given (error) location and it does so by analyzing the program backwards from the locations. We employ a variant of BSE with loop folding (BSELF) [8] which allows us to generate loop invariants and prove programs correct.

SE can very quickly identify easy-to-verify problems, so we use it with a short timeout as the weak verifier during splitting. Strong verifiers in the main verification phase are selected based on the property. For the property unreach-call, we use BSELF and SE (with no timeout) in parallel – BSELF to prove programs correct and SE to (mainly) find bugs. Other properties are not supported by BSELF. For checking termination properties, we run SE and termination with inductive invariants with progress (TIIP) [7]. For checking memory safety, we use only SE. Note that the splitting phase is executed for all properties.

Fig. 2.
figure 2

Result of splitting the program from Figure 1 at the first branching point.

Full size image

2 Software architecture

Bubaak runs verification tools in a combination of sequential and parallel portfolio. The verifiers are not composed into a fixed scheme, but they are invoked dynamically based on the information gathered during the verification process. In a bit more detail, the architecture of Bubaak is inspired by process algebras [4] and is centered about tasks and their rewriting. The tool starts with the execution of a set of initial tasks; upon finishing, each task either yields a result, or rewrites itself into a new task or a set of new tasks. Whenever a task rewrites itself into a set of new tasks, it also specifies how the results of the new tasks should be aggregated into a single result. The important feature is that generating new tasks is not fixed in a static scheme: a task can rewrite itself into new tasks based on the context and information hitherto gathered about the program during the verification process.

What tasks are executed and how they are being rewritten is defined by a selected workflow. The workflow for splitting in SV-COMP 2024 is depicted in Figure 3. It defines the task Split(P) that takes program P and splits it into two parts as described in Section 1. This task is invoked as the initial task on the input program. After splitting the program, Split rewrites itself into two identical tasks CCAndCheckWeak that are invoked on those two splits. As the name suggests, the input split is compiled (into LLVM [1]) and the weak verifier is ran on it to check if the split is easy to solve. If a split is not easy to solve, the task Split is invoked on the split recursively, and this process continues until a pre-defined depth is reached, at which point instead of splitting further the workflow invokes the strong verifier.

Fig. 3.
figure 3

The workflow of Bubaak-SpLit for SV-COMP 2024. For brevity, the scheme does not show errors handling and the result propagation.

Full size image

Workflows are only an abstraction: internally, task execution and rewriting is implemented using an event loop that handles events coming from tasks, task creation and destruction, and the results aggregation.

The weak verifier is based on Bubaak-Lee and we run a combination of Bubaak-Lee and Slowbeast during the main verification phase. Bubaak and Slowbeast are implemented in Python, Bubaak-Lee is in C++. Both Slowbeast and Bubaak-Lee use Z3 [9] as the SMT solver.

3 Strengths and Weaknesses

Program splitting has been shown to improve the runtime efficiency [15] and verification effectiveness [11] of symbolic execution engines. By splitting the program into several parts, SE and BSE can analyze different parts of the program at the same time, which can lead to results being decided more quickly. In SV-COMP 2024 [6], Bubaak-SpLit was able to solve 60 benchmarks that Bubaak was not able to solve and 456 benchmarks were solved faster, often significantly. In comparison to Bubaak, Bubaak-SpLit misses several violations on the ReachSafety benchmarks. Most of them are due to the fact that we severely limit the execution time of SE during verification. Another problem is the scalability of our approach in the restricted setting of the SV-COMP. By splitting the program up to n times, we currently run up to \(2^n\) verifiers at the same time. While in praxis this might significantly reduce the walltime, it also significantly reduces the cputime available to each verifier. Overall, Bubaak-SpLit inherits the strengths of the underlying analyses which allows the tool to perform well in the categories ReachSafety and SoftwareSystems. After SV-COMP 2024, we have found and fixed several bugs in the implementation of Bubaak-SpLit which might have severely limited its performance.