Verification Artifacts in Cooperative Verification: Survey and Unifying Component Framework

The goal of cooperative verification is to combine verification approaches in such a way that they work together to verify a system model. In particular, cooperative verifiers provide exchangeable information (verification artifacts) to other verifiers or consume such information from other verifiers with the goal of increasing the overall effectiveness and efficiency of the verification process. This paper first gives an overview over approaches for leveraging strengths of different techniques, algorithms, and tools in order to increase the power and abilities of the state of the art in software verification. Second, we specifically outline cooperative verification approaches and discuss their employed verification artifacts. We formalize all artifacts in a uniform way, thereby fixing their semantics and providing verifiers with a precise meaning of the exchanged information.

In contrast to these extremely tight or extremely loose combinations, cooperative verification is a combination of approaches that cooperate, that is, work together to achieve the verification goal, but leave the existing tools (mostly) untouched. Cooperative verifiers communicate with each other in order to maximize the common strength, in particular, by exchanging information about intermediate results. In a framework for cooperative verification, the integration of a new technique requires some implementation to make it understand the communication, viz. be able to use intermediate results, but it can avoid a new re-implementation of the combination -from the conceptual as well as from the practical viewpoint. If the intermediate results come in a format already accepted by the tool (e.g. as a program), the tool can even be employed as is.
In this paper, we provide a classification of verification approaches according to the interface and type of combination employed; we briefly survey combination approaches, for portfolio, selection, cooperative, and conceptual combination of verification approaches. We then discuss a number of aspects relevant to cooperative verification, in particular its objectives and prerequisites. In the following, we provide a classification of verification approaches according to their way of interfacing and combining verification components. By the term "verification approach" we understand an automatic or automatable formal method for solving verification tasks, i.e., for evaluating the proposition "Program p satisfies behavioral specification ϕ b " and returning a result r, which can be true (p |= ϕ b ), false (p |= ϕ b ), or unknown, and an (optional) witness ω, which contains proof hints, as depicted in Fig. 1.

Overview over Interfaces
Verifier Consumption by humans Consumption by machines

Fig. 2: Output Interfaces
Output. The goal of a verification tool is to solve a verification task and to deliver the computed results to either a verification engineer for manual inspection or to a machine for further automated processing (Fig. 2). Depending on how the results are consumed (by human or by machine), the tool needs to use different formats. While researchers mainly concentrated on improving the (internal) verification algorithms during the past two decades, it is understood since recently that it is (at least) equally important to provide not only true/false answers, but more details about the reasoning and the process of the verification. Human. Almost all verification tools provide some kind of statistics to the user, for example, about the number of iterations, of number of proof facts, or consumed resources. Execution reports [30] present an underapproximation of Information from humans Information from machines

Fig. 3: Input Interfaces
Input. Similar to the output, there are different interfaces for the kind of input that is given to the verification tools, some from users, some from machines, see Fig. 3. Human. From the very beginning of programming, assertions were added to programs [73] in order to make it easier to prove correctness. Nowadays, assertions, invariants, pre-and post-conditions, are annotated in programs in a way that machines (interactive verifiers) can read. There are several languages and tools that support this, and a nice overview over such tools and their application opportunities are given in the annual competition on interactive software verification VerifyThis [41].
There were also attempts to support the splitting of specifications and programs into modular parts, in order to make the verification task for the model checkers easier, such as in the Blast query language [8,68]. Last not least, and this is one of the most difficult parts, each verifier expects a set of parameters that the user has to set, in order to choose how the verifier should solve its task. However, finding the right parameters is a challenging task, which could use tool support itself (such as SMAC [52] or Tuner [71]).

Machine.
A classic approach to make additional information available to a tool is by transforming the original input, e.g., by simplification or enhancement. The advantage is that there is no additional input (no extra parser, no need to implement additional features). For example, the first software model checkers did not have a specification language, but the specification was weaved into the program in a preprocessing step (as was done for the Slam [2] specification language Slic [1] and the Blast [16] query language [8]). Even programs were made simpler [63].
Verification witnesses and conditions were discussed already above as example implementations for output interfaces. Verification witnesses can be taken as input by validation tools that re-establish the verification result using independent technology. Also, the error path described by the violation witness can be replayed and a test case can be derived from the path constraints along the found error path [13].
Conditional model checking is not widespread yet because it was considered difficult to extend a verifier such that it understands conditions as input and reduces the state space accordingly before running the verification engine. This problem was solved by the reducer-based construction of conditional verifiers: Reducers [23,37] can be used to construct (without implementation effort) conditional model checkers from off-the-shelf verifiers that do not understand conditions themselves, by reducing the original input program to a residual program that contains all the behavior that is not yet covered by the condition and removes as much as possible from the already-verified state space.

Overview over Combinations
In the early days of automatic program verification, tools implemented a single technique for verification (e.g., explicit enumeration of state space or dataflow analysis using a fixed abstract domain). In our classification (see Fig. 4) these are represented as Basic. Later, the tools implementing these techniques were considerably generalized, for instance by allowing abstract domains to be flexibly set via tool parameters. Still, during one verification run a single basic technique was employed.
It soon turned out that a single verification technique may work well for some verification tasks, but fail for others. This immediately triggered the application of Combination techniques, in order to benefit from the different strengths. Combinations can come in two sorts: A combination either treats techniques or tools as Black Box objects and runs them (mainly) as they are without implementation-specific integrations for which it matters what's inside the box, or a combination views a component as White Box, conceptually integrating two or more techniques within a new tool. We distinguish three forms of black-box combinations, without and with communication, and classify all white-box approaches into one category.  Portfolio combinations are motivated by the portfolio idea from economics [51], which is a means of distribution of risk: if one investment (here: of computational resources in a certain technique) fails, there are other investments (techniques) that will be successful. A portfolio combination has a number of approaches available, and on a given verification task executes the approaches in a fixed order sequentially (Fig. 5, top), or runs all approaches in parallel (Fig. 5, bottom). The overall approach terminates if one component analysis was successful in obtaining the result. The big advantage of this approach is that it requires no knowledge about the components and there is almost no effort for implementing the combination. Therefore, we placed this most loosely coupled approach on the very left in the bottom row of our Fig. 4. The big disadvantage of portfolio approaches is that the resources invested on unsuccessful tools or approaches are lost.
Algorithm Selection [67] is a solution to the problem of wasted resources of portfolio approaches: Algorithm-selection approaches have a number of approaches available, and on a given verification task choose one and execute it (Fig. 6). That is, before starting an approach, a selection model is extracted from the input, based on which a selector function predicts which approach would be best, and only the potentially best approach is selected and executed. This requires some knowledge about the (black box) characterization of the components, but does not require any change of the implementation of the components.  Cooperation approaches enable the possibility of solving the problem together. Typically, tools exchange intermediate results (e.g., the state space which has already been searched) in order to achieve a division of labor. Such cooperative combinations range from two or more basic techniques running in parallel and combining the information obtained for certain program locations (e.g., combining partial verification results to proof witnesses [55]) to approaches executing different tools in turns with each specializing to specific tasks (e.g., a testing tool trying to achieve coverage together with a model checker constructing counter examples to non-reachability [38]).
Conceptual Integration is most intensively coupled approach and therefore put on the very right end of the bottom row in our Fig. 4. The components are not communicating via clear interfaces, but are tightly integrated and exchange data structures via procedure calls and not via interfaces that could be externalized [15].
In the following subsections, we describe some forms of non-cooperative verification approaches in more detail. In the next section we explain some examples for cooperative verification approaches.

Examples for Portfolio Combinations
While it seems obvious that combinations of verification techniques have a large potential, for software verification the topic is not yet systematically investigated, while it is used in other areas since many years [51].
Sequential Combinations. Examples of sequential combinations are SDV and CPAchecker. The static driver verification (SDV) [3] tool chain at Microsoft used a sequential combination (described in [72]) which first runs Corral [59] for up to 1 400 s and then Yogi [64]. CPAchecker [24] won the competition on software verification 2013 (SV-COMP'13, [5]) using a sequential combination [76] that started with explicit-state model checking for up to 100 s and then switched to a predicate analysis [25].
Parallel Combinations. Examples of parallel combinations are the verifiers Ufo [47] and PredatorHP [62], which start several different strategies simultaneously and take the result from the approach that terminates first.

Examples for Algorithm Selection
Algorithm selection [67] first extracts a selection model from the input. In the case of software verification, the input is the verification task (program and its specification). The selection model describes some characteristics of the verification task, for example, feature vectors (measurement values for certain measures that map verification tasks to values). Based on the selection model, the strategy selector chooses one strategy from a set of given verification strategies.
Approaches without Machine Learning. Strategy selection can be very simple and yet effective. For example, a recent work has shown that it is possible with a few boolean features to effectively improve the overall verification progress [10]. The disadvantage is that the strategy selector needs to be explicitly defined by the developer or user. This leads to approaches that use machine learning, in order to automatically learn the strategy selector from training instances.
Machine-Learning-Based Approaches. The technique MUX [72] can be used to synthesize a strategy selector for a set of features of the input program and a given number of strategies. The strategies are verification tools in this case, and the feature values are statically extracted from the source code of the input program. Later, a technique that uses more sophisticated features was proposed [39,40]. While the above techniques use explicit features (defined by measures on the source code), a more recently developed technique [36] leaves it up to the machine learning to obtain insights from the input program. The advantage is that there is no need to define the features: the learner is given the control-flow graph, the data-dependency graph, and the abstract syntax tree, and automatically derives internally the characteristics that it needs. Also, the technique predicts a ranking, that is, the strategy selector is not a function that maps verification tasks to a strategy, but to a sequence of strategies.

Examples for Conceptual Integrations
Conceptual integrations tightly combine two or more approaches into a new tool, typically re-implementing the basic techniques. A frequent combination of this type is integrating an overapproximating (static) may-analysis with an underapproximating (dynamic) must-analysis. The tool SMASH [44] at the same time maintains an over and an under approximation of the state space of programs. Building on the same idea, Beckman et al. [4] (tool Yogi, first proposal of the algorithm under name Synergy in [45]) in addition specifically employs testing to derive alias information which is costly to precisely compute by a static analysis.
A second form of conceptual integration is offered by tools running different analysis in parallel in a form of "product" construction. A prototypical example is the tool CPAchecker [24] with the possibility of specifying and running composite analyses. A composite analysis could for instance combine two sorts of data-flow analyses (e.g., an interval analysis and an available-expression analysis). The analyses are then jointly run and jointly derive analysis information for program locations. The same idea was classically hard-coded as reduced product [35] and further improved [20,34,42,46,60].
All those combinations have in common that they exchange information, but they are more intertwinned, hardcoded combinations, rather than interface-based combinations. More approaches are described in the Handbook on Model Checking, in the chapters on combining model checking with data-flow analysis [15], with deductive verification [69], and with testing [43].

Cooperative Verification Approaches
In the following, we discuss approaches for cooperative verification, structured according to the kind of information objects which are exchanged, and then explain a few applications and their effects.

Exchangeable Objects for Communication and Information Transfer
We now classify the approaches for cooperative verification according to the kinds of communication interfaces that they use. While our text always refers to software verification for concrete examples, cooperative verification is in no way limited to software. Conditions and Residual Programs. Conditional model checking (CMC) [17] means to produce a condition as output that describes the state-space that was successfully verified. The (temporal) condition can be represented as an automaton. This information can be passed on to another verifier as input, instructing this verifier to not verify again those parts of the state space that are covered by the condition. Using a reducer [23], a program can be reduced to those parts of its state space that still has to be verified; the result is called residual program. Symbiotic [31] can be seen as reducer-based cooperation (slicer + Klee).
Witnesses. Exchangeable witnesses serve as envelopes for error paths and invariants in a way that makes it possible to exchange the information between different tools. A violation witness [6,12,27] explains the specification violation, by describing a path through the program that violates the specification. A correctness witness [11] explains why the program satisfied the specification, by describing invariants that are useful to have in a correctness proof. Figure 7 illustrates the process: The first analyzer verifies the program p according to specification ϕ b , and produces a result r and a witness ω. The second analyzer (re-)verifies the same program and specification using information from the witness. If the result r matches the result r', then the result is confirmed.
Precisions. Verification approaches that are based on counterexample-guided abstraction refinement (CEGAR) [33] iteratively construct an abstract model of the system. The "abstraction facts" that define the level of abstraction are often formalized and expressed as precision [21,26,27]. The precision can be exported as output, such that later verification runs can start from such a given definition of the abstraction level.
Abstract States / Certificates. Extreme model checking [48] dumps the abstract reachability graph (ARG) to a file when the verification process terminates. Configurable certificates [54] are sets of abstract states that cover all reachable states of the system. ARGs and configurable certificates can be used by a different verifier to check its validity (completeness and soundness).

Path Programs and Path Invariants.
Path programs [19] are programs (for example, written in the same programming language as the input program) that were invented to incorporate external invariant generators into CEGAR-based approaches and are produced after a verifier has found an infeasible error path (often called infeasible counterexample). The path program contains that path in question, but usually also longer paths that use the same program operations, that is, unrollings of a certain loop. The path program can now be given to a tool for invariant synthesis (e.g., [18]) in order to obtain path invariants [19], which are invariants for the whole path program, but in particular also for the original path. The path invariants can then be fed back into the CEGAR-based approach that was encountering the original path.

Objectives and Applications
Having exchangeable objects about (partial) verification results [28] available is important to overcome a variety of practical problems. In the following, we highlight a few of the objectives and applications that we can aim for. Precisions that are stored and in later verification runs read and reused can significantly improve the performance of regression verification [26]. The setup of this strategy is the following: the first version of a module is verified and at the end, the precision is written to a file. When the i-th version is verified, then the verifier reads the precision that the verification run for version i−1 has written, in order to save time discovering the right abstraction level for the abstract model. [54] can reduce the validation time, because the verifier that performs the validation of the certificate does "only" need to check for the set of abstract states that all initial states are contained and that the set is closed under successor transitions.

Configurable certificates
Also caching-based approaches to improve the efficiency can be seen as a stateful way of performing computation. For example, Green [74] makes symbolic execution more efficient by caching previous intermediate results.
Stateless Verification and Parallelization. The previous argument was based on having a state that contains the intermediate results. It is also possible to speed up verification processes in a stateless way. The technique of conditional model checking is used to split programs into parts that can be independently verified [70]. [11,12] can be used to increase the confidence in the results of verification tools, because it is possible to take a witness-based results validator to "replay" the verification. That is, for a violation witness, the validator tries to find and confirm the error path that the witness describes, and for a correctness witness, the validator tries to use the invariants in the witness to re-establish the proof of correctness.

Improvement of Precision and Confidence. Witness-based results validation
Execution-based results validation [13] extracts a test case from a violation witness and executes it, in order to confirm that the specification violation is observable in the executed program as well. Explainability. The existence and availability of exchangeable objects with information about the verification process makes it possible to develop approaches for explaining what the verification process did and why the user should be more confident about the verification result. There are preliminary results on explaining and visualizing counterexamples, e.g., for SPIN models [61] and for C programs [9], but due to the exchangeable witness format, many more approaches are possible.

Verification Artifacts
This section outlines a construction framework for cooperation. We study verification artifacts, classify several verification tools as verification actors according to their usage of artifacts, and define the semantics of some important artifacts.

Artifacts of Verification Tools
Verification artifacts are central to cooperation as they provide the means of information exchange. A number of artifacts exist already, most notably of course the programs themselves. We identified the following artifacts: Program p. Defines the implemented behavior of the system. Syntax: C programming language (for example). We represent programs as control-flow automata in Sect. 4.3. Behavior Specification ϕ b . Defines requirements that all executions of a given program have to satisfy, often as conjunction of several properties. Syntax: The competition SV-COMP established a minimal set of properties that participants of the competition have to support 1 , which is based on LTL [66], but some tools also support monitor automata as specification. We represent properties by property automata in Sect. 4.3. Test Specification ϕ t . Defines requirements that a given test suite has to satisfy. Syntax: The competition Test-Comp established a minimal set of coverage criteria that participants of the competition have to support 2 , which is based on FQL [49,50], but some tools also offer parameters for hard-coded coverage criteria. We represent coverage criteria via test-goal automata in Sect. 4.3. Verification Result r. Verification tools return an evaluation of the statement "Program p satisfies specification ϕ b ." as answer, which is from the set {true, false, unknown}. Witness ω. Verification witnesses are used to witness an outcome of a verification run, and thus can come in the form of violation and correctness witnesses. Syntax: XML-based witness format 3 that is supported by all available validators of verification results.

Test case t. Defines a sequence of values for all calls of external functions,
i.e., inputs for the program. Syntax: XML-based test-case format 4 that is supported by all test-case generators that participate in Test-Comp. Condition ψ. Defines the part of the program behavior that does not need to be further explored. For verification, ψ describes the already verified parts. For testing, ψ describes the parts of the program that are already covered by an existing test suite. Syntax: Condition automata using a notation similar to the Blast query language [8].
We use the corresponding capital letters to denote the types (i.e., sets of artifacts of a kind), for example, the type P is the set of all C programs. Many tools generate different forms of verification artifacts, but currently only very few understand more than the artifact "program" as input.

Classification of Verification Tools as Actors
Based on the identified artifacts, we classify existing tools according to their usage of artifacts into three sorts of verification actors:

Analyzers.
Produce new knowledge about programs, for example verification results or test suites. Transformers. Translate one artifact into another, in order to implement a certain feature or support cooperation. Presenters. Prepare information from artifacts such that it can be presented in a human-readable form.  To convey a better understanding of these concepts, consider the following examples: A verifier is an analyzer of type P × Φ b → R × Ω, which takes as input a program p and a behavior specification ϕ b , and produces as output a result r and a witness ω 5 i.e., a verifier that supports also input and output conditions. A validator is of type P × Φ b × Ω → R × Ω, i.e., a verifier that takes as input in addition a witness. A test-case generator is also an analyzer, but of type P × Φ t → 2 T , which takes as input a program p and a test specification ϕ t , and produces as output a set ts ∈ 2 T of test cases.
Transformers are largely lacking today, only a few exist already [13,23]. Transformers are, however, key to cooperation: only if a transformer can bring the artifact into a form understandable by the next tool without implementing an extension of this tool, cooperation can be put into practice. A test-case extractor is a transformer of type P ×Φ b ×Ω → T , which translates a program, specification, and violation witness to a test case. The identity function is also a transformer (for any given type). A reducer is a transformer of type P × Ψ → P , which takes a program and a condition as input, and transforms it to a residual program.
Presenters form the interface to the user. A test-case executor is a presenter of type P × T → {}, which takes a program p and a test case T as input, and shows a particular program execution to the software engineer. Now we can construct, for example, a conditional verifier from a reducer red and an off-the-shelf verifier ver by composition. For inputs p and ϕ b , the expression ver (red(p, ϕ b ), ϕ b ) runs the construction. For a verification with an execution- With our construction framework, it is possible to identify the gaps of meaningful transformers, and propose solutions to close these gaps, as far as needed for cooperation.

Semantics
We now develop the theoretical foundations of artifacts and actors. Artifacts describe some information about a program (or a program itself), and for sound cooperation we need to define the semantics of artifacts. For instance, a violation witness of a program describes a path of the program on which a specific specification is violated, a condition describes a set of paths of a program which have (or have not been) inspected by an analyzer. When employing cooperation as a means for sharing the work load of validation, the cooperating tools need to agree on the meaning of the exchanged artifacts. Without this, cooperation might easily get unsound, e.g., returning a result true for a program and specification although the combined usage of tools has failed to inspect the whole state space of the program. By defining the semantics of artifacts, we also implicitly define the desired semantics of the various actors operating on artifacts.
We start the formalization of artifacts with the definition of programs, our prime artifact. We denote the set of all program locations by Loc. Formally, a program p is described by a control-flow automaton (CFA) A p = (L, 0 , G) that consists of a set of locations L ⊆ Loc, an initial location 0 ∈ L, and a set of control-flow edges G ⊆ L×Ops×L, where Ops is the set of operations. Operations can be (a) assignments, (b) assume statements (arising out of branches), and (c) calls to functions retrieving inputs. Here, we assume to have a single such function, called input. We let G = L × Ops × L be the set of all control-flow edges.
We let X be the set of variables occurring in the operations Ops. For simplicity, we restrict the type of variables to integers. A concrete data state c : X −→ • Z is thus a partial mapping from X to Z. In the left of Fig. 9 we see our running example of the simple program p and its control-flow automaton on the right. The program starts by retrieving an input for variable x, sets variables a and b to 0, and then increments both while the value of a is less than that of x.
A concrete program path of a program A p = (L, 0 , G) is a sequence We let paths(A p ) be the set of all concrete program paths.
We allow artifacts to state assumptions on program variables. These are given as state conditions (from a set Γ of predicates over a certain theory). We write c |= γ to say that a concrete state c satisfies a state condition γ ∈ Γ . Artifacts on a program p are represented by artifact automata: −−− → q for (q, (D, γ), q ) ∈ δ. In figures, we often elide invariants as well as the assumptions on edges when they are true. We furthermore elide the set notation when the element of 2 G is a singleton.
Artifact automata describe paths of a program. Depending on the sort of artifact automaton, these could for instance be paths allowed or disallowed by a specification, or paths already checked by a verifier. A path of the program can be accepted (if the automaton reaches a final state) or covered by the automaton.

Definition 2. An artifact automaton
The artifact automaton A accepts the path π if A matches π and q k ∈ F , and A covers π if A matches π and k = n. We let L(A) be the set of paths accepted by the automaton A (its language) and paths(A) be the set of paths covered by A. As we will see below, some artifact automata might have an empty set of final states and just describe a set of paths that they cover.
Artifact Automata as Representation of Artifacts. We consider different specializations of artifact automata and use the notation A s to denote the automaton that represents the syntactical object s.
(1) A property automaton (or, observer automaton) A ¬ϕ b = (Q, Σ, δ, q 0 , Inv, F ) is an artifact automaton that satisfies the following conditions: Condition 2 ensures that property automata only observe the state of the program (when running in parallel with the program). They do not block, except for the case when the final state is reached where blocking is allowed. Final states denote the reaching of property violations (or, targets).
(2) A test-goal automaton A ϕt = (Q, Σ, δ, q 0 , Inv, F ) is an artifact automaton that has only trivial state invariants, i.e., ∀q ∈ Q : Inv(q) = true. If a final state is reached, the test goal is fulfilled. Figure 10 shows two specification automata: In Fig. 10a we see a property automaton specifying that variables a and b have to be equal when the loop terminates, i.e., the error state is reached if there is a transition from location 3 to 6 at which a = b. The label o/w (otherwise) denotes all transitions other than the ones explicitly depicted. Figure 10b depicts a test-goal automaton for the branch condition entering the loop.
Violation witnesses are used to describe the part of a program's state space which contains the error. The final state is reached if an error is detected. Counter examples are a specific form of violation witnesses which describe a single path.
(4) A correctness-witness automaton A ω = (Q, Σ, δ, q 0 , Inv, F ) is an artifact automaton that has only trivial transition assumptions, i.e., ∀(q, (D, γ), q ) ∈ δ : γ = true , and there are no final states ( F = ∅ ). A correctness witness typically gives information about the state space of the program (like a loop invariant) in order to facilitate its verification.
In Fig. 11 we see both a correctness and a violation witness. The correctness witness belongs to program p and, e.g., certifies that at location 3 variables a and b are equal (via the invariant for q 3 ). The violation witness on the right belongs to program p with line 5 removed, i.e., a program which does not satisfy the property stated in Fig. 10a. The violation witness states that an input value of x being greater or equal to 1 is needed for directing the verifier towards the error.
A condition is typically used to describe parts of the state space of a program, e.g., the part already explored during verification. Final states are thus used to fix which paths have already been explored.
A test case is a sequence of input values consecutively supplied to the calls of function input. Such a test case is encoded as artifact automaton using a special template variable χ that can be instantiated with every program variable.

(6)
A test-case automaton A t = (Q, Σ, δ, q 0 , Inv, F ) for a test case z 1 , . . . , z n is an artifact automaton with the following components: For matching these special transitions (G i , γ i ) = (( * , χ = input(), * ), χ = z) with program paths, the program transitions g i have to be of the form ( , x = input(), ) and the next state needs to satisfy c i (x) = z, c i (y) = c i−1 (y) for y = x. Figure 12a gives a condition stating the exploration of the state space for inputs less or equal to 0. This could for instance be the output of a verifier having checked that the property holds for inputs x ≤ 0. Figure 12b is the test-case automaton for the test case 4 .
Semantics of Artifact Automata. The above definitions fix the syntactical structure of artifact automata. In addition, we need to state their semantics, i.e., the meaning of particular artifacts for a given program. In the following, we let A p = (L, 0 , G) be the CFA for a program p and A ¬ϕ b , A ω , and A ϕt be artifact automata.  paths(A p )∩L(A ¬ϕ b ), the correctness witness helps in proving the program correct. The correctness witness in Fig. 11a is valid for p and the property in Fig. 10a.
(iii) A violation witness ω is valid for a program p and a property specification ϕ b if paths(A p ) ∩ L(A ω ) ∩ L(A ¬ϕ b ) = ∅ . During verification, violation witnesses can thus steer state-space exploration towards the property violation. Looking again at the running example: If we elide the statement in location 5 of our program, the automaton in Fig. 11b is a valid violation witness. It restricts the state-space exploration to inputs for variable x which are greater or equal to 1.
(iv) A condition ψ is correct for a program p and property ϕ b if paths(A p ) ∩ L(A ψ ) ∩ L(A ¬ϕ b ) = ∅ . All program paths accepted by the condition fulfill the specification given by the property automaton. The condition in Fig. 12a describes all paths of the program p which initially started with input x less or equal to 0. This condition is correct for p and the property automaton in Fig. 10a. Finally, for testing we are interested in test inputs covering certain test goals.
(v) A test-case t for a program p covers the goals of a test-goal specification ϕ t if paths(A p ) ∩ paths(A t ) ∩ L(A ϕt ) = ∅ . Basically, we require that the inputs provided by the test case guarantee program execution to reach (at least one) test goal. If there are more than one final state in the test-goal automaton (or the final state can be reached via different paths), the test-goal automaton specifies several test goals. In this case, the test case covers only some of these goals. The test-case automaton in Fig. 12b for p covers the (single) goal of the test-goal automaton in Fig. 10b.

Conclusion
Different verification approaches have different strengths, and the only way to benefit from a variety of approaches is to combine them. The two classic approaches of combining approaches either in white-box manner via a tight conceptual integration or in black-bock manner via loosely coupled combinations, such as portfolio or selection, are both insufficient. We propose that the right direction to go is the way of cooperation: a looselycoupled combination of tools that interact via clear interfaces and exchange formats, in order to achieve the verification goal together. To this end, we provide a classification and an overview of existing techniques, which we briefly describe, while giving most importance to cooperative approaches.