Constraint-based Relational Verification

In recent years they have been numerous works that aim to automate relational verification. Meanwhile, although Constrained Horn Clauses (CHCs) empower a wide range of verification techniques and tools, they lack the ability to express hyperproperties beyond $k$-safety such as generalized non-interference and co-termination. This paper describes a novel and fully automated constraint-based approach to relational verification. We first introduce a new class of predicate Constraint Satisfaction Problems called pfwCSP where constraints are represented as clauses modulo first-order theories over predicate variables of three kinds: ordinary, well-founded, or functional. This generalization over CHCs permits arbitrary (i.e., possibly non-Horn) clauses, well-foundedness constraints, functionality constraints, and is capable of expressing these relational verification problems. Our approach enables us to express and automatically verify problem instances that require non-trivial (i.e., non-sequential and non-lock-step) self-composition by automatically inferring appropriate schedulers (or alignment) that dictate when and which program copies move. To solve problems in this new language, we present a constraint solving method for pfwCSP based on stratified CounterExample-Guided Inductive Synthesis (CEGIS) of ordinary, well-founded, and functional predicates. We have implemented the proposed framework and obtained promising results on diverse relational verification problems that are beyond the scope of the previous verification frameworks.

A key challenge in relational property verification is the discovery of relational invariants which relate the states of multiple program executions.However, whereas most prior approaches must fix the execution schedule5 (e.g., lockstep or sequential) [8,20,21,42,54,57], a recent work by Shemer et al. [50] has proposed a method to automatically infer sufficient fair schedulers to prove the goal relational property.Importantly, the schedulers in their approach can be semantic in which the choice of which program to execute can depend on the states of the programs as opposed to the classic syntactic schedulers such as lock-step and sequential that can only depend on the control locations.However, their approach requires the user to provide appropriate atomic predicates and is not fully automatic.Moreover, they only support k-safety properties.A recent work has proposed a method for automatically verifying non-hypersafety relational properties but only for finite state systems [19].
Meanwhile, today's constraint-based frameworks are also insufficient at automating relational verification.The class of predicate constraints called Constrained Horn Clauses (CHCs) [13] has been widely adopted as a "common intermediate language" for uniformly expressing verification problems for various programming paradigms, such as functional and object-oriented languages.Example uses of the CHCs framework include safety property verification [29,30,35] and refinement type inference [33,36,53,56,66].The separation of constraint generation and solving has facilitated the rapid development of constraint generation tools such as RCaml [56], SeaHorn [30], and JayHorn [35] as well as efficient constraint solving tools such as SPACER [37], Eldarica [32], and HoIce [14].Unfortunately, CHCs lack the ingredients to sufficiently express these relational verification problems.
In this paper we introduce automated support for relational verification by generalizing CHCs and introducing a new class of predicate Constraint Satisfaction Problems called pfwCSP.This language allows constraints that are arbitrary (i.e., possibly non-Horn) clauses modulo first-order theories over predicate variables that can be functional predicates, well-founded predicates or ordinary predicates.We then show that, thanks to the enhanced predicate variables, pfwCSP can express non-hypersafety relational properties such as cotermination [11], termination-sensitive non-interference (TS-NI) [63], and generalized non-interference (GNI) [40].In addition, our approach effectively quantifies over the schedule, expressing arbitrary fair semantic scheduling thanks to non-Horn clauses and functional predicates (functional predicates are needed to express fairness in the presence of non-termination which is needed for properties like co-termination and TS-GNI).The flexibility allows our approach to automatically discover a fair semantic schedule and verify difficult relational problem instances that require non-trivial schedules.We prove that our encodings are sound and complete.Expressing relational invariants with such flexible scheduling is not possible with CHCs.However, pfwCSP retains a key benefit of CHCs: the idea of separating constraint generation from solving.
We next present a novel constraint solving method for pfwCSP based on stratified CounterExample-Guided Inductive Synthesis (CEGIS) of ordinary, wellfounded, and functional predicates.In our method, ordinary predicates represent relational inductive invariants, well-founded predicates witness synchronous termination, and functional predicates represent Skolem functions witnessing existential quantifiers that encode angelic non-determinism.These witnesses for a relational property are often mutually dependent and involve many variables in a complicated way (see the extended report [58] for examples).The synthesis thus needs to use expressive templates without compromising the efficiency.Stratified CEGIS combines CEGIS [51] with stratified families of templates [55] (i.e., decomposing templates into a series of increasingly expressive templates) to achieve completeness in the sense of [34,55], a theoretical guarantee of convergence, and a faster and stable convergence by avoiding the overfitting problem of expressive templates to counterexamples [44].The constraint solving method naturally generalizes a number of previous techniques developed for CHCs solving and invariant/ranking function synthesis, addressing the challenges due to the generality of pfwCSP that is essential for relational verification.
We have implemented the above framework and have applied our tool PC-Sat to a diverse collection of 20 relational verification problems and obtained promising results.The benchmark problems go beyond the capabilities of the existing related tools (such as CHCs solvers and program verification tools).PCSat has solved 15 problems fully automatically by synthesizing complex witnesses for relational properties, and for the 5 problems that could not be solved fully automatically within the time limit, PCSat was able to solve them semiautomatically provided that a part of an invariant is manually given as a hint.

Relational verification problems
k-safety Consider the following program taken from [50] that uses a summation to calculate the square of x, and then doubles it.
This program also takes another input h and, if the value of h is true, calculates the result differently.The classical relational property termination-insensitive non-interference (TI-NI) says that, roughly, an observer cannot infer the value of high security variables (h in this case) by observing the outputs (y).This is a 2-safety property [17,54]: it relates two executions of the same program.In this example, we ask whether two executions that initially agree on x (i.e., x 1 = x 2 ) will agree on the resulting y (i.e., y 1 = y 2 ).The subscripts in these relations indicate copies of the program: x 1 is variable x in the first copy of the program and x 2 is variable x in the second copy.More generally, k-safety means that if the initial states of a k-tuple of programs satisfy a pre-relation Pre, then when they all terminate the k-tuple of post states will satisfy post-relation Post.
The literature proposes many ways to reason about k-safety including methods of reducing a multi-program problem to a single-program problem, such as through self-composition [8,54,57], product programs [7], and their variants [21,46,50,52,59].Their key challenge is that of scheduling: how to interleave the programs' executions so that invariants in the combined program are able to effectively describe cross-program relationships.Indeed, as proved by [50], verifying this example with the naïve lock-step scheduling is impossible with only linear arithmetic invariants while linear arithmetic invariants suffice with a more "semantic" scheduling that schedules the copy with h 1 = false to iterate the loop twice per each iteration of the loop in the copy with h 2 = true.
In this paper, we will describe a way to pose the scheduling problem as a part of a series of constraints so that the search for an effective scheduler is relegated to the solver level.In our approach, a k-safety verification problem is encoded as a set of constraints containing (ordinary) predicate variables that represent the scheduler to be discovered and a relational invariant preserved by the scheduler.Specially, we introduce a predicate variable inv that represents a relational invariant and for each A ⊆ {1, . . ., k}, a predicate variable sch A ( V 1 , . . ., V k ) where V i are the variables of the ith program, and add constraints that say that if the predicate is true, then the programs whose index are in A will step forward while the rest remain still and also inv is preserved by the step.For soundness, it is important to constrain the scheduler to be fair, i.e., at least one program that can progress must be scheduled to progress if there is a program that can progress.As we shall show in Sec. 4, non-Horn clauses are essential to expressing the fairness constraint.Roughly, the idea is to use a clause with multiple positive predicate variables (i.e., head disjunction) to say "if the relational invariant holds, then at least one of the unfinished programs must be scheduled to progress." Our approach is similar to and is inspired by the approach of [50] that also infers a fair semantic scheduler.However, their approach requires the user to provide sufficient atomic predicates manually and is not fully automated.By contrast, our approach soundly-and-completely encodes the k-safety verification problem together with scheduler inference as a set of constraints thanks to the expressiveness of pfwCSP, and automatically solves those constraints by the stratified CEGIS algorithm (cf.Sec.7 for further comparison).
Co-termination Now consider the following pair of programs.
A (non-safety) relational question is whether these programs P cot 1 and P cot 2 agree on termination [6,10].In general they do not: if, for example, P cot 1 is executed with x < 0 and P cot 2 with x > 0 ∧ y = 0, the first will terminate while the second will diverge.However, under the pre-relation Pre ≡ x 1 = x 2 ∧ y 1 = y 2 , they will agree on termination: the first program terminates iff the second one does.The property falls outside of the k-safety fragment as it cannot be refuted by finite execution traces.It is worth noting that termination-sensitive non-interference (TS-NI) is the conjunction of TI-NI and co-termination of two copies of the same target program with Pre equating the copies' low inputs.
Proving co-termination, like k-safety, can be aided by scheduler and we can again use our constraints over predicate variables.But this is not enough.We need additional constraints to ensure that whenever one of the two has terminated, the other is also guaranteed to terminate.To address this, we next introduce well-founded predicate variables.These predicate variables will appear in our generalized language of constraints as terms of the form wfr( V i , V i ), where the relation wfr must be discovered by the constraint solving method.(In Sec. 5 we describe how to achieve this through our stratified CEGIS algorithm.)For the above example, our stratified CEGIS algorithm and our tool PCSat automatically discovers (1) a schedule where the two programs step together when x 1 > 0 and x 2 > 0, (2) a relational invariant that implies that if the first program is terminated, then either the second program is terminated or y 2 ≥ 1 (and vice-versa), and (3) well-founded relations that (combined with the relational invariant) witness that if the loop has terminated in the second program (x 2 ≤ 0) but not in the first (x 1 > 0), then a transition in the first is wellfounded (and vice-versa).In Sec. 4, we show how co-termination problems can be soundly-and-completely encoded in pfwCSP.
Generalized non-interference.Now consider the following program.The * int (resp.* bool ) above indicates an integer (resp.a binary) non-deterministic choice.Termination-insensitive generalized non-interference (TI-GNI) [40] is an extension of non-interference to non-deterministic programs, and it says that for any two copies of the program with possibly different values for the high security input (high in this example) and with the same value for the low security input (low in this example), if one copy has a terminating execution that ends in some output (the final value of x in this example), then the other copy has either a terminating execution ending in the same output or a non-terminating execution.The termination-sensitive variant (TS-GNI) strengthens the condition by asserting that if one copy has a terminating execution then the other copy has a terminating execution that ends in the same output.Both GNI variants are ∀∃ hyperproperties and fall outside of the k-safety fragment.
Verifying GNI requires handling non-determinism.Note that non-determinism occurs both demonically (i.e., as ∀) and angelically (i.e., as ∃) in GNI.While handling demonic non-determinism is straightforward in a constraint-based verification since the term variables are implicitly universally quantified, handling angelic non-determinism is less straightforward.Our approach handles finitary angelic non-determinism like * bool by adding non-Horn clauses with head disjunctions that roughly express the condition "the relational invariant remains true in one of the finitely many next step choices".To handle infinitary non-determinism like * int , we introduce functional predicate variables denoted f( V , r).In these terms, f is a predicate variable to be discovered but with a new wrinkle: this predicate involves a return value r and the interpretation of f is a total function over V .For this example, we introduce the term f( V , r) where r represents the value chosen non-deterministically at * int and V are program variables and prophecy variables that represent the final return values of the demonic copy.For this example, PCSat automatically discovers the predicate r = ret 1 where ret 1 is the prophecy variable for the return value of the demonic copy.With it, PCSat is able to verify TS-GNI and TI-GNI of this example.We remark that functional predicates are also used to encode scheduler fairness in the presence of non-termination and is needed to ensure soundness for properties like co-termination and TS-GNI.In Sec.4.3, we show how TI-GNI and TS-GNI can be soundly-and-completely encoded in pfwCSP.

Challenges & Contributions
There are several challenges that we face in supporting relational verification problems with a constraint-based approach.The subsequent sections of this paper are organized around addressing those challenges as follows: -We first ask how to generalize the constraint language to go beyond CHCs to express a more general class of relational verification problems.To this end, in Sec. 3, we present a new language called predicated constraint satisfaction problems (pfwCSP), which incorporate non-Horn clauses, (ordinary) predicate variables, well-founded predicate variables, and functional predicate variables.-We next return to the above relational verification problems -k-safety, cotermination, and generalized non-interference-and describe how pfwCSP can express each of them in a sound and complete manner in Sec. 4. -The next major contribution of our research is a novel stratified CEGIS algorithm for solving pfwCSP constraints.Our approach integrates advanced verification techniques: stratified family of templates [55] and CEGIS of invariants/ranking functions [14,26,28,45].While the individual ideas have been proposed previously, they have only been designed for less expressive frameworks such as CHCs, and substantial extensions are needed to combine and apply them to the new pfwCSP framework as we shall show in Sec. 5. -We next turn to an implementation and experimental validation on a diverse collection of 20 relational verification problems, consisting of k-safety problems from Shemer et al. [50] and new co-termination and GNI problems in Sec. 6.As far as we know, none of the existing automated tools other than our new tool called PCSat can solve them.
In sum, Fig. 1 depicts each of these sections and how, together, they enable relational verification.For space, extra materials are deferred to the extended report [58].

Predicate Constraint Satisfaction Problems pfwCSP
As discussed in Sec. 2, CHCs are insufficient to express important relational verification problems.In the section we introduced a generalized language of constraints called pfwCSP.The language of constraint satisfaction problems (CSP) permits non-Horn clauses, predicate variable terms, including those for functional predicates and well-founded relations (pfw).We now define pfwCSP.
Let T be a (possibly many-sorted) first-order theory with the signature Σ.The syntax of T -formulas and T -terms is: Here, the meta-variables x and X respectively range over term and predicate variables.The meta-variables p and f respectively denote predicate and function symbols of Σ.We use s as a meta-variable ranging over sorts of the signature Σ.We write for the sort of propositions and s 1 → s 2 for the sort of functions from s 1 to s 2 .We write ar(o) and sort(o) respectively for the arity and the sort of a syntactic element o.A function f represents a constant if ar(f ) = 0. We write ftv (φ) and fpv (φ) respectively for the set of free term and predicate variables that occur in φ.We write x for a sequence of term variables, | x| for the length of x, and for the empty sequence.We often abbreviate ¬φ 1 ∨ φ 2 as φ 1 ⇒ φ 2 .We henceforth consider only well-sorted formulas and terms.We use ϕ as a meta-variable ranging over T -formulas without predicate variables.
We now define a pCSP C (with ordinary but without well-founded and functional predicate variables) to be a finite set of clauses of the form where 0 ≤ ≤ m.We write ftv (c) for the set of free term variables of a clause c.The set of free term variables of C is defined by ftv (C) = c∈C ftv (c).We regard the variables in ftv (c) as implicitly universally quantified.We write fpv (C) for the set of free predicate variables that occur in C. A predicate substitution σ is a finite map from predicate variables X to closed predicates of the form λx 1 , . . ., x ar(X) .ϕ.We write σ(C) for the application of σ to C and dom(σ) for the domain of σ.We call σ a syntactic solution for C if fpv (C) ⊆ dom(σ) and |= σ(C).Similarly, we call a predicate interpretation ρ a semantic solution for C if fpv (C) ⊆ dom(ρ) and ρ |= C.
Remark 1.The language pCSP generalizes over existing languages of constraints.
CHCs can be obtained as a restriction of pCSP where ≤ 1 in (1) for all clauses.
We can also define coCHCs as pCSP but with the restriction that m ≤ + 1 for all clauses.A linear CHCs is a pCSP that is both CHCs and coCHCs.
We next extend pCSP to pfwCSP by adding well-foundedness and functionness constraints.A pfwCSP (C, K) consists of a finite set C of pCSP-clauses over predicate variables and a kinding function K that maps each predicate variable X ∈ fpv (C) to its kind: any one of •, ⇓, or λ which respectively represent ordinary, wellfounded, and functional predicate variables.
We write ρ |= WF (X) if the interpretation ρ(X) of the predicate variable X is well-founded, that is, sort(X) = ( s, s) → for some s and there is no infinite sequence v 1 , v 2 , . . . of sequences v i of values of the sorts s such that ( v i , v i+1 ) ∈ ρ(X) for all i ≥ 1.We write ρ |= FN (X) if X is functional, that is, sort(X) = ( s, s) → for some s and s, and ρ |= ∀ x : s.(∃y : s.X( x, y))∧∀y 1 , y 2 : s.(X( x, y 1 )∧ X( x, y 2 ) ⇒ y 1 = y 2 ) holds.We call a predicate interpretation ρ a semantic solution for (C, K) if ρ is a semantic solution of C, ρ |= WF (X) for all X such that K(X) =⇓, and ρ |= FN (X) for all X such that K(X) = λ.The notion of syntactic solution can be similarly generalized to pfwCSP.
Definition 1 (Satisfiability of pfwCSP).The predicate satisfiability problem of a pfwCSP (C, K) is that of deciding whether it has a semantic solution.
Remark 2. Recall that we assume that the T -formulas ϕ in pCSP clauses do not contain quantifiers.The assumption, however, is not a restriction for pfwCSP because we can Skolemize quantifiers using functional predicates.

Relational Verification with Constraints
We now present reductions from relational verification problems to pfwCSP, thus enabling a new route to automation of these problems.We begin with k-safety, and then move toward liveness and non-determinism, which are thorny problems in the relational setting.We first provide some basic definitions and notations.
Programs.We consider programs P 1 ,. . .,P k on variables V 1 ,. . ., V k , respectively.A state of the program P i is a valuation of the variables V i .We represent such a valuation by a sequence of values v such that | v| = | V i |.We assume that each P i is defined by the predicate T i ( V i , V i ) denoting its one-step transition relation i.e., T i ( v, v ) implies that evaluating P i one step from the state v reaches the state v .We also assume that there is a predicate F i ( V i ) that represents the final states of the program such that F i ( v) and T i ( v, v ) implies v = v , i.e., the program self-loops when it reaches a final state.We say that a state v (multistep) reaches a final state v in the evaluation of P i , written v i v , if there exists a non-empty finite sequence of states π such that π ) for all 1 < j ≤ |π|, and F i ( v ).We write v i ⊥ if there exists a non-terminating evaluation from v in P i , i.e., if there exists an infinite sequence of states such that ) for all 1 < j, and ¬F i ( [j]) for all 0 < j.We note that a program may be non-deterministic, that is, T i ( v, v ) and T i ( v, v ) may both be true for some v = v .

k-Safety
A k-safety property is given by predicates Pre( V ) and Post( V ) that respectively denote the pre and the post relations across the k-tuple.
Definition 2 (k-safety).The k-safety property verification problem is to decide if the following holds: That is, any k-tuple of final states reachable from a k-tuple of states satisfying the precondition satisfies the post condition.For instance, the TI-NI verification from Sec. 2.1 is a 2-safety property where P 1 and P 2 are copies of the same program, Pre states that the low inputs of the two programs are equal (i.e., x 1 = x 2 in the example), and Post states that the low outputs of the two programs are equal (i.e., y 1 = y 2 in the example).We now describe a new way to pose the k-safety relational verification problem via constraints written in pfwCSP.We write [k] for the set {1, . . ., k}.We define corresponding to the variables of the k programs.
Definition 3 (k-safety through constraints).We define pfwCSP constraints C S be the set of following clauses: Here, inv and sch A (for each A ∈ P + [k]) are ordinary predicate variables.Roughly, the predicate variables sch A describe a scheduler.The scheduler stipulates that when sch A ( v 1 , . . ., v k ) is true, each P i such that i ∈ A takes a step from the state v i while the others remain still.Note that the scheduler is semantic in the sense that which programs are scheduled to be executed next can depend on the current states of the programs.Clauses (1)-( 3) assert that inv is an invariant sufficient to prove the given safety property with the scheduler defined by sch A 's. Clauses (4) say that if an inv-satisfying state is such that the processes in A are allowed to move and some program has not yet terminated, then at least one process in A has not yet terminated.Clause (5) says that any state satisfying inv has to satisfy some sch A .Clauses ( 4) and ( 5) ensure the fairness of the scheduler, that is, at least one unfinished program is scheduled to make progress if there is an unfinished program.
Theorem 1 (Soundness and Completeness of C S ).The given k-tuple of programs satisfies the given k-safety property iff C S is satisfiable.
We note that the soundness direction crucially relies on scheduler fairness.The completeness is with respect to semantic solutions (cf.Def. 1) and it is only "relative" with respect to syntactic solutions: a syntactic solution only exists when the predicates of the background theory are able to express sufficient invariants and schedulers (impossible in general for any decidable theory when the class of programs is Turing-powerful as in our case when the background theory of predicates is QFLIA).
It is important to note that C S is not CHCs because clause (5) has a head disjunction.C S may be seen as a constraint-based formulation of the approach proposed in [50].However, their approach requires the user to provide sufficient predicates manually and is not fully automated, while our approach can fully automatically solve the problems by constraint solving (cf.Sec. 5).
Example 1.The formalization allows flexible scheduling.For instance, for the TI-NI example from Sec. 2.1, our approach is able to infer the predicate substitution that maps sch {1} , sch {2} , and sch {1,2} to λ V .h respectively, where V is the list of the variables in the two program copies.The inferred predicates stipulate that the copy with h = true is scheduled to execute the loop two times per every loop iteration of the copy with h = false.The extended report [58] shows the pfwCSP encoding of the example.A solution generated by PCSat is also shown in [58].

Co-termination
Intuitively, co-termination means that if one program terminates, then a second program must terminate [6,10].This can also be thought of as a form of relational termination problem. 6efinition 4 (Co-Termination).The co-termination verification problem is to decide if for all v 1 , v 2 such that Pre( Roughly, the property says that from any pair of states related by Pre, if P 1 terminates, then P 2 must also terminate.Note that this is an asymmetric property.A symmetric version can be obtained by also asserting the property with the positions of the two programs exchanged.The symmetric version implies, assuming that there is at least one execution from any Pre-related state, that from any pair of Pre-related states, all executions from one state terminates iff all executions from the other one do as well.We now present an encoding of conditional co-termination in pfwCSP. Definition 5 (Co-termination through constraints).
We define pfwCSP constraints C CoT be the set of following clauses: Here, sch TT , sch FT , and sch TF are 2-specialization of the k-safety scheduler of Def. 3. Clauses (3x)'s are similar to (3) of Def. 3 and assert that inv is an invariant under the scheduler.Clauses (4x)'s and (5), like (4) and (5) of Def. 3, are used to ensure the scheduler fairness.However, they are insufficient for cotermination as a non-terminating copy can be scheduled indefinitely leaving the other copy unscheduled.Clauses (1) and ( 2) are added to amend the issue.In (1), fnb is a functional predicate variable that is used to select a bound b, and (2) asserts that the difference d between the numbers of steps taken by the two copies is within b in any state in inv when neither copy has terminated.Note that d is initialized to 0 by (1) and properly updated in (3x)'s.Finally, by using the well-founded predicate variable wfr, (6) asserts that if P 1 has terminated, then so must eventually P 2 .
Theorem 2 (Soundness and Completeness of C CoT ).The given pair of programs co-terminate iff C CoT is satisfiable.
As with Theorem 1, the soundness direction relies on scheduler fairness.
Example 2. Via the encoding, our PCSat tool is able to verify the symmetric co-termination example from Sec. 2.1 by automatically inferring the solution described there.For space, the concrete constraint set and solution are given in the extended report [58].

Generalized Non-Interference
We now turn to another relational property that cannot simply be captured by k-safety or co-termination.So-called termination-insensitive (resp.-sensitive) generalized non-interference (resp.TI-GNI, TS-GNI) are ∀∃ hyperproperties: from any pre-related pair of states whenever one side can take a move to a post state, there must be a way for the other side to also move to a post state such that the post-relation holds.As remarked in Sec. 2, verifying GNI requires reasoning about both demonic (i.e., for all) and angelic (i.e., exists) non-determinism.

Definition 6 (TI/TS-GNI).
The GNI verification problem is to decide if the following holds.If Note that our definition is parameterized by Pre and Post.The standard GNI definitions [40] can be obtained by letting P 1 and P 2 be copies of the same target program and letting Pre be the predicate equating the low inputs of the copies and Post be the predicate equating the low outputs of the copies.
To formalize the pfwCSP encodings of the GNI verification problems, we define a relation U 2 to be one such that

a function version of the transition relation
T 2 with the extra parameter r to make the non-deterministic choices explicit.
We now show the pfwCSP encodings of TI-GNI and TS-GNI.The key idea is to augment the encodings for k-safety and/or co-termination with functional predicate variables and prophecy variables that respectively represent the nondeterministic choices of the angelic side (i.e., P 2 ) and the final outputs of the demonic side (i.e., P 1 ).
Definition 7 (TI-GNI through constraints).We define pfwCSP constraints C TIGNI as C S in Def. 3 for k = 2 but with the following modifications: (m1) The parameters representing the inputs and outputs of P 1 is extended with prophecy variables p where i.e., the prophecy values are unconstrained in the precondition.
i.e., if the prophecy was correct then the original post condition must hold.
where fnr is a functional predicate variable.
Modifications (m1)-(m5) concern prophecy variables.They are initialized arbitrarily as shown in (m2), propagated unmodified through the transitions as shown in (m4), and finally checked if they match P 1 's outputs in (m5).Modification (m6) adds functional predicate variables to express the angelic nondeterministic choices of P 2 .The functional predicate variables shift the onus of making the right choices to the solver's task of discovering sufficient assignments to them.Importantly, the functional predicate takes the prophecy variables as parameters, thus allowing dependence on the final outputs of the demonic side.
Definition 8 (TS-GNI through constraints).We define pfwCSP constraints C TSGNI as C CoT in Def. 5 but with modifications of Def. 7 except (m3) and (m5), and with the following modifications: C TSGNI is similar to C TIGNI except that it contains the difference bound and wellfoundedness constraints to handle the "co-termination" aspect of TS-GNI, i.e., if P 1 terminates and makes an output then P 2 must also be able terminate and make a matching output.One subtle aspect of the encoding is that (m3') modifies the final state predicate for P 1 to enforce co-termination only when the prophecy is correct.However, it is worth noting that TS-GNI is not a conjunction of TI-GNI and co-termination.For instance, the GNI example from Sec. 2.1 satisfies TS-GNI but does not satisfy co-termination.
Theorem 3 (Soundess and Completeness of of TI-GNI).The given pair of programs satisfy TI-GNI iff C TIGNI is satisfiable.
Theorem 4 (Soundess and Completeness of TS-GNI).The given pair of programs satisfy TS-GNI iff C TSGNI is satisfiable.
The soundness directions are proven by "determinizing" the angelic choices by solutions to the functional predicate variables and reducing the argument to those of k-safety and co-termination.The completeness directions are proven by "synthesizing" sufficient angelic choice functions from program executions.
Example 3. Via the encoding, our PCSat tool is able to verify the TS-GNI example from Sec. 2.1 by automatically inferring not only the functional predicate described there but also relational invariants and well-founded relations given in the extended report [58].For space, the concrete constraint set is also given in [58].
Remark 3. The angelic non-determinism encoding can be optimized by using head disjunctions when the non-determinism is finitary (i.e., max v |{ v | T 2 ( v, v )}| is finite) instead of using functional predicate variables.For this, we modify clauses (3) and (3x)'s of Def. 7 and 8 to contain multiple positive occurrences of inv where each occurrence represents one of the finitely many possible choices.
Remark 4. Recall that we allow any program to be non-deterministic.The ksafety and co-termination encodings treat non-determinism in all programs as demonic, whereas the GNI encodings treat those in one program (i.e., P 1 ) as demonic and those in the other program (i.e., P 2 ) as angelic.In general, an arbitrary program can be made angelic by applying the transformation done in the angelic side of GNI encodings (to factor out non-determinism).
We describe a CEGIS-based method for finding a (syntactic) solution of the given pfwCSP (C, K).Our method iterates the following phases until convergence.The iteration maintains and builds a sequence σ of candidate solutions and a sequence E of example instances where E (i) are ground clauses obtained from C by instantiating the term variables and serve as a counterexample to the candidate solution σ (i−1) , for each i-th iteration.The iteration starts from E (1) = ∅.
Synthesis Phase: We check if (E (i) , K) is unsatisfiable.If so, we stop by returning E (i) as a genuine counterexample to the input problem (C, K).Otherwise, we use the synthesizer S T B (cf. Sec.5.1) to find a solution σ (i) of (E (i) , K), which will be used as the next candidate solution.
Validation Phase: We check if σ (i) is a genuine solution to (C, K) by using an SMT solver.If so, we stop by returning σ (i) as a solution.Otherwise, for each clause c ∈ C not satisfied by σ (i) , we obtain a term substitution θ c such that dom(θ c ) = ftv (c) and |= θ c (σ (i) (c)).We then update the example set by adding a new example instance for each unsatisfied clause (i.e., , and proceed to the next iteration. The above procedure satisfies the usual progress property of CEGIS: discovered counterexamples and candidate solutions are not discovered again in succeeding iterations.Furthermore, as discussed in Sec.5.1, by carefully designing the synthesizer S T B by incorporating stratified CEGIS, we achieve completeness in the sense of [34,55]: if the given pfwCSP (C, K) has a syntactic solution expressible in the stratified families of templates, a solution of the pfwCSP is eventually found by the procedure.In Sec.5.1, we discuss the details of the synthesis phase.There, for space, we focus on the theory of quantifier-free linear integer arithmetic (QFLIA).For space, we defer the details of the unsatisfiability checking process to the extended report [58].
Remark 5.The implementation described in Sec.6 contains an additional phase called resolution phase for accelerating the convergence.There, we first apply unit propagation repeatedly to the given E (i) to obtain positive examples E (i)+ of the form X( v) and negative examples E (i)− of the form ¬X( v).We then repeatedly apply resolution principle to the clauses in the input clauses C and the clauses E (i)+ ∪ E (i)− to obtain additional positive and negative examples.

Predicate Synthesis with Stratified Families of Templates
We describe our candidate solution synthesizer S T B .S T B performs a templatebased search for a solution to the given example instances.As we shall show, our approach allows searching for assignments to all predicate variables (of all three kinds) in the given instance which is important because satisfying assignments to different predicate variables often inter-dependent.There, however, is a trade-off between expressiveness and generalizability.With less expressive templates like intervals, we may miss actual solutions.But with very expressive templates like polyhedra, there could be many solutions, and a solution thus returned is liable Stratified Template Family for Ordinary Predicate Variables: T • X (nd , nc, ac, ad ) λ(x1, . . ., x ar(X) ).nd i=1 nc j=1 ci,j,0 + ar(X) Stratified Template Family for Well-Founded Predicate Variables: T ⇓ X (np, nl , nc, rc, rd , dc, dd ) λ( x, y).np Stratified Template Family for Functional Predicate Variables: to overfitting, i.e., the solution to the example instances becomes too specific to be an actual solution to the original input clauses.[44] discusses a similar overfitting issue in the context of grammar-based synthesis.
Our remedy to the issue is stratified families of predicate templates, inspired by a similar approach proposed in the context of predicate abstraction with CEGAR [34,55].Initially, we assign each predicate variable a less expressive template and gradually refine it in a counterexample-guided manner: if no solution exists in the current template, we generate and analyze an unsat core to identify the parameters of the families of templates that should be updated.The stratification of templates thus automatically pushes the template to an expressive one (e.g., polyhedra) when it needs to.Importantly, with our approach, expressive templates are not always used but only when they should be used.

Stratified Families of Templates
We have designed three stratified families of templates shown in Fig. 2, respectively for ordinary (•), well-founded (⇓), and functional (λ) predicate variables.First, for each ordinary predicate variable X, we prepare the stratified family of templates T • X (nd , nc, ac, ad ) with unknowns c i,j,k 's to be inferred and its accompanying constraint φ • X (nd , nc, ac, ad ).The body of T • X is a DNF with affine inequalities as atoms.The parameter nd (resp.nc) is the number of disjuncts (resp.conjuncts).The parameter ac is the upper bound of the sum of the absolute values of coefficients c i,j,k (k > 0), and ad is the upper bound of the absolute value of c i,j,0 .
Secondly, for each functional predicate variable X, we prepare the stratified family of templates T ⇓ X (np, nl , nc, rc, rd , dc, dd ) with unknowns c i,j,k 's and c i,j,k 's and its accompanying constraint φ ⇓ X (np, nl , nc, rc, rd , dc, dd ).T ⇓ X represents the well-founded relation induced by a piecewise-defined lexicographic affine ranking function [2,39,39,60,61] where r i,j is the affine ranking function template for the j-th lexicographic component of the i-th region specified by the discriminator D i .The parameter np (resp.nl ) is the number of regions (resp.lexicographic components).The parameters rc, rd , dc, dd are the upper bounds of (the sums of) the absolute values of unknowns, similar to ac and ad of T • X .The first conjunct of T ⇓ X asserts that the return value of each ranking functions is non-negative.The second and the third conjuncts assert that the discriminators cover all relevant states.Note that discriminators may overlap, and for such overlapping regions, the maximum return value of the ranking functions is used.The fourth conjunct asserts that the return value of the piecewise-defined ranking function strictly decreases from x to y.Here, DEC i,j ( x, y) asserts that the return value of the lexicographic ranking function for the i-th region at x is greater than that for the j-th region at y.It follows that for any substitution θ for the unknowns in T ⇓ X , θ(T ⇓ X ) represents a well-founded relation.Our implementation PCSat uses a refined version of T ⇓ X shown in the extended report [58].Finally, for each functional predicate variable X, we prepare the stratified family of templates T λ X (nd , nc, dc, dd , ec, ed ) with unknowns c i,j 's and c i,j,k 's and its accompanying constraint φ λ F (nd , nc, dc, dd , ec, ed ).T λ X characterizes a piecewise-defined affine function with discriminators D 1 , . . ., D nd−1 and branch expressions e 1 , . . ., e nd .The parameter nc is the number of conjuncts in each discriminator.The parameters dc, dd , ec, ed are the upper bounds of (the sums of) the absolute values of unknown, similar to ac and ad of T • X .Note that for any substitution θ for the unknowns in T λ X , θ(T λ X )( x, r) expresses a total function that maps x to r.
Next, we give the details of the candidate solution synthesis process.Let p ∈ Z n where n is the number of parameters summed across all templates, and let T α X ( p) and φ α X ( p) (for α ∈ {•, ⇓, λ}) project the corresponding parameters.Each p ∈ Z n induces a solution space p ( p).Let p 1 ≤ p 2 be the point-wise ordering.Note that p is a finite set for any p ∈ Z n , and p 1 ≤ p 2 implies p 1 ⊆ p 2 .We start the CEGIS process with some small initial parameters p (0) (the parameters will be maintained as a state of the CEGIS process).The synthesis phase of each iteration tries to find a solution θ ∈ p (i) to the given example instances (E, K) where p (i) are the current parameters.This is done by using an SMT solver for QFLIA to find θ satisfying T ( p (i) )[θ](E) ∧ θ(Con( p (i) )).If such θ is found, we return T ( p (i) )[θ] as the candidate solution for the next validation phase of the CEGIS process.Note that, by construction of the templates, the solution is guaranteed to assign each well-founded (resp.functional) predicate variable a well-founded relation (resp.total function).Otherwise, no solutions to the given example instances (E, K) can be found in p (i) , and we update the parameters to some p (i+1) > p (i) such that p (i+1) contains a solution for (E, K).Here, it is important to do the update in a fair manner [34,55], that is, in any infinite series of updates p (0) , p (1) , . . ., every parameter is updated infinitely often (the details are deferred to below).By the progress property and the fact that every p is finite, this ensures that every parameter is updated infinitely often in an infinite series of CEGIS iterations.We thus obtain the following property.
Theorem 5. Our CEGIS-procedure based on stratified families of templates is complete in the sense of [34,55]: if there is p and σ ∈ p such that σ is a syntactic solution to the given pfwCSP (C, K), a syntactic solution to (C, K) is eventually found by the procedure.
Note that, while the solution space of each stratum (i.e., p (i) ) is finite, our procedure searches the infinite solution space obtained by taking the infinite union of the solution spaces of the template family strata (i.e., i∈ω p (i) ).Remark 6.Our template-based synthesis simultaneously finds ordinary, wellfounded, and functional predicates that are mutually dependent through the given (E, K).This means that templates for different kinds of predicate variables are updated in a synchronized and balanced manner, which benefits the synthesis of mutually dependent witnesses for a relational property (see the extended report [58] for examples).
Updating Parameters of Template Families via Unsat Cores.We now describe the parameter update process.We first obtain the unsat core of the unsatisfiability of T ( p (i) )[θ](E) ∧ θ(Con( p (i) )) from the SMT solver.We then analyze the core to obtain the parameters of template families, such as the number of conjuncts and disjuncts, that have caused the unsatisfiability.Here, there could be a dependency between predicate variables and in such a case our unsat core analysis enumerates all the involved predicate variables from which we obtain the parameters of template families to be updated.We then increment these parameters in some fair manner, by limiting the maximum differences between different parameters to some bounded threshold, and repeatedly solve the resulting constraint until a solution is found.Thus, the parameters of stratified families of templates are grown on-the-fly guided by the reasons for unsatisfiability.We found that a careful design of parameter update strategies important for scaling the stratified CEGIS to hard relational verification problems.The manual tuning, however, is tiresome and suboptimal.We leave as future work to investigate methods for automatic tuning of parameter update strategies.

Evaluation
To evaluate the presented verification framework, we have implemented PCSat, a satisfiability checking tool for pfwCSP based on stratified CEGIS.PCSat supports the theory of Booleans and the quantifier-free theory of linear inequalities over integers and rationals.The tool is implemented in OCaml, using Z3 [41] as the backend SMT solver.We ran the tool on a diverse collection of 20 relational verification problems, consisting of k-safety, co-termination, and GNI problems.Though we have manually reduced them to pfwCSP using the presented method in Sec. 4, this process can be easily automated.The full benchmark set is provided in the extended report [58].All experiments have been conducted on 3.1GHz Intel Xeon Platinum 8000 CPU and 32 GiB RAM with the time limit of 600 seconds.
The experimental results are summarized in Table 1.The columns "Time (s)" and "#Iters" respectively show elapsed wall clock time in seconds and numbers of CEGIS iterations.PCSat solved 15 verification problems fully automatically and 5 problems labeled with the symbol † and/or ‡ semi-automatically.For the 4 problems labeled with †, we manually provided small hints for invariant synthesis (interested readers are referred to [58]).The provided hints for all but SquareSum are non-relational invariants that can be inferred prior to relational verification by using a CHCs solver or an invariant synthesizer.For the 2 problems labeled with ‡, we manually chose the initial value for the parameters of the template family for ordinary predicate variables to reduce the elapsed time.This can be automated by running PCSat with different initial values in parallel.
The problems DoubleSquareNI _ h ** , HalfSquareNI, ArrayInsert, and SquareSum are k-safety verification problems obtained from [50] that require non-lock-step scheduling. 7The problems DoubleSquareNI _ h ** are generated from Example 1 by a case analysis of the valuation for the boolean variables h 1 and h 2 .PC-Sat solved all the k-safety problems but SquareSum fully automatically.The tool Pdsc proposed in [50] can verify them but requires the user to provide the atomic predicates for expressing relational invariants and schedulers.The problems CotermIntro1 and CotermIntro2 are asymmetric co-termination problems obtained from the symmetric problem in Example 2 and are verified by PCSat fully automatically.The problems TS _ GNI _ h ** are generated from Example 3 by a case analysis and are verified by PCSat with small non-relational hints.We have also tested PCSat on various TS-GNI (SimpleTS _ GNI1, SimpleTS _ GNI2, InfBranchTS _ GNI) and TI-GNI problems (TI _ GNI _ h ** ) and obtained promising results.As far as we know, no existing tools can verify these non-k-safety relational problems.
Furthermore, manual inspection of the PCSat's output logs for the GNI problems that required hints revealed that the functional predicate synthesis appears to be the main bottleneck of the current version.In fact, we confirmed that the problems can be solved in less than 10 seconds if appropriate functional predicates for angelic non-determinism are manually provided.As future work, we plan to investigate methods for improved functional predicate synthesis.
7 Related Work

Relational Verification
There has been substantial work on verifying relational properties.They include program logics, type systems, or program analysis frameworks such as abstract  [3,18,23,46,59].We refer to [43] for an excellent survey.However, most prior automatic approaches address only the k-safety fragment [17,54] and cannot verify non-k-safety (actually, not even hypersafety) properties such as co-termination, TS-NI, TI-GNI, and TS-GNI [6,11,40].The only exception that we are aware is the recent work by Coenen et al. [19] that proposes a sound method for automatically verifying ∀∃ hyperproperties such as GNI for finite state systems.To our knowledge, we are the first to propose a sound-and-complete approach to automatically verifying these non-hypersafety properties for infinite state programs. 8 key task in many relational verification methods, including ours, is the discovery of relational invariants which relate the states of multiple program executions.While most prior approaches are limited to fixed execution schedule (or alignment) such as lock-step and sequential [7,8,20,21,42,54,57], a recent work by Shemer et al. [50] has proposed a k-safety property verification method that automatically infers fair schedulers sufficient to prove the goal property.Importantly, the schedulers in their approach can be semantic in which the choice of which program to execute can depend on the states of the programs as opposed to the classic syntactic schedulers such as lock-step and sequential that can only depend on the control locations.Our approach also infers such fair semantic schedulers, and as remarked before, they enable solving instances like doubleSquare that are difficult for previous approaches.However, [50] requires the user to provide appropriate atomic predicates and is not fully automatic.By contrast, our approach soundly and completely encodes the problem as a constraint satisfaction problem and fully automatically verifies hard instances like doubleSquare by constraint solving.
Furthermore, our work extends the fair semantic scheduler synthesis to beyond k-safety problems like co-termination, TI-GNI and TS-GNI, in a sound and complete manner.We note that the extensions are non-trivial and involves delicate uses of functional predicate variables and well-founded predicate variables to ensure scheduler fairness in the presence of non-termination as well as uses of prophecy variables and functional predicate variables to handle angelic non-determinism.The higher-degree of automation and the extension to nonk-safety properties are thanks to the expressive power of our novel constraint framework pfwCSP.

Predicate Constraint Solving
Our pfwCSP solving technique builds on and generalizes a number of techniques developed for CHCs solving as well as invariant and ranking function discovery.Most closely related to our constraint solving method are CEGIS-based [51] and data-driven approaches to solving CHCs [14,22,24,26,27,38,44,45,48,49,65].As remarked before, the new pfwCSP framework is strictly more expressive than CHCs and extending the prior techniques to the new framework is non-trivial.
Our stratified CEGIS is inspired by the idea of stratified languages of predicates proposed in the context of predicate abstraction with CEGAR [34,55].It is also similar in spirit to the work by Padhi et al. [44], but they use a stratified family of grammars.Also none of these prior works use unsat cores for updating the language/grammar stratum, synthesize well-founded relations and functional predicates, or support non-Horn clauses.
Our class of pfwCSP constraints is related to existentially-quantified Horn clauses (E-CHCs) introduced by Beyene et al. [12].E-CHCs does not have non-Horn clauses or functional predicate variables.However, it has disjunctive well-foundedness constraints which are similar to our well-founded predicate variables.Also, existential quantifiers can be used to encode head disjunctions and functional predicates.We conjecture that pfwCSP and E-CHCs are interreducible, but it is not trivial to fill the gap.Also, inter-reducibility is a desirable feature: different formats may have different benefits.For relational verification, as we have shown, pfwCSP enables direct sound-and-complete encodings of the problems.For instance, head disjunctions allow direct encoding of scheduler fairness and finitary angelic non-determinism (cf.Remark 3).And, functional predicate variables can be explicitly given necessary-and-sufficient parameters to encode angelic non-determinism and difference bounds for ensuring scheduler fairness in the presence of non-termination.The tight encodings also lead to reduction in search space and benefited the constraint solving.

Conclusion
We have introduced the class pfwCSP of predicate constraint satisfaction problems that generalizes CHCs with arbitrary clauses, well-foundedness constraints, and functionality constraints.We have then established a program verification framework based on pfwCSP by showing that (1) pfwCSP can soundly-andcompletely encode various classes of relational problems of infinite-state nondeterministic programs, including hard instances of k-safety, co-termination, and termination-sensitive generalized non-interference that benefit from statedependent scheduling/alignment (Theorems 1-4), and (2) existing CHCs solving and invariants/ranking function synthesis techniques can be adopted to pfwCSP solving and further improved with the idea of stratified CEGIS for simultaneously achieving completeness (Theorem 5) and practical effectiveness.
In future work we plan to investigate ways to improve functional predicate synthesis, automatic tuning of parameter update strategies for constraint solving, and whether a constraint-based approach (and the techniques presented in the present paper) can be extended to reason about relational temporal properties such as the ones expressed in hyper temporal logics [16,25].

Fig. 1 .
Fig. 1.Overview of the contributions and how they achieve a constraint-based strategy for relational verification.

Table 1 .
Experimental results of PCSat on the relational verification benchmarks