Verification of Threshold-Based Distributed Algorithms by Decomposition to Decidable Logics

Verification of fault-tolerant distributed protocols is an immensely difficult task. Often, in these protocols, thresholds on set cardinalities are used both in the process code and in its correctness proof, e.g., a process can perform an action only if it has received an acknowledgment from at least half of its peers. Verification of threshold-based protocols is extremely challenging as it involves two kinds of reasoning: first-order reasoning about the unbounded state of the protocol, together with reasoning about sets and cardinalities. In this work, we develop a new methodology for decomposing the verification task of such protocols into two decidable logics: EPR and BAPA. Our key insight is that such protocols use thresholds in a restricted way as a means to obtain certain properties of"intersection"between sets. We define a language for expressing such properties, and present two translations: to EPR and BAPA. The EPR translation allows verifying the protocol while assuming these properties, and the BAPA translation allows verifying the correctness of the properties. We further develop an algorithm for automatically generating the properties needed for verifying a given protocol, facilitating fully automated deductive verification. Using this technique we have verified several challenging protocols, including Byzantine one-step consensus, hybrid reliable broadcast and fast Byzantine Paxos.


Introduction
Fault-tolerant distributed protocols play an important role in the avionic and automotive industries, medical devices, cloud systems, blockchains, etc. Their unexpected behavior might put human lives at risk or cause a huge financial loss. Therefore, their correctness is of ultimate importance.
Ensuring correctness of distributed protocols is a notoriously difficult task, due to the unbounded number of processes and messages, as well as the non-deterministic behavior caused by the presence of faults, concurrency, and message delays. In general, the problem of verifying such protocols is undecidable. This imposes two directions for attacking the problem: (i) developing fully-automatic verification techniques for restricted classes of protocols, or (ii) designing deductive techniques for a wide range of systems that require user assistance. Within the latter approach, recently emerging techniques [28] leverage decidable logics that are supported by mature automated solvers to significantly reduce user effort, and increase verification productivity. Such logics bring several key benefits: (i) their solvers usually enjoy stable performance, and (ii) whenever annotations provided by the user are incorrect, the automated solvers can provide a counterexample for the user to examine.
Deductive verification based on decidable logic requires a logical formalism that satisfies two conflicting criteria: the formalism should be expressive enough to capture the protocol, its correctness properties, its inductive invariants, and ultimately its verification conditions. At the same time, the formalism should be decidable and have an effective automated tool for checking verification conditions.
In this paper we develop a methodology for deductive verification of thresholdbased distributed protocols using decidable logic, where we use decomposition into two well-established decidable logics to settle the tension explained above.
In threshold-based protocols, a process may take different actions based on the number of processes from which it received certain messages. This is often used to achieve fault-tolerance. For example, a process may take a certain step once it has received an acknowledgment from a strict majority of its peers, that is, from more than n/2 processes, where n is the total number of processes. Such expressions as n/2, are called thresholds, and in general they can depend on additional parameters, such as the maximal number of crashed processes, or the maximal number of Byzantine processes.
Verification of such protocols requires two flavors of reasoning, as demonstrated by the following example. Consider the Paxos [19] protocol, in which each process proposes a value and all must agree on a common proposal. The protocol tolerates up to t process crashes, and ensures that every two processes that decide agree on the decided value. The protocol requires n > 2t processes, and each process must obtain confirmation messages from n − t processes before making a decision. The protocol is correct due to, among others, the fact that if n > 2t then any two sets of n − t processes have a process in common. To verify this protocol we need to express (i) relationships between an unbounded number of processes and values, which typically requires quantification over uninterpreted domains ("every two processes"), and (ii) properties of sets of certain cardinalities ("any two sets of n−t processes intersect"). Crucially, these two types of reasoning are intertwined, as the sets of processes for which we need to capture cardinalities may be defined by their relations with other state components ("messages from at least n − t processes"). While uninterpreted first-order logic (FOL) seems like the natural fit for the first type of reasoning, it is seemingly a poor fit for the second type, since it cannot express set cardinalities and the arithmetic used to define thresholds. Typically, logics that combine both types of reasoning are either undecidable or not flexible enough to capture protocols as intricate as the ones we consider.
The approach we present relies on the observation that threshold-based protocols and their correctness proofs use set cardinality thresholds in a restricted way as a means to obtain certain properties between sets, and that these properties can be expressed in FOL via a suitable encoding. In the example above, the important property is that every two sets of cardinality at least n − t have a non-empty intersection. This property can be encoded in FOL by modeling sets of cardinality at least n − t using an uninterpreted sort along with a corresponding membership relation between this sort and the sort for processes. However, the validity of the intersection property under the assumption that n > 2t cannot be verified in FOL.
The key idea of this paper is, hence, to decompose the verification problem of threshold-based protocols into the following problems: (i) Checking protocol correctness assuming certain intersection properties, which can be reduced to verification conditions expressed in the Effectively Propositional (EPR) fragment of FOL [24,34].
(ii) Checking that sets with cardinalities adhering to the thresholds satisfy the intersection properties (under the protocol assumptions), which can be reduced to validity checks in quantifier-free Boolean Algebra with Presburger Arithmetic (BAPA) [18]. Both BAPA and EPR are decidable logics, and are supported by mature automated solvers.
A crucial step in employing this decomposition is finding suitable intersection properties that are strong enough to imply the protocol's correctness (i.e., imply the FOL verification conditions), and are also implied by the precise definitions of the thresholds and the protocol's assumptions. Thus, these intersection properties can be viewed as interpolants between the FOL verification conditions and the thresholds in the context of the protocol's assumptions. We present fully automated procedures to find such intersection property interpolants, either eagerly or lazily.
The main contributions of this paper are: 1. We define a threshold intersection property (TIP) language for expressing properties of sets whose cardinalities adhere to certain thresholds; TIP is expressive enough to capture the properties required to prove the correctness of challenging threshold-based protocols. 2. We develop two encodings of TIP, one in BAPA, and another in EPR. These encodings facilitate decomposition of protocol verification into decidable EPR and (quantifier-free) BAPA queries. 3. We show that there are only finitely many TIP formulas (up to equivalence) that are valid for any given protocol. Moreover, we present an effective algorithm for computing all TIP formulas valid for a given protocol, as well as an algorithm for lazily finding a set of TIP formulas that suffice to prove a given protocol. 4. Put together, we obtain an effective deductive verification approach for thresholdbased protocols: the user models the protocol and its inductive invariants in EPR using a suitable encoding of thresholds, and defines the thresholds and the protocol's assumptions using arithmetic; verification is carried out automatically via decomposition to well-established decidable logics. 5. We implement the approach, leveraging mature existing solvers (Z3 and CVC4), and evaluate it by verifying several challenging threshold-based protocols with sophisticated thresholds and assumptions. Our evaluation shows the effectiveness and flexibility of our approach in modeling and verifying complex protocols, including the feasibility of automatically inferring threshold intersection properties.

Preliminaries
In this section we present the necessary background on the formalization of transition systems using FOL, as well as on the decidable logics used in our work: EPR and BAPA.
Transition Systems in FOL We model distributed protocols as transition systems expressed in many-sorted FOL. A state of the system is a first-order (FO) structure s = (D, I) over a vocabulary Σ that consists of sorted constant, function and relation symbols, s.t. s satisfies a finite set of axioms Θ in the form of closed formulas over Σ. D is the domain of s mapping each sort to a set of objects (elements), and I is the interpretation function. A FO transition system is a tuple (Σ, Θ, I, TR), where Σ and Θ are as above, I is the initial condition given by a closed formula over Σ, and TR is the transition relation given by a closed formula over Σ Σ where Σ describes the source state of the transition and Σ = {a | a ∈ Σ} describes the target state. We require that TR does not modify any symbol that appears in Θ. The set of initial states and the set of transitions of the system consist of the states, respectively, pairs of states, that satisfy I, respectively, TR. The set of reachable states is defined as usual. In practice, we define FO transition systems using a modeling language with a convenient syntax [28].
Properties and inductive invariants. A safety property is expressed by a closed FO formula P over Σ. The system is safe if all of its reachable states satisfy P . A closed FO formula Inv over Σ is an inductive invariant for a transition system (Σ, Θ, I, TR) and property P if the following formulas, called the verification conditions, are valid (equivalently, their negations are unsatisfiable): where Inv results from substituting every symbol in Inv by its primed version. Requirements (i) and (ii) ensure that Inv represents a superset of the reachable states, hence together with (iii) safety follows. We also use inductive invariants to verify arbitrary first-order LTL formulas via the reduction of [29,30].
Effectively Propositional Logic (EPR) The effectively-propositional (EPR) fragment of FOL is restricted to formulas without function symbols and with a quantifier prefix ∃ * ∀ * in prenex normal form. Satisfiability of EPR formulas is decidable [24]. Moreover, EPR formulas enjoy the finite model property, meaning that a satisfiable formula is guaranteed to have a finite model. The size of this model is bounded by the total number of existential quantifiers and constants in the formula. While EPR does not allow any function symbols nor quantifier alternation except ∃ * ∀ * , we consider a straightforward extension of EPR that maintains these properties and is supported by mature solvers such as Z3 [4]. The extension allows function symbols and quantifier alternations as long as the formula's quantifier alternation graph, denoted QA(ϕ), is acyclic. For ϕ in negation normal form, QA(ϕ) is a directed graph where the set of vertices is the set of sorts and the set of edges is defined as follows: -Function edges: let f be a function in ϕ from sorts s 1 , . . . , s k to sort s. Then there is a ∀∃ edge from s i to s for every 1 ≤ i ≤ k. -Quantifier edges: let ∃x : s be an existential quantifier that resides in the scope of the universal quantifiers ∀x 1 : s 1 , . . . , ∀x k : s k in ϕ. Then there is a ∀∃ edge from s i to s for every 1 ≤ i ≤ k. The quantifier alternation graph is extended to sets of formulas as expected.
Boolean Algebra with Presburger Arithmetic (BAPA) Boolean Algebra with Presburger Arithmetic (BAPA) [18] is a FO theory defined over two sorts: int, representing integers, and set, representing subsets of a finite universe. The language is defined by the following grammar: where L defines linear integer terms, where u denotes an integer variable, k ∈ K defines an (interpreted) integer constant symbol . . . , −2, −1, 0, 1, 2 . . ., n is an integer constant symbol that represents the size of the finite set universe, i is an uninterpreted integer constant symbol (as opposed to the constant symbols from K), and |b| denotes set cardinality; B defines set terms, where x denotes a set variable, ∅ is a (interpreted) set constant symbol that represents the empty set, and a is an uninterpreted set constant symbol; and F defines the set of BAPA formulas, where 1 = 2 and 1 < 2 are atomic arithmetic formulas and b 1 = b 2 is an atomic set formula. (Other set constraints such as b 1 ⊆ b 2 can be encoded in the usual way). Note that BAPA formulas are closed under Boolean operations and allow quantification over integer variables (u), and over set variables (x). In the sequel, we also allow arithmetic terms of the form k where k ∈ K is a positive integer and ∈ L, as any formula that contains such terms can be translated to an equivalent BAPA formula by multiplying by k.
A BAPA structure s B = (D, I) consists of a domain D mapping sort int to the set of all integers and mapping the sort set to the set of all subsets of a finite universe U , called the universal set, and an interpretation function I of the symbols for integer and set operations. The semantics of terms and formulas is as expected. The interpretation of the complement operation is defined with respect to U . In particular, this means that I(∅ c ) = U . The integer constant n is interpreted to the size of U , i.e. I(n) = |U |. The uninterpreted constant symbols may be interpreted arbitrarily.
Both validity and satisfiability of BAPA formulas (with arbitrary quantification) are decidable [18], and the quantifier-free fragment is supported by CVC4 [2], a mature SMT solver.

First-Order Modeling of Threshold-Based Protocols
Next we explain our modeling of threshold-based protocols as transition systems in FOL (Note that FOL cannot directly express set cardinality constraints). The idea is to capture each threshold by a designated sort, such that elements of this sort represent sets of nodes that satisfy the threshold. Elements of the threshold sort are then used instead of the actual threshold in the description of the protocol and in the verification conditions. For verification to succeed, some properties of the sets satisfying the cardinality threshold must be captured in FOL. This is done by introducing additional assumptions (formally, axioms of the transition system) expressed in FOL, as discussed in Sec. 4.
Running Example. We illustrate our approach using the example of Bosco-an asynchronous Byzantine fault-tolerant (BFT) consensus algorithm [38]. Its modeling in firstorder logic using our technique appears alongside an informal pseudo-code in  Bosco: a one-step asynchronous Byzantine consensus algorithm [38], and an excerpt RML (relational modeling language) code of the main transition. Note that we overload the member relation for all threshold sorts. The formula In the BFT consensus problem, each node proposes a value and correct nodes must decide on a unique proposal. BFT consensus algorithms typically require at least two communication rounds to reach a decision. In Bosco, nodes execute a preliminary communication step which, under favorable conditions, reaches an early decision, and then call an underlying BFT consensus algorithm to ensure reaching a decision even if these conditions are not met. Bosco is safe when n > 3t; it guarantees that a preliminary decision will be reached if all nodes are non-faulty and propose the same value when n > 5t (weakly one-step condition), and even if some nodes are faulty, as long as all non-faulty nodes propose the same value, when n > 7t (strongly one-step condition).
Bosco achieves consensus by ensuring that (a) no two correct nodes decide differently in the preliminary step, and (b) if a correct node decides value v in the preliminary step then every correct process calls the underlying BFT consensus algorithm with proposal v. Property (a) is ensured by the fact that a node decides in the preliminary step only if more than n+3t 2 nodes proposed the same value. When n > 3t, two sets of cardinality greater than n+3t 2 have at least one non-faulty node in common, and therefore no two different values can be proposed by more than n+3t 2 nodes. Similarly, we can derive property (b) from the fact that a set of more than n+3t 2 nodes and a set of n − t nodes intersect in n+t 2 nodes, which, after removing t nodes which may be faulty, still leaves us with more than n−t 2 nodes, satisfying the condition in line 9.

Threshold-based protocols
Parameters and resilience conditions. We consider protocols whose definitions depend on a set of parameters, Prm, divided into integer parameters, Prm I , and set parameters, Prm S . Prm I always includes n, used to represent the total number of nodes (assumed to be finite). Protocol correctness is ensured under a set of assumptions Γ called resilience conditions, formulated as BAPA formulas over Prm (this means that all the uninterpreted constants appearing in Γ are from Prm). In Bosco, Prm I = {n, t}, where t is the maximal number of Byzantine failures tolerated by the algorithm, and Threshold conditions. Both the description of the protocol and the inductive invariant may include conditions that require the size of some set of nodes to be "at least t", "at most t", and so on, where the threshold t is of the form t = k , where k is a positive integer, and is a ground BAPA integer term over Prm (we restrict k to ensure that the guard can be translated to BAPA). Comparing sizes of two sets is not allowedwe observe that it is not needed for threshold-based protocols. We denote the set of thresholds by T . For example, in Bosco, }. Without loss of generality, we assume that all conditions on set cardinalities are of the form "at least t". This is because every condition can be written in this form, possibly by introducing new threshold expressions and complementing the set which the condition refers to, according to the following rules:

Modeling in FOL
FO vocabulary for modeling threshold-based protocols. We describe the protocol's states (e.g., pending messages, votes, etc.) using a core FO vocabulary Σ C that includes sort node and additional sorts and symbols. Parameters Prm are not part of the FO vocabulary used to model the protocol. Also, we do not model set cardinality directly. Instead, we encode the cardinality thresholds in FOL by defining a FO vocabulary Σ Prm T : -For every threshold t we introduce a threshold sort set t with the intended meaning that elements of this sort are sets of nodes whose size is at least t. -As the threshold sorts represent sets, each sort set t is equipped with a binary relation symbol member t between sort node and sort set t that captures the membership relation of a node (first argument) in a set (second argument). -For each set parameter a ∈ Prm S we introduce a unary relation symbol member a over sort node that captures membership of a node in the set a. We then model the protocol as a transition system (Σ, Θ, I, TR) where Σ = Σ C Σ Prm T . We are interested only in states (FO structures over Σ) where the interpretation of the threshold sorts and membership relations is according to their intended meaning in a corresponding BAPA structure. The intended meaning is captured by a BAPA structure (over Prm) that satisfies the resilience condition, and whose universal set coincides with the set of nodes. Formally, these are T -extensions, defined as follows: where P is the powerset operator. For such compatible structures and a set of thresholds T over Prm, the T -extension of s C by s B is the structure s = (D, I) over Σ defined as follows: Note that for the T -extension s to be well defined as a FO structure, we must have that D(set t ) = ∅ for every threshold t ∈ T . This means that a T -extension by s B only exists if {A ⊆ D(node) | |A| ≥ I B (t)} = ∅, i.e., there exists at least one set that satisfies each threshold in T . This is ensured by the following condition: Expressing threshold constraints. Cardinality constraints can be expressed in FOL over the vocabulary Σ = Σ C Σ Prm T using quantification. To express the condition that |{n : node | ϕ(n,ū)}| ≥ t, i.e., that there are at least t nodes that satisfy the FO formula ϕ over Σ C (whereū are additional free variables in ϕ), we use the following first-order formula over Σ: ∃q : set t . ∀n : node. member t (n, q) → ϕ(n,ū), where set t is the threshold sort assigned to t. Similarly, to express the property that a node is a member of a set parameter a (e.g., to check if n ∈ f , i.e., a node is faulty) we use the FO formula member a (n). For example, in Fig. 1, line 5 (right) uses the FO modeling to express the condition in line 5 (left). This modeling is sound in the following sense: 1. For every a ∈ Prm S and FO valuation ι: s, ι |= member a (n) iff ι(n) ∈ I B (a), 2. For every t ∈ T , formula ϕ, and FO valuation ι: s, ι |= ∃q : Incompleteness. Lem. 1 ensures that the FO modeling can be soundly used to verify the protocol. It also ensures that the modeling is precise on threshold-faithful structures (Def. 1). Yet, the FO transition system is not restricted to such states, making it an abstraction of the actual protocol. To have any hope to verify the protocol, we must capture some of the intended meaning of the threshold sorts and relations. This is obtained by adding FO axioms to the FO transition system. Soundness is maintained as long as the axioms hold in all threshold-faithful structures. We note that the set of all FO formulas over Σ that hold in all threshold-faithful structures is not recursively enumerable 5 , which is where the essential incompleteness of our approach lies.

Decomposition via Threshold Intersection Properties
In this section, we identify a set of properties we call threshold intersection properties. When captured via FO axioms, these properties suffice for verifying many thresholdbased protocols (all the ones we considered). Importantly, these are properties of sets adhering to the thresholds that do not involve the protocol state. As a result, they can be expressed both in FOL and in BAPA. This allows us to decompose the verification task into: (i) checking that certain threshold properties are valid in all threshold-faithful structures by checking that they are implied by Γ (carried out using quantifier free BAPA), and (ii) checking that the verification conditions of the FO transition-system with the same threshold properties taken as axioms are valid (carried out in first-order logic, and in EPR if quantifier alternations are acyclic).

Threshold Intersection Property Language
Threshold properties are expressed in the threshold intersection property language (TIP). TIP is essentially a subset of BAPA, specialized to have the properties listed above.
Syntax. We define TIP as follows, where t ∈ T is a threshold (of the form k ) and a ∈ Prm S : TIP restricts the use of set cardinality to threshold guards g ≥t (b) with the meaning |b| ≥ t. No other arithmetic atomic formulas ( 1 = 2 or 1 < 2 ) are allowed. TIP only allows guarded quantifiers over set variables, that is, quantification is restricted to sets of a certain cardinality. We exclude negation, disjunction and existential quantification in formulas. We restrict comparison atomic formulas to b = ∅ and b c = ∅, which correspond to asserting that the cardinality of the set represented by b is at least 1, respectively n. Furthermore, we forbid set union and restrict complementation to atomic set terms. We refer to such formulas as intersection properties since they express properties of intersections of (atomic) sets.
Example 1. In Bosco, the following property captures the fact that the intersection of a set of at least n − t nodes and a set of more than n+3t 2 nodes consists of at least n−t 2 non-faulty nodes. This is needed for establishing correctness of the protocol. ∀x : g ≥n−t . ∀y : Semantics. As TIP is essentially a subset of BAPA, we define its semantics by translating its formulas to BAPA, where most constructs directly correspond to BAPA constructs, and guards are translated to cardinality constraints: Set terms (derived from B) are also set terms of BAPA, and most set formula constructs map to constructs of BAPA directly. The only constructs that are not in BAPA are those involving guards, which correspond to a special case of cardinality constraints. We therefore define the following translation B: The notions of structures, satisfaction, equivalence, validity, satisfiability, etc. are inherited from BAPA. In particular, given a set of BAPA resilience conditions Γ over the parameters Prm, we say that a TIP formula ϕ is Γ -valid, denoted Γ |= ϕ, if Γ |= B(ϕ).
If Γ is quantifier-free (which is the typical case), Γ -validity of TIP formulas can be checked via validity checks of quantifier-free BAPA formulas, supported by mature solvers. Note that Γ -validity of a formula of the form ∀x : g ≥t1 . |x ∩ b| ≥ t 2 is equivalent to Γ |= ∀u. u ≥ t 1 → u + |b| − n ≥ t 2 , allowing to replace quantification over sets by quantification over integers, thus improving performance of existing solvers.

Translation to FOL
To verify threshold-based protocols, we translate TIP formulas to FO axioms, using the threshold sorts and relations. To translate g ≥t (b), we follow the principle in (Sec. 3.2): We lift FO to sets of formulas: Soundness of the translation to FOL. Next, we state the soundness of the translation, which intuitively means that FO(ϕ) is "equivalent" to ϕ over threshold-faithful FO structures (Def. 1). This justifies adding FO(ϕ) as a FO axiom whenever ϕ is Γ -valid. Corollary 1. For every closed TIP formula ϕ such that Γ |= ϕ, we have that FO(ϕ) is satisfied by every threshold-faithful first-order structure.
This justifies using the translation to FOL in order to generate first-order axioms from TIP formulas ϕ that are entailed by the resilience conditions. Namely, if Γ |= ϕ, then FO(ϕ) may be safely added as a first-order axiom.

Automatically Inferring Threshold Intersection Properties
To apply the approach described in Sec. 3 and 4 for verifying threshold-based protocols, it is crucial to find suitable threshold properties for a given protocol.That is, given the resilience conditions Γ and a FO transition system modeling the protocol, we need to find a set ∆ of TIP formulas such that (i) Γ |= ϕ for every ϕ ∈ ∆, and (ii) the VCs of the transition system with the axioms FO(∆) are valid.
In this section, we address the problem of automatically inferring such a set ∆. In particular, we prove that for any protocol that satisfies a natural condition, there are finitely many Γ -valid TIP formulas (up to equivalence), enabling a complete automatic inference algorithm. Furthermore, we show that (under certain reasonable conditions formalized in this section), the FO axioms resulting from the inferred TIP properties have an acyclic quantifier alternation graph, facilitating protocol verification in EPR.
Notation. For the rest of this section, we fix a set Prm of parameters, a set Γ of resilience conditions over Prm, and a set T of thresholds. Note that b = ∅ ≡ g ≥1 (b) and b c = ∅ ≡ g ≥n (b). Therefore, for uniformity of the presentation, given a set T of thresholds, we defineT def = T ∪ {1, n} and replace atomic formulas of the form b = ∅ and b c = ∅ by the corresponding guard formulas. As such, the only atomic formulas are of the form g ≥t (b) where t ∈T . Note that guards in quantifiers are still restricted to g ≥t where t ∈ T . Given a set Prm S , we also denotePrm S = Prm S ∪ {a c | a ∈ Prm S }.

Finding Consequences in the Threshold Intersection Property Language
In this section, we present AIP-an algorithm for inferring all Γ -valid TIP formulas. A naïve (non-terminating) algorithm would iteratively check Γ -validity of every TIP formula. Instead, AIP prunes the search space relying on the following condition, which essentially states that no guard is "equivalent" to g ≥0 with respect to Γ .
If Γ |= t ≤ 0 then t is degenerate in the sense that g ≥t (b) is always Γ -valid, and ∀x : g ≥t . g ≥t (x ∩ b) is never Γ -valid unless t is also degenerate. (Note that checking if a threshold is degenerate amounts to a (non) entailment check in BAPA, which is decidable.) We observe that we can (i) push conjunctions outside of formulas (since ∀ distributes over ∧), and assuming non-degeneracy, (ii) ignore terms of the form x c as, under the assumption that T is non-degenerate, they will not appear in Γ -valid formulas. These observations are formalized by the following lemma.
We refer to ϕ i of the form above as simple, and refer to g ≥t as its atomic guard.
Proof (sketch). Let ϕ be a formula in TIP such that Γ |= ϕ. Since universal quantification distributes over conjunction, there exist ϕ 1 , . . . , ϕ m such that ϕ ≡ m i=1 ϕ i and for every 1 ≤ i ≤ m, the formula ϕ i is conjunction-free, i.e., ϕ i is a quantified atomic formula.
Since ϕ is Γ -valid, each of the ϕ i 's is Γ -valid as well. This means, because T is Γnon-degenerate, that the term ∅ cannot appear in the formula. Furthermore, the term ∅ c as well as duplicated instances of terms in ϕ i can be omitted, resulting in an equivalent formula. As a result, the essence of the proof is to show that x c will not appear in Γ -valid formulas.
Consider such ϕ i with atomic guard t b and a universally quantified variable x with guard t a . Recall that T is Γ -non-degenerate and let s B = (D, I) be such that s B |= Γ but s B |= t b ≤ 0, i.e., I(t b ) > 0. Since T is also Γ -feasible, we have that in particular I(t a ) ≤ I(n). Therefore, taking a valuation where ι( By Lem. 2, it suffices to generate all simple Γ -valid formulas. Next, we show that this can be done more efficiently by pruning the search space based on a subsumption relation that is checked syntactically avoiding Γ -validity checks.  When h 1 , h 2 ∈T , h 1 Γ h 2 means that Γ |= ∀x : g ≥h1 . g ≥h2 (x), and when h 1 ∈ Prm S and h 2 ∈T , h 1 Γ h 2 means that Γ |= g ≥h2 (h 1 ). We lift the relation Γ to a partial order over simple formulas: Definition 6. Given simple formulas Lemma 3. Let α, β be simple formulas such that α Γ β. If Γ |= α then Γ |= β.
Corollary 2. If no simple formula with q quantifiers is Γ -valid then no simple formula with more than q quantifiers is Γ -valid.
Alg. 1 depicts AIP that generates all Γ -valid simple formulas, relying on Lem. 3. Alg. 1 is designed to emit all Γ -valid simple formulas, including ones that are subsumed or entailed by others, since BAPA entailment does not imply entailment between the FO translations 6 . AIP uses a naïve search strategy; different strategies can be used (e.g. [25]). Based on Cor. 2, AIP terminates if for some number of quantifiers no Γ -valid formula is discovered.
Lemma 4 (Soundness). Every formula ϕ that is returned by the algorithm is Γ -valid.
The proof to Lem. 4 follows directly from Lem. 3. Proof Lem. 5 is obtained by using Lem. 2 and 3, and Cor. 2. Next, we characterize the cases in which there are finitely many Γ -valid TIP formulas, up to equivalence, and thus, AIP is guaranteed to terminate.
Sanity implies that no threshold in T is equivalent to 0 or to n under Γ (in particular, this implies non-degeneracy). In fact, it captures a stronger requirement that for every pair of thresholds, there is a model of Γ in which one of them is not interpreted as 0 and the other is not interpreted as n.
Theorem 2. Assume that T is Γ -feasible. Then the following conditions are equivalent: 1. There are finitely many Γ -valid simple formulas. 2. There are finitely many Γ -valid TIP formulas, up to equivalence.
The proof that (3) implies (1) uses the following lemma. Lemma 6. If T is Γ -sane, then for every t a ∈ T and t b ∈T , there exists a number Proof (of Lem. 6). Let t a ∈ T, t b ∈T be arbitrary guards. As T is Γ -sane, there is a structure s B = (D, I) such that s B |= Γ and We define Q ta,t b = I(n). Wlog the universal set of s B is U = {e 1 . . . e Qt a ,t b }. We define a valuation ι for s B in which for every Lem. 3 this also holds for any q ≥ Q ta,t b .
Proof (of Thm. 2). By Lem. 2, (1) implies (2). Then (2) implies (3) since whenever Γ |= t a > n − 1 ∨ t b ≤ 0, then any formula of the form ∀x 1 : g ≥ta . . . . ∀x q : where Q ta,t b is the number defined by Lem. 6. Then every Γvalid simple TIP formula α must have less than Q quantifiers, as otherwise there exists t a ∈ T such that at least max t b ∈T Q ta,t b quantifiers have guard g ≥ta . Suppose that g ≥t is the atomic guard of α, then this means that at least Q ta,t quantifiers have guard g ≥ta , and thus, from Lem. 3 and Lem. 6, we get that Γ |= α.

From TIP to Axioms in EPR
The set of simple formulas generated by AIP, ∆, is translated to FOL axioms as described in Sec. 4.2. Next, we show how to ensure that the quantifier alternation graph (Sec. 2) of FO(∆) is acyclic. Observation 1. A simple formula ϕ with atomic guard g ≥t induces quantifier alternation edges in QA(FO(ϕ)) from the threshold sorts of the guards of its universal quantifiers (these are thresholds in T ) to the threshold sort of t if t ∈ T or to the node sort if t = 1. If t = n, no quantifier alternation edges are induced by ϕ.
Therefore, from Lem. 3, for a Γ -valid ϕ, cycles in QA(FO(ϕ)) may only occur if they occur in the graph obtained by Γ . Furthermore, if QA(FO(ϕ)) is not acyclic, then the atomic guard must be equal to one of the quantifier guards. We refer to such a formula as a cyclic formula. We show that, under the following assumption, we can eliminate all cyclic formulas from ∆.
Intuitively, if T is not Γ -acyclic, then it has (at least) two "equivalent" thresholds, making one of them redundant. If that is the case, we can alter the protocol and its proof so that one of these guards is eliminated and the other one is used instead.
To facilitate elimination of cyclic formulas, we also need to strengthen the sanity requirement (Def. 7) to refer to the set parameters as well: T is Γ -sane and, in addition, for every t ∈ T and a ∈Prm S , Γ |= t ≤ 0 ∨ |a| = n.
Theorem 3. Assume that T is Γ -feasible and Γ -acyclic and that (T, Prm S ) is Γ -sane. Let ∆ be the set of simple Γ -valid formulas returned by AIP, and let ∆ = {ϕ ∈ ∆ | ϕ is not cyclic}. Then the verification conditions of the first-order transition system with axioms FO(∆) are valid if and only if they are valid with axioms FO(∆ ). Further, the quantifier alternation graph of FO(∆ ) is acyclic.
To prove the theorem we first prove the following lemma: Lemma 7. If T is Γ -feasible and (T, Prm S ) is Γ -sane, then for every Γ -valid simple formula ϕ, if ϕ is cyclic, then FO(ϕ) ≡ true.
Proof of Thm. 3. Let ∆ be the set of all Γ -valid TIP formulas, and let ∆ be defined as in the theorem. Lem. 7 shows that FO(∆ ) |= FO(∆) as required. Assume QA(FO(∆ )) is cyclic with a cycle α. Following Observation 1, the sort node cannot be in α. Let t 1 , t 2 ∈ T be such that set t1 , set t2 ∈ α. Because Γ is transitive, we get that t 1 Γ t 2 and t 2 Γ t 1 , which by definition of Γ means that Γ |= t 1 = t 2 , and because T is Γ -acyclic, we get that t 1 = t 2 , i.e., α corresponds to a self loop over a single sort set t1 . By Observation 1, a self loop may only arise from a cyclic formula. Hence, each cycle in QA(FO(∆ )) is induced by a single cyclic formula. As ∆ contains no cyclic formulas, we conclude that QA(FO(∆ )) contains no cycles.

Finding Minimal Properties Required for a Protocol
If ∆ consists of all acyclic Γ -valid TIP formulas returned by AIP, using FO(∆) as FO axioms leads to divergence of the verifier. To overcome this, we propose two variants.
Minimal Equivalent ∆ min . Some of the formulas in FO(∆) are implied by others, making them redundant. We remove such formulas using a greedy procedure that for every ϕ i ∈ ∆, checks whether FO(∆ \ {ϕ i }) |= FO(ϕ i ), and if so, removes ϕ i from ∆. Note that if QA(FO(∆)) is acyclic, the check translates to (un)satisfiability in EPR.
This procedure results in ∆ min ⊆ ∆ s.t. FO(∆ min ) |= FO(∆) and no strict subset of ∆ min satisfies this condition. That is, ∆ min is a local minimum for that property.
suffices to prove the first-order VCs, and enables to discharge the VCs more efficiently. We compute such a set ∆ int iteratively. Initially, ∆ int = ∅. In each iteration, we check the VCs. If a counterexample to induction (CTI) is found, we add to ∆ int a formula from ∆ not satisfied by the CTI. In this approach, ∆ is not pre-computed. Instead, AIP is invoked lazily to generate candidate formulas in reaction to CTIs.

Evaluation
We evaluate the approach by verifying several challenging threshold-based distributed protocols that use sophisticated thresholds: Bosco [38] (presented in Sec. 3), Hybrid Reliable Broadcast [39], and Byzantine Fast Paxos [22].

Protocols
Bosco Bosco was explained in detail in Sec. 3. We verified it under 3 different resilience conditions. The condition n > 3t is required in order to ensure correctness. The condition n > 5t allows Bosco to guarantee that if there are no Byzantine processes and all processes have the same input value, then every processor would reach a decision in a single network step (weakly one-step). If n > 7t then Bosco ensures that even if there are some faulty processes, when all non-faulty processes start with the same initial value, they would reach a decision within a single network step (strongly one-step). To evaluate our approach, we verify the safety and liveness (using the liveness to safety reduction presented in [29]) of Bosco.
Hybrid Reliable Broadcast We consider the asynchronous reliable broadcast from [39], that is designed to tolerate t b < n 3 Byzantine faulty processes. Hybrid reliable broadcast, in its extended version, tolerates four different types of faults (namely Byzantine faults, symmetric faults, clean crash, and crash faults), with associated constants denoted in Fig. 2 by f b , f s , f c , and f i , respectively. The protocol constitutes the core of the clock synchronization algorithm presented in [42]. Interestingly, the protocol is correct under several different threshold conditions [23]. Nevertheless, the threshold intersection properties are same in all cases, which confirms that we capture the essence of thresholds, independently of their arithmetic representation. We verify the safety and liveness of Hybrid Reliable Broadcast.
Byzantine Fast Paxos Byzantine Fast Paxos [22] is a fast-learning [20,21] Byzantine fault-tolerant consensus protocols for an asynchronous system equipped with a leaderelection module; "fast-learning" means that under favourable timing, a decision can be reached in a single round of communication (when each node is simultaneously a proposer, acceptor, and learner) despite q crash failures. Moreover, Byzantine Fast Paxos is optimal in the sense that, in general, no fast-learning protocol can improve the bounds on n [20]. Byzantine Fast Paxos is safe if at most t Byzantine failures occur in a system of n nodes where n > 3t + 2q for some parameter q ≤ t (Lamport [22] assumes that at most m out of the t failures are Byzantine; we only consider the case m = t).
We verified the safety of Byzantine Fast Paxos, i.e., agreement.
Note that Byzantine fault-tolerant fast-learning consensus protocols are notoriously tricky to develop and verify. For example, two such algorithms, Zyzzyva [16] and FaB [27], were recently revealed incorrect [1] despite having been published at major systems conferences. The verification of Fast Byzantine Paxos therefore shows that the methodology presented in this paper is able to handle some of the most intricate distributed protocols.

Implementation
We implemented both algorithms described in Sec. 5.3. AIP EAGER eagerly constructs ∆ by running AIP, and then uses EPR reasoning to remove redundant formulas (whose FO representation is implied by the FO representation of others). To reduce the number of EPR validity checks used during this minimization step, we implemented an optimization that allows us to prove redundancy of TIP formulas internally based on an extension of the notion of subsumption from Sec. 5. AIP LAZY computes a subset of ∆ while using AIP in a lazy fashion, guided by CTIs obtained from attempting to verify the FO transition system. Our implementations use CVC4 to discharge BAPA queries, and Z3 to discharge EPR queries. Verification of first-order transition systems is performed using Ivy, which internally uses Z3 as well. All experiments reported were performed on a laptop running 64-bit Windows 10, with a Core-i5 2.2 GHz CPU, using Z3 version 4.8.4, CVC4 version 1.7, and the latest version of Ivy. Fig. 2 lists the protocols we verified and the details of the evaluation. Each experiment was repeated 10 times, and we will report the mean time (µ) and standard deviation (σ). The figure's caption explains the presented information, and we discuss the results below.

Results
AIP EAGER For all protocols, running AIP took less than 1 minute (column t C ), and generated all Γ -valid simple TIP formulas. We observe that for most formulas, (in)validity is deduced from other formulas by subsumption, and less than 2%-5% of the formulas are actually checked using a BAPA query. With the optimization of the redundancy check, minimization of the set is performed in negligible time. The resulting set, ∆ EAGER , contains 3-5 formulas, compared to 39-79 before minimization.
Due to the optimization described in Sec. 4 for the BAPA validity queries, the number of quantifiers in the TIP formulas that are checked by AIP does not affect the time needed to compute the full ∆. For example, Bosco under the Strongly One-step resilience condition contains Γ -valid simple TIP formulas with up to 7 quantifiers (as n > 7t and t 1 = n − t), but AIP does not take significantly longer to find ∆. Interestingly, in this example the Γ -valid TIP formulas with more than 3 quantifiers are implied (in FOL) by formulas with at most 3 quantifiers, as indicated by the fact that these are the only formulas that remain in ∆ Bosco Strongly One-step EAGER . AIP LAZY With the lazy approach based on CTIs, the time for finding the set of TIP formulas, ∆ LAZY , is generally longer. This is because the run time is dominated by calls to Ivy with FO axioms that are too weak for verifying the protocol. However, the resulting ∆ LAZY has a significant benefit: it lets Ivy prove the protocol much faster compared to using ∆ EAGER . Comparing t V in AIP EAGER vs. AIP LAZY shows that when the former takes a minute, the latter takes a few seconds, and when the former times out after 1 hour, the latter terminates, usually in under 1 minute. Comparing the formulas of ∆ EAGER and ∆ LAZY reveals the reason. While the FO translation of both yields EPR formulas, the formulas resulting from ∆ EAGER contain more quantifiers and generate much more ground terms, which degrades the performance of Z3.
Another advantage of the lazy approach is that during the search, it avoids considering formulas with many quantifiers unless those are actually needed. Comparing the 3 versions of Bosco we see that AIP LAZY is not sensitive to the largest number of quantifiers that may appear in a Γ -valid simple TIP formula. The downside is that AIP LAZY performs many Ivy checks in order to compute the final ∆ LAZY . The total duration of finding CTIs varies significantly (as demonstrated under the column t I ), in part because it is very sensitive to the CTIs returned by Ivy, which are in turn affected by the random seed used in the heuristics of the underlying solver.
Finally, ∆ LAZY provides more insight into the protocol design, since it presents minimal assumptions that are required for protocol correctness. Thus, it may be useful in designing and understanding protocols.

Related Work
Fully automatic verification of threshold-based protocols. Algorithms modeled as Threshold automata (TA) [13] have been studied in [15,12], and verified using an automated tool ByMC [14]. The tool also automatically synthesizes thresholds as arithmetic expressions [23]. Reachability properties of TAs for more general thresholds are studied in [17]. There have been recent advances in verification of synchronous threshold-based algorithms using TAs [40], and of asynchronous randomized algorithms where TAs support coin tosses and unboundedly many rounds [3]. Still, this modeling is very restrictive and not as faithful to the pseudo-code as our modeling.
Another approach for full automation is to use sound and incomplete procedures for deduction and invariant search for logics that combine quantifiers and set cardinalities [9,7]. However, distributed systems of the level of complexity we consider here (e.g., Byzantine Fast Paxos) are beyond the reach of these techniques.
Verification of distributed protocols using decidable logics. Padon et al. [32] introduced an interactive approach for the safety verification of distributed protocols based on EPR using the Ivy [28] verification tool. Later works extended the approach to more complex protocols [31], their implementations [41], and liveness properties [29,30]. Those works verified some threshold protocols using ad-hoc first-order modeling and axiomatization of threshold-intersection properties, whereas we develop a systematic methodology. Moreover, the axioms were not mechanically verified, except in [41], where a simple intersection property-intersection of two sets with more than n 2 nodesrequires a proof by induction over n. The proof relies on a user provided induction hypothesis that is automatically checked using the FAU decidable fragment [8]. This approach requires user ingenuity even for a simple intersection property, and we expect that it would not scale to the more complex properties required for e.g. Bosco or Fast Byzantine Paxos. In contrast, our approach completely automates both verification and inference of threshold-intersection properties required to verify protocol correctness.
Dragoi et al. [5] propose a decidable logic supporting cardinalities, uninterpreted functions, and universal quantifiers for verifying consensus algorithms expressed in the partially synchronous Heard-Of Model. As in this paper, the user is expected to provide an inductive invariant. The PSync framework [6] extends the approach to protocol implementations. Compared to our approach, the approach of Dragoi et al. is less flexible due to the specialized logic used and the restrictions of the Heard-Of Model.
Our approach decomposes verification into EPR and BAPA. Piskac [33] presents a decidable logic that combines BAPA and EPR, with some restrictions. The verification conditions of the protocols we consider are outside the scope of this fragment since they include cardinality constraints in the scope of quantifiers. Furthermore, this logic is not supported by mature solvers. Instead of looking for a specialized logic per protocol, we rely on a decomposition which allows more flexibility.
Recently, [10] presented an approach for verifying asynchronous algorithms by reduction to synchronous verification. This technique is largely orthogonal and complementary to our approach, which is focused on the challenge of cardinality thresholds.
Verification using interactive theorem provers. We are not aware of works based on interactive theorem provers that verified protocols with complex thresholds as we do in this work (although doing so is of course possible). However, many works used interactive theorem provers to verify related protocols, e.g., [43,37,26,35,11,36] (the most related protocols use either n 2 or 2n 3 as the only thresholds, other protocols do not involve any thresholds). The downside of verification using interactive theorem provers is that it requires tremendous human efforts and skills. For example, the Verdi proof of Raft included 50,000 lines of proof in Coq for 500 lines of code [44].

Conclusion
This paper proposes a new deductive verification approach for threshold-based distributed protocols by decomposing the verification problem into two well-established decidable logics, BAPA and EPR, thus allowing greater flexibility compared to monolithic approaches based on domain-specific, specialized logics. The user models their protocol in EPR, defines the thresholds and resilience conditions using arithmetic in BAPA, and provides an inductive invariant. An automatic procedure infers threshold intersection properties expressed in TIP that are both (1) sound w.r.t. the resilience conditions and (2) sufficient to discharge the VCs. Soundness is automatically checked in (quantifier-free) BAPA, and the VCs are automatically discharged using EPR. Both logics are decidable, supported by mature solvers, and allow providing the user with an understandable counterexample in case verification fails.
We evaluate the approach by formally verifying the correctness (both safety and liveness) of intricate protocols, including notoriously tricky fast-learning consensus protocols such as Byzantine Fast Paxos. The experimental results show that threshold intersection properties are inferred in a matter of minutes. While this may be too slow for interactive use, we expect improvements such as memoization and parallelism to provide response times of a few seconds in an iterative, interactive setting. Another potential future direction is combining our inference algorithm with automated invariant inference algorithms.  is the set of thresholds and Γ is the resilience condition. AIPEAGER lists metrics for the procedure of finding all Γ -valid TIP formulas (taking time t C ), and verifying the transition system using the resulting properties (taking time tv).
Obtaining a minimal subset that FO-implies the rest takes negligible time, so we did not include it in the shows c v , where c is the number of Γ -valid simple formulas that were checked using the BAPA solver (CVC4), and v is the total number of Γ -valid simple formulas. Namely v − c simple formulas were inferred to be valid via subsumption. I reports the analogous metric for Γ -invalid simple formulas. Finally, Q reports the maximal number of quantifiers considered (for which all formulas were Γ -invalid).
AIPLAZY lists metrics for the procedure of finding a set of Γ -valid TIP formulas sufficient to prove the protocol based on counterexamples. The resulting set is listed in ∆ Protocol LAZY , and t I lists the total Ivy runtime, with the standard deviation specified below. V