Partial Quantifier Elimination and Property Generation

Abstract

We study partial quantifier elimination (PQE) for propositional CNF formulas with existential quantifiers. PQE is a generalization of quantifier elimination where one can limit the set of clauses taken out of the scope of quantifiers to a small subset of clauses. The appeal of PQE is that many verification problems (e.g., equivalence checking and model checking) can be solved in terms of PQE and the latter can be dramatically simpler than full quantifier elimination. We show that PQE can be used for property generation that one can view as a generalization of testing. The objective here is to produce an unwanted property of a design implementation, thus exposing a bug. We introduce two PQE solvers called $EG \text {-} PQE$ and $EG \text {-} PQE ^+$. $EG \text {-} PQE$ is a very simple SAT-based algorithm. $EG \text {-} PQE ^+$ is more sophisticated and robust than $EG \text {-} PQE$. We use these PQE solvers to find an unwanted property (namely, an unwanted invariant) of a buggy FIFO buffer. We also apply them to invariant generation for sequential circuits from a HWMCC benchmark set. Finally, we use these solvers to generate properties of a combinational circuit that mimic symbolic simulation.

1 Introduction

In this paper, we consider the following problem. Let F(X, Y) be a propositional formula in conjunctive normal form (CNF)¹ where X, Y are sets of variables. Let G be a subset of clauses of F. Given a formula $\exists {X} [F]$, find a quantifier-free formula H(Y) such that $\exists {X} [F]\equiv H\wedge \exists {X} [F \setminus G]$. In contrast to full quantifier elimination (QE), only the clauses of G are taken out of the scope of quantifiers here. So, we call this problem partial QE (PQE) [1]. (In this paper, we consider PQE only for formulas with existential quantifiers.) We will refer to H as a solution to PQE. Like SAT, PQE is a way to cope with the complexity of QE. But in contrast to SAT that is a special case of QE (where all variables are quantified), PQE generalizes QE. The latter is just a special case of PQE where $G = F$ and the entire formula is unquantified. Interpolation [2, 3] can be viewed as a special case of PQE as well [4, 5].

The appeal of PQE is threefold. First, it can be much more efficient than QE if G is a small subset of F. Second, many verification problems like SAT, equivalence checking, model checking can be solved in terms of PQE [1, 6,7,8]. So, PQE can be used to design new efficient methods for solving known problems. Third, one can apply PQE to solving new problems like property generation considered in this paper. In practice, to perform PQE, it suffices to have an algorithm that takes a single clause out of the scope of quantifiers. Namely, given a formula $\exists {X} [F(X,Y)]$ and a clause $C \in F$, this algorithm finds a formula H(Y) such that $\exists {X} [F] \equiv H \wedge \exists {X} [F \setminus \{C\}]$. To take out k clauses, one can apply this algorithm k times. Since $H \wedge \exists {X} [F] \equiv H\wedge \exists {X} [ F \setminus \{C\}]$, solving the PQE above reduces to finding H(Y) that makes C redundant in $H \wedge \exists {X} [F]$. So, the PQE algorithms we present here employ redundancy based reasoning. We describe two PQE algorithms called $EG \text {-} PQE$ and $EG \text {-} PQE ^+$ where “EG” stands for “Enumerate and Generalize”. $EG \text {-} PQE$ is a very simple SAT-based algorithm that can sometimes solve very large problems. $EG \text {-} PQE ^+$ is a modification of $EG \text {-} PQE$ that makes the algorithm more powerful and robust.

In [7], we showed the viability of an equivalence checker based on PQE. In particular, we presented instances for which this equivalence checker outperformed ABC [9], a high quality tool. In this paper, we describe and check experimentally one more important application of PQE called property generation. Our motivation here is as follows. Suppose a design implementation Imp meets the set of specification properties $P_1,\dots ,P_m$. Typically, this set is incomplete. So, Imp can still be buggy even if every $P_i,i=1,\dots ,m$ holds. Let $P^*_{m+1},\dots ,P^*_n$ be desired properties adding which makes the specification complete. If Imp meets the properties $P_1,\dots ,P_m$ but is still buggy, a missed property $P^*_i$ above fails. That is, Imp has the unwanted property $\overline{P^*_i}$. So, one can detect bugs by generating unspecified properties of Imp and checking if there is an unwanted one.

Currently, identification of unwanted properties is mostly done by massive testing. (As we show later, the input/output behavior specified by a single test can be cast as a simple property of Imp.) Another technique employed in practice is guessing unwanted properties that may hold and formally checking if this is the case. The problem with these techniques is that they can miss an unwanted property. In this paper, we describe property generation by PQE. The benefit of PQE is that it can produce much more complex properties than those corresponding to single tests. So, using PQE one can detect bugs that testing overlooks or cannot find in principle. Importantly, PQE generates properties covering different parts of Imp. This makes the search for unwanted properties more systematic and facilitates discovering bugs that can be missed if one simply guesses unwanted properties that may hold.

In this paper, we experimentally study generation of invariants of a sequential circuit N. An invariant of N is unwanted if a state that is supposed to be reachable in N falsifies this invariant and hence is unreachable. Note that finding a formal proof that N has no unwanted invariants is impractical. (It is hard to efficiently prove a large set of states reachable because different states are reached by different execution traces.) So developing practical methods for finding unwanted invariants if very important. We also study generation of properties mimicking symbolic simulation for a combinational circuit obtained by unrolling a sequential circuit. An unwanted property here exposes a wrong execution trace.

This paper is structured as follows. (Some additional information can be found in the supporting technical report [5].) In Sect. 2, we give basic definitions. Section 3 presents property generation for a combinational circuit. In Sect. 4, we describe invariant generation for a sequential circuit. Sections 5 and 6 present $EG \text {-} PQE$ and $EG \text {-} PQE ^+$ respectively. In Sect. 7, invariant generation is used to find a bug in a FIFO buffer. Experiments with invariant generation for HWMCC benchmarks are described in Sect. 8. Section 9 presents an experiment with property generation for combinational circuits. In Sect. 10 we give some background. Finally, in Sect. 11, we make conclusions and discuss directions for future research.

2 Basic Definitions

In this section, when we say “formula” without mentioning quantifiers, we mean “a quantifier-free formula”.

Definition 1

We assume that formulas have only Boolean variables. A literal of a variable v is either v or its negation. A clause is a disjunction of literals. A formula F is in conjunctive normal form (CNF) if $F = C_1 \wedge \dots \wedge C_k$ where $C_1,\dots ,C_k$ are clauses. We will also view F as the set of clauses $\{C_1,\dots ,C_k\}$. We assume that every formula is in CNF.

Definition 2

Let F be a formula. Then $\boldsymbol{ Vars (F)}$ denotes the set of variables of F and $\boldsymbol{ Vars (\exists {X} [F])}$ denotes $Vars (F)\!\setminus \!X$.

Definition 3

Let V be a set of variables. An assignment to V is a mapping $V'~\rightarrow \{0,1\}$ where $V' \subseteq V$. We will denote the set of variables assigned in   as $\boldsymbol{ Vars (\textbf{q})}$. We will refer to as a full assignment to V if $Vars (\textbf{q})=V$. We will denote as the fact that a) $Vars (\textbf{q}) \subseteq Vars (\textbf{r})$ and b) every variable of $Vars (\textbf{q})$ has the same value in and .

Definition 4

A literal, a clause and a formula are said to be satisfied (respectively falsified) by an assignment if they evaluate to 1 (respectively 0) under .

Definition 5

Let C be a clause. Let H be a formula that may have quantifiers, and be an assignment to $Vars (H)$. If C is satisfied by , then $\boldsymbol{C_{\textbf{q}} \equiv 1}$. Otherwise, $\boldsymbol{C_{\textbf{q}}}$ is the clause obtained from C by removing all literals falsified by . Denote by $\boldsymbol{H_{\textbf{q}}}$ the formula obtained from H by removing the clauses satisfied by and replacing every clause C unsatisfied by with $C_{\textbf{q}}$.

Definition 6

Given a formula $\exists {X} [F(X,Y)]$, a clause C of F is called a quantified clause if $Vars (C)$ $\cap ~X~\ne ~\emptyset$. If $Vars (C) \cap X = \emptyset$, the clause C depends only on free, i.e., unquantified variables of F and is called a free clause.

Definition 7

Let G, H be formulas that may have existential quantifiers. We say that G, H are equivalent, written $\boldsymbol{G \equiv H}$, if $G_{\textbf{q}} = H_{\textbf{q}}$ for all full assignments to $Vars (G) \cup Vars (H)$.

Definition 8

Let F(X, Y) be a formula and $G \subseteq F$ and $G \ne \emptyset$. The clauses of G are said to be redundant in $\boldsymbol{\exists {X} [F]}$ if $\exists {X} [F] \equiv \exists {X} [F \setminus G]$. Note that if $F \setminus G$ implies G, the clauses of G are redundant in $\exists {X} [F]$.

Definition 9

Given a formula $\exists {X} [F(X,Y))]$ and G where $G \subseteq F$, the Partial Quantifier Elimination (PQE) problem is to find H(Y) such that$\boldsymbol{\exists {X} [F]\equiv H\wedge \exists {X} [F \setminus G]}$. (So, PQE takes G out of the scope of quantifiers.) The formula H is called a solution to PQE. The case of PQE where $G = F$ is called Quantifier Elimination (QE).

Example 1

Consider the formula $F = C_1 \wedge C_2 \wedge C_3 \wedge C_4$ where $C_1=\overline{x}_3 \vee x_4$, $C_2\!=\!y_1\!\vee \!x_3$, $C_3=y_1 \vee \overline{x}_4$, $C_4\!=\!y_2\!\vee \!x_4$. Let Y denote $\{y_1,y_2\}$ and X denote $\{x_3,x_4\}$. Consider the PQE problem of taking $C_1$ out of $\exists {X} [F]$, i.e., finding H(Y) such that $\exists {X} [F] \equiv H \wedge \exists {X} [F \setminus \{C_1\}]$. As we show later, $\exists {X} [F] \equiv y_1 \wedge \exists {X} [F \setminus \{C_1\}]$. That is, $H\! =\!y_1$ is a solution to the PQE problem above.

Remark 1

Let D be a clause of a solution H to the PQE problem of Definition 9. If $F \setminus G$ implies D, then $H \setminus \{D\}$ is a solution to this PQE problem too.

Proposition 1

Let H be a solution to the PQE problem of Definition 9. That is, $\exists {X} [F]\equiv H\wedge \exists {X} [F \setminus G]$. Then $F \Rightarrow H$ (i.e., F implies H).

The proofs of propositions can be found in [5].

Definition 10

Let clauses $C'$,$C''$ have opposite literals of exactly one variable $w\!\in \! Vars (C')\!\cap \! Vars (C'')$. Then $C'$,$C''$ are called resolvable on w. Let C be a clause of a formula G and $w \in Vars (C)$. The clause C is said to be blocked [10] in G with respect to the variable w if no clause of G is resolvable with C on w.

Proposition 2

Let a clause C be blocked in a formula F(X, Y) with respect to a variable $x \in X$. Then C is redundant in $\exists {X} [F]$, i.e., $\exists {X} [F \setminus \{C\}]$ $\equiv$ $\exists {X} [F]$.

3 Property Generation by PQE

Many known problems can be formulated in terms of PQE, thus facilitating the design of new efficient algorithms. In [5], we give a short summary of results on solving SAT, equivalence checking and model checking by PQE presented in [1, 6,7,8]. In this section, we describe application of PQE to property generation for a combinational circuit. The objective of property generation is to expose a bug via producing an unwanted property.

Let M(X, V, W) be a combinational circuit where X, V, W specify the sets of the internal, input and output variables of M respectively. Let F(X, V, W) denote a formula specifying M. As usual, this formula is obtained by Tseitin’s transformations [11]. Namely, F equals $F_{G_1} \wedge \dots \wedge F_{G_k}$ where $G_1,\dots ,G_k$ are the gates of M and $F_{G_i}$ specifies the functionality of gate $G_i$.

Example 2

Let G be a 2-input AND gate defined as $x_3 = x_1 \wedge x_2$ where $x_3$ denotes the output value and $x_1,x_2$ denote the input values of G. Then G is specified by the formula $F_G\!=\!(\overline{x}_1 \vee \overline{x}_2\vee x_3) \wedge (x_1 \vee \overline{x}_3) \wedge (x_2 \vee \overline{x}_3)$. Every clause of $F_G$ is falsified by an inconsistent assignment (where the output value of G is not implied by its input values). For instance, $x_1\!\vee \overline{x}_3$ is falsified by the inconsistent assignment $x_1\!=\!0, x_3\!=\!1$. So, every assignment satisfying $F_G$ corresponds to a consistent assignment to G and vice versa. Similarly, every assignment satisfying the formula F above is a consistent assignment to the gates of M and vice versa.

3.1 High-Level View of Property Generation by PQE

One generates properties by PQE until an unwanted property exposing a bug is produced. (Like in testing, one runs tests until a bug-exposing test is encountered.) The benefit of property generation by PQE is fourfold. First, by property generation, one can identify bugs that are hard or simply impossible to find by testing. Second, using PQE makes property generation efficient. Third, by taking out different clauses one can generate properties covering different parts of the design. This increases the probability of discovering a bug. Fourth, every property generated by PQE specifies a large set of high-quality tests.

In this paper (Sects. 7, 9), we consider cases where identifying an unwanted property is easy. However, in general, such identification is not trivial. A discussion of this topic is beyond the scope of this paper. (An outline of a procedure for deciding if a property is unwanted is given in [5].)

3.2 Property Generation as Generalization of Testing

The behavior of M corresponding to a single test can be cast as a property. Let $w_i \in W$ be an output variable of M and be a test, i.e., a full assignment to the input variables V of M. Let $B^{\textbf{v}}$ denote the longest clause falsified by , i.e., $Vars (B^{\textbf{v}}) = V$. Let $l(w_i)$ be the literal satisfied by the value of $w_i$ produced by M under input . Then the clause $B^{\textbf{v}} \vee l(w_i)$ is satisfied by every assignment satisfying F, i.e., $B^{\textbf{v}} \vee l(w_i)$ is a property of M. We will refer to it as a single-test property (since it describes the behavior of M for a single test). If the input is supposed to produce the opposite value of $w_i$ (i.e., the one falsifying $l(w_i)$), then exposes a bug in M. In this case, the single-test property above is an unwanted property of M exposing the same bug as the test .

A single-test property can be viewed as a weakest property of M as opposed to the strongest property specified by $\exists {X} [F]$. The latter is the truth table of M that can be computed explicitly by performing QE on $\exists {X} [F]$. One can use PQE to generate properties of M that, in terms of strength, range from the weakest ones to the strongest property inclusively. (By combining clause splitting with PQE one can generate single-test properties, see the next subsection.) Consider the PQE problem of taking a clause C out of $\exists {X} [F]$. Let H(V, W) be a solution to this problem, i.e., $\exists {X} [F] \equiv H \wedge \exists {X} [F \setminus \{C\}]$. Since H is implied by F, it can be viewed as a property of M. If H is an unwanted property, M has a bug. (Here we consider the case where a property of M is obtained by taking a clause out of formula $\exists {X} [F]$ where only the internal variables of M are quantified. Later we consider cases where some external variables of M are quantified too.)

We will assume that the property H generated by PQE has no redundant clauses (see Remark 1). That is, if $D \in H$, then $F \setminus \{C\} \not \Rightarrow D$. Then one can view H as a property that holds due to the presence of the clause C in F.

3.3 Computing Properties Efficiently

If a property H is obtained by taking only one clause out of $\exists {X} [F]$, its computation is much easier than performing QE on $\exists {X} [F]$. If computing H still remains too time-consuming, one can use the two methods below that achieve better performance at the expense of generating weaker properties. The first method applies when a PQE solver forms a solution incrementally, clause by clause (like the algorithms described in Sects. 5 and 6). Then one can simply stop computing H as soon as the number of clauses in H exceeds a threshold. Such a formula H is still implied by F and hence specifies a property of M.

The second method employs clause splitting. Here we consider clause splitting on input variables $v_1,\dots ,v_p$, i.e., those of V (but one can split a clause on any subset of variables from $Vars (F)$). Let $F'$ denote the formula F where a clause C is replaced with $p+1$ clauses: $C_1 = C \vee \overline{l(v_1)}$,..., $C_p = C \vee \overline{l(v_p)}$, $C_{p+1} = C \vee l(v_1) \vee \dots \vee l(v_p)$, where $l(v_i)$ is a literal of $v_i$. The idea is to obtain a property H by taking the clause $C_{p+1}$ out of $\exists {X} [F']$ rather than C out of $\exists {X} [F]$. The former PQE problem is simpler than the latter since it produces a weaker property H. One can show that if $\{v_1,\dots ,v_p\}\! =\!V$, then a) the complexity of PQE reduces to linear; b) taking out $C_{p+1}$ actually produces a single-test property. The latter specifies the input/output behavior of M for the test falsifying the literals $l(v_1),\dots , l(v_p)$. (The details can be found in [5].)

3.4 Using Design Coverage for Generation of Unwanted Properties

Arguably, testing is so effective in practice because one verifies a particular design. Namely, one probes different parts of this design using some coverage metric rather than sampling the truth table (which would mean verifying every possible design). The same idea works for property generation by PQE for the following two reasons. First, by taking out a clause, PQE generates a property inherent to the specific circuit M. (If one replaces M with an equivalent but structurally different circuit, PQE will generate different properties.) Second, by taking out different clauses of F one generates properties corresponding to different parts of M thus “covering” the design. This increases the chance to take out a clause corresponding to the buggy part of M and generate an unwanted property.

3.5 High-Quality Tests Specified by a Property Generated by PQE

In this subsection, we show that a property H generated by PQE, in general, specifies a large set of high-quality tests. Let H(V, W) be obtained by taking C out of $\exists {X} [F(X,V,W)]$. Let Q(V, W) be a clause of H. As mentioned above, we assume that $F \setminus \{C\} \not \Rightarrow Q$. Then there is an assignment ( , , ) satisfying formula $(F \setminus \{C\}) \wedge \overline{Q}$ where , , are assignments to X, V, W respectively. (Note that by definition, ( , ) falsifies Q.) Let be the execution trace of M under the input . So, satisfies F. Note that the output assignments and must be different because has to satisfy Q. (Otherwise, satisfies $F \wedge \overline{Q}$ and so $F \not \Rightarrow Q$ and hence $F \not \Rightarrow H$.) So, one can view as a test “detecting” disappearance of the clause C from F. Note that different assignments satisfying $(F \setminus \{C\}) \wedge \overline{Q}$ correspond to different tests . So, the clause Q of H, in general, specifies a very large number of tests. One can show that these tests are similar to those detecting stuck-at faults and so have very high quality [5].

4 Invariant Generation by PQE

In this section, we extend property generation for combinational circuits to sequential ones. Namely, we generate invariants. Note that generation of desired auxiliary invariants is routinely used in practice to facilitate verification of a predefined property. The problem we consider here is different in that our goal is to produce an unwanted invariant exposing a bug. We picked generation of invariants (over that of weaker properties just claiming that a state cannot be reached in k transitions or less) because identification of an unwanted invariant is, arguably, easier.

4.1 Bugs Making States Unreachable

Let N be a sequential circuit and S denote the state variables of N. Let I(S) specify the initial state (i.e., ). Let $T(S',V,S'')$ denote the transition relation of N where $S',S''$ are the present and next state variables and V specifies the (combinational) input variables. We will say that a state of N is reachable if there is an execution trace leading to . That is, there is a sequence of states where , and there exist $i=0,\dots ,k\!-\!1$ for which . Let N have to satisfy a set of invariants $P_0(S),\dots ,P_m(S)$. That is, $P_i$ holds iff for every reachable state of N. We will denote the aggregate invariant $P_0 \wedge \dots \wedge P_m$ as $\boldsymbol{ P _{ agg }}$. We will call a bad state of N if . If $P _{ agg }$ holds, no bad state is reachable. We will call a good state of N if .

Typically, the set of invariants $P_0,\dots ,P_m$ is incomplete in the sense that it does not specify all states that must be unreachable. So, a good state can well be unreachable. We will call a good state operative (or op-state for short) if it is supposed to be used by N and so should be reachable. We introduce the term an operative state just to factor out “useless” good states. We will say that N has an op-state reachability bug if an op-state is unreachable in N. In Sect. 7, we consider such a bug in a FIFO buffer. The fact that $P _{ agg }$ holds says nothing about reachability of op-states. Consider, for instance, a trivial circuit $N _{ triv }$ that simply stays in the initial state and . Then $P _{ agg }$ holds for $N _{ triv }$ but the latter has op-state reachability bugs (assuming that the correct circuit must reach states other than ).

Let be the predicate satisfied only by a state . In terms of CTL, identifying an op-state reachability bug means finding for which the property must hold but it does not. The reason for assuming to be unknown is that the set of op-states is typically too large to explicitly specify every property to hold. This makes finding op-state reachability bugs very hard. The problem is exacerbated by the fact that reachability of different states is established by different traces. So, in general, one cannot efficiently prove many properties (for different states) at once.

4.2 Proving Operative State Unreachability by Invariant Generation

In practice, there are two methods to check reachability of op-states for large circuits. The first method is testing. Of course, testing cannot prove a state unreachable, however, the examination of execution traces may point to a potential problem. (For instance, after examining execution traces of the circuit $N _{ triv }$ above one realizes that many op-states look unreachable.) The other method is to check unwanted invariants, i.e., those that are supposed to fail. If an unwanted invariant holds for a circuit, the latter has an op-state reachability bug. For instance, one can check if a state variable $s_i \in S$ of a circuit never changes its initial value. To break this unwanted invariant, one needs to find an op-state where the initial value of $s_i$ is flipped. (For the circuit $N _{ triv }$ above this unwanted invariant holds for every state variable.) The potential unwanted invariants are formed manually, i.e., simply guessed.

The two methods above can easily overlook an op-state reachability bug. Testing cannot prove that an op-state is unreachable. To correctly guess an unwanted invariant that holds, one essentially has to know the underlying bug. Below, we describe a method for invariant generation by PQE that is based on property generation for combinational circuits. The appeal of this method is twofold. First, PQE generates invariants “inherent” to the implementation at hand, which drastically reduces the set of invariants to explore. Second, PQE is able to generate invariants related to different parts of the circuit (including the buggy one). This increases the probability of generating an unwanted invariant. We substantiate this intuition in Sect. 7.

Let formula $\boldsymbol{F_k}$ specify the combinational circuit obtained by unfolding a sequential circuit N for k time frames and adding the initial state constraint $I(S_0)$. That is, $F_k = I(S_0) \wedge T(S_0,V_0,S_1) \wedge \dots \wedge T(S_{k-1},V_{k-1},S_k)$ where $S_j, V_j$ denote the state and input variables of j-th time frame respectively. Let $H(S_k)$ be a solution to the PQE problem of taking a clause C out of $\exists {X_{k}} [F_k]$ where $X_{k} = S_0 \cup V_0 \cup \dots \cup S_{k-1} \cup V_{k-1}$. That is, $\exists {X_{k}} [F_k]$ $\equiv H \wedge$ $\exists {X_{k}} [F_k \setminus \{C\}]$. Note that in contrast to Sect. 3, here some external variables of the combinational circuit (namely, the input variables $V_0,\dots ,V_{k-1}$) are quantified too. So, H depends only on state variables of the last time frame. H can be viewed as a local invariant asserting that no state falsifying H can be reached in k transitions.

One can use H to find global invariants (holding for every time frame) as follows. Even if H is only a local invariant, a clause Q of H can be a global invariant. The experiments of Sect. 8 show that, in general, this is true for many clauses of H. (To find out if Q is a global invariant, one can simply run a model checker to see if the property Q holds.) Note that by taking out different clauses of $F_k$ one can produce global single-clause invariants Q relating to different parts of N. From now on, when we say “an invariant” without a qualifier we mean a global invariant.

5 Introducing $EG \text {-} PQE$

In this section, we describe a simple SAT-based algorithm for performing PQE called $EG \text {-} PQE$. Here ‘EG’ stands for ‘Enumerate and Generalize’. $EG \text {-} PQE$ accepts a formula $\exists {X} [F(X,Y)]$ and a clause $C \in F$. It outputs a formula H(Y) such that $\exists {X} [ F _{ ini }] \equiv H \wedge \exists {X} [ F _{ ini } \setminus \{C\}]$ where $F _{ ini }$ is the initial formula F. (This point needs clarification because $EG \text {-} PQE$ changes F by adding clauses.)

5.1 An Example

Before describing the pseudocode of $EG \text {-} PQE$, we explain how it solves the PQE problem of Example 1. That is, we consider taking clause $C_1$ out of $\exists {X} [F(X,Y)]$ where $F = C_1 \wedge \dots \wedge C_4$, $C_1=\overline{x}_3 \vee x_4$, $C_2\!=\!y_1\!\vee \!x_3$, $C_3=y_1 \vee \overline{x}_4$, $C_4\!=\!y_2\!\vee \!x_4$ and $Y=\{y_1,y_2\}$ and $X=\{x_3,x_4\}$.

$EG \text {-} PQE$ iteratively generates a full assignment to Y and checks if $(C_1)_{\textbf{y}}$ is redundant in $\exists {X} [F_{\textbf{y}}]$ (i.e., if $C_1$ is redundant in $\exists {X} [F]$ in subspace ). Note that if $(F \setminus \{C_1\})_{\textbf{y}}$ implies $(C_1)_{\textbf{y}}$, then $(C_1)_{\textbf{y}}$ is trivially redundant in $\exists {X} [F_{\textbf{y}}]$. To avoid such subspaces, $EG \text {-} PQE$ generates by searching for an assignment ( , ) satisfying the formula $(F \setminus \{C_1\}) \wedge \overline{C}_1$. (Here and are full assignments to Y and X respectively.) If such ( , ) exists, it satisfies $F \setminus \{C_1\}$ and falsifies $C_1$ thus proving that $(F \setminus \{C_1\})_{\textbf{y}}$ does not imply $(C_1)_{\textbf{y}}$.

Assume that $EG \text {-} PQE$ found an assignment

$(y_1\!=\!0,y_2\!=\!1,x_3\!=\!1,x_4\!=\!0)$ satisfying $(F \setminus \{C_1\}) \wedge \overline{C}_1$. So . Then $EG \text {-} PQE$ checks if $F_{\textbf{y}}$ is satisfiable. $F_{\textbf{y}}$ = $(\overline{x}_3 \vee x_4) \wedge x_3 \wedge \overline{x}_4$ and so it is unsatisfiable. This means that $(C_1)_{\textbf{y}}$ is not redundant in $\exists {X} [F_{\textbf{y}}]$. (Indeed, $(F \setminus \{C_1\})_{\textbf{y}}$ is satisfiable. So, removing $C_1$ makes F satisfiable in subspace .) $EG \text {-} PQE$ makes $(C_1)_{\textbf{y}}$ redundant in $\exists {X} [F_{\textbf{y}}]$ by adding to F a clause B falsified by . The clause B equals $y_1$ and is obtained by identifying the assignments to individual variables of Y that made $F_{\textbf{y}}$ unsatisfiable. (In our case, this is the assignment $y_1 = 0$.) Note that derivation of clause $y_1$ generalizes the proof of unsatisfiability of F in subspace $(y_1\!=\!0,y_2\!=\!1)$ so that this proof holds for subspace $(y_1\!=\!0,y_2\!=\!0)$ too.

Now $EG \text {-} PQE$ looks for a new assignment satisfying $(F \setminus \{C_1\}) \wedge \overline{C}_1$. Let the assignment $(y_1 =1,y_2=1,x_3=1,x_4=0)$ be found. So, . Since $(y_1\!=\!1,y_2\!=\!1,x_3=0)$ satisfies F, the formula $F_{\textbf{y}}$ is satisfiable. So, $(C_1)_{\textbf{y}}$ is already redundant in $\exists {X} [F_{\textbf{y}}]$. To avoid re-visiting the subspace , $EG \text {-} PQE$ generates the plugging clause $D = \overline{y}_1 \vee \overline{y}_2$ falsified by .

$EG \text {-} PQE$ fails to generate a new assignment because the formula$D \wedge (F \setminus \{C_1\}) \wedge \overline{C}_1$ is unsatisfiable. Indeed, every full assignment we have examined so far falsifies either the clause $y_1$ added to F or the plugging clause D. The only assignment $EG \text {-} PQE$ has not explored

yet is . Since $(F \setminus \{C_1\})_{\textbf{y}} = x_4$ and $(C_1)_{\textbf{y}}$ = $\overline{x}_3 \vee x_4$, the formula $(F \setminus \{C_1\}) \wedge \overline{C}_1$ is unsatisfiable in subspace . In other words, $(C_1)_{\textbf{y}}$ is implied by $(F \setminus \{C_1\})_{\textbf{y}}$ and hence is redundant. Thus, $C_1$ is redundant in $\exists {X} [ F _{ ini } \wedge y_1]$ for every assignment to Y where $F _{ ini }$ is the initial formula F. That is, $\exists {X} [ F _{ ini }]$ $\equiv y_1 \wedge$ $\exists {X} [ F _{ ini } \setminus \{C_1\}]$ and so the clause $y_1$ is a solution H to our PQE problem.

5.2 Description of $EG \text {-} PQE$

Fig. 1.

Pseudocode of $EG \text {-} PQE$

The pseudo-code of $EG \text {-} PQE$ is shown in Fig. 1. $EG \text {-} PQE$ starts with storing the initial formula F and initializing formula $Plg$ that accumulates the plugging clauses generated by $EG \text {-} PQE$ (line 1). As we mentioned in the previous subsection, plugging clauses are used to avoid re-visiting the subspaces where the formula F is proved satisfiable.

All the work is carried out in a while loop. First, $EG \text {-} PQE$ checks if there is a new subspace where $\exists {X} [(F \setminus \{C\})_{\textbf{y}}]$ does not imply $F_{\textbf{y}}$. This is done by searching for an assignment ( , ) satisfying $Plg \wedge (F \setminus \{C\}) \wedge \overline{C}$ (lines 3–4). If such an assignment does not exist, the clause C is redundant in $\exists {X} [F]$. (Indeed, let be a full assignment to Y. The formula $Plg \wedge (F \setminus \{C\}) \wedge \overline{C}$ is unsatisfiable in subspace for one of the two reasons. First, falsifies $Plg$. Then $C_{\textbf{y}}$ is redundant because $F_{\textbf{y}}$ is satisfiable. Second, $(F \setminus \{C\})_{\textbf{y}} \wedge \overline{C_{\textbf{y}}}$ is unsatisfiable. In this case, $(F \setminus \{C\})_{\textbf{y}}$ implies $C_{\textbf{y}}$.) Then $EG \text {-} PQE$ returns the set of clauses added to the initial formula F as a solution H to the PQE problem (lines 5–6).

If the satisfying assignment ( , ) above exists, $EG \text {-} PQE$ checks if the formula $F_{\textbf{y}}$ is satisfiable (line 7). If not, then the clause $C_{\textbf{y}}$ is not redundant in $\exists {X} [F_{\textbf{y}}]$ (because $(F \setminus \{C\})_{\textbf{y}}$ is satisfiable). So, $EG \text {-} PQE$ makes $C_{\textbf{y}}$ redundant by generating a clause B(Y) falsified by and adding it to F (line 9). Note that adding B also prevents $EG \text {-} PQE$ from re-visiting the subspace again. The clause B is built by finding an unsatisfiable subset of $F_{\textbf{y}}$ and collecting the literals of Y removed from clauses of this subset when obtaining $F_{\textbf{y}}$ from F.

If $F_{\textbf{y}}$ is satisfiable, $EG \text {-} PQE$ generates an assignment to X such that satisfies F (line 7). The satisfiability of $F_{\textbf{y}}$ means that every clause of $F_{\textbf{y}}$ including $C_{\textbf{y}}$ is redundant in $\exists {X} [F_{\textbf{y}}]$. At this point, $EG \text {-} PQE$ uses the longest clause D(Y) falsified by as a plugging clause (line 11). The clause D is added to $Plg$ to avoid re-visiting subspace . Sometimes it is possible to remove variables from to produce a shorter assignment such that still satisfies F. Then one can use a shorter plugging clause D that is falsified by and involves only the variables assigned in .

5.3 Discussion

$EG \text {-} PQE$ is similar to the QE algorithm presented at CAV-2002 [12]. We will refer to it as $CAV02 \text {-} QE$. Given a formula $\exists {X} [F(X,Y)]$, $CAV02 \text {-} QE$ enumerates full assignments to Y. In subspace , if $F_{\textbf{y}}$ is unsatisfiable, $CAV02 \text {-} QE$ adds to F a clause falsified by . Otherwise, $CAV02 \text {-} QE$ generates a plugging clause D. (In [12], D is called “a blocking clause”. This term can be confused with the term “blocked clause” specifying a completely different kind of a clause. So, we use the term “the plugging clause” instead.) To apply the idea of $CAV02 \text {-} QE$ to PQE, we reformulated it in terms of redundancy based reasoning.

The main flaw of $EG \text {-} PQE$ inherited from $CAV02 \text {-} QE$ is the necessity to use plugging clauses produced from a satisfying assignment. Consider the PQE problem of taking a clause C out of $\exists {X} [F(X,Y)]$. If F is proved unsatisfiable in subspace , typically, only a small subset of clauses of $F_{\textbf{y}}$ is involved in the proof. Then the clause generated by $EG \text {-} PQE$ is short and thus proves C redundant in many subspaces different from . On the contrary, to prove F satisfiable in subspace , every clause of F must be satisfied. So, the plugging clause built off a satisfying assignment includes almost every variable of Y. Despite this flaw of $EG \text {-} PQE$, we present it for two reasons. First, it is a very simple SAT-based algorithm that can be easily implemented. Second, $EG \text {-} PQE$ has a powerful advantage over $CAV02 \text {-} QE$ since it solves PQE rather than QE. Namely, $EG \text {-} PQE$ does not need to examine the subspaces where C is implied by $F \setminus \{C\}$. Surprisingly, for many formulas this allows $EG \text {-} PQE$ to completely avoid examining subspaces where F is satisfiable. In this case, $EG \text {-} PQE$ is very efficient and can solve very large problems. Note that when $CAV02 \text {-} QE$ performs complete QE on $\exists {X} [F]$, it cannot avoid subspaces where $F_{\textbf{y}}$ is satisfiable unless F itself is unsatisfiable (which is very rare in practical applications).

6 Introducing $EG \text {-} PQE ^+$

In this section, we describe $EG \text {-} PQE ^+$, an improved version of $EG \text {-} PQE$.

6.1 Main Idea

The pseudocode of $EG \text {-} PQE ^+$ is shown in Fig. 2. It is different from that of $EG \text {-} PQE$ only in line 11 marked with an asterisk. The motivation for this change is as follows. Line 11 describes proving redundancy of C for the case where $C_{\textbf{y}}$ is not implied by $(F \setminus \{C\})_{\textbf{y}}$ and $F_{\textbf{y}}$ is satisfiable. Then $EG \text {-} PQE$ simply uses a satisfying assignment as a proof of redundancy of C in subspace . This proof is unnecessarily strong because it proves that every clause of F (including C) is redundant in $\exists {X} [F]$ in subspace . Such a strong proof is hard to generalize to other subspaces.

Fig. 2.

Pseudocode of $EG \text {-} PQE ^+$

The idea of $EG \text {-} PQE ^+$ is to generate a proof for a much weaker proposition namely a proof of redundancy of C (and only C). Intuitively, such a proof should be easier to generalize. So, $EG \text {-} PQE ^+$ calls a procedure PrvClsRed generating such a proof. $EG \text {-} PQE ^+$ is a generic algorithm in the sense that any suitable procedure can be employed as PrvClsRed. In our current implementation, the procedure $DS$-$PQE$  [1] is used as PrvClsRed. $DS$-$PQE$ generates a proof stating that C is redundant in $\exists {X} [F]$ in subspace . Then the plugging clause D falsified by is generated. Importantly, can be much shorter than . (A brief description of $DS$-$PQE$ in the context of $EG \text {-} PQE ^+$ is given in [5].)

Example 3

Consider the example solved in Subsect. 5.1. That is, we consider taking clause $C_1$ out of $\exists {X} [F(X,Y)]$ where $F = C_1 \wedge \dots \wedge C_4$, $C_1=\overline{x}_3 \vee x_4$, $C_2\!=\!y_1\!\vee \!x_3$, $C_3=y_1 \vee \overline{x}_4$, $C_4\!=\!y_2\!\vee \!x_4$ and $Y=\{y_1,y_2\}$ and $X=\{x_3,x_4\}$. Consider the step where $EG \text {-} PQE$ proves redundancy of $C_1$ in subspace . $EG \text {-} PQE$ shows that $(y_1\!=\!1,y_2\!=\!1,\!x_3=0)$ satisfies F, thus proving every clause of F (including $C_1$) redundant in $\exists {X} [F]$ in subspace . Then $EG \text {-} PQE$ generates the plugging clause $D = \overline{y}_1 \vee \overline{y}_2$ falsified by .

In contrast to $EG \text {-} PQE$, $EG \text {-} PQE ^+$ calls PrvClsRed to produce a proof of redundancy for the clause $C_1$ alone. Note that F has no clauses resolvable with $C_1$ on $x_3$ in subspace . (The clause $C_2$ containing $x_3$ is satisfied by .) This means that $C_1$ is blocked in subspace and hence redundant there (see Proposition 2). Since , $EG \text {-} PQE ^+$ produces a more general proof of redundancy than $EG \text {-} PQE$. To avoid re-examining the subspace , $EG \text {-} PQE ^+$ generates a shorter plugging clause $D = \overline{y}_1$.

6.2 Discussion

Consider the PQE problem of taking a clause C out of $\exists {X} [F(X,Y)]$. There are two features of PQE that make it easier than QE. The first feature mentioned earlier is that one can ignore the subspaces where $F \setminus \{C\}$ implies C. The second feature is that when $F_{\textbf{y}}$ is satisfiable, one only needs to prove redundancy of the clause C alone. Among the three algorithms we run in experiments, namely, $DS$-$PQE$, $EG \text {-} PQE$ and $EG \text {-} PQE ^+$ only the latter exploits both features. (In addition to using $DS$-$PQE$ inside $EG \text {-} PQE ^+$ we also run it as a stand-alone PQE solver.) $DS$-$PQE$ does not use the first feature [1] and $EG \text {-} PQE$ does not exploit the second one. As we show in Sects. 7 and 8, this affects the performance of $DS$-$PQE$ and $EG \text {-} PQE$.

7 Experiment with FIFO Buffers

In this and the next two sections we describe some experiments with$DS$-$PQE$, $EG \text {-} PQE$ and $EG \text {-} PQE ^+$ (their sources are available at [13, 14] and [15] respectively). We used Minisat2.0 [16] as an internal SAT-solver. The experiments were run on a computer with Intel Core i5-8265U CPU of 1.6 GHz.

Fig. 3.

A buggy fragment of Verilog code describing $Fifo$

In this section, we give an example of bug detection by invariant generation for a FIFO buffer. Our objective here is threefold. First, we want to give an example of a bug that can be overlooked by testing and guessing the unwanted properties to check (see Subsect. 7.3). Second, we want to substantiate the intuition of Subsect. 3.4 that property generation by PQE (in our case, invariant generation by PQE) has the same reasons to be effective as testing. In particular, by taking out different clauses one generates invariants relating to different parts of the design. So, taking out a clause of the buggy part is likely to produce an unwanted invariant. Third, we want to give an example of an invariant that can be easily identified as unwanted².

7.1 Buffer Description

Consider a FIFO buffer that we will refer to as $Fifo$. Let n be the number of elements of $Fifo$ and $Data$ denote the data buffer of $Fifo$. Let each $Data [i],i=1,\dots ,n$ have p bits and be an integer where $0 \le Data [i] < 2^p$. A fragment of the Verilog code describing $Fifo$ is shown in Fig. 3. This fragment has a buggy line marked with an asterisk. In the correct version without the marked line, a new element $dataIn$ is added to $Data$ if the write flag is on and $Fifo$ has less than n elements. Since $Data$ can have any combination of numbers, all $Data$ states are supposed to be reachable. However, due to the bug, the number $Val$ cannot appear in $Data$. (Here $Val$ is some constant $0\!<\! Val \!<\! 2^p$. We assume that the buffer elements are initialized to 0.) So, $Fifo$ has an op-state reachability bug since it cannot reach operative states where an element of $Data$ equals $Val$.

7.2 Bug Detection by Invariant Generation

Let N be a circuit implementing $Fifo$. Let S be the set of state variables of N and $\boldsymbol{ S _{ data } \subset S}$ be the subset corresponding to the data buffer $Data$. We used $DS$-$PQE$, $EG \text {-} PQE$ and $EG \text {-} PQE ^+$ to generate invariants of N as described in Sect. 4. Note that an invariant Q depending only on $S _{ data }$ is an unwanted one. If Q holds for N, some states of $Data$ are unreachable. Then $Fifo$ has an op-state reachability bug since every state of $Data$ is supposed to be reachable. To generate invariants, we used the formula $F_k = I(S_0) \wedge T(S_0,V_0,S_1) \wedge \dots \wedge T(S_{k-1},V_{k-1},S_k)$ introduced in Subsect. 4.2. Here I and T describe the initial state and the transition relation of N respectively and $S_j$ and $V_j$ denote state variables and combinational input variables of j-th time frame respectively. First, we used a PQE solver to generate a local invariant $H(S_k)$ obtained by taking a clause C out of $\exists {X_{k}} [F_k]$ where $X_{k} = S_0 \cup V_0 \cup \dots \cup S_{k-1} \cup V_{k-1}$. So, $\exists {X_{k}} [F_k]\equiv$ $H \wedge$ $\exists {X_{k}} [F_k \setminus \{C\}]$. (Since $F_k \Rightarrow H$, no state falsifying H can be reached in k transitions.) In the experiment, we took out only clauses of $F_k$ containing an unquantified variable, i.e., a state variable of the k-th time frame. The time limit for solving the PQE problem of taking out a clause was set to 10 s.

Table 1. FIFO buffer with n elements of 32 bits. Time limit is 10 s per PQE problem

For each clause Q of every local invariant H generated by PQE, we checked if Q was a global invariant. Namely, we used a public version of IC3  [17, 18] to verify if the property Q held (by showing that no reachable state of N falsified Q). If so, and Q depended only on variables of $S _{ data }$, N had an unwanted invariant. Then we stopped invariant generation. The results of the experiment for buffers with 32-bit elements are given in Table 1. When picking a clause to take out, i.e., a clause with a state variable of k-th time frame, one could make a good choice by pure luck. To address this issue, we picked clauses to take out randomly and performed 10 different runs of invariant generation and then computed the average value. So, the columns four to twelve of Table 1 actually give the average value of 10 runs.

Let us use the first line of Table 1 to explain its structure. The first two columns show the number of elements in $Fifo$ implemented by N and the number of latches in N (8 and 300). The third column gives the number k of time frames (i.e., 5). The next three columns show the total number of PQE problems solved by a PQE solver before an unwanted invariant was generated. For instance, $EG \text {-} PQE ^+$ found such an invariant after solving 8 problems. On the other hand, $DS$-$PQE$ failed to find an unwanted invariant and had to solve all 1,236 PQE problems of taking out a clause of $F_k$ with an unquantified variable. The following three columns show the share of PQE problems finished in the time limit of 10 s. For instance, $EG \text {-} PQE$ finished 36% of 311 problems. The next three columns show if an unwanted invariant was generated by a PQE solver. ($EG \text {-} PQE$ and $EG \text {-} PQE ^+$ found one whereas $DS$-$PQE$ did not.) The last three columns give the total run time. Table 1 shows that only $EG \text {-} PQE ^+$ managed to generate an unwanted invariant for all four instances of $Fifo$. This invariant asserted that $Fifo$ cannot reach a state where an element of $Data$ equals $Val$.

7.3 Detection of the Bug by Conventional Methods

The bug above (or its modified version) can be overlooked by conventional methods. Consider, for instance, testing. It is hard to detect this bug by random tests because it is exposed only if one tries to add Val to $Fifo$. The same applies to testing using the line coverage metric [19]. On the other hand, a test set with 100% branch coverage [19] will find this bug. (To invoke the else branch of the if statement marked with ‘*’ in Fig. 3, one must set $dataIn$ to $Val$.) However, a slightly modified bug can be missed even by tests with 100% branch coverage [5].

Now consider, manual generation of unwanted properties. It is virtually impossible to guess an unwanted invariant of $Fifo$ exposing this bug unless one knows exactly what this bug is. However, one can detect this bug by checking a property asserting that the element $dataIn$ must appear in the buffer if $Fifo$ is ready to accept it. Note that this is a non-invariant property involving states of different time frames. The more time frames are used in such a property the more guesswork is required to pick it. Let us consider a modified bug. Suppose $Fifo$ does not reject the element $Val$. So, the non-invariant property above holds. However, if $dataIn == Val$, then $Fifo$ changes the previous accepted element if that element was $Val$ too. So, $Fifo$ cannot have two consecutive elements $Val$. Our method will detect this bug via generating an unwanted invariant falsified by states with consecutive elements $Val$. One can also identify this bug by checking a property involving two consecutive elements of $Fifo$. But picking it requires a lot of guesswork and so the modified bug can be easily overlooked.

8 Experiments with HWMCC Benchmarks

In this section, we describe three experiments with 98 multi-property benchmarks of the HWMCC-13 set [20]. (We use this set because it has a multi-property track, see the explanation below.) The number of latches in those benchmarks range from 111 to 8,000. More details about the choice of benchmarks and the experiments can be found in [5]. Each benchmark consists of a sequential circuit N and invariants $P_0,\dots ,P_m$ to prove. Like in Sect. 4, we call $P _{ agg }=P_0 \wedge \dots \wedge P_m$ the aggregate invariant. In experiments 2 and 3 we used PQE to generate new invariants of N. Since every invariant P implied by $P _{ agg }$ is a desired one, the necessary condition for P to be unwanted is $P _{ agg } \not \Rightarrow P$. The conjunction of many invariants $P_i$ produces a stronger invariant $P _{ agg }$, which makes it harder to generate P not implied by $P _{ agg }$. (This is the reason for using multi-property benchmarks in our experiments.) The circuits of the HWMCC-13 set are anonymous, so, we could not know if an unreachable state is supposed to be reachable. For that reason, we just generated invariants not implied by $P _{ agg }$ without deciding if some of them were unwanted.

Similarly to the experiment of Sect. 7, we used the formula $F_k = I(S_0) \wedge T(S_0,V_0,S_1)$ $\wedge \dots \wedge T(S_{k-1},V_{k-1},S_k)$ to generate invariants. The number k of time frames was in the range of $2\!\le \!k\! \le \!10$. As in the experiment of Sect. 7, we took out only clauses containing a state variable of the k-th time frame. In all experiments, the time limit for solving a PQE problem was set to 10 s.

8.1 Experiment 1

In the first experiment, we generated a local invariant H by taking out a clause C of $\exists {X_{k}} [F_k]$ where $X_{k}=S_0 \cup V_0 \cup \dots \cup S_{k-\!1}\cup V_{k-\!1}$. The formula H asserts that no state falsifying H can be reached in k transitions. Our goal was to show that PQE can find H for large formulas $F_k$ that have hundreds of thousands of clauses. We used $EG \text {-} PQE$ to partition the PQE problems we tried into two groups. The first group consisted of 3,736 problems for which we ran $EG \text {-} PQE$ with the time limit of 10 s and it never encountered a subspace where $F_k$ was satisfiable. Here is a full assignment to $S_k$. Recall that only the variables $S_k$ are unquantified in $\exists {X_{k}} [F_k]$. So, in every subspace , formula $F_k$ was either unsatisfiable or $(F_k \setminus \{C\}) \Rightarrow C$. (The fact that so many problems meet the condition of the first group came as a big surprise.) The second group consisted of 3,094 problems where $EG \text {-} PQE$ encountered subspaces where $F_k$ was satisfiable.

For the first group, $DS$-$PQE$ finished only 30% of the problems within 10 s whereas $EG \text {-} PQE$ and $EG \text {-} PQE ^+$ finished 88% and 89% respectively. The poor performance of $DS$-$PQE$ is due to not checking if $(F_k \setminus \{C\}) \Rightarrow C$ in the current subspace. For the second group, $DS$-$PQE$, $EG \text {-} PQE$ and $EG \text {-} PQE ^+$ finished 15%, 2% and 27% of the problems respectively within 10 s. $EG \text {-} PQE$ finished far fewer problems because it used a satisfying assignment as a proof of redundancy of C (see Subsect. 6.2).

To contrast PQE and QE, we employed a high-quality tool $CADET$  [21, 22] to perform QE on the 98 formulas $\exists {X_{k}} [F_k]$ (one formula per benchmark). That is, instead of taking a clause out of $\exists {X_{k}} [F_k]$ by PQE, we applied $CADET$ to perform full QE on this formula. (Performing QE on $\exists {X_{k}} [F_k]$ produces a formula $H(S_k)$ specifying all states unreachable in k transitions.) $CADET$ finished only 25% of the 98 QE problems with the time limit of 600 s. On the other hand, $EG \text {-} PQE ^+$ finished 60% of the 6,830 problems of both groups (generated off $\exists {X_{k}} [F_k]$) within 10 s. So, PQE can be much easier than QE if only a small part of the formula gets unquantified.

8.2 Experiment 2

The second experiment was an extension of the first one. Its goal was to show that PQE can generate invariants for realistic designs. For each clause Q of a local invariant H generated by PQE we used IC3 to verify if Q was a global invariant. If so, we checked if $P _{ agg } \not \Rightarrow Q$ held. To make the experiment less time consuming, in addition to the time limit of 10 s per PQE problem we imposed a few more constraints. The PQE problem of taking a clause out of $\exists {X_{k}} [F_k]$ terminated as soon as H accumulated 5 clauses or more. Besides, processing a benchmark aborted when the summary number of clauses of all formulas H generated for this benchmark reached 100 or the total run time of all PQE problems generated off $\exists {X_{k}} [F_k]$ exceeded 2,000 s.

Table 2. Invariant generation

Table 2 shows the results of the experiment. The third column gives the number of local single-clause invariants (i.e., the total number of clauses in all H over all benchmarks). The fourth column shows how many local single-clause invariants turned out to be global. (Since global invariants were extracted from H and the summary size of all H could not exceed 100, the number of global invariants per benchmark could not exceed 100.) The last column gives the number of global invariants not implied by $P _{ \!agg }$. So, these invariants are candidates for checking if they are unwanted. Table 2 shows that $EG \text {-} PQE$ and $EG \text {-} PQE ^+$ performed much better than $DS$-$PQE$.

8.3 Experiment 3

To prove an invariant P true, IC3 conjoins it with clauses $Q_1\!,\dots ,\!Q_n$ to make $P\!\wedge Q_1\!\wedge \dots \wedge Q_n$ inductive. If IC3 succeeds, every $Q_i$ is an invariant. Moreover, $Q_i$ may be an unwanted invariant. The goal of the third experiment was to demonstrate that PQE and IC3, in general, produce different invariant clauses. The intuition here is twofold. First, IC3 generates clauses $Q_i$ to prove a predefined invariant rather than find an unwanted one. Second, the closer P to being inductive, the fewer new invariant clauses are generated by IC3. Consider the circuit $N _{ triv }$ that simply stays in the initial state (Sect. 4). Any invariant satisfied by is already inductive for $N _{ triv }$. So, IC3 will not generate a single new invariant clause. On the other hand, if the correct circuit is supposed to leave the initial state, $N _{ triv }$ has unwanted invariants that our method will find.

In this experiment, we used IC3 to generate $P^*_{ agg }$, an inductive version of $P _{ agg }$. The experiment showed that in 88% cases, an invariant clause generated by $EG \text {-} PQE ^+$ and not implied by $P _{ agg }$ was not implied by $P^*_{ agg }$ either. (More details about this experiment can be found in [5].)

9 Properties Mimicking Symbolic Simulation

Let M(X, V, W) be a combinational circuit where X, V, W are internal, input and output variables. In this section, we describe generation of properties of M that mimic symbolic simulation [23]. Every such a property Q(V) specifies a cube of tests that produce the same values for a given subset of variables of W. We chose generation of such properties because deciding if Q is an unwanted property is, in general, simple. The procedure for generation of these properties is slightly different from the one presented in Sect. 3.

Let F(X, V, W) be a formula specifying M. Let B(W) be a clause. Let H(V) be a solution to the PQE problem of taking a clause $C \in F$ out of $\exists {X}\exists {W}[F \wedge B]$. That is, $\exists {X}\exists {W}[F \wedge B] \equiv H \wedge$ $\exists {X}\exists {W}[(F \setminus \{C\}) \wedge B]$. Let Q(V) be a clause of H. Then M has the property that for every full assignment to V falsifying Q, it produces an output falsifying B (a proof of this fact can be found in [5]). Suppose, for instance, $Q\!=\!v_1 \vee \, \overline{v}_{10} \vee v_{30}$ and $B\!=\!w_2 \vee \, \overline{w}_{40}$. Then for every where $v_1\!=\!0,v_{10}\!=\!1,\!v_{30}\!=\!0$, the circuit M produces an output where $w_2 = 0, w_{40}=1$. Note that Q is implied by $F \wedge B$ rather than F. So, it is a property of M under constraint B rather than M alone. The property Q is unwanted if there is an input falsifying Q that should not produce an output falsifying B.

To generate combinational circuits, we unfolded sequential circuits of the set of 98 benchmarks used in Sect. 8 for invariant generation. Let N be a sequential circuit. (We reuse the notation of Sect. 4). Let $M_k(S_0,V_0,\dots ,S_{k-1},V_{k-1},S_k)$ denote the combinational circuit obtained by unfolding N for k time frames. Here $S_j,V_j$ are state and input variables of j-th time frame respectively. Let $F_k$ denote the formula $I(S_0) \wedge T(S_0,V_0,S_1) \wedge \dots \wedge T(S_{k-1},V_{k-1},S_k)$ describing the unfolding of N for k time frames. Note that $F_k$ specifies the circuit $M_k$ above under the input constraint $I(S_0)$. Let $B(S_k)$ be a clause. Let $H(S_0,V_0,\dots ,V_{k-1})$ be a solution to the PQE problem of taking a clause $C \in F_k$ out of formula $\exists {S_{1,k}} [F_k \wedge B]$. Here $S_{1,k} = S_1 \cup \dots \cup S_k$. That is, $\exists {S_{1,k}} [F_k \wedge B] \equiv H \wedge$ $\exists {S_{1,k}} [(F_k \setminus \{C\}) \wedge B]$. Let Q be a clause of H. Then for every assignment ( ) falsifying Q, the circuit $M_k$ outputs falsifying B. (Here is the initial state of N and is a state of the last time frame.)

Table 3. Property generation for combinational circuits

In the experiment, we used $DS$-$PQE$,$EG \text {-} PQE$ and $EG \text {-} PQE ^+$ to solve 1,586 PQE problems described above. In Table 3, we give a sample of results by $EG \text {-} PQE ^+$. (More details about this experiment can be found in [5].) Below, we use the first line of Table 3 to explain its structure. The first column gives the benchmark name (6s326). The next column shows that 6s326 has 3,342 latches. The third column gives the number of time frames used to produce a combinational circuit $M_k$ (here $k=20$). The next column shows that the clause B introduced above consisted of 15 literals of variables from $S_k$. (Here and below we still use the index k assuming that $k = 20$.) The literals of B were generated randomly. When picking the length of B we just tried to simulate the situation where one wants to set a particular subset of output variables of $M_k$ to specified values. The next two columns give the size of the subcircuit $M'_k$ of $M_k$ that feeds the output variables present in B. When computing a property H we took a clause out of formula $\exists {S_{1,k}} [F'_k \wedge B]$ where $F'_k$ specifies $M'_k$ instead of formula $\exists {S_{1,k}} [F_k \wedge B]$ where $F_k$ specifies $M_k$. (The logic of $M_k$ not feeding a variable of B is irrelevant for computing H.) The first column of the pair gives the number of gates in $M'_k$ (i.e., 348,479). The second column provides the number of input variables feeding $M'_k$ (i.e., 1,774). Here we count only variables of $V_0 \cup \dots \cup V_{k-1}$ and ignore those of $S_0$ since the latter are already assigned values specifying the initial state  of N.

The next four columns show the results of taking a clause out of $\exists {S_{1,k}} [F'_k\!\wedge \!B]$. For each PQE problem the time limit was set to 10 s. Besides, $EG \text {-} PQE ^+$ terminated as soon as 5 clauses of property $H(S_0,V_0,\dots ,V_{k-1})$ were generated. The first three columns out of four describe the minimum and maximum sizes of clauses in H and the run time of $EG \text {-} PQE ^+$. So, it took for $EG \text {-} PQE ^+$ 2.9 s. to produce a formula H containing clauses of sizes from 27 to 28 variables. A clause Q of H with 27 variables, for instance, specifies $2^{1747}$ tests falsifying Q that produce the same output of $M'_k$ (falsifying the clause B). Here $1747 = 1774-27$ is the number of input variables of $M'_k$ not present in Q. The last column shows that at least one clause Q of H specifies a property that cannot be produced by 3-valued simulation (a version of symbolic simulation [23]). To prove this, one just needs to set the input variables of $M'_k$ present in Q to the values falsifying Q and run 3-valued simulation. (The remaining input variables of $M'_k$ are assigned a don’t-care value.) If after 3-valued simulation some output variable of $M'_k$ is assigned a don’t-care value, the property specified by Q cannot be produced by 3-valued simulation.

Running $DS$-$PQE$, $EG \text {-} PQE$ and $EG \text {-} PQE ^+$ on the 1,586 PQE problems mentioned above showed that a) $EG \text {-} PQE$ performed poorly producing properties only for 28% of problems; b) $DS$-$PQE$ and $EG \text {-} PQE ^+$ showed much better results by generating properties for 62% and 66% of problems respectively. When $DS$-$PQE$ and $EG \text {-} PQE ^+$ succeeded in producing properties, the latter could not be obtained by 3-valued simulation in 74% and 78% of cases respectively.

10 Some Background

In this section, we discuss some research relevant to PQE and property generation. Information on BDD based QE can be found in [24, 25]. SAT based QE is described in [12, 21, 26,27,28,29,30,31,32]. Our first PQE solver called $DS$-$PQE$ was introduced in [1]. It was based on redundancy based reasoning presented in [33] in terms of variables and in [34] in terms of clauses. The main flaw of $DS$-$PQE$ is as follows. Consider taking a clause C out of $\exists {X} [F]$. Suppose $DS$-$PQE$ proved C redundant in a subspace where F is satisfiable and some quantified variables are assigned. The problem is that $DS$-$PQE$ cannot simply assume that C is redundant every time it re-enters this subspace [35]. The root of the problem is that redundancy is a structural rather than semantic property. That is, redundancy of a clause in a formula $\xi$ (quantified or not) does not imply such redundancy in every formula logically equivalent to $\xi$. Since our current implementation of $EG \text {-} PQE ^+$ uses $DS$-$PQE$ as a subroutine, it has the same learning problem. We showed in [36] that this problem can be addressed by the machinery of certificate clauses. So, the performance of PQE can be drastically improved via enhanced learning in subspaces where F is satisfiable.

We are unaware of research on property generation for combinational circuits. As for invariants, the existing procedures typically generate some auxiliary desired invariants to prove a predefined property (whereas our goal is to generate invariants that are unwanted). For instance, they generate loop invariants [37] or invariants relating internal points of circuits checked for equivalence [38]. Another example of auxiliary invariants are clauses generated by IC3 to make an invariant inductive [17]. As we showed in Subsect. 8.3, the invariants produced by PQE are, in general, different from those built by IC3.

11 Conclusions and Directions for Future Research

We consider Partial Quantifier Elimination (PQE) on propositional CNF formulas with existential quantifiers. In contrast to complete quantifier elimination, PQE allows to unquantify a part of the formula. We show that PQE can be used to generate properties of combinational and sequential circuits. The goal of property generation is to check if a design has an unwanted property and thus is buggy. We used PQE to generate an unwanted invariant for a FIFO buffer exposing a non-trivial bug. We also applied PQE to invariant generation for HWMCC benchmarks. Finally, we used PQE to generate properties of combinational circuits mimicking symbolic simulation. Our experiments show that PQE can efficiently generate properties for realistic designs.

There are at least three directions for future research. The first direction is to improve the performance of PQE solving. As we mentioned in Sect. 10, the most promising idea here is to enhance the power of learning in subspaces where the formula is satisfiable. The second direction is to use the improved PQE solvers to design new, more efficient algorithms for well-known problems like SAT, model checking and equivalence checking. The third direction is to look for new problems that can be solved by PQE.