Abstract
In reactive synthesis, one begins with a temporal specification \(\varphi \), and automatically synthesizes a system \(M\) such that \(M\models \varphi \). As many systems can satisfy a given specification, it is natural to seek ways to force the synthesis tool to synthesize systems that are of a higher quality, in some welldefined sense. In this article we focus on a wellknown measure of the way in which a system satisfies its specification, namely vacuity. Our conjecture is that if the synthesized system M satisfies \(\varphi \) nonvacuously, then M is likely to be closer to the user’s intent, because it satisfies \(\varphi \) in a more “meaningful” way. Narrowing the gap between the formal specification and the designer’s intent in this way, automatically, is the topic of this article. Specifically, we propose a bounded synthesis method for achieving this goal. The notion of vacuity as defined in the context of model checking, however, is not necessarily refined enough for the purpose of synthesis. Hence, even when the synthesized system is technically nonvacuous, there are yet more interesting (equivalently, less vacuous) systems, and we would like to be able to synthesize them. To that end, we cope with the problem of synthesizing a system that is as nonvacuous as possible, given that the set of interesting behaviours with respect to a given specification induce a partial order on transition systems. On the theoretical side we show examples of specifications for which there is a single maximal element in the partial order (i.e., the most interesting system), a set of equivalent maximal elements, or a number of incomparable maximal elements. We also show examples of specifications that induce infinite chains of increasingly interesting systems. These results have implications on how nonvacuous the synthesized system can be. We implemented the new procedure in our synthesis tool PARTY. For this purpose we added to it the capability to synthesize a system based on a property which is a conjunction of universal and existential LTL formulas.
Introduction
Given a temporal specification \(\varphi \), the goal of reactive synthesis [9, 17] is to build a transition system M such that \(M \models \varphi \). The motivation of synthesis is clear: rather than building a design and then checking whether it adheres to the specification, focus on the specification alone, and generate automatically a design that satisfies it. In recent years, the theory and especially the tools for synthesis have made significant progress [11].
As many systems can satisfy a given specification, it is natural to seek ways to force the synthesis tool to synthesize systems that are of a higher quality, in some welldefined sense. In this article we focus on a wellknown measure of the way in which a system satisfies its specification, namely vacuity [2]. It is a standard concept in modelchecking, for detecting errors in the specification itself, the model, or both. So far it has not been used in the context of synthesis, however. Our conjecture is that if the synthesized system M satisfies \(\varphi \) nonvacuously, then M is likely to be closer to the user’s intent, because it satisfies \(\varphi \) in a more “meaningful” way. Narrowing the gap between the formal specification and the designer’s intent in this way, automatically, is the topic of this article.
Consider, for example, the property
A system M with one state satisfying grant (regardless of req) satisfies \(\varphi \), and is indeed a legitimate outcome of synthesising (1). However, M also satisfies stronger properties such as \(\text {{G}}\text {{F}}grant\). When a system satisfies a property regardless of some of its subformulas, as in this example where the behavior of req is immaterial for the satisfaction of \(\varphi \), the specification is said to be satisfied vacuously (see below a formal definition).
It is not likely that M captures the user’s intent: the intent is probably that the system also permits a path \(\pi \) in which there are no grants from a point in which there are no requests. Such a path is called an interesting witness [2]: it is an execution that demonstrates nonvacuous satisfaction of the original property. However, the requirement that there exists an interesting witness cannot be expressed in LTL, since it implicitly relies on an existential quantifier.
Previous work addressing the lack of expressibility of LTL in specifying highquality systems suggested the extension to quantitative specifications, in order to make it easier to specify certain properties [4], and to be able to synthesize systems that are robust against environment errors, even if the way to react to such errors has not been specified explicitly [3, 18].
There are multiple definitions of vacuity in the literature [1, 2, 6, 7, 14, 15]; the vast majority of them is based on the concept of strengthening the specification by changing a part of it and checking whether the strengthened version is still satisfied in the system; if not, then the system is considered to satisfy the specification nonvacuously. While the general method that we will describe in this paper is orthogonal to the chosen definition as long as the analysed formulas are still in LTL, for ease of exposition we will choose one definition and use it throughout the paper. Most commercially used vacuitydetection tools use the generalised definition by Kupferman and Vardi [14], which is what we will follow here: Let \(\psi \) be a subformula in \(\varphi \). The strengthening of \(\varphi \) with respect to \(\psi \) is \(\varphi [\psi \leftarrow \bot ]\).^{Footnote 1} If \(M \models \varphi [\psi \leftarrow \bot ]\) then \(\psi \) is irrelevant for the satisfaction of \(\varphi \) in M, and we say that \(\varphi \) is satisfied in M vacuously with respect to \(\psi \). It follows that M satisfies \(\varphi \) nonvacuously with respect to \(\psi \) iff \(M \models \text {{E}}\lnot \varphi [\psi \leftarrow \bot ]\). As shown in [14], it is sufficient to consider strengthenings of \(\varphi \) with respect to atomic propositions (literals, in fact) rather than all subformulas. We note that the definitions of vacuity in the literature, including [14], did not consider the division of the atomic propositions into inputs and outputs, as such division is immaterial in modelchecking. As we argue later, in synthesis this division is in fact important.
Our synthesis method requires systems with at least one interesting witness for every possible strengthening of \(\varphi \). More formally, if \(\varphi \) is a specification in LTL, a model M satisfies \(\varphi \) non vacuously if it satisfies a formula in a simple fragment of CTL\(^*\) consisting of a conjunction of universal and existential formulas:
where \(\text {Lit}\left( \varphi \right) \) denotes the literals of \(\varphi \). One of the contributions of this article is the extension of the bounded synthesis [10] algorithm to handle this fragment, based on a new ranking function (the original bounded synthesis algorithm handles only universal formulas).
Even when the system satisfies the specification nonvacuously, our tool is capable of improving it by synthesizing a system that has additional interesting witnesses. The users decides when the system reflects their intent. In Sect. 3 we define a partial order of vacuity on transition systems, stating that system \(M'\) is less vacuous than M if it contains all of the interesting witnesses permitted by M and at least one more. This condition can be stated as a formula in the same fragment of CTL\(^*\) mentioned above. We describe a synthesis procedure for generating increasingly nonvacuous systems for a given number of states.
In Sect. 4, we revisit the partial order of vacuity and prove that this partial order can be mapped to a partial order on subsets of input traces. This allows us to formally introduce the concept of equivalent systems w.r.t. their degree of vacuity and discuss cases where there is a single top element to the partial order of vacuity, multiple equivalent top elements, or multiple incomparable top elements. This is also the main addition to the paper compared to its conference version [5].
We have implemented the nonvacuous bounded synthesis algorithm on top of the PARTY synthesizer [13], which is available for download.^{Footnote 2} Given the informal goal we stated (“capturing the user’s intent”) naturally it is difficult to prove that our approach works, especially since there are no users in the industry that specify real system for the purpose of synthesis. Our experiments were based, then, on starting from previously published complete specifications, removing parts of them, and activating nonvacuous synthesis. In our experiments, which we describe in Sect. 5, the removed parts of the specification were compensated by our tool. In fact, the generated models not only satisfy the original, complete specifications, but they also realize them less vacuously.
Motivating example
The following example will be used in the rest of the article as a running example. It is a specification of an arbiter with two types of requests and two types of grants (i. e., \(\varphi _1\) and \(\varphi _2\)) and a mutual exclusion between the grants (i. e., \(\varphi _3\)). The specification \(\varphi \) is a conjunction of the following three properties:
where \(r_1\) and \(r_2\) are inputs (the ‘requests’) and \(g_1\) and \(g_2\) are outputs (the ‘grants’). The smallest system \(M_0\) satisfying \(\varphi \), synthesised by our tool, is depicted in Fig. 1a. It consists of two states, \(s_0\) and \(s_1\), where in each state exactly one of the grants is up. It is easy to see that \(M_0\) satisfies \(\varphi \) vacuously. In particular \(M_0 \models \varphi _1[r_1 \leftarrow \bot ]\) and \(M_0 \models \varphi _2[r_2 \leftarrow \bot ]\), where the \(\bot \) value for \(r_1\) and \(r_2\) is \({\text {{true}}}\) in both \(\varphi _1\) and \(\varphi _2\), respectively.
The system generated by our tool in the next step is \(M_1\), depicted in Fig. 1b.^{Footnote 3} This system satisfies \(\varphi \) nonvacuously in all its subformulas. Indeed:

1.
\(M_1 \not \models \varphi _1[r_1 \leftarrow \bot ]\), as the path \(\pi _1 = s_0^\omega \) corresponding to the output trace \(\{\lnot {g_1},g_2\}^\omega \) falsifies \(\text {{G}}\text {{F}}g_1\);

2.
\(M_1 \not \models \varphi _2[r_2 \leftarrow \bot ]\), as the path \(\pi _2 = s_0 s_1^\omega \) corresponding to the output trace \(\{\lnot {g_1},g_2\},\{g_1,\lnot {g_2}\}^\omega \) falsifies \(\text {{G}}\text {{F}}g_2\);

3.
The formulas obtained by replacing one of the grants with false are unrealisable, i. e., there is no system that can satisfy, for example, \(\text {{G}}(\lnot {r_1})\), because we have no control over the inputs.
\(\square \)
Preliminaries
Labeled transition systems
For the remainder of the paper, let us fix an input alphabet I and a disjoint output alphabet O, and let us define \(\text {AP}= I \cup O\), \(\Upsilon = \text {{P}}(I)\), \(\Sigma = \text {{P}}(O)\), and \(\Gamma = \text {{P}}(\text {AP})\).
Definition 1
(Labeled Transition System) A finite, \(\Sigma \)labeled \(\Upsilon \)transition system is a tuple \(M= (S, s_0, \tau , o)\), where S is nonempty set of states, \(s_0 \in S\) is the initial state, \(\tau : S \times \Upsilon \rightarrow S\) is a transition function, and \(o:S \rightarrow \Sigma \) is a labelling function.
Definition 2
(An LTS word) A word p of a labeled transition system M, is a sequence \((s_0,\upsilon _0, \sigma _0),(s_1,\upsilon _1,\sigma _1),\ldots \) in \((S \times \Upsilon \times \Sigma )^\omega \) such that \(\sigma _i = o(s_i)\) and \(s_{i+1} = \tau (s_i,\upsilon _i)\).
Definition 3
(Path) A path of a word p is the sequence of states induced by p.
Definition 4
(Trace) A trace of a word p is the sequence of input/output pairs induced by p.
We denote by \(\text {traces}\left( M\right) \) the set of all traces of \(M\). The projection of a trace \(\pi \in \text {traces}\left( M\right) \) on \(\Upsilon ^{\omega }\) (resp. \(\Sigma ^\omega \)) is an input (resp. output) trace denoted by \(\upsilon = \mathop {\left( \pi \downharpoonright \Upsilon \right) }\) (resp. \(\sigma = \mathop {\left( \pi \downharpoonright \Sigma \right) }\)). Similarly, for an input trace \(\upsilon \in \Upsilon ^{\omega }\), we denote by \(M(\upsilon )\) the (unique) trace \(\pi \in \text {traces}\left( M\right) \) s. t. \(\upsilon = \mathop {\left( \pi \downharpoonright \Upsilon \right) }\).
Definition 5
(Parallel Composition) Given input trace \(\upsilon \in \Upsilon ^\omega \) and output trace \(\sigma \in \Sigma ^\omega \), their parallel composition denoted by \(\left( \upsilon ~\Vert ~\sigma \right) \) is an infinite word \(\omega = \upsilon _0 \cup \sigma _0, \upsilon _1 \cup \sigma _1,\dots \) over \(\Gamma \).
Temporal logic
Throughout the paper, we denote by \(\varphi \) an LTL formula in negation normal form (NNF), over the set \(\text {AP}\) of atomic propositions [16]. The semantics of LTL is defined over \(\text {AP}\) with respect to infinite paths of \(M\) in a standard way. In this paper, we synthesize systems that satisfy the following simple fragment of CTL\(^*\):
where \(\varphi \) is an LTL formula. The semantics of the universal and existential quantifiers over LTL formulas are defined as expected:
So far we denoted by \(\text {traces}\left( M\right) \) the set of traces of M, where a trace corresponds to a run, which in itself must start from an initial state of M (see Definition 2 and 4). We similarly use \(\text {traces}\left( M,s\right) \) to refer to runs that begin at an arbitrary state \(s\in S\) of M.
Definition 6
For a state s of a transition system \(M\),
A transition system \(M\) satisfies a formula \(\varphi \), written \(M\models \varphi \), if all its initial states do.
Nondeterministic Büchi automata
An LTL formula can be represented by a nondeterministic Büchi automaton [19]: a tuple \(\mathcal {A}=(Q, q_0, \rho , \alpha )\), where Q is a finite set of states, \(q_0 \in Q\) is the initial state, \(\rho : Q \times \Upsilon \times \Sigma \rightarrow \text {{P}}(Q)\) is the transition relation, and \(\alpha \) is the set of accepting states; recall \(\Sigma \) and \(\Upsilon \) are defined in Sect. 2.1.
Definition 7
(Run of a Büchi automaton) Given an infinite word \(\omega = \upsilon _0\cup \sigma _0,\upsilon _1\cup \sigma _1,\dots \) over \(\Gamma \), The corresponding runs of a Büchi automaton \(\mathcal {A}\), denoted by \(\text {run}\left( w\right) \), are the infinite paths \(\pi =q_0,q_1, \dots \in Q^{\omega }\) where for all \(i \ge 0\), \(q_{i+1} \in \rho (q_i,\upsilon ^i,\sigma ^i)\).
Definition 8
(Accepting run) An accepting run of \(\mathcal {A}\) is a run that visits some accepting state infinitely often; a trace is accepted by \(\mathcal {A}\) if it has a corresponding accepting run, and the language of \(\mathcal {A}\) is the set of all accepted traces.
From this point forward, we denote by \(\mathcal {A}_\varphi \) the nondeterministic Büchi automata that accepts exactly the traces that satisfy \(\varphi \).
Vacuity detection
Informally speaking, a transition system \(M\) satisfies a property \(\varphi \) vacuously if not all parts of \(\varphi \) are instrumental for the satisfaction of \(\varphi \) in \(M\) (in other words, \(M\) satisfies \(\varphi \) in an uninteresting way). As proved in [14], it is sufficient to check vacuity with respect to atomic propositions of \(\varphi \), which, in case of an atomic proposition p appearing in \(\varphi \) in a pure polarity, amounts to modelchecking \(\varphi [p \leftarrow \bot ]\), that is, \(\varphi \) where p is replaced with its bottom value.
Hence, we use the following definition of vacuity that allows for efficient detection algorithm:
Definition 9
(Vacuity [2, 14]) A transition system \(M\) satisfies an LTL property \(\varphi \) vacuously iff \(M\models \varphi \) and there exists a literal \(\psi \) of \(\varphi \) which is of pure polarity and \( M\models \varphi [\psi \leftarrow \bot ]\).
The formula \(\varphi [\psi \leftarrow \bot ]\) is a strengthening of \(\varphi \) since \(\varphi [\psi \leftarrow \bot ] \implies \varphi \) and we call the negation \(\varphi _\psi = \lnot {\varphi [\psi \leftarrow \bot ]}\) of a strengthening a witness formula. A trace \(\pi \) of \(M\) that satisfies \(\varphi _\psi \) is called an interesting witness for \(\psi \), since it demonstrates that \(\psi \) is instrumental to the satisfaction of \(\varphi \) in \(M\); \(\pi \) is an interesting witness of \(M\) if it is an interesting witness for some subformula \(\psi \) of \(\varphi \). We note that if \(M\models \bigwedge _i \varphi _i\) then \(M\not \models \bigvee _i\varphi _i[\psi \leftarrow \bot ]\) is relevant to check only for those \(\varphi _i\) in which \(\psi \) appears.
The concept of witnesses and strengthenings is not restricted to Definition 9, and it lends itself, in theory, to other definitions of vacuity [1, 7, 8]. The framework proposed in this paper is orthogonal to the particular definition of vacuity, as long as the strengthenings are \(\omega \)regular.
Bounded synthesis
Bounded synthesis is a method to construct a finitestate labeled transition system that not only satisfies a given temporal specification \(\varphi \), but also fulfills a constraint on its size [10]. The idea is to let an SMT solver synthesize a transition system M (i.e., choose the transitions and the labeling of the states), such that \(M \times \mathcal {A}_{\lnot \varphi }\) has an empty language.
The synchronous product \(\mathcal {G}\) of a transition system \(M= (S, s_0, \tau , o)\) and a Büchi automaton \(\mathcal {A}_{\lnot \varphi }=(Q, q_0, \rho , \alpha )\) is called the run graph of \(\mathcal {A}_{\lnot \varphi }\) on \(M\).^{Footnote 4} The states of \(\mathcal {G}\) are annotated with two functions: a reachability function \(\lambda ^\mathbb {B}: Q \times S \rightarrow \mathbb {B}\) and a ranking function \(\lambda ^\#: Q \times S \rightarrow C\subset \mathbb {N}\), where \(C = \{0, \dots , Q\times S1\}\). Annotations of \(\mathcal {G}\) (i. e., \(\lambda ^\#\) and \(\lambda ^\mathbb {B}\) functions) are valid if they satisfy the following constraints. First, the initial state is reachable:
Second, the reachability predicate and the transition system are compatible:
Finally, the ranking function guarantees that the constraint is satisfiable only if the language of the run graph is empty: For accepting states, we require that the labelling on the target state is strictly larger than on the source (accepting) state:
and for nonaccepting states the labelling on the target states is larger or equal than on the source state:
The intuition behind the ranking function is as follows: if the language is not empty, then there is an accepting path (i.e., a lassoshaped path in the product automaton that includes an accepting state), and then it is impossible to satisfy these constraints over that path. This is because the ranks of states on the cycle cannot be strictly descending. The two automata in Fig. 2 illustrate this point—see caption. Hence, (5)–(8) are satisfiable if and only if the language of the product automaton is empty. The correctness of this construction was proven in [10].
Theorem 1
(Finkbeiner et al. [10]) Given a Büchi automaton \(\mathcal {A}=(Q, q_0, \rho , \alpha )\) constructed from \(\lnot \varphi \), transition system \(M=(S, s_0, \tau , o)\) satisfies \(\text {{A}}\varphi \) iff it corresponds to a solution to the constraints (5)–(8).
Initially, the LTL specification \(\varphi \) is negated and translated to a Büchi automaton \(\mathcal {A}_{\lnot \varphi }\). In the next step, (5)–(8) are solved with an SMT solver based on \(\mathcal {A}_{\lnot \varphi }\). Being unknown, \(\tau \), \(\lambda ^\mathbb {B}\), \(\lambda ^\#\) and o (the labeling function) are represented by uninterpreted functions; thus, the quest for finding \(M\) is reduced to the problem of satisfiability modulo finite integer arithmetic with uninterpreted functions, which is an NPcomplete problem.
Nonvacuous bounded synthesis
In this section we describe nonvacuous bounded synthesis—our method for constructing a finitestate labeled transition system that fulfils a constraint on its size and satisfies a given temporal specification nonvacuously.
A specification for nonvacuous satisfaction
A specification \(\varphi \) is satisfied nonvacuously in \(M\) if and only if \(M\) contains a witness for each strengthening of \(\varphi \). In other words, as we stated earlier in (2),
(note that (9) is based on our choice of definition for vacuity—see 9). We call \(\varphi _\psi = \lnot \varphi [\psi \leftarrow \bot ]\) the witness formulas for nonvacuity of \(\varphi \) with respect to \(\psi \).
Note that not all witness formulas add usable information. For instance, for \(\varphi \) as defined in (3), the witness formula \(\varphi _{g_1}\) (i. e., \(\lnot \varphi _1[g_1 \leftarrow \bot ] = \text {{F}}r_1\)) is clearly satisfied by a trace of any system, and the same holds for any satisfiable witness formula that contains only input signals.
We continue in the next subsection by showing how existentiallyquantified formulas can be synthesized. Then, we can use this technique to synthesise formulas of the form defined in (9).
Bounded synthesis for existential formulas
Our goal is to synthesize a finitestate labeled transition system with a bound on its size, in which there exists an execution path that satisfies a given temporal specification \(\varphi \). We will define a set of constraints that is different than the case described in Sect. 2.5 to achieve this. Initially, we translate \(\varphi \) to a nondeterministic Büchi automaton \(\mathcal {A}_\varphi \) and create the run graph \(\mathcal {G}\) of \(\mathcal {A}_\varphi \) on \(M\). Then, we use a Boolean marking function \(\lambda ^\mathbb {*}:Q \times S \rightarrow \mathbb {B}\) to indicate that a state is on our selected path in \(\mathcal {G}\). On that selected path, we impose a ranking function that can only be satisfied if it corresponds to an accepting run.
First, the initial state is marked:
Next, if a nonaccepting state is marked, then at least one of its successors is marked, and the ranking of the destination state is strictly smaller:
On the other hand if an accepting state is marked, then we only require that one of its successors is marked (but in contrast to the previous case, here there is no restriction on the ranking of its successor):
The two automata in Fig. 3 illustrate our construction—see caption. The following theorem states that these constraints are correct.
Theorem 2
Given a Büchi automaton \(\mathcal {A}=(Q, q_0, \rho , \alpha )\) constructed from a formula \(\varphi '\), a transition system \(M=(S, s_0, \tau , o)\) satisfies \(\text {{E}}\varphi '\) iff it corresponds to a solution to constraints (10)–(12).
Proof
\((\Rightarrow )\) There is a unique run graph \(\mathcal {G}=(G,E)\) for \(\mathcal {A}\) on \(M\). Assume \(M\) is accepted by \(\mathcal {A}\); therefore, \(\mathcal {G}\) contains at least one lassoshaped path \(\pi = (q_0,s_0)(q_1,s_1) \dots [(q_n,s_n)\dots (q_m,s_m)]^\omega \) such that \(q_i\) is accepting for some \(i\in [n,m]\). We have to show that in such a case (10)(12) are satisfiable. Marking all the states on the path clearly satisfies (10), and the \(\lambda ^\mathbb {*}\) predicate is true along this path as required by constraints (11) and (12). It is left to show that there exists a ranking function that satisfies (11). Indeed the following function, which annotates each state on \(\pi \) by its distance to \(q_i\), is a valid ranking function:
Indeed, \(\lambda ^\#(q_j,s_j)>\lambda ^\#(q_k,s_k)\) for all \(((q_j,s_j)(q_k,s_k))\in \pi \), unless \(j = i\). Recall that only nonaccepting states are bound by constraint (11). The figure below demonstrates this ranking for \(n=3,m=6\), and \(i=5\).
\((\Leftarrow )\) Assume that (10)–(12) are satisfiable. The set of marked states must include a lassoshaped path beginning from the initial state, and the fact that (11) is satisfied means that there exists an accepting state in the loop. Hence the run graph must contain an accepting path. \(\square \)
Finally, synthesising a nonvacuous system—a system that satisfies (2)—amounts to solving the conjunction of the constraints that were described in Sect. 2.5 (for the universal part), and the constraints in Sect. 3.2 for each \(\psi \in \text {Lit}\left( \varphi \right) \) (for the existential part). A separate discrete ranking function is required for \(\varphi \) and each of its witness formulas.
Corollary 1
A finitestate transition system \(M=(S, s_0, \tau , o)\) satisfies a temporal specification in the form of the CTL\({}^*\) fragment defined in (4) iff it corresponds to a solution to constraints (5)–(8) and (10)–(12).
We note that the addition of (10)–(12) to the bounded synthesis formula, does not change the asymptotic complexity of the problem.
Beyond vacuity
In the introduction we argued that nonvacuous systems are preferable to vacuous systems because they are more likely to fulfill the designer’s intent. This guarantees that for specifications like \(\varphi = \text {{G}}(r \rightarrow \text {{F}}g)\), there will be at least one path on which \(\text {{G}}\text {{F}}g\) does not hold. Intuitively, this corresponds to the idea that an input r should trigger the output g. However, the definition of vacuity is somewhat too coarse for our purpose. We need a more refined notion, which will enable us to distinguish between nonsystems that are nonvacuous. In the following subsection we introduce a partial order between systems realizing a given specification. We consider a system A strictly less vacuous (or, equivalently, more interesting) than another system B if the set of interesting witnesses in A properly contains the set of interesting witnesses in B. For the property above, for example, this corresponds to additional witnesses to \(\lnot \text {{G}}\text {{F}}g\), i.e., additional traces in which g does not occur without first being ‘triggered’ by r. For some specifications, there exist least vacuous systems, that is, maximal systems in the partial order of vacuity. In Sect. 4.2, however, we show that for some other specifications the partial order gives rise to infinite vacuity chains, i.e., infinite chains of ever less vacuous systems.
In Sect. 4.3 we show that given a system, we can use a variant of bounded synthesis to synthesize a less vacuous one, which naturally leads to a most interesting (leastvacuous) system of a given size, when such a system exists. Finally, in Sect. 4.4, we show that for our running example there exists such a leastvacuous system.
Partial order on nonvacuous systems
Let \(M_1\) and \(M_2\) be transition systems that satisfy \(\varphi \). Given a witness formula \(\varphi _\psi \), we define a relation \(M_1 \preccurlyeq _\psi M_2\) to indicate that \(M_2\) has at least the same set of interesting witnesses according to \(\varphi _\psi \) as \(M_1\). Formally, given a specification \(\varphi \) and a witness formula \(\varphi _\psi \) of \(\varphi \), we define
We say that \(M_2\) is strictly less vacuous than \(M_1\) if in addition there is at least one input sequence that leads to an interesting witness only in \(M_2\):
If \(M_1 \preccurlyeq _\psi M_2\) and \(M_2 \preccurlyeq _\psi M_1\), we say that \(M_1\) and \(M_2\) are equivalent in the partial order of vacuity with respect to \(\psi \), denoted \(M_1 \equiv _\psi M_2\).
By extending the relation \(\prec _\psi \) to the set of all witness formulas, we can compare two transition systems in terms of vacuity. Let \(\Psi \) be the set of all witness formulas for \(\varphi \). We define the partial order \(\preccurlyeq _\varphi \) as
and the strict partial order \(\prec _\varphi \) as
In other words, \(M_2\) is at least as nonvacuous as \(M_1\) w.r.t. all possible witnesses and is strictly less vacuous than \(M_1\) w.r.t. at least one witness formula. Similarly to the above, we say that \(M_1\) is equivalent to \(M_2\) in the partial order of vacuity, denoted \(M_1 \equiv _\varphi M_2\), if \(M_1 \preccurlyeq _\varphi M_2\) and \(M_2 \preccurlyeq _\varphi M_1\).
Since there is a finite number of transition systems of any size N, for a given LTL formula \(\varphi \), there exists at least one least vacuous system \(M^\varphi _N\) of size N, according to \(\prec \), assuming that \(\varphi \) is realizable by a system with N states. This system may not be unique. Moreover, if there are several leastvacuous systems of size N, they can be equivalent or incomparable.
It is easier to reason about equivalent and incomparable systems if we redefine the partial order on systems as follows. We can view the above partial order on systems realizing \(\varphi \) as a partial order on subsets of the (infinite) set of all input traces \(\mathcal{V} = \{ \upsilon : \upsilon \in \Upsilon ^\omega \}\). A system \(M\) realizing \(\varphi \) can be mapped to the set of input traces \(V(M,\varphi )\) that induce interesting witness traces in \(M\) with respect to some subformula of \(\varphi \). We can similarly denote by \(V(M,\varphi _\psi )\) the set of input traces that induce interesting witness traces in \(M\) with respect to a particular witness formula \(\varphi _\psi \). Then, it is easy to see that (13) is equivalent to
and the strict inequality holds if the set \(V(M_1,\varphi _\psi )\) is a strict subset of \(V(M_2,\varphi _\psi )\). Also, \(M_1 \equiv _\psi M_2\) if \(V(M_1,\varphi _\psi ) = V(M_2,\varphi _\psi )\).
Similarly, (15) can be rewritten as
and the strict inequality holds if the set \(V(M_1,\varphi _\psi )\) is a strict subset of \(V(M_2,\varphi _\psi )\) for some \(\varphi _\psi \in \Psi \). Two systems \(M_1\) and \(M_2\) realizing \(\varphi \) are equivalent with respect to \(\varphi \) (i.e., \(M_1 \equiv _\varphi M_2\)) if and only if the following holds:
Finally, if \(M_1 \not \preccurlyeq _\varphi M_2\) and \(M_2 \not \preccurlyeq _\varphi M_1\), we say that \(M_1\) and \(M_2\) are incomparable.
The following theorem clarifies the difference between incomparable and equivalent leastvacuous systems using (18) and (19).
Theorem 3
The following claims hold for any LTL formula \(\varphi \).

(a)
If there exists a system \(M_{max}\) realizing \(\varphi \) such that
$$\begin{aligned} \forall \varphi _\psi \in \Psi .\ V(M_{max},\varphi _\psi ) = \bigcup _{M\textit{ realizes } \varphi } V(M,\varphi _\psi ), \end{aligned}$$(20)then \(M_{max}\) is a leastvacuous system realizing \(\varphi \), and any other leastvacuous system \(M'\) realizing \(\varphi \) is equivalent to \(M_{max}\).

(b)
If \(\varphi \) is realizable and there is no system \(M_{max}\) that realizes \(\varphi \) and satisfies (20), then either

there is an infinite vacuity chain; or

there are several incomparable leastvacuous systems.

Proof
We prove the claims separately.

(a)
If there exists \(M_{max}\) realizing \(\varphi \) such that (20) holds, then by (18), for any other \(M\) realizing \(\varphi \), we have \(M\preccurlyeq _\varphi M_{max}\), hence \(M_{max}\) is a leastvacuous system realizing \(\varphi \). Now assume that there is another system \(M'\) that is a leastvacuous system realizing \(\varphi \). Then,
$$\begin{aligned} \forall \varphi _\psi \in \Psi .\ V(M', \varphi _\psi ) \subseteq \bigcup _{M\textit{ realizes } \varphi } V(M,\varphi _\psi ) \end{aligned}$$(21)(as \(M'\) is one of the systems in the union), hence \(M' \preccurlyeq _\varphi M_{max}\). Therefore, \(M'\) and \(M_{max}\) are not incomparable. Since \(M'\) is least vacuous, it also holds that \(M'\succeq M_{max}\), and hence they are equivalent in the vacuity preorder.

(b)
Assume that there is no system that satisfies (20) and let \(M_1\) be a leastvacuous system realizing \(\varphi \) (if there is no such \(M_1\), then there is an infinite vacuity chain, and the claim holds trivially). In particular, since \(M_1\) does not satisfy (20), we have
$$\begin{aligned} V(M_1, \varphi _\psi ) \subset \bigcup _{M\textit{ realizes } \varphi } V(M,\varphi _\psi ) \end{aligned}$$for at least one witness formula \(\varphi _\psi \) of \(\varphi \). Consider an input trace
$$\begin{aligned} \upsilon \in \Big ( \bigcup _{M\textit{ realizes } \varphi } V(M,\varphi _\psi ) \Big ) \setminus V(M_1, \varphi _\psi ). \end{aligned}$$Assume that there exists \(M_2\) realizing \(\varphi \) that contains an interesting witness trace for \(\varphi _\psi \) induced by \(\upsilon \). By construction, \(M_1\) and \(M_2\) are incomparable. Hence, either \(M_2\) is also a leastvacuous system, which is incomparable to \(M_1\) (which concludes the proof), or there exists another system \(M_3\) such that \(M_2 \prec _\varphi M_3\) and \(M_3\) is a leastvacuous system realizing \(\varphi \) (recall that we assumed that there are no infinite vacuity chains). Then, \(M_1\) and \(M_3\) are two incomparable leastvacuous systems realizing \(\varphi \).
\(\square \)
The following examples illustrate the notion of leastvacuous systems. Example 1 shows a specification for which there exist two equivalent leastvacuous systems. Example 2 and the example in “Appendix 1” show specifications for which there are two incomparable leastvacuous systems.
Example 1
[Equivalent leastvacuous systems] Consider the following specification:
where the sole interesting witness formula is \(\varphi _r = \text {{F}}\text {{G}}(\lnot g)\). Figure 4 demonstrates two systems of size 2 realizing \(\varphi \). It is easy to see that
that is, all nonvacuous traces of \(M_0\) are still nonvacuous in \(M_1\) and viceversa. In other words, \(M_0 \equiv _\varphi M_1\). Yet, the systems are not isomorphic. As an example of the difference between \(M_0\) and \(M_1\), consider the run of \(M_0\) on the input trace \((\lnot r)^\omega \):
On the other hand, the run of \(M_1\) on the same input trace is:
Proposition 1
\(M_0\) and \(M_1\) are leastvacuous systems realizing \(\varphi \).
Proof
As \(M_0 \equiv _\varphi M_1\), it suffices to prove that \(M_0\) is a leastvacuous system realizing \(\varphi \). Let \(M\) be an arbitrary system that realizes \(\varphi \). For an input sequence \(\upsilon \in \Upsilon ^{\omega }\), assume that \(\upsilon \) induces a path in \(M\) that satisfies \(\varphi _r = \text {{F}}\text {{G}}(\lnot g)\). Since this path, in particular, satisfies \(\varphi \), it also satisfies \(\text {{F}}\text {{G}}( \lnot r)\) (otherwise there would have been requests that are never granted). Observing Fig. 4, it is easy to see that the same input sequence \(\upsilon \) would induce a path in \(M_0\) with an infinite suffix \(\{s_0\}^\omega \), hence, in particular, it satisfies \(\text {{F}}\text {{G}}(\lnot g)\). Hence, \(M\) is not less vacuous than \(M_0\). \(\square \)
The following is an example of two incomparable leastvacuous systems, illustrating the second case in 3(b).
Example 2
[Incomparable leastvacuous systems] Consider the specification \(\varphi \), which is the conjunction of the following three properties:
where r is an input, and \(g_1\) and \(g_2\) are outputs. The interesting witness formulas for \(\varphi \) are
In any system that realizes \(\varphi \), satisfying the witness formula \(\text {{X}}(\lnot g_1)\) implies that the second state must be labelled with \(\lnot g_1 \wedge g_2\) if \(\lnot r\) is its input (because of \(\varphi _3\)). At the same time in order to satisfy \(\text {{X}}(\lnot g_2)\) the second state of this system must be labelled with \(g_1\wedge \lnot g_2\), which clearly contradicts the previous requirement.
On the other hand, leastvacuous systems for \(\varphi \) exist. For example, consider the two systems in Fig. 5: the left satisfies \(\text {{E}}\text {{X}}(\lnot g_1)\) and the right satisfies \(\text {{E}}\text {{X}}(\lnot g_2)\), but neither satisfies both. Furthermore, it is not hard to see that both \(M_1\) and \(M_2\) are leastvacuous with respect to \(\varphi \) (the proof would be similar to the proof of Proposition 1).
\(\square \)
Infinite vacuity chains
For some formulas, there is an infinite chain of ever less vacuous (and ever larger) systems. As an example, consider the following LTL specification:
The only useful witness formula for \(\varphi \) is
Figure 6 depicts an abstract transition system \(M_k\) of arbitrary size (i. e., \(k+3\)) that realizes specification \(\varphi \) nonvacuously for any k.
Proposition 2
\(\forall k.\ M_k \prec _\psi M_{k+1}\).
Proof
We have to show that \(M_{k+1}\) is as nonvacuous as \(M_k\) and that there exists an input trace that makes \(M_{k+1}\) less vacuous w.r.t. \(\varphi _r\).
First we show \(\forall k.\ M_k \preccurlyeq _\psi M_{k+1}\). For each input trace \(\pi \in \Upsilon ^\omega \), if \(M_k(\pi ) \models \text {{G}}\lnot g\), then \(\pi \models r^j(\lnot r)^+(\lnot r + r)^\omega \) for some \(j\le k\), so \(M_{k+1}(\pi ) \models \text {{G}}\lnot g\).
To see that \(\forall k.\ M_k \prec _\psi M_{k+1}\) holds, note that the input trace \(r^{k+1}(\lnot r)^\omega \) leads to an interesting trace in \(M_{k+1}\) but not in \(M_k\). \(\square \)
Synthesizing a less vacuous system
We now discuss how to synthesize a less vacuous system \(M_2\) given a correct system \(M_1\). We do this by expressing the partial order defined above in the simple fragment of CTL\({}^*\)defined in (4).
Given a formula \(\varphi \) or a system M, we use a primed version (\(\varphi '\) or \(M'\), respectively) to denote the formula/system obtained by replacing all output literals by primed versions. Given a system \(M_1\) that satisfies \(\varphi \), we have \(M_1 \prec _\psi M_2\) iff
Equation (25) follows directly from Eq. (14) after renaming the variables of \(M_1\) with their primed versions. Note that \(\varphi \) and \(\varphi _\psi \) refer to the outputs of \(M_2\) and \(\varphi '_{\psi }\) refers to the outputs of \(M_1\), while both systems receive the same inputs. The following theorem generalizes (25) to all subformulas of \(\varphi \).
Theorem 4
\(M_1\) is strictly less vacuous than \(M_2\) iff
Note that this equation has the form of Eq. (2) and can thus be solved as described in Sect. 3.
We iteratively synthesize less and less vacuous systems by applying (26) where \(M_2\) is the existing system. The result is \(M_1'\), which then becomes \(M_2\) of the next iteration. We repeat this process until (26) becomes unrealizable with the current bound on the number of states. We can then either stop or increase the bound. Every such step adds a linear number of variables. Since the asymptotic complexity has an exponential dependence on the number of variables, eventually this formula may become too hard to solve in practice.
A leastvacuous system for the running example
Consider once again our running example from the introduction. Figure 7 shows a leastvacuous system \(M_2\) with the bound 4 on the number of states (one of the intermediate iterations resulted in \(M_1\) depicted in Fig. 7).
System \(M_2\) is strictly less vacuous than \(M_1\). Recall that the two witness formulas are \(\varphi _{r_1} = \text {{F}}\text {{G}}\lnot g_1\) and \(\varphi _{r_2} = \text {{F}}\text {{G}}\lnot g_2\). It is not hard to verify that all interesting paths in \(M_1\) w.r.t. to \(\varphi _{r_1}\) (w.r.t. to \(\varphi _{r_2}\)) are also interesting w.r.t. to \(\varphi _{r_1}\) (w.r.t. to \(\varphi _{r_2}\), resp.) in \(M_2\). Also, the trace that results from leaving \(r_1\) and \(r_2\) low all the time is interesting w.r.t. \(\varphi _{r_2}\) in \(M_2\) but not in \(M_1\).
Proposition 3
\(M_2\) is a leastvacuous system with respect to \(\{\varphi _{r_1}, \varphi _{r_2}\}\).
Proof
Let \(M\) be an arbitrary system that realizes \(\varphi \). For an input sequence \(\upsilon \in \Upsilon ^{\omega }\), assume that \(\upsilon \) induces a path in \(M\) that satisfies \(\varphi _1[r_1 \leftarrow \bot ] = \text {{F}}\text {{G}}\lnot g_1\). Since this path, in particular, satisfies \(\varphi \), it also satisfies \(\text {{F}}\text {{G}}\lnot r_1\) (otherwise there would have been requests that are never granted). Observing Fig. 7, it is easy to see that the same input sequence \(\upsilon \) would induce a path in \(M_2\) with an infinite suffix \({\{s_0,s_2\}}^\omega \), hence, in particular, it satisfies \(\text {{F}}\text {{G}}\lnot g_1\). A similar argument holds for \(\varphi _2[r_2 \leftarrow \bot ]\). Hence, \(M\) is not less vacuous than \(M_2\). \(\square \)
The question whether a given system is a leastvacuous one (again, such systems may not be unique) is equivalent to asking whether a less vacuous one exists, which, by (26) can be reduced to a CTL\({}^*\)realizability question.
Experimental evaluation
We implemented the described technique in the PARTY synthesizer [13]. We cannot check directly whether nonvacuous synthesis leads to a system which is closer to ‘the user intent’. However, we can check something with a similar flavor: the ability of this technique to guess missing parts of a specification. Hence, we conducted the following experiment: first, we synthesized models for three complete and correct specifications; then, we made them incomplete by removing some of the conjuncts in the specification; finally, we ran nonvacuous synthesis on these incomplete specifications, and checked whether the resulting system satisfies the original, complete specification; A positive answer indicates that nonvacuous synthesis can accelerate the convergence towards the desired system.
Indeed, in the three experiments that we conducted, nonvacuous synthesis was able to synthesize a system that satisfies the original, full specification, although we emphasize that this is not guaranteed in general. The synthesized system in all three cases is not identical to the one synthesized according to the full specification, which reflects the fact that many systems can satisfy the same specification. It is up to the user to choose between them.
More details about the experimental setting, as well as a performance comparison, is given in 1. It is evident from the comparison that, as expected, on average nonvacuous bounded synthesis takes more time than bounded synthesis.
A ‘next’ arbiter
The ‘next’ arbiter of two clients issues a grant for each client in the next step if and only if the client sends a request. The assumption is that clients never send requests simultaneously; thus, issued grants should be mutually exclusive. The complete and incomplete specification of this arbiter for two clients is shown in Fig. 8. The specification should be interpreted as ‘every run that satisfies the \(\text {{assume}}\) predicates should also satisfy the \(\text {{guarantee}}\) predicates’.
As depicted in Fig. 9a, b, even a slight modification in the specification results in a large gap in the behaviors of the synthesized systems. On the other hand starting from the system depicted in Fig. 9b, three iterations of the nonvacuous synthesis process result in the system shown in Fig. 9c, which satisfies the original, full specification. Figure 10 depicts the runtime breakdown of bounded synthesis and nonvacuous bounded synthesis for both the complete and incomplete versions of the ‘next’ arbiter.
A ‘full’ arbiter
A ‘full’ arbiter of two clients eventually issues a grant for each client if the client sends a request. The complete specification appears in Fig. 11 (left), and a partial specification appears in Fig. 11 (right). The properties that are removed in the partial specification state that grants are never given “unnecessarily”.
The transition systems that are synthesized for the full and partial specification appear in Fig. 12a, b respectively. On the other hand, starting from the partial specification, after four iterations of the nonvacuous synthesis the system we get is as shown in Fig. 12c, which again satisfies the full specification. Figure 10 depicts the runtime breakdown of bounded synthesis and nonvacuous bounded synthesis for both the complete and incomplete versions of the ‘full’ arbiter (Fig. 13).
A ‘Pnueli’ arbiter
A ‘Pnueli’ arbiter of two clients is a handshake mechanism such that whenever a client sets a request the arbiter will set and keep the corresponding grant high as long as the request is high [12]. The complete and incomplete specification of a ‘Pnueli’ arbiter of two clients is shown in Fig. 14. The incomplete specification allows the arbiter to set a grant and never unset it; therefore, the synthesized system may issue vacuous grants for each client infinitely often unless the other client sends a request—see Fig. 15b.
The result of our nonvacuous synthesis from the partial specification again satisfies the full specification, as shown in Fig. 15c, and is synthesized in one step. This system also satisfies the specification in a less vacuous way than the system synthesized from the complete specification using the previous version of PARTY, without the new functionality of nonvacuity. Figure 10 depicts runtime breakdown of both synthesis methods for complete and incomplete versions of the ‘Pneuli’ arbiter (Fig. 16).
Conclusion
In synthesis, it is hard to expect the designer to think of a complete specification. As a result, the large range of possible systems that satisfy the specification permits designs that stand in contrast to the designer’s intent. We proposed in this article to apply the concept of vacuity to address this problem. Our method narrows down the range of legitimate synthesised system to those that satisfy the (partial) specification in a meaningful way, a wellknown concept from using vacuity in modelchecking. But as we argued, we do not have to commit to the Boolean nature of the classical definition of vacuity: we showed how a system can be made less vacuous, even if it already satisfies the specification nonvacuously. Our experiments showed that our method is capable of synthesising better designs, in the sense that they even satisfy parts of the specification that we deliberately removed and were hence inaccessible to the synthesis algorithm. Perhaps in the future synthesis will be used in the industry, and then our conjecture that this process can save time to the designer will be tested with a userstudy.
Our solution is based on a novel bounded synthesis technique that combines universal and existential properties; It paves the way for generalizing our technique to full CTL\(^*\). Our tool PARTY is available on the web to try and improve.
Notes
 1.
This means that we swap \(\psi \) with falseif \(\psi \) is in positive polarity, and with trueotherwise. Hence, e.g., if \(\varphi \equiv \psi _1 \Rightarrow \Psi _2\), then \(\varphi [\psi _1 \leftarrow \bot ] \equiv \psi _2\).
 2.
 3.
Note that these are Moore machines. As such the output of a transition leaving a state appears in the state itself. Hence, for example, in the trace \(\{g_2, r_1\}, \{g_1,r_1,r_2\},\{g_2\}^{\omega }\), the request \(r_1\) on the outgoing edge of \(s_1\) is granted by the label \(g_1\) in \(s_1\).
 4.
Since \(\mathcal {G}\) is only used for checking emptiness, the labels are immaterial, and it is customary to use a oneletter automaton (i. e., \(\vert \Sigma \vert = \vert \Upsilon \vert = 1\)).
References
 1.
Armoni R, Fix L, Flaisher A, Grumberg O, Piterman N, Tiemeyer A, Vardi MY (2003) Enhanced vacuity detection in linear temporal logic. In: CAV, LNCS, Springer, vol 2725, pp 368–380
 2.
Beer I, BenDavid S, Eisner C, Rodeh Y (2001) Efficient detection of vacuity in ACTL formulas. Formal Methods Syst Des 18(2):279–290
 3.
Bloem R, Chatterjee K, Greimel K, Henzinger TA, Hofferek G, Jobstmann B, Könighofer B, Könighofer R (2014) Synthesizing robust systems. Acta Inf 51:193–220
 4.
Bloem R, Chatterjee K, Henzinger TA, Jobstmann B (2009) Better quality in synthesis through quantitative objectives. In: CAV, LNCS, Springer, vol 5643, pp 140–156
 5.
Bloem R, Chockler H, Ebrahimi M, Strichman O (2017) Synthesizing nonvacuous systems. In: Proceedings of the 18th international conference in verification, model checking, and abstract interpretation (VMCAI), Lecture notes in computer science, Springer, vol 10145, pp 55–72
 6.
Bustan D, Flaisher A, Grumberg O, Kupferman O, Vardi M (2005) Regular vacuity. In: CHARME, LNCS, Springer, vol 3725, pp 191–206
 7.
Chechik M, Gurfinkel A (2004) Extending extended vacuity. In: FMCAD, LNCS, vol 3312
 8.
Chockler H, Gurfinkel A, Strichman O (2013) Beyond vacuity: towards the strongest passing formula. Formal Methods Syst Des 43(3):1–8
 9.
Church A (1963) Logic, arithmetics, and automata. In: ICM, institut MittagLeffler, pp 23–35
 10.
Finkbeiner B, Schewe S (2012) Bounded synthesis. Int J Softw Tools Technol Transf 15(5):519–539
 11.
Jacobs S, Bloem R, Brenguier R, Könighofer R, Pérez GA, Raskin J, Ryzhyk L, Sankur O, Seidl M, Tentrup L, Walker A (2015) The second reactive synthesis competition. In: SYNT
 12.
Jobstmann B, Staber S, Griesmayer A, Bloem R (2012) Finding and fixing faults. J Comput Syst Sci 78(2):35–49
 13.
Khalimov A, Jacobs S, Bloem R (2013) PARTY parameterized synthesis of token rings. In: CAV, pp 928–933
 14.
Kupferman O, Vardi M (2003) Vacuity detection in temporal model checking. J Softw Tools Technol Transf 4(2):224–233
 15.
Namjoshi KS (2004) An efficiently checkable, proofbased formulation of vacuity in model checking. In: CAV, LNCS, Springer, vol 3114, pp 57–69
 16.
Pnueli A (1977) The temporal logic of programs. In: FOCS, pp 46–57
 17.
Pnueli A, Rosner R (1989) On the synthesis of a reactive module. In: POPL, Austin, pp 179–190
 18.
Samanta R, Deshmukh JV, Chaudhuri S (2013) Robustness analysis of networked systems. In: VMCAI, pp 229–247
 19.
Vardi M, Wolper P (1994) Reasoning about infinite computations. Inf Comput 115(1):1–37
Acknowledgements
We would like to thank Nir Piterman for his insights on infinite chains of ever less vacuous systems and Ayrat Khalimov for his comments on existential bounded synthesis and his valuable assistance with the implementation.
Funding
Open access funding provided by Graz University of Technology.
Author information
Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work was supported by the Austrian Research Promotion Agency (FFG) through project TRUSTED (867558), Graz University of Technology’s LEAD project “Dependable Internet of Things in Adverse Environments”.
Appendices
A: An example of incomparable leastvacuous systems
Consider the specification \(\varphi \) that is a conjunction of the following four properties:
where \(r_i\) is an input and \(g_i\) is an output. The following interesting witness formulas
give rise to incomparable leastvacuous systems as described in Fig. 17. Indeed, the witness formula \(\text {{F}}\text {{G}}(\lnot g_1)\) (resp. \(\text {{F}}\text {{G}}(\lnot g_2)\)) requires a trace that satisfies it to have a suffix \(\lnot g_1^\omega \) (resp. \(\lnot g_2^\omega \)), hence in order to satisfy the property \(\varphi _2\) (resp. \(\varphi _4\)), the same suffix should satisfy \(\text {{X}}(g_2^\omega )\) (resp. \(\text {{X}}(g_1^\omega )\)). Thus, each trace either satisfies the interesting witness \(\text {{F}}\text {{G}}(\lnot g_1)\) or \(\text {{F}}\text {{G}}(\lnot g_2)\), but not both, hence only one of the two sets of witness traces can be maximized. The proof of the least vacuity is similar to the proofs of Propositions 1 and 3.
B: Performance comparison
Here we compare the performance of nonvacuous bounded synthesis to ‘normal’ bounded synthesis, using a benchmark set of 164 specifications. The set includes the LTLReal benchmarks of synthcomp 2020, lily, genbuf, acacia+, and partyelli. Our implementation uses Z3 4.8.10 and SPOT 2.9. It is worth mentioning that our implementation uses integer encoding of cobüchi automata for both methods.
We ran each experiment twice and each time for a total of 60 minutes. In both runs, 70 experiments timed out, 85 were realizable, and 9 were unrealizable. We averaged the runtime profile of both runs and computed the performance loss in making the synthesis nonvacuous.
Of all realizable experiments the nonvacuous bounded synthesis gained negligible performance in three cases. Meanwhile, nonvacuous bounded synthesis timed out on 3 experiments that were realizable through bounded synthesis. Table 1 gives a more detailed comparison.
The biggest factor in the performance loss is the solver time. This is evident in the plots shown in Figs. 18 and 19. Other than the solver time, much of the runtime is spent on translating each specification to a cobüchi automaton, which is shared by both synthesis methods; thus, the total performance loss is typically less than the solver performance loss.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Bloem, R., Chockler, H., Ebrahimi, M. et al. Vacuity in synthesis. Form Methods Syst Des 57, 473–495 (2021). https://doi.org/10.1007/s10703021003815
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10703021003815
Keywords
 Reactive synthesis
 Vacuity
 Bounded synthesis