Satisfiability Checking for MissionTime LTL
 2 Citations
 4k Downloads
Abstract
Missiontime LTL (MLTL) is a bounded variant of MTL over naturals designed to generically specify requirements for missionbased system operation common to aircraft, spacecraft, vehicles, and robots. Despite the utility of MLTL as a specification logic, major gaps remain in analyzing MLTL, e.g., for specification debugging or model checking, centering on the absence of any complete MLTL satisfiability checker. We prove that the MLTL satisfiability checking problem is NEXPTIMEcomplete and that satisfiability checking Open image in new window , the variant of MLTL where all intervals start at 0, is PSPACEcomplete. We introduce translations for MLTLtoLTL, Open image in new window , MLTLtoSMV, and MLTLtoSMT, creating four options for MLTL satisfiability checking. Our extensive experimental evaluation shows that the MLTLtoSMT transition with the Z3 SMT solver offers the most scalable performance.
1 Introduction
Missiontime LTL (MLTL) [34] has the syntax of Linear Temporal Logic with the option of integer bounds on the temporal operators. It was created as a generalization of the variations [3, 14, 25] on finitelybounded linear temporal logic, ideal for specification of missions carried out by aircraft, spacecraft, rovers, and other vehicular or robotic systems. MLTL provides the readability of LTL [32], while assuming, when a different duration is not specified, that all requirements must be upheld during the (a priori known) length of a given mission, such as during the halfhour battery life of an Unmanned Aerial System (UAS). Using integer bounds instead of realnumber or realtime bounds leads to more generic specifications that are adaptable to model checking at different levels of abstraction, or runtime monitoring on different platforms (e.g., in software vs in hardware). Integer bounds should be read as generic time units, referring to the basic temporal resolution of the system, which can generically be resolved to units such as clock ticks or seconds depending on the mission. Integer bounds also allow generic specification with respect to different granularities of time, e.g., to allow easy updates to modelchecking models, and reusable specifications for the same requirements on different embedded systems that may have different resource limits for storing runtime monitors. MLTL has been used in many industrial case studies [18, 28, 34, 37, 42, 43, 44], and was the official logic of the 2018 Runtime Verification Benchmark Competition [1]. Many specifications from other case studies, in logics such as MTL [3] and STL [25], can be represented in MLTL. We intuitively relate MLTL to LTL and MTLovernaturals as follows: (1) MLTL formulas are LTL formulas with bounded intervals over temporal operators, and interpreted over finite traces. (2) MLTL formulas are MTLovernaturals formulas without any unbounded intervals, and interpreted over finite traces.
Despite the practical utility of MLTL, no model checker currently accepts this logic as a specification language. The model checker nuXmv encodes a related logic for use in symbolic model checking, where the \(\Box \) and \(\Diamond \) operators of an Open image in new window can have integer bounds [21], though bounds cannot be placed on the \(\mathcal {U}\) or \(\mathcal {V}\) (the Release operator of nuXmv) operators.
We also critically need an MLTL satisfiability checker to enable specification debugging. Specification is a major bottleneck to the formal verification of missionbased, especially autonomous, systems [35], with a key part of the problem being the availability of good tools for specification debugging. Satisfiability checking is an integral tool for specification debugging: [38, 39] argued that for every requirement \(\varphi \) we need to check \(\varphi \) and \(\lnot \varphi \) for satisfiability; we also need to check the conjunction of all requirements to ensure that they can all be true of the same system at the same time. Specification debugging is essential to model checking [39, 40, 41] because a positive answer may not mean there is no bug and a negative answer may not mean there is a bug if the specification is valid/unsatisfiable, respectively. Specification debugging is critical for synthesis and runtime verification (RV) since in these cases there is no model; synthesis and RV are both entirely dependent on the specification. For synthesis, satisfiability checking is the bestavailable specificationdebugging technique, since other techniques, such as vacuity checking (cf. [6, 10]) reference a model in addition to the specification. While there are artifacts one can use in RV, specification debugging is still limited outside of satisfiability checking yet central to correct analysis. A false positive due to RV of an incorrect specification can have disastrous consequences, such as triggering an abort of an (otherwise successful) mission to Mars. Arguably, the biggest challenge to creating an RV algorithm or tool is the dearth of benchmarks for checking correctness or comparatively analyzing these [36], where a benchmark consists of some runtime trace, a temporal logic formula reasoning about that trace, and some verdict designating whether the trace at a given time satisfies the requirement formula. A MLTL satisfiability solver is useful for RV benchmark generation [22].
Despite the critical need for an MLTL satisfiability solver, no such tool currently exists. To the best of our knowledge, there is only one available solver (zot [8]) for checking the satisfiability of MTLovernaturals formulas, interpreted over infinite traces. Since MLTL formulas are interpreted over finite traces and there is no trivial reduction from one to another, zot cannot be directly applied to MLTL satisfiability checking.
Our approach is inspired by satisfiabilitychecking algorithms from other logics. For LTL satisfiability solving, we observe that there are multiple efficient translations from LTL satisfiability to model checking, using nuXmv [40]; we therefore consider here translations to nuXmv model checking, both indirectly (as a translation to LTL), and directly using the new KLIVE [13] backend and the BMC backend, taking advantage of the bounded nature of MLTL. The bounded nature of MLTL enables us to also consider a direct encoding at the wordlevel, suitable as input to an SMT solver. Our contribution is both theoretic and experimental. We first consider the complexity of such translations. We prove that the MLTL satisfiability checking problem is NEXPTIMEcomplete and that satisfiability checking Open image in new window , the variant of MLTL where all intervals start at 0, is PSPACEcomplete. Secondly, we introduce translation algorithms for Open image in new window (LTL over finite traces [14]), MLTLtoLTL, MLTLtoSMV, and MLTLtoSMT, thus creating four options for MLTL satisfiability checking. Our results show that the MLTLtoSMT transition with the Z3 SMT solver offers the most scalable performance, though the MLTLtoSMV translation with an SMV model checker can offer the best performance when the intervals in the MLTL formulas are restricted to small ranges less than 100.
2 Preliminaries
Given two MLTL formulas \(\varphi , \psi \), we denote \(\varphi = \psi \) iff they are syntactically equivalent, and \(\varphi \equiv \psi \) iff they are semantically equivalent, i.e., \(\pi \,\models \,\varphi \) iff \(\pi \,\models \,\psi \) for a finite trace \(\pi \). In MLTL semantics, we define \(\mathsf{false}\equiv \lnot \mathsf{true}\), \(\varphi \vee \psi \equiv \lnot (\lnot \varphi \wedge \lnot \psi )\), \(\lnot (\varphi \ \mathcal {U}_I \ \psi )\equiv (\lnot \varphi \mathcal {R}_I\lnot \psi )\) and \(\lnot \Diamond _I\varphi \equiv \Box _I\lnot \varphi \). MLTL keeps the standard operator equivalences from LTL, including \((\Diamond _I \varphi ) \equiv (true \ \mathcal {U}_I \varphi )\), \((\Box _I \varphi ) \equiv (false \ \mathcal {R}_I \ \varphi )\), and \((\varphi \ \mathcal {R}_I \ \psi ) \equiv (\lnot (\lnot \varphi \ \mathcal {U}_I \ \lnot \psi ))\). Notably, MLTL discards the neXt (\(\mathcal {X}\)) operator, which is essential in LTL [32], since \(\mathcal {X}\varphi \) is semantically equivalent to \(\Box _{[1,1]}\varphi \).

\(\pi \,\models \,p\) iff \(p\in \pi [0]\);

\(\pi \,\models \, \lnot \varphi \) iff Open image in new window ;

\(\pi \,\models \,\varphi \wedge \psi \) iff \(\pi \,\models \,\varphi \) and \(\pi \,\models \,\psi \);

\(\pi \,\models \, \varphi \ \mathcal {U}_{[a,b]} \ \psi \) iff \(\pi > a\) and, there exists \(i\in [a,b]\), \(i<\pi \) such that \(\pi _i\,\models \,\psi \) and for every \(j\in [a,b], j<i\) it holds that \(\pi _j\,\models \,\varphi \);
Compared to the traditional MTLovernaturals^{1} [16], the Until formula in MLTL is interpreted in a slightly different way. In MTLovernaturals, the satisfaction of \(\varphi \ \mathcal {U}_I \ \psi \) requires \(\varphi \) to hold from position 0 to the position where \(\psi \) holds (in I), while in MLTL \(\varphi \) is only required to hold within the interval I, before the time \(\psi \) holds. From the perspective of writing specifications, cf. [34, 37], this adjustment is more userfriendly. It is not hard to see that MLTL is as expressive as the standard MTLovernaturals: the formula \(\varphi \ \mathcal {U}_{[a, b]} \ \psi \) in MTLovernaturals can be represented as \((\Box _{[0,a1]}\varphi )\wedge (\varphi \ \mathcal {U}_{[a, b]}\ \psi )\) in MLTL; \(\varphi \ \mathcal {U}_{[a, b]} \ \psi \) in MLTL can be represented as \(\Diamond _{[a, a]}(\varphi \ \mathcal {U}_{[0, ba]} \ \psi )\) in MTLovernaturals.
We say an MLTL formula is in BNF if the formula contains only \(\lnot \), \(\wedge \) and \(\mathcal {U}_I\) operators. It is trivial to see that every MLTL formula can be converted to its (semantically) equivalent BNF with a linear cost. Consider \(\varphi = (\lnot a)\vee ((\lnot b) \mathcal {R}_I (\lnot c))\) as an example. Its BNF form is \(\lnot (a\wedge (b \ \mathcal {U}_I \ c))\). Without explicit clarification, this paper assumes that every MLTL formula is in BNF.
The closure of an MLTL formula \(\varphi \), denoted as \(cl(\varphi )\), is a set of formulas such that: (1) \(\varphi \in cl(\varphi )\); (2) \(\varphi \in cl(\varphi )\) if \(\lnot \varphi \in cl(\varphi )\); (3) \(\varphi ,\psi \in cl(\varphi )\) if \(\varphi \ op\ \psi \in cl(\varphi )\), where op can be \(\wedge \) or \(\mathcal {U}_I\). Let \(cl(\varphi )\) be the size of \(cl(\varphi )\). Since the definition of \(cl(\varphi )\) ignores the intervals in \(\varphi \), \(cl(\varphi )\) is linear in the number of operators in \(\varphi \). We also define the closure(*) of an MLTL formula \(\varphi \), denoted \(cl^*(\varphi )\), as the set of formulas such that: (1) \(cl(\varphi )\subseteq cl^*(\varphi )\); (2) if \(\varphi \ \mathcal {U}_{[a, b]}\ \psi \in cl^*(\varphi )\) for \(0<a\le b\), then \(\varphi \ \mathcal {U}_{[a1, b1]}\ \psi \) is in \(cl^*(\varphi )\); (3) if \(\varphi \ \mathcal {U}_{[0, b]}\ \psi \in cl^*(\varphi )\) for \(0<b\), then \(\varphi \ \mathcal {U}_{[0, b1]}\ \psi \) is in \(cl^*(\varphi )\). Let \(cl^*(\varphi )\) be the size of \(cl^*(\varphi )\) and K be the maximal natural number in the intervals of \(\varphi \). It is not hard to see that \(cl^*(\varphi )\) is at most \(K\cdot cl(\varphi )\).
We also consider a fragment of MLTL, namely Open image in new window , which is more frequently used in practice, cf. [18, 34]. Informally speaking, Open image in new window formulas are MLTL formulas in which all intervals start from 0. For example, \(\Diamond _{[0,4]}a\wedge (a \ \mathcal {U}_{[0,1]} \ b)\) is a Open image in new window formula, while \(\Diamond _{[2,4]}a\) is not.
Given an MLTL formula \(\varphi \), the satisfiability problem asks whether there is a finite trace \(\pi \) such that \(\pi \,\models \,\varphi \) holds. To solve this problem, we can reduce it to the satisfiability problem of the related logics LTL and Open image in new window (LTL over finite traces [14]), and leverage the offtheshelf satisfiability checking solvers for these wellexplored logics. We abbreviate MLTL, LTL, and Open image in new window satisfiability checking as MLTLSAT, LTLSAT, and Open image in new window respectively.

\(t(p) = p\) where p is an atom;

\(t(\lnot \psi ) = \lnot t(\psi )\);

\(t(\mathcal {X}\psi ) = \lnot Tail\wedge \mathcal {X}t(\psi )\);

\(t(\psi _1\wedge \psi _2) = t(\psi _1)\wedge t(\psi _2)\);

\(t(\psi _1 U\psi _2) = t(\lnot Tail\wedge \psi _1) U t(\psi _2)\).
In the above reduction, \(\varphi \) is in BNF. Since the reduction is linear in the size of the original Open image in new window formula and LTLSAT is PSPACEcomplete [45], Open image in new window is also a PSPACEcomplete problem [14].
3 Complexity of MLTLSAT
It is known that the complexity of MITL (Metric Interval Temporal Logic) satisfiability is EXPSPACEcomplete, and the satisfiability complexity of the fragment of MITL named Open image in new window is PSPACEcomplete [2]. MLTL (resp. Open image in new window ) can be viewed as a variant of MITL (resp. Open image in new window ) that is interpreted over the naturals. We show that MLTL satisfiability checking is NEXPTIMEcomplete, via a reduction from MLTL to Open image in new window .
Lemma 1
Let \(\varphi \) be an MLTL formula, and K be the maximal natural appearing in the intervals of \(\varphi \) (K is set to 1 if there are no intervals in \(\varphi \)). There is an Open image in new window formula \(\theta \) that recognizes the same language as \(\varphi \). Moreover, the size of \(\theta \) is in \(O(K\cdot cl(\varphi ))\).
Proof
(Sketch). For an MLTL formula \(\varphi \), we define the Open image in new window formula \(f(\varphi )\) recursively as follows:

If \(\varphi = \mathsf{true}, \mathsf{false}\), or an atom p, \(f(\varphi ) = \varphi \);

If \(\varphi = \lnot \psi \), \(f(\varphi ) = \lnot f(\psi )\);

If \(\varphi = \xi \wedge \psi \), \(f(\varphi ) = f(\xi )\wedge f(\psi )\);

If \(\varphi = \xi \ \mathcal {U}_{[a,b]} \ \psi \),
We use the construction shown in Lemma 1 to explore several useful properties of MLTL. For instance, the Open image in new window formula translated from an MLTL formula contains only the \(\mathcal {X}\) temporal operator or its dual \(\mathcal {N}\), which represents weak Next [19, 23], and the number of these operators is strictly smaller than \(K\cdot cl(\varphi )\). Every \(\mathcal {X}\) or \(\mathcal {N}\) subformula in the Open image in new window formula corresponds to some temporal formula in \(cl^*(\varphi )\). Notably, because the naturalnumber intervals in \(\varphi \) are written in base 10 (decimal) notation, the blowup in the translation of Lemma 1 is exponential.
The next lower bound is reminiscent of the NEXPTIMElower bound shown in [31] for a fragment of Metric Interval Temporal Logic (MITL), but is different in the details of the proof as the two logics are quite different.
Theorem 1
The complexity of \(\mathsf{MLTL} \) satisfiability checking is NEXPTIMEcomplete.
Proof
(Sketch). By Lemma 1, there is an Open image in new window formula \(\theta \) that accepts the same traces as MLTL formula \(\varphi \), and the size of \(\theta \) is in \(O(K\cdot cl(\varphi ))\). The only temporal connectives used in \(\theta \) are \(\mathcal {X}\) and \(\mathcal {N}\), since the translation to Open image in new window reduces all MLTL temporal connectives in \(\varphi \) to nested \(\mathcal {X}\)’s or \(\mathcal {N}\)’s (produced by simplifying \(\lnot \mathcal {X}\)). Thus, if \(\theta \) is satisfiable, then it is satisfiable by a trace whose length is bounded by the length of \(\theta \). Thus, we can just guess a trace \(\pi \) of exponential length of \(\theta \) and check that it satisfies \(\varphi \). As a result, the upper bound for MLTLSAT is NEXPTIME.
Before proving the NEXPTIME lower bound, recall the PSPACElower bound proof in [45] for LTL satisfiability. The proof reduces the acceptance problem for a linearspace bounded Turing machine M to LTL satisfiability. Given a Turing machine M and an integer k, we construct a formula \(\varphi _M\) such that \(\varphi _M\) is satisfiable iff M accepts the empty tape using k tape cells. The argument is that we can encode such a spacebounded computation of M by a trace \(\pi \) of length \(c^k\) for some constant c, and then use \(\varphi _M\) to force \(\pi \) to encode an accepting computation of M. The formula \(\varphi _M\) has to match corresponding points in successive configurations of M, which can be expressed using a O(k)nested \(\mathcal {X}\)’s, since such points are O(k) points apart.
To prove a NEXPTIMElower bound for MLTL, we reduce the acceptance problem for exponentially bounded nondeterministic Turing machines to MLTL satisfiability. Given a nondeterministic Turing machine M and an integer k, we construct an MLTL formula \(\varphi _M\) of length O(k) such that \(\varphi _M\) is satisfiable iff M accepts the empty tape in time \(2^k\). Note that such a computation of a \(2^k\)time bounded Turing machines consists of \(2^k\) many configurations of length \(2^k\) each, so the whole computation is of exponential length – \(4^k\), and can be encoded by a trace \(\pi \) of length \(4^k\), where every point of \(\pi \) encodes one cell in the computation of M. Unlike the reduction in [45], in the encoding here corresponding points in successive configurations are exponentially far (\(2^k\)) from each other, because each configuration has \(2^k\) cells, so the relationship between such successive points cannot be expressed in LTL. Because, however, the constants in the intervals of MLTL are written in base10 (decimal) notation, we can write formulas of size O(k), e.g., formulas of the form \(p \ \mathcal {U}_{[0,{2^k}]} \ q\), that relate points that are \(2^k\) apart.
The key is to express the fact that one Turing machine configuration is a proper successor of another configuration using a formula of size O(k). In the PSPACElowerbound proof of [45], LTL formulas of size O(k) relate successive configurations of kspacebounded machines. Here MLTL formulas of size O(k) relate successive configurations of \(2^k\)timebounded machines. Thus, we can write a formula \(\varphi _M\) of length O(k) that forces trace \(\pi \) to encode a computation of M of length \(2^k\). \(\square \)
Now we consider Open image in new window formulas, and prove that the complexity of checking the satisfiability of Open image in new window formulas is PSPACEcomplete. We first introduce the following lemma to show an inherent feature of Open image in new window formulas.
Lemma 2
The conjunction of identical Open image in new window \(\mathcal {U}\)rooted formulas is equivalent to the conjunct with the smallest interval range: \((\xi \ \mathcal {U}_{[0, a]} \ \psi ) \wedge (\xi \ \mathcal {U}_{[0, b]} \ \psi ) \equiv (\xi \ \mathcal {U}_{[0, a]} \ \psi ) \), where \(b > a\).
Proof
Lemma 3
\(\mathcal {X}\)free Open image in new window is reducible to Open image in new window at a linear cost.
Proof

\(m (p) = p\) where p is an atom;

\(m(\lnot \xi ) = \lnot m(\xi )\);

\(m(\xi \wedge \psi ) = m(\xi ) \wedge m(\psi )\);

\(m(\xi \ \mathcal {U}\ \psi ) = m (\xi ) \ \mathcal {U}_{[0,2^{\varphi }]} \ m(\psi )\).
Notably for the Until Open image in new window formula, we bound it with the interval \([0, 2^{\varphi }]\), where \(\varphi \) is the original \(\mathcal {X}\)free Open image in new window formula, in the corresponding MLTL formula, which is motivated by the fact that every satisfiable Open image in new window formula has a finite model whose length is less than \(2^{\varphi }\) [14]. The above translation has linear blowup, because the integers in intervals use the decimal notation. Now we prove by induction over the type of \(\varphi \) that \(\varphi \) is satisfiable iff \(m(\varphi )\) is satisfiable. That is, we prove that (\(\Rightarrow \)) \(\pi \,\models \,\varphi \) implies \(\pi \,\models \, m(\varphi )\) and (\(\Leftarrow \)) \(\pi \,\models \, m(\varphi )\) implies \(\pi \,\models \,\varphi \), for some finite trace \(\pi \).
We consider the Until formula \(\eta =\xi \ \mathcal {U}\ \psi \) (noting that \(\varphi \) is fixed to the original Open image in new window formula), and the proofs are trivial for other types. (\(\Rightarrow \)) \(\eta \) is satisfiable implies there is a finite trace \(\pi \) such that \(\pi \,\models \,\eta \) and \(\pi \le 2^{\varphi }\) [14]. Moreover, \(\pi \,\models \,\eta \) holds iff there is \(0\le i\) such that \(\pi _i\,\models \,\psi \) and for every \(0\le j<i\), \(\pi _j\,\models \,\xi \) is true (from Open image in new window semantics). By the induction hypothesis, \(\pi _i \,\models \,\psi \) implies \(\pi _i\,\models \, m(\psi )\) and \(\pi _j\,\models \,\xi \) implies \(\pi _j\,\models \, m(\xi )\). Also, \(i\le 2^{\varphi }\) is true because of \(\pi \le 2^{\varphi }\). As a result, \(\pi \,\models \,\eta \) implies that there is \(0\le i\le 2^{\varphi }\) such that \(\pi _i\,\models \, m(\psi )\) and for every \(0\le j<i\), \(\pi _j\,\models \, m(\xi )\) is true. According to the MLTL semantics, \(\pi \,\models \, m(\eta )\) is true. (\(\Leftarrow \)) \(m (\eta )\) is satisfiable implies there is a finite trace \(\pi \) such that \(\pi \,\models \, m(\eta )\). According to MLTL semantics, there is \(0\le i\le 2^{\varphi }\) such that \(\pi _i\,\models \, m(\psi )\) and for every \(0\le j < i\) it holds that \(\pi _j\,\models \, m(\xi )\). By hypothesis assumption, \(\pi _i\,\models \, m(\psi )\) implies \(\pi _i\,\models \,\psi \) and \(\pi _j\,\models \, m(\xi )\) implies \(\pi _j\,\models \, \xi \). Also, \(0\le i\le 2^{\varphi }\) implies \(0\le i\). As a result, \(\pi \,\models \, m(\eta )\) implies that there is \(0\le i\) such that \(\pi _i\,\models \, \psi \) and for every \(0\le j < i\) it holds that \(\pi _j\,\models \, \xi \). From Open image in new window semantics, it is true that \(\pi \,\models \,\eta \). \(\square \)
Theorem 2
The complexity of checking the satisfiability of Open image in new window is PSPACEcomplete.
Proof
Since Lemma 3 shows a linear reduction from \(\mathcal {X}\)free Open image in new window to Open image in new window and \(\mathcal {X}\)free Open image in new window is PSPACEcomplete [14], it directly implies that the lower bound of Open image in new window is PSPACEhard.
For the upper bound, recall from the proof of Theorem 1 that an \(\mathsf{MLTL} \) formula \(\varphi \) is translated to an Open image in new window formula \(\theta \) of length \(K\cdot cl(\varphi )\), which, as we commented, involved an exponential blowup in the notation for K. Following the automatatheoretic approach for satisfiability, one would translate \(\theta \) to an NFA and check its nonemptiness [14]. Normally, such a translation would involve another exponential blowup. We show that this is not the case for Open image in new window . Recalling from the automaton construction in [14] that every state of the automaton is a set of subformulas of \(\theta \), the size of a state is at most \(K\cdot cl(\varphi )\). In the general case, if \(\psi _1,\psi _2\) are two subformulas of \(\theta \) corresponding to the MLTL formulas \(\xi \ \mathcal {U}_{I_1} \ \psi \) and \(\xi \ \mathcal {U}_{I_2} \ \psi \), \(\psi _1\) and \(\psi _2\) can be in the same state of the automaton, which implies that the size of the state can be at most \(K\cdot cl(\varphi )\). When the formula \(\varphi \) is restricted to Open image in new window , we show that the exponential blowup can be avoided. Lemma 2 shows that either \(\psi _1\) or \(\psi _2\) in the state is enough, since assuming \(I_1\subseteq I_2\), then \((\psi _1\wedge \psi _2)\equiv \psi _1\), by Lemma 2. So the size of the state in the automaton for a Open image in new window formula \(\varphi \) is at most \(cl(\varphi )\). For each subformula in the state, there can be K possible values (e.g., for \(\Diamond _I \xi \) in the state, we can have \(\Diamond _{[0, 1]}\xi \), \(\Diamond _{[0,2]}\xi \), etc.). Therefore the size of the automaton is in \(O(2^{cl(\varphi )}\cdot K^{cl(\varphi )})\approx 2^{O(cl(\varphi ))}\). Therefore, Open image in new window satisfiability checking is a PSPACEcomplete problem. \(\square \)
4 Implementation of MLTLSAT
We first show how to reduce MLTLSAT to the wellexplored Open image in new window and LTLSAT. Then we introduce two new satisfiabilitychecking strategies based on the inherent properties of MLTL formulas, which are able to leverage the stateofart modelchecking and SMTsolving techniques.
4.1 MLTLSAT via Logic Translation
For a formula \(\varphi \) from one logic, and \(\psi \) from another logic, we say \(\varphi \) and \(\psi \) are equisatisfiable when \(\varphi \) is satisfiable under its semantics iff \(\psi \) is satisfiable under its semantics. Based on Lemma 1 and Theorem 1, we have the following corollary,
Corollary 1
(MLTLSAT to Open image in new window ). MLTLSAT can be reduced to Open image in new window with an exponential blowup.
From Corollary 1, MLTLSAT is reducible to Open image in new window , enabling use of the offtheshelf Open image in new window satisfiability solvers, cf. aaltaf [23]. It is also straightforward to consider MLTLSAT via LTLSAT; LTLSAT has been studied for more than a decade, and there many offtheshelf LTL solvers are available, cf. [24, 38, 40].
Theorem 3
(MLTL to LTL). For an MLTL formula \(\varphi \), there is an LTL formula \(\theta \) such that \(\varphi \) and \(\theta \) are equisatisfiable, and the size of \(\theta \) is in \(O(K\cdot cl(\varphi ))\), where K is the maximal integer in \(\varphi \).
Proof
Lemma 1 provides a translation from the MLTL formula \(\varphi \) to the equivalent Open image in new window formula \(\varphi '\), with a blowup of \(O(K\cdot cl(\varphi ))\). As shown in Sect. 2, there is a linear translation from the Open image in new window formula \(\varphi '\) to its equisatisfiable LTL formula \(\theta \) [14]. Therefore, the blowup from \(\varphi \) to \(\theta \) is in \(O(K\cdot cl(\varphi ))\). \(\square \)
Corollary 2
(MLTLSAT to LTLSAT). MLTLSAT can be reduced to LTLSAT with an exponential blowup.
Since MLTLSAT is reducible to LTLSAT, MLTLSAT can also benefit from the power of LTL satisfiability solvers. Moreover, the reduction from MLTLSAT to LTLSAT enables leveraging modern modelchecking techniques to solve the MLTLSAT problem, due to the fact that LTLSAT has been shown to be reducible to model checking with a linear blowup [38, 39].
Corollary 3
(MLTLSAT to Open image in new window MLTLSAT can be reduced to LTL model checking with an exponential blowup.
In our implementation, we choose the model checker nuXmv [12] for LTL satisfiability checking, as it allows an LTL formula to be directly input as the temporal specification together with a universal model as described in [38, 39].
4.2 Model Generation
Using the LTL formula as the temporal specification in nuXmv has been shown, however, to not be the most efficient way to use model checking for satisfiability checking [40]. Consider the MLTL formula \(\Diamond _{[0,10]}a \wedge \Diamond _{[1,11]}a\). The translated Open image in new window formula is \(f(\Diamond _{[0,10]}a) \wedge \mathcal {X}(f(\Diamond _{[0,10]}a))\), where \(f(\Diamond _{[0,10]}a)\) has to be constructed twice. To avoid such redundant construction, we follow [40] and encode directly the input MLTL formula as an SMV model (the input model of nuXmv) rather than treating the LTL formula, which is obtained from the input MLTL formula, as a specification.
An SMV [27] model consists of a Boolean transition system \(Sys=(V, I, T)\), where V is a set of Boolean variables, I is a Boolean formula representing the initial states of Sys, and T is the Boolean transition formula. Moreover, a specification to be verified against the system is also contained in the SMV model (here we focus on the LTL specification). Given the input MLTL formula \(\varphi \), we construct the corresponding SMV model \(M_{\varphi }\) as follows.

Introduce a Boolean variable for each atom in \(\varphi \) as well as for “Tail” (new variable identifying the end of a finite trace).

Introduce a Boolean variable \(\mathcal {X}\_\psi \) for each \(\mathcal {U}\) formula \(\psi \) in \(cl^*(\varphi )\), which represents the intermediate temporal formula \(\mathcal {X}\psi \).

Introduce a temporary Boolean variable^{2} \(T\_\psi \) for each \(\mathcal {U}\) formula in \(cl^*(\varphi )\).

A Boolean formula \(e(\psi )\) is used to represent the formula \(\psi \) in \(cl^*(\varphi )\) in the SMV model, which is defined recursively as follows.
 1.
\(e(\psi ) = \psi \), if \(\psi \) is an Boolean atom;
 2.
\(e (\psi ) = \lnot e (\psi _1)\), if \(\psi =\lnot \psi _1\);
 3.
\(e (\psi ) = e (\psi _1)\wedge e (\psi _2)\), if \(\psi = \psi _1\wedge \psi _2\);
 4.
\(e (\psi ) = T\_\psi \), if \(\psi \) is an \(\mathcal {U}\) formula.
 1.

Let the initial Boolean formula of the system Sys be \(e(\varphi )\).
 For each temporary variable \(T\_\psi \), create a DEFINE statement according to the type and interval of \(\psi \), as follows.$$ T_{\psi _1 \mathcal {U}_{[a, b]}\psi _2} = {\left\{ \begin{array}{ll} \mathcal {X}\_(\psi _1 \mathcal {U}_{[a1, b1]}\psi _2), \ \ \text {if } 0< a\le b;\\ e(\psi _2) \vee (e(\psi _1)\wedge \mathcal {X}\_(\psi _1 \mathcal {U}_{[0, b1]}\psi _2)), \text {if } a = 0 \text { and } 0 < b;\\ e(\psi _2), \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \text {if } a = 0 \text { and } b = 0.\\ \end{array}\right. } $$

Create the Boolean formula \((\mathcal {X}\_\psi \leftrightarrow (\lnot Tail\wedge next(e(\psi ))))\) for each \(\mathcal {X}\_\psi \) in the VAR list (the set V in Sys) of the SMV model.

Finally, designate the LTL formula \(\Box \lnot Tail\) as the temporal specification of the SMV model \(M_{\varphi }\) (which implies that a counterexample trace satisfies \(\Diamond Tail\)).
Theorem 4
The MLTL formula \(\varphi \) is satisfiable iff the corresponding SMV model \(M_{\varphi }\) violates the LTL property \(\Box \lnot Tail\).
There are different techniques that can be used for LTL model checking. Based on the latest evaluation of LTL satisfiability checking [24], the KLIVE [13] backend implemented in the SMV model checker nuXmv [12] produces the best performance. We thus choose KLIVE as our modelchecking technique for MLTLSAT.
Bounded MLTLSAT. Although MLTLSAT is reducible to the satisfiability problem of other wellexplored logics, with established offtheshelf satisfiability solvers, a dedicated solution based on inherent properties of MLTL may be superior. One intuition is, since all intervals in MLTL formulas are bounded, the satisfiability of the formula can be reduced to Bounded Model Checking (BMC) [9].
Theorem 5
Given an MLTL formula \(\varphi \) with K as the largest natural in the intervals of \(\varphi \), \(\varphi \) is satisfiable iff there is a finite trace \(\pi \) with \(\pi  \le K\cdot cl(\varphi )\) such that \(\pi \,\models \,\varphi \).
Theorem 5 states that the satisfiability of a given MLTL formula can be reduced to checking for the existence of a satisfying trace. To apply the BMC technique in nuXmv, we compute and set the maximal depth of BMC to be the value of \(K\cdot cl(\varphi )\) for a given MLTL formula \(\varphi \). The input SMV model for BMC is still \(M_{\varphi }\), as described in Sect. 4.2. However to ensure correct BMC checking in nuXmv, the constraint “FAIRNESS TRUE” has to be added into the SMV model.^{3} The Open image in new window remains \(\Box \lnot Tail\). According to Theorem 5, \(\varphi \) is satisfiable iff the model checker returns a counterexample by using the BMC technique within the maximal depth of \(K\cdot cl(\varphi )\).
4.3 MLTLSAT via SMT Solving
Another approach to solve MLTLSAT is via SMT solving, considering that using SMT solvers to handle intervals in MLTL formulas is straightforward. Since the input logic of SMT solvers is FirstOrder Logic, we must first translate the MLTL formula to its equisatisfiable formula in FirstOrder Logic over the natural domain N. We assume that readers are familiar with FirstOrder Logic and only focus on the translation. Given an MLTL formula \(\varphi \) and the alphabet \(\varSigma \), we construct the corresponding formula in FirstOrder Logic over N in the following way.
 1.
For each \(p\in \varSigma \), define a corresponding function \(f_p: Int\rightarrow Bool\) such that \(f_p(k)\) is true (\(k\in N\)) iff there is a satisfying (finite) trace \(\pi \) of \(\varphi \) and p is in \(\pi [k]\).
 2.The FirstOrder Logic formula \(\textsf {fol} (\varphi , k, len)\) for \(\varphi \) (\(k, len\in N\)) is constructed recursively as below:

\(\textsf {fol} (\mathsf{true}, k, len) = (len > k)\) and \(\textsf {fol} (\mathsf{false}, k, len) = false\);

\(\textsf {fol} (p, k, len) = (len > k)\wedge f_p(k)\) for \(p\in \varSigma \);

\(\textsf {fol} (\lnot \xi , k, len) = (len > k)\wedge \lnot \textsf {fol} (\xi , k, len)\);

\(\textsf {fol} (\xi \wedge \psi , k, len)\) \(= (len >k) \wedge \textsf {fol} (\xi , k, len) \wedge \textsf {fol} (\psi , k, len)\);

\(\textsf {fol} (\xi \ \mathcal {U}_{[a,b]} \ \psi , k, len)\) \(= (len > a+k)\wedge \) \(\exists i.(\) \((a+k \le i\le b+k) \wedge \) \(\textsf {fol} (\psi , i, leni) \wedge \) \(\forall j. (\) \((a+k \le j< i)\rightarrow \) \(\textsf {fol} (\xi ,j, lenj)))\);

In the formula \(\textsf {fol} (\varphi , k, len)\), k represents the index of the (finite) trace from which \(\varphi \) is evaluated, and len indicates the length of the suffix of the trace starting from the index k. Since the formula is constructed recursively, we need to introduce k to record the index. Meanwhile, len is necessary because the \(\mathsf{MLTL} \) semantics, which is interpreted over finite traces, constrains the lengths of the satisfying traces of the Until formulas. The following theorem guarantees that MLTLSAT is reducible to the satisfiability of FirstOrder Logic.
Theorem 6
For an MLTL formula \(\varphi \), \(\varphi \) is satisfiable iff the corresponding FirstOrder Logic formula \(\exists len.\textsf {fol} (\varphi , 0, len)\) is satisfiable.
Proof
Let the alphabet of \(\varphi \) be \(\varSigma \), and \(\pi \in (2^{\varSigma })^*\) be a finite trace. For each \(p\in \varSigma \), we define the function \(f_p : Int \rightarrow Bool\) as follows: \(f_p (k) = \mathsf{true}\) iff \(p\in \pi [k]\) if \(0\le k< \pi \). We now prove by induction over the type of \(\varphi \) and the construction of \(\textsf {fol} (\varphi , k, len)\) with respect to \(\varphi \) that \(\pi _k\,\models \,\varphi \) holds iff \(\{f_p  p\in \varSigma \}\) is a model of \(\textsf {fol} (\varphi , k, \pi )\): here \(\pi \) is the length of \(\pi \). The cases when \(\varphi \) is \(\mathsf{true}\) or \(\mathsf{false}\) are trivial.

If \(\varphi = p\) is an atom, \(\pi _k\,\models \,\varphi \) holds iff \(p\in \pi [k]\) (i.e., \(\pi _k[0]\)) is true, which means \(f_p (k)=\mathsf{true}\). As a result, \(\{f_p\}\) is a model of \(\textsf {fol} (\varphi , k, \pi )\), which implies that \(\pi _k\,\models \,\varphi \) holds iff \(\{f_p  p\in \varSigma \}\) is a model of \(\textsf {fol} (\varphi , k, \pi )\).

If \(\varphi = \lnot \xi \), \(\pi _k\,\models \,\varphi \) holds iff Open image in new window holds. By hypothesis assumption, \(\pi _k\,\models \,\xi \) holds iff \(\{f_p  p\in \varSigma \}\) is a model of \(\textsf {fol} (\xi , k, \pi )\), which is equivalent to saying Open image in new window holds iff \(\{f_p  p\in \varSigma \}\) is not a model of \(\textsf {fol} (\xi , k, \pi )\). As a result, \(\pi _k\,\models \,\lnot \xi \) holds iff \(\{f_p  p\in \varSigma \}\) is a model of \(\lnot \textsf {fol} (\xi , k, \pi )\).

If \(\varphi = \xi \wedge \psi \), \(\pi _k\,\models \,\varphi \) holds iff \(\pi _k\,\models \,\xi \) and \(\pi _k\,\models \,\psi \). By hypothesis assumption, \(\pi _k\,\models \,\xi \) (resp. \(\pi _k\,\models \,\psi \)) holds iff \(\{f_pp\in \varSigma \}\) is a model of \(\textsf {fol} (\xi , k, \pi )\) (resp. \(\textsf {fol} (\psi , k, \pi )\)). According to the construction of the \(\textsf {fol} \) function, \(\{f_pp\in \varSigma \}\) is a model of \(\textsf {fol} (\xi \wedge \psi , k, \pi )\). As a result, \(\pi _k\,\models \,\xi \wedge \psi \) holds iff \(\{f_pp\in \varSigma \}\) is a model of \(\textsf {fol} (\xi \wedge \psi , k, \pi )\).

If \(\varphi = \xi \ \mathcal {U}_{[a,b]} \ \psi \), \(\pi _k\,\models \,\varphi \) holds iff there is \(a+k\le i\le b+k\) such that \(\pi _i\,\models \,\psi \) and \(\pi _j\,\models \,\xi \) holds for every \(a+k\le j < i\). By hypothesis assumption, \(\pi _i\,\models \,\psi \) holds iff \(\{f_pp\in \varSigma \}\) is a model of \(\textsf {fol} (\psi , i, leni)\) (the length of \(\pi _i\) is \(leni\)), and \(\pi ,j\,\models \,\xi \) holds iff \(\{f_pp\in \varSigma \}\) is a model of \(\textsf {fol} (\xi , j, \pi j)\) (the length of \(\pi _j\) is \(\pi j\)). Moreover, \(\pi  > a+k\) must be true according to the MLTL semantics. As a result, \(\{f_pp\in \varSigma \}\) is a model of \(\textsf {fol} (\varphi , k, \pi )\), which implies that \(\pi _k\,\models \,\xi \ \mathcal {U}_{[a,b]}\psi \) holds iff \(\{f_pp\in \varSigma \}\) is a model of \(\textsf {fol} (\xi \ \mathcal {U}_{[a,b]} \ \psi , k, \pi )\).
This proof holds for all values of k, including the special case where \(k = 0\). \(\square \)
We then encode \(\exists len.\textsf {fol} (\varphi , 0, len)\) into the SMTLIB v2 format [7], which is the input of most modern SMT solvers; we call the full SMTLIB v2 encoding \(\mathsf{SMT} (\varphi )\). We first use the “declarefun” command to declare a function \(f_a : Int\rightarrow Bool\) for each \(p\in \varSigma \). We also define the function \(f_{\varphi }: Int\times Int\rightarrow Bool\) for the FirstOrder Logic formula \(\textsf {fol} (\varphi , k, len)\). The corresponding SMTLIB v2 command is “definefun \(f_{\varphi }\ ((k\ Int)\ (len\ Int))\ Bool\ S (\textsf {fol} (\varphi , k, len))\)”, where \(S (\textsf {fol} (\varphi , k, len))\) is the SMTLIB v2 implementation of \(\textsf {fol} (\varphi , k, len)\). In detail, \(S (\textsf {fol} (\varphi , k, len))\) is acquired recursively as follows.

\(S (\textsf {fol} (p, k, len)) \longrightarrow \) (and (\(> len\ k\)) (\(f_p\ k\)))

\(S(\lnot \textsf {fol} (\varphi , k, len)) \longrightarrow \) (and (\(> len\ k\)) (not \(S (\textsf {fol} (\varphi , k)\))))

\(S (\textsf {fol} (\varphi _1\wedge \psi , k, len) \longrightarrow \) (and (\(>\ len\ k\)) (and \(S (\textsf {fol} (\varphi _1, k, len))\) \(S(\textsf {fol} (\psi , k, len))\)))

\(S (\textsf {fol} (\varphi _1\ \mathcal {U}_{[a,b]} \ \psi , k, len)) \longrightarrow \) (and (\(>\ len\ a+k\)) (exists (\(i\ Int\)) (and (\(\le \ (+\ a\ k)\ i\)) (\(\ge \ i\ (+\ b\ k)\)) \(S(\textsf {fol} (\psi , i, leni))\) (forall (\(j\ Int\)) (\(\Rightarrow \) (and \((\le \ (+\ a\ k)\ j)\ (<\ j\ i)\)) \(S(\textsf {fol} (\varphi _1, j, lenj))\))))))
The SMTLIB v2 template for \(\mathsf{SMT} (\varphi )\).
Finally, we use the “assert” command “(assert (exists ((\(len\ Int\))) \((f_{\varphi }\ 0\ len)\)))” together with the “(checksat)” command to request SMT solvers for the satisfiability of \(\exists len.\textsf {fol} (\varphi , 0, len)\). In a nutshell, the general framework of the SMTLIB v2 format for \(\mathsf{SMT} (\varphi )\) (i.e., \(\exists len.\textsf {fol} (\varphi , 0, len)\)) is shown in Table 1, and the correctness is guaranteed by Theorem 7 below.
Theorem 7
The FirstOrder Logic formula \(\exists len.fol(\varphi , 0, len)\) is satisfiable iff the SMT solver returns SAT with the input \(\mathsf{SMT} (\varphi )\).
An inductive proof for the theorem can be conducted according to the construction of \(\mathsf{SMT} (\varphi )\). Notably, there is no difference between the SMT encoding for MLTL formulas and that for Open image in new window formulas, as the SMTbased encoding does not require unrolling the temporal operators in the formula.
5 Experimental Evaluations
List of solvers and their runtime flags.
Encoding  MLTLconverter flag  Solver  Solver flag 

LTL  ltl  aalta  default 
ltlf  aaltaf  default  
SMV  smv  nuXmv  source bmc.cmd (BMC) 
source klive.cmd (KLIVE)  
SMTLIB v2  smtlib  Z3  smt2 
All experiments were executed on Rice University’s NOTS cluster,^{4} running RedHat 5, with 226 dual socket compute blades housed within HPE s6500, HPE Apollo 2000, and Dell PowerEdge C6400 chassis. All the nodes are interconnected with 10 GigE network. Each satisfiability check over one MLTL formula and one solver was executed with exclusive access to one CPU and 8 GB RAM with a timeout of one hour, as measured by the Linux Open image in new window command. We assigned a time penalty of one hour to benchmarks that segmentation fault or timeout.
Experimental Goals. We evaluate performance along three metrics. (1) Each satisfiability check has two parts: the encoding time (consumed by MLTLconverter) and the solving time (consumed by solvers). We evaluate how each encoding affects the performance of both stages of MLTLSAT. (2) We comparatively analyze the performance and scalability of endtoend MLTLSAT via LTLSAT, Open image in new window , LTL model checking, and our new SMTbased approach. (3) We evaluate the performance and scalability for Open image in new window satisfiability checking using Open image in new window encoding heuristics (Lemma 2).
Benchmarks. There are few MLTL (or even MTLovernaturals) benchmarks available for evaluation. Previous works on MTLovernaturals [2, 3, 4] mainly focus on the theoretic exploration of the logic. To enable rigorous experimental evaluation, we develop three types of benchmarks, motivated by the generation of LTL benchmarks [38].^{5}
(1) Random MLTL Formulas (R): We generated 10,000 R formulas, varying the formula length L (20, 40, 60, 80, 100), the number of variables N (1, 2, 3, 4, 5), and the probability of the appearance of the \(\mathcal {U}\) operator P (0.33, 0,5, 0.7, 0.95); for each (L, N, P) we generated 100 formulas. For every \(\mathcal {U}\) operator, we randomly chose an interval [i, j] where \(i\ge 0\) and \(j\le 100\).
(2) NASABoeing MLTL Formulas (NB): We use challenging benchmarks [15] created from projects at NASA [17, 26] and Boeing [11]. We extract 63 reallife LTL requirements from the SMV models of the benchmarks, and then randomly generate an interval for each temporal operator. (We replace each \(\mathcal {X}\) with \(\Box _{[1,1]}\).) We create 3 groups of such formulas (63 in each) to test the scalability of different approaches, by restricting the maximal number of the intervals to be 1,000, 10,000, and 100,000 respectively.
Correctness Checking. We compared the verdicts from all solvers for every test instance and found no inconsistencies, excluding segmentation faults. This exercise aided with verification of our implementations of the translators, including diagnosing the need for including FAIRNESS TRUE in BMC models.
Experimental Results. Figure 2 compares encoding times for the R benchmark formulas. We find that (1) Encoding MLTL as either LTL and Open image in new window is not scalable even when the intervals in the formula are small; (2) The cost of MLTLtoSMV encoding is comparable to that from MLTL to SMTLIB v2. Although the cost of encoding MLTL as Open image in new window and SMV are in \(O(K\cdot cl(\varphi ))\), where K is the maximal interval length in \(\varphi \), the practical gap between the Open image in new window encodings and SMV encoding affirms our conjecture that the SMV model is more compact in general than the corresponding Open image in new window formulas. Also because K is kept small in the R formulas, the encoding cost between SMV and SMTLIB v2 becomes comparable.
Finally, we evaluated the performance of modelcheckingbased approaches on the R0 formulas, observing that there is an exponential complexity gap between MLTLSAT and Open image in new window . Figure 6 compares the performance of satisfiability solving via the BMC and KLIVE approaches. There is no significant improvement when the SMV encoding heuristics for Open image in new window are applied. For the BMC solving approach, performance is largely unaffected by encoding heuristics. For the KLIVE solving approach, encoding heuristics decrease solving performance. The results support the wellknown phenomenon that the theoretic analysis and the practical evaluations do not always match.
We summarize with three conclusions. (1) For satisfiability checking of MLTL formulas, the new SMTbased approach is best. (2) For satisfiability checking of MLTL formulas with interval ranges less than 100, the MLTLSAT via KLIVE approach is fastest. (3) The dedicated encoding heuristics for Open image in new window do not significantly improve the satisfiability checking time of Open image in new window over MLTLSAT. They do not solve the nuXmv scalability problem.
6 Discussion and Conclusion
Metric Temporal Logic (MTL) was first introduced in [3], for describing continuous behaviors interpreted over infinite realtime traces. The later variants Metric Interval Temporal Logic (MITL) [5], and Bounded Metric Temporal Logic (BMTL) [30] are also interpreted over infinite traces. Intuitively, MLTL is a combination of MITL and BMTL that allows only bounded, discrete (over natural domain) intervals that are interpreted over finite traces. There are several previous works on the satisfiability of MITL, though their tools only support the infinite semantics. Bounded satisfiability checking for MITL formulas is proposed in [33], and the reduction from MITL to LTL is presented in [20]. Since previous works focus on MITL over infinite traces and there is no trivial way to reduce MLTL over finite traces to MITL over infinite traces, the previous methodologies are not comparable to those presented in this paper. This includes the SMTbased solution of reducing MITL formulas to equisatisfiable Constraint LTL formulas [8]. Compared to that, our new SMTbased approach more directly encodes MLTL formulas into the SMT language without translation through an intermediate language.
The contribution of a complete, correct, and opensource MLTL satisfiability checking algorithm and tool opens up avenues for a myriad of future directions, as we have now made possible specification debugging MLTL formulas in designtime verification and benchmark generation for runtime verification. We plan to explore alternative encodings for improving the performance of MLTL satisfiability checking and work toward developing an optimized multiencoding approach, following the style of the previous study for LTL [40]; the current SMT model generated from the MLTL formula uses a relatively simple theory (uninterpreted functions). We also plan to explore lazy encodings from MLTL formulas to SMT models. For example, instead of encoding the whole MLTL formula into a monolithic SMT model, we may be able to decrease overall satisfiabilitysolving time by encoding the MLTL formula in parts with dynamic ordering similar to [15]. To make the output of SMTbased MLTL satisfiability checking more usable, we plan to investigate translations from the functions returned from Z3 for satisfiable instances into more easily parsable satisfying assignments.
Footnotes
 1.
In this paper, MTLovernaturals is interpreted over finite traces.
 2.
A temporary variable is introduced in the DEFINE statement rather than the VAR statement of the SMV model, as it will be automatically replaced with those in VAR statements.
 3.
Based on comments in emails from the nuXmv developers.
 4.
 5.
All experimental materials are at http://temporallogic.org/research/CAV19/. The plots are best viewed online.
Notes
Acknowledgment
We thank anonymous reviewers for their helpful comments. This work is supported by NASA ECF NNX16AR57G, NSF CAREER Award CNS1552934, NSF grants IIS1527668, IIS1830549, and by NSF Expeditions in Computing project “ExCAPE: Expeditions in Computer Augmented Program Engineering.”
References
 1.Runtime Verification Benchmark Competition (2018). https://www.rvcompetition.org/20182/
 2.Alur, R., Feder, T., Henzinger, T.A.: The benefits of relaxing punctuality. J. ACM 43(1), 116–146 (1996)MathSciNetCrossRefGoogle Scholar
 3.Alur, R., Henzinger, T.A.: Realtime logics: complexity and expressiveness. In: LICS, pp. 390–401. IEEE (1990)Google Scholar
 4.Alur, R., Henzinger, T.A.: A really temporal logic. J. ACM 41(1), 181–204 (1994)MathSciNetCrossRefGoogle Scholar
 5.Alur, R., Henzinger, T.A.: Reactive modules. In: Proceedings of the 11th IEEE Symposium on Logic in Computer Science, pp. 207–218 (1996)Google Scholar
 6.Armoni, R., Fix, L., Flaisher, A., Grumberg, O., Piterman, N., Vardi, M.Y.: Enhanced vacuity detection in linear temporal logic. In: Hunt, W.A., Somenzi, F. (eds.) CAV 2003. LNCS, vol. 2725, pp. 368–380. Springer, Heidelberg (2003). https://doi.org/10.1007/9783540450696_35
 7.Barrett, C., Stump, A., Tinelli, C.: The SMTLIB standard: version 2.0. In: Workshop on Satisfiability Modulo Theories (2010)Google Scholar
 8.Bersani, M., Rossi, M., San Pietro, P.: An SMTbased approach to satisfiability checking of MITL. Inf. Comput. 245(C), 72–97 (2015)MathSciNetCrossRefGoogle Scholar
 9.Biere, A., Cimatti, A., Clarke, E., Zhu, Y.: Symbolic model checking without BDDs. In: Cleaveland, W.R. (ed.) TACAS 1999. LNCS, vol. 1579, pp. 193–207. Springer, Heidelberg (1999). https://doi.org/10.1007/3540490590_14CrossRefGoogle Scholar
 10.Bloem, R., Chockler, H., Ebrahimi, M., Strichman, O.: Synthesizing nonvacuous systems. In: Bouajjani, A., Monniaux, D. (eds.) VMCAI 2017. LNCS, vol. 10145, pp. 55–72. Springer, Cham (2017). https://doi.org/10.1007/9783319522340_4CrossRefGoogle Scholar
 11.Bozzano, M., et al.: Formal design and safety analysis of AIR6110 wheel brake system. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015, Part I. LNCS, vol. 9206, pp. 518–535. Springer, Cham (2015). https://doi.org/10.1007/9783319216904_36CrossRefGoogle Scholar
 12.Cavada, R., et al.: The nuXmv symbolic model checker. In: Biere, A., Bloem, R. (eds.) CAV 2014. LNCS, vol. 8559, pp. 334–342. Springer, Cham (2014). https://doi.org/10.1007/9783319088679_22CrossRefGoogle Scholar
 13.Claessen, K., Sörensson, N.: A liveness checking algorithm that counts. In: FMCAD, pp. 52–59. IEEE (2012)Google Scholar
 14.De Giacomo, G., Vardi, M.: Linear temporal logic and linear dynamic logic on finite traces. In: IJCAI, pp. 2000–2007. AAAI Press (2013)Google Scholar
 15.Dureja, R., Rozier, K.Y.: More scalable LTL model checking via discovering designspace dependencies (\(D^{3}\)). In: Beyer, D., Huisman, M. (eds.) TACAS 2018, Part I. LNCS, vol. 10805, pp. 309–327. Springer, Cham (2018). https://doi.org/10.1007/9783319899602_17CrossRefGoogle Scholar
 16.Furia, C.A., Spoletini, P.: Tomorrow and all our yesterdays: MTL satisfiability over the integers. In: Fitzgerald, J.S., Haxthausen, A.E., Yenigun, H. (eds.) ICTAC 2008. LNCS, vol. 5160, pp. 126–140. Springer, Heidelberg (2008). https://doi.org/10.1007/9783540857624_9CrossRefGoogle Scholar
 17.Gario, M., Cimatti, A., Mattarei, C., Tonetta, S., Rozier, K.Y.: Model checking at scale: automated air traffic control design space exploration. In: Chaudhuri, S., Farzan, A. (eds.) CAV 2016, Part II. LNCS, vol. 9780, pp. 3–22. Springer, Cham (2016). https://doi.org/10.1007/9783319415406_1CrossRefGoogle Scholar
 18.Geist, J., Rozier, K.Y., Schumann, J.: Runtime observer pairs and bayesian network reasoners onboard FPGAs: flightcertifiable system health management for embedded systems. In: Bonakdarpour, B., Smolka, S.A. (eds.) RV 2014. LNCS, vol. 8734, pp. 215–230. Springer, Cham (2014). https://doi.org/10.1007/9783319111643_18CrossRefGoogle Scholar
 19.De Giacomo, G., Vardi, M.: Synthesis for LTL and LDL on finite traces. In: IJCAI, pp. 1558–1564 (2015)Google Scholar
 20.Hustadt, U., Ozaki, A., Dixon, C.: Theorem proving for metric temporal logic over the naturals. In: de Moura, L. (ed.) CADE 2017. LNCS (LNAI), vol. 10395, pp. 326–343. Springer, Cham (2017). https://doi.org/10.1007/9783319630465_20CrossRefGoogle Scholar
 21.Kessler, F.B.: nuXmv 1.1.0 (20160510) Release Notes (2016). https://esstatic.fbk.eu/tools/nuxmv/downloads/NEWS.txt
 22.Li, J., Rozier, K.Y.: MLTL benchmark generation via formula progression. In: Colombo, C., Leucker, M. (eds.) RV 2018. LNCS, vol. 11237, pp. 426–433. Springer, Cham (2018). https://doi.org/10.1007/9783030037697_25CrossRefGoogle Scholar
 23.Li, J., Zhang, L., Pu, G., Vardi, M.Y., He, J.: LTL\(_f\) satisfibility checking. In: ECAI, pp. 91–98 (2014)Google Scholar
 24.Li, J., Zhu, S., Pu, G., Vardi, M.Y.: SATbased explicit LTL reasoning. In: Piterman, N. (ed.) HVC 2015. LNCS, vol. 9434, pp. 209–224. Springer, Cham (2015). https://doi.org/10.1007/9783319262871_13CrossRefGoogle Scholar
 25.Maler, O., Nickovic, D.: Monitoring temporal properties of continuous signals. In: Lakhnech, Y., Yovine, S. (eds.) FORMATS/FTRTFT 2004. LNCS, vol. 3253, pp. 152–166. Springer, Heidelberg (2004). https://doi.org/10.1007/9783540302063_12CrossRefzbMATHGoogle Scholar
 26.Mattarei, C., Cimatti, A., Gario, M., Tonetta, S., Rozier, K.Y.: Comparing different functional allocations in automated air traffic control design. In: Proceedings of Formal Methods in ComputerAided Design (FMCAD 2015), Austin, Texas, USA. IEEE/ACM, September 2015Google Scholar
 27.McMillan, K.: Symbolic model checking: an approach to the state explosion problem. Ph.D. thesis, Carnegie Mellon University, Pittsburgh, PA, USA (1992). UMI Order No. GAX9224209Google Scholar
 28.Moosbrugger, P., Rozier, K.Y., Schumann, J.: R2U2: monitoring and diagnosis of security threats for unmanned aerial systems. In: FMSD, pp. 1–31, April 2017Google Scholar
 29.de Moura, L., Bjørner, N.: Z3: an efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008). https://doi.org/10.1007/9783540788003_24CrossRefGoogle Scholar
 30.Ouaknine, J., Worrell, J.: Some recent results in metric temporal logic. In: Cassez, F., Jard, C. (eds.) FORMATS 2008. LNCS, vol. 5215, pp. 1–13. Springer, Heidelberg (2008). https://doi.org/10.1007/9783540857785_1CrossRefGoogle Scholar
 31.Pandya, P.K., Shah, S.S.: The unary fragments of metric interval temporal logic: bounded versus lower bound constraints. In: Chakraborty, S., Mukund, M. (eds.) ATVA 2012. LNCS, pp. 77–91. Springer, Heidelberg (2012). https://doi.org/10.1007/9783642333866_8CrossRefGoogle Scholar
 32.Pnueli, A.: The temporal logic of programs. In: IEEE FOCS, pp. 46–57 (1977)Google Scholar
 33.Pradella, M., Morzenti, A., Pietro, P.: Bounded satisfiability checking of metric temporal logic specifications. ACM Trans. Softw. Eng. Methodol. 22(3), 20:1–20:54 (2013)CrossRefGoogle Scholar
 34.Reinbacher, T., Rozier, K.Y., Schumann, J.: Temporallogic based runtime observer pairs for system health management of realtime systems. In: Ábrahám, E., Havelund, K. (eds.) TACAS 2014. LNCS, vol. 8413, pp. 357–372. Springer, Heidelberg (2014). https://doi.org/10.1007/9783642548628_24CrossRefGoogle Scholar
 35.Rozier, K.Y.: Specification: the biggest bottleneck in formal methods and autonomy. In: Blazy, S., Chechik, M. (eds.) VSTTE 2016. LNCS, vol. 9971, pp. 8–26. Springer, Cham (2016). https://doi.org/10.1007/9783319488691_2CrossRefGoogle Scholar
 36.Rozier, K.Y.: On the evaluation and comparison of runtime verification tools for hardware and cyberphysical systems. In: RVCUBES, vol. 3, pp. 123–137. Kalpa Publications (2017)Google Scholar
 37.Rozier, K.Y., Schumann, J., Ippolito, C.: Intelligent hardwareenabled sensor and software safety and health management for autonomous UAS. Technical Memorandum NASA/TM2015218817, NASA Ames Research Center, Moffett Field, CA 94035, May 2015Google Scholar
 38.Rozier, K.Y., Vardi, M.Y.: LTL satisfiability checking. In: Bošnački, D., Edelkamp, S. (eds.) SPIN 2007. LNCS, vol. 4595, pp. 149–167. Springer, Heidelberg (2007). https://doi.org/10.1007/9783540733706_11CrossRefGoogle Scholar
 39.Rozier, K.Y., Vardi, M.Y.: LTL satisfiability checking. Int. J. Softw. Tools Technol. Transf. 12(2), 123–137 (2010)CrossRefGoogle Scholar
 40.Rozier, K.Y., Vardi, M.Y.: A multiencoding approach for LTL symbolic satisfiability checking. In: Butler, M., Schulte, W. (eds.) FM 2011. LNCS, vol. 6664, pp. 417–431. Springer, Heidelberg (2011). https://doi.org/10.1007/9783642214370_31CrossRefGoogle Scholar
 41.Rozier, K.Y., Vardi, M.Y.: Deterministic compilation of temporal safety properties in explicit state model checking. In: Biere, A., Nahir, A., Vos, T. (eds.) HVC 2012. LNCS, vol. 7857, pp. 243–259. Springer, Heidelberg (2013). https://doi.org/10.1007/9783642396113_23CrossRefGoogle Scholar
 42.Schumann, J., Moosbrugger, P., Rozier, K.Y.: R2U2: monitoring and diagnosis of security threats for unmanned aerial systems. In: Bartocci, E., Majumdar, R. (eds.) RV 2015. LNCS, vol. 9333, pp. 233–249. Springer, Cham (2015). https://doi.org/10.1007/9783319238203_15CrossRefGoogle Scholar
 43.Schumann, J., Moosbrugger, P., Rozier, K.Y.: Runtime analysis with R2U2: a tool exhibition report. In: Falcone, Y., Sánchez, C. (eds.) RV 2016. LNCS, vol. 10012, pp. 504–509. Springer, Cham (2016). https://doi.org/10.1007/9783319469829_35CrossRefGoogle Scholar
 44.Schumann, J., Rozier, K.Y., Reinbacher, T., Mengshoel, O.J., Mbaya, T., Ippolito, C.: Towards realtime, onboard, hardwaresupported sensor and software health management for unmanned aerial systems. IJPHM 6(1), 1–27 (2015)Google Scholar
 45.Sistla, A.P., Clarke, E.M.: The complexity of propositional linear temporal logic. J. ACM 32, 733–749 (1985)MathSciNetCrossRefGoogle Scholar
Copyright information
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.