We have introduced the Hemiola DSL in Sect. 3 and provided an intuition that rule templates ensure general noninterference, i.e., interleavings among any transactions are safe. That said, we have not yet showed how the rule templates guarantee such noninterference in a formal way. We also have not explained how noninterference eases the verification of cache-coherence protocols.
In this section, we provide the semantics of the Hemiola DSL and the formal meaning of general noninterference called serializability. We then introduce our novel approach to proving invariants called predicate messages, which eliminates the burden of considering interference while proving invariants.
4.1 Semantics of the Hemiola DSL
A system in Hemiola follows so-called “one-rule-at-a-time semantics” [4, 5, 10, 34], i.e., any state transition by concurrent rule executions can be interpreted as a serial execution of rules. Thus, it is fair to consider that a state transition happens by executing a single rule.
Transition Steps. Figure 4 describes the complete semantics for transition steps of the Hemiola DSL. The semantics for a step is presented as a judgment \(s_0 \mathop {\rightarrow }\limits _{S}^{l} s_1\), where S is the system to execute, \(s_0\) is a prestate, \(s_1\) is a poststate, and l is a label generated by the state transition. The state of a system (in domain \(\mathbb {S}{}\)) is a pair \(\langle \overline{c}, M \rangle \) of cache states (\(\overline{c}\)) and message states (M). Cache states are represented in a finite map from cache indices to cache states, and message states are represented in a finite map from channel indices to ordered queues of messages.
Rule [SSilent] represents the case where no state transition happens in the current step; an empty label (\(l_\epsilon \)) is generated in this case. From now on, we assume that all the input/output messages used in the step definitions do not share the same channel, i.e., \((\textsf {List.NoDup}\, \overline{im.i})\). [SIns] describes the case for external input messages coming to the system; an external-inputs label (\(l_{\text {in}}(\overline{im})\)) is generated in this case. [SOuts] describes the opposite case, for output messages being released to the external world, generating an external-outputs label (\(l_{\text {out}}(\overline{im})\)).
Lastly, [SInt] deals with a state transition by a rule (r) in a cache (C). It nondeterministically chooses a cache and a rule in the cache, checks that the precondition holds, and applies the transition to update the state of the system; an internal label (\(l_{\text {int}}(C.i, r.i, \overline{im^{\text {ins}}}, \overline{im^{\text {outs}}})\)) is generated in this case, which records a cache index, a rule index, input messages, and output messages. Note that the semantics is based on ordered channels, so messages are enqueued and dequeued in each state-transition case.
The step semantics is naturally lifted to one for multiple steps, presented as a judgment \(s_0 \mathop {\Rightarrow }\limits _{S}^{\overline{l}} s_1\), where \(\overline{l}\) is a sequence of labels generated by executions of the steps in order. We will sometimes call such a sequence of labels a history.
We say that a state s is reachable iff there is a history \(\overline{l}\) such that \(S_{\textsf {init}} \mathop {\Rightarrow }\limits _{S}^{\overline{l}} s\) holds, where \(S_{\textsf {init}}\) is the initial state of the system S. We use a simpler notation \(S \Rightarrow s\) for reachable states. We also call such a history \(\overline{l}\) legal, denoted as \(S \mathop {\Rightarrow }\limits ^{\overline{l}} \bullet \). We call \(\mathcal {I}: \mathbb {S}{} \rightarrow \mathbb {P}{}\)Footnote 2 an invariant over a system S if \(\mathcal {I}\) holds for all reachable states, i.e., \(\forall s.\; (S \Rightarrow s) \rightarrow \mathcal {I}(s)\).
Behaviors and Correctness. A system S has a behavior \(\lfloor \overline{l} \rfloor \) (denoted as \(S \Downarrow \lfloor \overline{l} \rfloor \)) iff \(S_{\textsf {init}} \mathop {\Rightarrow }\limits _{S}^{\overline{l}} s\) holds, where \(\lfloor \cdot \rfloor \) filters out silent (\(l_\epsilon \)) and internal (\(l_{\text {int}}\)) labels so only the external parts remain. We call such a sequence of labels a trace. Lastly, we say that a system I (“implementation”) trace-refines another system S (“specification”), written as \(I \sqsubseteq S\), iff every trace of I is also a trace of S:
$$\begin{aligned} I \sqsubseteq S \triangleq \forall \overline{t}.\; I \Downarrow \overline{t} \rightarrow S \Downarrow \overline{t}. \end{aligned}$$
In order to prove trace refinement, we usually establish a simulation relation [6] between the implementation and the spec states and prove that the relation is preserved over steps, and it is crucial to state and prove proper invariants of the implementation for the simulation proof. Since the invariant proof is indeed the most significant part of the whole correctness proof, in this paper we would like to focus on how Hemiola helps a user state and prove invariants.
4.2 Serializability in Hemiola
Serializability [3, 28] is a celebrated notion of concurrency correctness. While each transaction in a system affects multiple values, serializability guarantees that interleaved execution of such transactions is correct in that the effect (state change) is the same as if the transactions were executed serially, i.e., atomically in some order with no interleaving.
In order to define serializability formally, we first provide basic definitions of atomic histories and transactions. A history h is atomic iff it satisfies the predicate
with initial messages \(\overline{im^{\text {init}}}\) and live messages \(\overline{im^{\text {end}}}\), constructed inductively by the following two cases:
-
Any singleton history with an internal label is an atomic history with its input and output messages as initial and live messages, respectively.
-
If h is an atomic history, \((h + l)\) is also an atomic history if l consumes its input messages from the live messages of h. The new live messages are constructed by subtracting the input messages and adding the output messages of l to the previous live messages.
Figure 5 presents an atomic history already shown in Fig. 1. h is generated by executions of three rules, \(r_1 \in C_1.\overline{r}\), \(r_2 \in P.\overline{r}\), and \(r_3 \in C_2.\overline{r}\). Rule \(r_1\) takes an input message \((1, \textsf {rqWr})\) (from the channel with index 1) as an initial message of the history. Rule \(r_2\) takes \((3, \textsf {rqM})\), the output message from \(r_1\). Finally, \(r_3\) takes \((8, \textsf {rqI})\), the output message from \(r_2\). Summing up all the rule executions, by the definition of an atomic history we get the predicate lower-right in Fig. 5.
This example shows that an atomic history intuitively captures a transaction flow triggered by the initial messages. Note that an atomic history does not need to be completed, e.g., h in the example is incomplete in the sense that the live message (\(\textsf {rsI}\)) is not a response sent to an external channel.
We call an atomic history
a transaction if its initial messages are external requests (\(\overline{im^{\text {init}}.i} \subseteq S.\overline{i_{\text {rq}}}\)); we denote it as
.
With a clear notion of transactions, we can now easily define sequential histories and serializability. A history h is sequential iff the history is a concatenation of transactions:
A legal history h is serializable in the system S iff there exists a sequential history that reaches the same state:
$$\begin{aligned} \mathsf {Serializable}\ S\ h \triangleq \; \forall s.\; S_{\textsf {init}} \mathop {\Rightarrow }\limits _{S}^{h} s \rightarrow \exists \, h_{\text {seq}}.\; \mathsf {Sequential}\ S\ h_{\text {seq}} \wedge S_{\textsf {init}} \mathop {\Rightarrow }\limits _{S}^{h_{\text {seq}}} s. \end{aligned}$$
A system S is serializable iff every legal history is serializable:
$$\begin{aligned} \mathsf {Serializable}\ S \triangleq \forall h.\; \mathsf {Serializable}\ S\ h. \end{aligned}$$
4.3 Predicate Messages
Now we discuss how to exploit our notion of serializability: how does it help prove global invariants of a system? In proving the correctness of a cache-coherence protocol, it is very common to state an invariant like “an important property holds whenever the system includes a certain message in a certain channel.” We call such an invariant a predicate message, giving the intuition of messages that logically carry predicates that must be true so long as those messages remain in play. More formally, \(S \vdash im\{P\} \triangleq \forall s.\; (S \Rightarrow s) \rightarrow im \in s.M \rightarrow P(s)\), where s.M refers to the message state of the system. We will write just \(im\{P\}\) when the system S is clear from context, also often using a shorter version \(\textsf {id}{}\{P\}\) (considering only messages with a given ID) when it is not ambiguous.
Figure 6 presents an example of a predicate message. When a child \(C_2\) is about to handle a response message
, which is a permission to change the cache status to M, we expect the parent and the other child \(C_1\) to have I status (like
in the figure). However, between the sending of that message and receipt by \(C_2\), the predicate may be broken by another transaction; for instance, the predicate no longer holds if a state transition happens by
, which takes another
and updates the status of \(C_1\) to M.
Investigating this corner case carefully, we find that actually no two different \(\textsf {rsM}\) messages can be in the system at the same time. It implies that now the predicate message for \(\textsf {rsM}\) should have a much-more-complicated form, which considers all possible noninterference cases. The complete desired predicate message for \((8, \textsf {rsM})\) will then look like:
It is indeed a burden to consider all possible interleavings per predicate message. We would not have faced such a complication if we could ensure that no other transactions interfere while handling a transaction. Serializability guarantees exactly that simplification, and Hemiola provides a way of designing and proving predicate messages in the simpler form, not taking any interference into account.
Our novel approach to employing predicate messages in atomic histories begins with formalizing the notion of atomic invariants. We say that \(\mathcal {I}_A: \overline{\mathbb {I}\mathbb {M}{}} \times \mathbb {S}{} \rightarrow \mathbb {P}{}\) is an atomic invariant iff \(\mathcal {I}_A\, (\overline{im_o},\, s_1)\) holds for any atomic history h with \(s_0 \mathop {\Rightarrow }\limits _{S}^{h} s_1\) and
.
Figure 7 shows an example of predicate messages defined in an atomic history, formalized as an atomic invariant. An atomic invariant \(\mathcal {I}_A\) is a conjunction of clauses \((im \in \overline{im_o} \rightarrow P(s))\), each claiming that the predicate P holds when im is in the live messages \(\overline{im_o}\). We can prove that the atomic invariant \(\mathcal {I}_A\) holds by induction on state-transition steps through the atomic history in the figure:
-
The initial step of the atomic history is the one by \(r_1\). The live messages are \([(4, \textsf {rsI}) ]\). Since \(r_1\) changes the status of \(C_1\) to I, it is straightforward to prove \(\mathcal {I}_A\).
-
The next step is by \(r_p\), and at this point the live messages are \([(8, \textsf {rsM}) ]\). By the induction hypothesis, we obtain the predicate message \((4, \textsf {rsI})\{C_1.\text {st} = I\}\). Since \(r_p\) changes the status of P to I, we can prove the predicate for \((8, \textsf {rsM})\).
-
The last step is by \(r_2\), and the live messages are \([(10, \textsf {rsWr}) ]\). \(\mathcal {I}_A\) trivially holds here since it does not contain any predicate for \((10, \textsf {rsWr})\).
Note that the invariant proof was straightforward since no other state transitions interfere with an atomic history.
How do atomic invariants help prove conventional invariants? If the system S is serializable, by definition, for every reachable state there is a sequential history that reaches the same state. Since the sequential history is a concatenation of transactions, an invariant can be proven by showing that any transaction preserves it.
Since a transaction is an (external) atomic history, we can make use of corresponding atomic invariants. In other words, we can employ both conventional/atomic invariants (\(\mathcal {I}\) and \(\mathcal {I}_A\)) to prove the ones for the next state (\(s_{i+1}\)):
$$\begin{aligned} \mathcal {I}_A\, (\overline{m_i}, s_i) \wedge \mathcal {I}(s_i) \rightarrow (\mathcal {I}_A\, (\overline{m_{i+1}}, s_{i+1}) \wedge \mathcal {I}(s_{i+1})). \end{aligned}$$
For instance, in proving a cache-coherence protocol, we usually want to have an invariant claiming that at most one node of the system has M status at a time. The predicate messages defined in Fig. 7 will play a crucial role here, e.g., the one for \((8, \textsf {rsM})\) says that \(C_1\) and P both have I status, which means that the state transition by \((r_2: C_2.\text {st} \leftarrow M)\) preserves the invariant. We will see more comprehensive uses of predicate messages in our case studies (Sect. 5).
4.4 Serializability Guarantee by the Hemiola DSL
The biggest contribution of the Hemiola framework includes the serializability proof. The highest-level theorem simply claims that use of good topology (\(\mathsf {OnTree}\ S\ t\)) and the rule templates (\(\mathsf {GoodRules}\ S\ t\)) guarantees serializability:
$$\begin{aligned} \forall S, t.\; \mathsf {OnTree}\ S\ t \wedge \mathsf {GoodRules}\ S\ t \rightarrow \mathsf {Serializable}\ S. \end{aligned}$$
In the proof we used a well-established technique called commuting reductions [15], showing that any interleaving transactions can be serialized by performing a finite number of reductions. Interested readers are referred to Choi’s dissertation [9], which describes more details of the proof.