Active Objects Based on Algebraic Effects

Abstract

Algebraic effects are a long-studied programming language construct allowing the implementation of complex control flow in a structured way. With OCaml 5, such features are finally available in a mainstream programming language, giving us a great opportunity to experiment with varied concurrency constructs implemented as simple libraries. In this article, we explore how to implement concurrency features such as futures and active objects using algebraic effects, both in theory and in practice. On the practical side, we present a library of active objects implemented in OCaml, with futures, cooperative scheduling of active objects, and thread-level parallelism. On the theoretical side, we formalise the compilation of a future calculus that models our library into an effect calculus similar to the primitives available in OCaml; we then prove the correctness of the compilation scheme.

A Proof of the Bisimulation Theorem (Theorem 2)

A proof of bisimulation involves two simulation proofs for the same relation. We detail the proof for the first direction: the behaviour of the compiled program is one of the behaviours of the original one. This direction is more complex because of the middle-step semantics and is also more important as it states that the behaviour of the compiled program is a valid one. The other direction is done very similarly with the same arguments as the ones used in the first direction. It however has a different structure as the SOS semantics provides more different cases (but the proof below often needs to distinguish cases according to the current state of the configuration, leading to a similar set of cases overall). We omit the other direction.

Consider , and . Let i be the thread identifier of the thread involved in the reduction (in case of spawn i is the thread that performs the spawn).

We have and for some $Q_1$ and $Q'_1$. Additionally, and .

We do a case analysis on the rule used to prove the $\text {R}_i$ relation; two cases are possible:

• Continue:

  

  In this case, the top level of continue is a handle thanks to the context $C_c$. can result from three possible rules (modulo a seq rule at the configuration level and a $\lambda$-calculus context rule to reach the reducible statement):

  • HANDLE-RETURN must be of the form (and is inside a handle because of $C_c$).

    We have . Its first visible reduction rule must be:

    

    Where:

    

    The hidden rules then update the appropriate task in the store and start the run function. Overall, we obtain:

    

    Where

    

    Since $e= C_{rec}[\texttt {handle}{(v)}\{h\}]$, by case analysis on the compilation rules, we must have the source expression $e' = v'$ be a Fut value with . Then we have:

    

    We then need to establish that the new future map and stores are in relation, i.e., $\sigma _2, \ell _{threads} \text {R}_e \sigma '_1, F_1\!\left[ f\mapsto v'\right]$.

    We recall the env rule below:

    

    By inversion on , we obtain three maps $T_{e,1} \uplus T_{e,2} \uplus T_v$ that ensure the relation. We extend $T_v$ so that $T_v[f]\mapsto \mathbb {V}(v)$ to obtain the relation.

    Recall that ; this is sufficient to conclude that

    

  • HANDLE-STEP must be of the form where can only be reduced by a $\lambda$-calculus reduction.

    We have . Its first visible reduction rule must be:

    

    Where:

    

    The translation leave $\lambda$-calculus terms unchanged, without any hiding, thus there are no follow up hidden rules.

    Overall, we obtain:

    

    We know that $\sigma _1, \ell _{threads} ~\text {R}_e~ \sigma '_1, F_1$. By definition, this means that $\sigma _1 = \sigma '_1 \cup \{ \ell _{threads} \mapsto T \}$ for some map T. By definition of the translation, $\ell _{threads}$ is not accessible by user code, and thus left unchanged by the reduction on . As such, we have:

    

    By case analysis on the translation and the $\lambda$-calculus reduction rules, $e'$ must be reduced by the same $\lambda$-calculus reduction rule than . Thus:

    

    This case analysis and by determinism of our $\lambda$-calculus, we have . We also have $\sigma _2, \ell _{threads} \text {R}_e \sigma '_2, F_1$.

    This is sufficient to conclude that

    

  • HANDLE-EFFECT must be of the form (and is inside a handle because of $C_c$). We distinguish by the effect captured:

    • ${\boldsymbol{Async}}({\boldsymbol{job}},t')$. We have . Its first visible reduction rule must be:

      

      Where:

      

      By definition of the translation, and because the reduction is possible, the arguments of the Async effect must be a thunk task, and its second argument must be a thread identifier (it can be an expression but this one is entirely evaluated before triggering the effect). This as some consequences on the considered Fut configuration, e.g. $e'$ is of the form $\texttt {AsyncAt}(e_0,t)$. Additionally, t is the same on both side as thread identifiers are preserved by the translation (this can be proven by case analysis on the definition of $\text {R}_i$).

      The hidden rules apply then update the suspended tasks in the store and start the continue function. The last hidden reduction rule is the beta-reduction that de-thunks the continuation $\lambda ().(\lambda y.C[y])(\textit{fut}')$ inside the handler of continue and puts $\textit{fut}'$ back into the invocation context.

      Overall, we obtain:

      

      Where

      

      Since , by case analysis on the compilation rules, we must have the source expression where and by Lemma 1. Note also that the set of future identifiers are the same in the Fut program and in its translation, and thus $\textit{fut}' = (t,\textit{fresh}())$ is a fresh future in the Fut configuration. Then we have:

      

      We then need to establish that the new future map and stores are in relation, i.e., $\sigma _2, \ell _{threads} \text {R}_e \sigma '_1, F_1\!\left[ \textit{fut}'\mapsto e'_1\right]$.

      We recall the env rule below:

      

      By inversion on $\sigma _1,\ell _{threads} ~\text {R}_{e}~ \sigma _1',F_1$, we obtain tree maps $T_{e,1} \uplus T_{e,2} \uplus T_v$ that ensure the relation. We then extend $T_{e,1}$ so that to obtain the relation.

      This is sufficient to conclude that

      

    • ${\boldsymbol{Get}}(f')$. We have . Its first visible reduction rule must be:

      

      Where⁶:

      

      The argument of the Get effect must be a future reference that is totally evaluated for the rule to succeed. If it is not a future the evaluation of poll fails. If it is not fully evaluated, the reduction should first occur inside the argument of the Get effect.

      Details on Poll Reductions. At this point, we look at hidden reductions, which must start in the body of poll. If the future is unresolved, poll loops forever and the medium step reduction diverges. This means either that the future never resolves, and this divergence in Eff simulates a deadlock in Fut; or that we could make reductions in other threads to resolve the deadlock. In the second case, the semantics for Eff would interleave loops in poll and reduction in other threads. Such interleaving is equivalent to triggering the Get event at the end, with a single loop in poll. The current theorem only consider this last interleaving. Overall, if there is a medium step reduction it means that the future is resolved.

      In this case, the future has been resolved, and, by bisimilarity on the stores ($\text {R}_e$) we have $F_1(f')=v$ and $\sigma _1(\ell _{threads})[f']=v$ for some v. We obtain after a couple of steps of beta-reduction:

      

      Since $e= C_{rec}[\texttt {handle}{(C[\texttt {throw}({\textit{Get}(f'))}]}){h}]$, by case analysis on the compilation rules, we have where by Lemma 1. Then we have:

      

      This is sufficient to conclude that

      

    • Await$(f')$ . The case when the awaited future is resolved is similar to the case of the Get effect just above. We only detail the proof in case the future is still unresolved.

      We have . Its first visible reduction rule must be:

      

      Where:

      

      Like in the Get case, the argument of the Await effect must be a future reference that is totally evaluated for the rule to succeed.

      When the future is unresolved, $\ell _{threads}[f']$ is not a value (it is not mapped or mapped to a $\mathbb {C}{}$). By definition of $\texttt {R}_e$ we necessarily have: $\not \exists v.\, (f' \mapsto v)\in F_1$. Then a few hidden beta reduction steps lead to the following configuration:

      

      Where

      

      Since $e= C_{rec}[\texttt {handle}{(C[\texttt {throw}({\textit{Await}(f'))}]}){h}]$, by case analysis on the compilation rules, we have where by Lemma 1. Thus on the Fut side, we have:

      

      We easily obtain that by expanding the environment $T_{e,2}$ in the env rule.

      With the arguments above and the case run of we conclude:

      

    • ${\boldsymbol{Spawn}}()$. We have . Its first visible reduction rule must be:

      

      With: $C_{rec}$ the “let ... rec” context of the continue handler inside $C_c$, h the effect handlers defined in Continue, additionally:

      

      The first hidden rule applied is

      

      Where $e_2= C_2[\texttt {spawn}({\textit{run}})]$. This is followed by steps of beta reduction to reduce the $\texttt{let}\ t' =\ldots$ construct, trigger continue, pass the associated $\textit{tid}$ and de-thunk the $\lambda ().\lambda y.C[y](\textit{tid})$ inside continue. We obtain the following configuration

      

      Finally, by a step of beta reduction in the thread $\textit{tid}$ we obtain the right evaluation context $C_r$

      

      This configuration is not reducible by a hidden transition. Thus

      

      By case analysis on the terms involved in we have where by Lemma 1. We then have:

      

      Note that by definition of the set of used thread identifiers is the same in both configurations, wo we can take the same fresh $\textit{tid}$. Note also that the store and the future map are unchanged. Comparing thread by thread, we can directly apply rule run and rule cont for the two processes $\textit{tid}$ and i, which leads to the conclusion:

      

• Run:

  

  The only first applicable rule is the pop operation reduction that picks a new available thread:

  

  Note that pop ensures that $f_2$ is of the form $f_2=(i, \textit{lf})$. Using only reductions in the thread i and such that: ⁷ by definition of $\text {R}_i$ and ⁸ by definition of pop. Note that the last step of reduction is inside continue and de-thunks the new task ( )⁹. We additionally have:

  

  From the points above, we obtain (with $f_2=(i, \textit{lf})$):

  

  Note that by construction of the equivalence on stores (Fig. 16). Finally (the equivalence on the store can be trivially checked):

  

  This immediately concludes by adding the other threads (in $Q_1$ and $Q'_1$) and obtaining the relation on the obtained configurations.    $\square$