Nested Session Types

Session types statically describe communication protocols between concurrent message-passing processes. Unfortunately, parametric polymorphism even in its restricted prenex form is not fully understood in the context of session types. In this paper, we present the metatheory of session types extended with prenex polymorphism and, as a result, nested recursive datatypes. Remarkably, we prove that type equality is decidable by exhibiting a reduction to trace equivalence of deterministic first-order grammars. Recognizing the high theoretical complexity of the latter, we also propose a novel type equality algorithm and prove its soundness. We observe that the algorithm is surprisingly efficient and, despite its incompleteness, sufficient for all our examples. We have implemented our ideas by extending the Rast programming language with nested session types. We conclude with several examples illustrating the expressivity of our enhanced type system.


Introduction
Session types express and enforce interaction protocols in message-passing systems [28,43].In this work, we focus on binary session types that describe bilateral protocols between two endpoint processes performing dual actions.Binary session types obtained a firm logical foundation since they were shown to be in a Curry-Howard correspondence with linear logic propositions [7,8,46].This allows us to rely on properties of cut reduction to derive type safety properties such as progress (deadlock freedom) and preservation (session fidelity), which continue to hold even when extended to recursive types and processes [16].
However, the theory of session types is still missing a crucial piece: a general understanding of prenex (or ML-style) parametric polymorphism, encompassing recursively defined types, polymorphic type constructors, and nested types.We abbreviate the sum of these features simply as nested types [3].Prior work has restricted itself to parametric polymorphism either: in prenex form without nested types [25,44]; with explicit higher-rank quantifiers [6,37] (including bounded ones [23]) but without general recursion; or in specialized form for iteration at the type level [45].None of these allow a free, nested use of polymorphic type constructors combined with prenex polymorphism.
In this paper, we develop the metatheory of this rich language of nested session types.Nested types are reasonably well understood in the context of functional languages [3,31] and have a number of interesting applications [10,27,36].
One difficult point is the interaction of nested types with polymorphic recursion and type inference [35].By adopting bidirectional type-checking we avoid this particular set of problems altogether, at the cost of some additional verbosity.However, we have a new problem namely that session type definitions are generally equirecursive and not generative.This means that even before we consider nesting, with the definitions We have that Tree[α] ⊗ κ is isomorphic to STree[α, κ] and that the processes witnessing the isomorphism can be easily implemented (see Section 9).At the core of type checking such programs lies type equality.We show that we can translate type equality for nested session types to the trace equivalence problem for deterministic first-order grammars, which was shown to be decidable by Jančar, albeit with doubly-exponential complexity [30].Solomon [41] already proved a related connection between inductive type equality for nested types and language equality for DPDAs.The difference is that session type equality must be defined coinductively, as a bisimulation, rather than via language equivalence [22].This is because session types capture communication behavior rather than the structure of closed values so a type such as R = ⊕{a : R} is not equal to the empty type E = ⊕{}.The reason is that the former type can send infinitely many a's while the latter cannot (due to the coinductive interpretation).Interestingly, if we imagine a lazy functional language such as Haskell with non-generative recursive types, then R and E would also be different.In fact, nothing in our analysis of equirecursive nested types depends on linearity, just on the coinductive interpretation of types.Several of our key results, namely decidability of type equality and a practical algorithm for it, apply to lazy functional languages!Open in this different setting would still be the question of type inference, including the treatment of polymorphic recursion.
The decision procedure for deterministic first-order grammars does not appear to be directly suitable for implementation, in part due to its doublyexponential complexity bound.Instead we develop an algorithm combining loop detection [22] with instantiation [17] and a special treatment of reflexivity to handle all cases that would have passed in a nominal system.The algorithm is sound, but incomplete, and reports success, a counterexample, or an inconclusive outcome (which counts as failure).In our experience, the algorithm is surprisingly efficient and sufficient for all our examples.
We have implemented nested session types and integrated them with the Rast language that is based on session types [16,17,18].We have evaluated our prototype on several examples such as the Dyck language [20], an expression server [44] and serializing binary trees, and standard polymorphic data structures such as lists, stacks and queues.
Most closely related to our work is context-free session types (CFSTs) [44].CFSTs also enhance the expressive power of binary session types by extending types with a notion of sequential composition of types.In connection with CFSTs, we identified a proper fragment of nested session types closed under sequential composition and therefore nested session types are strictly more expressive than CFSTs.
The main technical contributions of our work are: -A uniform language of session types supporting prenex polymorphism, type constructors, and nested types and its type safety proof (Sections 3, 6).-A proof of decidability of type equality (Section 4).
-A practical algorithm for type equality and its soundness proof (Section 5).
-A proper fragment of nested session types that is closed under sequential composition, the main feature of context-free session types (Section 7).-An implementation and integration with the Rast language (Section 8).

Overview of Nested Session Types
The main motivation for studying nested types is quite practical and generally applicable to programming languages with structural type systems.We start by applying parametric type constructors for a standard polymorphic queue data structure.We also demonstrate how the types can be made more precise using nesting.A natural consequence of having nested types is the ability to capture (communication) patterns characterized by context-free languages.As an illustration, we express the Dyck language of balanced parentheses and show how nested types are connected to DPDAs also.
Queues A standard application of parameterized types is the definition of polymorphic data structures such as lists, stacks, or queues.As a simple example, consider the nested type: The type queue, parameterized by α, represents a queue with values of type α.
A process providing this type offers an external choice ( ) enabling the client to either insert a value of type α in the queue (label ins), or to delete a value from the queue (label del).After receiving label ins, the provider expects to receive a value of type α (the ⊸ operator) and then proceeds to offer queue[α].
Upon reception of the label del, the provider queue is either empty, in which case it sends the label none and terminates the session (as prescribed by type 1), or is non-empty, in which case it sends a value of type α (the ⊗ operator) and recurses with queue[α].
Although parameterized type definitions are sufficient to express the standard interface to polymorphic data structures, we propose nested session types which are considerably more expressive.For instance, we can use type parameters to track the number of elements in the queue in its type!queue [α, x] {ins The second type parameter x tracks the number of elements.This parameter can be understood as a symbol stack.On inserting an element, we recurse to queue[α, Some[α, x]] denoting the push of Some symbol on stack x.We initiate the empty queue with the type queue[α, None] where the second parameter denotes an empty symbol stack.Thus, a queue with n elements would have the type queue[α, Some n [α, None]].On receipt of the del label, the type transitions to x which can either be None (if the queue is empty) or Some[α, x] (if the queue is non-empty).In the latter case, the type sends label some followed by an element, and transitions to queue[α, x] denoting a pop from the symbol stack.
In the former case, the type sends the label none and terminates.Both these behaviors are reflected in the definitions of types Some and None.
Context-Free Languages Recursive session types capture the class of regular languages [44].However, in practice, many useful languages are beyond regular.
As an illustration, suppose we would like to express a balanced parentheses language, also known as the Dyck language [20] with the end-marker $.We use L to denote an opening symbol, and R to denote a closing symbol (in a sessiontyped mindset, L can represent client request and R is server response).We need to enforce that each L has a corresponding closing R and they are properly nested.To express this, we need to track the number of L's in the output with the session type.However, this notion of memory is beyond the expressive power of regular languages, so mere recursive session types will not suffice.We utilize the expressive power of nested types to express this behavior.
The nested type T [x] takes x as a type parameter and either outputs L and continues with T [T [x]], or outputs R and continues with x.The type D either outputs L and continues with T [D], or outputs $ and terminates.The type D expresses a Dyck word with end-marker $ [33].
The key idea here is that the number of T 's in the type of a word tracks the number of unmatched L's in it.Whenever the type T [x] outputs L, it recurses with T [T [x]] incrementing the number of T 's in the type by 1. Dually, whenever the type outputs R, it recurses with x decrementing the number of T 's in the type by 1.The type D denotes a balanced word with no unmatched L's.Moreover, since we can only output $ (or L) at the type D and not R, we obtain the invariant that any word of type D must be balanced.If we imagine the parameter x as the symbol stack, outputting an L pushes T on the stack, while outputting R pops T from the stack.The definition of D ensures that once an L is outputted, the symbol stack is initialized with T [D] indicating one unmatched L.
Nested session types do not restrict communication so that the words represented have to be balanced.To this end, the type D ′ can model the cropped Dyck language, where unbalanced words can be captured.
The only difference between types T [x] and T ′ [x] is that T ′ [x] allows us to terminate at any point using the $ label which immediately transitions to type 1. Nested session types can not only capture the class of deterministic context-free languages recognized by DPDAs that accept by empty stack (balanced words), but also the class of deterministic context-free languages recognized by DPDAs that accept by final state (cropped words).

Multiple Kinds of Parentheses
We can use nested types to express more general words with different kinds of parentheses.Let L and L ′ denote two kinds of opening symbols, while R and R ′ denote their corresponding closing symbols respectively.We define the session types We push symbols S and S ′ to the stack on outputting L and L ′ respectively.Dually, we pop S and S ′ from the stack on outputting R and R ′ respectively.Then, the type E defines an empty stack, thereby representing a balanced Dyck word.This technique can be generalized to any number of kinds of brackets.

Multiple States as Multiple Parameters
Using defined type names with multiple type parameters, we enable types to capture the language of DPDAs with several states.Consider the language proposed by Korenjak and Hopcroft [33].A word in this language starts with a sequence of opening symbols L, followed by an intermediate symbol, either a or b.Then, the word contains as many closing symbols R as there were Ls and terminates with the symbol a or b matching the intermediate symbol.
The L 3 language is characterized by session type  [44]) using two type parameters.More broadly, nested types can neatly capture complex server-client interactions.For instance, client requests can be captured using labels L, L ′ while server responses can be captured using labels R, R ′ expressing multiple kinds of requests.Balanced words will then represent that all requests have been handled.The types can also guarantee that responses do not exceed requests.

Concatenation of Dyck Words
We conclude this section by proving some standard properties on balanced parentheses: closure under concatenation and closure under wrapping.If w 1 $ and w 2 $ are two balanced words, then so is w 1 w 2 $.Similarly, if w$ is a balanced word, then so is LwR$.These two properties can be proved by implementing append and wrap processes capturing the former and latter properties.The above declarations describe the type for the two processes.The append process uses two channels w 1 and w 2 of type D and provides w : D, whereas wrap uses w : D and provides w ′ : D. The underlying base system of session types is derived from a Curry-Howard interpretation [7,8] of intuitionistic linear logic [24].Below we describe the session types, their operational interpretation and the continuation type.
A, B, C ::= ⊕{ℓ : defined type name The basic type operators have the usual interpretation: the internal choice operator ⊕{ℓ : A ℓ } ℓ∈L selects a branch with label k ∈ L with corresponding continuation type A k ; the external choice operator {ℓ : A ℓ } ℓ∈L offers a choice with labels ℓ ∈ L with corresponding continuation types A ℓ ; the tensor operator A ⊗ B represents the channel passing type that consists of sending a channel of type A and proceeding with type B; dually, the lolli operator A ⊸ B consists of receiving a channel of type A and continuing with type B; the terminated session 1 is the operator that closes the session.
We also support type constructors to define new type names.A type name V is defined according to a type definition V [α] = A that is parameterized by a sequence of distinct type variables α that the type A can refer to.We can use type names in a type expression using V [B].Type expressions can also refer to parameter α available in scope.The free variables in type A refer to the set of type variables that occur freely in A. Types without any free variables are called closed types.We call any type not of the form V [B] to be structural.
All type definitions are stored in a finite global signature Σ defined as In a valid signature, all definitions V [α] = A are contractive, meaning that A is structural, i.e. not itself a type name.This allows us to take an equirecursive view of type definitions, which means that unfolding a type definition does not require communication.More concretely, the type V [B] is considered equivalent to its unfolding A[B/α].We can easily adapt our definitions to an isorecursive view [34,19] with explicit unfold messages.All type names V occurring in a valid signature must be defined, and all type variables defined in a valid definition must be distinct.Furthermore, for a valid definition V [α] = A, the free variables occurring in A must be contained in α.This top-level scoping of all type variables is what we call the prenex form of polymorphism.

Type Equality
Central to any practical type checking algorithm is type equality.In our system, it is necessary for the rule of identity (forwarding) and process spawn, as well as the channel-passing constructs for types A⊗B and A ⊸ B. However, with nested polymorphic recursion, checking equality becomes challenging.We first develop the underlying theory of equality providing its definition, and then establish its reduction to checking trace equivalence of deterministic first-order grammars.

Type Equality Definition
Intuitively, two types are equal if they permit exactly the same communication behavior.Formally, type equality is captured using a coinductive definition following seminal work by Gay and Hole [22].
Definition 1.We first define unfold Σ (A) as Unfolding a structural type simply returns A. Since type definitions are contractive [22], the result of unfolding is never a type name application and it always terminates in one step.

Definition 2. Let Type be the set of closed type expressions (no free variables
When the signature Σ is not clear from context we add a subscript, A ≡ Σ B. This definition only applies to types with no free type variables.Since we allow parameters in type definitions, we need to define equality in the presence of free type variables.To this end, we define the notation ∀V.A ≡ B where V is a collection of type variables and A and B are valid types w.r.t.V (i.e., free variables in A and B are contained in V).Definition 4. We define ∀V.A ≡ B iff for all closed type substitutions σ : V,

Decidability of Type Equality
Solomon [41] proved that types defined using parametric type definitions with an inductive interpretation can be translated to DPDAs, thus reducing type equality to language equality on DPDAs.However, our type definitions have a coinductive interpretation.As an example, consider the types A = ⊕{a : A} and B = ⊕{b : B}.With an inductive interpretation, types A and B are empty (because they do not have terminating symbols) and, thus, are equal.However, with a coinductive interpretation, type A will send an infinite number of a's, and B will send an infinite number of b's, and are thus not equal.Our reduction needs to account for this coinductive behavior.
We show that type equality of nested session types is decidable via a reduction to the trace equivalence problem of deterministic first-order grammars [29].A first-order grammar is a structure (N , A, S) where N is a set of non-terminals, A is a finite set of actions, and S is a finite set of production rules.The arity of non-terminal X ∈ N is written as arity(X) ∈ N. Production rules rely on a countable set of variables V, and on the set T N of regular terms over N ∪ V.A term is regular if the set of subterms is finite (see [29]).
Each production rule has the form Xα a − → E where X ∈ N is a non-terminal, a ∈ A is an action, and α ∈ V * are variables that the term E ∈ T N can refer to.A grammar is deterministic if for each pair of X ∈ N and a ∈ A, there is at most one rule of the form Xα Given a set of rules S, the trace of a term T is defined as The crux of the reduction lies in the observation that session types can be translated to terms and type definitions can be translated to production rules of a first-order grammar.We start the translation of nested session types to grammars by first making an initial pass over the signature and introducing fresh internal names such that the new type definitions alternate between structural (except 1 and α) and non-structural types.These internal names are parameterized over their free type variables, and their definitions are added to the signature.This internal renaming simplifies the next step where we translate this extended signature to grammar production rules.
Example 1.As a running example, consider the queue type from Section 2: After performing internal renaming for this type, we obtain the following signature: We introduce the fresh internal names X 0 , X 1 and X 2 (parameterized with free variable α) to represent the continuation type in each case.Note the alternation between structural and non-structural types (of the form V [B]).
Next, we translate this extended signature to the grammar G = (N , A, S) aimed at reproducing the behavior prescribed by the types as grammar actions.
Essentially, each defined type name is translated to a fresh non-terminal.Each type definition then corresponds a sequence of rules: one for each possible continuation type with the appropriate label that leads to that continuation.For instance, the type Q[α] has two possible continuations: transition to X 0 [α] with action &ins or to X 1 [α] with action &del.The rules for all other type names is analogous.When the continuation is 1, we transition to the nullary non-terminal ⊥ disabling any further action.When the continuation is α, we transition to α.Since each type name is defined once, the produced grammar is deterministic.
Formally, the translation from an (extended) signature to a grammar is handled by two simultaneous tasks: translating type definitions into production rules (function τ below), and converting type names, variables and the terminated session into grammar terms (function • ).The function • : OType → T N from open session types to grammar terms is defined by: type names translate to first-order terms Due to this mapping, throughout this section we will use type names indistinctly as type names or as non-terminal first-order symbols.
The function τ converts a type definition V [α] = A into a set of production rules and is defined according to the structure of A as follows: The function τ identifies the actions and continuation types corresponding to A and translates them to grammar rules.Internal and external choices lead to actions ⊕ℓ and ℓ, for each ℓ ∈ L, with A ℓ as the continuation type.The type A 1 ⊗ A 2 enables two possible actions, ⊗ 1 and ⊗ 2 , with continuation A 1 and A 2 respectively.Similarly A 1 ⊸ A 2 produces the actions ⊸ 1 and ⊸ 2 with A 1 and A 2 as respective continuations.Contractiveness ensures that there are no definitions of the form Our internal renaming ensures that we do not encounter cases of the form V [α] = 1 or V [α] = α because we do not generate internal names for them.For the same reason, the • function is only defined on the complement types 1, α and V The τ function is extended to translate a signature by being applied pointwise.Formally, Connecting all pieces, we define the fog function that translates a signature to a grammar as: fog(Σ) = (N , A, S), where: The grammar is constructed by first computing τ (Σ) to obtain all the production rules.N and A are constructed by collecting the set of non-terminals and actions from these rules.The finite representation of session types and uniqueness of definitions ensure that fog(Σ) is a deterministic first-order grammar.Checking equality of types A and B given signature Σ finally reduces to (i) internal renaming of Σ to produce Σ ′ , and (ii) checking trace-equivalence of terms A and B given grammar fog(Σ ′ ).If A and B are themselves structural, we generate internal names for them also during the internal renaming process.Since we assume an equirecursive and non-generative view of types, it is easy to show that internal renaming does not alter the communication behavior of types and preserves type equality.Formally, and Σ ′ is the extended signature for Σ.
Proof.For the direct implication, assume that A ∼ S B .Pick a sequence of actions in the difference of the traces and let w 0 be its greatest prefix occurring in both traces.Either w 0 is a maximal trace for one of the terms, or we have where In both cases, with a simple case analysis on the definition of the translation τ , we conclude that A ′ ≡ B ′ and so A ≡ B. For the reciprocal implication, assume that A ∼ S B .Consider the relation Obviously, (A, B) ∈ R. To prove that R is a type bisimulation, let (A 0 , B 0 ) ∈ R and proceed by case analysis on A 0 and B 0 .We sketch a couple of cases for A 0 .The other cases are analogous.
However, type equality is not only restricted to closed types (see Definition 4).To decide equality for open types, i.e. ∀V.A ≡ B given signature Σ, we introduce a fresh label ℓ α and type A α for each α ∈ V. We extend the signature with type definitions: We then replace all occurrences of α in A and B with A α and check their equality with signature Σ * .We prove that this substitution preserves equality.
Proof.The direct implication is immediate because σ * is a closed substitution.For the reciprocal implication, assume that ∀V.A ≡ Σ B. Either for any closed substitution σ : In the latter, there is a distinct trace for A[σ ′ ] and B[σ ′ ], resulting from the substitution.Thus, a maximal trace w 1 belonging to both trace(A[σ ′ ]) and trace(B[σ ′ ]) leads to a subterm C of σ ′ (β) and to a subterm D of σ ′ (γ), respectively: Proof.Theorem 2 reduces equality of open types to equality of closed types.Theorem 1 reduces equality of closed nested session types to trace equivalence of first-order grammars.Jančar [29] proved that trace equivalence for first-order grammars is decidable, hence establishing the decidability of equality for nested session types.

Practical Algorithm for Type Equality
Although type equality can be reduced to trace equivalence for first-order grammars (Theorem 1), the latter problem has a very high theoretical complexity with no known practical algorithm [29].In response, we have designed a coinductive algorithm for approximating type equality.Taking inspiration from Gay and Hole [22], we attempt to construct a bisimulation.Our proposed algorithm is sound but incomplete and can terminate in three states: (i) types are proved equal by constructing a bisimulation, (ii) counterexample detected by identifying the position where types differ, or (iii) terminated without a conclusive answer due to incompleteness.We interpret both (ii) and (iii) as a failure of type-checking (but there is a recourse; see Section 5.1).The algorithm is deterministic (no backtracking) and the implementation is quite efficient in practice.For all our examples, type checking is instantaneous (see Section 8).
The fundamental operation in the equality algorithm is loop detection where we determine if we have already added an equation A ≡ B to the bisimulation we are constructing.Due to the presence open types with free type variables, determining if we have considered an equation already becomes a difficult operation.
To that purpose, we make an initial pass over the given types and introduce fresh internal names abstracted over their free type variables.In the resulting signature defined type names and type operators alternate and we can perform loop detection entirely on defined type names (whether internal or external).
Based on the invariants established by internal names, the algorithm only needs to alternately compare two type names or two structural types, i.e., types with an operator on the head.The rules are shown in Figure 1.The judgment has the form V ; Γ ⊢ Σ A ≡ B where V contains the free type variables in the types A and B, Σ is a fixed valid signature containing type definitions of the form If a derivation can be constructed, all closed instances of all closures are included in the resulting bisimulation (see the proof of Theorem 5).A closed instance of closure ] have no free type variables.Because the signature Σ is fixed, we elide it from the rules in Figure 1.
In the type equality algorithm, the rules for type operators simply compare the components.If the type constructors (or the label sets in the ⊕ and rules) do not match, then type equality fails having constructed a counterexample to bisimulation.Similarly, two type variables are considered equal iff they have the Fig. 1: Algorithmic Rules for Type Equality same name, as exemplified by the var rule.Finally, to account for α-renaming, when comparing explicitly quantified types (rule ∃ γ , ∀ γ ), we substitute α and β by the same fresh variable γ.
The rule of reflexivity is needed explicitly here (but not in the version of Gay and Hole) due to the incompleteness of the algorithm: we may otherwise fail to recognize type names parameterized with equal types as equal.Note that the refl rule checks a sequence of types.Now we come to the key rules, expd and def.In the expd rule we expand the definitions of must hold for all its closed instances, the extension of Γ with the corresponding closure remembers exactly that.
The def rule only applies when there already exists a closure in Γ with the same type names V 1 and V 2 .In that case, we try to find a substitution Immediately after, the refl rule applies and recursively calls the equality algorithm on both type parameters.Existence of such a substitution ensures that any closed instance of , which are already present in the constructed type bisimulation, and we can terminate our equality check, having successfully detected a loop.
The algorithm so far is sound, but potentially non-terminating.There are two points of non-termination: (i) when encountering name/name equations, we can use the expd rule indefinitely, and (ii) we call the type equality recursively in the def rule.To ensure termination in the former case, we restrict the expd rule so that for any pair of type names V 1 and V 2 there is an upper bound on the number of closures of the form − ; allowed in Γ .We define this upper bound as the depth bound of the algorithm and allow the programmer to specify this depth bound.Surprisingly, a depth bound of 1 suffices for all of our examples.In the latter case, instead of calling the general type equality algorithm, we introduce the notion of rigid equality, denoted by V ; Γ A ≡ B. The only difference between general and rigid equality is that we cannot employ the expd rule for rigid equality.Since the size of the types reduce in all equality rules except for expd, this algorithm terminates.When comparing two instantiated type names, our algorithm first tries reflexivity, then tries to close a loop with def, and only if neither of these is applicable or fails do we expand the definitions with the expd rule.Note that if type names have no parameters, our algorithm specializes to Gay and Hole's (with the small optimizations of reflexivity and internal naming), which means our algorithm is sound and complete on monomorphic types.
Soundness.We establish the soundness of the equality algorithm by constructing a type bisimulation from a derivation of V ; Γ ⊢ A ≡ B by (i) collecting the conclusions of all the sequents, and (ii) forming all closed instances from them.Finally, consider the case . If the last applied rule is expd, then V ; A ≡ B ∈ S(D 0 ), and thus (A[σ], B[σ]) ∈ R, satisfying the closure condition.If the last applied rule is def, since ) ∈ R for any substituion σ over V ′ .The second premise of the def rule ensures that there exists a substition σ ′ over Further note that extending a relation by its reflexive transitive closure preserves its bisimulation properties.

Type Equality Declarations
One of the primary sources of incompleteness in our algorithm is its inability to generalize the coinductive hypothesis.As an illustration, consider the following two types D and D ′ , which only differ in the names, but have the same structure.To allow a recourse, we permit the programmer to declare (concrete syntax)

eqtype T[x] = T'[x]
an equality constraint easily verified by our algorithm.We then seed the Γ in the equality algorithm with the corresponding closure from the eqtype constraint which can then be used to establish which, upon exploring the L branch reduces to which holds because under the substitution [D/x] as required by the def rule.the implementation, we first collect all the eqtype declarations in the program into a global set of closures Γ 0 .We then validate every eqtype declaration by checking V ; Γ 0 ⊢ A ≡ B for every pair (A, B) (with free variables V) in the eqtype declarations.Essentially, this ensures that all equality declarations Table 1: Session types with operational description are valid w.r.t. each other.Finally, all equality checks are then performed under this more general Γ 0 .The soundness of this approach can be proved with the following more general theorem.
Our soundness proof can easily be modified to accommodate this requirement.Intuitively, since Γ 0 is valid, all closed instances of Γ 0 are already proven to be bisimilar.Thus, all properties of a type bisimulation are still preserved if all closed instances of Γ 0 are added to it.
One final note on the rule of reflexivity: a type name may not actually depend on its parameter.As a simple example, we have V [α] = 1; a more complicated one would be V When applying reflexivity, we would like to conclude that V [A] ≡ V [B] regardless of A and B. This could be easily established with an equality type declaration eqtype V [α] = V [β].In order to avoid this syntactic overhead for the programmer, we determine for each parameter α of each type name V whether its definition is nonvariant in α.This information is recorded in the signature and used when applying the reflexivity rule by ignoring nonvariant arguments.

Formal Language Description
In this section, we present the program constructs we have designed to realize nested polymorphism which have also been integrated with the Rast language [16,17,18] to support general-purpose programming.The underlying base system of session types is derived from a Curry-Howard interpretation [7,8] of intuitionistic linear logic [24].The key idea is that an intuitionistic linear sequent A 1 A 2 . . .A n ⊢ A is interpreted as the interface to a process P .We label each of the antecedents with a channel name x i and the succedent with channel name z.The x i 's are channels used by P and z is the channel provided by P .
The resulting judgment formally states that process P provides a service of session type C along channel z, while using the services of session types A 1 , . . ., A n provided along channels x 1 , . . ., x n respectively.All these channels must be distinct.We abbreviate the antecedent of the sequent by ∆.
Due to the presence of type variables, the formal typing judgment is extended with V and written as V ; ∆ ⊢ Σ P :: (x : A) where V stores the type variables α, ∆ represents the linear antecedents x i : A i , P is the process expression and x : A is the linear succedent.We propose and maintain that all free type variables in ∆, P , and A are contained in V. Finally, Σ is a fixed valid signature containing type and process definitions.Table 1 overviews the session types, their associated process terms, their continuation (both in types and terms) and operational description.For each type, the first line describes the provider's viewpoint, while the second line describes the client's matching but dual viewpoint.
We formalize the operational semantics as a system of multiset rewriting rules [9].We introduce semantic objects proc(c, P ) and msg(c, M ) which mean that process P or message M provide along channel c.A process configuration is a multiset of such objects, where any two provided channels are distinct.

Basic Session Types
We briefly review the structural types already existing in the Rast language, focusing on explicit quantifier operators that we introduce.

Structural Types
The internal choice type constructor ⊕{ℓ : A ℓ } ℓ∈L is an n-ary labeled generalization of the additive disjunction A ⊕ B. Operationally, it requires the provider of x : ⊕{ℓ : A ℓ } ℓ∈L to send a label label k ∈ L on channel x and continue to provide type A k .The corresponding process term is written as (x.k ; P ) where the continuation P provides type x : A k .Dually, the client must branch based on the label received on x using the process term case x (ℓ ⇒ Q ℓ ) ℓ∈L where Q ℓ is the continuation in the ℓ-th branch.
Communication is asynchronous, so that the client (c.k ; Q) sends a message k along c and continues as Q without waiting for it to be received.As a technical device to ensure that consecutive messages on a channel arrive in order, the sender also creates a fresh continuation channel c ′ so that the message k is actually represented as (c.k ; c ↔ c ′ ) (read: send k along c and continue along c ′ ).When the message k is received along c, we select branch k and also substitute the continuation channel c ′ for c.
The external choice constructor {ℓ : A ℓ } ℓ∈L generalizes additive conjunction and is the dual of internal choice reversing the role of the provider and client.Thus, the provider branches on the label k ∈ L sent by the client.
Rules S and C below describe the operational behavior of the provider and client respectively (c ′ fresh). ( The tensor operator A ⊗ B prescribes that the provider of x : A ⊗ B sends a channel, say w of type A and continues to provide type B. The corresponding process term is send x w ; P where P is the continuation.Correspondingly, its client must receive a channel on x using the term y ← recv x ; Q, binding it to variable y and continuing to execute Q.
The dual operator A ⊸ B allows the provider to receive a channel of type A and continue to provide type B. The client of A ⊸ B, on the other hand, sends the channel of type A and continues to use B using dual process terms as ⊗.
V ; ∆, (y : A) ⊢ P :: (x : B) V ; ∆ ⊢ (y ← recv x ; P ) :: (x : The type 1 indicates termination requiring that the provider of x : 1 send a close message, formally written as close x followed by terminating the communication.Correspondingly, the client of x : 1 uses the term wait x ; Q to wait for x to terminate before continuing with executing Q. Linearity enforces that the provider does not use any channels.
Operationally, the provider waits for the closing message, which has no continuation channel since the provider terminates.
A forwarding process x ↔ y identifies the channels x and y so that any further communication along either x or y will be along the unified channel.Its typing rule corresponds to the logical rule of identity.
V ; y : A ⊢ (x ↔ y) :: (x : A) id Operationally, a process c ↔ d forwards any message M that arrives on d to c and vice-versa.Since channels are used linearly, the forwarding process can then terminate, ensuring proper renaming, as exemplified in the rules below.
We write M (c) to indicate that c must occur in message M ensuring that M is the sole client of c.
Process Definitions Process definitions have the form ∆ ⊢ f [α] = P :: (x : A) where f is the name of the process and P its definition, with ∆ being the channels used by f and x : A being the offered channel.In addition, α is a sequence of type variables that ∆, P and A can refer to.All definitions are collected in the fixed global signature Σ.For a valid signature, we require that α ; ∆ ⊢ P :: (x : A) for every definition, thereby allowing definitions to be mutually recursive.A new instance of a defined process f can be spawned with the expression x ← f [A] y ; Q where y is a sequence of channels matching the antecedents ∆ and A is a sequence of types matching the type variables α.The newly spawned process will use all variables in y and provide x to the continuation Q.
The declaration of f is looked up in the signature Σ (first premise), and A is substituted for α while matching the types in ∆ ′ and y (second premise).
Similarly, the freshly created channel x has type A from the signature with A substituted for α.The corresponding semantics rule also performs a similar substitution (a fresh). (defC where y ′ : Sometimes a process invocation is a tail call, written without a continuation as x ← f [A] y.This is a short-hand for x ′ ← f [A] y ; x ↔ x ′ for a fresh variable x ′ , that is, we create a fresh channel and immediately identify it with x.

Type Safety
The extension of session types with nested polymorphism is proved type safe by the standard theorems of preservation and progress, also known as session fidelity and deadlock freedom.At runtime, a program is represented using a multiset of semantic objects denoting processes and messages defined as a configuration.
We say that proc(c, P ) (or msg(c, M )) provide channel c.We stipulate that no two distinct semantic objects in a configuration provide the same channel.

Type Preservation
The key to preservation is defining the rules to type a configuration.We define a well-typed configuration using the judgment ∆ 1 Σ S :: ∆ 2 denoting that configuration S uses channels ∆ 1 and provides channels ∆ 2 .A configuration is always typed w.r.t. a valid signature Σ.Since the signature Σ is fixed, we elide it from the presentation.
The rules for typing a configuration are defined in Figure 2. The emp rule states that an empty configuration does not consume any channels provides all channels it uses.The comp rule composes two configurations S 1 and S 2 ; S 1 provides channels ∆ 2 while S 2 uses channels ∆ 2 .The rule proc creates a singleton configuration out of a process.Since configurations are runtime objects, they do not refer to any free variables and V is empty.The msg rule is analogous.
Global Progress To state progress, we need to define a poised process [38].A process proc(c, P ) is poised if it is trying to receive a message on c. Dually, a message msg(c, M ) is poised if it is sending along c.A configuration is poised if every message or process in the configuration is poised.Intuitively, this represents that the configuration is trying to communicate externally along one of the channels it uses or provides.
Theorem 6 (Type Safety).For a well-typed configuration ∆ 1 Σ S :: Proof.Preservation is proved by case analysis on the rules of operational semantics.First, we invert the derivation of the current configuration S and use the premises to assemble a new derivation for S ′ .Progress is proved by induction on the right-to-left typing of S so that either S is empty (and therefore poised) or S = (D, proc(c, P )) or S = (D, msg(c, M )).By the induction hypothesis, either D → D ′ or D is poised.In the former case, S takes a step (since D does).In the latter case, we analyze the cases for P and M , applying multiple steps of inversion to show that in each case either S can take a step or is poised.

Relationship to Context-Free Session Types
As ordinarily formulated, session types express communication protocols that can be described by regular languages [44].In particular, the type structure is necessarily tail recursive.Context-free session types (CFSTs) were introduced by Thiemann and Vascoconcelos [44] as a way to express a class of communication protocols that are not limited to tail recursion.CFSTs express protocols that can be described by single-state, real-time DPDAs that use the empty stack acceptance criterion [1,33].
Despite their name, the essence of CFSTs is not their connection to a particular subset of the (deterministic) context-free languages.Rather, the essence of CFSTs is that session types are enriched to admit a notion of sequential composition.Nested session types are strictly more expressive than CFSTs, in the sense that there exists a proper fragment of nested session types that is closed under a notion of sequential composition.(In keeping with process algebras like ACP [2], we define a sequential composition to be an operation that satisfies the laws of a right-distributive monoid.) Consider (up to α,β,η-equivalence) the linear, tail functions from types to types with unary type constructors only: The linear, tail nature of these functions allows the type α to be thought of as a continuation type for the session.The functions S are closed under function composition, and the identity function, λα.α, is included in this class of functions.Moreover, because these functions are tail functions, composition rightdistributes over the various logical connectives in the following sense: and similarly for and ⊸.Together with the monoid laws of function composition, these distributive properties justify defining sequential composition as S; T = S • T .
This suggests that although many details distinguish our work from CF-STs, nested session types cover the essence of sequential composition underlying context-free session types.However, even stating a theorem that every CFST process can be translated into a well-typed process in our system of nested session types is difficult because the two type systems differ in many details: we include ⊗ and ⊸ as session types, but CFSTs do not; CFSTs use a complex kinding system to incorporate unrestricted session types and combine session types with ordinary function types; the CFST system uses classical typing for session types and a procedure of type normalization, whereas our types are intuitionistic and do not rely on normalization; and the CFST typing rules are based on natural deduction, rather than the sequent calculus.With all of these differences, a formal translation, theorem, and proof would not be very illuminating beyond the essence already described here.Empirically, we can also give analogues of the published examples for CFSTs (see, e.g., the first two examples of Section 9).
Finally, nested session types are strictly more expressive than CFSTs.Recall from Section 2 the language L 3 = {L n a R n a ∪ L n b R n b | n > 0}, which can be expressed using nested session types with two type parameters used in an essential way.Moreover, Korenjak and Hopcroft [33] observe that this language cannot be recognized by a single-state, real-time DPDA that uses empty stack acceptance, and thus, CFSTs cannot express the language L 3 .More broadly, nested types allow for finitely many states and acceptance by empty stack or final state, while CFSTs only allow a single state and empty stack acceptance.

Implementation
We have implemented a prototype for nested session types and integrated it with the open-source Rast system [16].Rast (Resource-aware session types) is a programming language which implements the intuitionistic version of session types [7] with support for arithmetic refinements [17], ergometric [15] and temporal [14] types for complexity analysis.Our prototype extension is implemented in Standard ML (8011 lines of code) containing a lexer and parser (1214 lines), a type checker (3001 lines) and an interpreter (201 lines) and is well-documented.The prototype is available in the Rast repository [13].The first line is a type definition, where V is the type name parameterized by type variables x 1 , . . ., x k and A is its definition.The second line is a process declaration, where f is the process name (parameterized by type variables x 1 , . . ., x k ), (c 1 : A 1 ) . . .(c n : A n ) are the used channels and corresponding types, while the offered channel is c of type A. Finally, the last line is a process definition for the same process f defined using the process expression P .We use a hand-written lexer and shift-reduce parser to read an input file and generate the corresponding abstract syntax tree of the program.The reason to use a hand-written parser instead of a parser generator is to anticipate the most common syntax errors that programmers make and respond with the best possible error messages.
Once the program is parsed and its abstract syntax tree is extracted, we perform a validity check on it.This includes checking that type definitions, and process declarations and definitions are closed w.r.t. the type variables in scope.To simplify and improve the efficiency of the type equality algorithm, we also assign internal names to type subexpressions parameterized over their free index variables.These internal names are not visible to the programmer.

Type Checking and Error Messages
The implementation is carefully designed to produce precise error messages.To that end, we store the extent (source location) information with the abstract syntax tree, and use it to highlight the source of the error.We also follow a bi-directional type checking [39] algorithm reconstructing intermediate types starting with the initial types provided in the declaration.This helps us precisely identify the source of the error.Another particularly helpful technique has been type compression.Whenever the type checker expands a type V When printing types for error messages this mapping is consulted, and complex types may be compressed to much simpler forms, greatly aiding readability of error messages.

More Examples
Expression Server We adapt the example of an arithmetic expression from prior work on context-free session types [44].The type of the server is defined as type bin = +{ b0 : bin, b1 : bin, $ : 1 } type tm The type bin represents a constant binary natural number.A process providing a binary number sends a stream of bits, b0 and b1, starting with the least significant bit and eventually terminated by $.
An arithmetic term, parameterized by continuation type K can have one of three forms: a constant, the sum of two terms, or the double of a term.Consequently, the type tm[K] ensures that a process providing tm[K] is a well-formed term: it either sends the const label followed by sending a constant binary number of type bin and continues with type K; or it sends the add label and continues with tm[tm[K]], where the two terms denote the two summands; or it sends the double label and continues with tm[K].In particular, the continuation type tm[tm [K]] in the add branch enforces that the process must send exactly two summands for sums.
As a first illustration, consider two binary constants a and b, and suppose that we want to create the expression a + 2b.We can issue commands to the expression server in a prefix notation to obtain a + 2b, as shown in the following exp[K] process, which is parameterized by a continuation type K.In prefix notation, a + 2b would be written + (a) (2 b), which is exactly the form followed by the exp process: The process sends add, followed by const and the number a, followed by double, const, and b.Finally, the process continues at type K by forwarding k to e (intermediate typing contexts on the right).
To evaluate a term, we can define an eval process, parameterized by type K: The eval process uses channel t : tm[K] as argument, and offers v : bin * K.
The process evaluates term t and sends its binary value along v.
decl eval[K] : (t : tm Intuitively, the process evaluates term t and sends its binary value along v.If t is a constant, then eval[K] receives the constant n, sends it along v and forwards. The interesting case is the add branch.We evaluate the first summand by spawning a new eval[K] process on t.Note that since the type of t (indicated on the right) is tm[tm[K]] and hence, the recursive call to eval is at parameter tm[K].This is in contrast with nominal polymorphism in functional programming languages, where the recursive call must also be at parameter K.We store the value of the first summand at channel n1 : bin.Then, we continue to evaluate the second summand by calling eval[K] on t again and storing its value in n2 : bin.Finally, we add n1 and n2 by calling the plus process, and send the result bin along v.We follow a similar approach for the double branch.Serializing binary trees Another example from [44] is serializing binary trees.Here we adapt that example to our system.Binary trees can be described by: type Tree[a] = +{ node : Tree[a] * a * Tree[a] , leaf : 1 } These trees are polymorphic in the type of data stored at each internal node.A tree is either an internal node or a leaf, with the internal nodes storing channels that emit the left subtree, data, and right subtree.Owing to the multiple channels stored at each node, these trees do not exist in a serial form.
We can, however, use a different type to represent serialized trees: A serialized tree is a stream of node and leaf labels, nd and lf, parameterized by a continuation type K. Like add in the expression server, the label nd continues with type STree[a][a * STree[K]]: the label nd is followed by the serialized left subtree, which itself continues by sending the data stored at the internal node and then the serialized right subtree, which continues with type K. 3Using these types, it is relatively straightforward to implement processes that serialize and deserialize such trees.The process serialize can be declared with: This process uses channels t and k that hold the tree and continuation, and offers that tree's serialization along channel s.If the tree is only a leaf, then the process forwards to the continuation.Otherwise, if the tree begins with a node, then the serialization begins with nd.A recursive call to serialize serves to serialize the right subtree with the given continuation.A subsequent recursive call serializes the left subtree with the data together with the right subtree's serialization as the new continuation.
It is also possible to implement a process for deserializing trees, but because of space limitations, we will not describe deserialize here.Generalized tries for binary trees Using nested types in Haskell, prior work [27] describes an implementation of generalized tries that represent mappings on binary trees.Our type system is expressive enough to represent such generalized tries.We can reuse the type Tree To lookup a tree in a trie, first determine whether that tree is a leaf or a node.
If the tree is a leaf, then sending lookup_leaf to the trie will return the value of type b associated with that tree in the trie.Otherwise, if the tree is a node, then sending lookup_node to the trie results in a trie of type Trie We can define a process that constructs a trie from a function on trees: Both lookup_tree and build_trie can be seen as analogues to deserialize and serialize, respectively, converting a lower-level representation to a higherlevel representation and vice versa.These types and declarations mean that tries represent total mappings; partial mappings are also possible, at the expense of some additional complexity.
All our examples have been implemented and type checked in the opensource Rast repository [13].We have also further implemented the standard polymorphic data structures such as lists, stacks and queues.

Further Related Work
To our knowledge, our work is the first proposal of polymorphic recursion using nested type definitions in session types.Thiemann and Vasconcelos [44] use polymorphic recursion to update the channel between successive recursive calls but do not allow type constructors or nested types.An algorithm to check type equivalence for the non-polymorphic fragment of context-free session types has been proposed by Almeida et al. [1].
Other forms of polymorphic session types have also been considered in the literature.Gay [23] studies bounded polymorphism associated with branch and choice types in the presence of subtyping.He mentions recursive types (which are used in some examples) as future work, but does not mention parametric type definitions or nested types.Bono and Padovani [4,5] propose (bounded) polymorphism to type the endpoints in copyless message-passing programs inspired by session types, but they do not have nested types.Following Kobayashi's approach [32], Dardha et al. [12] provide an encoding of session types relying on linear and variant types and present an extension to enable parametric and bounded polymorphism (to which recursive types were added separately [11]) but not parametric type definitions nor nested types.Caires et al. [6] and Perez et al. [37] provide behavioral polymorphism and a relational parametricity principle for session types, but without recursive types or type constructors.
Nested session types bear important similarities with first-order cyclic terms, as observed by Jančar.Jančar [29] proves that the trace equivalence problem of first-order grammars is decidable, following the original ideas by Stirling for the language equality problem in deterministic pushdown automata [42].These ideas were also reformulated by Sénizergues [40].Henry and Sénizergues [26] proposed the only practical algorithm to decide the language equivalence problem on deterministic pushdown automata that we are aware of.Preliminary experiments show that such a generic implementation, even if complete in theory, is a poor match for the demands made by our type checker.

Conclusion
Nested session types extend binary session types with parameterized type definitions.This extension enables us to express polymorphic data structures just as naturally as in functional languages.The proposed types are able to capture sequences of communication actions described by deterministic context-free languages recognized by deterministic pushdown automata with several states, that accept by empty stack or by final state.In this setting, we show that type equality is decidable.To offset the complexity of type equality, we give a practical type equality algorithm that is sound, efficient, but incomplete.
In the future, we are planning to explore subtyping for nested types.In particular, since the language inclusion problem for simple languages [21] is undecidable, we believe subtyping can be reduced to inclusion and would also be undecidable.Despite this negative result, it would be interesting to design an algorithm to approximate subtyping.That would significantly increase the programs that can be type checked in the system.In another direction, since Rast [16] supports arithmetic refinements for lightweight verification, it would be interesting to explore how refinements interact with polymorphic type parameters, namely in the presence of subtyping.We would also like to explore examples where the current type equality is not adequate.Finally, protocols in distributed algorithms such as consensus or leader election (Raft, Paxos, etc.) depend on unbounded memory and cannot usually be expressed with finite control structure.In future work, we would like to see if these protocols can be expressed with nested session types.
our algorithm explores the L branch and checks T [D] ≡ T ′ [D ′ ].A corresponding closure • ; T [D] ≡ T ′ [D ′ ] is added to Γ , and our algorithm then checks T [T [D]] ≡ T ′ [T ′ [D ′ ]].This process repeats until it exceeds the depth bound and terminates with an inconclusive answer.What the algorithm never realizes is that T [x] ≡ T ′ [x] for all x ∈ Type; it fails to generalize to this hypothesis and is always inserting closed equality constraints to Γ .

c
: ⊕{ℓ : A ℓ } ℓ∈L c : A k c.k ; P P send label k on c case c (ℓ ⇒ Q ℓ ) ℓ∈L Q k receive label k on c c : {ℓ : A ℓ } ℓ∈L c : A k case c (ℓ ⇒ P ℓ ) ℓ∈L P k receive label k on c c.k ; Q Q send labelk on c c : A ⊗ B c : B send c w ; P P send channel w : A on c y ← recv c ; Qy Qy[w/y] receive channel w : A on c c : A ⊸ B c : B y ← recv c ; Py Py[w/y] receive channel w : A on c send c w ; Q Q send channel w : A on c c : ∃α.A c : A[B/α] send c [B] ; P P send type B on c [α] ← recv c ; Qα Qα[B/α] receive type B on c c : ∀α.A c : A [α] ← recv c ; Pα Pα[B/α] receive type B on c send c [B] ; Q Q send type B on c c : 1 close c send close on c wait c ; Q Q receive close on c

V
; ∆ ⊢ P :: (x : B) V ; ∆, (y : A) ⊢ (send x y ; P ) :: (x : A ⊗ B) ⊗R V ; ∆, (y : A), (x : B) ⊢ Q :: (z : C) V ; ∆, (x : A ⊗ B) ⊢ (y ← recv x ; Q) :: (z : C) ⊗L Operationally, the provider (send c d ; P ) sends the channel d and the continuation channel c ′ along c as a message and continues with executing P .The client receives the channel d and continuation channel c ′ and substitutes d for x and c ′ for c. (⊗S) : proc(c, send c d
decl build_trie[a][b] : (f : Tree[a] -o b) |-(m : Trie[a][b]) U .Since the type U is unaware of which intermediate symbol among a or b would eventually be chosen, it cleverly maintains two symbol stacks in the two type parameters x and y of O.