Keywords

1 Introduction

1.1 Multi-stage Programming

Program generation is a technique useful for several purposes such as improving performance or enhancing maintainability of programs; it can be used for optimizing programs by exploiting information available only at runtime, and also works as a basis for macros for eliminating so-called boilerplate code. Various studies and implementations provide features for code generation, and their formalizations differ from one another. Among these, a considerable amount of studies have been done for proposing multi-stage programming (MSP) languages [5, 6, 12, 13, 16, 26, 31,32,33,34,35, 38]. They enable users to write code-generating programs in a less error-prone manner with the aid of their type systems.

For a brief introduction to MSP, we use a simple setting similar to MetaML [32, 33] or \(\lambda ^\bigcirc \) [5, 6] here. In addition to ordinary constructs for typed functional languages, two special constructs, bracket and escape , are provided: . Brackets and escapes correspond to (hygienic) quasi-quotes and unquotes in Lisp, respectively. Owing to brackets and escapes, every subpart of expressions has its stage; a subexpression inside a bracket has one higher stage than outside, and conversely, a subexpression inside an escape has one lower stage. The lowermost stage is called stage \(0\), and seen from stage \(n\), subexpressions at stage \((n + 1)\) intuitively represent code fragments used for building the resulting code. One can intuitively understand the notion of stages in a graphical manner like (a) and (b) below, considering that expressions are bumpy; brackets are convex, and escapes are concave.

figure d

The essentials of the evaluation rules are the following: (i) The ordinary (call-by-value) \(\beta \)-reduction is performed only at stage \(0\); expressions inside brackets are not evaluated in a usual sense except for ones inside escapes, since they represent code fragments. (ii) Escape cancels bracket; an expression of the form is evaluated to \(e\) when \(e\) is a “completed” code fragment, i.e., does not contain further escapes. Such code fragments are dubbed as code values henceforth. An example evaluation step is shown in (c) above.

After repeated reduction, a program at stage 0 hopefully reaches a code value , which intuitively corresponds to the end of macro expansion. Then the bracketed expression \(e\) is “put down” to stage 0, and the evaluation of the expression starts in turn. Especially when the number of stages is \(2\), stage \(0\) is for code generation, and the generated program at stage \(1\) is for ordinary evaluation.

As an example multi-stage program, let us consider \( \texttt{ genpower } \), a function that takes a natural number \(n\) and produces code for the \(n\)-th power function. This function can be implemented like the following:

figure g

The application \( \texttt{genpower } \ \texttt{2 } \) is, for instance, evaluated as follows (which does not precisely conform to the operational semantics but describes overall steps):

figure h

A minimal type system for asserting the safety of staged computation is actually quite concise; it basically suffices to equip types of the form , which is the type for code fragments that will be expressions of type \(\tau \) at the next stage. For example, \( \texttt{ genpower } \) is assigned type since it takes an integer and returns a code value bracketing an expression of type \( \texttt{ int } \rightarrow \texttt{ int } \) at stage \(1\). Thanks to such typing, we can assert that programs do not get stuck during code generation and neither does the evaluation of produced code, once code-generating programs are successfully type-checked.

1.2 Point at Issue: Modularity

Although possibly used as an intermediate language to handle the results of some transformations such as those guided by binding-time analysis [30], MSP languages are intended to be written by hand as well. As long as written and read directly by users, multi-stage programs are desirably composed of loosely coupled smaller components for code maintainability. To this end, as widely known, ML-family languages such as Standard ML or OCaml conventionally have an encapsulation mechanism called a module system [7,8,9, 14, 19, 21, 23, 24]. One can naturally expect that such a module system is also useful for MSP.

In this paper, we propose a new module system for the purpose above. Our contributions can be summarized as follows: (1) Language design: We design a module system named MetaFM that enables us to decompose multi-stage programs in a manner natural with respect to type abstraction. Unlike some existing module systems equipped with staging features [16, 26, 35], our module system does not assign stages to modules; instead, it allows values at different stages to be bound in the same structure (i.e., \( \textbf{struct} \ \cdots \ \textbf{end} \)). We observe that this language design is crucial for the natural decomposition of multi-stage programs into loosely coupled components, as exemplified in Sects. 23. (2) Reconciliation of staging with full-fledged ML-style modules: Based on the language design above, we accommodate staging with some advanced features for modules such as higher-order functors, sealing, the \(\textbf{with}\ \textbf{type}\)-construct, or higher-kinded types. Our module system also supports cross-stage persistence (CSP) [33, 38], an important staging feature that enables the reuse of one common value at more than one stage. We give semantics to our language through an elaboration, i.e., a type-directed translation to a target language, and show that the elaboration is sound, i.e., that any target terms produced by the elaboration are well-typed. (3) Target type safety: As a target language, we define System F\(\omega ^{\langle \rangle }\), a multi-stage extension of System F\(\omega \) [11, 22], and prove its type safety. The language has its own stage polymorphism to provide CSP for MetaFM.

On the other hand, our method has the following limitations so far: (a) It does not support the \( \textbf{run} \)-primitive [13, 26, 32, 34, 35], which executes code like \( ( \textbf{run} \ ( \texttt{genpower } \ \texttt{3 } ) ) \ \texttt{5 } \longrightarrow ^*\texttt{ 125 } \). This is not a severe limitation because \( \textbf{run} \) is not necessary if one wants to do only compile-time code generation, and may well be covered by some techniques orthogonal to ours, such as \(\lambda ^{\triangleright }\) [34]. (b) It cannot handle first-class modules [20, 24, 25]. This is perhaps difficult to overcome within our language design because unpacking them can be done only at runtime while modules in our formalization are not staged and considered stage-\( 0 \). (c) It cannot straightforwardly extend with features that have side effects such as mutable references due to a lack of sophistication in the semantics. We have a promising solution to this issue nonetheless, and discuss it as part of our future work.

The rest of the paper is organized as follows: First, Sect. 2 displays examples that motivate our language design. Section 3 compares related work and ours, and discusses the necessity of the design. Section 4 formalizes the source language, omitting CSP for clarity for the moment. After Sect. 5 defines the target language and proves its type safety, Sect. 6 explains the elaboration rules and proves their soundness. Section 7 extends our language with CSP without breaking the previously proved properties. Finally, Sect. 8 mentions future work and its provisional implementation, and Sect. 9 concludes the paper. Many definitions and proofs are in Appendix due to space limitations.

Fig. 1.
figure 1

Example definition of modules in MetaFM

2 Motivating Examples

To showcase our motivation, consider a module \( \texttt{ Timestamp } \) that handles absolute timestamps as data of abstract type \( \texttt{Timestamp } . \texttt{t } \), under the two-stage setting, i.e., where we only have stages \( 0 \) and \( 1 \), which are for compile-time macro expansion and runtime, respectively. The module has, for instance, a predicate \( \texttt{ precedes } \) of type \( \texttt{ t } \rightarrow \texttt{ t } \rightarrow \texttt{ bool } \) that takes two timestamps and judges which is earlier. It would be useful if the module provides a macro \( \texttt{ gen } \) that transforms datetime texts into the corresponding timestamps beforehand like the following:

figure k

In our language, one can implement such \( \texttt{ Timestamp } \) as shown in Fig. 1(a)Footnote 1. Timestamps are represented by Unix time integers, and functions are implemented as simple arithmetics. The macro \( \texttt{ gen } \) is implemented by using an auxiliary function \( \texttt{ parseDatetime } \) of type \( \texttt{ string } \rightarrow \texttt{ option } \ \texttt{ int } \) at stage \(0\); Although its definition is omitted, \( \texttt{ parseDatetime } \) parses a text and returns \( \texttt{Some } \ \texttt{ts } \) if the given text is a valid datetime, where \( \texttt{ ts } \) is the corresponding Unix timestamp, or returns \( \texttt{ None } \) otherwise. Most importantly, \( \texttt{ gen } \) does not expose the internal representation of \( \texttt{Timestamp } . \texttt{t } \)-typed values, as the ordinary functions do not.

As a side note, two additional constructs excluded from the formalization are used in the example above. One is \( \textbf{fail} \), which simply aborts the program when evaluated. Note that allowing the use of \( \textbf{fail} \) at stage \(0\) is much less harmful than that at stage \(1\); such an abort occurs before runtime. The other is \( \textbf{lift} \ e \), which evaluates \(e\) to a value \(v\), and “lifts” it to for the next stage. We should note here that lifting and CSP are different despite their apparent resemblance, and that one cannot lift arbitrary values; lifting functions, for example, makes variable occurrences in the function body inconsistent as to their stage. It is thus desirable to rule out such lifting, but that topic is out of the scope of this paper.

Another example is shown in Fig. 1(b). We here implement a functor \( \texttt{ MakeMap } \) for handling (finite) maps, which is equivalent to OCaml’s Map.Make; it takes a module \( \texttt{ Key } \) of signature \( \texttt{ Ord } \), which requires \( \texttt{ Key } \) to have a type \( \texttt{Key } . \texttt{t } \) of mapping keys and a comparison function \( \texttt{Key } . \texttt{compare } \) of type \( \texttt{Key } . \texttt{t } \rightarrow \texttt{Key } . \texttt{t } \rightarrow \texttt{int } \) used for efficient access to values, and produces a module that provides a type \( \texttt{ t } \ \alpha \) for maps whose keys and values are of type \( \texttt{Key } . \texttt{t } \) and \(\alpha \), respectively. We add to the resulting module a macro \( \texttt{ gen } \) that produces a map from a list of key–value pairs beforehand. Since module expressions are not staged, we do not have any additional difficulty in the reconciliation of functors and staging features. One can use this macro as follows, for instance:

figure m

One important thing here is that \( \texttt{Key } . \texttt{compare } \) should be available both at stage \( 0 \) and \( 1 \) so that both \( \texttt{ gen } \) and ordinary functions can use it at compile-time and runtime, respectively. Such capability is called cross-stage persistence or CSP for short [33, 38] and is known as one of the vital staging features for practical use. We support it by bindings of the form \( \textbf{val} ^{\ge n } \ X = E \ \), which defines a value \(X\) for any stage \(n'\) such that \(n' \ge n\). Based on this language design, \( \texttt{ Ord } \) will be \( ( \textbf{sig} \ \textbf{type} \ \texttt{t } \, {::} \, *; \textbf{val} ^{\ge 0 }\ \texttt{compare } : \texttt{t } \rightarrow \texttt{t } \rightarrow \texttt{int } \ \textbf{end} ) \), for example.

3 Related Work and Our Approach

There are some existing studies that mix staging features with module systems, though their goals differ from ours: Inoue et al. [16] indicated that by staging modules we can eliminate abstraction overheads due to the use of functors, and then Watanabe et al. [35] and Sato et al. [26] followed the approach and proposed the formalization of such type systems. Despite the difference in the purpose, it is apparently worth considering that we can possibly utilize them for our goal. However, such reuse seems somewhat unsatisfactory; to implement modules equivalent to \( \texttt{ Timestamp } \) (or \( \texttt{ MakeMap } \)) in such languages where only entire modules are staged, one could do one of the following: (1) separate the module into two, i.e., \( \texttt{ Timestamp } \) at stage \( 1 \) for ordinary functions, and \( \texttt{ GenTimestamp } \) at stage \( 0 \) for defining macros; (2) do basically the same as (1), but define the type for representing timestamps internally in a separate module \( \texttt{ TimestampImpl } \), and both \( \texttt{ Timestamp } \) and \( \texttt{ GenTimestamp } \) include it; or (3) define \( \texttt{ Timestamp } \) at stage \( 0 \), and ordinary functions are bound as code values. Each option has a kind of drawback, unfortunately. First, consider implementing the macro \( \texttt{ gen } \) in \( \texttt{ GenTimestamp } \) based on (1). To assign type to the macro, we must provide a “backdoor” function \( \lambda x .\ x \) as \( \texttt{Timestamp } . \texttt{make } \) of type \( \texttt{int } \rightarrow \texttt{Timestamp } . \texttt{t } \) and leave its application in code fragments produced by \( \texttt{ gen } \). Things get worse when considering the \( \texttt{ MakeMap } \) example; we cannot even provide such a backdoor so that \( \texttt{ GenMakeMap } \) can use it (at least when functors are generative, not applicative). Option (2) seems better in that it does not require a backdoor, but it is a kind of expediency; it reveals \( \texttt{ TimestampImpl } \) (or \( \texttt{ MapImpl } \)) to outside and thus requires another mechanism than modules that conceals the implementation. Option (3) is good in terms of modularity, but it makes every occurrence of ordinary values at stage \( 1 \) be like .

Perhaps some of the most similar existing work would be Modular Macros [37] and MacoCaml [36]; the former informally suggests a language design similar to ours by giving some examples, and the latter gives a formalization that supports a subset of that language design. MacoCaml offers structures (i.e., \( \textbf{struct} \ \cdots \ \textbf{end} \)) in which both values and macros can be bound, and supports CSP by level-shifting imports \(\textbf{import}^{\downarrow }\) inspired by Flatt [10]. Major differences between ours and MacoCaml are that the number of stages in a structure is not limited to two, that the type abstraction by signatures is taken into account, and that functors can be handled. Aside from how semantics is precisely defined, ours can perhaps be seen as an extension of MacoCaml (without mutable references) with functors, higher-kinded types, sealing, the \(\textbf{with}\ \textbf{type}\)-construct, etc.

A technical challenge lies in giving semantics to our language and, at the same time, proving that our language is type-safe and does not break type abstraction. Indeed, defining semantics conforming to full-fledged ML-style module systems has long been an issue by itself [4, 7, 14, 15, 23, 27]. Among the line of studies, F-ing Modules [23, 24] elegantly formalizes such semantics through an elaboration (i.e., a type-directed translation) to System F\(\omega \) [11, 22]. This approach seems better than giving semantics directly on module expressions in that it does not suffer at all from the avoidance problem [4, 7, 14] caused by locally defined types under the combination of sealing (\( X \mathrel {:>} S \)) and projection (\( M . X \)).

To define semantics for our language, we follow the elaboration approach taken by F-ing Modules. Specifically, we translate the source language MetaFM into System F\(\omega ^{\langle \rangle }\), a multi-stage extension of System F\(\omega \). This is in contrast to MacoCaml, which gives semantics directly to module expressions and might well induce the avoidance problem when extending with features of full-fledged ML-style modules such as sealing and projection. Although the elaboration intensively utilizes the existential quantification offered by System F\(\omega ^{\langle \rangle }\) to demonstrate that type abstraction is properly done, its operational essence is rather simple; in a sense, our elaboration performs the option (3) above internally.

4 Source Language

The following defines the entire syntax for our source language MetaFM, where meta-level notations \([\mu ]^{*}\) and \([\mu \mapsto \nu ]\) range over the (possibly empty) finite sequences of \(\mu \) and the finite maps from \(\mu \) to \(\nu \), respectively:

figure p

The metavariables \(M\), \(S\), \(B\), and \(D\) stand for modules, signatures, bindings, and declarations, respectively. For brevity, we use the same metavariable \(X\) in common for names of values, types, and modules bound as members of structures. The module language is quite similar to that of F-ing Modules [23, 24]; modules consist of identifiers \(X\), projections \( M . X \), structures \( \textbf{struct} \ \boldsymbol{B} \ \textbf{end} \), functor abstractions \( \textbf{fun} ( X : S ) \rightarrow M \), functor applications \( \ X_{{\textrm{1}}} \ X_{{\textrm{2}}} \), and sealing \( X \mathrel {:>} S \). The sole essential difference is that value bindings \( \textbf{val} ^{ n }\ X = E \) and declarations \( \textbf{val} ^{ n }\ X : T \) have an annotation \(n\) that specifies for which stage the value \(X\) is defined. The binding forms and \( \textbf{val} \ X = E \ \) used in Sect. 1 were actually syntax sugars of \( \textbf{val} ^{ 0 }\ X = E \) and \( \textbf{val} ^{ 1 }\ X = E \), respectively. We do not specify the core language in detail, but both expressions \(E\) and types \(T\) are equipped with paths \(P\) to items in structures (e.g. \( \texttt{Timestamp } . \texttt{precedes } \)). We only formalize generative functors (i.e., ones that produce fresh abstract types each time even if applied to the same module); we can perhaps handle applicative ones as well in an F-ing Modules-like manner, but omit them for simplicity.

For defining elaboration rules, we employ semantic signatures \( \varSigma \) and \(\xi \) [23, 24], as internal representations, and make type environments \( \varGamma \) track them:

figure r

where \(n\) ranges over the set of stage numbers (i.e. the set of natural numbers). Although \(s\) ranges over exactly the same set as \(n\) for the moment, it will be extended for CSP in Sect. 7. Basically, and correspond to value and type items in structures, respectively, and \( \mathopen {\{\!|} R \mathclose {|\!\} } \) and \( \forall \boldsymbol{b} .\ \varSigma \rightarrow \xi \) work respectively as structure and functor signatures. Existentially quantified type variables intuitively correspond to abstract types. For simplicity, we assume that source variables \(X\) can be injectively embedded into the set of labels as \( l_{ X } \).

Fig. 2.
figure 2

Signature elaboration rules (selective; see Fig. 11 for omitted ones)

Figure 2 displays the elaboration rules for the judgment \( \varGamma \vdash S \rightsquigarrow \xi \), which converts syntactic signature \(S\) into semantic signature \(\xi \). Among the rules, only D-Val is essentially new, compared to those of F-ing Modules [23, 24]; it handles declarations of value items of type \(\tau \) at stage \(n\) by signatures .

The set of typing rules for modules and bindings is displayed in Fig. 3. The judgment \( \varGamma \vdash M : \xi \rightsquigarrow e \) intuitively states that \(M\) is assigned signature \(\xi \) under type environment \( \varGamma \), aside from the elaboration part \(\rightsquigarrow e\) for the moment; indeed, one can read the rules just ignoring the portions with a gray background. We explain how the elaboration works later in Sect. 5. While most of the rules are essentially the same as those of F-ing Modules, only B-Val is new; it type-checks the left-hand side \(E\) of a binding \( \textbf{val} ^{ n }\ X = E \) as an expression at stage \(n\), and assigns signature to the value item \(X\), where \(\tau \) is the resulting type.

Fig. 3.
figure 3

Module elaboration rules (selective; see Fig. 10 for omitted ones)

Although we do not fix the core language, Fig. 4 displays foundational or basic rules for judgments such as \( \varGamma \vdash ^{ s } E : \tau \rightsquigarrow e \). Among the rules, E-Path essentially uses signatures for value items; it limits the occurrence of paths by stage number \(n\) as well as by types so that value items are used only at the stage for which they are bound. The rules for staging, i.e., E-Brkt and E-Esc, are fairly standard as an MSP language; an expression of the form has a code type if the subexpression \(E\) is assigned the type \(\tau \) at the next stage, and a subexpression \(E\) of is expected to be of type at the previous stage.

As is usual with module systems, the rules M-App and M-Seal depend on signature matching \( \varGamma \vdash \varSigma \leqslant \exists \boldsymbol{b} .\ \varSigma ' \uparrow \boldsymbol{\tau } \rightsquigarrow e \), where \(\boldsymbol{\tau }\) ranges over the set of finite sequences of types. This judgment intuitively asserts that \( \varSigma \) can be a subtype of \( \exists \boldsymbol{b} .\ \varSigma ' \) when the type variables in \(\boldsymbol{b}\) (i.e., abstract types) are respectively instantiated by the types listed in \(\boldsymbol{\tau }\). Selected rules for signature matching are shown in Fig. 5. Again, only U-Val is essentially new; the others are exactly the same as the corresponding rules of F-ing Modules.

What we should note lastly is decidability. While most of the rules are syntax-directed, U-Val requires type equivalence \( \tau \equiv \tau ' \), and U-Match depends on the guess at each \(\tau _{ i }\), which makes the decidability of the whole type-checking non-trivial. The former is easy: we can prove that the evident type-level reduction relation on well-kinded types is confluent and strongly normalizing, and can thus check type equivalence by comparing normal forms. The latter requires a more complicated technique, but we can indeed infer each \(\tau _{ i }\) as discussed in [24].

In the forthcoming two sections, we define a target language, see how the elaboration part \(\rightsquigarrow e\) works, and show that \(e\) is well-typed in the target language.

5 Target Language and Its Type Safety

This section introduces System F\(\omega ^{\langle \rangle }\), a multi-stage extension of System F\(\omega \) [11, 22] used as a target language. The following defines the syntax:

figure ab

Expressions \(e\) have bracket and escape for staging in addition to the standard constructs for typed lambda calculi, record construction \( \{ l_{{\textrm{1}}} = e_{{\textrm{1}}} , \ldots , l_{ m } = e_{ m } \} \) (where the labels are assumed to be pairwise distinct), record projection \( e \# l \), type variable abstraction \( \mathrm {\Lambda } \alpha \mathrel {::} \kappa .\ e \), type application \( e \ \tau \), and pack/unpack expressions for existential quantification. Higher-kinded types \(\tau \) consist of type variables \(\alpha \), function types \( \tau \rightarrow \tau \), record types \( \{ r \} \), type-level abstractions \( \mathrm {\Lambda } \alpha \mathrel {::} \kappa .\ \tau \), type-level applications \( \tau \ \tau \), existential (resp. universal) quantification \( \exists \alpha \mathrel {::} \kappa .\ \tau \) (resp. \( \forall \alpha \mathrel {::} \kappa .\ \tau \)), and code types .

Figure 6 shows the typing rules for \( \gamma \vdash ^{ s } e : \tau \), which states that \(e\) is assigned type \(\tau \) at stage \(s\) under the type environment \(\gamma \). The judgments \( \vdash \gamma \) and \( \gamma \vdash \tau \mathrel {::} \kappa \), which are defined in Appendix, assert that \(\gamma \) is well-formed and that \(\tau \) is assigned the kind \(\kappa \) under \(\gamma \), respectively. The set of rules is basically a natural integration of System F\(\omega \) with staging features, but pack/unpack-expressions are allowed only at stage \( 0 \). This is just because it suffices for our purpose; we could possibly allow them at arbitrary stages, but we don’t have to.

Fig. 4.
figure 4

Core language elaboration rules

Fig. 5.
figure 5

Signature subtyping rules (selective; see Fig. 12 for omitted ones)

Fig. 6.
figure 6

Typing rules for System F\(\omega ^{\langle \rangle }\) (selective; see Fig. 14 for omitted ones)

Fig. 7.
figure 7

Operational semantics of System F\(\omega ^{\langle \rangle }\) (selective; see Fig. 15 for omitted ones)

The small-step call-by-value operational semantics of System F\(\omega ^{\langle \rangle }\) is shown in Fig. 7. The judgment \( e \, {\mathop {\longrightarrow }\limits ^{ n }}\, e' \) stands for the reduction of the expression \(e\) at stage \(n\). As is usual in multi-stage languages, essential reductions, such as ordinary \(\beta \)-reduction or access to record fields, are defined only at stage \(0\), and the cancellation of brackets by escapes happens only at stage \(1\). Here, values \( v ^{( 0 )} \) (resp. \( v ^{( n )} \)) at stage \( 0 \) (resp. at stage \(n \ge 1 \)) are defined by the following:

figure af

We have the following standard type safety properties of System F\(\omega ^{\langle \rangle }\). Here, we write \( \vdash ^{\ge n } \gamma \) if all entries in \(\gamma \) of the form \(x : \tau ^{ s } \) satisfy \(s \ge n\). Since the language has type equivalence, we have to take care of the so-called inversion lemma by using an argument similar to the one in Chapter 30 of [22].

Theorem 1

(Preservation). If \( \gamma \vdash ^{ n } e : \tau \) and \( e \,{\mathop {\longrightarrow }\limits ^{ n }} \, e' \), then \( \gamma \vdash ^{ n } e' : \tau \).

Theorem 2

(Progress). If \( \vdash ^{\ge 1 } \gamma \) and \( \gamma \vdash ^{ n } e : \tau \), then \(e\) is a value at stage \(n\), or there exists \(e'\) such that \( e \,{\mathop {\longrightarrow }\limits ^{ n }} \, e' \).

6 Elaboration and Its Soundness

This section explains the elaboration of MetaFM programs into System F\(\omega ^{\langle \rangle }\), i.e., discusses the \(\rightsquigarrow e\) part of the rules in Figs. 3, 4, and 5.

Taking the elaboration part into account, the judgment \( \varGamma \vdash ^{ s } E : \tau \rightsquigarrow e \) intuitively states that the core language expression \(E\) at stage \(s\) is translated to the term \(e\) at stage \(s\) in System F\(\omega ^{\langle \rangle }\), in addition to typing \(E\). Basically the same holds for the judgment \( \varGamma \vdash M : \xi \rightsquigarrow e \); it illustrates that the module \(M\) is converted to the term \(e\). What should be noted here is that the resulting terms \(e\) of the module elaboration are at stage \( 0 \) in System F\(\omega ^{\langle \rangle }\); through elaboration, we virtually deal with modules as if they were at stage \( 0 \). In this sense, particularly, functor applications are resolved by stage-\( 0 \) computation. Lastly, terms \(e\) produced by judgments for signature subtyping such as \( \varGamma \vdash \varSigma \leqslant \varSigma ' \rightsquigarrow e \) are intuitively “upcast functions” from the subtype to the supertype. This intuition is justified afterwards by Theorem 3. For simplicity, in Figs. 3, 4, and 5, we assume that source variables \(X\) can be injectively embedded into the target variables, which is ranged over by \(x\). The figures also use the following syntax sugars as well as let-expressions and pack/unpack-expressions generalized for sequences, the precise definitions of which are shown in Appendix:

figure ag

We embed semantic signatures into System F\(\omega ^{\langle \rangle }\) types by the following \(\lfloor -\rfloor \), and use them for describing the rules and for proving type safety:

figure ah

The intuition for the elaboration rules involved in staging is fairly easy; leaving types out of account, B-Val and E-Path convert bindings \( \textbf{val} ^{ n }\ X = E \) and variable occurrences \(X\) at stage \(n\) into and in essence, respectively. The rule U-Val does similar things for building upcast functions.

Example 1

The elaboration translates \( \texttt{ Timestamp } \) to the following in essence:

figure ak

Although the translation above suffices for giving semantics that fulfills type safety as shown by the theorems below, we do not assert that resulting terms are evaluated in the same manner as programmers’ intuition on source programs. This is a common downside of the elaboration approach, which defines semantics only through translationFootnote 2. Indeed, our translation sometimes causes a counterintuitive evaluation order due to its naïveness; it does not bind identifiers to values but to code fragments in general. This might be fine for typical items defined by immediate values such as lambda abstractions, but some cases are essentially unsatisfactory. For example, one may expect that \( \textbf{val} ^{ 1 }\ \texttt{a } = \texttt{1 } + \texttt{2 } \) computes \( \texttt{1 } + \texttt{2 } \) once and replaces all the occurrences of \( \texttt{ a } \) with \( \texttt{ 3 } \) at runtime, but this is not the case; it replaces \( \texttt{ a } \) with the expression \( \texttt{1 } + \texttt{2 } \) at compile-time. In a sense, value items for stage \(\ge 1 \) are used in a CBN-like manner. For the same reason, the translation prevents the extension with features that have side effects, such as mutable referencesFootnote 3, in a straightforward manner. It may also enlarge the generated code and make it less performant since code fragments are copied to every occurrence. Section 8 discusses possible improvements in the elaboration.

Fig. 8.
figure 8

Extension of System F\(\omega ^{\langle \rangle }\) with stage polymorphism

Nonetheless, we have the following theorems that prove that the elaboration is sound in the sense that every produced term is well-typed (under some moderate requirements for the core language; see Assumption 5 in Appendix).

Theorem 3

(Soundness of Signature Subtyping). If \( \varGamma \vdash \varSigma \leqslant \exists \boldsymbol{b} .\ \varSigma ' \uparrow ( \tau _{ i } ) _{i = 1}^{m} \rightsquigarrow e \) and \( \lfloor \varGamma \rfloor , \boldsymbol{b} \vdash \lfloor \varSigma ' \rfloor \mathrel {::} *\), then \( \lfloor \varGamma \rfloor \vdash ^{ 0 } e : \lfloor \varSigma \rfloor \rightarrow \lfloor [ \tau _{ i } / \alpha _{ i } ] _{i = 1}^{m} \varSigma ' \rfloor \) and \( \lfloor \varGamma \rfloor \vdash \tau _{ i } \mathrel {::} \kappa _{ i } \) for each \(i\), where \(\boldsymbol{b} = ( \alpha _{ i } {::} \kappa _{ i } ) _{i = 1}^{m} \).

Theorem 4

(Soundness of Elaboration).

  1. 1.

    \( \varGamma \vdash T {::} \kappa \rightsquigarrow \tau \) implies \( \lfloor \varGamma \rfloor \vdash \tau \mathrel {::} \kappa \).

  2. 2.

    \( \varGamma \vdash ^{ s } E : \tau \rightsquigarrow e \) implies \( \lfloor \varGamma \rfloor \vdash ^{ s } e : \tau \).

  3. 3.

    \( \varGamma \vdash P : \varSigma \rightsquigarrow e \) implies \( \lfloor \varGamma \rfloor \vdash ^{ 0 } e : \lfloor \varSigma \rfloor \).

  4. 4.

    \( \varGamma \vdash D \rightsquigarrow \exists \boldsymbol{b} .\ R \) (resp. \( \varGamma \vdash \boldsymbol{D} \rightsquigarrow \exists \boldsymbol{b} .\ R \)) implies \( \lfloor \varGamma \rfloor \vdash \lfloor \exists \boldsymbol{b} .\ \mathopen {\{\!|} R \mathclose {|\!\} } \rfloor \mathrel {::} *\).

  5. 5.

    \( \varGamma \vdash B : \exists \boldsymbol{b} .\ R \rightsquigarrow e \) (resp. \( \varGamma \vdash \boldsymbol{B} : \exists \boldsymbol{b} .\ R \rightsquigarrow e \)) implies \( \lfloor \varGamma \rfloor \vdash ^{ 0 } e : \lfloor \exists \boldsymbol{b} .\ \mathopen {\{\!|} R \mathclose {|\!\} } \rfloor \).

  6. 6.

    \( \varGamma \vdash S \rightsquigarrow \xi \) implies \( \lfloor \varGamma \rfloor \vdash \lfloor \xi \rfloor \mathrel {::} *\).    7.  \( \varGamma \vdash M : \xi \rightsquigarrow e \) implies \( \lfloor \varGamma \rfloor \vdash ^{ 0 } e : \lfloor \xi \rfloor \).

7 Extension with Cross-Stage Persistence

As mentioned earlier in Sect. 1, we support cross-stage persistence (CSP) [13, 33, 38] by the following binding (resp. declaration) form, which binds (resp. declares) \(X\) as a value available at any stages \(n'\) such that \(n' \ge n\):

figure al

To achieve this, we first extend System F\(\omega ^{\langle \rangle }\) with stage variables \(\sigma \):

figure am

Intuitively, stage variables work for “stage polymorphism” and can be instantiated with an arbitrary natural number \(k\). Stages \(s\) can newly be of the form \( n \dotplus \sigma \), which stands for any stages greater than or equal to \(n\). Bracket with a stage variable expresses arbitrarily nested brackets and is instantiated to . We correspondingly have escape , which can be instantiated to . Stage abstractions \( ( \mathrm {\Lambda } \sigma .\ e ) \) and stage applications \( e \mathbin {\uparrow } s \) perform generalization and instantiation of the stage variables, respectively. For typing, we use stage-polymorphic types \( \forall \sigma .\ \tau \) and persistent code types . Figure 8 displays additional rules for extending System F\(\omega ^{\langle \rangle }\) with the stage polymorphism. The rule TT-BrktVar allows brackets with a stage variable \(\sigma \) to be used in “fixed” stages \(n\) (as long as \(\sigma \) is valid in that scope), and expressions inside them are regarded as being at “polymorphic” stages \( n \dotplus \sigma \). TT-EscVar does the converse by requiring persistent code types to expressions inside escapes with \(\sigma \). One may see that stage variables are a minimal version of transition variables [13, 34] or environment classifiers [31]. We can keep the stage polymorphism minimal here because we do not have to provide something like \(\sigma + \sigma '\) for the elaboration. Most importantly, adding these rules can be done without breaking the type safety of System F\(\omega ^{\langle \rangle }\), i.e., Theorems 1 and 2.

Fig. 9.
figure 9

Extension of MetaFM with cross-stage persistence

Now that the target language supports stage polymorphism, we can utilize it to support CSP in MetaFM by extending rules as displayed in Fig. 9. Here, we add a new semantic signature for persistent value items that can be embedded into System F\(\omega ^{\langle \rangle }\) by using stage-polymorphic types, and \( \varGamma \) newly tracks stage variables for the soundness of the expression elaboration:

figure au

An essential part of the extended rules lies in those for paths in expressions, i.e., E-Path, E-PersInNonpers, and E-PersInPers. These rules permit paths to persistent values to occur at any expressions but prevent definitions of persistent value items from depending on non-persistent ones in order for CSP to work correctly, i.e., we do not allow expressions \(E\) of \( \textbf{val} ^{\ge n } \ X = E \ \) to contain paths to non-persistent values. Again, owing to such typing, we successfully extend MetaFM with CSP without breaking the soundness shown by Theorems 3 and 4.

8 Future Work and Provisional Implementation

Though we have successfully given semantics to our language and proved its type safety, several aspects can be improved further. To remedy the issues pointed out in Sect. 6, we should desirably modify the translation about when to bind value items. Possible solutions would be the following: (1) use the genlet primitive [17, 26] for performing let-insertion during code generation; or (2) define conversion of programs into a flat list of bindings of the form \(\textbf{val} ^{n}\ x = e\) with functor applications resolved, by using static interpretation (which is dubbed as SI here) [1, 8, 9]. Because the former appears less suitable for proving type safety in that it complicates formal semantics, we have been intensively studying the latter. Our ongoing study is implying that mixing staging features with SI is fine, but the soundness of SI itself seems not so well-established. Elsman [8] first showed the soundness of SI for first-order functors, which is quite sufficient for typical use cases, but it did not support higher-order ones. The SI for Futhark [9] claims its support for higher-order ones, but it seems that its current mechanized proof only covers functors assigned a signature of the form \( \forall \varepsilon .\ \varSigma \rightarrow \exists \varepsilon .\ \varSigma ' \) in essence.

Although its type safety has not yet been proved, we implemented a two-stage versionFootnote 4 of MetaFM with an SI-based elaboration for [29]. [28] is a statically-typed domain-specific language for typesetting documents where commands for the markup, which are equivalent to control sequences in like , can be implemented in an OCaml-like manner. Programs in this language are basically functional, but mutable references are exceptionally used for numbering sections, for example. Because the module system incorporated in appears working fine with mutable references so far, we believe that the SI approach is promising for solving the binding-time issue.

9 Conclusion

We have proposed MetaFM, a module system for decomposing multi-stage programs into loosely coupled components without breaking type abstraction, by defining semantics and proving its type safety through an elaboration to System F\(\omega ^{\langle \rangle }\). It supports several important features such as sealing, functors, higher-kinded types, or CSP. Further improvements on the elaboration are nonetheless desirable for real-world use, which can probably be done by static interpretation.