A Formal C Memory Model for Separation Logic
 1.6k Downloads
Abstract
The core of a formal semantics of an imperative programming language is a memory model that describes the behavior of operations on the memory. Defining a memory model that matches the description of C in the C11 standard is challenging because C allows both highlevel (by means of typed expressions) and lowlevel (by means of bit manipulation) memory accesses. The C11 standard has restricted the interaction between these two levels to make more effective compiler optimizations possible, at the expense of making the memory model complicated. We describe a formal memory model of the (nonconcurrent part of the) C11 standard that incorporates these restrictions, and at the same time describes lowlevel memory operations. This formal memory model includes a rich permission model to make it usable in separation logic and supports reasoning about program transformations. The memory model and essential properties of it have been fully formalized using the Coq proof assistant.
Keywords
ISO C11 standard C verification Memory models Separation logic Interactive theorem proving Coq1 Introduction

Reading a value at a given address.

Storing a value at a given address.

Allocating a new object to hold a local variable or storage obtained via malloc.

Deallocating a previously allocated object.
This duality makes the memory model of C more complicated than the memory model of nearly any other programming language. For example, more mathematically oriented languages such as Java and ML feature only highlevel data access, in which case the memory can be modeled in a relatively simple and structured way, whereas assembly languages feature only lowlevel data access, in which case the memory can be modeled as an array of bits.
The situation becomes more complicated as the C11 standard allows compilers to perform optimizations based on a highlevel view of data access that are inconsistent with the traditional lowlevel view of data access. This complication has lead to numerous ambiguities in the standard text related to aliasing, uninitialized memory, endofarray pointers and typepunning that cause problems for C code when compiled with widely used compilers. See for example the message [42] on the standard committee’s mailing list, Defect Reports #236, #260, and #451 [26], and the various examples in this paper.
Contribution This paper describes the \(\mathrm{CH}_2\mathrm{O}\) memory model, which is part of the the \(\mathrm{CH}_2\mathrm{O}\) project [30, 31, 32, 33, 34, 35, 36, 37]. \(\mathrm{CH}_2\mathrm{O}\) provides an operational, executable and axiomatic semantics in Coq for a large part of the nonconcurrent fragment of C, based on the official description of C as given by the C11 standard [27].

Close to C11 \(\mathrm{CH}_2\mathrm{O}\) is faithful to the C11 standard in order to be compiler independent. When one proves something about a given program with respect to \(\mathrm{CH}_2\mathrm{O}\), it should behave that way with any C11 compliant compiler (possibly restricted to certain implementationdefined choices).

Static type system Given that C is a statically typed language, \(\mathrm{CH}_2\mathrm{O}\) does not only capture the dynamic semantics of C11 but also its type system. We have established properties such as type preservation of the memory operations.

Proof infrastructure All parts of the \(\mathrm{CH}_2\mathrm{O}\) memory model and semantics have been formalized in Coq (without axioms). This is essential for its application to program verification in proof assistants. Also, considering the significant size of \(\mathrm{CH}_2\mathrm{O}\) and its memory model, proving metatheoretical properties of the language would have been intractable without the support of a proof assistant.
Despite our choice to use Coq, we believe that nearly all parts of \(\mathrm{CH}_2\mathrm{O}\) could be formalized in any proof assistant based on higherorder logic.

Executable To obtain more confidence in the accuracy of \(\mathrm{CH}_2\mathrm{O}\) with respect to C11, the \(\mathrm{CH}_2\mathrm{O}\) memory model is executable. An executable memory model allows us to test the \(\mathrm{CH}_2\mathrm{O}\) semantics on example programs and to compare the behavior with that of widely used compilers [33, 37].

Separation logic In order to reason about concrete C programs, one needs a program logic. To that end, the \(\mathrm{CH}_2\mathrm{O}\) memory model incorporates a complex permission model suitable for separation logic. This permission system, as well as the memory model itself, forms a separation algebra.

Memory refinements \(\mathrm{CH}_2\mathrm{O}\) has an expressive notion of memory refinements that relates memory states. All memory operations are proven invariant under this notion. Memory refinements form a general way to validate many commonsense properties of the memory model in a formal way. They also open the door to reasoning about program transformations, which is useful if one were to use the memory model as part of a verified compiler frontend.
Parts of this paper also appear in the author’s PhD thesis [33], which describes the entire \(\mathrm{CH}_2\mathrm{O}\) project including its operational, executable and axiomatic semantics, and metatheoretical results about these.

Unspecified behavior: two or more behaviors are allowed. For example: the execution order of expressions. The choice may vary for each use of the construct.

Implementationdefined behavior: like unspecified behavior, but the compiler has to document its choice. For example: size and endianness of integers.

Undefined behavior: the standard imposes no requirements at all, the program is even allowed to crash. For example: dereferencing a NULL pointer, or signed integer overflow.
Modeling underspecification in a formal semantics is folklore: unspecified behavior corresponds to nondeterminism, implementationdefined behavior corresponds to parameterization, and undefined behavior corresponds to a program having no semantics. However, the extensive amount of underspecification in the C11 standard [27, Annex J], and especially that with respect to the memory model, makes the situation challenging. We will give a motivating example of subtle underspecification in the introduction of this paper. Section 3 provides a more extensive overview.
When f is called with pointers p and q that are aliased, the assignment to *p also affects *q. As a result, one cannot transform the function body of f into the shorter *p = 10; return (*q);. The shorter function will return 10 in case p and q are aliased, whereas the original f will always return the original value of *q.
Unlike this example, there are many situations in which pointers can be assumed not to alias. It is essential for an optimizing compiler to determine where aliasing cannot occur, and use this information to generate faster code. The technique of determining whether pointers can alias or not is called alias analysis.
Here, a compiler is allowed to assume that p and q are not aliased because they point to objects of different types. The compiler is therefore allowed to transform the function body of g into the shorter *p = 10; return (*q);.
The above program is valid according to the rules of the C11 type system, but has undefined behavior during execution of g. This is caused by the standard’s notion of effective types [27, 6.5p67] (also called strictaliasing restrictions) that assigns undefined behavior to incorrect usage of aliased pointers to different types.
The assignment *p = 10 violates the rules for effective types. The memory area where p points to contains a union whose variant is y of type short, but is accessed through a pointer to variant x of type int. This causes undefined behavior.
Effective types form a clear tension between the lowlevel and highlevel way of data access in C. The lowlevel representation of the memory is inherently untyped and unstructured and therefore does not contain any information about variants of unions. However, the standard treats the memory as if it were typed.
Approach Most existing C formalizations (most notably Norrish [45], Leroy et al. [39, 40] and Ellison and Roşu [19]) use an unstructured untyped memory model where each object in the formal memory model consists of an array of bytes. These formalizations therefore cannot assign undefined behavior to violations of the rules for effective types, among other things.
In order to formalize the interaction between lowlevel and highlevel data access, and in particular effective types, we represent the formal memory state as a forest of welltyped trees whose structure corresponds to the structure of data types in C. The leaves of these trees consist of bits to capture lowlevel aspects of the language.
 Memory trees (Sect. 6.3) are used to represent each object in memory. They are abstract trees whose structure corresponds to the shape of C data types. The memory tree of struct S { short x, *r; } s = {33,&s.x } might be (the precise shape and the bit representations are implementation defined): The leaves of memory trees contain permission annotated bits (Sect. 6.2). Bits are represented symbolically: the integer value 33 is represented as its binary representation 1000010000000000, the padding bytes as symbolic indeterminate bits Open image in new window (whose actual value should not be used), and the pointer &s.x as a sequence of symbolic pointer bits.
The memory itself is a forest of memory trees. Memory trees are explicit about type information (in particular the variants of unions) and thus give rise to a natural formalization of effective types.

Pointers (Sect. 6.1) are formalized using paths through memory trees. Since we represent pointers as paths, the formal representation contains detailed information about how each pointer has been obtained (in particular which variants of unions were used). A detailed formal representation of pointers is essential to describe effective types.
 Abstract values (Definition 6.4) are trees whose structure is similar to memory trees, but have base values (mathematical integers and pointers) on their leaves. The abstract value of struct S { short x, *r; } s = { 33,&s.x } is: Abstract values hide internal details of the memory such as permissions, padding and object representations. They are therefore used in the external interface of the memory model and throughout the operational semantics.
Abstract values  Memory trees  Bits with permissions  

Permissions  
Padding  Always Open image in new window  
Variants of union  
Mathematical values 
This table indicates that abstract values and sequences of bits are complementary. Memory trees are a middle ground, and therefore suitable to describe both the lowlevel and highlevel aspects of the C memory.

Section 3 describes some challenges that a C11 memory model should address; these include endofarray pointers, bytelevel operations, indeterminate memory, and typepunning.

Section 4 describes the types of C. Our formal development is parameterized by an abstract interface to characterize implementationdefined behavior.

Section 5 describes the permission model using a variant of separation algebras that is suitable for formalization in Coq. The permission model is built compositionally from simple separation algebras.

Section 6 contains the main part of this paper, it describes a memory model that can accurately deal with the challenges posed in Sect. 3.

Section 7 demonstrates that our memory model is suitable for formal proofs. We prove that the standard’s notion of effective types has the desired effect of allowing typebased alias analysis (Sect. 7.1), we present a method to reason compositionally about memory transformations (Sect. 7.2), and prove that the memory model has a separation algebra structure (Sect. 7.4).

Section 8 describes the Coq formalization: all proofs about our memory model have been fully formalized using Coq.
2 Notations
This section introduces some common mathematical notions and notations that will be used throughout this paper.
Definition 2.1
We let \(\mathbb {N}\) denote the type of natural numbers (including 0), let \(\mathbb {Z}\) denote the type of integers, and let \(\mathbb {Q}\) denote the type of rational numbers. We let Open image in new window denote that \(i \in \mathbb {N}\) is a divisor of \(j \in \mathbb {N}\).
Definition 2.2
We let \(\mathsf{Prop}\) denote the type of propositions, and let \(\mathsf{bool}\) denote the type of Booleans whose elements are \(\mathsf{true}\) and \(\mathsf{false}\). Most propositions we consider have a corresponding Booleanvalued decision function. In Coq we use type classes to keep track of these correspondences, but in this paper we leave these correspondences implicit.
Definition 2.3
We let \(\mathsf{option}\;A\) denote the option type over A, whose elements are inductively defined as either \(\bot \) or \(x\) for some \(x \in A\). We implicitly lift operations to operate on the option type, and often omit cases of definitions that yield \(\bot \). This is formally described using the option monad in the Coq formalization.
Definition 2.4
A partial function f from A to B is a function \(f : A \rightarrow \mathsf{option}\;B\).
Definition 2.5
A partial function f is called a finite partial function or a finite map if its domain \(\mathsf{dom}\;f :=\{ x \;\;\exists y\in B\,.\,f\,x = y\} \) is finite. The type of finite partial functions is denoted as \(A \rightarrow _{\mathsf{fin}} B\). The operation \({f}[ {x} := {y} ]\) yields f with the value y for argument x.
Definition 2.6
We let \(A \times B\) denote the product of types A and B. Given a pair \((x,\,y) \in A \times B\), we let \({(x,\,y)}_{\mathbf {1}} :=x\) and \({(x,\,y)}_{\mathbf {2}} :=y\) denote the first and second projection of \((x,\,y)\).
Definition 2.7
We let \(\mathsf{list}\;A\) denote the list type over A, whose elements are inductively defined as either \(\varepsilon \) or \(x\,{x}\) for some \(x \in A\) and \({{x}} \in \mathsf{list}\;A\). We let \(x_i \in A\) denote the ith element of a list \({{x}} \in \mathsf{list}\;A\) (we count from 0). Lists are sometimes denoted as \([\, x_0,\ldots ,x_{n1} \,] \in \mathsf{list}\;A\) for \(x_0,\ldots ,x_{n1} \in A\).

We often implicitly lift a function \(f : A_0 \rightarrow \cdots \rightarrow A_n\) pointwise to the function \(f : \mathsf{list}\;A_0 \rightarrow \cdots \rightarrow \mathsf{list}\;A_n\). The resulting list is truncated to the length of the smallest input list in case \(n > 1\).

We often implicitly lift a predicate \(P : A_0 \rightarrow A_{n1} \rightarrow \mathsf{Prop}\) to the predicate \(P : \mathsf{list}\;A_0 \rightarrow \cdots \rightarrow \mathsf{list}\;A_{n1} \rightarrow \mathsf{Prop}\) that guarantees that P holds for all (pairs of) elements of the list(s). The lifted predicate requires all lists to have the same length in case \(n > 1\).

We let \({{x}} \in \mathbb {N}\) denote the length of \({{x}} \in \mathsf{list}\;A\).

We let \({{x}}_{[i,\,j)} \in \mathsf{list}\;A\) denote the sublist \(x_i \ldots x_{j1}\) of \({{x}} \in \mathsf{list}\;A\).

We let \(x ^ {n} \in \mathsf{list}\;A\) denote the list consisting of n times \(x \in A\).

We let \({({{{x}}} {y}^\infty )}_{[i,\,j)} \in \mathsf{list}\;A\) denote the sublist \(x_i \ldots x_{j1}\) of \({{x}} \in \mathsf{list}\;A\) which is padded with \(y \in A\) in case \({{x}}\) is too short.

Given lists \({{x}} \in \mathsf{list}\;A\) and \({{y}} \in \mathsf{list}\;B\) with \({{x}} = {{y}}\), we let Open image in new window denote the pointwise pairing of \({{x}}\) and \({{y}}\).
3 Challenges

In a highlevel way using arrays, structs and unions.

In a lowlevel way using unstructured and untyped byte representations.
3.1 ByteLevel Operations and Object Representations
On 32bit computing architectures such as x86 (with _Alignof(short*) \(=4\)), the object representation of s1 might be:
The above object representation contains a hole due to alignment of objects. The bytes belonging to such holes are called padding bytes.
Alignment is the way objects are arranged in memory. In modern computing architectures, accesses to addresses that are a multiple of a word sized chunk (often a multiple of 4 bytes on a 32bit computing architecture) are significantly faster due to the way the processor interacts with the memory. For that reason, the C11 standard has put restrictions on the addresses at which objects may be allocated [27, 6.2.8]. For each type \(\tau \), there is an implementationdefined integer constant _Alignof(\(\tau \)), and objects of type \(\tau \) are required to be allocated at addresses that are a multiple of that constant. In case _Alignof(short*) \(=4\), there are thus two bytes of padding in between the fields of struct S.
In the above code, size_t is an unsigned integer type, which is able to hold the results of the sizeof operator [27, 7.19p2].
Manipulation of object representations of structs also involves access to padding bytes, which are not part of the highlevel representation. In particular, in the example the padding bytes are also being copied. The problematic part is that padding bytes have indeterminate values, whereas in general, reading an indeterminate value has undefined behavior (for example, reading from an uninitialized int variable is undefined). The C11 standard provides an exception for unsigned char [27, 6.2.6.1p5], and the above example thus has defined behavior.
Our memory model uses a symbolic representation of bits (Definition 6.19) to distinguish determinate and indeterminate memory. This way, we can precisely keep track of the situations in which access to indeterminate memory is permitted.
3.2 Padding of Structs and Unions
The following excerpt from the C11 standard points out another challenge with respect to padding bytes [27, 6.2.6.1p6]:
When a value is stored in an object of structure or union type, including in a member object, the bytes of the object representation that correspond to any padding bytes take unspecified values.
On architectures with sizeof(struct S) = 4, objects of type struct S have one byte of padding. The object representation may be as follows:
Now, the assignments to fields of s by the function f affect also the padding bytes of s, including the one ((unsigned char*)&s)[3] that we have assigned to. As a consequence, the returned value is unspecified.
From a highlevel perspective this behavior makes sense. Padding bytes are not part of the abstract value of a struct, so their actual value should not matter. However, from a lowlevel perspective it is peculiar. An assignment to a specific field of a struct affects the object representation of parts not assigned to.
None of the currently existing C formalizations describes this behavior correctly. In our tree based memory model we enforce that padding bytes always have an indeterminate value, and in turn we have the desired behavior implicitly. Note that if the function call f(&s) would have been removed, the behavior of the example program remains unchanged in \(\mathrm{CH}_2\mathrm{O}\).
3.3 TypePunning
This code will reinterpret the bit representation of the int value 3 of u.x as a value of type short. The reinterpreted value that is printed is implementationdefined (on architectures where shorts do not have trap values).
Since C11 is ambiguous about the exact conditions under which typepunning is allowed,^{1} we follow the interpretation by the GCC documentation [20]:
Typepunning is allowed, provided the memory is accessed through the union type.
According to this interpretation the above program indeed has implementation defined behavior because the variant y is accessed via the expression u.y that involves the variable u of the corresponding union type.
We formalize the interpretation of C11 by GCC by decorating pointers and lvalues to subobjects with annotations (Definition 6.4). When a pointer to a variant of a union is stored in memory, or used as the argument of a function, the annotations are changed to ensure that typepunning no longer has defined behavior via that pointer. In Sect. 7.1 we formally establish that this approach is correct by showing that a compiler can perform typebased alias analysis (Theorem 7.2 on p. 51).
3.4 Indeterminate Memory and Pointers
In this code malloc(sizeof(int)) yields a pointer to a newly allocated memory area that may hold an integer, or yields a NULL pointer in case no memory is available. The function free deallocates memory allocated by malloc. In the example we assert that both calls to malloc succeed.
After execution of the second call to malloc it may happen that the memory area of the first call to malloc is reused: we have used free to deallocate it after all. This would lead to the following situation in memory:
Both GCC (version 4.9.2) or Clang (version 3.5.0) use the fact that p and q are obtained via different calls to malloc as a license to assume that p and q do not alias. As a result, the value 10 of *q is inlined, and the program prints the value 10 instead of the naively expected value 14.
The situation becomes more subtle because when the object a pointer points to has been deallocated, not just the argument of free becomes indeterminate, but also all other copies of that pointer. This is therefore yet another example where highlevel representations interact subtly with their lowlevel counterparts.
In our memory model we represent pointer values symbolically (Definition 6.4), and keep track of memory areas that have been previously deallocated. The behavior of operations like == depends on the memory state, which allows us to accurately capture the described undefined behaviors.
3.5 EndofArray Pointers
The way the C11 standard deals with pointer equality is subtle. Consider the following excerpt [27, 6.5.9p6]:
Two pointers compare equal if and only if [...] or one is a pointer to one past the end of one array object and the other is a pointer to the start of a different array object that happens to immediately follow the first array object in the address space.
The pointer p initially refers to the first element of the array a. The value p points to, as well as p itself, is being increased as long as p is before the endofarray pointer a + 4. This code thus increases the values of the array a. The initial state of the memory is displayed below:
Based on the aforementioned excerpt of the C11 standard [27, 6.5.9p6], one would naively say that the value of &x + 1 ==&y is uniquely determined by the way x and y are allocated in the address space. However, the GCC implementers disagree.^{2} They claim that Defect Report #260 [26] allows them to take the derivation of a pointer value into account.
When compiled with GCC (version 4.9.2), we have observed that the string x and y are still adjacent is being printed, wheras x and y are adjacent is not being printed. This means that the value of &x + 1 ==&y is not consistent among different occurrences of the comparison.
Due to these discrepancies we assign undefined behavior to questionable uses of endofarray pointers while assigning the correct defined behavior to pointer comparisons involving endofarray pointers when looping through arrays (such as in the first example above). Our treatment is similar to our extension of CompCert [34].
3.6 Sequence Point Violations and Nondeterminism
Instead of having to follow a specific execution order, the execution order of expressions is unspecified in C. This is a common cause of portability problems because a compiler may use an arbitrary execution order for each expression, and each time that expression is executed. Hence, to ensure correctness of a C program with respect to an arbitrary compiler, one has to verify that each possible execution order is free of undefined behavior and gives the correct result.
By considering all possible execution orders, one would naively expect this program to print 4 7 or 3 7, depending on whether the assignment x = 3 or x = 4 is executed first. However, x is modified twice within the same expression, and thus both execution orders have undefined behavior. The program is thereby allowed to exhibit any behavior. Indeed, when compiled with gcc O2 (version 4.9.2), the compiled program prints 4 8, which does not correspond to any of the execution orders.
Our approach to nondeterminism and sequence points is inspired by Norrish [44] and Ellison and Roşu [19]. Each bit in memory carries a permission (Definition 5.5) that is set to a special locked permission when a store has been performed. The memory model prohibits any access (read or store) to objects with locked permissions. At the next sequence point, the permissions of locked objects are changed back into their original permission, making future accesses possible again.
It is important to note that we do not have nondeterminism in the memory model itself, and have set up the memory model in such a way that all nondeterminism is on the level of the smallstep operational semantics.
4 Types in C
This section describes the types used in the \(\mathrm{CH}_2\mathrm{O}\) memory model. We support integer, pointer, function pointer, array, struct, union and void types. More complicated types such as enum types and typedefs are defined by translation [33, 37].
This section furthermore describes an abstract interface, called an implementation environment, that describes properties such as size and endianness of integers, and the layout of structs and unions. The entire \(\mathrm{CH}_2\mathrm{O}\) memory model and semantics will be parameterized by an implementation environment.
4.1 Integer Representations
This section describes the part of implementation environments corresponding to integer types and the encoding of integer values as bits. Integer types consist of a rank (\(\mathsf{char}\), \(\mathsf{short}\), \(\mathsf{int}\) ...) and a signedness (\(\mathsf{signed}\) or \(\mathsf{unsigned}\)). The set of available ranks as well as many of their properties are implementationdefined. We therefore abstract over the ranks in the definition of integer types.
Definition 4.1
Definition 4.2
Definition 4.3
The rank \(\mathsf{char}\) is the rank of the smallest integer type, whose unsigned variant corresponds to bytes that constitute object representations (see Sect. 3.1). Its bit size is \(\mathsf{char\_bits}\) (called CHAR_BIT in the standard library header files [27, 5.2.4.2.1]), and its signedness \(\mathsf{char\_signedness}\) is implementationdefined [27, 6.2.5p15].
The rank \(\mathsf{ptr\_rank}\) is the rank of the integer types size_t and ptrdiff_t, which are defined in the standard library header files [27, 7.19p2]. The type ptrdiff_t is a signed integer type used to represent the result of subtracting two pointers, and the type size_t is an unsigned integer type used to represent sizes of types.
An integer coding environment can have an arbitrary number of integer ranks apart from the standard ones \(\mathsf{char}\), \(\mathsf{short}\), \(\mathsf{int}\), \(\mathsf{long}\), \(\mathsf{long}\;\mathsf{long}\), and \(\mathsf{ptr\_rank}\). This way, additional integer types like those describe in [27, 7.20] can easily be included.
The function \(\mathsf{rank\_size}\) gives the byte size of an integer of a given rank. Since we require \(\mathsf{rank\_size}\) to be monotone rather than strictly monotone, integer types with different ranks can have the same size [27, 6.3.1.1p1]. For example, on many implementations int and long have the same size, but are in fact different.
The C11 standard allows implementations to use either signmagnitude, 1’s complement or 2’s complement signed integers representations. It moreover allows integer representations to contain padding or parity bits [27, 6.2.6.2]. However, since all current machine architectures use 2’s complement representations, this is more of a historic artifact. Current machine architectures use 2’s complement representations because these do not suffer from positive and negative zeros and thus enjoy unique representations of the same integer. Hence, \(\mathrm{CH}_2\mathrm{O}\) restricts itself to implementations that use 2’s complement signed integers representations.
Integer representations in \(\mathrm{CH}_2\mathrm{O}\) can solely differ with respect to endianness (the order of the bits). The function \(\mathsf{endianize}\) takes a list of bits in little endian order and permutes them accordingly. We allow \(\mathsf{endianize}\) to yield an arbitrary permutation and thus we not just support big and littleendian, but also mixedendian variants.
Definition 4.4
Lemma 4.5
 1.
We have Open image in new window and Open image in new window provided that \(x : {\tau }_{\mathsf{i}}\).
 2.
We have Open image in new window and \(({\beta })_{{\tau }_{\mathsf{i}}} : {\tau }_{\mathsf{i}}\) provided that \({\beta } = \mathsf{rank\_size}\;{\tau }_{\mathsf{i}}\).
4.2 Definition of Types
We support integer, pointer, function pointer, array, struct, union and void types. The translation that we have described in [33, 37] translates more complicated types, such as typedefs and enums, into these simplified types. This translation also alleviates other simplifications of our simplified definition of types, such as the use of unnamed struct and union fields. Floating point types and type qualifiers like const and volatile are not supported.
All definitions in this section are implicitly parameterized by an integer coding environment with ranks K (Definition 4.2).
Definition 4.6
Tags \(t \in \mathsf{tag}\) (sometimes called struct/union names) and function names \(f \in \mathsf{funname}\) are represented as strings.
Definition 4.7
The three mutual inductive parts of types correspond to the different components of the memory model. Addresses and pointers have pointto types (Definitions 6.8 and 6.10), base values have base types (Definition 6.40), and memory trees and values have full types (Definitions 6.25 and 6.46).
The void type of C is used for two entirely unrelated purposes: void is used for functions without return type and void* is used for pointers to objects of unspecified type. In \(\mathrm{CH}_2\mathrm{O}\) this distinction is explicit in the syntax of types. The type \(\mathsf{void}\) is used for function without return value. Like the mathematical unit type it has one value called \(\mathsf{nothing}\) (Definition 6.39). The type \(\mathsf{any}{*}\) is used for pointers to objects of unspecified type.
Unlike more modern programming languages C does not provide first class functions. Instead, C provides function pointers which are just addresses of executable code in memory instead of closures. Function pointers can be used in a way similar to ordinary pointers: they can be used as arguments and return value of functions, they can be part of structs, unions and arrays, etc.
Definition 4.8
Struct and union names on the one hand, and function names on the other, have their own name space in accordance with the C11 standard [27, 6.2.3p1].
Notation 4.9
We often write an environment as a mixed sequence of struct and union declarations \(t : {\tau }\), and function declarations \(f : ({\tau },\,\tau )\). This is possible because environments are finite.
Since we represent the fields of structs and unions as lists, fields are nameless. For example, the C type struct S1 { int x; struct S1 *p; } is translated into the environment \(\mathtt {S1} : [\, {{\mathsf{signed}\;\mathsf{int}}}, {{\mathsf{struct}\;\mathtt {S1}}{*}} \,]\).
Although structs and unions are semantically very different (products versus sums, respectively), environments do not keep track of whether a tag has been used for a struct or a union type. Structs and union types with the same tag are thus allowed. The translator in [33, 37] forbids the same name being used to declare both a struct and union type.
Definition 4.10
 The judgment \(\Gamma \vdash _{*} {{\tau }_{\mathsf{p}}}\) describes pointto types \({\tau }_{\mathsf{p}}\) to which a pointer may point:
 The judgment \(\Gamma \vdash _{\mathsf{b}} {{\tau }_{\mathsf{b}}}\) describes valid base types \({\tau }_{\mathsf{b}}\):
Definition 4.11
Note that \(\Gamma \vdash {\tau }\) does not imply \(\vdash \Gamma \). Most results therefore have \(\vdash \Gamma \) as a premise. These premises are left implicit in this paper.
Wellformedness \(\vdash \Gamma \) of the environment \(\Gamma :=\mathtt {S3} : [\, {{\mathsf{struct}\;\mathtt {S3}}{*}} \,]\) can be derived using the judgments \(\emptyset \vdash _{*} {\mathsf{struct}\;\mathtt {S3}}\), \(\emptyset \vdash _{\mathsf{b}} {{\mathsf{struct}\;\mathtt {S3}}{*}}\), \(\emptyset \vdash {{{\mathsf{struct}\;\mathtt {S3}}{*}}}\), and thus \(\vdash \Gamma \). The environment \(\mathtt {S2} : [\, \mathsf{struct}\;\mathtt {S2} \,]\) is illformed because we do not have \(\emptyset \vdash {\mathsf{struct}\;\mathtt {S2}}\).
Affeldt et al. [1, 2] have formalized noncyclicity of types using a complex constraint on paths through types. Our definition of validity of environments (Definition 4.11) follows the structure of type environments, and is therefore wellsuited to implement the aforementioned recursor and induction principle.
There is a close correspondence between array and pointer types in C. Arrays are not first class types, and except for special cases such as initialization, manipulation of arrays is achieved via pointers. We consider arrays as first class types so as to avoid having to make exceptions for the case of arrays all the time.
The corresponding type of the function f is thus \({{({{{\mathsf{signed}\;\mathsf{int}}}}){*}} \rightarrow {\mathsf{void}}}\). Note that the type \({{({{{\mathsf{signed}\;\mathsf{int}}}})}[10] \rightarrow {\mathsf{void}}}\) is also valid, but entirely different, and never generated by the translator in [33, 37].
4.3 Implementation Environments
We finish this section by extending integer coding environments to describe implementationdefined properties related the layout of struct and union types. The author’s PhD thesis [33] also considers the implementationdefined behavior of integer operations (such as addition and division) and defines inhabitants of this interface corresponding to actual computing architectures.
Definition 4.12
Notation 4.13
We let \(\mathsf{sizeof}_{\Gamma }\;\tau \) specify the number of bytes out of which the object representation of an object of type \(\tau \) consists. Objects of type \(\tau \) should be allocated at addresses that are a multiple of \(\mathsf{alignof}_{\Gamma }\;\tau \). We will prove that our abstract notion of addresses satisfies this property (see Lemma 6.18). The functions \(\mathsf{sizeof}_{\Gamma }\), \(\mathsf{alignof}_{\Gamma }\) correspond to the sizeof and _Alignof operators [27, 6.5.3.4], and \(\mathsf{offsetof}_{\Gamma }\) corresponds to the offsetof macro [27, 7.19p3]. The list \(\mathsf{fieldsizes}_{\Gamma }\;{\tau }\) specifies the layout of a struct type with fields \({\tau }\) as follows:
5 Permissions and Separation Algebras

Permissions are used to formalize the sequence point restriction which assigns undefined behavior to programs in which an object in memory is modified more than once in between two sequence points.

Permissions are used to distinguish objects in memory that are writable from those that are readonly (const qualified in C terminology).

Writable objects are unique to each subexpression.

Readonly objects may be shared between subexpressions.
5.1 Separation Logic and Share Accounting
5.2 Separation Algebras

Finite partial functions Open image in new window , where \(\emptyset \) is the empty finite partial function, and Open image in new window the disjoint union on finite partial functions.

The Booleans \((\mathsf{bool},\mathsf{false},\vee )\).

Boyland’s fractional permissions \(([0,1]_\mathbb {Q},0,+)\) where 0 denotes no access, 1 denotes writable access, and \(0 < \_ < 1\) denotes readonly access [11, 12].

Dealing with partial operations such as Open image in new window is cumbersome, see Sect. 8.3.

Dealing with subset types (modeled as \(\Sigma \)types) is inconvenient.

Operations such as the difference operation Open image in new window cannot be defined constructively from the laws of a separation algebra.
Definition 5.1

An element \(\emptyset : A\)

A predicate \(\mathsf{valid}: A \rightarrow \mathsf{Prop}\)

Binary relations Open image in new window

Binary operations Open image in new window
 1.
If \(\mathsf{valid}\;x\), then \(\emptyset \, {\mathrel {\bot }}\, x\) and \(\emptyset \,{\mathrel {\cup }}\, x = x\)
 2.
If \(x\, {\mathrel {\bot }}\, y\), then \(y\, {\mathrel {\bot }}\, x\) and \(x \,{\mathrel {\cup }}\, y = y \,{\mathrel {\cup }}\, x\)
 3.
If \(x\, {\mathrel {\bot }}\, y\) and \(x \,{\mathrel {\cup }}\, y\, {\mathrel {\bot }}\, z\), then \(y\, {\mathrel {\bot }}\, z\), \(x\, {\mathrel {\bot }}\, y \,{\mathrel {\cup }}\, z\), and \(x \,{\mathrel {\cup }}\, (y \,{\mathrel {\cup }}\, z) = (x \,{\mathrel {\cup }}\, y) \,{\mathrel {\cup }}\, z\)
 4.
If \(z\, {\mathrel {\bot }}\, x\), \(z\, {\mathrel {\bot }}\, y\) and \(z \,{\mathrel {\cup }}\, x = z \,{\mathrel {\cup }}\, y\), then \(x = y\)
 5.
If \(x\, {\mathrel {\bot }}\, y\), then \(\mathsf{valid}\;x\) and \(\mathsf{valid}\;(x \,{\mathrel {\cup }}\, y)\)
 6.
If \(x\, {\mathrel {\bot }}\, y\) and \(x \,{\mathrel {\cup }}\, y = \emptyset \), then \(x = \emptyset \)
 7.
If \(x\, {\mathrel {\bot }}\, y\), then Open image in new window
 8.
Laws 1–4 describe the traditional laws of a separation algebra: identity, commutativity, associativity and cancellativity. Law 5 ensures that \(\mathsf{valid}\) is closed under the Open image in new window operation. Law 6 describes positivity. Laws 7 and 8 fully axiomatize the Open image in new window relation and Open image in new window operation. Using the positivity and cancellation law, we obtain that Open image in new window is a partial order in which Open image in new window is order preserving and respecting.
In case of permissions, the \(\emptyset \) element is used to split objects of compound types (arrays and structs) into multiple parts. We thus use separation algebras instead of permission algebras [47], which are a variant of separation algebras without an \(\emptyset \) element.
Definition 5.2
In the case of fractional permissions \([0,1]_\mathbb {Q}\) the problem of partiality and subset types already clearly appears. The Open image in new window operation (here \(+\)) can ‘overflow’. We remedy this problem by having all operations operate on preterms (here \(\mathbb {Q}\)) and the predicate \(\mathsf{valid}\) describes validity of preterms (here \(0 \le \_ \le 1\)).
Definition 5.3

We include a predicate \(\mathsf{valid}\) to prevent having to deal with subset types.

They have weaker premises for associativity (law 3), namely \(x\, {\mathrel {\bot }}\, y\), \(y\, {\mathrel {\bot }}\, z\) and \(x\, {\mathrel {\bot }}\, z\) instead of \(x\, {\mathrel {\bot }}\, y\) and \(x \,{\mathrel {\cup }}\, y\, {\mathrel {\bot }}\, z\). Ours are more natural, e.g. for fractional permissions one has \(0.5\, {\mathrel {\bot }}\, 0.5\) but not \(0.5 + 0.5\, {\mathrel {\bot }}\, 0.5\), and it thus makes no sense to require \(0.5 \,{\mathrel {\cup }}\, (0.5 \,{\mathrel {\cup }}\, 0.5) = (0.5 \,{\mathrel {\cup }}\, 0.5) \,{\mathrel {\cup }}\, 0.5\) to hold.

Since Coq (without axioms) does not have a choice operator, the Open image in new window operation cannot be defined in terms of Open image in new window . Isabelle has a choice operator.
Bengtson et al. [6] have formalized separation algebras in Coq to reason about objectoriented programs. They have treated Open image in new window as a partial function, and have not defined any complex permission systems.
5.3 Permissions
In this section we define the \(\mathrm{CH}_2\mathrm{O}\) permission system and show that it forms a separation algebra. We furthermore define permission kinds, which are used to classify the abilities of the permissions.
Definition 5.4
The lattice of permission kinds \((\mathsf{pkind},\subseteq )\) is defined as:

\(\mathsf{Writable}\). Writable permissions allow reading and writing.

\(\mathsf{Readable}\). Readonly permissions allow solely reading.

\(\mathsf{Existing}\). Existence permissions [11] are used for objects that are known to exist but whose value cannot be used. Existence permissions are used to model that C only allows pointer arithmetic on pointers that refer to objects that have not been previously deallocated (see Sect. 3.4 for discussion).

\(\mathsf{Locked}\). Locked permissions are used to formalize the sequence point restriction. When an object is modified during the execution of an expression, it is temporarily given a locked permission to forbid any read/write accesses until the next sequence point. For example, in (x = 3) + *p; the assignment x = 3 locks the permissions of the object x. Since future read/write accesses to x are forbidden, accessing *p results in undefined in case p points to x. At the sequence point “;”, the original permission of x is restored.
Locked permissions are different from existence permissions because the operational semantics can change writable permissions into locked permissions and vice versa, but cannot do that with existence permissions.

\(\bot \). Empty permissions allow no operations.

We need to subdivide objects with writable or readonly permission into multiple parts with readonly permission. For example, in the expression x + x, both subexpressions require x to have at least readonly permission.

We need to subdivide objects with writable permission into a part with existence permission and a part with writable permission. For example, in the expression *(p + 1) = (*p = 1), the subexpression *p = 1 requires *p to have writable permission, and the subexpression *(p + 1) requires *p to have at least existence permission in order to perform pointer arithmetic on p.
Definition 5.5
\(\mathrm{CH}_2\mathrm{O}\) permissions \(\mathsf{perm}\) are defined as:
where Open image in new window and \(\mathcal {C}(A) :=\mathbb {Q}\times A\).
The author’s PhD thesis [33] gives the exact definition of the separation algebra structure of permissions by defining it one by one for the counting separation algebra \(\mathcal {C}\), the lockable separation algebra \(\mathcal {L}\), and the separation algebra on sums \(+\). We omit the formal definitions of these separation algebras in this paper.

Unlocked permissions Open image in new window where \(x \in \mathbb {Q}\) counts the number existence permissions, and \(y \in \mathbb {Q}\) is a fractional permission accounting for the read/write share. Permissions Open image in new window with \(x < 0\) are existence permissions (see also Definitions 5.6 and 5.9). Note that the counter x is not a fractional permission and is thus not restricted to the interval \([0,1]_\mathbb {Q}\).

Locked permissions Open image in new window where \(x \in \mathbb {Q}\) counts the number existence permissions, and \(y \in \mathbb {Q}\) is a fractional permission accounting for the read/write share.

Const permissions \(\gamma \in \mathbb {Q}\), which are used for const qualified objects. Modifying an object with const permissions results in undefined behavior. Const permissions do not have a locked variant or an existence counter as they do not allow writing.
Definition 5.6
Definition 5.7
The operation Open image in new window on permissions is defined as pointwise addition of the counting permission and the fractional permission, and the operation Open image in new window is defined as pointwise subtraction. The exact definitions can be found in [33]. Apart from the common separation algebra connectives, we define an operation \(\tfrac{1}{2}\) to subdivide a writable or readonly permission into two readonly permissions.
Definition 5.8
Given a writable or readonly permission \(\gamma \), the subdivided readonly permission \(\tfrac{1}{2}\gamma \) enjoys \(\tfrac{1}{2}\gamma \, {\mathrel {\bot }}\, \tfrac{1}{2}\gamma \) and \(\tfrac{1}{2}\gamma \,{\mathrel {\cup }}\, \tfrac{1}{2}\gamma = \gamma \). The split operation will be described abstractly by extended separation algebras in Sect. 5.4.
Definition 5.9
The existence permission \(\mathsf{token}\) is defined as Open image in new window .
Existence permissions are used to subdivide objects with writable permission into a part with existence permission and a part with writable permission. For example, in *(p + 1) = (*p = 1), the subexpression *p = 1 requires writable permission of *p, and *(p + 1) requires an existence permission of *p to perform pointer arithmetic. Subdivision is achieved using the Open image in new window operation, which can be used to split a writable permission \(\gamma \) into an existence permission \(\mathsf{token}\) and writable permission Open image in new window . Law 8 of separation algebras guarantees that combining the subdivided permissions gives back the original permission, i.e. Open image in new window . Note that because \(\mathsf{token}\)s can be combined using Open image in new window and subdivided using \(\tfrac{1}{2}\), the counter x in the permissions Open image in new window and Open image in new window is an arbitrary rational number.
As ensured by Definition 6.58, only objects with the full Open image in new window permission can be deallocated, whereas objects with Open image in new window permission cannot. This is to model that expressions such as (p == p) + (free(p),0) have undefined behavior.
The following lemma shows that the operations on permissions interact accordingly and respect the permission kinds.
Lemma 5.10
5.4 Extended Separation Algebras
We extend the notion of a separation algebra with a split operation \(\tfrac{1}{2}\), and predicates \(\mathsf{unmapped}\) and \(\mathsf{exclusive}\) that associate – in an abstract way – an intended semantics to elements of a separation algebra. Recall that the split operation \(\tfrac{1}{2}\) plays an important role in the \(\mathrm{CH}_2\mathrm{O}\) separation logic [31, 33] to subdivide objects with writable or readonly permission into multiple parts with readonly permission.
Definition 5.11

Predicates \(\mathsf{splittable}, \mathsf{unmapped}, \mathsf{exclusive}: A \rightarrow \mathsf{Prop}\)

A unary operation \(\tfrac{1}{2}: A \rightarrow A\)
 9.
If \(x\, {\mathrel {\bot }}\, x\), then \(\mathsf{splittable}\;(x \,{\mathrel {\cup }}\, x)\)
 10.
If \(\mathsf{splittable}\;x\), then \(\tfrac{1}{2}x\, {\mathrel {\bot }}\, \tfrac{1}{2}x\) and \(\tfrac{1}{2}x \,{\mathrel {\cup }}\, \tfrac{1}{2}x = x\)
 11.
If \(\mathsf{splittable}\;y\) and Open image in new window , then \(\mathsf{splittable}\;x\)
 12.
If \(x\, {\mathrel {\bot }}\, y\) and \(\mathsf{splittable}\;(x \,{\mathrel {\cup }}\, y)\), then \(\tfrac{1}{2}(x \,{\mathrel {\cup }}\, y) = \tfrac{1}{2}x \,{\mathrel {\cup }}\, \tfrac{1}{2}y\)
 13.
\(\mathsf{unmapped}\;\emptyset \), and if \(\mathsf{unmapped}\;x\), then \(\mathsf{valid}\;x\)
 14.
If \(\mathsf{unmapped}\;y\) and Open image in new window , then \(\mathsf{unmapped}\;x\)
 15.
If \(x\, {\mathrel {\bot }}\, y\), \(\mathsf{unmapped}\;x\) and \(\mathsf{unmapped}\;y\), then \(\mathsf{unmapped}\;(x \,{\mathrel {\cup }}\, y)\)
 16.
\(\mathsf{exclusive}\;x\) iff \(\mathsf{valid}\;x\) and for all y with \(x\, {\mathrel {\bot }}\, y\) we have \(\mathsf{unmapped}\;y\)
 17.
Not both \(\mathsf{exclusive}\;x\) and \(\mathsf{unmapped}\;x\)
 18.
There exists an x with \(\mathsf{valid}\;x\) and not \(\mathsf{unmapped}\;x\)
Note that \(\tfrac{1}{2}\) is described by a total function whose result, \(\tfrac{1}{2}x\), is only meaningful if \(\mathsf{splittable}\;x\) holds. This is to account for locked permissions, which cannot be split. Law 11 ensures that splittable permissions are infinitely splittable, and law 12 ensures that \(\tfrac{1}{2}\) distributes over Open image in new window .
The predicates \(\mathsf{unmapped}\) and \(\mathsf{exclusive}\) associate an intended semantics to the elements of a separation algebra in an abstract way. The predicate \(\mathsf{unmapped}\) describes whether the permission allows its content to be used, as will become clear in the definition of the tagged separation algebra (Definition 5.13). The predicate \(\mathsf{exclusive}\) is the dual of \(\mathsf{unmapped}\). Let us consider the separation algebra of fractional permissions to describe the intended meaning of these predicates.
Definition 5.12
Remember that permissions will be used to annotate each individual bit in memory. Unmapped permissions are on the bottom: they do not allow their bit to be used in any way. Exclusive permissions are on the top: they are the sole owner of a bit and can do anything to that bit without affecting disjoint bits.
In order to formalize the intuitive meaning of the \(\mathsf{unmapped}\) predicate and to abstractly describe bits annotated with permissions, we introduce the tagged separation algebra \(\mathcal {T}_{T}^{t}(A)\). In the memory model it is instantiated as Open image in new window (Definition 6.21). The elements \((\gamma ,\,b)\) consist of a permission \(\gamma \in \mathsf{perm}\) and bit \(b \in \mathsf{bit}\). We use the symbolic bit Open image in new window that represents indeterminate storage to ensure that bits with \(\mathsf{unmapped}\) permissions have no usable value.
Definition 5.13
6 The Memory Model
Notation 6.1
We let \(m \langle a \rangle _{\Gamma } :=\mathsf{lookup}_{\Gamma }\,a\;m\) and \(m \langle a := v \rangle _{\Gamma } :=\mathsf{insert}_{\Gamma }\,a\;v\;m\).
Many of these operations depend on the typing environment \(\Gamma \) which assigns fields to structs and unions (Definition 4.8). This dependency is required because these operations need to be aware of the layout of structs and unions.
The operation \(m \langle a \rangle _{\Gamma }\) yields the value stored at address a in memory m. It fails with \(\bot \) if the permissions are insufficient, effective types are violated, or a is an endofarray address. Reading from (the abstract) memory is not a pure operation. Although it does not affect the memory contents, it may affect the effective types [27, 6.5p67]. This happens for example in case typepunning is performed (see Sect. 3.3). This impurity is factored out by the operation \(\mathsf{force}_{\Gamma }\,a\;m\).
The operation \(m \langle a := v \rangle _{\Gamma }\) stores the value v at address a in memory m. A store is only permitted in case permissions are sufficient, effective types are not violated, and a is not an endofarray address. The proposition \(\mathsf{writable}_{\Gamma }\,a\;m\) describes the sideconditions necessary to perform a store.
After a successful store, the operation \(\mathsf{lock}_{\Gamma }\,a\;m\) is used to lock the object at address a in memory m. The lock operation temporarily reduces the permissions to \(\mathsf{Locked}\) so as to prohibit future accesses to a. Locking yields a formal treatment of the sequence point restriction (which states that modifying an object more than once between two sequence points results in undefined behavior, see Sect. 3.6).
The operational semantics accumulates a set \(\Omega \in \mathsf{lockset}\) of addresses that have been written to (Definition 6.54) and uses the operation \(\mathsf{unlock}\;\Omega \;m\) at the subsequent sequence point (which may be at the semicolon that terminates a full expression). The operation \(\mathsf{unlock}\;\Omega \;m\) restores the permissions of the addresses in \(\Omega \) and thereby makes future accesses to the addresses in \(\Omega \) possible again. The author’s PhD thesis [33] describes in detail how sequence points and locks are treated in the operational semantics.
The operation \(\mathsf{alloc}_{\Gamma }\,o\;v\;\mu \;m\) allocates a new object with value v in memory m. The object has object identifier \(o \notin \mathsf{dom}\;m\) which is nondeterministically chosen by the operation semantics. The Boolean \(\mu \) expresses whether the new object is allocated by malloc.
Accompanying \(\mathsf{alloc}_{\Gamma }\), the operation \(\mathsf{free}\;o\;m\) deallocates a previously allocated object with object identifier o in memory m. In order to deallocate dynamically obtained memory via free, the sidecondition \(\mathsf{freeable}\;a\;m\) describes that the permissions are sufficient for deallocation, and that a points to a malloced object.
6.1 Representation of Pointers
Adapted from CompCert [40, 41], we represent memory states as finite partial functions from object identifiers to objects. Each local, global and static variable, as well as each invocation of malloc, is associated with a unique object identifier of a separate object in memory. This approach separates unrelated objects by construction, and is therefore wellsuited for reasoning about memory transformations.
We improve on CompCert by modeling objects as structured trees instead of arrays of bytes to keep track of padding bytes and the variants of unions. This is needed to faithfully describe C11’s notion of effective types (see page 4 of Sect. 1 for an informal description). This approach allows us to describe various undefined behaviors of C11 that have not been considered by others (see Sects. 3.1 and 3.3).
In the CompCert memory model, pointers are represented as pairs (o, i) where o is an object identifier and i is a byte offset into the object with object identifier o. Since we represent objects as trees instead of as arrays of bytes, we represent pointers as paths through these trees rather than as byte offsets.
Definition 6.2
Object identifiers \(o \in \mathsf{index}\) are elements of a fixed countable set. In the Coq development we use binary natural numbers, but since we do not rely on any properties apart from countability, we keep the representation opaque.
We first introduce a typing environment to relate the shape of paths representing pointers to the types of objects in memory.
Definition 6.3
 1.
An object identifier o has type \(\tau \), notation \(\Delta \vdash o : \tau \), if \({\Delta }\,o= (\tau ,\,\beta )\) for a \(\beta \).
 2.
An object identifier o is alive, notation \(\Delta \vdash o\ \mathsf{alive}\), if \({\Delta }\,o= (\tau ,\,\mathsf{false})\) for a \(\tau \).
Here, \(o_1\) is the object identifier of the variable x, \(o_2\) is the object identifier of the variable p and \(o_3\) is the object identifier of the storage obtained via malloc.
Memory typing environments also keep track of objects that have been deallocated. Although one cannot directly create a pointer to a deallocated object, existing pointers to such objects remain in memory after deallocation (see the pointer p in the above example). These pointers, also called dangling pointers, cannot actually be used.
Definition 6.4

The reference Open image in new window is used to select the ith element of a \(\tau \)array of length n.

The reference Open image in new window is used to select the ith field of a struct t.

The reference Open image in new window is used to select the ith variant of a union t.

An object identifier o with type \(\tau \).

A reference \({{r}}\) to a subobject of type \(\sigma \) in the entire object of type \(\tau \).

An offset i to a particular byte in the subobject of type \(\sigma \) (note that one cannot address individual bits in C).

The type \({\sigma }_{\mathsf{p}}\) to which the address is cast. We use a pointsto type in order to account for casts to the anonymous void* pointer, which is represented as the pointsto type \(\mathsf{any}\). This information is needed to define, for example, pointer arithmetic, which is sensitive to the type of the address.
The annotations \(q \in \{ \circ , \bullet \}\) on references Open image in new window describe whether typepunning is allowed or not. The annotation \(\bullet \) means that typepunning is allowed, i.e. accessing another variant than the current one has defined behavior. The annotation \(\circ \) means that typepunning is forbidden. A pointer whose annotations are all of the shape \(\circ \), and thereby does not allow typepunning at all, is called frozen.
Definition 6.5
Here, the pointers s.a + 3 and s.b have different representations in the \(\mathrm{CH}_2\mathrm{O}\) memory model. The author’s PhD thesis [33] gives the formal definition of pointer subtraction.
We will now define typing judgments for references, addresses and pointers. The judgment for references \(\Gamma \vdash {{r}} : \tau \rightarrowtail \sigma \) states that \(\sigma \) is a subobject type of \(\tau \) which can be obtained via the reference \({{r}}\) (see also Definition 7.1). For example, int[2] is a subobject type of struct S { int x[2]; int y[3]; } via Open image in new window .
Definition 6.6
Definition 6.7
The relation \({\tau } >_{\!*} {{\sigma }_{\mathsf{p}}}\), type \(\tau \) is pointer castable to \({\sigma }_{\mathsf{p}}\), is inductively defined by \({\tau } >_{\!*} {{\tau }}\), \({\tau } >_{\!*} {{{{\mathsf{unsigned}\;\mathsf{char}}}}}\), and \({\tau } >_{\!*} {\mathsf{any}}\).
Definition 6.8
We use an intrinsic encoding of syntax, which means that terms contain redundant type annotations so we can read off types. Functions to read off types are named \(\mathsf{typeof}\) and will not be defined explicitly. Type annotations make it more convenient to define operations that depend on types (such as \(\mathsf{offset}\) and \(\mathsf{size}\) in Definition 6.8). As usual, typing judgments ensure that type annotations are consistent.
The premises \(i \le \mathsf{sizeof}_{\Gamma }\;\sigma \cdot \mathsf{size}\;{{r}}\) and Open image in new window of the typing rule ensure that the byte offset i is aligned and within range. The inequality \(i \le \mathsf{sizeof}_{\Gamma }\;\sigma \cdot \mathsf{size}\;{{r}}\) is nonstrict so as to allow endofarray pointers.
Definition 6.9
An address \(a = (o : \tau , r, i)_{{\sigma } >_{\!*} {{\sigma }_{\mathsf{p}}}}\) is called strict, notation \(\Gamma \vdash a\ \mathsf{strict}\), in case it satisfies \(i < \mathsf{sizeof}_{\Gamma }\;\sigma \cdot \mathsf{size}\;{{r}}\).
Definition 6.10
Addresses Open image in new window that point to an element of \({\tau }[n]\) always have their reference point to the first element of the array, i.e. \(j = 0\). For some operations we use the normalized reference which refers to the actual array element.
Definition 6.11
Let us display the above definition graphically. Given an address \((o : \tau , {{r}}, i)_{{\sigma } >_{\!*} {{\sigma }_{\mathsf{p}}}}\), the normalized reference and normalized byte offset are as follows:
For endofarray addresses the normalized reference is illtyped because references cannot be endofarray. For strict addresses the normalized reference is welltyped.
Definition 6.12
Definition 6.13
An address \((o : \tau , {{r}}, i)_{{\sigma } >_{\!*} {{\sigma }_{\mathsf{p}}}}\) is a byte address if \({\sigma }\ne {\sigma }_{\mathsf{p}}\).
The pointers &u1 and &u2 are disjoint because they point to separate memory objects. Writing to one does not affect the value of the other and vice versa. Likewise, &u1.s.x and &u1.s.y are disjoint because they point to different fields of the same struct, and as such do not affect each other. The pointers &u1.s.x and &u1.z are not disjoint because they point to overlapping objects and thus do affect each other.
Definition 6.14
Note that we do not require a special case for \(\,{{r}}_1\,_\circ \ne \,{{r}}_2\,_\circ \). Such a case is implicit because disjointness is defined in terms of prefixes.
Definition 6.15
The first inference rule accounts for addresses whose object identifiers are different, the second rule accounts for addresses whose references are disjoint, and the third rule accounts for addresses that point to different bytes of the same subobject.
Definition 6.16
Disjointness implies nonoverlapping bitoffsets, but the reverse implication does not always hold because references to different variants of unions are not disjoint. For example, given the declaration union { struct { int x, y; } s; int z; } u, the pointers corresponding to &u.s.y and &u.z are not disjoint.
Lemma 6.17
 1.
\(\mathsf{bitoffset}_{\Gamma }\,a_1 + \mathsf{bitsizeof}_{\Gamma }\,\sigma _1 \le \mathsf{bitoffset}_{\Gamma }\,a_2\), or
 2.
\(\mathsf{bitoffset}_{\Gamma }\,a_2 + \mathsf{bitsizeof}_{\Gamma }\,\sigma _2 \le \mathsf{bitoffset}_{\Gamma }\,a_1\).
Lemma 6.18
(Welltyped addresses are properly aligned) If \(\Gamma ,\Delta \vdash a : \sigma \), then Open image in new window .
6.2 Representation of Bits
As shown in Sect. 3.1, each object in C can be interpreted as an unsigned char array called the object representation. On actual computing architectures, the object representation consists of a sequence of concrete bits (zeros and ones). However, so as to accurately describe all undefined behaviors, we need a special treatment for the object representations of pointers and indeterminate memory in the formal memory model. To that end, \(\mathrm{CH}_2\mathrm{O}\) represents the bits belonging to the object representations of pointers and indeterminate memory symbolically.
Definition 6.19
Definition 6.20
A bit is either a concrete bit \({0}\) or \({1}\), the ith fragment bit \({(\mathsf{ptr}\;{p})_{{i}}}\) of a pointer p, or the indeterminate bit Open image in new window . Integers are represented using concrete sequences of bits, and pointers as sequences of fragment bits. Assuming \(\mathsf{bitsizeof}_{}\,({{{\mathsf{signed}\;\mathsf{int}}}{*}}) = 32\), a pointer p to a \({{\mathsf{signed}\;\mathsf{int}}}\) will be represented as the bit sequence \({(\mathsf{ptr}\;{p})_{{0}}} \ldots {(\mathsf{ptr}\;{p})_{{31}}}\), and assuming \(\mathsf{bitsizeof}_{}\,({{\mathsf{signed}\;\mathsf{int}}}) = 32\) on a littleendian architecture, the integer \(33 : {{\mathsf{signed}\;\mathsf{int}}}\) will be represented as the bit sequence \(\mathtt {1000010000000000}\).

Symbolic bit representations for pointers avoid the need to clutter the memory model with subtle, implementationdefined, and runtime dependent operations to decode and encode pointers as concrete bit sequences.

We can precisely keep track of memory areas that are uninitialized. Since these memory areas consist of arbitrary concrete bit sequences on actual machines, most operations on them have undefined behavior.

While reasoning about program transformations one has to relate the memory states during the execution of the source program to those during the execution of the target program. Program transformations can, among other things, make more memory defined (that is, transform some indeterminate Open image in new window bits into determinate bits) and relabel the memory. Symbolic bit representations make it easy to deal with such transformations (see Sect. 7.2).

It vastly decreases the amount of nondeterminism, making it possible to evaluate the memory model as part of an executable semantics [33, 37].

The use of concrete bit representations for integers still gives a semantics to many lowlevel operations on integer representations.
Objects in our memory model are annotated with permissions. We use permission annotations on the level of individual bits, rather than on the level of bytes or entire objects, to obtain the most precise way of permission handling.
Definition 6.21
In the above definition, \(\mathcal {T}\) is the tagged separation algebra that has been defined in Definition 5.13. We have spelled out its definition for brevity’s sake.
Definition 6.22
6.3 Representation of the Memory
Memory trees are abstract trees whose structure corresponds to the shape of data types in C. They are used to describe individual objects (base values, arrays, structs, and unions) in memory. The memory is a forest of memory trees.
Definition 6.23
The memory tree representing the object s with object identifier \(o_{\mathtt s}\) may be as follows (permissions are omitted for brevity’s sake, and integer encoding and padding are subject to implementationdefined behavior):

The memory tree \(\mathsf{union}_{t}\,(i,w,{\mathbf {{b}}})\) represents a union whose variant is i. Unions of variant i can only be accessed through a pointer to variant i. This is essential for effective types. The list \({\mathbf {{b}}}\) represents the padding after the element w.

The memory tree \(\overline{\mathsf{union}}_{t}\,{{\mathbf {{b}}}}\) represents a union whose variant is yet unspecified. Whenever the union is accessed through a pointer to variant i, the list \({\mathbf {{b}}}\) will be interpreted as a memory tree of the type belonging to the ith variant.
Although unions are tagged in the formal memory, actual compilers implement untagged unions. Information about variants should thus be internal to the formal memory model. In Sect. 7.2 we prove that this is indeed the case.
The additional structure of memory trees, namely type annotations, variants of unions, and structured information about padding, can be erased by flattening. Flattening just appends the bytes on the leaves of the tree.
Definition 6.24
Definition 6.25
Although padding bits should be kept indeterminate (see Sect. 3.1), padding bits are explicitly stored in memory trees for uniformity’s sake. The typing judgment ensures that the value of each padding bit is Open image in new window and that the padding thus only have a permission. Storing a value in padding is a noop (see Definition 6.35).
The sidecondition \(\lnot \mathsf{unmapped}\;(\overline{w}\,{{\mathbf {{b}}}})\) in the typing rule for a union \(\mathsf{union}_{t}\,(i,w,{\mathbf {{b}}})\) of a specified variant ensures canonicity. Unions whose permissions are unmapped cannot be accessed and should therefore be in an unspecified variant. This condition is essential for the separation algebra structure, see Sect. 7.4.
Definition 6.26
Each object \((w,\,\mu )\) in memory is annotated with a Boolean \(\mu \) to describe whether it has been allocated using malloc (in case \(\mu = \mathsf{true}\)) or as a block scope local, static, or global variable (in case \(\mu = \mathsf{false}\)). The types of deallocated objects are kept to ensure that dangling pointers (which may remain to exist in memory, but cannot be used) have a unique type.
Definition 6.27
 1.For each o and \(\tau \) with \({m}\,o = \tau \) we have:
 (a)
\(\Delta \vdash o : \tau \), (b) \(\Delta \not \vdash o\ \mathsf{alive}\), and (c) \(\Gamma \vdash {\tau }\).
 (a)
 2.For each o, w and \(\mu \) with \({m}\,o = (w,\,\mu )\) we have:
 (a)
\(\Delta \vdash o : \tau \), (b) \(\Delta \vdash o\ \mathsf{alive}\), (c) \(\Gamma ,\Delta \vdash {w} : {\tau }\), and (d) not \(\overline{w}\) all Open image in new window .
 (a)
Definition 6.28
Notation 6.29
We let \(\Gamma \vdash m\) denote \(\Gamma ,\overline{m} \vdash m\).
Using an assertion of separation logic we can describe the memory induced by the above program as Open image in new window . The separation conjunction \(\mathrel {*}\) describes that the memory can be subdivided into two parts, a part for x and another part for p. When considering Open image in new window in isolation, which is common in separation logic, we have a pointer that refers outside the part itself. This isolated part is thus not typeable by \(\Gamma \vdash m\), but it is typeable in the context of a the memory environment corresponding to the whole memory. See also Lemma 7.26.
Here, we poke some bytes into the object representation of u, and interpret these as a memory tree of type short.
We have defined the flatten operation \(\overline{w}\) that takes a memory tree w and yields its bit representation already in Definition 6.24. We now define the operation which goes in opposite direction, called the unflatten operation.
Definition 6.30

We do not have \((\overline{w})^{\tau }_{\Gamma } = w\) for each w with \(\Gamma ,\Delta \vdash {w} : {\tau }\). Variants of unions are destroyed by flattening w.

We do not have \(\overline{({{\mathbf {{b}}}})^{\tau }_{\Gamma }} = {\mathbf {{b}}}\) for each \({\mathbf {{b}}}\) with \({\mathbf {{b}}} = \mathsf{bitsizeof}_{\Gamma }\,\tau \) either. Padding bits become indeterminate due to Open image in new window by unflattening.
Definition 6.31
The memory tree \(\mathsf{new}_{\Gamma }^{\gamma }\,\tau \) that consists of indeterminate bits with permission \(\gamma \) is used for objects with indeterminate value. We have defined \(\mathsf{new}_{\Gamma }^{\gamma }\,\tau \) in terms of the unflattening operation for simplicity’s sake. This definition enjoys desirable structural properties such as \(\mathsf{new}_{\Gamma }^{\gamma }\,({\tau }[n]) = (\mathsf{new}_{\Gamma }^{\gamma }\,\tau ) ^ {n}\).
We will now define the lookup operation \(m [ a ]_{\Gamma }\) that yields the subtree at address a in the memory m. The lookup function is partial, it will fail in case a is endofarray or violates effective types. We first define the counterpart of lookup on memory trees and then lift it to memories.
Definition 6.32

The annotation \(q = \bullet \) allows a union to be accessed via a reference whose variant is unequal to the current one. This is called typepunning.

The annotation \(q = \circ \) allows a union to be accessed only via a reference whose variant is equal to the current one. This means, it rules out typepunning.
Definition 6.33
In this code, we obtain the first byte ((unsigned char*)&s)[0] of the struct s. This is formalized by flattening the entire memory tree of the struct s, and selecting the appropriate byte.
In this code, the variant of the union u is initially unspecified. The read *q in g forces its variant to y, making the assignment *p to variant x undefined. Note that it is important that we also assign undefined behavior to this example, a compiler may assume p and q to not alias regardless of how g is called.
We factor these sideeffects out using a function \(\mathsf{force}_{\Gamma }: \mathsf{addr}\rightarrow \mathsf{mem}\rightarrow \mathsf{mem}\) that updates the effective types (that is the variants of unions) after a successful lookup. The \(\mathsf{force}_{\Gamma }\) function, as defined in Definition 6.5, can be described in terms of the alter operation \(m[a / f]_{\Gamma }\) that applies the function \(f : \mathsf{mtree}\rightarrow \mathsf{mtree}\) to the object at address a in the memory m and update variants of unions accordingly to a. To define \(\mathsf{force}_{\Gamma }\) we let f be the identify.
Definition 6.34
Definition 6.35
The lookup and alter operation enjoy various properties; they preserve typing and satisfy laws about their interaction. We list some for illustration.
Lemma 6.36

\(\Gamma ,\Delta \vdash a_1 : \tau _1\), \(m [ a_1 ]_{\Gamma } = w_1\), and \(\Gamma ,\Delta \vdash {f_1\,w_1} : {\tau _1}\), and

\(\Gamma ,\Delta \vdash a_2 : \tau _2\), \(m [ a_2 ]_{\Gamma } = w_2\), and \(\Gamma ,\Delta \vdash {f_2\,w_2} : {\tau _2}\),
Lemma 6.37
A variant of Lemma 6.37 for byte addresses is more subtle because a byte address can be used to modify padding. Since modifications of padding are masked, a successive lookup may yield a memory tree with more indeterminate bits. In Sect. 7.2 we present an alternative lemma that covers this situation.
We conclude this section with a useful helper function that zips a memory tree and a list. It is used in for example Definitions 6.58 and 7.22.
Definition 6.38
6.4 Representation of Values
Memory trees (Definition 6.23) are still rather lowlevel and expose permissions and implementation specific properties such as bit representations. In this section we define abstract values, which are like memory trees but have mathematical integers and pointers instead of bit representations as leaves. Abstract values are used in the external interface of the memory model.
Definition 6.39
While performing bytewise operations (for example, bytewise copying a struct containing pointer values), abstraction is broken, and pointer fragment bits have to reside outside of memory. The value \(\mathsf{byte}\,{{{b}}}\) is used for this purpose.
Definition 6.40
The sideconditions of the typing rule for \(\mathsf{byte}\,{{{b}}}\) ensure canonicity of representations of base values. It ensures that the construct \(\mathsf{byte}\,{{{b}}}\) is only used if \({{b}}\) cannot be represented as an integer \(\mathsf{int}_{\mathsf{unsigned}\;\mathsf{char}}\,{x}\) or \(\mathsf{indet}\,({\mathsf{unsigned}\;\mathsf{char}})\).
In Definition 6.44 we define abstract values by extending base values with constructs for arrays, structs and unions. In order to define the operations to look up and store values in memory, we define conversion operations between abstract values and memory trees. Recall that the leaves of memory trees, which represent base values, are just sequences of bits. We therefore first define operations that convert base values to and from bits. These operations are called flatten and unflatten.
Definition 6.41
Definition 6.42
The encoding of pointers is an important aspect of the flatten operation related to our treatment of effective types. Pointers are encoded as sequences of frozen pointer fragment bits \((\mathsf{ptr}\;{\,p\,_\circ })_{{i}}\) (see Definition 6.5 for the definition of frozen pointers). Recall that the flatten operation is used to store base values in memory, whereas the unflatten operation is used to retrieve them. This means that whenever a pointer p is stored and read back, the frozen variant \(\,p\,_\circ \) is obtained.
Lemma 6.43
For each \(\Gamma ,\Delta \vdash _{\mathsf b} {v}_{\mathsf{b}} : {\tau }_{\mathsf{b}}\) we have Open image in new window .
Here, typepunning is allowed because it is performed directly via u.y, which has not been stored in memory, and thus has not been frozen.
Definition 6.44
Definition 6.45
Definition 6.46
Definition 6.47
Definition 6.48
The operation \(\mathsf{ofval}_{\Gamma }: \mathsf{list}\;\mathsf{perm}\rightarrow \mathsf{val}\rightarrow \mathsf{mtree}\), which converts a value v of type \(\tau \) into a memory tree \(\mathsf{ofval}_{\Gamma }\,{\gamma }\,v\), is albeit technical fairly straightforward. In principle it is just a recursive definition that uses the flatten operation Open image in new window for base values \({{v}_{\mathsf{b}}}\) and the flatten operation Open image in new window for unions \(\overline{\mathsf{union}}_{t}\,{{{v}}}\) of an unspecified variant.
The technicality is that abstract values do not contain permissions, so we have to merge the given value with permissions. The sequence \({\gamma }\) with \({\gamma } = \mathsf{bitsizeof}_{\Gamma }\,\tau \) represents a flattened sequence of permissions. In the definition of the memory store \(m \langle a := v \rangle _{\Gamma }\) (see Definition 6.58), we convert v into the stored memory tree \(\mathsf{ofval}_{\Gamma }\,{\gamma }\,v\) where \(\gamma \) constitutes the old permissions of the object at address a.
Definition 6.49
Converting a memory tree into a value is as expected: permissions are removed and unions are interpreted as values corresponding to each variant.
Definition 6.50
The function \(\mathsf{toval}_{\Gamma }\) is an inverse of \(\mathsf{ofval}_{\Gamma }\) up to freezing of pointers. Freezing is intended, it makes indirect typepunning illegal.
Lemma 6.51
We finish this section by defining the indeterminate abstract value \(\mathsf{new}_{\Gamma }\,\tau \), which consists of indeterminate base values. The definition is similar to its counterpart on memory trees (Definition 6.31).
Definition 6.52
Lemma 6.53
6.5 Memory Operations
Now that we have all primitive definitions in place, we can compose these to implement the actual memory operations as described in the beginning of this section. The last part that is missing is a data structure to keep track of objects that have been locked. Intuitively, this data structure should represent a set of addresses, but up to overlapping addresses.
Definition 6.54
Elements of locksets are pairs \((o,\,i)\) where \(o \in \mathsf{index}\) describes the object identifier and \(i \in \mathbb {N}\) a bitoffset in the object described by o. We introduce a typing judgment to describe that the structure of locksets matches up with the memory layout.
Definition 6.55
Definition 6.56
Lemma 6.57
Definition 6.58
The lookup operation \(m \langle a \rangle _{\Gamma }\) uses the lookup operation \(m [ a ]_{\Gamma }\) that yields a memory tree w (Definition 6.33), and then converts w into the value \(\mathsf{toval}_{\Gamma }\,w\). The operation \(m [ a ]_{\Gamma }\) already yields \(\bot \) in case effective types are violated or a is an endofarray address. The additional condition of \(m \langle a \rangle _{\Gamma }\) ensures that the permissions allow for a read access. Performing a lookup affects the effective types of the object at address a. This is factored out by the operation \(\mathsf{force}_{\Gamma }\,a\;m\) which applies the identity function to the subobject at address a in the memory m. Importantly, this does not change the memory contents, but merely changes the variants of the involved unions.
The store operation \(m \langle a := v \rangle _{\Gamma }\) uses the alter operation \(m[a / \lambda w \,.\,\mathsf{ofval}_{\Gamma }\,({\overline{w}}_{\mathbf {1}})\,v]_{\Gamma }\) on memories (Definition 6.35) to apply \(\lambda w \,.\,\mathsf{ofval}_{\Gamma }\,({\overline{w}}_{\mathbf {1}})\,v\) to the subobject at address a. The stored value v is converted into a memory tree while retaining the permissions \({\overline{w}}_{\mathbf {1}}\) of the previously stored memory tree w at address a.
The definition of \(\mathsf{lock}_{\Gamma }\,a\;m\) is straightforward. In the Coq development we use a map operation on memory trees to apply the function \(\mathsf{lock}\) (Definition 5.5) to the permission of each bit of the memory tree at address a.
The operation \(\mathsf{unlock}\;\Omega \;m\) unlocks a whole lockset \(\Omega \), rather than an individual address, in memory m. For each memory tree w at object identifier o, it converts \(\Omega \) to a Boolean vector \({{y}} = ((o,\,0) \in \Omega ) \ldots ((o,\,\mathsf{bitsizeof}_{\Gamma }\,(\mathsf{typeof}\;w)  1) \in \Omega )\) and merges w with \({{y}}\) (using Definition 6.38) to apply \(\mathsf{unlock}\) (Definition 5.5) to the permissions of bits that should be unlocked in w. We show some lemmas to illustrate that the operations for locking and unlocking enjoy the intended behavior:
Lemma 6.59
Lemma 6.60
If \(\Omega \subseteq \mathsf{locks}\;m\), then \(\mathsf{locks}\;(\mathsf{unlock}\;\Omega \;m) = \mathsf{locks}\;m \setminus \Omega \).
Provided \(o \notin \mathsf{dom}\;m\), allocation \(\mathsf{alloc}_{\Gamma }\,o\;v\;\mu \;m\) extends the memory with a new object holding the value v and full permissions Open image in new window . Typically we use \(v = \mathsf{new}_{\Gamma }\,\tau \) for some \(\tau \), but global and static variables are allocated with a specific value v.
The operation \(\mathsf{free}\;o\;m\) deallocates the object o in m, and keeps track of the type of the deallocated object. In order to deallocate dynamically obtained memory via free, the sidecondition \(\mathsf{freeable}\;a\;m\) describes that the permissions are sufficient for deallocation, and that a points to the first element of a malloced array.
All operations preserve typing and satisfy the expected laws about their interaction. We list some for illustration.
Fact 6.61
If \(\mathsf{writable}_{\Gamma }\,a\;m\), then there exists a value v with \(a \langle m \rangle _{\Gamma } = v\).
Lemma 6.62

\(\Gamma ,\Delta \vdash a_1 : \tau _1\), \(\mathsf{writable}_{\Gamma }\,a_1\;m\), and \(\Gamma ,\Delta \vdash {v_1} : {\tau _1}\), and

\(\Gamma ,\Delta \vdash a_2 : \tau _2\), \(\mathsf{writable}_{\Gamma }\,a_2\;m\), and \(\Gamma ,\Delta \vdash {v_2} : {\tau _2}\),
Lemma 6.63
Storing a value v in memory and then retrieving it, does not necessarily yield the same value v. It intentionally yields the value \(\,v\,_\circ \) whose pointers have been frozen. Note that the above result does not hold for byte addresses, which may store a value in a padding byte, in which case the resulting value is indeterminate.
Lemma 6.64
7 Formal Proofs
7.1 TypeBased Alias Analysis
Here, a compiler should be able to assume that p and q are not aliased because they point to objects with different types (although the integer types \(\mathsf{signed}\;\mathsf{short}\) and \(\mathsf{signed}\;\mathsf{int}\) may have the same representation, they have different integer ranks, see Definition 4.2, and are thus different types). If g is called with aliased pointers, execution of the function body should have undefined behavior in order to allow a compiler to soundly assume that p and q are not aliased.
From the C11 standard’s description of effective types it is not immediate that calling g with aliased pointers results in undefined behavior. We prove an abstract property of our memory model that shows that this is indeed a consequence, and that indicates a compiler can perform typebased alias analysis. This also shows that our interpretation of effective types of the C11 standard, in line with the interpretation from the GCC documentation [20], is sensible.
Definition 7.1
A type \(\tau \) is a subobject type of \(\sigma \), notation \(\tau \subseteq _{\Gamma } \sigma \), if there exists some reference \(r\) with \(\Gamma \vdash {{r}} : \sigma \rightarrowtail \tau \).
For example, int[2] is a subobject type of struct S { int x[2]; int y[3]; } and int[2][2], but not of struct S { short x[2]; }, nor of int(*)[2].
Theorem 7.2
 1.
We have \(\sigma _1 \subseteq _{\Gamma } \sigma _2\) or \(\sigma _2 \subseteq _{\Gamma } \sigma _1\).
 2.
We have Open image in new window .
 3.Accessing \(a_1\) after accessing \(a_2\) and vice versa fails. That means:
 (a)
\((\mathsf{force}_{\Gamma }\,a_2\;m) \langle a_1 \rangle _{\Gamma } = \bot \) and \((\mathsf{force}_{\Gamma }\,a_1\;m) \langle a_2 \rangle _{\Gamma } = \bot \), and
 (b)
\(m \langle a_2 := v_1 \rangle _{\Gamma } \langle a_1 \rangle _{\Gamma } = \bot \) and \(m \langle a_1 := v_2 \rangle _{\Gamma } \langle a_2 \rangle _{\Gamma } = \bot \) for all stored values \(v_1\) and \(v_2\).
 (a)
If p and q are aliased, then calling g yields undefined behavior because the assignment *p = 10 violates effective types. Let m be the initial memory while executing g, and let \(a_{\mathtt p}\) and \(a_{\mathtt q}\) be the addresses corresponding to p and q, then the condition \(\mathsf{writable}_{\Gamma }\,a_{\mathtt p}\;(\mathsf{force}_{\Gamma }\,a_{\mathtt q}\;m)\) does not hold by Theorem 7.2 and Fact 6.61.
7.2 Memory Refinements
This section defines the notion of memory refinements that allows us to relate memory states. The author’s PhD thesis [33] shows that the \(\mathrm{CH}_2\mathrm{O}\) operational semantics is invariant under this notion. Memory refinements form a general way to validate many commonsense properties of the memory model in a formal way. For example, they show that the memory is invariant under relabeling. More interestingly, they show that symbolic information (such as variants of unions) cannot be observed.
Memory refinements also open the door to reason about program transformations. We demonstrate their usage by proving soundness of constant propagation and by verifying an abstract version of memcpy.
 1.
The function \(f : \mathsf{index}\rightarrow \mathsf{option}\;(\mathsf{index}\times \mathsf{ref})\) is used to rename object identifiers and to coalesce multiple objects into subobjects of a compound object.
 2.
Deallocated objects in \(m_1\) may be replaced by arbitrary objects in \(m_2\).
 3.
Indeterminate bits Open image in new window in \(m_1\) may be replaced by arbitrary bits in \(m_2\).
 4.
Pointer fragment bits \({(\mathsf{ptr}\;{p})_{{i}}}\) that belong to deallocated pointers in \(m_1\) may be replaced by arbitrary bits in \(m_2\).
 5.
Effective types may be weakened. That means, unions with a specific variant in \(m_1\) may be replaced by unions with an unspecified variant in \(m_2\), and pointers with frozen union annotations \(\circ \) in \(m_1\) may be replaced by pointers with unfrozen union annotations \(\bullet \) in \(m_2\).
The main judgment \(m_1 \sqsubseteq _{\Gamma }^{f : \Delta _1 \mapsto \Delta _2} m_2\) of memory refinements will be built using a series of refinement relations on the structures out of which the memory consists (addresses, pointers, bits, memory trees, values). All of these judgments should satisfy some basic properties, which are captured by the judgment \(\Delta _1 \sqsubseteq _{\Delta }^{f} \Delta _2\).
Definition 7.3
 1.
If \(f\,o_1 = (o,\,{{r}}_1)\) and \(f\,o_2 = (o,\,{{r}}_2)\), then \(o_1 = o_2\) or Open image in new window (injectivity).
 2.
If \(f\,o_1 = (o_2,\,{{r}})\), then \(\mathsf{frozen}\;{{r}}\).
 3.
If \(f\,o_1 = (o_2,\,{{r}})\) and \(\Delta _1 \vdash o_1 : \sigma \), then \(\Delta _2 \vdash o_2 : \tau \) and \(\Gamma \vdash {{r}} : \tau \rightarrowtail \sigma \) for a \(\tau \).
 4.
If \(f\,o_1 = (o_2,\,{{r}})\) and \(\Delta _2 \vdash o_2 : \tau \), then \(\Delta _1 \vdash o_1 : \sigma \) and \(\Gamma \vdash {{r}} : \tau \rightarrowtail \sigma \) for a \(\sigma \).
 5.
If \(f\,o_1 = (o_2,\,{{r}})\) and \(\Delta _1 \vdash o_1\ \mathsf{alive}\), then \(\Delta _2 \vdash o_2\ \mathsf{alive}\).
The renaming function \(f : \mathsf{index}\rightarrow \mathsf{option}\;(\mathsf{index}\times \mathsf{ref})\) is the core of all refinement judgments. It is used to rename object identifiers and to coalesce multiple source objects into subobjects of a single compound target object.
Consider a renaming f with Open image in new window and Open image in new window , and an environment \(\Gamma \) with \({\Gamma }\,{t}= [\, \tau _1,\tau _2 \,]\). This gives rise to following refinement:
Injectivity of renaming functions guarantees that distinct source objects are coalesced into disjoint target subobjects. In the case of Blazy and Leroy, the renaming functions have type \(\mathsf{index}\rightarrow \mathsf{option}\;(\mathsf{index}\times \mathbb {N})\), but we replaced the natural number by a reference since our memory model is structured using trees.
Since memory refinements rearrange the memory layout, addresses should be rearranged accordingly. The judgment \(a_1 \sqsubseteq _{\Gamma }^{f : \Delta _1 \mapsto \Delta _2} a_2 : {\tau }_{\mathsf{p}}\) describes how \(a_2\) is obtained by renaming \(a_1\) according to the renaming f, and moreover allows frozen union annotations \(\circ \) in \(a_1\) to be changed into unfrozen ones \(\bullet \) in \(a_2\). The index \({\tau }_{\mathsf{p}}\) in the judgment \(a_1 \sqsubseteq _{\Gamma }^{f : \Delta _1 \mapsto \Delta _2} a_2 : {\tau }_{\mathsf{p}}\) corresponds to the type of \(a_1\) and \(a_2\).
Lemma 7.4
If \(w_1 \sqsubseteq _{\Gamma }^{f : \Delta _1 \mapsto \Delta _2} w_2 : \tau \), then \(\Gamma ,\Delta _1 \vdash {w_1} : {\tau }\) and \(\Gamma ,\Delta _2 \vdash {w_2} : {\tau }\).
This lemma is useful because it removes the need for simultaneous inductions on both typing and refinement judgments.
We define \(m_1 \sqsubseteq _{\Gamma }^{f} m_2\) as \(m_1 \sqsubseteq _{\Gamma }^{f : \overline{m_1} \mapsto \overline{m_2}} m_2\), where the judgment \(m_1 \sqsubseteq _{\Gamma }^{f : \Delta _1 \mapsto \Delta _2} m_2\) is defined such that if \(f\,o_1 = (o_2,\,{{r}})\), then:
The above definition makes sure that objects are renamed, and possibly coalesced into subobjects of a compound object, as described by the renaming function f.
In order to reason about program transformations modularly, we show that memory refinements can be composed.
Lemma 7.5
Memory refinements are reflexive for valid memories, that means, if \(\Gamma ,\Delta \vdash m\), then \(m \sqsubseteq _{\Gamma }^{\mathsf{id} : \Delta \mapsto \Delta } m\) where \(\mathsf{id}\,o :=(o,\,\varepsilon )\).
Lemma 7.6
All memory operations are preserved by memory refinements. This property is not only useful for reasoning about program transformations, but also indicates that the memory interface does not expose internal details (such as variants of unions) that are unavailable in the memory of a (concrete) machine.
Lemma 7.7
If \(m_1 \sqsubseteq _{\Gamma }^{f : \Delta _1 \mapsto \Delta _2} m_2\) and \(a_1 \sqsubseteq _{\Gamma }^{f : \Delta _1 \mapsto \Delta _2} a_2 : \tau \) and \(m_1 \langle a_1 \rangle _{\Gamma } = v_1\), then there exists a value \(v_2\) with \(m_2 \langle a_2 \rangle _{\Gamma } = v_2\) and \(v_1 \sqsubseteq _{\Gamma }^{f : \Delta _1 \mapsto \Delta _2} v_2 : \tau \).
Lemma 7.8
 1.
We have \(\mathsf{writable}_{\Gamma }\,m_2\;a_2\).
 2.
We have \(m_1 \langle a_1 := v_1 \rangle _{\Gamma } \sqsubseteq _{\Gamma }^{f : \Delta _1 \mapsto \Delta _2} m_2 \langle a_2 := v_2 \rangle _{\Gamma }\).
As shown in Lemma 6.63, storing a value v in memory and then retrieving it, does not necessarily yield the same value v. In case of a byte address, the value may have been stored in padding and therefore have become indeterminate. Secondly, it intentionally yields the value \(\,v\,_\circ \) in which all pointers are frozen. However, the widely used compiler optimization of constant propagation, which substitutes values of known constants at compile time, is still valid in our memory model.
Lemma 7.9
If \(\Gamma ,\Delta \vdash v : \tau \), then \(\,v\,_\circ \sqsubseteq _{\Gamma }^{\Delta } v : \tau \).
Theorem 7.10
Copying an object w by an assignment results in it being converted to a value \(\mathsf{toval}_{\Gamma }\,w\) and back. This conversion makes invalid representations of base values indeterminate. Copying an object w bytewise results in it being converted to bits \(\overline{w}\) and back. This conversion makes all variants of unions unspecified. The following theorem shows that a copy by assignment can be transformed into a bytewise copy.
Theorem 7.11
Unused reads cannot be removed unconditionally in the \(\mathrm{CH}_2\mathrm{O}\) memory model because these have sideeffects in the form of uses of the \(\mathsf{force}_{\Gamma }\) operation that updates effective types. We show that uses of \(\mathsf{force}_{\Gamma }\) can be removed for frozen addresses.
Theorem 7.12
If \(\Gamma ,\Delta \vdash m\) and \(m \langle a \rangle _{\Gamma } \ne \bot \) and \(\mathsf{frozen}\;a\), then \(\mathsf{force}_{\Gamma }\,a\;m \sqsubseteq _{\Gamma }^{\Delta } m\).
7.3 Reasoning About Disjointness
In order to prove soundness of the \(\mathrm{CH}_2\mathrm{O}\) axiomatic semantics, we often needed to reason about preservation of disjointness under memory operations [33]. This section describes some machinery to ease reasoning about disjointness. We show that our machinery, as originally developed in [31], extends to any separation algebra.
Definition 7.13
 1.
\(\bot \,\varepsilon \)
 2.
If \(\bot \,{{x}}\) and \(x\, {\mathrel {\bot }}\, \bigcup {{x}}\), then \(\bot \,(x\,{{x}})\)
Notice that \(\bot \,{{x}}\) is stronger than having \(x_i\, {\mathrel {\bot }}\, x_j\) for each \(i \ne j\). For example, using fractional permissions, we do not have \(\bot \,[\, 0.5,\,0.5,\,0.5 \,]\) whereas \(0.5\, {\mathrel {\bot }}\, 0.5\) clearly holds. Using disjointness of lists we can for example state the associativity law (law 3 of Definition 5.1) in a symmetric way:
Fact 7.14
If \(\bot \,(x\;y\;z)\), then \(x \,{\mathrel {\cup }}\, (y \,{\mathrel {\cup }}\, z) = (x \,{\mathrel {\cup }}\, y) \,{\mathrel {\cup }}\, z\).
We define a relation Open image in new window that expresses that \({{x}}_1\) and \({{x}}_2\) behave equivalently with respect to disjointness.
Definition 7.15
It is straightforward to show that Open image in new window is reflexive and transitive, is respected by concatenation of lists, and is preserved by list containment. Hence, Open image in new window is an equivalence relation, a congruence with respect to concatenation of lists, and is preserved by permutations. The following results (on arbitrary separation algebras) allow us to reason algebraically about disjointness.
Fact 7.16
If Open image in new window , then \(\bot \,{{x}}_1\) implies \(\bot \,{{x}}_2\).
Fact 7.17
If Open image in new window , then \(\bot \,{{x}}_1\) iff \(\bot \,{{x}}_2\).
Theorem 7.18
In Sect. 7.4 we show that we have similar properties as the above for the specific operations of our memory model.
7.4 The Memory as a Separation Algebra
We show that the \(\mathrm{CH}_2\mathrm{O}\) memory model is a separation algebra, and that the separation algebra operations interact appropriately with the memory operations that we have defined in Sect. 6.
In order to define the separation algebra relations and operations on memories, we first define these on memory trees. Memory trees do not form a separation algebra themselves due to the absence of a unique \(\emptyset \) element (memory trees have a distinct identity element \(\mathsf{new}_{\Gamma }^{\tau }\,\) for each type \(\tau \), see Definition 6.31). The separation algebra of memories is then defined by lifting the definitions on memory trees to memories (which are basically finite functions to memory trees).
Definition 7.19
Fact 7.20
If \(\Gamma ,\Delta \vdash {w} : {\tau }\), then \(\mathsf{valid}\;w\).
The \(\mathsf{valid}\) predicate specifies the subset of memory trees on which the separation algebra structure is defined. The definition basically lifts the \(\mathsf{valid}\) predicate from the leaves to the trees. The sidecondition \(\lnot \mathsf{unmapped}\;(\overline{w}\,{{\mathbf {{b}}}})\) on \(\mathsf{union}_{t}\,(i,w,{\mathbf {{b}}})\) memory trees ensures canonicity, unions whose permissions are unmapped cannot be accessed and are thus kept in unspecified variant. Unmapped unions \(\overline{\mathsf{union}}_{t}\,{{\mathbf {{b}}}}\) can be combined with other unions using Open image in new window . The rationale for doing so will become clear in the context of the separation logic in the author’s PhD thesis [33].
Definition 7.21
Definition 7.22
The definitions of \(\mathsf{valid}\), Open image in new window and Open image in new window on memory trees satisfy all laws of a separation algebra (see Definition 5.1) apart from those involving \(\emptyset \). We prove the cancellation law explicitly since it involves the aforementioned sideconditions on unions.
Lemma 7.23
Proof
Definition 7.24
The emptiness conditions ensure canonicity. Objects that solely consist of indeterminate bits with \(\emptyset \) permission are meaningless and should not be kept at all. These conditions are needed for cancellativity.
Fact 7.25
If \(\Gamma ,\Delta \vdash m\), then \(\mathsf{valid}\;m\).
Lemma 7.26
Notice that the memory typing environment \(\Delta \) is not subdivided among \(m_1\) and \(m_2\). Consider the memory state corresponding to int x = 10, *p =&x:
Here, w is the memory tree that represents the integer value 10. The pointer on the right hand side is welltyped in the memory environment \(\overline{\phantom {X}o_{\mathtt x} \mapsto w,\ o_{\mathtt p} \mapsto \bullet }\) of the whole memory, but not in \(\overline{\phantom {X}o_{\mathtt p} \mapsto \bullet }\).
We prove some essential properties about the interaction between the separation algebra operations and the memory operations. These properties have been used in the soundness proof of the separation logic in the author’s PhD thesis [33].
Lemma 7.27
Lemma 7.28
Lemma 7.29
Memory trees and memories can be generalized to contain elements of an arbitrary separation algebra as leaves instead of just permission annotated bits [32]. These generalized memories form a functor that lifts the separation algebra structure on the leaves to entire trees. We have taken this approach in the Coq development, but for brevity’s sake, we have refrained from doing so in this paper.
8 Formalization in Coq
Realworld programming language have a large number of features that require large formal descriptions. As this paper has shown, the C programming language is not different in this regard. On top of that, the C semantics is very subtle due to an abundance of delicate corner cases. Designing a semantics for C and proving properties about such a semantics therefore inevitably requires computer support.
For these reasons, we have used Coq [15] to formalize all results in this paper. Although Coq does not guarantee the absence of mistakes in our definitions, it provides a rigorous set of checks on our definitions, for example by its type checking of definitions. On top of that, we have used Coq to prove all metatheoretical results stated in this paper. Last but not least, using Coq’s program extraction facility we have extracted an exploration tool to test our memory model on small example programs [33, 37]. Despite our choice to use Coq, we believe that nearly all parts of \(\mathrm{CH}_2\mathrm{O}\) could be formalized in any proof assistant based on higherorder logic.
8.1 Overloaded Typing Judgments
8.2 ImplementationDefined Behavior
8.3 Partial Functions
8.4 Automation
The proof style deployed in the \(\mathrm{CH}_2\mathrm{O}\) development combines interactive proofs with automated proofs. In this section we describe some tactics and forms of proof automation deployed in the \(\mathrm{CH}_2\mathrm{O}\) development.
Small inversions Coq’s inversion tactic has two serious shortcomings on inductively defined predicates with many constructors. It is rather slow and its way of controlling of names for variables and hypotheses is deficient. Hence, we often used the technique of small inversions by Monin and Shi [43] that improves on both shortcomings.
 1.
Simplify hypotheses using Theorem 7.18.
 2.
Solve sideconditions by simplification using Theorem 7.18 and a solver for list containment (implemented by reflection).
 3.
Repeat these steps until no further simplification is possible.
 4.
Finally, solve the goal by simplification using Theorem 7.18 and list containment.
Firstorder logic Many sideconditions we have encountered involve simple entailments of firstorder logic such as distributing logical quantifiers combined with some propositional reasoning. Coq does not provide a solver for firstorder logic apart from the firstorder tactic whose performance is already insufficient on small goals.
We have used Ltac to implemented an adhoc solver called naive_solver, which performs a simple breathfirst search proof search. Although this tactic is inherently incomplete and suffers from some limitations, it turned out to be sufficient to solve many uninteresting sideconditions (without the need for classical axioms).
8.5 Overview of the Coq Development
9 Related Work
The idea of using a memory model based on trees instead of arrays of plain bits, and the idea of using pointers based on paths instead of offsets, has already been used for object oriented languages. It goes back at least to Rossie and Friedman [51], and has been used by Ramananandro et al. [48] for C++. Furthermore, many researchers have considered connections between unstructured and structured views of data in C [2, 14, 21, 56] in the context of program logics.
However, a memory model that combines an abstract tree based structure with lowlevel object representations in terms of bytes has not been explored before. In this section we will describe other formalizations of the C memory model.
Norrish (1998) Norrish has formalized a significant fragment of the C89 standard using the proof assistant HOL4 [44, 45]. He was the first to describe nondeterminism and sequence points formally. Our treatment of these features has partly been based on his work. Norrish’s formalization of the C type system has some similarities with our type system: he has also omitted features that can be desugared and has proven type preservation.
Contrary to our work, Norrish has used an unstructured memory model based on sequences of bytes. Since he has considered the C89 standard in which effective types (and similar notions) were not introduced yet, his choice is appropriate. For C99 and beyond, a more detailed memory model like ours is needed, see also Sect. 3 and Defect Report #260 and #451 [26].
Another interesting difference is that Norrish represents abstract values (integers, pointers and structs) as sequences of bytes instead of mathematical values. Due to this, padding bytes retain their value while structs are copied. This is not faithful to the C99 standard and beyond.
Leroy et al. (2006) Leroy et al. have formalized a significant part of C using the Coq proof assistant [38, 39]. Their part of C, which is called CompCert C, covers most major features of C and can be compiled into assembly (PowerPC, ARM and x86) using a compiler written in Coq. Their compiler, called CompCert, has been proven correct with respect to the CompCert C and assembly semantics.
The goal of CompCert is essentially different from \(\mathrm{CH}_2\mathrm{O}\)’s. What can be proven with respect to the CompCert semantics does not have to hold for any C11 compiler, it just has to hold for the CompCert compiler. CompCert is therefore in its semantics allowed to restrict implementation defined behaviors to be very specific (for example, it uses 32bit ints since it targets only 32bit computing architectures) and allowed to give a defined semantics to various undefined behaviors (such as sequence point violations, violations of effective types, and certain uses of dangling pointers).
The CompCert memory model is used by all languages (from C until assembly) of the CompCert compiler [40, 41]. The CompCert memory is a finite partial function from object identifiers to objects. Each local, global and static variable, and invocation of malloc is associated with a unique object identifier of a separate object in memory. We have used the same approach in \(\mathrm{CH}_2\mathrm{O}\), but there are some important differences. The paragraphs below discuss the relation of \(\mathrm{CH}_2\mathrm{O}\) with the first and second version of the CompCert memory model.
Leroy and Blazy (2008) In the first version of the CompCert memory model [41], objects were represented as arrays of typeannotated fragments of base values. Examples of bytes are thus “the 2nd byte of the short 13” or “the 3rd byte of the pointer \((o,\,i)\)”. Pointers were represented as pairs \((o,\,i)\) where o is an object identifier and i the byte offset into the object o.
This code strongly resembles example [27, 6.5.2.3p9] from the C11 standard, which is stated to have undefined behavior.^{6} GCC and Clang optimize this code to print 10, which differs from the value assigned by Leroy and Blazy.
Apart from assigning too much defined behavior, Leroy and Blazy’s treatment of effective types also prohibits any form of “bit twiddling”.
Leroy and Blazy have introduced the notion of memory injections in [41]. This notion allows one to reason about memory transformations in an elegant way. Our notion of memory refinements (Sect. 7.2) generalize the approach of Leroy and Blazy to a tree based memory model.
Leroy et al. (2012) The second version of CompCert memory model [40] is entirely untyped and is extended with permissions. Symbolic bytes are only used for pointer values and indeterminate storage, whereas integer and floating point values are represented as numerical bytes (integers between 0 and \(2^81\)).
We have extended this approach by analogy to bitrepresentations, representing indeterminate storage and pointer values using symbolic bits, and integer values using concrete bits. This choice is detailed in Sect. 6.2.
As an extension of CompCert, Robert and Leroy have formally proven soundness of an alias analysis [50]. Their alias analysis is untyped and operates on the RTL intermediate language of CompCert.
Beringer et al. [7] have developed an extension of CompCert’s memory injections to reason about program transformations in the case of separate compilation. The issues of separate compilation are orthogonal to those that we consider.
Appel et al. (2014) The Verified Software Toolchain (VST) by Appel et al. provides a higherorder separation logic for Verifiable C, which is a variant of CompCert’s intermediate language Clight [3].
The VST is intended to be used together with the CompCert compiler. It gives very strong guarantees when done so. The soundness proof of the VST in conjunction with the correctness proof of the CompCert compiler ensure that the proven properties also hold for the generated assembly.
In case the verified program is compiled with a compiler different from CompCert, the trust in the program is still increased, but no full guarantees can be given. That is caused by the fact that CompCert’s intermediate language Clight uses a specific evaluation order and assigns defined behavior to many undefined behaviors of the C11 standard. For example, Clight assigns defined behavior to violations of effective types and sequence point violations. The VST inherits these defined behaviors from CompCert and allows one to use them in proofs.
Since the VST is linked to CompCert, it uses CompCert’s coarse permission system on the level of the operational semantics. Stewart and Appel [3, Chapter 42] have introduced a way to use a more fine grained permission system at the level of the separation logic without having to modify the Clight operational semantics. Their approach shows its merits when used for concurrency, in which case the memory model contains ghost data related to the conditions of locks [23, 24].
Besson et al. (2014) Besson et al. have proposed an extension of the CompCert memory model that assigns a defined semantics to operations that rely on the numerical values of uninitialized memory and pointers [8].
The semantics of Besson et al. is deterministic by definition. Normalization of symbolic expressions has defined behavior if and only if the expression can be normalized to a unique value under any choice of numeral values for pointer representations and uninitialized storage. In the second if statement this is not the case.
The approach of Besson et al. gives a semantics to some programming techniques that rely on the numerical representations of pointers and uninitialized memory. For example, it gives an appropriate semantics to pointer tagging in which unused bits of a pointer representation are used to store additional information.
However, as already observed by Kang et al. [28], Besson et al. do not give a semantics to many other useful cases. For example, printing the object representation of a struct, or computing the hash of a pointer value, is inherently nondeterministic. The approach of Besson et al. assigns undefined behavior to these use cases.
The goal of Besson et al. is inherently different from ours. Our goal is to describe the C11 standard faithfully whereas Besson et al. focus on de facto versions of C. They intentionally assign defined behavior to many constructs involving uninitialized memory that are clearly undefined according to the C11 standard, but that are nonetheless faithfully compiled by specific compilers.
Ellison and Roşu (2012) Ellison and Roşu [18, 19] have developed an executable semantics of the C11 standard using the \(\mathbb {K}\)framework.^{7} Their semantics is very comprehensive and describes all features of a freestanding C implementation [27, 4p6] including some parts of the standard library. It furthermore has been thoroughly tested against test suites (such as the GCC torture test suite), and has been used as an oracle for compiler testing [49].
Ellison and Roşu support more C features than we do, but they do not have infrastructure for formal proofs, and thus have not established any metatheoretical properties about their semantics. Their semantics, despite being written in a formal framework, should more be seen as a debugger, a state space search tool, or possibly, as a model checker. It is unlikely to be of practical use in proof assistants because it is defined on top of a large C abstract syntax and uses a rather adhoc execution state that contains over 90 components.
Similar to our work, Ellison and Roşu’s goal is to exactly describe the C11 standard. However, for some programs their semantics is less precise than ours, which is mainly caused by their memory model, which is less principled than ours. Their memory model is based on CompCert’s: it is essentially a finite map of objects consisting of unstructured arrays of bytes.
Hathhorn et al. (2015) Hathhorn et al. [22] have extended the work of Ellison and Roşu to handle more underspecification of C11. Most importantly, the memory model has been extended and support for the type qualifiers const, restrict and volatile has been added.
The above program has undefined behavior due to a violation of effective types. This is captured by our tree based memory model, but Hathhorn et al. require the program to return the value 11. When compiled with GCC or Clang with optimization level O2, the compiled program returns the value 10.
The restrictions on paddings bytes are implicit in our memory model based on structured trees, and thus handled correctly. The above examples provide evidence that a structured approach, especially combined with metatheoretical results, is more reliable than depending on adhoc decorations.
Kang et al. (2015) Kang et al. [28] have proposed a memory model that gives a semantics to pointer to integer casts. Their memory model uses a combination of numerical and symbolic representations of pointer values (whereas CompCert and \(\mathrm{CH}_2\mathrm{O}\) always represent pointer values symbolically). Initially each pointer is represented symbolically, but whenever the numerical representation of a pointer is needed (due to a pointer to integer cast), it is nondeterministically realized.
In a concrete memory model, there is the possibility that the function g is able to guess the numerical representation of &a, and thereby access or even modify a. This is undesirable, because it prevents the widely used optimization of constant propagation, which optimizes the variable a out.
In the CompCert and \(\mathrm{CH}_2\mathrm{O}\) memory model, where pointers are represented symbolically, it is guaranteed that f has exclusive control over a. Since &a has not been leaked, g can impossibly access a. In the memory model of Kang et al. a pointer will only be given a numerical representation when it is cast to an integer. In the above code, no such casts appear, and g cannot access a.
Here we loop through the range of integers of type uintptr_t until we have found the integer representation i of &x, which we then assign to the pointer p.
When compiled with gcc O2 (version 4.9.2), the generated assembly no longer contains a loop, and the pointers p and &x are assumed not to alias. As a result, the program prints the old value of x, namely 0. In the memory model of Kang et al. the pointer obtained via the cast (int*)i is exactly the same as &x. In their model the program thus has defined behavior and is required to print 15.
We have reported this issue to the GCC bug tracker.^{8} However it unclear whether the GCC developers consider this a bug or not. Some developers seem to believe that this program has undefined behavior and that GCC’s optimizations are thus justified. Note that the cast (intptr_t)&x is already forbidden by the type system of \(\mathrm{CH}_2\mathrm{O}\).
10 Conclusion
In this paper we have given a formal description of a significant part of the nonconcurrent C11 memory model. This formal description has been used in [33, 37] as part of an an operational, executable and axiomatic semantics of C. On top of this formal description, we have provided a comprehensive collection of metatheoretical results. All of these results have been formalized using the Coq proof assistant.
It would be interesting to investigate whether our memory model can be used to help the standard committee to improve future versions of the standard. For example, whether it could help to improve the standard’s prose description of effective types. As indicated on page 4 of Sect. 1, the standard’s description is not only ambiguous, but also does not cover its intent to enable typebased alias analysis. The description of our memory model is unambiguous and allows one to express intended consequences formally. We have formally proven soundness of an abstract version of typebased alias analysis with respect to our memory model (Theorem 7.2).

Floating point arithmetic Representations of floating point numbers and the behaviors of floating point arithmetic are subject to a considerable amount of implementation defined behavior [27, 5.2.4.2.2].
First of all, one could restrict to IEEE754 floating point arithmetic, which has a clear specification [25] and a comprehensive formalization in Coq [10]. Boldo et al. have taken this approach in the context of CompCert [9] and we see no fundamental problems applying it to \(\mathrm{CH}_2\mathrm{O}\) as well.
Alternatively, one could consider formalizing all implementationdefined aspects of the description of floating arithmetic in the C11 standard.

Bitfields Bitfields are fields of struct types that occupy individual bits [27, 6.7.2.1p9]. We do not foresee fundamental problems adding bitfields to \(\mathrm{CH}_2\mathrm{O}\) as bits already constitute the smallest unit of storage in our memory model.

Untyped malloc \(\mathrm{CH}_2\mathrm{O}\) supports dynamic memory allocation via an operator \(\mathsf{alloc}_{\tau }\;e\) close to C++’s new operator. The \(\mathsf{alloc}_{\tau }\;e\) operator yields a \(\tau {*}\) pointer to storage for a \(\tau \)array of length e. This is different from C’s malloc function that yields a void* pointer to storage of unknown type [27, 7.22.3.4].
Dynamic memory allocation via the untyped malloc function is closely related to unions and effective types. Only when dynamically allocated storage is actually used, it will receive an effective type. We expect one could treat malloced objects as unions that range over all possible types that fit.

Restrict qualifiers The restrict qualifier can be applied to any pointer type to express that the pointers do not alias. Since the description in the C11 standard [27, 6.7.3.1] is ambiguous (most notably, it is unclear how it interacts with nested pointers and data types), formalization and metatheoretical proofs may provide prospects for clarification.

Volatile qualifiers The volatile qualifier can be applied to any type to indicate that its value may be changed by an external process. It is meant to prevent compilers from optimizing away data accesses or reordering these [27, footnote 134]. Volatile accesses should thus be considered as a form of I/O.

Concurrency and atomics Sharedmemory concurrency and atomic operations are the main omission from the C11 standard in the \(\mathrm{CH}_2\mathrm{O}\) semantics. Although sharedmemory concurrency is a relatively new addition to the C and C++ standards, there is already a large body of ongoing work in this direction, see for example [4, 5, 52, 53, 57]. These works have led to improvements of the standard text.
There are still important open problems in the area of concurrent memory models for already small sublanguages of C [4]. Current memory models for these sublanguages involve just features specific to threads and atomic operations whereas we have focused on structs, unions, effective types and indeterminate memory. We hope that both directions are largely orthogonal and will eventually merge into a fully fledged C11 memory model and semantics.
Footnotes
 1.
 2.
 3.The array size is ignored unless the static keyword is used. In case f would have the prototype , the pointer a should provide access to an array of at least 10 elements [27, 6.7.6.3]. The static keyword is not supported by \(\mathrm{CH}_2\mathrm{O}\).
 4.
Contrary to traditional separation logic, we do not give local variables a special status of being stack allocated. We do so, because in C even local variables are allowed to have pointers to them.
 5.
Note that the C11 standard does not guarantee that the NULL pointer is represented as zeros, thus u.p is not necessarily NULL.
 6.
We have modified the example from the standard slightly in order to trigger optimizations by GCC and Clang.
 7.
This work has been superseded by Hathhorn et al. [22], which is described below.
 8.
Notes
Acknowledgments
I thank my supervisors Freek Wiedijk and Herman Geuvers for their helpful suggestions. I thank Xavier Leroy, Andrew Appel, Lennart Beringer and Gordon Stewart for many discussions on the CompCert memory model, and the anonymous reviewers for their feedback. This work is financed by the Netherlands Organisation for Scientific Research (NWO), Project 612.001.014.
References
 1.Affeldt, R., Marti, N.: Towards formal verification of TLS network packet processing written in C. In: PLPV, pp. 35–46 (2013)Google Scholar
 2.Affeldt, R., Sakaguchi, K.: An intrinsic encoding of a subset of C and its application to TLS network packet processing. JFR. 7(1), 63–104 (2014)Google Scholar
 3.Appel, A.W. (ed.): Program Logics for Certified Compilers. Cambridge University Press, Cambridge (2014)MATHGoogle Scholar
 4.Batty, M., Memarian, K., Nienhuis, K. PichonPharabod, J., Sewell, P.: The problem of programming language concurrency semantics. In: ESOP, volume 9032 of LNCS, pp. 283–307 (2015)Google Scholar
 5.Batty, M., Owens, S., Sarkar, S., Sewell, P., Weber, T.: Mathematizing C++ concurrency. In: POPL, pp. 55–66 (2011)Google Scholar
 6.Bengtson, J., Jensen, J.B., Sieczkowski, F., Birkedal, L.: Verifying Objectoriented programs with higherorder separation logic in Coq. In: ITP, volume 6898 of LNCS, pp. 22–38 (2011)Google Scholar
 7.Beringer, L., Stewart, G., Dockins, R., Appel, A.W.: Verified compilation for sharedmemory C. In: ESOP, volume 8410 of LNCS, pp. 107–127 (2014)Google Scholar
 8.Besson, F., Blazy, S., Wilke, P.: A precise and abstract memory model for C using symbolic values. In: APLAS, volume 8858 of LNCS, pp. 449–468 (2014)Google Scholar
 9.Boldo, S., Jourdan, J.H., Leroy, X., Melquiond, G.: A formallyverified C compiler supporting floatingpoint arithmetic. In: ARITH, pp. 107–115 (2013)Google Scholar
 10.Boldo, S., Melquiond, G.: Flocq: a unified library for proving floatingpoint algorithms in Coq. In: ARITH, pp. 243–252 (2011)Google Scholar
 11.Bornat, R., Calcagno, C., O’Hearn, P.W., Parkinson, M.J.: Permission accounting in separation logic. In: POPL, pp. 259–270 (2005)Google Scholar
 12.Boyland, J.: Checking interference with fractional permissions. In: SAS, volume 2694 of LNCS, pp. 55–72 (2003)Google Scholar
 13.Calcagno, C., O’Hearn, P.W., Yang, H.: Local action and abstract separation logic. In: LICS, pp. 366–378 (2007)Google Scholar
 14.Cohen, E., Moskal, M., Tobies, S., Schulte, W.: A precise yet efficient memory model for C. ENTCS 254, 85–103 (2009)Google Scholar
 15.Coq Development Team: The Coq proof assistant reference manual. Available at https://coq.inria.fr/doc/ (2015)
 16.Dijkstra, E.W.: Cooperating sequential processes. In: Genuys, F. (ed.) Programming Languages: NATO Advanced Study Institute, pp. 43–112. Academic Press, Cambridge (1968)Google Scholar
 17.Dockins, R., Hobor, A., Appel, A.W.: A fresh look at separation algebras and share accounting. In: APLAS, volume 5904 of LNCS, pp. 161–177 (2009)Google Scholar
 18.Ellison, C.: A Formal Semantics of C with Applications. PhD thesis, University of Illinois (2012)Google Scholar
 19.Ellison, C., Roşu, G.: An executable formal semantics of C with applications. In: POPL, pp. 533–544 (2012)Google Scholar
 20.GCC: The GNU Compiler Collection. Website, available at http://gcc.gnu.org/
 21.Greenaway, D., Lim, J., Andronick, J., Klein, G.: Don’t sweat the small stuff: formal verification of C code without the pain. In: PLDI, pp. 429–439 (2014)Google Scholar
 22.Hathhorn, C., Ellison, C., Roşu, G.: Defining the undefinedness of C. In: PLDI, pp. 336–345 (2015)Google Scholar
 23.Hobor, A.: Oracle Semantics. PhD thesis, Princeton University, (2008)Google Scholar
 24.Hobor, A., Appel, A.W., Nardelli, F.Z.: Oracle semantics for concurrent separation logic. In: ESOP, volume 4960 of LNCS, pp. 353–367 (2008)Google Scholar
 25.IEEE Computer Society: 7542008: IEEE Standard for Floating Point Arithmetic. IEEE (2008)Google Scholar
 26.ISO: WG14 Defect Report Summary. Website, available at http://www.openstd.org/jtc1/sc22/wg14/www/docs/
 27.ISO: ISO/IEC 98992011: Programming languages—C. ISO Working Group 14 (2012)Google Scholar
 28.Kang, J., Hur, C.K., Mansky, W., Garbuzov, D., Zdancewic, S., Vafeiadis, V.: A formal c memory model supporting integerpointer casts. In: PLDI, pp. 326–335 (2015)Google Scholar
 29.Klein, G., Kolanski, R., Boyton, A.: Mechanised separation algebra. In: ITP, volume 7406 of LNCS, pp. 332–337 (2012)Google Scholar
 30.Krebbers, R.: Aliasing restrictions of C11 formalized in Coq. In: CPP, volume 8307 of LNCS (2013)Google Scholar
 31.Krebbers, R.: An operational and axiomatic semantics for nondeterminism and sequence points in C. In: POPL, pp. 101–112 (2014)Google Scholar
 32.Krebbers, R.: Separation algebras for C verification in Coq. In: VSTTE, volume 8471 of LNCS, pp. 150–166 (2014)Google Scholar
 33.Krebbers, R.: The C standard formalized in Coq. PhD thesis, Radboud University (2015)Google Scholar
 34.Krebbers, R., Leroy, X., Wiedijk, F.: Formal C semantics: CompCert and the C standard. In: ITP, volume 8558 of LNCS, pp. 543–548 (2014)Google Scholar
 35.Krebbers, R., Wiedijk, F.: A formalization of the C99 standard in HOL, Isabelle and Coq. In: CICM, volume 6824 of LNCS, pp. 297–299 (2011)Google Scholar
 36.Krebbers, R., Wiedijk, F.: Separation logic for nonlocal control flow and block scope variables. In: FoSSaCS, volume 7794 of LNCS, pp. 257–272 (2013)Google Scholar
 37.Krebbers, R., Wiedijk, F.: A typed C11 semantics for interactive theorem proving. In: CPP, pp. 15–27 (2015)Google Scholar
 38.Leroy, X.: Formal certification of a compiler backend or: programming a compiler with a proof assistant. In: POPL, pp. 42–54 (2006)Google Scholar
 39.Leroy, X.: Formal verification of a realistic compiler. CACM 52(7), 107–115 (2009)CrossRefGoogle Scholar
 40.Leroy, X., Appel, A.W., Blazy, S., Stewart, G.: The CompCert Memory Model, Version 2. Research report RR7987, INRIA. Revised version available as Chapter 32 of [3] (2012)Google Scholar
 41.Leroy, X., Blazy, S.: Formal verification of a Clike memory model and its uses for verifying program transformations. JAR 41(1), 1–31 (2008)MathSciNetCrossRefMATHGoogle Scholar
 42.Maclaren, N.: What is an Object in C Terms? Mailing list message. Available at http://www.openstd.org/jtc1/sc22/wg14/9350 (2001)
 43.Monin, J., Shi, X.: Handcrafted Inversions made operational on operational semantics. In: ITP, volume 7998 of LNCS, pp. 338–353 (2013)Google Scholar
 44.Norrish, M.: C formalised in HOL. PhD thesis, University of Cambridge (1998)Google Scholar
 45.Norrish, M.: Deterministic expressions in C. In: ESOP, volume 1576 of LNCS, pp. 147–161 (1999)Google Scholar
 46.O’Hearn, P.W.: Resources, concurrency and local reasoning. In: CONCUR, volume 3170 of LNCS, pp. 49–67 (2004)Google Scholar
 47.O’Hearn, P.W., Reynolds, J.C., Yang., H.: Local reasoning about programs that alter data structures. In: CSL, volume 2142 of LNCS, pp. 1–19 (2001)Google Scholar
 48.Ramananandro, T., Dos Reis, G., Leroy, X.: Formal verification of object layout for C++ multiple inheritance. In: POPL, pp. 67–80 (2011)Google Scholar
 49.Regehr, J., Chen, Y., Cuoq, P., Eide, E., Ellison, C., Yang, X.: Testcase reduction for C compiler bugs. In: PLDI, pp. 335–346 (2012)Google Scholar
 50.Robert, V., Leroy, X.: A formallyverified alias analysis. In: CPP, volume 7679 of LNCS, pp. 11–26 (2012)Google Scholar
 51.Rossie, J.G., Friedman, D.P.: An algebraic semantics of subobjects. In: OOPSLA, pp. 187–199 (1995)Google Scholar
 52.Sevcík, J., Vafeiadis, V., Nardelli, F.Z., Jagannathan, S., Sewell, P.: CompCertTSO: a verified compiler for relaxedmemory concurrency. JACM 60(3), 22 (2013)MathSciNetCrossRefMATHGoogle Scholar
 53.Sewell, P., Sarkar, S., Owens, S., Nardelli, F.Z., Myreen, M.O.: x86TSO: a rigorous and usable programmer’s model for x86 multiprocessors. CACM 53(7), 89–97 (2010)CrossRefGoogle Scholar
 54.Sozeau, M.: A new look at generalized rewriting in type theory. JFR. 2(1), 41–62 (2009)Google Scholar
 55.Spitters, B., van der Weegen, E.: Type classes for mathematics in type theory. Math. Struct. Comput. Sci. 21(4), 795–825 (2011)MathSciNetCrossRefMATHGoogle Scholar
 56.Tuch, H., Klein, G., Norrish, M.: Types, bytes, and separation logic. In: POPL, pp. 97–108 (2007)Google Scholar
 57.Vafeiadis, V., Balabonski, T., Chakraborty, S., Morisset, R., Nardelli, F.Z.: Common compiler optimisations are invalid in the C11 memory model and what we can do about it. In: POPL, pp. 209–220 (2015)Google Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.