Journal of Mathematical Imaging and Vision

, Volume 59, Issue 2, pp 318–340 | Cite as

Overview of Shelling for 2-Manifold Surface Reconstruction Based on 3D Delaunay Triangulation

Article

Abstract

Recently, methods have been proposed to reconstruct a 2-manifold surface from a sparse cloud of points estimated from an image sequence. Once a 3D Delaunay triangulation is computed from the points, the surface is searched by growing a set of tetrahedra whose boundary is maintained 2-manifold. Shelling is a step that adds one tetrahedron at once to the growing set. This paper surveys properties that helps to understand the shelling performances: shelling provides most tetrahedra enclosed by the final surface, but it can “get stuck” or block in unexpected cases.

Keywords

Reconstruction Volumetric models Shellability 3D Delaunay triangulation Star shapes 

1 Introduction

In the recent years, a family of 2-manifold surface reconstruction methods have been proposed to deal with a sparse cloud of input points and their visibility information estimated from an image sequence. There are both batch [13] and incremental [15, 16, 18, 23] variations. First, a 3D Delaunay triangulation is computed from the sparse point cloud to divide up the space by a set T of tetrahedra. Then, the visibility information is used to associate a free space score to every tetrahedron \(\Delta \in T\), e.g. by counting the rays (line segments between points and camera poses used to reconstruct these points) that intersect \(\Delta \). Last, these scores are used for growing a set O of tetrahedra in T such that the boundary \(\partial O\) of O is maintained 2-manifold.

Several operations are designed for the O growing. One of them adds one tetrahedron at once to O and is closely related to the “shelling” process that is studied in Combinatorial Topology: Algorithm 1 in [13] generates a shelling in the 3D case if O is initialized by a single tetrahedron. This greedy algorithm tries to add to O the tetrahedron in \(T{\setminus } O\) that has the largest visibility score (and that has a face triangle in \(\partial O\)) if \(\partial O\) is still a 2-manifold. The goal of the paper is to survey the shelling properties in our surface reconstruction context, where O is included in a 3D Delaunay triangulation, and the O initialization is not restricted to a single tetrahedron.

First, Sect. 2 introduces prerequisites. Second, Sect. 3 provides our shelling definition and shows that it does not change the topology (number of connected components and genus) of \(\partial O\) in almost all cases.

Third, Sect. 4 overviews shelling properties known in Combinatorial Topology, i.e. if O is initialized by a single tetrahedron. At first glance, one could guess that a shelling started by a single tetrahedron in T can reach/have every end value \(O_e\subseteq T\) such that tetrahedron union \(\cup O_e\) is homeomorphic to one tetrahedron. Surprisingly, this is wrong: some of these \(O_e\) do not have such a shelling. This implies that no greedy algorithm [13, 18] (whatever the free space scoring of the tetrahedra) can reach \(O_e\). Furthermore, even if there is a shelling that reaches \(O_e\), a greedy algorithm can “get stuck” or block: it can generate another shelling with other tetrahedron choice(s) at the beginning such that the end value is strictly included in \(O_e\).

Fourth, Sect. 5 studies shelling in the favorable case of star shapes in a 3D Delaunay triangulation. We remind that \(\cup O\) is star-shaped with respect to the center \(\mathbf {c}\in \mathbb {R}^3\), if for every point \(\mathbf {x}\in \cup O\), the line segment \(\mathbf {xc}\subseteq \cup O\). More precisely, we show that there is a shelling that starts from \(O_s\) and ends to \(O_e\subseteq T\) if both \(\cup O_s\) and \(\cup O_e\) are star-shaped with respect to the same center. This generalizes a known case where \(\cup O_e\) is convex (indeed, a convex set is star-shaped with respect to every of its point) and \(O_s\) has a single tetrahedron. Then, Sect. 5 explains why shelling provides most of the tetrahedra enclosed by the final surface in our context (shelling does not have excessive blocking).

Last, Sect. 6 provides examples of shelling blocking: a shelling starts from \(O_s\) but cannot end to \(O_e\) although \(\cup O_s\) and \(\cup O_e\) have the same topology. This should convince the reader that such cases really exist, although they don’t in the 2D case [4] (i.e. by replacing tetrahedra by triangles). The examples are chosen to be as simple as possible. Section 6 introduces a family of sets O that cannot be reached by a shelling started from a single tetrahedron (as in Sect. 4), which generalizes an example in [24]. It also introduces examples of visual artifact similar to that in Fig. 1 of [16], which cannot be removed by shelling alone (against the intuition).

2 Main Prerequisites

The majority of prerequisites are in [5, 9, 11, 14, 21].

2.1 Simplicial Complexes in \(\mathbb {R}^n\)

Let integers \(k\ge 0\) and \(n> 0\). A simplex \(\sigma \) is the convex hull of \(k+1\) points \(\mathbf {v}_0, \ldots , \mathbf {v}_k\) in general position in \(\mathbb {R}^n\); \(\sigma \) is a vertex if \(k=0\), an edge if \(k=1\), a triangle if \(k=2\), a tetrahedron if \(k=3\). The dimension of \(\sigma \) is k. A simplex \(\sigma '\) is a face of \(\sigma \) if \(\sigma '\) is the convex hull of some of the \(\mathbf {v}_i\) above (we have \(\sigma '\subseteq \sigma \)). A simplicial complex K in \(\mathbb {R}^n\) is a finite set of simplices in \(\mathbb {R}^n\) such that (1) \(\sigma '\in K\) if \(\sigma '\) is a face of \(\sigma \in K\) and (2) \(\sigma \cap \sigma '\) is empty or a face of \(\sigma \) and \(\sigma '\) if both \(\sigma \) and \(\sigma '\) are in K.

A 3D Delaunay triangulation is a simplicial complex K in \(\mathbb {R}^3\) that meets several conditions. There is a set T of tetrahedra such that the faces of the tetrahedra are the simplices in K. Let V be the vertex set of K. The circumscribing sphere of every tetrahedron in T does not contain a vertex in V in its interior. The convex hull of V is the union of the tetrahedra in T.

2.2 Closure, Pure Simplicial Complex, Boundary

Let A be included in a simplicial complex. The closure c(A) of A is the set of all faces (including the vertices) of the simplices in A. If both A and B are included in a same simplicial complex, \(c(A\cup B)= c(A)\cup c(B)\), \(c(A\cap B)\subseteq c(A)\cap c(B)\). If \(A\subseteq B\), \(c(A)\subseteq c(B)\). We use notation \(c(\sigma _1,\ldots ,\sigma _k)= c(\{ \sigma _1,\ldots ,\sigma _k \})\) for simplices \(\sigma _i\).

Let X be a simplicial complex. We say that X is kD pure if there are simplices(s) \(\sigma _i\in X\) with a same dimension k such that \(X= c(\sigma _1,\ldots , \sigma _m)\). A 3D Delaunay triangulation K is 3D pure and meets \(K=c(T)\) where T is the set of all tetrahedra in K.

If A is a set of simplices with a same dimension \(k>0\), the boundary \(\partial A\) of A is the set of all simplices of dimension \(k-1\) in c(A) that are faces of exactly one simplex in A. The boundary \(\partial \sigma \) of a simplex \(\sigma \) is \(\partial \{\sigma \}\).

2.3 Abstract Simplicial Complexes

Let \(\mathcal{V}\) be a finite set. An abstract simplicial complex \(\mathcal{S}\) is a set of subsets of \(\mathcal{V}\) such that \(A\in \mathcal{S}\) and \(B\subset A\) imply \(B\in \mathcal{S}\). It is implicitly defined by every simplicial complex K as follows: \(\mathcal{V}\) is the vertex set of K and \(\mathcal{S}\) is the set of the vertex sets of the K simplices. Conversely, K is a realization of \(\mathcal{S}\) in \(\mathbb {R}^n\). The elements of \(\mathcal{S}\) are called (abstract) simplices; their faces are their subsets.

The definitions of edge/triangle/tetrahedron, closure, kD pure and boundary still hold for abstract simplices and abstract simplicial complexes, replacing simplices by abstract simplices and using their same inclusion relations. Same notations are used for a simplicial complex and its abstract version, and also for a simplex and its abstract version. We use bold fonts for vertices, e.g. \(\mathbf {ab}\) is an edge (abstract or non-abstract).

Let c(T) be a 3D Delaunay triangulation using a non-empty set T of tetrahedra. Every triangle in c(T) is included in exactly two tetrahedra in c(T), except those in \(\partial T\) (we have \(\partial T\ne \emptyset \)). These exceptions are removed as in [2, 14] to make easier proofs. Let \(\mathbf {v}_\infty \) be an abstract vertex that is different to those in c(T). For every triangle \(\mathbf {abc}\in \partial T\), we create a new tetrahedron \(\mathbf {abcv}_\infty \) by adding \(\mathbf {v}_\infty \) to the set \(\mathbf {abc}\). Let \(T^\infty = T\cup \{\mathbf {abcv}_\infty , \mathbf {abc}\in \partial T\}\). Now \(T\subset T^\infty \) and every triangle in \(c(T^\infty )\) is a face of exactly 2 tetrahedra in \(T^\infty \). Notations T and \(T^\infty \) are used in the whole paper.

2.4 2-Manifolds (with Boundary), k-Spheres, k-Balls

Let \(A\subseteq \mathbb {R}^n\) and \(B\subseteq \mathbb {R}^m\). If there is a bijective and continuous function between A and B such that the inverse function is also continuous, A and B are homeomorphic. We say that A is a 2-manifold with boundary if every point \(\mathbf {x}\in A\) has a neighborhood in A that is homeomorphic to \(\mathbb {R}^2\) or \(\mathbb {R}\times \mathbb {R}^+\) (\(\mathbb {R}^+\) includes 0); A is a 2-manifold if every point \(\mathbf {x}\in A\) has a neighborhood in A that is homeomorphic to \(\mathbb {R}^2\); A is a k-ball (\(k\in \{1,2,3\}\)) if it is homeomorphic to \(\{ \mathbf {x}\in \mathbb {R}^k, ||\mathbf {x}||\le 1\}\); A is a k-sphere (\(k\in \{1,2\}\)) if it is homeomorphic to \(\{ \mathbf {x}\in \mathbb {R}^{k+1}, ||\mathbf {x}||= 1\}\). 2-spheres are 2-manifolds, 2-manifolds and 2-balls are 2-manifolds with boundary.

Let \(Y\subseteq c(T)\). We define \(|Y|=\cup Y= \cup _{\sigma \in Y} \sigma \) (here \(\sigma \) is a convex hull, it is not an abstract simplex). We say that Y is homeomorphic to B if |Y| is homeomorphic to B. Thus Y can be a 2-manifold/2-manifold with boundary/k-sphere/k-ball. Here are examples: a triangle in c(T) is a 2-ball, \(\partial T\) is a 2-sphere, \(\partial \Delta \) is a 2-sphere and \(\Delta \) is a 3-ball if tetrahedron \(\Delta \in c(T)\).

Let vertex series \(\mathbf {v}_1\cdots \mathbf {v}_m\) where \(m\ge 2\). We use notation \(\mathbf {v}_1\hbox {-}\mathbf {v}_2\hbox {-}\cdots \hbox {-}\mathbf {v}_m\) for the set of edges \(\mathbf {v}_1\mathbf {v}_2\), \(\mathbf {v}_2\mathbf {v}_3,\ldots \), \(\mathbf {v}_{m-1}\mathbf {v}_m\). If the \(\mathbf {v}_i\) are distinct and every \(\mathbf {v}_i\mathbf {v}_{i+1}\in c(T)\), \(\mathbf {v}_1\hbox {-}\mathbf {v}_2\hbox {-}\cdots \hbox {-}\mathbf {v}_m\) is a 1-ball. A cycle is an edge set \(\mathbf {v}_1\hbox {-}\cdots \hbox {-}\mathbf {v}_m\hbox {-}\mathbf {v}_1\) such that \(m\ge 3\) and the \(\mathbf {v}_i\) are distinct. A cycle is a 1-sphere if all its edges are in c(T).

Here we merge Theorem 4 and Lemma 4 in [14]:

Theorem 1

Let \(O\subset T^\infty \) be such that \(\partial O\subset c(T)\) and \(\partial O\) is a 2-manifold. If \(\Delta \) is a tetrahedron in \(T{\setminus } O\), \(\partial (O\cup \{\Delta \})\) is a 2-manifold iff \(c(O)\cap c(\Delta )\) is 2D pure.

2.5 2-Manifold Criterion for Simplicial Complexes

If \(\mathbf {v}\) is a vertex and E is a set of edges, \(\mathbf {v}\times E\) is the set of the triangles defined by joining \(\mathbf {v}\) and every edge in E: \(\mathbf {v}\times E=\{ \mathbf {vab}, \mathbf {ab}\in E \}\). Let X be a set of triangles in c(T). If a vertex \(\mathbf {v}\in c(X)\), we define \(X_\mathbf {v}\) by the set of all triangles in X that include \(\mathbf {v}\). There is a set \(E_\mathbf {v}\) of edges such that \(X_\mathbf {v}= \mathbf {v}\times E_\mathbf {v}\). If \(E_\mathbf {v}\) is not a cycle, \(\mathbf {v}\) is a singular vertex of X.

According to [1, 14, 21], X is a 2-manifold iff for every vertex \(\mathbf {v}\in c(X)\), there is a cycle \(\mathbf {a}_*\) such that \(X_\mathbf {v}= \mathbf {v}\times \mathbf {a}_*\). Thus there are distinct vertices \(\mathbf {a}_i\in c(X_\mathbf {v})\) such that \(\mathbf {a}_*= \mathbf {a}_1\hbox {-}\cdots \hbox {-}\mathbf {a}_m\hbox {-}\mathbf {a}_1\). This implies that every edge \(\mathbf {vw}\in c(X)\) is included in exactly two triangles in X (indeed, \(\mathbf {w}\) is an \(\mathbf {a}_i\) and all \(\mathbf {a}_i\) are distinct).

According to [5, 9], X is a 2-manifold with boundary iff for every vertex \(\mathbf {v}\in c(X)\), there is a non-empty set of edge(s) \(\mathbf {a}_*\) such that \(X_\mathbf {v}= \mathbf {v}\times \mathbf {a}_*\) with two cases: there are distinct vertices \(\mathbf {a}_i\) such that \(\mathbf {a}_*= \mathbf {a}_1\hbox {-}\cdots \mathbf {a}_m\) or \(\mathbf {a}_*= \mathbf {a}_1\hbox {-}\cdots \mathbf {a}_m\hbox {-}\mathbf {a}_1\) is a cycle (\(m\ge 2\) in the first case and \(m\ge 3\) in the second case). Thus every \(\mathbf {a}_i\in c(X_\mathbf {v})\) and every edge \(\mathbf {vw}\in c(X)\) is included in one or two triangle(s) in X (but not more).

2.6 Connectivity and Genus

Let \(X\subseteq c(T^\infty )\). We say that X is connected if for all vertices \(\mathbf {v}\) and \(\mathbf {v}'\) in c(X), there are vertices \(\mathbf {v}_i\in c(X)\) such that \(1\le i\le m\), \(\mathbf {v}=\mathbf {v}_1\), \(\mathbf {v}'=\mathbf {v}_m\) and \(\mathbf {v}_1\hbox {-}\mathbf {v}_2\hbox {-}\cdots \hbox {-}\mathbf {v}_m \subseteq c(X)\). A connected component X is a connected subset of X that is maximal in the inclusion sense. If two sets are homeomorphic, they have the same number of connected components. If \(X\subseteq c(T)\), X is connected iff |X| is connected in \(\mathbb {R}^3\).

If X is a set of simplices with a same dimension k, X is strongly connected if for all \(\sigma \) and \(\sigma '\) in X, there are \(\sigma _i\in X\) such that \(1\le i\le m\), \(\sigma _1= \sigma \), \(\sigma _m= \sigma '\) and every \(\sigma _i\cap \sigma _{i+1}\) is a \(k-1\) dimensional simplex. For example, \(T^\infty \), T and \(\partial T\) are strongly connected. If X is strongly connected, X is connected.

Let \(X\subset c(T)\) be a connected 2-manifold. The genus g of X meets \(2(1-g)= v-e+t\) where v, e and t are the numbers of the vertices, edges and triangles in c(X), respectively. Two connected 2-manifolds in c(T) are homeomorphic iff they have the same genus. The genus is the number of handle(s) of X, e.g. \(g=0\) if X is a 2-sphere, \(g=1\) if X is homeomorphic to a torus. The genus of a non-connected 2-manifold in c(T) is the sum of genuses of its connected components.

3 Shelling Definition and Properties

First, Sect. 3.1 provides our definition and explains the shelling algorithm in [13]. Then, Sect. 3.2 describes the two cases of shelling steps. Last, Sect. 3.3 details the topological changes in these two cases.

3.1 Definition

In our context (surface reconstruction), a shelling is a series of tetrahedron sets \(O_0, O_1, \ldots , O_n\) such that
  1. 1.

    \(O_0\subseteq T^\infty \) and \(\partial O_0\) is a non-empty 2-manifold

     
  2. 2.

    \(O_{i+1}= O_i\cup \{\Delta _i\}\) where a tetrahedron \(\Delta _i\in T{\setminus } O_i\) and \(c(O_i)\cap c(\Delta _i)\) is a non-empty 2D-pure simplicial complex if \(0\le i< n\).

     
Thus every \(\partial O_i\) is a 2-manifold thanks to Theorem 1.
Fig. 1

A shelling example in the 2D case: O is a growing set of triangles such that \(\partial O\) is a cycle (drawn with bold edges)

Figure 1 shows a shelling in the 2D case to help intuition: replace tetrahedra by triangles, 2-manifold by 1-manifold (a disjoint union of cycles), 2D-pure by 1D-pure.

There is another definition in remark 8.3.ii of [25] (see also remark 2 in chapter 7 of [9], and definition 2.7 of [12]): a shelling of a d-D pure simplicial complex K is a series \(\sigma _0, \sigma _1,\ldots , \sigma _n\) of d-D simplices such that \(K= c(\sigma _0, \sigma _1,\ldots , \sigma _n)\) and every \(c(\sigma _0, \sigma _1, \cdots \sigma _{j-1})\cap c(\sigma _j)\) is a non-empty \((d-1)\)-D-pure simplicial complex. Both definitions are equivalent if \(d=3\) and \(O_0=\{ \sigma _0\}\subseteq T\) and \(K\subseteq c(T)\).

Our definition is different by two ways. First we prefer a cumulative formulation based on tetrahedron sets \(O_i\) since our application computes surfaces \(\partial O_i\). Second a more general initialization \(O_0\) is used since shellings are started at several steps of the surface reconstruction in [13]. Indeed,

Theorem 2

Algorithm 1 in [13] generates a shelling.

We describe this algorithm before the proof of Theorem 2. Let \(F\subseteq T\) be the free space: a tetrahedron \(\Delta \in F\) iff there is a point \(\mathbf {x}\) reconstructed from a viewpoint \(\mathbf {c}\) using Computer Vision techniques such that \(\Delta \cap \mathbf {cx}\ne \emptyset \). Let \(n_\Delta \) be the number of line segments \(\mathbf {cx}\) intersecting \(\Delta \). The goal is the computation of \(O\subseteq F\) that maximizes \(\sum _{\Delta \in O} n_\Delta \) subject to the constraint that \(\partial O\) is a 2-manifold. Let \(Q\subseteq F{\setminus } O\) be a set of tetrahedra that should be added to the current estimate of O. Furthermore, Q is included in the immediate neighborhood of O: every tetrahedron \(\Delta \in Q\) has a common triangle with a tetrahedron in O (if \(O\ne \emptyset \)), i.e. \(\partial \Delta \cap \partial O\ne \emptyset \). At each step of Algorithm 1, we remove a tetrahedron \(\Delta \) from Q with the largest \(n_\Delta \), then we add \(\Delta \) to O if \(\partial O\) remains a 2-manifold. If \(\Delta \) is added to O, we also add to Q every tetrahedron \(\Delta '\in F{\setminus } O\) such that \(\partial \Delta \cap \partial \Delta '\ne \emptyset \). This algorithm stops when \(Q= \emptyset \). It is used several times in [13] for several initializations of Q and O.

Proof

This algorithm computes a series \(O_i\) (consecutive values of O) such that \(O_{i+1}= O_i\cup \{\Delta _i\}\), \(\Delta _i\in T{\setminus } O_i\), \(\partial \Delta _i\cap \partial O_i\ne \emptyset \) and \(\partial O_i\) is a 2-manifold for all i. According to Theorem 1, \(c(O_i)\cap c(\Delta _i)\) is 2D pure. Since \(\partial \Delta _i\cap \partial O_i\subseteq c(O_i)\cap c(\Delta _i)\), \(c(O_i)\cap c(\Delta _i)\) is non-empty 2D pure. Furthermore, \(\partial O_0\) is a non-empty 2-manifold (if the algorithm starts with \(O=\emptyset \), set \(O_0\) as the first tetrahedron added to O). \(\square \)

3.2 Basic Properties

Several remarks can be done about shelling thanks to the following Lemma (this is Lemma 1 in [14]).

Lemma 1

Let \(O\subseteq T^\infty \) and \(\Delta \) be a tetrahedron in \(T{\setminus } O\). Then \(c(O)\cap c(\Delta )\) is a simplicial complex in \(\mathbb {R}^3\). Furthermore, \(c(O)\cap c(\Delta )= c(\partial O)\cap c(\partial \Delta )\) and the triangles in \(c(O)\cap c(\Delta )\) are exactly those in \(\partial O\cap \partial \Delta \) (Fig. 2).

Fig. 2

Lemma 1 in the 2D case. Left \(c(\Delta )\) and \(c(\partial \Delta )\). Middle c(O) and \(c(\partial O)\). Right \(c(\Delta )\cap c(O)\). Both \(\Delta \) and O are in gray, T is the set of all triangles, \(\Delta \in T{\setminus } O\), the edges of \(c(\partial \Delta )\) and \(c(\partial O)\) are thickened, the vertices of \(c(\Delta )\) and c(O) are black squares. We see that \(c(\Delta )\cap c(O)\) has 1 edge and 3 vertices, \(c(\Delta )\cap c(O)= c(\partial \Delta )\cap c(\partial O)\), \(\partial O\cap \partial \Delta \) has 1 edge

First the shelling definition is the same if we replace \(c(O_i)\cap c(\Delta _i)\) by \(c(\partial O_i)\cap c(\partial \Delta _i)\). Intuitively, this is the simplicial complex “between” \(O_i\) and \(\Delta _i\). Second the number of the triangles in \(\partial O_i\cap \partial \Delta _i\) cannot be 0.

Now we provide the two cases of shelling steps. They are in a corollary of the following theorem:

Theorem 3

Let \(\Delta \) be a tetrahedron in T. Let \(K\subseteq c(\Delta )\) be a simplicial complex. Then K is a non-empty 2D-pure simplicial complex iff it is a 2-ball or a 2-sphere. Let f be the number of the triangles in K. In the 2-ball case, \(f\in \{ 1,2,3 \}\). In the 2-sphere case, \(f=4\).

Proof

We show that assertions AK is a non-empty 2D-pure simplicial complex” and BK is a 2-ball or a 2-sphere” are simultaneously true or false for every simplicial complex \(K\subseteq c(\Delta )\).

If \(K=\emptyset \) or \(K= c(\Delta )\) or K has only one vertex, both A and B are wrong. Otherwise K has at least two vertices. If one of these vertices is not in an edge of K, K is not connected. Thus, both A and B are wrong.

Now we consider the case where an edge \(\mathbf {ab}\in K\) and triangles \(\mathbf {abc}\notin K\) and \(\mathbf {abd}\notin K\) using notation \(\mathbf {abcd}=\Delta \). Here A is wrong. Let points \(\mathbf {x}= {2\over 3}\mathbf {a}+{1\over 3}\mathbf {b}\) and \(\mathbf {y}= {1\over 3}\mathbf {a}+{2\over 3}\mathbf {b}\). Assume (reductio ad absurdum) that B is true. Thus \(|K|{\setminus }\{ \mathbf {x}, \mathbf {y}\}\) is homeomorphic to a 2-ball or a 2-sphere minus two points, which is connected. This is impossible since \(|K|{\setminus }\{ \mathbf {x}, \mathbf {y}\}\) has (at least) two connected components: \(\mathbf {xy}{\setminus }\{ \mathbf {x}, \mathbf {y}\}\) and another one included in \((\mathbf {ax}{\setminus }\{\mathbf {x}\})\cup (\mathbf {by}{\setminus }\{\mathbf {y}\})\cup \mathbf {cda}\cup \mathbf {cdb}\).

Assume that we are not in the previous cases. Thus A is true. We show that B is true. We have triangles \(t_i\in \partial \Delta \) and \(K=c(t_1,t_2, \ldots , t_f)\) where \(f\ge 1\). If \(f=4\), \(K= c(\partial \Delta )\) and K is a 2-sphere. If \(f=1\), \(|K|= t_1\) is a 2-ball. If \(f=3\), \(|K|= t_1\cup t_2\cup t_3\) is homeomorphic to \(t_4\) (use the piecewise linear function that is the identity at the \(t_4\) vertices and maps the \(t_4\) center to vertex \(t_1\cap t_2\cap t_3\)). If \(f=2\), \(|K|= t_1\cup t_2\) is homeomorphic to \(t_3\) (thanks to a similar piecewise linear function). \(\square \)

Corollary 1

A simplicial complex \(c(O_i)\cap c(\Delta _i)\) is non-empty and 2D-pure (in the shelling definition) iff it is a 2-ball or a 2-sphere. Let f be the number of the triangles in \(\partial O_i\cap \partial \Delta _i\). In the ball case, \(1\le f\le 3\), otherwise \(f=4\).

Proof

The set \(K=c(O_i)\cap c(\Delta _i)\) is a simplicial complex (Lemma 1) included in \(c(\Delta _i)\). According to Theorem 3, K is non-empty and 2D-pure iff it is a 2-ball (\(f\in \{ 1,2,3 \}\)) or a 2-sphere (\(f=4\)). \(\square \)

3.3 Topology of Surfaces \(\partial O_i\)

Theorem 4 describes all topology change(s) by adding one tetrahedron in \(O_i\). The two following lemmas are needed in its proof (the former is Lemma 3 in [14].

Lemma 2

Let \(O\subset T^\infty \) and \(\Delta \) be a tetrahedron in \(T{\setminus } O\). Then \(\partial (O\cup \{\Delta \})= (\partial \Delta {\setminus }\partial O)\cup (\partial O{\setminus }\partial \Delta )\).

Lemma 3

Let \(O\subset T^\infty \) be such that \(\partial O\subset c(T)\) and \(\partial O\) is a 2-manifold. Let \(\Delta \) be a tetrahedron in \(T{\setminus } O\) such that \(c(O)\cap c(\Delta )\) is 2D pure. Let f be the number of the triangles in \(\partial O\cap \partial \Delta \). If \(1\le f\le 3\), \(\partial O\) and \(\partial (O\cup \{\Delta \})\) are homeomorphic.

Proof

We find a homeomorphism \(\varphi \) such that \(\varphi (|\partial O|)= |\partial (O\cup \{\Delta \})|\). First \(\varphi \) is defined by its values on vertices and linear interpolation on the \(\partial O\) triangles (or their subdivisions) as follows. For every vertex \(\mathbf {v}\) in \(c(\partial O)\), we set \(\varphi (\mathbf {v})= \mathbf {v}\). We use notation \(\Delta =\mathbf {a}\mathbf {b}\mathbf {c}\mathbf {d}\).

If \(f=1\), \(\partial O\cap \partial \Delta = \{\mathbf {a}\mathbf {b}\mathbf {c}\}\), we split \(\mathbf {a}\mathbf {b}\mathbf {c}\) into triangles \(\mathbf {a}\mathbf {b}\mathbf {e}\), \(\mathbf {b}\mathbf {c}\mathbf {e}\), \(\mathbf {c}\mathbf {a}\mathbf {e}\) where \(\mathbf {e}= (\mathbf {a}+\mathbf {b}+\mathbf {c})/3\), and set \(\varphi (\mathbf {e})= \mathbf {d}\). We obtain \(\varphi (\mathbf {abc})= \mathbf {abd}\cup \mathbf {bcd}\cup \mathbf {cad}\).

If \(f=2\), \(\partial O\cap \partial \Delta = \{ \mathbf {a}\mathbf {b}\mathbf {c}, \mathbf {b}\mathbf {c}\mathbf {d} \}\), we split \(\mathbf {b}\mathbf {c}\) into edges \(\mathbf {b}\mathbf {e}\) and \(\mathbf {e}\mathbf {c}\) where \(\mathbf {e}= (\mathbf {b}+\mathbf {c})/2\), we split \(\mathbf {a}\mathbf {d}\) into edges \(\mathbf {a}\mathbf {g}\) and \(\mathbf {g}\mathbf {d}\) where \(\mathbf {g}= (\mathbf {a}+\mathbf {d})/2\), and set \(\varphi (\mathbf {e})= \mathbf {g}\). Note that this scheme also splits every triangle including \(\mathbf {bc}\) or \(\mathbf {ad}\) into two other triangles. We obtain \(\varphi (\mathbf {abc}\cup \mathbf {bcd})= \mathbf {adb}\cup \mathbf {adc}\).

If \(f=3\), \(\partial O\cap \partial \Delta = \{ \mathbf {d}\mathbf {a}\mathbf {b}, \mathbf {d}\mathbf {b}\mathbf {c}, \mathbf {d}\mathbf {c}\mathbf {a} \}\), we split \(\mathbf {a}\mathbf {b}\mathbf {c}\) into triangles \(\mathbf {a}\mathbf {b}\mathbf {e}\), \(\mathbf {b}\mathbf {c}\mathbf {e}\), \(\mathbf {c}\mathbf {a}\mathbf {e}\) where \(\mathbf {e}= (\mathbf {a}+\mathbf {b}+\mathbf {c})/3\), and reset \(\varphi (\mathbf {d})= \mathbf {e}\). We obtain \(\varphi (\mathbf {d}\mathbf {a}\mathbf {b}\cup \mathbf {d}\mathbf {b}\mathbf {c}\cup \mathbf {d}\mathbf {c}\mathbf {a})= \mathbf {abc}\). The three cases \(f\in \{ 1,2,3\}\) are shown in Fig. 3.
Fig. 3

Triangle splitting for Lemma 3 in cases \(f=1\) (left), \(f=2\) (middle) and \(f=3\) (right). There are \(\partial O\cap \partial \Delta \) on the top and \(\partial \Delta {\setminus }\partial O\) on the bottom (\(\varphi \) maps the former to the latter)

Second we check that every triangle \(t\in \partial O{\setminus }\partial \Delta \) and \(\varphi _{|t}\) (\(\varphi \) restricted to t) are unchanged by the modifications above. If \(t\cap \Delta =\emptyset \), both t and \(\varphi _{|t}\) are unchanged. Now we assume \(t\cap \Delta \ne \emptyset \). Since c(T) is a simplicial complex, \(t\cap \Delta \) is a vertex or an edge of \(\Delta \). We consider several cases. If \(f=1\), \(t\ne \mathbf {abc}\) and \(\mathbf {e}\) is not a vertex of t. Thus both t and \(\varphi _{|t}\) are unchanged. Assume (reductio ad absurdum) that \(f=2\) and \(\mathbf {bc}\subset t\). Thus \(\mathbf {bc}\) is a face of three distinct triangles t, \(\mathbf {abc}\) and \(\mathbf {bcd}\) in \(\partial O\), although \(\partial O\) is a 2-manifold (impossible). Assume (reductio ad absurdum) that \(f=2\) and \(\mathbf {ad}\subset t\). Thus \(\mathbf {ad}\in c(O)\cap c(\Delta )\) although triangles \(\mathbf {adb}\) and \(\mathbf {adc}\) are not in \(c(O)\cap c(\Delta )\), i.e. \(c(O)\cap c(\Delta )\) is not 2D pure (impossible). Therefore both t and \(\varphi _{|t}\) are unchanged if \(f=2\). Assume (reductio ad absurdum) that \(f=3\) and \(\mathbf {d}\in t\). Thus \(\{ t\}\cup \mathbf {d}\times \mathbf {a}\hbox {-}\mathbf {b}\hbox {-} \mathbf {c}\hbox {-}\mathbf {a}\subseteq (\partial O)_\mathbf {d}\) and \(t\notin \mathbf {d}\times \mathbf {a}\hbox {-}\mathbf {b}\hbox {-}\mathbf {c}\hbox {-}\mathbf {a}\), i.e. \(\mathbf {d}\) is a singular vertex of \(\partial O\) (impossible). Therefore both t and \(\varphi _{|t}\) are unchanged if \(f=3\).

Third we show that \(\varphi \) is surjective. Thanks to Lemma 2 and since \(\varphi (|\partial O\cap \partial \Delta |)=|\partial \Delta {\setminus }\partial O|\) and \(\varphi \) is the identity in \(|\partial O{\setminus }\partial \Delta |\),
$$\begin{aligned} \varphi (|\partial O|)= & {} \varphi (|\partial O\cap \partial \Delta | \cup |\partial O{\setminus }\partial \Delta |) \end{aligned}$$
(1)
$$\begin{aligned}= & {} |\partial \Delta {\setminus }\partial O|\cup |\partial O{\setminus }\partial \Delta | \end{aligned}$$
(2)
$$\begin{aligned}= & {} |\partial (O\cup \{\Delta \})|. \end{aligned}$$
(3)
Fourth we show that \(\varphi \) is a homeomorphism. We only need to check that \(\varphi \) maps distinct vertices to distinct vertices [19]. This is true if \(f=2\) and \(f=3\). Assume (reductio ad absurdum) that \(f=1\) and \(\mathbf {d}\in c(\partial O)\). Thus \(\mathbf {d}\in c(O)\cap c(\Delta )\). Since triangles \(\mathbf {dab}\), \(\mathbf {dbc}\) and \(\mathbf {dca}\) are not in \(c(O)\cap c(\Delta )\), \(c(O)\cap c(\Delta )\) is not 2D pure (impossible). If \(f=1\), \(\mathbf {d}=\varphi (\mathbf {e})\) is different to all vertices in \(c(\partial O)\) and we obtain the result. \(\square \)

Note that Appendix B of [14] provides a first version of the proof of Lemma 3 without details on steps 2 and 4.

Theorem 4

In a shelling, all \(\partial O_i\) are 2-manifolds with the same genus. If \(c(O_i)\cap c(\Delta _i)\) is a 2-ball, \(\partial O_{i+1}\) and \(\partial O_i\) have the same number of connected components. If \(c(O_i)\cap c(\Delta _i)\) is a 2-sphere, \(\partial O_i\) has one connected component more than \(\partial O_{i+1}\) (Fig. 4).

Fig. 4

Theorem 4 in the 2D case (O is gray). Left O encloses a cavity defined by two white triangles. Middle add a triangle to O such that \(c(O)\cap c(\Delta )\) is a ball. Right add a triangle to O such that \(c(O)\cap c(\Delta )\) is a sphere. The number of the connected component(s) of \(\partial O\) is two (left and middle) or one (right)

Proof

Every \(\partial O_i\) is a 2-manifold thanks to Theorem 1 and since \(\partial O_0\) is a 2-manifold. Let f be the number of the triangles in \(\partial O_i\cap \partial \Delta _i\). If \(f\in \{1,2,3\}\), \(\partial O_{i+1}\) and \(\partial O_i\) are homeomorphic thanks to Lemma 3. Therefore, \(\partial O_{i+1}\) and \(\partial O_i\) have the same genus and the same number of connected component(s).

Now we study case \(f=4\). We have \(\partial \Delta _i\subseteq \partial O_i\) and Lemma 2 implies \(\partial O_{i+1}= \partial O_i{\setminus }\partial \Delta _i\). Therefore \(\partial O_i=\partial O_{i+1}\cup \partial \Delta _i\) and \(\partial O_{i+1}\cap \partial \Delta _i= \emptyset \). Assume (reductio ad absurdum) that \(\partial \Delta _i\) is not a connected component of \(\partial O_i\). Thus these is an edge \(\mathbf {vv}'\in c(\partial O_i)\) such that \(\mathbf {v}\in c(\partial \Delta _i)\) and \(\mathbf {v}'\notin c(\partial \Delta _i)\). This implies that there is a triangle in \(\partial O_i{\setminus }\partial \Delta _i\) that includes \(\mathbf {vv}'\). Therefore \(\mathbf {v}\in c(\partial \Delta _i)\cap c(\partial O_{i+1})\). Since \(\partial O_{i+1}\) is a 2-manifold, there is a cycle \(\mathbf {a}_*\) such that \((\partial O_{i+1})_\mathbf {v}= \mathbf {v}\times \mathbf {a}_*\). Thus \((\mathbf {v}\times \mathbf {a}_*)\cup (\partial \Delta _i)_\mathbf {v}\subseteq (\partial O_i)_\mathbf {v}\) and \((\partial \Delta _i)_\mathbf {v}{\setminus } \mathbf {v}\times \mathbf {a}_*\ne \emptyset \), which imply that \(\mathbf {v}\) is a singular vertex of \(\partial O_i\) (impossible). Since \(\partial \Delta _i\) is a connected component of \(\partial O_i\) and \(\partial O_i=\partial O_{i+1}\cup \partial \Delta _i\), \(\partial O_i\) has one connected component more than \(\partial O_{i+1}\). Last \(\partial O_{i+1}\) and \(\partial O_i\) have the same genus since \(\partial \Delta _i\) has zero genus. \(\square \)

In most cases, \(c(O_i)\cap c(\Delta _i)\) is a 2-ball. In the few cases where \(\Delta _i\) is a cavity of \(O_i\) (\(O_0\) can be like a piece of Swiss cheese that has cavities as in Fig. 4), \(c(O_i)\cap c(\Delta _i)\) is a 2-sphere.

Corollary 2 describes a frequent case where shelling does not change the topology of \(\partial O_i\). We need

Lemma 4

If \(\emptyset \ne O\subsetneq T^\infty \), \(\partial O\ne \emptyset \).

The proof of Lemma 4 is straightforward (\(T^\infty \) is strongly connected) and is in the supplementary material.

Corollary 2

Assume that \(O_i\subset T^\infty \) and \(\partial O_i\) is a 2-sphere in a shelling. If \(i\le j< n\) and \(O_{j+1}\ne T^\infty \), \(c(O_j)\cap c(\Delta _j)\) is a 2-ball and \(\partial O_{j+1}\) is a 2-sphere.

Proof

We assume (induction) that \(\partial O_j\) is a 2-sphere (this is OK if \(j=i\)) and show that \(c(O_j)\cap c(\Delta _j)\) is a 2-ball and \(\partial O_{j+1}\) is a 2-sphere if \(O_{j+1}\ne T^\infty \).

Assume (reductio ad absurdum) that \(c(O_j)\cap c(\Delta _j)\) is a 2-sphere. Since \(\partial O_j\) has one connected component, Theorem 4 implies that \(\partial O_{j+1}\) does not have a connected component, i.e. \(\partial O_{j+1}= \emptyset \). Since \(\emptyset \ne O_{j+1}\subsetneq T^\infty \), this contradicts Lemma 4.

Thus \(c(O_j)\cap c(\Delta _j)\) is a 2-ball (Corollary 1) and \(\partial O_{j+1}\) is a 2-manifold with the same genus and the same number of connected component than \(\partial O_j\) (Theorem 4), i.e. \(\partial O_{j+1}\) is a 2-sphere. \(\square \)

4 Overview of 3-Ball Shellability

This section summarizes previous works in Combinatorial Topology. Here we would like to know if \(O\subseteq T\) is shellable, i.e. if O can be the end value of a shelling started from a single tetrahedron. First Sect. 4.1 describes and compares several kinds of shellability and non-shellability. Second Sect. 4.2 presents invariant numbers and a necessary condition for the O shellability. Last Sects. 4.3 and 4.4 list positive and negative cases of shellability.

4.1 Definitions

According to Theorem 5 (supplementary material),

Theorem 5

Let \(O\subseteq T\). Then O is a 3-ball iff \(\partial O\) is a 2-sphere.

assertions “O is a 3-ball” and “\(\partial O\) is a 2-sphere” are equivalent in our context. Although the introduction of 3-balls seems useless, we mention it (and use it) since this formulation is often used in the bibliography.

A tetrahedron set \(O\subseteq T\) is shellable if there is a tetrahedron \(\Delta \in T\) and a shelling \(O_0\cdots O_n\) such that \(O_0=\{\Delta \}\) and \(O=O_n\). We also say that \(O_0\cdots O_n\) is a shelling of O. Thanks to Corollary 2, we note that all \(\partial O_i\) are 2-spheres and all \(c(O_i)\cap c(\Delta _i)\) are 2-balls. Thus O is a 3-ball.

A 3-ball \(O\subseteq T\) is extendably shellable if for every shellable \(O'\subset O\), there is a shelling \(O_0\cdots O_k\) such that \(O_0= O'\) and \(O_k= O\). In other words, every partial shelling of O can be completed to reach O. Therefore O is shellable if O is extendably shellable.

Extendable shellability is a convenient case of shellability. If O is extendably shellable and if a greedy algorithm constructs a shelling of O by choosing successive tetrahedra in O (e.g. Algorithm 1 in [13] using \(F=O\)), the algorithm always succeeds, i.e. its final value is always \(O_n= O\). If O is shellable but non-extendably shellable, the greedy algorithm can choose tetrahedra at the shelling beginning such that it “gets stuck.” This means that there is \(O_i\subsetneq O\) in the shelling computation such that \(c(O_i)\cap c(\Delta )\) is never non-empty 2D-pure for every tetrahedron \(\Delta \) tried by the algorithm (thus the algorithm stops). Thanks to Theorem 1, this means that \(\partial (O_i\cup \{\Delta \})\) is not a 2-manifold (i.e. \(\partial (O_i\cup \{\Delta \})\) has a singular vertex) or \(c(O_i)\cap c(\Delta )=\emptyset \).

A 3-ball \(O\subseteq T\) is strongly non-shellable if O has at least two tetrahedra and if for every tetrahedron \(\Delta \in O\), \(c(\Delta )\cap c(\partial O)\) is not a 2-ball. Here we note that \(\Delta \in O\) in contrast to the shelling definition where \(\Delta _i\notin O_i\). According to Proposition 2.4.iv in [24], O is non-shellable if O is strongly non-shellable (principle of the proof: if \(O_0\cdots O_n\) is a shelling of O, we have \(O= O_{n-1}\cup \{\Delta \}\), and see that \(c(\partial O)\cap c(\Delta )\) is a 2-ball as the complement of 2-ball \(c(\partial O_{n-1})\cap c(\Delta )\) in 2-sphere \(c(\partial \Delta )\)).

Strongly non-shellability is an easy case of non-shellability for several reasons. First it is not difficult to check that a given O is strongly non-shellable. In comparison, it is quite more difficult to check that O is non-shellable: we don’t know whether a polynomial-time algorithm can do that [8]. Second, a strongly non-shellable O can be build with a few tetrahedra. Third, non-shellability and strongly non-shellability are related by Proposition 2.4.v in [24]: if O is non-shellable, there is a strongly non-shellable 3-ball \(O'\subseteq O\) (principle of the proof: construct a shelling of O in the reverse order if every 3-ball \(O'\subseteq O\) is not strongly non-shellable). More details about the relations between strongly non-shellability and non-shellability are in the supplementary material.

4.2 h-Numbers of O

The result described in this section is known for every dimension (Theorem 8.19 in [25], see also Proposition 7.7 in [9]). Here we present a simpler (specialized) proof in our 3D case.

Let O be a 3-ball in T and \(f_0\) (respectively, \(f_1\), \(f_2\) and \(f_3\)) be the number of the vertices (respectively, edges, triangles and tetrahedra) in c(O). According to [9, 25], the definition of h-numbers \(h_1\), \(h_2\) and \(h_3\) of O is
$$\begin{aligned} \begin{pmatrix} 1\\ h_1\\ h_2\\ h_3\end{pmatrix}= \mathtt {H} \begin{pmatrix} 1\\ f_0\\ f_1\\ f_2\end{pmatrix}\hbox { where } \mathtt {H}=\begin{pmatrix} +1 &{} 0 &{} 0 &{} 0\\ -4 &{} +1 &{} 0 &{} 0\\ +6 &{} -3 &{} +1 &{} 0 \\ -4 &{} +3 &{} -2 &{} +1\\ \end{pmatrix}. \end{aligned}$$
(4)
If O is shellable, we classify the shelling steps in three cases and all steps in a case update the numbers of the simplices by the same way. Thanks to this, Theorem 6 shows that the number of the steps in the j-th case is a function of the \(f_i\)s, more precisely it is \(h_j\). The number of the steps in the j-th case is invariant to the shelling choice since \(h_j\) does not depend on it.

Theorem 6

If O is shellable, every shelling of O has \(h_j\) step(s) \(O_{i+1}= O_i\cup \{\Delta _i \}\) such that the number of the triangles in \(\partial O_i\cap \partial \Delta _i\) is equal to j.

Proof

If \(X\subseteq c(T)\), we define the vector \(f(X)\in \mathbb {N}^4\) whose k-th coordinate is the number of \(k-1\)-dimensional simplices in c(X). Let \(O_0\cdots O_n\) be a shelling of O and the tetrahedron \(\Delta _i\in T{\setminus } O_i\) such that \(O_{i+1}= O_i\cup \{ \Delta _i \}\). We will calculate \(\delta _i= f(c(O_{i+1}))-f(c(O_i))\).

Since \(c(O_{i+1})= c(O_i)\cup (c(\Delta _i){\setminus } c(O_i))\), \(\delta _i= f(c(\Delta _i){\setminus } c(O_i))\). Let \(n_i\) be the number of the triangles in \(\partial O_i\cap \partial \Delta _i\) and \(\mathbf {abcd}= \Delta _i\). Since \(c(O_i)\cap c(\Delta _i)\) is a 2-ball, there are three cases according to Corollary 1.

If \(n_i=1\), \(c(O_i)\cap c(\Delta _i)= c(\mathbf {abc})\). Thus \(c(\Delta _i){\setminus } c(O_i)= \{ \mathbf {d}, \mathbf {da}, \mathbf {db}, \mathbf {dc}, \mathbf {dab}, \mathbf {dbc}, \mathbf {dca}, \mathbf {abcd}\}\). We obtain \(\delta _i= \begin{pmatrix} 1&3&3&1\end{pmatrix}^T\).

If \(n_i=2\), \(c(O_i)\cap c(\Delta _i)= c(\mathbf {abc}, \mathbf {bcd})\). Thus \(c(\Delta _i){\setminus } c(O_i)= \{ \mathbf {ad}, \mathbf {adb}, \mathbf {adc}, \mathbf {abcd} \}\). We obtain \(\delta _i= \begin{pmatrix} 0&1&2&1\end{pmatrix}^T\).

If \(n_i=3\), \(c(O_i)\cap c(\Delta _i)=c(\mathbf {abc},\mathbf {acd},\mathbf {adb})\). Thus \(c(\Delta _i){\setminus } c(O_i)= \{ \mathbf {bcd}, \mathbf {abcd} \}\). We obtain \(\delta _i= \begin{pmatrix} 0&0&1&1\end{pmatrix}^T\).

Let \(j\in \{ 1,2,3\}\) and \(h'_j\) be the number of the step(s) \(O_{i+1}= O_i\cup \{ \Delta _i \}\) such that \(n_i= j\). Let
$$\begin{aligned} \mathtt {A}= \begin{pmatrix} 4 &{} 1 &{} 0 &{} 0 \\ 6 &{} 3 &{} 1 &{} 0 \\ 4 &{} 3 &{} 2 &{} 1 \\ 1 &{} 1 &{} 1 &{} 1 \end{pmatrix}\hbox { and } \mathtt {B}= \begin{pmatrix} 0 &{} 1 &{} 0 &{} 0\\ 0 &{} 0 &{} 1 &{} 0\\ 0 &{} 0 &{} 0 &{} 1\\ -1 &{} 1 &{} -1 &{} 1 \end{pmatrix}. \end{aligned}$$
(5)
Since \(O_0\) has one tetrahedron, \(f(c(O_0))=\begin{pmatrix} 4&6&4&1 \end{pmatrix}^T\). Now we have
$$\begin{aligned} f(c(O))= f(c(O_0))+\sum _{i=0}^{n-1}\delta _i= \mathtt {A}\begin{pmatrix} 1&h'_1&h'_2&h'_3\end{pmatrix}^\top . \end{aligned}$$
(6)
Furthermore, Eqs. 6 and 5 imply
$$\begin{aligned} f_0-f_1+f_2-f_3= \begin{pmatrix} 1&-1&1&-1\end{pmatrix}f(c(O))= 1, \end{aligned}$$
(7)
which in turn implies
$$\begin{aligned} f(c(O))= \begin{pmatrix} f_0&f_1&f_2&f_3\end{pmatrix}^\top = \mathtt {B}\begin{pmatrix} 1&f_0&f_1&f_2\end{pmatrix}^\top . \end{aligned}$$
(8)
Thanks to Eqs. 6 and 8 and since \(\mathtt {H}=\mathtt {A}^{-1}\mathtt {B}\), we obtain
$$\begin{aligned} \begin{pmatrix} 1&h'_1&h'_2&h'_3\end{pmatrix}^\top = \mathtt {H}\begin{pmatrix} 1&f_0&f_1&f_2 \end{pmatrix}^\top . \end{aligned}$$
(9)
Therefore \(h'_j=h_j\) for \(j\in \{ 1,2,3\}\). \(\square \)

We also see that a necessary condition for shellability is \(h_1\ge 0\) and \(h_2\ge 0\) and \(h_3\ge 0\) (O is non-shellable if there is j such that \(h_j< 0\)). Furthermore, the \(h_j\) are consistent even if O is non-shellable:

Property 1

If \(O\subseteq T\) is a 3-ball, \(1+h_1+h_2+h_3=f_3\).

Proof

According to Eq. 4,
$$\begin{aligned} 1+h_1+h_2+h_3= & {} \begin{pmatrix} 1&1&1&1\end{pmatrix}\mathtt {H} \begin{pmatrix} 1&f_0&f_1&f_2 \end{pmatrix}^\top \end{aligned}$$
(10)
$$\begin{aligned}= & {} -1+f_0-f_1+f_2. \end{aligned}$$
(11)
Last we obtain the result thanks to Euler’s relation for 3-balls [9], i.e. \(f_0-f_1+f_2-f_3= 1\). \(\square \)

4.3 Positive Cases

First, all 3-balls with less than 9 vertices are extendably shellable and almost all 3-balls with 9 vertices are shellable [17] (there are only 29 non-shellable 3-balls over the 2,451,305 combinatorial 3-balls with 9 vertices). Second, every 3D Delaunay T is a shellable 3-ball [3, 24]. Thus every \(O\subseteq T\) such that |O| is convex is shellable. Indeed, such an O is the set of all tetrahedra of the 3D Delaunay triangulation defined by using only the vertices in c(O). Third Sect. 5 shows that \(O\subseteq T\) is shellable if |O| is star-shaped. This generalizes the convex case above, since a convex set is star-shaped with respect to every point in the convex set.

4.4 Negative Cases

There is a survey [24] of non-shellable 3-balls in a more general context than ours: some of them are constructed using a polytopal complex instead of a simplicial complex, i.e. the tetrahedra are replaced by convex hulls with \(n\ge 4\) vertices likes cubes. Since we focus on 3-balls included in a 3D Delaunay triangulation, they should be converted to analog examples in a 3D Delaunay. There are two kinds of examples: strongly non-shellable 3-balls defined by a small (ideally minimal) set of simplices, and non-shellable 3-balls defined by an important (not explicit) set of simplices and a global property. In the former, there is a 3-ball that has only 10 vertices and 21 tetrahedra (or 12 vertices and 25 tetrahedra). In the latter, there is a 3-ball called “Knotted hole ball” (K); K is non-shellable thanks to a property based on the knot theory: there is a knotted curve (a cycle of edges that cannot be included in a 2-sphere) with all edges on the boundary of K except one edge that is in K.

There are other negative cases. A 3D Delaunay triangulation T can be non-extendably shellable [24]. Thus a greedy shelling algorithm can fail and miss O if \(O\subseteq T\) and |O| is convex. Last Sect. 6 provides a family of strongly non-shellable 3-balls that are not difficult to visualize in 3D. It includes the 3-ball with 12 vertices and 25 tetrahedra in [24], which is in a 3D Delaunay triangulation.

5 Star-Shape Shelling

In Sect. 5, there are tetrahedron sets \(O,O'\) and a point \(\mathbf {c}\in |O|\) such that \(O\subsetneq O'\subseteq T\) and both \(O'\) and O are \(\mathbf {c}\)-star-shaped. We show that there is a shelling \(O_0\cdots O_n\) such that \(O_0= O\) and \(O_n= O'\). Such a result is suggested in [7] if \(O'=T\) and O has a single tetrahedron. Here we need a non-degenerate case: \(\mathbf {c}\) is not in a plane including a triangle of c(T).

5.1 Summary

The proof has three main steps. First Sect. 5.2 provides useful lemmas and a star-shape criterion for a set of tetrahedra included in T (Theorem 7). Second Sect. 5.3 describes a visibility relation between two tetrahedra in T with respect to a viewpoint \(\mathbf {c}\): \(\Delta \) is behind \(\Delta '\). Then Sect. 5.3 shows that there is a “front” tetrahedron in \(O'{\setminus } O\) for this relation (Theorem 8), i.e. this tetrahedron is not behind another one in \(O'{\setminus } O\). Third Sect. 5.4 shows that the front tetrahedron is added to O such that we obtain a shelling step (Theorem 9) thanks to Theorems 7 and 8 and lemmas. Then Sect. 5.4 shows the existence of a shelling between O and \(O'\) (Corollary 3).

Last Sect. 5.5 presents a qualitative argument based on Corollary 3 to explain why the first shelling in [13] does not have excessive blocking.

5.2 Prerequisites

We give definitions before lemmas and theorems. If \(\mathbf {a}\) and \(\mathbf {b}\) are distinct points in \(\mathbb {R}^3\), we distinguish the line segment \(\mathbf {ab}\) and the full line \((\mathbf {ab})\) including \(\mathbf {a}\) and \(\mathbf {b}\):
$$\begin{aligned} \mathbf {ab}= & {} \{ (1-\lambda )\mathbf {a}+\lambda \mathbf {b},0\le \lambda \le 1\}\hbox { and } \end{aligned}$$
(12)
$$\begin{aligned} (\mathbf {ab})= & {} \{ (1-\lambda )\mathbf {a}+\lambda \mathbf {b}, \lambda \in \mathbb {R} \}. \end{aligned}$$
(13)
A point \(\mathbf {d}\) is between \(\mathbf {a}\) and \(\mathbf {b}\) if \(\mathbf {d}\in \mathbf {ab}{\setminus }\{ \mathbf {a}, \mathbf {b}\}\).
A set \(X\subset \mathbb {R}^3\) is star-shaped with respect to \(\mathbf {c}\) (or X is \(\mathbf {c}\)-star-shaped) if \(\mathbf {x}\in X\) implies that \(\mathbf {xc}\subseteq X\). We say that \(O\subseteq T\) is \(\mathbf {c}\)-star-shaped if |O| is \(\mathbf {c}\)-star-shaped. If t is a triangle in \(\mathbb {R}^3\), the half-spaces of t are the two half-spaces of \(\mathbb {R}^3\) separated by the plane including t (each half-space includes the plane itself). Note that our non-degenerate case implies that \(\mathbf {c}\) is in exactly one half-space of every triangle in c(T).
Fig. 5

A star-shape shelling example in the 2D case: O is a growing set of triangles such that \(\partial O\) is a cycle (bold edges) and O is always star-shaped with respect to the black dot

Figure 5 shows a star-shape shelling in the 2D case (replace tetrahedron by triangle etc, as in Fig. 1).

We also use other notations in Sect. 5. Let \(\sigma \) be a simplex in \(\mathbb {R}^3\) and \(\mathring{\sigma }=\sigma {\setminus } |\partial \sigma |\), i.e. \(\mathring{\sigma }\) is the interior of \(\sigma \). Let \(B(\mathbf {x},\epsilon )\) be the 3D ball centered at \(\mathbf {x}\in \mathbb {R}^3\) with the radius \(\epsilon > 0\). If \(\sigma \) is a tetrahedron in \(\mathbb {R}^3\), \(\mathring{\sigma }\) is an open set in \(\mathbb {R}^3\), i.e. every point \(\mathbf {x}\in \mathring{\sigma }\) is such that there is \(\epsilon > 0\) and \(B(\mathbf {x},\epsilon )\subseteq \mathring{\sigma }\).

We need the three following lemmas to show Theorem 7 (the star-shape criterion) and others. The first two help to escape from degenerate configurations. The third one is a ray-tracing lemma, which provides a series of adjacent tetrahedra in T covering a line segment between two points in |T|.

Lemma 5

If a tetrahedron \(\Delta \in T\) and a point \(\mathbf {x}\in \mathring{\Delta }\), \(\Delta \) is the only tetrahedron in T that contains \(\mathbf {x}\), and \(\mathbf {x}\) is not in a triangle of c(T). If a triangle \(t\in c(T)\) and \(\mathbf {x}\in \mathring{t}\), t is the only triangle in c(T) that contains \(\mathbf {x}\).

The proof of Lemma 5 is straightforward (c(T) is a simplicial complex) and is in the supplementary material.

Lemma 6

Let \(A\subseteq T\) and points \(\mathbf {c}\in |A|\), \(\mathbf {x}\in |A|{\setminus }\mathbf {c}\), \(\mathbf {y}\in \mathbf {cx}\) such that \(\mathbf {y}\notin |A|\). Let \(\Delta _x\) be a tetrahedron in A such that \(\mathbf {x}\in \Delta _x\). There is a point \(\tilde{\mathbf {x}}\in \mathring{\Delta }_x\) and a tetrahedron \(\Delta _y\in T{\setminus } A\) and a point \(\tilde{\mathbf {y}}\in \mathring{\Delta }_y\cap \mathbf {c}\tilde{\mathbf {x}}\) such that \((\mathbf {c}\tilde{\mathbf {x}})\) does not intersect the edges in c(T) (Fig. 6).

Fig. 6

Lemma 6 in the 2D case. Left A is enclosed by the bold edges, \(\mathbf {c}\in |A|\), \(\mathbf {x}\in \Delta _x\in A\), \(\Delta _x\) is gray, \((\mathbf {cx})\) intersects a vertex of c(T) (an edge in 3D), \(\mathbf {y}\in \mathbf {cx}{\setminus } |A|\). Right replace \(\mathbf {x}\) by \(\tilde{\mathbf {x}}\in \mathring{\Delta }_x\) such that \((\mathbf {c}\tilde{\mathbf {x}})\) does not intersect the c(T) vertices, replace \(\mathbf {y}\) by \(\tilde{\mathbf {y}}\in \mathring{\Delta }_y\cap \mathbf {c}\tilde{\mathbf {x}}\) such that \(\Delta _y\in T{\setminus } A\), \(\Delta _y\) is gray

Lemma 7

Let \(\Delta _x\) be a tetrahedron in T and points \(\mathbf {x}\in \mathring{\Delta }_x\), \(\mathbf {y}\in T{\setminus }\{\mathbf {x}\}\) such that \((\mathbf {xy})\) does not intersect the edges in c(T). Let \(\mathbf {x}_\lambda = (1-\lambda )\mathbf {x}+\lambda \mathbf {y}\) if \(\lambda \in \mathbb {R}\). There are integer \(k\ge 1\), reals \(\lambda _1\cdots \lambda _{k+1}\), tetrahedra \(\Delta _1\cdots \Delta _k\) in T such that \(\lambda _i< \lambda _{i+1}\) if \(1\le i\le k\), \(\mathbf {x}_\lambda \in \Delta _i\) iff \(\lambda \in [\lambda _i,\lambda _{i+1}]\), every \(\Delta _i\cap \Delta _{i+1}\) is a triangle. If \(\Delta \in T\) and \((\mathbf {xy})\cap \Delta \ne \emptyset \), \(\Delta \) is a \(\Delta _i\) and \((\mathbf {xy})\cap \mathring{\Delta }\ne \emptyset \) (Fig. 7).

Fig. 7

Lemma 7 in the 2D case. Left \((\mathbf {xy})\) successively intersects \(\Delta _1\cdots \Delta _6\) in T, every \(\Delta _i\cap \Delta _{i+1}\) is an edge (a triangle in 3D). Right \(\mathbf {x}_\lambda \) maps \([\lambda _i,\lambda _{i+1}]\) in \(\Delta _i\)

The Proofs of Lemmas 6 and 7 are in “Appendix 1”). Theorem 7 converts the star-shape definition to a more tractable condition in our proof.

Theorem 7

Let \(O\subseteq T\) and a point \(\mathbf {c}\in |O|\). Then O is \(\mathbf {c}\)-star-shaped iff every triangle \(t\in \partial O\) is a face of a tetrahedron \(\Delta \in O\) such that \(\Delta \) and \(\mathbf {c}\) are in the same t half-space (Fig. 8).

This is similar to star-shape tests in [20, 22], which use outward-pointing normals at points on a volume boundary and a scalar product instead of our condition based on simplices. The proof is summarized in Fig. 8 and detailed in “Proof of Theorem 7” of “Appendix 1.”
Fig. 8

Theorem 7 in the 2D case (tetrahedra are drawn as triangles). The streaked triangle is \(\Delta \) and the bold edge is t. If O is \(\mathbf {c}\)-star-shaped (left), \(\Delta \) and \(\mathbf {c}\) are in the same t half-space. If O is not \(\mathbf {c}\)-star-shaped (right), there is \(\mathbf {x}\in |O|\) such that we have not \(\mathbf {xc}\subset |O|\). Thus there are \(\Delta \) and t such that \(\Delta \) and \(\mathbf {c}\) are not in the same t half-space

5.3 Visibility in a 3D Delaunay Triangulation

Let tetrahedra \(\Delta \in T\) and \(\Delta '\in T\). Let l be a half-line starting at \(\mathbf {c}\). We say that \(\Delta \) is behind [7] \(\Delta '\) if there is l such that \(l\cap \mathring{\Delta }\ne \emptyset \) and \(l\cap \mathring{\Delta }'\ne \emptyset \) and \(\Delta \ne \Delta '\) and every point in \((l{\setminus }\{\mathbf {c}\})\cap \mathring{\Delta }'\) is between \(\mathbf {c}\) and every point of \((l{\setminus }\{\mathbf {c}\})\cap \mathring{\Delta }\). Figure 9 shows \(\mathbf {c}\), l, \(\Delta \) and \(\Delta '\).
Fig. 9

Notations for Lemma 8 in the 2D case (tetrahedra are drawn as triangles). The two streaked triangles are \(\Delta \) and \(\Delta '\), the triangle with dotted edges is in O. Here \(\Delta \) is behind \(\Delta '\)

The goal of Sect. 5.3 is Theorem 8: the existence of a “front” tetrahedron in \(O'{\setminus } O\), i.e. a tetrahedron which is not behind another one in \(O'{\setminus } O\). The proof is based on the fact that the relation “is behind” is acyclic in the 3D Delaunay triangulation T [7]. It also needs the two following lemmas to generate “behind” relations.

Lemma 8

Assume that \(O'\) is \(\mathbf {c}\)-star-shaped and \(\Delta \in O'{\setminus } O\) has a triangle face \(t\notin \partial O\) such that \(\Delta \) and \(\mathbf {c}\) are not in the same t half-space. There is \(\Delta '\in O'{\setminus } O\) such that \(\partial \Delta '\cap \partial O\ne \emptyset \) and \(\Delta \) is behind \(\Delta '\) (Fig. 9).

A sketch proof is provided before the proof for intuition. Let \(\mathbf {y}\) be a point in \(\Delta \). Since \(O'\) is \(\mathbf {c}\)-star-shaped, \(\mathbf {y}+\lambda (\mathbf {c}-\mathbf {y})\) successively intersects tetrahedra \(\Delta _0, \Delta _1\cdots \Delta _k\) in \(O'\) when \(\lambda \) increases from 0 to 1, and every \(\Delta _i\cap \Delta _{i+1}\) is not empty. We choose \(\mathbf {y}\) such that \(\Delta _i\cap \Delta _{i+1}\) is a triangle for every i (\(\mathbf {cy}\) does not intersect the edges in c(T)). Since \(\mathbf {y}\in \Delta =\Delta _0\in O'{\setminus } O\) and \(\mathbf {c}\in \Delta _k\in O\), there is i such that \(\Delta _i\in O'{\setminus } O\) and \(\Delta _{i+1}\in O\). We obtain \(\Delta '= \Delta _i\) (we have \(\Delta \ne \Delta '\) if \(\partial \Delta \cap \partial O=\emptyset \)).

Proof

First we set \(A= O\cup \{\Delta \}\) and find points \(\mathbf {x}'\) and \(\mathbf {y}'\) that meet the assumptions of Lemma 6. Let \(\mathbf {x}'\) be the barycentre of t and \(\Delta _x= \Delta \in A\). If \(t\in \partial T\) (reductio ad absurdum), both \(\Delta \) and \(\mathbf {c}\) are in the same t half-space since T is convex (impossible). Thus there is a tetrahedron \(\Delta ''\in T{\setminus }\{\Delta \}\) such that \(t= \Delta \cap \Delta ''\). Since \(t\notin \partial O\) and \(\Delta \notin O\), \(\Delta ''\in T{\setminus } O\) and thus \(\Delta ''\in T{\setminus } A\). Since \(\mathbf {c}\) and \(\Delta \) are in different half-spaces of t, there is a point \(\mathbf {y}'\in \mathbf {cx}'\cap \mathring{\Delta }''\). Since \(\mathbf {y}'\) cannot be in a tetrahedron of A (Lemma 5), \(\mathbf {y}'\notin |A|\).

Second we find a tetrahedron series covering \(\mathbf {xc}\) (\(\mathbf {x}\in \Delta \)) using Lemma 7. According to Lemma 6, there are a tetrahedron \(\Delta _y\in T{\setminus } A\) and points \(\mathbf {x}\in \mathring{\Delta }_x\) and \(\mathbf {y}\in \mathring{\Delta }_y\cap \mathbf {cx}\) such that \((\mathbf {cx})\) does not intersect the edges in c(T). Now we have a tetrahedron series \(\Delta _i\in T\) that covers \(|T|\cap (\mathbf {xy})\) as defined by Lemma 7. Since \(\mathbf {xc}\subseteq |T|\) (indeed |T| is convex and include both \(\mathbf {x}\) and \(\mathbf {c}\)) and \((\mathbf {xc})= (\mathbf {xy})\), the tetrahedron series also covers \(\mathbf {xc}\).

Third we find distinct \(\Delta _i\) that include \(\mathbf {x}\), \(\mathbf {y}\) and \(\mathbf {c}\). There are j, n and l such that \(\Delta _j=\Delta _x\), \(\Delta _n=\Delta _y\) and \(\mathbf {c}\in \Delta _l\in O\) (since \(\mathbf {c}\in |O|\)). Since \(\mathbf {y}\in \mathbf {x}\mathbf {c}\) and \(\mathbf {x}= \mathbf {x}_0\) and \(\mathbf {y}= \mathbf {x}_1\), we have \(j\le n\le l\). Since \(\Delta _j\in A\) and \(\Delta _n\notin A\) and \(\Delta _l\in A\), we have \(j< n < l\).

Fourth we show that \(j\le m\le l\) implies \(\Delta _m\in O'\). Since \(O'\) is \(\mathbf {c}\)-star-shaped and both \(\mathbf {x}\) and \(\mathbf {c}\) are in \(|O'|\), \(\mathbf {cx}\in |O'|\). Thus \(\mathring{\Delta }_m\cap |O'|\ne \emptyset \) (Lemma 7) and \(\Delta _m\in O'\) (Lemma 5).

Last we conclude. Since \(\Delta _n\in O'{\setminus } O\) and \(\Delta _l\in O\), there is m such that \(\Delta _m\in O'{\setminus } O\) and \(\Delta _{m+1}\in O\) and \(n\le m< l\). Let \(\Delta '= \Delta _m\). The triangle \(\Delta _m\cap \Delta _{m+1}\) is in \(\partial \Delta '\cap \partial O\). Note that \(\mathbf {x}_\lambda \in \Delta \) iff \(\lambda \in [\lambda _j,\lambda _{j+1}]\), \(\mathbf {x}_\lambda \in \Delta '\) iff \(\lambda \in [\lambda _m,\lambda _{m+1}]\), \(\lambda _{j+1}\le \lambda _m\) (since \(j<n\le m\)), \(\mathbf {c}= \mathbf {x}_\mu \) with \(\mu \in [\lambda _l,\lambda _{l+1}]\) and \(\lambda _{m+1}\le \lambda _l\) (since \(m< l\)). Thus \(\Delta \) is behind \(\Delta '\). \(\square \)

Lemma 9

If \(O'\) is \(\mathbf {c}\)-star-shaped and \(O'{\setminus } O\ne \emptyset \), there is a tetrahedron \(\Delta '\in O'{\setminus } O\) such that \(\partial \Delta '\cap \partial O\ne \emptyset \).

Proof

Let \(\Delta \) be a tetrahedron in \(O'{\setminus } O\). If \(\partial \Delta \cap \partial O\ne \emptyset \), we obtain \(\Delta '=\Delta \). Otherwise, \(\partial \Delta \cap \partial O=\emptyset \). Note that \(\mathbf {c}\notin \Delta \) (\(\mathbf {c}\notin \partial \Delta \) in our non-degenerate case and \(\mathbf {c}\notin \mathring{\Delta }\) by Lemma 5 and since \(\mathbf {c}\in |O|\)). Thus there is a triangle \(t\in \partial \Delta \) such that \(\mathbf {c}\) and \(\Delta \) are not in the same t half-space. Since \(t\notin \partial O\), Lemma 8 provides \(\Delta '\). \(\square \)

Theorem 8

There is a tetrahedron \(\Delta \in O'{\setminus } O\) such that \(\partial \Delta \cap \partial O\ne \emptyset \) and every triangle \(t\in \partial \Delta {\setminus }\partial O\) is such that \(\mathbf {c}\) and \(\Delta \) are in the same t half-space (Fig. 10).

Fig. 10

Theorem 8 in the 2D case where O forms an octagon that is \(\mathbf {c}\)-star-shaped. Left \(\Delta \) meets all conditions of the theorem. Middle \(\Delta \) does not. Right \(\Delta \) never does (reductio ad absurdum). We see that \(\Delta _1\) is behind \(\Delta _2\), which is behind \(\Delta _3\), \(\cdots \) which is behind \(\Delta _8\), which is behind \(\Delta _1\). However such a cycle is impossible in a Delaunay triangulation [7]

Proof

Assume (reductio ad absurdum) that
$$\begin{aligned}&\forall \Delta \in O'{\setminus } O, \partial \Delta \cap \partial O=\emptyset \hbox { or } \exists t\in \partial \Delta {\setminus } \partial O \hbox { such that } \\&\quad \Delta \hbox { and }\mathbf {c}\hbox { are not in the same half-space of }t. \end{aligned}$$
Thus \(\Delta \in O'{\setminus } O\) and \(\partial \Delta \cap \partial O\ne \emptyset \) imply that \(\Delta \) meets the assumptions of Lemma 8.

The principle of the proof is the following. We find an infinite series of tetrahedra \(\Delta _i\in O'{\setminus } O\) such that \(\partial \Delta _i\cap \partial O\ne \emptyset \) and \(\Delta _i\) is behind \(\Delta _{i+1}\) for all i using Lemma 8. Since \(O'\) is finite, there are \(m\ne n\) such that \(\Delta _m= \Delta _n\), i.e. the relation “is behind” has a cycle. This contradicts [7] which shows that this relation is acyclic in Delaunay triangulation T.

Here is the series. First Lemma 9 provides \(\Delta _0\in O'{\setminus } O\) such that \(\partial \Delta _0\cap \partial O\ne \emptyset \). Now \(\Delta _{i+1}\) is defined from \(\Delta _i\) as follow. If \(\Delta _i\in O'{\setminus } O\) and \(\partial \Delta _i\cap \partial O\ne \emptyset \), \(\Delta _i\) meets the assumptions of Lemma 8 (thanks to the reductio ad absurdum). Thus Lemma 8 provides \(\Delta _{i+1}\in O'{\setminus } O\) such that \(\partial \Delta _{i+1}\cap \partial O\ne \emptyset \) and \(\Delta _i\) is behind \(\Delta _{i+1}\).\(\square \)

5.4 Proof of Star-Shape Shelling

First we show in Theorem 9 that \(c(O)\cap c(\Delta )\) is a 2-ball using the tetrahedron \(\Delta \) provided by Theorem 8. The proof highly relies on the fact that we are not in a degenerate case, i.e. \(\mathbf {c}\) is not in a plane including a triangle of c(T). It also requires the three following lemmas. Then a shelling between O and \(O'\) is obtained in a corollary of Theorem 9.

Lemma 10

If O is \(\mathbf {c}\)-star-shaped and \(\Delta \) is defined by Theorem 8, \(O\cup \{\Delta \}\) is also \(\mathbf {c}\)-star-shaped.

Proof

According to Theorem 7 applied to O, every triangle \(t\in \partial O{\setminus }\partial \Delta \) is a face of a tetrahedron \(\Delta '\in O\) such that \(\Delta '\) and \(\mathbf {c}\) are in the same t half-space. Theorem 8 implies that every triangle \(t\in \partial \Delta {\setminus }\partial O\) is such that \(\Delta \) and \(\mathbf {c}\) are in the same t half-space. Thanks to Lemma 2, every triangle \(t\in \partial (O\cup \{\Delta \})\) is a face of a tetrahedron \(\Delta '\in O\cup \{\Delta \}\) such that \(\Delta '\) and \(\mathbf {c}\) are in the same t half-space. Last we apply Theorem 7 to \(O\cup \{\Delta \}\) and see that \(O\cup \{\Delta \}\) is \(\mathbf {c}\)-star-shaped. \(\square \)

The proofs of the two following technical lemmas are in the supplementary material and “Appendix 2.”

Lemma 11

Let \(\mathbf {abcd}\) be a tetrahedron and K be a simplicial complex such that \(K\subseteq c(\mathbf {abcd})\). If \(c(\mathbf {abc})\subsetneq K\), \(\mathbf {d}\in K\). If \(c(\mathbf {abc},\mathbf {bcd})\subsetneq K\), \(\mathbf {ad}\in K\).

Lemma 12

Let \(\Delta = \mathbf {v}_1\mathbf {v}_2\mathbf {v}_3\mathbf {v}_4\) and triangles \(t_i\in \partial \Delta \) such that \(\mathbf {v}_i\notin t_i\). Let \(H_i\) and \(H'_i\) be the two half-spaces of \(t_i\) such that \(\Delta \subset H_i\). We have \(H'_1\cap H'_2\cap H'_3\cap H'_4=\emptyset \). If \(\mathbf {c}\in H'_1\cap H_2\cap H_3\cap H_4\), \(\mathbf {v}_1\mathbf {c}\cap t_1\ne \emptyset \). Let \(\mathbf {v}= (\mathbf {v}_1+\mathbf {v}_2)/2\). If \(\mathbf {c}\in H'_1\cap H'_2\cap H_3\cap H_4\), \(\mathbf {vc}\cap (t_1\cup t_2)\ne \emptyset \) (Fig. 11).

Fig. 11

The point \(\mathbf {vc}\cap t_1\) according to Lemma 12. We have \(t_1= \mathbf {v}_2\mathbf {v}_3\mathbf {v}_4\), \(t_2= \mathbf {v}_3\mathbf {v}_4\mathbf {v}_1\), \(t_3= \mathbf {v}_4\mathbf {v}_1\mathbf {v}_2\) and \(t_4= \mathbf {v}_1\mathbf {v}_2\mathbf {v}_3\). Left \(\mathbf {v}= \mathbf {v}_1\) and \(\mathbf {c}\in H'_1\cap H_2\cap H_3\cap H_4\). The half-spaces \(H'_1\), \(H_2\), \(H_3\) and \(H_4\) are below \(t_1\), \(t_2\), \(t_3\) and \(t_4\), respectively. Right \(\mathbf {v}= (\mathbf {v}_1+\mathbf {v}_2)/2\) and \(\mathbf {c}\in H'_1\cap H'_2\cap H_3\cap H_4\). The half-spaces \(H'_1\), \(H'_2\), \(H_3\) and \(H_4\) are below \(t_1\), \(t_2\), \(t_3\) and \(t_4\), respectively

Theorem 9

If O is \(\mathbf {c}\)-star-shaped and \(\Delta \) is defined by Theorem 8, \(c(\partial \Delta )\cap c(\partial O)\) is a 2-ball.

Proof

Let triangles \(t_i\) be such that \(\partial \Delta =\{ t_1,t_2,t_3,t_4 \}\) and \(\partial \Delta \cap \partial O=\{ t_1,\ldots , t_n\}\). Thanks to Theorem 8, \(n\ge 1\). Let \(K= c(\partial \Delta )\cap c(\partial O)\). We show that \(K=c(t_1,\ldots , t_n)\) where \(1\le n\le 3\) and conclude using Theorem 3. Note that \(c(t_1,\ldots , t_n)= c(\partial \Delta \cap \partial O)\subseteq K\).

Let \(H_i\) and \(H'_i\) be the two half-spaces of \(t_i\) such that \(\Delta \subset H_i\). First we show that \(i\le n\) implies \(\mathbf {c}\in H'_i\) and \(i> n\) implies \(\mathbf {c}\in H_i\). If \(i\le n\), there is a tetrahedron \(\Delta _i\in O\) such that \(t_i=\Delta \cap \Delta _i\) and \(\Delta _i\subset H'_i\). Since O is \(\mathbf {c}\)-star-shaped, \(\mathbf {c}\) and \(\Delta _i\) are in the same \(t_i\) half-space (Theorem 7), i.e. \(\mathbf {c}\in H'_i\). If \(i> n\), \(t_i\in \partial \Delta {\setminus }\partial O\). Thanks to Lemma 2, \(t_i\in \partial (O\cup \{\Delta \})\). Since \(\partial (O\cup \{\Delta \})\) is \(\mathbf {c}\)-star-shaped (Lemma 10), \(\mathbf {c}\) and \(\Delta \) are in the same \(t_i\) half-space (Theorem 7), i.e. \(\mathbf {c}\in H_i\).

Second we show that \(n\ne 4\). If \(n=4\) (reductio ad absurdum), \(\mathbf {c}\in H'_1\cap H'_2\cap H'_3\cap H'_4\), which is impossible thanks to Lemma 12.

Third we show that \(n=3\) implies \(K= c(t_1,t_2,t_3)\). We have \(c(t_1,t_2,t_3)\subseteq K\subseteq c(\partial \Delta )= c(t_1,t_2,t_3)\cup \{ t_4\}\). If \(c(t_1,t_2,t_3)\subsetneq K\) (reductio ad absurdum), \(K= c(\partial \Delta )\), i.e. \(n=4\) (impossible).

Fourth we show that \(n=1\) implies \(K= c(t_1)\). Thus \(\mathbf {c}\in H'_1\cap H_2\cap H_3\cap H_4\). Let \(\mathbf {v}_1\) be the \(\Delta \) vertex that is not in \(t_1\), i.e. \(\mathbf {v}_1= t_2\cap t_3\cap t_4\). Thanks to Lemma 12, there is a point \(\mathbf {w}\in t_1\cap \mathbf {cv}_1\). Assume (reductio ad absurdum) that \(c(t_1)\subsetneq K\). According to Lemma 11, \(\mathbf {v}_1\in K\subseteq c(O)\). Since O is \(\mathbf {c}\)-star-shaped, \(\mathbf {cv}_1\subset |O|\). We obtain \(\mathbf {wv}_1\subseteq \Delta \cap |O|\). Since \(\Delta \notin O\), Lemma 5 implies \(\mathring{\Delta }\cap |O|= \emptyset \). Therefore \(\mathbf {wv}_1\subset |\partial \Delta |\). There is a triangle \(t_i\in \partial \Delta \) that includes at least two distinct points in \(\mathbf {wv}_1\). Since \(\mathbf {c}\in (\mathbf {wv}_1)\), \(\mathbf {c}\) is in the \(t_i\) plane: we are in a degenerate case (contradiction).

Last we show that \(n=2\) implies \(K= c(t_1,t_2)\). Thus \(\mathbf {c}\in H'_1\cap H'_2\cap H_3\cap H_4\). Let \(\mathbf {v}\) be the center of the edge \(t_3\cap t_4\). Thanks to Lemma 12, there is a point \(\mathbf {w}\in (t_1\cup t_2)\cap \mathbf {cv}\). Assume (reductio ad absurdum) that \(c(t_1,t_2)\subsetneq K\). According to Lemma 11, \(t_3\cap t_4\in K\subseteq c(O)\) and \(\mathbf {v}\in |O|\). Since O is \(\mathbf {c}\)-star-shaped, \(\mathbf {cv}\subset |O|\). Therefore \(\mathbf {wv}\subseteq \Delta \cap |O|\). Last we show that \(\mathbf {c}\) is in a \(t_i\) plane as in the previous case \(n=1\) (contradiction). \(\square \)

Now we obtain the result of Sect. 5.

Corollary 3

Let \(\mathbf {c}\) be a point in |T| that is not in a plane including a triangle of c(T). Let \(O\subseteq T\) such that O is non-empty \(\mathbf {c}\)-star-shaped. Then \(\partial O\) is a 2-manifold. Let \(O'\subseteq T\) such that \(O\subsetneq O'\) and \(O'\) is \(\mathbf {c}\)-star-shaped. Then there is a shelling \(O_0\cdots O_n\) such that \(O_0= O\), \(O_n= O'\) and every \(O_i\subseteq T\) is \(\mathbf {c}\)-star-shaped.

Proof

First we assume that both \(\tilde{O}\) and \(O_i\) are \(\mathbf {c}\)-star-shaped such that \(\emptyset \ne O_i\subsetneq \tilde{O}\subseteq T\) and \(\partial O_i\) is a 2-manifold. We show that there is a tetrahedron \(\Delta _i\in \tilde{O}{\setminus } O_i\) such that \(\partial \Delta _i\cap \partial O_i\ne \emptyset \) and, using notation \(O_{i+1}= O_i\cup \{\Delta _i\}\), \(O_{i+1}\) is \(\mathbf {c}\)-star-shaped and \(\partial O_{i+1}\) is a 2-manifold. Theorem 8 provides the tetrahedron \(\Delta _i\). Let \(O_{i+1}= O_i\cup \{\Delta _i\}\). Now \(O_{i+1}\) is \(\mathbf {c}\)-star-shaped (Lemma 10) and \(c(\partial \Delta _i)\cap c(\partial O_i)\) is a 2-ball (Theorem 9). Thus \(c(\Delta _i)\cap c(O_i)\) is non-empty 2D-pure (Theorem 3) and \(\partial O_{i+1}\) is a 2-manifold (Theorem 1).

Second we show that \(\partial O\) is a 2-manifold. Let \(\Delta '\) be a tetrahedron such that \(\mathbf {c}\in \Delta '\in T\). Let \(O_0= \{\Delta '\}\). Note that \(O_0\) is \(\mathbf {c}\)-star-shaped (\(\Delta '\) is convex) and \(\partial O_0\) is a 2-manifold (\(\partial O_0\) is a 2-sphere). By successive use(s) of the first step (above) with \(\tilde{O}= O\), we find a series \(O_0\cdots O_n\) in T such that \(O_n= O\) and every \(\partial O_i\) is a 2-manifold.

Third we show that there is a shelling \(O_0\cdots O_n\) such that \(O_0= O\), \(O_n= O'\) and every \(O_i\) is \(\mathbf {c}\)-star-shaped. We set \(O_0= O\). Note that \(O_0\) is \(\mathbf {c}\)-star-shaped and \(\partial O_0\) is a 2-manifold. By successive use(s) of the first step (above) with \(\tilde{O}= O'\), we find a series \(O_0\cdots O_n\) in T such that \(O_n= O'\) and every \(O_i\) is \(\mathbf {c}\)-star-shaped. This series is a shelling since \(O_{i+1}= O_i\cup \{\Delta _i\}\) and \(c(\Delta _i)\cap c(O_i)\) is non-empty 2D-pure for all \(i< n\), and \(\partial O_0\) is a non-empty 2-manifold. \(\square \)

According to “Appendix 3,” there are non-shellable star shapes in degenerate cases.

5.5 A Qualitative Argument for Surface Reconstruction

Here we provide a qualitative argument (this is not a proof as in the paper remainder) to explain that the first shelling in [13], i.e. Algorithm 1 started from \(O=\emptyset \), does not have excessive blocking and provides most of the tetrahedra enclosed by the final surface. Section 3.1 gives a reminder of this algorithm and useful notations F and \(n_\Delta \).

The set F looks like a star-shape: for every point \(\mathbf {x}\) reconstructed from a camera location \(\mathbf {c}\), \(\mathbf {cx}\subset \cup F\) (F is not exactly a star shape since \(\mathbf {x}\) is restricted to the vertices in c(T) and there are several \(\mathbf {c}\)). Assuming that \(\mathbf {c}\) follows a curve and the distance between \(\mathbf {x}\) and \(\mathbf {c}\) is small compared to the curve length, F can be seen as a tubular neighborhood of the curve.

Furthermore, O also looks like a star shape during the first shelling in [13]. Since both \(\Delta \) size and \(\mathbf {cx}\) density increase when we go toward the curve, \(n_\Delta \) increases when we go toward the curve. Since this shelling tries to add first to O the tetrahedra \(\Delta \) having the highest \(n_\Delta \), O essentially grows in a front-to-back order relatively to the curve. As a consequence, the points in \(\cup F\) with a small distance to the curve enter in \(\cup O\) during the shelling before those with a greater distance, i.e. there is a \(\mathbf {c}\) in the curve such that \(\mathbf {cx}\subset \cup O\) if \(\mathbf {x}\in \cup O\).

Now O is growing in F by the shelling such that both O and F look like star shapes with the same curve as “center.” Then Corollary 3 suggests that O fills F, if we replace the point center by the curve center.

Let O / F be the ratio of the number of the tetrahedra in O generated by the first shelling alone and the number of the tetrahedra in F. Let \(O/O_e\) be the ratio of the number of the tetrahedra in O generated by the first shelling alone and the final number of the tetrahedra generated by all operators (including the first shelling). In Tab. 2 of [13], \(O/F= 87\%\) and \(O/O_e=94\%\).

6 Shelling Blocking

In this section, we provide a family of 3-balls that are strongly non-shellable with small numbers of simplices (Sect. 6.5). We also provide other cases of blocking (Sect. 6.6) for a shelling started from a general \(O_0\subseteq T\). All these cases of shelling blocking are generated by disjoint unions of tetrahedron sets in T that are called “pipe” (Sect. 6.1) and “slice” (Sect. 6.3). Both are 3-balls and their “gluing” (union of such a 3-ball with another tetrahedron set in \(T^\infty \)) is described in Sects. 6.2 and 6.4.

6.1 Basic Component: Pipe

If triangles \(\mathbf {a}_0\mathbf {b}_0\mathbf {c}_0\) and \(\mathbf {a}_1\mathbf {b}_1\mathbf {c}_1\) have distinct vertices, the tetrahedron set \(P= p(\mathbf {a}_0\mathbf {b}_0\mathbf {c}_0, \mathbf {a}_1\mathbf {b}_1\mathbf {c}_1)\) is
$$\begin{aligned} \{ \mathbf {a}_0\mathbf {a}_1\mathbf {b}_1\mathbf {c}_0, \mathbf {a}_0\mathbf {a}_1\mathbf {b}_0\mathbf {c}_0, \mathbf {a}_1\mathbf {b}_0\mathbf {b}_1\mathbf {c}_0, \mathbf {a}_0\mathbf {a}_1\mathbf {b}_1\mathbf {c}_1, \mathbf {a}_0\mathbf {b}_1\mathbf {c}_0\mathbf {c}_1 \}.\nonumber \\ \end{aligned}$$
(14)
Fig. 12

The pipe \(p(\mathbf {a}_0\mathbf {b}_0\mathbf {c}_0, \mathbf {a}_1\mathbf {b}_1\mathbf {c}_1)\) (right) and its decomposition (left). See the internal tetrahedron \(\mathbf {a}_0\mathbf {a}_1\mathbf {b}_1\mathbf {c}_0\)

We assume that \(P\subseteq T\), show P in Fig. 12, and see that \(\mathbf {a}_0\mathbf {a}_1\mathbf {b}_1\mathbf {c}_0\) is an internal tetrahedron of P. Indeed, every triangle face of \(\mathbf {a}_0\mathbf {a}_1\mathbf {b}_1\mathbf {c}_0\) is join to \(\mathbf {b}_0\) or \(\mathbf {c}_1\) by another tetrahedron in P:
$$\begin{aligned} P= & {} \{ \mathbf {a}_0\mathbf {a}_1\mathbf {b}_1\mathbf {c}_0 \} \cup \mathbf {b}_0 \times \{ \mathbf {a}_0\mathbf {a}_1\mathbf {c}_0, \mathbf {a}_1\mathbf {b}_1\mathbf {c}_0\} \nonumber \\&\cup \, \mathbf {c}_1\times \{ \mathbf {a}_0\mathbf {a}_1\mathbf {b}_1, \mathbf {a}_0\mathbf {b}_1\mathbf {c}_0\}. \end{aligned}$$
(15)

Lemma 13

We have
$$\begin{aligned} \partial P= & {} \{ \mathbf {a}_0\mathbf {b}_0\mathbf {c}_0, \mathbf {a}_1\mathbf {b}_1\mathbf {c}_1\} \cup \{ \mathbf {a}_0\mathbf {a}_1\mathbf {b}_0, \mathbf {a}_1\mathbf {b}_0\mathbf {b}_1\} \nonumber \\&\cup \, \{ \mathbf {b}_0\mathbf {b}_1\mathbf {c}_0, \mathbf {b}_1\mathbf {c}_0\mathbf {c}_1\} \cup \{ \mathbf {a}_0\mathbf {c}_0\mathbf {c}_1, \mathbf {a}_0\mathbf {a}_1\mathbf {c}_1\}. \end{aligned}$$
(16)

Proof

Equation 15 implies that the triangles in c(P), that have neither \(\mathbf {b}_0\) nor \(\mathbf {c}_1\) as vertex, are faces of the internal tetrahedron \(\mathbf {a}_0\mathbf {a}_1\mathbf {b}_1\mathbf {c}_0\). Since \(\partial P\subset c(P)\), every triangle in \(\partial P\) has vertex \(\mathbf {b}_0\) or \(\mathbf {c}_1\), i.e.
$$\begin{aligned} \partial P\subseteq & {} \mathbf {b}_0 \times \mathbf {a}_0\hbox {-}\mathbf {a}_1\hbox {-}\mathbf {c}_0\hbox {-}\mathbf {a}_0\cup \mathbf {b}_0 \times \mathbf {a}_1\hbox {-}\mathbf {b}_1\hbox {-}\mathbf {c}_0\hbox {-}\mathbf {a}_1 \nonumber \\&\cup \, \mathbf {c}_1 \times \mathbf {a}_0\hbox {-}\mathbf {a}_1\hbox {-}\mathbf {b}_1\hbox {-}\mathbf {a}_0\cup \mathbf {c}_1 \times \mathbf {a}_0\hbox {-}\mathbf {b}_1\hbox {-}\mathbf {c}_0\hbox {-}\mathbf {a}_0. \end{aligned}$$
(17)
Since a triangle in \(\partial P\) is in a single tetrahedron of P,
$$\begin{aligned} \partial P= & {} \mathbf {b}_0 \times \mathbf {c}_0\hbox {-}\mathbf {a}_0\hbox {-}\mathbf {a}_1\cup \mathbf {b}_0 \times \mathbf {a}_1\hbox {-}\mathbf {b}_1\hbox {-}\mathbf {c}_0 \nonumber \\&\cup \, \mathbf {c}_1 \times \mathbf {a}_0\hbox {-}\mathbf {a}_1\hbox {-}\mathbf {b}_1\cup \mathbf {c}_1 \times \mathbf {b}_1\hbox {-}\mathbf {c}_0\hbox {-}\mathbf {a}_0 \nonumber \\= & {} \mathbf {b}_0 \times \mathbf {c}_0\hbox {-}\mathbf {a}_0\hbox {-}\mathbf {a}_1 \hbox {-}\mathbf {b}_1\hbox {-}\mathbf {c}_0\cup \mathbf {c}_1 \times \mathbf {a}_0\hbox {-}\mathbf {a}_1\hbox {-}\mathbf {b}_1 \hbox {-}\mathbf {c}_0\hbox {-}\mathbf {a}_0 \end{aligned}$$
(18)
and we obtain the result. \(\square \)

We say that P is a pipe. Intuitively, the two cross sections of the pipe are triangles \(\mathbf {a}_0\mathbf {b}_0\mathbf {c}_0\) and \(\mathbf {a}_1\mathbf {b}_1\mathbf {c}_1\). The lateral section of the pipe is a strip of 6 triangles (in Eq. 16) having edges \(\mathbf {a}_0\mathbf {a}_1\), \(\mathbf {b}_0\mathbf {b}_1\), \(\mathbf {c}_0\mathbf {c}_1\) and three others. A pipe series \(p(\mathbf {a}_0\mathbf {b}_0\mathbf {c}_0,\mathbf {a}_1\mathbf {b}_1\mathbf {c}_1)\), \(p(\mathbf {a}_1\mathbf {b}_1\mathbf {c}_1,\mathbf {a}_2\mathbf {b}_2\mathbf {c}_2)\), \(p(\mathbf {a}_2\mathbf {b}_2\mathbf {c}_2,\mathbf {a}_3\mathbf {b}_3\mathbf {c}_3)\), \(\cdots \) will be used to form a longer “pipe” and build strongly non-shellable 3-balls.

Lemma 14

If \(p(\mathbf {a}_0\mathbf {b}_0\mathbf {c}_0,\mathbf {a}_1\mathbf {b}_1\mathbf {c}_1) \subseteq T\), P is a 3-ball.

The proof ot this lemma is in the supplementary material (use a shelling, or see that \(\partial P\) is an octahedron).

6.2 Gluing a Pipe

The goal of Sect. 6.2 is Theorem 11: we define an union of a pipe P and another tetrahedron set \(A\subseteq T^\infty \) such that the criterion of the strongly non-shellability (Sect. 4.1) applied to \(A\cup P\) is met for every tetrahedron in P. The proof of this theorem (and others) needs the following gluing lemma and theorem.

Lemma 15

If \(A\subseteq T^\infty \) and \(B\subseteq T^\infty \) and \(A\cap B=\emptyset \), \(\partial (A\cup B)= (\partial A{\setminus }\partial B)\cup (\partial B{\setminus }\partial A)\).

Proof

Let \(t\in c(T^\infty )\) and \(C= T^\infty {\setminus } (A\cup B)\). Let \(\Delta \) and \(\Delta '\) be the two tetrahedra in \(T^\infty \) such that \(t= \Delta \cap \Delta '\). We show that \(t\in \partial (A\cup B)\) iff \(t\in (\partial A{\setminus }\partial B)\cup (\partial B{\setminus }\partial A)\) in all cases.

If \((\Delta ,\Delta ')\in A\times A\), the triangle t is neither in \(\partial A{\setminus }\partial B\) nor in \(\partial (A\cup B)\). Since \(A\cap B=\emptyset \), we also have \(t\notin \partial B{\setminus }\partial A\). The case \((\Delta ,\Delta ')\in B\times B\) is similar. If \((\Delta ,\Delta ')\in C\times C\), t is neither in \(\partial (A\cup B)\) nor in \((\partial A{\setminus }\partial B)\cup (\partial B{\setminus }\partial A)\). If \((\Delta ,\Delta ')\in A\times B\), \(t\notin \partial (A\cup B)\). Since \(A\cap B=\emptyset \), \(t\in \partial A\cap \partial B\) and thus \(t\notin (\partial A{\setminus }\partial B)\cup (\partial B{\setminus }\partial A)\). If \((\Delta ,\Delta ')\in A\times C\), \(t\in \partial A{\setminus }\partial B\) and \(t\in \partial (A\cup B)\). The case \((\Delta ,\Delta ')\in B\times C\) is similar. \(\square \)

Theorem 10

Let \(A\subseteq T^\infty \) and \(B\subseteq T{\setminus } A\) such that both \(\partial A\) and \(\partial B\) are 2-manifolds, and \(c(A)\cap c(B)\) is a 2-manifold with boundary. Then \(\partial (A\cup B)\) is a 2-manifold.

This theorem means that volumes A and B can be glued properly, i.e. by maintaining the 2-manifold property of their boundaries, if their intersection is well behaved (Fig. 13). The proof is technical and is in “Appendix 4.”
Fig. 13

Gluing \(A\subset T\) (left) and \(B\subset T\) (middle) using Theorem 10. Both \(\partial A\) and \(\partial B\) are 2-manifolds, \(A\cap B=\emptyset \) and \(\partial A\cap \partial B\) is the union of two 2-balls. Right: the result \(A\cup B\). If \(c(A)\cap c(B)\) is a 2-manifold with boundary, \(\partial (A\cup B)\) is a 2-manifold (top). If \(c(A)\cap c(B)\) is 2D pure and non-manifold, \(\partial (A\cup B)\) can have a singular vertex (bottom)

Theorem 11

Let \(A\subseteq T^\infty \) such that \(\partial A\) is a 2-manifold. Let \(P= p(\mathbf {a}_0\mathbf {b}_0\mathbf {c}_0,\mathbf {a}_1\mathbf {b}_1\mathbf {c}_1)\) such that \(P\subseteq T{\setminus } A\) and
$$\begin{aligned} c(A)\cap c(P)= c(\mathbf {a}_0\mathbf {b}_0\mathbf {c}_0, \mathbf {a}_1\mathbf {b}_1\mathbf {c}_1, \mathbf {b}_0\mathbf {b}_1\mathbf {c}_0, \mathbf {b}_1\mathbf {c}_0\mathbf {c}_1). \end{aligned}$$
(19)
Then \(\partial (A\cup P)\) is a 2-manifold. If a tetrahedron \(\Delta \in P\), \(c(\partial (A\cup P))\cap c(\Delta )\) is not a 2-ball (Fig. 14).
Fig. 14

Assumptions of Theorem 11: P (left), A (middle) and \(A\cup P\) (right)

Proof

Since the triangles in \(c(A)\cap c(P)\) form a hexagon \(\mathbf {a}_0\mathbf {b}_0\mathbf {b}_1\mathbf {a}_1\mathbf {c}_1\mathbf {c}_0\), \(c(A)\cap c(P)\) is homeomorphic to a 2-ball. Then Lemma 14 and Theorem 10 imply that \(\partial (A\cup P)\) is a 2-manifold.

We will show that \(c(\partial (A\cup P))\cap c(\Delta )\) has zero or one triangle and includes four vertices if \(\Delta \in P\). This implies that \(c(\partial (A\cup P))\cap c(\Delta )\) is not 2D pure, which in turn implies that \(c(\partial (A\cup P))\cap c(\Delta )\) is not a 2-ball thanks to Theorem 3.

Since \(\partial A\cap \partial P= \{ \mathbf {a}_0\mathbf {b}_0\mathbf {c}_0, \mathbf {a}_1\mathbf {b}_1\mathbf {c}_1, \mathbf {b}_0\mathbf {b}_1\mathbf {c}_0, \mathbf {b}_1\mathbf {c}_0\mathbf {c}_1\}\), \(\partial P{\setminus } \partial A= \{ \mathbf {a}_0\mathbf {a}_1\mathbf {b}_0, \mathbf {a}_1\mathbf {b}_0\mathbf {b}_1, \mathbf {a}_0\mathbf {c}_0\mathbf {c}_1, \mathbf {a}_0\mathbf {a}_1\mathbf {c}_1 \}\) (Eq. 16). Therefore
$$\begin{aligned} \{ \mathbf {a}_0, \mathbf {a}_1, \mathbf {b}_0, \mathbf {b}_1, \mathbf {c}_0, \mathbf {c}_1 \}\subset c(\partial P{\setminus } \partial A) \end{aligned}$$
(20)
and
$$\begin{aligned} \Delta = \mathbf {a}_0\mathbf {a}_1\mathbf {b}_1\mathbf {c}_0\Rightarrow & {} \partial \Delta \cap (\partial P{\setminus }\partial A)= \emptyset \nonumber \\ \Delta = \mathbf {a}_0\mathbf {a}_1\mathbf {b}_0\mathbf {c}_0\Rightarrow & {} \partial \Delta \cap (\partial P{\setminus }\partial A)= \{ \mathbf {a}_0\mathbf {a}_1\mathbf {b}_0 \} \nonumber \\ \Delta = \mathbf {a}_1\mathbf {b}_0\mathbf {b}_1\mathbf {c}_0\Rightarrow & {} \partial \Delta \cap (\partial P{\setminus }\partial A)= \{ \mathbf {a}_1\mathbf {b}_0\mathbf {b}_1 \} \nonumber \\ \Delta = \mathbf {a}_0\mathbf {a}_1\mathbf {b}_1\mathbf {c}_1\Rightarrow & {} \partial \Delta \cap (\partial P{\setminus }\partial A)= \{ \mathbf {a}_0\mathbf {a}_1\mathbf {c}_1 \} \nonumber \\ \Delta = \mathbf {a}_0\mathbf {b}_1\mathbf {c}_0\mathbf {c}_1\Rightarrow & {} \partial \Delta \cap (\partial P{\setminus }\partial A)= \{ \mathbf {a}_0\mathbf {c}_0\mathbf {c}_1\}. \end{aligned}$$
(21)
Since \(\Delta \in P\) has its four vertices in \(\{ \mathbf {a}_0, \mathbf {a}_1, \mathbf {b}_0, \mathbf {b}_1, \mathbf {c}_0, \mathbf {c}_1 \}\) and thanks to Eq. 20, \(c(\partial P{\setminus } \partial A)\cap c(\Delta )\) includes four vertices. Furthermore, Eq. 21 implies that \((\partial P{\setminus } \partial A)\cap (\partial \Delta )\) has zero or one triangle for every \(\Delta \in P\).

Assume (reductio ad absurdum) that there is a triangle \(t\in (\partial A{\setminus }\partial P)\cap \partial \Delta \). Thus \(t\subseteq \Delta \in P\) and \(t\subseteq \Delta '\in A\). Since \(A\cap P=\emptyset \) (Eq. 19), we have \(\Delta '\notin P\) and obtain \(t\in \partial P\), which contradicts \(t\in \partial A{\setminus }\partial P\).

Thanks to Lemma 15, \((\partial P{\setminus }\partial A)\cup (\partial A{\setminus }\partial P)= \partial (A\cup P)\). Therefore \(c(\partial (A\cup P))\) includes the four vertices of \(\Delta \) and has zero or one triangle of \(\Delta \). \(\square \)

6.3 Basic Component: Slice

Let integer \(n\ge 3\) and vertices \(\mathbf {b}_0,\ldots , \mathbf {b}_n\) and \(\mathbf {c}_0,\ldots , \mathbf {c}_n\) in c(T) that are distinct, with exceptions \(\mathbf {b}_n= \mathbf {b}_0\) and \(\mathbf {c}_n= \mathbf {c}_0\). We have cycles \(\mathbf {b}_*= \mathbf {b}_0\hbox {-}\mathbf {b}_1\hbox {-}\cdots \mathbf {b}_{n-1} \hbox {-} \mathbf {b}_0\) and \(\mathbf {c}_*= \mathbf {c}_0\hbox {-}\mathbf {c}_1\hbox {-}\cdots \mathbf {c}_{n-1} \hbox {-} \mathbf {c}_0\). Let
$$\begin{aligned} N= & {} \bigcup _{0\le i< n} N_i\hbox { where } \nonumber \\ N_i= & {} \{ \mathbf {b}_i\mathbf {b}_{i+1}\mathbf {c}_i, \mathbf {b}_{i+1}\mathbf {c}_i\mathbf {c}_{i+1} \}\hbox { or } \nonumber \\ N_i= & {} \{ \mathbf {c}_{i+1}\mathbf {b}_i\mathbf {b}_{i+1}, \mathbf {c}_i\mathbf {c}_{i+1}\mathbf {b}_i \}. \end{aligned}$$
(22)
The set N is a strip of triangles that form an annulus and we have \(\partial N= \mathbf {b}_*\cup \mathbf {c}_*\).
Fig. 15

Annulus (left) and slice (right) using \(n=6\)

We say that \(S\subseteq T\) is a slice if S is a 3-ball such that \(N\subseteq \partial S\) (see Fig. 15).

Slice properties are in the following theorem:

Theorem 12

Let S be a slice with an annulus \(N\subseteq \partial S\). There are B and C such that \(\partial S= N\cup B\cup C\), \(\partial B= \mathbf {b}_*\), \(\partial C= \mathbf {c}_*\), \(c(B)\cap c(C)=\emptyset \), \(c(B)\cap c(N)= c(\mathbf {b}_*)\), \(c(C)\cap c(N)= c(\mathbf {c}_*)\), both B and C are strongly connected.

Intuitively, the boundary of the 3-ball S is a 2-sphere segmented by N in three connected parts BNC, whose common boundaries are \(\mathbf {b}_*\) or \(\mathbf {c}_*\). Furthermore, S has two “sides” |B| and |C| which do not intersect. In the right of Fig. 15, B is on the bottom and C is on the top of the slice. The proof is technical (see “Appendix 5”).

6.4 Gluing a Slice

Similarly as in Sect. 6.2, the goal of Sect. 6.4 is Corollary 4: we define an union of a slice \(S'\) and another tetrahedron set \(A\subseteq T^\infty \) such that the criterion of the strongly non-shellability applied to \(A\cup S'\) is met for every tetrahedron in \(S'\). Here \(S'\) is computed by “shrinking” an initial slice S: we progressively remove every tetrahedron \(\Delta \) from S that does not meet this criterion. The proof needs Theorem 13, which guarantees that \(S'\) is a slice (a 3-ball with the same annulus as S). This theorem needs the following lemma.

Lemma 16

Let \(A\subseteq T^\infty \) such that \(\partial A\) is a 2-manifold. Let \(S\subset T{\setminus } A\) be a 3-ball such that \(c(A)\cap c(S)= c(N)\) and N is an annulus. Then there are B and C as in Theorem 12 such that \(c(\partial (A\cup S))\cap c(S)= c(B)\cup c(C)\).

Proof

Since \(c(A)\cap c(S)= c(N)\) and \(A\cap S=\emptyset \), every triangle in N is in a tetrahedron in A and another in S. Thus S is a slice with the annulus \(N\subset \partial S\). Let B and C as in Theorem 12. Now \(N=\partial A\cap \partial S\) and Lemma 15 imply \(\partial (A\cup S)= (\partial A{\setminus } N)\cup (\partial S{\setminus } N)\). We use shortened notation \(\bar{S}= c(S)\). Since \(\partial (A\cup S)= (\partial A{\setminus } N)\cup (\partial S{\setminus } N)\) and \(\partial S{\setminus } N= B\cup C\) and \(B\subset \bar{S}\) and \(C\subset \bar{S}\),
$$\begin{aligned} c(\partial (A\cup S))\cap \bar{S}= & {} (c(\partial A{\setminus } N)\cap \bar{S}) \cup (c(\partial S{\setminus } N)\cap \bar{S}) \nonumber \\= & {} (c(\partial A{\setminus } N)\cap \bar{S})\cup c(B)\cup c(C). \end{aligned}$$
(23)
Let \(\sigma \) be a simplex in \(c(\partial A{\setminus } N)\cap \bar{S}\) and show that \(\sigma \in c(B)\cup c(C)\). Since \(c(\partial A{\setminus } N)\cap \bar{S}\subseteq c(A)\cap \bar{S}= c(N)\), we have \(\sigma \in c(N)\). We have \(c(N)= c(\mathbf {b}_*)\cup c(\mathbf {c}_*)\cup \tilde{N}\) using
$$\begin{aligned} \tilde{N}= & {} \cup _i \tilde{N}_i\hbox { where } \nonumber \\ \tilde{N}_i= & {} \{ \mathbf {b}_i\mathbf {c}_i, \mathbf {b}_{i+1}\mathbf {c}_{i+1}, \mathbf {b}_{i+1}\mathbf {c}_i, \mathbf {b}_i\mathbf {b}_{i+1}\mathbf {c}_i, \mathbf {b}_{i+1}\mathbf {c}_i\mathbf {c}_{i+1} \}\hbox { or } \nonumber \\ \tilde{N}_i= & {} \{ \mathbf {b}_i\mathbf {c}_i, \mathbf {b}_{i+1}\mathbf {c}_{i+1}, \mathbf {b}_i\mathbf {c}_{i+1}, \mathbf {c}_{i+1}\mathbf {b}_i\mathbf {b}_{i+1}, \mathbf {c}_i\mathbf {c}_{i+1}\mathbf {b}_i\}.\nonumber \\ \end{aligned}$$
(24)
If \(\sigma \in c(\mathbf {b}_*)\cup c(\mathbf {c}_*)\), \(\sigma \in c(B)\cup c(C)\). Otherwise \(\sigma \in \tilde{N}\). This implies that there is an edge \(\mathbf {b}_i\mathbf {c}_i\) or \(\mathbf {b}_{i+1}\mathbf {c}_i\) or \(\mathbf {c}_{i+1}\mathbf {b}_i\) in \(c(\partial (A\cup S))\cap c(N)\). Let e be this edge. There are distinct triangles \(t_1\) and \(t_2\) in \(N=\partial S\cap \partial A\) such that \(e= t_1\cap t_2\). Since \(e\in c(\partial (A\cup S))= c((\partial A\cup \partial S){\setminus } N)\), there is another triangle \(t_3\in (\partial A\cup \partial S){\setminus } N\) such that \(e\subset t_3\). If \(t_3\in \partial A\), e is in 3 distinct triangles in \(\partial A\). This is impossible since \(\partial A\) is a 2-manifold. If \(t_3\in \partial S\), e is in 3 distinct triangles in \(\partial S\). This is impossible since \(\partial S\) is a 2-manifold. \(\square \)

Theorem 13

Let \(A\subseteq T^\infty \) such that \(\partial A\) is a 2-manifold. Let \(S\subset T{\setminus } A\) be a 3-ball such that \(c(A)\cap c(S)= c(N)\) and N is an annulus. Let \(\Delta \) be a tetrahedron in S such that \(c(\partial (A\cup S))\cap c(\Delta )\) is a 2-ball. Then \(S{\setminus } \{\Delta \}\) is a 3-ball and \(c(A)\cap c(S{\setminus } \{\Delta \})= c(N)\).

Proof

Let B and C be as in Lemma 16. Let \(X= c(\Delta )\cap c(B)\) and \(Y= c(\Delta )\cap c(C)\). First we show that \(X=\emptyset \) (or similarly, \(Y=\emptyset \)). Since \(\Delta \in S\),
$$\begin{aligned} c(\Delta )\cap c(\partial (A\cup S))= & {} c(\Delta )\cap (c(\partial (A\cup S))\cap c(S)) \nonumber \\= & {} X\cup Y. \end{aligned}$$
(25)
Assume (reductio ad absurdum) that \(X\ne \emptyset \) and \(Y\ne \emptyset \). Since \(c(B)\cap c(C)=\emptyset \), \(X\cap Y=\emptyset \) and \(c(\Delta )\cap c(\partial (A\cup S))\) is not connected. This contradicts that \(c(\Delta )\cap c(\partial (A\cup S))\) is a 2-ball.

Second we show that \(c(\Delta )\cap c(N)\subseteq c(\Delta )\cap c(\mathbf {c}_*)\). Since \(c(\Delta )\cap c(\mathbf {b}_*)\subseteq X= \emptyset \), \(\Delta \) does not have a vertex in \(c(\mathbf {b}_*)\). Since every vertex in \(c(\Delta )\cap c(N)\) is a \(\mathbf {b}_i\) or a \(\mathbf {c}_i\), every vertex in \(c(\Delta )\cap c(N)\) is a \(\mathbf {c}_i\). Thus \(c(\Delta )\cap c(N)\subseteq c(\mathbf {c}_*)\). We obtain \(c(\Delta )\cap c(N)\subseteq c(\Delta )\cap c(\mathbf {c}_*)\).

Third we show that \(S{\setminus } \{\Delta \}\) is a 3-ball. Since \(\partial S= B\cup C\cup N\) and \(c(\Delta )\cap c(N)\subseteq c(\Delta )\cap c(\mathbf {c}_*)\subseteq c(\Delta )\cap c(C)= Y\),
$$\begin{aligned} c(\Delta )\cap c(\partial S)= & {} X\cup Y\cup (c(\Delta )\cap c(N))= Y \nonumber \\= & {} c(\Delta )\cap c(\partial (A\cup S)). \end{aligned}$$
(26)
Therefore \(c(\Delta )\cap c(\partial S)\) is a 2-ball. Since S is a 3-ball, we see that \(S{\setminus } \{\Delta \}\) is a 3-ball (use Corollary 2 for the shelling \(O_0= T^\infty {\setminus } S\), \(O_1= O_0\cup \{\Delta \}\) and Theorem 5).
Last we show that \(c(A)\cap c(S)= c(A)\cap c(S{\setminus }\{\Delta \})\). We have
$$\begin{aligned} c(A)\cap c(S)= (c(A)\cap c(S{\setminus }\{\Delta \}))\cup (c(A)\cap c(\Delta )). \end{aligned}$$
(27)
Since \(\Delta \in S\) and \(c(N)= c(A)\cap c(S)\),
$$\begin{aligned} c(A)\cap c(\Delta )= c(N)\cap c(\Delta )\subseteq c(\Delta )\cap c(\mathbf {c}_*). \end{aligned}$$
(28)
Let \(\sigma \) be a simplex in \(c(A)\cap c(\Delta )\). Thanks to Eq. 28, \(\sigma \) is a vertex or an edge. Since \(c(N)= c(A)\cap c(S)\) and thanks to Eq. 27, \(\sigma \) is in a triangle \(t\in N\). Since \(c(A)\cap c(\Delta )\) does not have a triangle, \(t\in c(A)\cap c(S{\setminus }\{\Delta \})\). Thus \(c(A)\cap c(\Delta )\subseteq c(A)\cap c(S{\setminus }\{\Delta \})\). \(\square \)

Corollary 4

Let \(A\subseteq T^\infty \) such that \(\partial A\) is a 2-manifold. Let \(S\subset T{\setminus } A\) be a 3-ball such that \(c(A)\cap c(S)= c(N)\) and N is an annulus. There is a 3-ball \(S'\subseteq S\) such that \(c(A)\cap c(S')= c(N)\) and \(c(\Delta )\cap c(\partial (A\cup S'))\) is not a 2-ball for every tetrahedron \(\Delta \in S'\).

Proof

Let \(S'=S\). We consider the following loop algorithm: while there is a tetrahedron \(\Delta \in S'\) such that \(c(\Delta )\cap c(\partial (A\cup S'))\) is a 2-ball, remove \(\Delta \) from \(S'\). Thanks to Theorem 13, \(S'\) is always a 3-ball and we always have \(c(A)\cap c(S')= c(N)\). The algorithm stops since S is finite, and it stops if \(c(\Delta )\cap c(\partial (A\cup S'))\) is not a 2-ball for every tetrahedron \(\Delta \in S'\). \(\square \)

6.5 Family of Strongly Non-shellable 3-Balls

First we need a specialization of Theorem 10 (proof in “Appendix 6”):

Theorem 14

Let \(A\subseteq T^\infty \) and \(B\subseteq T{\setminus } A\) such that \(\partial A\) is a connected 2-manifold, \(\partial B\) is a 2-sphere, \(c(A)\cap c(B)\) is a 2-ball. Then \(\partial (A\cup B)\) is homeomorphic to \(\partial A\).

Then Theorem 15 presents a family of strongly non-shellable 3-balls.

Theorem 15

Let \(P_i= p(\mathbf {a}_i\mathbf {b}_i\mathbf {c}_i, \mathbf {a}_{i+1}\mathbf {b}_{i+1}\mathbf {c}_{i+1})\) or \(P_i= p(\mathbf {a}_{i+1}\mathbf {b}_{i+1}\mathbf {c}_{i+1}, \mathbf {a}_i\mathbf {b}_i\mathbf {c}_i)\) such that \(P_i\subseteq T\) if \(0\le i< n\). Let \(S\subseteq T{\setminus } \cup _i P_i\) be a 3-ball such that \(c(\cup _i P_i)\cap c(S)= c(N)\) and N is an annulus using notations in Eq. 22 (top of Fig. 16). Then there is a 3-ball \(S'\subseteq S\) such that \(c(\cup _i P_i)\cap c(S')= c(N)\) and \(\cup _i P_i\cup S'\) is a strongly non-shellable 3-ball (bottom of Fig. 16).

Fig. 16

Illustrations for Theorem 15. Top S (right) and \(\cup _i P_i\cup S\) (left). The pipes \(P_i\) form a torus; we have \(c(\cup _i P_i)\cap c(S)= c(N)\) where N is the annulus whose boundary is \(\mathbf {b}_*\cup \mathbf {c}_*\). Bottom cross section by a plane including gray triangles \(\mathbf {a}_0\mathbf {b}_0\mathbf {c}_0\) and \(\mathbf {a}_3\mathbf {b}_3\mathbf {c}_3\). The white triangles are tetrahedra in S (left), \(S'\) is obtained by removing some of these tetrahedra such that \(c(\cup _i P_i)\cap c(S')= c(N)\) (middle), a singular vertex appears if we remove a tetrahedron from \(S'\) (right)

Intuitively (Fig. 16), \(\partial (\cup _i P_i)\) is a torus whose intersection with \(\partial S\) (and also \(\partial S'\)) is the annulus N. Furthermore, \(\partial (\cup _i P_i\cup S')\) is a 2-sphere, with both “sides” separated by the cycle \(\mathbf {a}_*\) (B is in one side and C is in the other). If a tetrahedron \(\Delta \) having a triangle in one side is removed from \(\cup _i P_i\cup S'\), a singular vertex appears in both sides. We use Theorem 11 if \(\Delta \in P_i\) and Corollary 4 if \(\Delta \in S'\) to show that \(c(\Delta )\cap c(\partial (\cup _i P_i\cup S'))\) is not a 2-ball, i.e. to show that \(\cup _i P_i\cup S'\) is a strongly non-shellable 3-ball.

Proof

First we find \(S'\). Every \(\partial P_i\) is a 2-manifold and every \(c(\partial (\cup _{k=0}^{i-1} P_k))\cap c(\partial P_i)\) is a 2-manifold with boundary (\(\{\mathbf {a}_i\mathbf {b}_i\mathbf {c}_i\}\) if \(i< n-1\), \(\{ \mathbf {a}_{n-1}\mathbf {b}_{n-1}\mathbf {c}_{n-1}, \mathbf {a}_0\mathbf {b}_0\mathbf {c}_0\}\) if \(i= n-1\)). Thus successive uses of Theorem 10 imply that \(\partial (\cup _{i=0}^{n-1} P_i)\) is a 2-manifold. Now we use Corollary 4 using \(A= \cup _i P_i\) and obtain a 3-ball \(S'\subseteq S\) such that \(c(A)\cap c(S')= c(N)\) and \(c(\Delta )\cap c(\partial (A\cup S'))\) is not a 2-ball for every tetrahedron \(\Delta \in S'\).

Second we show that \(A_i= \cup _{j\ne i} P_j\cup S'\) and \(O= \cup _j P_j\cup S'\) are 3-balls. By successive uses of Theorem 14 (as in the previous case above), \(\partial (\cup _{j\ne i} P_j)\) is a 2-manifold that is homeomorphic to a 2-sphere. Then Theorem 14 implies that \(\partial A_i\) is a 2-sphere (use \(A= \cup _{j\ne i} P_j\) and \(B= S'\)). Now Theorem 14 implies that \(\partial O\) is a 2-sphere (use \(A= A_i\) and \(B= P_i\)). Since \(A_i\subseteq T\) and \(O\subseteq T\), Theorem 5 implies that \(A_i\) and O are 3-balls.

Last we show that O is strongly non-shellable. We already know that \(c(\Delta )\cap c(\partial O)\) is not a 2-ball for every tetrahedron \(\Delta \in S'\). We use Theorem 11 using \(A= A_i\) and \(P= P_i\) and see that \(c(\Delta )\cap c(\partial O)\) is not a 2-ball for every tetrahedron \(\Delta \in P_i\). \(\square \)

“Appendix 7” shows that there is a strongly non-shellable 3-ball as described in Theorem 15: the strongly non-shellable 3-ball Z with 12 vertices and 25 tetrahedra in Sect. 4 of [24] (Z is the symmetric version of the 3-ball with 10 vertices and 21 tetrahedra).

6.6 Other Shelling Blocking

After a lemma, Theorem 16 provides examples of residual 3-balls in the free space that cannot be carved by a shelling during the surface reconstruction.

Lemma 17

If a 3-ball \(O\subseteq T\) has at least two tetrahedra and \(\Delta \in O\), there is a tetrahedron \(\Delta '\in O{\setminus }\{\Delta \}\) such that \(\Delta \cap \Delta '\) is a triangle.

Proof

Assume (reductio ad absurdum) that \(\partial \Delta \subseteq \partial O\). Let \(O_0= T^\infty {\setminus } O\) and \(O_1= O_0\cup \{\Delta \}\). We have \(\Delta \in T{\setminus } O_0\) and \(c(O_0)\cap c(\Delta )= c(\partial O)\cap c(\partial \Delta )=c(\partial \Delta )\) is non-empty 2D-pure. Thus \((O_0,O_1)\) is a shelling. Since \(O_1= T^\infty {\setminus } (O{\setminus }\{\Delta \})\) and O has at least two tetrahedra, \(O_1\ne T^\infty \). Since \(\partial O_0\) is a 2-sphere and \(O_1\ne T^\infty \), Corollary 2 implies that \(c(O_0)\cap c(\Delta )= c(\partial \Delta )\) is a 2-ball (impossible). \(\square \)

Theorem 16

Let \(M\subseteq T^\infty \) such that \(\partial M\) is a 2-manifold. Let \(S\subseteq T{\setminus } M\) be a 3-ball such that \(c(M)\cap c(S)= c(N{\setminus } \{\mathbf {b}_0\mathbf {b}_1\mathbf {c}_0, \mathbf {b}_1\mathbf {c}_0\mathbf {c}_1 \})\) and N is an annulus (Eq. 22). Let \(P= p(\mathbf {a}_0\mathbf {b}_0\mathbf {c}_0,\mathbf {a}_1\mathbf {b}_1\mathbf {c}_1)\) such that \(P\subseteq T{\setminus } (M\cup S)\) and \(c(M)\cap c(P)= c(\mathbf {a}_0\mathbf {b}_0\mathbf {c}_0, \mathbf {a}_1\mathbf {b}_1\mathbf {c}_1)\) and \(c(S)\cap c(P)= c(\mathbf {b}_0\mathbf {b}_1\mathbf {c}_0, \mathbf {b}_1\mathbf {c}_0\mathbf {c}_1)\). Then there is a 3-ball \(S'\subseteq S\) such that \(c(M\cup P)\cap c(S')= c(M\cup P)\cap c(S)\) and, using notation \(O= T^\infty {\setminus } (M\cup P\cup S')\), \(\partial O\) is a 2-manifold, \(c(O)\cap c(\Delta )\) is not 2D-pure for every tetrahedron \(\Delta \in P\cup S'\) such that \(\partial \Delta \cap \partial O\ne \emptyset \).

Fig. 17

Two examples for Theorem 16: M includes a horizontal ground and a vertical wall (top) or includes two pillars on the ground (bottom). Left \(\partial M\) including triangles \(\mathbf {a}_0\mathbf {b}_0\mathbf {c}_0\) and \(\mathbf {a}_1\mathbf {b}_1\mathbf {c}_1\). Middle \(\partial M\) and S. Right \(\partial M\) and S and P

Two examples of M, S and P are shown in Fig. 17. The first one (on the top) is inspired by the real example in Fig. 1 of [16] such that M is “the matter” (set of the tetrahedra non-intersected by rays and including the ground and walls of a city), P, S and \(P\cup S\) are 3-balls that cannot be carved by a shelling although they are in the free space (set of the tetrahedra intersected by rays). The second one (on the bottom) is similar and is inspired by the real example in Fig. 3.4 of [10].

Proof

First we find \(S'\). Let \(A= M\cup P\). Since \(c(M)\cap c(P)\) is two disjoint 2-balls and both \(\partial M\) and \(\partial P\) are 2-manifolds and \(P\subset T{\setminus } M\), Theorem 10 implies that \(\partial A\) is a 2-manifold. Furthermore, \(c(A)\cap c(S)= (c(M)\cap c(S))\cup (c(P)\cap c(S))= c(N)\). Thus Corollary 4 implies that there is a 3-ball \(S'\subseteq S\) such that \(c(A)\cap c(S')= c(N)\) and \(c(\Delta )\cap c(\partial (A\cup S'))\) is not a 2-ball for every tetrahedron \(\Delta \in S'\).

Second we show that \(\partial O\) is a 2-manifold. Since both \(\partial A\) and \(\partial S'\) are 2-manifold and \(c(A)\cap c(S')\) is a 2-manifold with boundary (the annulus N), Theorem 10 implies that \(\partial (A\cup S')\) is a 2-manifold. Therefore \(\partial O= \partial (M\cup P\cup S')\) is a 2-manifold.

Third we show that \(\Delta \in P\cup S'\) implies that \(c(\Delta )\cap c(\partial O)\) is not a 2-sphere. If \(\Delta \in S'\) and since \(S'\) is a 3-ball with more that one tetrahedron, there is another tetrahedron \(\Delta '\in S'\) such that \(\Delta '\cap \Delta \) is a triangle (Lemma 17). We have a similar case if \(\Delta \in P\). In all cases, \(\Delta \) has a triangle face t between two tetrahedra in \(M\cup P\cup S'\). Since \(t\notin \partial O\), \(c(\Delta )\cap c(\partial O)\ne c(\partial \Delta )\). According to Theorem 3 and since \(\Delta \notin O\), \(c(\Delta )\cap c(\partial O)\) is not 2D-pure or is a 2-ball, but it is not a 2-sphere.

Last we conclude. We use Theorem 11 using \(A= M\cup S'\) and see that \(c(\Delta )\cap c(\partial O)\) is not a 2-ball for every tetrahedron \(\Delta \in P\). Now \(c(\Delta )\cap c(\partial O)\) is not a 2-ball for every tetrahedron \(\Delta \in P\cup S'\). Since \(c(\Delta )\cap c(\partial O)\) is neither 2-ball nor 2-sphere, \(c(\Delta )\cap c(\partial O)\) is not non-empty 2D-pure (Theorem 3). Since \(\partial \Delta \cap \partial O\ne \emptyset \), \(c(\Delta )\cap c(\partial O)\) is not 2D pure. \(\square \)

7 Conclusion

This article overviews shelling properties for the surface reconstruction application. In this context, a shelling is a series of tetrahedron sets \(O_0\cdots O_n\) in a 3D Delaunay triangulation T, generated by a greedy algorithm that adds one tetrahedron to every \(O_i\), such that all boundaries \(\partial O_i\) have the same topology in almost all cases.

If \(O_0\) includes only one tetrahedron, all \(O_i\) are 3-balls and Combinatorial Topology works study the shellability of a 3-ball O: is there a shelling such that \(O_n=O\)? Several cases are possible for greedy shelling algorithms that try to generate O: always success (O is extendably shellable), sometimes success (O is shellable but non-extendably shellable), and always failure (O is non-shellable). Furthermore, there are 3 invariant numbers (the h-numbers) for all shellings of O.

We also show that there is a shelling started from \(O_0\) and ended to O, if \(O_0\subset O\subseteq T\) and both \(O_0\) and O are star-shaped with respect to a same point. This generalizes a known result of shellability (if \(O= T\) and \(O_0\) has a single tetrahedron), and we use this property to qualitatively explain why the greedy algorithm does not have excessive failures and provides most of the tetrahedra enclosed by the final surface.

Last we provide a family of non-shellable 3-balls and examples of visual artifacts that a shelling alone cannot remove. Such visual artifacts occur in previous surface reconstruction results and are due to an incomplete growing of O in the free space. Actually non-shelling algorithms remove some of them, but future work is needed to remove all of them.

Supplementary material

10851_2017_734_MOESM1_ESM.pdf (142 kb)
Supplementary material 1 (pdf 143 KB)

References

  1. 1.
    Boissonnat, J.D.: Geometric structures for three-dimensional shape representation. ACM Trans. Graph. 3(4), 266–286 (1984)Google Scholar
  2. 2.
    Boissonnat, J.D., Devillers, O., Pion, S., Teillaud, M., Yvinec, M.: Triangulations in CGAL. Comput. Geom. Theory Appl. 22, 5–19 (2002)Google Scholar
  3. 3.
    Bruggesser, H., Mani, P.: Shellable decomposition of cells and spheres. Math. Scand. 29, 197–205 (1971)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Danaraj, G., Klee, V.: A representation of 2-dimensional pseudomanifolds and its use in the design of a linear-time shelling algorithm. Algorithmic aspects of combinatorics. Ann. Discrete Math. 2, 53–63 (1978)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    DeRose, T., Duchamp, T., Hoppe, H., McDonald, J.A., Stuetzle, W: Reconstructing two-dimensional manifolds from scattered data: motivations and background. Technical report 215, University of Washington (1991)Google Scholar
  6. 6.
    Diestel, R.: Graph Theory. Graduated Texts in Mathematics. Springer, Heidelberg (2010)Google Scholar
  7. 7.
    Edelsbrunner, H.: An acyclicity theorem for cell complexes in d dimensions. Combinatorica 10(3), 251–260 (1990)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Edelsbrunner, H.: Geometry and Topology for Mesh Generation. Cambridge University Press, Cambridge (2001)CrossRefMATHGoogle Scholar
  9. 9.
    Gallier, J.: Notes on convex sets, polytopes, polyhedra, combinatorial topology, voronoi diagrams and Delaunay triangulation. Research report RR-6379, INRIA (2007)Google Scholar
  10. 10.
    Gezahegne, A.: Surface reconstruction with constrained sculpting. Master of science thesis, University of California (2005)Google Scholar
  11. 11.
    Giblin, P.: Graphs, Surfaces and Homology, 3rd edn. Cambridge University Press, Cambridge (2010)CrossRefMATHGoogle Scholar
  12. 12.
    Hachimori, M.: Combinatorics of constructible complexes. PhD. Thesis, University of Tokyo (2000)Google Scholar
  13. 13.
    Lhuillier, M., Yu, S.: Manifold surface reconstruction of an environment from sparse structure-from-motion data. CVIU 117(11), 1628–1644 (2013)Google Scholar
  14. 14.
    Lhuillier, M.: 2-manifold tests for 3D Delaunay triangulation-based surface reconstruction. J. Math. Imaging Vis. 51(1), 98–105 (2015)Google Scholar
  15. 15.
    Litvinov, V., Lhuillier, M.: Incremental solid modeling from sparse and omnidirectional structure-from-motion data. British Machine Vision Conference, Bristol (2013)Google Scholar
  16. 16.
    Litvinov, V., Lhuillier, M.: Incremental solid modeling from sparse structure-from-motion data with improved visual artifacts removal. International Conference on Pattern Recognition, Stockholm (2014)Google Scholar
  17. 17.
    Lutz, F.H.: Combinatorial 3-manifold with 10 vertices. Beitrage zur Algebra und Geometrie 49(1), 97–106 (2008)MathSciNetMATHGoogle Scholar
  18. 18.
    Romanoni A., Matteucci M.: Incremental reconstruction of urban environments by edge-point Delaunay triangulation. International Conference on Intelligent Robots and Systems, Hamburg (2015)Google Scholar
  19. 19.
    Sakai, K.: Simplicial Homology: A Short Course. Institute of Mathematics, University of Tsukuba, Tsukuba (2010)Google Scholar
  20. 20.
    Varadhan G., Krishnan S., Sriram TVN., Manocha D.: Topology preserving surface extraction using adaptive subdivision. In: Eurographics Symposium on Geometry Processing (2004)Google Scholar
  21. 21.
    Vegter, G.: Computational topology. In: GoodMan, J., Rourke, J.O. (eds.) Handbook of Discrete and Computational Geometry. CRS Press, Boca Raton (2004)Google Scholar
  22. 22.
    Xia, J., He, Y., Han, S., Fu, C.W., Luo, F., Gu, X.: Parameterization of a star-shaped volumes using green’s functions. Lect. Notes Comput. Sci. 6130, 219–235 (2010)CrossRefGoogle Scholar
  23. 23.
    Yu S., Lhuillier M.: Incremental reconstruction of a manifold surface from sparse visual mapping. Joint 3DIM/3DPVT conference: 3D Imaging-Modeling-Processing-Visualization-Transmission, Zurich (2012)Google Scholar
  24. 24.
    Ziegler, G.M.: Shelling polyhedral 3-balls and 4-polytopes. Discrete Comput. Geom. 19, 159–174 (1998)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Ziegler, G.M.: Lectures on Polytope. Graduate Texts in Mathematics. Springer, Berlin (2007)Google Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.UMR 6602 CNRS/UCA/SIGMAInstitut PascalAubière CedexFrance

Personalised recommendations