On the Space Between Critical Points

Abstract

The vertices of the neighborhood graph of a digital picture P can be interpolated to form a 2-manifold M with critical points (maxima, minima, saddles), slopes and plateaus being the ones recognized by local binary patterns (LBPs). Neighborhood graph produces a cell decomposition of M: each 0-cell is a vertex in the neighborhood graph, each 1-cell is an edge in the neighborhood graph and, if P is well-composed, each 2-cell is a slope region in M in the sense that every pair of s in the region can be connected by a monotonically increasing or decreasing path. In our previous research, we produced superpixel hierarchies (combinatorial graph pyramids) that are multiresolution segmentations of the given picture. Critical points of P are preserved along the pyramid. Each level of the pyramid produces a slope complex which is a cell decomposition of M preserving critical points of P and such that each 2-cell is a slope region. Slope complexes in different levels of the pyramid are always homeomorphic. Our aim in this research is to explore the configuration at the top level of the pyramid which consists of a slope complex with vertices being only the critical points of P. We also study the number of slope regions on the top.

1 Introduction

Local binary patterns (LBPs) can efficiently recognize local configurations of digital images as the critical points (maxima, minima, saddles) as well as slopes and plateaus. Of the wide range of successful applications of LBPs the classification of textures, segmentation and feature extraction have shown the great potential of these simple features (see, for example, [11]). Adjacent LBPs are redundant through the inequalities shared by the two adjacent windows. This redundancy applies to all edges of monotonic paths along which all bits of the LBP are same. This fact has been used in our previous research [2, 3] to produce a hierarchy of successively smaller graphs (combinatorial graph pyramid) such that all critical points are preserved wile providing a multiresolution segmentations of an image. The main differences from most of previous multiresolution structures are that they preserve the critical points and they preserve textures and high frequency image features allowing image reconstructions with only a small percentage of regions.

At the top of this pyramid we expect only to have critical points and their adjacencies. Since our original graphs are planar and since our reduction algorithms preserve the embedding of the graphs in the plane we automatically generate also the dual graphs built by the regions between the critical points.

In [4], Edelsbrunner et al. presented algorithms for constructing a hierarchy of increasingly coarse Morse-Smale complexes that decompose a piecewise linear 2-manifold M. Morse-Smale regions are defined in the continuous setting and consist in the set of points of M that lie in integral lines with common origin and destination. In that paper, they simplify Morse-Smale complexes by canceling pairs of critical points in order of increasing persistence. Each region of the Morse-Smale complex is a quadrangle with vertices of index 0, 1, 2, 1, in this order around the region. The boundary is possibly glued to itself along vertices and arcs.

In this paper, we define the concept of slope complex which can be seen as an extension of the concept of Morse-Smale complex of Edelsbrunner et al [4]. We compute a combinatorial graph pyramid by merging slope regions only if the resulting region is also a slope. At the top of the pyramid, only slope regions bounded by critical points should remain. We study the possible configurations of slope regions at the top of the pyramid and give bounds for the number of that regions on the top.

The paper is organized as follows. Section 2 is devoted to recall the main concepts needed to understand the rest of the paper. In Sect. 3, we study the configuration of slope complexes which are cell decompositions with 2-cells being slope regions. Section 4 is the main section of the paper. In that section we study the structure (in terms of the number of 2-cells and configurations of the 2-cells) of the cell decomposition with vertices being only critical points. Last section is devoted to conclusions and future work.

2 Preliminaries

Given a grayscale digital picture $P=(\mathbb {Z}^2,F_P,g)$, being $F_P$ a finite subset of $\mathbb {Z}^2$, the intensity of a pixel p of $F_P$, denoted by g(p), is expressed within a given range between a minimum and a maximum. Without loss of generality we suppose that the range is [0, 255]. The Local Binary Pattern LBP(P) [10] is again a grayscale digital picture which represents the texture element at each pixel in P. This is currently the most frequently used texture descriptor¹ with outstanding results in applications ranging from object detection to segmentation and classification [11].

The vertices of the adjacency graph of a digital picture P can be interpolated to form a 2-manifold M with critical points (maxima, minima, saddles), slopes and plateaus being the ones that are recognized by LBPs.

In [2, 3], we proposed an equivalent LBP encoding which transfers the code from the pixels to the neighbor relations thus only using one bit per edge of the adjacency graph. That LBP encoding is the one that we follow in this paper.

Below, we provide the background needed to understand the rest of the paper.

Primal and Dual Graphs. A neighborhood graph, denoted by G, encodes the 4-adjacency of pixels of P. A vertex v of G is associated to each pixel p of P and it is labeled with the gray value of p, i.e., $g(v):=g(p)$. Vertices of neighboring pixels are connected by edges of G.

The dual graph of G is denoted by $\overline{G}$. The vertices of $\overline{G}$ are associated with the faces of G and the edges of $\overline{G}$ corresponds to the border between the two faces in G. In simple words there is there is a one-to-one correspondence between the edges of G and $\overline{G}$ so as the faces of G and vertices of $\overline{G}$ as mentioned in [9, Sect. 4.6].

Thus the operation edge contraction in G is equivalent to edge removal in $\overline{G}$. Conversely, edge removal in G is equivalent to edge contraction in $\overline{G}$. The graphs G and $\bar{G}$ are planar if they represent a 2D decomposition of M into regions. In the original picture P, a vertex v in G is a border vertex if the number of edges incident to it, denoted by deg(v) is 2 or 3. Otherwise ($deg(v)=4$), it is an inner vertex. A border edge connects two border vertices. Otherwise, it is an inner edge as mentioned in [9, Sect. 1.3].

Monotonically Increasing (Decreasing) Path. A path in G is a finite sequence of vertices $v_1, v_2\dots ,v_n$ and edges $(v_1,v_2)$, $(v_2,v_3)$, $\dots$, $(v_{n-1},v_n)$. A path in G is monotonically increasing (resp. decreasing) if $g(v_{i})\le g(v_{i+1})$ (resp. $g(v_{i})\ge g(v_{i+1})$), for $1\le i < n$. Two monotonically increasing (resp. decreasing) paths can be concatenated if both have the same orientation. Observe that if $g(v_{i})=g(v_{i+1})$, for $1 \le i < n$, then the path is a level curve. For simplicity we will use monotonic paths irrespective whether it is increasing or decreasing.

LBP Codes. The adjacent vertices of a vertex v in H are compared with v to obtain the LBP code of v. Given a vertex v in H, follow these adjacent vertices along a circle (for example, clockwise orientation). Whenever the gray value of v is greater than the gray value of the adjacent vertex w, i.e., when $g(w)<g(v)$, write LBP value 0. Otherwise (i.e., when $g(w)>g(w)$), write LBP value 1. Observe that the case $g(w)=g(v)$ is not considered when computing LBP codes.

If vertices of G are points (pixels) in $F_P$, then a direction can be associated to an edge (u, v) in G between vertices $v \ne u$ when $g(u)>g(v)$:

In this way LBP codes are moved from the vertices to the orientation of edges. As a result the LBP-codes are independent of the degree of the vertices. In this particular case, the one-to-one correspondence between edges in G and edges in $\bar{G}$ can be geometrically obtained by rotating $90^0$ (for example clockwise orientation). Then, we can associate a direction to an edge in $\bar{G}$ if the corresponding edge in G also has a direction.

Graph Pyramids. A (dual) irregular graph pyramid [6, 7] is a stack of successively reduced planar graphs $\mathcal{P} = \{(G_0,\bar{G}_0),\dots ,$ $(G_n,\bar{G}_n)\}$. Each level $(G_k,\bar{G}_k)$, $0 < k \le n$ is obtained by first contracting edges in selected contraction kernels of $G_{k-1}$, if the corresponding regions should be merged, and then removing edges in $G_{k-1}$ to simplify redundant structure (such as regions completely bordered by a self-loop or by two parallel edges).

Merging Plateaus. In order to characterize each vertex in the neighborhood graph in terms of its LBP code, we first contract edges in G (i.e., merge regions in $\bar{G}$) having the same gray value. Therefore, after merging plateaus, we obtain a new neighborhood graph denoted by H satisfying that $g(v)\ne g(w)$, for any two adjacent vertices v, w in H. Plateau regions may contain holes. They can degenerate when the width of the region shrinks to 0. The degenerate case is a level curve. We will contract plateau regions to a single vertex. Every hole in a plateau region generates a self-loop surrounding the hole after contraction.

Critical Vertices. A vertex v of H is a local maximum if its LBP code is composed just by 0s. A local minimum produces an LBP code only with 1s. The LBP code describes a slope vertex if there are exactly two transitions from 0s to 1s or 1s to 0s in the code, when traversed circularly. More transitions identify a saddle vertex. In the particular case of v being a border vertex, if the two border edges incident to v produce equal LBP values, it counts as an extra transition (in order to later get monotonic paths along the boundary of the slope complex, see next section). Self-loops are not taken into account when computing the LBP code. Since LBP codes can also be described by the orientation of their incident edges, we can also associate an LBP code to each vertex in $\bar{H}$. Besides, observe that self-loops in H are never oriented whereas self-loops in $\bar{H}$ can be oriented. In the case of a self-loop incident to a vertex $\bar{v}$ in H is oriented, it counts twice when computing the LBP code of $\bar{v}$.

Remark 1

Vertices in $\bar{H}$ can only be slopes and saddles vertices.

Non Well-Composed Configurations. A gray scale digital picture is called well-composed [8] if it does not contain the following non-well composed configuration of pixels (modulo reflection and 90-degree rotation). Look at the drawings on the left of Fig. 1 where $g(a)<g(b)$, $g(a)<g(c)$, $g(d)<g(b)$ and $g(d)<g(c)$, being a, b, c and d vertices in G.

Fig. 1.

Repairing non-well composed configurations.

In order to avoid the non-well composed configurations (see [2, 3]), we insert a new dummy vertex v adjacent to vertices a, b, c, d in G creating new four regions that correspond to four slope vertices in $\bar{G}$. The new vertex v with new gray value g(v) reflects the relation between a, b, c, d:

$$\begin{array}{rcl} g(v)&{}=&{}(g(a)+g(b)+g(c)+g(d))/2\\ &{}&{}-(\max \{g(a),g(b),g(c),g(d)\}+\min \{g(a),g(b),g(c),g(d)\})/2. \end{array}$$

From now on, to repair a non-well composed configuration means that such vertex v has been inserted.

Remark 2

After repairing all non-well composed configurations, we obtain a new neighborhood graph J for which all the vertices in $\bar{J}$ are slope vertices.

3 Slope Complexes

This section is devoted to introduce the most important concept of our paper: slope complexes. Roughly, a slope complex is a cell complex formed by slope regions. We provide a different definition of a slope region which helps us explore the top of the pyramid.

We can naturally associate an abstract cell complex K[G] to a neighborhood graph denoted by G, where the 0-cells of K[G] are the vertices of G, the 1-cells of K[G] are the edges (or loops) of G, and the 2-cells of K[G] are the regions of M (which correspond to vertices of $\bar{G}$). The border of a 2-cell is the set of edges (1-cells) delimiting such 2-cell. The boundary of the cell complex K[G], denoted by $\partial K[G]$ is the subcomplex of K[G] formed by the border vertices and the border edges of G. See [5] for more details.

Definition 1

A 2-cell in K[G] is a slope region if the corresponding vertex $\bar{v}$ in $\bar{G}$ is a slope vertex. Besides, K[G] is a slope complex if all its 2-cells are slope regions.

After merging plateau regions, we obtain a new cell complex K[H] which is homeomorphic to K[G] (see [1]). The following result provides us an important characterization for slope regions.

Proposition 1

A 2-cell in K[H] is a slope region if and only if there exist two vertices $u \ne v$ in K[H] such that the border of the 2-cell is composed exactly by two monotonic paths between u and v with the same orientation.

Proof

Let r be a 2-cell in K[H] and let u, v be two vertices of K[H]. Assume first that r is a slope region in K[H]. Then, the corresponding vertex $\bar{v}$ in $\bar{H}$ is a slope vertex. It means that there are exactly one transition from 0s to 1s and one from 1s to 0s in the LBP code for $\bar{v}$, when traversed circularly. Then, the corresponding edges in H can be grouped in two disjoint sets, say $E_1$ and $E_2$, where all the edges in each set have the same orientation. These two sets correspond to two monotonic paths. Hence, there exist two different vertices u and v in H which are connected by two monotonic paths. These two paths have the same orientation. By contradiction, suppose that one path is monotonically increasing and the other path is monotonically decreasing. Then, it holds $g(v_{i})< g(v_{i+1})$, for $1\le i \le m$, for the first path and $g(v_{i})> g(v_{i+1})$), for $1\le i \le n$, for the second path. Then, the union of both paths generates a directed cycle where $g(u) \le g(v)$ and also $g(u) \ge g(v)$, which is a contradiction because H contains no plateaus.

Suppose now that the border of r is composed exactly by two monotonic paths between two different vertices u and v with the same orientation. Then, the corresponding vertex $\bar{v}$ in $\bar{H}$ has $deg(\bar{v})$ (here, with loops counted twice) equal to the number of edges that are in both paths. Moreover, computing the LBP code of $\bar{v}$ we add bit 1 for each edge that corresponds to one path and 0 for each edge that correspond to the other. Hence, $\bar{v}$ is a slope vertex and r is a slope region.    $\square$

After repairing non-well composed configurations in H, we obtain a new cell complex K[J] which is homeomorphic² to K[H]. Let us highlight an important property for this level of the pyramid.

Proposition 2

The cell complex K[J] is a slope complex.

Proof

Let J be the new neighborhood graph obtained by repairing all the non-well composed configurations in H. Then, by Remark 2, every vertex in $\bar{J}$ is a slope vertex, so every 2-cell in K[J] is a slope region, and therefore, K[J] is a slope complex.    $\square$

In order to study the decomposition of a slope complex, we introduce the concept of minimal slope regions.

Definition 2

A minimal slope region is a slope region whose border has a length of three edges.

We differentiate between simple and non-simple minimal slope regions. A simple minimal slope region is bounded by three edges which does not contain self-loop (see Fig. 2a). A non-simple minimal slope region contains a self-loop (see Fig. 2b and c) but the dual face also has degree three.

Figure 2c exchanges the inside and the outside of the self-loop. To follow the boundary of the non-simple slope region showed in Fig. 2b, cross the edge AC twice while in Fig. 2c, the edge is split and forms a double edge. If the inside of the self-loop is empty, it is not a slope region because it is contracted in the construction of the graph pyramid when it appears. Consequently, the double edge does not surround any substructure and can be collapsed to a single edge. Hence the self-loop must be filled with a non-empty subgraph “?" (Fig. 2c).

Fig. 2.

Types of minimal slope regions

We are interested in finding tight bounds on the number of slope regions in a slope complex. We finish this section with an upper bound on the number of slope regions in a slope complex.

Denote by |F| the number of slope regions (faces), by |E| the number of edges, and by |V| the number of vertices of K[G]. Denote by B and I, the number of border edges and inner edges of $\partial K[G]$, respectively. Notice that $E = B \cup I$.

Since K[G] is a cellular decomposition of a given 2-manifold M (with boundary), it is possible to use the Euler characteristic formula in [9, Theorem 4.2.7] (without the outer-face): $|V|-|E|+|F|=1.$

The next result shows that the number of slope regions in a slope complex depends on the number of vertices and the length of the boundary of such slope complex.

Proposition 3

Let K[G] be a slope complex where all the slope regions are minimal. Then $|F| = 2|V| - |B| - 2$.

Proof

Let K[G] be a slope complex where all the slope regions are minimal. Notice that every inner edge bounds two minimal slope regions and every border edge bounds only one. Since $E = B \cup I$ and every slope region is bounded by three edges, then $2 |I| + |B| = 3 |F|.$ This allows us to express the number of edges depending on the number of edges on the boundary and the number of faces:

$2|I| + |B| = 3|F|; \,\,\, 2|I| + 2|B| = 3|F| + |B|; \,\,\, 2(|I| + |B|) = 3|F| + |B|; \,\,\, |E| = (|B| + 3 |F|)/2.$ Finally, the Euler-equation yields $|V|-(|B| + 3 |F|)/2 + |F| = 1$, from which we derive the number of slope regions $|F| = 2|V| - |B| - 2$.    $\square$

To see an example where $|F| = 2 |V| - |B| - 2$, it is sufficient to consider a neighborhood graph (denoted by G) where all the vertices in $\bar{G}$ are saddles. Then, after repairing all the non-well composed configurations in G, all the vertices in $\bar{G}$ are slope vertices and the corresponding regions in G are minimal slope regions.

As an immediate consequence, next result follows.

Corollary 1

Let K[G] be a slope complex. Then $|F| \le 2|V| - |B| - 2$, with equality if and only if all the slope regions are minimal.

4 Critical Slope Complex

Our goal is to describe the slope complex on the top of the pyramid. As said in the introduction each cell of the complex should only contain critical vertices in its border.

Definition 3

A critical slope region is a slope region with only critical vertices in its border.

Definition 4

Given a neighborhood graph, denoted by G, a slope complex homeomorphic to K[G] with only critical slope regions, is called critical slope complex.

The following result gives an upper bound on the number of 2-cells in a given critical slope complex.

Proposition 4

A critical slope complex with |V| critical vertices, |W| of them being in the boundary of the complex, has at most $2|V|-|W|-2$ critical slope regions.

Proof

The result is a direct consequence of Corollary 1. Observe that since all the vertices in a critical slope complex are critical, the number of critical vertices in the boundary is equal to the number of edges in the boundary of the complex.

   $\square$

4.1 On the Number of 2-cells in a Critical Slope Complex

Now, let us study the number of 2-cells in a critical slope complex, depending on the number of edges in the border of each 2-cell.

Proposition 5

A critical slope complex with |V| critical vertices, |W| of them being in the boundary of the complex, has at most $2(|V| - 1) - |W|$ critical slope regions.

Proof

Let F denote the number of 2-cells of the critical slope complex (excluding the outer-face), $F_d$ the number of 2-cells with d edges in their border and |E| the number of edges. Clearly $F = \sum _{d\ge 3} F_d$ (since special cases of $d \in \{ 1,2 \}$ are removed in the construction of the pyramid). Therefore, $2 |E| - |W| = \sum _{d\ge 3} d F_d.$ Every inner edge $E \setminus W$ bounds two different faces and the edges W on the boundary is part of only one face. Expressing the number of edges |E| of the critical slope complex in terms of the edges on the boundary and the faces yields $|E| = (|W| + \sum _{d\ge 3} d F_d)/2$. Combining it with the Euler characteristic equation we get

$$\begin{array}{rcl} 1 &{}=&{} |V| - (|W| + \sum \nolimits _{d\ge 3} d F_d)/2 + \sum \nolimits _{d\ge 3} F_d \\ 2 &{}=&{} 2|V| - |W| - \sum \nolimits _{d\ge 3} d F_d + 2 \sum \nolimits _{d\ge 3} F_d = 2|V| - |W| - \sum \nolimits _{d\ge 3} (d-2) F_d \end{array} $$

and then $\sum _{d\ge 3} (d-2) F_d = 2(|V| - 1) - |W|.$    $\square$

In other words, we can now calculate the number of faces depending on the total number of critical vertices of the critical slope complex and number of critical vertices of its boundary with the choice of the number of edges in the border of the 2-cells.

For example, assume that we have only critical slope regions with 3 or 4 edges in their border. Then $F_3 + 2F_4 = 2(|V| - 1) - |W|$. So the total number of critical slope regions becomes $F_3 + F_4 = 2(|V| - 1) - |W| - F_4$. That means that the number of critical slope regions is reduced by $F_4$ with respect to its upper bound. That motivates the study of slope configurations with respect to the number of edges in their border since this would reduce the total number of critical slope regions necessary to cover the complete area.

4.2 Critical Vertices in the Border of a Critical Slope Region

In this subsection we explore how the critical vertices are distributed along the border of a critical slope region.

Proposition 6

A critical slope region cannot contain more than one minimum or more than one maximum in its border.

Proof

Let us proceed by contradiction, assume that there are two maxima in the border of a slope region. Then, the corresponding vertex of such region in $\bar{G}$ is a saddle since there are at least four transitions in its LBP code.    $\square$

As an immediate consequence of the previous result, next we describe all the possible critical slope regions. Other different configurations are not possible because they would produce no slope vertices in the dual graph, which contradicts the slope region definition. First, we focus on the critical slope regions which are minimal. The smallest critical slope region is a minimal slope region with only critical vertices. The configurations comes from to the minimal slope regions depicted in Fig. 2.

A simple minimal slope region can be formed by two saddles and one extremum. The alternating sequence of orientation of the edges attached to a saddle would allow a minimal slope region with three saddles. However in this case the triangle would form an oriented cycle which contradicts the transitivity of the inequalities (if $A>B$ and $B>C$ then $A>C$). Then, the only possibility for a minimal slope region with only critical vertices is to contain all three types of critical points (Fig. 3a).

For non-simple minimal slope regions, there are two options: Fig. 2b or c. In Fig. 2b, it is a critical slope region if the inner vertex B in Fig. 2b is an extremum and the vertex A is either the opposite extremum or a saddle. In Fig. 2c, the inside of the self-loop cannot be empty, then the vertex B have to be an extremum and the vertex A the opposite extremum or a saddle (see examples in Fig. 3b and c).

Fig. 3.

Types of minimal slope regions with critical vertices in their border

Now we describe the critical slope regions which are not minimal, that is, those which have more than three edges in their border. By Proposition 6, the only possibilities are the generalized triangle and generalized quadrangle (see Fig. 4).

Fig. 4.

Critical slope regions which are not minimal.

4.3 Configurations of Critical Slope Regions

We explore now possible configurations of critical slope regions which can compose a critical slope complex.

First, we combine only simple critical slope regions, it means to combine simple triangles. Then, the possible configurations are those a saddle is surrounded by an even and alternating sequences of maxima and minima, or a maximum (minimum) is surrounded by alternating sequences of minima (maxima) and saddles (see Fig. 5). Notice that by combining critical slope regions with minimum number of edges produce the maximum number of 2-cells in the critical slope complex.

Fig. 5.

Combination of simple critical slope regions, denoted by T.

Now let us study possible configurations that can be obtained by combining non-simple critical slope regions shown in Fig. 2b and c.

• 1. Combination of self-loop inside a double edge anchored at an extremum: Region $S_+$ in Fig. 6a contains on the outside a self-loop attached to a maximum. This self-loop is required in order to match the outside configuration. Hence it can only be a self-loop anchored at a maximum, another region $S_+$. The case of $S_-$ in Fig. 6b is symmetric to $S_+$. This combination contains on the outside an alternating sequence of maximum and minimum. Again we have only a single primitive configuration that can encapsulate $S_-$ and the result is an outer boundary with alternating minima and maxima.

• 2. In Fig. 7, the self-loops attached to saddles show a simple ($S_{\times +}, S_{\times -}$) and a more complex configuration involving two more simple critical slope regions. The simple configurations can be combined similar to the self-loops at extrema:

  In this case $S_{\times +}$ and $S_{\times -}$ complement each other to yield as outside a double edge between the saddle and the extremum. The pending edge must be complemented by the opposite extremum. It can be completed by a cycle around $\otimes$ similar to the simple triangles T in Fig. 5. The complex configurations $(S_{\times +},T,T), (S_{\times -},T,T)$ have on the outside a self-loop attached to the saddle. The self-loop of $S_{\times +}$ can be encapsulated into a $S_{\times -}$-configuration and the self-loop $S_{\times -}$ into a $S_{\times +}$- configuration. In both cases the outside is a double edge connecting two different extrema. Hence it can be combined easily with any of the other configurations with alternating extrema on the outside.

• 3. Alternatively, any of the configurations bounded by a double edge can be recursively embedded into a self-loop generating one more of the above primitive self-loop configurations. The completion towards the outside is then analogously as in the primitive configurations.

Fig. 6.

Combination of non-simple critical slope regions (two maxima and two minima).

Fig. 7.

Combination of non-simple critical slope regions (a saddle with maximum $S_{\times +}$ and minimum $S_{\times -}$).

5 Conclusion and Future Work

In this paper, our starting point is the neighborhood graph associated to a grayscale digital picture $P=({\mathbb Z}^2,F_P,g)$ with critical vertices computed from the LBP code of $F_P$. These critical vertices coincide with the ones of the 2-manifold obtained by interpolating the points of $F_P$. We study the structure (in terms of the number of 2-cells and configurations of the 2-cells) of the critical cell decomposition K of M in the sense that the vertices in the border of each 2-cell in K are critical.

As future work we plan to show that the minimum number of slope regions in a slope complex is the one of the critical slope complex and that critical slope complexes are in some sense unique with respect to the initial configuration of the critical points of P. In this sense, critical slope complexes can be seen as a cell-complex representation of the image P.