Hierarchical stack filtering: a bitplane-based algorithm for massively parallel processors

Abstract

With the development of novel parallel architectures for image processing, the implementation of well-known image operators needs to be reformulated to take advantage of the so-called massive parallelism. In this work, we propose a general algorithm that implements a large class of nonlinear filters, called stack filters, with a 2D-array processor. The proposed method consists of decomposing an image into bitplanes with the bitwise decomposition and then processes every bitplane hierarchically. The filtered image is reconstructed by simply stacking the filtered bitplanes according to their order of significance. Owing to its hierarchical structure, our algorithm allows us to trade-off between image quality and processing time, and to significantly reduce the computation time of low-entropy images. Also, experimental tests show that the processing time of our method is substantially lower than that of classical methods when using large structuring elements. All these features are of interest to a variety of real-time applications based on morphological operations such as video segmentation and video enhancement.

Appendices

Appendix 1: Derivation of Eq. (5)

The bitwise decomposition function, stated in (2), can be expressed in terms of floor functions using the modulo operator $x \mod y = x - y \left\lfloor {x}/{y} \right\rfloor$ and the identity $\left\lfloor {\left\lfloor {x}/{p} \right\rfloor }/{q} \right\rfloor = \left\lfloor {x}/{pq} \right\rfloor$ as follows

$$\begin{aligned} B_{k}= \left\lfloor \frac{I}{2^{k}} \right\rfloor - 2\left\lfloor \frac{I}{2^{k+1}} \right\rfloor . \end{aligned}$$

(16)

In general, a floor function has a staircase shape, where every step can be expressed in terms of threshold functions. Therefore, we can state that

$$\begin{aligned} \left\lfloor \frac{I}{p} \right\rfloor = \sum _{j} j\left( T_{jp}-T_{(j+1)p}\right) , \end{aligned}$$

(17)

where p is an integer number that determines the stair width. Consequently, the floor functions in (16) can be expressed as

$$\begin{aligned} \left\lfloor \frac{I}{2^{k}} \right\rfloor= & {} \sum _{j' = 0}^{N_{k}} j'\left( T_{j'2^{k}}-T_{(j'+1)2^{k}}\right) , \end{aligned}$$

(18)

$$\begin{aligned} \left\lfloor \frac{I}{2^{k+1}} \right\rfloor= & {} \sum _{j = 0}^{N_{k+1}} j\left( T_{j2^{k+1}} - T_{(j+1)2^{k+1}} \right) , \end{aligned}$$

(19)

where $N_{k}=2^{K-k}-1$ and K represents the bit depth of the image. As the stair width in (19) is two times larger than in (18), the indices j and $j'$ point to different thresholding levels. Therefore, to use a common index, we express (18) with the index j as follows

$$\begin{aligned} \left\lfloor \frac{I}{2^{k}} \right\rfloor = \sum _{j = 0}^{N_{k+1}} 2j \left( T_{j2^{k+1}} - T_{(2j+1)2^{k}} \right) + (2j+1)\left( T_{(2j+1)2^{k}} - T_{(j+1)2^{k+1}} \right) . \end{aligned}$$

(20)

By substituting (19) and (20) into (16), we obtain the relation between decompositions as

$$\begin{aligned} B_{k} = \sum _{j = 0}^{N_{k+1}} T_{(2j+1)2^{k}} - T_{(j+1)2^{k+1}}. \end{aligned}$$

(21)

Equivalently, we can express this relation in terms of Boolean operators as shown below

$$\begin{aligned} B_{k} = \bigvee _{j = 0}^{N_{k+1}} \left( T_{(2j+1)2^{k}}\wedge \lnot T_{(j+1)2^{k+1}}\right) . \end{aligned}$$

(22)

Appendix 2: Derivation of Eq. (6)

As stated in (1), the threshold function is based on the comparison of the image I and a gray level $\ell$. The equality comparator, on the other hand, is a more fundamental operation defined as shown below

$$\begin{aligned} C_{\ell }(n,m) = \left\{ \begin{array}{ll} 1,\quad \mathrm {if} \ I(n,m) = \ell ,\\ 0,\quad \mathrm {otherwise.} \end{array} \right. \end{aligned}$$

(23)

If we apply the bitwise decomposition to I and $\ell$, we get $\left( B_{K-1},\ldots ,B_{0}\right) _{2}$ and $\left( \beta _{K-1,\ell },\ldots ,\beta _{0,\ell }\right) _{2}$, respectively. By comparing their K bit levels of decomposition, we can also express the equality comparator as follows

$$\begin{aligned} C_{\ell }(n,m) = \bigwedge _{k=0}^{K-1} B_{k}(n,m)\leftrightarrow \beta _{k,\ell }. \end{aligned}$$

(24)

Based on equality comparisons, we can reformulate the threshold function (1) as shown below

$$\begin{aligned} T_{l}(n,m) = \bigvee _{\ell =l}^{2^{K}-1} C_{\ell }(n,m), \end{aligned}$$

(25)

where $C_\ell$ can be substituted by (24) leading to

$$\begin{aligned} T_{l}(n,m) = \bigvee _{\ell =l}^{2^{K}-1} \bigwedge _{k=0}^{K-1} \left( B_{k}(n,m)\leftrightarrow \beta _{k,\ell }\right) . \end{aligned}$$

(26)

This Boolean function has the canonical form of a sum of products (SoP) [13], and it can be minimized yielding a simplified representation. In the following paragraphs, we will perform the minimization of (26) by assuming that $T_{l}$ returns the minimized expression and $F_{l}(B_{K-1},B_{K-2},\ldots ,B_{0})$ is the sum-of-products representation. In this way, we start applying the Shannon’s expansion theorem [14] to $F_{l}$ as follows

$$\begin{aligned} F_{l}(B_{K-1},B_{K-2},\ldots ,B_{0}) = \left( B_{K-1} \wedge F_{l}({\mathbf {1}},B_{K-2},\ldots ,B_{0})\right) \vee \left( \lnot B_{K-1} \wedge F_{l}({\mathbf {0}},B_{K-2},\ldots ,B_{0})\right) , \end{aligned}$$

(27)

where ${\mathbf {1}}$ and ${\mathbf {0}}$ represent bitplanes of ones and zeros, respectively. This expression can be reduced for two possible cases:

Case 1 :

If $l < 2^{K-1}$, then $F_{l}\left( {\mathbf {1}},B_{K-2},\ldots ,B_{0}\right) = {\mathbf {1}}$ by the uniting theorem [13]. As a result, we can reduce (27) to $F_{l}\left( B_{K-1},B_{K-2},\ldots ,B_{0}\right) = B_{K-1} \vee (\lnot B_{K-1} \wedge F_{l}({\mathbf {0}},B_{K-2},\ldots ,B_{0}))$, and then use the elimination theorem [13] to get

$$\begin{aligned} F_{l}\left( B_{K-1},B_{K-2},\ldots ,B_{0}\right) = B_{K-1} \vee F_{l}\left( {\mathbf {0}},B_{K-2},\ldots ,B_{0}\right) . \end{aligned}$$

(28)

Case 2 :

If $l\ge 2^{K-1}$, then $F_{l}\left( {\mathbf {0}},B_{K-2},\ldots ,B_{0}\right) = {\mathbf {0}}$. For this case, Eq. (27) is reduced to

$$\begin{aligned} F_{l}\left( B_{K-1},B_{K-2},\ldots ,B_{0}\right) = B_{K-1} \wedge F_{l}\left( {\mathbf {1}},B_{K-2},\ldots ,B_{0}\right) . \end{aligned}$$

(29)

By gathering (28) and (29) into a single function, we can express (27) as follows

$$\begin{aligned} T_{l} = {\left\{ \begin{array}{ll} B_{K-1} \vee F_{l}({\mathbf {0}},B_{K-2},\ldots ,B_{0}), &{} \mathrm {if} \ \lambda _{K-1} = 0, \\ B_{K-1} \wedge F_{l}({\mathbf {1}},B_{K-2},\ldots ,B_{0}), &{} \mathrm {if} \ \lambda _{K-1} = 1, \end{array}\right. } \end{aligned}$$

(30)

where $\lambda _{K-1}$ is the most significant bit of l given that $(l)_{10}=(\lambda _{K-1}\lambda _{K-2}\ldots \lambda _{0})_{2}$. Note that $\lambda _{K-1} = 0$ implies that $l < 2^{K-1}$, while $\lambda _{K-1} = 1$ implies that $l\ge 2^{K-1}$. This piecewise function can be redefined more compactly as follows

$$\begin{aligned} T_{l} = B_{K-1}\underset{\lambda _{K-1}}{\bowtie } F_{l}\left( \varvec{\lambda }_{K-1},B_{K-2},\ldots ,B_{0}\right) , \end{aligned}$$

(31)

where $\varvec{\lambda }_{K-1}$ represents a bitplane of either zeros or ones depending on the value of $\lambda _{K-1}$, while the bow tie operator is defined by

$$\begin{aligned} \underset{x}{\bowtie } \ = \left\{ \begin{array}{ll} \vee , \quad \mathrm {if} \;x=0, \\ \wedge , \quad \mathrm {if} \;x=1. \end{array} \right. \end{aligned}$$

(32)

The expression in (31) can be further worked out by applying the Shannon’s expansion theorem to $F_{l}(\varvec{\lambda }_{K-1},B_{K-2},\ldots ,B_{0})$, which leads to

$$\begin{aligned} F_{l}\left( \varvec{\lambda }_{K-1},B_{K-2},\ldots ,B_{0}\right)&= \left( B_{K-2} \wedge F_{l}\left( \varvec{\lambda }_{K-1},{\mathbf {1}},B_{K-3},\ldots ,B_{0}\right) \right) \vee \nonumber \\&\quad\left( \lnot B_{K-2} \wedge F_{l}\left( \varvec{\lambda }_{K-1},{\mathbf {0}},B_{K-3},\ldots ,B_{0}\right) \right) . \end{aligned}$$

(33)

This expression can be simplified for four possible cases:

Case 1 :

If $l<2^{K-2}$, then $F_{l}\left( {\mathbf {0}},{\mathbf {1}},B_{K-3},\ldots ,B_{0}\right) = {\mathbf {1}}$, which after substituted in (33) we get

$$\begin{aligned} F_{l}\left( {\mathbf {0}},B_{K-2},\ldots ,B_{0}\right) = B_{K-2} \vee F_{l}\left( {\mathbf {0}},{\mathbf {0}},B_{K-3},\ldots ,B_{0}\right) . \end{aligned}$$

(34)

Case 2 :

If $l\ge 2^{K-2}$, then $F_{l}({\mathbf {0}},{\mathbf {0}},B_{K-3},\ldots ,B_{0})= {\mathbf {0}}$, which reduces (33) to

$$\begin{aligned} F_{l}\left( {\mathbf {0}},B_{K-2},\ldots ,B_{0}\right) = B_{K-2} \wedge F_{l}\left( {\mathbf {0}},{\mathbf {1}},B_{K-3},\ldots ,B_{0}\right) . \end{aligned}$$

(35)

Case 3 :

If $l<3\cdot 2^{K-2}$, then $F_{l}\left( {\mathbf {1}},{\mathbf {1}},B_{K-3},\ldots ,B_{0}\right) = {\mathbf {1}}$, and the expansion is simplified as

$$\begin{aligned} F_{l}\left( {\mathbf {1}},B_{K-2},\ldots ,B_{0}\right) = B_{K-2} \vee F_{l}\left( {\mathbf {1}},{\mathbf {0}},B_{K-3},\ldots ,B_{0}\right) . \end{aligned}$$

(36)

Case 4 :

If $l\ge 3\cdot 2^{K-2}$, then $F_{l}({\mathbf {1}},{\mathbf {0}},B_{K-3},\ldots ,B_{0})= {\mathbf {0}}$, which after substituted in (33) returns

$$\begin{aligned} F_{l}\left( {\mathbf {1}},B_{K-2},\ldots ,B_{0}\right) = B_{K-2} \wedge F_{l}\left( {\mathbf {1}},{\mathbf {1}},B_{K-3},\ldots ,B_{0}\right) . \end{aligned}$$

(37)

All these cases are put together in the following form:

$$\begin{aligned} T_{l}=\left( B_{K-1}\underset{\lambda _{K-1}}{\bowtie }\left( B_{K-2}\underset{\lambda _{K-2}}{\bowtie }F_{l}\left( \varvec{\lambda }_{K-1},\varvec{\lambda }_{K-2},B_{K-3},\ldots ,B_{0}\right) \right) \right) \end{aligned}$$

(38)

In the light of (31) and (38), we can easily deduce the last Shannon’s expansion as follows

$$\begin{aligned} T_{l}=\left( B_{K-1}\underset{\lambda _{K-1}}{\bowtie }\left( B_{K-2}\underset{\lambda _{K-2}}{\bowtie }\cdots \left( B_{0}\underset{\lambda _{0}}{\bowtie } F_{l}\left( \varvec{\lambda }_{K-1},\varvec{\lambda }_{K-2},\ldots ,\varvec{\lambda }_{0}\right) \right) \right) \right) \end{aligned}$$

(39)

Given that $F_{l}(\varvec{\lambda }_{K-1},\varvec{\lambda }_{K-2},\ldots ,\varvec{\lambda }_{0})={\mathbf {1}}$, then Eq. (39) can be simplified if the least significant bits of l are set to zero. For instance, if z is the number of the least significant bits of l set to zero, then $\lambda _{z-1}=\lambda _{z-2}=\cdots =\lambda _{0}=0$ and the rightmost part of (39) is reduced as follows $B_{z-1} \vee B_{z-2} \vee \cdots \vee {\mathbf {1}}={\mathbf {1}}$. Finally, the minimization of the sum of products stated in (26) is shown below

$$\begin{aligned} T_{l}=\left( B_{K-1}\underset{\lambda _{K-1}}{\bowtie }\left( B_{K-2}\underset{\lambda _{K-2}}{\bowtie }\cdots \left( B_{K-q+1}\underset{\lambda _{K-q+1}}{\bowtie } B_{K-q}\right) \right) \right) \end{aligned}$$

(40)

where $q = K - z$.