A Klein-Bottle-Based Dictionary for Texture Representation

Abstract

A natural object of study in texture representation and material classification is the probability density function, in pixel-value space, underlying the set of small patches from the given image. Inspired by the fact that small $n\times n$ high-contrast patches from natural images in gray-scale accumulate with high density around a surface $\fancyscript{K}\subset {\mathbb {R}}^{n^2}$ with the topology of a Klein bottle (Carlsson et al. International Journal of Computer Vision 76(1):1–12, 2008), we present in this paper a novel framework for the estimation and representation of distributions around $\fancyscript{K}$, of patches from texture images. More specifically, we show that most $n\times n$ patches from a given image can be projected onto $\fancyscript{K}$ yielding a finite sample $S\subset \fancyscript{K}$, whose underlying probability density function can be represented in terms of Fourier-like coefficients, which in turn, can be estimated from $S$. We show that image rotation acts as a linear transformation at the level of the estimated coefficients, and use this to define a multi-scale rotation-invariant descriptor. We test it by classifying the materials in three popular data sets: The CUReT, UIUCTex and KTH-TIPS texture databases.

Appendix: Main Results and Proofs

Proof

(Proposition 1)

• 1. The Cauchy-Schwartz inequality implies that for every $\mathbf {v}\in S^1$ and every almost-everywhere differentiable $I_P$ (not necessarily purely directional) one has that

  $$\begin{aligned}Q_P(\mathbf {v}) \le \int \int \limits _{[-1 ,1]^2} \Vert \nabla I_P \Vert ^2 dxdy.\end{aligned}$$

  Since the equality holds when $I_P(x,y) = g(ax + by)$ and $\mathbf {v} = \begin{bmatrix}a\\ b\end{bmatrix}$, the result follows.

• 2. Let $\lambda _{max} \ge \lambda _{min} \ge 0$ be the eigenvalues of $A_P$, and let $B_P$ be a unitary matrix such that

  $$\begin{aligned}A_PB_P = \begin{bmatrix}\lambda _{max}&0 \\ 0&\lambda _{min}\end{bmatrix}.\end{aligned}$$

  If $\mathbf {v}\in S^1$ and $\mathbf {v} = B_P\begin{bmatrix}v_x \\ v_y\end{bmatrix}$, then $1 = \Vert \mathbf {v}\Vert ^2 = v_x^2 +v_y^2$, $Q_P(\mathbf {v}) = \lambda _{max} v_x^2 + \lambda _{min}v_y^2$ and therefore

  $$\begin{aligned}\max _{\Vert \mathbf {v}\Vert =1}Q_P(\mathbf {v})= \lambda _{max}.\end{aligned}$$

  Finally, since $Q_P(\mathbf {v}) = \lambda _{max}$ for $\mathbf {v}\in S^1$, if and only if $\mathbf {v} \in E_{max}(A_P)$, we obtain the result.

• 3. If the eigenvalues of $A_P$ are distinct, then $E_{max}(A_P)$ is one dimensional and thus intercepts $S^1$ at exactly two antipodal points.$\square$

Proof

(Proposition 2) Since $u$ and $\sqrt{3}u^2$ are orthonormal with respect to $\langle \cdot , \cdot \rangle _D$, then it follows that

$$\begin{aligned} \mathop {\text {argmin }}\limits _{c^2 + d^2 = 1} \varPhi (c,d)&= \mathop {\text {argmin }}\limits _{c^2 +d^2 =1} \varPhi (c,d)^2\\&= \mathop {\text {argmax }}\limits _{c^2 + d^2 =1} \left\langle I_P , cu + d\sqrt{3}u^2 \right\rangle _D \\&= \mathop {\text {argmax }}\limits _{c^2 + d^2 =1} \left\langle \begin{bmatrix}c \\ d\end{bmatrix}, \begin{bmatrix}\langle I_P ,u \rangle _D \\ \\ \langle I_P, \sqrt{3}u^2\rangle _D\end{bmatrix} \right\rangle \end{aligned}$$

which can be found via the usual condition of parallelism, provided $\varphi (I_P,\alpha ) \ne 0$. $\square$

Proof

(Proposition  3) Let $g\in L^2(T)$ and consider

$$\begin{aligned} g^\perp (z,w)= \frac{g(z,w) + g\left( -z,-\overline{w}\right) }{2} \end{aligned}$$

It follows that $g^\perp$ is square-integrable on $T$, and that:

• 1. $g^\perp \left( -z,-\overline{w}\right) = g^\perp (z,w)$ for every $(z,w)\in T$. Hence $g^\perp \in L^2(K)$ for every $g\in L^2(T)$.

• 2. If $g_1,g_2\in L^2(T)$ and $a_1,a_2\in {\mathbb {C}}$ then

  $$\begin{aligned} (a_1g_1 + a_2g_2)^\perp = a_1(g_1^\perp ) + a_2 (g_2)^{\perp } \end{aligned}$$

• 3. $g^\perp = g$ for every $g\in L^2(K)$.

We claim that $g^\perp$ is the orthogonal projection of $g\in L^2(T)$ onto $L^2(K)$, and all we have to check is that $g -g^\perp$ is perpendicular to every $h\in L^2(K)$. To this end, let us introduce the notation

$$\begin{aligned} g^*(z,w) = g(-z,-\overline{w}). \end{aligned}$$

By writing the inner product of $L^2(T)$ in polar coordinates, using the substitution $(\alpha ,\theta )\mapsto (\alpha + \pi ,\pi - \theta )$, and the fact that $h$ satisfies Eq. 3.3, one obtains that

$$\begin{aligned} \langle g^*, h\rangle _T&= \frac{1}{(2\pi )^2} \int _{-\frac{\pi }{2}}^{\frac{3\pi }{2}} \int _{\frac{\pi }{4}}^{\frac{9\pi }{4}} g(\alpha + \pi ,\pi -\theta )\overline{h}(\alpha ,\theta ) d\alpha d\theta \\&= \frac{1}{(2\pi )^2} \int _{-\frac{\pi }{2}}^{\frac{3\pi }{2}} \int _{\frac{\pi }{4}}^{\frac{9\pi }{4}} g(\alpha ,\theta )\overline{h} (\alpha + \pi ,\pi -\theta )d\alpha d\theta \\&= \frac{1}{(2\pi )^2} \int _{-\frac{\pi }{2}}^{\frac{3\pi }{2}} \int _{\frac{\pi }{4}}^{\frac{9\pi }{4}} g(\alpha ,\theta )\overline{h} (\alpha ,\pi )d\alpha d\theta =\langle g,h \rangle _T \end{aligned}$$

Therefore $2\langle g - g^\perp , h\rangle _T = \langle g, h\rangle _T - \langle g^*, h\rangle _T = 0$ and we get the result. $\square$

Proof

[Theorem 1] By continuity, $\varPi$ takes spanning sets to spanning sets which is the first part of the theorem. Now, if we consider the decomposition

$$\begin{aligned} L^{2}(T) = L^{2}(K)\oplus L^{2}(K)^{\perp } \end{aligned}$$

where $L^2(K)^\perp$ denotes the orthogonal linear complement of $L^2(K)$ in $L^2(T)$, and $\fancyscript{B}$ is an orthonormal basis for $L^2(K)$, then for any orthonormal basis $\fancyscript{B}^\prime$ of $L^2(K)^\perp$ we have that $\fancyscript{B}\cup \fancyscript{B}^\prime$ is an orthonormal basis for $L^2(T)$. The thing to notice is that since $\ker (\varPi ) = L^2(K)^\perp$ and $\varPi$ restricted to $L^2(K)$ is the identity, then $\varPi (\fancyscript{B}^\prime ) = \{\mathbf {0}\}$ and therefore $\varPi (\fancyscript{B}\cup \fancyscript{B}^\prime ) = \fancyscript{B}\cup \{\mathbf {0}\}$. It follows that the only subset of $\fancyscript{B}\cup \{\mathbf {0}\}$ which can be a basis for $L^2(K)$ is $\fancyscript{B}$, and that it is invariant when applying Gram-Schmidt.

Proof

(Theorem  3) If $f,g\in L^2(K,{\mathbb {R}})$ then from a change of coordinates it follows that $\langle f^\tau , g \rangle _K = \langle f, g^{-\tau }\rangle _K$, and therefore

$$\begin{aligned} a_m^\tau&= a_m \\ b_n^\tau&= \langle f^\tau , \sqrt{2}\cos (2n\alpha )\rangle _K \\&= \langle f, \sqrt{2}\cos (2n(\alpha + \tau )) \rangle _K \!=\! \cos (2n\tau ) b_n \!-\! \sin (2n\tau ) c_n \\ c_n^\tau&= \langle f, \sqrt{2} \sin (2n(\alpha + \tau )) \rangle _K \\&= \cos (2n\tau ) c_n + \sin (2n\tau )b_n \end{aligned}$$

Similarly

$$\begin{aligned} d_{n,m}^\tau&= \cos (n\tau )d_{n,m} -\sin (n\tau )e_{n,m} \\ e_{n,m}^\tau&= \cos (n\tau )e_{n,m} +\sin (n\tau )d_{n,m}. \end{aligned}$$

$\square$

Corollary 4

Let $T_\tau :\ell ^2({\mathbb {R}}) \longrightarrow \ell ^2({\mathbb {R}})$ be as in Theorem 3. Then

$$\begin{aligned} T_{\tau + \beta } = T_\tau \circ T_\beta . \end{aligned}$$

Proof

(Theorem 4) From the description of $T_\tau$ as a block diagonal matrix whose blocks are rotation matrices (Theorem 3), it follows that

$$\begin{aligned} \mathop {\text {argmin }}\limits _{\tau } \varPsi (-\tau )&= \mathop {\text {argmax }}\limits _{\tau } \left\langle K\fancyscript{F}_w(f), T_\tau \left( K\fancyscript{F}_w(g)\right) \right\rangle \\&= \mathop {\text {argmax }}\limits _{\tau } \sum _{n=1}^{w} x_n \cos (n\tau ) + y_n\sin (n\tau ) \end{aligned}$$

where $x_n$ and $y_n$ depend solely on $K\fancyscript{F}_w(f)$ and $K\fancyscript{F}_w(g)$. By taking the derivative with respect to $\tau$, we get that if $\tau ^*$ is a minimizer for $\varPsi (\tau )$ then we must have

$$\begin{aligned} \sum _{n=1}^w ny_n \cos (n\tau ^*) - nx_n\sin (n\tau ^*) = 0 \end{aligned}$$

which is equivalent to having

$$\begin{aligned} Re\left( \sum _{n=1}^w n(y_n + i x_n)\xi _*^n\right) = 0. \end{aligned}$$

Let $q(z)= \sum \limits _{n=1}^w n(y_n + ix_n)z^n,\; \bar{q}(z)= \sum \limits _{n=1}^w n(y_n - ix_n)z^n$ and

$$\begin{aligned} p(z) = z^{w}\cdot \left( q(z) + \bar{q}\left( \frac{1}{z}\right) \right) . \end{aligned}$$

It follows that $p(z)$ is a complex polynomial of degree less than or equal to $2w$, and so that

$$\begin{aligned} p\left( \xi _*\right)&= \xi _*^w \left( q\left( \xi _*\right) + \bar{q}\left( \frac{1}{\xi _*}\right) \right) = \xi _*^w \left( q\left( \xi _*\right) + \bar{q}\left( \overline{\xi _*}\right) \right) \\&= 2\xi _*^w \cdot Re\left( q\left( \xi _*\right) \right) = 0. \end{aligned}$$

$\square$

Corollary 5

The vector $T_\tau \left( \widehat{K\fancyscript{F}}(f)\right)$ is a componentwise unbiased estimator for $K\fancyscript{F}(f^\tau )$, which converges almost surely as the sample size tends to infinity.

Proof

(Proposition  4)Let $\widehat{\mathbf {v}}^\tau$ be the vector with entries $\widehat{d}_{1,1}^\tau$ and $\widehat{e}_{1,1}^\tau$ for $\widehat{K\fancyscript{F}}(f^\tau ,S^\tau )$. It follows from Theorem 3 and Corollaries 4 and 5 that

$$\begin{aligned} \widehat{\mathbf {v}}^\tau&= \begin{bmatrix}\cos (\tau )&-\sin (\tau ) \\ \sin (\tau )&\cos (\tau )\end{bmatrix} \mathbf {\widehat{v}} \\&= \begin{bmatrix}\cos \left( \tau -\sigma (f)\right)&-\sin \left( \tau -\sigma (f)\right) \\ \sin \left( \tau -\sigma (f)\right)&\cos \left( \tau -\sigma (f)\right) \end{bmatrix} \begin{bmatrix}\Vert \mathbf {\widehat{v}}\Vert \\ \\ 0\end{bmatrix}\\&= \begin{bmatrix}\cos \left( \tau -\sigma (f)\right)&-\sin \left( \tau -\sigma (f)\right) \\ \sin \left( \tau -\sigma (f)\right)&\cos \left( \tau -\sigma (f)\right) \end{bmatrix} \begin{bmatrix}\left\| \mathbf {\widehat{v}}^\tau \right\| \\ \\ 0\end{bmatrix} \end{aligned}$$

from which we get $\sigma \left( f^\tau \right) \equiv \sigma (f) - \tau \;\; (\mathrm {mod}\;\;2\pi )$, and

$$\begin{aligned} T_{\sigma \left( f^\tau \right) }\left( \widehat{K\fancyscript{F}}(f^\tau ,S^\tau )\right)&= T_{\sigma (f) - \tau } \circ T_\tau \left( \widehat{K\fancyscript{F}}(f,S)\right) \\&= T_{\sigma (f)} \left( \widehat{K\fancyscript{F}}(f,S)\right) \end{aligned}$$

as claimed. $\square$