A Multi-Orientation Analysis Approach to Retinal Vessel Tracking

Abstract

This paper presents a method for retinal vasculature extraction based on biologically inspired multi-orientation analysis. We apply multi-orientation analysis via so-called invertible orientation scores, modeling the cortical columns in the visual system of higher mammals. This allows us to generically deal with many hitherto complex problems inherent to vessel tracking, such as crossings, bifurcations, parallel vessels, vessels of varying widths and vessels with high curvature. Our approach applies tracking in invertible orientation scores via a novel geometrical principle for curve optimization in the Euclidean motion group SE(2). The method runs fully automatically and provides a detailed model of the retinal vasculature, which is crucial as a sound basis for further quantitative analysis of the retina, especially in screening applications.

Appendices

Appendix A: A Mathematical Underpinning of Optimization in the e _η −e _θ Tangent Planes $\mathcal{V}$

Before we provide the key-geometrical principle behind our ETOS algorithm, we make some comments on the moving frame of reference that lives in the domain of an orientation score.

The domain of an orientation requires a curved (non-Euclidean) geometry as can be seen in Fig. 1. To this end we identify the coupled space of positions and orientations $\mathbb{R} ^{2}\rtimes S^{1}$ with SE(2). This needs a bit of explanation. To see whether a local line element $(\mathbf{x}',\theta') \in \mathbb{R} ^{2}\times S^{1}$ with position $\mathbf{x}'\in\mathbb{R}^{2}$ and orientation θ′ is aligned with its neighbors one applies a rigid body motion g=(x,R _θ )∈SE(2), with counter-clockwise rotation and translation via Eq. (10). This puts a 1-to-1 correspondence between rigid body motions (x,y,R _θ ) and elements from $\mathbb{R}^{2}\times S^{1}$ as every local line element (x,y,θ) can be obtained by a local line element (0,0,0) positioned at the origin 0=(0,0) pointing in say x-direction:

$$(\mathbf{x},\theta) = (\mathbf{x},R_{\theta}) (\mathbf{0},0) \quad {\Leftrightarrow}\quad (\mathbf {x},\theta) \equiv(\mathbf{x},R_{\theta}). $$

Via this identification we see that the curved domain of an orientation score is due to the non-commutative group structure of the rigid body motion (also known as roto-translation group or Euclidean motion group) SE(2) in the plane. Indeed, a concatenation of two rigid body motions is again a rigid body motion and this induces the group-product given by Eq. (11)

It is a semi-direct product of rotations SO(2) and translations $\mathbb{R}^{2}$, as the rotation part affects the position part. As a result this group product is non-commutative, i.e. gg′ can be very different from g′g. Therefore it is conceptually wrong to consider the domain of an orientation scores as the flat Cartesian space $\mathbb{R}^{2} \times S^{1}$ with Euclidean metric and it is wrong to just take derivatives of orientation scores only in ∂ _x direction or only in ∂ _y direction. For formal underpinning of this statement see [19, Thm. 21], for a short illustration see [33, Fig. 2.6]. Instead one must differentiate along the moving frame of reference {∂ _θ ,∂ _ξ ,∂ _η } with

$$\partial_{\theta}, \partial_{\xi}=\cos\theta \partial_{x} + \sin \theta\partial_{y},\qquad \partial_{\eta}= -\sin\theta\partial_{x} +\cos\theta \partial_{y}. $$

This a priori moving frame of reference coincides with the so-called Lie algebra of left-invariant vector fields on the group SE(2). The curved geometry in the orientation score requires us to apply parallel transport along this moving frame of reference. For instance in Fig. 1d we see examples of auto-parallel curves whose tangent vector is always constant to the moving frame of reference. The non-commutative structure of SE(2) (due to the fact that rotations and translations do not commute) is reflected by the commutators in the Lie-algebra

$$[\partial_{\theta},\partial_{\xi}]=\partial_{\eta}, \quad [\partial_{\theta},\partial_{\eta}]= -\partial_{\xi}, \quad [\partial_{\xi},\partial_{\eta}]= 0, $$

where [A,B]=AB−BA. As a result first moving in ξ direction and then moving in θ direction brings us to a different point in $\mathit{SE}(2)\equiv\mathbb{R}^{2} \times S^{1}$ than first moving in θ direction and then moving in ξ direction (as can be deduced in Fig. 1b).

1.1 A.1 The Geometrical Principle Behind the ETOS-Algorithm

The ETOS-algorithm presented in Sect. 3.1.1 heavily relies on local optimization in each transversal 2D-tangent plane $\mathcal{V}=\textrm{span}\{\partial_{\theta}, \partial_{\eta}\}$ spanned by ∂ _θ and ∂ _η =−sinθ∂ _x +cosθ∂ _y in tangent-bundle

$$T\bigl(\mathit{SE}(2)\bigr)= \bigcup_{g \in \mathit{SE}(2)} T_{g}\bigl(\mathit{SE}(2) \bigr), $$

where T _g (SE(2)) denotes the tangent space at g = (x,y,θ)∈SE(2).

To this end we recall Fig. 7 where one can observe that each tangent vector to the lifted curve s↦(x(s),y(s),θ(s)) with $\theta(s)=\arg(\dot{x}(s)+i \dot{y}(s))$ (e.g. the curve following the right edge of a blood vessel) in SE(2) is pointing orthogonal to a plane $\mathcal{V}$ (plotted in yellow). Within $\mathcal{V}$ we see that the maximum values of the absolute value of the imaginary part of the score is located at the origin of the yellow plane in the tangent space.

Definition 1

A smooth curve s↦γ(s)=(x(s),y(s),θ(s)) in SE(2) is called the lifted curve of a smooth planar curve iff for all s∈[0,ℓ] we have $\theta(s)=\arg(\dot{x}(s) +i \, \dot{y}(s))$.

Definition 2

If a curve γ is equal to the lift of its spatial projection (i.e. if its satisfies Eq. (46)) it is called horizontal.

Tangent vectors to horizontal curves always lay in the tangent plane H=span{∂ _θ ,∂ _ξ :=cosθ∂ _x +sinθ∂ _y }, spanned by ∂ _ξ and ∂ _θ , since one has

$$ \begin{aligned}[b] & \theta(s)=\arg\bigl(\dot{x}(s) +i \dot{y}(s)\bigr) \quad {\Leftrightarrow} \\ & \dot{\gamma}(s) \in\textrm{span}\{\partial_{\xi} \vert _{\gamma(s)},\partial_{\theta}\}. \end{aligned} $$

(46)

In this appendix we will underpin and discuss our fundamental venture point: The most salient curves in the smooth imaginary part/real part/absolute value $C:\mathit{SE}(2) \to \mathbb{R}$ of the smooth orientation score $U:\mathit{SE}(2)\to\mathbb{C}$ are given by

$$\gamma\textrm{ is horizontal such that } \partial_{\theta}C(\gamma )=0 \textrm{ and } (\partial_{\eta} \vert _{\gamma }C) (\gamma)=0. $$

The intuitive idea behind this is as follows. Let $C:\mathit{SE}(2)\mapsto\mathbb{R}^{+}$ denote an a priori given cost. Now, if a horizontal curve s↦γ(s):=(x(s),θ(s)) satisfies

$$\begin{aligned} \partial_{\eta} C\bigl( \mathbf{x}(s),\theta(s)\bigr) =& \bigl(-\sin\theta(s) \partial _{x}C + \cos\theta\partial_{y} C\bigr) \bigl( \mathbf{x}(s),\theta(s)\bigr) \\ =& \partial_{\theta}C\bigl(\mathbf{x}(s),\theta(s)\bigr)=0 \end{aligned}$$

(47)

with $\partial_{\eta}^{2} C(\mathbf{x}(s),\theta(s)) <0$ and $\partial _{\theta}^{2} C(\mathbf{x}(s),\theta(s)) <0$, then there is no gain in moving⁶ the curve s↦γ(s)=(x(s),θ(s)) in directions orthogonal to the spatial propagation vector ∂ _ξ | _γ(s)=cosθ(s)∂ _x +sinθ(s)∂ _y .

In fact, we expect the curve γ(s)=(x(s),θ(s)) satisfying (47) to be optimal in some sense within the sub-Riemannian manifold

$$M=\bigl(\mathit{SE}(2), \textrm{span}\{\partial_{\xi},\partial_{\theta} \}, G_{\beta}\bigr) $$

with metric tensor $G_{\beta}:\mathit{SE}(2) \times T(\mathit{SE}(2)) \times T(\mathit{SE}(2)) \to\mathbb{R}$ is given by

$$ \begin{aligned}[b] & G_{\beta} \vert _{(x,y,\theta)} \\ &\quad = {\rm d}\theta\otimes{\rm d}\theta \\ &\qquad {}+\beta^{2}(\cos\theta{\rm d }x + \sin\theta{\rm d}y) \otimes(\cos\theta{\rm d }x + \sin\theta{\rm d}y), \end{aligned} $$

(48)

where ⊗ denotes the tensor product. Note that

$$ \begin{aligned}[b] & (\cos\theta{\rm d }x +\sin\theta{\rm d}y) \bigl( \dot{\gamma}(s)\bigr)= \dot{x}(s) \cos\theta(s) +\dot{y}(s)\sin\theta(s) \\ &\quad {\Rightarrow}\quad G_{\beta}\bigl(\dot{\gamma}(s),\dot{\gamma}(s)\bigr)= \beta^{2}\bigl| \cos\theta(s)\dot{x}(s) +\sin\theta(s) \dot{y}(s)\bigr|^2 \\ & \quad \phantom{{\Rightarrow}\quad G_{\beta}\bigl(\dot{\gamma}(s),\dot{\gamma}(s)\bigr)=} {}+\bigl|\dot{\theta}(s)\bigr|^{2}. \end{aligned} $$

If s equals arclength (i.e. we have $\|\dot{\mathbf{x}}(s)\| ^{2}=1$) of the spatial part s↦x(s)=(x(s),y(s)) of the horizontal curve s↦γ(s)=(x(s),θ(s)) we have $\kappa ^{2}(s)=|\dot{\theta}(s)|^{2}$, so that

$$ \bigl\| \dot{\gamma}(s)\bigr\| = \sqrt{G_{\beta}\bigl(\dot{ \gamma}(s),\dot{\gamma}(s)\bigr)}= \sqrt{\kappa ^{2}(s) + \beta^{2}}. $$

(49)

Intuitively, such a sub-Riemannian manifold M equals the group SE(2) where one restricts oneself to horizontal curves with a constant relative penalty for bending and stretching determined by β>0 which has physical dimension one over length.

For special cases of C we can show that our geometrical principle indeed produces optimal curves in SE(2), as we will show in the subsequent section.

1.2 A.2 Application of the Geometrical Principle to Completion Fields

In general the real part, imaginary part, or absolute value of an orientation score is a complicated function on SE(2). Therefore we will consider the case where $C: \mathit{SE}(2) \to\mathbb{R}^{+}$ is a so-called “completion field” [5, 6, 59, 60, 67]. This corresponds to collision probability densities of a source particle g ₁∈SE(2) and a sink particle g ₂∈SE(2).

There exist remarkable relations [22, 24] between optimal curves (i.e. curves minimizing an optimal control problem on SE(2)) and solutions of Eq. (47) for special cases where C denotes a so-called completion distribution (or “completion field”). Given two sources at the origin g ₁=(0,0,0) and at g ₂=(x ₁,y ₁,θ ₁), such completion fields are defined as products of resolvent Green’s functions of stochastic processes for contour completion [47] and contour enhancement [18, 23] in SE(2):

$$ C(g):= \lambda^{2} \cdot R_{\lambda} \bigl(g_{1}^{-1}g\bigr) R_{\lambda,*} \bigl(g_{2}^{-1}g\bigr), $$

(50)

where R _λ (g) denotes the probability density of finding a random walker g in the underlying stochastic process [23, 47] given that it started at g=(x,y,θ)=(0,0,0) regardless its memoryless traveling time T which is negatively exponentially distributed with expectation E(T)=λ ⁻¹. Furthermore, g↦R _λ,∗(g) denotes the adjoint resolvent kernel (i.e. the resolvent that arises by taking the adjoint of the generator [24, Ch. 4.4].

For a concise overview on (Green’s functions of ) contour completion and enhancement see [21]. Exact formulas for the resolvent Green’s function R _λ for contour enhancement can be found in [23], whereas exact formulae for resolvent kernels R _λ for contour completion (direction process) can be found in [24]. In both cases there are representations involving 4 Mathieu functions. For a visual impression of exact Green’s functions see Fig. 20 (where in dashed lines we have depicted level sets of the corresponding Heisenberg approximations that we will discuss and employ in the next subsection).

Fig. 20

Top: A contour completion process and the corresponding Green’s function (in SE(2)) and its planar θ-integrated version. Bottom: A contour enhancement process the corresponding Green’s function (in SE(2)) and its planar θ-integrated version. In the completion process one has randomness in θ with variance 2D ₁₁>0 and non-random advection in ξ. In the enhancement process one has randomness both in θ-direction (with variance 2D ₁₁>0) and in ξ-direction (with variance 2D ₂₂>0)

These completion fields relate to the well-known Brownian bridges (in probability theory) where the traveling time is integrated out. This relation is relevant, since it is known that such Brownian bridges concentrate on geodesics [62].

If $G_{t}:\mathit{SE}(2) \to\mathbb{R}^{+}$ denotes the time dependent Green’s function of the Fokker-Planck equation of the underlying time dependent stochastic process [21, 23, 47] at time t>0 we have

$$C(g)= \lambda^2 \!\int_{0}^{\infty} \!\int_{0}^{t} G_{t-s}\bigl(g_{1}^{-1}g\bigr) e^{-\lambda(t-s)} G_{s}\bigl(g_{2}^{-1}g\bigr) e^{-\lambda s} {\rm d}s {\rm d}t . $$

This identity follows from two facts. Firstly, the resolvent Green’s function follow from the time dependent Green’s function via Laplace transform with respect to time

$$R_{\lambda}(g)= \lambda\mathcal{L}\bigl(t \mapsto G_{t}(g) \bigr) (\lambda) = \lambda\int_0^\infty G_t(g) e^{-\lambda t}dt. $$

Secondly, a temporal convolution relates to a product in the Laplace domain. As Brownian bridge measures concentrate on geodesics when λ→0, cf. [62], [20, App. B], the completion field for contour enhancement concentrates on sub-Riemannian geodesics (shortest distance curves) within M.

Such a sub-Riemannian geodesic γ=(x,θ) connecting g ₁ and g ₂ (with g ₁ and g ₂ sufficiently close⁷) is the lifted curve of the minimizer to the following optimal control problem

$$ \begin{aligned}[b] & \min_{\substack{\mathbf{x} \in C^{1}([0,\ell],\mathbb{R}^{2}), \\ \ell>0, \\ (\mathbf{x}(0),\dot{\mathbf{x}}(0))=g_1, \\ (\mathbf{x}(\ell),\dot{\mathbf{x}}(\ell))=g_2 }} \int_{0}^{\ell} \sqrt{\kappa^{2}(s)+\beta^{2}}\, {\rm d}s \\ &\quad = \min_{\substack{ \gamma=(\mathbf{x},\theta) \in C^{1}([0,\ell], \mathit{SE}(2)),\\ \ell>0, \\ (\mathbf{x}(0),\dot{\mathbf{x}}(0))=g_1, \\ (\mathbf{x}(\ell),\dot{\mathbf{x}}(\ell))=g_2, \\ \theta(s)= \arg(\dot{x}(s)+i \dot{y}(s)) \\ }} \int_{0}^{\ell} \sqrt{G_{\beta}\bigl(\dot{\gamma}(s),\dot {\gamma}(s)\bigr)}\, {\rm d}s \end{aligned} $$

(51)

with spatial arc-length parameter s>0 and with free total length ℓ>0 and with curvature $\kappa(s)=\|\ddot {x}(s)\|$ and where metric tensor G _ξ is defined by Eq. (48).

On the other hand, in his paper [47] Mumford showed that the modes of the direction process (also known as the contour completion process) coincide with elastica curves which are the solutions to the following optimal control problem

$$ \inf_{\substack{ \mathbf{x} \in C^{1}[0,\ell], \ell>0 \\ (\mathbf{x}(0),\dot{\mathbf{x}}(0))=g_0, \\ (\mathbf{x}(l),\dot{\mathbf{x}}(\ell))=g_1 }} \int _{0}^{\ell} \kappa^{2}(s)+ \beta^{2}\, {\rm d}s $$

(52)

Similar to the Onsager-Machlup approach to optimal paths [57] he obtains these modes by looking at the most probable/likely realization of discretized versions of the direction process.

As this cannot be (efficiently) realized in practice, one needs a more tangible description of the mode. To this end, we will call solution curves of (47) the “modes”. In case of the direction process and in case one uses the completion distribution (Eq. (50)) for the function C(g), the solution curves of (47) indeed seem to numerically coincide with the elastica. This experimentally underpins our conjecture in [24], where we have shown such a result does hold exactly for the corresponding Heisenberg approximations as we will explain next.

1.3 A.3 In the Heisenberg Approximation of Completion Fields Our Approach Produces B-Splines

The Heisenberg group approximation (obtained by contraction [23]) of the Green’s functions and induced completion field arises by replacing the moving frame of left-invariant vector fields

$$\{\cos\theta\partial x + \sin\theta\partial_y,-\sin\theta \partial_{x} +\cos\theta\partial_{y},\partial_{\theta} \} $$

on SE(2) by the moving frame of reference of left-invariant vector fields on the Heisenberg group

$$\{\partial_{x}+ \theta\partial_{y},\partial_{y}, \partial_{\theta }\} $$

and by replacing spatial arc-length parametrization via s by spatial coordinate x. Intuitively, such replacement boils down to replacing the space of positions and orientations by the space of positions and velocities.

When contracting (for details on this contraction see [23, Ch. 5.4]) our fundamental equation (47) with cost C given by Eq. (50) towards the Heisenberg group H ₃ we obtain

$$ \frac{d}{dy} \tilde{C}(x,y,\theta) = \frac{d}{d\theta} \tilde {C}(x,y,\theta) =0, $$

(53)

where again in the Heisenberg group each tangent plane {∂ _θ ,∂ _y } is orthogonal to the propagation direction ∂ _x +θ∂ _y and where the completion field is the product of two resolvent Green’s functions

$$ \begin{aligned} & \tilde{C}(x,y,\theta) \\ &\quad = R_{\lambda}(x,y,\theta) R_{\lambda} \bigl(-x+x_{1},y-y_{1}-\theta_{1}(x-x_{1}),- \theta\bigr) \end{aligned} $$

which can be derived in exact form as

$$R_{\lambda}(x,y,\theta)= \frac{\lambda\sqrt{3}}{2 D_{11} \pi x^2} e^{-\lambda x}e^{-\frac{3(x\theta-2y)^2 +x^2\theta^2 }{4x^3 D_{11}}} {\rm u}(x), $$

where D ₁₁>0 stands for the amount of diffusion in θ-direction and ${\rm u}$ for the unit step function, for details⁸ see (cf. [24, Thm. 4.6]).

Interestingly, the solution of (53) is a third order polynomial y(x) naturally lifted to (the corresponding sub-Riemannian manifold within) H(3) by setting θ(x)=y′(x). The solutions of Eq. (53) are therefore cubic B-splines which are the solutions of the Euler-Lagrange equation

$$y^{(4)}(x)=\theta^{(3)}(x)=0 $$

of a curve optimization problem which arises by contracting (52) towards the Heisenberg group. This gives the following “Heisenberg group equivalent” of control problem (52):

$$ \begin{aligned} & \min_{\substack{ \gamma(\cdot)=(\cdot, y(\cdot),\theta(\cdot)) \in C^{1}(H_{3}), \\ \gamma(0)=(0,0,0), \\ \gamma(x_{2})=(x_2, y_{2},\theta_{2}), \\ \theta(x)=y'(x), }} \int_{0}^{x_{2}} \beta^2+\bigl|\theta'(x)\bigr|^{2} \, {\rm d}x \\ &\quad =\beta^{2} x_{2}+ \frac{4(3y^2_{2}+3x_{2}y_{2}\theta_{2}+ x_{2}^{2}\theta_{2}^{2})}{x^{3}_2}, \end{aligned} $$

which takes the minimum along a lifted cubic-B spline (x,y(x),y′(x)) (with y(x) a third order polynomial matching the boundary conditions). For details see [24, Eq. (4.6.2)] and [20, Ch. 9.1.1]. See Fig. 21.

Fig. 21

The intersection of the planes $\{(x,y,\theta) \in\mathbb{R}^{3} \mid \partial_{\theta}\tilde{C}(x,y,\theta)=0\}$ and $\{(x,y,\theta) \in\mathbb{R}^{3} \mid \partial_{y}\tilde {C}(x,y,\theta)=0\}$ of the Heisenberg approximation $\tilde{C}(g)$ of the completion field C(g) given by Eq. (50), produces a cubic B-spline lifted in H(3), i.e. (x,y(x),θ(x)=y′(x)) with y ⁽⁴⁾(x)=0. Boundary conditions have been set to x ₁=y ₁=0, θ ₁=0.4, x ₂=2, y ₂=0, θ ₂=−0.4, D ₁₁=1/8

1.4 A.4 Concluding Remarks

By the results of the previous section the conjecture rises whether elastica curves coincide with Eq. (50) as such relation holds for their counterparts in the Heisenberg group. Numeric computations seem to provide a confirmation of this conjecture, see Fig. 22.

Fig. 22

The intersection of the planes {g∈SE(2)∣∂ _θ C(g)=0} (depicted in red) and {g∈SE(2)∣∂ _η C(g)=0} (depicted in yellow) of the exact completion field g=(x,y,θ)↦C(g) given by Eq. (50), with x ₁=y ₁=θ ₁=0 and x ₂=3, y ₂=−1, θ ₂=7/4π λ=0.1, D ₁₁=1/32. The intersection of these planes seems to coincides with an elastica (with β ²=4λD ₁₁), which we plotted in dashed red in the top figure and by white balls in the bottom figures (Color figure online)

On the one hand, we expect with respect to the contour enhancement process that our approach produces the sub-Riemannian geodesics (based on the results in [20, 22, 62]), but this is a point for future investigation. On the other hand, the conjecture together with the result in [47] (and the result that B-splines solve Eq. (53)) would underpin our alternative applicable definition of modes as solution curves of Eq. (47).

In fact this means that optimization in each (η,θ)-plane $\mathcal{V}$, as is done in our ETOS algorithm, produces the most probable curves (in the sense of [47, 57]) in direction processes.

Appendix B: The Closure of the Distributional Orientation Score Transform is an $\mathbb{L}_{2}$-Isometry on the Whole $\mathbb{L}_{2}(\mathbb {R}^{2})$ Space

As mentioned in Sect. 2.1, the condition given by Eq. (6) violates the condition $\psi\in\mathbb{L}_{1}(\mathbb{R}^{2}) \cap\mathbb{L}_{2}(\mathbb {R}^{2})$, as for such ψ, the function $M_{\psi}:\mathbb{R}^{2} \rightarrow\mathbb{R}^{+}$ is a continuous function vanishing at infinity. In fact this follows by [53, Thm. 7.5], Fubini’s theorem and compactness of SO(2).

To avoid technicalities one can restrict oneself a priori to bandlimited/disklimited images f, but this imposes a depending on sampling rate and induced Nyquist frequency ϱ. However, as we will show in this section such a restriction to band limited images is not crucial.

Akin to the unitary Fourier transform $\mathcal{F}:\mathbb {L}_{2}(\mathbb{R}^{2}) \rightarrow\mathbb{L}_{2}(\mathbb{R}^{2})$, whose kernel k(ω,x)=e ^−iω⋅x is not square integrable, we can allow non-square integrable kernels and rely on Gelfand-Triples [63] as we will explain next.

To this end we first drop the constraint $\psi\in\mathbb {L}_{1}(\mathbb{R}^{d}) \cap\mathbb{L}_{2}(\mathbb{R}^{d})$ by imposing ψ to be in a dual Sobolev-space:

$$ \psi\in\mathbb{H}_{-I}\bigl(\mathbb{R}^2\bigr) = \mathbb{H}_I^*\bigl(\mathbb {R}^2\bigr) = \bigl( \mathcal{D}(D_\beta)\bigr)^*, $$

(54)

where

$$\begin{aligned} \begin{aligned}[b] \mathbb{H}_I\bigl(\mathbb{R}^2\bigr) & = \mathcal{D}(D_\beta) \\ & = \bigl\{ f \in\mathbb{L}_2\bigl( \mathbb{R}^2\bigr) \mid f \mbox{ admits generalized derivatives} \\ & \phantom{{}= \bigl\{ } \mbox{s.t. } D_\beta f~\in~\mathbb {L}_2\bigl(\mathbb{R}^2\bigr) \bigr\} \end{aligned} \end{aligned}$$

(55)

equipped with inner product

$$(\cdot,\cdot)_{\mathbb{H}_I(\mathbb{R}^2)}=(D_\beta\cdot, D_\beta \cdot)_{\mathbb{L}_2(\mathbb{R}^2)} $$

and where $\beta:\mathbb{R}^{2} \mapsto\mathbb{R}^{+}$ is continuous, bounded from below, isotropic, and with differential operator

$$D_\beta= \mathcal{F}^{-1} \beta\mathcal{F}. $$

Now D _β is an unbounded self adjoint, positive operator with bounded inverse. This means that D _β defines a so-called Gelfand-Triple

$$\mathbb{H}_I\bigl(\mathbb{R}^2\bigr) \hookrightarrow \mathbb{L}_2\bigl(\mathbb {R}^2\bigr) \hookrightarrow \mathbb{H}^*_I\bigl(\mathbb{R}^2\bigr) = \mathbb {H}_{-I}\bigl(\mathbb{R}^2\bigr), $$

where $\mathbb{H}_{-I}(\mathbb{R}^{2})$ is equipped with inner product

$$(\cdot,\cdot)_{\mathbb{H}_{-I}(\mathbb{R}^2)} = \bigl(D_\beta^{-1} \cdot,D_\beta^{-1} \cdot\bigr)_{\mathbb{L}_2(\mathbb{R}^2)}, $$

and where all embeddings are dense.

Subsequently, we define the distributional orientation score transform ⁹

$$(\mathfrak {W}_\psi f) (g) = \langle \psi, \mathcal{U}_{g^{-1}} f \rangle , $$

for all $f \in\mathbb{H}_{I}(\mathbb{R}^{2})$ and all g∈SE(2), where we applied the notation 〈b,a〉=b(a) for functional b acting on vector a. Note that our non-distributional orientation score transform can be rewritten as

$$(\mathcal{W}_\psi f) (g) = (\mathcal{U}_g \psi, f) = \bigl(\psi,\mathcal {U}_g^* f\bigr) = (\psi, \mathcal{U}_{g^{-1}} f). $$

Under a certain condition on ψ, we show that operator $\mathfrak {W}$ is an isometry from $\mathbb{H}_{I}(\mathbb{R}^{2})$ (with $\mathbb {L}_{2}$-norm) into $\mathbb{L}_{2}(\mathit{SE}(2))$. Therefore this operator is closable and its closure is an isometry. This bring us to the main result.

Theorem 1

Let $\psi\in\mathbb{H}_{-I}(\mathbb{R}^{2})$. If $M_{D_{\beta}^{-1}\psi} = \beta^{-2}$ then $\mathfrak {W}_{\psi}$ maps $\mathcal {D}(D_{\beta})$ isometrically (w.r.t. $\mathbb{L}_{2}$-norm) onto a closed subspace of $\mathbb{L}_{2}(\mathit{SE}(2))$. Moreover, this operator is closable and its isometric closure is given by $D_{\beta}\mathcal {W}_{D_{\beta}^{-1}\psi}:\mathbb{L}_{2}(\mathbb{R}^{2}) \rightarrow \mathbb{L}_{2}(\mathit{SE}(2))$, where $\mathcal{W}_{D_{\beta}^{-1}\psi}$ is the normed non-distributional orientation score transform, w.r.t. kernel $D_{\beta}^{-1} \psi\in\mathbb{L}_{2}(\mathbb{R}^{2})$.

Proof

First we provide some preliminaries. Operator D _β is an unbounded, self-adjoint (thereby closed) operator that is bounded from below, with bounded inverse. Therefore, $\mathbb{H}_{I}(\mathbb{R}^{2})$ is again a Hilbert space:

• Let $(f_{n})_{n\in\mathbb{N}}$ be Cauchy in $\mathbb {H}_{I}(\mathbb{R}^{2})$. Then $(D_{\beta}f_{n})_{n\in\mathbb{N}}$ is Cauchy in $\mathbb{L}_{2}(\mathbb{R}^{2})$. Because $\mathbb {L}_{2}(\mathbb{R}^{2})$ is complete we have D _β f _n →g in $\mathbb{L}_{2}(\mathbb{R}^{2})$. But then, since $D_{\beta}^{-1}$ is bounded, f _n is also Cauchy in $\mathbb{L}_{2}(\mathbb{R}^{2})$, so f _n →f in $\mathbb{L}_{2}(\mathbb{R}^{2})$ to some $f \in \mathbb{L}_{2}(\mathbb{R}^{2})$. Now D _β is self adjoint and therefore closed so $f \in\mathcal{D}(D_{\beta})$ and D _β f=g. So we have D _β f _n →D _β f in $\mathbb {L}_{2}(\mathbb{R}^{2})$, so f _n →f in $\mathbb{H}_{I}(\mathbb {R}^{2})$, and $f \in\mathbb{H}_{I}(\mathbb{R}^{2})$.

The space $\mathbb{H}_{-I}(\mathbb{R}^{2})$ is defined as the completion of $\mathbb{H}_{I}(\mathbb{R}^{2})$ and is equipped with inverse product $(f,g)_{-I} = (D_{\beta}^{-1} f, D_{\beta}^{-1} g)_{\mathbb{L}_{2}(\mathbb{R}^{2})}$. This space is isomorphic to the dual space of $\mathbb{H}_{I}(\mathbb{R}^{2})$ under the pairing

$$ \langle F,f\rangle = \bigl(R^{-1} F, R f \bigr)_{\mathbb{L}_2(\mathbb{R}^2)} $$

(56)

for all $F \in\mathbb{H}_{-I}(\mathbb{R}^{2})$ and $f \in\mathbb {H}_{I}(\mathbb{R}^{2})$. In fact, all embeddings in $\mathbb {H}_{I}(\mathbb{R}^{2}) \hookrightarrow\mathbb{L}_{2}(\mathbb{R}^{2}) \hookrightarrow\mathbb{H}_{-I}(\mathbb{R}^{2})$ are dense. Now every $\mathbb{L}_{2}(\mathbb{R}^{2})$ element is the limit of $\mathbb {H}_{I}(\mathbb{R}^{2})$ elements, i.e., $\mathbb{H}_{I}(\mathbb{R}^{2})$ is dense in $\mathbb{L}_{2}(\mathbb{R}^{2})$. Furthermore, since $D_{\beta}^{-1}$ is bounded we have $\mathbb{H}_{I}(\mathbb{R}^{2}) = D_{\beta}^{-1}(\mathbb{L}_{2}(\mathbb{R}^{2}))$.

Now after these preliminaries, let us continue with the proof. Consider the associated normal orientation score transform

$$\mathcal{W}_{\tilde{\psi}}: \mathbb{L}_2\bigl(\mathbb{R}^2 \bigr) \mapsto \mathbb{C}_K^{\mathit{SE}(2)} $$

with $\tilde{\psi} = D_{\beta}^{-1} \psi\in\mathbb{L}_{2}(\mathbb {R}^{2})$ associated to $\psi\in\mathbb{H}_{-I}(\mathbb{R}^{2})$. Then by the results in [19, Thm. 18 and 19] this transform is unitary. This transform maps $\mathbb{L}_{2}(\mathbb{R}^{2})$ onto the unique reproducing kernel subspace $\mathbb{C}_{K}^{\mathit{SE}(2)}$, with reproducing kernel $K(g,h) = (\mathcal{U}_{g}\tilde{\psi},\mathcal {U}_{h}\tilde{\psi})$. In fact we have

$$\begin{aligned} \Vert\mathcal{W}_{\tilde{\psi}} f \Vert_{\mathbb{C}_k^{\mathit{SE}(2)}}^2 &= \int_{\mathbb{R}^2} \int_{S^1} \bigl|\mathcal{F} \mathcal{W}_{\tilde{\psi}} f (\boldsymbol {\omega },\theta) \bigr|^2 d\theta M_{\tilde {\psi}}^{-1}(\boldsymbol {\omega }) d\boldsymbol {\omega }\\ &= \int_{\mathbb{R}^2} \bigl|f(\mathbf{x})\bigr|^2 d\mathbf{x}, \end{aligned} $$

for all $f\in\mathbb{L}_{2}(\mathbf{R}^{2})$. Therefore $D_{\beta}\mathcal{W}_{\tilde{\psi}} = D_{\beta}\mathcal {W}_{D_{\beta}^{-1}\psi}$ is an isometry from $\mathbb{L}_{2}(\mathbb {R}^{2})$ into $\mathbb{L}_{2}(\mathit{SE}(2))$ if $M_{D_{\beta}^{-1}\psi}=\beta ^{-2}$ (since $\beta^{2} M_{D_{\beta}^{-1}\psi} = 1$), and moreover if $f \in\mathcal{D}(D_{\beta})=\mathbb{H}_{I}(\mathbb{R}^{2})$ we have (using Eq. (56) and $D_{\beta}\mathcal{U}_{g} = \mathcal{U}_{g} D_{\beta}$) that

$$ \begin{aligned}[b] (D_\beta\mathcal{W}_{D_\beta^{-1}\psi}f) (g) & =(\mathcal{W}_{D_\beta^{-1}\psi}D_\beta f) (g) \\ & = \bigl(\mathcal{U}_g D_\beta^{-1} \psi, D_\beta f \bigr)_{\mathbb {L}_2(\mathbb{R}^2)} \\ & = \bigl(D_\beta^{-1} \psi, D_\beta \mathcal{U}_{g^{-1}}f \bigr)_{\mathbb {L}_2(\mathbb{R}^2)} \\ & =\langle \psi,\mathcal{U}_{g^{-1}}f\rangle =(\mathfrak {W}_\psi f) (g), \end{aligned} $$

(57)

for all g∈SE(2) and for all $f \in\mathbb{H}_{I}(\mathbb{R}^{2})$.

Now Hilbert space $\mathbb{H}_{I}(\mathbb{R}^{2})$ is dense in $\mathbb{L}_{2}(\mathbb{R}^{2})$ and $D_{\beta}\mathcal {W}_{D_{\beta}^{-1}\psi} \vert _{\mathbb{H}_{I}(\mathbb{R}^{2})} = \mathfrak {W}_{\psi}$ maps $\mathbb{H}_{I}(\mathbb{R}^{2})$ (with $\mathbb {L}_{2}$-norm) isometrically into $\mathbb{L}_{2}(\mathit{SE}(2))$. So $\mathfrak {W}_{\psi}$ is closable as it admits the closed extension $D_{\beta}\mathcal{W}_{D_{\beta}^{-1}\psi}$ as an extension. □

Concluding Remark

By the result of the previous theorem, $\psi\in\mathbb {H}_{-I}(\mathbb{R}^{2})$ with $M_{D_{\beta}^{-1}\psi}=\beta^{-2}$ can be called proper distributional wavelets. When insisting on an $\mathbb{L}_{2}$-isometric mapping between image and score one has to fall back on these kind of wavelets. In case of cake wavelets (proper wavelets of class I [19, Ch. 4.6.1]), when ϱ→∞ such wavelets typically become oriented δ-distributions.

In case of proper wavelets of class II [19, Ch. 4.6.2] (including the kernel proposed by Kalitzin [43]) such wavelets concentrate around and explode along the x-axis when N→∞, [19, Fig. 4.11 and Ch. 7.3]. In both cases the limits do not exists in $\mathbb{L}_{2}$-sense, but they do exist both pointwise and in $\mathbb{H}_{-I}$-sense.

We conclude from the results in this section that the orientation score framework does not insist on images to be bandlimited, and remains valid regardless the sampling size/rate.