Iterative Graph Cuts for Image Segmentation with a Nonlinear Statistical Shape Prior

Abstract

Shape-based regularization has proven to be a useful method for delineating objects within noisy images where one has prior knowledge of the shape of the targeted object. When a collection of possible shapes is available, the specification of a shape prior using kernel density estimation is a natural technique. Unfortunately, energy functionals arising from kernel density estimation are of a form that makes them impossible to directly minimize using efficient optimization algorithms such as graph cuts. Our main contribution is to show how one may recast the energy functional into a form that is minimizable iteratively and efficiently using graph cuts.

Appendices

Appendix A: Shape Alignment

In order to align the shape templates, we choose the affine transformation $\mathbb{T}(\mathbf {s})=\alpha\mathbf{R}(\boldsymbol{\omega}) (\mathbf {s}-\mathbf {c})$ that minimizes the shape energy

In this expression, we have rewritten the contour integral on ∂Ω in terms of an integral over the zero-level set of ϕ _Ω . Here, we develop a local Newton-Raphson algorithm for finding the optimal transform. Let us denote the 2⋅d column vector of transformation parameters φ=[α c ω]. We estimate φ using the iterative updates

$$\boldsymbol{\varphi}_{n+1} = \boldsymbol{\varphi}_n - \bigl[ \mathbf {H}_{\boldsymbol{\varphi}} (\boldsymbol{\varphi}_n ) \bigr]^{-1}\nabla_{\boldsymbol{\varphi}} U_{\textrm{shape}}(\boldsymbol{\varphi}_n) $$

where

$$\nabla_{\boldsymbol{\varphi}}E = \biggl[ \frac{\partial}{\partial\alpha}\quad \nabla_{\mathbf{c}}^\textrm{T} \quad \nabla_{\boldsymbol{\omega}}^\textrm {T} \biggr]^\textrm{T}U_{\textrm{shape}} $$

and

To populate the matrix, we need to calculate the gradients with respect to α, c, and ω. For λ>0, the first-order derivatives take the form

The sign function comes about from differentiation of the absolute value function in the distributional sense.

For λ≠1, the second-order derivatives constitute tensors that take the form

If λ=1, the first term is zero and the following term is added

The contour integrals in these expressions can be calculated using a regularized version of the Dirac delta function such as

$$\delta_\epsilon(x) = \frac{1}{\pi}\frac{\epsilon}{\epsilon^2+x^2} \quad \epsilon \to0^+. $$

Similarly, the characteristic function can be interpreted as the Heaviside function acting on the signed-distance shape embedding, which can be approximated using an approximation of the Heaviside function such as

$$H_\epsilon(x) = \frac{1}{2} + \frac{1}{\pi}\arctan \frac{x}{\epsilon} \quad\epsilon\to0^+. $$

The nonzero transformation derivatives are as follows

The results of this alignment method are shown in the third column Fig. 3. As an example, the raw input templates for the vans in this figure are shown in Fig. 5.

Fig. 5

Inputted shape templates for a van. These templates are aligned using the method detailed in Appendix A in order to generate the aligned templates of Fig. 3

Appendix B: MM Algorithm for Iterative Graph Cuts

The shape term log∑w _j K(Ω,Ω _j ) can make minimization of the energy difficult, since its formulation involves a sum within a logarithm, making the energy functional nonlinear with respect to the labeling of pixels (into background and foreground) in the image.

To linearize the shape contribution, we will derive a surrogate function with separated terms. A function f(x|x _k ), with fixed x _k , is said to majorize a function g(x) at x _k if the following holds [17]

(16)

(17)

We wish to perform iterative inference by finding a sequence of segmentations Ω ⁽ⁿ⁺¹⁾=argmin _Ω Q(Ω|Ω ⁽ⁿ⁾), where Q(Ω|Ω ⁽ⁿ⁾) majorizes Eq. (7). By the descent property of the MM algorithm [17], this sequence converges to a local minimum.

For any convex function f(x), the following holds [17]

$$f \biggl(\sum_i \alpha_it_i \biggr)\leq\sum_i\alpha_i f(t_i). $$

Noting that −log(⋅) is convex, we have

(18)

verifying that the inequality condition (Eq. (16)) holds. In the case that Ω=Ω ⁽ⁿ⁾, we have

verifying that the equality condition (Eq. (17)) holds. So the two majorizing conditions (16), (17) are met, and we can minimize our original energy by iteratively minimizing

(19)

Because the distance function can be written as a sum over vertices and edges of a graph, so can Eq. (19). As a result, it is possible to minimize Eq. (19) within the graph cuts framework.