Feature matching based on unsupervised manifold alignment

Abstract

Feature-based methods for image registration frequently encounter the correspondence problem. In this paper, we formulate feature-based image registration as a manifold alignment problem, and present a novel matching method for finding the correspondences among different images containing the same object. Different from the semi-supervised manifold alignment, our methods map the data sets to the underlying common manifold without using correspondence information. An iterative multiplicative updating algorithm is proposed to optimize the objective, and its convergence is guaranteed theoretically. The proposed approach has been tested for matching accuracy, and robustness to outliers. Its performance on synthetic and real images is compared with the state-of-the-art reference algorithms.

Appendix (Proofs of Theorem 1)

To prove Theorem 1, we need to show that $J$ is non-increasing under the update steps in Eqs. (19) and (20). First, we need to prove that $J$ is non-increasing under the update step in Eq. (19). We will follow the similar procedure described in [29]. We use the auxiliary function approach to prove the convergence. Here we first introduce the definition of auxiliary function.

Definition

$G(u,u^{{\prime }})$ is an auxiliary function for $F(u)$ if the conditions

$$\begin{aligned} G(u,u^{{\prime }})\ge F(u),\quad G(u,u)=F(u) \end{aligned}$$

(30)

are satisfied.

Lemma 1

If $G$ is an auxiliary function of $F$, then $F$ is non-increasing under the update

$$\begin{aligned} u^{(t+1)}=\mathop {\arg \min }\limits _u G(u,u^{(t)}) \end{aligned}$$

(31)

Proof

$F(u^{(t+1)})\!\le \! G(u^{(t+1)},u^{(t)})\!\le \! G(u^{(t)},u^{(t)})\!=\!F(u^{(t)})$

Considering any element $a_{kl}$ in $A$, we use $F$ to denote the part of $J$ which is only relevant to $a_{kl}$. It is easy to check that

$$\begin{aligned} F^{{\prime }}(a_{kl} )&\!= \left( {\frac{\partial J}{\partial A}} \right)_{kl} \!=\!(2XH^{x}X^{T}A\nonumber \\&\qquad \qquad \qquad \!-\!2XHY^{T}B\!+\!2\lambda _1 XM^{x}X^{T}A)_{kl}\end{aligned}$$

(32)

$$\begin{aligned} F^{{\prime }{\prime }}(a_{kl} ) = \frac{\partial (2XH^{x}X^{T}A\!-\!2XHY^{T}B\!+\!2\lambda _1 XM^{x}X^{T}A)_{kl} }{\partial a_{kl} } \nonumber \\&\!\!= 2\frac{\partial (XH^{x}X^{T}A)_{kl} }{\partial a_{kl} }\!\!+\!2\lambda _1 \frac{\partial (XM^{x}X^{T}A)_{kl} }{\partial a_{kl} } \nonumber \\&\!\!= 2\frac{\partial \sum _i {(XH^{x}X^{T})_{ki} a_{il} } }{\partial a_{kl} }\!+\!\!2\lambda _1 \frac{\partial \sum _j {(XM^{x}X^{T})_{kj} a_{jl} } }{\partial a_{kl} } \nonumber \\&\!\!= 2(XH^{x}X^{T})_{kk} \!+\!2\lambda _1 (XM^{x}X^{T})_{kk} \end{aligned}$$

(33)

Lemma 2

Function

$$\begin{aligned} G(a,a_{kl}^{(t)} ) = F(a_{kl}^{(t)} )\!+\!F^{{\prime }}(a_{kl}^{(t)} )(a\!-\!a_{kl}^{(t)} ) \nonumber \\&\!+\frac{(XH^{x}X^{T}A)_{kl} \!+\!\lambda _1 (XM^{x}X^{T}A)_{kl} }{a_{kl}^{(t)} }(a\!-\!a_{kl}^{(t)} )^{2}\nonumber \\ \end{aligned}$$

(34)

is an auxiliary function for $F(a)$.

Proof

Since $G(a,a)=F(a)$ is obvious, we need only show that $G(a,a_{kl}^{(t)} )\ge F(a)$. To do this, we compare the Taylor series expansion of $F(a)$

$$\begin{aligned} F(a) = F(a_{kl}^{(t)} )\!+\!F^{{\prime }}(a_{kl}^{(t)} )(a\!-\!a_{kl}^{(t)} )\!+\!\frac{1}{2}F^{{\prime }{\prime }}(a_{kl}^{(t)} )(a\!-\!a_{kl}^{(t)} )^{2} \nonumber \\ = F(a_{kl}^{(t)} )\!+\!F^{{\prime }}(a_{kl}^{(t)} )(a\!-\!a_{kl}^{(t)} )\!+\} )^{2} \end{aligned}$$

(35)

With Eq. (34) to find that $G(a,a_{kl}^{(t)} )\ge F(a)$ is equivalent to [30]

$$\begin{aligned}&\frac{(XH^{x}X^{T}A)_{kl} \!+\lambda _1 (XM^{x}X^{T}A)_{kl} }{a_{kl}^{(t)} }\nonumber \\&\quad \ge (XH^{x}X^{T})_{kk}+\lambda _1 (XM^{x}X^{T})_{kk} \end{aligned}$$

(36)

Since

$$\begin{aligned} (XH^{x}X^{T}A)_{kl} =\sum _{i=1} {(XH^{x}X^{T})_{ki} a_{il}^{(t)} } \ge (XH^{x}X^{T})_{kk} a_{kl}^{(t)}\nonumber \\ \end{aligned}$$

(37)

and

$$\begin{aligned} \begin{aligned} \lambda _1 (XM^{x}X^{T}A)_{kl}=\,&\lambda _1 \sum _{j=1} {(XM^{x}X^{T})_{kj} a_{jl}^{(t)} }\\ \ge \,&\lambda _1 (XM^{x}X^{T})_{kk} a_{kl}^{(t)} \end{aligned} \end{aligned}$$

(38)

Thus, $G(a,a_{kl}^{(t)} )\ge F(a)$ $\square$

We can now demonstrate the convergence of Theorem 1:

Proof of Theorem 1

$a^{(t+1)}=\mathop {\arg \min }\limits _a G(a,a^{(t)})$, and

$$\begin{aligned} G(a,a_{kl}^{(t)} )&= F(a_{kl}^{(t)} )+F^{{\prime }}(a_{kl}^{(t)} )(a-a_{kl}^{(t)} ) \nonumber \\&+\frac{(XH^{x}X^{T}A)_{kl}\!+\!\lambda _1 (XM^{x}X^{T}A)_{kl} }{a_{kl}^{(t)} }(a\!-\!a_{kl}^{(t)} )^{2}\nonumber \\ \end{aligned}$$

(39)

The minimum of $G(a,a^{(t)})$ with respect to $a$ is determined by setting the gradient to zeros $\frac{\partial G(a,a^{(t)})}{\partial a}=0$:

$$\begin{aligned}&\frac{\partial G(a,a_{kl}^{(t)} )}{\partial a}\!=\!F^{{\prime }}(a_{kl}^{(t)} )\nonumber \\&\;\;+ \frac{2(XH^{x}X^{T}A)_{kl} \!+\!2\lambda _1 (XM^{x}X^{T}A)_{kl} }{a_{kl}^{(t)} }(a\!-\!a_{kl}^{(t)} )\!=\!0 \nonumber \\ \end{aligned}$$

(40)

So, we get

$$\begin{aligned} a_{kl}^{(t+1)} =a_{kl}^{(t)} \frac{(XHY^{T}B)_{kl} }{(XH^{x}X^{T}A)_{kl} +(\lambda _1 XM^{x}X^{T}A)_{kl} } \end{aligned}$$

(41)

Since $G(a,a^{(t)})$ is an auxiliary function, $F$ is non-increasing under this update rule. $J$ can similarly be shown to be non-increasing under the update rules for $B$. $\square$